Add conflict resolution diff UI and app branding
This commit is contained in:
1
.project/.last_state_hash
Normal file
1
.project/.last_state_hash
Normal file
@@ -0,0 +1 @@
|
||||
a65cd76244a613f2505eb2212effb30f
|
||||
94
.project/README.md
Normal file
94
.project/README.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# Cairnquire - Documentation Index
|
||||
|
||||
## Project Planning
|
||||
|
||||
| Document | Description |
|
||||
|----------|-------------|
|
||||
| [project-plan.md](project-plan.md) | Master project plan with milestones, timelines, and acceptance criteria |
|
||||
| [architecture-overview.md](architecture-overview.md) | System architecture, component diagrams, and data flow |
|
||||
|
||||
## Architecture Decision Records (ADRs)
|
||||
|
||||
| Document | Decision |
|
||||
|----------|----------|
|
||||
| [ADR-001: Tech Stack](adrs/adr-001-tech-stack.md) | Go backend + Preact frontend |
|
||||
| [ADR-002: Sync Protocol](adrs/adr-002-sync-protocol.md) | SimpleSync content-addressed protocol |
|
||||
| [ADR-003: SSR Strategy](adrs/adr-003-ssr-strategy.md) | Go templates with Preact hydration |
|
||||
| [ADR-004: Auth Strategy](adrs/adr-004-auth-strategy.md) | Passkeys primary, passwords fallback |
|
||||
| [ADR-005: Storage Strategy](adrs/adr-005-storage-strategy.md) | libsql + content-addressed filesystem |
|
||||
| [ADR-006: Email Strategy](adrs/adr-006-email-strategy.md) | Postmark gateway, plain-text only |
|
||||
| [ADR-007: Security Model](adrs/adr-007-security-model.md) | Defense in depth, minimal dependencies |
|
||||
|
||||
## Milestones
|
||||
|
||||
| Document | Duration | Focus |
|
||||
|----------|----------|-------|
|
||||
| [Milestone 1: Foundation](milestones/milestone-01-foundation.md) | Weeks 1-2 | Server, database, markdown rendering |
|
||||
| [Milestone 2: Sync Protocol](milestones/milestone-02-sync-protocol.md) | Weeks 3-4 | File sync, offline support, WebSocket |
|
||||
| [Milestone 3: Authentication](milestones/milestone-03-authentication.md) | Weeks 5-6 | Passkeys, sessions, RBAC |
|
||||
| [Milestone 4: Collaboration](milestones/milestone-04-collaboration.md) | Weeks 7-8 | Comments, email, conflict resolution |
|
||||
| [Milestone 5: Search & UI](milestones/milestone-05-search-ui.md) | Weeks 9-10 | Full-text search, design system, responsive |
|
||||
| [Milestone 6: Production](milestones/milestone-06-production.md) | Weeks 11-12 | Testing, security audit, ops docs |
|
||||
|
||||
## Technical Specifications
|
||||
|
||||
| Document | Topic |
|
||||
|----------|-------|
|
||||
| [SimpleSync Protocol](specs/simplesync-protocol.md) | Message formats, state machine, security |
|
||||
| [Security Specification](specs/security-specification.md) | Threat model, controls, implementation |
|
||||
| [Deployment Guide](specs/deployment-guide.md) | Docker setup, configuration, operations |
|
||||
|
||||
## Key Metrics
|
||||
|
||||
- **Timeline**: 12 weeks (3 months)
|
||||
- **Test Coverage Target**: >80%
|
||||
- **Performance Targets**:
|
||||
- Initial page load: <100ms
|
||||
- Search query: <50ms
|
||||
- File sync: <5s
|
||||
- **Security**: Zero critical/high vulnerabilities
|
||||
- **Dependencies**: Minimal, all pinned and audited
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Review all documentation
|
||||
2. Approve project plan
|
||||
3. Set up repository and CI pipeline
|
||||
4. Begin Milestone 1: Foundation
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
cairnquire/
|
||||
├── docs/
|
||||
│ ├── README.md (this file)
|
||||
│ ├── project-plan.md
|
||||
│ ├── architecture-overview.md
|
||||
│ ├── adrs/
|
||||
│ │ ├── adr-001-tech-stack.md
|
||||
│ │ ├── adr-002-sync-protocol.md
|
||||
│ │ ├── adr-003-ssr-strategy.md
|
||||
│ │ ├── adr-004-auth-strategy.md
|
||||
│ │ ├── adr-005-storage-strategy.md
|
||||
│ │ ├── adr-006-email-strategy.md
|
||||
│ │ └── adr-007-security-model.md
|
||||
│ ├── milestones/
|
||||
│ │ ├── milestone-01-foundation.md
|
||||
│ │ ├── milestone-02-sync-protocol.md
|
||||
│ │ ├── milestone-03-authentication.md
|
||||
│ │ ├── milestone-04-collaboration.md
|
||||
│ │ ├── milestone-05-search-ui.md
|
||||
│ │ └── milestone-06-production.md
|
||||
│ └── specs/
|
||||
│ ├── simplesync-protocol.md
|
||||
│ ├── security-specification.md
|
||||
│ └── deployment-guide.md
|
||||
├── apps/
|
||||
│ ├── server/ # Go application
|
||||
│ └── web/ # Preact frontend
|
||||
├── packages/
|
||||
│ └── protocol/ # Shared types
|
||||
├── docker-compose.yml
|
||||
├── Dockerfile
|
||||
└── mise.toml
|
||||
```
|
||||
51
.project/STATUS.md
Normal file
51
.project/STATUS.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# Project Status Report
|
||||
|
||||
**Generated:** 2026-04-29 14:57:37 UTC
|
||||
**Monitoring Duration:** 14h 41m
|
||||
**Iteration:** 86
|
||||
**Branch:** codex/foundation-bootstrap
|
||||
|
||||
## Quick Stats
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Total Files | 101 |
|
||||
| Go Files | 22 ( 2304 lines) |
|
||||
| TypeScript Files | 3 ( 761 lines) |
|
||||
| Markdown Files | 134 ( 27723 lines) |
|
||||
| Docker Config | yes |
|
||||
| Uncommitted Changes | 9 |
|
||||
| Milestones Complete | 0/6 (0%) |
|
||||
|
||||
## Recent Activity
|
||||
|
||||
**Last Commit:** 3d8ca8b - refactor: focus admin dashboard layout (3 hours ago)
|
||||
|
||||
## Milestone Progress
|
||||
|
||||
- [ ] Milestone 1: Foundation
|
||||
- [ ] Milestone 2: Sync Protocol
|
||||
- [ ] Milestone 3: Authentication
|
||||
- [ ] Milestone 4: Collaboration
|
||||
- [ ] Milestone 5: Search & UI
|
||||
- [ ] Milestone 6: Production
|
||||
|
||||
**Current Focus:** Not Started
|
||||
|
||||
### Error Handling (10 ignored errors)
|
||||
|
||||
Several errors are being silently ignored. Consider handling or logging these.
|
||||
|
||||
## Notes for Human Review
|
||||
|
||||
>> **Active development.** 22 Go files with 2304 lines of code.
|
||||
>> **9 uncommitted changes** detected. Codex may be mid-work.
|
||||
>> **Code review items found.** See Action Items above.
|
||||
|
||||
## Files Changed This Session
|
||||
|
||||
|
||||
---
|
||||
|
||||
*This report auto-generated by Cairnquire progress monitor.*
|
||||
*Next update in ~10 minutes or when filesystem changes detected.*
|
||||
53
.project/adrs/adr-001-tech-stack.md
Normal file
53
.project/adrs/adr-001-tech-stack.md
Normal file
@@ -0,0 +1,53 @@
|
||||
# ADR-001: Go Backend with Preact Frontend
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
We need to choose a backend language/framework and frontend technology for a documentation platform with these requirements:
|
||||
- Ultra-fast performance
|
||||
- Minimal external dependencies
|
||||
- No vendor lock-in
|
||||
- Battle-tested security
|
||||
- Server-side rendering for non-interactive pages
|
||||
- Offline-capable web client
|
||||
|
||||
## Decision
|
||||
**Backend**: Go 1.22+ with standard library plus minimal, well-audited packages
|
||||
**Frontend**: Preact 10 with TypeScript, Vite build system
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- Go's standard library is comprehensive and security-audited
|
||||
- Single static binary deployment — no runtime dependencies
|
||||
- Preact is 10KB (vs 40KB+ React) with identical API
|
||||
- Go's `html/template` provides safe SSR with auto-escaping
|
||||
- TypeScript gives type safety without runtime overhead
|
||||
- Both ecosystems have excellent supply-chain security (Go modules with checksums, npm audit)
|
||||
|
||||
### Negative
|
||||
- Go template syntax is less expressive than JSX
|
||||
- WebSocket sync logic must be implemented manually (no framework like Socket.io)
|
||||
- Preact ecosystem smaller than React (mitigated by using mostly native APIs)
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### Rust + Axum
|
||||
- **Pros**: Memory safety guarantees, excellent performance
|
||||
- **Cons**: Longer compile times, smaller talent pool, steeper learning curve for team
|
||||
- **Rejected**: Go's pragmatic balance of safety and velocity better fits project timeline
|
||||
|
||||
### Deno Fresh
|
||||
- **Pros**: Island architecture, native TypeScript, no build step
|
||||
- **Cons**: Newer ecosystem, fewer audited libraries, Deno runtime less battle-tested than Go
|
||||
- **Rejected**: Go's maturity and deployment simplicity preferred
|
||||
|
||||
### Next.js / React Server Components
|
||||
- **Pros**: Mature ecosystem, built-in SSR
|
||||
- **Cons**: Heavy bundle size, complex build system, RSC lock-in, many dependencies
|
||||
- **Rejected**: Violates "minimal dependencies" and "no vendor lock-in" principles
|
||||
|
||||
## References
|
||||
- Go Security Policy: https://go.dev/security
|
||||
- Preact Size Comparison: https://bundlephobia.com/package/preact@10.19.3
|
||||
81
.project/adrs/adr-002-sync-protocol.md
Normal file
81
.project/adrs/adr-002-sync-protocol.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# ADR-002: SimpleSync Protocol
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
Users need to edit markdown files on their local filesystem (using any editor) and have changes reflected on the web automatically. Multiple users should be able to work on different files simultaneously without conflicts. We need a sync mechanism simpler than git but robust enough for production use.
|
||||
|
||||
## Decision
|
||||
Implement **SimpleSync**: a content-addressed, Merkle-tree-inspired sync protocol over WebSocket with the following characteristics:
|
||||
|
||||
1. **Content-addressed storage**: All files identified by SHA-256 hash
|
||||
2. **File-level granularity**: Parallel editing of different files is conflict-free
|
||||
3. **Automatic sync**: File watcher detects changes and pushes to server
|
||||
4. **Three-way merge**: Only when same file edited concurrently
|
||||
5. **No branches/commits**: Just current state + history log
|
||||
|
||||
## Protocol Flow
|
||||
|
||||
### Client → Server: Sync Request
|
||||
```json
|
||||
{
|
||||
"type": "sync_request",
|
||||
"client_root_hash": "abc123...",
|
||||
"client_files": {
|
||||
"getting-started.md": "hash1",
|
||||
"api-reference.md": "hash2"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Server → Client: Sync Response
|
||||
```json
|
||||
{
|
||||
"type": "sync_response",
|
||||
"server_root_hash": "def456...",
|
||||
"missing_from_client": ["hash3", "hash4"],
|
||||
"missing_from_server": ["hash5"],
|
||||
"conflicts": ["getting-started.md"]
|
||||
}
|
||||
```
|
||||
|
||||
### Content Transfer
|
||||
Files transferred by hash: `GET /content/{hash}` or WebSocket binary frames
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- Dramatically simpler mental model than git
|
||||
- Automatic — no user commands needed
|
||||
- Works with any text editor
|
||||
- Immutable file store provides natural deduplication
|
||||
- Easy to implement offline queue: just store hashes and content
|
||||
|
||||
### Negative
|
||||
- No offline branching (by design)
|
||||
- Conflict resolution UI must be built
|
||||
- No rebase/squash history manipulation (intentional simplicity)
|
||||
- Large binary files could bloat content store (mitigated by size limits)
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### Git-based sync
|
||||
- **Pros**: Mature, well-understood, existing tools
|
||||
- **Cons**: Requires git on client, complex merge UX, repo-level locking
|
||||
- **Rejected**: Too complex for notes; overkill for target use case
|
||||
|
||||
### CRDTs (Yjs, Automerge)
|
||||
- **Pros**: Real-time collaborative editing, automatic conflict resolution
|
||||
- **Cons**: Complex to implement correctly, memory overhead, designed for real-time not async
|
||||
- **Rejected**: Requirement is async-only; CRDTs are overkill
|
||||
|
||||
### rsync / Unison
|
||||
- **Pros**: Battle-tested file sync
|
||||
- **Cons**: No concept of "versions" or "comments", no web integration
|
||||
- **Rejected**: Doesn't meet collaboration requirements
|
||||
|
||||
## References
|
||||
- Content-addressable storage: https://en.wikipedia.org/wiki/Content-addressable_storage
|
||||
- Merkle trees: https://en.wikipedia.org/wiki/Merkle_tree
|
||||
- WebSocket RFC: https://tools.ietf.org/html/rfc6455
|
||||
67
.project/adrs/adr-003-ssr-strategy.md
Normal file
67
.project/adrs/adr-003-ssr-strategy.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# ADR-003: SSR Strategy - Go Templates with Preact Hydration
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
The application needs to serve document pages that are readable without JavaScript (for performance and accessibility), while providing rich interactivity (comments, editing, search) when JavaScript is available. We need a server-side rendering strategy that works with our Go backend and Preact frontend.
|
||||
|
||||
## Decision
|
||||
**Strategy C**: Go's `html/template` renders initial HTML for all pages. Preact hydrates interactive components on the client.
|
||||
|
||||
### Implementation Details
|
||||
|
||||
1. **Server Rendering**:
|
||||
- Go handlers query database for document content
|
||||
- Markdown parsed to HTML server-side (Goldmark)
|
||||
- `html/template` renders full page with document HTML embedded
|
||||
- No JavaScript required for reading
|
||||
|
||||
2. **Client Hydration**:
|
||||
- Preact components mount into designated DOM containers
|
||||
- Comments, search, editor are Preact islands
|
||||
- Static content remains server-rendered HTML
|
||||
- `preact-iso` handles client-side navigation between pages
|
||||
|
||||
3. **Data Passing**:
|
||||
- Initial state embedded as JSON in `<script type="application/json">` tags
|
||||
- Preact reads initial state during hydration, avoids extra API calls
|
||||
- CSP nonce applied to all inline scripts
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- First contentful paint is instant (pure HTML)
|
||||
- SEO-friendly without extra complexity
|
||||
- Graceful degradation: works without JS
|
||||
- Go templates are auto-escaping XSS-safe
|
||||
- No V8/JS runtime embedded in Go binary
|
||||
- Smaller client bundle (only interactive components)
|
||||
|
||||
### Negative
|
||||
- Some code duplication: markdown parsing logic in both Go and JS (for preview)
|
||||
- Template syntax is verbose compared to JSX
|
||||
- Hydration can cause flicker if not carefully managed
|
||||
- Client state must reconcile with server-rendered HTML
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### Go Templates Only (No Preact)
|
||||
- **Pros**: Zero client JS, maximum performance
|
||||
- **Cons**: No interactive features (comments, real-time sync, search)
|
||||
- **Rejected**: Doesn't meet collaboration requirements
|
||||
|
||||
### Embedded JS Runtime (QuickJS/V8)
|
||||
- **Pros**: True isomorphic rendering, JSX templates
|
||||
- **Cons**: Adds C dependency, complex build, larger binary, security surface area
|
||||
- **Rejected**: Violates minimal dependency principle
|
||||
|
||||
### Preact SSR via WASM
|
||||
- **Pros**: Same templates on server and client
|
||||
- **Cons**: WASM overhead, complex build pipeline, slower than Go templates
|
||||
- **Rejected**: Go templates are faster and simpler
|
||||
|
||||
## References
|
||||
- Go html/template security: https://pkg.go.dev/html/template
|
||||
- Preact hydration: https://preactjs.com/guide/v10/server-side-rendering
|
||||
- Islands Architecture: https://jasonformat.com/islands-architecture/
|
||||
74
.project/adrs/adr-004-auth-strategy.md
Normal file
74
.project/adrs/adr-004-auth-strategy.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# ADR-004: Authentication Strategy
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
The platform needs secure authentication for both technical and non-technical users. Requirements include: primary passkey support, password fallback, session management, and resistance to common attacks (phishing, credential stuffing, session hijacking).
|
||||
|
||||
## Decision
|
||||
**Passkeys as primary authentication method, Argon2id-hashed passwords as fallback.**
|
||||
|
||||
### Implementation
|
||||
|
||||
1. **Registration Flow**:
|
||||
- User provides email
|
||||
- System generates WebAuthn registration challenge
|
||||
- Browser creates passkey ( Touch ID, Windows Hello, YubiKey)
|
||||
- Server stores credential ID and public key
|
||||
- Optional: user sets password fallback
|
||||
|
||||
2. **Login Flow (Passkey)**:
|
||||
- User enters email
|
||||
- Server returns WebAuthn assertion challenge
|
||||
- Browser signs challenge with passkey
|
||||
- Server verifies signature, creates session
|
||||
|
||||
3. **Login Flow (Password)**:
|
||||
- User enters email + password
|
||||
- Server verifies Argon2id hash (constant-time comparison)
|
||||
- If valid, creates session
|
||||
- Prompts to set up passkey (progressive enhancement)
|
||||
|
||||
4. **Session Management**:
|
||||
- Signed cookies with 24-hour expiry
|
||||
- Refresh tokens rotate on each use
|
||||
- Sessions stored in database (revocable)
|
||||
- HttpOnly, Secure, SameSite=Strict flags
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- Passkeys are phishing-resistant by design
|
||||
- No password database to breach (if passkey-only)
|
||||
- Argon2id is winner of Password Hashing Competition
|
||||
- Session tokens are revocable server-side
|
||||
- Progressive enhancement: passkey users never type passwords
|
||||
|
||||
### Negative
|
||||
- Passkey support varies by browser/OS
|
||||
- Users may lose access if they lose all devices (recovery flow needed)
|
||||
- WebAuthn implementation complexity (mitigated by go-webauthn library)
|
||||
- Backup codes or recovery email needed for account recovery
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### OAuth 2.0 / SSO Only
|
||||
- **Pros**: No password management, familiar UX
|
||||
- **Cons**: Vendor lock-in to identity providers, privacy concerns, not self-hosted
|
||||
- **Rejected**: Violates "no vendor lock-in" principle
|
||||
|
||||
### Passwords Only (with 2FA)
|
||||
- **Pros**: Simple, universally supported
|
||||
- **Cons**: Vulnerable to phishing, credential stuffing, SIM swapping
|
||||
- **Rejected**: Passkeys are strictly superior
|
||||
|
||||
### Magic Links
|
||||
- **Pros**: No passwords, simple UX
|
||||
- **Cons**: Email dependency, slower, inbox security issues
|
||||
- **Rejected**: Less secure than passkeys, poor for frequent login
|
||||
|
||||
## References
|
||||
- WebAuthn spec: https://www.w3.org/TR/webauthn-2/
|
||||
- go-webauthn library: https://github.com/go-webauthn/webauthn
|
||||
- Argon2id RFC: https://datatracker.ietf.org/doc/html/rfc9106
|
||||
68
.project/adrs/adr-005-storage-strategy.md
Normal file
68
.project/adrs/adr-005-storage-strategy.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# ADR-005: Storage Strategy
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
The system needs to store: structured data (users, documents, comments), document content, file attachments, and search indexes. Requirements include: zero external dependencies, backup simplicity, and resistance to data corruption.
|
||||
|
||||
## Decision
|
||||
**libsql (SQLite) for structured data + content-addressed filesystem for files**
|
||||
|
||||
### Implementation
|
||||
|
||||
1. **Database (libsql/SQLite)**:
|
||||
- Single file: `data/db.sqlite`
|
||||
- WAL mode for concurrent reads during writes
|
||||
- Embedded — no separate process needed
|
||||
- Backup: `cp db.sqlite db.sqlite.backup` (atomic with WAL checkpoint)
|
||||
|
||||
2. **File Storage (Content-Addressed)**:
|
||||
- Directory: `data/files/`
|
||||
- Layout: `data/files/ab/cd/abcdef1234567890abcdef1234567890abcdef12`
|
||||
- SHA-256 hash as filename (hex encoded)
|
||||
- First two characters as subdirectory (for filesystem performance)
|
||||
- Immutable: write-once, never modify
|
||||
|
||||
3. **Document Mapping**:
|
||||
- Database stores: `document_path → current_hash`
|
||||
- History table stores all previous hashes
|
||||
- Attachment metadata stores original filename + content hash
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- SQLite is the most tested database in the world (billions of deployments)
|
||||
- libsql adds replication option without changing the API
|
||||
- Content-addressed files are naturally deduplicated
|
||||
- Filesystem storage is trivial to backup (rsync, tar)
|
||||
- No network dependencies, works offline
|
||||
- Single directory contains all state — easy to migrate
|
||||
|
||||
### Negative
|
||||
- SQLite has write concurrency limits (mitigated by WAL + short transactions)
|
||||
- Filesystem storage doesn't scale horizontally without shared storage
|
||||
- No built-in file expiration/garbage collection (must implement orphan cleanup)
|
||||
- Large attachments can bloat filesystem (mitigated by quotas)
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### PostgreSQL
|
||||
- **Pros**: Excellent concurrency, rich feature set, horizontal scaling
|
||||
- **Cons**: Requires separate process, more complex backup, network dependency
|
||||
- **Rejected**: Overkill for anticipated scale; SQLite is simpler and faster for read-heavy workloads
|
||||
|
||||
### Object Storage (S3/MinIO)
|
||||
- **Pros**: Unlimited scale, CDN-friendly
|
||||
- **Cons**: Network dependency, vendor lock-in, complex local development
|
||||
- **Rejected**: Violates zero-dependency and simple backup requirements
|
||||
|
||||
### SQLite for Everything (BLOBs)
|
||||
- **Pros**: Single file for everything
|
||||
- **Cons**: Large BLOBs slow down database, harder to serve directly, backup size bloat
|
||||
- **Rejected**: Separation of concerns — database for metadata, filesystem for content
|
||||
|
||||
## References
|
||||
- SQLite WAL mode: https://sqlite.org/wal.html
|
||||
- Content-addressable storage: https://git-scm.com/book/en/v2/Git-Internals-Git-Objects
|
||||
- libsql: https://github.com/tursodatabase/libsql
|
||||
79
.project/adrs/adr-006-email-strategy.md
Normal file
79
.project/adrs/adr-006-email-strategy.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# ADR-006: Email Integration
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
The platform needs to send email notifications for: file changes (to watchers), new comments, conflict alerts, and digest summaries. It must also receive email replies and convert them to comments. Requirement: use Postmark email gateway with plain-text only.
|
||||
|
||||
## Decision
|
||||
**Postmark HTTP API for outgoing email, Postmark inbound webhook for incoming email.**
|
||||
|
||||
### Implementation
|
||||
|
||||
1. **Outgoing Email**:
|
||||
- Go adapter using `github.com/keighl/postmark`
|
||||
- Plain-text templates (no HTML)
|
||||
- Message-ID generated per email for threading
|
||||
- Templates: notification, digest, conflict-alert, welcome
|
||||
|
||||
2. **Incoming Email**:
|
||||
- Postmark inbound webhook POSTs to `/webhooks/email`
|
||||
- Webhook payload parsed for: from, subject, body, in-reply-to
|
||||
- Reply matched to comment thread via In-Reply-To header
|
||||
- Body converted to plain text comment
|
||||
- Authentication: webhook signature verification
|
||||
|
||||
3. **Email Format**:
|
||||
```
|
||||
From: Cairnquire <notifications@example.com>
|
||||
To: user@example.com
|
||||
Subject: [docs/getting-started.md] Comment from Alice
|
||||
Message-ID: <comment-123@example.com>
|
||||
In-Reply-To: <comment-122@example.com>
|
||||
|
||||
Alice commented on "Getting Started":
|
||||
|
||||
> This section needs clarification...
|
||||
|
||||
Reply to this email to add a comment.
|
||||
|
||||
View online: https://example.com/docs/getting-started#comment-123
|
||||
```
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- Postmark has excellent deliverability reputation
|
||||
- Plain-text emails are accessible and lightweight
|
||||
- Email threading works in all clients
|
||||
- No HTML parsing/rendering security risks
|
||||
- Webhook integration is simple and reliable
|
||||
|
||||
### Negative
|
||||
- Postmark is a paid service (though inexpensive)
|
||||
- Requires DNS setup (SPF, DKIM) for deliverability
|
||||
- Plain-text limits formatting options (intentional trade-off)
|
||||
- Rate limits on Postmark API (mitigated by queue + retry)
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### SMTP Direct Delivery
|
||||
- **Pros**: No third-party dependency, full control
|
||||
- **Cons**: Complex deliverability management, IP reputation, spam folder issues
|
||||
- **Rejected**: Operational burden too high for small team
|
||||
|
||||
### SendGrid / Mailgun
|
||||
- **Pros**: Similar features to Postmark
|
||||
- **Cons**: More expensive at scale, complex APIs
|
||||
- **Rejected**: Postmark has better reputation for transactional email
|
||||
|
||||
### No Email (In-App Only)
|
||||
- **Pros**: Zero email complexity
|
||||
- **Cons**: Users must check app for notifications; poor for async collaboration
|
||||
- **Rejected**: Email is core to collaboration workflow
|
||||
|
||||
## References
|
||||
- Postmark API docs: https://postmarkapp.com/developer
|
||||
- keighl/postmark Go library: https://github.com/keighl/postmark
|
||||
- Email threading RFC: https://www.ietf.org/rfc/rfc2822.txt
|
||||
69
.project/adrs/adr-007-security-model.md
Normal file
69
.project/adrs/adr-007-security-model.md
Normal file
@@ -0,0 +1,69 @@
|
||||
# ADR-007: Security Model
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
The platform handles sensitive documentation and must resist supply-chain attacks, unauthorized access, data tampering, and common web vulnerabilities. The user explicitly requested a "paranoid" approach.
|
||||
|
||||
## Decision
|
||||
Implement defense in depth with the following layers:
|
||||
|
||||
### 1. Supply Chain Security
|
||||
- **Minimal dependencies**: Only well-established, security-audited libraries
|
||||
- **Pinned versions**: All Go modules pinned with go.sum checksums
|
||||
- **Vendoring option**: `go mod vendor` for air-gapped builds
|
||||
- **Centralized automation**: `mise run ...` tasks only — no ad hoc shell scripts, no curl | bash
|
||||
- **Reproducible builds**: Same source → same binary hash (via `-trimpath`)
|
||||
|
||||
### 2. Authentication Security
|
||||
- **Passkeys primary**: Phishing-resistant WebAuthn/FIDO2
|
||||
- **Argon2id fallback**: Password Hashing Competition winner
|
||||
- **Constant-time comparison**: All secret verification uses `subtle.ConstantTimeCompare`
|
||||
- **Rate limiting**: Auth endpoints: 5 attempts per minute per IP
|
||||
- **Session hardening**: Signed, HttpOnly, Secure, SameSite=Strict cookies
|
||||
|
||||
### 3. Authorization Security
|
||||
- **RBAC**: Read/write/admin per document and collection
|
||||
- **Principle of least privilege**: Default to no access, explicitly grant
|
||||
- **Content verification**: SHA-256 hash verified on every read
|
||||
- **Audit logging**: Every access logged with actor, resource, timestamp, IP
|
||||
|
||||
### 4. Input Validation
|
||||
- **Markdown sanitization**: Goldmark renders to safe HTML; no raw HTML injection
|
||||
- **File upload validation**: Magic number check, extension whitelist, size limits
|
||||
- **Path canonicalization**: All file paths resolved to absolute, checked against allowlist
|
||||
- **Query parameterization**: All SQL uses parameterized queries (no string concatenation)
|
||||
|
||||
### 5. Transport Security
|
||||
- **TLS 1.3**: Minimum version enforced
|
||||
- **HSTS**: Strict-Transport-Security header with preload
|
||||
- **CSP**: Content-Security-Policy restricts script sources to nonce-tagged inline and same-origin
|
||||
- **CORS**: Restrictive — only same-origin by default
|
||||
|
||||
### 6. Operational Security
|
||||
- **No secrets in logs**: All tokens, passwords redacted
|
||||
- **No secrets in environment**: Configuration from file or encrypted env vars
|
||||
- **Read-only container**: Filesystem mounted read-only except `/data`
|
||||
- **Non-root user**: Container runs as unprivileged user
|
||||
- **Health checks**: `/health` endpoint verifies all dependencies
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- Resistant to OWASP Top 10 vulnerabilities
|
||||
- Supply-chain attacks mitigated by minimal dependencies
|
||||
- Audit trail provides accountability
|
||||
- Container hardening reduces attack surface
|
||||
|
||||
### Negative
|
||||
- CSP can block legitimate resources (requires careful tuning)
|
||||
- Rate limiting may affect legitimate users behind NAT (mitigated by user-aware limits)
|
||||
- Security reviews required for any new dependency
|
||||
- More complex local development (TLS certs, CSP headers)
|
||||
|
||||
## References
|
||||
- OWASP Top 10: https://owasp.org/www-project-top-ten/
|
||||
- Go Security Best Practices: https://go.dev/security
|
||||
- WebAuthn Security: https://www.w3.org/TR/webauthn-2/#security-considerations
|
||||
- Content Security Policy: https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
|
||||
316
.project/architecture-overview.md
Normal file
316
.project/architecture-overview.md
Normal file
@@ -0,0 +1,316 @@
|
||||
# Architecture Overview
|
||||
|
||||
## System Context
|
||||
|
||||
Cairnquire is a self-hosted documentation platform. A single Go binary serves HTTP requests, manages a libsql database, watches a filesystem directory, and synchronizes content with connected web clients.
|
||||
|
||||
## Component Diagram
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Client Layer │
|
||||
│ │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────┐ │
|
||||
│ │ Browser │ │ File Editor │ │ Mobile / PWA │ │
|
||||
│ │ (Preact SPA) │ │ (VS Code, │ │ (Responsive Web) │ │
|
||||
│ │ │ │ Vim, etc.) │ │ │ │
|
||||
│ └──────┬───────┘ └──────┬───────┘ └────────────┬─────────────┘ │
|
||||
│ │ │ │ │
|
||||
│ │ HTTP/WS │ File System │ HTTP/WS │
|
||||
└─────────┼──────────────────┼────────────────────────┼────────────────┘
|
||||
│ │ │
|
||||
┌─────────┼──────────────────┼────────────────────────┼────────────────┐
|
||||
│ │ │ Go Application Server │
|
||||
│ │ │ │
|
||||
│ ┌──────▼──────┐ ┌──────▼──────┐ │
|
||||
│ │ HTTP Router │ │ File Watcher │ │
|
||||
│ │ (chi) │ │ (fsnotify) │ │
|
||||
│ └──────┬──────┘ └──────┬──────┘ │
|
||||
│ │ │ │
|
||||
│ ┌──────▼──────────────────▼──────┐ │
|
||||
│ │ Request Processing │ │
|
||||
│ │ ┌────────────┐ ┌───────────┐ │ │
|
||||
│ │ │ Auth Layer │ │ Markdown │ │ │
|
||||
│ │ │(WebAuthn) │ │ Renderer │ │ │
|
||||
│ │ └────────────┘ └───────────┘ │ │
|
||||
│ │ ┌────────────┐ ┌───────────┐ │ │
|
||||
│ │ │ Sync Engine│ │ Search │ │ │
|
||||
│ │ │(WebSocket) │ │ (FTS5) │ │ │
|
||||
│ │ └────────────┘ └───────────┘ │ │
|
||||
│ │ ┌────────────┐ ┌───────────┐ │ │
|
||||
│ │ │ Email │ │ File │ │ │
|
||||
│ │ │ (Postmark) │ │ Store │ │ │
|
||||
│ │ └────────────┘ └───────────┘ │ │
|
||||
│ └──────────┬─────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌──────────▼─────────────────────┐ │
|
||||
│ │ Data Layer │ │
|
||||
│ │ ┌────────────┐ ┌───────────┐ │ │
|
||||
│ │ │ libsql │ │ Content- │ │ │
|
||||
│ │ │ (SQLite) │ │ Addressed │ │ │
|
||||
│ │ │ │ │ Files │ │ │
|
||||
│ │ └────────────┘ └───────────┘ │ │
|
||||
│ └────────────────────────────────┘ │
|
||||
└────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Component Descriptions
|
||||
|
||||
### 1. HTTP Router (chi)
|
||||
- Serves static assets (CSS, JS, fonts)
|
||||
- Routes API requests to handlers
|
||||
- Serves server-rendered HTML for document pages
|
||||
- Enforces middleware chain: logging, recovery, CORS, CSP, auth
|
||||
|
||||
### 2. File Watcher (fsnotify)
|
||||
- Watches configured directory for .md file changes
|
||||
- Debounces rapid changes (1 second)
|
||||
- Computes SHA-256 hash of new/changed files
|
||||
- Triggers sync events to connected clients
|
||||
|
||||
### 3. Authentication Layer
|
||||
- WebAuthn/passkey primary (FIDO2 compliant)
|
||||
- Argon2id password fallback
|
||||
- Session cookies: signed, HttpOnly, Secure, SameSite=Strict
|
||||
- RBAC middleware checks permissions on every request
|
||||
|
||||
### 4. Markdown Renderer
|
||||
- Goldmark parser with custom extensions:
|
||||
- Wiki-links: `[[Page Name]]` and `[[Page Name|Display Text]]`
|
||||
- Tags: `#tagname` parsed and linked
|
||||
- Admonitions: `> [!NOTE]`, `> [!WARNING]`, etc.
|
||||
- Mermaid diagrams: fenced code blocks with `mermaid` language
|
||||
- Math: `$inline$` and `$$block$$` via KaTeX
|
||||
- Server-side rendering produces semantic HTML
|
||||
- Client-side hydration adds interactivity (folding, copy buttons)
|
||||
|
||||
### 5. Sync Engine (WebSocket)
|
||||
- Implements SimpleSync protocol (see spec)
|
||||
- Bidirectional: server pushes changes, clients request/pull
|
||||
- Content-addressed: files identified by SHA-256 hash
|
||||
- Conflict detection: three-way merge or manual resolution
|
||||
- Offline queue: client stores pending changes in IndexedDB
|
||||
|
||||
### 6. Search (FTS5)
|
||||
- SQLite FTS5 virtual table indexes document content
|
||||
- Server provides search API for initial load
|
||||
- Client syncs index to IndexedDB for offline search
|
||||
- Index rebuilds incrementally on file changes
|
||||
|
||||
### 7. Email (Postmark)
|
||||
- Go adapter using Postmark HTTP API
|
||||
- Plain-text emails only (no HTML)
|
||||
- Threading via Message-ID / In-Reply-To headers
|
||||
- Webhook endpoint receives email replies, converts to comments
|
||||
- Templates: notification, digest, conflict alert
|
||||
|
||||
### 8. File Store (Content-Addressed)
|
||||
- Files stored by SHA-256 hash: `store/ab/cd/abcdef1234...`
|
||||
- Immutable: once written, never modified
|
||||
- Deduplication: identical content stored once
|
||||
- Metadata in database maps filenames to current hash
|
||||
- Attachments served with strict content-type validation
|
||||
|
||||
## Data Flow: Document Rendering
|
||||
|
||||
```
|
||||
1. Request: GET /docs/getting-started
|
||||
│
|
||||
▼
|
||||
2. Auth Middleware: Check session, verify read permission
|
||||
│
|
||||
▼
|
||||
3. Document Lookup: Query DB for current hash of "getting-started.md"
|
||||
│
|
||||
▼
|
||||
4. Cache Check: Is rendered HTML in cache? (TTL: 1 minute for dynamic)
|
||||
│
|
||||
├── Yes → Serve cached HTML
|
||||
│
|
||||
└── No → Continue
|
||||
│
|
||||
▼
|
||||
5. File Read: Read content from `store/ab/cd/...hash...`
|
||||
│
|
||||
▼
|
||||
6. Markdown Parse: Goldmark with extensions → AST
|
||||
│
|
||||
▼
|
||||
7. Transform AST: Resolve wiki-links, extract tags, validate admonitions
|
||||
│
|
||||
▼
|
||||
8. Render HTML: AST → html/template with layout
|
||||
│
|
||||
▼
|
||||
9. Response: HTML with CSP nonce, Preact hydration marker
|
||||
```
|
||||
|
||||
## Data Flow: File Sync
|
||||
|
||||
```
|
||||
User saves file on laptop
|
||||
│
|
||||
▼
|
||||
File Watcher detects change
|
||||
│
|
||||
▼
|
||||
Compute SHA-256 hash
|
||||
│
|
||||
▼
|
||||
Store in content-addressed filesystem
|
||||
│
|
||||
▼
|
||||
Update database: filename → new hash
|
||||
│
|
||||
▼
|
||||
Broadcast to WebSocket subscribers
|
||||
│
|
||||
▼
|
||||
Connected clients receive update notification
|
||||
│
|
||||
▼
|
||||
Client requests new content by hash (if not cached)
|
||||
│
|
||||
▼
|
||||
Server sends file content
|
||||
│
|
||||
▼
|
||||
Client verifies hash, updates UI
|
||||
```
|
||||
|
||||
## Database Schema (Simplified)
|
||||
|
||||
```sql
|
||||
-- Documents: current state and metadata
|
||||
CREATE TABLE documents (
|
||||
id TEXT PRIMARY KEY,
|
||||
path TEXT UNIQUE NOT NULL, -- relative path like "getting-started.md"
|
||||
current_hash TEXT NOT NULL, -- SHA-256 of current content
|
||||
title TEXT NOT NULL,
|
||||
created_at DATETIME NOT NULL,
|
||||
updated_at DATETIME NOT NULL,
|
||||
updated_by TEXT REFERENCES users(id),
|
||||
permission_read TEXT NOT NULL DEFAULT 'public', -- public, authenticated, private
|
||||
permission_write TEXT NOT NULL DEFAULT 'authenticated'
|
||||
);
|
||||
|
||||
-- Document versions: immutable history
|
||||
CREATE TABLE document_versions (
|
||||
id TEXT PRIMARY KEY,
|
||||
document_id TEXT NOT NULL REFERENCES documents(id),
|
||||
hash TEXT NOT NULL, -- content hash
|
||||
previous_hash TEXT, -- for diffing
|
||||
created_at DATETIME NOT NULL,
|
||||
created_by TEXT REFERENCES users(id),
|
||||
change_summary TEXT, -- auto-generated or user-provided
|
||||
signature TEXT -- optional Ed25519 signature
|
||||
);
|
||||
|
||||
-- Users: minimal, no PII beyond email
|
||||
CREATE TABLE users (
|
||||
id TEXT PRIMARY KEY,
|
||||
email TEXT UNIQUE NOT NULL,
|
||||
display_name TEXT,
|
||||
password_hash TEXT, -- NULL if passkey-only
|
||||
passkey_credential_id TEXT, -- WebAuthn credential
|
||||
created_at DATETIME NOT NULL,
|
||||
last_seen_at DATETIME
|
||||
);
|
||||
|
||||
-- Sessions: server-side state
|
||||
CREATE TABLE sessions (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES users(id),
|
||||
token_hash TEXT UNIQUE NOT NULL, -- SHA-256 of session token
|
||||
created_at DATETIME NOT NULL,
|
||||
expires_at DATETIME NOT NULL,
|
||||
ip_address TEXT,
|
||||
user_agent TEXT
|
||||
);
|
||||
|
||||
-- Comments: linked to specific versions
|
||||
CREATE TABLE comments (
|
||||
id TEXT PRIMARY KEY,
|
||||
document_id TEXT NOT NULL REFERENCES documents(id),
|
||||
version_hash TEXT NOT NULL, -- which version this comments on
|
||||
parent_id TEXT REFERENCES comments(id), -- threading
|
||||
author_id TEXT NOT NULL REFERENCES users(id),
|
||||
content TEXT NOT NULL,
|
||||
created_at DATETIME NOT NULL,
|
||||
resolved_at DATETIME,
|
||||
resolved_by TEXT REFERENCES users(id)
|
||||
);
|
||||
|
||||
-- Notifications: user preferences and queue
|
||||
CREATE TABLE notification_settings (
|
||||
user_id TEXT PRIMARY KEY REFERENCES users(id),
|
||||
global_enabled BOOLEAN NOT NULL DEFAULT true,
|
||||
digest_mode TEXT CHECK(digest_mode IN ('instant', 'hourly', 'daily')) DEFAULT 'instant',
|
||||
email_enabled BOOLEAN NOT NULL DEFAULT true
|
||||
);
|
||||
|
||||
CREATE TABLE document_watchers (
|
||||
user_id TEXT REFERENCES users(id),
|
||||
document_id TEXT REFERENCES documents(id),
|
||||
folder_path TEXT, -- watch entire folder
|
||||
created_at DATETIME NOT NULL,
|
||||
PRIMARY KEY (user_id, document_id, folder_path)
|
||||
);
|
||||
|
||||
-- FTS5 Search Index
|
||||
CREATE VIRTUAL TABLE document_search USING fts5(
|
||||
content,
|
||||
document_id UNINDEXED,
|
||||
tokenize='porter'
|
||||
);
|
||||
|
||||
-- Audit Log: everything
|
||||
CREATE TABLE audit_log (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
occurred_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
actor_id TEXT REFERENCES users(id),
|
||||
action TEXT NOT NULL, -- read, write, delete, login, logout, sync
|
||||
resource_type TEXT NOT NULL, -- document, user, comment, etc.
|
||||
resource_id TEXT NOT NULL,
|
||||
details TEXT, -- JSON metadata
|
||||
ip_address TEXT
|
||||
);
|
||||
```
|
||||
|
||||
## Deployment Architecture
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml (simplified)
|
||||
version: "3.8"
|
||||
services:
|
||||
app:
|
||||
build: .
|
||||
ports:
|
||||
- "8080:8080"
|
||||
volumes:
|
||||
- ./data:/data # Database + attachments
|
||||
- ./notes:/notes:ro # Source markdown files (optional)
|
||||
environment:
|
||||
- DB_PATH=/data/db.sqlite
|
||||
- FILESTORE_PATH=/data/files
|
||||
- NOTES_PATH=/notes
|
||||
- POSTMARK_API_KEY=${POSTMARK_API_KEY}
|
||||
- DOMAIN=${DOMAIN}
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "--spider", "http://localhost:8080/health"]
|
||||
```
|
||||
|
||||
## Scaling Considerations
|
||||
|
||||
This architecture is intentionally single-node:
|
||||
|
||||
- **SQLite** handles read-heavy workloads excellently; write concurrency managed via WAL mode
|
||||
- **Content-addressed files** are immutable and cache-friendly
|
||||
- **Server-rendered HTML** reduces client compute
|
||||
|
||||
If horizontal scaling becomes necessary:
|
||||
1. libsql replication (read replicas)
|
||||
2. NFS or object storage for content-addressed files
|
||||
3. Load balancer with sticky sessions for WebSocket
|
||||
|
||||
These are documented as future enhancements, not current requirements.
|
||||
118
.project/milestones/milestone-01-foundation.md
Normal file
118
.project/milestones/milestone-01-foundation.md
Normal file
@@ -0,0 +1,118 @@
|
||||
# Milestone 1: Foundation
|
||||
|
||||
**Duration:** 2 weeks
|
||||
**Goal:** Working server that can serve markdown files as HTML
|
||||
|
||||
## Tasks
|
||||
|
||||
### Week 1: Project Setup & Infrastructure
|
||||
|
||||
- [ ] Initialize monorepo structure
|
||||
- `apps/server/` with Go module
|
||||
- `apps/web/` with Vite + Preact + TypeScript
|
||||
- `packages/protocol/` with protobuf schema
|
||||
- `docs/` with existing documentation
|
||||
|
||||
- [ ] Go server skeleton
|
||||
- `main.go` with graceful shutdown
|
||||
- Chi router with middleware chain
|
||||
- Configuration management (environment variables + config file)
|
||||
- Structured logging (slog)
|
||||
|
||||
- [ ] Database layer
|
||||
- libsql connection setup
|
||||
- Migration system (golang-migrate)
|
||||
- Initial schema (users, documents, document_versions)
|
||||
- Query builder / repository pattern
|
||||
|
||||
- [ ] Docker setup
|
||||
- Multi-stage Dockerfile for Go app
|
||||
- `docker-compose.yml` with volume mounts
|
||||
- Health check endpoint
|
||||
- Non-root user in container
|
||||
|
||||
### Week 2: Markdown Rendering & Static Serving
|
||||
|
||||
- [ ] Markdown parser integration
|
||||
- Goldmark with all extensions
|
||||
- Wiki-link parser: `[[Page Name]]` → internal link
|
||||
- Tag extractor: `#tagname` → tag link
|
||||
- Admonition support: `> [!NOTE]` blocks
|
||||
- Mermaid diagram containers
|
||||
- Math markup containers (KaTeX)
|
||||
|
||||
- [ ] Template system
|
||||
- Base layout template (header, nav, footer)
|
||||
- Document template with rendered markdown
|
||||
- Error page templates (404, 500)
|
||||
- Design system CSS custom properties
|
||||
|
||||
- [ ] Static file serving
|
||||
- Content-addressed filesystem setup
|
||||
- File upload endpoint (basic)
|
||||
- Security headers middleware
|
||||
- MIME type detection
|
||||
|
||||
- [ ] Initial frontend
|
||||
- Preact app setup with Vite
|
||||
- Basic routing (preact-iso)
|
||||
- Hydration entry point
|
||||
- CSS design system foundation
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
### Functional
|
||||
- [ ] Server starts and listens on configurable port (default 8080)
|
||||
- [ ] Database auto-migrates on startup with no manual intervention
|
||||
- [ ] Markdown files render correctly with all extensions:
|
||||
- [ ] Wiki-links resolve to internal document paths
|
||||
- [ ] Tags are extracted and displayed
|
||||
- [ ] Admonitions render with appropriate styling
|
||||
- [ ] Mermaid diagrams have syntax-highlighted containers
|
||||
- [ ] Math markup has proper delimiters for KaTeX
|
||||
- [ ] Attachments served with correct Content-Type headers
|
||||
- [ ] 404 and 500 pages render correctly
|
||||
|
||||
### Non-Functional
|
||||
- [ ] Docker compose brings up full stack with `docker-compose up -d`
|
||||
- [ ] Container health check passes within 30 seconds
|
||||
- [ ] All Go dependencies pinned in go.sum
|
||||
- [ ] Build is reproducible (same commit → same binary hash)
|
||||
- [ ] No plaintext secrets in code or logs
|
||||
- [ ] Security headers present on all responses:
|
||||
- [ ] Content-Security-Policy
|
||||
- [ ] X-Content-Type-Options: nosniff
|
||||
- [ ] X-Frame-Options: DENY
|
||||
- [ ] Referrer-Policy: strict-origin-when-cross-origin
|
||||
|
||||
### Performance
|
||||
- [ ] Initial page load (TTFB) <100ms for cached documents
|
||||
- [ ] Markdown rendering <50ms for documents <100KB
|
||||
- [ ] Static file serving supports 1000 concurrent requests
|
||||
|
||||
## Deliverables
|
||||
|
||||
1. Working development environment (README with setup instructions)
|
||||
2. CI pipeline (GitHub Actions or similar) with:
|
||||
- Go tests
|
||||
- TypeScript type checking
|
||||
- Dockerfile build
|
||||
- Security scan (govulncheck)
|
||||
3. API documentation (OpenAPI spec)
|
||||
4. Database migration files
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| Goldmark extension conflicts | Test each extension in isolation, then combined |
|
||||
| Docker volume permissions | Document UID/GID requirements, provide setup script |
|
||||
| libsql compatibility | Test on both native SQLite and Turso cloud |
|
||||
|
||||
## Definition of Done
|
||||
|
||||
- [ ] All acceptance criteria pass
|
||||
- [ ] Code review completed
|
||||
- [ ] Documentation updated
|
||||
- [ ] CI pipeline green
|
||||
- [ ] Security scan shows no critical/high vulnerabilities
|
||||
143
.project/milestones/milestone-02-sync-protocol.md
Normal file
143
.project/milestones/milestone-02-sync-protocol.md
Normal file
@@ -0,0 +1,143 @@
|
||||
# Milestone 2: Sync Protocol
|
||||
|
||||
**Duration:** 2 weeks
|
||||
**Goal:** Bidirectional file sync between server and filesystem/web client
|
||||
|
||||
## Tasks
|
||||
|
||||
### Week 3: Server-Side Sync
|
||||
|
||||
- [ ] Content-addressed filesystem implementation
|
||||
- SHA-256 hash computation
|
||||
- Directory layout: `store/ab/cd/abcdef...`
|
||||
- Write-once semantics
|
||||
- Deduplication verification
|
||||
|
||||
- [ ] File watcher (fsnotify)
|
||||
- Watch configured directory recursively
|
||||
- Debounce rapid changes (1s window)
|
||||
- Ignore patterns (.git, temp files)
|
||||
- Compute hash on change, update database
|
||||
|
||||
- [ ] WebSocket server
|
||||
- Connection management (connection pool)
|
||||
- Message framing and parsing
|
||||
- Authentication over WebSocket (token validation)
|
||||
- Heartbeat/ping-pong for connection health
|
||||
|
||||
- [ ] SimpleSync protocol server implementation
|
||||
- Sync request/response handler
|
||||
- Root hash computation (Merkle tree)
|
||||
- Missing content detection
|
||||
- Conflict detection logic
|
||||
|
||||
### Week 4: Client-Side Sync & Offline Support
|
||||
|
||||
- [ ] WebSocket client (Preact)
|
||||
- Connection management with auto-reconnect
|
||||
- Message queue for offline periods
|
||||
- Exponential backoff for reconnection
|
||||
|
||||
- [ ] IndexedDB cache layer
|
||||
- Store file content by hash
|
||||
- Store sync state and pending changes
|
||||
- LRU eviction for cache management
|
||||
|
||||
- [ ] File sync UI
|
||||
- Sync status indicator
|
||||
- Offline mode banner
|
||||
- Pending changes list
|
||||
- Manual sync trigger button
|
||||
|
||||
- [ ] Conflict detection display
|
||||
- Visual indicator for conflicting files
|
||||
- Basic conflict resolution UI (choose local/server)
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
### Functional
|
||||
- [ ] File changes on disk automatically sync to server within 5 seconds
|
||||
- [ ] Web client receives real-time updates when files change on server
|
||||
- [ ] Client can make changes offline and sync when reconnected
|
||||
- [ ] Concurrent edits to different files succeed without conflict
|
||||
- [ ] Concurrent edits to same file detected as conflict
|
||||
- [ ] Content hash verified on every transfer (client and server)
|
||||
- [ ] Sync works across browser refresh (IndexedDB persistence)
|
||||
|
||||
### Non-Functional
|
||||
- [ ] WebSocket connections authenticated (reject unauthenticated)
|
||||
- [ ] File watcher doesn't consume >1% CPU on idle
|
||||
- [ ] IndexedDB storage cleared on logout
|
||||
- [ ] Sync protocol documented with sequence diagrams
|
||||
|
||||
### Performance
|
||||
- [ ] Sync of 100 files completes in <10 seconds
|
||||
- [ ] WebSocket message overhead <1KB per sync cycle
|
||||
- [ ] File watcher detects changes within 500ms
|
||||
|
||||
## Protocol Specification
|
||||
|
||||
### Message Types
|
||||
|
||||
```typescript
|
||||
// Client → Server
|
||||
interface SyncRequest {
|
||||
type: 'sync_request';
|
||||
clientId: string;
|
||||
rootHash: string;
|
||||
files: Record<string, string>; // path → hash
|
||||
}
|
||||
|
||||
// Server → Client
|
||||
interface SyncResponse {
|
||||
type: 'sync_response';
|
||||
serverRootHash: string;
|
||||
missingFromClient: string[]; // hashes to download
|
||||
missingFromServer: string[]; // hashes to upload
|
||||
conflicts: string[]; // file paths with conflicts
|
||||
}
|
||||
|
||||
// Content request
|
||||
interface ContentRequest {
|
||||
type: 'content_request';
|
||||
hash: string;
|
||||
}
|
||||
|
||||
// Content response
|
||||
interface ContentResponse {
|
||||
type: 'content_response';
|
||||
hash: string;
|
||||
content: string; // base64 encoded
|
||||
}
|
||||
```
|
||||
|
||||
### State Machine
|
||||
|
||||
```
|
||||
[Idle] --sync_request--> [Comparing] --has_differences--> [Transferring] --complete--> [Idle]
|
||||
--no_differences--> [Idle]
|
||||
[Idle] --file_change--> [LocalChange] --sync--> [Comparing]
|
||||
```
|
||||
|
||||
## Deliverables
|
||||
|
||||
1. SimpleSync protocol specification document
|
||||
2. Sequence diagrams for all sync scenarios
|
||||
3. WebSocket API documentation
|
||||
4. Offline sync test suite
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| WebSocket connection drops frequently | Auto-reconnect with exponential backoff, queue offline |
|
||||
| Large files cause sync delays | Chunked transfer, progress indicators, size limits |
|
||||
| Conflicts in frequently edited files | Optimistic UI, clear conflict resolution, email notification |
|
||||
|
||||
## Definition of Done
|
||||
|
||||
- [ ] All acceptance criteria pass
|
||||
- [ ] Sync protocol tested with 2+ concurrent clients
|
||||
- [ ] Offline/online transition tested
|
||||
- [ ] Performance benchmarks meet targets
|
||||
- [ ] Documentation complete
|
||||
134
.project/milestones/milestone-03-authentication.md
Normal file
134
.project/milestones/milestone-03-authentication.md
Normal file
@@ -0,0 +1,134 @@
|
||||
# Milestone 3: Authentication & Authorization
|
||||
|
||||
**Duration:** 2 weeks
|
||||
**Goal:** Secure access control with passkeys and granular permissions
|
||||
|
||||
## Tasks
|
||||
|
||||
### Week 5: Passkey Authentication
|
||||
|
||||
- [ ] WebAuthn server implementation
|
||||
- Relying party configuration
|
||||
- Challenge generation and storage
|
||||
- Attestation verification
|
||||
- Credential storage (credential ID, public key, sign count)
|
||||
|
||||
- [ ] Passkey registration flow
|
||||
- Initiate registration endpoint
|
||||
- Verify registration response
|
||||
- Store credential with user association
|
||||
- Support multiple passkeys per user
|
||||
|
||||
- [ ] Passkey authentication flow
|
||||
- Initiate login endpoint
|
||||
- Verify assertion response
|
||||
- Create session on success
|
||||
|
||||
- [ ] Session management
|
||||
- Signed cookie generation
|
||||
- Session storage in database
|
||||
- Session validation middleware
|
||||
- Session revocation endpoint
|
||||
- Automatic session refresh
|
||||
|
||||
### Week 6: Password Fallback & Authorization
|
||||
|
||||
- [ ] Password authentication
|
||||
- Argon2id password hashing
|
||||
- Password validation endpoint
|
||||
- Password change endpoint
|
||||
- Password strength requirements
|
||||
|
||||
- [ ] Role-based access control
|
||||
- Permission model: read, write, admin
|
||||
- Scope: global, collection, document
|
||||
- Middleware for permission checking
|
||||
- Admin UI for managing permissions
|
||||
|
||||
- [ ] User management
|
||||
- User registration (email + passkey/password)
|
||||
- User profile management
|
||||
- Account deletion (GDPR compliance)
|
||||
- User listing (admin only)
|
||||
|
||||
- [ ] Content signing (optional)
|
||||
- Ed25519 key pair generation
|
||||
- Document signing on publish
|
||||
- Signature verification display
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
### Functional
|
||||
- [ ] Users can register with passkey on supported browsers
|
||||
- [ ] Passkey login works with Touch ID, Windows Hello, YubiKey
|
||||
- [ ] Password fallback works when passkey unavailable
|
||||
- [ ] Sessions expire after configurable timeout (default 24h)
|
||||
- [ ] Sessions can be revoked (logout all devices)
|
||||
- [ ] Read/write/admin permissions enforced on all endpoints
|
||||
- [ ] Users can only access documents they have permission for
|
||||
- [ ] Admin users can manage other users' permissions
|
||||
|
||||
### Non-Functional
|
||||
- [ ] Argon2id parameters: time=3, memory=64MB, parallelism=4
|
||||
- [ ] Session cookies: HttpOnly, Secure, SameSite=Strict
|
||||
- [ ] Rate limiting: 5 auth attempts per minute per IP
|
||||
- [ ] No timing attacks on password comparison (constant-time)
|
||||
- [ ] All auth events logged to audit_log table
|
||||
- [ ] Ed25519 signatures verified on document read (if enabled)
|
||||
|
||||
### Security
|
||||
- [ ] WebAuthn challenges single-use and time-limited (5 minutes)
|
||||
- [ ] Credential IDs are unpredictable (128-bit random)
|
||||
- [ ] Password reset requires email verification
|
||||
- [ ] No enumeration attacks (same response for exist/non-exist user)
|
||||
|
||||
## Database Schema Additions
|
||||
|
||||
```sql
|
||||
-- WebAuthn credentials
|
||||
CREATE TABLE webauthn_credentials (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES users(id),
|
||||
credential_id BLOB NOT NULL UNIQUE,
|
||||
public_key BLOB NOT NULL,
|
||||
sign_count INTEGER NOT NULL DEFAULT 0,
|
||||
created_at DATETIME NOT NULL,
|
||||
last_used_at DATETIME
|
||||
);
|
||||
|
||||
-- Permissions
|
||||
CREATE TABLE permissions (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES users(id),
|
||||
resource_type TEXT NOT NULL CHECK(resource_type IN ('global', 'collection', 'document')),
|
||||
resource_id TEXT, -- NULL for global
|
||||
permission TEXT NOT NULL CHECK(permission IN ('read', 'write', 'admin')),
|
||||
granted_by TEXT REFERENCES users(id),
|
||||
created_at DATETIME NOT NULL
|
||||
);
|
||||
```
|
||||
|
||||
## Deliverables
|
||||
|
||||
1. Authentication API documentation
|
||||
2. WebAuthn flow diagrams
|
||||
3. Permission system documentation
|
||||
4. Security test results (penetration test guide)
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| Passkey not supported on user's device | Password fallback always available |
|
||||
| User loses all passkeys | Recovery email + backup codes |
|
||||
| Permission system too complex | Start with simple roles, iterate |
|
||||
| Session hijacking | Short expiry, rotation, revocation |
|
||||
|
||||
## Definition of Done
|
||||
|
||||
- [ ] All acceptance criteria pass
|
||||
- [ ] Security review completed
|
||||
- [ ] Auth flow tested on Chrome, Firefox, Safari, Edge
|
||||
- [ ] Mobile passkey tested (iOS, Android)
|
||||
- [ ] Rate limiting verified
|
||||
- [ ] Audit logs verified
|
||||
171
.project/milestones/milestone-04-collaboration.md
Normal file
171
.project/milestones/milestone-04-collaboration.md
Normal file
@@ -0,0 +1,171 @@
|
||||
# Milestone 4: Collaboration Features
|
||||
|
||||
**Duration:** 2 weeks
|
||||
**Goal:** Comments, notifications, email integration, and conflict resolution
|
||||
|
||||
## Tasks
|
||||
|
||||
### Week 7: Comments & Notifications
|
||||
|
||||
- [ ] Comment system
|
||||
- Comment creation endpoint (web)
|
||||
- Comment threading (parent/child)
|
||||
- Line/section anchoring (hash-based)
|
||||
- Comment resolution (mark as resolved)
|
||||
- Comment display in Preact UI
|
||||
|
||||
- [ ] Notification system
|
||||
- Notification queue in database
|
||||
- Per-file watcher subscriptions
|
||||
- Per-folder watcher subscriptions
|
||||
- Global notification settings
|
||||
- Digest mode (hourly/daily batches)
|
||||
|
||||
- [ ] Web notification UI
|
||||
- Notification bell with unread count
|
||||
- Notification list dropdown
|
||||
- Mark as read functionality
|
||||
- Notification preferences page
|
||||
|
||||
### Week 8: Email Integration & Conflict Resolution
|
||||
|
||||
- [ ] Postmark integration
|
||||
- Go adapter setup
|
||||
- Plain-text email templates
|
||||
- Outgoing email queue with retry
|
||||
- Webhook endpoint for inbound email
|
||||
|
||||
- [ ] Email notification types
|
||||
- File changed notification
|
||||
- New comment notification
|
||||
- Mention notification (@username)
|
||||
- Conflict alert notification
|
||||
- Digest summary email
|
||||
|
||||
[ ] Email reply handling
|
||||
- Parse inbound email (from, subject, body)
|
||||
- Match to comment thread via In-Reply-To
|
||||
- Create comment from email body
|
||||
- Validate sender permission
|
||||
|
||||
- [ ] Conflict resolution UI
|
||||
- Side-by-side diff view
|
||||
- Accept local/server/merge options
|
||||
- Manual merge editor
|
||||
- Conflict notification email
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
### Functional
|
||||
- [ ] Users can add comments to specific lines in a document
|
||||
- [ ] Comments are linked to document version (hash)
|
||||
[ ] Comment threading works (reply to reply)
|
||||
- [ ] Users can watch files or folders for changes
|
||||
- [ ] Email notifications sent within 1 minute of event
|
||||
- [ ] Replying to notification email creates a comment
|
||||
- [ ] Digest emails batch notifications by time window
|
||||
- [ ] Conflicts display side-by-side diff
|
||||
- [ ] Users can resolve conflicts via web UI
|
||||
- [ ] Resolved comments are hidden but accessible in history
|
||||
|
||||
### Non-Functional
|
||||
- [ ] Emails are plain-text only (no HTML)
|
||||
- [ ] Email threading works in Gmail, Outlook, Apple Mail
|
||||
- [ ] Webhook signature verified for inbound email
|
||||
- [ ] Failed emails retried 3 times with exponential backoff
|
||||
- [ ] Email queue doesn't block web requests (async processing)
|
||||
|
||||
### Email Template Examples
|
||||
|
||||
**File Change Notification:**
|
||||
```
|
||||
Subject: [docs/api-reference.md] Updated by Alice
|
||||
|
||||
Alice updated "API Reference" at 2024-01-15 14:30 UTC.
|
||||
|
||||
Changed sections:
|
||||
- Authentication
|
||||
- Rate Limiting
|
||||
|
||||
View changes: https://example.com/docs/api-reference?v=abc123
|
||||
Unsubscribe: https://example.com/unsubscribe/123
|
||||
```
|
||||
|
||||
**Comment Notification:**
|
||||
```
|
||||
Subject: [docs/getting-started.md] Comment from Bob
|
||||
|
||||
Bob commented on "Getting Started":
|
||||
|
||||
> This step is unclear. Can we add an example?
|
||||
|
||||
Reply to this email to respond.
|
||||
|
||||
View online: https://example.com/docs/getting-started#comment-456
|
||||
```
|
||||
|
||||
## Database Schema Additions
|
||||
|
||||
```sql
|
||||
-- Comments
|
||||
CREATE TABLE comments (
|
||||
id TEXT PRIMARY KEY,
|
||||
document_id TEXT NOT NULL REFERENCES documents(id),
|
||||
version_hash TEXT NOT NULL,
|
||||
parent_id TEXT REFERENCES comments(id),
|
||||
author_id TEXT NOT NULL REFERENCES users(id),
|
||||
content TEXT NOT NULL,
|
||||
anchor_line INTEGER, -- line number for anchoring
|
||||
anchor_hash TEXT, -- hash of specific content for anchoring
|
||||
created_at DATETIME NOT NULL,
|
||||
resolved_at DATETIME,
|
||||
resolved_by TEXT REFERENCES users(id)
|
||||
);
|
||||
|
||||
-- Notifications
|
||||
CREATE TABLE notifications (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES users(id),
|
||||
type TEXT NOT NULL, -- file_changed, comment, mention, conflict
|
||||
resource_type TEXT NOT NULL,
|
||||
resource_id TEXT NOT NULL,
|
||||
message TEXT NOT NULL,
|
||||
read_at DATETIME,
|
||||
emailed_at DATETIME,
|
||||
created_at DATETIME NOT NULL
|
||||
);
|
||||
|
||||
-- Watcher subscriptions
|
||||
CREATE TABLE watchers (
|
||||
user_id TEXT REFERENCES users(id),
|
||||
document_id TEXT REFERENCES documents(id),
|
||||
folder_path TEXT,
|
||||
created_at DATETIME NOT NULL,
|
||||
PRIMARY KEY (user_id, document_id, folder_path)
|
||||
);
|
||||
```
|
||||
|
||||
## Deliverables
|
||||
|
||||
1. Email template documentation
|
||||
2. Webhook integration guide
|
||||
3. Comment API documentation
|
||||
4. Conflict resolution flow diagrams
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| Email deliverability issues | Postmark reputation monitoring, SPF/DKIM setup |
|
||||
| Comment spam | Rate limiting, auth required, moderation tools |
|
||||
| Email parsing errors | Robust parser, fallback to plain text extraction |
|
||||
| Conflict resolution UI confusion | Clear visual design, undo option, help text |
|
||||
|
||||
## Definition of Done
|
||||
|
||||
- [ ] All acceptance criteria pass
|
||||
- [ ] Email flows tested with real Postmark account
|
||||
- [ ] Comment threading tested with 3+ levels
|
||||
- [ ] Conflict resolution tested with 2+ concurrent editors
|
||||
- [ ] Notification preferences persist across sessions
|
||||
- [ ] Digest mode tested with 24-hour window
|
||||
195
.project/milestones/milestone-05-search-ui.md
Normal file
195
.project/milestones/milestone-05-search-ui.md
Normal file
@@ -0,0 +1,195 @@
|
||||
# Milestone 5: Search & UI Polish
|
||||
|
||||
**Duration:** 2 weeks
|
||||
**Goal:** Fast search, design system, responsive layout, and performance optimization
|
||||
|
||||
## Tasks
|
||||
|
||||
### Week 9: Search Implementation
|
||||
|
||||
- [ ] Server-side search index
|
||||
- FTS5 virtual table setup
|
||||
- Index population from documents
|
||||
- Incremental index updates on file changes
|
||||
- Search query endpoint
|
||||
|
||||
- [ ] Client-side search
|
||||
- Flexsearch or Pagefind integration
|
||||
- Index sync from server on load
|
||||
- Incremental index updates via WebSocket
|
||||
- Search UI (input, results, filters)
|
||||
|
||||
- [ ] Search features
|
||||
- Full-text content search
|
||||
- Tag filtering
|
||||
- Title search (boosted)
|
||||
- Recent results caching
|
||||
- Search suggestions (autocomplete)
|
||||
|
||||
### Week 10: Design System & Performance
|
||||
|
||||
- [ ] Design system
|
||||
- CSS custom properties for theming
|
||||
- Component library (buttons, inputs, cards, navigation)
|
||||
- Typography scale
|
||||
- Color system (light/dark mode)
|
||||
- Spacing scale
|
||||
- `/design` route for browser-based design tool
|
||||
|
||||
- [ ] Responsive layout
|
||||
- Mobile-first CSS
|
||||
- Container queries for components
|
||||
- Navigation adapts to screen size
|
||||
- Touch-friendly targets (min 44px)
|
||||
- Print styles for documents
|
||||
|
||||
- [ ] Performance optimization
|
||||
- Bundle analysis and optimization
|
||||
- Lazy loading for non-critical components
|
||||
- Image optimization (if applicable)
|
||||
- CSS critical path extraction
|
||||
- Service worker for offline caching
|
||||
|
||||
- [ ] Accessibility
|
||||
- ARIA labels on interactive elements
|
||||
- Keyboard navigation
|
||||
- Focus management
|
||||
- Color contrast compliance (WCAG AA)
|
||||
- Screen reader testing
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
### Functional
|
||||
- [ ] Search returns results in <50ms after initial sync
|
||||
- [ ] Search works offline after first load
|
||||
- [ ] Search supports exact phrases (quoted)
|
||||
- [ ] Tag filtering narrows search results
|
||||
- [ ] `/design` route displays all components with editable CSS
|
||||
- [ ] Dark mode toggles correctly (no flash)
|
||||
- [ ] All pages responsive down to 320px width
|
||||
- [ ] Keyboard navigation works throughout app
|
||||
|
||||
### Non-Functional
|
||||
- [ ] Lighthouse score >90 on all metrics
|
||||
- [ ] Bundle size <100KB gzipped (JS + CSS)
|
||||
- [ ] Time to Interactive <200ms
|
||||
- [ ] No layout shift during hydration
|
||||
- [ ] CSS custom properties work without JS
|
||||
|
||||
### Performance Targets
|
||||
|
||||
| Metric | Target | Measurement |
|
||||
|--------|--------|-------------|
|
||||
| Initial Page Load | <100ms | TTFB + FCP |
|
||||
| Time to Interactive | <200ms | Hydration complete |
|
||||
| Search Query | <50ms | Client-side after sync |
|
||||
| Bundle Size | <100KB | Gzipped JS + CSS |
|
||||
| Lighthouse Performance | >90 | Chrome DevTools |
|
||||
| Accessibility | >90 | Lighthouse a11y |
|
||||
|
||||
## Design System Specification
|
||||
|
||||
### CSS Custom Properties
|
||||
|
||||
```css
|
||||
:root {
|
||||
/* Colors */
|
||||
--color-primary: #2563eb;
|
||||
--color-primary-hover: #1d4ed8;
|
||||
--color-text: #1f2937;
|
||||
--color-text-muted: #6b7280;
|
||||
--color-background: #ffffff;
|
||||
--color-surface: #f9fafb;
|
||||
--color-border: #e5e7eb;
|
||||
|
||||
/* Typography */
|
||||
--font-sans: system-ui, -apple-system, sans-serif;
|
||||
--font-mono: ui-monospace, monospace;
|
||||
--text-sm: 0.875rem;
|
||||
--text-base: 1rem;
|
||||
--text-lg: 1.125rem;
|
||||
--text-xl: 1.25rem;
|
||||
|
||||
/* Spacing */
|
||||
--space-1: 0.25rem;
|
||||
--space-2: 0.5rem;
|
||||
--space-4: 1rem;
|
||||
--space-8: 2rem;
|
||||
|
||||
/* Shadows */
|
||||
--shadow-sm: 0 1px 2px rgba(0,0,0,0.05);
|
||||
--shadow-md: 0 4px 6px rgba(0,0,0,0.1);
|
||||
|
||||
/* Radii */
|
||||
--radius-sm: 0.25rem;
|
||||
--radius-md: 0.5rem;
|
||||
}
|
||||
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--color-text: #f9fafb;
|
||||
--color-text-muted: #9ca3af;
|
||||
--color-background: #111827;
|
||||
--color-surface: #1f2937;
|
||||
--color-border: #374151;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Component Examples
|
||||
|
||||
**Button:**
|
||||
```html
|
||||
<button class="btn btn-primary">Click me</button>
|
||||
```
|
||||
|
||||
```css
|
||||
.btn {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: var(--space-2);
|
||||
padding: var(--space-2) var(--space-4);
|
||||
border-radius: var(--radius-md);
|
||||
font-size: var(--text-base);
|
||||
font-weight: 500;
|
||||
cursor: pointer;
|
||||
transition: all 0.2s;
|
||||
}
|
||||
|
||||
.btn-primary {
|
||||
background: var(--color-primary);
|
||||
color: white;
|
||||
border: none;
|
||||
}
|
||||
|
||||
.btn-primary:hover {
|
||||
background: var(--color-primary-hover);
|
||||
}
|
||||
```
|
||||
|
||||
## Deliverables
|
||||
|
||||
1. Design system documentation
|
||||
2. Component showcase at `/design`
|
||||
3. Performance audit report
|
||||
4. Accessibility audit report
|
||||
5. Search API documentation
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| Bundle size exceeds target | Tree-shaking, code splitting, lazy loading |
|
||||
| Search index too large for client | Pagination, server-side fallback, index pruning |
|
||||
| Design system inconsistent | CSS custom properties, component tests, design tokens |
|
||||
| Mobile layout issues | Test on real devices, container queries, touch targets |
|
||||
|
||||
## Definition of Done
|
||||
|
||||
- [ ] All acceptance criteria pass
|
||||
- [ ] Lighthouse audit green on all metrics
|
||||
- [ ] Manual testing on Chrome, Firefox, Safari, Edge
|
||||
- [ ] Mobile testing on iOS Safari and Android Chrome
|
||||
- [ ] Screen reader tested (VoiceOver, NVDA)
|
||||
- [ ] Performance benchmarks documented
|
||||
- [ ] Design system reviewed by team
|
||||
150
.project/milestones/milestone-06-production.md
Normal file
150
.project/milestones/milestone-06-production.md
Normal file
@@ -0,0 +1,150 @@
|
||||
# Milestone 6: Production Hardening
|
||||
|
||||
**Duration:** 2 weeks
|
||||
**Goal:** Deploy with confidence through testing, security audit, and operational documentation
|
||||
|
||||
## Tasks
|
||||
|
||||
### Week 11: Testing & Security
|
||||
|
||||
- [ ] Unit tests
|
||||
- Go: >80% coverage for all packages
|
||||
- Preact: Component testing with uvu
|
||||
- Mock external dependencies (email, filesystem)
|
||||
|
||||
- [ ] Integration tests
|
||||
- API endpoint testing
|
||||
- Database operation testing
|
||||
- Auth flow testing
|
||||
- Sync protocol testing
|
||||
|
||||
- [ ] End-to-end tests
|
||||
- User registration and login
|
||||
- Document creation and editing
|
||||
- Comment and notification flow
|
||||
- Conflict resolution
|
||||
- Search functionality
|
||||
|
||||
- [ ] Security audit
|
||||
- Dependency vulnerability scan (govulncheck, npm audit)
|
||||
- OWASP ZAP scan
|
||||
- Manual penetration testing guide
|
||||
- Rate limiting verification
|
||||
- Session security testing
|
||||
- File upload security testing
|
||||
|
||||
### Week 12: Operations & Documentation
|
||||
|
||||
- [ ] Backup and restore
|
||||
- Automated backup script (database + files)
|
||||
- Point-in-time recovery procedure
|
||||
- Backup encryption (optional)
|
||||
- Restore testing
|
||||
|
||||
- [ ] Monitoring
|
||||
- Health check endpoint (`/health`)
|
||||
- Metrics endpoint (`/metrics`) with Prometheus format
|
||||
- Error tracking and alerting
|
||||
- Log aggregation (structured JSON)
|
||||
|
||||
- [ ] Documentation
|
||||
- Installation guide
|
||||
- Upgrade procedures
|
||||
- Backup/restore procedures
|
||||
- Security incident response guide
|
||||
- Troubleshooting guide
|
||||
- API reference
|
||||
|
||||
- [ ] Load testing
|
||||
- 1000 concurrent readers
|
||||
- 100 concurrent editors
|
||||
- File sync stress test
|
||||
- Search performance under load
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
### Functional
|
||||
- [ ] >80% test coverage (Go + TypeScript)
|
||||
- [ ] All integration tests pass
|
||||
- [ ] All e2e tests pass
|
||||
- [ ] Backup script creates restorable archive
|
||||
- [ ] Restore procedure tested and documented
|
||||
- [ ] Health check verifies: database, filesystem, email connectivity
|
||||
|
||||
### Non-Functional
|
||||
- [ ] No critical or high-severity vulnerabilities
|
||||
- [ ] No medium-severity vulnerabilities without documented mitigations
|
||||
- [ ] Rate limiting prevents brute force attacks
|
||||
- [ ] File uploads cannot execute code
|
||||
- [ ] SQL injection impossible (parameterized queries only)
|
||||
- [ ] XSS prevented (auto-escaping, CSP)
|
||||
|
||||
### Performance
|
||||
- [ ] 1000 concurrent readers: p99 latency <500ms
|
||||
- [ ] 100 concurrent editors: no sync conflicts lost
|
||||
- [ ] Search performance: p99 <100ms under load
|
||||
- [ ] Memory usage stable over 24-hour test (no leaks)
|
||||
|
||||
## Security Checklist
|
||||
|
||||
### Authentication
|
||||
- [ ] Passkey registration works securely
|
||||
- [ ] Passkey login validates signature
|
||||
- [ ] Password hashing uses Argon2id
|
||||
- [ ] Session cookies are signed and HttpOnly
|
||||
- [ ] Session expiry is enforced
|
||||
- [ ] Rate limiting on auth endpoints
|
||||
|
||||
### Authorization
|
||||
- [ ] Permission checks on all endpoints
|
||||
- [ ] No horizontal privilege escalation
|
||||
- [ ] No vertical privilege escalation
|
||||
- [ ] Content only served to authorized users
|
||||
|
||||
### Input Validation
|
||||
- [ ] Markdown sanitized (no raw HTML injection)
|
||||
- [ ] File uploads validated (magic numbers, size)
|
||||
- [ ] Path traversal prevented
|
||||
- [ ] SQL injection impossible
|
||||
- [ ] XSS prevented
|
||||
|
||||
### Transport
|
||||
- [ ] TLS 1.3 enforced
|
||||
- [ ] HSTS header present
|
||||
- [ ] CSP header restricts scripts
|
||||
- [ ] CORS restrictive
|
||||
|
||||
### Operational
|
||||
- [ ] No secrets in logs
|
||||
- [ ] No secrets in environment (config file or encrypted)
|
||||
- [ ] Container runs as non-root
|
||||
- [ ] Read-only filesystem except /data
|
||||
- [ ] Resource limits configured
|
||||
|
||||
## Deliverables
|
||||
|
||||
1. Test suite (unit, integration, e2e)
|
||||
2. Security audit report
|
||||
3. Backup/restore scripts
|
||||
4. Monitoring dashboard configuration
|
||||
5. Operator documentation
|
||||
6. Load test results
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| Security vulnerability found | Patch immediately, assess impact, notify users |
|
||||
| Data loss | Automated backups, point-in-time recovery tested |
|
||||
| Performance degradation | Monitoring alerts, load testing, scaling plan |
|
||||
| Dependency vulnerability | Automated scans, rapid patching process |
|
||||
|
||||
## Definition of Done
|
||||
|
||||
- [ ] All acceptance criteria pass
|
||||
- [ ] Security audit signed off
|
||||
- [ ] Load tests meet targets
|
||||
- [ ] Documentation complete and reviewed
|
||||
- [ ] CI/CD pipeline green
|
||||
- [ ] Rollback procedure tested
|
||||
- [ ] Team trained on operations procedures
|
||||
9
.project/notes/discrepancies.md
Normal file
9
.project/notes/discrepancies.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# Documentation Discrepancies
|
||||
|
||||
## 2026-04-28
|
||||
|
||||
1. The plan and ADRs repeatedly describe a single Go binary, while Milestone 1 also requires `apps/web/` as a separate Vite/Preact app. The current implementation treats the web app as build-time static assets served by the Go binary.
|
||||
2. Milestone 1 asks for `packages/protocol/` with a protobuf schema, but the sync ADR and protocol spec describe JSON messages over WebSocket. The protocol package currently preserves both a `.proto` placeholder and TypeScript-facing JSON model notes.
|
||||
3. ADR-005 stores content under `data/files/...`, while Milestone 2 examples use `store/...`. The implementation follows ADR-005 and uses `data/files`.
|
||||
4. The frontend stack still requires a Node package install step, but local automation is now centralized in `mise` tasks and the lockfile-backed `pnpm` workflow rather than ad hoc commands.
|
||||
5. The foundation milestone specifies `golang-migrate`, but `go-libsql` integration constraints need a clean validation path. A lightweight internal migrator is used now so the project can move forward.
|
||||
7
.project/notes/problems.md
Normal file
7
.project/notes/problems.md
Normal file
@@ -0,0 +1,7 @@
|
||||
# Implementation Problems Log
|
||||
|
||||
## 2026-04-28
|
||||
|
||||
- `go-libsql` is the active official Go binding, but it requires CGO and precompiled native libraries. That weakens the "same source -> same binary hash" goal until the exact build environment is locked down.
|
||||
- The embedded-replica connector in `go-libsql` rejects an empty primary URL. The foundation therefore uses the local `file:` libsql driver path when no primary is configured, and reserves embedded-replica setup for later sync work.
|
||||
- Docker verification is currently blocked by the local environment because the daemon socket at `/Users/tim/.orbstack/run/docker.sock` was not reachable during `docker build`.
|
||||
371
.project/project-plan.md
Normal file
371
.project/project-plan.md
Normal file
@@ -0,0 +1,371 @@
|
||||
# Cairnquire - Project Plan
|
||||
|
||||
**Version:** 1.1\
|
||||
**Date:** 2026-04-29\
|
||||
**Status:** In Progress — Milestone 1 Complete, Milestone 2 Started
|
||||
|
||||
## Vision
|
||||
|
||||
A minimal-dependency, ultra-fast documentation platform that publishes markdown notes as a responsive web application. Features wiki-style linking, collaborative comments, offline-capable sync, and fine-grained access control — all deployable via a single `docker-compose.yml` with zero vendor lock-in.
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. **Paranoid Security**: Cryptographic content verification, battle-tested dependencies only, supply-chain attack mitigation through minimal external code
|
||||
2. **Operational Simplicity**: Single binary, single SQLite database, filesystem storage — nothing that can't be understood in an afternoon
|
||||
3. **Performance First**: Sub-100ms initial page loads, instant client-side navigation, no JavaScript required for reading
|
||||
4. **Offline Capability**: Read cached, sync when connected
|
||||
5. **No Vendor Lock-in**: Works on any Docker host; no cloud-specific APIs
|
||||
|
||||
---
|
||||
|
||||
## Milestones
|
||||
|
||||
### Milestone 1: Foundation ✅ COMPLETE
|
||||
|
||||
**Goal:** Working server that can serve markdown files as HTML
|
||||
|
||||
- [x] Go HTTP server with chi router
|
||||
- [x] libsql database with schema migration system
|
||||
- [x] Markdown-to-HTML pipeline with most common extensions enabled
|
||||
- [x] Static file serving for attachments
|
||||
- [x] Basic Dockerfile and docker-compose.yml
|
||||
- [x] Dev mode with `air` live reload + Vite HMR proxy
|
||||
- [x] Miller column document browser with folder navigation
|
||||
- [x] `index.md` default rendering for folders
|
||||
- [x] Content-addressed attachment storage with secure serving
|
||||
- [x] WebSocket real-time updates (document change notifications)
|
||||
|
||||
**Acceptance Criteria:**
|
||||
|
||||
- [x] Server starts and listens on configurable port
|
||||
- [x] Database auto-migrates on startup
|
||||
- [x] Markdown files with wiki-links, tags, callouts, mermaid, math render correctly
|
||||
- [x] Attachments served securely with content-type headers
|
||||
- [x] Docker compose brings up full stack with one command
|
||||
- [x] All dependencies pinned with checksum verification
|
||||
- [x] Build reproducible (same input → same binary hash)
|
||||
|
||||
**Notes:**
|
||||
- Mermaid and math blocks render as code blocks (no client-side JS rendering yet). Will be addressed in Milestone 5 with client-side hydration.
|
||||
- Dev mode uses `air` for Go live reload and Vite proxy for frontend HMR.
|
||||
|
||||
---
|
||||
|
||||
### Milestone 2: Sync Protocol (Weeks 3-4) — IN PROGRESS
|
||||
|
||||
**Goal:** Bidirectional file sync between server and filesystem/web client
|
||||
|
||||
- [x] Content-addressed storage (SHA-256 filesystem layout)
|
||||
- [x] WebSocket sync protocol implementation
|
||||
- [ ] File watcher for local filesystem sync
|
||||
- [ ] Client-side IndexedDB cache
|
||||
- [ ] Offline queue with conflict detection
|
||||
|
||||
**Acceptance Criteria:**
|
||||
|
||||
- [x] File changes on disk automatically sync to server within 5 seconds
|
||||
- [ ] Web client can sync after reconnection, even with changes made offline
|
||||
- [ ] Concurrent edits to different files succeed without conflict
|
||||
- [ ] Concurrent edits to same file detected and queued for resolution
|
||||
- [x] All synced content verified against SHA-256 hashes
|
||||
- [ ] Sync protocol documented with state machine diagrams
|
||||
|
||||
**What's Done:**
|
||||
- Real-time file watcher using `fsnotify` — detects create/write/remove/rename events on `.md` files
|
||||
- Changes trigger immediate `SyncSourceDir()` and broadcast `document_version` events via WebSocket
|
||||
- Browser shows reload notification when current document or folder structure changes
|
||||
- Fallback polling every 30 seconds ensures nothing is missed
|
||||
|
||||
**What's Next:**
|
||||
- Client-side IndexedDB cache for offline reading
|
||||
- Offline edit queue with conflict resolution UI
|
||||
- State machine diagram for sync protocol
|
||||
|
||||
---
|
||||
|
||||
### Milestone 3: Authentication & Authorization (Weeks 5-6)
|
||||
|
||||
**Goal:** Secure access control with passkeys and granular permissions
|
||||
|
||||
- Passkey (WebAuthn) primary authentication
|
||||
- Password fallback with Argon2id
|
||||
- Session management with signed cookies
|
||||
- Role-based access control (RBAC)
|
||||
- Content signing with Ed25519 (optional per-user)
|
||||
|
||||
**Acceptance Criteria:**
|
||||
|
||||
- [ ] Users can register and log in with passkeys on modern browsers
|
||||
- [ ] Password fallback works with proper hashing
|
||||
- [ ] Sessions expire securely and are revocable
|
||||
- [ ] Read/write/admin permissions work per-document and per-collection
|
||||
- [ ] All auth endpoints rate-limited and logged
|
||||
- [ ] Security audit: no plaintext secrets, no timing attacks on auth
|
||||
|
||||
---
|
||||
|
||||
### Milestone 4: Collaboration Features (Weeks 7-8)
|
||||
|
||||
**Goal:** Comments, notifications, and conflict resolution
|
||||
|
||||
- Comment system linked to file versions (by hash)
|
||||
- Email notifications via Postmark adapter
|
||||
- Web-based conflict resolution UI
|
||||
- Notification preferences (per-file, per-folder, global, digest)
|
||||
- Comment threading and email replies
|
||||
|
||||
**Acceptance Criteria:**
|
||||
|
||||
- [ ] Users can comment on specific lines/sections of a document
|
||||
- [ ] Comments persist and are linked to document versions
|
||||
- [ ] Email notifications sent for watched file changes
|
||||
- [ ] Replying to notification email creates a comment
|
||||
- [ ] Conflicts display side-by-side diff with resolution actions
|
||||
- [ ] Digest mode batches notifications by time window
|
||||
|
||||
---
|
||||
|
||||
### Milestone 5: Search & UI Polish (Weeks 9-10)
|
||||
|
||||
**Goal:** Fast search, design system, and production readiness
|
||||
|
||||
- Full-text search with SQLite FTS5
|
||||
- Client-side search index sync
|
||||
- Design system with browser-based CSS editor
|
||||
- Responsive mobile layout
|
||||
- Error boundaries and loading states
|
||||
- Performance benchmarking
|
||||
- Client-side Mermaid and KaTeX rendering
|
||||
|
||||
**Acceptance Criteria:**
|
||||
|
||||
- [ ] Search returns results in <50ms on client
|
||||
- [ ] Search works offline after initial sync
|
||||
- [ ] Design system documented at `/design` route
|
||||
- [ ] All screens responsive down to 320px width
|
||||
- [ ] Lighthouse score >90 on all metrics
|
||||
- [ ] Initial page load <100ms (server-rendered HTML)
|
||||
- [ ] Mermaid diagrams render client-side
|
||||
- [ ] Math blocks render with KaTeX
|
||||
|
||||
---
|
||||
|
||||
### Milestone 6: Production Hardening (Weeks 11-12)
|
||||
|
||||
**Goal:** Deploy with confidence
|
||||
|
||||
- Comprehensive test suite (unit, integration, e2e)
|
||||
- Security audit and penetration testing guide
|
||||
- Backup and restore procedures
|
||||
- Monitoring and health checks
|
||||
- Documentation for operators
|
||||
|
||||
**Acceptance Criteria:**
|
||||
|
||||
- [ ] 80% test coverage
|
||||
- [ ] No critical or high-severity security findings
|
||||
- [ ] Backup script exports database + attachments as tar.gz
|
||||
- [ ] Health check endpoint returns status of all dependencies
|
||||
- [ ] Operator docs cover: install, upgrade, backup, security incident response
|
||||
- [ ] Load test: 1000 concurrent readers, 100 concurrent editors
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Client Browser │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌─────────────────┐ │
|
||||
│ │ Preact UI │ │ File Sync │ │ Search Index │ │
|
||||
│ │ (hydrated) │ │ (IndexedDB) │ │ (FTS5) │ │
|
||||
│ └──────────────┘ └──────────────┘ └─────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
WebSocket / HTTP
|
||||
│
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Go Application Server │
|
||||
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌──────────┐ │
|
||||
│ │ HTTP │ │ WebSocket │ │ Markdown │ │ Auth │ │
|
||||
│ │ Router │ │ Sync │ │ Renderer │ │ (WebAuthn)│ │
|
||||
│ └────────────┘ └────────────┘ └────────────┘ └──────────┘ │
|
||||
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │
|
||||
│ │ Email │ │ Search │ │ File Watch │ │
|
||||
│ │ (Postmark)│ │ (FTS5) │ │ (fsnotify)│ │
|
||||
│ └────────────┘ └────────────┘ └────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────┴──────────────────┐
|
||||
│ │
|
||||
┌──────────────┐ ┌──────────────┐
|
||||
│ libsql │ │ Content- │
|
||||
│ (SQLite) │ │ Addressed │
|
||||
│ │ │ Filesystem │
|
||||
└──────────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
See [architecture-overview.md](architecture-overview.md) for detailed component design.
|
||||
|
||||
---
|
||||
|
||||
## Technology Stack
|
||||
|
||||
### Backend (Go 1.24+)
|
||||
|
||||
| Component | Library | Justification |
|
||||
| ------------- | ------------------------------------------- | ---------------------------------------- |
|
||||
| HTTP Router | `github.com/go-chi/chi` | Minimal, idiomatic, stdlib-compatible |
|
||||
| Database | `github.com/tursodatabase/go-libsql` | Embedded SQLite with replication option |
|
||||
| Migrations | Internal migration runner | Simple, no extra dependency |
|
||||
| Auth/Passkeys | `github.com/go-webauthn/webauthn` | Reference implementation, well-audited |
|
||||
| Password Hash | `golang.org/x/crypto/argon2` | Winner of Password Hashing Competition |
|
||||
| Markdown | `github.com/yuin/goldmark` + extensions | Fast, extensible, CommonMark compliant |
|
||||
| WebSocket | `github.com/gorilla/websocket` | Most widely used Go WS library |
|
||||
| File Watch | `github.com/fsnotify/fsnotify` | Cross-platform, stdlib-adjacent |
|
||||
| Email | `github.com/keighl/postmark` + `net/smtp` | Postmark adapter + SMTP fallback |
|
||||
| Crypto | `crypto/ed25519`, `crypto/sha256` | Standard library, no external deps |
|
||||
| Templates | `html/template` | Stdlib, auto-escaping, SSR-safe |
|
||||
|
||||
### Frontend (Preact + TypeScript)
|
||||
|
||||
| Component | Library | Justification |
|
||||
| --------- | ------------------------------ | --------------------------------------- |
|
||||
| Framework | `preact` (10KB) | React API, tiny bundle, fast hydration |
|
||||
| Routing | `preact-iso` | Lightweight isomorphic routing |
|
||||
| Markdown | `marked` + `micromark` | Server+client compatible |
|
||||
| Search | `flexsearch` | Client-side full-text, zero server load |
|
||||
| Styling | Native CSS + custom properties | No build-time CSS processing needed |
|
||||
| Icons | Inline SVG | No icon font dependency |
|
||||
| Build | `vite` | Fast HMR, modern output, minimal config |
|
||||
|
||||
### Infrastructure
|
||||
|
||||
| Component | Technology | Justification |
|
||||
| ------------- | ------------------ | --------------------------------------------- |
|
||||
| Container | Docker + Compose | Universal deployment target |
|
||||
| Reverse Proxy | Traefik (optional) | Auto-HTTPS, included in compose |
|
||||
| Database | libsql/SQLite | Single file, zero external dependency |
|
||||
| Storage | Local filesystem | Content-addressed, immutable, backup-friendly |
|
||||
|
||||
---
|
||||
|
||||
## Security Model
|
||||
|
||||
### Threats Addressed
|
||||
|
||||
1. **Supply Chain Attacks**: Minimal external dependencies; all pinned with go.sum; vendored if needed
|
||||
2. **Data Tampering**: All document versions SHA-256 hashed; optional Ed25519 signatures
|
||||
3. **Unauthorized Access**: RBAC with principle of least privilege; content never served without auth check
|
||||
4. **Session Hijacking**: Signed cookies with HttpOnly, Secure, SameSite=Strict; short expiry with refresh
|
||||
5. **CSRF**: Double-submit cookie pattern for state-changing operations
|
||||
6. **XSS**: Go html/template auto-escaping; strict CSP headers; no inline scripts except nonce-tagged
|
||||
7. **DoS**: Rate limiting on auth endpoints; file upload size limits; query timeouts
|
||||
|
||||
### Content Security
|
||||
|
||||
- Attachment uploads: Magic number validation, extension whitelist, size limits
|
||||
- Markdown rendering: Sanitized HTML output; no raw HTML injection
|
||||
- File paths: Canonicalization prevents directory traversal
|
||||
|
||||
### Audit & Compliance
|
||||
|
||||
- Every data modification logged with user, timestamp, IP, and content hash
|
||||
- Immutable history table — updates are inserts
|
||||
- GDPR-compliant: full data export and deletion capability
|
||||
|
||||
---
|
||||
|
||||
## Performance Targets
|
||||
|
||||
| Metric | Target | Measurement |
|
||||
| ------------------- | ------------------------- | -------------------------------- |
|
||||
| Initial Page Load | <100ms | TTFB + First Contentful Paint |
|
||||
| Time to Interactive | <200ms | Hydration complete |
|
||||
| Search Query | <50ms | Client-side after initial sync |
|
||||
| File Sync | <5s | From disk save to server persist |
|
||||
| Bundle Size | <100KB | Gzipped JS + CSS |
|
||||
| Concurrent Users | 1000 readers, 100 editors | Load tested |
|
||||
|
||||
---
|
||||
|
||||
## Monorepo Structure
|
||||
|
||||
```
|
||||
cairnquire/
|
||||
├── README.md
|
||||
├── docker-compose.yml
|
||||
├── Dockerfile
|
||||
├── mise.toml
|
||||
├── docs/ # Project documentation
|
||||
│ ├── project-plan.md # This file
|
||||
│ ├── architecture-overview.md # System architecture
|
||||
│ ├── adrs/ # Architecture Decision Records
|
||||
│ ├── milestones/ # Detailed milestone plans
|
||||
│ └── specs/ # Technical specifications
|
||||
├── apps/
|
||||
│ ├── server/ # Go HTTP application
|
||||
│ │ ├── cmd/cairnquire/
|
||||
│ │ ├── internal/
|
||||
│ │ │ ├── app/ # Application orchestration
|
||||
│ │ │ ├── config/ # Configuration management
|
||||
│ │ │ ├── database/ # Database & migrations
|
||||
│ │ │ ├── docs/ # Document service & repository
|
||||
│ │ │ ├── httpserver/ # HTTP handlers & templates
|
||||
│ │ │ ├── markdown/ # Markdown rendering pipeline
|
||||
│ │ │ ├── realtime/ # WebSocket hub
|
||||
│ │ │ └── store/ # Content-addressed filesystem
|
||||
│ │ ├── static/ # Static assets
|
||||
│ │ └── templates/ # Go HTML templates
|
||||
│ └── web/ # Preact frontend application
|
||||
│ ├── src/
|
||||
│ ├── index.html
|
||||
│ └── vite.config.ts
|
||||
├── content/ # Markdown source files
|
||||
├── data/ # SQLite database & attachments
|
||||
└── packages/
|
||||
└── protocol/ # Shared sync protocol spec
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Risk Register
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
| -------------------------------- | ---------- | ------ | ----------------------------------------------------------------------------------- |
|
||||
| WebAuthn/passkey adoption issues | Medium | Medium | Password fallback always available; extensive browser testing |
|
||||
| SQLite performance at scale | Low | High | libsql replication path; FTS5 for search; benchmark early |
|
||||
| Markdown extension parser bugs | Medium | Medium | Fuzz testing; strict CommonMark base; gradual feature rollout |
|
||||
| Email deliverability | Medium | Medium | Postmark reputation monitoring; SPF/DKIM setup; SMTP fallback |
|
||||
| Frontend bundle bloat | Medium | Medium | Bundle analysis in CI; tree-shaking enforced; lazy loading |
|
||||
| File sync conflicts in teams | High | Medium | Clear UX for conflict resolution; email notification; automatic merge when possible |
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ Complete Milestone 1: Foundation
|
||||
2. 🔄 Continue Milestone 2: Sync Protocol
|
||||
- Implement client-side IndexedDB cache
|
||||
- Build offline edit queue
|
||||
- Create sync protocol state machine diagram
|
||||
3. Begin Milestone 3: Authentication & Authorization
|
||||
- Set up WebAuthn/passkey infrastructure
|
||||
- Implement session management
|
||||
4. Add client-side Mermaid and KaTeX rendering (Milestone 5)
|
||||
5. Implement SQLite FTS5 search (Milestone 5)
|
||||
6. Add Renovate with dependency update cooldown policies
|
||||
|
||||
---
|
||||
|
||||
## Glossary
|
||||
|
||||
- **ADR**: Architecture Decision Record
|
||||
- **CRDT**: Conflict-free Replicated Data Type (not used, but considered)
|
||||
- **CSRF**: Cross-Site Request Forgery
|
||||
- **CSP**: Content Security Policy
|
||||
- **FTS5**: SQLite Full-Text Search version 5
|
||||
- **RBAC**: Role-Based Access Control
|
||||
- **SSR**: Server-Side Rendering
|
||||
- **TTFB**: Time to First Byte
|
||||
- **WebAuthn**: Web Authentication API (passkeys)
|
||||
272
.project/specs/deployment-guide.md
Normal file
272
.project/specs/deployment-guide.md
Normal file
@@ -0,0 +1,272 @@
|
||||
# Deployment Guide
|
||||
|
||||
## Requirements
|
||||
|
||||
- Docker 24.0+
|
||||
- Docker Compose 2.20+
|
||||
- 1GB RAM minimum (2GB recommended)
|
||||
- 10GB disk space
|
||||
- TLS certificate (Let's Encrypt or custom)
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/your-org/cairnquire.git
|
||||
cd cairnquire
|
||||
|
||||
# Copy and edit configuration
|
||||
cp .env.example .env
|
||||
# Edit .env with your domain, email, Postmark API key
|
||||
|
||||
# Start services
|
||||
docker-compose up -d
|
||||
|
||||
# Verify health
|
||||
curl https://your-domain.com/health
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Required | Default | Description |
|
||||
|----------|----------|---------|-------------|
|
||||
| `DOMAIN` | Yes | — | Public domain name |
|
||||
| `DB_PATH` | No | `/data/db.sqlite` | Database file path |
|
||||
| `FILESTORE_PATH` | No | `/data/files` | Content-addressed file storage |
|
||||
| `NOTES_PATH` | No | `/notes` | Watched markdown directory |
|
||||
| `POSTMARK_API_KEY` | Yes | — | Postmark server API token |
|
||||
| `POSTMARK_FROM` | No | `notifications@DOMAIN` | From email address |
|
||||
| `SESSION_SECRET` | Yes | — | 32-byte hex for session signing |
|
||||
| `ENV` | No | `production` | `development` or `production` |
|
||||
|
||||
### Docker Compose
|
||||
|
||||
```yaml
|
||||
version: "3.8"
|
||||
|
||||
services:
|
||||
app:
|
||||
build: .
|
||||
ports:
|
||||
- "8080:8080"
|
||||
volumes:
|
||||
- ./data:/data
|
||||
- ./notes:/notes:ro
|
||||
environment:
|
||||
- DOMAIN=${DOMAIN}
|
||||
- DB_PATH=/data/db.sqlite
|
||||
- FILESTORE_PATH=/data/files
|
||||
- NOTES_PATH=/notes
|
||||
- POSTMARK_API_KEY=${POSTMARK_API_KEY}
|
||||
- SESSION_SECRET=${SESSION_SECRET}
|
||||
env_file:
|
||||
- .env
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "--spider", "http://localhost:8080/health"]
|
||||
interval: 30s
|
||||
timeout: 3s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
read_only: true
|
||||
tmpfs:
|
||||
- /tmp:noexec,nosuid,size=100m
|
||||
|
||||
# Optional: Traefik for auto-HTTPS
|
||||
traefik:
|
||||
image: traefik:v3.0
|
||||
command:
|
||||
- "--api.insecure=false"
|
||||
- "--providers.docker=true"
|
||||
- "--providers.docker.exposedbydefault=false"
|
||||
- "--entrypoints.web.address=:80"
|
||||
- "--entrypoints.websecure.address=:443"
|
||||
- "--certificatesresolvers.letsencrypt.acme.tlschallenge=true"
|
||||
- "--certificatesresolvers.letsencrypt.acme.email=${EMAIL}"
|
||||
- "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
|
||||
ports:
|
||||
- "80:80"
|
||||
- "443:443"
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
- ./letsencrypt:/letsencrypt
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
restart: unless-stopped
|
||||
```
|
||||
|
||||
## Production Checklist
|
||||
|
||||
### Pre-Deployment
|
||||
- [ ] Change default `SESSION_SECRET` (generate: `openssl rand -hex 32`)
|
||||
- [ ] Configure `POSTMARK_API_KEY` with valid token
|
||||
- [ ] Set `DOMAIN` to public hostname
|
||||
- [ ] Configure DNS A/AAAA records
|
||||
- [ ] Set up SPF and DKIM for email domain
|
||||
- [ ] Review `.env` file — no default secrets
|
||||
|
||||
### Security
|
||||
- [ ] TLS 1.3 enforced (Traefik or reverse proxy)
|
||||
- [ ] HSTS header configured
|
||||
- [ ] Firewall rules: only 80/443 open
|
||||
- [ ] Fail2ban or rate limiting on SSH
|
||||
- [ ] Automatic security updates enabled
|
||||
- [ ] Backup encryption key stored securely
|
||||
|
||||
### Monitoring
|
||||
- [ ] Health check endpoint accessible
|
||||
- [ ] Log rotation configured
|
||||
- [ ] Disk space alerting (>80%)
|
||||
- [ ] Memory usage alerting
|
||||
- [ ] Uptime monitoring (external)
|
||||
|
||||
## Backup Procedures
|
||||
|
||||
### Automated Backup
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# backup.sh
|
||||
|
||||
BACKUP_DIR="/backups"
|
||||
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
|
||||
BACKUP_FILE="$BACKUP_DIR/md-hub-$TIMESTAMP.tar.gz"
|
||||
|
||||
# Create backup
|
||||
tar czf "$BACKUP_FILE" \
|
||||
-C /data \
|
||||
db.sqlite \
|
||||
files/
|
||||
|
||||
# Encrypt (optional)
|
||||
gpg --symmetric --cipher-algo AES256 "$BACKUP_FILE"
|
||||
rm "$BACKUP_FILE"
|
||||
|
||||
# Keep only last 30 days
|
||||
find "$BACKUP_DIR" -name "md-hub-*.tar.gz.gpg" -mtime +30 -delete
|
||||
```
|
||||
|
||||
**Cron**: `0 2 * * * /opt/cairnquire/backup.sh`
|
||||
|
||||
### Restore from Backup
|
||||
|
||||
```bash
|
||||
# Stop application
|
||||
docker-compose stop app
|
||||
|
||||
# Extract backup
|
||||
tar xzf md-hub-20240115_020000.tar.gz -C /data
|
||||
|
||||
# Restart
|
||||
docker-compose up -d app
|
||||
|
||||
# Verify
|
||||
curl https://your-domain.com/health
|
||||
```
|
||||
|
||||
## Upgrade Procedures
|
||||
|
||||
### Minor Update (patch version)
|
||||
```bash
|
||||
docker-compose pull
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
### Major Update
|
||||
1. Read release notes for breaking changes
|
||||
2. Create backup: `./backup.sh`
|
||||
3. Pull new image: `docker-compose pull`
|
||||
4. Run database migrations (auto on startup)
|
||||
5. Update configuration if needed
|
||||
6. Deploy: `docker-compose up -d`
|
||||
7. Verify health and functionality
|
||||
8. Rollback if issues: restore from backup
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Application Won't Start
|
||||
```bash
|
||||
# Check logs
|
||||
docker-compose logs -f app
|
||||
|
||||
# Verify configuration
|
||||
docker-compose exec app /server -check-config
|
||||
|
||||
# Test database
|
||||
docker-compose exec app sqlite3 /data/db.sqlite ".tables"
|
||||
```
|
||||
|
||||
### Sync Not Working
|
||||
```bash
|
||||
# Check file watcher
|
||||
docker-compose exec app ls -la /notes
|
||||
|
||||
# Verify WebSocket
|
||||
curl -i -N \
|
||||
-H "Connection: Upgrade" \
|
||||
-H "Upgrade: websocket" \
|
||||
-H "Sec-WebSocket-Key: $(openssl rand -base64 16)" \
|
||||
-H "Sec-WebSocket-Version: 13" \
|
||||
https://your-domain.com/ws
|
||||
```
|
||||
|
||||
### Email Not Sending
|
||||
```bash
|
||||
# Test Postmark connection
|
||||
curl -X POST \
|
||||
https://api.postmarkapp.com/server \
|
||||
-H "X-Postmark-Server-Token: YOUR_TOKEN" \
|
||||
-H "Accept: application/json"
|
||||
|
||||
# Check email queue
|
||||
docker-compose exec app sqlite3 /data/db.sqlite \
|
||||
"SELECT * FROM email_queue WHERE status='failed'"
|
||||
```
|
||||
|
||||
## Scaling
|
||||
|
||||
### Vertical Scaling
|
||||
- Increase container memory/CPU limits
|
||||
- SQLite WAL mode handles concurrent reads well
|
||||
- For heavy write loads, consider libsql replication
|
||||
|
||||
### Horizontal Scaling (Advanced)
|
||||
```yaml
|
||||
# Requires shared storage and load balancer
|
||||
services:
|
||||
app1:
|
||||
# ... same config ...
|
||||
environment:
|
||||
- DB_PATH=/shared/db.sqlite
|
||||
- FILESTORE_PATH=/shared/files
|
||||
|
||||
app2:
|
||||
# ... same config ...
|
||||
environment:
|
||||
- DB_PATH=/shared/db.sqlite
|
||||
- FILESTORE_PATH=/shared/files
|
||||
|
||||
nginx:
|
||||
image: nginx:alpine
|
||||
volumes:
|
||||
- ./nginx.conf:/etc/nginx/nginx.conf
|
||||
ports:
|
||||
- "80:80"
|
||||
```
|
||||
|
||||
**Note**: Horizontal scaling requires:
|
||||
- Shared filesystem (NFS, EFS)
|
||||
- libsql replication or external database
|
||||
- Sticky sessions for WebSocket
|
||||
|
||||
## Support
|
||||
|
||||
- **Documentation**: `/docs` route (when authenticated)
|
||||
- **Health Check**: `GET /health`
|
||||
- **Metrics**: `GET /metrics` (Prometheus format)
|
||||
- **Logs**: `docker-compose logs -f app`
|
||||
- **Issues**: https://github.com/your-org/cairnquire/issues
|
||||
456
.project/specs/security-specification.md
Normal file
456
.project/specs/security-specification.md
Normal file
@@ -0,0 +1,456 @@
|
||||
# Security Specification
|
||||
|
||||
## Threat Model
|
||||
|
||||
### Assets
|
||||
1. **Document Content**: Markdown files and attachments
|
||||
2. **User Credentials**: Passkeys, passwords, session tokens
|
||||
3. **Collaboration Data**: Comments, notifications, history
|
||||
4. **System State**: Database, configuration, logs
|
||||
|
||||
### Threat Actors
|
||||
1. **External Attacker**: No system access, attempts remote exploitation
|
||||
2. **Malicious User**: Authenticated, attempts unauthorized access
|
||||
3. **Compromised Client**: Legitimate user's device infected
|
||||
4. **Insider**: Administrator or developer with system access
|
||||
|
||||
### Threats
|
||||
1. **Supply Chain Attack**: Malicious dependency injection
|
||||
2. **Authentication Bypass**: Session hijacking, credential theft
|
||||
3. **Authorization Bypass**: Horizontal/vertical privilege escalation
|
||||
4. **Data Exfiltration**: Unauthorized document access
|
||||
5. **Data Tampering**: Modification without authorization
|
||||
6. **Denial of Service**: Resource exhaustion
|
||||
7. **Information Disclosure**: Secrets in logs/responses
|
||||
|
||||
## Security Controls
|
||||
|
||||
### 1. Supply Chain Security
|
||||
|
||||
**Policy**:
|
||||
- Only dependencies with >1M downloads/month or official Go team
|
||||
- All versions pinned in go.sum
|
||||
- No transitive dependencies without review
|
||||
- Prefer standard library when possible
|
||||
|
||||
**Implementation**:
|
||||
```
|
||||
# go.mod
|
||||
require (
|
||||
github.com/go-chi/chi/v5 v5.0.12
|
||||
github.com/tursodatabase/libsql-client-go v0.0.0-20240416075031-555ce55511f7
|
||||
// All versions pinned
|
||||
)
|
||||
|
||||
# Build
|
||||
RUN go mod verify # Verify checksums
|
||||
RUN go mod vendor # Optional: vendor for air-gapped
|
||||
```
|
||||
|
||||
**Audit**:
|
||||
- Weekly `govulncheck` scan
|
||||
- Dependency review in CI
|
||||
- No build-time network access (vendored or proxy with checksum)
|
||||
|
||||
### 2. Authentication Security
|
||||
|
||||
#### Passkey (WebAuthn)
|
||||
```go
|
||||
// Registration
|
||||
func registerPasskey(user User) (*Credential, error) {
|
||||
options, session, err := webauthn.BeginRegistration(user)
|
||||
// Store session data server-side (never client-side)
|
||||
// Return options to client
|
||||
}
|
||||
|
||||
func verifyRegistration(sessionID string, response ProtocolCredentialCreation) (*Credential, error) {
|
||||
session := getSessionData(sessionID)
|
||||
credential, err := webauthn.FinishRegistration(user, session, response)
|
||||
// Verify attestation
|
||||
// Store credential: ID, public key, sign count
|
||||
}
|
||||
```
|
||||
|
||||
#### Password (Argon2id)
|
||||
```go
|
||||
import "golang.org/x/crypto/argon2"
|
||||
|
||||
const (
|
||||
timeCost = 3
|
||||
memoryCost = 64 * 1024 // 64MB
|
||||
parallelism = 4
|
||||
saltLength = 16
|
||||
keyLength = 32
|
||||
)
|
||||
|
||||
func hashPassword(password string) (string, error) {
|
||||
salt := make([]byte, saltLength)
|
||||
if _, err := rand.Read(salt); err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
hash := argon2.IDKey([]byte(password), salt, timeCost, memoryCost, parallelism, keyLength)
|
||||
|
||||
// Encode to string: $argon2id$v=19$m=65536,t=3,p=4$<salt>$<hash>
|
||||
return encodeHash(salt, hash), nil
|
||||
}
|
||||
|
||||
func verifyPassword(password, encodedHash string) (bool, error) {
|
||||
salt, hash, params, err := decodeHash(encodedHash)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
computedHash := argon2.IDKey([]byte(password), salt, params.time, params.memory, params.parallelism, uint32(len(hash)))
|
||||
|
||||
// Constant-time comparison
|
||||
return subtle.ConstantTimeCompare(hash, computedHash) == 1, nil
|
||||
}
|
||||
```
|
||||
|
||||
#### Session Management
|
||||
```go
|
||||
func createSession(userID string, r *http.Request) (*Session, error) {
|
||||
token := make([]byte, 32)
|
||||
if _, err := rand.Read(token); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
session := &Session{
|
||||
ID: uuid.New().String(),
|
||||
UserID: userID,
|
||||
TokenHash: sha256.Sum256(token),
|
||||
CreatedAt: time.Now(),
|
||||
ExpiresAt: time.Now().Add(24 * time.Hour),
|
||||
IP: r.RemoteAddr,
|
||||
UserAgent: r.UserAgent(),
|
||||
}
|
||||
|
||||
// Store in database
|
||||
if err := db.CreateSession(session); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Set cookie
|
||||
http.SetCookie(w, &http.Cookie{
|
||||
Name: "session",
|
||||
Value: base64.URLEncoding.EncodeToString(token),
|
||||
Expires: session.ExpiresAt,
|
||||
HttpOnly: true,
|
||||
Secure: true, // HTTPS only
|
||||
SameSite: http.SameSiteStrictMode,
|
||||
Path: "/",
|
||||
})
|
||||
|
||||
return session, nil
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Authorization
|
||||
|
||||
#### Permission Model
|
||||
```go
|
||||
type Permission string
|
||||
|
||||
const (
|
||||
PermissionRead Permission = "read"
|
||||
PermissionWrite Permission = "write"
|
||||
PermissionAdmin Permission = "admin"
|
||||
)
|
||||
|
||||
type ResourceType string
|
||||
|
||||
const (
|
||||
ResourceGlobal ResourceType = "global"
|
||||
ResourceCollection ResourceType = "collection"
|
||||
ResourceDocument ResourceType = "document"
|
||||
)
|
||||
|
||||
func checkPermission(userID string, resourceType ResourceType, resourceID string, required Permission) (bool, error) {
|
||||
// Check explicit permission
|
||||
perm, err := db.GetPermission(userID, resourceType, resourceID)
|
||||
if err == nil && hasPermission(perm.Permission, required) {
|
||||
return true, nil
|
||||
}
|
||||
|
||||
// Check collection permission for documents
|
||||
if resourceType == ResourceDocument {
|
||||
collectionID := getDocumentCollection(resourceID)
|
||||
perm, err = db.GetPermission(userID, ResourceCollection, collectionID)
|
||||
if err == nil && hasPermission(perm.Permission, required) {
|
||||
return true, nil
|
||||
}
|
||||
}
|
||||
|
||||
// Check global permission
|
||||
perm, err = db.GetPermission(userID, ResourceGlobal, "")
|
||||
if err == nil && hasPermission(perm.Permission, required) {
|
||||
return true, nil
|
||||
}
|
||||
|
||||
// Check document public status
|
||||
if resourceType == ResourceDocument && required == PermissionRead {
|
||||
doc, err := db.GetDocument(resourceID)
|
||||
if err == nil && doc.PublicRead {
|
||||
return true, nil
|
||||
}
|
||||
}
|
||||
|
||||
return false, nil
|
||||
}
|
||||
```
|
||||
|
||||
#### Middleware
|
||||
```go
|
||||
func RequirePermission(resourceType ResourceType, required Permission) func(http.Handler) http.Handler {
|
||||
return func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
userID := getUserID(r.Context())
|
||||
resourceID := chi.URLParam(r, "id")
|
||||
|
||||
allowed, err := checkPermission(userID, resourceType, resourceID, required)
|
||||
if err != nil || !allowed {
|
||||
http.Error(w, "Forbidden", http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Usage:
|
||||
r.With(RequirePermission(ResourceDocument, PermissionRead)).
|
||||
Get("/docs/{id}", getDocumentHandler)
|
||||
```
|
||||
|
||||
### 4. Input Validation
|
||||
|
||||
#### Markdown Sanitization
|
||||
```go
|
||||
import "github.com/yuin/goldmark"
|
||||
import "github.com/yuin/goldmark/extension"
|
||||
import "github.com/yuin/goldmark/renderer/html"
|
||||
|
||||
func renderMarkdown(source []byte) ([]byte, error) {
|
||||
md := goldmark.New(
|
||||
goldmark.WithExtensions(
|
||||
extension.Table,
|
||||
extension.Strikethrough,
|
||||
extension.Linkify,
|
||||
// No raw HTML extension — we control HTML output
|
||||
),
|
||||
goldmark.WithRendererOptions(
|
||||
html.WithHardWraps(),
|
||||
html.WithXHTML(),
|
||||
html.WithUnsafe(), // DISABLED — no raw HTML passthrough
|
||||
),
|
||||
)
|
||||
|
||||
var buf bytes.Buffer
|
||||
if err := md.Convert(source, &buf); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return buf.Bytes(), nil
|
||||
}
|
||||
```
|
||||
|
||||
#### File Upload Validation
|
||||
```go
|
||||
func validateUpload(file io.Reader, filename string, maxSize int64) ([]byte, string, error) {
|
||||
// Size limit
|
||||
limited := io.LimitReader(file, maxSize+1)
|
||||
data, err := io.ReadAll(limited)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
if int64(len(data)) > maxSize {
|
||||
return nil, "", fmt.Errorf("file too large")
|
||||
}
|
||||
|
||||
// Magic number validation
|
||||
contentType := detectContentType(data)
|
||||
allowedTypes := map[string]bool{
|
||||
"image/jpeg": true,
|
||||
"image/png": true,
|
||||
"image/gif": true,
|
||||
"image/webp": true,
|
||||
"application/pdf": true,
|
||||
}
|
||||
if !allowedTypes[contentType] {
|
||||
return nil, "", fmt.Errorf("file type not allowed")
|
||||
}
|
||||
|
||||
// Extension whitelist
|
||||
ext := strings.ToLower(filepath.Ext(filename))
|
||||
allowedExts := map[string]bool{
|
||||
".jpg": true, ".jpeg": true, ".png": true,
|
||||
".gif": true, ".webp": true, ".pdf": true,
|
||||
}
|
||||
if !allowedExts[ext] {
|
||||
return nil, "", fmt.Errorf("file extension not allowed")
|
||||
}
|
||||
|
||||
// Compute hash
|
||||
hash := sha256.Sum256(data)
|
||||
hashStr := hex.EncodeToString(hash[:])
|
||||
|
||||
return data, hashStr, nil
|
||||
}
|
||||
```
|
||||
|
||||
#### Path Sanitization
|
||||
```go
|
||||
func sanitizePath(userPath string) (string, error) {
|
||||
// Clean the path
|
||||
clean := filepath.Clean(userPath)
|
||||
|
||||
// Prevent traversal
|
||||
if strings.Contains(clean, "..") {
|
||||
return "", fmt.Errorf("path traversal detected")
|
||||
}
|
||||
|
||||
// Ensure it's relative
|
||||
if filepath.IsAbs(clean) {
|
||||
return "", fmt.Errorf("absolute paths not allowed")
|
||||
}
|
||||
|
||||
// Join with base and verify it's within base
|
||||
fullPath := filepath.Join(baseDir, clean)
|
||||
if !strings.HasPrefix(filepath.Clean(fullPath), filepath.Clean(baseDir)) {
|
||||
return "", fmt.Errorf("path outside allowed directory")
|
||||
}
|
||||
|
||||
return clean, nil
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Transport Security
|
||||
|
||||
#### TLS Configuration
|
||||
```go
|
||||
tlsConfig := &tls.Config{
|
||||
MinVersion: tls.VersionTLS13,
|
||||
CurvePreferences: []tls.CurveID{
|
||||
tls.X25519,
|
||||
tls.CurveP256,
|
||||
},
|
||||
CipherSuites: []uint16{
|
||||
tls.TLS_AES_256_GCM_SHA384,
|
||||
tls.TLS_CHACHA20_POLY1305_SHA256,
|
||||
tls.TLS_AES_128_GCM_SHA256,
|
||||
},
|
||||
PreferServerCipherSuites: true,
|
||||
}
|
||||
```
|
||||
|
||||
#### Security Headers
|
||||
```go
|
||||
func securityHeaders(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Security-Policy",
|
||||
"default-src 'self'; "+
|
||||
"script-src 'self' 'nonce-{nonce}'; "+
|
||||
"style-src 'self' 'unsafe-inline'; "+
|
||||
"img-src 'self' data:; "+
|
||||
"connect-src 'self' wss:; "+
|
||||
"frame-ancestors 'none'; "+
|
||||
"base-uri 'self'; "+
|
||||
"form-action 'self';")
|
||||
w.Header().Set("X-Content-Type-Options", "nosniff")
|
||||
w.Header().Set("X-Frame-Options", "DENY")
|
||||
w.Header().Set("X-XSS-Protection", "1; mode=block")
|
||||
w.Header().Set("Referrer-Policy", "strict-origin-when-cross-origin")
|
||||
w.Header().Set("Strict-Transport-Security", "max-age=63072000; includeSubDomains; preload")
|
||||
w.Header().Set("Permissions-Policy", "camera=(), microphone=(), geolocation=()")
|
||||
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
### 6. Operational Security
|
||||
|
||||
#### Logging
|
||||
```go
|
||||
func logAudit(event AuditEvent) {
|
||||
entry := slog.With(
|
||||
"event_type", event.Type,
|
||||
"actor_id", redactIfSecret(event.ActorID),
|
||||
"resource_type", event.ResourceType,
|
||||
"resource_id", redactIfSecret(event.ResourceID),
|
||||
"ip", event.IP,
|
||||
"timestamp", event.Timestamp,
|
||||
)
|
||||
|
||||
// Never log: passwords, tokens, session IDs, API keys
|
||||
entry.Info("audit event")
|
||||
}
|
||||
|
||||
func redactIfSecret(value string) string {
|
||||
if looksLikeSecret(value) {
|
||||
return "[REDACTED]"
|
||||
}
|
||||
return value
|
||||
}
|
||||
```
|
||||
|
||||
#### Container Security
|
||||
```dockerfile
|
||||
# Dockerfile
|
||||
FROM golang:1.22-alpine AS builder
|
||||
# ... build ...
|
||||
|
||||
FROM alpine:3.19
|
||||
RUN addgroup -g 1000 appgroup && \
|
||||
adduser -u 1000 -G appgroup -s /bin/sh -D appuser
|
||||
|
||||
# Read-only root filesystem
|
||||
COPY --from=builder /app/server /server
|
||||
COPY --from=builder /app/web/dist /web
|
||||
|
||||
# Data directory (writable)
|
||||
RUN mkdir /data && chown appuser:appgroup /data
|
||||
VOLUME /data
|
||||
|
||||
USER appuser
|
||||
EXPOSE 8080
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=3s \
|
||||
CMD wget -q --spider http://localhost:8080/health || exit 1
|
||||
|
||||
ENTRYPOINT ["/server"]
|
||||
```
|
||||
|
||||
## Incident Response
|
||||
|
||||
### Detection
|
||||
- Automated alerts on: failed auth spikes, permission errors, large file uploads
|
||||
- Log analysis for anomalies
|
||||
- Health check failures
|
||||
|
||||
### Response
|
||||
1. **Isolate**: Block IP, revoke sessions, disable account if compromised
|
||||
2. **Assess**: Determine scope of breach, affected data
|
||||
3. **Contain**: Patch vulnerability, rotate secrets
|
||||
4. **Recover**: Restore from clean backup if necessary
|
||||
5. **Document**: Post-incident review, update security measures
|
||||
|
||||
## Compliance
|
||||
|
||||
### GDPR
|
||||
- Data minimization: only email stored, no other PII
|
||||
- Right to erasure: `/api/user/delete` endpoint
|
||||
- Data portability: `/api/user/export` endpoint
|
||||
- Consent: explicit opt-in for email notifications
|
||||
|
||||
### Audit Trail
|
||||
All security-relevant events logged:
|
||||
- Authentication attempts (success/failure)
|
||||
- Permission changes
|
||||
- Document access (if private)
|
||||
- Admin actions
|
||||
- Configuration changes
|
||||
|
||||
Retention: 90 days active, 1 year archive
|
||||
297
.project/specs/simplesync-protocol.md
Normal file
297
.project/specs/simplesync-protocol.md
Normal file
@@ -0,0 +1,297 @@
|
||||
# SimpleSync Protocol Specification
|
||||
|
||||
## Overview
|
||||
|
||||
SimpleSync is a content-addressed file synchronization protocol designed for asynchronous collaboration on markdown documents. It provides automatic bidirectional sync between a server (source of truth) and clients (filesystem watchers or web browsers).
|
||||
|
||||
## Design Goals
|
||||
|
||||
1. **Simplicity**: No branches, commits, or manual sync commands
|
||||
2. **Automatic**: Changes propagate without user intervention
|
||||
3. **Conflict-aware**: Detects concurrent edits, never silently loses data
|
||||
4. **Offline-capable**: Queue changes locally, sync when connected
|
||||
5. **Secure**: All content verified by cryptographic hash
|
||||
|
||||
## Concepts
|
||||
|
||||
### Content Addressing
|
||||
Every file is identified by the SHA-256 hash of its content. The content is immutable — if a file changes, it gets a new hash.
|
||||
|
||||
### Merkle Tree
|
||||
The directory state is represented as a Merkle tree where:
|
||||
- Leaf nodes are file hashes
|
||||
- Internal nodes are hashes of concatenated child hashes
|
||||
- The root hash represents the entire directory state
|
||||
|
||||
### Sync State
|
||||
Each client maintains:
|
||||
- `root_hash`: Hash of current directory Merkle root
|
||||
- `files`: Map of file paths to content hashes
|
||||
- `pending`: Queue of local changes not yet synced
|
||||
|
||||
## Message Format
|
||||
|
||||
All messages are JSON over WebSocket.
|
||||
|
||||
### Client → Server
|
||||
|
||||
#### Sync Request
|
||||
```json
|
||||
{
|
||||
"type": "sync_request",
|
||||
"client_id": "uuid-v4",
|
||||
"root_hash": "sha256-hex-64-chars",
|
||||
"files": {
|
||||
"getting-started.md": "sha256...",
|
||||
"api-reference.md": "sha256..."
|
||||
},
|
||||
"timestamp": "2024-01-15T14:30:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### Content Upload
|
||||
```json
|
||||
{
|
||||
"type": "content_upload",
|
||||
"hash": "sha256...",
|
||||
"content": "base64-encoded-content",
|
||||
"encoding": "base64"
|
||||
}
|
||||
```
|
||||
|
||||
#### Acknowledgment
|
||||
```json
|
||||
{
|
||||
"type": "ack",
|
||||
"message_id": "uuid-of-original-message",
|
||||
"status": "ok"
|
||||
}
|
||||
```
|
||||
|
||||
### Server → Client
|
||||
|
||||
#### Sync Response
|
||||
```json
|
||||
{
|
||||
"type": "sync_response",
|
||||
"server_root_hash": "sha256...",
|
||||
"missing_from_client": ["hash1", "hash2"],
|
||||
"missing_from_server": ["hash3"],
|
||||
"conflicts": ["getting-started.md"],
|
||||
"timestamp": "2024-01-15T14:30:01Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### Content Push
|
||||
```json
|
||||
{
|
||||
"type": "content_push",
|
||||
"hash": "sha256...",
|
||||
"content": "base64-encoded-content",
|
||||
"encoding": "base64",
|
||||
"path": "getting-started.md"
|
||||
}
|
||||
```
|
||||
|
||||
#### Conflict Notification
|
||||
```json
|
||||
{
|
||||
"type": "conflict",
|
||||
"path": "getting-started.md",
|
||||
"server_hash": "sha256...",
|
||||
"client_hash": "sha256...",
|
||||
"server_modified_at": "2024-01-15T14:30:00Z",
|
||||
"client_modified_at": "2024-01-15T14:25:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### Error
|
||||
```json
|
||||
{
|
||||
"type": "error",
|
||||
"code": "unauthorized",
|
||||
"message": "Session expired",
|
||||
"retryable": false
|
||||
}
|
||||
```
|
||||
|
||||
## Protocol Flow
|
||||
|
||||
### Normal Sync (No Conflicts)
|
||||
|
||||
```
|
||||
Client Server
|
||||
| |
|
||||
|-- sync_request -------------->|
|
||||
| root_hash: abc |
|
||||
| |
|
||||
| |-- Compare with server state
|
||||
| |-- Calculate deltas
|
||||
| |
|
||||
|<- sync_response --------------|
|
||||
| missing_from_client: [def] |
|
||||
| missing_from_server: [] |
|
||||
| conflicts: [] |
|
||||
| |
|
||||
|-- content_request(def) ------>|
|
||||
| |
|
||||
|<- content_push(def) ----------|
|
||||
| |
|
||||
|-- ack ----------------------->|
|
||||
```
|
||||
|
||||
### Upload Changes
|
||||
|
||||
```
|
||||
Client Server
|
||||
| |
|
||||
|-- sync_request -------------->|
|
||||
| root_hash: abc |
|
||||
| files: {a: hash1, b: hash2} |
|
||||
| |
|
||||
|<- sync_response --------------|
|
||||
| missing_from_server: [hash2] |
|
||||
| |
|
||||
|-- content_upload(hash2) ----->|
|
||||
| |
|
||||
| |-- Verify hash
|
||||
| |-- Store content
|
||||
| |-- Update database
|
||||
| |-- Broadcast to others
|
||||
| |
|
||||
|<- ack ------------------------|
|
||||
| |
|
||||
```
|
||||
|
||||
### Conflict Detection
|
||||
|
||||
```
|
||||
Client A Server Client B
|
||||
| | |
|
||||
| | |-- Edit file X
|
||||
| | |-- sync_request
|
||||
| |-- Update file X |
|
||||
| |-- Broadcast to A |
|
||||
| | |
|
||||
|-- Edit file X | |
|
||||
|-- sync_request -------------->| |
|
||||
| |-- Detect conflict |
|
||||
| |-- A has old hash |
|
||||
| |-- B has new hash |
|
||||
|<- conflict -------------------| |
|
||||
| path: X | |
|
||||
| server_hash: B_hash | |
|
||||
| client_hash: A_hash | |
|
||||
| | |
|
||||
```
|
||||
|
||||
## State Machine
|
||||
|
||||
```
|
||||
+--------+ sync_request
|
||||
| +-----------+
|
||||
| Idle | |
|
||||
| |<----------+
|
||||
+---+----+
|
||||
|
|
||||
| connect
|
||||
v
|
||||
+---------+---------+
|
||||
| |
|
||||
| Connected |
|
||||
| |
|
||||
+---------+---------+
|
||||
|
|
||||
| sync_request
|
||||
v
|
||||
+---------+---------+
|
||||
| |
|
||||
| Comparing |
|
||||
| |
|
||||
+---------+---------+
|
||||
|
|
||||
+-----------+-----------+
|
||||
| |
|
||||
| has_changes | no_changes
|
||||
v v
|
||||
+---------+---------+ +---------+---------+
|
||||
| | | |
|
||||
| Transferring | | Idle |
|
||||
| | | |
|
||||
+---------+---------+ +-------------------+
|
||||
|
|
||||
| complete
|
||||
v
|
||||
+---------+---------+
|
||||
| |
|
||||
| Idle |
|
||||
| |
|
||||
+-------------------+
|
||||
^
|
||||
| conflict
|
||||
|
|
||||
+---------+---------+
|
||||
| |
|
||||
| Conflicted |
|
||||
| |
|
||||
+-------------------+
|
||||
```
|
||||
|
||||
## Error Codes
|
||||
|
||||
| Code | Description | Retryable |
|
||||
|------|-------------|-----------|
|
||||
| `unauthorized` | Session expired or invalid | No (re-authenticate) |
|
||||
| `forbidden` | Insufficient permissions | No |
|
||||
| `not_found` | Requested content hash unknown | Yes |
|
||||
| `too_large` | Content exceeds size limit | No |
|
||||
| `hash_mismatch` | Content doesn't match claimed hash | Yes |
|
||||
| `rate_limited` | Too many requests | Yes (with backoff) |
|
||||
| `server_error` | Internal server error | Yes |
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Authentication**: All WebSocket connections must authenticate via token in initial HTTP upgrade request
|
||||
2. **Authorization**: Server verifies read/write permissions before serving or accepting content
|
||||
3. **Hash Verification**: Server recomputes SHA-256 of received content and rejects mismatches
|
||||
4. **Size Limits**: Maximum file size enforced (configurable, default 10MB)
|
||||
5. **Rate Limiting**: Sync requests limited per client (configurable, default 10/minute)
|
||||
6. **Path Validation**: All file paths canonicalized and checked against allowlist
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### Hash Computation
|
||||
```go
|
||||
import "crypto/sha256"
|
||||
|
||||
func computeHash(content []byte) string {
|
||||
h := sha256.Sum256(content)
|
||||
return hex.EncodeToString(h[:])
|
||||
}
|
||||
```
|
||||
|
||||
### Merkle Root Computation
|
||||
```go
|
||||
func computeRootHash(files map[string]string) string {
|
||||
// Sort paths for determinism
|
||||
paths := sortedKeys(files)
|
||||
|
||||
hasher := sha256.New()
|
||||
for _, path := range paths {
|
||||
hasher.Write([]byte(path))
|
||||
hasher.Write([]byte(files[path]))
|
||||
}
|
||||
|
||||
return hex.EncodeToString(hasher.Sum(nil))
|
||||
}
|
||||
```
|
||||
|
||||
### WebSocket Connection
|
||||
- Ping/pong every 30 seconds
|
||||
- Connection timeout after 60 seconds without response
|
||||
- Auto-reconnect with exponential backoff (1s, 2s, 4s, 8s, max 60s)
|
||||
|
||||
## Version
|
||||
|
||||
**Protocol Version**: 1.0
|
||||
**Last Updated**: 2024-01-15
|
||||
285
.project/specs/sync-protocol-v1.md
Normal file
285
.project/specs/sync-protocol-v1.md
Normal file
@@ -0,0 +1,285 @@
|
||||
# Sync Protocol v1 Specification
|
||||
|
||||
## Overview
|
||||
|
||||
The Native Sync Protocol enables bidirectional filesystem synchronization between a native macOS client and the cairnquire server. The browser client continues to operate via the existing WebSocket + REST API.
|
||||
|
||||
## Data Model
|
||||
|
||||
### Snapshot
|
||||
|
||||
A point-in-time tree of all files in the sync scope:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "snap:device-1:1234567890",
|
||||
"deviceId": "device-1",
|
||||
"createdAt": "2024-01-15T10:30:00Z",
|
||||
"files": [
|
||||
{
|
||||
"path": "hello.md",
|
||||
"hash": "a1b2c3...",
|
||||
"size": 42,
|
||||
"modified": "2024-01-15T10:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Delta
|
||||
|
||||
A list of changes since a snapshot:
|
||||
|
||||
```json
|
||||
{
|
||||
"changes": [
|
||||
{
|
||||
"type": "create|update|delete|rename",
|
||||
"path": "new-file.md",
|
||||
"oldPath": "old-file.md",
|
||||
"hash": "a1b2c3...",
|
||||
"size": 42,
|
||||
"modified": "2024-01-15T10:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Conflict
|
||||
|
||||
When server and client both changed the same file since the last common snapshot:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "hello.md",
|
||||
"serverHash": "a1b2c3...",
|
||||
"clientHash": "d4e5f6...",
|
||||
"serverModified": "2024-01-15T10:00:00Z",
|
||||
"clientModified": "2024-01-15T10:05:00Z",
|
||||
"strategy": "last-write-wins"
|
||||
}
|
||||
```
|
||||
|
||||
### Resolution Strategies
|
||||
|
||||
- `last-write-wins` (default): Use the most recently modified version
|
||||
- `rename-both`: Keep both versions, renaming one as `file (conflict).md`
|
||||
- `manual-merge`: Flag for UI resolution
|
||||
- `server-wins`: Always use server version
|
||||
- `client-wins`: Always use client version
|
||||
|
||||
## Endpoints
|
||||
|
||||
All endpoints require authentication via `X-API-Key` header.
|
||||
|
||||
### POST /api/sync/init
|
||||
|
||||
Initialize a new sync session.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"deviceId": "macbook-pro-1",
|
||||
"rootPath": "/Users/alice/Documents/md-hub"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"snapshotId": "snap:macbook-pro-1:1234567890",
|
||||
"serverSnapshot": [
|
||||
{
|
||||
"path": "hello.md",
|
||||
"hash": "a1b2c3...",
|
||||
"size": 42,
|
||||
"modified": "2024-01-15T10:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/sync/delta
|
||||
|
||||
Upload client changes and receive server changes + conflicts.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"snapshotId": "snap:macbook-pro-1:1234567890",
|
||||
"clientDelta": [
|
||||
{
|
||||
"type": "update",
|
||||
"path": "hello.md",
|
||||
"hash": "d4e5f6...",
|
||||
"size": 50,
|
||||
"modified": "2024-01-15T10:05:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"serverDelta": [
|
||||
{
|
||||
"type": "create",
|
||||
"path": "new-guide.md",
|
||||
"hash": "g7h8i9...",
|
||||
"size": 100,
|
||||
"modified": "2024-01-15T10:02:00Z"
|
||||
}
|
||||
],
|
||||
"conflicts": [
|
||||
{
|
||||
"path": "hello.md",
|
||||
"serverHash": "a1b2c3...",
|
||||
"clientHash": "d4e5f6...",
|
||||
"serverModified": "2024-01-15T10:00:00Z",
|
||||
"clientModified": "2024-01-15T10:05:00Z",
|
||||
"strategy": "last-write-wins"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/sync/resolve
|
||||
|
||||
Resolve conflicts and finalize the sync.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"snapshotId": "snap:macbook-pro-1:1234567890",
|
||||
"resolutions": [
|
||||
{
|
||||
"path": "hello.md",
|
||||
"strategy": "server-wins"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"newSnapshotId": "snap:macbook-pro-1:1234567999"
|
||||
}
|
||||
```
|
||||
|
||||
### GET /api/content/{hash}
|
||||
|
||||
Fetch raw file content by content hash (immutable, cache-friendly).
|
||||
|
||||
**Response:** Raw file bytes with headers:
|
||||
```
|
||||
Content-Type: application/octet-stream
|
||||
Cache-Control: public, max-age=31536000, immutable
|
||||
ETag: "a1b2c3..."
|
||||
```
|
||||
|
||||
## Protocol Flow
|
||||
|
||||
```
|
||||
1. Client calls POST /api/sync/init
|
||||
→ Receives server snapshot
|
||||
|
||||
2. Client computes local delta
|
||||
(fsevents + local state DB)
|
||||
|
||||
3. Client POSTs delta to /api/sync/delta
|
||||
→ Receives serverDelta + conflicts
|
||||
|
||||
4. If conflicts exist:
|
||||
a. Client resolves conflicts (auto or UI)
|
||||
b. Client POSTs resolutions to /api/sync/resolve
|
||||
→ Receives newSnapshotId
|
||||
|
||||
5. Client applies serverDelta to local filesystem
|
||||
(downloads content via /api/content/{hash})
|
||||
|
||||
6. Both sides now have the same snapshot
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
Sync endpoints require API key authentication:
|
||||
|
||||
```
|
||||
X-API-Key: <api-key>
|
||||
```
|
||||
|
||||
API keys are long-lived tokens scoped to a user and device. They are stored in the `api_keys` table with SHA-256 hashes.
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Server Architecture
|
||||
|
||||
```
|
||||
internal/sync/
|
||||
├── protocol.go # Types and constants
|
||||
├── service.go # Business logic (conflict resolution, snapshot management)
|
||||
├── store.go # Database operations
|
||||
```
|
||||
|
||||
### Database Schema
|
||||
|
||||
```sql
|
||||
-- API keys for native client authentication
|
||||
CREATE TABLE api_keys (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
key_hash TEXT NOT NULL UNIQUE,
|
||||
scopes TEXT NOT NULL DEFAULT 'sync:read,sync:write',
|
||||
created_at TEXT NOT NULL,
|
||||
expires_at TEXT,
|
||||
last_used_at TEXT,
|
||||
FOREIGN KEY(user_id) REFERENCES users(id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
-- Sync snapshots (point-in-time file trees)
|
||||
CREATE TABLE sync_snapshots (
|
||||
id TEXT PRIMARY KEY,
|
||||
device_id TEXT NOT NULL,
|
||||
user_id TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
FOREIGN KEY(user_id) REFERENCES users(id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
-- Files within a snapshot
|
||||
CREATE TABLE sync_files (
|
||||
snapshot_id TEXT NOT NULL,
|
||||
path TEXT NOT NULL,
|
||||
hash TEXT NOT NULL,
|
||||
size_bytes INTEGER NOT NULL DEFAULT 0,
|
||||
modified_at TEXT NOT NULL,
|
||||
PRIMARY KEY (snapshot_id, path),
|
||||
FOREIGN KEY(snapshot_id) REFERENCES sync_snapshots(id) ON DELETE CASCADE
|
||||
);
|
||||
```
|
||||
|
||||
### Conflict Detection
|
||||
|
||||
A conflict occurs when:
|
||||
1. The server file existed in the snapshot
|
||||
2. The server file has changed since the snapshot (hash changed)
|
||||
3. The client also changed the same file (different hash in delta)
|
||||
|
||||
### Content-Addressed Storage
|
||||
|
||||
All file content is stored in a content-addressable store using SHA-256 hashes:
|
||||
- Storage path: `<store-dir>/<hash[0:2]>/<hash[2:4]>/<hash>`
|
||||
- Immutable: once stored, content never changes
|
||||
- Deduplication: identical content shares the same storage
|
||||
|
||||
## Testing
|
||||
|
||||
The sync protocol includes table-driven tests that simulate client/server scenarios:
|
||||
|
||||
- `TestInitSyncCreatesSnapshot`: Validates snapshot generation
|
||||
- `TestApplyDeltaDetectsServerChanges`: Detects server-side modifications
|
||||
- `TestApplyDeltaDetectsConflicts`: Identifies concurrent modifications
|
||||
- `TestResolveConflictsCreatesNewSnapshot`: Validates conflict resolution
|
||||
- `TestGetContentReturnsFileBytes`: Tests content retrieval
|
||||
279
.project/sync-plan-browser.md
Normal file
279
.project/sync-plan-browser.md
Normal file
@@ -0,0 +1,279 @@
|
||||
# Plan B: Browser-First Sync (Progressive Web App)
|
||||
|
||||
> **AGENT INSTRUCTION — READ FIRST**
|
||||
>
|
||||
> When you complete a phase or are ready for a code review, you **must** notify the opencode session that created this plan. Use the opencode HTTP API to send a message to session `ses_225742b62ffeOSelYyCe4IPwxL`.
|
||||
>
|
||||
> **Message format:**
|
||||
> ```
|
||||
> Subject: Plan B Review Ready — Phase X Complete
|
||||
> Body:
|
||||
> - Phase completed: [e.g., Phase 1]
|
||||
> - Files touched: [list of files added/modified]
|
||||
> - Summary: [2-3 sentence description of what was implemented]
|
||||
> - Tests: [which tests were added or updated]
|
||||
> - Blockers: [any issues or decisions needed]
|
||||
> ```
|
||||
>
|
||||
> **OpenCode API Specification:**
|
||||
> ```
|
||||
> POST https://api.opencode.ai/v1/sessions/ses_225742b62ffeOSelYyCe4IPwxL/messages
|
||||
> Content-Type: application/json
|
||||
> Authorization: Bearer <OPENCODE_API_KEY>
|
||||
>
|
||||
> Request body:
|
||||
> {
|
||||
> "role": "assistant",
|
||||
> "content": "Subject: Plan B Review Ready — Phase X Complete\n\nBody:\n- Phase completed: ...\n- Files touched: ...\n- Summary: ...\n- Tests: ...\n- Blockers: ..."
|
||||
> }
|
||||
> ```
|
||||
>
|
||||
> Wait for a response before starting the next phase unless explicitly told otherwise.
|
||||
|
||||
## Goal
|
||||
Extend the existing web application so it works offline, supports concurrent editing, and syncs changes when reconnected. No native client needed — works on macOS, Windows, Linux, iOS, Android via browser.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Current State (Already Built)
|
||||
- Server: Go HTTP + SQLite + file watcher + WebSocket hub
|
||||
- Browser: Preact + IndexedDB cache + `navigator.onLine` detection
|
||||
- Documents served at `/:path*` with `index.md` default
|
||||
- Real-time updates pushed via WebSocket when files change on disk
|
||||
|
||||
### What We Add
|
||||
|
||||
```
|
||||
Browser (existing + new):
|
||||
├── cache.js (exists) # IndexedDB: docs, content, hashes
|
||||
├── realtime.js (exists) # WebSocket, offline indicator
|
||||
├── sync.js (new) # Sync engine: queue, conflict detection, state machine
|
||||
├── editor.js (new) # ProseMirror or textarea with operational transforms
|
||||
└── sw.js (new) # Service Worker for offline serving
|
||||
|
||||
Server (existing + new):
|
||||
├── internal/sync/ (new)
|
||||
│ ├── protocol.go # Browser sync protocol (simpler than Plan A)
|
||||
│ ├── queue.go # Server-side edit queue per document
|
||||
│ ├── conflict.go # Three-way merge for text
|
||||
│ └── broadcast.go # WebSocket broadcast for live collaboration
|
||||
├── internal/auth/ (new) # Sessions + WebSocket auth
|
||||
└── httpserver/ (existing)
|
||||
├── handlers.go # Add: POST /api/documents/:path, WebSocket auth
|
||||
└── websocket.go # Extend: handle edit messages
|
||||
```
|
||||
|
||||
## Sync Protocol v1 (Browser)
|
||||
|
||||
### Data Model
|
||||
- **Document**: `{path, content, hash, version}`
|
||||
- **Client Edit**: `{path, baseVersion, newHash, patch}` where patch = diff from base
|
||||
- **Server Ack**: `{path, version, status: accepted|conflict}`
|
||||
- **Server Broadcast**: `{type: update, path, newContent, newVersion, author}`
|
||||
|
||||
### Endpoints
|
||||
|
||||
```
|
||||
GET /api/documents # List all docs (cached in IndexedDB)
|
||||
GET /api/documents/:path # Get document content (cached in IndexedDB)
|
||||
POST /api/documents/:path # Save document (requires auth)
|
||||
Body: {content, baseVersion, clientHash}
|
||||
Response: {version, status, conflict?}
|
||||
|
||||
WS /api/ws # WebSocket (exists, extend for auth + edits)
|
||||
Client → Server: {type: "edit", path, patch, baseVersion}
|
||||
Server → Client: {type: "ack", path, version}
|
||||
Server → Client: {type: "broadcast", path, patch, author}
|
||||
Client → Server: {type: "cursor", path, position} // Phase 2
|
||||
```
|
||||
|
||||
### Protocol Flow (Online)
|
||||
```
|
||||
1. User opens /some-doc
|
||||
2. Browser fetches content, caches in IndexedDB
|
||||
3. User edits → client generates patch (diff)
|
||||
4. Client sends WS: {type: "edit", path, patch, baseVersion}
|
||||
5. Server validates: baseVersion == current version?
|
||||
- Yes: apply patch, increment version, broadcast to other clients
|
||||
- No: return conflict, client must merge
|
||||
6. Other clients receive broadcast, update editor if viewing same doc
|
||||
```
|
||||
|
||||
### Protocol Flow (Offline)
|
||||
```
|
||||
1. User goes offline (detected by navigator.onLine + fetch failure)
|
||||
2. Offline banner appears
|
||||
3. User edits → changes queued in IndexedDB: `pending_edits` store
|
||||
4. User navigates → served from IndexedDB cache
|
||||
5. User comes back online
|
||||
6. Client replays queue: for each pending edit, POST to server
|
||||
7. Server responds with ack or conflict
|
||||
8. Conflicts shown in UI for user resolution
|
||||
```
|
||||
|
||||
### Conflict Resolution (Text)
|
||||
- Three-way merge using diff3 algorithm
|
||||
- If auto-merge succeeds: apply silently, show subtle notification
|
||||
- If auto-merge fails: show side-by-side diff in UI
|
||||
- User chooses: keep local, keep server, or edit merged result
|
||||
- Markdown-aware: try to merge at paragraph/section boundaries
|
||||
|
||||
## Browser Sync Engine
|
||||
|
||||
### IndexedDB Schema
|
||||
```js
|
||||
const DB_NAME = 'mdhub';
|
||||
const DB_VERSION = 2;
|
||||
|
||||
const stores = {
|
||||
documents: 'path', // {path, title, hash, version, syncedAt}
|
||||
content: 'path', // {path, content, hash, fetchedAt}
|
||||
pending_edits: 'id', // {id, path, patch, baseVersion, createdAt, retries}
|
||||
sync_state: 'key' // {key: 'lastSync', value: timestamp}
|
||||
};
|
||||
```
|
||||
|
||||
### sync.js Responsibilities
|
||||
- `queueEdit(path, patch)`: Add to pending_edits, try to send immediately
|
||||
- `syncPending()`: Replay queue when online, handle acks/conflicts
|
||||
- `applyBroadcast(path, patch)`: Apply real-time update from other user
|
||||
- `detectConflict(base, local, server)`: Run diff3, return merged or null
|
||||
- `saveToIndexedDB(path, content)`: Cache for offline reading
|
||||
|
||||
### Service Worker (sw.js)
|
||||
- Intercept navigation requests (`/:path*`)
|
||||
- If offline: serve from IndexedDB cache
|
||||
- If online: network first, update cache in background
|
||||
- Cache static assets (JS, CSS, fonts) for app shell
|
||||
- Enables "Add to Home Screen" PWA behavior
|
||||
|
||||
### Editor
|
||||
- Start with `<textarea>` + diff generation (simple, robust)
|
||||
- Phase 2: ProseMirror for structured editing, operational transforms
|
||||
- Auto-save: debounced 2s, queues edit immediately
|
||||
- Show sync status: unsaved → saving → saved → (offline: queued)
|
||||
|
||||
## Server Changes
|
||||
|
||||
### 1. Auth Middleware
|
||||
```go
|
||||
// Extend existing session system
|
||||
func authMiddleware(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
// Check session cookie
|
||||
// Set user in context
|
||||
// For WebSocket: authenticate on connection open
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Document Write Endpoint
|
||||
```go
|
||||
func handleDocumentSave(w http.ResponseWriter, r *http.Request) {
|
||||
path := chi.URLParam(r, "*")
|
||||
var req struct {
|
||||
Content string `json:"content"`
|
||||
BaseVersion int `json:"baseVersion"`
|
||||
ClientHash string `json:"clientHash"`
|
||||
}
|
||||
|
||||
// Verify baseVersion matches current
|
||||
// If mismatch: return 409 Conflict with server version
|
||||
// If match: save to disk, update version, broadcast WS
|
||||
}
|
||||
```
|
||||
|
||||
### 3. WebSocket Extension
|
||||
```go
|
||||
type WSMessage struct {
|
||||
Type string `json:"type"` // edit, ack, broadcast, cursor
|
||||
Path string `json:"path"`
|
||||
Patch string `json:"patch"` // unified diff
|
||||
BaseVersion int `json:"baseVersion"`
|
||||
Author string `json:"author"`
|
||||
}
|
||||
|
||||
func (h *Hub) handleEdit(msg WSMessage, client *Client) {
|
||||
// Validate auth
|
||||
// Apply to document
|
||||
// Broadcast to other clients viewing same path
|
||||
// Send ack to sender
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Versioned Documents
|
||||
- Add `version` column to `documents` table (integer, auto-increment per doc)
|
||||
- Add `document_versions` table for history:
|
||||
```sql
|
||||
CREATE TABLE document_versions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
path TEXT NOT NULL,
|
||||
version INTEGER NOT NULL,
|
||||
content_hash TEXT NOT NULL,
|
||||
author TEXT,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
UNIQUE(path, version)
|
||||
);
|
||||
```
|
||||
- Current content always on disk; versions are content hashes referencing `store`
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Offline Reading (1-2 weeks)
|
||||
- [x] IndexedDB cache for documents and content (done)
|
||||
- [x] Offline indicator UI (done)
|
||||
- [x] Service Worker for offline navigation
|
||||
- [x] Cache static assets for PWA
|
||||
- [x] Show "cached" badge on offline-loaded pages
|
||||
|
||||
### Phase 2: Edit + Queue (2 weeks)
|
||||
- [ ] Simple textarea editor at `/:path/edit`
|
||||
- [ ] Auto-save with debounce
|
||||
- [ ] Queue edits in IndexedDB when offline
|
||||
- [ ] Replay queue on reconnect with exponential backoff
|
||||
- [ ] Visual "sync pending" indicator
|
||||
|
||||
### Phase 3: Conflict Resolution (1-2 weeks)
|
||||
- [ ] Server versioning (add `version` to documents table)
|
||||
- [ ] `POST /api/documents/:path` with version check
|
||||
- [ ] Three-way merge on client (diff3 library)
|
||||
- [ ] Conflict UI: side-by-side or inline diff
|
||||
- [ ] Auto-merge for non-overlapping changes
|
||||
|
||||
### Phase 4: Real-Time Collaboration (2 weeks)
|
||||
- [ ] WebSocket edit messages
|
||||
- [ ] Operational transforms or patch-based sync
|
||||
- [ ] Presence awareness (who's editing, cursor positions)
|
||||
- [ ] Broadcast updates to all connected clients
|
||||
- [ ] Optimistic UI updates (show edit immediately, rollback on conflict)
|
||||
|
||||
### Phase 5: PWA Polish (1 week)
|
||||
- [ ] Manifest.json for "Add to Home Screen"
|
||||
- [ ] App shell architecture (instant load)
|
||||
- [ ] Background sync API for reliable offline queue
|
||||
- [ ] Push notifications for mentions/comments (future)
|
||||
|
||||
## Pros
|
||||
- Builds directly on existing architecture (no new binary)
|
||||
- Works everywhere with a browser (no platform-specific code)
|
||||
- Simpler auth (reuse session cookies)
|
||||
- Real-time collaboration possible
|
||||
- PWA can feel like native app
|
||||
|
||||
## Cons
|
||||
- Requires internet for editing (unless Phase 2 offline queue works well)
|
||||
- Browser storage limits (~60MB+ with storage permission)
|
||||
- Can't use native editors (VS Code, etc.)
|
||||
- WebSocket reliability on mobile/bad networks
|
||||
- Conflict UX constrained by browser capabilities
|
||||
|
||||
## Decision Needed
|
||||
- Do we need native editor integration (Plan A) or is browser enough?
|
||||
- Can we build Plan B first, then extract core sync for Plan A later?
|
||||
|
||||
## Shared with Plan A
|
||||
Both plans eventually need:
|
||||
- Authentication/authorization (API keys vs sessions)
|
||||
- Document versioning
|
||||
- Conflict resolution algorithms
|
||||
- Content-addressed storage (already exists)
|
||||
210
.project/sync-plan-native.md
Normal file
210
.project/sync-plan-native.md
Normal file
@@ -0,0 +1,210 @@
|
||||
# Plan A: Native Sync Protocol (macOS + Server)
|
||||
|
||||
> **AGENT INSTRUCTION — READ FIRST**
|
||||
>
|
||||
> When you complete a phase or are ready for a code review, you **must** notify the opencode session that created this plan. Use the opencode HTTP API to send a message to session `ses_225742b62ffeOSelYyCe4IPwxL`.
|
||||
>
|
||||
> **Message format:**
|
||||
> ```
|
||||
> Subject: Plan A Review Ready — Phase X Complete
|
||||
> Body:
|
||||
> - Phase completed: [e.g., Phase 1]
|
||||
> - Files touched: [list of files added/modified]
|
||||
> - Summary: [2-3 sentence description of what was implemented]
|
||||
> - Tests: [which tests were added or updated]
|
||||
> - Blockers: [any issues or decisions needed]
|
||||
> ```
|
||||
>
|
||||
> **OpenCode API Specification:**
|
||||
> ```
|
||||
> POST https://api.opencode.ai/v1/sessions/ses_225742b62ffeOSelYyCe4IPwxL/messages
|
||||
> Content-Type: application/json
|
||||
> Authorization: Bearer <OPENCODE_API_KEY>
|
||||
>
|
||||
> Request body:
|
||||
> {
|
||||
> "role": "assistant",
|
||||
> "content": "Subject: Plan A Review Ready — Phase X Complete\n\nBody:\n- Phase completed: ...\n- Files touched: ...\n- Summary: ...\n- Tests: ...\n- Blockers: ..."
|
||||
> }
|
||||
> ```
|
||||
>
|
||||
> Wait for a response before starting the next phase unless explicitly told otherwise.
|
||||
|
||||
## Goal
|
||||
Build a bidirectional filesystem sync protocol that lets a native macOS client keep a local folder in sync with the cairnquire server. The browser client continues to work via the existing WebSocket + REST API.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Server: Sync Service (`internal/sync/`)
|
||||
New package alongside existing services:
|
||||
|
||||
```
|
||||
internal/sync/
|
||||
├── protocol.go # Sync protocol types and state machine
|
||||
├── service.go # Business logic: resolve conflicts, apply changes
|
||||
├── handlers.go # HTTP endpoints for sync (wrapped in httpserver)
|
||||
├── store.go # Abstract over docs + store for sync operations
|
||||
└── watcher.go # Bridge file watcher events into sync protocol
|
||||
```
|
||||
|
||||
### macOS Client
|
||||
A separate Go binary in `apps/macos-sync/` (or `apps/sync/`):
|
||||
|
||||
```
|
||||
apps/macos-sync/
|
||||
├── cmd/md-sync/ # Main entry point
|
||||
├── internal/
|
||||
│ ├── client.go # HTTP client for sync protocol
|
||||
│ ├── fs.go # macOS filesystem watcher (fsevents)
|
||||
│ ├── sync.go # Local sync engine
|
||||
│ └── state.go # Local state DB (SQLite or JSON)
|
||||
```
|
||||
|
||||
## Sync Protocol v1
|
||||
|
||||
### Data Model
|
||||
- **Snapshot**: A point-in-time tree of `{path, hash, size, mtime}`
|
||||
- **Delta**: List of changes since a snapshot: `{type: create|update|delete|rename, path, oldPath?, hash, size}`
|
||||
- **Conflict**: When server and client both changed same file since last common snapshot
|
||||
|
||||
### Endpoints
|
||||
|
||||
```
|
||||
POST /api/sync/init
|
||||
Body: {deviceId, rootPath}
|
||||
Response: {snapshotId, serverSnapshot}
|
||||
|
||||
POST /api/sync/delta
|
||||
Body: {snapshotId, clientDelta}
|
||||
Response: {serverDelta, conflicts[]}
|
||||
|
||||
POST /api/sync/resolve
|
||||
Body: {snapshotId, resolutions[]}
|
||||
Response: {newSnapshotId}
|
||||
|
||||
GET /api/sync/content?hash=<hash>
|
||||
Response: raw file bytes (content-addressed, immutable)
|
||||
```
|
||||
|
||||
### Protocol Flow
|
||||
```
|
||||
1. Client calls /sync/init → gets server snapshot
|
||||
2. Client computes local delta (fsevents + local state)
|
||||
3. Client POSTs delta to /sync/delta
|
||||
4. Server responds with:
|
||||
- serverDelta (changes client needs to apply)
|
||||
- conflicts[] (need user resolution or auto-merge)
|
||||
5. Client resolves conflicts locally or via UI
|
||||
6. Client POSTs resolutions to /sync/resolve
|
||||
7. Server returns new snapshotId, both sides updated
|
||||
```
|
||||
|
||||
### Conflict Resolution Strategies
|
||||
- `last-write-wins` (default for non-conflicting files)
|
||||
- `rename-both` (keep both as `file.md` and `file (conflict).md`)
|
||||
- `manual-merge` (flag for UI)
|
||||
- `server-wins` / `client-wins` (per-file or global preference)
|
||||
|
||||
### Authentication
|
||||
- API keys (long-lived, revocable) — simpler for native clients
|
||||
- OAuth2 device flow — better for multi-user
|
||||
- Start with API keys: `X-API-Key: <token>` header
|
||||
|
||||
## macOS Client Design
|
||||
|
||||
### File Watcher
|
||||
- Use `github.com/fsnotify/fsevents` (Cgo wrapper around FSEvents)
|
||||
- Debounce rapid changes (500ms window)
|
||||
- Ignore `.git`, `.DS_Store`, temp files
|
||||
|
||||
### Local State
|
||||
- SQLite DB at `~/Library/Application Support/md-sync/state.db`
|
||||
- Tables: `snapshots`, `files`, `pending_deltas`, `conflicts`
|
||||
- Or simpler: JSON files + file hashes on disk
|
||||
|
||||
### Sync Engine
|
||||
- Runs as daemon / menubar app
|
||||
- Configurable sync interval (default: continuous via fsevents)
|
||||
- Offline queue: store deltas locally, sync when reconnected
|
||||
- Bandwidth-aware: skip large files on cellular
|
||||
|
||||
### UI (Optional Phase 2)
|
||||
- Menubar icon: sync status, last sync time, conflicts badge
|
||||
- Settings window: root folder, server URL, API key, exclusions
|
||||
- Conflict resolution dialog
|
||||
|
||||
## Server Changes Required
|
||||
|
||||
### 1. Sync Service
|
||||
```go
|
||||
type SyncService struct {
|
||||
docStore *docs.Service
|
||||
fileStore *store.Service
|
||||
watcher *fsnotify.Watcher
|
||||
}
|
||||
|
||||
func (s *SyncService) InitSync(deviceID string) (*Snapshot, error)
|
||||
func (s *SyncService) ApplyDelta(snapshotID string, delta Delta) (*DeltaResult, error)
|
||||
func (s *SyncService) ResolveConflicts(snapshotID string, resolutions []Resolution) (*Snapshot, error)
|
||||
```
|
||||
|
||||
### 2. Authentication
|
||||
- Add API key table to database
|
||||
- Middleware to accept `X-API-Key` or session cookie
|
||||
- API keys scoped to user + device
|
||||
|
||||
### 3. Content-Addressed Fetch
|
||||
- `GET /api/content/<hash>` — immutable, cache-friendly
|
||||
- Uses existing `store.Service` (already content-addressed)
|
||||
|
||||
### 4. Push Notifications (Optional)
|
||||
- Server-Sent Events or WebSocket to tell client "server changed"
|
||||
- Avoids client polling
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Server Sync Protocol (2-3 weeks)
|
||||
- [ ] Define protocol types and state machine
|
||||
- [ ] Implement `/api/sync/init` and snapshot generation
|
||||
- [ ] Implement `/api/sync/delta` with conflict detection
|
||||
- [ ] Add content-addressed fetch endpoint
|
||||
- [ ] Write sync protocol tests (table-driven, simulate client/server)
|
||||
- [ ] Document protocol in `docs/specs/sync-protocol-v1.md`
|
||||
|
||||
### Phase 2: macOS Sync Engine (2-3 weeks)
|
||||
- [ ] Project scaffold: Go module, build scripts
|
||||
- [ ] FSEvents watcher with debouncing
|
||||
- [ ] Local state SQLite DB
|
||||
- [ ] HTTP client for sync endpoints
|
||||
- [ ] Basic sync loop: init → delta → apply
|
||||
- [ ] Offline queue (store deltas, retry with backoff)
|
||||
- [ ] CLI tool: `md-sync --folder ./docs --server https://...`
|
||||
|
||||
### Phase 3: Conflict Resolution + Polish (1-2 weeks)
|
||||
- [ ] Auto-resolve strategies
|
||||
- [ ] Conflict file generation (`file (conflict).md`)
|
||||
- [ ] Menubar app wrapper (Swift or `github.com/progrium/macdriver`)
|
||||
- [ ] Settings/preferences
|
||||
- [ ] Binary signing + notarization for distribution
|
||||
|
||||
### Phase 4: Advanced Features (Future)
|
||||
- [ ] Selective sync (ignore patterns)
|
||||
- [ ] Bandwidth limiting
|
||||
- [ ] Version history / time machine from server snapshots
|
||||
- [ ] iOS client (shares core sync engine)
|
||||
|
||||
## Pros
|
||||
- Native performance, works with any editor (VS Code, Obsidian, etc.)
|
||||
- Familiar sync model (Dropbox, iCloud Drive)
|
||||
- Content-addressed storage makes deduplication trivial
|
||||
- macOS client opens door to iOS, Windows, Linux
|
||||
|
||||
## Cons
|
||||
- New authentication system (API keys)
|
||||
- Complex conflict resolution UX
|
||||
- macOS development overhead (codesign, notarization, FSEvents)
|
||||
- Diverges from browser-first architecture
|
||||
|
||||
## Decision Needed
|
||||
- Should we support both browser sync AND native sync, or pick one?
|
||||
- If both: shared sync service, different transport (WebSocket vs HTTP)
|
||||
Reference in New Issue
Block a user