Add conflict resolution diff UI and app branding
This commit is contained in:
272
.project/specs/deployment-guide.md
Normal file
272
.project/specs/deployment-guide.md
Normal file
@@ -0,0 +1,272 @@
|
||||
# Deployment Guide
|
||||
|
||||
## Requirements
|
||||
|
||||
- Docker 24.0+
|
||||
- Docker Compose 2.20+
|
||||
- 1GB RAM minimum (2GB recommended)
|
||||
- 10GB disk space
|
||||
- TLS certificate (Let's Encrypt or custom)
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/your-org/cairnquire.git
|
||||
cd cairnquire
|
||||
|
||||
# Copy and edit configuration
|
||||
cp .env.example .env
|
||||
# Edit .env with your domain, email, Postmark API key
|
||||
|
||||
# Start services
|
||||
docker-compose up -d
|
||||
|
||||
# Verify health
|
||||
curl https://your-domain.com/health
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Required | Default | Description |
|
||||
|----------|----------|---------|-------------|
|
||||
| `DOMAIN` | Yes | — | Public domain name |
|
||||
| `DB_PATH` | No | `/data/db.sqlite` | Database file path |
|
||||
| `FILESTORE_PATH` | No | `/data/files` | Content-addressed file storage |
|
||||
| `NOTES_PATH` | No | `/notes` | Watched markdown directory |
|
||||
| `POSTMARK_API_KEY` | Yes | — | Postmark server API token |
|
||||
| `POSTMARK_FROM` | No | `notifications@DOMAIN` | From email address |
|
||||
| `SESSION_SECRET` | Yes | — | 32-byte hex for session signing |
|
||||
| `ENV` | No | `production` | `development` or `production` |
|
||||
|
||||
### Docker Compose
|
||||
|
||||
```yaml
|
||||
version: "3.8"
|
||||
|
||||
services:
|
||||
app:
|
||||
build: .
|
||||
ports:
|
||||
- "8080:8080"
|
||||
volumes:
|
||||
- ./data:/data
|
||||
- ./notes:/notes:ro
|
||||
environment:
|
||||
- DOMAIN=${DOMAIN}
|
||||
- DB_PATH=/data/db.sqlite
|
||||
- FILESTORE_PATH=/data/files
|
||||
- NOTES_PATH=/notes
|
||||
- POSTMARK_API_KEY=${POSTMARK_API_KEY}
|
||||
- SESSION_SECRET=${SESSION_SECRET}
|
||||
env_file:
|
||||
- .env
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "--spider", "http://localhost:8080/health"]
|
||||
interval: 30s
|
||||
timeout: 3s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
read_only: true
|
||||
tmpfs:
|
||||
- /tmp:noexec,nosuid,size=100m
|
||||
|
||||
# Optional: Traefik for auto-HTTPS
|
||||
traefik:
|
||||
image: traefik:v3.0
|
||||
command:
|
||||
- "--api.insecure=false"
|
||||
- "--providers.docker=true"
|
||||
- "--providers.docker.exposedbydefault=false"
|
||||
- "--entrypoints.web.address=:80"
|
||||
- "--entrypoints.websecure.address=:443"
|
||||
- "--certificatesresolvers.letsencrypt.acme.tlschallenge=true"
|
||||
- "--certificatesresolvers.letsencrypt.acme.email=${EMAIL}"
|
||||
- "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
|
||||
ports:
|
||||
- "80:80"
|
||||
- "443:443"
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
- ./letsencrypt:/letsencrypt
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
restart: unless-stopped
|
||||
```
|
||||
|
||||
## Production Checklist
|
||||
|
||||
### Pre-Deployment
|
||||
- [ ] Change default `SESSION_SECRET` (generate: `openssl rand -hex 32`)
|
||||
- [ ] Configure `POSTMARK_API_KEY` with valid token
|
||||
- [ ] Set `DOMAIN` to public hostname
|
||||
- [ ] Configure DNS A/AAAA records
|
||||
- [ ] Set up SPF and DKIM for email domain
|
||||
- [ ] Review `.env` file — no default secrets
|
||||
|
||||
### Security
|
||||
- [ ] TLS 1.3 enforced (Traefik or reverse proxy)
|
||||
- [ ] HSTS header configured
|
||||
- [ ] Firewall rules: only 80/443 open
|
||||
- [ ] Fail2ban or rate limiting on SSH
|
||||
- [ ] Automatic security updates enabled
|
||||
- [ ] Backup encryption key stored securely
|
||||
|
||||
### Monitoring
|
||||
- [ ] Health check endpoint accessible
|
||||
- [ ] Log rotation configured
|
||||
- [ ] Disk space alerting (>80%)
|
||||
- [ ] Memory usage alerting
|
||||
- [ ] Uptime monitoring (external)
|
||||
|
||||
## Backup Procedures
|
||||
|
||||
### Automated Backup
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# backup.sh
|
||||
|
||||
BACKUP_DIR="/backups"
|
||||
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
|
||||
BACKUP_FILE="$BACKUP_DIR/md-hub-$TIMESTAMP.tar.gz"
|
||||
|
||||
# Create backup
|
||||
tar czf "$BACKUP_FILE" \
|
||||
-C /data \
|
||||
db.sqlite \
|
||||
files/
|
||||
|
||||
# Encrypt (optional)
|
||||
gpg --symmetric --cipher-algo AES256 "$BACKUP_FILE"
|
||||
rm "$BACKUP_FILE"
|
||||
|
||||
# Keep only last 30 days
|
||||
find "$BACKUP_DIR" -name "md-hub-*.tar.gz.gpg" -mtime +30 -delete
|
||||
```
|
||||
|
||||
**Cron**: `0 2 * * * /opt/cairnquire/backup.sh`
|
||||
|
||||
### Restore from Backup
|
||||
|
||||
```bash
|
||||
# Stop application
|
||||
docker-compose stop app
|
||||
|
||||
# Extract backup
|
||||
tar xzf md-hub-20240115_020000.tar.gz -C /data
|
||||
|
||||
# Restart
|
||||
docker-compose up -d app
|
||||
|
||||
# Verify
|
||||
curl https://your-domain.com/health
|
||||
```
|
||||
|
||||
## Upgrade Procedures
|
||||
|
||||
### Minor Update (patch version)
|
||||
```bash
|
||||
docker-compose pull
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
### Major Update
|
||||
1. Read release notes for breaking changes
|
||||
2. Create backup: `./backup.sh`
|
||||
3. Pull new image: `docker-compose pull`
|
||||
4. Run database migrations (auto on startup)
|
||||
5. Update configuration if needed
|
||||
6. Deploy: `docker-compose up -d`
|
||||
7. Verify health and functionality
|
||||
8. Rollback if issues: restore from backup
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Application Won't Start
|
||||
```bash
|
||||
# Check logs
|
||||
docker-compose logs -f app
|
||||
|
||||
# Verify configuration
|
||||
docker-compose exec app /server -check-config
|
||||
|
||||
# Test database
|
||||
docker-compose exec app sqlite3 /data/db.sqlite ".tables"
|
||||
```
|
||||
|
||||
### Sync Not Working
|
||||
```bash
|
||||
# Check file watcher
|
||||
docker-compose exec app ls -la /notes
|
||||
|
||||
# Verify WebSocket
|
||||
curl -i -N \
|
||||
-H "Connection: Upgrade" \
|
||||
-H "Upgrade: websocket" \
|
||||
-H "Sec-WebSocket-Key: $(openssl rand -base64 16)" \
|
||||
-H "Sec-WebSocket-Version: 13" \
|
||||
https://your-domain.com/ws
|
||||
```
|
||||
|
||||
### Email Not Sending
|
||||
```bash
|
||||
# Test Postmark connection
|
||||
curl -X POST \
|
||||
https://api.postmarkapp.com/server \
|
||||
-H "X-Postmark-Server-Token: YOUR_TOKEN" \
|
||||
-H "Accept: application/json"
|
||||
|
||||
# Check email queue
|
||||
docker-compose exec app sqlite3 /data/db.sqlite \
|
||||
"SELECT * FROM email_queue WHERE status='failed'"
|
||||
```
|
||||
|
||||
## Scaling
|
||||
|
||||
### Vertical Scaling
|
||||
- Increase container memory/CPU limits
|
||||
- SQLite WAL mode handles concurrent reads well
|
||||
- For heavy write loads, consider libsql replication
|
||||
|
||||
### Horizontal Scaling (Advanced)
|
||||
```yaml
|
||||
# Requires shared storage and load balancer
|
||||
services:
|
||||
app1:
|
||||
# ... same config ...
|
||||
environment:
|
||||
- DB_PATH=/shared/db.sqlite
|
||||
- FILESTORE_PATH=/shared/files
|
||||
|
||||
app2:
|
||||
# ... same config ...
|
||||
environment:
|
||||
- DB_PATH=/shared/db.sqlite
|
||||
- FILESTORE_PATH=/shared/files
|
||||
|
||||
nginx:
|
||||
image: nginx:alpine
|
||||
volumes:
|
||||
- ./nginx.conf:/etc/nginx/nginx.conf
|
||||
ports:
|
||||
- "80:80"
|
||||
```
|
||||
|
||||
**Note**: Horizontal scaling requires:
|
||||
- Shared filesystem (NFS, EFS)
|
||||
- libsql replication or external database
|
||||
- Sticky sessions for WebSocket
|
||||
|
||||
## Support
|
||||
|
||||
- **Documentation**: `/docs` route (when authenticated)
|
||||
- **Health Check**: `GET /health`
|
||||
- **Metrics**: `GET /metrics` (Prometheus format)
|
||||
- **Logs**: `docker-compose logs -f app`
|
||||
- **Issues**: https://github.com/your-org/cairnquire/issues
|
||||
456
.project/specs/security-specification.md
Normal file
456
.project/specs/security-specification.md
Normal file
@@ -0,0 +1,456 @@
|
||||
# Security Specification
|
||||
|
||||
## Threat Model
|
||||
|
||||
### Assets
|
||||
1. **Document Content**: Markdown files and attachments
|
||||
2. **User Credentials**: Passkeys, passwords, session tokens
|
||||
3. **Collaboration Data**: Comments, notifications, history
|
||||
4. **System State**: Database, configuration, logs
|
||||
|
||||
### Threat Actors
|
||||
1. **External Attacker**: No system access, attempts remote exploitation
|
||||
2. **Malicious User**: Authenticated, attempts unauthorized access
|
||||
3. **Compromised Client**: Legitimate user's device infected
|
||||
4. **Insider**: Administrator or developer with system access
|
||||
|
||||
### Threats
|
||||
1. **Supply Chain Attack**: Malicious dependency injection
|
||||
2. **Authentication Bypass**: Session hijacking, credential theft
|
||||
3. **Authorization Bypass**: Horizontal/vertical privilege escalation
|
||||
4. **Data Exfiltration**: Unauthorized document access
|
||||
5. **Data Tampering**: Modification without authorization
|
||||
6. **Denial of Service**: Resource exhaustion
|
||||
7. **Information Disclosure**: Secrets in logs/responses
|
||||
|
||||
## Security Controls
|
||||
|
||||
### 1. Supply Chain Security
|
||||
|
||||
**Policy**:
|
||||
- Only dependencies with >1M downloads/month or official Go team
|
||||
- All versions pinned in go.sum
|
||||
- No transitive dependencies without review
|
||||
- Prefer standard library when possible
|
||||
|
||||
**Implementation**:
|
||||
```
|
||||
# go.mod
|
||||
require (
|
||||
github.com/go-chi/chi/v5 v5.0.12
|
||||
github.com/tursodatabase/libsql-client-go v0.0.0-20240416075031-555ce55511f7
|
||||
// All versions pinned
|
||||
)
|
||||
|
||||
# Build
|
||||
RUN go mod verify # Verify checksums
|
||||
RUN go mod vendor # Optional: vendor for air-gapped
|
||||
```
|
||||
|
||||
**Audit**:
|
||||
- Weekly `govulncheck` scan
|
||||
- Dependency review in CI
|
||||
- No build-time network access (vendored or proxy with checksum)
|
||||
|
||||
### 2. Authentication Security
|
||||
|
||||
#### Passkey (WebAuthn)
|
||||
```go
|
||||
// Registration
|
||||
func registerPasskey(user User) (*Credential, error) {
|
||||
options, session, err := webauthn.BeginRegistration(user)
|
||||
// Store session data server-side (never client-side)
|
||||
// Return options to client
|
||||
}
|
||||
|
||||
func verifyRegistration(sessionID string, response ProtocolCredentialCreation) (*Credential, error) {
|
||||
session := getSessionData(sessionID)
|
||||
credential, err := webauthn.FinishRegistration(user, session, response)
|
||||
// Verify attestation
|
||||
// Store credential: ID, public key, sign count
|
||||
}
|
||||
```
|
||||
|
||||
#### Password (Argon2id)
|
||||
```go
|
||||
import "golang.org/x/crypto/argon2"
|
||||
|
||||
const (
|
||||
timeCost = 3
|
||||
memoryCost = 64 * 1024 // 64MB
|
||||
parallelism = 4
|
||||
saltLength = 16
|
||||
keyLength = 32
|
||||
)
|
||||
|
||||
func hashPassword(password string) (string, error) {
|
||||
salt := make([]byte, saltLength)
|
||||
if _, err := rand.Read(salt); err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
hash := argon2.IDKey([]byte(password), salt, timeCost, memoryCost, parallelism, keyLength)
|
||||
|
||||
// Encode to string: $argon2id$v=19$m=65536,t=3,p=4$<salt>$<hash>
|
||||
return encodeHash(salt, hash), nil
|
||||
}
|
||||
|
||||
func verifyPassword(password, encodedHash string) (bool, error) {
|
||||
salt, hash, params, err := decodeHash(encodedHash)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
computedHash := argon2.IDKey([]byte(password), salt, params.time, params.memory, params.parallelism, uint32(len(hash)))
|
||||
|
||||
// Constant-time comparison
|
||||
return subtle.ConstantTimeCompare(hash, computedHash) == 1, nil
|
||||
}
|
||||
```
|
||||
|
||||
#### Session Management
|
||||
```go
|
||||
func createSession(userID string, r *http.Request) (*Session, error) {
|
||||
token := make([]byte, 32)
|
||||
if _, err := rand.Read(token); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
session := &Session{
|
||||
ID: uuid.New().String(),
|
||||
UserID: userID,
|
||||
TokenHash: sha256.Sum256(token),
|
||||
CreatedAt: time.Now(),
|
||||
ExpiresAt: time.Now().Add(24 * time.Hour),
|
||||
IP: r.RemoteAddr,
|
||||
UserAgent: r.UserAgent(),
|
||||
}
|
||||
|
||||
// Store in database
|
||||
if err := db.CreateSession(session); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Set cookie
|
||||
http.SetCookie(w, &http.Cookie{
|
||||
Name: "session",
|
||||
Value: base64.URLEncoding.EncodeToString(token),
|
||||
Expires: session.ExpiresAt,
|
||||
HttpOnly: true,
|
||||
Secure: true, // HTTPS only
|
||||
SameSite: http.SameSiteStrictMode,
|
||||
Path: "/",
|
||||
})
|
||||
|
||||
return session, nil
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Authorization
|
||||
|
||||
#### Permission Model
|
||||
```go
|
||||
type Permission string
|
||||
|
||||
const (
|
||||
PermissionRead Permission = "read"
|
||||
PermissionWrite Permission = "write"
|
||||
PermissionAdmin Permission = "admin"
|
||||
)
|
||||
|
||||
type ResourceType string
|
||||
|
||||
const (
|
||||
ResourceGlobal ResourceType = "global"
|
||||
ResourceCollection ResourceType = "collection"
|
||||
ResourceDocument ResourceType = "document"
|
||||
)
|
||||
|
||||
func checkPermission(userID string, resourceType ResourceType, resourceID string, required Permission) (bool, error) {
|
||||
// Check explicit permission
|
||||
perm, err := db.GetPermission(userID, resourceType, resourceID)
|
||||
if err == nil && hasPermission(perm.Permission, required) {
|
||||
return true, nil
|
||||
}
|
||||
|
||||
// Check collection permission for documents
|
||||
if resourceType == ResourceDocument {
|
||||
collectionID := getDocumentCollection(resourceID)
|
||||
perm, err = db.GetPermission(userID, ResourceCollection, collectionID)
|
||||
if err == nil && hasPermission(perm.Permission, required) {
|
||||
return true, nil
|
||||
}
|
||||
}
|
||||
|
||||
// Check global permission
|
||||
perm, err = db.GetPermission(userID, ResourceGlobal, "")
|
||||
if err == nil && hasPermission(perm.Permission, required) {
|
||||
return true, nil
|
||||
}
|
||||
|
||||
// Check document public status
|
||||
if resourceType == ResourceDocument && required == PermissionRead {
|
||||
doc, err := db.GetDocument(resourceID)
|
||||
if err == nil && doc.PublicRead {
|
||||
return true, nil
|
||||
}
|
||||
}
|
||||
|
||||
return false, nil
|
||||
}
|
||||
```
|
||||
|
||||
#### Middleware
|
||||
```go
|
||||
func RequirePermission(resourceType ResourceType, required Permission) func(http.Handler) http.Handler {
|
||||
return func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
userID := getUserID(r.Context())
|
||||
resourceID := chi.URLParam(r, "id")
|
||||
|
||||
allowed, err := checkPermission(userID, resourceType, resourceID, required)
|
||||
if err != nil || !allowed {
|
||||
http.Error(w, "Forbidden", http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Usage:
|
||||
r.With(RequirePermission(ResourceDocument, PermissionRead)).
|
||||
Get("/docs/{id}", getDocumentHandler)
|
||||
```
|
||||
|
||||
### 4. Input Validation
|
||||
|
||||
#### Markdown Sanitization
|
||||
```go
|
||||
import "github.com/yuin/goldmark"
|
||||
import "github.com/yuin/goldmark/extension"
|
||||
import "github.com/yuin/goldmark/renderer/html"
|
||||
|
||||
func renderMarkdown(source []byte) ([]byte, error) {
|
||||
md := goldmark.New(
|
||||
goldmark.WithExtensions(
|
||||
extension.Table,
|
||||
extension.Strikethrough,
|
||||
extension.Linkify,
|
||||
// No raw HTML extension — we control HTML output
|
||||
),
|
||||
goldmark.WithRendererOptions(
|
||||
html.WithHardWraps(),
|
||||
html.WithXHTML(),
|
||||
html.WithUnsafe(), // DISABLED — no raw HTML passthrough
|
||||
),
|
||||
)
|
||||
|
||||
var buf bytes.Buffer
|
||||
if err := md.Convert(source, &buf); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return buf.Bytes(), nil
|
||||
}
|
||||
```
|
||||
|
||||
#### File Upload Validation
|
||||
```go
|
||||
func validateUpload(file io.Reader, filename string, maxSize int64) ([]byte, string, error) {
|
||||
// Size limit
|
||||
limited := io.LimitReader(file, maxSize+1)
|
||||
data, err := io.ReadAll(limited)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
if int64(len(data)) > maxSize {
|
||||
return nil, "", fmt.Errorf("file too large")
|
||||
}
|
||||
|
||||
// Magic number validation
|
||||
contentType := detectContentType(data)
|
||||
allowedTypes := map[string]bool{
|
||||
"image/jpeg": true,
|
||||
"image/png": true,
|
||||
"image/gif": true,
|
||||
"image/webp": true,
|
||||
"application/pdf": true,
|
||||
}
|
||||
if !allowedTypes[contentType] {
|
||||
return nil, "", fmt.Errorf("file type not allowed")
|
||||
}
|
||||
|
||||
// Extension whitelist
|
||||
ext := strings.ToLower(filepath.Ext(filename))
|
||||
allowedExts := map[string]bool{
|
||||
".jpg": true, ".jpeg": true, ".png": true,
|
||||
".gif": true, ".webp": true, ".pdf": true,
|
||||
}
|
||||
if !allowedExts[ext] {
|
||||
return nil, "", fmt.Errorf("file extension not allowed")
|
||||
}
|
||||
|
||||
// Compute hash
|
||||
hash := sha256.Sum256(data)
|
||||
hashStr := hex.EncodeToString(hash[:])
|
||||
|
||||
return data, hashStr, nil
|
||||
}
|
||||
```
|
||||
|
||||
#### Path Sanitization
|
||||
```go
|
||||
func sanitizePath(userPath string) (string, error) {
|
||||
// Clean the path
|
||||
clean := filepath.Clean(userPath)
|
||||
|
||||
// Prevent traversal
|
||||
if strings.Contains(clean, "..") {
|
||||
return "", fmt.Errorf("path traversal detected")
|
||||
}
|
||||
|
||||
// Ensure it's relative
|
||||
if filepath.IsAbs(clean) {
|
||||
return "", fmt.Errorf("absolute paths not allowed")
|
||||
}
|
||||
|
||||
// Join with base and verify it's within base
|
||||
fullPath := filepath.Join(baseDir, clean)
|
||||
if !strings.HasPrefix(filepath.Clean(fullPath), filepath.Clean(baseDir)) {
|
||||
return "", fmt.Errorf("path outside allowed directory")
|
||||
}
|
||||
|
||||
return clean, nil
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Transport Security
|
||||
|
||||
#### TLS Configuration
|
||||
```go
|
||||
tlsConfig := &tls.Config{
|
||||
MinVersion: tls.VersionTLS13,
|
||||
CurvePreferences: []tls.CurveID{
|
||||
tls.X25519,
|
||||
tls.CurveP256,
|
||||
},
|
||||
CipherSuites: []uint16{
|
||||
tls.TLS_AES_256_GCM_SHA384,
|
||||
tls.TLS_CHACHA20_POLY1305_SHA256,
|
||||
tls.TLS_AES_128_GCM_SHA256,
|
||||
},
|
||||
PreferServerCipherSuites: true,
|
||||
}
|
||||
```
|
||||
|
||||
#### Security Headers
|
||||
```go
|
||||
func securityHeaders(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Security-Policy",
|
||||
"default-src 'self'; "+
|
||||
"script-src 'self' 'nonce-{nonce}'; "+
|
||||
"style-src 'self' 'unsafe-inline'; "+
|
||||
"img-src 'self' data:; "+
|
||||
"connect-src 'self' wss:; "+
|
||||
"frame-ancestors 'none'; "+
|
||||
"base-uri 'self'; "+
|
||||
"form-action 'self';")
|
||||
w.Header().Set("X-Content-Type-Options", "nosniff")
|
||||
w.Header().Set("X-Frame-Options", "DENY")
|
||||
w.Header().Set("X-XSS-Protection", "1; mode=block")
|
||||
w.Header().Set("Referrer-Policy", "strict-origin-when-cross-origin")
|
||||
w.Header().Set("Strict-Transport-Security", "max-age=63072000; includeSubDomains; preload")
|
||||
w.Header().Set("Permissions-Policy", "camera=(), microphone=(), geolocation=()")
|
||||
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
### 6. Operational Security
|
||||
|
||||
#### Logging
|
||||
```go
|
||||
func logAudit(event AuditEvent) {
|
||||
entry := slog.With(
|
||||
"event_type", event.Type,
|
||||
"actor_id", redactIfSecret(event.ActorID),
|
||||
"resource_type", event.ResourceType,
|
||||
"resource_id", redactIfSecret(event.ResourceID),
|
||||
"ip", event.IP,
|
||||
"timestamp", event.Timestamp,
|
||||
)
|
||||
|
||||
// Never log: passwords, tokens, session IDs, API keys
|
||||
entry.Info("audit event")
|
||||
}
|
||||
|
||||
func redactIfSecret(value string) string {
|
||||
if looksLikeSecret(value) {
|
||||
return "[REDACTED]"
|
||||
}
|
||||
return value
|
||||
}
|
||||
```
|
||||
|
||||
#### Container Security
|
||||
```dockerfile
|
||||
# Dockerfile
|
||||
FROM golang:1.22-alpine AS builder
|
||||
# ... build ...
|
||||
|
||||
FROM alpine:3.19
|
||||
RUN addgroup -g 1000 appgroup && \
|
||||
adduser -u 1000 -G appgroup -s /bin/sh -D appuser
|
||||
|
||||
# Read-only root filesystem
|
||||
COPY --from=builder /app/server /server
|
||||
COPY --from=builder /app/web/dist /web
|
||||
|
||||
# Data directory (writable)
|
||||
RUN mkdir /data && chown appuser:appgroup /data
|
||||
VOLUME /data
|
||||
|
||||
USER appuser
|
||||
EXPOSE 8080
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=3s \
|
||||
CMD wget -q --spider http://localhost:8080/health || exit 1
|
||||
|
||||
ENTRYPOINT ["/server"]
|
||||
```
|
||||
|
||||
## Incident Response
|
||||
|
||||
### Detection
|
||||
- Automated alerts on: failed auth spikes, permission errors, large file uploads
|
||||
- Log analysis for anomalies
|
||||
- Health check failures
|
||||
|
||||
### Response
|
||||
1. **Isolate**: Block IP, revoke sessions, disable account if compromised
|
||||
2. **Assess**: Determine scope of breach, affected data
|
||||
3. **Contain**: Patch vulnerability, rotate secrets
|
||||
4. **Recover**: Restore from clean backup if necessary
|
||||
5. **Document**: Post-incident review, update security measures
|
||||
|
||||
## Compliance
|
||||
|
||||
### GDPR
|
||||
- Data minimization: only email stored, no other PII
|
||||
- Right to erasure: `/api/user/delete` endpoint
|
||||
- Data portability: `/api/user/export` endpoint
|
||||
- Consent: explicit opt-in for email notifications
|
||||
|
||||
### Audit Trail
|
||||
All security-relevant events logged:
|
||||
- Authentication attempts (success/failure)
|
||||
- Permission changes
|
||||
- Document access (if private)
|
||||
- Admin actions
|
||||
- Configuration changes
|
||||
|
||||
Retention: 90 days active, 1 year archive
|
||||
297
.project/specs/simplesync-protocol.md
Normal file
297
.project/specs/simplesync-protocol.md
Normal file
@@ -0,0 +1,297 @@
|
||||
# SimpleSync Protocol Specification
|
||||
|
||||
## Overview
|
||||
|
||||
SimpleSync is a content-addressed file synchronization protocol designed for asynchronous collaboration on markdown documents. It provides automatic bidirectional sync between a server (source of truth) and clients (filesystem watchers or web browsers).
|
||||
|
||||
## Design Goals
|
||||
|
||||
1. **Simplicity**: No branches, commits, or manual sync commands
|
||||
2. **Automatic**: Changes propagate without user intervention
|
||||
3. **Conflict-aware**: Detects concurrent edits, never silently loses data
|
||||
4. **Offline-capable**: Queue changes locally, sync when connected
|
||||
5. **Secure**: All content verified by cryptographic hash
|
||||
|
||||
## Concepts
|
||||
|
||||
### Content Addressing
|
||||
Every file is identified by the SHA-256 hash of its content. The content is immutable — if a file changes, it gets a new hash.
|
||||
|
||||
### Merkle Tree
|
||||
The directory state is represented as a Merkle tree where:
|
||||
- Leaf nodes are file hashes
|
||||
- Internal nodes are hashes of concatenated child hashes
|
||||
- The root hash represents the entire directory state
|
||||
|
||||
### Sync State
|
||||
Each client maintains:
|
||||
- `root_hash`: Hash of current directory Merkle root
|
||||
- `files`: Map of file paths to content hashes
|
||||
- `pending`: Queue of local changes not yet synced
|
||||
|
||||
## Message Format
|
||||
|
||||
All messages are JSON over WebSocket.
|
||||
|
||||
### Client → Server
|
||||
|
||||
#### Sync Request
|
||||
```json
|
||||
{
|
||||
"type": "sync_request",
|
||||
"client_id": "uuid-v4",
|
||||
"root_hash": "sha256-hex-64-chars",
|
||||
"files": {
|
||||
"getting-started.md": "sha256...",
|
||||
"api-reference.md": "sha256..."
|
||||
},
|
||||
"timestamp": "2024-01-15T14:30:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### Content Upload
|
||||
```json
|
||||
{
|
||||
"type": "content_upload",
|
||||
"hash": "sha256...",
|
||||
"content": "base64-encoded-content",
|
||||
"encoding": "base64"
|
||||
}
|
||||
```
|
||||
|
||||
#### Acknowledgment
|
||||
```json
|
||||
{
|
||||
"type": "ack",
|
||||
"message_id": "uuid-of-original-message",
|
||||
"status": "ok"
|
||||
}
|
||||
```
|
||||
|
||||
### Server → Client
|
||||
|
||||
#### Sync Response
|
||||
```json
|
||||
{
|
||||
"type": "sync_response",
|
||||
"server_root_hash": "sha256...",
|
||||
"missing_from_client": ["hash1", "hash2"],
|
||||
"missing_from_server": ["hash3"],
|
||||
"conflicts": ["getting-started.md"],
|
||||
"timestamp": "2024-01-15T14:30:01Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### Content Push
|
||||
```json
|
||||
{
|
||||
"type": "content_push",
|
||||
"hash": "sha256...",
|
||||
"content": "base64-encoded-content",
|
||||
"encoding": "base64",
|
||||
"path": "getting-started.md"
|
||||
}
|
||||
```
|
||||
|
||||
#### Conflict Notification
|
||||
```json
|
||||
{
|
||||
"type": "conflict",
|
||||
"path": "getting-started.md",
|
||||
"server_hash": "sha256...",
|
||||
"client_hash": "sha256...",
|
||||
"server_modified_at": "2024-01-15T14:30:00Z",
|
||||
"client_modified_at": "2024-01-15T14:25:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### Error
|
||||
```json
|
||||
{
|
||||
"type": "error",
|
||||
"code": "unauthorized",
|
||||
"message": "Session expired",
|
||||
"retryable": false
|
||||
}
|
||||
```
|
||||
|
||||
## Protocol Flow
|
||||
|
||||
### Normal Sync (No Conflicts)
|
||||
|
||||
```
|
||||
Client Server
|
||||
| |
|
||||
|-- sync_request -------------->|
|
||||
| root_hash: abc |
|
||||
| |
|
||||
| |-- Compare with server state
|
||||
| |-- Calculate deltas
|
||||
| |
|
||||
|<- sync_response --------------|
|
||||
| missing_from_client: [def] |
|
||||
| missing_from_server: [] |
|
||||
| conflicts: [] |
|
||||
| |
|
||||
|-- content_request(def) ------>|
|
||||
| |
|
||||
|<- content_push(def) ----------|
|
||||
| |
|
||||
|-- ack ----------------------->|
|
||||
```
|
||||
|
||||
### Upload Changes
|
||||
|
||||
```
|
||||
Client Server
|
||||
| |
|
||||
|-- sync_request -------------->|
|
||||
| root_hash: abc |
|
||||
| files: {a: hash1, b: hash2} |
|
||||
| |
|
||||
|<- sync_response --------------|
|
||||
| missing_from_server: [hash2] |
|
||||
| |
|
||||
|-- content_upload(hash2) ----->|
|
||||
| |
|
||||
| |-- Verify hash
|
||||
| |-- Store content
|
||||
| |-- Update database
|
||||
| |-- Broadcast to others
|
||||
| |
|
||||
|<- ack ------------------------|
|
||||
| |
|
||||
```
|
||||
|
||||
### Conflict Detection
|
||||
|
||||
```
|
||||
Client A Server Client B
|
||||
| | |
|
||||
| | |-- Edit file X
|
||||
| | |-- sync_request
|
||||
| |-- Update file X |
|
||||
| |-- Broadcast to A |
|
||||
| | |
|
||||
|-- Edit file X | |
|
||||
|-- sync_request -------------->| |
|
||||
| |-- Detect conflict |
|
||||
| |-- A has old hash |
|
||||
| |-- B has new hash |
|
||||
|<- conflict -------------------| |
|
||||
| path: X | |
|
||||
| server_hash: B_hash | |
|
||||
| client_hash: A_hash | |
|
||||
| | |
|
||||
```
|
||||
|
||||
## State Machine
|
||||
|
||||
```
|
||||
+--------+ sync_request
|
||||
| +-----------+
|
||||
| Idle | |
|
||||
| |<----------+
|
||||
+---+----+
|
||||
|
|
||||
| connect
|
||||
v
|
||||
+---------+---------+
|
||||
| |
|
||||
| Connected |
|
||||
| |
|
||||
+---------+---------+
|
||||
|
|
||||
| sync_request
|
||||
v
|
||||
+---------+---------+
|
||||
| |
|
||||
| Comparing |
|
||||
| |
|
||||
+---------+---------+
|
||||
|
|
||||
+-----------+-----------+
|
||||
| |
|
||||
| has_changes | no_changes
|
||||
v v
|
||||
+---------+---------+ +---------+---------+
|
||||
| | | |
|
||||
| Transferring | | Idle |
|
||||
| | | |
|
||||
+---------+---------+ +-------------------+
|
||||
|
|
||||
| complete
|
||||
v
|
||||
+---------+---------+
|
||||
| |
|
||||
| Idle |
|
||||
| |
|
||||
+-------------------+
|
||||
^
|
||||
| conflict
|
||||
|
|
||||
+---------+---------+
|
||||
| |
|
||||
| Conflicted |
|
||||
| |
|
||||
+-------------------+
|
||||
```
|
||||
|
||||
## Error Codes
|
||||
|
||||
| Code | Description | Retryable |
|
||||
|------|-------------|-----------|
|
||||
| `unauthorized` | Session expired or invalid | No (re-authenticate) |
|
||||
| `forbidden` | Insufficient permissions | No |
|
||||
| `not_found` | Requested content hash unknown | Yes |
|
||||
| `too_large` | Content exceeds size limit | No |
|
||||
| `hash_mismatch` | Content doesn't match claimed hash | Yes |
|
||||
| `rate_limited` | Too many requests | Yes (with backoff) |
|
||||
| `server_error` | Internal server error | Yes |
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Authentication**: All WebSocket connections must authenticate via token in initial HTTP upgrade request
|
||||
2. **Authorization**: Server verifies read/write permissions before serving or accepting content
|
||||
3. **Hash Verification**: Server recomputes SHA-256 of received content and rejects mismatches
|
||||
4. **Size Limits**: Maximum file size enforced (configurable, default 10MB)
|
||||
5. **Rate Limiting**: Sync requests limited per client (configurable, default 10/minute)
|
||||
6. **Path Validation**: All file paths canonicalized and checked against allowlist
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### Hash Computation
|
||||
```go
|
||||
import "crypto/sha256"
|
||||
|
||||
func computeHash(content []byte) string {
|
||||
h := sha256.Sum256(content)
|
||||
return hex.EncodeToString(h[:])
|
||||
}
|
||||
```
|
||||
|
||||
### Merkle Root Computation
|
||||
```go
|
||||
func computeRootHash(files map[string]string) string {
|
||||
// Sort paths for determinism
|
||||
paths := sortedKeys(files)
|
||||
|
||||
hasher := sha256.New()
|
||||
for _, path := range paths {
|
||||
hasher.Write([]byte(path))
|
||||
hasher.Write([]byte(files[path]))
|
||||
}
|
||||
|
||||
return hex.EncodeToString(hasher.Sum(nil))
|
||||
}
|
||||
```
|
||||
|
||||
### WebSocket Connection
|
||||
- Ping/pong every 30 seconds
|
||||
- Connection timeout after 60 seconds without response
|
||||
- Auto-reconnect with exponential backoff (1s, 2s, 4s, 8s, max 60s)
|
||||
|
||||
## Version
|
||||
|
||||
**Protocol Version**: 1.0
|
||||
**Last Updated**: 2024-01-15
|
||||
285
.project/specs/sync-protocol-v1.md
Normal file
285
.project/specs/sync-protocol-v1.md
Normal file
@@ -0,0 +1,285 @@
|
||||
# Sync Protocol v1 Specification
|
||||
|
||||
## Overview
|
||||
|
||||
The Native Sync Protocol enables bidirectional filesystem synchronization between a native macOS client and the cairnquire server. The browser client continues to operate via the existing WebSocket + REST API.
|
||||
|
||||
## Data Model
|
||||
|
||||
### Snapshot
|
||||
|
||||
A point-in-time tree of all files in the sync scope:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "snap:device-1:1234567890",
|
||||
"deviceId": "device-1",
|
||||
"createdAt": "2024-01-15T10:30:00Z",
|
||||
"files": [
|
||||
{
|
||||
"path": "hello.md",
|
||||
"hash": "a1b2c3...",
|
||||
"size": 42,
|
||||
"modified": "2024-01-15T10:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Delta
|
||||
|
||||
A list of changes since a snapshot:
|
||||
|
||||
```json
|
||||
{
|
||||
"changes": [
|
||||
{
|
||||
"type": "create|update|delete|rename",
|
||||
"path": "new-file.md",
|
||||
"oldPath": "old-file.md",
|
||||
"hash": "a1b2c3...",
|
||||
"size": 42,
|
||||
"modified": "2024-01-15T10:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Conflict
|
||||
|
||||
When server and client both changed the same file since the last common snapshot:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "hello.md",
|
||||
"serverHash": "a1b2c3...",
|
||||
"clientHash": "d4e5f6...",
|
||||
"serverModified": "2024-01-15T10:00:00Z",
|
||||
"clientModified": "2024-01-15T10:05:00Z",
|
||||
"strategy": "last-write-wins"
|
||||
}
|
||||
```
|
||||
|
||||
### Resolution Strategies
|
||||
|
||||
- `last-write-wins` (default): Use the most recently modified version
|
||||
- `rename-both`: Keep both versions, renaming one as `file (conflict).md`
|
||||
- `manual-merge`: Flag for UI resolution
|
||||
- `server-wins`: Always use server version
|
||||
- `client-wins`: Always use client version
|
||||
|
||||
## Endpoints
|
||||
|
||||
All endpoints require authentication via `X-API-Key` header.
|
||||
|
||||
### POST /api/sync/init
|
||||
|
||||
Initialize a new sync session.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"deviceId": "macbook-pro-1",
|
||||
"rootPath": "/Users/alice/Documents/md-hub"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"snapshotId": "snap:macbook-pro-1:1234567890",
|
||||
"serverSnapshot": [
|
||||
{
|
||||
"path": "hello.md",
|
||||
"hash": "a1b2c3...",
|
||||
"size": 42,
|
||||
"modified": "2024-01-15T10:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/sync/delta
|
||||
|
||||
Upload client changes and receive server changes + conflicts.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"snapshotId": "snap:macbook-pro-1:1234567890",
|
||||
"clientDelta": [
|
||||
{
|
||||
"type": "update",
|
||||
"path": "hello.md",
|
||||
"hash": "d4e5f6...",
|
||||
"size": 50,
|
||||
"modified": "2024-01-15T10:05:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"serverDelta": [
|
||||
{
|
||||
"type": "create",
|
||||
"path": "new-guide.md",
|
||||
"hash": "g7h8i9...",
|
||||
"size": 100,
|
||||
"modified": "2024-01-15T10:02:00Z"
|
||||
}
|
||||
],
|
||||
"conflicts": [
|
||||
{
|
||||
"path": "hello.md",
|
||||
"serverHash": "a1b2c3...",
|
||||
"clientHash": "d4e5f6...",
|
||||
"serverModified": "2024-01-15T10:00:00Z",
|
||||
"clientModified": "2024-01-15T10:05:00Z",
|
||||
"strategy": "last-write-wins"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/sync/resolve
|
||||
|
||||
Resolve conflicts and finalize the sync.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"snapshotId": "snap:macbook-pro-1:1234567890",
|
||||
"resolutions": [
|
||||
{
|
||||
"path": "hello.md",
|
||||
"strategy": "server-wins"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"newSnapshotId": "snap:macbook-pro-1:1234567999"
|
||||
}
|
||||
```
|
||||
|
||||
### GET /api/content/{hash}
|
||||
|
||||
Fetch raw file content by content hash (immutable, cache-friendly).
|
||||
|
||||
**Response:** Raw file bytes with headers:
|
||||
```
|
||||
Content-Type: application/octet-stream
|
||||
Cache-Control: public, max-age=31536000, immutable
|
||||
ETag: "a1b2c3..."
|
||||
```
|
||||
|
||||
## Protocol Flow
|
||||
|
||||
```
|
||||
1. Client calls POST /api/sync/init
|
||||
→ Receives server snapshot
|
||||
|
||||
2. Client computes local delta
|
||||
(fsevents + local state DB)
|
||||
|
||||
3. Client POSTs delta to /api/sync/delta
|
||||
→ Receives serverDelta + conflicts
|
||||
|
||||
4. If conflicts exist:
|
||||
a. Client resolves conflicts (auto or UI)
|
||||
b. Client POSTs resolutions to /api/sync/resolve
|
||||
→ Receives newSnapshotId
|
||||
|
||||
5. Client applies serverDelta to local filesystem
|
||||
(downloads content via /api/content/{hash})
|
||||
|
||||
6. Both sides now have the same snapshot
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
Sync endpoints require API key authentication:
|
||||
|
||||
```
|
||||
X-API-Key: <api-key>
|
||||
```
|
||||
|
||||
API keys are long-lived tokens scoped to a user and device. They are stored in the `api_keys` table with SHA-256 hashes.
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Server Architecture
|
||||
|
||||
```
|
||||
internal/sync/
|
||||
├── protocol.go # Types and constants
|
||||
├── service.go # Business logic (conflict resolution, snapshot management)
|
||||
├── store.go # Database operations
|
||||
```
|
||||
|
||||
### Database Schema
|
||||
|
||||
```sql
|
||||
-- API keys for native client authentication
|
||||
CREATE TABLE api_keys (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
key_hash TEXT NOT NULL UNIQUE,
|
||||
scopes TEXT NOT NULL DEFAULT 'sync:read,sync:write',
|
||||
created_at TEXT NOT NULL,
|
||||
expires_at TEXT,
|
||||
last_used_at TEXT,
|
||||
FOREIGN KEY(user_id) REFERENCES users(id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
-- Sync snapshots (point-in-time file trees)
|
||||
CREATE TABLE sync_snapshots (
|
||||
id TEXT PRIMARY KEY,
|
||||
device_id TEXT NOT NULL,
|
||||
user_id TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
FOREIGN KEY(user_id) REFERENCES users(id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
-- Files within a snapshot
|
||||
CREATE TABLE sync_files (
|
||||
snapshot_id TEXT NOT NULL,
|
||||
path TEXT NOT NULL,
|
||||
hash TEXT NOT NULL,
|
||||
size_bytes INTEGER NOT NULL DEFAULT 0,
|
||||
modified_at TEXT NOT NULL,
|
||||
PRIMARY KEY (snapshot_id, path),
|
||||
FOREIGN KEY(snapshot_id) REFERENCES sync_snapshots(id) ON DELETE CASCADE
|
||||
);
|
||||
```
|
||||
|
||||
### Conflict Detection
|
||||
|
||||
A conflict occurs when:
|
||||
1. The server file existed in the snapshot
|
||||
2. The server file has changed since the snapshot (hash changed)
|
||||
3. The client also changed the same file (different hash in delta)
|
||||
|
||||
### Content-Addressed Storage
|
||||
|
||||
All file content is stored in a content-addressable store using SHA-256 hashes:
|
||||
- Storage path: `<store-dir>/<hash[0:2]>/<hash[2:4]>/<hash>`
|
||||
- Immutable: once stored, content never changes
|
||||
- Deduplication: identical content shares the same storage
|
||||
|
||||
## Testing
|
||||
|
||||
The sync protocol includes table-driven tests that simulate client/server scenarios:
|
||||
|
||||
- `TestInitSyncCreatesSnapshot`: Validates snapshot generation
|
||||
- `TestApplyDeltaDetectsServerChanges`: Detects server-side modifications
|
||||
- `TestApplyDeltaDetectsConflicts`: Identifies concurrent modifications
|
||||
- `TestResolveConflictsCreatesNewSnapshot`: Validates conflict resolution
|
||||
- `TestGetContentReturnsFileBytes`: Tests content retrieval
|
||||
Reference in New Issue
Block a user