8.6 KiB
8.6 KiB
SimpleSync Protocol Specification
Overview
SimpleSync is a content-addressed file synchronization protocol designed for asynchronous collaboration on markdown documents. It provides automatic bidirectional sync between a server (source of truth) and clients (filesystem watchers or web browsers).
Design Goals
- Simplicity: No branches, commits, or manual sync commands
- Automatic: Changes propagate without user intervention
- Conflict-aware: Detects concurrent edits, never silently loses data
- Offline-capable: Queue changes locally, sync when connected
- Secure: All content verified by cryptographic hash
Concepts
Content Addressing
Every file is identified by the SHA-256 hash of its content. The content is immutable — if a file changes, it gets a new hash.
Merkle Tree
The directory state is represented as a Merkle tree where:
- Leaf nodes are file hashes
- Internal nodes are hashes of concatenated child hashes
- The root hash represents the entire directory state
Sync State
Each client maintains:
root_hash: Hash of current directory Merkle rootfiles: Map of file paths to content hashespending: Queue of local changes not yet synced
Message Format
All messages are JSON over WebSocket.
Client → Server
Sync Request
{
"type": "sync_request",
"client_id": "uuid-v4",
"root_hash": "sha256-hex-64-chars",
"files": {
"getting-started.md": "sha256...",
"api-reference.md": "sha256..."
},
"timestamp": "2024-01-15T14:30:00Z"
}
Content Upload
{
"type": "content_upload",
"hash": "sha256...",
"content": "base64-encoded-content",
"encoding": "base64"
}
Acknowledgment
{
"type": "ack",
"message_id": "uuid-of-original-message",
"status": "ok"
}
Server → Client
Sync Response
{
"type": "sync_response",
"server_root_hash": "sha256...",
"missing_from_client": ["hash1", "hash2"],
"missing_from_server": ["hash3"],
"conflicts": ["getting-started.md"],
"timestamp": "2024-01-15T14:30:01Z"
}
Content Push
{
"type": "content_push",
"hash": "sha256...",
"content": "base64-encoded-content",
"encoding": "base64",
"path": "getting-started.md"
}
Conflict Notification
{
"type": "conflict",
"path": "getting-started.md",
"server_hash": "sha256...",
"client_hash": "sha256...",
"server_modified_at": "2024-01-15T14:30:00Z",
"client_modified_at": "2024-01-15T14:25:00Z"
}
Error
{
"type": "error",
"code": "unauthorized",
"message": "Session expired",
"retryable": false
}
Protocol Flow
Normal Sync (No Conflicts)
Client Server
| |
|-- sync_request -------------->|
| root_hash: abc |
| |
| |-- Compare with server state
| |-- Calculate deltas
| |
|<- sync_response --------------|
| missing_from_client: [def] |
| missing_from_server: [] |
| conflicts: [] |
| |
|-- content_request(def) ------>|
| |
|<- content_push(def) ----------|
| |
|-- ack ----------------------->|
Upload Changes
Client Server
| |
|-- sync_request -------------->|
| root_hash: abc |
| files: {a: hash1, b: hash2} |
| |
|<- sync_response --------------|
| missing_from_server: [hash2] |
| |
|-- content_upload(hash2) ----->|
| |
| |-- Verify hash
| |-- Store content
| |-- Update database
| |-- Broadcast to others
| |
|<- ack ------------------------|
| |
Conflict Detection
Client A Server Client B
| | |
| | |-- Edit file X
| | |-- sync_request
| |-- Update file X |
| |-- Broadcast to A |
| | |
|-- Edit file X | |
|-- sync_request -------------->| |
| |-- Detect conflict |
| |-- A has old hash |
| |-- B has new hash |
|<- conflict -------------------| |
| path: X | |
| server_hash: B_hash | |
| client_hash: A_hash | |
| | |
State Machine
+--------+ sync_request
| +-----------+
| Idle | |
| |<----------+
+---+----+
|
| connect
v
+---------+---------+
| |
| Connected |
| |
+---------+---------+
|
| sync_request
v
+---------+---------+
| |
| Comparing |
| |
+---------+---------+
|
+-----------+-----------+
| |
| has_changes | no_changes
v v
+---------+---------+ +---------+---------+
| | | |
| Transferring | | Idle |
| | | |
+---------+---------+ +-------------------+
|
| complete
v
+---------+---------+
| |
| Idle |
| |
+-------------------+
^
| conflict
|
+---------+---------+
| |
| Conflicted |
| |
+-------------------+
Error Codes
| Code | Description | Retryable |
|---|---|---|
unauthorized |
Session expired or invalid | No (re-authenticate) |
forbidden |
Insufficient permissions | No |
not_found |
Requested content hash unknown | Yes |
too_large |
Content exceeds size limit | No |
hash_mismatch |
Content doesn't match claimed hash | Yes |
rate_limited |
Too many requests | Yes (with backoff) |
server_error |
Internal server error | Yes |
Security Considerations
- Authentication: All WebSocket connections must authenticate via token in initial HTTP upgrade request
- Authorization: Server verifies read/write permissions before serving or accepting content
- Hash Verification: Server recomputes SHA-256 of received content and rejects mismatches
- Size Limits: Maximum file size enforced (configurable, default 10MB)
- Rate Limiting: Sync requests limited per client (configurable, default 10/minute)
- Path Validation: All file paths canonicalized and checked against allowlist
Implementation Notes
Hash Computation
import "crypto/sha256"
func computeHash(content []byte) string {
h := sha256.Sum256(content)
return hex.EncodeToString(h[:])
}
Merkle Root Computation
func computeRootHash(files map[string]string) string {
// Sort paths for determinism
paths := sortedKeys(files)
hasher := sha256.New()
for _, path := range paths {
hasher.Write([]byte(path))
hasher.Write([]byte(files[path]))
}
return hex.EncodeToString(hasher.Sum(nil))
}
WebSocket Connection
- Ping/pong every 30 seconds
- Connection timeout after 60 seconds without response
- Auto-reconnect with exponential backoff (1s, 2s, 4s, 8s, max 60s)
Version
Protocol Version: 1.0
Last Updated: 2024-01-15