docs: add project planning documents
This commit is contained in:
297
docs/specs/simplesync-protocol.md
Normal file
297
docs/specs/simplesync-protocol.md
Normal file
@@ -0,0 +1,297 @@
|
||||
# SimpleSync Protocol Specification
|
||||
|
||||
## Overview
|
||||
|
||||
SimpleSync is a content-addressed file synchronization protocol designed for asynchronous collaboration on markdown documents. It provides automatic bidirectional sync between a server (source of truth) and clients (filesystem watchers or web browsers).
|
||||
|
||||
## Design Goals
|
||||
|
||||
1. **Simplicity**: No branches, commits, or manual sync commands
|
||||
2. **Automatic**: Changes propagate without user intervention
|
||||
3. **Conflict-aware**: Detects concurrent edits, never silently loses data
|
||||
4. **Offline-capable**: Queue changes locally, sync when connected
|
||||
5. **Secure**: All content verified by cryptographic hash
|
||||
|
||||
## Concepts
|
||||
|
||||
### Content Addressing
|
||||
Every file is identified by the SHA-256 hash of its content. The content is immutable — if a file changes, it gets a new hash.
|
||||
|
||||
### Merkle Tree
|
||||
The directory state is represented as a Merkle tree where:
|
||||
- Leaf nodes are file hashes
|
||||
- Internal nodes are hashes of concatenated child hashes
|
||||
- The root hash represents the entire directory state
|
||||
|
||||
### Sync State
|
||||
Each client maintains:
|
||||
- `root_hash`: Hash of current directory Merkle root
|
||||
- `files`: Map of file paths to content hashes
|
||||
- `pending`: Queue of local changes not yet synced
|
||||
|
||||
## Message Format
|
||||
|
||||
All messages are JSON over WebSocket.
|
||||
|
||||
### Client → Server
|
||||
|
||||
#### Sync Request
|
||||
```json
|
||||
{
|
||||
"type": "sync_request",
|
||||
"client_id": "uuid-v4",
|
||||
"root_hash": "sha256-hex-64-chars",
|
||||
"files": {
|
||||
"getting-started.md": "sha256...",
|
||||
"api-reference.md": "sha256..."
|
||||
},
|
||||
"timestamp": "2024-01-15T14:30:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### Content Upload
|
||||
```json
|
||||
{
|
||||
"type": "content_upload",
|
||||
"hash": "sha256...",
|
||||
"content": "base64-encoded-content",
|
||||
"encoding": "base64"
|
||||
}
|
||||
```
|
||||
|
||||
#### Acknowledgment
|
||||
```json
|
||||
{
|
||||
"type": "ack",
|
||||
"message_id": "uuid-of-original-message",
|
||||
"status": "ok"
|
||||
}
|
||||
```
|
||||
|
||||
### Server → Client
|
||||
|
||||
#### Sync Response
|
||||
```json
|
||||
{
|
||||
"type": "sync_response",
|
||||
"server_root_hash": "sha256...",
|
||||
"missing_from_client": ["hash1", "hash2"],
|
||||
"missing_from_server": ["hash3"],
|
||||
"conflicts": ["getting-started.md"],
|
||||
"timestamp": "2024-01-15T14:30:01Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### Content Push
|
||||
```json
|
||||
{
|
||||
"type": "content_push",
|
||||
"hash": "sha256...",
|
||||
"content": "base64-encoded-content",
|
||||
"encoding": "base64",
|
||||
"path": "getting-started.md"
|
||||
}
|
||||
```
|
||||
|
||||
#### Conflict Notification
|
||||
```json
|
||||
{
|
||||
"type": "conflict",
|
||||
"path": "getting-started.md",
|
||||
"server_hash": "sha256...",
|
||||
"client_hash": "sha256...",
|
||||
"server_modified_at": "2024-01-15T14:30:00Z",
|
||||
"client_modified_at": "2024-01-15T14:25:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### Error
|
||||
```json
|
||||
{
|
||||
"type": "error",
|
||||
"code": "unauthorized",
|
||||
"message": "Session expired",
|
||||
"retryable": false
|
||||
}
|
||||
```
|
||||
|
||||
## Protocol Flow
|
||||
|
||||
### Normal Sync (No Conflicts)
|
||||
|
||||
```
|
||||
Client Server
|
||||
| |
|
||||
|-- sync_request -------------->|
|
||||
| root_hash: abc |
|
||||
| |
|
||||
| |-- Compare with server state
|
||||
| |-- Calculate deltas
|
||||
| |
|
||||
|<- sync_response --------------|
|
||||
| missing_from_client: [def] |
|
||||
| missing_from_server: [] |
|
||||
| conflicts: [] |
|
||||
| |
|
||||
|-- content_request(def) ------>|
|
||||
| |
|
||||
|<- content_push(def) ----------|
|
||||
| |
|
||||
|-- ack ----------------------->|
|
||||
```
|
||||
|
||||
### Upload Changes
|
||||
|
||||
```
|
||||
Client Server
|
||||
| |
|
||||
|-- sync_request -------------->|
|
||||
| root_hash: abc |
|
||||
| files: {a: hash1, b: hash2} |
|
||||
| |
|
||||
|<- sync_response --------------|
|
||||
| missing_from_server: [hash2] |
|
||||
| |
|
||||
|-- content_upload(hash2) ----->|
|
||||
| |
|
||||
| |-- Verify hash
|
||||
| |-- Store content
|
||||
| |-- Update database
|
||||
| |-- Broadcast to others
|
||||
| |
|
||||
|<- ack ------------------------|
|
||||
| |
|
||||
```
|
||||
|
||||
### Conflict Detection
|
||||
|
||||
```
|
||||
Client A Server Client B
|
||||
| | |
|
||||
| | |-- Edit file X
|
||||
| | |-- sync_request
|
||||
| |-- Update file X |
|
||||
| |-- Broadcast to A |
|
||||
| | |
|
||||
|-- Edit file X | |
|
||||
|-- sync_request -------------->| |
|
||||
| |-- Detect conflict |
|
||||
| |-- A has old hash |
|
||||
| |-- B has new hash |
|
||||
|<- conflict -------------------| |
|
||||
| path: X | |
|
||||
| server_hash: B_hash | |
|
||||
| client_hash: A_hash | |
|
||||
| | |
|
||||
```
|
||||
|
||||
## State Machine
|
||||
|
||||
```
|
||||
+--------+ sync_request
|
||||
| +-----------+
|
||||
| Idle | |
|
||||
| |<----------+
|
||||
+---+----+
|
||||
|
|
||||
| connect
|
||||
v
|
||||
+---------+---------+
|
||||
| |
|
||||
| Connected |
|
||||
| |
|
||||
+---------+---------+
|
||||
|
|
||||
| sync_request
|
||||
v
|
||||
+---------+---------+
|
||||
| |
|
||||
| Comparing |
|
||||
| |
|
||||
+---------+---------+
|
||||
|
|
||||
+-----------+-----------+
|
||||
| |
|
||||
| has_changes | no_changes
|
||||
v v
|
||||
+---------+---------+ +---------+---------+
|
||||
| | | |
|
||||
| Transferring | | Idle |
|
||||
| | | |
|
||||
+---------+---------+ +-------------------+
|
||||
|
|
||||
| complete
|
||||
v
|
||||
+---------+---------+
|
||||
| |
|
||||
| Idle |
|
||||
| |
|
||||
+-------------------+
|
||||
^
|
||||
| conflict
|
||||
|
|
||||
+---------+---------+
|
||||
| |
|
||||
| Conflicted |
|
||||
| |
|
||||
+-------------------+
|
||||
```
|
||||
|
||||
## Error Codes
|
||||
|
||||
| Code | Description | Retryable |
|
||||
|------|-------------|-----------|
|
||||
| `unauthorized` | Session expired or invalid | No (re-authenticate) |
|
||||
| `forbidden` | Insufficient permissions | No |
|
||||
| `not_found` | Requested content hash unknown | Yes |
|
||||
| `too_large` | Content exceeds size limit | No |
|
||||
| `hash_mismatch` | Content doesn't match claimed hash | Yes |
|
||||
| `rate_limited` | Too many requests | Yes (with backoff) |
|
||||
| `server_error` | Internal server error | Yes |
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Authentication**: All WebSocket connections must authenticate via token in initial HTTP upgrade request
|
||||
2. **Authorization**: Server verifies read/write permissions before serving or accepting content
|
||||
3. **Hash Verification**: Server recomputes SHA-256 of received content and rejects mismatches
|
||||
4. **Size Limits**: Maximum file size enforced (configurable, default 10MB)
|
||||
5. **Rate Limiting**: Sync requests limited per client (configurable, default 10/minute)
|
||||
6. **Path Validation**: All file paths canonicalized and checked against allowlist
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### Hash Computation
|
||||
```go
|
||||
import "crypto/sha256"
|
||||
|
||||
func computeHash(content []byte) string {
|
||||
h := sha256.Sum256(content)
|
||||
return hex.EncodeToString(h[:])
|
||||
}
|
||||
```
|
||||
|
||||
### Merkle Root Computation
|
||||
```go
|
||||
func computeRootHash(files map[string]string) string {
|
||||
// Sort paths for determinism
|
||||
paths := sortedKeys(files)
|
||||
|
||||
hasher := sha256.New()
|
||||
for _, path := range paths {
|
||||
hasher.Write([]byte(path))
|
||||
hasher.Write([]byte(files[path]))
|
||||
}
|
||||
|
||||
return hex.EncodeToString(hasher.Sum(nil))
|
||||
}
|
||||
```
|
||||
|
||||
### WebSocket Connection
|
||||
- Ping/pong every 30 seconds
|
||||
- Connection timeout after 60 seconds without response
|
||||
- Auto-reconnect with exponential backoff (1s, 2s, 4s, 8s, max 60s)
|
||||
|
||||
## Version
|
||||
|
||||
**Protocol Version**: 1.0
|
||||
**Last Updated**: 2024-01-15
|
||||
Reference in New Issue
Block a user