144 lines
4.0 KiB
Markdown
144 lines
4.0 KiB
Markdown
# Milestone 2: Sync Protocol
|
|
|
|
**Duration:** 2 weeks
|
|
**Goal:** Bidirectional file sync between server and filesystem/web client
|
|
|
|
## Tasks
|
|
|
|
### Week 3: Server-Side Sync
|
|
|
|
- [ ] Content-addressed filesystem implementation
|
|
- SHA-256 hash computation
|
|
- Directory layout: `store/ab/cd/abcdef...`
|
|
- Write-once semantics
|
|
- Deduplication verification
|
|
|
|
- [ ] File watcher (fsnotify)
|
|
- Watch configured directory recursively
|
|
- Debounce rapid changes (1s window)
|
|
- Ignore patterns (.git, temp files)
|
|
- Compute hash on change, update database
|
|
|
|
- [ ] WebSocket server
|
|
- Connection management (connection pool)
|
|
- Message framing and parsing
|
|
- Authentication over WebSocket (token validation)
|
|
- Heartbeat/ping-pong for connection health
|
|
|
|
- [ ] SimpleSync protocol server implementation
|
|
- Sync request/response handler
|
|
- Root hash computation (Merkle tree)
|
|
- Missing content detection
|
|
- Conflict detection logic
|
|
|
|
### Week 4: Client-Side Sync & Offline Support
|
|
|
|
- [ ] WebSocket client (Preact)
|
|
- Connection management with auto-reconnect
|
|
- Message queue for offline periods
|
|
- Exponential backoff for reconnection
|
|
|
|
- [ ] IndexedDB cache layer
|
|
- Store file content by hash
|
|
- Store sync state and pending changes
|
|
- LRU eviction for cache management
|
|
|
|
- [ ] File sync UI
|
|
- Sync status indicator
|
|
- Offline mode banner
|
|
- Pending changes list
|
|
- Manual sync trigger button
|
|
|
|
- [ ] Conflict detection display
|
|
- Visual indicator for conflicting files
|
|
- Basic conflict resolution UI (choose local/server)
|
|
|
|
## Acceptance Criteria
|
|
|
|
### Functional
|
|
- [ ] File changes on disk automatically sync to server within 5 seconds
|
|
- [ ] Web client receives real-time updates when files change on server
|
|
- [ ] Client can make changes offline and sync when reconnected
|
|
- [ ] Concurrent edits to different files succeed without conflict
|
|
- [ ] Concurrent edits to same file detected as conflict
|
|
- [ ] Content hash verified on every transfer (client and server)
|
|
- [ ] Sync works across browser refresh (IndexedDB persistence)
|
|
|
|
### Non-Functional
|
|
- [ ] WebSocket connections authenticated (reject unauthenticated)
|
|
- [ ] File watcher doesn't consume >1% CPU on idle
|
|
- [ ] IndexedDB storage cleared on logout
|
|
- [ ] Sync protocol documented with sequence diagrams
|
|
|
|
### Performance
|
|
- [ ] Sync of 100 files completes in <10 seconds
|
|
- [ ] WebSocket message overhead <1KB per sync cycle
|
|
- [ ] File watcher detects changes within 500ms
|
|
|
|
## Protocol Specification
|
|
|
|
### Message Types
|
|
|
|
```typescript
|
|
// Client → Server
|
|
interface SyncRequest {
|
|
type: 'sync_request';
|
|
clientId: string;
|
|
rootHash: string;
|
|
files: Record<string, string>; // path → hash
|
|
}
|
|
|
|
// Server → Client
|
|
interface SyncResponse {
|
|
type: 'sync_response';
|
|
serverRootHash: string;
|
|
missingFromClient: string[]; // hashes to download
|
|
missingFromServer: string[]; // hashes to upload
|
|
conflicts: string[]; // file paths with conflicts
|
|
}
|
|
|
|
// Content request
|
|
interface ContentRequest {
|
|
type: 'content_request';
|
|
hash: string;
|
|
}
|
|
|
|
// Content response
|
|
interface ContentResponse {
|
|
type: 'content_response';
|
|
hash: string;
|
|
content: string; // base64 encoded
|
|
}
|
|
```
|
|
|
|
### State Machine
|
|
|
|
```
|
|
[Idle] --sync_request--> [Comparing] --has_differences--> [Transferring] --complete--> [Idle]
|
|
--no_differences--> [Idle]
|
|
[Idle] --file_change--> [LocalChange] --sync--> [Comparing]
|
|
```
|
|
|
|
## Deliverables
|
|
|
|
1. SimpleSync protocol specification document
|
|
2. Sequence diagrams for all sync scenarios
|
|
3. WebSocket API documentation
|
|
4. Offline sync test suite
|
|
|
|
## Risk Mitigation
|
|
|
|
| Risk | Mitigation |
|
|
|------|-----------|
|
|
| WebSocket connection drops frequently | Auto-reconnect with exponential backoff, queue offline |
|
|
| Large files cause sync delays | Chunked transfer, progress indicators, size limits |
|
|
| Conflicts in frequently edited files | Optimistic UI, clear conflict resolution, email notification |
|
|
|
|
## Definition of Done
|
|
|
|
- [ ] All acceptance criteria pass
|
|
- [ ] Sync protocol tested with 2+ concurrent clients
|
|
- [ ] Offline/online transition tested
|
|
- [ ] Performance benchmarks meet targets
|
|
- [ ] Documentation complete
|