69 lines
2.7 KiB
Markdown
69 lines
2.7 KiB
Markdown
# ADR-005: Storage Strategy
|
|
|
|
## Status
|
|
Accepted
|
|
|
|
## Context
|
|
The system needs to store: structured data (users, documents, comments), document content, file attachments, and search indexes. Requirements include: zero external dependencies, backup simplicity, and resistance to data corruption.
|
|
|
|
## Decision
|
|
**libsql (SQLite) for structured data + content-addressed filesystem for files**
|
|
|
|
### Implementation
|
|
|
|
1. **Database (libsql/SQLite)**:
|
|
- Single file: `data/db.sqlite`
|
|
- WAL mode for concurrent reads during writes
|
|
- Embedded — no separate process needed
|
|
- Backup: `cp db.sqlite db.sqlite.backup` (atomic with WAL checkpoint)
|
|
|
|
2. **File Storage (Content-Addressed)**:
|
|
- Directory: `data/files/`
|
|
- Layout: `data/files/ab/cd/abcdef1234567890abcdef1234567890abcdef12`
|
|
- SHA-256 hash as filename (hex encoded)
|
|
- First two characters as subdirectory (for filesystem performance)
|
|
- Immutable: write-once, never modify
|
|
|
|
3. **Document Mapping**:
|
|
- Database stores: `document_path → current_hash`
|
|
- History table stores all previous hashes
|
|
- Attachment metadata stores original filename + content hash
|
|
|
|
## Consequences
|
|
|
|
### Positive
|
|
- SQLite is the most tested database in the world (billions of deployments)
|
|
- libsql adds replication option without changing the API
|
|
- Content-addressed files are naturally deduplicated
|
|
- Filesystem storage is trivial to backup (rsync, tar)
|
|
- No network dependencies, works offline
|
|
- Single directory contains all state — easy to migrate
|
|
|
|
### Negative
|
|
- SQLite has write concurrency limits (mitigated by WAL + short transactions)
|
|
- Filesystem storage doesn't scale horizontally without shared storage
|
|
- No built-in file expiration/garbage collection (must implement orphan cleanup)
|
|
- Large attachments can bloat filesystem (mitigated by quotas)
|
|
|
|
## Alternatives Considered
|
|
|
|
### PostgreSQL
|
|
- **Pros**: Excellent concurrency, rich feature set, horizontal scaling
|
|
- **Cons**: Requires separate process, more complex backup, network dependency
|
|
- **Rejected**: Overkill for anticipated scale; SQLite is simpler and faster for read-heavy workloads
|
|
|
|
### Object Storage (S3/MinIO)
|
|
- **Pros**: Unlimited scale, CDN-friendly
|
|
- **Cons**: Network dependency, vendor lock-in, complex local development
|
|
- **Rejected**: Violates zero-dependency and simple backup requirements
|
|
|
|
### SQLite for Everything (BLOBs)
|
|
- **Pros**: Single file for everything
|
|
- **Cons**: Large BLOBs slow down database, harder to serve directly, backup size bloat
|
|
- **Rejected**: Separation of concerns — database for metadata, filesystem for content
|
|
|
|
## References
|
|
- SQLite WAL mode: https://sqlite.org/wal.html
|
|
- Content-addressable storage: https://git-scm.com/book/en/v2/Git-Internals-Git-Objects
|
|
- libsql: https://github.com/tursodatabase/libsql
|