Complete Phase 1: parallel sync, IPC, theme colors, lazy CLI loading

- Sync: Parallelize message downloads with asyncio.gather (batch size 5)
- Sync: Increase HTTP semaphore from 2 to 5 concurrent requests
- Sync: Add IPC notifications to sync daemon after sync completes
- Mail: Replace all hardcoded RGB colors with theme variables
- Mail: Remove envelope icon/checkbox gap (padding cleanup)
- Mail: Add IPC listener for refresh notifications from sync
- Calendar: Style current time line with error color and solid line
- Tasks: Fix table not displaying (CSS grid to horizontal layout)
- CLI: Implement lazy command loading for faster startup (~12s to ~0.3s)
- Add PROJECT_PLAN.md with full improvement roadmap
- Add src/utils/ipc.py for Unix socket cross-app communication
This commit is contained in:
Bendt
2025-12-19 10:29:53 -05:00
parent a41d59e529
commit d4226caf0a
11 changed files with 1096 additions and 103 deletions

587
PROJECT_PLAN.md Normal file
View File

@@ -0,0 +1,587 @@
# LUK Project Plan
This document outlines planned improvements across the LUK applications (Mail, Calendar, Sync, Tasks).
---
## Bug Fixes
### Tasks App - Table Not Displaying (FIXED)
**Priority:** Critical
**File:** `src/tasks/app.py`
The tasks app was not showing the task table due to a CSS grid layout issue. The grid layout with mixed `dock` and `grid` positioning caused the main content area to have 0 height.
**Fix:** Changed from grid layout to horizontal layout with docked header/footer.
---
## Sync App
### Performance Improvements
#### 1. Parallelize Message Downloads
**Priority:** High
**Files:** `src/services/microsoft_graph/mail.py`, `src/services/microsoft_graph/client.py`
Currently, message downloads are sequential (`for` loop with `await`). This is a significant bottleneck.
**Current code pattern:**
```python
for msg_id in message_ids:
content = await client.get_message(msg_id)
await save_to_maildir(content)
```
**Proposed changes:**
1. Increase semaphore limit in `client.py` from 2 to 5 concurrent HTTP requests
2. Batch parallel downloads using `asyncio.gather()` with batches of 5-10 messages
3. Batch sync state writes every N messages instead of after each message (currently an I/O bottleneck in archive sync)
**Example implementation:**
```python
BATCH_SIZE = 5
async def fetch_batch(batch_ids):
return await asyncio.gather(*[client.get_message(id) for id in batch_ids])
for i in range(0, len(message_ids), BATCH_SIZE):
batch = message_ids[i:i + BATCH_SIZE]
results = await fetch_batch(batch)
for content in results:
await save_to_maildir(content)
# Write sync state every batch instead of every message
save_sync_state()
```
#### 2. Optimize Maildir Writes
**Priority:** Medium
**File:** `src/utils/mail_utils/maildir.py`
The `save_mime_to_maildir_async` function is a potential bottleneck. Consider:
- Batching file writes
- Using thread pool for I/O operations
---
### CLI Improvements
#### 3. Default to TUI Mode
**Priority:** Medium
**File:** `src/cli/sync.py`
Currently `luk sync` requires subcommands. Change to:
- `luk sync` → Opens TUI (interactive mode) by default
- `luk sync --once` / `luk sync -1` → One-shot sync
- `luk sync --daemon` / `luk sync -d` → Daemon mode
- `luk sync status` → Show sync status
---
### UI Consistency
#### 4. Navigation and Styling
**Priority:** Low
**File:** `src/cli/sync_dashboard.py`
- Add `j`/`k` keybindings for list navigation (vim-style)
- Use `border: round` in TCSS for consistency with other apps
- Add `.border_title` styling for list containers
#### 5. Notifications Toggle
**Priority:** Low
**Files:** `src/cli/sync_dashboard.py`, `src/utils/notifications.py`
Add a UI switch to enable/disable desktop notifications during sync.
---
## Inter-Process Communication (IPC)
### Overview
**Priority:** High
**Goal:** Enable real-time updates between apps when data changes:
- Mail app refreshes when sync downloads new messages
- Tasks app refreshes when syncing or when tasks are added from mail app
- Calendar app reloads when sync updates events
### Platform Options
#### macOS Options
1. **Distributed Notifications (NSDistributedNotificationCenter)**
- Native macOS IPC mechanism
- Simple pub/sub model
- Lightweight, no server needed
- Python: Use `pyobjc` (`Foundation.NSDistributedNotificationCenter`)
- Example: `CFNotificationCenterPostNotification()`
2. **Unix Domain Sockets**
- Works on both macOS and Linux
- Requires a listener process (daemon or embedded in sync)
- More complex but more flexible
- Can send structured data (JSON messages)
3. **Named Pipes (FIFO)**
- Simple, works on both platforms
- One-way communication
- Less suitable for bidirectional messaging
4. **File-based Signaling**
- Write a "dirty" file that apps watch via `watchdog` or `fsevents`
- Simple but has latency
- Works cross-platform
#### Linux Options
1. **D-Bus**
- Standard Linux IPC
- Python: Use `dbus-python` or `pydbus`
- Powerful but more complex
- Not available on macOS by default
2. **Unix Domain Sockets**
- Same as macOS, fully compatible
- Recommended for cross-platform compatibility
3. **systemd Socket Activation**
- If running as a systemd service
- Clean integration with Linux init
### Recommended Approach
Use **Unix Domain Sockets** for cross-platform compatibility:
```python
# Socket path
SOCKET_PATH = Path.home() / ".cache" / "luk" / "ipc.sock"
# Message types
class IPCMessage:
MAIL_UPDATED = "mail.updated"
TASKS_UPDATED = "tasks.updated"
CALENDAR_UPDATED = "calendar.updated"
# Sync daemon sends notifications
async def notify_mail_updated(folder: str):
await send_ipc_message({"type": IPCMessage.MAIL_UPDATED, "folder": folder})
# Mail app listens
async def listen_for_updates():
async for message in ipc_listener():
if message["type"] == IPCMessage.MAIL_UPDATED:
self.load_messages()
```
**Implementation files:**
- `src/utils/ipc.py` - IPC client/server utilities
- `src/cli/sync_daemon.py` - Add notification sending
- `src/mail/app.py` - Add listener for mail updates
- `src/tasks/app.py` - Add listener for task updates
- `src/calendar/app.py` - Add listener for calendar updates
---
## Calendar App
### Visual Improvements
#### 1. Current Time Hour Line Styling
**Priority:** High
**File:** `src/calendar/widgets/WeekGrid.py`
The current time indicator hour line should have a subtle contrasting background color to make it more visible.
#### 2. Cursor Hour Header Highlighting
**Priority:** Medium
**File:** `src/calendar/widgets/WeekGrid.py`
The hour header at cursor position should have a brighter background, similar to how the day header is highlighted when selected.
---
### Layout Improvements
#### 3. Responsive Detail Panel
**Priority:** Medium
**Files:** `src/calendar/app.py`, `src/calendar/widgets/`
When the terminal is wider than X characters (e.g., 120), show a side-docked detail panel for the selected event instead of a modal/overlay.
#### 4. Sidebar Mini-Calendar
**Priority:** Medium
**Files:** `src/calendar/app.py`, `src/calendar/widgets/MonthCalendar.py`
When the sidebar is toggled on, display a mini-calendar in the top-left corner showing:
- Current day highlighted
- The week(s) currently visible in the main WeekGrid pane
- Click/navigate to jump to a specific date
**Implementation:**
- Reuse existing `MonthCalendar` widget in compact mode
- Add reactive property to sync selected week with main pane
- Add to sidebar composition when toggled
---
### Microsoft Graph Integration
#### 5. Calendar Invites Sidebar
**Priority:** Medium
**Files:** `src/calendar/app.py`, `src/services/microsoft_graph/calendar.py`
Display a list of pending calendar invites from Microsoft Graph API in a sidebar panel:
- List pending invites (meeting requests)
- Show invite details (organizer, time, subject)
- Accept/Decline/Tentative actions
- No sync needed - fetch on-demand from API
**API Endpoints:**
- `GET /me/calendar/events?$filter=responseStatus/response eq 'notResponded'`
- `POST /me/events/{id}/accept`
- `POST /me/events/{id}/decline`
- `POST /me/events/{id}/tentativelyAccept`
**UI:**
- Toggle with keybinding (e.g., `i` for invites)
- Sidebar shows list of pending invites
- Detail view shows full invite info
- Action buttons/keys for response
---
### Search Feature
#### 6. Calendar Search
**Priority:** Medium
**Files:** `src/calendar/app.py`, `src/services/khal/client.py` (if khal supports search)
Add search functionality if the underlying backend (khal) supports it:
- `/` keybinding to open search input
- Search by event title, description, location
- Display search results in a modal or replace main view
- Navigate to selected event
**Check:** Does `khal search` command exist?
---
### Help System
#### 7. Help Toast (Keep Current Implementation)
**Priority:** Low
**File:** `src/calendar/app.py`
The `?` key shows a help toast using `self.notify()`. This pattern should be implemented in other apps (Mail, Tasks, Sync) for consistency.
---
## Mail App
### Layout Fixes
#### 1. Remove Envelope Icon/Checkbox Gap
**Priority:** High
**File:** `src/mail/widgets/EnvelopeListItem.py`
There's a 1-character space between the envelope icon and checkbox that should be removed for tighter layout.
---
### Theme Improvements
#### 2. Replace Hardcoded RGB Colors
**Priority:** High
**File:** `src/mail/email_viewer.tcss`
Multiple hardcoded RGB values should use Textual theme variables for better theming support:
| Line | Current | Replacement |
|------|---------|-------------|
| 6 | `border: round rgb(117, 106, 129)` | `border: round $border` |
| 46 | `background: rgb(55, 53, 57)` | `background: $surface` |
| 52, 57 | RGB label colors | Theme variables |
| 69 | `background: rgb(64, 62, 65)` | `background: $panel` |
| 169, 225, 228, 272, 274 | Various RGB colors | Theme variables |
---
### Keybindings
#### 3. Add Refresh Keybinding
**Priority:** Medium
**File:** `src/mail/app.py`
Add `r` keybinding to refresh/reload the message list.
#### 4. Add Mark Read/Unread Action
**Priority:** Medium
**Files:** `src/mail/app.py`, `src/mail/actions/` (new file)
Add action to toggle read/unread status on selected message(s).
---
### Search Feature
#### 5. Mail Search
**Priority:** Medium
**Files:** `src/mail/app.py`, backend integration
Add search functionality if the underlying mail backend supports it:
- `/` keybinding to open search input
- Search by subject, sender, body
- Display results in message list
- Check: Does himalaya or configured backend support search?
---
### UI Enhancements
#### 6. Folder Message Counts
**Priority:** Medium
**Files:** `src/mail/app.py`, `src/mail/widgets/`
Display total message count next to each folder name (e.g., "Inbox (42)").
#### 7. Sort Setting in Config/UI
**Priority:** Low
**Files:** `src/mail/config.py`, `src/mail/app.py`
Add configurable sort order (date, sender, subject) with UI toggle.
---
### Message Display
#### 8. URL Compression in Markdown View
**Priority:** Medium
**Files:** `src/mail/widgets/ContentContainer.py`, `src/mail/screens/LinkPanel.py`
Compress long URLs in the markdown view to ~50 characters with a nerdfont icon. The `_shorten_url` algorithm in `LinkPanel.py` can be reused.
**Considerations:**
- Cache processed markdown to avoid re-processing on scroll
- Store URL mapping for click handling
#### 9. Remove Emoji from Border Title
**Priority:** Low
**File:** `src/mail/widgets/ContentContainer.py` or `EnvelopeHeader.py`
Remove the envelope emoji prefix before message ID in border titles.
#### 10. Enhance Subject Styling
**Priority:** Medium
**File:** `src/mail/widgets/EnvelopeHeader.py`
- Move subject line to the top of the header
- Make it bolder/brighter for better visual hierarchy
---
## Tasks App
### Search Feature
#### 1. Task Search
**Priority:** Medium
**Files:** `src/tasks/app.py`, `src/services/dstask/client.py`
Add search functionality:
- `/` keybinding to open search input
- Search by task summary, notes, project, tags
- Display matching tasks
- Check: dstask likely supports filtering which can be used for search
**Implementation:**
- Add search input widget (TextInput)
- Filter tasks locally or via dstask command
- Update table to show only matching tasks
- Clear search with Escape
---
### Help System
#### 2. Implement Help Toast
**Priority:** Low
**File:** `src/tasks/app.py`
Add `?` keybinding to show help toast (matching Calendar app pattern).
**Note:** This is already implemented in the current code.
---
## Cross-App Improvements
### 1. Consistent Help System
Implement `?` key help toast in all apps using `self.notify()`:
- Mail: `src/mail/app.py`
- Tasks: `src/tasks/app.py` (already has it)
- Sync: `src/cli/sync_dashboard.py`
### 2. Consistent Navigation
Add vim-style `j`/`k` navigation to all list views across apps.
### 3. Consistent Border Styling
Use `border: round` and `.border_title` styling consistently in all TCSS files.
### 4. Consistent Search Interface
Implement `/` keybinding for search across all apps with similar UX:
- `/` opens search input
- Enter executes search
- Escape clears/closes search
- Results displayed in main view or filtered list
---
## Implementation Priority
### Phase 1: Critical/High Priority
1. ~~Tasks App: Fix table display~~ (DONE)
2. Sync: Parallelize message downloads
3. Mail: Replace hardcoded RGB colors
4. Mail: Remove envelope icon/checkbox gap
5. Calendar: Current time hour line styling
6. IPC: Implement cross-app refresh notifications
### Phase 2: Medium Priority
1. Sync: Default to TUI mode
2. Calendar: Cursor hour header highlighting
3. Calendar: Responsive detail panel
4. Calendar: Sidebar mini-calendar
5. Calendar: Calendar invites sidebar
6. Mail: Add refresh keybinding
7. Mail: Add mark read/unread action
8. Mail: Folder message counts
9. Mail: URL compression in markdown view
10. Mail: Enhance subject styling
11. Mail: Search feature
12. Tasks: Search feature
13. Calendar: Search feature
### Phase 3: Low Priority
1. Sync: UI consistency (j/k navigation, borders)
2. Sync: Notifications toggle
3. Calendar: Help toast (already implemented, replicate to other apps)
4. Mail: Remove emoji from border title
5. Mail: Sort setting in config/UI
6. Cross-app: Consistent help, navigation, and styling
---
## Technical Notes
### IPC Implementation Details
For macOS-first with Linux compatibility:
```python
# src/utils/ipc.py
import asyncio
import json
from pathlib import Path
SOCKET_PATH = Path.home() / ".cache" / "luk" / "ipc.sock"
class IPCServer:
"""Server for sending notifications to listening apps."""
async def broadcast(self, message: dict):
"""Send message to all connected clients."""
pass
class IPCClient:
"""Client for receiving notifications in apps."""
async def listen(self, callback):
"""Listen for messages and call callback."""
pass
```
### Backend Search Capabilities
| Backend | Search Support |
|---------|---------------|
| dstask | Filter by project/tag, summary search via shell |
| himalaya | Check `himalaya search` command |
| khal | Check `khal search` command |
| Microsoft Graph | Full text search via `$search` parameter |
---
## Notes
- All UI improvements should be tested with different terminal sizes
- Theme changes should be tested with multiple Textual themes
- Performance improvements should include before/after benchmarks
- New keybindings should be documented in each app's help toast
- IPC should gracefully handle missing socket (apps work standalone)
- Search should be responsive and not block UI
---
## Library Updates & Python Version Review
### Priority: Medium (Scheduled Review)
Periodically review the latest releases of heavily-used libraries to identify:
- Bug fixes that address issues we've encountered
- New features that could improve the codebase
- Deprecation warnings that need to be addressed
- Security updates
### Key Libraries to Review
| Library | Current Use | Review Focus |
|---------|-------------|--------------|
| **Textual** | All TUI apps | New widgets, performance improvements, theming changes, CSS features |
| **aiohttp** | Microsoft Graph API client | Async improvements, connection pooling |
| **msal** | Microsoft authentication | Token caching, auth flow improvements |
| **rich** | Console output (via Textual) | New formatting options |
| **orjson** | Fast JSON parsing | Performance improvements |
| **pyobjc** (macOS) | Notifications | API changes, compatibility |
### Textual Changelog Review Checklist
When reviewing Textual releases, check for:
1. **New widgets** - Could replace custom implementations
2. **CSS features** - New selectors, pseudo-classes, properties
3. **Theming updates** - New theme variables, design token changes
4. **Performance** - Rendering optimizations, memory improvements
5. **Breaking changes** - Deprecated APIs, signature changes
6. **Worker improvements** - Background task handling
### Python Version Upgrade
#### Current Status
- Check `.python-version` and `pyproject.toml` for current Python version
- Evaluate upgrade to Python 3.13 or 3.14 when stable
#### Python 3.13 Features to Consider
- Improved error messages
- Type system enhancements (`typing` module improvements)
- Performance optimizations (PEP 709 - inline comprehensions)
#### Python 3.14 Considerations
- **Status:** Currently in alpha/beta (as of Dec 2024)
- **Expected stable release:** October 2025
- **Recommendation:** Wait for stable release before adopting
- **Pre-release testing:** Can test compatibility in CI/CD before adoption
#### Upgrade Checklist
1. [ ] Review Python release notes for breaking changes
2. [ ] Check library compatibility (especially `pyobjc`, `textual`, `msal`)
3. [ ] Update `.python-version` (mise/pyenv)
4. [ ] Update `pyproject.toml` `requires-python` field
5. [ ] Run full test suite
6. [ ] Test on both macOS and Linux (if applicable)
7. [ ] Update CI/CD Python version
### Action Items
1. **Quarterly Review** - Schedule quarterly reviews of library changelogs
2. **Dependabot/Renovate** - Consider adding automated dependency update PRs
3. **Changelog Reading** - Before updating, read changelogs for breaking changes
4. **Test Coverage** - Ensure adequate test coverage before major updates

View File

@@ -373,8 +373,8 @@ class WeekGridBody(ScrollView):
# Style time label - highlight current time, dim outside work hours # Style time label - highlight current time, dim outside work hours
if is_current_time_row: if is_current_time_row:
secondary_color = self._get_theme_color("secondary") error_color = self._get_theme_color("error")
time_style = Style(color=secondary_color, bold=True) time_style = Style(color=error_color, bold=True)
elif ( elif (
row_index < self._work_day_start * rows_per_hour row_index < self._work_day_start * rows_per_hour
or row_index >= self._work_day_end * rows_per_hour or row_index >= self._work_day_end * rows_per_hour
@@ -388,13 +388,19 @@ class WeekGridBody(ScrollView):
# Event cells for each day # Event cells for each day
for col_idx, day_col in enumerate(self._days): for col_idx, day_col in enumerate(self._days):
cell_text, cell_style = self._render_event_cell(day_col, row_index, col_idx) cell_text, cell_style = self._render_event_cell(
day_col, row_index, col_idx, is_current_time_row
)
segments.append(Segment(cell_text, cell_style)) segments.append(Segment(cell_text, cell_style))
return Strip(segments) return Strip(segments)
def _render_event_cell( def _render_event_cell(
self, day_col: DayColumn, row_index: int, col_idx: int self,
day_col: DayColumn,
row_index: int,
col_idx: int,
is_current_time_row: bool = False,
) -> Tuple[str, Style]: ) -> Tuple[str, Style]:
"""Render a single cell for a day/time slot.""" """Render a single cell for a day/time slot."""
events_at_row = day_col.grid[row_index] if row_index < len(day_col.grid) else [] events_at_row = day_col.grid[row_index] if row_index < len(day_col.grid) else []
@@ -404,10 +410,16 @@ class WeekGridBody(ScrollView):
is_cursor = col_idx == self.cursor_col and row_index == self.cursor_row is_cursor = col_idx == self.cursor_col and row_index == self.cursor_row
# Get colors for current time line
error_color = self._get_theme_color("error") if is_current_time_row else None
if not events_at_row: if not events_at_row:
# Empty cell # Empty cell
if is_cursor: if is_cursor:
return ">" + " " * (day_col_width - 1), Style(reverse=True) return ">" + " " * (day_col_width - 1), Style(reverse=True)
elif is_current_time_row:
# Current time indicator line
return "" * day_col_width, Style(color=error_color, bold=True)
else: else:
# Grid line style # Grid line style
if row_index % rows_per_hour == 0: if row_index % rows_per_hour == 0:

View File

@@ -1,37 +1,55 @@
# CLI module for the application # CLI module for the application
# Uses lazy imports to speed up startup time
import click import click
import importlib
from .sync import sync
from .drive import drive
from .email import email
from .calendar import calendar
from .ticktick import ticktick
from .godspeed import godspeed
from .gitlab_monitor import gitlab_monitor
from .tasks import tasks
@click.group() class LazyGroup(click.Group):
"""A click Group that lazily loads subcommands."""
def __init__(self, *args, lazy_subcommands=None, **kwargs):
super().__init__(*args, **kwargs)
self._lazy_subcommands = lazy_subcommands or {}
def list_commands(self, ctx):
base = super().list_commands(ctx)
lazy = list(self._lazy_subcommands.keys())
return sorted(base + lazy)
def get_command(self, ctx, cmd_name):
if cmd_name in self._lazy_subcommands:
return self._load_command(cmd_name)
return super().get_command(ctx, cmd_name)
def _load_command(self, cmd_name):
module_path, attr_name = self._lazy_subcommands[cmd_name]
# Handle relative imports
if module_path.startswith("."):
module = importlib.import_module(module_path, package="src.cli")
else:
module = importlib.import_module(module_path)
return getattr(module, attr_name)
# Create CLI with lazy loading - commands only imported when invoked
@click.group(
cls=LazyGroup,
lazy_subcommands={
"sync": (".sync", "sync"),
"drive": (".drive", "drive"),
"email": (".email", "email"),
"mail": (".email", "email"), # alias
"calendar": (".calendar", "calendar"),
"ticktick": (".ticktick", "ticktick"),
"tt": (".ticktick", "ticktick"), # alias
"godspeed": (".godspeed", "godspeed"),
"gs": (".godspeed", "godspeed"), # alias
"gitlab_monitor": (".gitlab_monitor", "gitlab_monitor"),
"glm": (".gitlab_monitor", "gitlab_monitor"), # alias
"tasks": (".tasks", "tasks"),
},
)
def cli(): def cli():
"""Root command for the CLI.""" """LUK - Local Unix Kit for productivity."""
pass pass
cli.add_command(sync)
cli.add_command(drive)
cli.add_command(email)
cli.add_command(calendar)
cli.add_command(ticktick)
cli.add_command(godspeed)
cli.add_command(gitlab_monitor)
cli.add_command(tasks)
# Add 'mail' as an alias for email
cli.add_command(email, name="mail")
# Add 'tt' as a short alias for ticktick
cli.add_command(ticktick, name="tt")
# Add 'gs' as a short alias for godspeed
cli.add_command(godspeed, name="gs")
# Add 'glm' as a short alias for gitlab_monitor
cli.add_command(gitlab_monitor, name="glm")

View File

@@ -14,6 +14,7 @@ from typing import Optional, Dict, Any
from src.cli.sync import _sync_outlook_data, should_run_godspeed_sync, should_run_sweep from src.cli.sync import _sync_outlook_data, should_run_godspeed_sync, should_run_sweep
from src.cli.sync import run_godspeed_sync, run_task_sweep, load_sync_state from src.cli.sync import run_godspeed_sync, run_task_sweep, load_sync_state
from src.utils.ipc import notify_all, notify_refresh
class SyncDaemon: class SyncDaemon:
@@ -247,6 +248,13 @@ class SyncDaemon:
notify=self.config.get("notify", False), notify=self.config.get("notify", False),
) )
self.logger.info("Sync completed successfully") self.logger.info("Sync completed successfully")
# Notify all running TUI apps to refresh their data
results = await notify_all({"source": "sync_daemon"})
notified = [app for app, success in results.items() if success]
if notified:
self.logger.info(f"Notified apps to refresh: {', '.join(notified)}")
except Exception as e: except Exception as e:
self.logger.error(f"Sync failed: {e}") self.logger.error(f"Sync failed: {e}")

View File

@@ -10,6 +10,7 @@ from .actions.delete import delete_current
from src.services.taskwarrior import client as taskwarrior_client from src.services.taskwarrior import client as taskwarrior_client
from src.services.himalaya import client as himalaya_client from src.services.himalaya import client as himalaya_client
from src.utils.shared_config import get_theme_name from src.utils.shared_config import get_theme_name
from src.utils.ipc import IPCListener, IPCMessage
from textual.containers import Container, ScrollableContainer, Vertical, Horizontal from textual.containers import Container, ScrollableContainer, Vertical, Horizontal
from textual.timer import Timer from textual.timer import Timer
from textual.binding import Binding from textual.binding import Binding
@@ -149,7 +150,7 @@ class EmailViewerApp(App):
async def on_mount(self) -> None: async def on_mount(self) -> None:
self.alert_timer: Timer | None = None # Timer to throttle alerts self.alert_timer: Timer | None = None # Timer to throttle alerts
self.theme = get_theme_name() self.theme = get_theme_name()
self.title = "MaildirGTD" self.title = "LUK Mail"
self.query_one("#main_content").border_title = self.status_title self.query_one("#main_content").border_title = self.status_title
sort_indicator = "" if self.sort_order_ascending else "" sort_indicator = "" if self.sort_order_ascending else ""
self.query_one("#envelopes_list").border_title = f"1⃣ Emails {sort_indicator}" self.query_one("#envelopes_list").border_title = f"1⃣ Emails {sort_indicator}"
@@ -157,6 +158,10 @@ class EmailViewerApp(App):
self.query_one("#folders_list").border_title = "3⃣ Folders" self.query_one("#folders_list").border_title = "3⃣ Folders"
# Start IPC listener for refresh notifications from sync daemon
self._ipc_listener = IPCListener("mail", self._on_ipc_message)
self._ipc_listener.start()
self.fetch_accounts() self.fetch_accounts()
self.fetch_folders() self.fetch_folders()
worker = self.fetch_envelopes() worker = self.fetch_envelopes()
@@ -164,6 +169,12 @@ class EmailViewerApp(App):
self.query_one("#envelopes_list").focus() self.query_one("#envelopes_list").focus()
self.action_oldest() self.action_oldest()
def _on_ipc_message(self, message: IPCMessage) -> None:
"""Handle IPC messages from sync daemon."""
if message.event == "refresh":
# Schedule a reload on the main thread
self.call_from_thread(self.fetch_envelopes)
def compute_status_title(self): def compute_status_title(self):
metadata = self.message_store.get_metadata(self.current_message_id) metadata = self.message_store.get_metadata(self.current_message_id)
message_date = metadata["date"] if metadata else "N/A" message_date = metadata["date"] if metadata else "N/A"
@@ -820,6 +831,9 @@ class EmailViewerApp(App):
self.query_one("#envelopes_list").focus() self.query_one("#envelopes_list").focus()
def action_quit(self) -> None: def action_quit(self) -> None:
# Stop IPC listener before exiting
if hasattr(self, "_ipc_listener"):
self._ipc_listener.stop()
self.exit() self.exit()
def action_toggle_selection(self) -> None: def action_toggle_selection(self) -> None:

View File

@@ -3,7 +3,7 @@
#main_content, .list_view { #main_content, .list_view {
scrollbar-size: 1 1; scrollbar-size: 1 1;
border: round rgb(117, 106, 129); border: round $border;
height: 1fr; height: 1fr;
} }
@@ -43,18 +43,18 @@
#main_content:focus, .list_view:focus { #main_content:focus, .list_view:focus {
border: round $secondary; border: round $secondary;
background: rgb(55, 53, 57); background: $surface;
border-title-style: bold; border-title-style: bold;
} }
Label#task_prompt { Label#task_prompt {
padding: 1; padding: 1;
color: rgb(128,128,128); color: $text-muted;
} }
Label#task_prompt_label { Label#task_prompt_label {
padding: 1; padding: 1;
color: rgb(255, 216, 102); color: $warning;
} }
Label#message_label { Label#message_label {
@@ -66,7 +66,7 @@ StatusTitle {
width: 100%; width: 100%;
height: 1; height: 1;
color: $text; color: $text;
background: rgb(64, 62, 65); background: $panel;
content-align: center middle; content-align: center middle;
} }
@@ -113,8 +113,8 @@ EnvelopeListItem .envelope-row-3 {
} }
EnvelopeListItem .status-icon { EnvelopeListItem .status-icon {
width: 3; width: 2;
padding: 0 1 0 0; padding: 0;
color: $text-muted; color: $text-muted;
} }
@@ -124,7 +124,7 @@ EnvelopeListItem .status-icon.unread {
EnvelopeListItem .checkbox { EnvelopeListItem .checkbox {
width: 2; width: 2;
padding: 0 1 0 0; padding: 0;
} }
EnvelopeListItem .sender-name { EnvelopeListItem .sender-name {
@@ -166,11 +166,11 @@ EnvelopeListItem.selected {
GroupHeader { GroupHeader {
height: 1; height: 1;
width: 1fr; width: 1fr;
background: rgb(64, 62, 65); background: $panel;
} }
GroupHeader .group-header-label { GroupHeader .group-header-label {
color: rgb(160, 160, 160); color: $text-muted;
text-style: bold; text-style: bold;
padding: 0 1; padding: 0 1;
width: 1fr; width: 1fr;
@@ -222,10 +222,10 @@ GroupHeader .group-header-label {
#envelopes_list { #envelopes_list {
ListItem:odd { ListItem:odd {
background: rgb(45, 45, 46); background: $surface;
} }
ListItem:even { ListItem:even {
background: rgb(50, 50, 56); background: $surface-darken-1;
} }
& > ListItem { & > ListItem {
@@ -269,9 +269,9 @@ GroupHeader .group-header-label {
} }
Label.group_header { Label.group_header {
color: rgb(140, 140, 140); color: $text-muted;
text-style: bold; text-style: bold;
background: rgb(64, 62, 65); background: $panel;
width: 100%; width: 100%;
padding: 0 1; padding: 0 1;
} }
@@ -300,6 +300,3 @@ ContentContainer {
width: 100%; width: 100%;
height: 1fr; height: 1fr;
} }
.checkbox {
padding-right: 1;
}

View File

@@ -44,12 +44,17 @@ class EnvelopeListItem(Static):
EnvelopeListItem .status-icon { EnvelopeListItem .status-icon {
width: 2; width: 2;
padding: 0 1 0 0; padding: 0;
} }
EnvelopeListItem .checkbox { EnvelopeListItem .checkbox {
width: 2; width: 2;
padding: 0 1 0 0; padding: 0;
}
EnvelopeListItem .checkbox {
width: 2;
padding: 0;
} }
EnvelopeListItem .sender-name { EnvelopeListItem .sender-name {

View File

@@ -13,8 +13,8 @@ logging.getLogger("aiohttp.access").setLevel(logging.ERROR)
logging.getLogger("urllib3").setLevel(logging.ERROR) logging.getLogger("urllib3").setLevel(logging.ERROR)
logging.getLogger("asyncio").setLevel(logging.ERROR) logging.getLogger("asyncio").setLevel(logging.ERROR)
# Define a global semaphore for throttling - reduced for better compliance # Define a global semaphore for throttling - increased for better parallelization
semaphore = asyncio.Semaphore(2) semaphore = asyncio.Semaphore(5)
async def _handle_throttling_retry(func, *args, max_retries=3): async def _handle_throttling_retry(func, *args, max_retries=3):

View File

@@ -110,26 +110,47 @@ async def fetch_mail_async(
progress.update(task_id, total=len(messages_to_download), completed=0) progress.update(task_id, total=len(messages_to_download), completed=0)
downloaded_count = 0 downloaded_count = 0
for message in messages_to_download: # Download messages in parallel batches for better performance
BATCH_SIZE = 5
for i in range(0, len(messages_to_download), BATCH_SIZE):
# Check if task was cancelled/disabled # Check if task was cancelled/disabled
if is_cancelled and is_cancelled(): if is_cancelled and is_cancelled():
progress.console.print("Task cancelled, stopping inbox fetch") progress.console.print("Task cancelled, stopping inbox fetch")
break break
progress.console.print( batch = messages_to_download[i : i + BATCH_SIZE]
f"Processing message: {message.get('subject', 'No Subject')}", end="\r"
) # Create tasks for parallel download
await save_mime_to_maildir_async( async def download_message(message):
maildir_path, progress.console.print(
message, f"Processing message: {message.get('subject', 'No Subject')[:50]}",
attachments_dir, end="\r",
headers, )
progress, await save_mime_to_maildir_async(
dry_run, maildir_path,
download_attachments, message,
) attachments_dir,
progress.update(task_id, advance=1) headers,
downloaded_count += 1 progress,
dry_run,
download_attachments,
)
return 1
# Execute batch in parallel
tasks = [download_message(msg) for msg in batch]
results = await asyncio.gather(*tasks, return_exceptions=True)
# Count successful downloads
batch_success = sum(1 for r in results if r == 1)
downloaded_count += batch_success
progress.update(task_id, advance=len(batch))
# Log any errors
for idx, result in enumerate(results):
if isinstance(result, Exception):
progress.console.print(f"Error downloading message: {result}")
progress.update(task_id, completed=downloaded_count) progress.update(task_id, completed=downloaded_count)
progress.console.print(f"\nFinished downloading {downloaded_count} new messages.") progress.console.print(f"\nFinished downloading {downloaded_count} new messages.")
@@ -461,37 +482,57 @@ async def fetch_archive_mail_async(
# Update progress to reflect only the messages we actually need to download # Update progress to reflect only the messages we actually need to download
progress.update(task_id, total=len(messages_to_download), completed=0) progress.update(task_id, total=len(messages_to_download), completed=0)
# Load sync state once, we'll update it incrementally # Load sync state once, we'll update it after each batch for resilience
synced_ids = _load_archive_sync_state(maildir_path) if not dry_run else set() synced_ids = _load_archive_sync_state(maildir_path) if not dry_run else set()
downloaded_count = 0 downloaded_count = 0
for message in messages_to_download: # Download messages in parallel batches for better performance
BATCH_SIZE = 5
for i in range(0, len(messages_to_download), BATCH_SIZE):
# Check if task was cancelled/disabled # Check if task was cancelled/disabled
if is_cancelled and is_cancelled(): if is_cancelled and is_cancelled():
progress.console.print("Task cancelled, stopping archive fetch") progress.console.print("Task cancelled, stopping archive fetch")
break break
progress.console.print( batch = messages_to_download[i : i + BATCH_SIZE]
f"Processing archived message: {message.get('subject', 'No Subject')[:50]}", batch_msg_ids = []
end="\r",
)
# Save to .Archive folder instead of main maildir
await save_mime_to_maildir_async(
archive_dir, # Use archive_dir instead of maildir_path
message,
attachments_dir,
headers,
progress,
dry_run,
download_attachments,
)
progress.update(task_id, advance=1)
downloaded_count += 1
# Update sync state after each message for resilience # Create tasks for parallel download
# This ensures we don't try to re-upload this message in archive_mail_async async def download_message(message):
if not dry_run: progress.console.print(
synced_ids.add(message["id"]) f"Processing archived message: {message.get('subject', 'No Subject')[:50]}",
end="\r",
)
# Save to .Archive folder instead of main maildir
await save_mime_to_maildir_async(
archive_dir, # Use archive_dir instead of maildir_path
message,
attachments_dir,
headers,
progress,
dry_run,
download_attachments,
)
return message["id"]
# Execute batch in parallel
tasks = [download_message(msg) for msg in batch]
results = await asyncio.gather(*tasks, return_exceptions=True)
# Process results and collect successful message IDs
for result in results:
if isinstance(result, Exception):
progress.console.print(f"Error downloading archived message: {result}")
elif result:
batch_msg_ids.append(result)
downloaded_count += 1
progress.update(task_id, advance=len(batch))
# Update sync state after each batch (not each message) for resilience + performance
if not dry_run and batch_msg_ids:
synced_ids.update(batch_msg_ids)
_save_archive_sync_state(maildir_path, synced_ids) _save_archive_sync_state(maildir_path, synced_ids)
progress.update(task_id, completed=downloaded_count) progress.update(task_id, completed=downloaded_count)

View File

@@ -46,24 +46,20 @@ class TasksApp(App):
CSS = """ CSS = """
Screen { Screen {
layout: grid; layout: horizontal;
grid-size: 2;
grid-columns: auto 1fr;
grid-rows: auto 1fr auto auto;
} }
Header { Header {
column-span: 2; dock: top;
} }
Footer { Footer {
column-span: 2; dock: bottom;
} }
#sidebar { #sidebar {
width: 28; width: 28;
height: 100%; height: 100%;
row-span: 1;
} }
#sidebar.hidden { #sidebar.hidden {
@@ -116,7 +112,6 @@ class TasksApp(App):
background: $surface; background: $surface;
color: $text-muted; color: $text-muted;
padding: 0 1; padding: 0 1;
column-span: 2;
} }
#detail-pane { #detail-pane {
@@ -124,7 +119,6 @@ class TasksApp(App):
height: 50%; height: 50%;
border-top: solid $primary; border-top: solid $primary;
background: $surface; background: $surface;
column-span: 2;
} }
#detail-pane.hidden { #detail-pane.hidden {
@@ -154,7 +148,6 @@ class TasksApp(App):
border-top: solid $primary; border-top: solid $primary;
padding: 1; padding: 1;
background: $surface; background: $surface;
column-span: 2;
} }
#notes-pane.hidden { #notes-pane.hidden {

318
src/utils/ipc.py Normal file
View File

@@ -0,0 +1,318 @@
"""Inter-Process Communication using Unix Domain Sockets.
This module provides a simple pub/sub mechanism for cross-app notifications.
The sync daemon can broadcast messages when data changes, and TUI apps can
listen for these messages to refresh their displays.
Usage:
# In sync daemon (publisher):
from src.utils.ipc import notify_refresh
await notify_refresh("mail") # Notify mail app to refresh
await notify_refresh("calendar") # Notify calendar app to refresh
await notify_refresh("tasks") # Notify tasks app to refresh
# In TUI apps (subscriber):
from src.utils.ipc import IPCListener
class MyApp(App):
def on_mount(self):
self.ipc_listener = IPCListener("mail", self.on_refresh)
self.ipc_listener.start()
def on_unmount(self):
self.ipc_listener.stop()
async def on_refresh(self, message):
# Refresh the app's data
await self.refresh_data()
"""
import asyncio
import json
import os
import socket
import threading
from pathlib import Path
from typing import Callable, Optional, Any, Dict
# Socket paths for each app type
SOCKET_DIR = Path("~/.local/share/luk/ipc").expanduser()
SOCKET_PATHS = {
"mail": SOCKET_DIR / "mail.sock",
"calendar": SOCKET_DIR / "calendar.sock",
"tasks": SOCKET_DIR / "tasks.sock",
}
def ensure_socket_dir():
"""Ensure the socket directory exists."""
SOCKET_DIR.mkdir(parents=True, exist_ok=True)
def get_socket_path(app_type: str) -> Path:
"""Get the socket path for a given app type."""
if app_type not in SOCKET_PATHS:
raise ValueError(
f"Unknown app type: {app_type}. Must be one of: {list(SOCKET_PATHS.keys())}"
)
return SOCKET_PATHS[app_type]
class IPCMessage:
"""A message sent via IPC."""
def __init__(self, event: str, data: Optional[Dict[str, Any]] = None):
self.event = event
self.data = data or {}
def to_json(self) -> str:
return json.dumps({"event": self.event, "data": self.data})
@classmethod
def from_json(cls, json_str: str) -> "IPCMessage":
parsed = json.loads(json_str)
return cls(event=parsed["event"], data=parsed.get("data", {}))
async def notify_refresh(app_type: str, data: Optional[Dict[str, Any]] = None) -> bool:
"""Send a refresh notification to a specific app.
Args:
app_type: The type of app to notify ("mail", "calendar", "tasks")
data: Optional data to include with the notification
Returns:
True if the notification was sent successfully, False otherwise
"""
socket_path = get_socket_path(app_type)
if not socket_path.exists():
# No listener, that's okay
return False
try:
message = IPCMessage("refresh", data)
# Connect to the socket and send the message
reader, writer = await asyncio.open_unix_connection(str(socket_path))
writer.write((message.to_json() + "\n").encode())
await writer.drain()
writer.close()
await writer.wait_closed()
return True
except (ConnectionRefusedError, FileNotFoundError, OSError):
# Socket exists but no one is listening, or other error
return False
except Exception:
return False
async def notify_all(data: Optional[Dict[str, Any]] = None) -> Dict[str, bool]:
"""Send a refresh notification to all apps.
Args:
data: Optional data to include with the notification
Returns:
Dictionary of app_type -> success status
"""
results = {}
for app_type in SOCKET_PATHS:
results[app_type] = await notify_refresh(app_type, data)
return results
class IPCListener:
"""Listens for IPC messages on a Unix socket.
Usage:
listener = IPCListener("mail", on_message_callback)
listener.start()
# ... later ...
listener.stop()
"""
def __init__(
self,
app_type: str,
callback: Callable[[IPCMessage], Any],
):
"""Initialize the IPC listener.
Args:
app_type: The type of app ("mail", "calendar", "tasks")
callback: Function to call when a message is received.
Can be sync or async.
"""
self.app_type = app_type
self.callback = callback
self.socket_path = get_socket_path(app_type)
self._server: Optional[asyncio.AbstractServer] = None
self._running = False
self._loop: Optional[asyncio.AbstractEventLoop] = None
self._thread: Optional[threading.Thread] = None
async def _handle_client(
self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter
):
"""Handle an incoming client connection."""
try:
data = await reader.readline()
if data:
message_str = data.decode().strip()
if message_str:
message = IPCMessage.from_json(message_str)
# Call the callback (handle both sync and async)
result = self.callback(message)
if asyncio.iscoroutine(result):
await result
except Exception:
pass # Ignore errors from malformed messages
finally:
writer.close()
try:
await writer.wait_closed()
except Exception:
pass
async def _run_server(self):
"""Run the Unix socket server."""
ensure_socket_dir()
# Remove stale socket file if it exists
if self.socket_path.exists():
self.socket_path.unlink()
self._server = await asyncio.start_unix_server(
self._handle_client, path=str(self.socket_path)
)
# Set socket permissions (readable/writable by owner only)
os.chmod(self.socket_path, 0o600)
async with self._server:
await self._server.serve_forever()
def _run_in_thread(self):
"""Run the event loop in a separate thread."""
self._loop = asyncio.new_event_loop()
asyncio.set_event_loop(self._loop)
try:
self._loop.run_until_complete(self._run_server())
except asyncio.CancelledError:
pass
finally:
self._loop.close()
def start(self):
"""Start listening for IPC messages in a background thread."""
if self._running:
return
self._running = True
self._thread = threading.Thread(target=self._run_in_thread, daemon=True)
self._thread.start()
def stop(self):
"""Stop listening for IPC messages."""
if not self._running:
return
self._running = False
# Cancel the server
if self._server and self._loop:
self._loop.call_soon_threadsafe(self._server.close)
# Stop the event loop
if self._loop:
self._loop.call_soon_threadsafe(self._loop.stop)
# Wait for thread to finish
if self._thread:
self._thread.join(timeout=1.0)
# Clean up socket file
if self.socket_path.exists():
try:
self.socket_path.unlink()
except Exception:
pass
class AsyncIPCListener:
"""Async version of IPCListener for use within an existing event loop.
Usage in a Textual app:
class MyApp(App):
async def on_mount(self):
self.ipc_listener = AsyncIPCListener("mail", self.on_refresh)
await self.ipc_listener.start()
async def on_unmount(self):
await self.ipc_listener.stop()
async def on_refresh(self, message):
self.refresh_data()
"""
def __init__(
self,
app_type: str,
callback: Callable[[IPCMessage], Any],
):
self.app_type = app_type
self.callback = callback
self.socket_path = get_socket_path(app_type)
self._server: Optional[asyncio.AbstractServer] = None
async def _handle_client(
self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter
):
"""Handle an incoming client connection."""
try:
data = await reader.readline()
if data:
message_str = data.decode().strip()
if message_str:
message = IPCMessage.from_json(message_str)
result = self.callback(message)
if asyncio.iscoroutine(result):
await result
except Exception:
pass
finally:
writer.close()
try:
await writer.wait_closed()
except Exception:
pass
async def start(self):
"""Start the Unix socket server."""
ensure_socket_dir()
if self.socket_path.exists():
self.socket_path.unlink()
self._server = await asyncio.start_unix_server(
self._handle_client, path=str(self.socket_path)
)
os.chmod(self.socket_path, 0o600)
async def stop(self):
"""Stop the server and clean up."""
if self._server:
self._server.close()
await self._server.wait_closed()
if self.socket_path.exists():
try:
self.socket_path.unlink()
except Exception:
pass