luk/PROJECT_PLAN.md

# LUK Project Plan

This document outlines planned improvements across the LUK applications (Mail, Calendar, Sync, Tasks).

---

## Bug Fixes

### Tasks App - Table Not Displaying (FIXED)
**Priority:** Critical  
**File:** `src/tasks/app.py`

The tasks app was not showing the task table due to a CSS grid layout issue. The grid layout with mixed `dock` and `grid` positioning caused the main content area to have 0 height.

**Fix:** Changed from grid layout to horizontal layout with docked header/footer.

---

## Sync App

### Performance Improvements

#### 1. Parallelize Message Downloads
**Priority:** High  
**Files:** `src/services/microsoft_graph/mail.py`, `src/services/microsoft_graph/client.py`

Currently, message downloads are sequential (`for` loop with `await`). This is a significant bottleneck.

**Current code pattern:**
```python
for msg_id in message_ids:
    content = await client.get_message(msg_id)
    await save_to_maildir(content)
```

**Proposed changes:**
1. Increase semaphore limit in `client.py` from 2 to 5 concurrent HTTP requests
2. Batch parallel downloads using `asyncio.gather()` with batches of 5-10 messages
3. Batch sync state writes every N messages instead of after each message (currently an I/O bottleneck in archive sync)

**Example implementation:**
```python
BATCH_SIZE = 5

async def fetch_batch(batch_ids):
    return await asyncio.gather(*[client.get_message(id) for id in batch_ids])

for i in range(0, len(message_ids), BATCH_SIZE):
    batch = message_ids[i:i + BATCH_SIZE]
    results = await fetch_batch(batch)
    for content in results:
        await save_to_maildir(content)
    # Write sync state every batch instead of every message
    save_sync_state()
```

#### 2. Optimize Maildir Writes
**Priority:** Medium  
**File:** `src/utils/mail_utils/maildir.py`

The `save_mime_to_maildir_async` function is a potential bottleneck. Consider:
- Batching file writes
- Using thread pool for I/O operations

---

### CLI Improvements

#### 3. Default to TUI Mode
**Priority:** Medium  
**File:** `src/cli/sync.py`

Currently `luk sync` requires subcommands. Change to:
- `luk sync` → Opens TUI (interactive mode) by default
- `luk sync --once` / `luk sync -1` → One-shot sync
- `luk sync --daemon` / `luk sync -d` → Daemon mode
- `luk sync status` → Show sync status

---

### UI Consistency

#### 4. Navigation and Styling
**Priority:** Low  
**File:** `src/cli/sync_dashboard.py`

- Add `j`/`k` keybindings for list navigation (vim-style)
- Use `border: round` in TCSS for consistency with other apps
- Add `.border_title` styling for list containers

#### 5. Notifications Toggle
**Priority:** Low  
**Files:** `src/cli/sync_dashboard.py`, `src/utils/notifications.py`

Add a UI switch to enable/disable desktop notifications during sync.

---

## Inter-Process Communication (IPC)

### Overview
**Priority:** High  
**Goal:** Enable real-time updates between apps when data changes:
- Mail app refreshes when sync downloads new messages
- Tasks app refreshes when syncing or when tasks are added from mail app
- Calendar app reloads when sync updates events

### Platform Options

#### macOS Options

1. **Distributed Notifications (NSDistributedNotificationCenter)**
   - Native macOS IPC mechanism
   - Simple pub/sub model
   - Lightweight, no server needed
   - Python: Use `pyobjc` (`Foundation.NSDistributedNotificationCenter`)
   - Example: `CFNotificationCenterPostNotification()`

2. **Unix Domain Sockets**
   - Works on both macOS and Linux
   - Requires a listener process (daemon or embedded in sync)
   - More complex but more flexible
   - Can send structured data (JSON messages)

3. **Named Pipes (FIFO)**
   - Simple, works on both platforms
   - One-way communication
   - Less suitable for bidirectional messaging

4. **File-based Signaling**
   - Write a "dirty" file that apps watch via `watchdog` or `fsevents`
   - Simple but has latency
   - Works cross-platform

#### Linux Options

1. **D-Bus**
   - Standard Linux IPC
   - Python: Use `dbus-python` or `pydbus`
   - Powerful but more complex
   - Not available on macOS by default

2. **Unix Domain Sockets**
   - Same as macOS, fully compatible
   - Recommended for cross-platform compatibility

3. **systemd Socket Activation**
   - If running as a systemd service
   - Clean integration with Linux init

### Recommended Approach

Use **Unix Domain Sockets** for cross-platform compatibility:

```python
# Socket path
SOCKET_PATH = Path.home() / ".cache" / "luk" / "ipc.sock"

# Message types
class IPCMessage:
    MAIL_UPDATED = "mail.updated"
    TASKS_UPDATED = "tasks.updated"
    CALENDAR_UPDATED = "calendar.updated"

# Sync daemon sends notifications
async def notify_mail_updated(folder: str):
    await send_ipc_message({"type": IPCMessage.MAIL_UPDATED, "folder": folder})

# Mail app listens
async def listen_for_updates():
    async for message in ipc_listener():
        if message["type"] == IPCMessage.MAIL_UPDATED:
            self.load_messages()
```

**Implementation files:**
- `src/utils/ipc.py` - IPC client/server utilities
- `src/cli/sync_daemon.py` - Add notification sending
- `src/mail/app.py` - Add listener for mail updates
- `src/tasks/app.py` - Add listener for task updates
- `src/calendar/app.py` - Add listener for calendar updates

---

## Calendar App

### Visual Improvements

#### 1. Current Time Hour Line Styling
**Priority:** High  
**File:** `src/calendar/widgets/WeekGrid.py`

The current time indicator hour line should have a subtle contrasting background color to make it more visible.

#### 2. Cursor Hour Header Highlighting
**Priority:** Medium  
**File:** `src/calendar/widgets/WeekGrid.py`

The hour header at cursor position should have a brighter background, similar to how the day header is highlighted when selected.

---

### Layout Improvements

#### 3. Responsive Detail Panel
**Priority:** Medium  
**Files:** `src/calendar/app.py`, `src/calendar/widgets/`

When the terminal is wider than X characters (e.g., 120), show a side-docked detail panel for the selected event instead of a modal/overlay.

#### 4. Sidebar Mini-Calendar
**Priority:** Medium  
**Files:** `src/calendar/app.py`, `src/calendar/widgets/MonthCalendar.py`

When the sidebar is toggled on, display a mini-calendar in the top-left corner showing:
- Current day highlighted
- The week(s) currently visible in the main WeekGrid pane
- Click/navigate to jump to a specific date

**Implementation:**
- Reuse existing `MonthCalendar` widget in compact mode
- Add reactive property to sync selected week with main pane
- Add to sidebar composition when toggled

---

### Microsoft Graph Integration

#### 5. Calendar Invites Sidebar
**Priority:** Medium  
**Files:** `src/calendar/app.py`, `src/services/microsoft_graph/calendar.py`

Display a list of pending calendar invites from Microsoft Graph API in a sidebar panel:
- List pending invites (meeting requests)
- Show invite details (organizer, time, subject)
- Accept/Decline/Tentative actions
- No sync needed - fetch on-demand from API

**API Endpoints:**
- `GET /me/calendar/events?$filter=responseStatus/response eq 'notResponded'`
- `POST /me/events/{id}/accept`
- `POST /me/events/{id}/decline`
- `POST /me/events/{id}/tentativelyAccept`

**UI:**
- Toggle with keybinding (e.g., `i` for invites)
- Sidebar shows list of pending invites
- Detail view shows full invite info
- Action buttons/keys for response

---

### Search Feature

#### 6. Calendar Search
**Priority:** Medium  
**Files:** `src/calendar/app.py`, `src/services/khal/client.py` (if khal supports search)

Add search functionality if the underlying backend (khal) supports it:
- `/` keybinding to open search input
- Search by event title, description, location
- Display search results in a modal or replace main view
- Navigate to selected event

**Check:** Does `khal search` command exist?

---

### Help System

#### 7. Help Toast (Keep Current Implementation)
**Priority:** Low  
**File:** `src/calendar/app.py`

The `?` key shows a help toast using `self.notify()`. This pattern should be implemented in other apps (Mail, Tasks, Sync) for consistency.

---

## Mail App

### Layout Fixes

#### 1. Remove Envelope Icon/Checkbox Gap
**Priority:** High  
**File:** `src/mail/widgets/EnvelopeListItem.py`

There's a 1-character space between the envelope icon and checkbox that should be removed for tighter layout.

---

### Theme Improvements

#### 2. Replace Hardcoded RGB Colors
**Priority:** High  
**File:** `src/mail/email_viewer.tcss`

Multiple hardcoded RGB values should use Textual theme variables for better theming support:

| Line | Current | Replacement |
|------|---------|-------------|
| 6 | `border: round rgb(117, 106, 129)` | `border: round $border` |
| 46 | `background: rgb(55, 53, 57)` | `background: $surface` |
| 52, 57 | RGB label colors | Theme variables |
| 69 | `background: rgb(64, 62, 65)` | `background: $panel` |
| 169, 225, 228, 272, 274 | Various RGB colors | Theme variables |

---

### Keybindings

#### 3. Add Refresh Keybinding
**Priority:** Medium  
**File:** `src/mail/app.py`

Add `r` keybinding to refresh/reload the message list.

#### 4. Add Mark Read/Unread Action
**Priority:** Medium  
**Files:** `src/mail/app.py`, `src/mail/actions/` (new file)

Add action to toggle read/unread status on selected message(s).

---

### Search Feature

#### 5. Mail Search
**Priority:** Medium  
**Files:** `src/mail/app.py`, backend integration

Add search functionality if the underlying mail backend supports it:
- `/` keybinding to open search input
- Search by subject, sender, body
- Display results in message list
- Check: Does himalaya or configured backend support search?

---

### UI Enhancements

#### 6. Folder Message Counts
**Priority:** Medium  
**Files:** `src/mail/app.py`, `src/mail/widgets/`

Display total message count next to each folder name (e.g., "Inbox (42)").

#### 7. Sort Setting in Config/UI
**Priority:** Low  
**Files:** `src/mail/config.py`, `src/mail/app.py`

Add configurable sort order (date, sender, subject) with UI toggle.

---

### Message Display

#### 8. URL Compression in Markdown View
**Priority:** Medium  
**Files:** `src/mail/widgets/ContentContainer.py`, `src/mail/screens/LinkPanel.py`

Compress long URLs in the markdown view to ~50 characters with a nerdfont icon. The `_shorten_url` algorithm in `LinkPanel.py` can be reused.

**Considerations:**
- Cache processed markdown to avoid re-processing on scroll
- Store URL mapping for click handling

#### 9. Remove Emoji from Border Title
**Priority:** Low  
**File:** `src/mail/widgets/ContentContainer.py` or `EnvelopeHeader.py`

Remove the envelope emoji prefix before message ID in border titles.

#### 10. Enhance Subject Styling
**Priority:** Medium  
**File:** `src/mail/widgets/EnvelopeHeader.py`

- Move subject line to the top of the header
- Make it bolder/brighter for better visual hierarchy

---

## Tasks App

### Search Feature

#### 1. Task Search
**Priority:** Medium  
**Files:** `src/tasks/app.py`, `src/services/dstask/client.py`

Add search functionality:
- `/` keybinding to open search input
- Search by task summary, notes, project, tags
- Display matching tasks
- Check: dstask likely supports filtering which can be used for search

**Implementation:**
- Add search input widget (TextInput)
- Filter tasks locally or via dstask command
- Update table to show only matching tasks
- Clear search with Escape

---

### Help System

#### 2. Implement Help Toast
**Priority:** Low  
**File:** `src/tasks/app.py`

Add `?` keybinding to show help toast (matching Calendar app pattern).
**Note:** This is already implemented in the current code.

---

## Cross-App Improvements

### 1. Consistent Help System
Implement `?` key help toast in all apps using `self.notify()`:
- Mail: `src/mail/app.py`
- Tasks: `src/tasks/app.py` (already has it)
- Sync: `src/cli/sync_dashboard.py`

### 2. Consistent Navigation
Add vim-style `j`/`k` navigation to all list views across apps.

### 3. Consistent Border Styling
Use `border: round` and `.border_title` styling consistently in all TCSS files.

### 4. Consistent Search Interface
Implement `/` keybinding for search across all apps with similar UX:
- `/` opens search input
- Enter executes search
- Escape clears/closes search
- Results displayed in main view or filtered list

---

## Implementation Priority

### Phase 1: Critical/High Priority
1. ~~Tasks App: Fix table display~~ (DONE)
2. ~~Sync: Parallelize message downloads~~ (DONE - connection pooling + batch size increase)
3. ~~Mail: Replace hardcoded RGB colors~~ (DONE - already using theme variables)
4. ~~Mail: Remove envelope icon/checkbox gap~~ (DONE)
5. ~~Calendar: Current time hour line styling~~ (DONE - added surface background)
6. ~~IPC: Implement cross-app refresh notifications~~ (DONE)

### Phase 2: Medium Priority
1. ~~Sync: Default to TUI mode~~ (DONE - already implemented)
2. ~~Calendar: Cursor hour header highlighting~~ (DONE)
3. Calendar: Responsive detail panel
4. Calendar: Sidebar mini-calendar
5. Calendar: Calendar invites sidebar
6. ~~Mail: Add refresh keybinding~~ (DONE - `r` key)
7. ~~Mail: Add mark read/unread action~~ (DONE - `u` key)
8. ~~Mail: Folder message counts~~ (DONE)
8. ~~Mail: URL compression in markdown view~~ (DONE)
9. ~~Mail: Enhance subject styling~~ (DONE)
10. Mail: Search feature
11. ~~Tasks: Search feature~~ (DONE - `/` key with live filtering)
12. ~~Calendar: Search feature~~ (DONE - `/` key using khal search)

### Phase 3: Low Priority
1. Sync: UI consistency (j/k navigation, borders)
2. Sync: Notifications toggle
3. Calendar: Help toast (already implemented, replicate to other apps)
4. Mail: Remove emoji from border title
5. Mail: Sort setting in config/UI
6. Cross-app: Consistent help, navigation, and styling

---

## Technical Notes

### IPC Implementation Details

For macOS-first with Linux compatibility:

```python
# src/utils/ipc.py
import asyncio
import json
from pathlib import Path

SOCKET_PATH = Path.home() / ".cache" / "luk" / "ipc.sock"

class IPCServer:
    """Server for sending notifications to listening apps."""
    
    async def broadcast(self, message: dict):
        """Send message to all connected clients."""
        pass

class IPCClient:
    """Client for receiving notifications in apps."""
    
    async def listen(self, callback):
        """Listen for messages and call callback."""
        pass
```

### Backend Search Capabilities

| Backend | Search Support |
|---------|---------------|
| dstask | Filter by project/tag, summary search via shell |
| himalaya | Check `himalaya search` command |
| khal | Check `khal search` command |
| Microsoft Graph | Full text search via `$search` parameter |

---

## Notes

- All UI improvements should be tested with different terminal sizes
- Theme changes should be tested with multiple Textual themes
- Performance improvements should include before/after benchmarks
- New keybindings should be documented in each app's help toast
- IPC should gracefully handle missing socket (apps work standalone)
- Search should be responsive and not block UI

---

## Mail Rendering Improvements

Is there a way to improve readability of emails in a terminal? I get a lot of "notification style emails", and they aren't easy to comprehend or read in plain text. At the very least they don't look good. Maybe we can find an algorithm to reduce the visual noise. Or maybe an AI summary view option using copilot APIs? Research best options. Perhaps embedding a small pre-trained model that can do the summary?

---

## Note-taking integration and more cross-app integrations

I like the `tdo` (https://github.com/2KAbhishek/tdo) program for managing markdown notes with fzf and my terminal text editor. It makes it easy to have a "today" note and a list of todos. Perhaps we can gather those todos from the text files in the $NOTES_DIR and put them into the task list during regular sync - and when users mark a task complete the sync can find the text file it was in and mark it complete on that line of markdown text. We need a little ore features for the related annotations then, because when I press `n` in the notes app we would want to actually open the textfile that task came from, not just make another annotation. So we would need a special cross-linking format for knowing which tasks came from a $NOTES_DIR sync. And then we can work on the same IPC scenario for tasks that were created in the email app. Then those tasks should be connected so that when the user views the notes on those tasks they see the whole email. That would be simpe enough if we just copied the email text into an annotation. But maybe we need a way to actually change the selected message ID in the mail app if it's open. So if the user activates the "open" feature on a task the related email will be displayed in the other terminal window where the user has mail open.

---

## Polish for release and Auth

- Authentication isn't working if the user is in a different directory than expected. Store the auth token in a ~/.local directory so it's consistently availabe in every dir.
- Setup scripts, look at popular modern python CLI or TUI projects and create an install or first run script that works for most scenarios, using `uv` or `pip` to install globally for users, or maybe get a homebrew installation to work? Get users to set up himalaya and khal and dstask but ask them for their choice once we have other backends available. 
- Documentation files in the repo, and documentation site to publish.



---

## Library Updates & Python Version Review

### Priority: Medium (Scheduled Review)

Periodically review the latest releases of heavily-used libraries to identify:
- Bug fixes that address issues we've encountered
- New features that could improve the codebase
- Deprecation warnings that need to be addressed
- Security updates

### Key Libraries to Review

| Library | Current Use | Review Focus |
|---------|-------------|--------------|
| **Textual** | All TUI apps | New widgets, performance improvements, theming changes, CSS features |
| **aiohttp** | Microsoft Graph API client | Async improvements, connection pooling |
| **msal** | Microsoft authentication | Token caching, auth flow improvements |
| **rich** | Console output (via Textual) | New formatting options |
| **orjson** | Fast JSON parsing | Performance improvements |
| **pyobjc** (macOS) | Notifications | API changes, compatibility |

### Textual Changelog Review Checklist

When reviewing Textual releases, check for:
1. **New widgets** - Could replace custom implementations
2. **CSS features** - New selectors, pseudo-classes, properties
3. **Theming updates** - New theme variables, design token changes
4. **Performance** - Rendering optimizations, memory improvements
5. **Breaking changes** - Deprecated APIs, signature changes
6. **Worker improvements** - Background task handling

### Python Version Upgrade

#### Current Status
- Check `.python-version` and `pyproject.toml` for current Python version
- Evaluate upgrade to Python 3.13 or 3.14 when stable

#### Python 3.13 Features to Consider
- Improved error messages
- Type system enhancements (`typing` module improvements)
- Performance optimizations (PEP 709 - inline comprehensions)

#### Python 3.14 Considerations
- **Status:** Currently in alpha/beta (as of Dec 2024)
- **Expected stable release:** October 2025
- **Recommendation:** Wait for stable release before adopting
- **Pre-release testing:** Can test compatibility in CI/CD before adoption

#### Upgrade Checklist
1. [ ] Review Python release notes for breaking changes
2. [ ] Check library compatibility (especially `pyobjc`, `textual`, `msal`)
3. [ ] Update `.python-version` (mise/pyenv)
4. [ ] Update `pyproject.toml` `requires-python` field
5. [ ] Run full test suite
6. [ ] Test on both macOS and Linux (if applicable)
7. [ ] Update CI/CD Python version

### Action Items

1. **Quarterly Review** - Schedule quarterly reviews of library changelogs
2. **Dependabot/Renovate** - Consider adding automated dependency update PRs
3. **Changelog Reading** - Before updating, read changelogs for breaking changes
4. **Test Coverage** - Ensure adequate test coverage before major updates