feat(macos): add native desktop controls and hotkey integration

- Introduced MacOsDesktopControlManifest to manage desktop control configurations. - Implemented MacOsDesktopControlService to handle the lifecycle of the macOS helper process. - Created MacOsWorkflowRulesEditorWindowService for managing workflow rules through a native window. - Developed a Swift helper for macOS that integrates with AppKit and Carbon for menu-bar controls and global hotkeys. - Added support for recording lifecycle actions via HTTP endpoints, maintaining consistency with existing Windows functionality. - Enhanced user experience by allowing macOS users to start/stop recordings and access settings through a menu-bar interface. - Documented design decisions, implementation evidence, and verification tasks for the new macOS features.
2026-08-06 08:18:52 +02:00
parent 5f39923c27
commit 9542c05ea4
20 changed files with 981 additions and 11 deletions
@@ -0,0 +1,23 @@
+## Context
+
+The application already exposes local HTTP endpoints for recording lifecycle actions. Windows desktop integrations invoke the coordinator in-process, while macOS currently has no desktop process capable of participating in AppKit or Carbon event loops.
+
+## Decisions
+
+### Native helper owns macOS desktop APIs
+
+A bundled Swift helper owns the AppKit status item and Carbon global-hotkey registrations. The .NET host starts the helper only on macOS and terminates it during shutdown.
+
+### Existing HTTP endpoints remain the control seam
+
+The helper invokes loopback endpoints for recording and diagnostic actions. It does not duplicate recording state or lifecycle policy. It polls the status endpoint to keep the menu-bar presentation current.
+
+### Configuration is passed as an explicit manifest
+
+The .NET host derives a manifest from configured launch profiles. Shortcut strings are passed unchanged so macOS uses the same bindings displayed and registered on Windows.
+
+## Verification
+
+- Behavior tests prove the manifest preserves configured shortcuts and maps actions to the existing endpoints.
+- The Swift helper supports a describe/self-test mode for deterministic packaging validation.
+- Runtime verification launches the application, starts a new recording, and captures the visible macOS menu-bar recording state.
@@ -0,0 +1,98 @@
+# Implementation Evidence
+
+## Automated
+
+- `dotnet test MeetingAssistant.Tests/MeetingAssistant.Tests.csproj --filter FullyQualifiedName~MacOsDesktopControlManifestTests --no-restore`
+  - Passed: 1, Failed: 0.
+- `dotnet build MeetingAssistant/MeetingAssistant.csproj -f net10.0 --no-restore`
+  - Passed and produced `Native/macos-desktop-controls` as an arm64 Mach-O executable.
+- `openspec validate add-macos-desktop-controls --strict`
+  - Passed.
+- Full portable test project:
+  - Passed: 427.
+  - Failed: 9 existing macOS/environment-dependent tests: Windows path expectations, unavailable GDI+, and user-environment key resolution.
+- Whole solution:
+  - Cannot build the Windows target on macOS without enabling cross-Windows targeting (`NETSDK1100`).
+
+## Runtime
+
+- Installed the rebuilt application assembly and both native helpers into the existing local LaunchAgent application directory.
+- LaunchAgent restarted successfully.
+- `/health` returned `ok`.
+- `macos-desktop-controls` ran as a child companion process.
+- The helper manifest contained six configured hotkeys, including the unchanged defaults `Ctrl+Alt+M`, `Ctrl+Alt+L`, `Ctrl+Alt+Z`, and `Ctrl+Alt+S`.
+- A fresh default-profile recording request created the `20260724-2015` meeting note and linked artifacts.
+- Screenshot evidence: `tmp/macos-desktop-recording-verification.png`.
+
+## Live desktop-control proof (2026-07-24)
+
+- The installed LaunchAgent ran:
+  - `.NET host` PID `23023`
+  - native `macos-desktop-controls` PID `23097`
+- The native helper manifest exposed:
+  - `Ctrl+Alt+M` -> `/profiles/default/recording/toggle`
+  - `Ctrl+Alt+L` -> `/profiles/english/recording/toggle`
+  - `Ctrl+Alt+Z` -> `/profiles/default/recording/abort`
+  - `Ctrl+Alt+S` -> `/profiles/default/meetings/screenshot/capture`
+- Accessibility inspection of the real AppKit menu returned:
+  - `Meeting Assistant — idle`
+  - `Open agent`
+  - `Start meeting recording (default)`
+  - `Start meeting recording (english)`
+  - `Exit`
+- Visible menu screenshot:
+  - `tmp/proof-macos-menu-idle.png`
+  - The menu visibly displays the macOS equivalents `⌃⌥M` and `⌃⌥L`.
+- Clicking `Start meeting recording (default)` through the real menu produced the sampled status transition `false,true,false`, with `launchProfile: default`, and created:
+  - `Meetings/Notes/20260724-2022-note.md`
+  - `Meetings/Transcripts/20260724-2022-transcript.md`
+  - `Meetings/Assistant Context/20260724-2022-context.md`
+- Posting the HID-level `Ctrl+Alt+M` combination produced the sampled status transition `false,true,false`, with `launchProfile: default`.
+- Posting the HID-level `Ctrl+Alt+L` combination produced an active sampled state with `launchProfile: english`.
+- Raw sampled status evidence:
+  - `tmp/proof-menu-recording-status.jsonl`
+  - `tmp/proof-hotkey-recording-status.jsonl`
+  - `tmp/proof-english-hotkey-status.jsonl`
+
+## Runtime limitation
+
+The new recording did not remain active because the local audio capture process stopped shortly after artifact creation. The desktop-control helper, menu-bar item, endpoint routing, and meeting creation were verified; sustained audio capture was not.
+
+## Open agent regression and runtime proof
+
+- Original deterministic regression:
+  - `MacOsApplicationRegistersARealInteractiveAgentWindow`
+  - Red result: portable registration resolved exactly `NoopWorkflowRulesEditorWindowService`.
+- Fix:
+  - macOS now registers `MacOsWorkflowRulesEditorWindowService`.
+  - `Open agent` in the native helper creates an AppKit `NSWindow` titled `Meeting Summary Agent`.
+  - The window hosts a WebKit chat page backed by the existing `WorkflowRulesEditorChatViewModel` and agent pipeline.
+- Green regression:
+  - Passed: 1, Failed: 0.
+- Live installed runtime:
+  - Clicking the actual menu item created an on-screen CoreGraphics window:
+    - owner: `macos-desktop-controls`
+    - title: `Meeting Summary Agent`
+    - layer: `0`
+    - bounds: `756x714`
+  - `GET /diagnostics/settings-and-logs` returned the interactive agent HTML page.
+
+## Open agent prompt execution proof (2026-07-25)
+
+- Original live failure:
+  - `Meeting Summary Agent failed: No workflow rules editor API key configured.`
+- Root cause:
+  - The LaunchAgent did not inherit the interactive shell's model API key.
+  - Its configured `127.0.0.1:4021` model endpoint had no running service.
+- Runtime configuration:
+  - The protected LaunchAgent environment file now supplies the local LiteLLM gateway key.
+  - `MeetingAssistant__WorkflowRulesEditor__Endpoint` targets the running local gateway on port `4000`.
+  - The local gateway exposes the Responses-compatible `chatgpt-gpt-5.5` model alias.
+- Event-stream compatibility regression:
+  - `ClientReconstructsResponseFromServerSentEvents` failed before the fix with `JsonReaderException`.
+  - The client now reconstructs completed response items from LiteLLM server-sent events.
+  - All 11 `LiteLlmResponsesChatClientTests` pass.
+- Live installed proof:
+  - `POST /diagnostics/settings-and-logs/chat` with `Reply with exactly: key-ok`
+    returned `{"role":"agent","content":"key-ok"}`.
+  - `/health` returned successfully after deployment.
@@ -0,0 +1,24 @@
+## Why
+
+Meeting Assistant can record audio on macOS, but the portable build does not register the configured global shortcuts or provide the menu-bar controls used for the normal Windows desktop workflow. macOS users must call HTTP endpoints manually.
+
+## What Changes
+
+- Add a native macOS menu-bar companion that starts and stops with Meeting Assistant.
+- Register the same configured recording, abort, and screenshot shortcuts used on Windows.
+- Expose recording/profile controls, `Open agent`, and exit from the macOS menu bar.
+- Keep recording lifecycle behavior behind the existing local HTTP/service surface.
+
+## Capabilities
+
+### Modified Capabilities
+
+- `meeting-recording`: configured global recording controls work on macOS.
+- `meeting-session`: the desktop control menu is available from the macOS menu bar.
+
+## Impact
+
+- `MeetingAssistant/Native/MacOsDesktopControls`
+- portable macOS build/publish output
+- startup service registration
+- desktop-control behavior tests and macOS runtime documentation
@@ -0,0 +1,20 @@
+## MODIFIED Requirements
+
+### Requirement: Recording mode is controlled by a configurable hotkey
+Meeting Assistant SHALL use normal .NET configuration to define global hotkeys that work on Windows and macOS and toggle recording/transcription mode.
+
+Meeting Assistant SHALL preserve the configured shortcut strings across Windows and macOS, including the defaults `Ctrl+Alt+M`, `Ctrl+Alt+L`, `Ctrl+Alt+Z`, and `Ctrl+Alt+S`.
+
+Meeting Assistant SHALL expose a configurable abort/discard hotkey for an active recording on Windows and macOS.
+
+#### Scenario: macOS uses the configured Windows-equivalent shortcuts
+- **GIVEN** the default and English profiles use `Ctrl+Alt+M` and `Ctrl+Alt+L`
+- **AND** abort and screenshot use `Ctrl+Alt+Z` and `Ctrl+Alt+S`
+- **WHEN** Meeting Assistant starts on macOS
+- **THEN** it registers those same global shortcut combinations
+- **AND** routes them to toggle, abort, and screenshot actions through the local service surface
+
+#### Scenario: macOS recording hotkey starts recording
+- **GIVEN** Meeting Assistant is idle on macOS
+- **WHEN** the user presses the configured recording hotkey
+- **THEN** Meeting Assistant starts recording/transcription mode
@@ -0,0 +1,28 @@
+## MODIFIED Requirements
+
+### Requirement: Workflow rules and speaker identities can be edited through a tray-launched assistant
+Meeting Assistant SHALL expose desktop controls from the Windows tray icon and the macOS menu bar.
+
+The macOS menu-bar control SHALL start with Meeting Assistant, show current recording state, expose each configured launch profile with its configured hotkey, expose stop and abort controls while recording, expose `Open agent`, and expose exit.
+
+Selecting `Open agent` on macOS SHALL open a native window titled `Meeting Summary Agent` containing the interactive settings-and-logs chat surface.
+
+#### Scenario: macOS menu bar shows idle recording controls
+- **GIVEN** Meeting Assistant is idle on macOS
+- **WHEN** the user opens the menu-bar control
+- **THEN** it shows `Open agent`
+- **AND** shows a start action for every configured launch profile
+- **AND** displays each profile's configured toggle hotkey
+- **AND** shows an exit action
+
+#### Scenario: macOS menu bar shows active recording controls
+- **GIVEN** a meeting is recording on macOS
+- **WHEN** the user opens the menu-bar control
+- **THEN** it shows the active recording state
+- **AND** exposes stop and abort actions
+
+#### Scenario: macOS menu opens the interactive agent
+- **GIVEN** Meeting Assistant is running on macOS
+- **WHEN** the user selects `Open agent` from the menu-bar control
+- **THEN** a `Meeting Summary Agent` window opens
+- **AND** the user can submit a chat message and receive the interactive agent response
@@ -0,0 +1,18 @@
+## 1. Contract and behavior
+
+- [x] 1.1 Define macOS desktop-control requirements and design.
+- [x] 1.2 Add a failing behavior test for configured shortcut and endpoint preservation.
+- [x] 1.3 Implement the desktop-control manifest and make the behavior test pass.
+
+## 2. Native macOS integration
+
+- [x] 2.1 Add and package the native macOS menu-bar/global-hotkey helper.
+- [x] 2.2 Start and stop the helper with the .NET application on macOS.
+- [x] 2.3 Add deterministic helper packaging/self-test coverage.
+
+## 3. Verification and closeout
+
+- [x] 3.1 Run narrow and full tests.
+- [x] 3.2 Run strict OpenSpec validation.
+- [x] 3.3 Launch Meeting Assistant, start a new recording, and save screenshot evidence.
+- [x] 3.4 Perform required DRY, SOLID, and KISS refactoring reviews and re-run verification.