feat(macos): enable meeting integrations

2026-08-06 08:19:26 +02:00
parent 9542c05ea4
commit 2af7933a1b
27 changed files with 1999 additions and 227 deletions
@@ -0,0 +1,37 @@
+## Context
+
+The macOS build already bundles Swift helpers for audio and menu-bar controls. Calendar and screenshot integrations are currently replaced with no-op providers at compile time, while the ASR, diarization, speaker, OCR, and workflow-agent implementations themselves are portable.
+
+## Decisions
+
+### EventKit is the preferred macOS calendar source
+
+A bundled Swift helper reads events through EventKit. This covers calendars exposed to macOS, including Outlook accounts synchronized into Calendar, without linking Windows COM into the portable target. If Calendar Full Access is denied, the managed client falls back to the already-authorized Calendar application's automation dictionary. The helper uses a stable permission-denied exit code rather than coupling the fallback to human-readable error text. The managed providers retain the existing Teams-marker and current-meeting selection rules.
+
+### Native AppKit owns the recording prompt
+
+The same helper displays a record/skip AppKit alert and returns the selected response to the existing `IMeetingStartPromptService` callback. Recording policy remains in the existing scheduler/controller.
+
+### CoreGraphics captures the active app window
+
+The helper resolves the frontmost application and captures its foremost on-screen window as PNG. The managed screenshot service continues to own attachment naming, note updates, OCR, cropping, attendee enrichment, retries, and waits.
+
+### Portable managed features stay managed
+
+Screenshot OCR, speaker identity, Docker-backed FunASR, FunASR diarization, Whisper-local pyannote diarization, and the AppKit/WebKit workflow editor use the same managed implementations on macOS and Windows. Platform branching is limited to the native input providers.
+
+### Diagnostics report effective behavior
+
+A read-only endpoint returns the host platform, concrete provider names, and effective enabled switches. This supplies deterministic proof without starting a recording or mutating meeting artifacts.
+
+## Windows compatibility
+
+The Windows target keeps its existing conditional registrations and does not compile or invoke the macOS helper. Verification must build the Windows target in addition to the portable target and tests.
+
+## Verification
+
+- Regression tests prove macOS resolves real providers and reports enabled capabilities.
+- Provider behavior tests prove EventKit JSON maps to calendar metadata, native prompt responses reach the callback, and PNG screenshot output reaches the screenshot interface.
+- The Swift helper provides deterministic self-test output.
+- A live macOS instance proves health, provider diagnostics, native helper packaging, calendar access, and screenshot capture.
+- A Windows-target build proves the existing Windows client still compiles with its native providers.
@@ -0,0 +1,47 @@
+## Verification evidence
+
+Verified on macOS 26 on 2026-07-25.
+
+### Automated checks
+
+- Focused macOS/calendar/screenshot/OCR tests: 35 passed, 0 failed.
+- Full portable test project: 441 passed, 6 failed, 447 total. The six failures are existing macOS-incompatible tests for Windows path assumptions, GDI+ taskbar rendering, and user-environment fallback; none cover this change.
+- Portable `net10.0` build: succeeded.
+- Windows `net10.0-windows10.0.19041.0` target build with Windows targeting enabled: succeeded with 0 warnings and 0 errors.
+- `openspec validate add-macos-meeting-integrations --strict`: valid.
+- Sequential DRY, SOLID, and KISS reviews were completed and their findings implemented.
+- Final independent standards and spec reviews reported no findings.
+
+### Live macOS installation
+
+The recording status was idle before deployment. The repository updater published the portable build, installed it into `~/Library/Application Support/MeetingAssistant/app`, restarted the LaunchAgent, and passed its health check.
+
+`GET /diagnostics/platform-capabilities` returned:
+
+```json
+{
+  "host": "macOS",
+  "meetingMetadataProvider": "MacOsMeetingMetadataProvider",
+  "calendarMeetingProvider": "MacOsCalendarMeetingProvider",
+  "meetingStartPromptProvider": "MacOsMeetingStartPromptService",
+  "screenshotProvider": "MacOsDesktopIntegrationClient",
+  "workflowEditorProvider": "MacOsWorkflowRulesEditorWindowService",
+  "workflowEditorAvailable": true,
+  "calendarRecordingPromptsEnabled": true,
+  "speakerIdentificationEnabled": true,
+  "screenshotOcrEnabled": true,
+  "funAsrBackendEnabled": true,
+  "funAsrDiarizationEnabled": true,
+  "whisperLocalDiarizationEnabled": true
+}
+```
+
+The installed arm64 Mach-O helper returned:
+
+```json
+{"features":["calendar","prompt","screenshot","crop"],"status":"ok"}
+```
+
+Direct native screenshot capture selected the foreground ChatGPT window and produced a valid 3258 by 1920 PNG. The asymmetric native crop regression test also proves OCR top-left pixel coordinates are preserved.
+
+The live calendar diagnostic returned `provider=MacOsMeetingMetadataProvider`, `timedOut=false`, and a normal no-matching-event result. EventKit reported missing Full Calendar Access with stable exit code 77, after which the Calendar automation fallback completed; the scheduler logged `Synced 0 Teams calendar meetings`.
@@ -0,0 +1,29 @@
+## Why
+
+The portable macOS application can capture meeting audio and expose menu-bar controls, but it still registers unavailable/no-op implementations for calendar metadata, recording prompts, and active-window screenshots. The local macOS environment also disables screenshot OCR, speaker identification, the managed FunASR backend, FunASR diarization, and Whisper-local diarization even though their managed implementations are portable.
+
+## What Changes
+
+- Read Teams meeting metadata and prompt candidates from calendars available through macOS EventKit, including Outlook-synced calendars.
+- Show a native macOS record/skip prompt for due calendar meetings.
+- Capture the foreground macOS application window through a bundled native helper so screenshot OCR can use the existing portable OCR pipeline.
+- Register the real macOS integrations instead of no-op/unavailable providers.
+- Enable the portable OCR, speaker identification, FunASR, and diarization configuration in the macOS runtime environment.
+- Expose a diagnostic capability surface that proves the effective platform providers and enabled feature switches.
+- Preserve all Windows-specific COM, notification, screenshot, tray, hotkey, and audio registrations.
+
+## Capabilities
+
+### Modified Capabilities
+
+- `meeting-session`: calendar enrichment, recording prompts, workflow editor access, and capability diagnostics work on macOS.
+- `meeting-summary`: active-window screenshots and screenshot OCR work on macOS.
+- `meeting-transcription`: portable speaker identification, FunASR, and local diarization can be enabled on macOS.
+
+## Impact
+
+- portable service registration in `MeetingAssistant/Program.cs`
+- a bundled native macOS desktop-integration helper
+- calendar, meeting-metadata, screenshot, and diagnostic providers
+- macOS runtime configuration and operational documentation
+- macOS behavior tests plus Windows-target build verification
@@ -0,0 +1,41 @@
+## MODIFIED Requirements
+
+### Requirement: Meeting metadata is captured when available
+Meeting Assistant SHALL enrich a meeting from a matching calendar appointment when platform calendar metadata is available.
+
+On Windows, Meeting Assistant SHALL retain Outlook Classic COM metadata lookup.
+
+On macOS, Meeting Assistant SHALL read calendars available through EventKit, including Outlook-synced calendars, and SHALL apply the existing Teams-marker and current-meeting selection rules.
+
+#### Scenario: macOS meeting uses calendar metadata
+- **GIVEN** an EventKit calendar contains a Teams appointment spanning the meeting start time
+- **WHEN** Meeting Assistant enriches the meeting on macOS
+- **THEN** it applies the appointment title, attendees, agenda, and scheduled end time
+
+#### Scenario: Windows metadata provider remains unchanged
+- **GIVEN** Meeting Assistant is compiled for the Windows target
+- **WHEN** meeting metadata is registered
+- **THEN** it uses the Outlook Classic COM provider
+- **AND** does not invoke or require the macOS native helper
+
+### Requirement: Meeting Assistant can prompt to record scheduled Teams meetings
+Meeting Assistant SHALL optionally inspect the host calendar for scheduled Teams meetings and show a native record-or-skip prompt when a meeting enters the configured prompt window.
+
+On Windows, Meeting Assistant SHALL retain Outlook Classic COM lookup and Windows app notifications.
+
+On macOS, Meeting Assistant SHALL use EventKit calendar events and a native AppKit record-or-skip prompt.
+
+#### Scenario: macOS calendar meeting becomes due
+- **GIVEN** calendar recording prompts are enabled on macOS
+- **AND** an EventKit Teams appointment enters the prompt window
+- **WHEN** the scheduler checks due meetings
+- **THEN** Meeting Assistant shows the native macOS prompt
+- **AND** starts recording with the appointment metadata when the user chooses record
+
+### Requirement: Platform capabilities are diagnosable
+Meeting Assistant SHALL expose a read-only diagnostic response containing the detected host, effective integration provider names, and enabled state of calendar prompts, speaker identification, screenshot OCR, FunASR backend, FunASR diarization, Whisper-local diarization, and workflow editor availability.
+
+#### Scenario: macOS reports real integration providers
+- **WHEN** the capability diagnostic is requested on macOS
+- **THEN** it reports real macOS calendar metadata, calendar prompt, screenshot, and workflow editor providers
+- **AND** does not report the no-op or unavailable provider names
@@ -0,0 +1,23 @@
+## MODIFIED Requirements
+
+### Requirement: Meeting screenshots are captured into assistant context
+Meeting Assistant SHALL capture the active application window on supported Windows and macOS hosts and append it to the active meeting assistant context.
+
+On Windows, Meeting Assistant SHALL retain the existing foreground-window capture implementation.
+
+On macOS, Meeting Assistant SHALL capture the foremost on-screen window belonging to the frontmost application through native macOS APIs.
+
+The existing screenshot OCR, crop, attendee enrichment, retry, and summary-wait behavior SHALL apply unchanged to macOS screenshots.
+
+#### Scenario: macOS captures the active window for OCR
+- **GIVEN** an active meeting on macOS and Screen Recording permission
+- **WHEN** the screenshot hotkey or endpoint is invoked
+- **THEN** Meeting Assistant saves a PNG of the frontmost application window
+- **AND** appends it to assistant context
+- **AND** starts OCR when screenshot OCR is enabled
+
+#### Scenario: Windows screenshot provider remains unchanged
+- **GIVEN** Meeting Assistant is compiled for the Windows target
+- **WHEN** active-window screenshot capture is registered
+- **THEN** it uses the existing Windows foreground-window provider
+- **AND** does not invoke or require the macOS native helper
@@ -0,0 +1,18 @@
+## MODIFIED Requirements
+
+### Requirement: Portable transcription backends and speaker processing are available on macOS
+Meeting Assistant SHALL allow speaker identification, managed FunASR startup, FunASR diarization, and Whisper-local pyannote diarization to run on macOS using the same managed implementations and configuration contracts as Windows.
+
+These capabilities SHALL remain independently configurable and SHALL report actionable dependency errors when Docker, model files, API credentials, or model-access credentials are missing.
+
+#### Scenario: macOS enables configured portable processing
+- **GIVEN** the portable build is running on macOS
+- **AND** speaker identification, managed FunASR, FunASR diarization, or Whisper-local diarization is enabled
+- **WHEN** the corresponding transcription path runs
+- **THEN** Meeting Assistant invokes the existing managed implementation
+- **AND** does not disable the capability solely because the host is macOS
+
+#### Scenario: Windows processing behavior remains unchanged
+- **GIVEN** Meeting Assistant is compiled for the Windows target
+- **WHEN** portable transcription and speaker-processing services are registered
+- **THEN** it retains the existing managed implementations and configuration behavior
@@ -0,0 +1,26 @@
+## 1. Contract and regression coverage
+
+- [x] 1.1 Define macOS integration requirements and design.
+- [x] 1.2 Add failing registration and capability-diagnostic tests.
+- [x] 1.3 Add failing calendar, prompt, screenshot, and helper-contract tests.
+
+## 2. Native and managed implementation
+
+- [x] 2.1 Add and package the native macOS calendar/prompt/screenshot helper.
+- [x] 2.2 Add managed macOS calendar metadata and prompt providers.
+- [x] 2.3 Add managed macOS active-window screenshot provider.
+- [x] 2.4 Register the real providers on macOS while preserving unsupported-host no-ops.
+- [x] 2.5 Add the read-only platform-capability diagnostic endpoint.
+
+## 3. Configuration and documentation
+
+- [x] 3.1 Enable the portable OCR, speaker, FunASR, and diarization switches in the macOS runtime.
+- [x] 3.2 Document macOS permissions, calendar source behavior, and diagnostics.
+
+## 4. Verification and closeout
+
+- [x] 4.1 Run narrow tests and portable build.
+- [x] 4.2 Run the full test project and strict OpenSpec validation.
+- [x] 4.3 Build the Windows target.
+- [x] 4.4 Verify the live macOS application through direct operational surfaces.
+- [x] 4.5 Perform required DRY, SOLID, KISS, and final code reviews.