Files

T

dh 68db6955a2 Merge remote-tracking branch 'origin/main' into codex/macos-support

# Conflicts:
#	README.md

2026-08-31 18:49:49 +02:00

16 KiB

Raw Blame History

Meeting Assistant

Meeting Assistant is Manuel's local .NET meeting capture and knowledge service. It runs on Windows and macOS, records microphone plus system audio, maintains meeting artifacts in an Obsidian vault, and runs transcription, speaker attribution, screenshot OCR, and agentic summarization without depending on a meeting-platform API for the primary flow.

Scope And Runtime Boundary

This repository owns the application source, tests, OpenSpec requirements, checked-in configuration, and CI validation. It does not own workstation startup automation, a homelab deployment stack, Traefik routing, Docker Compose, or a deployed image tag.

Windows builds provide the tray icon, global hotkeys, NAudio capture, Outlook Classic COM enrichment, active-window screenshots, and notifications.
Portable builds on macOS provide a native menu-bar icon and global hotkeys, and capture the default microphone through AVFoundation plus computer output through ScreenCaptureKit.
Portable builds on other hosts keep the server/testable service surface but do not provide meeting audio capture.
The normal runtime endpoint is local HTTP on port 5090, with /health and /recording/status as the safe first checks. Accepted requirements live under openspec/specs. Relevant active changes under openspec/changes can describe implemented behavior that has not yet been folded into the accepted specs.

Quick Start

The project requires the .NET 10 SDK. Build and test it with:

dotnet restore MeetingAssistant.slnx
dotnet build MeetingAssistant.slnx
dotnet test MeetingAssistant.slnx

Do not start a second copy over the live workstation instance. Check the local control surface first:

Invoke-RestMethod http://127.0.0.1:5090/health
Invoke-RestMethod http://127.0.0.1:5090/recording/status

On macOS, build the portable target on the Mac that will run it:

dotnet build MeetingAssistant/MeetingAssistant.csproj -f net10.0
dotnet run --project MeetingAssistant/MeetingAssistant.csproj -f net10.0

The macOS build compiles bundled Swift helpers into Native/macos-meeting-audio-capture, Native/macos-desktop-controls, and Native/macos-meeting-integrations. On first use, allow Microphone, Screen & System Audio Recording, and Calendar Full Access under System Settings > Privacy & Security. The menu-bar icon and global hotkeys use the same configured bindings as Windows and call the local service surface.

For a foreground Windows development run when port 5090 is free:

dotnet run --project MeetingAssistant --framework net10.0-windows10.0.19041.0

The durable workstation instance is published and replaced through automation owned by Manuel/snippets:

powershell -ExecutionPolicy Bypass -File C:\Manuel\snippets\restart-meeting-assistant.ps1

The helper publishes before stopping the old process, starts the timestamped Windows publish on loopback port 5090, waits for /health, and records stdout, stderr, and the PID under %LOCALAPPDATA%\MeetingAssistant\Logs. Always confirm /recording/status is idle before invoking it.

Recording And Control

Recording can be controlled through global hotkeys, the Windows tray icon, the macOS menu-bar icon, or local HTTP endpoints. The default hotkeys are:

Ctrl+Alt+M: toggle the default recording profile.
Ctrl+Alt+L: start or switch to the configured english profile.
Ctrl+Alt+Z: abort the active run and delete its artifacts.
Ctrl+Alt+S: capture the active window into the meeting context.

The Windows tray and macOS menu-bar menu present Finish meeting as the primary action during capture; microphone selection, cancel/discard, and profile switching remain separate controls. Exit is always available and requires confirmation while any meeting is recording or processing.

The loopback HTTP surface has no application authentication, so port 5090 must remain a trusted loopback-only control surface. The main endpoints are:

GET /health
GET /recording/status
POST /recording/start, /recording/stop, /recording/toggle, /recording/abort
POST /profiles/{launchProfile}/recording/start, /stop, /toggle, /abort
POST /asr/transcribe-file and /asr/diarize-file for diagnostic WAV checks
POST /diagnostics/workflow/reload
POST /diagnostics/settings-and-logs/show
POST /diagnostics/workflow/rules-editor/show
GET /diagnostics/platform-capabilities
POST /meetings/current/summary/run
POST or GET /meetings/summary/retry
POST or GET /meetings/screenshot-ocr/retry

Some generated retry links use GET while starting work.

Stopping capture lets buffered transcription, speaker work, meeting-note image OCR, screenshot OCR, and summary generation finish. Another meeting can start while an older stopped run finalizes; each run retains isolated options and artifact paths.

During an active run, microphone creation failures and disconnects are retried every second with fresh endpoint selection. The meeting and system-loopback capture stay active, with microphone silence mixed in until capture resumes. This recovery does not cover a failed system-loopback source.

Outlook enrichment selects an unambiguous current or imminent appointment. A scheduled prompt shown during an active recording can apply that exact appointment's title, eligible attendees, agenda, and scheduled end without interrupting capture; explicit prompt metadata wins over a slower background lookup.

Data And Side Effects

Meeting Assistant writes meeting notes, transcripts, assistant context, summaries, and project knowledge into the configured Obsidian vault. Assistant context is persistent meeting-specific memory: the summarizer records problems and assumptions, and the interactive agent can read it and append later repairs and conclusions.

Agents are intentionally stateful. Depending on the invoked tools, they can change workflow rules and appsettings, create or update project and meeting files, change frontmatter, merge or delete speaker identities and samples, run diagnostics, and trigger transcription or summary work. Screenshot OCR can add recognized attendee names to the meeting note after workflow transformation; OCR of images already embedded in the meeting note does not add attendees or modify that note.

Local runtime state outside the vault includes:

%LOCALAPPDATA%\MeetingAssistant\Recordings: mixed WAV files are normally deleted after completion, and unqueued stale files are deleted at startup. If an Azure stop cannot drain within Recording:StopProcessingTimeout, the WAV plus a JSON item under offline-transcription-backlog are retained and retried every minute. Both are removed only after successful replay, transcript finalization, and summary processing.
%LOCALAPPDATA%\MeetingAssistant\SpeakerIdentity\speaker-identities.db: SQLite identities, aliases, meeting references, and bounded voice snippets.
%LOCALAPPDATA%\MeetingAssistant\FunASR\models and %LOCALAPPDATA%\MeetingAssistant\Pyannote\models: persistent model, hotword, Hugging Face, and torch caches for optional local backends.
%TEMP%\MeetingAssistant\Logs\meeting-assistant.log: application log with four rotated predecessors. Paths, transcript text, agent diagnostics, and provider errors can make these logs sensitive.

Abort is destructive: it removes the active run's note, transcript, context, summary, and linked screenshot attachments and skips summarization. A normal stop below Recording:MinimumCompletedMeetingDuration, or an otherwise content-empty stop, can also remove generated artifacts.

External Data Boundaries

“Local” describes control and durable storage, not every processing step:

The default azure-speech provider sends mixed meeting audio and dictation phrase hints to Azure AI Speech. Azure-backed speaker matching also sends selected voice audio.
Summary, screenshot OCR, and interactive-agent requests go to the configured OpenAI-compatible Responses endpoint. They can include meeting/transcript/project text, screenshots, configuration, logs, and speaker samples when corresponding tools are used. The checked-in endpoint is a loopback proxy; its ultimate provider, data path, and retention policy are outside this repository.
Outlook Classic access on Windows is local COM. EventKit access on macOS reads calendars synchronized into the Calendar app. Neither provides the primary capture path.
A managed FunASR run pulls its configured image, removes any same-named container, starts a disposable container privileged by default, publishes the configured host port, and mounts the model/hotword cache. Pyannote may build a local image and starts disposable containers with the input WAV mounted read-only and its model cache read/write. These paths require Docker Desktop or a compatible Docker CLI and may download images/models from external registries.

Configuration

MeetingAssistant/appsettings.json is the canonical configuration example. Keep secret values out of tracked JSON and local rule/config files out of source control.

The settings with the largest operational effect are:

Vault: selects the durable vault and artifact/project locations.
Recording:TranscriptionProvider: selects azure-speech, funasr, or whisper-local; the latter requires a local Whisper model file.
Recording:MicrophoneDeviceId, mix gains, stop timeout, minimum duration, and temporary folder: control capture selection, audio, cleanup, and Azure backlog behavior. Microphone-device selection is Windows-only; macOS follows the system default input device.
Recording:InactivitySafeguard: prompts and can auto-finish a run after no new transcript text; it is not an audio-silence detector.
LaunchProfiles: overlay named recording/ASR/agent settings and require distinct hotkeys.
Automation:RulesPath: points to the local YAML workflow-rules file, normally ignored meeting-rules.local.yaml.
CalendarRecordingPrompts and Screenshots: control Outlook prompts on Windows, EventKit prompts on macOS, capture, attachments, and configured OCR.
Agent and WorkflowRulesEditor: select the Responses endpoint/model, streaming or non-streaming transport, reasoning, retry, output, and compaction behavior; the available tools are defined by the application.

Required or commonly used secrets:

AZURE_SPEECH_KEY for Azure Speech live transcription and speaker matching.
LITELLM_API_KEY for summary, OCR, and workflow editor agents when their effective endpoint requires an API key.
HF_TOKEN for pyannote model access when pyannote diarization or validation is enabled.

See docs/meeting-assistant-configuration.md for the full configuration reference.

Integrations

Obsidian vault: primary durable store for notes, transcripts, summaries, assistant context, project files, and generated links.
Outlook Classic on Windows: optional COM metadata lookup and scheduled Teams-meeting start prompts.
EventKit on macOS: metadata lookup and native recording prompts for Teams events in calendars available to macOS, including Outlook-synced calendars.
Native active-window capture: Windows foreground-window capture or macOS frontmost-application window capture, feeding the same screenshot/OCR pipeline.
Azure AI Speech: default checked-in ASR path, live diarized conversation transcription, and speaker identity matching.
FunASR: optional WebSocket streaming ASR. When managed backend startup is enabled, the app pulls and runs the configured Docker image as meeting-assistant-funasr on port 10095.
Whisper.NET plus pyannote: optional local Whisper fallback and Docker-backed final diarization.
LiteLLM/OpenAI-compatible Responses endpoint: summary generation, screenshot OCR fallback, project tools, the tray-launched assistant, and retry flows.
Docker Desktop or compatible Docker CLI: required only for managed FunASR and pyannote paths.

Workflow Rules And Agents

Meeting-specific automation lives in a local YAML file, not in committed personal rules. Rules can trigger on meeting creation, assistant-context state transitions, identified speakers, transcript-line writes, and added attendees. They can add/remove attendees, set supported properties, add context, add projects, rewrite a transcript line, and transform an attendee name before it is stored.

The Windows tray and macOS menu-bar menus expose Open agent, which opens the Meeting Summary Agent window. It can edit workflow rules with validation, inspect logs and health/status, manage speaker identities and samples, run ASR diagnostics, and read/write scoped meeting/project artifacts through explicit tools. Windows renders the chat with WPF; macOS renders the same agent pipeline in a native AppKit window backed by WebKit.

At startup, Meeting Assistant detects whether the host is Windows or macOS and includes that immutable runtime context in the interactive settings/logs assistant instructions. On Windows the assistant uses Windows commands and concepts such as PowerShell, Windows paths, services, and Task Manager. On macOS it uses zsh, POSIX paths, launchd/LaunchAgents, and Activity Monitor. This guidance is also appended when a custom interactive-agent prompt is configured. macOS audio capture, menu-bar controls, global hotkeys, EventKit calendar enrichment/prompts, active-window screenshots, screenshot OCR, speaker identification, FunASR, local diarization, and the AppKit/WebKit workflow editor are available in the portable build. Outlook Classic COM and Windows toast notifications remain Windows-specific implementations.

Detailed workflow syntax and extension guidance live in docs/meeting-workflow-engine.md.

Development And CI

Behavior changes are OpenSpec-driven and test-first: update the relevant requirement/scenario, add a failing public behavior test, implement the smallest passing change, run focused tests and then the justified broader suite, and validate the active change with openspec validate <change-id> --strict. Documentation-only maintenance does not need a new OpenSpec change.

The Gitea workflow runs for pull requests, pushes, and manual dispatch. It builds the Windows target on an Ubuntu runner, installs Wine plus a matching Windows .NET SDK, and runs the test project through the Windows host under Wine. It validates source; it does not publish or deploy the workstation application.

Operations And Limitations

Treat recording, transcription drain, speaker finalization, OCR, and summarization as live user work. Never restart, kill, or clean runtime files until /recording/status is idle unless interruption is explicitly intended.
Summary and screenshot retry links use Api:PublicBaseUrl, which defaults to http://localhost:5090.
Workstation startup is external to this repository. On NA-EXC765X84 as of 2026-08-11, the installed Windows Startup shortcut invokes the restart helper at sign-in, so it republishes and replaces the process; it differs from the start-only shortcut tracked in Manuel/snippets, and the reason for that local deviation is not documented.
No public hostname, homelab ingress, always-on deployment, or remote-service availability contract is owned here.

16 KiB Raw Blame History