Meeting Assistant

Meeting Assistant is Manuel's local .NET meeting capture and knowledge service. It runs on Windows and macOS, records microphone plus system audio, maintains meeting artifacts in an Obsidian vault, and runs transcription, speaker attribution, screenshot OCR, and agentic summarization without depending on a meeting-platform API for the primary flow.

Scope And Runtime Boundary

This repository owns the application source, tests, OpenSpec requirements, checked-in configuration, and CI validation. It does not own workstation startup automation, a homelab deployment stack, Traefik routing, Docker Compose, or a deployed image tag.

• Windows builds provide the tray icon, global hotkeys, NAudio capture, Outlook Classic COM enrichment, active-window screenshots, and notifications.
• Portable builds on macOS provide a native menu-bar icon and global hotkeys, and capture the default microphone through AVFoundation plus computer output through ScreenCaptureKit.
• Portable builds on other hosts keep the server/testable service surface but do not provide meeting audio capture.
• The normal runtime endpoint is local HTTP on port 5090, with /health and /recording/status as the safe first checks. Accepted requirements live under openspec/specs. Relevant active changes under openspec/changes can describe implemented behavior that has not yet been folded into the accepted specs.

Quick Start

The project requires the .NET 10 SDK. Build and test it with:

dotnet restore MeetingAssistant.slnx
dotnet build MeetingAssistant.slnx
dotnet test MeetingAssistant.slnx

Do not start a second copy over the live workstation instance. Check the local control surface first:

Invoke-RestMethod http://127.0.0.1:5090/health
Invoke-RestMethod http://127.0.0.1:5090/recording/status

On macOS, build the portable target on the Mac that will run it:

dotnet build MeetingAssistant/MeetingAssistant.csproj -f net10.0
dotnet run --project MeetingAssistant/MeetingAssistant.csproj -f net10.0

The macOS build compiles bundled Swift helpers into Native/macos-meeting-audio-capture, Native/macos-desktop-controls, and Native/macos-meeting-integrations. On first use, allow Microphone, Screen & System Audio Recording, and Calendar Full Access under System Settings > Privacy & Security. The menu-bar icon and global hotkeys use the same configured bindings as Windows and call the local service surface.

For a foreground Windows development run when port 5090 is free:

dotnet run --project MeetingAssistant --framework net10.0-windows10.0.19041.0

The durable workstation instance is published and replaced through automation owned by Manuel/snippets:

powershell -ExecutionPolicy Bypass -File C:\Manuel\snippets\restart-meeting-assistant.ps1

The helper publishes before stopping the old process, starts the timestamped Windows publish on loopback port 5090, waits for /health, and records stdout, stderr, and the PID under %LOCALAPPDATA%\MeetingAssistant\Logs. Always confirm /recording/status is idle before invoking it.

Recording And Control

Recording can be controlled through global hotkeys, the Windows tray icon, the macOS menu-bar icon, or local HTTP endpoints. The default hotkeys are:

• Ctrl+Alt+M: toggle the default recording profile.
• Ctrl+Alt+L: start or switch to the configured english profile.
• Ctrl+Alt+Z: abort the active run and delete its artifacts.
• Ctrl+Alt+S: capture the active window into the meeting context.

The Windows tray presents Finish meeting as the primary action during capture; microphone selection, cancel/discard, profile switching, and pause/unpause transcription remain separate controls. A paused meeting stays active and can still be finished or canceled normally. Windows Exit requires confirmation while any meeting is recording or processing. The macOS menu offers profile start actions while idle and Stop meeting recording and transcribe plus cancel/discard during capture; its Exit action currently exits directly.

The loopback HTTP surface has no application authentication, so port 5090 must remain a trusted loopback-only control surface. The main endpoints are:

• GET /health
• GET /recording/status
• POST /recording/start, /recording/stop, /recording/toggle, /recording/abort
• POST /profiles/{launchProfile}/recording/start, /stop, /toggle, /abort
• POST /asr/transcribe-file and /asr/diarize-file for diagnostic WAV checks
• POST /diagnostics/workflow/reload
• POST /diagnostics/settings-and-logs/show
• POST /diagnostics/workflow/rules-editor/show
• GET /diagnostics/platform-capabilities
• POST /meetings/current/summary/run
• POST or GET /meetings/summary/retry
• POST or GET /meetings/screenshot-ocr/retry

Some generated retry links use GET while starting work.

Stopping capture lets buffered transcription, speaker work, meeting-note image OCR, screenshot OCR, and summary generation finish. Another meeting can start while an older stopped run finalizes; each run retains isolated options and artifact paths.

Pausing transcription keeps the active recognition pipeline and meeting session alive, including Azure conversation transcription and the run's speaker context. Captured audio is replaced with equal-length silence before it reaches the temporary WAV or transcription backend, so real audio from the paused interval is discarded while provider continuity and meeting-relative timing are preserved. Normal transcript-inactivity notifications and auto-stop are suppressed during pause, while a separate four-hour maximum continuous pause prevents a forgotten paused meeting from running indefinitely. The recording status response exposes pause state as isPaused.

During an active run, microphone creation failures and disconnects are retried every second with fresh endpoint selection. The meeting and system-loopback capture stay active, with microphone silence mixed in until capture resumes. This recovery does not cover a failed system-loopback source.

Outlook enrichment selects an unambiguous current or imminent appointment. A scheduled prompt shown during an active recording can apply that exact appointment's title, eligible attendees, agenda, and scheduled end without interrupting capture; explicit prompt metadata wins over a slower background lookup.

Data And Side Effects

Meeting Assistant writes meeting notes, transcripts, assistant context, summaries, and project knowledge into the configured Obsidian vault. Assistant context is persistent meeting-specific memory: the summarizer records problems and assumptions, and the interactive agent can read it and append later repairs and conclusions.

Agents are intentionally stateful. Depending on the invoked tools, they can change workflow rules and appsettings, create or update project and meeting files, change frontmatter, merge or delete speaker identities and samples, run diagnostics, and trigger transcription or summary work. Screenshot OCR can add recognized attendee names to the meeting note after workflow transformation; OCR of images already embedded in the meeting note does not add attendees or modify that note.

Local runtime state outside the vault includes:

• %LOCALAPPDATA%\MeetingAssistant\Recordings: mixed WAV files are normally deleted after completion, and unqueued stale files are deleted at startup. If an Azure stop cannot drain within Recording:StopProcessingTimeout, the WAV plus a JSON item under offline-transcription-backlog are retained and retried every minute. Both are removed only after successful replay, transcript finalization, and summary processing.
• %LOCALAPPDATA%\MeetingAssistant\SpeakerIdentity\speaker-identities.db: SQLite identities, aliases, meeting references, bounded WAV snippets for the existing matcher, and separate versioned voice vectors for the optional Resemblyzer matcher.
• %LOCALAPPDATA%\MeetingAssistant\Resemblyzer: content-versioned managed Python environments, the local encoder script, and temporary encoder inputs. Per-meeting WAV inputs are deleted after each encoding command.
• %LOCALAPPDATA%\MeetingAssistant\FunASR\models and %LOCALAPPDATA%\MeetingAssistant\Pyannote\models: persistent model, hotword, Hugging Face, and torch caches for optional local backends.
• %TEMP%\MeetingAssistant\Logs\meeting-assistant.log: application log with four rotated predecessors. Paths, transcript text, agent diagnostics, and provider errors can make these logs sensitive.

Abort is destructive: it removes the active run's note, transcript, context, summary, and linked screenshot attachments and skips summarization. A normal stop below Recording:MinimumCompletedMeetingDuration, or an otherwise content-empty stop, can also remove generated artifacts.

External Data Boundaries

“Local” describes control and durable storage, not every processing step:

• The default azure-speech provider sends mixed meeting audio and dictation phrase hints to Azure AI Speech. Azure-backed speaker matching also sends selected voice audio.
• Summary, screenshot OCR, and interactive-agent requests go to the configured OpenAI-compatible Responses endpoint. They can include meeting/transcript/project text, screenshots, configuration, logs, and speaker samples when corresponding tools are used. The checked-in endpoint is a loopback proxy; its ultimate provider, data path, and retention policy are outside this repository.
• Outlook Classic access on Windows is local COM. EventKit access on macOS reads calendars synchronized into the Calendar app. Neither provides the primary capture path.
• A managed FunASR run pulls its configured image, removes any same-named container, starts a disposable container privileged by default, publishes the configured host port, and mounts the model/hotword cache. Pyannote may build a local image and start disposable containers with audio inputs mounted read-only; those paths require Docker Desktop or a compatible Docker CLI. The opt-in Resemblyzer speaker matcher instead provisions an isolated local Python venv with CPU-only dependencies. First use may download images, Python packages, or models from external registries.

Configuration

MeetingAssistant/appsettings.json is the canonical configuration example. Keep secret values out of tracked JSON and local rule/config files out of source control.

The settings with the largest operational effect are:

• Vault: selects the durable vault and artifact/project locations.
• Recording:TranscriptionProvider: selects azure-speech, funasr, or whisper-local; the latter requires a local Whisper model file.
• Recording:MicrophoneDeviceId, mix gains, stop timeout, minimum duration, and temporary folder: control capture selection, audio, cleanup, and Azure backlog behavior. Microphone-device selection is Windows-only; macOS follows the system default input device.
• Recording:InactivitySafeguard: prompts and can auto-finish a run after no new transcript text; it is not an audio-silence detector.
• LaunchProfiles: overlay named recording/ASR/agent settings and require distinct hotkeys.
• SpeakerIdentification:Resemblyzer:Enabled: selects the local, managed-Python-venv vector matcher for the whole application; when disabled, the existing WAV/Azure path stays active. Five vectors unlock matching by default without capping retained evidence, and mature profiles use configurable fail-safe density clustering to remove likely mixed-speaker outliers.
• Automation:RulesPath: points to the local YAML workflow-rules file, normally ignored meeting-rules.local.yaml.
• CalendarRecordingPrompts and Screenshots: control Outlook prompts on Windows, EventKit prompts on macOS, capture, attachments, and configured OCR.
• Agent and WorkflowRulesEditor: select the Responses endpoint/model, streaming or non-streaming transport, reasoning, retry, output, and compaction behavior; the available tools are defined by the application.

Required or commonly used secrets:

• AZURE_SPEECH_KEY for Azure Speech live transcription and speaker matching.
• LITELLM_API_KEY for summary, OCR, and workflow editor agents when their effective endpoint requires an API key.
• HF_TOKEN for pyannote model access when pyannote diarization or validation is enabled.

See docs/meeting-assistant-configuration.md for the full configuration reference.

Integrations

• Obsidian vault: primary durable store for notes, transcripts, summaries, assistant context, project files, and generated links.
• Outlook Classic on Windows: optional COM metadata lookup and scheduled Teams-meeting start prompts.
• EventKit on macOS: metadata lookup and native recording prompts for Teams events in calendars available to macOS, including Outlook-synced calendars.
• Native active-window capture: Windows foreground-window capture or macOS frontmost-application window capture, feeding the same screenshot/OCR pipeline.
• Azure AI Speech: default checked-in ASR path, live diarized conversation transcription, and speaker identity matching.
• FunASR: optional WebSocket streaming ASR. When managed backend startup is enabled, the app pulls and runs the configured Docker image as meeting-assistant-funasr on port 10095.
• Whisper.NET plus pyannote: optional local Whisper fallback and Docker-backed final diarization.
• LiteLLM/OpenAI-compatible Responses endpoint: summary generation, screenshot OCR fallback, project tools, the tray-launched assistant, and retry flows.
• Docker Desktop or compatible Docker CLI: required only for managed FunASR and pyannote paths.

Workflow Rules And Agents

Meeting-specific automation lives in a local YAML file, not in committed personal rules. Rules can trigger on meeting creation, assistant-context state transitions, identified speakers, transcript-line writes, and added attendees. They can add/remove attendees, set supported properties, add context, add projects, rewrite a transcript line, and transform an attendee name before it is stored.

The Windows tray and macOS menu-bar menus expose Open agent, which opens the Meeting Summary Agent window. It can edit workflow rules with validation, inspect logs and health/status, manage speaker identities and samples, run ASR diagnostics, and read/write scoped meeting/project artifacts through explicit tools. Windows renders the chat with WPF; macOS renders the same agent pipeline in a native AppKit window backed by WebKit.

At startup, Meeting Assistant detects whether the host is Windows or macOS and includes that immutable runtime context in the interactive settings/logs assistant instructions. On Windows the assistant uses Windows commands and concepts such as PowerShell, Windows paths, services, and Task Manager. On macOS it uses zsh, POSIX paths, launchd/LaunchAgents, and Activity Monitor. This guidance is also appended when a custom interactive-agent prompt is configured. macOS audio capture, menu-bar controls, global hotkeys, EventKit calendar enrichment/prompts, active-window screenshots, screenshot OCR, speaker identification, FunASR, local diarization, and the AppKit/WebKit workflow editor are available in the portable build. Outlook Classic COM and Windows toast notifications remain Windows-specific implementations.

The macOS publish output includes a signed MeetingAssistant.app bundle. Install that bundle in /Applications and launch the background service through its native executable so macOS microphone, calendar, and Screen/System Audio privacy grants are attributed to the stable Meeting Assistant application identity.

Detailed workflow syntax and extension guidance live in docs/meeting-workflow-engine.md.

Development And CI

Behavior changes are OpenSpec-driven and test-first: update the relevant requirement/scenario, add a failing public behavior test, implement the smallest passing change, run focused tests and then the justified broader suite, and validate the active change with openspec validate <change-id> --strict. Documentation-only maintenance does not need a new OpenSpec change.

The Gitea workflow runs for pull requests, pushes, and manual dispatch. Its Linux job builds the Windows target, installs Wine plus a matching Windows .NET SDK, and runs the test project through the Windows host under Wine. A separate runs-on: macos job builds the portable target and bundled Swift helpers, verifies the native helper executables, and runs the complete portable unit-test suite on macOS. The Gitea runner serving that job must run natively on a Mac and expose the macos:host label; Docker-OSX is not used because it requires nested KVM that is normally unavailable on CI runners. CI validates source; it does not publish or deploy the workstation application.

Operations And Limitations

• Treat recording, transcription drain, speaker finalization, OCR, and summarization as live user work. Never restart, kill, or clean runtime files until /recording/status is idle unless interruption is explicitly intended.
• Summary and screenshot retry links use Api:PublicBaseUrl, which defaults to http://localhost:5090.
• Workstation startup is external to this repository. On NA-EXC765X84 as of 2026-08-11, the installed Windows Startup shortcut invokes the restart helper at sign-in, so it republishes and replaces the process; it differs from the start-only shortcut tracked in Manuel/snippets, and the reason for that local deviation is not documented.
• No public hostname, homelab ingress, always-on deployment, or remote-service availability contract is owned here.

More Documentation

• docs/meeting-assistant-configuration.md: complete configuration and backend behavior.
• docs/meeting-workflow-engine.md: workflow triggers, conditions, templates, steps, and editor behavior.
• openspec/specs: accepted behavioral requirements.
• openspec/changes: active change specs and designs, including implemented work not yet archived.
• openspec/changes/archive: historical proposals and decisions behind accepted behavior.