meeting-assistant/README.md

Meeting Assistant

Meeting Assistant is Manuel's local Windows meeting capture and knowledge service. It records microphone plus system audio, maintains meeting artifacts in an Obsidian vault, enriches them from local context, and runs transcription, speaker attribution, screenshot OCR, and agentic summarization without depending on a meeting-platform API for the primary flow.

Scope And Runtime Boundary

This repository owns the application source, tests, OpenSpec requirements, checked-in configuration, and CI validation. It does not own workstation startup automation, a homelab deployment stack, Traefik routing, Docker Compose, or a deployed image tag.

The Windows target provides audio capture, the tray icon, global hotkeys, Outlook Classic COM access, active-window screenshots, notifications, and the workflow-agent window. The cross-platform target keeps the HTTP and testable service surface but substitutes unavailable/no-op implementations for Windows integrations.

Accepted requirements live under openspec/specs. Relevant active changes under openspec/changes can describe implemented behavior that has not yet been folded into the accepted specs.

Quick Start

The project requires the .NET 10 SDK. Build and test it with:

dotnet restore MeetingAssistant.slnx
dotnet build MeetingAssistant.slnx
dotnet test MeetingAssistant.slnx

Do not start a second copy over the live workstation instance. Check the local control surface first:

Invoke-RestMethod http://127.0.0.1:5090/health
Invoke-RestMethod http://127.0.0.1:5090/recording/status

For a foreground Windows development run when port 5090 is free:

dotnet run --project MeetingAssistant --framework net10.0-windows10.0.19041.0

The durable workstation instance is published and replaced through automation owned by Manuel/snippets:

powershell -ExecutionPolicy Bypass -File C:\Manuel\snippets\restart-meeting-assistant.ps1

The helper publishes before stopping the old process, starts the timestamped Windows publish on loopback port 5090, waits for /health, and records stdout, stderr, and the PID under %LOCALAPPDATA%\MeetingAssistant\Logs. Always confirm /recording/status is idle before invoking it.

Recording And Control

The default global controls are:

• Ctrl+Alt+M: toggle the default recording profile.
• Ctrl+Alt+L: start or switch to the configured english profile.
• Ctrl+Alt+Z: abort the active run and delete its artifacts.
• Ctrl+Alt+S: capture the active window into the meeting context.

The tray presents Finish meeting as the primary action during capture; pause/unpause transcription, microphone selection, cancel/discard, and profile switching remain separate controls. A paused meeting stays active and can still be finished or canceled normally. Exit is always available and requires confirmation while any meeting is recording or processing.

The loopback HTTP surface exposes health and recording status, recording start/stop/toggle/abort operations, profile-specific equivalents, and diagnostics/retry operations. It has no application authentication. Some generated retry links use GET while starting work, so port 5090 must remain a trusted loopback-only control surface.

Stopping capture lets buffered transcription, speaker work, meeting-note image OCR, screenshot OCR, and summary generation finish. Another meeting can start while an older stopped run finalizes; each run retains isolated options and artifact paths.

Pausing transcription keeps the active recognition pipeline and meeting session alive, including Azure conversation transcription and the run's speaker context. Captured audio is replaced with equal-length silence before it reaches the temporary WAV or transcription backend, so real audio from the paused interval is discarded while provider continuity and meeting-relative timing are preserved. Normal transcript-inactivity notifications and auto-stop are suppressed during pause, while a separate four-hour maximum continuous pause prevents a forgotten paused meeting from running indefinitely. The recording status response exposes pause state as isPaused.

During an active run, microphone creation failures and disconnects are retried every second with fresh endpoint selection. The meeting and system-loopback capture stay active, with microphone silence mixed in until capture resumes. This recovery does not cover a failed system-loopback source.

A newly available microphone during a meeting triggers a Windows notification with its name and Yes/No actions, including while transcription is paused. Yes switches the current microphone within the same meeting; No or no response keeps capture unchanged. Each device is offered once per meeting, and unanswered prompts expire after one minute.

Outlook enrichment selects an unambiguous current or imminent appointment. A scheduled prompt shown during an active recording can apply that exact appointment's title, eligible attendees, agenda, and scheduled end without interrupting capture; explicit prompt metadata wins over a slower background lookup.

Data And Side Effects

Meeting Assistant writes meeting notes, transcripts, assistant context, summaries, and project knowledge into the configured Obsidian vault. Assistant context is persistent meeting-specific memory: the summarizer records problems and assumptions, and the interactive agent can read it and append later repairs and conclusions.

Project discovery uses the name and short description in each project's PROJECT.md. The summarizer can request missing associations through a Yes/No notification that waits up to ten minutes; only approval adds the project and enables its file access. Workflow-change requests return immediately and, when approved, open a separate settings-agent conversation with the proposed change.

Agents are intentionally stateful. Depending on the invoked tools, they can change workflow rules and appsettings, create or update project and meeting files, change frontmatter, merge or delete speaker identities and samples, run diagnostics, and trigger transcription or summary work. Screenshot OCR can add recognized attendee names to the meeting note after workflow transformation; OCR of images already embedded in the meeting note does not add attendees or modify that note.

Local runtime state outside the vault includes:

• %LOCALAPPDATA%\MeetingAssistant\Recordings: mixed WAV files are normally deleted after completion, and unqueued stale files are deleted at startup. If an Azure stop cannot drain within Recording:StopProcessingTimeout, the WAV plus a JSON item under offline-transcription-backlog are retained and retried every minute. Both are removed only after successful replay, transcript finalization, and summary processing.
• %LOCALAPPDATA%\MeetingAssistant\SpeakerIdentity\speaker-identities.db: SQLite identities, aliases, meeting references, bounded WAV snippets for the existing matcher, and separate versioned voice vectors for the optional Resemblyzer matcher.
• %LOCALAPPDATA%\MeetingAssistant\Resemblyzer: content-versioned managed Python environments, the local encoder script, and temporary encoder inputs. Per-meeting WAV inputs are deleted after each encoding command.
• %LOCALAPPDATA%\MeetingAssistant\FunASR\models and %LOCALAPPDATA%\MeetingAssistant\Pyannote\models: persistent model, hotword, Hugging Face, and torch caches for optional local backends.
• %TEMP%\MeetingAssistant\Logs\meeting-assistant.log: application log with four rotated predecessors. Paths, transcript text, agent diagnostics, and provider errors can make these logs sensitive.

Abort is destructive: it removes the active run's note, transcript, context, summary, and linked screenshot attachments and skips summarization. A normal stop below Recording:MinimumCompletedMeetingDuration, or an otherwise content-empty stop, can also remove generated artifacts.

External Data Boundaries

“Local” describes control and durable storage, not every processing step:

• The default azure-speech provider sends mixed meeting audio and dictation phrase hints to Azure AI Speech. Azure-backed speaker matching also sends selected voice audio.
• Summary, screenshot OCR, and interactive-agent requests go to the configured OpenAI-compatible Responses endpoint. They can include meeting/transcript/project text, screenshots, configuration, logs, and speaker samples when corresponding tools are used. The checked-in endpoint is a loopback proxy; its ultimate provider, data path, and retention policy are outside this repository.
• Outlook Classic access is local COM and reads appointment metadata; it does not provide the primary capture path.
• A managed FunASR run pulls its configured image, removes any same-named container, starts a disposable container privileged by default, publishes the configured host port, and mounts the model/hotword cache. Pyannote may build a local image and start disposable containers with audio inputs mounted read-only; those paths require Docker Desktop or a compatible Docker CLI. The opt-in Resemblyzer speaker matcher instead provisions an isolated local Python venv with CPU-only dependencies. First use may download images, Python packages, or models from external registries.

Configuration

MeetingAssistant/appsettings.json is the canonical configuration example. Keep secret values out of tracked JSON and local rule/config files out of source control.

The settings with the largest operational effect are:

• Vault: selects the durable vault and artifact/project locations.
• Recording:TranscriptionProvider: selects azure-speech, funasr, or whisper-local; the latter requires a local Whisper model file.
• Recording:MicrophoneDeviceId, mix gains, stop timeout, minimum duration, and temporary folder: control capture selection, audio, cleanup, and Azure backlog behavior.
• Recording:InactivitySafeguard: prompts and can auto-finish a run after no new transcript text; it is not an audio-silence detector.
• LaunchProfiles: overlay named recording/ASR/agent settings and require distinct hotkeys.
• SpeakerIdentification:Resemblyzer:Enabled: selects the local, managed-Python-venv vector matcher for the whole application; when disabled, the existing WAV/Azure path stays active. Five vectors unlock matching by default without capping retained evidence, and mature profiles use configurable fail-safe density clustering to remove likely mixed-speaker outliers.
• Automation:RulesPath: points to the local YAML rules file, normally ignored meeting-rules.local.yaml.
• CalendarRecordingPrompts and Screenshots: control Outlook prompts, capture, attachments, and configured OCR.
• Agent and WorkflowRulesEditor: select the Responses endpoint/model, streaming or non-streaming transport, reasoning, retry, output, and compaction behavior; the available tools are defined by the application.

Common secret environment variables are AZURE_SPEECH_KEY, LITELLM_API_KEY, and HF_TOKEN. See docs/meeting-assistant-configuration.md for the full setting reference and docs/meeting-workflow-engine.md for rule syntax and safety behavior.

Development And CI

Behavior changes are OpenSpec-driven and test-first: update the relevant requirement/scenario, add a failing public behavior test, implement the smallest passing change, run focused tests and then the justified broader suite, and validate the active change with openspec validate <change-id> --strict. Documentation-only maintenance does not need a new OpenSpec change.

The Gitea workflow runs for pull requests, pushes, and manual dispatch. It builds the Windows target on an Ubuntu runner, installs Wine plus a matching Windows .NET SDK, and runs the test project through the Windows host under Wine. It validates source; it does not publish or deploy the workstation application.

Operations And Limitations

• Treat recording, transcription drain, speaker finalization, OCR, and summarization as live user work. Never restart, kill, or clean runtime files until /recording/status is idle unless interruption is explicitly intended.
• Summary and screenshot retry links use Api:PublicBaseUrl, which defaults to http://localhost:5090.
• Workstation startup is external to this repository. On NA-EXC765X84 as of 2026-08-11, the installed Windows Startup shortcut invokes the restart helper at sign-in, so it republishes and replaces the process; it differs from the start-only shortcut tracked in Manuel/snippets, and the reason for that local deviation is not documented.
• No public hostname, homelab ingress, always-on deployment, or remote-service availability contract is owned here.

More Documentation

• docs/meeting-assistant-configuration.md: complete configuration and backend behavior.
• docs/meeting-workflow-engine.md: workflow triggers, conditions, templates, steps, and editor behavior.
• openspec/specs: accepted behavioral requirements.
• openspec/changes: active change specs and designs, including implemented work not yet archived.
• openspec/changes/archive: historical proposals and decisions behind accepted behavior.