forked from Manuel/meeting-assistant
124 lines
11 KiB
Markdown
124 lines
11 KiB
Markdown
# Meeting Assistant
|
|
|
|
Meeting Assistant is Manuel's local Windows meeting capture and knowledge service. It records microphone plus system audio, maintains meeting artifacts in an Obsidian vault, enriches them from local context, and runs transcription, speaker attribution, screenshot OCR, and agentic summarization without depending on a meeting-platform API for the primary flow.
|
|
|
|
## Scope And Runtime Boundary
|
|
|
|
This repository owns the application source, tests, OpenSpec requirements, checked-in configuration, and CI validation. It does not own workstation startup automation, a homelab deployment stack, Traefik routing, Docker Compose, or a deployed image tag.
|
|
|
|
The Windows target provides audio capture, the tray icon, global hotkeys, Outlook Classic COM access, active-window screenshots, notifications, and the workflow-agent window. The cross-platform target keeps the HTTP and testable service surface but substitutes unavailable/no-op implementations for Windows integrations.
|
|
|
|
Accepted requirements live under `openspec/specs`. Relevant active changes under `openspec/changes` can describe implemented behavior that has not yet been folded into the accepted specs.
|
|
|
|
## Quick Start
|
|
|
|
The project requires the .NET 10 SDK. Build and test it with:
|
|
|
|
```powershell
|
|
dotnet restore MeetingAssistant.slnx
|
|
dotnet build MeetingAssistant.slnx
|
|
dotnet test MeetingAssistant.slnx
|
|
```
|
|
|
|
Do not start a second copy over the live workstation instance. Check the local control surface first:
|
|
|
|
```powershell
|
|
Invoke-RestMethod http://127.0.0.1:5090/health
|
|
Invoke-RestMethod http://127.0.0.1:5090/recording/status
|
|
```
|
|
|
|
For a foreground Windows development run when port `5090` is free:
|
|
|
|
```powershell
|
|
dotnet run --project MeetingAssistant --framework net10.0-windows10.0.19041.0
|
|
```
|
|
|
|
The durable workstation instance is published and replaced through automation owned by `Manuel/snippets`:
|
|
|
|
```powershell
|
|
powershell -ExecutionPolicy Bypass -File C:\Manuel\snippets\restart-meeting-assistant.ps1
|
|
```
|
|
|
|
The helper publishes before stopping the old process, starts the timestamped Windows publish on loopback port `5090`, waits for `/health`, and records stdout, stderr, and the PID under `%LOCALAPPDATA%\MeetingAssistant\Logs`. Always confirm `/recording/status` is idle before invoking it.
|
|
|
|
## Recording And Control
|
|
|
|
The default global controls are:
|
|
|
|
- `Ctrl+Alt+M`: toggle the default recording profile.
|
|
- `Ctrl+Alt+L`: start or switch to the configured `english` profile.
|
|
- `Ctrl+Alt+Z`: abort the active run and delete its artifacts.
|
|
- `Ctrl+Alt+S`: capture the active window into the meeting context.
|
|
|
|
The tray presents `Finish meeting` as the primary action during capture; microphone selection, cancel/discard, and profile switching remain separate controls. `Exit` is always available and requires confirmation while any meeting is recording or processing.
|
|
|
|
The loopback HTTP surface exposes health and recording status, recording start/stop/toggle/abort operations, profile-specific equivalents, and diagnostics/retry operations. It has no application authentication. Some generated retry links use `GET` while starting work, so port `5090` must remain a trusted loopback-only control surface.
|
|
|
|
Stopping capture lets buffered transcription, speaker work, meeting-note image OCR, screenshot OCR, and summary generation finish. Another meeting can start while an older stopped run finalizes; each run retains isolated options and artifact paths.
|
|
|
|
During an active run, microphone creation failures and disconnects are retried every second with fresh endpoint selection. The meeting and system-loopback capture stay active, with microphone silence mixed in until capture resumes. This recovery does not cover a failed system-loopback source.
|
|
|
|
Outlook enrichment selects an unambiguous current or imminent appointment. A scheduled prompt shown during an active recording can apply that exact appointment's title, eligible attendees, agenda, and scheduled end without interrupting capture; explicit prompt metadata wins over a slower background lookup.
|
|
|
|
## Data And Side Effects
|
|
|
|
Meeting Assistant writes meeting notes, transcripts, assistant context, summaries, and project knowledge into the configured Obsidian vault. Assistant context is persistent meeting-specific memory: the summarizer records problems and assumptions, and the interactive agent can read it and append later repairs and conclusions.
|
|
|
|
Agents are intentionally stateful. Depending on the invoked tools, they can change workflow rules and appsettings, create or update project and meeting files, change frontmatter, merge or delete speaker identities and samples, run diagnostics, and trigger transcription or summary work. Screenshot OCR can add recognized attendee names to the meeting note after workflow transformation; OCR of images already embedded in the meeting note does not add attendees or modify that note.
|
|
|
|
Local runtime state outside the vault includes:
|
|
|
|
- `%LOCALAPPDATA%\MeetingAssistant\Recordings`: mixed WAV files are normally deleted after completion, and unqueued stale files are deleted at startup. If an Azure stop cannot drain within `Recording:StopProcessingTimeout`, the WAV plus a JSON item under `offline-transcription-backlog` are retained and retried every minute. Both are removed only after successful replay, transcript finalization, and summary processing.
|
|
- `%LOCALAPPDATA%\MeetingAssistant\SpeakerIdentity\speaker-identities.db`: SQLite identities, aliases, meeting references, and bounded voice snippets.
|
|
- `%LOCALAPPDATA%\MeetingAssistant\FunASR\models` and `%LOCALAPPDATA%\MeetingAssistant\Pyannote\models`: persistent model, hotword, Hugging Face, and torch caches for optional local backends.
|
|
- `%TEMP%\MeetingAssistant\Logs\meeting-assistant.log`: application log with four rotated predecessors. Paths, transcript text, agent diagnostics, and provider errors can make these logs sensitive.
|
|
|
|
Abort is destructive: it removes the active run's note, transcript, context, summary, and linked screenshot attachments and skips summarization. A normal stop below `Recording:MinimumCompletedMeetingDuration`, or an otherwise content-empty stop, can also remove generated artifacts.
|
|
|
|
## External Data Boundaries
|
|
|
|
“Local” describes control and durable storage, not every processing step:
|
|
|
|
- The default `azure-speech` provider sends mixed meeting audio and dictation phrase hints to Azure AI Speech. Azure-backed speaker matching also sends selected voice audio.
|
|
- Summary, screenshot OCR, and interactive-agent requests go to the configured OpenAI-compatible Responses endpoint. They can include meeting/transcript/project text, screenshots, configuration, logs, and speaker samples when corresponding tools are used. The checked-in endpoint is a loopback proxy; its ultimate provider, data path, and retention policy are outside this repository.
|
|
- Outlook Classic access is local COM and reads appointment metadata; it does not provide the primary capture path.
|
|
- A managed FunASR run pulls its configured image, removes any same-named container, starts a disposable container privileged by default, publishes the configured host port, and mounts the model/hotword cache. Pyannote may build a local image and starts disposable containers with the input WAV mounted read-only and its model cache read/write. These paths require Docker Desktop or a compatible Docker CLI and may download images/models from external registries.
|
|
|
|
## Configuration
|
|
|
|
`MeetingAssistant/appsettings.json` is the canonical configuration example. Keep secret values out of tracked JSON and local rule/config files out of source control.
|
|
|
|
The settings with the largest operational effect are:
|
|
|
|
- `Vault`: selects the durable vault and artifact/project locations.
|
|
- `Recording:TranscriptionProvider`: selects `azure-speech`, `funasr`, or `whisper-local`; the latter requires a local Whisper model file.
|
|
- `Recording:MicrophoneDeviceId`, mix gains, stop timeout, minimum duration, and temporary folder: control capture selection, audio, cleanup, and Azure backlog behavior.
|
|
- `Recording:InactivitySafeguard`: prompts and can auto-finish a run after no new transcript text; it is not an audio-silence detector.
|
|
- `LaunchProfiles`: overlay named recording/ASR/agent settings and require distinct hotkeys.
|
|
- `Automation:RulesPath`: points to the local YAML rules file, normally ignored `meeting-rules.local.yaml`.
|
|
- `CalendarRecordingPrompts` and `Screenshots`: control Outlook prompts, capture, attachments, and configured OCR.
|
|
- `Agent` and `WorkflowRulesEditor`: select the Responses endpoint/model, streaming or non-streaming transport, reasoning, retry, output, and compaction behavior; the available tools are defined by the application.
|
|
|
|
Common secret environment variables are `AZURE_SPEECH_KEY`, `LITELLM_API_KEY`, and `HF_TOKEN`. See `docs/meeting-assistant-configuration.md` for the full setting reference and `docs/meeting-workflow-engine.md` for rule syntax and safety behavior.
|
|
|
|
## Development And CI
|
|
|
|
Behavior changes are OpenSpec-driven and test-first: update the relevant requirement/scenario, add a failing public behavior test, implement the smallest passing change, run focused tests and then the justified broader suite, and validate the active change with `openspec validate <change-id> --strict`. Documentation-only maintenance does not need a new OpenSpec change.
|
|
|
|
The Gitea workflow runs for pull requests, pushes, and manual dispatch. It builds the Windows target on an Ubuntu runner, installs Wine plus a matching Windows .NET SDK, and runs the test project through the Windows host under Wine. It validates source; it does not publish or deploy the workstation application.
|
|
|
|
## Operations And Limitations
|
|
|
|
- Treat recording, transcription drain, speaker finalization, OCR, and summarization as live user work. Never restart, kill, or clean runtime files until `/recording/status` is idle unless interruption is explicitly intended.
|
|
- Summary and screenshot retry links use `Api:PublicBaseUrl`, which defaults to `http://localhost:5090`.
|
|
- Workstation startup is external to this repository. On `NA-EXC765X84` as of 2026-08-11, the installed Windows Startup shortcut invokes the restart helper at sign-in, so it republishes and replaces the process; it differs from the start-only shortcut tracked in `Manuel/snippets`, and the reason for that local deviation is not documented.
|
|
- No public hostname, homelab ingress, always-on deployment, or remote-service availability contract is owned here.
|
|
|
|
## More Documentation
|
|
|
|
- `docs/meeting-assistant-configuration.md`: complete configuration and backend behavior.
|
|
- `docs/meeting-workflow-engine.md`: workflow triggers, conditions, templates, steps, and editor behavior.
|
|
- `openspec/specs`: accepted behavioral requirements.
|
|
- `openspec/changes`: active change specs and designs, including implemented work not yet archived.
|
|
- `openspec/changes/archive`: historical proposals and decisions behind accepted behavior.
|