Meeting Assistant
Meeting Assistant is a .NET 10 server application for capturing and enriching meetings.
Its core purpose is to work with any kind of meeting, including in-person meetings. Integrations with platforms such as Teams, Zoom, or similar tools may augment the experience later, but the system must not depend on those product APIs for its primary meeting flow.
The application is intended to:
- create an Obsidian markdown note before transcription starts
- keep meeting metadata, user notes, detected context, and links to generated output in that note
- toggle recording/transcription mode through a configurable global hotkey
- switch the active transcription profile during a meeting with a profile-specific toggle hotkey
- abort and discard an active recording through a configurable global hotkey
- show a Windows taskbar notification icon with state-aware recording controls
- capture active-window screenshots through a configurable global hotkey
- capture microphone input and computer output into one transcription stream
- transcribe meetings with speaker attribution
- use a configurable speech recognition pipeline, with local Whisper as the first version 1 fallback
- use FunASR or local Whisper plus pyannote as configurable speaker-attribution paths
- write transcript markdown files into the configured vault folder
- generate summaries, decisions, and next steps
- build and maintain a project knowledge base from meetings and future context sources
- use Microsoft Agent Framework-based agents to reason over meeting and project context
- give agents tools for project lookup, keyword lookup, retrieval-augmented generation, and project file updates in existing projects
Repository Status
This repository currently contains the initial .NET 10 application skeleton, a health endpoint, and the first OpenSpec change for version 1.
Local Development
Restore, build, and test:
dotnet restore MeetingAssistant.slnx
dotnet build MeetingAssistant.slnx
dotnet test MeetingAssistant.slnx
Run the server:
dotnet run --project MeetingAssistant
Local Autostart and Restart
For the local Windows workstation, use the snippets restart helper instead of running the app directly when you want a durable background instance:
powershell -ExecutionPolicy Bypass -File C:\Manuel\snippets\restart-meeting-assistant.ps1
The restart helper publishes MeetingAssistant\MeetingAssistant.csproj as net10.0-windows into a fresh temp folder under C:\Manuel\meeting-assistant\tmp\meeting-assistant-runtime. Only after that publish succeeds does it stop any active Meeting Assistant instance on port 5090 and start the newly published executable detached from the script.
The detached starter writes logs and the last process id to:
%LOCALAPPDATA%\MeetingAssistant\Logs\meeting-assistant.out.log
%LOCALAPPDATA%\MeetingAssistant\Logs\meeting-assistant.err.log
%LOCALAPPDATA%\MeetingAssistant\Logs\meeting-assistant.pid
This script is also the intended autostart target. Add a Windows Startup shortcut to C:\Manuel\snippets\restart-meeting-assistant.ps1 when the app should rebuild and restart automatically after login.
The initial endpoints are:
GET /health
GET /recording/status
POST /recording/toggle
POST /recording/start
POST /recording/stop
POST /recording/abort
POST /asr/transcribe-file
POST /asr/diarize-file
POST /diagnostics/workflow/reload
POST /meetings/current/summary/run
POST /meetings/summary/retry
GET /meetings/summary/retry?summaryPath=...
Configuration
Meeting Assistant uses normal .NET configuration under the MeetingAssistant section:
{
"MeetingAssistant": {
"Hotkey": {
"Toggle": "Ctrl+Alt+M",
"Abort": "Ctrl+Alt+Z"
},
"Vault": {
"TranscriptsFolder": "%USERPROFILE%\\OpenCloud\\Persönlich\\Vault\\Meetings\\Transcripts",
"MeetingNotesFolder": "%USERPROFILE%\\OpenCloud\\Persönlich\\Vault\\Meetings\\Notes",
"AssistantContextFolder": "%USERPROFILE%\\OpenCloud\\Persönlich\\Vault\\Meetings\\Assistant Context",
"SummariesFolder": "%USERPROFILE%\\OpenCloud\\Persönlich\\Vault\\Meetings\\Summaries",
"ProjectsFolder": "%USERPROFILE%\\OpenCloud\\Persönlich\\Vault\\Projects",
"DictationWordsPath": "%USERPROFILE%\\OpenCloud\\Persönlich\\Vault\\Meetings\\dictation-words.md"
},
"Recording": {
"TranscriptionProvider": "funasr",
"SampleRate": 16000,
"Channels": 1,
"StopProcessingTimeout": "00:10:00",
"MaxMetadataAttendeeImportCount": 30,
"TemporaryRecordingsFolder": "%LOCALAPPDATA%\\MeetingAssistant\\Recordings"
},
"FunAsr": {
"Endpoint": "ws://127.0.0.1:10095",
"Mode": "2pass",
"ChunkSize": [5, 10, 5],
"ChunkInterval": 10,
"EncoderChunkLookBack": 4,
"DecoderChunkLookBack": 1,
"Itn": true,
"EmitOnlinePartials": false,
"WavName": "meeting",
"FinalResultTimeout": "00:00:10",
"Backend": {
"Enabled": true,
"DockerCommand": "docker",
"Image": "registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.13",
"ContainerName": "meeting-assistant-funasr",
"ModelsFolder": "%LOCALAPPDATA%\\MeetingAssistant\\FunASR\\models",
"ContainerPort": 10095,
"Privileged": true,
"CommandTimeout": "00:15:00",
"StartupTimeout": "00:10:00",
"StopOnDispose": true,
"ServerCommand": "cd /workspace/FunASR/runtime && bash run_server_2pass.sh --download-model-dir /workspace/models --vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx --model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx --online-model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx --punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx --itn-dir thuduj12/fst_itn_zh --certfile 0 --hotword /workspace/models/hotwords.txt && tail -f /dev/null"
},
"Diarization": {
"Enabled": true,
"AsrModel": "paraformer-zh",
"VadModel": "fsmn-vad",
"PunctuationModel": "ct-punc",
"SpeakerModel": "cam++",
"BatchSizeSeconds": 300,
"BatchSizeThresholdSeconds": 60,
"CommandTimeout": "00:30:00"
}
},
"WhisperLocal": {
"ModelPath": "models\\ggml-base.bin",
"Language": "auto",
"WindowSeconds": 15,
"Diarization": {
"Enabled": true,
"DockerCommand": "docker",
"BaseImage": "python:3.11-slim",
"Image": "meeting-assistant-pyannote:local",
"BuildImage": true,
"ModelsFolder": "%LOCALAPPDATA%\\MeetingAssistant\\Pyannote\\models",
"Model": "pyannote/speaker-diarization-3.1",
"AnnotationSource": "SpeakerDiarization",
"AlignmentMode": "PyannoteTurns",
"TokenEnv": "HF_TOKEN",
"CommandTimeout": "00:30:00"
}
},
"AzureSpeech": {
"Endpoint": "https://germanywestcentral.api.cognitive.microsoft.com/",
"Region": "germanywestcentral",
"Language": "en-US",
"KeyEnv": "AZURE_SPEECH_KEY",
"RecognitionStopTimeout": "00:03:00",
"DiarizeIntermediateResults": true
},
"SpeakerIdentification": {
"MinimumSampleSpeechDuration": "00:00:30",
"MaximumSampleSegmentGap": "00:00:01",
"PyannoteValidation": {
"Enabled": false,
"MinimumSingleSpeakerCoverage": 0.9,
"MinimumMatchingKnownSnippetRatio": 0.5,
"Diarization": {
"Enabled": true,
"DockerCommand": "docker",
"Image": "meeting-assistant-pyannote:local",
"BuildImage": true,
"ModelsFolder": "%LOCALAPPDATA%\\MeetingAssistant\\Pyannote\\models",
"Model": "pyannote/speaker-diarization-3.1",
"TokenEnv": "HF_TOKEN",
"CommandTimeout": "01:00:00"
}
}
},
"Automation": {
"RulesPath": "meeting-rules.local.yaml"
},
"WorkflowRulesEditor": {
"Endpoint": "",
"KeyEnv": "",
"Model": ""
},
"Screenshots": {
"Hotkey": "Ctrl+Alt+S",
"AttachmentsFolder": "Attachments",
"Ocr": {
"Enabled": false,
"Endpoint": "",
"KeyEnv": "LITELLM_API_KEY",
"Model": "",
"Timeout": "00:02:00",
"Prompt": "This screenshot was captured during a meeting..."
}
},
"Agent": {
"Endpoint": "https://litellm.schweigert.cloud",
"KeyEnv": "LITELLM_API_KEY",
"Model": "chatgpt/gpt-5.5",
"EnableThinking": true,
"ReasoningEffort": "Medium",
"ReconnectionAttempts": 2,
"ReconnectionDelay": "00:00:02",
"ContextWindowTokens": 128000,
"MaxOutputTokens": 8192,
"EnableCompaction": true,
"CompactionRemainingRatio": 0.10,
"ResponsesCompactPath": "responses/compact"
},
"Api": {
"PublicBaseUrl": "http://localhost:5090"
}
}
}
funasr streams 16 kHz PCM audio to the configured FunASR WebSocket endpoint. When FunAsr:Backend:Enabled is true, Meeting Assistant manages a local Docker-backed FunASR runtime: on application startup it begins a non-blocking warm-up for the default launch profile and any named launch profile that selects funasr. That warm-up pulls the configured online streaming image, prepares the mounted model and hotword folder, replaces any stale container with the configured name, starts the two-pass WebSocket server mapped to the endpoint port, keeps the container alive after the FunASR startup script backgrounds the server process, and stops it when the app shuts down. Recording can start while the backend is still warming up; captured audio is queued and then streamed once the WebSocket backend is healthy. Docker commands are bounded by CommandTimeout, and backend WebSocket readiness is bounded by StartupTimeout. FunASR response fields such as spk_name, spk, and sentence-level speaker metadata are preserved in transcript segments; missing speaker fields are written as Unknown.
During recording, Meeting Assistant writes the mixed PCM stream to a temporary WAV under TemporaryRecordingsFolder only so the final diarization pass has audio to process. After capture stops and the streaming ASR provider drains the already-captured audio, FunASR Python AutoModel can run with VAD, punctuation, and CAMPPlus (spk_model="cam++") over that temporary WAV. If sentence-level speaker labels are returned, the transcript markdown is rewritten with the final speaker-attributed segments. If final diarization is disabled or returns no segments, the live streaming transcript is kept. Temporary WAV files are deleted after the run completes, and stale temporary recordings from interrupted runs are deleted when the application starts.
whisper-local uses Whisper.NET with a local ggml model and remains available by setting Recording:TranscriptionProvider to whisper-local. The model file is not committed to this repository; place it at the configured ModelPath before using live transcription. When WhisperLocal:Diarization:Enabled is true, the final post-processing pass runs pyannote in Docker over the temporary WAV and reads the Hugging Face token from WhisperLocal:Diarization:Token or TokenEnv (HF_TOKEN by default). AnnotationSource selects pyannote's speaker_diarization or exclusive_speaker_diarization output. AlignmentMode selects whether Meeting Assistant assigns the best-overlap pyannote speaker label to each Whisper segment or rebuilds the finished transcript as pyannote speaker-turn segments with matching Whisper text. When BuildImage is true, Meeting Assistant builds the configured local pyannote Docker image if Docker does not already have it, so Python packages are installed once instead of during every diarization call. The configured ModelsFolder is mounted as the persistent pyannote model cache and stores Hugging Face, torch, and pip cache artifacts so model downloads are reused across runs. Active pyannote runtimes are also warmed up on application start so image setup and model download do not wait for the first diarization request. If pyannote is disabled, has no token, or returns no turns, the live Whisper transcript is kept.
azure-speech streams captured PCM audio to Azure AI Speech using the Speech SDK ConversationTranscriber. Configure AzureSpeech:Endpoint, Region, Language, and either Key or KeyEnv; this repository uses AZURE_SPEECH_KEY by default and does not store the key in appsettings. DiarizeIntermediateResults enables diarization for intermediate and final conversation transcription results. Azure returns generic speaker IDs such as Guest-1 and Guest-2, which Meeting Assistant writes directly into live transcript segments. RecognitionStopTimeout bounds SDK shutdown after the captured audio stream has been closed; the default is 3 minutes so Azure has time to finalize longer speaker-recognition samples without letting diagnostic runs wait indefinitely. Since Azure Speech emits diarized live transcript segments, the Azure pipeline keeps those segments as the finished transcript instead of running pyannote.
Speaker identity matching keeps candidate samples only after a diarized speaker has at least SpeakerIdentification:MinimumSampleSpeechDuration of continuous speech, defaulting to 30 seconds. Adjacent same-speaker segments may be combined when the gap is no larger than MaximumSampleSegmentGap, but a different speaker resets the pending span. SpeakerIdentification:PyannoteValidation is an optional secondary confidence layer. When enabled, pyannote rejects multi-speaker samples and must confirm an Azure-confirmed identity match before Meeting Assistant accepts it. It uses the same Docker-based pyannote runtime shape as local Whisper finalization, reads its Hugging Face token from the nested diarization options, warms that runtime on application start, and defaults the validation command timeout to 1 hour because the local model can need substantial setup time.
ASR backends are exposed downstream as speech recognition pipelines. A pipeline can initialize, wait for readiness, accept audio chunks, emit live transcript segments, and produce a finished transcript after capture completes. The configured implementation is selected from Recording:TranscriptionProvider and registered through DI so recording and diagnostic endpoints do not need to know whether the backend uses FunASR, Whisper.NET, Azure Speech, pyannote, or a later provider.
POST /asr/transcribe-file is a local diagnostic endpoint for the configured ASR backend. Send { "path": "C:\\path\\to\\sample.wav" } with a 16-bit PCM WAV file to stream it through the configured speech recognition pipeline and return the emitted live transcript segments.
For real meeting recordings, Meeting Assistant derives the optional finished-transcript numSpeakers hint from the meeting note attendees frontmatter. The hint is set to the attendee count only when at least two attendees are listed; empty and single-person attendee lists are ignored. For pyannote-backed finalization, that hint is passed to pyannote as num_speakers.
POST /asr/diarize-file streams a local 16-bit PCM WAV through the configured speech recognition pipeline and returns the finished transcript. This is useful for checking the FunASR CAMPPlus path or the local Whisper plus pyannote path without starting a live recording. The request can include an optional speaker-count hint: { "path": "C:\\path\\to\\sample.wav", "numSpeakers": 5 }.
DictationWordsPath is reserved for providers that can accept vocabulary hints. It points to a markdown word list containing unusual project, person, product, or domain terms that should be biased during transcription when the provider supports it.
Meeting notes link to separate transcript, assistant context, and summary markdown artifacts using the configured vault folders.
Hotkey:Abort configures a global discard shortcut. The default is Ctrl+Alt+Z. Aborting an active recording stops capture/transcription, deletes the meeting note, transcript, assistant context, summary file if present, and linked screenshot attachments for that run, and skips automatic summary generation.
Launch profile toggle hotkeys start a meeting when idle. When a different launch profile toggle is pressed during an active meeting, Meeting Assistant keeps the same meeting note, transcript, assistant context, and metadata; drains the current speech recognition pipeline without summarizing; appends a transcript marker; clears run-local speaker mappings; and continues transcription through the target profile. Pressing the same active profile toggle still stops the meeting normally.
On Windows, Meeting Assistant shows a taskbar notification icon. The icon indicates idle, recording, or post-recording summarizing/processing state. Its right-click menu can start a recording for any configured launch profile, stop and transcribe the active recording, cancel and discard the active recording, or switch an active recording to another configured launch profile. A newer active recording takes priority in the icon state even if an older stopped run is still summarizing.
Meeting note, transcript, assistant context, and summary artifacts use frontmatter links so they backlink to each other. Meeting note frontmatter includes start_time when recording starts and end_time when transcription processing finishes. Transcript and summary frontmatter copy the meeting title, start time, and end time from the meeting note; if the meeting has no title, the summary agent can provide one when writing the summary.
Recording:MaxMetadataAttendeeImportCount limits how many attendees Outlook metadata enrichment imports into meeting-note frontmatter. The default is 30; when an appointment has more attendees than that, Meeting Assistant still imports title, agenda, and scheduled end time, but leaves attendees empty because large invites are usually presentation-style meetings.
Assistant context notes keep their artifact links in frontmatter and expose a lifecycle state: collecting metadata, transcribing, speaker recognition, summarizing, finished, or error. Meeting Assistant updates this state as metadata lookup, recording, final speaker recognition, and summary generation progress.
ProjectsFolder contains project knowledge. Each direct subfolder is treated as one project, and a meeting binds projects by listing those subfolder names in the meeting note projects frontmatter.
Automation:RulesPath points to an optional local YAML rules file. The default meeting-rules.local.yaml is ignored by git. Rules can trigger on meeting creation, assistant-context state transitions, or identified speakers; conditions are evaluated with NCalc-style expressions and step values can use Razor syntax against Model.Meeting, Model.Event, and Model.Speaker. The initial supported steps are add_attendee, remove_attendee, set_property, add_context, and add_project.
See docs/meeting-workflow-engine.md for the detailed YAML format, supported variables, examples, and extension notes.
POST /diagnostics/workflow/reload reloads application configuration so workflow automation settings such as Automation:RulesPath can be changed without restarting the application. A meeting run that already captured its options keeps using those options.
Screenshots:Hotkey configures a global hotkey that captures the currently active window during an active meeting. Screenshots are written under Screenshots:AttachmentsFolder, which defaults to an Attachments folder beside the assistant context note, and each capture appends a timestamped markdown image link to the assistant context. Launch profiles can override the screenshot hotkey with LaunchProfiles:{profile}:Screenshots:Hotkey; only explicitly configured profile hotkeys are registered, so profile-specific hotkeys do not silently inherit and collide with the default profile.
Screenshots:Ocr optionally enables vision extraction for screenshots. When Enabled is true, Meeting Assistant sends the screenshot and prompt to an OpenAI-compatible Responses endpoint and replaces the screenshot OCR placeholder with the model output. Blank Endpoint or Model values fall back to the summary Agent endpoint and model, which keeps local OCR on the same LiteLLM backend as summarization. Key or KeyEnv can be set specifically for OCR; otherwise the summary agent key configuration is used. The default prompt asks the model to say whether visible participants are clearly the exact meeting participants or only a partial result, and to return pixel crop coordinates when it can confidently isolate only the presentation or shared-screen content; valid crop boxes are saved as *-cropped.png beside the original screenshot and linked before the OCR block. Automatic summarization waits for pending screenshot OCR work to complete or hit Timeout before the assistant context moves to summarizing.
Agent configures the Microsoft Agent Framework summary pipeline. Key may be set directly for local testing, but KeyEnv is preferred; by default the API key is read from LITELLM_API_KEY. After transcription has fully finished, Meeting Assistant automatically runs the summary pipeline for the meeting. The summary agent writes the full markdown summary through write_summary and must provide a required oneliner value, which is stored in summary frontmatter and must not contain line breaks. The summary agent can add and remove meeting-note attendees when transcript or OCR evidence is clear; partial participant evidence from screenshots should not be used to remove invitees by itself. It can also override a transcript speaker label with a named speaker only when the evidence is very certain, such as a user note, OCR at a correlating meeting timestamp, or a clear context cue. If that named speaker already exists in the transcript, override_speaker refuses by default; the agent must pass merge=true only when it is certain both labels are the same identity. When it is certain that a speaker identity was wrongfully matched, it can delete that identity, relabeling transcript speaker labels to Removed-<n> and removing the local speaker identity database row before future matching. Final speaker identity learning and candidate updates run after the summary pipeline finishes so they use the summary-refined attendee list and any recorded speaker identity changes. ReconnectionAttempts and ReconnectionDelay control retries for transient model endpoint failures such as LiteLLM token refresh or 5xx responses. If summary generation still fails, Meeting Assistant overwrites the configured summary note with a markdown failure report containing the exception details and linked meeting artifacts.
ContextWindowTokens, MaxOutputTokens, EnableCompaction, and CompactionRemainingRatio configure summary-agent context monitoring. Meeting Assistant estimates the outgoing Responses payload size, logs the estimated token count and remaining context, and compacts the conversation when only the configured remaining ratio is left. It first attempts POST /v1/{ResponsesCompactPath} on the configured OpenAI-compatible endpoint. If that endpoint is unavailable or returns invalid data, it falls back to Microsoft Agent Framework compaction: collapse old tool results, summarize older message groups, preserve only the last four user turns if needed, and finally drop oldest groups until the request is back under budget.
The summary endpoints are:
POST /meetings/current/summary/run
POST /meetings/summary/retry
GET /meetings/summary/retry?summaryPath=...
POST /meetings/summary/retry accepts { "summaryPath": "20260519-124110-summary.md" } or an absolute path under the configured summaries folder. It resolves the linked meeting note from frontmatter and reruns the summary pipeline, overwriting the existing summary file with either the new summary or a new failure report.
Failure summary notes include a clickable retry link using Api:PublicBaseUrl, for example http://localhost:5090/meetings/summary/retry?summaryPath=20260519-124110-summary.md. The GET endpoint exists for Obsidian/browser clicks and performs the same overwrite behavior as the POST endpoint. Meeting note bodies stay reserved for user-authored notes.
The summary agent currently has meeting-scoped tools to read the transcript, full meeting note including frontmatter, assistant context, user notes, glossary, and project files. Every read tool that returns file-like content can read the whole content or a clamped inclusive 1-based line range with from and to, so the agent can inspect large transcripts incrementally. The assistant context note is the agent's notebook and user-visible context window; the agent can write internal notes, discovered context, missing tool requests, suggested improvements, or follow-up task ideas there using append-by-default, explicit whole-file replacement, replace, and insert modes. Project lookup and search tools are constrained to the projects listed in the meeting note frontmatter:
When assistant context contains cropped screenshot links produced by OCR, the summary instructions tell the agent to include only the most relevant cropped screenshots in the summary and markdown-link them near the related summary text.
read_meetingnotelist_projectslist_projectfilesread_projectfilewrite_projectfilesearchwrite_contextadd_attendeeremove_attendeeoverride_speakerdelete_identity
search accepts ripgrep syntax and returns matches as filename:line text. If one project is searched, paths are relative to that project; if multiple projects are searched, paths include the project folder name.
write_projectfile writes inside an existing project folder. With no line arguments it appends or creates the file. With replace_file: true it replaces the whole file. With from and to it replaces that inclusive 1-based line range. With insert it inserts content at that 1-based line position.
Spec Workflow
This repository is OpenSpec-driven. Before changing behavior, read:
README.mdopenspec/config.yaml- existing specs in
openspec/specs - active change specs in
openspec/changes/*/specs
The initial active change is:
openspec/changes/define-meeting-assistant-v1
Validate it with:
openspec validate define-meeting-assistant-v1 --strict