meeting-assistant/openspec/changes/archive/2026-05-19-define-meeting-assistant-v1/specs/meeting-transcription/spec.md

ADDED Requirements

Requirement: Meeting Assistant transcribes meetings with speaker attribution

Meeting Assistant SHALL transcribe meeting audio and associate transcript segments with speakers when speaker attribution is available.

The transcription contract SHALL remain independent of the meeting source so that local audio, in-person meetings, and future meeting-platform integrations can use the same downstream processing.

Scenario: Speaker-attributed transcript is produced

• WHEN meeting audio is transcribed and speaker attribution is available
• THEN the transcript contains ordered speaker-attributed segments

Scenario: Speaker attribution is unavailable

• WHEN meeting audio is transcribed but speaker attribution is unavailable
• THEN Meeting Assistant keeps the transcript and marks speaker identity as unknown rather than discarding the meeting

Requirement: Speech recognition pipelines are streamable and configurable

Meeting Assistant SHALL route audio through a provider-independent speech recognition pipeline contract.

The configured pipeline SHALL be selected through DI from the ASR backend configuration and SHALL expose operations to initialize, wait for readiness, write captured audio chunks, read live transcript segments, and read the finished transcript after capture completes.

Reading the finished transcript SHALL accept optional pipeline options, including a requested speaker count hint.

For real meeting recordings, Meeting Assistant SHALL derive the requested speaker count hint from the meeting note attendees frontmatter when at least two attendees are listed.

The configured pipeline SHALL receive audio chunks as they are captured rather than requiring the recording coordinator to finish a complete meeting audio file first.

Scenario: Streaming provider receives live audio

• WHEN recording mode is active and the combined audio source emits a chunk
• THEN Meeting Assistant can pass that chunk to the configured speech recognition pipeline before the audio source completes

Scenario: Pipeline readiness is separated from recording capture

• WHEN recording mode starts before the configured speech recognition backend is ready
• THEN Meeting Assistant starts audio capture immediately and lets the pipeline finish readiness before it emits live transcript output

Scenario: Pipeline is changed by configuration

• WHEN the configured ASR backend setting is changed
• THEN Meeting Assistant selects the configured speech recognition pipeline without changing recording control or transcript persistence code

Scenario: ASR backend is tested with a WAV file

• WHEN the user submits a local PCM WAV file to the ASR diagnostic endpoint
• THEN Meeting Assistant streams that file through the configured speech recognition pipeline and returns the emitted live transcript segments

Scenario: ASR diagnostic backend is unavailable

• WHEN the configured speech recognition pipeline fails while processing the diagnostic WAV file
• THEN Meeting Assistant returns a diagnostic backend failure response instead of hiding the pipeline error

Scenario: Final diarization backend is tested with a WAV file

• WHEN the user submits a local PCM WAV file to the final diarization diagnostic endpoint
• THEN Meeting Assistant streams the file through the configured speech recognition pipeline and returns the finished transcript segments

Scenario: Final diarization receives a speaker count hint

• WHEN the user submits a local PCM WAV file and numSpeakers to the final diarization diagnostic endpoint
• THEN Meeting Assistant passes the requested speaker count to the configured speech recognition pipeline as a finished-transcript option

Scenario: Meeting attendee count provides final speaker hint

• WHEN a meeting note has two or more entries in attendees
• THEN Meeting Assistant passes that attendee count to the configured speech recognition pipeline as the finished-transcript speaker count hint

Scenario: Empty or single-attendee meeting note has no final speaker hint

• WHEN a meeting note has zero or one entry in attendees
• THEN Meeting Assistant leaves the finished-transcript speaker count hint unset

Requirement: Version 1 keeps local Whisper transcription available

Meeting Assistant SHALL provide a local Whisper speech recognition pipeline as a fallback ASR backend for version 1.

The local Whisper adapter SHALL use configurable model path, language, and streaming window settings.

When local Whisper is the configured speech recognition pipeline, Meeting Assistant SHALL be able to run a final pyannote diarization pass over the temporary mixed audio and either assign pyannote speaker turns back to live Whisper transcript segments by timestamp overlap or build finished transcript segments from pyannote speaker turns.

The pyannote finalization backend SHALL allow configuration to use either pyannote's normal speaker_diarization annotation or pyannote's exclusive_speaker_diarization annotation.

When a speaker count hint is provided through the speech recognition pipeline options, the pyannote finalization backend SHALL pass it to pyannote as num_speakers.

The pyannote finalization backend SHALL use a reusable runtime image for Python dependencies and a persistent configured cache folder for Hugging Face models, torch artifacts, and pip artifacts so repeated finalization runs do not redownload the diarization model or reinstall the full Python dependency stack.

Scenario: Local Whisper model is configured

• WHEN the configured provider is whisper-local
• THEN Meeting Assistant uses the configured local Whisper model path to transcribe incoming audio windows

Scenario: Local Whisper transcript is finalized with pyannote speaker turns

• WHEN the configured provider is whisper-local, live Whisper transcript segments exist, and pyannote returns speaker turns for the temporary recording
• THEN Meeting Assistant rewrites the transcript with the original Whisper text and the best-overlap pyannote speaker label for each segment

Scenario: Local Whisper transcript is segmented by pyannote speaker turns

• WHEN the configured provider is whisper-local, pyannote turn alignment is configured, live Whisper transcript segments exist, and pyannote returns speaker turns for the temporary recording
• THEN Meeting Assistant rewrites the transcript as ordered pyannote speaker-turn segments with matching Whisper text assigned to each turn

Scenario: Local Whisper uses exclusive pyannote diarization

• WHEN the configured provider is whisper-local and exclusive pyannote annotation is configured
• THEN Meeting Assistant reads speaker turns from pyannote's exclusive_speaker_diarization output

Scenario: Local Whisper pyannote runtime cache is reused

• WHEN local Whisper finalization runs pyannote more than once
• THEN Meeting Assistant uses the configured reusable pyannote runtime image and persistent pyannote model cache instead of treating each run as a clean download environment

Scenario: Local Whisper pyannote diarization is unavailable

• WHEN the configured provider is whisper-local but pyannote is disabled, has no configured token, or returns no speaker turns
• THEN Meeting Assistant keeps the live Whisper transcript instead of deleting it

Requirement: Azure Speech can provide streaming transcription

Meeting Assistant SHALL provide an Azure Speech speech recognition pipeline that streams captured PCM audio to Azure AI Speech through the Azure Speech SDK conversation transcription API.

The Azure Speech adapter SHALL use configurable endpoint, region, language, key, and key environment variable settings.

The Azure Speech adapter SHALL support configurable diarization of intermediate conversation transcription results.

The Azure Speech adapter SHALL bound recognizer shutdown after the input audio stream is closed.

When Azure Speech is the configured speech recognition pipeline, Meeting Assistant SHALL emit live transcript segments from Azure conversation transcription events with Azure speaker IDs when available.

When Azure Speech is the configured speech recognition pipeline, Meeting Assistant SHALL use the live Azure conversation transcription segments as the finished transcript without running pyannote finalization.

Scenario: Azure Speech pipeline is configured

• WHEN the configured provider is azure-speech
• THEN Meeting Assistant streams captured PCM chunks to Azure AI Speech conversation transcription through the configured speech recognition pipeline without changing recording control or transcript persistence code

Scenario: Azure Speech returns speaker IDs

• WHEN Azure Speech conversation transcription returns transcript text with speaker identity
• THEN Meeting Assistant emits ordered transcript segments with those Azure speaker identities

Scenario: Azure Speech key is resolved from environment

• WHEN Azure Speech is configured with KeyEnv
• THEN Meeting Assistant reads the Azure Speech key from that environment variable

Scenario: Azure Speech stream shutdown is bounded

• WHEN the captured audio stream has ended
• THEN Meeting Assistant stops Azure Speech continuous recognition within the configured stop timeout instead of waiting indefinitely

Scenario: Azure Speech transcript is finalized from live conversation segments

• WHEN the configured provider is azure-speech and live Azure transcript segments exist
• THEN Meeting Assistant uses the live Azure conversation transcript segments as the finished transcript

Requirement: FunASR can provide speaker-attributed streaming transcription

Meeting Assistant SHALL provide a FunASR speech recognition pipeline that streams PCM audio to a configured FunASR WebSocket endpoint.

The FunASR provider SHALL preserve speaker attribution from FunASR response fields when they are present and SHALL fall back to Unknown when no speaker field is returned.

Meeting Assistant SHALL write the mixed recording audio only as a temporary processing artifact while streaming transcription is active. After capture stops and the streaming provider has drained already-captured audio, Meeting Assistant SHALL be able to run a final FunASR Python AutoModel pass with VAD, punctuation, and CAMPPlus speaker modeling over the temporary audio. When that final pass returns sentence-level speaker labels, Meeting Assistant SHALL rewrite the transcript markdown with the final speaker-attributed sentence segments.

Meeting Assistant SHALL delete temporary recording audio after the run completes, whether final diarization succeeds, fails, or produces no speaker output. On application startup, Meeting Assistant SHALL delete stale temporary recording audio left behind by previous interrupted runs.

When configured, Meeting Assistant SHALL manage a local FunASR backend lifecycle for the FunASR provider by starting a non-blocking backend warm-up when the application starts, installing the configured streaming runtime image, preparing the mounted model and hotword folder, starting the backend, keeping the container alive for the backgrounded FunASR server process, and stopping it when the application stops. Recording SHALL be able to start while the backend is still warming up; captured audio SHALL be queued and streamed once the backend is healthy. Backend installation, startup commands, and WebSocket readiness SHALL be bounded by configurable timeouts so startup failures are diagnosable.

Scenario: FunASR pipeline is configured

• WHEN the configured provider is funasr
• THEN Meeting Assistant streams captured PCM chunks to the configured FunASR WebSocket endpoint through the configured speech recognition pipeline without changing recording control or transcript persistence code

Scenario: FunASR returns speaker fields

• WHEN FunASR returns transcript text with speaker identity fields
• THEN Meeting Assistant emits ordered transcript segments with those speaker identities

Scenario: Final FunASR diarization rewrites transcript

• WHEN recording stops after mixed audio was temporarily stored and the final FunASR diarization pass returns sentence-level speaker labels
• THEN Meeting Assistant rewrites the transcript markdown with the final speaker-attributed sentence segments

Scenario: Final FunASR diarization has no speaker output

• WHEN recording stops but the final diarization pass is disabled or returns no sentence-level speaker labels
• THEN Meeting Assistant keeps the live streaming transcript instead of deleting it

Scenario: Temporary recording audio is deleted after completion

• WHEN a recording run completes after using temporary audio for final processing
• THEN Meeting Assistant deletes the temporary audio file from disk

Scenario: Stale temporary recordings are deleted on startup

• WHEN Meeting Assistant starts and stale temporary recording files exist from a previous interrupted run
• THEN Meeting Assistant deletes those stale temporary recording files before accepting new recordings

Scenario: Managed FunASR backend starts with the application

• WHEN the configured provider is funasr and managed backend startup is enabled
• THEN Meeting Assistant begins installing the configured FunASR backend image and starting the two-pass streaming backend without blocking recording startup

Scenario: Recording starts while managed FunASR backend is warming up

• WHEN recording starts before the managed FunASR backend is ready
• THEN Meeting Assistant starts audio capture immediately and queues captured audio until the backend WebSocket is healthy

Scenario: Managed FunASR backend installation stalls

• WHEN a managed FunASR backend command exceeds its configured timeout
• THEN Meeting Assistant reports a backend startup timeout instead of waiting indefinitely

Scenario: Managed FunASR backend WebSocket is not healthy

• WHEN the managed backend container has started but the FunASR WebSocket endpoint is not accepting sessions
• THEN Meeting Assistant continues waiting for backend readiness until the configured startup timeout is reached

Scenario: Managed FunASR backend stops with the application

• WHEN Meeting Assistant stops after starting a managed FunASR backend
• THEN Meeting Assistant stops the managed backend process or container