meeting-assistant/openspec/changes/archive/2026-06-26-azure-speech-offline-resilience/specs/meeting-transcription/spec.md

MODIFIED Requirements

Requirement: Azure Speech can provide streaming transcription

Meeting Assistant SHALL provide an Azure Speech speech recognition pipeline that streams captured PCM audio to Azure AI Speech through the Azure Speech SDK conversation transcription API.

The Azure Speech adapter SHALL use configurable endpoint, region, language, key, and key environment variable settings.

The Azure Speech adapter SHALL support configurable diarization of intermediate conversation transcription results.

The Azure Speech adapter SHALL bound recognizer shutdown after the input audio stream is closed.

When Azure Speech is the configured speech recognition pipeline, Meeting Assistant SHALL emit live transcript segments from Azure conversation transcription events with Azure speaker IDs when available.

When Azure Speech is the configured speech recognition pipeline, Meeting Assistant SHALL use the live Azure conversation transcription segments as the finished transcript without running pyannote finalization.

When Azure Speech reports a transient connectivity interruption, Meeting Assistant SHALL keep accepting captured audio into the active in-process pipeline buffer.

When Azure Speech is reconnecting, Meeting Assistant SHALL write a transcript marker in the form <Reconnecting... n/m>.

When Azure Speech emits the next real transcript segment after reconnecting, Meeting Assistant SHALL replace the latest reconnect marker with that transcript segment.

When Azure Speech cannot reconnect after the configured immediate retry attempts, Meeting Assistant SHALL write a transcript marker explaining that Azure Speech is disconnected, recording can continue, and transcription/summarization will continue after Azure reconnects.

After Azure Speech reconnects through a new SDK session, Meeting Assistant SHALL clear live speaker-label assumptions for future Azure speaker labels.

When Azure Speech is still unavailable after recording stops and transcription cannot drain before the configured stop-processing timeout, Meeting Assistant SHALL persist the stopped meeting as a durable transcription backlog item that references the completed mixed WAV and meeting artifacts.

When a stopped meeting is persisted to the durable transcription backlog, Meeting Assistant SHALL release the active recording slot so another meeting can be recorded while the stopped meeting waits for Azure Speech to become available.

When Azure Speech becomes available again, Meeting Assistant SHALL retry durable backlog items, append the recovered transcript from the recorded WAV without replacing the existing transcript body, run the normal post-transcription meeting completion and summary flow, and remove the backlog item after successful completion.

Before appending recovered lines, Meeting Assistant SHALL insert a visible recovery marker explaining that the following section was recovered by replaying the entire recording and may duplicate earlier transcript passages. The marker SHALL tell the summarizer to treat repeated passages as the same discussion. The marker and appended lines SHALL be available to the summary agent together with the preserved earlier transcript. Existing transcript text, speaker labels, and user edits SHALL remain unchanged.

An offline replay that returns no speech segments with non-blank text SHALL be treated as an unsuccessful attempt. This includes empty results, blank speech segments, and results containing only status markers. Meeting Assistant SHALL preserve the existing transcript and meeting artifacts unchanged, SHALL skip meeting completion and summarization, and SHALL retain the durable backlog item and its WAV for a later retry.

When Meeting Assistant starts, it SHALL preserve WAV files that are referenced by durable transcription backlog items instead of deleting them as stale temporary recordings.

Scenario: Azure Speech pipeline is configured

• WHEN the configured provider is azure-speech
• THEN Meeting Assistant streams captured PCM chunks to Azure AI Speech conversation transcription through the configured speech recognition pipeline without changing recording control or transcript persistence code

Scenario: Azure Speech returns speaker IDs

• WHEN Azure Speech conversation transcription returns transcript text with speaker identity
• THEN Meeting Assistant emits ordered transcript segments with those Azure speaker identities

Scenario: Azure Speech key is resolved from environment

• WHEN Azure Speech is configured with KeyEnv
• THEN Meeting Assistant reads the Azure Speech key from that environment variable

Scenario: Azure Speech stream shutdown is bounded

• WHEN the captured audio stream has ended
• THEN Meeting Assistant stops Azure Speech continuous recognition within the configured stop timeout instead of waiting indefinitely

Scenario: Azure Speech transcript is finalized from live conversation segments

• WHEN the configured provider is azure-speech and live Azure transcript segments exist
• THEN Meeting Assistant uses the live Azure conversation transcript segments as the finished transcript

Scenario: Azure reconnect marker is replaced by next transcript line

• GIVEN Azure Speech reports a transient connectivity interruption
• WHEN Meeting Assistant writes <Reconnecting... 1/5> and Azure later emits a transcript segment
• THEN Meeting Assistant replaces the reconnect marker with the transcript segment
• AND keeps later transcript lines intact

Scenario: Azure disconnected marker keeps meeting recording alive

• GIVEN Azure Speech cannot reconnect after the configured immediate retry attempts
• WHEN meeting audio continues to be captured
• THEN Meeting Assistant writes a transcript marker explaining that Azure Speech is disconnected
• AND keeps buffering captured audio in the active process for later transcription

Scenario: Stopped Azure meeting is queued durably while offline

• GIVEN Azure Speech cannot finish transcription before the recording stop-processing timeout
• WHEN the user stops the meeting
• THEN Meeting Assistant persists a durable backlog item for the stopped meeting
• AND keeps the completed mixed WAV referenced by that backlog item
• AND returns to an idle recording state so another meeting can start

Scenario: Durable Azure backlog resumes after connectivity returns

• GIVEN a stopped Azure meeting exists in the durable transcription backlog
• WHEN Azure Speech can transcribe the recorded WAV
• THEN Meeting Assistant preserves the existing transcript body and appends a recovery marker followed by the transcript from the recorded WAV
• AND the marker explains the recovery and possible duplicates to the summary agent
• AND runs meeting completion and summarization
• AND removes the durable backlog item and its temporary WAV after successful completion

Scenario: Empty offline replay preserves the live transcript and recovery files

• GIVEN a stopped Azure meeting has an existing live transcript and a durable backlog item referencing its WAV
• WHEN the offline replay returns no transcript segments
• THEN Meeting Assistant leaves the transcript and meeting artifacts unchanged
• AND does not run meeting completion or summarization
• AND retains the backlog item and WAV for the next retry
• WHEN a later retry produces a transcript from the WAV
• THEN Meeting Assistant appends the recovery marker and recovered transcript without replacing earlier content and completes the meeting normally
• AND removes the backlog item and WAV only after that successful completion

Scenario: Offline status markers or blank speech do not count as successful transcription

• GIVEN a stopped Azure meeting has an existing live transcript and a durable backlog item referencing its WAV
• WHEN the offline replay returns only status markers or speech segments with blank text
• THEN Meeting Assistant preserves the existing transcript and meeting artifacts
• AND skips meeting completion and summarization
• AND retains the backlog item and WAV for another retry