Warm up pyannote and FunASR runtimes

2026-05-28 13:05:27 +02:00
parent 7ff93b73b3
commit c46c587e65
14 changed files with 696 additions and 20 deletions
@@ -0,0 +1,13 @@
+# Change: Warm up pyannote on startup
+
+## Why
+Local pyannote validation can spend significant time building the Docker image and downloading Hugging Face model artifacts. If that work happens on the first speaker-validation request, the request can time out before validation starts.
+
+## What Changes
+- Increase the local pyannote validation command timeout default.
+- Add a startup warm-up path that prepares enabled pyannote diarization runtimes and downloads the configured model into the persistent cache.
+- Keep startup non-blocking and keep diarization requests bounded by configured timeouts.
+
+## Impact
+- First speaker validation is less likely to fail due to one-time model setup.
+- Startup logs can surface pyannote setup failures before the first meeting needs validation.
@@ -0,0 +1,94 @@
+## MODIFIED Requirements
+### Requirement: Speech recognition pipelines are streamable and configurable
+Meeting Assistant SHALL route audio through a provider-independent speech recognition pipeline contract.
+
+The configured pipeline SHALL be selected through DI from the ASR backend configuration and SHALL expose operations to initialize, wait for readiness, write captured audio chunks, read live transcript segments, and read the finished transcript after capture completes.
+
+Reading the finished transcript SHALL accept optional pipeline options, including a requested speaker count hint.
+
+For real meeting recordings, Meeting Assistant SHALL derive the requested speaker count hint from the meeting note `attendees` frontmatter when at least two attendees are listed.
+
+The configured pipeline SHALL receive audio chunks as they are captured rather than requiring the recording coordinator to finish a complete meeting audio file first.
+
+On application startup, Meeting Assistant SHALL start non-blocking warm-up for the default launch profile's speech recognition pipeline and for any named launch profile that selects the managed FunASR provider.
+
+#### Scenario: Streaming provider receives live audio
+- **WHEN** recording mode is active and the combined audio source emits a chunk
+- **THEN** Meeting Assistant can pass that chunk to the configured speech recognition pipeline before the audio source completes
+
+#### Scenario: Pipeline readiness is separated from recording capture
+- **WHEN** recording mode starts before the configured speech recognition backend is ready
+- **THEN** Meeting Assistant starts audio capture immediately and lets the pipeline finish readiness before it emits live transcript output
+
+#### Scenario: Pipeline is changed by configuration
+- **WHEN** the configured ASR backend setting is changed
+- **THEN** Meeting Assistant selects the configured speech recognition pipeline without changing recording control or transcript persistence code
+
+#### Scenario: ASR backend is tested with a WAV file
+- **WHEN** the user submits a local PCM WAV file to the ASR diagnostic endpoint
+- **THEN** Meeting Assistant streams that file through the configured speech recognition pipeline and returns the emitted live transcript segments
+
+#### Scenario: ASR diagnostic backend is unavailable
+- **WHEN** the configured speech recognition pipeline fails while processing the diagnostic WAV file
+- **THEN** Meeting Assistant returns a diagnostic backend failure response instead of hiding the pipeline error
+
+#### Scenario: Final diarization backend is tested with a WAV file
+- **WHEN** the user submits a local PCM WAV file to the final diarization diagnostic endpoint
+- **THEN** Meeting Assistant streams the file through the configured speech recognition pipeline and returns the finished transcript segments
+
+#### Scenario: Final diarization receives a speaker count hint
+- **WHEN** the user submits a local PCM WAV file and `numSpeakers` to the final diarization diagnostic endpoint
+- **THEN** Meeting Assistant passes the requested speaker count to the configured speech recognition pipeline as a finished-transcript option
+
+#### Scenario: Meeting attendee count provides final speaker hint
+- **WHEN** a meeting note has two or more entries in `attendees`
+- **THEN** Meeting Assistant passes that attendee count to the configured speech recognition pipeline as the finished-transcript speaker count hint
+
+#### Scenario: Empty or single-attendee meeting note has no final speaker hint
+- **WHEN** a meeting note has zero or one entry in `attendees`
+- **THEN** Meeting Assistant leaves the finished-transcript speaker count hint unset
+
+#### Scenario: Launch profile FunASR pipeline warms on startup
+- **GIVEN** a named launch profile selects `funasr`
+- **WHEN** Meeting Assistant starts
+- **THEN** it begins warming that profile's speech recognition pipeline without blocking startup
+
+#### Scenario: Non-default non-FunASR profile is not warmed
+- **GIVEN** a named launch profile selects a provider other than `funasr`
+- **WHEN** Meeting Assistant starts
+- **THEN** it does not create a startup warm-up pipeline for that named profile
+
+### Requirement: Speaker identity matching can use pyannote secondary validation
+Meeting Assistant SHALL support an optional configurable pyannote secondary validation layer for speaker identity matching.
+
+When pyannote secondary validation is enabled, Meeting Assistant SHALL verify candidate speaker samples before retaining them for identity matching. Samples that pyannote reports as containing multiple speakers SHALL be rejected.
+
+When pyannote secondary validation is enabled and the primary identity matcher confirms a speaker, Meeting Assistant SHALL run a second validation pass through pyannote before accepting the match.
+
+If pyannote secondary validation cannot confirm that the unknown live sample and matched identity samples belong to one speaker, Meeting Assistant SHALL reject the match.
+
+When pyannote secondary validation is disabled, Meeting Assistant SHALL preserve the primary identity matching behavior.
+
+When pyannote secondary validation is enabled, Meeting Assistant SHALL start a non-blocking startup warm-up that builds or verifies the configured pyannote runtime image and downloads the configured model into the persistent model cache before the first validation request when possible.
+
+#### Scenario: Multi-speaker sample is rejected
+- **GIVEN** pyannote secondary validation is enabled
+- **WHEN** pyannote reports multiple speakers in a candidate sample
+- **THEN** Meeting Assistant does not retain that sample for identity matching
+
+#### Scenario: Pyannote rejects primary match
+- **GIVEN** pyannote secondary validation is enabled
+- **AND** the primary identity matcher confirms `Guest03` as `Chris`
+- **WHEN** pyannote reports that the unknown `Guest03` sample and known `Chris` samples contain different speakers
+- **THEN** Meeting Assistant rejects the match
+
+#### Scenario: Disabled pyannote validation preserves primary match
+- **GIVEN** pyannote secondary validation is disabled
+- **WHEN** the primary identity matcher confirms `Guest03` as `Chris`
+- **THEN** Meeting Assistant accepts the match without running pyannote secondary validation
+
+#### Scenario: Pyannote validation warms up on startup
+- **GIVEN** pyannote secondary validation is enabled
+- **WHEN** Meeting Assistant starts
+- **THEN** it begins preparing the configured pyannote runtime image and model cache without waiting for the first validation request
+- **AND** application startup is not blocked by the warm-up task
@@ -0,0 +1,9 @@
+## Implementation
+- [x] Add behavior tests for pyannote warm-up command generation.
+- [x] Add behavior tests for non-blocking startup warm-up of enabled pyannote validation.
+- [x] Implement pyannote warm-up in the finalizer/runtime.
+- [x] Register a startup hosted service for enabled pyannote runtime warm-up.
+- [x] Warm named launch-profile FunASR pipelines on startup.
+- [x] Increase local pyannote validation timeout in code/config/docs.
+- [x] Run focused tests.
+- [x] Validate OpenSpec change.