fix: unify pyannote validation toggle
PR and Push Build/Test / build-and-test (push) Successful in 12m8s

This commit is contained in:
2026-09-02 13:46:51 +02:00
parent 70ffaa7357
commit b1365b202d
14 changed files with 329 additions and 81 deletions
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-09-02
@@ -0,0 +1,52 @@
## Context
Speaker identity validation reuses the general pyannote diarization option type. That type includes an `Enabled` property for the optional Whisper finalization feature, while speaker validation already has its own outer `Enabled` property. The checked-in and deployed configuration currently sets those properties to opposite values. The validator observes the outer value, calls the finalizer, and receives an empty result because the finalizer observes the nested value, causing every sample to be rejected before Azure matching.
## Goals / Non-Goals
**Goals:**
- Expose one authoritative enable switch for speaker identity pyannote validation.
- Keep Whisper diarization independently configurable.
- Make validation, matching, and startup warm-up observe the same speaker-validation state.
- Preserve the existing pyannote runtime settings and validation thresholds.
**Non-Goals:**
- Change speaker sample duration or gap thresholds.
- Change Azure Speech matching semantics.
- Enable pyannote validation without the required Docker runtime and Hugging Face token.
## Decisions
### Give speaker validation a runtime-options type without an enable flag
Extract the shared pyannote runtime properties into `PyannoteRuntimeOptions`. Keep `PyannoteDiarizationOptions` as the Whisper-facing derived type that adds `Enabled`, and type `SpeakerIdentityPyannoteValidationOptions.Diarization` as `PyannoteRuntimeOptions`.
This makes contradictory speaker-validation state unrepresentable through the typed configuration model. The alternative—retaining the nested flag and overriding it at runtime—would leave a misleading configuration surface and permit the same mistake to recur.
### Gate once at the feature boundary
`PyannoteSpeakerIdentityMatchValidator` bypasses pyannote when the application-level outer validation switch is off. When it is on, the validator calls an enabled-runtime finalization path that does not evaluate another toggle. The warm-up service selects the same application-level speaker-validation runtime instead of resolving speaker validation independently for each launch profile.
Whisper finalization continues checking `WhisperLocal:Diarization:Enabled` before it invokes the shared runtime path.
### Migrate configuration by removing the nested key
Remove `SpeakerIdentification:PyannoteValidation:Diarization:Enabled` from the canonical configuration and document `SpeakerIdentification:PyannoteValidation:Enabled` as the sole switch. Existing unknown nested keys are ignored by .NET configuration binding after the typed property is removed; deployments should republish from the canonical configuration to remove the stale key.
## Risks / Trade-offs
- [Enabling the outer switch now really invokes Docker/pyannote] → Preserve the existing token/runtime error reporting and document that disabling the outer switch is the supported bypass.
- [The shared options refactor touches Whisper code] → Preserve its independent `Enabled` property on the derived type and run focused Whisper/pyannote tests plus the full suite.
- [Old deployed appsettings may retain the removed nested key] → The binder ignores it, so runtime behavior remains controlled by the outer switch; republishing removes it from the canonical deployed file.
## Migration Plan
1. Publish the updated application configuration with the nested key removed.
2. Keep `SpeakerIdentification:PyannoteValidation:Enabled` on only where Docker, the pyannote model, and `HF_TOKEN` are available.
3. Roll back by deploying the prior build and configuration together if necessary.
## Open Questions
None.
@@ -0,0 +1,27 @@
## Why
Speaker identity matching currently exposes two independent pyannote validation switches. Enabling the outer validation switch while disabling the nested diarization switch silently rejects every otherwise usable speaker sample, so the default configuration can prevent all automatic identity matches.
## What Changes
- Make `SpeakerIdentification:PyannoteValidation:Enabled` the only switch controlling secondary pyannote validation.
- Remove the nested `SpeakerIdentification:PyannoteValidation:Diarization:Enabled` configuration setting.
- Ensure enabling validation also enables its pyannote runtime and startup warm-up, while disabling validation preserves primary Azure identity matching without invoking pyannote.
- Add regression coverage and configuration documentation for both toggle states.
## Capabilities
### New Capabilities
None.
### Modified Capabilities
- `meeting-transcription`: Clarify that secondary pyannote validation has one authoritative enable setting and cannot be partially enabled.
## Impact
- Speaker identification options and pyannote runtime invocation.
- Pyannote startup warm-up selection.
- Checked-in application configuration and configuration reference.
- Speaker validation and warm-up behavior tests.
@@ -0,0 +1,60 @@
## MODIFIED Requirements
### Requirement: Speaker identity matching can use pyannote secondary validation
Meeting Assistant SHALL support an optional configurable pyannote secondary validation layer for speaker identity matching.
`SpeakerIdentification:PyannoteValidation:Enabled` SHALL be the only enable setting for speaker identity pyannote validation. The nested pyannote runtime settings SHALL NOT expose or honor a second enable setting.
Speaker identity pyannote validation SHALL use the application-level setting consistently for matching and startup warm-up. Launch profiles SHALL NOT override this validation setting or its runtime configuration.
When pyannote secondary validation is enabled, Meeting Assistant SHALL verify candidate speaker samples before retaining them for identity matching. Samples that pyannote reports as containing multiple speakers SHALL be rejected.
When pyannote secondary validation is enabled, Meeting Assistant SHALL verify speaker-override samples before retaining them on speaker identities. Speaker overrides whose source samples are rejected SHALL NOT create a new speaker identity from that rejected sample.
When pyannote secondary validation is enabled and the primary identity matcher confirms a speaker, Meeting Assistant SHALL run a second validation pass through pyannote before accepting the match.
If pyannote secondary validation cannot confirm that the unknown live sample and matched identity samples belong to one speaker, Meeting Assistant SHALL reject the match.
When pyannote secondary validation is disabled, Meeting Assistant SHALL preserve the primary identity matching behavior.
When pyannote secondary validation is enabled, Meeting Assistant SHALL start a non-blocking startup warm-up that builds or verifies the configured pyannote runtime image and downloads the configured model into the persistent model cache before the first validation request when possible.
#### Scenario: Multi-speaker sample is rejected
- **GIVEN** pyannote secondary validation is enabled
- **WHEN** pyannote reports multiple speakers in a candidate sample
- **THEN** Meeting Assistant does not retain that sample for identity matching
#### Scenario: Multi-speaker speaker-override sample is rejected
- **GIVEN** pyannote secondary validation is enabled
- **WHEN** a summary speaker override resolves a source sample that pyannote reports as containing multiple speakers
- **THEN** Meeting Assistant does not retain that sample on a speaker identity
- **AND** does not create a new speaker identity from that rejected sample
#### Scenario: Pyannote rejects primary match
- **GIVEN** pyannote secondary validation is enabled
- **AND** the primary identity matcher confirms `Guest03` as `Chris`
- **WHEN** pyannote reports that the unknown `Guest03` sample and known `Chris` samples contain different speakers
- **THEN** Meeting Assistant rejects the match
#### Scenario: Enabled pyannote validation invokes its runtime
- **GIVEN** speaker identity pyannote validation is enabled
- **WHEN** Meeting Assistant validates a readable speaker sample
- **THEN** it invokes the configured pyannote runtime without requiring another enable setting
#### Scenario: Launch profile cannot override speaker validation
- **GIVEN** application-level speaker identity pyannote validation is disabled
- **AND** a named launch profile contains different speaker-validation settings
- **WHEN** Meeting Assistant starts or matches a speaker for that profile
- **THEN** it keeps application-level validation disabled
- **AND** does not warm or invoke the named profile's speaker-validation runtime
#### Scenario: Disabled pyannote validation preserves primary match
- **GIVEN** pyannote secondary validation is disabled
- **WHEN** the primary identity matcher confirms `Guest03` as `Chris`
- **THEN** Meeting Assistant accepts the match without running pyannote secondary validation
#### Scenario: Pyannote validation warms up on startup
- **GIVEN** pyannote secondary validation is enabled
- **WHEN** Meeting Assistant starts
- **THEN** it begins preparing the configured pyannote runtime image and model cache without waiting for the first validation request
- **AND** application startup is not blocked by the warm-up task
@@ -0,0 +1,15 @@
## 1. Single-toggle behavior
- [x] 1.1 Add a failing behavior test proving enabled speaker validation invokes pyannote without a nested enable setting
- [x] 1.2 Introduce toggle-free speaker-validation runtime options and make the validator use them
- [x] 1.3 Add or update behavior coverage proving disabled validation bypasses pyannote and enabled validation warms the runtime
- [x] 1.4 Keep application-level validation and warm-up consistent when named launch profiles contain speaker-validation overrides
## 2. Configuration migration
- [x] 2.1 Remove the nested speaker-validation diarization toggle from canonical configuration and document the outer toggle as authoritative
## 3. Verification
- [x] 3.1 Run focused speaker validation, warm-up, and pyannote finalizer tests
- [x] 3.2 Run the full solution test suite and validate the OpenSpec change strictly