Make meeting lifecycle stateful

openspec/changes/archive/2026-05-21-add-speaker-identity-learning/proposal.md

@@ -0,0 +1,17 @@
+# Add Speaker Identity Learning
+
+## Why
+Meeting Assistant currently receives diarized speaker labels from ASR backends, but those labels are backend-local names such as `Guest01` or `Speaker 0`. Users need transcripts to converge toward real attendee names without manually maintaining a roster.
+
+## What Changes
+- Add a local SQLite-backed speaker identity store under the user's application data folder.
+- Store a small bounded set of WAV snippets per learned speaker identity.
+- Match unknown diarized speakers against known identities during and after transcription using a dedicated Azure Speech diarization verifier component separate from the configured ASR pipeline.
+- Confirm matches before applying them, then rewrite transcript speaker labels and use the known name for later segments.
+- Learn new identities automatically from unmatched diarized speakers using meeting attendees as candidate names.
+- Eliminate candidate names across future meetings until a canonical speaker name is known.
+
+## Impact
+- Adds EF Core SQLite as a runtime dependency.
+- Adds speaker identity services and a dedicated Azure Speech identity matching component to the transcription flow.
+- Adds persisted local voice snippets; these are explicit identity artifacts, not temporary full recordings.

openspec/changes/archive/2026-05-21-add-speaker-identity-learning/specs/meeting-transcription/spec.md

@@ -0,0 +1,165 @@
+## ADDED Requirements
+
+### Requirement: Meeting Assistant learns speaker identities locally
+Meeting Assistant SHALL maintain a local SQLite speaker identity database in the user's application data folder.
+
+The speaker identity database SHALL store speaker identities, optional canonical names, aliases, candidate names, a transcription participation counter, and a bounded set of WAV snippets per identity.
+
+Each speaker identity SHALL store a last-modified timestamp used by active-age filtering, and Meeting Assistant SHALL update it whenever the identity is created or modified by identification, candidate updates, snippet changes, or merge operations.
+
+The configured maximum snippet count per identity SHALL prevent unbounded growth.
+
+#### Scenario: Unknown speaker is learned from meeting attendees
+- **WHEN** a finished transcript contains an unmatched diarized speaker and the meeting note has attendees
+- **THEN** Meeting Assistant stores a new unnamed speaker identity with candidate names from the attendees that were not already matched in that meeting
+
+#### Scenario: Speaker snippets are bounded
+- **WHEN** Meeting Assistant adds a snippet for an identity that already has the configured maximum number of snippets
+- **THEN** Meeting Assistant does not store more snippets for that identity
+
+#### Scenario: Identity modification updates active-age timestamp
+- **WHEN** Meeting Assistant creates, identifies, updates candidates for, stores snippets for, or merges a speaker identity
+- **THEN** Meeting Assistant updates that identity's last-modified timestamp
+
+### Requirement: Speaker identities can be merged diagnostically
+Meeting Assistant SHALL expose a diagnostic endpoint that merges duplicate speaker identities.
+
+The merge process SHALL compare recently-created identities, using a configurable recent age that defaults to two weeks, against all other identities in matcher batches bounded by the configured batch size.
+
+The merge process SHALL require a match and a second validation match using a different source sample before merging two identities.
+
+When identities are merged, Meeting Assistant SHALL retain one identity, move useful names from the merged identity into aliases, combine transcription counts, and retain a bounded set of snippets from both identities.
+
+#### Scenario: Recently-created duplicate identity is merged
+- **GIVEN** a recently-created identity and an older identity have matching speaker snippets
+- **WHEN** the diagnostic merge endpoint is triggered
+- **THEN** Meeting Assistant validates the match twice with different source snippets
+- **AND** merges the recent identity into the older identity
+- **AND** stores the recent identity name as an alias on the retained identity
+
+#### Scenario: Old identities are not used as merge sources
+- **GIVEN** two identities older than the configured recent age
+- **WHEN** the diagnostic merge endpoint is triggered
+- **THEN** Meeting Assistant does not compare them as source identities
+
+### Requirement: Speaker candidates are eliminated across meetings
+Meeting Assistant SHALL update candidate names for a matched unnamed identity using the intersection of its existing candidate names and the current meeting attendees.
+
+Meeting Assistant SHALL treat identity aliases as acceptable names during candidate elimination and when excluding already-matched attendees from new unmatched speaker candidates.
+
+When the candidate intersection leaves exactly one candidate, Meeting Assistant SHALL promote that candidate to the canonical speaker name.
+
+When the candidate intersection is empty, Meeting Assistant SHALL replace the oldest stored snippet for that identity and reset candidates from the current meeting attendees, because the previous snippet may have been dirty.
+
+#### Scenario: Candidate elimination promotes canonical name
+- **GIVEN** an unnamed identity has candidate names `John` and `Mike`
+- **WHEN** that identity matches a speaker in a later meeting with attendees `Jane`, `John`, and `Chris`
+- **THEN** Meeting Assistant removes `Mike` from the identity candidates and promotes `John` as the canonical name
+
+#### Scenario: Alias participates in candidate elimination
+- **GIVEN** an unnamed identity has candidate name `Michael` and alias `Mike`
+- **WHEN** that identity matches a speaker in a later meeting with attendee `Mike`
+- **THEN** Meeting Assistant treats `Mike` as matching `Michael`
+- **AND** promotes `Michael` as the canonical name
+
+#### Scenario: Empty candidate intersection resets candidates
+- **GIVEN** an unnamed identity has candidate names `John` and `Mike`
+- **WHEN** that identity matches a speaker in a later meeting with attendees `Jane` and `Chris`
+- **THEN** Meeting Assistant resets the identity candidates to `Jane` and `Chris`
+- **AND** replaces the oldest stored snippet for that identity with the current speaker snippet
+
+### Requirement: Speaker identity matches relabel transcripts
+Meeting Assistant SHALL attempt to match unknown diarized speaker snippets against known speaker identities ordered by transcription participation count.
+
+Speaker identity matching SHALL use a dedicated Azure Speech diarization verifier component rather than the configured speech recognition pipeline.
+
+The matcher SHALL test at most the configured batch size of known people per diarization session and continue with later batches until a match is found or no candidates remain.
+
+The matcher SHALL prioritize identities whose canonical name or aliases match current meeting attendees.
+
+After attendee-matched identities, the matcher SHALL order identities by transcription participation count, filter out non-attendee identities whose last update is older than the configured active age, and cap the candidate set at the configured maximum match candidate count.
+
+When a match is confirmed and the identity has a canonical name, Meeting Assistant SHALL rewrite finished transcript segments for that diarized speaker with the canonical name.
+
+When a match is confirmed and the matched speaker is not already listed in meeting note attendees by display name or alias, Meeting Assistant SHALL add the speaker display name to the attendee list.
+
+When Meeting Assistant writes attendees from calendar metadata, it SHALL match attendee display names exactly against known identity canonical names and aliases, replace matches with the identity display name, and deduplicate attendees that map to the same identity.
+
+#### Scenario: Finished transcript is relabeled after a confirmed match
+- **GIVEN** the speaker identity database contains canonical speaker `Chris`
+- **WHEN** a finished transcript has diarized speaker `Guest03` and the matching backend confirms it is `Chris`
+- **THEN** Meeting Assistant rewrites `Guest03` segments in the transcript as `Chris`
+
+#### Scenario: Confirmed match adds missing attendee
+- **GIVEN** the speaker identity database contains canonical speaker `Chris`
+- **AND** the meeting note attendees do not contain `Chris` or one of that identity's aliases
+- **WHEN** live or final speaker matching confirms a diarized speaker is `Chris`
+- **THEN** Meeting Assistant adds `Chris` to the meeting note attendees
+
+#### Scenario: Confirmed match does not duplicate attendee aliases
+- **GIVEN** the speaker identity database contains canonical speaker `Christopher` with alias `Chris`
+- **AND** the meeting note attendees already contain `Chris <chris@example.com>`
+- **WHEN** live or final speaker matching confirms a diarized speaker is `Christopher`
+- **THEN** Meeting Assistant does not add another attendee for `Christopher`
+
+#### Scenario: Calendar attendee aliases are deduplicated
+- **GIVEN** the speaker identity database contains canonical speaker `Karl Berger` with alias `Berger, Karl`
+- **WHEN** calendar metadata provides attendees `Karl Berger` and `Berger, Karl`
+- **THEN** Meeting Assistant writes a single attendee `Karl Berger` to the meeting note
+
+#### Scenario: Attendee identities are tried first
+- **GIVEN** the meeting attendees include names matching known identity display names or aliases
+- **WHEN** Meeting Assistant tries to identify a diarized speaker
+- **THEN** attendee-matched identities are tried before non-attendee identities
+
+#### Scenario: Stale non-attendee identities are skipped
+- **GIVEN** a known identity has not matched within the configured active age
+- **AND** that identity does not match the current meeting attendees
+- **WHEN** Meeting Assistant tries to identify a diarized speaker
+- **THEN** that stale identity is not included in the match candidates
+
+### Requirement: Speaker matching runs during active transcription
+Meeting Assistant SHALL start speaker identity matching only after the configured initial transcription duration has elapsed.
+
+For backends that emit live diarized transcript segments, Meeting Assistant SHALL keep a bounded in-memory sliding audio buffer with chunk timestamps and extract candidate WAV snippets from that buffer when live diarized segments arrive.
+
+Meeting Assistant SHALL keep only the configured best candidate snippets per diarized speaker in memory. Better snippets SHALL be preferred when the segment looks like a continuous medium-length sentence.
+
+Meeting Assistant SHALL periodically match unresolved diarized speaker samples while transcription is active and attempt to match them against the local identity database.
+
+Meeting Assistant SHALL run live matching incrementally at the configured interval only when at least one new unmapped diarized speaker sample appears or the meeting note attendee frontmatter changes while unmapped speaker samples still exist.
+
+When a speaker is matched during transcription, Meeting Assistant SHALL rewrite already-written live transcript segments for that diarized speaker and write future transcript segments using the canonical name.
+
+Live speaker matching SHALL be read-only with respect to the speaker identity database. Candidate elimination, canonical promotion, transcription counters, stored snippet updates, and new unmatched identity creation SHALL happen only after transcription is finished, using the latest meeting note frontmatter.
+
+For backends that only provide diarization after finalization, Meeting Assistant SHALL defer speaker identity matching until finished diarization is available, extract candidate snippets from the completed temporary recording, complete identity matching, and only then allow summary generation to start.
+
+#### Scenario: Matching waits for useful speech duration
+- **WHEN** transcription has been active for less than the configured speaker identification initial delay
+- **THEN** Meeting Assistant does not run speaker identity matching yet
+
+#### Scenario: Live matching uses in-memory speaker samples
+- **WHEN** a live diarized transcript segment identifies an unresolved speaker
+- **THEN** Meeting Assistant extracts a WAV snippet for that segment from the in-memory sliding audio buffer
+- **AND** uses retained speaker samples for live identity matching without reading the temporary recording file
+
+#### Scenario: Live match rewrites current and future transcript writes
+- **WHEN** periodic matching confirms that diarized speaker `Guest03` is canonical speaker `Chris`
+- **THEN** already-written live transcript segments for `Guest03` are rewritten as `Chris`
+- **AND** later live transcript segments for `Guest03` are written as `Chris`
+
+#### Scenario: New live speaker triggers another identification round
+- **GIVEN** live matching already checked the current unresolved speaker samples
+- **WHEN** a new unmapped diarized speaker sample appears
+- **THEN** Meeting Assistant runs another live matching round at the next configured interval
+
+#### Scenario: Attendee changes trigger another identification round
+- **GIVEN** live matching already checked unresolved speaker samples
+- **WHEN** the meeting note attendee frontmatter changes
+- **THEN** Meeting Assistant runs another live matching round at the next configured interval using the latest attendees
+
+#### Scenario: Live matching does not make final identity decisions
+- **WHEN** live matching finds a possible speaker identity during transcription
+- **THEN** Meeting Assistant does not change candidate names, promote canonical names, increment counters, store snippets, or create unmatched identities
+- **AND** the final speaker identity pass uses the latest meeting note attendees after transcription finishes

openspec/changes/archive/2026-05-21-add-speaker-identity-learning/tasks.md

@@ -0,0 +1,36 @@
+## 1. Specification
+- [x] 1.1 Define requirements for local speaker identity storage and candidate elimination.
+- [x] 1.2 Define requirements for live periodic matching and transcript relabeling.
+
+## 2. Implementation
+- [x] 2.1 Add SQLite speaker identity database in app data.
+- [x] 2.2 Add speaker snippet extraction and matching orchestration.
+- [x] 2.3 Learn unmatched speaker identities from meeting attendees at transcript completion.
+- [x] 2.4 Eliminate candidate names on future matches and promote a canonical name.
+- [x] 2.5 Relabel finished transcript segments when canonical identity matches are found.
+- [x] 2.6 Add live periodic matching after an initial delay and apply matches to future transcript writes.
+- [x] 2.7 Keep timestamped in-memory audio samples for live diarized speaker matching and reserve recording-file extraction for final-only diarization.
+- [x] 2.8 Add speaker aliases and a diagnostic duplicate-identity merge process.
+- [x] 2.9 Use aliases during candidate elimination and unmatched-candidate filtering.
+- [x] 2.10 Keep live speaker matching read-only so final identity decisions use the latest meeting note frontmatter.
+- [x] 2.11 Rewrite already-written live transcript segments when a live speaker match is found.
+- [x] 2.12 Prioritize attendee-matched speaker identities, filter stale non-attendees, and cap match candidates.
+- [x] 2.13 Run live speaker identification incrementally when new unmapped speakers appear or attendees change.
+- [x] 2.14 Maintain speaker identity last-modified timestamps for active-age filtering.
+- [x] 2.15 Add identified speaker display names to meeting note attendees without duplicating display names or aliases.
+- [x] 2.16 Canonicalize calendar attendee names through identity canonical names and aliases before writing meeting notes.
+
+## 3. Validation
+- [x] 3.1 Add tests for candidate creation, elimination, canonical promotion, and reset on empty candidate intersection.
+- [x] 3.2 Add tests for recording coordinator transcript relabel integration.
+- [x] 3.3 Add tests for in-memory speaker samples and rolling audio buffer extraction.
+- [x] 3.4 Add tests for duplicate speaker identity merging and the diagnostic endpoint.
+- [x] 3.5 Add tests for alias-aware candidate elimination.
+- [x] 3.6 Add tests that live matching does not mutate speaker identities.
+- [x] 3.7 Add tests that live speaker matches relabel current and future transcript segments.
+- [x] 3.8 Add tests for speaker identity candidate ordering, active-age filtering, and candidate caps.
+- [x] 3.9 Add tests for incremental live speaker identification triggers.
+- [x] 3.10 Add tests for speaker identity last-modified timestamp creation, updates, and schema upgrade.
+- [x] 3.11 Add tests for adding matched speaker identities to meeting attendees and alias duplicate suppression.
+- [x] 3.12 Add tests for calendar attendee canonicalization and alias duplicate suppression.
+- [x] 3.13 Run `dotnet test`.