meeting-assistant/openspec/changes/add-speaker-identity-learning/specs/meeting-transcription/spec.md

ADDED Requirements

Requirement: Meeting Assistant learns speaker identities locally

Meeting Assistant SHALL maintain a local SQLite speaker identity database in the user's application data folder.

The speaker identity database SHALL store speaker identities, optional canonical names, aliases, candidate names, a transcription participation counter, and a bounded set of WAV snippets per identity.

Each speaker identity SHALL store a last-modified timestamp used by active-age filtering, and Meeting Assistant SHALL update it whenever the identity is created or modified by identification, candidate updates, snippet changes, or merge operations.

The configured maximum snippet count per identity SHALL prevent unbounded growth.

Scenario: Unknown speaker is learned from meeting attendees

• WHEN a finished transcript contains an unmatched diarized speaker and the meeting note has attendees
• THEN Meeting Assistant stores a new unnamed speaker identity with candidate names from the attendees that were not already matched in that meeting

Scenario: Speaker snippets are bounded

• WHEN Meeting Assistant adds a snippet for an identity that already has the configured maximum number of snippets
• THEN Meeting Assistant does not store more snippets for that identity

Scenario: Identity modification updates active-age timestamp

• WHEN Meeting Assistant creates, identifies, updates candidates for, stores snippets for, or merges a speaker identity
• THEN Meeting Assistant updates that identity's last-modified timestamp

Requirement: Speaker identities can be merged diagnostically

Meeting Assistant SHALL expose a diagnostic endpoint that merges duplicate speaker identities.

The merge process SHALL compare recently-created identities, using a configurable recent age that defaults to two weeks, against all other identities in matcher batches bounded by the configured batch size.

The merge process SHALL require a match and a second validation match using a different source sample before merging two identities.

When identities are merged, Meeting Assistant SHALL retain one identity, move useful names from the merged identity into aliases, combine transcription counts, and retain a bounded set of snippets from both identities.

Scenario: Recently-created duplicate identity is merged

• GIVEN a recently-created identity and an older identity have matching speaker snippets
• WHEN the diagnostic merge endpoint is triggered
• THEN Meeting Assistant validates the match twice with different source snippets
• AND merges the recent identity into the older identity
• AND stores the recent identity name as an alias on the retained identity

Scenario: Old identities are not used as merge sources

• GIVEN two identities older than the configured recent age
• WHEN the diagnostic merge endpoint is triggered
• THEN Meeting Assistant does not compare them as source identities

Requirement: Speaker candidates are eliminated across meetings

Meeting Assistant SHALL update candidate names for a matched unnamed identity using the intersection of its existing candidate names and the current meeting attendees.

Meeting Assistant SHALL treat identity aliases as acceptable names during candidate elimination and when excluding already-matched attendees from new unmatched speaker candidates.

When the candidate intersection leaves exactly one candidate, Meeting Assistant SHALL promote that candidate to the canonical speaker name.

When the candidate intersection is empty, Meeting Assistant SHALL replace the oldest stored snippet for that identity and reset candidates from the current meeting attendees, because the previous snippet may have been dirty.

Scenario: Candidate elimination promotes canonical name

• GIVEN an unnamed identity has candidate names John and Mike
• WHEN that identity matches a speaker in a later meeting with attendees Jane, John, and Chris
• THEN Meeting Assistant removes Mike from the identity candidates and promotes John as the canonical name

Scenario: Alias participates in candidate elimination

• GIVEN an unnamed identity has candidate name Michael and alias Mike
• WHEN that identity matches a speaker in a later meeting with attendee Mike
• THEN Meeting Assistant treats Mike as matching Michael
• AND promotes Michael as the canonical name

Scenario: Empty candidate intersection resets candidates

• GIVEN an unnamed identity has candidate names John and Mike
• WHEN that identity matches a speaker in a later meeting with attendees Jane and Chris
• THEN Meeting Assistant resets the identity candidates to Jane and Chris
• AND replaces the oldest stored snippet for that identity with the current speaker snippet

Requirement: Speaker identity matches relabel transcripts

Meeting Assistant SHALL attempt to match unknown diarized speaker snippets against known speaker identities ordered by transcription participation count.

Speaker identity matching SHALL use a dedicated Azure Speech diarization verifier component rather than the configured speech recognition pipeline.

The matcher SHALL test at most the configured batch size of known people per diarization session and continue with later batches until a match is found or no candidates remain.

The matcher SHALL prioritize identities whose canonical name or aliases match current meeting attendees.

After attendee-matched identities, the matcher SHALL order identities by transcription participation count, filter out non-attendee identities whose last update is older than the configured active age, and cap the candidate set at the configured maximum match candidate count.

When a match is confirmed and the identity has a canonical name, Meeting Assistant SHALL rewrite finished transcript segments for that diarized speaker with the canonical name.

When a match is confirmed and the matched speaker is not already listed in meeting note attendees by display name or alias, Meeting Assistant SHALL add the speaker display name to the attendee list.

Scenario: Finished transcript is relabeled after a confirmed match

• GIVEN the speaker identity database contains canonical speaker Chris
• WHEN a finished transcript has diarized speaker Guest03 and the matching backend confirms it is Chris
• THEN Meeting Assistant rewrites Guest03 segments in the transcript as Chris

Scenario: Confirmed match adds missing attendee

• GIVEN the speaker identity database contains canonical speaker Chris
• AND the meeting note attendees do not contain Chris or one of that identity's aliases
• WHEN live or final speaker matching confirms a diarized speaker is Chris
• THEN Meeting Assistant adds Chris to the meeting note attendees

Scenario: Confirmed match does not duplicate attendee aliases

• GIVEN the speaker identity database contains canonical speaker Christopher with alias Chris
• AND the meeting note attendees already contain Chris <chris@example.com>
• WHEN live or final speaker matching confirms a diarized speaker is Christopher
• THEN Meeting Assistant does not add another attendee for Christopher

Scenario: Attendee identities are tried first

• GIVEN the meeting attendees include names matching known identity display names or aliases
• WHEN Meeting Assistant tries to identify a diarized speaker
• THEN attendee-matched identities are tried before non-attendee identities

Scenario: Stale non-attendee identities are skipped

• GIVEN a known identity has not matched within the configured active age
• AND that identity does not match the current meeting attendees
• WHEN Meeting Assistant tries to identify a diarized speaker
• THEN that stale identity is not included in the match candidates

Requirement: Speaker matching runs during active transcription

Meeting Assistant SHALL start speaker identity matching only after the configured initial transcription duration has elapsed.

For backends that emit live diarized transcript segments, Meeting Assistant SHALL keep a bounded in-memory sliding audio buffer with chunk timestamps and extract candidate WAV snippets from that buffer when live diarized segments arrive.

Meeting Assistant SHALL keep only the configured best candidate snippets per diarized speaker in memory. Better snippets SHALL be preferred when the segment looks like a continuous medium-length sentence.

Meeting Assistant SHALL periodically match unresolved diarized speaker samples while transcription is active and attempt to match them against the local identity database.

Meeting Assistant SHALL run live matching incrementally at the configured interval only when at least one new unmapped diarized speaker sample appears or the meeting note attendee frontmatter changes while unmapped speaker samples still exist.

When a speaker is matched during transcription, Meeting Assistant SHALL rewrite already-written live transcript segments for that diarized speaker and write future transcript segments using the canonical name.

Live speaker matching SHALL be read-only with respect to the speaker identity database. Candidate elimination, canonical promotion, transcription counters, stored snippet updates, and new unmatched identity creation SHALL happen only after transcription is finished, using the latest meeting note frontmatter.

For backends that only provide diarization after finalization, Meeting Assistant SHALL defer speaker identity matching until finished diarization is available, extract candidate snippets from the completed temporary recording, complete identity matching, and only then allow summary generation to start.

Scenario: Matching waits for useful speech duration

• WHEN transcription has been active for less than the configured speaker identification initial delay
• THEN Meeting Assistant does not run speaker identity matching yet

Scenario: Live matching uses in-memory speaker samples

• WHEN a live diarized transcript segment identifies an unresolved speaker
• THEN Meeting Assistant extracts a WAV snippet for that segment from the in-memory sliding audio buffer
• AND uses retained speaker samples for live identity matching without reading the temporary recording file

Scenario: Live match rewrites current and future transcript writes

• WHEN periodic matching confirms that diarized speaker Guest03 is canonical speaker Chris
• THEN already-written live transcript segments for Guest03 are rewritten as Chris
• AND later live transcript segments for Guest03 are written as Chris

Scenario: New live speaker triggers another identification round

• GIVEN live matching already checked the current unresolved speaker samples
• WHEN a new unmapped diarized speaker sample appears
• THEN Meeting Assistant runs another live matching round at the next configured interval

Scenario: Attendee changes trigger another identification round

• GIVEN live matching already checked unresolved speaker samples
• WHEN the meeting note attendee frontmatter changes
• THEN Meeting Assistant runs another live matching round at the next configured interval using the latest attendees

Scenario: Live matching does not make final identity decisions

• WHEN live matching finds a possible speaker identity during transcription
• THEN Meeting Assistant does not change candidate names, promote canonical names, increment counters, store snippets, or create unmatched identities
• AND the final speaker identity pass uses the latest meeting note attendees after transcription finishes