Public Access
Implement meeting assistant v1
This commit is contained in:
@@ -0,0 +1,61 @@
|
||||
# agent-project-tools Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change define-meeting-assistant-v1. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Agents can use project context tools
|
||||
Meeting Assistant SHALL expose tools that allow agents to look up project information, retrieve keyword-relevant context, and inspect meeting-derived knowledge.
|
||||
|
||||
Project tools SHALL treat each direct subfolder of the configured projects folder as one project. A meeting note binds projects by listing those subfolder names in the `projects` frontmatter field.
|
||||
|
||||
The summary agent SHALL expose these project tools:
|
||||
|
||||
- `list_projects`
|
||||
- `list_projectfiles`
|
||||
- `read_projectfile`
|
||||
- `write_projectfile`
|
||||
- `search`
|
||||
|
||||
#### Scenario: Agent looks up meeting project context
|
||||
- **WHEN** a meeting note identifies a project
|
||||
- **THEN** the agent can retrieve relevant project context through Meeting Assistant tools
|
||||
|
||||
#### Scenario: Agent lists projects bound to the meeting
|
||||
- **WHEN** the meeting note frontmatter lists project folder names
|
||||
- **THEN** `list_projects` returns the matching configured project subfolders
|
||||
|
||||
#### Scenario: Agent reads a clamped project file range
|
||||
- **WHEN** the agent reads a project file with optional line bounds outside the file length
|
||||
- **THEN** `read_projectfile` clamps the requested range to the available file lines without failing
|
||||
|
||||
#### Scenario: Agent searches project knowledge
|
||||
- **WHEN** the agent calls `search` with ripgrep syntax keywords
|
||||
- **THEN** Meeting Assistant runs the search against the requested projects or the meeting-bound projects and returns matches as `filename:line text`
|
||||
|
||||
### Requirement: Agents can write files in existing projects
|
||||
Meeting Assistant SHALL expose a `write_projectfile` tool that can create or update files inside an existing project folder.
|
||||
|
||||
The target project SHALL be an existing direct subfolder of the configured projects folder. The target file path SHALL stay inside that project folder.
|
||||
|
||||
When no line edit arguments are supplied, `write_projectfile` SHALL overwrite the whole file and SHALL create the file when it does not exist.
|
||||
|
||||
When both `from` and `to` are supplied, `write_projectfile` SHALL replace the inclusive 1-based line range with the supplied content.
|
||||
|
||||
When `insert` is supplied, `write_projectfile` SHALL insert the supplied content at that 1-based line position.
|
||||
|
||||
#### Scenario: Project file is overwritten or created
|
||||
- **WHEN** the agent writes a project file without line edit arguments
|
||||
- **THEN** Meeting Assistant writes the supplied content as the complete file content
|
||||
|
||||
#### Scenario: Project file line range is replaced
|
||||
- **WHEN** the agent writes a project file with `from` and `to` line numbers
|
||||
- **THEN** Meeting Assistant replaces the inclusive line range with the supplied content
|
||||
|
||||
#### Scenario: Project file content is inserted
|
||||
- **WHEN** the agent writes a project file with an `insert` line number
|
||||
- **THEN** Meeting Assistant inserts the supplied content at that line position
|
||||
|
||||
#### Scenario: Project file write target is invalid
|
||||
- **WHEN** the target project is missing or the target path escapes the project folder
|
||||
- **THEN** Meeting Assistant refuses the project file write
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
# meeting-recording Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change define-meeting-assistant-v1. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Recording mode is controlled by a configurable hotkey
|
||||
Meeting Assistant SHALL use normal .NET configuration to define the global hotkey that toggles recording/transcription mode.
|
||||
|
||||
#### Scenario: Hotkey starts recording
|
||||
- **WHEN** the configured hotkey is pressed while recording mode is inactive
|
||||
- **THEN** Meeting Assistant starts recording/transcription mode
|
||||
|
||||
#### Scenario: Hotkey stops recording
|
||||
- **WHEN** the configured hotkey is pressed while recording mode is active
|
||||
- **THEN** Meeting Assistant stops recording/transcription mode
|
||||
|
||||
### Requirement: Recording mode captures microphone and computer output
|
||||
Meeting Assistant SHALL capture microphone input and computer output and combine them into one audio stream for transcription.
|
||||
|
||||
#### Scenario: Both sources produce audio
|
||||
- **WHEN** microphone and computer output audio chunks are available
|
||||
- **THEN** Meeting Assistant mixes them into one PCM stream before transcription
|
||||
|
||||
#### Scenario: Device-level capture cannot be verified in tests
|
||||
- **WHEN** automated tests run without live audio devices
|
||||
- **THEN** Meeting Assistant verifies the audio mixer through deterministic source abstractions rather than depending on physical microphone or speaker devices
|
||||
|
||||
### Requirement: Stopping recording drains captured audio through transcription
|
||||
Meeting Assistant SHALL stop capturing new audio when recording mode is stopped, but it SHALL allow already captured audio to finish running through the configured speech recognition pipeline before the recording session completes.
|
||||
|
||||
#### Scenario: Hotkey stops recording with buffered audio
|
||||
- **WHEN** the user presses the hotkey to stop recording while captured audio is still buffered for transcription
|
||||
- **THEN** Meeting Assistant stops capturing new audio and appends transcription output for the already captured audio before reporting the recording stopped
|
||||
|
||||
### Requirement: Transcripts are written to the configured vault folder
|
||||
Meeting Assistant SHALL write live transcript text to a markdown file in the configured vault folder.
|
||||
|
||||
#### Scenario: Recording starts
|
||||
- **WHEN** recording mode starts
|
||||
- **THEN** Meeting Assistant creates the configured vault folder if necessary and creates a transcript markdown file
|
||||
|
||||
#### Scenario: Transcription segment arrives
|
||||
- **WHEN** the speech recognition pipeline emits a live transcript segment
|
||||
- **THEN** Meeting Assistant appends that segment to the active transcript text file
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
# meeting-session Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change define-meeting-assistant-v1. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Meeting Assistant creates a note before transcription
|
||||
Meeting Assistant SHALL create an Obsidian markdown note for a meeting session before transcription starts.
|
||||
|
||||
The note SHALL be the durable anchor for meeting metadata, user notes, assistant-discovered context, transcript references, summaries, decisions, and next steps.
|
||||
|
||||
#### Scenario: Session starts
|
||||
- **WHEN** the user starts a new meeting session
|
||||
- **THEN** Meeting Assistant creates a markdown note in the configured meetings folder before starting transcription
|
||||
|
||||
#### Scenario: Recording starts from hotkey
|
||||
- **WHEN** the user presses the recording hotkey while recording is inactive
|
||||
- **THEN** Meeting Assistant creates the transcript file, creates the meeting note with links to the transcript, assistant context, and summary notes, and opens the meeting note through an `obsidian://` system command
|
||||
|
||||
#### Scenario: User adds context before transcription
|
||||
- **WHEN** the note exists before transcription starts
|
||||
- **THEN** the user can add attendees, project context, and other notes to that file
|
||||
|
||||
### Requirement: Vault locations are configurable
|
||||
Meeting Assistant SHALL expose separate configuration settings for transcript files, meeting notes, assistant context, generated summaries, project knowledge, and dictation words.
|
||||
|
||||
The dictation words location SHALL point to a word-list artifact that speech recognition pipelines MAY use as vocabulary hints when supported.
|
||||
|
||||
#### Scenario: Transcript and note locations are configured separately
|
||||
- **WHEN** Meeting Assistant starts with configured vault locations
|
||||
- **THEN** transcripts, meeting notes, assistant context, generated summaries, project knowledge, and dictation words each have an independently configured path
|
||||
|
||||
#### Scenario: Provider supports dictation words
|
||||
- **WHEN** the configured speech recognition pipeline supports vocabulary hints
|
||||
- **THEN** Meeting Assistant can provide words from the configured dictation words path without changing the other vault locations
|
||||
|
||||
### Requirement: Meeting note template is coded and round-trippable
|
||||
Meeting Assistant SHALL own a coded meeting note template and a store that can save and read meeting notes.
|
||||
|
||||
The meeting note frontmatter SHALL include:
|
||||
|
||||
- `start_time` as the meeting recording start timestamp
|
||||
- `end_time` as the timestamp when transcript processing stops, empty while the meeting is active
|
||||
- `attendees` as an array of display strings, where email is optional and may use a format such as `Mike <m.ike@ibm.com>`
|
||||
- `projects` as an array because one meeting can cover multiple projects
|
||||
- `transcript` as text containing a markdown link to the transcript note
|
||||
- `assistant_context` as text containing a markdown link to the assistant context note
|
||||
- `summary` as text containing a markdown link to the generated summary note once Meeting Assistant creates it
|
||||
|
||||
The user notes SHALL be the markdown body. Decisions and next steps SHALL NOT be required as frontmatter fields.
|
||||
|
||||
#### Scenario: Meeting note is saved
|
||||
- **WHEN** Meeting Assistant saves a meeting note
|
||||
- **THEN** it writes the coded frontmatter template and the user notes body to a markdown file in the configured meeting notes folder
|
||||
|
||||
#### Scenario: Meeting note links generated artifacts
|
||||
- **WHEN** Meeting Assistant creates a meeting note for a new recording
|
||||
- **THEN** the note frontmatter links to the configured transcript, assistant context, and summary note locations
|
||||
|
||||
#### Scenario: Generated artifacts link back to each other
|
||||
- **WHEN** Meeting Assistant creates transcript, assistant context, and summary artifacts
|
||||
- **THEN** those artifacts use frontmatter links for the meeting note, transcript note, assistant context note, and summary note where applicable
|
||||
|
||||
#### Scenario: Meeting note records session times
|
||||
- **WHEN** Meeting Assistant creates a meeting note for a new recording
|
||||
- **THEN** the note frontmatter contains `start_time`
|
||||
- **AND** when transcript processing stops, Meeting Assistant writes `end_time` to the same note frontmatter
|
||||
- **AND** transcript and summary frontmatter copy the meeting note title, `start_time`, and `end_time`
|
||||
|
||||
#### Scenario: Meeting note is read
|
||||
- **WHEN** Meeting Assistant reads a meeting note created from the coded template
|
||||
- **THEN** it returns the frontmatter fields and user notes body without losing attendees, projects, transcript link, assistant context link, or summary
|
||||
|
||||
#### Scenario: Meeting note frontmatter is updated
|
||||
- **WHEN** Meeting Assistant reads a meeting note, changes frontmatter, and writes the note back
|
||||
- **THEN** the existing user-authored markdown body remains unchanged
|
||||
|
||||
### Requirement: Meeting notes preserve user-authored content
|
||||
Meeting Assistant SHALL preserve user-authored meeting notes and include them as input when generating summaries, decisions, and next steps.
|
||||
|
||||
#### Scenario: User writes notes during the meeting
|
||||
- **WHEN** the user updates the meeting note while the meeting is active
|
||||
- **THEN** Meeting Assistant keeps that content and incorporates it into generated meeting outputs
|
||||
|
||||
### Requirement: Meeting sessions are platform independent
|
||||
Meeting Assistant SHALL support meeting sessions that do not originate from Teams, Zoom, or any other meeting software API.
|
||||
|
||||
Meeting software integrations MAY augment the meeting experience, but they SHALL NOT be required for the primary meeting capture, transcription, or summary flow.
|
||||
|
||||
#### Scenario: In-person meeting is captured
|
||||
- **WHEN** the user starts transcription for an in-person meeting
|
||||
- **THEN** Meeting Assistant captures and processes the meeting without requiring a meeting platform integration
|
||||
|
||||
### Requirement: Windows Outlook enrichment is optional
|
||||
Meeting Assistant SHALL gate Outlook Classic COM enrichment behind the Windows compilation target.
|
||||
|
||||
When the Windows build starts a meeting and Outlook Classic has exactly one current Teams appointment, Meeting Assistant SHALL copy the appointment title and attendees to the meeting note and copy the appointment agenda and scheduled end time to the assistant-context frontmatter.
|
||||
|
||||
The agenda SHALL be extracted from the appointment body content before the Teams join separator or Teams join text.
|
||||
|
||||
#### Scenario: Current Teams appointment enriches meeting artifacts
|
||||
- **WHEN** a Windows build starts a meeting while Outlook Classic exposes exactly one current Teams appointment
|
||||
- **THEN** Meeting Assistant uses the appointment subject as the meeting title
|
||||
- **AND** writes the appointment attendees into meeting note frontmatter
|
||||
- **AND** writes the appointment agenda into assistant-context frontmatter
|
||||
- **AND** writes the appointment end time as `scheduled_end` into assistant-context frontmatter
|
||||
|
||||
#### Scenario: Outlook is unavailable or ambiguous
|
||||
- **WHEN** Outlook Classic is unavailable or more than one current Teams appointment is found
|
||||
- **THEN** Meeting Assistant starts the recording with the default generated meeting title and empty agenda
|
||||
- **AND** omits `scheduled_end` from assistant-context frontmatter
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
# meeting-summary Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change define-meeting-assistant-v1. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Meeting Assistant generates meeting outputs
|
||||
Meeting Assistant SHALL generate a summary, decisions, and next steps from the meeting transcript, metadata, user notes, and assistant-discovered context.
|
||||
|
||||
Meeting Assistant SHALL use a Microsoft Agent Framework pipeline for the first summary implementation. The pipeline SHALL use a configurable OpenAI-compatible endpoint, model, and API key source, including support for a direct key or an environment variable name.
|
||||
|
||||
The summary pipeline SHALL retry transient model endpoint failures according to configurable reconnection attempts and delay settings.
|
||||
|
||||
The summary pipeline SHALL track context-window usage for the configured model using response usage when available and request-size estimates otherwise. The context-window limit, maximum output reserve, compaction enablement, compaction threshold, and Responses compact endpoint path SHALL be configurable.
|
||||
|
||||
When only the configured remaining context ratio is available, the summary pipeline SHALL try to compact the conversation through the configured OpenAI-compatible `POST /v1/responses/compact` endpoint. If that endpoint is unavailable or returns invalid data, the pipeline SHALL fall back to Microsoft Agent Framework compaction.
|
||||
|
||||
Fallback compaction SHALL become increasingly aggressive: collapse old tool results, summarize older conversation spans, keep only the last four user turns, and drop oldest groups if still over budget.
|
||||
|
||||
When summary generation fails after retries, Meeting Assistant SHALL write a markdown failure report to the configured summary note path. The failure report SHALL include a clickable retry link, error details, and the meeting artifact paths needed to diagnose or retry the run.
|
||||
|
||||
After transcript processing finishes for a recording, Meeting Assistant SHALL automatically invoke the summary pipeline for that meeting.
|
||||
|
||||
Meeting Assistant SHALL expose an API operation to retry summary generation for a given summary note path. The retry operation SHALL resolve the linked meeting artifacts from the meeting note frontmatter and SHALL overwrite the existing summary note with either the new summary or a new failure report.
|
||||
|
||||
The summary pipeline SHALL expose tools scoped to the current meeting:
|
||||
|
||||
- `read_meetingnote`
|
||||
- `read_transcript`
|
||||
- `read_context`
|
||||
- `read_usernotes`
|
||||
- `read_glossary`
|
||||
- `write_summary`
|
||||
- `write_context`
|
||||
- `list_projects`
|
||||
- `list_projectfiles`
|
||||
- `read_projectfile`
|
||||
- `write_projectfile`
|
||||
- `search`
|
||||
|
||||
All summary-agent read tools that return file-like content SHALL support either reading the whole file or reading a clamped inclusive 1-based line range when `from` and `to` are supplied.
|
||||
|
||||
After writing the meeting summary, the summary agent SHALL update existing project files when the meeting produced project-relevant knowledge.
|
||||
|
||||
The assistant context note SHALL keep `meeting`, `transcript`, `summary`, `state`, and `agenda` in frontmatter. When a Teams appointment is detected at recording start, the assistant context note SHALL also keep `scheduled_end` in frontmatter. The `state` value SHALL be one of `transcribing`, `speaker recognition`, `summarizing`, `finished`, or `error`.
|
||||
|
||||
The summary note SHALL keep frontmatter links to meeting artifacts and SHALL copy `title`, `start_time`, and `end_time` from the meeting note when available. If the meeting note has no title, the summary agent SHALL be able to provide a title when writing the summary.
|
||||
|
||||
The summary agent SHALL be able to read and write the assistant context body as its own notebook using overwrite, replace, and insert line modes.
|
||||
|
||||
#### Scenario: Assistant context note is initialized
|
||||
- **WHEN** Meeting Assistant starts a meeting session
|
||||
- **THEN** it creates the assistant context note with frontmatter links to the meeting note, transcript note, and summary note
|
||||
- **AND** the assistant context frontmatter state is `transcribing`
|
||||
- **AND** the assistant context frontmatter includes `agenda`, empty when no agenda is known
|
||||
- **AND** the assistant context frontmatter includes `scheduled_end` when Teams metadata supplied a scheduled end time
|
||||
|
||||
#### Scenario: Assistant context state follows meeting processing
|
||||
- **WHEN** transcription, final speaker recognition, and summary generation progress
|
||||
- **THEN** Meeting Assistant updates assistant context frontmatter state to `transcribing`, `speaker recognition`, `summarizing`, `finished`, or `error` as appropriate
|
||||
- **AND** preserves existing `agenda` and `scheduled_end` frontmatter values
|
||||
|
||||
@@ -0,0 +1,190 @@
|
||||
# meeting-transcription Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change define-meeting-assistant-v1. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Meeting Assistant transcribes meetings with speaker attribution
|
||||
Meeting Assistant SHALL transcribe meeting audio and associate transcript segments with speakers when speaker attribution is available.
|
||||
|
||||
The transcription contract SHALL remain independent of the meeting source so that local audio, in-person meetings, and future meeting-platform integrations can use the same downstream processing.
|
||||
|
||||
#### Scenario: Speaker-attributed transcript is produced
|
||||
- **WHEN** meeting audio is transcribed and speaker attribution is available
|
||||
- **THEN** the transcript contains ordered speaker-attributed segments
|
||||
|
||||
#### Scenario: Speaker attribution is unavailable
|
||||
- **WHEN** meeting audio is transcribed but speaker attribution is unavailable
|
||||
- **THEN** Meeting Assistant keeps the transcript and marks speaker identity as unknown rather than discarding the meeting
|
||||
|
||||
### Requirement: Speech recognition pipelines are streamable and configurable
|
||||
Meeting Assistant SHALL route audio through a provider-independent speech recognition pipeline contract.
|
||||
|
||||
The configured pipeline SHALL be selected through DI from the ASR backend configuration and SHALL expose operations to initialize, wait for readiness, write captured audio chunks, read live transcript segments, and read the finished transcript after capture completes.
|
||||
|
||||
Reading the finished transcript SHALL accept optional pipeline options, including a requested speaker count hint.
|
||||
|
||||
For real meeting recordings, Meeting Assistant SHALL derive the requested speaker count hint from the meeting note `attendees` frontmatter when at least two attendees are listed.
|
||||
|
||||
The configured pipeline SHALL receive audio chunks as they are captured rather than requiring the recording coordinator to finish a complete meeting audio file first.
|
||||
|
||||
#### Scenario: Streaming provider receives live audio
|
||||
- **WHEN** recording mode is active and the combined audio source emits a chunk
|
||||
- **THEN** Meeting Assistant can pass that chunk to the configured speech recognition pipeline before the audio source completes
|
||||
|
||||
#### Scenario: Pipeline readiness is separated from recording capture
|
||||
- **WHEN** recording mode starts before the configured speech recognition backend is ready
|
||||
- **THEN** Meeting Assistant starts audio capture immediately and lets the pipeline finish readiness before it emits live transcript output
|
||||
|
||||
#### Scenario: Pipeline is changed by configuration
|
||||
- **WHEN** the configured ASR backend setting is changed
|
||||
- **THEN** Meeting Assistant selects the configured speech recognition pipeline without changing recording control or transcript persistence code
|
||||
|
||||
#### Scenario: ASR backend is tested with a WAV file
|
||||
- **WHEN** the user submits a local PCM WAV file to the ASR diagnostic endpoint
|
||||
- **THEN** Meeting Assistant streams that file through the configured speech recognition pipeline and returns the emitted live transcript segments
|
||||
|
||||
#### Scenario: ASR diagnostic backend is unavailable
|
||||
- **WHEN** the configured speech recognition pipeline fails while processing the diagnostic WAV file
|
||||
- **THEN** Meeting Assistant returns a diagnostic backend failure response instead of hiding the pipeline error
|
||||
|
||||
#### Scenario: Final diarization backend is tested with a WAV file
|
||||
- **WHEN** the user submits a local PCM WAV file to the final diarization diagnostic endpoint
|
||||
- **THEN** Meeting Assistant streams the file through the configured speech recognition pipeline and returns the finished transcript segments
|
||||
|
||||
#### Scenario: Final diarization receives a speaker count hint
|
||||
- **WHEN** the user submits a local PCM WAV file and `numSpeakers` to the final diarization diagnostic endpoint
|
||||
- **THEN** Meeting Assistant passes the requested speaker count to the configured speech recognition pipeline as a finished-transcript option
|
||||
|
||||
#### Scenario: Meeting attendee count provides final speaker hint
|
||||
- **WHEN** a meeting note has two or more entries in `attendees`
|
||||
- **THEN** Meeting Assistant passes that attendee count to the configured speech recognition pipeline as the finished-transcript speaker count hint
|
||||
|
||||
#### Scenario: Empty or single-attendee meeting note has no final speaker hint
|
||||
- **WHEN** a meeting note has zero or one entry in `attendees`
|
||||
- **THEN** Meeting Assistant leaves the finished-transcript speaker count hint unset
|
||||
|
||||
### Requirement: Version 1 keeps local Whisper transcription available
|
||||
Meeting Assistant SHALL provide a local Whisper speech recognition pipeline as a fallback ASR backend for version 1.
|
||||
|
||||
The local Whisper adapter SHALL use configurable model path, language, and streaming window settings.
|
||||
|
||||
When local Whisper is the configured speech recognition pipeline, Meeting Assistant SHALL be able to run a final pyannote diarization pass over the temporary mixed audio and either assign pyannote speaker turns back to live Whisper transcript segments by timestamp overlap or build finished transcript segments from pyannote speaker turns.
|
||||
|
||||
The pyannote finalization backend SHALL allow configuration to use either pyannote's normal `speaker_diarization` annotation or pyannote's `exclusive_speaker_diarization` annotation.
|
||||
|
||||
When a speaker count hint is provided through the speech recognition pipeline options, the pyannote finalization backend SHALL pass it to pyannote as `num_speakers`.
|
||||
|
||||
The pyannote finalization backend SHALL use a reusable runtime image for Python dependencies and a persistent configured cache folder for Hugging Face models, torch artifacts, and pip artifacts so repeated finalization runs do not redownload the diarization model or reinstall the full Python dependency stack.
|
||||
|
||||
#### Scenario: Local Whisper model is configured
|
||||
- **WHEN** the configured provider is `whisper-local`
|
||||
- **THEN** Meeting Assistant uses the configured local Whisper model path to transcribe incoming audio windows
|
||||
|
||||
#### Scenario: Local Whisper transcript is finalized with pyannote speaker turns
|
||||
- **WHEN** the configured provider is `whisper-local`, live Whisper transcript segments exist, and pyannote returns speaker turns for the temporary recording
|
||||
- **THEN** Meeting Assistant rewrites the transcript with the original Whisper text and the best-overlap pyannote speaker label for each segment
|
||||
|
||||
#### Scenario: Local Whisper transcript is segmented by pyannote speaker turns
|
||||
- **WHEN** the configured provider is `whisper-local`, pyannote turn alignment is configured, live Whisper transcript segments exist, and pyannote returns speaker turns for the temporary recording
|
||||
- **THEN** Meeting Assistant rewrites the transcript as ordered pyannote speaker-turn segments with matching Whisper text assigned to each turn
|
||||
|
||||
#### Scenario: Local Whisper uses exclusive pyannote diarization
|
||||
- **WHEN** the configured provider is `whisper-local` and exclusive pyannote annotation is configured
|
||||
- **THEN** Meeting Assistant reads speaker turns from pyannote's `exclusive_speaker_diarization` output
|
||||
|
||||
#### Scenario: Local Whisper pyannote runtime cache is reused
|
||||
- **WHEN** local Whisper finalization runs pyannote more than once
|
||||
- **THEN** Meeting Assistant uses the configured reusable pyannote runtime image and persistent pyannote model cache instead of treating each run as a clean download environment
|
||||
|
||||
#### Scenario: Local Whisper pyannote diarization is unavailable
|
||||
- **WHEN** the configured provider is `whisper-local` but pyannote is disabled, has no configured token, or returns no speaker turns
|
||||
- **THEN** Meeting Assistant keeps the live Whisper transcript instead of deleting it
|
||||
|
||||
### Requirement: Azure Speech can provide streaming transcription
|
||||
Meeting Assistant SHALL provide an Azure Speech speech recognition pipeline that streams captured PCM audio to Azure AI Speech through the Azure Speech SDK conversation transcription API.
|
||||
|
||||
The Azure Speech adapter SHALL use configurable endpoint, region, language, key, and key environment variable settings.
|
||||
|
||||
The Azure Speech adapter SHALL support configurable diarization of intermediate conversation transcription results.
|
||||
|
||||
The Azure Speech adapter SHALL bound recognizer shutdown after the input audio stream is closed.
|
||||
|
||||
When Azure Speech is the configured speech recognition pipeline, Meeting Assistant SHALL emit live transcript segments from Azure conversation transcription events with Azure speaker IDs when available.
|
||||
|
||||
When Azure Speech is the configured speech recognition pipeline, Meeting Assistant SHALL use the live Azure conversation transcription segments as the finished transcript without running pyannote finalization.
|
||||
|
||||
#### Scenario: Azure Speech pipeline is configured
|
||||
- **WHEN** the configured provider is `azure-speech`
|
||||
- **THEN** Meeting Assistant streams captured PCM chunks to Azure AI Speech conversation transcription through the configured speech recognition pipeline without changing recording control or transcript persistence code
|
||||
|
||||
#### Scenario: Azure Speech returns speaker IDs
|
||||
- **WHEN** Azure Speech conversation transcription returns transcript text with speaker identity
|
||||
- **THEN** Meeting Assistant emits ordered transcript segments with those Azure speaker identities
|
||||
|
||||
#### Scenario: Azure Speech key is resolved from environment
|
||||
- **WHEN** Azure Speech is configured with `KeyEnv`
|
||||
- **THEN** Meeting Assistant reads the Azure Speech key from that environment variable
|
||||
|
||||
#### Scenario: Azure Speech stream shutdown is bounded
|
||||
- **WHEN** the captured audio stream has ended
|
||||
- **THEN** Meeting Assistant stops Azure Speech continuous recognition within the configured stop timeout instead of waiting indefinitely
|
||||
|
||||
#### Scenario: Azure Speech transcript is finalized from live conversation segments
|
||||
- **WHEN** the configured provider is `azure-speech` and live Azure transcript segments exist
|
||||
- **THEN** Meeting Assistant uses the live Azure conversation transcript segments as the finished transcript
|
||||
|
||||
### Requirement: FunASR can provide speaker-attributed streaming transcription
|
||||
Meeting Assistant SHALL provide a FunASR speech recognition pipeline that streams PCM audio to a configured FunASR WebSocket endpoint.
|
||||
|
||||
The FunASR provider SHALL preserve speaker attribution from FunASR response fields when they are present and SHALL fall back to `Unknown` when no speaker field is returned.
|
||||
|
||||
Meeting Assistant SHALL write the mixed recording audio only as a temporary processing artifact while streaming transcription is active. After capture stops and the streaming provider has drained already-captured audio, Meeting Assistant SHALL be able to run a final FunASR Python `AutoModel` pass with VAD, punctuation, and CAMPPlus speaker modeling over the temporary audio. When that final pass returns sentence-level speaker labels, Meeting Assistant SHALL rewrite the transcript markdown with the final speaker-attributed sentence segments.
|
||||
|
||||
Meeting Assistant SHALL delete temporary recording audio after the run completes, whether final diarization succeeds, fails, or produces no speaker output. On application startup, Meeting Assistant SHALL delete stale temporary recording audio left behind by previous interrupted runs.
|
||||
|
||||
When configured, Meeting Assistant SHALL manage a local FunASR backend lifecycle for the FunASR provider by starting a non-blocking backend warm-up when the application starts, installing the configured streaming runtime image, preparing the mounted model and hotword folder, starting the backend, keeping the container alive for the backgrounded FunASR server process, and stopping it when the application stops. Recording SHALL be able to start while the backend is still warming up; captured audio SHALL be queued and streamed once the backend is healthy. Backend installation, startup commands, and WebSocket readiness SHALL be bounded by configurable timeouts so startup failures are diagnosable.
|
||||
|
||||
#### Scenario: FunASR pipeline is configured
|
||||
- **WHEN** the configured provider is `funasr`
|
||||
- **THEN** Meeting Assistant streams captured PCM chunks to the configured FunASR WebSocket endpoint through the configured speech recognition pipeline without changing recording control or transcript persistence code
|
||||
|
||||
#### Scenario: FunASR returns speaker fields
|
||||
- **WHEN** FunASR returns transcript text with speaker identity fields
|
||||
- **THEN** Meeting Assistant emits ordered transcript segments with those speaker identities
|
||||
|
||||
#### Scenario: Final FunASR diarization rewrites transcript
|
||||
- **WHEN** recording stops after mixed audio was temporarily stored and the final FunASR diarization pass returns sentence-level speaker labels
|
||||
- **THEN** Meeting Assistant rewrites the transcript markdown with the final speaker-attributed sentence segments
|
||||
|
||||
#### Scenario: Final FunASR diarization has no speaker output
|
||||
- **WHEN** recording stops but the final diarization pass is disabled or returns no sentence-level speaker labels
|
||||
- **THEN** Meeting Assistant keeps the live streaming transcript instead of deleting it
|
||||
|
||||
#### Scenario: Temporary recording audio is deleted after completion
|
||||
- **WHEN** a recording run completes after using temporary audio for final processing
|
||||
- **THEN** Meeting Assistant deletes the temporary audio file from disk
|
||||
|
||||
#### Scenario: Stale temporary recordings are deleted on startup
|
||||
- **WHEN** Meeting Assistant starts and stale temporary recording files exist from a previous interrupted run
|
||||
- **THEN** Meeting Assistant deletes those stale temporary recording files before accepting new recordings
|
||||
|
||||
#### Scenario: Managed FunASR backend starts with the application
|
||||
- **WHEN** the configured provider is `funasr` and managed backend startup is enabled
|
||||
- **THEN** Meeting Assistant begins installing the configured FunASR backend image and starting the two-pass streaming backend without blocking recording startup
|
||||
|
||||
#### Scenario: Recording starts while managed FunASR backend is warming up
|
||||
- **WHEN** recording starts before the managed FunASR backend is ready
|
||||
- **THEN** Meeting Assistant starts audio capture immediately and queues captured audio until the backend WebSocket is healthy
|
||||
|
||||
#### Scenario: Managed FunASR backend installation stalls
|
||||
- **WHEN** a managed FunASR backend command exceeds its configured timeout
|
||||
- **THEN** Meeting Assistant reports a backend startup timeout instead of waiting indefinitely
|
||||
|
||||
#### Scenario: Managed FunASR backend WebSocket is not healthy
|
||||
- **WHEN** the managed backend container has started but the FunASR WebSocket endpoint is not accepting sessions
|
||||
- **THEN** Meeting Assistant continues waiting for backend readiness until the configured startup timeout is reached
|
||||
|
||||
#### Scenario: Managed FunASR backend stops with the application
|
||||
- **WHEN** Meeting Assistant stops after starting a managed FunASR backend
|
||||
- **THEN** Meeting Assistant stops the managed backend process or container
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
# project-knowledge Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change define-meeting-assistant-v1. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Meeting Assistant maintains project knowledge
|
||||
Meeting Assistant SHALL maintain project knowledge based on meeting content and future generic information sources.
|
||||
|
||||
Project knowledge SHALL support retrieval by project context and relevant keywords.
|
||||
|
||||
Project knowledge SHALL be stored under the configured projects folder, where each direct subfolder is a project identified by its folder name.
|
||||
|
||||
#### Scenario: Meeting has project context
|
||||
- **WHEN** a meeting note identifies a project
|
||||
- **THEN** Meeting Assistant can associate transcript-derived and summary-derived knowledge with that project
|
||||
|
||||
#### Scenario: Meeting binds project folders
|
||||
- **WHEN** the meeting note `projects` frontmatter lists configured project subfolder names
|
||||
- **THEN** Meeting Assistant treats those subfolders as the projects available to meeting agents
|
||||
|
||||
#### Scenario: Agent retrieves project context
|
||||
- **WHEN** an agent needs context for a meeting or follow-up task
|
||||
- **THEN** Meeting Assistant can retrieve relevant project knowledge and source references
|
||||
|
||||
Reference in New Issue
Block a user