Public Access
Implement meeting assistant v1
This commit is contained in:
@@ -0,0 +1,15 @@
|
||||
## Why
|
||||
|
||||
Meeting Assistant currently focuses V1 runtime support on Windows capture and Outlook enrichment. We need a separate future change to define macOS and Linux build targets without expanding the V1 scope.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Define supported macOS and Linux build targets for Meeting Assistant.
|
||||
- Keep platform-specific capture, hotkey, calendar, and shell integrations isolated per target.
|
||||
- Ensure non-Windows builds compile without Windows COM, WASAPI, or user32 dependencies.
|
||||
- Define platform-specific test selection so Windows-only tests do not run on macOS/Linux and future macOS/Linux tests do not run on incompatible targets.
|
||||
|
||||
## Impact
|
||||
|
||||
- Affected specs: build-targets
|
||||
- Affected code: project target frameworks, platform service registration, platform integration tests
|
||||
@@ -0,0 +1,19 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Meeting Assistant defines macOS and Linux build targets
|
||||
Meeting Assistant SHALL define macOS and Linux build targets separately from the Windows build target.
|
||||
|
||||
Windows-only integrations such as WASAPI capture, global hotkeys through user32, and Outlook Classic COM SHALL NOT be compiled into macOS or Linux targets.
|
||||
|
||||
#### Scenario: Non-Windows target compiles
|
||||
- **WHEN** Meeting Assistant is built for macOS or Linux
|
||||
- **THEN** Windows-only source files and package dependencies are excluded
|
||||
- **AND** the application compiles with portable service registrations or explicit unsupported implementations
|
||||
|
||||
#### Scenario: Windows target keeps Windows integrations
|
||||
- **WHEN** Meeting Assistant is built for Windows
|
||||
- **THEN** WASAPI audio capture, Windows global hotkeys, and Outlook Classic COM enrichment remain available
|
||||
|
||||
#### Scenario: Platform-specific tests are gated
|
||||
- **WHEN** tests run for a platform target
|
||||
- **THEN** tests for incompatible platform APIs are excluded or compiled only for their matching target
|
||||
@@ -0,0 +1,11 @@
|
||||
## 1. Build Targets
|
||||
|
||||
- [ ] 1.1 Add explicit macOS and Linux target framework/runtime build coverage.
|
||||
- [ ] 1.2 Keep Windows-only implementations excluded from macOS and Linux builds.
|
||||
- [ ] 1.3 Add platform service registration for unsupported or future macOS/Linux capture and hotkey implementations.
|
||||
|
||||
## 2. Tests
|
||||
|
||||
- [ ] 2.1 Split platform-specific tests by compilation target or runtime guard.
|
||||
- [ ] 2.2 Add CI/build commands that verify Windows, macOS, and Linux compile targets.
|
||||
- [ ] 2.3 Add at least one portable compile test that proves Windows-only packages are not required by non-Windows builds.
|
||||
@@ -0,0 +1,77 @@
|
||||
## Context
|
||||
|
||||
Meeting Assistant is a local/server-side .NET 10 application that helps capture and process meetings. It should work for any meeting format, including in-person conversations, and should not require direct APIs from meeting products such as Teams or Zoom.
|
||||
|
||||
The central artifact is an Obsidian markdown note created before transcription begins. The user can add attendees, project context, and live notes while the meeting is happening. Meeting Assistant can add discovered context to the linked assistant-context note, then later append or link summaries, next steps, and knowledge base updates.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- Establish a .NET 10 application skeleton and OpenSpec workflow.
|
||||
- Model a platform-independent meeting session flow.
|
||||
- Toggle recording/transcription mode through a configurable hotkey.
|
||||
- Capture microphone input and computer output for transcription.
|
||||
- Stream audio into a provider-independent speech recognition pipeline.
|
||||
- Use FunASR for speaker-attributed transcription, while keeping local Whisper as a fallback provider.
|
||||
- Write transcript text into the configured vault folder.
|
||||
- Create a meeting note before transcription starts.
|
||||
- Preserve user-authored notes as first-class input to summarization.
|
||||
- Transcribe meeting audio with speaker attribution.
|
||||
- Generate summaries, decisions, and next steps.
|
||||
- Maintain project knowledge from meeting output and additional future sources.
|
||||
- Let agents retrieve project context and keyword context.
|
||||
- Allow agents to update files in existing project folders through explicit overwrite, replace, and insert tool modes.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- Require a Teams, Zoom, or calendar-specific integration in version 1.
|
||||
- Require cloud transcription APIs.
|
||||
- Define the final Obsidian vault path or note template completely in this first change.
|
||||
- Implement unrestricted autonomous file edits.
|
||||
|
||||
## Decisions
|
||||
|
||||
1. Build the source application as ASP.NET Core on .NET 10.
|
||||
- Rationale: It matches the requested runtime and keeps the application service-oriented.
|
||||
|
||||
2. Make the Obsidian markdown note the session anchor.
|
||||
- Rationale: The user can enrich the meeting context before and during transcription, and agents can work from a durable human-readable artifact.
|
||||
|
||||
3. Treat meeting-platform APIs as optional augmentation.
|
||||
- Rationale: The assistant must be usable for in-person meetings and for platforms where no integration is available or desired.
|
||||
|
||||
4. Keep project knowledge as a generic capability.
|
||||
- Rationale: Meeting-derived knowledge should later combine with other sources without coupling the design to transcripts only.
|
||||
|
||||
5. Keep project file mutation simple in version 1.
|
||||
- Rationale: Agents may update files in existing project folders, but project creation and scaffolded project templates are future work.
|
||||
|
||||
6. Use the normal .NET options configuration model for hotkey, vault, recording, and provider settings.
|
||||
- Rationale: These settings will need machine-specific values and should work through `appsettings.json`, environment variables, and future deployment configuration.
|
||||
|
||||
7. Model ASR backends as speech recognition pipelines.
|
||||
- Rationale: The application must not assume file-only transcription or split live transcription from final diarization at the coordinator boundary. A pipeline can initialize, wait for readiness, accept captured audio chunks, emit live transcript segments, and produce a finished transcript. FunASR consumes PCM over WebSocket, Whisper.NET remains available as a local windowed fallback with pyannote post-processing, and the app can replace either backend later without changing recording control or persistence.
|
||||
|
||||
8. Capture computer output through WASAPI loopback and microphone input through WASAPI capture, then mix PCM chunks before transcription.
|
||||
- Rationale: WASAPI loopback is the Windows-supported way to capture the system mix, and tests can cover the mixer without depending on live devices.
|
||||
|
||||
9. Gate Windows-only capture, hotkey, and Outlook Classic COM enrichment behind the Windows target.
|
||||
- Rationale: The portable build should continue to compile without Windows APIs, while the Windows build can augment meeting notes from the current Teams appointment when Outlook Classic is available.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [Risk] Speaker attribution quality depends on the selected transcription pipeline. -> Mitigation: keep attribution requirements observable and provider-independent.
|
||||
- [Risk] FunASR speaker metadata depends on the runtime/model configuration and may be absent. -> Mitigation: preserve speaker fields when returned and fall back to `Unknown` instead of dropping transcript text.
|
||||
- [Risk] Whisper.NET is windowed rather than a dedicated real-time recognizer. -> Mitigation: keep it as a configurable fallback and make the window size configurable.
|
||||
- [Risk] WASAPI loopback may not emit chunks when no system audio is playing. -> Mitigation: isolate the system-audio source so silence insertion can be added without changing the coordinator.
|
||||
- [Risk] The Obsidian note may become too dense. -> Mitigation: separate metadata, user notes, assistant context, transcript references, and generated outputs into clear sections.
|
||||
- [Risk] Agent file updates could affect the wrong project. -> Mitigation: restrict writes to existing project folders and prevent target paths from escaping the selected project.
|
||||
- [Risk] Retrieval can surface stale or irrelevant project context. -> Mitigation: include source references and update timestamps in retrieved context.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Should transcript text files eventually be merged into the meeting note, linked from it, or remain separate artifacts?
|
||||
- What metadata fields are required before transcription can start?
|
||||
- Which storage backend should hold embeddings and project knowledge?
|
||||
- What approval model should govern agent project file edits?
|
||||
+10
-2
@@ -8,22 +8,30 @@ Meeting Assistant should provide a meeting capture workflow that is not tied to
|
||||
- Establish OpenSpec as the source of truth for behavior changes.
|
||||
- Define the version 1 meeting flow around Obsidian markdown notes created before transcription starts.
|
||||
- Require the core flow to support in-person meetings and not depend on Teams, Zoom, or similar APIs.
|
||||
- Add a configurable global hotkey that toggles recording/transcription mode.
|
||||
- Capture microphone input and computer output into one transcription audio stream.
|
||||
- Require speech recognition pipelines to accept streamable audio, with FunASR configured for speaker-attributed transcription and local Whisper plus pyannote kept as a fallback pipeline.
|
||||
- Store the live transcript in a text file under the configured vault folder.
|
||||
- Add requirements for speaker-attributed transcription, summaries, decisions, and next steps.
|
||||
- Define project context capture and retrieval as first-class capabilities.
|
||||
- Define a guarded agent tool model for project lookup and controlled project file changes.
|
||||
- Define agent tools for project lookup and project file changes in existing projects.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `meeting-session`: creation and lifecycle of meeting notes and metadata.
|
||||
- `meeting-recording`: hotkey-controlled recording mode, mixed audio capture, and transcript persistence.
|
||||
- `meeting-transcription`: platform-independent transcription with speaker attribution.
|
||||
- `meeting-summary`: generated summaries, decisions, and next steps.
|
||||
- `project-knowledge`: project context, retrieval, and knowledge base updates from meetings.
|
||||
- `agent-project-tools`: agent tools for lookup, retrieval, and controlled project file updates.
|
||||
- `agent-project-tools`: agent tools for lookup, retrieval, and project file updates.
|
||||
|
||||
## Impact
|
||||
|
||||
- New .NET 10 solution, service project, tests, README, and OpenSpec structure.
|
||||
- New recording/transcription services using configurable .NET options.
|
||||
- NAudio is used for Windows microphone and WASAPI loopback capture.
|
||||
- FunASR is used as the configured speaker-attribution pipeline, and Whisper.NET plus pyannote remains available as a local transcription fallback.
|
||||
- Version 1 behavior is documented before feature implementation.
|
||||
- Future platform integrations remain optional augmentations instead of hard dependencies.
|
||||
+57
@@ -0,0 +1,57 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Agents can use project context tools
|
||||
Meeting Assistant SHALL expose tools that allow agents to look up project information, retrieve keyword-relevant context, and inspect meeting-derived knowledge.
|
||||
|
||||
Project tools SHALL treat each direct subfolder of the configured projects folder as one project. A meeting note binds projects by listing those subfolder names in the `projects` frontmatter field.
|
||||
|
||||
The summary agent SHALL expose these project tools:
|
||||
|
||||
- `list_projects`
|
||||
- `list_projectfiles`
|
||||
- `read_projectfile`
|
||||
- `write_projectfile`
|
||||
- `search`
|
||||
|
||||
#### Scenario: Agent looks up meeting project context
|
||||
- **WHEN** a meeting note identifies a project
|
||||
- **THEN** the agent can retrieve relevant project context through Meeting Assistant tools
|
||||
|
||||
#### Scenario: Agent lists projects bound to the meeting
|
||||
- **WHEN** the meeting note frontmatter lists project folder names
|
||||
- **THEN** `list_projects` returns the matching configured project subfolders
|
||||
|
||||
#### Scenario: Agent reads a clamped project file range
|
||||
- **WHEN** the agent reads a project file with optional line bounds outside the file length
|
||||
- **THEN** `read_projectfile` clamps the requested range to the available file lines without failing
|
||||
|
||||
#### Scenario: Agent searches project knowledge
|
||||
- **WHEN** the agent calls `search` with ripgrep syntax keywords
|
||||
- **THEN** Meeting Assistant runs the search against the requested projects or the meeting-bound projects and returns matches as `filename:line text`
|
||||
|
||||
### Requirement: Agents can write files in existing projects
|
||||
Meeting Assistant SHALL expose a `write_projectfile` tool that can create or update files inside an existing project folder.
|
||||
|
||||
The target project SHALL be an existing direct subfolder of the configured projects folder. The target file path SHALL stay inside that project folder.
|
||||
|
||||
When no line edit arguments are supplied, `write_projectfile` SHALL overwrite the whole file and SHALL create the file when it does not exist.
|
||||
|
||||
When both `from` and `to` are supplied, `write_projectfile` SHALL replace the inclusive 1-based line range with the supplied content.
|
||||
|
||||
When `insert` is supplied, `write_projectfile` SHALL insert the supplied content at that 1-based line position.
|
||||
|
||||
#### Scenario: Project file is overwritten or created
|
||||
- **WHEN** the agent writes a project file without line edit arguments
|
||||
- **THEN** Meeting Assistant writes the supplied content as the complete file content
|
||||
|
||||
#### Scenario: Project file line range is replaced
|
||||
- **WHEN** the agent writes a project file with `from` and `to` line numbers
|
||||
- **THEN** Meeting Assistant replaces the inclusive line range with the supplied content
|
||||
|
||||
#### Scenario: Project file content is inserted
|
||||
- **WHEN** the agent writes a project file with an `insert` line number
|
||||
- **THEN** Meeting Assistant inserts the supplied content at that line position
|
||||
|
||||
#### Scenario: Project file write target is invalid
|
||||
- **WHEN** the target project is missing or the target path escapes the project folder
|
||||
- **THEN** Meeting Assistant refuses the project file write
|
||||
+41
@@ -0,0 +1,41 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Recording mode is controlled by a configurable hotkey
|
||||
Meeting Assistant SHALL use normal .NET configuration to define the global hotkey that toggles recording/transcription mode.
|
||||
|
||||
#### Scenario: Hotkey starts recording
|
||||
- **WHEN** the configured hotkey is pressed while recording mode is inactive
|
||||
- **THEN** Meeting Assistant starts recording/transcription mode
|
||||
|
||||
#### Scenario: Hotkey stops recording
|
||||
- **WHEN** the configured hotkey is pressed while recording mode is active
|
||||
- **THEN** Meeting Assistant stops recording/transcription mode
|
||||
|
||||
### Requirement: Recording mode captures microphone and computer output
|
||||
Meeting Assistant SHALL capture microphone input and computer output and combine them into one audio stream for transcription.
|
||||
|
||||
#### Scenario: Both sources produce audio
|
||||
- **WHEN** microphone and computer output audio chunks are available
|
||||
- **THEN** Meeting Assistant mixes them into one PCM stream before transcription
|
||||
|
||||
#### Scenario: Device-level capture cannot be verified in tests
|
||||
- **WHEN** automated tests run without live audio devices
|
||||
- **THEN** Meeting Assistant verifies the audio mixer through deterministic source abstractions rather than depending on physical microphone or speaker devices
|
||||
|
||||
### Requirement: Stopping recording drains captured audio through transcription
|
||||
Meeting Assistant SHALL stop capturing new audio when recording mode is stopped, but it SHALL allow already captured audio to finish running through the configured speech recognition pipeline before the recording session completes.
|
||||
|
||||
#### Scenario: Hotkey stops recording with buffered audio
|
||||
- **WHEN** the user presses the hotkey to stop recording while captured audio is still buffered for transcription
|
||||
- **THEN** Meeting Assistant stops capturing new audio and appends transcription output for the already captured audio before reporting the recording stopped
|
||||
|
||||
### Requirement: Transcripts are written to the configured vault folder
|
||||
Meeting Assistant SHALL write live transcript text to a markdown file in the configured vault folder.
|
||||
|
||||
#### Scenario: Recording starts
|
||||
- **WHEN** recording mode starts
|
||||
- **THEN** Meeting Assistant creates the configured vault folder if necessary and creates a transcript markdown file
|
||||
|
||||
#### Scenario: Transcription segment arrives
|
||||
- **WHEN** the speech recognition pipeline emits a live transcript segment
|
||||
- **THEN** Meeting Assistant appends that segment to the active transcript text file
|
||||
+105
@@ -0,0 +1,105 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Meeting Assistant creates a note before transcription
|
||||
Meeting Assistant SHALL create an Obsidian markdown note for a meeting session before transcription starts.
|
||||
|
||||
The note SHALL be the durable anchor for meeting metadata, user notes, assistant-discovered context, transcript references, summaries, decisions, and next steps.
|
||||
|
||||
#### Scenario: Session starts
|
||||
- **WHEN** the user starts a new meeting session
|
||||
- **THEN** Meeting Assistant creates a markdown note in the configured meetings folder before starting transcription
|
||||
|
||||
#### Scenario: Recording starts from hotkey
|
||||
- **WHEN** the user presses the recording hotkey while recording is inactive
|
||||
- **THEN** Meeting Assistant creates the transcript file, creates the meeting note with links to the transcript, assistant context, and summary notes, and opens the meeting note through an `obsidian://` system command
|
||||
|
||||
#### Scenario: User adds context before transcription
|
||||
- **WHEN** the note exists before transcription starts
|
||||
- **THEN** the user can add attendees, project context, and other notes to that file
|
||||
|
||||
### Requirement: Vault locations are configurable
|
||||
Meeting Assistant SHALL expose separate configuration settings for transcript files, meeting notes, assistant context, generated summaries, project knowledge, and dictation words.
|
||||
|
||||
The dictation words location SHALL point to a word-list artifact that speech recognition pipelines MAY use as vocabulary hints when supported.
|
||||
|
||||
#### Scenario: Transcript and note locations are configured separately
|
||||
- **WHEN** Meeting Assistant starts with configured vault locations
|
||||
- **THEN** transcripts, meeting notes, assistant context, generated summaries, project knowledge, and dictation words each have an independently configured path
|
||||
|
||||
#### Scenario: Provider supports dictation words
|
||||
- **WHEN** the configured speech recognition pipeline supports vocabulary hints
|
||||
- **THEN** Meeting Assistant can provide words from the configured dictation words path without changing the other vault locations
|
||||
|
||||
### Requirement: Meeting note template is coded and round-trippable
|
||||
Meeting Assistant SHALL own a coded meeting note template and a store that can save and read meeting notes.
|
||||
|
||||
The meeting note frontmatter SHALL include:
|
||||
|
||||
- `start_time` as the meeting recording start timestamp
|
||||
- `end_time` as the timestamp when transcript processing stops, empty while the meeting is active
|
||||
- `attendees` as an array of display strings, where email is optional and may use a format such as `Mike <m.ike@ibm.com>`
|
||||
- `projects` as an array because one meeting can cover multiple projects
|
||||
- `transcript` as text containing a markdown link to the transcript note
|
||||
- `assistant_context` as text containing a markdown link to the assistant context note
|
||||
- `summary` as text containing a markdown link to the generated summary note once Meeting Assistant creates it
|
||||
|
||||
The user notes SHALL be the markdown body. Decisions and next steps SHALL NOT be required as frontmatter fields.
|
||||
|
||||
#### Scenario: Meeting note is saved
|
||||
- **WHEN** Meeting Assistant saves a meeting note
|
||||
- **THEN** it writes the coded frontmatter template and the user notes body to a markdown file in the configured meeting notes folder
|
||||
|
||||
#### Scenario: Meeting note links generated artifacts
|
||||
- **WHEN** Meeting Assistant creates a meeting note for a new recording
|
||||
- **THEN** the note frontmatter links to the configured transcript, assistant context, and summary note locations
|
||||
|
||||
#### Scenario: Generated artifacts link back to each other
|
||||
- **WHEN** Meeting Assistant creates transcript, assistant context, and summary artifacts
|
||||
- **THEN** those artifacts use frontmatter links for the meeting note, transcript note, assistant context note, and summary note where applicable
|
||||
|
||||
#### Scenario: Meeting note records session times
|
||||
- **WHEN** Meeting Assistant creates a meeting note for a new recording
|
||||
- **THEN** the note frontmatter contains `start_time`
|
||||
- **AND** when transcript processing stops, Meeting Assistant writes `end_time` to the same note frontmatter
|
||||
- **AND** transcript and summary frontmatter copy the meeting note title, `start_time`, and `end_time`
|
||||
|
||||
#### Scenario: Meeting note is read
|
||||
- **WHEN** Meeting Assistant reads a meeting note created from the coded template
|
||||
- **THEN** it returns the frontmatter fields and user notes body without losing attendees, projects, transcript link, assistant context link, or summary
|
||||
|
||||
#### Scenario: Meeting note frontmatter is updated
|
||||
- **WHEN** Meeting Assistant reads a meeting note, changes frontmatter, and writes the note back
|
||||
- **THEN** the existing user-authored markdown body remains unchanged
|
||||
|
||||
### Requirement: Meeting notes preserve user-authored content
|
||||
Meeting Assistant SHALL preserve user-authored meeting notes and include them as input when generating summaries, decisions, and next steps.
|
||||
|
||||
#### Scenario: User writes notes during the meeting
|
||||
- **WHEN** the user updates the meeting note while the meeting is active
|
||||
- **THEN** Meeting Assistant keeps that content and incorporates it into generated meeting outputs
|
||||
|
||||
### Requirement: Meeting sessions are platform independent
|
||||
Meeting Assistant SHALL support meeting sessions that do not originate from Teams, Zoom, or any other meeting software API.
|
||||
|
||||
Meeting software integrations MAY augment the meeting experience, but they SHALL NOT be required for the primary meeting capture, transcription, or summary flow.
|
||||
|
||||
#### Scenario: In-person meeting is captured
|
||||
- **WHEN** the user starts transcription for an in-person meeting
|
||||
- **THEN** Meeting Assistant captures and processes the meeting without requiring a meeting platform integration
|
||||
|
||||
### Requirement: Windows Outlook enrichment is optional
|
||||
Meeting Assistant SHALL gate Outlook Classic COM enrichment behind the Windows compilation target.
|
||||
|
||||
When the Windows build starts a meeting and Outlook Classic has exactly one current Teams appointment, Meeting Assistant SHALL copy the appointment title and attendees to the meeting note and copy the appointment agenda to the assistant-context frontmatter.
|
||||
|
||||
The agenda SHALL be extracted from the appointment body content before the Teams join separator or Teams join text.
|
||||
|
||||
#### Scenario: Current Teams appointment enriches meeting artifacts
|
||||
- **WHEN** a Windows build starts a meeting while Outlook Classic exposes exactly one current Teams appointment
|
||||
- **THEN** Meeting Assistant uses the appointment subject as the meeting title
|
||||
- **AND** writes the appointment attendees into meeting note frontmatter
|
||||
- **AND** writes the appointment agenda into assistant-context frontmatter
|
||||
|
||||
#### Scenario: Outlook is unavailable or ambiguous
|
||||
- **WHEN** Outlook Classic is unavailable or more than one current Teams appointment is found
|
||||
- **THEN** Meeting Assistant starts the recording with the default generated meeting title and empty agenda
|
||||
+98
@@ -0,0 +1,98 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Meeting Assistant generates meeting outputs
|
||||
Meeting Assistant SHALL generate a summary, decisions, and next steps from the meeting transcript, metadata, user notes, and assistant-discovered context.
|
||||
|
||||
Meeting Assistant SHALL use a Microsoft Agent Framework pipeline for the first summary implementation. The pipeline SHALL use a configurable OpenAI-compatible endpoint, model, and API key source, including support for a direct key or an environment variable name.
|
||||
|
||||
The summary pipeline SHALL retry transient model endpoint failures according to configurable reconnection attempts and delay settings.
|
||||
|
||||
The summary pipeline SHALL track context-window usage for the configured model using response usage when available and request-size estimates otherwise. The context-window limit, maximum output reserve, compaction enablement, compaction threshold, and Responses compact endpoint path SHALL be configurable.
|
||||
|
||||
When only the configured remaining context ratio is available, the summary pipeline SHALL try to compact the conversation through the configured OpenAI-compatible `POST /v1/responses/compact` endpoint. If that endpoint is unavailable or returns invalid data, the pipeline SHALL fall back to Microsoft Agent Framework compaction.
|
||||
|
||||
Fallback compaction SHALL become increasingly aggressive: collapse old tool results, summarize older conversation spans, keep only the last four user turns, and drop oldest groups if still over budget.
|
||||
|
||||
When summary generation fails after retries, Meeting Assistant SHALL write a markdown failure report to the configured summary note path. The failure report SHALL include a clickable retry link, error details, and the meeting artifact paths needed to diagnose or retry the run.
|
||||
|
||||
After transcript processing finishes for a recording, Meeting Assistant SHALL automatically invoke the summary pipeline for that meeting.
|
||||
|
||||
Meeting Assistant SHALL expose an API operation to retry summary generation for a given summary note path. The retry operation SHALL resolve the linked meeting artifacts from the meeting note frontmatter and SHALL overwrite the existing summary note with either the new summary or a new failure report.
|
||||
|
||||
The summary pipeline SHALL expose tools scoped to the current meeting:
|
||||
|
||||
- `read_meetingnote`
|
||||
- `read_transcript`
|
||||
- `read_context`
|
||||
- `read_usernotes`
|
||||
- `read_glossary`
|
||||
- `write_summary`
|
||||
- `write_context`
|
||||
- `list_projects`
|
||||
- `list_projectfiles`
|
||||
- `read_projectfile`
|
||||
- `write_projectfile`
|
||||
- `search`
|
||||
|
||||
All summary-agent read tools that return file-like content SHALL support either reading the whole file or reading a clamped inclusive 1-based line range when `from` and `to` are supplied.
|
||||
|
||||
After writing the meeting summary, the summary agent SHALL update existing project files when the meeting produced project-relevant knowledge.
|
||||
|
||||
The assistant context note SHALL keep `meeting`, `transcript`, `summary`, `state`, and `agenda` in frontmatter. The `state` value SHALL be one of `transcribing`, `speaker recognition`, `summarizing`, `finished`, or `error`.
|
||||
|
||||
The summary note SHALL keep frontmatter links to meeting artifacts and SHALL copy `title`, `start_time`, and `end_time` from the meeting note when available. If the meeting note has no title, the summary agent SHALL be able to provide a title when writing the summary.
|
||||
|
||||
The summary agent SHALL be able to read and write the assistant context body as its own notebook using overwrite, replace, and insert line modes.
|
||||
|
||||
#### Scenario: Meeting processing completes
|
||||
- **WHEN** transcription and context collection have completed
|
||||
- **THEN** Meeting Assistant generates a summary, decisions, and next steps for the meeting note
|
||||
|
||||
#### Scenario: Assistant context note is initialized
|
||||
- **WHEN** Meeting Assistant starts a meeting session
|
||||
- **THEN** it creates the assistant context note with frontmatter links to the meeting note, transcript note, and summary note
|
||||
- **AND** the assistant context frontmatter state is `transcribing`
|
||||
- **AND** the assistant context frontmatter includes `agenda`, empty when no agenda is known
|
||||
|
||||
#### Scenario: Assistant context state follows meeting processing
|
||||
- **WHEN** transcription, final speaker recognition, and summary generation progress
|
||||
- **THEN** Meeting Assistant updates assistant context frontmatter state to `transcribing`, `speaker recognition`, `summarizing`, `finished`, or `error` as appropriate
|
||||
|
||||
#### Scenario: Summary pipeline is invoked
|
||||
- **WHEN** transcript processing finishes for the current meeting
|
||||
- **THEN** the agent can read the transcript, assistant context, user notes, glossary, and bound project knowledge, can write the finished markdown summary to the configured summary note, and can update existing project files
|
||||
|
||||
#### Scenario: Summary agent reads meeting frontmatter
|
||||
- **WHEN** the summary pipeline runs
|
||||
- **THEN** the agent can read the full meeting note including frontmatter such as title, attendees, projects, start time, and end time
|
||||
- **AND** the agent can read the assistant-context frontmatter including agenda
|
||||
|
||||
#### Scenario: Summary agent reads large inputs by line range
|
||||
- **WHEN** the summary agent reads the meeting note, transcript, assistant context, user notes, glossary, or a project file with `from` and `to` line numbers
|
||||
- **THEN** Meeting Assistant returns only the clamped inclusive line range
|
||||
- **AND** omitting the line arguments returns the whole readable content
|
||||
|
||||
#### Scenario: Summary agent writes assistant context notes
|
||||
- **WHEN** the summary pipeline runs
|
||||
- **THEN** the agent can overwrite, replace lines in, or insert lines into the assistant context body while preserving assistant context frontmatter
|
||||
|
||||
#### Scenario: Transient summary model endpoint failure is retried
|
||||
- **WHEN** the model endpoint returns a retryable failure while the summary pipeline is running
|
||||
- **THEN** Meeting Assistant retries the model request until it succeeds or the configured reconnection attempts are exhausted
|
||||
|
||||
#### Scenario: Summary context window is compacted
|
||||
- **WHEN** the summary pipeline estimates that only the configured remaining context ratio is available
|
||||
- **THEN** Meeting Assistant attempts to compact the outgoing conversation with the configured Responses compact endpoint
|
||||
- **AND** if remote compaction fails, Meeting Assistant uses Microsoft Agent Framework compaction with increasingly aggressive fallback strategies
|
||||
|
||||
#### Scenario: Summary failure is written to the summary note
|
||||
- **WHEN** summary generation fails after configured retries
|
||||
- **THEN** Meeting Assistant writes a markdown failure report with a retry link to the configured summary note path
|
||||
|
||||
#### Scenario: Summary retry overwrites existing summary note
|
||||
- **WHEN** the user triggers summary retry for a configured summary note path
|
||||
- **THEN** Meeting Assistant resolves the linked meeting artifacts and overwrites the existing summary note with the retry result
|
||||
|
||||
#### Scenario: User notes conflict with transcript interpretation
|
||||
- **WHEN** user-authored notes provide context that changes the interpretation of the transcript
|
||||
- **THEN** Meeting Assistant considers those notes during summary and next-step generation
|
||||
+186
@@ -0,0 +1,186 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Meeting Assistant transcribes meetings with speaker attribution
|
||||
Meeting Assistant SHALL transcribe meeting audio and associate transcript segments with speakers when speaker attribution is available.
|
||||
|
||||
The transcription contract SHALL remain independent of the meeting source so that local audio, in-person meetings, and future meeting-platform integrations can use the same downstream processing.
|
||||
|
||||
#### Scenario: Speaker-attributed transcript is produced
|
||||
- **WHEN** meeting audio is transcribed and speaker attribution is available
|
||||
- **THEN** the transcript contains ordered speaker-attributed segments
|
||||
|
||||
#### Scenario: Speaker attribution is unavailable
|
||||
- **WHEN** meeting audio is transcribed but speaker attribution is unavailable
|
||||
- **THEN** Meeting Assistant keeps the transcript and marks speaker identity as unknown rather than discarding the meeting
|
||||
|
||||
### Requirement: Speech recognition pipelines are streamable and configurable
|
||||
Meeting Assistant SHALL route audio through a provider-independent speech recognition pipeline contract.
|
||||
|
||||
The configured pipeline SHALL be selected through DI from the ASR backend configuration and SHALL expose operations to initialize, wait for readiness, write captured audio chunks, read live transcript segments, and read the finished transcript after capture completes.
|
||||
|
||||
Reading the finished transcript SHALL accept optional pipeline options, including a requested speaker count hint.
|
||||
|
||||
For real meeting recordings, Meeting Assistant SHALL derive the requested speaker count hint from the meeting note `attendees` frontmatter when at least two attendees are listed.
|
||||
|
||||
The configured pipeline SHALL receive audio chunks as they are captured rather than requiring the recording coordinator to finish a complete meeting audio file first.
|
||||
|
||||
#### Scenario: Streaming provider receives live audio
|
||||
- **WHEN** recording mode is active and the combined audio source emits a chunk
|
||||
- **THEN** Meeting Assistant can pass that chunk to the configured speech recognition pipeline before the audio source completes
|
||||
|
||||
#### Scenario: Pipeline readiness is separated from recording capture
|
||||
- **WHEN** recording mode starts before the configured speech recognition backend is ready
|
||||
- **THEN** Meeting Assistant starts audio capture immediately and lets the pipeline finish readiness before it emits live transcript output
|
||||
|
||||
#### Scenario: Pipeline is changed by configuration
|
||||
- **WHEN** the configured ASR backend setting is changed
|
||||
- **THEN** Meeting Assistant selects the configured speech recognition pipeline without changing recording control or transcript persistence code
|
||||
|
||||
#### Scenario: ASR backend is tested with a WAV file
|
||||
- **WHEN** the user submits a local PCM WAV file to the ASR diagnostic endpoint
|
||||
- **THEN** Meeting Assistant streams that file through the configured speech recognition pipeline and returns the emitted live transcript segments
|
||||
|
||||
#### Scenario: ASR diagnostic backend is unavailable
|
||||
- **WHEN** the configured speech recognition pipeline fails while processing the diagnostic WAV file
|
||||
- **THEN** Meeting Assistant returns a diagnostic backend failure response instead of hiding the pipeline error
|
||||
|
||||
#### Scenario: Final diarization backend is tested with a WAV file
|
||||
- **WHEN** the user submits a local PCM WAV file to the final diarization diagnostic endpoint
|
||||
- **THEN** Meeting Assistant streams the file through the configured speech recognition pipeline and returns the finished transcript segments
|
||||
|
||||
#### Scenario: Final diarization receives a speaker count hint
|
||||
- **WHEN** the user submits a local PCM WAV file and `numSpeakers` to the final diarization diagnostic endpoint
|
||||
- **THEN** Meeting Assistant passes the requested speaker count to the configured speech recognition pipeline as a finished-transcript option
|
||||
|
||||
#### Scenario: Meeting attendee count provides final speaker hint
|
||||
- **WHEN** a meeting note has two or more entries in `attendees`
|
||||
- **THEN** Meeting Assistant passes that attendee count to the configured speech recognition pipeline as the finished-transcript speaker count hint
|
||||
|
||||
#### Scenario: Empty or single-attendee meeting note has no final speaker hint
|
||||
- **WHEN** a meeting note has zero or one entry in `attendees`
|
||||
- **THEN** Meeting Assistant leaves the finished-transcript speaker count hint unset
|
||||
|
||||
### Requirement: Version 1 keeps local Whisper transcription available
|
||||
Meeting Assistant SHALL provide a local Whisper speech recognition pipeline as a fallback ASR backend for version 1.
|
||||
|
||||
The local Whisper adapter SHALL use configurable model path, language, and streaming window settings.
|
||||
|
||||
When local Whisper is the configured speech recognition pipeline, Meeting Assistant SHALL be able to run a final pyannote diarization pass over the temporary mixed audio and either assign pyannote speaker turns back to live Whisper transcript segments by timestamp overlap or build finished transcript segments from pyannote speaker turns.
|
||||
|
||||
The pyannote finalization backend SHALL allow configuration to use either pyannote's normal `speaker_diarization` annotation or pyannote's `exclusive_speaker_diarization` annotation.
|
||||
|
||||
When a speaker count hint is provided through the speech recognition pipeline options, the pyannote finalization backend SHALL pass it to pyannote as `num_speakers`.
|
||||
|
||||
The pyannote finalization backend SHALL use a reusable runtime image for Python dependencies and a persistent configured cache folder for Hugging Face models, torch artifacts, and pip artifacts so repeated finalization runs do not redownload the diarization model or reinstall the full Python dependency stack.
|
||||
|
||||
#### Scenario: Local Whisper model is configured
|
||||
- **WHEN** the configured provider is `whisper-local`
|
||||
- **THEN** Meeting Assistant uses the configured local Whisper model path to transcribe incoming audio windows
|
||||
|
||||
#### Scenario: Local Whisper transcript is finalized with pyannote speaker turns
|
||||
- **WHEN** the configured provider is `whisper-local`, live Whisper transcript segments exist, and pyannote returns speaker turns for the temporary recording
|
||||
- **THEN** Meeting Assistant rewrites the transcript with the original Whisper text and the best-overlap pyannote speaker label for each segment
|
||||
|
||||
#### Scenario: Local Whisper transcript is segmented by pyannote speaker turns
|
||||
- **WHEN** the configured provider is `whisper-local`, pyannote turn alignment is configured, live Whisper transcript segments exist, and pyannote returns speaker turns for the temporary recording
|
||||
- **THEN** Meeting Assistant rewrites the transcript as ordered pyannote speaker-turn segments with matching Whisper text assigned to each turn
|
||||
|
||||
#### Scenario: Local Whisper uses exclusive pyannote diarization
|
||||
- **WHEN** the configured provider is `whisper-local` and exclusive pyannote annotation is configured
|
||||
- **THEN** Meeting Assistant reads speaker turns from pyannote's `exclusive_speaker_diarization` output
|
||||
|
||||
#### Scenario: Local Whisper pyannote runtime cache is reused
|
||||
- **WHEN** local Whisper finalization runs pyannote more than once
|
||||
- **THEN** Meeting Assistant uses the configured reusable pyannote runtime image and persistent pyannote model cache instead of treating each run as a clean download environment
|
||||
|
||||
#### Scenario: Local Whisper pyannote diarization is unavailable
|
||||
- **WHEN** the configured provider is `whisper-local` but pyannote is disabled, has no configured token, or returns no speaker turns
|
||||
- **THEN** Meeting Assistant keeps the live Whisper transcript instead of deleting it
|
||||
|
||||
### Requirement: Azure Speech can provide streaming transcription
|
||||
Meeting Assistant SHALL provide an Azure Speech speech recognition pipeline that streams captured PCM audio to Azure AI Speech through the Azure Speech SDK conversation transcription API.
|
||||
|
||||
The Azure Speech adapter SHALL use configurable endpoint, region, language, key, and key environment variable settings.
|
||||
|
||||
The Azure Speech adapter SHALL support configurable diarization of intermediate conversation transcription results.
|
||||
|
||||
The Azure Speech adapter SHALL bound recognizer shutdown after the input audio stream is closed.
|
||||
|
||||
When Azure Speech is the configured speech recognition pipeline, Meeting Assistant SHALL emit live transcript segments from Azure conversation transcription events with Azure speaker IDs when available.
|
||||
|
||||
When Azure Speech is the configured speech recognition pipeline, Meeting Assistant SHALL use the live Azure conversation transcription segments as the finished transcript without running pyannote finalization.
|
||||
|
||||
#### Scenario: Azure Speech pipeline is configured
|
||||
- **WHEN** the configured provider is `azure-speech`
|
||||
- **THEN** Meeting Assistant streams captured PCM chunks to Azure AI Speech conversation transcription through the configured speech recognition pipeline without changing recording control or transcript persistence code
|
||||
|
||||
#### Scenario: Azure Speech returns speaker IDs
|
||||
- **WHEN** Azure Speech conversation transcription returns transcript text with speaker identity
|
||||
- **THEN** Meeting Assistant emits ordered transcript segments with those Azure speaker identities
|
||||
|
||||
#### Scenario: Azure Speech key is resolved from environment
|
||||
- **WHEN** Azure Speech is configured with `KeyEnv`
|
||||
- **THEN** Meeting Assistant reads the Azure Speech key from that environment variable
|
||||
|
||||
#### Scenario: Azure Speech stream shutdown is bounded
|
||||
- **WHEN** the captured audio stream has ended
|
||||
- **THEN** Meeting Assistant stops Azure Speech continuous recognition within the configured stop timeout instead of waiting indefinitely
|
||||
|
||||
#### Scenario: Azure Speech transcript is finalized from live conversation segments
|
||||
- **WHEN** the configured provider is `azure-speech` and live Azure transcript segments exist
|
||||
- **THEN** Meeting Assistant uses the live Azure conversation transcript segments as the finished transcript
|
||||
|
||||
### Requirement: FunASR can provide speaker-attributed streaming transcription
|
||||
Meeting Assistant SHALL provide a FunASR speech recognition pipeline that streams PCM audio to a configured FunASR WebSocket endpoint.
|
||||
|
||||
The FunASR provider SHALL preserve speaker attribution from FunASR response fields when they are present and SHALL fall back to `Unknown` when no speaker field is returned.
|
||||
|
||||
Meeting Assistant SHALL write the mixed recording audio only as a temporary processing artifact while streaming transcription is active. After capture stops and the streaming provider has drained already-captured audio, Meeting Assistant SHALL be able to run a final FunASR Python `AutoModel` pass with VAD, punctuation, and CAMPPlus speaker modeling over the temporary audio. When that final pass returns sentence-level speaker labels, Meeting Assistant SHALL rewrite the transcript markdown with the final speaker-attributed sentence segments.
|
||||
|
||||
Meeting Assistant SHALL delete temporary recording audio after the run completes, whether final diarization succeeds, fails, or produces no speaker output. On application startup, Meeting Assistant SHALL delete stale temporary recording audio left behind by previous interrupted runs.
|
||||
|
||||
When configured, Meeting Assistant SHALL manage a local FunASR backend lifecycle for the FunASR provider by starting a non-blocking backend warm-up when the application starts, installing the configured streaming runtime image, preparing the mounted model and hotword folder, starting the backend, keeping the container alive for the backgrounded FunASR server process, and stopping it when the application stops. Recording SHALL be able to start while the backend is still warming up; captured audio SHALL be queued and streamed once the backend is healthy. Backend installation, startup commands, and WebSocket readiness SHALL be bounded by configurable timeouts so startup failures are diagnosable.
|
||||
|
||||
#### Scenario: FunASR pipeline is configured
|
||||
- **WHEN** the configured provider is `funasr`
|
||||
- **THEN** Meeting Assistant streams captured PCM chunks to the configured FunASR WebSocket endpoint through the configured speech recognition pipeline without changing recording control or transcript persistence code
|
||||
|
||||
#### Scenario: FunASR returns speaker fields
|
||||
- **WHEN** FunASR returns transcript text with speaker identity fields
|
||||
- **THEN** Meeting Assistant emits ordered transcript segments with those speaker identities
|
||||
|
||||
#### Scenario: Final FunASR diarization rewrites transcript
|
||||
- **WHEN** recording stops after mixed audio was temporarily stored and the final FunASR diarization pass returns sentence-level speaker labels
|
||||
- **THEN** Meeting Assistant rewrites the transcript markdown with the final speaker-attributed sentence segments
|
||||
|
||||
#### Scenario: Final FunASR diarization has no speaker output
|
||||
- **WHEN** recording stops but the final diarization pass is disabled or returns no sentence-level speaker labels
|
||||
- **THEN** Meeting Assistant keeps the live streaming transcript instead of deleting it
|
||||
|
||||
#### Scenario: Temporary recording audio is deleted after completion
|
||||
- **WHEN** a recording run completes after using temporary audio for final processing
|
||||
- **THEN** Meeting Assistant deletes the temporary audio file from disk
|
||||
|
||||
#### Scenario: Stale temporary recordings are deleted on startup
|
||||
- **WHEN** Meeting Assistant starts and stale temporary recording files exist from a previous interrupted run
|
||||
- **THEN** Meeting Assistant deletes those stale temporary recording files before accepting new recordings
|
||||
|
||||
#### Scenario: Managed FunASR backend starts with the application
|
||||
- **WHEN** the configured provider is `funasr` and managed backend startup is enabled
|
||||
- **THEN** Meeting Assistant begins installing the configured FunASR backend image and starting the two-pass streaming backend without blocking recording startup
|
||||
|
||||
#### Scenario: Recording starts while managed FunASR backend is warming up
|
||||
- **WHEN** recording starts before the managed FunASR backend is ready
|
||||
- **THEN** Meeting Assistant starts audio capture immediately and queues captured audio until the backend WebSocket is healthy
|
||||
|
||||
#### Scenario: Managed FunASR backend installation stalls
|
||||
- **WHEN** a managed FunASR backend command exceeds its configured timeout
|
||||
- **THEN** Meeting Assistant reports a backend startup timeout instead of waiting indefinitely
|
||||
|
||||
#### Scenario: Managed FunASR backend WebSocket is not healthy
|
||||
- **WHEN** the managed backend container has started but the FunASR WebSocket endpoint is not accepting sessions
|
||||
- **THEN** Meeting Assistant continues waiting for backend readiness until the configured startup timeout is reached
|
||||
|
||||
#### Scenario: Managed FunASR backend stops with the application
|
||||
- **WHEN** Meeting Assistant stops after starting a managed FunASR backend
|
||||
- **THEN** Meeting Assistant stops the managed backend process or container
|
||||
+6
@@ -5,10 +5,16 @@ Meeting Assistant SHALL maintain project knowledge based on meeting content and
|
||||
|
||||
Project knowledge SHALL support retrieval by project context and relevant keywords.
|
||||
|
||||
Project knowledge SHALL be stored under the configured projects folder, where each direct subfolder is a project identified by its folder name.
|
||||
|
||||
#### Scenario: Meeting has project context
|
||||
- **WHEN** a meeting note identifies a project
|
||||
- **THEN** Meeting Assistant can associate transcript-derived and summary-derived knowledge with that project
|
||||
|
||||
#### Scenario: Meeting binds project folders
|
||||
- **WHEN** the meeting note `projects` frontmatter lists configured project subfolder names
|
||||
- **THEN** Meeting Assistant treats those subfolders as the projects available to meeting agents
|
||||
|
||||
#### Scenario: Agent retrieves project context
|
||||
- **WHEN** an agent needs context for a meeting or follow-up task
|
||||
- **THEN** Meeting Assistant can retrieve relevant project knowledge and source references
|
||||
@@ -0,0 +1,58 @@
|
||||
## 1. Repository Setup
|
||||
|
||||
- [x] 1.1 Create the Gitea repository and clone it to `C:\Manuel\meeting-assistant`.
|
||||
- [x] 1.2 Add .NET 10 solution, service project, and test project.
|
||||
- [x] 1.3 Add repository README and agent instructions.
|
||||
- [x] 1.4 Add OpenSpec config and the version 1 proposal/design/specs.
|
||||
|
||||
## 2. Meeting Session Notes
|
||||
|
||||
- [x] 2.1 Define configurable vault locations for transcripts, meeting notes, assistant context, summaries, project knowledge, and dictation words.
|
||||
- [x] 2.2 Add a coded meeting note template and store with attendees, projects, transcript link, assistant context link, summary link, and body user notes.
|
||||
- [x] 2.3 Implement meeting session creation that writes the initial note before transcription starts and opens it through `obsidian://`.
|
||||
- [x] 2.4 Add tests proving user-authored notes remain preserved and included in later processing.
|
||||
- [x] 2.5 Add a configurable summary note location and store summary as a markdown link in meeting note frontmatter.
|
||||
|
||||
## 2A. Recording Control and Audio Capture
|
||||
|
||||
- [x] 2A.1 Add configurable hotkey settings using the normal .NET options model.
|
||||
- [x] 2A.2 Implement hotkey-controlled recording mode toggle.
|
||||
- [x] 2A.3 Capture microphone input and computer output through replaceable audio source abstractions.
|
||||
- [x] 2A.4 Mix microphone and system audio chunks into one PCM stream.
|
||||
- [x] 2A.5 Write transcript text into the configured vault folder.
|
||||
- [x] 2A.6 Add deterministic tests for streaming coordinator behavior, vault transcript persistence, hotkey parsing, and audio mixing.
|
||||
- [x] 2A.7 Stop capture while draining already captured audio through the speech recognition pipeline.
|
||||
|
||||
## 3. Transcription
|
||||
|
||||
- [x] 3.1 Select the first ASR backend.
|
||||
- [x] 3.2 Add provider-independent speech recognition pipeline contracts.
|
||||
- [x] 3.3 Implement transcription ingestion for platform-independent audio input.
|
||||
- [x] 3.4 Add FunASR speaker-attribution provider and tests.
|
||||
- [x] 3.5 Add local Whisper.NET provider adapter with configurable model path, language, and window size.
|
||||
- [x] 3.6 Add a small WAV fixture for transcription-provider tests.
|
||||
- [x] 3.7 Add an ASR diagnostic endpoint that streams a local WAV file through the configured provider.
|
||||
- [x] 3.8 Add managed FunASR backend install/start/stop lifecycle for the FunASR provider.
|
||||
- [x] 3.9 Temporarily write mixed recording audio and run a final FunASR CAMPPlus diarization pass that can rewrite the transcript.
|
||||
- [x] 3.10 Delete temporary recording audio after each run and clean up stale temporary recordings at startup.
|
||||
- [x] 3.11 Add a local Whisper final pyannote diarization pass that maps speaker turns onto live Whisper transcript segments.
|
||||
- [x] 3.12 Refactor ASR backends behind a speech recognition pipeline interface for initialization, readiness, chunk writes, live transcript reads, and finished transcript reads.
|
||||
|
||||
## 4. Summary and Next Steps
|
||||
|
||||
- [x] 4.1 Define the summary, decisions, open questions, and next steps output structure for the first summary note.
|
||||
- [x] 4.2 Create the assistant context note at session start and link it to the meeting, transcript, and summary notes.
|
||||
- [x] 4.3 Add a Microsoft Agent Framework summary pipeline with OpenAI-compatible configuration and meeting-scoped tools.
|
||||
- [x] 4.4 Add assistant-context agenda frontmatter and Windows-only Outlook Classic Teams appointment enrichment.
|
||||
|
||||
## 5. Project Knowledge and Agent Tools
|
||||
|
||||
- [x] 5.1 Define project binding metadata for meetings.
|
||||
- [x] 5.2 Define project knowledge storage and retrieval requirements.
|
||||
- [x] 5.3 Add agent tools for project lookup and keyword lookup.
|
||||
- [x] 5.4 Define and implement project file write modes for overwrite, replace, and insert.
|
||||
|
||||
## 6. Verification
|
||||
|
||||
- [x] 6.1 Run `dotnet test MeetingAssistant.slnx`.
|
||||
- [x] 6.2 Run `openspec validate define-meeting-assistant-v1 --strict`.
|
||||
@@ -0,0 +1,14 @@
|
||||
## Why
|
||||
|
||||
When Outlook Classic detects the current Teams appointment at transcription start, Meeting Assistant already copies title, attendees, and agenda into meeting artifacts. The assistant context should also record the appointment's scheduled end time so later agent steps can understand meeting timing without querying Outlook again.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add `scheduled_end` frontmatter to assistant context notes when a Teams appointment is detected at recording start.
|
||||
- Preserve `scheduled_end` when assistant context state changes.
|
||||
- Keep the field absent when no Teams appointment is detected.
|
||||
|
||||
## Impact
|
||||
|
||||
- Affected specs: meeting-summary, meeting-session
|
||||
- Affected code: Outlook meeting metadata, recording coordinator, assistant context artifact store
|
||||
+20
@@ -0,0 +1,20 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Windows Outlook enrichment is optional
|
||||
Meeting Assistant SHALL gate Outlook Classic COM enrichment behind the Windows compilation target.
|
||||
|
||||
When the Windows build starts a meeting and Outlook Classic has exactly one current Teams appointment, Meeting Assistant SHALL copy the appointment title and attendees to the meeting note and copy the appointment agenda and scheduled end time to the assistant-context frontmatter.
|
||||
|
||||
The agenda SHALL be extracted from the appointment body content before the Teams join separator or Teams join text.
|
||||
|
||||
#### Scenario: Current Teams appointment enriches meeting artifacts
|
||||
- **WHEN** a Windows build starts a meeting while Outlook Classic exposes exactly one current Teams appointment
|
||||
- **THEN** Meeting Assistant uses the appointment subject as the meeting title
|
||||
- **AND** writes the appointment attendees into meeting note frontmatter
|
||||
- **AND** writes the appointment agenda into assistant-context frontmatter
|
||||
- **AND** writes the appointment end time as `scheduled_end` into assistant-context frontmatter
|
||||
|
||||
#### Scenario: Outlook is unavailable or ambiguous
|
||||
- **WHEN** Outlook Classic is unavailable or more than one current Teams appointment is found
|
||||
- **THEN** Meeting Assistant starts the recording with the default generated meeting title and empty agenda
|
||||
- **AND** omits `scheduled_end` from assistant-context frontmatter
|
||||
+57
@@ -0,0 +1,57 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Meeting Assistant generates meeting outputs
|
||||
Meeting Assistant SHALL generate a summary, decisions, and next steps from the meeting transcript, metadata, user notes, and assistant-discovered context.
|
||||
|
||||
Meeting Assistant SHALL use a Microsoft Agent Framework pipeline for the first summary implementation. The pipeline SHALL use a configurable OpenAI-compatible endpoint, model, and API key source, including support for a direct key or an environment variable name.
|
||||
|
||||
The summary pipeline SHALL retry transient model endpoint failures according to configurable reconnection attempts and delay settings.
|
||||
|
||||
The summary pipeline SHALL track context-window usage for the configured model using response usage when available and request-size estimates otherwise. The context-window limit, maximum output reserve, compaction enablement, compaction threshold, and Responses compact endpoint path SHALL be configurable.
|
||||
|
||||
When only the configured remaining context ratio is available, the summary pipeline SHALL try to compact the conversation through the configured OpenAI-compatible `POST /v1/responses/compact` endpoint. If that endpoint is unavailable or returns invalid data, the pipeline SHALL fall back to Microsoft Agent Framework compaction.
|
||||
|
||||
Fallback compaction SHALL become increasingly aggressive: collapse old tool results, summarize older conversation spans, keep only the last four user turns, and drop oldest groups if still over budget.
|
||||
|
||||
When summary generation fails after retries, Meeting Assistant SHALL write a markdown failure report to the configured summary note path. The failure report SHALL include a clickable retry link, error details, and the meeting artifact paths needed to diagnose or retry the run.
|
||||
|
||||
After transcript processing finishes for a recording, Meeting Assistant SHALL automatically invoke the summary pipeline for that meeting.
|
||||
|
||||
Meeting Assistant SHALL expose an API operation to retry summary generation for a given summary note path. The retry operation SHALL resolve the linked meeting artifacts from the meeting note frontmatter and SHALL overwrite the existing summary note with either the new summary or a new failure report.
|
||||
|
||||
The summary pipeline SHALL expose tools scoped to the current meeting:
|
||||
|
||||
- `read_meetingnote`
|
||||
- `read_transcript`
|
||||
- `read_context`
|
||||
- `read_usernotes`
|
||||
- `read_glossary`
|
||||
- `write_summary`
|
||||
- `write_context`
|
||||
- `list_projects`
|
||||
- `list_projectfiles`
|
||||
- `read_projectfile`
|
||||
- `write_projectfile`
|
||||
- `search`
|
||||
|
||||
All summary-agent read tools that return file-like content SHALL support either reading the whole file or reading a clamped inclusive 1-based line range when `from` and `to` are supplied.
|
||||
|
||||
After writing the meeting summary, the summary agent SHALL update existing project files when the meeting produced project-relevant knowledge.
|
||||
|
||||
The assistant context note SHALL keep `meeting`, `transcript`, `summary`, `state`, and `agenda` in frontmatter. When a Teams appointment is detected at recording start, the assistant context note SHALL also keep `scheduled_end` in frontmatter. The `state` value SHALL be one of `transcribing`, `speaker recognition`, `summarizing`, `finished`, or `error`.
|
||||
|
||||
The summary note SHALL keep frontmatter links to meeting artifacts and SHALL copy `title`, `start_time`, and `end_time` from the meeting note when available. If the meeting note has no title, the summary agent SHALL be able to provide a title when writing the summary.
|
||||
|
||||
The summary agent SHALL be able to read and write the assistant context body as its own notebook using overwrite, replace, and insert line modes.
|
||||
|
||||
#### Scenario: Assistant context note is initialized
|
||||
- **WHEN** Meeting Assistant starts a meeting session
|
||||
- **THEN** it creates the assistant context note with frontmatter links to the meeting note, transcript note, and summary note
|
||||
- **AND** the assistant context frontmatter state is `transcribing`
|
||||
- **AND** the assistant context frontmatter includes `agenda`, empty when no agenda is known
|
||||
- **AND** the assistant context frontmatter includes `scheduled_end` when Teams metadata supplied a scheduled end time
|
||||
|
||||
#### Scenario: Assistant context state follows meeting processing
|
||||
- **WHEN** transcription, final speaker recognition, and summary generation progress
|
||||
- **THEN** Meeting Assistant updates assistant context frontmatter state to `transcribing`, `speaker recognition`, `summarizing`, `finished`, or `error` as appropriate
|
||||
- **AND** preserves existing `agenda` and `scheduled_end` frontmatter values
|
||||
@@ -0,0 +1,6 @@
|
||||
## 1. Scheduled End Metadata
|
||||
|
||||
- [x] 1.1 Extend Teams meeting metadata with scheduled end time.
|
||||
- [x] 1.2 Write `scheduled_end` to assistant context frontmatter only when Teams metadata is detected.
|
||||
- [x] 1.3 Preserve `scheduled_end` across assistant context state updates.
|
||||
- [x] 1.4 Add behavior tests and validate the OpenSpec change.
|
||||
@@ -1,58 +0,0 @@
|
||||
## Context
|
||||
|
||||
Meeting Assistant is a local/server-side .NET 10 application that helps capture and process meetings. It should work for any meeting format, including in-person conversations, and should not require direct APIs from meeting products such as Teams or Zoom.
|
||||
|
||||
The central artifact is an Obsidian markdown note created before transcription begins. The user can add attendees, project context, meeting purpose, and live notes while the meeting is happening. Meeting Assistant can add discovered context to the same note, then later append or link summaries, next steps, and knowledge base updates.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- Establish a .NET 10 application skeleton and OpenSpec workflow.
|
||||
- Model a platform-independent meeting session flow.
|
||||
- Create a meeting note before transcription starts.
|
||||
- Preserve user-authored notes as first-class input to summarization.
|
||||
- Transcribe meeting audio with speaker attribution.
|
||||
- Generate summaries, decisions, and next steps.
|
||||
- Maintain project knowledge from meeting output and additional future sources.
|
||||
- Let agents retrieve project context and keyword context.
|
||||
- Allow guarded project file updates when the project context or user explicitly permits them.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- Build a Teams, Zoom, or calendar-specific integration in version 1.
|
||||
- Require cloud transcription APIs.
|
||||
- Define the final Obsidian vault path or note template completely in this first change.
|
||||
- Implement unrestricted autonomous file edits.
|
||||
|
||||
## Decisions
|
||||
|
||||
1. Build the source application as ASP.NET Core on .NET 10.
|
||||
- Rationale: It matches the requested runtime and keeps the application service-oriented.
|
||||
|
||||
2. Make the Obsidian markdown note the session anchor.
|
||||
- Rationale: The user can enrich the meeting context before and during transcription, and agents can work from a durable human-readable artifact.
|
||||
|
||||
3. Treat meeting-platform APIs as optional augmentation.
|
||||
- Rationale: The assistant must be usable for in-person meetings and for platforms where no integration is available or desired.
|
||||
|
||||
4. Keep project knowledge as a generic capability.
|
||||
- Rationale: Meeting-derived knowledge should later combine with other sources without coupling the design to transcripts only.
|
||||
|
||||
5. Require explicit guardrails for project file mutation.
|
||||
- Rationale: Agents may update project files, but only when the project is known and the user or project context permits edits.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [Risk] Speaker attribution quality depends on the selected transcription pipeline. -> Mitigation: keep attribution requirements observable and provider-independent.
|
||||
- [Risk] The Obsidian note may become too dense. -> Mitigation: separate metadata, user notes, assistant context, transcript references, and generated outputs into clear sections.
|
||||
- [Risk] Agent file updates could affect the wrong project. -> Mitigation: require explicit project binding before mutation tools are available.
|
||||
- [Risk] Retrieval can surface stale or irrelevant project context. -> Mitigation: include source references and update timestamps in retrieved context.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- What exact Obsidian vault folder should hold meeting notes?
|
||||
- What metadata fields are required before transcription can start?
|
||||
- Which transcription provider should be used first?
|
||||
- Which storage backend should hold embeddings and project knowledge?
|
||||
- What approval model should govern agent project file edits?
|
||||
@@ -1,19 +0,0 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Agents can use project context tools
|
||||
Meeting Assistant SHALL expose tools that allow agents to look up project information, retrieve keyword-relevant context, and inspect meeting-derived knowledge.
|
||||
|
||||
#### Scenario: Agent looks up meeting project context
|
||||
- **WHEN** a meeting note identifies a project
|
||||
- **THEN** the agent can retrieve relevant project context through Meeting Assistant tools
|
||||
|
||||
### Requirement: Agents can only mutate project files with explicit project binding
|
||||
Meeting Assistant SHALL only expose project file mutation tools when the target project is known and mutation is permitted by user instruction or project context.
|
||||
|
||||
#### Scenario: Project mutation is permitted
|
||||
- **WHEN** the user or project context permits file updates for a known project
|
||||
- **THEN** the agent can use controlled tools to change files in that project
|
||||
|
||||
#### Scenario: Project mutation is not permitted
|
||||
- **WHEN** the target project is missing or mutation permission is absent
|
||||
- **THEN** Meeting Assistant does not expose project file mutation tools for that request
|
||||
@@ -1,30 +0,0 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Meeting Assistant creates a note before transcription
|
||||
Meeting Assistant SHALL create an Obsidian markdown note for a meeting session before transcription starts.
|
||||
|
||||
The note SHALL be the durable anchor for meeting metadata, user notes, assistant-discovered context, transcript references, summaries, decisions, and next steps.
|
||||
|
||||
#### Scenario: Session starts
|
||||
- **WHEN** the user starts a new meeting session
|
||||
- **THEN** Meeting Assistant creates a markdown note in the configured meetings folder before starting transcription
|
||||
|
||||
#### Scenario: User adds context before transcription
|
||||
- **WHEN** the note exists before transcription starts
|
||||
- **THEN** the user can add attendees, project context, meeting purpose, and other notes to that file
|
||||
|
||||
### Requirement: Meeting notes preserve user-authored content
|
||||
Meeting Assistant SHALL preserve user-authored meeting notes and include them as input when generating summaries, decisions, and next steps.
|
||||
|
||||
#### Scenario: User writes notes during the meeting
|
||||
- **WHEN** the user updates the meeting note while the meeting is active
|
||||
- **THEN** Meeting Assistant keeps that content and incorporates it into generated meeting outputs
|
||||
|
||||
### Requirement: Meeting sessions are platform independent
|
||||
Meeting Assistant SHALL support meeting sessions that do not originate from Teams, Zoom, or any other meeting software API.
|
||||
|
||||
Meeting software integrations MAY augment the meeting experience, but they SHALL NOT be required for the primary meeting capture, transcription, or summary flow.
|
||||
|
||||
#### Scenario: In-person meeting is captured
|
||||
- **WHEN** the user starts transcription for an in-person meeting
|
||||
- **THEN** Meeting Assistant captures and processes the meeting without requiring a meeting platform integration
|
||||
@@ -1,12 +0,0 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Meeting Assistant generates meeting outputs
|
||||
Meeting Assistant SHALL generate a summary, decisions, and next steps from the meeting transcript, metadata, user notes, and assistant-discovered context.
|
||||
|
||||
#### Scenario: Meeting processing completes
|
||||
- **WHEN** transcription and context collection have completed
|
||||
- **THEN** Meeting Assistant generates a summary, decisions, and next steps for the meeting note
|
||||
|
||||
#### Scenario: User notes conflict with transcript interpretation
|
||||
- **WHEN** user-authored notes provide context that changes the interpretation of the transcript
|
||||
- **THEN** Meeting Assistant considers those notes during summary and next-step generation
|
||||
@@ -1,14 +0,0 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Meeting Assistant transcribes meetings with speaker attribution
|
||||
Meeting Assistant SHALL transcribe meeting audio and associate transcript segments with speakers when speaker attribution is available.
|
||||
|
||||
The transcription contract SHALL remain independent of the meeting source so that local audio, in-person meetings, and future meeting-platform integrations can use the same downstream processing.
|
||||
|
||||
#### Scenario: Speaker-attributed transcript is produced
|
||||
- **WHEN** meeting audio is transcribed and speaker attribution is available
|
||||
- **THEN** the transcript contains ordered speaker-attributed segments
|
||||
|
||||
#### Scenario: Speaker attribution is unavailable
|
||||
- **WHEN** meeting audio is transcribed but speaker attribution is unavailable
|
||||
- **THEN** Meeting Assistant keeps the transcript and marks speaker identity as unknown rather than discarding the meeting
|
||||
@@ -1,38 +0,0 @@
|
||||
## 1. Repository Setup
|
||||
|
||||
- [x] 1.1 Create the Gitea repository and clone it to `C:\Manuel\meeting-assistant`.
|
||||
- [x] 1.2 Add .NET 10 solution, service project, and test project.
|
||||
- [x] 1.3 Add repository README and agent instructions.
|
||||
- [x] 1.4 Add OpenSpec config and the version 1 proposal/design/specs.
|
||||
|
||||
## 2. Meeting Session Notes
|
||||
|
||||
- [ ] 2.1 Define the Obsidian meeting note location and naming convention.
|
||||
- [ ] 2.2 Add a meeting note template with metadata, attendees, purpose, project context, user notes, assistant context, transcript, summary, decisions, and next steps.
|
||||
- [ ] 2.3 Implement meeting session creation that writes the initial note before transcription starts.
|
||||
- [ ] 2.4 Add tests proving user-authored notes remain preserved and included in later processing.
|
||||
|
||||
## 3. Transcription
|
||||
|
||||
- [ ] 3.1 Select the first transcription provider.
|
||||
- [ ] 3.2 Add provider-independent transcription contracts.
|
||||
- [ ] 3.3 Implement transcription ingestion for platform-independent audio input.
|
||||
- [ ] 3.4 Add speaker attribution handling and tests.
|
||||
|
||||
## 4. Summary and Next Steps
|
||||
|
||||
- [ ] 4.1 Define the summary, decisions, and next steps output structure.
|
||||
- [ ] 4.2 Implement summary generation from transcript plus user notes.
|
||||
- [ ] 4.3 Append or link generated outputs from the meeting note.
|
||||
|
||||
## 5. Project Knowledge and Agent Tools
|
||||
|
||||
- [ ] 5.1 Define project binding metadata for meetings.
|
||||
- [ ] 5.2 Define project knowledge storage and retrieval requirements.
|
||||
- [ ] 5.3 Add agent tools for project lookup and keyword lookup.
|
||||
- [ ] 5.4 Define and implement guardrails for project file mutation tools.
|
||||
|
||||
## 6. Verification
|
||||
|
||||
- [x] 6.1 Run `dotnet test MeetingAssistant.slnx`.
|
||||
- [x] 6.2 Run `openspec validate define-meeting-assistant-v1 --strict`.
|
||||
Reference in New Issue
Block a user