Public Access
Preserve recovery transcripts and add user-approved agent requests
PR and Push Build/Test / build-and-test (push) Successful in 12m0s
PR and Push Build/Test / build-and-test (push) Successful in 12m0s
Append successful offline recovery with a duplicate-warning marker and retain failed replay sources. Offer newly available microphones through confirmation notifications. Add project discovery metadata and blocking association approval, plus nonblocking workflow requests that start a separate settings-agent conversation. Include regression tests, OpenSpec changes, and verified local Release startup.
This commit is contained in:
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-09-25
|
||||
@@ -0,0 +1,35 @@
|
||||
## Context
|
||||
|
||||
Project IDs are direct subfolder names. BoundMeetingProjectResolver reads the meeting note on each access. The summarizer already has project read/write tools and an initial section for bound AGENTS.md instructions, but writes currently accept any existing project. Native actionable toasts and a WPF settings-agent window already exist.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:** Discover projects using compact metadata, extend project access only after approval, maintain metadata through the agent, and forward approved workflow requests to the interactive settings agent.
|
||||
|
||||
**Non-Goals:** Bulk-generate metadata for unassociated projects, let the summarizer edit workflows directly, change project IDs, migrate UI frameworks, or deploy/restart the workstation service.
|
||||
|
||||
## Decisions
|
||||
|
||||
- Store `name` and `description` in the root `PROJECT.md` frontmatter; `name` is the display name, and the directory name remains the stable ID. Missing/invalid metadata falls back to the ID and an empty description. Catalog results contain metadata, never AGENTS.md or arbitrary project content.
|
||||
- Reuse write_projectfile for metadata maintenance. Validate resulting PROJECT.md frontmatter (name and description, maximum 256 description characters) before writing. Preserve ordinary file edit modes. Restrict writes to associated projects, matching existing read/search permissions.
|
||||
- Append metadata and AGENTS.md at the existing initial project section. Add mandatory capability guidance even when a custom base prompt is configured. Instruct the agent to maintain metadata using grounded project knowledge and propose missing associations before finalizing the summary.
|
||||
- Project approval is an awaited tool call, with cancellation and a ten-minute deadline. Return approval plus project metadata and AGENTS.md only after updating the latest meeting note. Preserve other frontmatter and body content; recheck target and cancellation before writing. Write to a temporary file in the same directory and replace the note only after that write completes, so cancellation or a partial write cannot truncate the note. Existing associations return current context without another prompt.
|
||||
- Use an application-owned background request queue for workflow approvals. The tool returns requested immediately; the queue outlives a summary run. Requests also expire after ten minutes. Only acceptance opens a new settings-agent window and automatically submits the complete detailed prompt. A new conversation avoids overwriting drafts or interrupting an existing agent run.
|
||||
- Reuse the installed native toast toolkit behind a testable approval-prompt interface. Timeouts and cancellation remove pending toasts and invalidate late actions. Shutdown cancels pending approvals; requests are not persisted across process restarts.
|
||||
- Keep tool execution sequential so a blocking project approval cannot be bypassed by concurrent tool execution in the same model response.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [User ignores project approval] → continue after ten minutes as denied; no project scope or note change.
|
||||
- [User edits the note while a request is pending] → read again after acceptance and append only the project field, retaining all other current fields and notes.
|
||||
- [Project disappears or becomes invalid while waiting] → refuse the association without expanding access.
|
||||
- [Native UI cannot run in automated tests] → test approval, timeout, scope and handoff through controlled prompt/window services and build the Windows target; record hardware/UI verification limits.
|
||||
- [Summarizer completes before workflow approval] → the application queue retains the request until decision, timeout, or shutdown.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
No bulk migration. Existing folders remain discoverable without PROJECT.md, and associated projects gain metadata as the summarizer maintains them. Rollback removes the tools; PROJECT.md and normal meeting associations remain ordinary vault content.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. Workflow notifications use the same ten-minute validity as project notifications; workflow notification intentions are limited to 100 characters for readability.
|
||||
@@ -0,0 +1,25 @@
|
||||
## Why
|
||||
|
||||
The summarizer cannot currently discover missing project associations or propose workflow improvements with explicit user approval. Project folder names alone do not provide enough context for selecting the right project.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Maintain `PROJECT.md` frontmatter with `name` and a description of at most 256 characters through the summarizer's existing project-file tools.
|
||||
- Include available project metadata alongside bound project instructions and expose the full project metadata catalog through `list_projects`.
|
||||
- Add `request_project_association` with a project ID and a reason of at most 100 characters. Wait up to ten minutes for a native Yes/No notification; approval adds the association and returns project instructions.
|
||||
- Enforce meeting association for summarizer project-file writes as well as reads. Catalog visibility alone does not grant project access.
|
||||
- Add nonblocking `request_workflow_change` with a short intention and detailed scenario/specification. Approval opens a settings-agent conversation and immediately submits the detailed request; the summarizer cannot edit workflows directly.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `summary-agent-requests`: User approval for blocking project association and asynchronous workflow-change handoff.
|
||||
|
||||
### Modified Capabilities
|
||||
- `project-knowledge`: Project identity and discovery metadata in `PROJECT.md`.
|
||||
- `agent-project-tools`: Full metadata catalog and association-scoped writes.
|
||||
- `meeting-summary`: Project metadata in initial instructions and guidance for metadata maintenance and requests.
|
||||
|
||||
## Impact
|
||||
|
||||
Summary tools and agent instructions, project lookup and meeting frontmatter, native Windows notifications, the settings-agent window/prompt submission, DI registration, tests, and workflow/configuration documentation. Existing project IDs remain directory names; no bulk vault migration or live service restart is required.
|
||||
@@ -0,0 +1,91 @@
|
||||
## MODIFIED Requirements
|
||||
### Requirement: Agents can use project context tools
|
||||
Meeting Assistant SHALL expose tools that allow agents to look up project information, retrieve keyword-relevant context, and inspect meeting-derived knowledge.
|
||||
|
||||
Project tools SHALL treat each direct subfolder of the configured projects folder as one project. A meeting note binds projects by listing those subfolder names in the `projects` frontmatter field.
|
||||
|
||||
The summary agent SHALL expose these project tools:
|
||||
|
||||
- `list_projects`
|
||||
- `request_project_association`
|
||||
- `request_workflow_change`
|
||||
- `list_projectfiles`
|
||||
- `read_projectfile`
|
||||
- `write_projectfile`
|
||||
- `list_past_project_meetings`
|
||||
- `read_past_project_meeting_summary`
|
||||
- `search`
|
||||
|
||||
The `search` tool SHALL search the requested current-meeting project scope, or all projects assigned to the current meeting when no project scope is supplied.
|
||||
|
||||
The `search` tool SHALL search both configured project files for the scoped projects and past meeting summary notes in the configured summaries folder whose frontmatter projects intersect the scoped projects.
|
||||
|
||||
The `search` tool SHALL refuse requested project scopes that are not assigned to the current meeting.
|
||||
|
||||
#### Scenario: Agent looks up meeting project context
|
||||
- **WHEN** a meeting note identifies a project
|
||||
- **THEN** the agent can retrieve relevant project context through Meeting Assistant tools
|
||||
|
||||
#### Scenario: Agent discovers the full project catalog
|
||||
- **WHEN** the agent calls `list_projects`
|
||||
- **THEN** it receives every configured project with ID, display name, and description
|
||||
- **AND** this catalog does not grant access to unassociated project files or AGENTS.md
|
||||
|
||||
#### Scenario: Agent reads a clamped project file range
|
||||
- **WHEN** the agent reads a project file with optional line bounds outside the file length
|
||||
- **THEN** `read_projectfile` clamps the requested range to the available file lines without failing
|
||||
|
||||
#### Scenario: Agent searches project knowledge
|
||||
- **WHEN** the agent calls `search` with .NET regular expression keywords
|
||||
- **THEN** Meeting Assistant runs the search against the requested projects or the meeting-bound projects and returns matches as `filename:line text`
|
||||
|
||||
#### Scenario: Agent searches past project meeting summaries
|
||||
- **GIVEN** the current meeting note is assigned to `Project X`
|
||||
- **AND** a prior summary note in the configured summaries folder is assigned to `Project X`
|
||||
- **WHEN** the summary agent searches for a keyword
|
||||
- **THEN** Meeting Assistant returns matches from both `Project X` project files and matching past summary notes
|
||||
|
||||
#### Scenario: Agent search rejects out-of-scope project
|
||||
- **GIVEN** the current meeting note is assigned to `Project X`
|
||||
- **WHEN** the summary agent searches with project scope `Project Z`
|
||||
- **THEN** Meeting Assistant refuses the request because `Project Z` is not assigned to the current meeting
|
||||
|
||||
### Requirement: Agents can write files in existing projects
|
||||
Meeting Assistant SHALL expose a `write_projectfile` tool that can create or update files inside an existing project folder.
|
||||
|
||||
The target project SHALL be an existing direct subfolder of the configured projects folder and SHALL be associated with the current meeting. The target file path SHALL stay inside that project folder. Newly approved associations SHALL take effect immediately for all project tools.
|
||||
|
||||
When no line edit arguments are supplied and `replace_file` is not true, `write_projectfile` SHALL append the supplied content to the file and SHALL create the file when it does not exist.
|
||||
|
||||
When `replace_file` is true, `write_projectfile` SHALL replace the whole file with the supplied content.
|
||||
|
||||
When both `from` and `to` are supplied, `write_projectfile` SHALL replace the inclusive 1-based line range with the supplied content.
|
||||
|
||||
When `insert` is supplied, `write_projectfile` SHALL insert the supplied content at that 1-based line position.
|
||||
|
||||
Meeting Assistant SHALL refuse ambiguous writes that combine `replace_file` with line edit arguments.
|
||||
|
||||
#### Scenario: Project file is appended or created
|
||||
- **WHEN** the agent writes a project file without line edit arguments
|
||||
- **THEN** Meeting Assistant appends the supplied content to an existing file
|
||||
- **AND** creates the file with the supplied content when it does not exist
|
||||
|
||||
#### Scenario: Project file is explicitly replaced
|
||||
- **WHEN** the agent writes a project file with `replace_file` set to true
|
||||
- **THEN** Meeting Assistant writes the supplied content as the complete file content
|
||||
|
||||
#### Scenario: Project file line range is replaced
|
||||
- **WHEN** the agent writes a project file with `from` and `to` line numbers
|
||||
- **THEN** Meeting Assistant replaces the inclusive line range with the supplied content
|
||||
|
||||
#### Scenario: Project file content is inserted
|
||||
- **WHEN** the agent writes a project file with an `insert` line number
|
||||
- **THEN** Meeting Assistant inserts the supplied content at that line position
|
||||
|
||||
#### Scenario: Project file write target is invalid
|
||||
- **WHEN** the target project is missing, is not associated with the meeting, or the target path escapes the project folder
|
||||
- **THEN** Meeting Assistant refuses the project file write
|
||||
|
||||
#### Scenario: Project file write mode is ambiguous
|
||||
- **WHEN** the agent combines `replace_file` with `from`, `to`, or `insert`
|
||||
- **THEN** Meeting Assistant refuses the project file write
|
||||
@@ -0,0 +1,23 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Summary agent receives bound project instructions
|
||||
Meeting Assistant SHALL append a project section for projects bound to the meeting note frontmatter. Each entry SHALL identify the project ID and include its display name and description from PROJECT.md when available. For each bound project with a root AGENTS.md, the entry SHALL also include that file's content. Metadata SHALL appear even when AGENTS.md is absent. Unassociated project instructions SHALL NOT be appended.
|
||||
|
||||
#### Scenario: Project AGENTS files are appended
|
||||
- **GIVEN** the meeting note frontmatter lists projects Alpha and Beta with root AGENTS.md files
|
||||
- **WHEN** the summary agent is created
|
||||
- **THEN** its project section includes both project IDs and their instructions
|
||||
|
||||
#### Scenario: Project metadata without instructions is appended
|
||||
- **GIVEN** a bound project has PROJECT.md metadata but no AGENTS.md
|
||||
- **WHEN** the summary agent is created
|
||||
- **THEN** its entry contains ID, display name and description
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Summarizer maintains project metadata and proposes missing associations
|
||||
Both default and custom summary-agent instructions SHALL explain the full metadata catalog, the distinction between catalog visibility and project access, and the approval process. They SHALL instruct the agent to propose plausible missing associations with concise evidence before finalizing the summary and to maintain associated PROJECT.md name and description (maximum 256 characters). They SHALL explain that workflow improvements require a short intention and a detailed request and are executed only by the settings agent after user approval.
|
||||
|
||||
#### Scenario: Custom base prompt still receives capability guidance
|
||||
- **WHEN** a custom summary base prompt is configured
|
||||
- **THEN** the resulting instructions still explain metadata maintenance, association approvals and workflow-change requests
|
||||
@@ -0,0 +1,18 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Projects have compact discovery metadata
|
||||
Each project SHALL use a root `PROJECT.md` with YAML frontmatter `name` and `description` for agent-maintained discovery metadata. `name` SHALL be the display name; the project ID SHALL remain its direct subfolder name. The summarizer SHALL be instructed to create and maintain this metadata for associated projects from grounded project knowledge, preserving other frontmatter and body content. The description SHALL contain at most 256 characters.
|
||||
|
||||
Catalog lookup SHALL tolerate missing or malformed metadata by falling back to the ID as display name and an empty description. Metadata lookup SHALL NOT grant access to project files or instructions.
|
||||
|
||||
#### Scenario: Project has metadata
|
||||
- **WHEN** a project has valid name and description frontmatter
|
||||
- **THEN** catalog lookup exposes its ID, display name, and description
|
||||
|
||||
#### Scenario: Legacy project has no metadata
|
||||
- **WHEN** a project has no PROJECT.md or no usable frontmatter
|
||||
- **THEN** it remains discoverable with its ID as display name and an empty description
|
||||
|
||||
#### Scenario: Agent writes invalid metadata
|
||||
- **WHEN** the summarizer attempts a PROJECT.md update without valid name/description or with a description longer than 256 characters
|
||||
- **THEN** the write is refused without changing the existing file
|
||||
@@ -0,0 +1,56 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Summarizer can request project association
|
||||
The summarizer SHALL expose `request_project_association` accepting an existing project ID and a nonempty single-line reason of at most 100 characters. Invalid requests SHALL be refused without a notification or file changes.
|
||||
|
||||
The tool SHALL show a native notification identifying the meeting and project, explaining the association request and reason, with Yes/No actions. Tool completion and further summarizer work SHALL wait for the decision for up to ten minutes. No, expiration, notification unavailability, or canceled summarization SHALL NOT add the project. Expiration SHALL remove the notification, invalidate late activation, and return denial.
|
||||
|
||||
Once a stopped run has reached summarization, the recording stop-processing deadline SHALL return control while summary processing continues in the background. It SHALL NOT cancel a summary awaiting approval or queue an already-transcribed recording for offline replay.
|
||||
|
||||
On Yes, the tool SHALL append the canonical project ID to the latest meeting-note projects frontmatter without duplicates, preserving other fields and body. Project tools SHALL immediately honor the updated scope, and the result SHALL report approval with project metadata and the root AGENTS.md content when present. No unapproved AGENTS.md content SHALL be returned. Already associated projects SHALL return their current context without prompting.
|
||||
|
||||
#### Scenario: Approval extends scope during the same summary
|
||||
- **GIVEN** a catalog project is not associated with the meeting
|
||||
- **WHEN** the summarizer requests association and the user chooses Yes
|
||||
- **THEN** the meeting note gains that project and the same agent can read/write it
|
||||
- **AND** the result includes its AGENTS.md if present
|
||||
|
||||
#### Scenario: Denial or timeout preserves scope
|
||||
- **WHEN** the user chooses No or ten minutes elapse
|
||||
- **THEN** the waiting tool reports denial, the notification disappears, and scope and meeting note remain unchanged
|
||||
|
||||
#### Scenario: User edits note during approval
|
||||
- **WHEN** the user changes other meeting frontmatter or notes while an association request waits
|
||||
- **AND** then approves the project
|
||||
- **THEN** the project is added without losing those edits
|
||||
|
||||
#### Scenario: Invalid or stale request cannot grant access
|
||||
- **WHEN** the ID is unknown, reason exceeds 100 characters, the target disappears, or summarization is canceled
|
||||
- **THEN** the request cannot expand project access
|
||||
|
||||
#### Scenario: Stop wait expires while summary approval is pending
|
||||
- **GIVEN** transcription has completed and summarization is waiting for a user decision
|
||||
- **WHEN** the recording stop-processing timeout expires
|
||||
- **THEN** stopping returns with summarization still active
|
||||
- **AND** the summary continues with its own approval deadline without an offline transcription backlog item
|
||||
|
||||
### Requirement: Summarizer can request workflow changes without editing workflows
|
||||
The summarizer SHALL expose `request_workflow_change` accepting a nonempty single-line intention of at most 100 characters and a nonempty detailed change prompt, preferably a scenario or specification. The tool SHALL return requested without waiting for approval, and the summarizer SHALL continue normally without direct workflow-editing capabilities.
|
||||
|
||||
A native notification SHALL show `Workflow change requested: ...` with the intention and Yes/No actions. The detailed prompt SHALL be retained by the application independently of summary completion. No or ten-minute expiration SHALL discard the request without opening the editor or changing workflows. Late activation SHALL have no effect.
|
||||
|
||||
Yes SHALL open a new settings-agent conversation and immediately submit the full detailed prompt once. An existing open conversation or draft SHALL remain intact. Application shutdown SHALL cancel and remove pending notifications.
|
||||
|
||||
#### Scenario: Workflow request does not block the summary
|
||||
- **WHEN** the agent requests a workflow change
|
||||
- **THEN** the tool returns requested while approval remains pending
|
||||
- **AND** the summary can complete without opening the settings agent
|
||||
|
||||
#### Scenario: Approved workflow request starts settings agent
|
||||
- **GIVEN** the summary run has completed and its workflow request is pending
|
||||
- **WHEN** the user chooses Yes
|
||||
- **THEN** the settings-agent window opens and starts the detailed prompt exactly once
|
||||
|
||||
#### Scenario: Denied or expired workflow request does nothing
|
||||
- **WHEN** the user declines or ignores a workflow request for ten minutes
|
||||
- **THEN** its notification disappears and no editor or workflow change is started
|
||||
@@ -0,0 +1,35 @@
|
||||
## 1. Project catalog and metadata
|
||||
|
||||
- [x] 1.1 Test and implement full metadata catalog with legacy/malformed fallback and scoped file access.
|
||||
- [x] 1.2 Test PROJECT.md metadata writes and enforce the description limit without damaging existing files.
|
||||
- [x] 1.3 Include bound metadata beside AGENTS.md and instruct default/custom agents to maintain it and request missing associations.
|
||||
|
||||
## 2. Project approval
|
||||
|
||||
- [x] 2.1 Test blocking approval, immediate scope update, returned AGENTS.md and preservation of concurrent user note edits.
|
||||
- [x] 2.2 Handle invalid IDs/reasons, existing associations, denial, timeout, cancellation and stale actions.
|
||||
- [x] 2.3 Register the tool and native Yes/No notification with ten-minute validity and sequential summary tool execution.
|
||||
- [x] 2.4 Keep summarization alive after the recording stop wait expires so approval does not trigger transcription cancellation or offline replay.
|
||||
|
||||
## 3. Workflow-change requests
|
||||
|
||||
- [x] 3.1 Test and implement nonblocking application-owned workflow requests that survive summary completion.
|
||||
- [x] 3.2 Open a separate settings-agent conversation and automatically submit approved detail; preserve existing drafts and runs.
|
||||
- [x] 3.3 Cover rejection, expiration, shutdown, invalid requests and exactly-once acceptance.
|
||||
|
||||
## 4. Verification
|
||||
|
||||
- [x] 4.1 Update workflow and configuration documentation.
|
||||
- [x] 4.2 Review changed code and surrounding integration; fix actionable findings.
|
||||
- [x] 4.3 Run relevant tests, full solution tests, Windows build and strict OpenSpec validation.
|
||||
- [x] 4.4 Verify through the safest direct operational surface and document native UI/live-provider limits.
|
||||
|
||||
Verification (2026-09-25): all 552 solution tests pass, including 20 focused project/catalog/request tests. The solution builds the Windows target successfully with the four existing NAudio obsolete-API warnings. Strict validation passes for this change and the existing microphone change; `git diff --check` is clean.
|
||||
|
||||
Two independent review passes covered project metadata/access and notification/queue/UI lifecycle, including surrounding recording-stop integration. Incomplete metadata now consistently falls back to the project ID, and traversal coverage uses an associated project so it actually exercises path containment. Project association writes prepare a same-directory temporary file before replacing the meeting note; successful requests leave no temporary file behind.
|
||||
|
||||
Public-tool behavior tests inspect real generated meeting and project files with controlled approval/window services. They verify latest-note preservation, immediate scope changes, returned instructions, rejection/expiry/cancellation, nonblocking workflow requests, and automatic submission of approved detail to a fresh settings-agent conversation. A coordinator regression test verifies that the stop wait cannot cancel a summary awaiting approval or enqueue unnecessary offline replay.
|
||||
|
||||
The running application's recording-status endpoint was checked read-only and reported idle. The service was not updated or restarted. Native toast activation/removal, actual WPF window isolation, and a live model invoking these tools were not exercised, because those require running the new Windows build and interacting with its notifications. Sequential tool execution is configured and compiled; the model pipeline itself was not exercised with simultaneous tool calls. These operational checks remain required before acceptance or archival.
|
||||
|
||||
Authorized local start (2026-09-25): after confirming idle status and logs, the documented restart helper published and started the Release build in `tmp/meeting-assistant-runtime/run-20260925-115711`. `/health` reports `ok`, `/recording/status` reports idle, the tray icon was created, and Resemblyzer warm-up completed without stderr errors. The published application DLL matches the Release output by SHA-256. This verifies startup of the new build; native approval interactions and live-model tool invocation remain unverified.
|
||||
@@ -24,4 +24,8 @@ After configured reconnect attempts are exhausted, Azure should emit a longer di
|
||||
|
||||
If a stopped Azure Speech meeting cannot finish draining before `Recording:StopProcessingTimeout`, the coordinator queues the completed temporary WAV and artifact paths in a persisted offline backlog, releases the active recording slot, and keeps the WAV from startup cleanup. This lets the user start more meetings while Azure or the network is still unavailable.
|
||||
|
||||
The backlog processor retries queued items in the background. Each item creates a fresh speech recognition pipeline for the original launch profile, streams the queued WAV into that pipeline, rewrites the original transcript file with the finished lines, updates meeting-note and transcript metadata, runs transcript-line workflow transformations, transitions the assistant context through summarizing to finished/error, runs the normal summary pipeline, then removes the backlog item and deletes the temporary WAV. Failed processing leaves the backlog item and WAV in place for a later retry.
|
||||
The backlog processor retries queued items in the background. Each item creates a fresh speech recognition pipeline for the original launch profile and streams the queued WAV into that pipeline. Results without non-blank speech text leave all artifacts and recovery files unchanged for a later retry.
|
||||
|
||||
A usable replay preserves the existing transcript body and appends a visible recovery section before updating meeting metadata and running the normal summary pipeline. The recovery marker explains that the full recording was replayed, earlier passages may be duplicated, and repeated content should be treated as the same discussion when summarizing. Recovered lines still receive transcript-line workflow transformations; the explanatory marker is written directly so it remains visible. The application does not try to deduplicate or reconcile live and recovered recognition results.
|
||||
|
||||
After appending, the processor updates meeting-note and transcript metadata, transitions the assistant context through summarizing to finished/error, and runs the normal summary pipeline. Completed processing removes the backlog item and temporary WAV. Failed processing leaves both in place for a later retry; an additional replay may append another clearly marked recovery section without discarding earlier content.
|
||||
|
||||
+24
-2
@@ -27,7 +27,11 @@ When Azure Speech is still unavailable after recording stops and transcription c
|
||||
|
||||
When a stopped meeting is persisted to the durable transcription backlog, Meeting Assistant SHALL release the active recording slot so another meeting can be recorded while the stopped meeting waits for Azure Speech to become available.
|
||||
|
||||
When Azure Speech becomes available again, Meeting Assistant SHALL retry durable backlog items, rewrite the transcript from the recorded WAV, run the normal post-transcription meeting completion and summary flow, and remove the backlog item after successful completion.
|
||||
When Azure Speech becomes available again, Meeting Assistant SHALL retry durable backlog items, append the recovered transcript from the recorded WAV without replacing the existing transcript body, run the normal post-transcription meeting completion and summary flow, and remove the backlog item after successful completion.
|
||||
|
||||
Before appending recovered lines, Meeting Assistant SHALL insert a visible recovery marker explaining that the following section was recovered by replaying the entire recording and may duplicate earlier transcript passages. The marker SHALL tell the summarizer to treat repeated passages as the same discussion. The marker and appended lines SHALL be available to the summary agent together with the preserved earlier transcript. Existing transcript text, speaker labels, and user edits SHALL remain unchanged.
|
||||
|
||||
An offline replay that returns no speech segments with non-blank text SHALL be treated as an unsuccessful attempt. This includes empty results, blank speech segments, and results containing only status markers. Meeting Assistant SHALL preserve the existing transcript and meeting artifacts unchanged, SHALL skip meeting completion and summarization, and SHALL retain the durable backlog item and its WAV for a later retry.
|
||||
|
||||
When Meeting Assistant starts, it SHALL preserve WAV files that are referenced by durable transcription backlog items instead of deleting them as stale temporary recordings.
|
||||
|
||||
@@ -73,6 +77,24 @@ When Meeting Assistant starts, it SHALL preserve WAV files that are referenced b
|
||||
#### Scenario: Durable Azure backlog resumes after connectivity returns
|
||||
- **GIVEN** a stopped Azure meeting exists in the durable transcription backlog
|
||||
- **WHEN** Azure Speech can transcribe the recorded WAV
|
||||
- **THEN** Meeting Assistant rewrites the transcript from the recorded WAV
|
||||
- **THEN** Meeting Assistant preserves the existing transcript body and appends a recovery marker followed by the transcript from the recorded WAV
|
||||
- **AND** the marker explains the recovery and possible duplicates to the summary agent
|
||||
- **AND** runs meeting completion and summarization
|
||||
- **AND** removes the durable backlog item and its temporary WAV after successful completion
|
||||
|
||||
#### Scenario: Empty offline replay preserves the live transcript and recovery files
|
||||
- **GIVEN** a stopped Azure meeting has an existing live transcript and a durable backlog item referencing its WAV
|
||||
- **WHEN** the offline replay returns no transcript segments
|
||||
- **THEN** Meeting Assistant leaves the transcript and meeting artifacts unchanged
|
||||
- **AND** does not run meeting completion or summarization
|
||||
- **AND** retains the backlog item and WAV for the next retry
|
||||
- **WHEN** a later retry produces a transcript from the WAV
|
||||
- **THEN** Meeting Assistant appends the recovery marker and recovered transcript without replacing earlier content and completes the meeting normally
|
||||
- **AND** removes the backlog item and WAV only after that successful completion
|
||||
|
||||
#### Scenario: Offline status markers or blank speech do not count as successful transcription
|
||||
- **GIVEN** a stopped Azure meeting has an existing live transcript and a durable backlog item referencing its WAV
|
||||
- **WHEN** the offline replay returns only status markers or speech segments with blank text
|
||||
- **THEN** Meeting Assistant preserves the existing transcript and meeting artifacts
|
||||
- **AND** skips meeting completion and summarization
|
||||
- **AND** retains the backlog item and WAV for another retry
|
||||
|
||||
@@ -9,3 +9,17 @@
|
||||
- [x] Run focused tests, full solution tests, and `openspec validate azure-speech-offline-resilience --strict`.
|
||||
- [x] Add a durable offline backlog for stopped Azure meetings across process restarts.
|
||||
- [x] Replay queued WAV files through a fresh speech pipeline and finish transcript metadata, meeting context state, summary generation, and backlog cleanup.
|
||||
|
||||
## Bug-fix addendum: Preserve transcripts during offline replay
|
||||
|
||||
- [x] Specify that replay without speech text preserves the transcript, backlog, and WAV until a successful retry.
|
||||
- [x] Reproduce the data-loss bug with real transcript and backlog files, then reject empty, blank-text, and marker-only replay results before any artifact writes.
|
||||
- [x] Verify retained files survive repeated retries and a later successful replay still completes and cleans up normally.
|
||||
- [x] Run focused regression tests, full solution tests, and strict OpenSpec validation.
|
||||
|
||||
- [x] Preserve existing transcript content on successful replay and append a recovery marker explaining possible duplicates before the recovered text.
|
||||
- [x] Verify the summary agent receives earlier text, the recovery explanation, and appended text together, then rerun the relevant tests and strict validation.
|
||||
|
||||
Verification: the regression cases exercise the backlog processor with real WAV, transcript, meeting-note, and durable JSON files. Each case checks byte-for-byte preservation through two unsuccessful attempts and reloads the backlog from disk. A later usable replay preserves the original body, including a user correction without a trailing newline, and appends the marker and recovered text. A transcript-reading summary stub verifies that both versions, including an intentional duplicate passage, are available with the explanation before backlog/WAV cleanup. All 525 solution tests and strict validation of this change and the accepted transcription spec pass; independent review found no actionable issues. The Azure response is deterministic test input; no live Azure outage was induced and the running workstation service was not restarted.
|
||||
|
||||
Authorized local start (2026-09-25): the Release build containing this fix was subsequently published and started after confirming the workstation instance was idle. Health and startup logs are successful; the full solution now has 552 passing tests. No live Azure outage was induced.
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-09-25
|
||||
@@ -0,0 +1,33 @@
|
||||
## Context
|
||||
|
||||
MicrophoneAudioSource already retries failed capture sources without ending the composite microphone/system-audio stream. WindowsMicrophoneDeviceProvider enumerates active endpoints on demand. Native actionable Windows notifications already use CommunityToolkit.WinUI.Notifications.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:** Discover newly available microphones during active capture, ask before switching, and replace only the microphone capture lifetime when accepted. Preserve the recording, paused state, transcription session, and system audio.
|
||||
|
||||
**Non-Goals:** Change tray selection semantics, automatically prefer newly connected hardware, introduce a new notification framework, or restart the service.
|
||||
|
||||
## Decisions
|
||||
|
||||
- Poll active endpoints once per second within each microphone capture session. This reuses existing enumeration and works independently of incoming audio chunks. An OS-specific device-event subscription would add another platform lifecycle without improving the current requirement materially.
|
||||
- Establish the initial endpoint IDs as the baseline and offer each newly seen ID at most once per meeting. Declining or ignoring a prompt does not interrupt audio and does not cause repeated prompts.
|
||||
- Keep discovery/prompt handling in a separate monitor with a narrow prompt-service boundary. The microphone source owns replacement of its current capture token; the shared runtime selection is updated only on a still-valid affirmative response.
|
||||
- Bind monitoring and prompt cancellation to the originating capture session. Cancel prompts when their device disappears or the session ends. Prompts expire after one minute and stale actions cannot alter another meeting.
|
||||
- Use the existing native toast toolkit and a neutral no-op prompt implementation. Display the device name with explicit Yes/No actions.
|
||||
- Recheck prompt cancellation after device lookup and at the selection update. Capture sources that own native resources are disposed even when a switch cancels their stream before enumeration starts.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- Device discovery can lag by one polling interval. Capture continues during discovery and while waiting for a response.
|
||||
- Hardware can disappear after acceptance. The existing capture-recovery loop handles creation or capture failures and keeps system audio running.
|
||||
- Switching has an unavoidable short microphone gap. Only microphone capture is canceled; the enclosing recording and transcription are retained.
|
||||
- Windows controls actual notification display duration. One-minute expiration controls action validity; it does not guarantee on-screen persistence.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
No data migration or new configuration is needed. Verify through deterministic capture sources and a Windows build, with native UI/device verification only when it can avoid interrupting a real meeting.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None.
|
||||
@@ -0,0 +1,24 @@
|
||||
## Why
|
||||
|
||||
A microphone connected during a healthy recording is currently only discovered on later device enumeration or capture recovery. Users need an explicit choice to use a newly connected headset without ending the meeting.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Watch active microphone endpoints while microphone capture is running, including paused transcription.
|
||||
- Offer a native Windows notification naming each newly available microphone with Yes/No actions.
|
||||
- Switch only the microphone capture source when accepted, preserving system audio, transcription, and meeting artifacts.
|
||||
- Ignore declined, expired, disconnected-device, and ended-meeting prompts.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
None.
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
- `meeting-recording`: prompt for newly available microphones and safely switch the active microphone on acceptance.
|
||||
|
||||
## Impact
|
||||
|
||||
Microphone capture orchestration, Windows notifications, dependency registration, recording documentation, and deterministic capture/notification tests. Reuses the existing Windows notification toolkit; no new UI framework or configuration migration.
|
||||
@@ -0,0 +1,41 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Newly available microphones can be selected during a meeting
|
||||
While microphone capture for a meeting is active, including while transcription is paused, Meeting Assistant SHALL monitor newly available microphone endpoints and show a native Windows notification naming each newly discovered microphone and offering affirmative and negative actions.
|
||||
|
||||
Devices already available when capture starts SHALL NOT trigger a notification. Each device ID SHALL be offered at most once per meeting. Discovery and unanswered prompts SHALL NOT block audio capture.
|
||||
|
||||
An affirmative response SHALL select the named microphone and replace only the active microphone capture source. The meeting, transcript session, artifacts, transcription pause state, and system-audio capture SHALL remain active. The selection SHALL also be used by later recording starts until another runtime selection or application exit.
|
||||
|
||||
A negative response or expiration SHALL leave microphone selection and capture unchanged. Prompts SHALL expire after one minute and SHALL become invalid when the originating capture ends or the target device disappears. A stale response SHALL NOT change a later meeting's microphone selection.
|
||||
|
||||
#### Scenario: Newly connected microphone is offered
|
||||
- **GIVEN** a meeting is recording from an existing microphone
|
||||
- **WHEN** another microphone becomes available
|
||||
- **THEN** a notification names that microphone and offers Yes and No
|
||||
- **AND** audio continues while the notification is unanswered
|
||||
- **AND** repeated observations of the same device do not produce more notifications
|
||||
|
||||
#### Scenario: User accepts the new microphone
|
||||
- **GIVEN** the notification refers to a still-available microphone in the active meeting
|
||||
- **WHEN** the user chooses Yes
|
||||
- **THEN** subsequent microphone audio comes from that microphone
|
||||
- **AND** the existing meeting, system audio, and transcription session continue
|
||||
|
||||
#### Scenario: User declines or ignores the new microphone
|
||||
- **WHEN** the user chooses No or the prompt expires
|
||||
- **THEN** capture and selection remain on the existing microphone
|
||||
|
||||
#### Scenario: Device disappears or meeting ends before acceptance
|
||||
- **GIVEN** a new-microphone prompt is pending
|
||||
- **WHEN** its device disappears or its meeting capture ends
|
||||
- **THEN** the prompt is canceled and later activation cannot switch the microphone
|
||||
|
||||
#### Scenario: Existing devices do not prompt on start
|
||||
- **WHEN** a meeting starts with multiple available microphones
|
||||
- **THEN** the initial device list becomes the discovery baseline without showing notifications
|
||||
|
||||
#### Scenario: Paused transcription still offers a new microphone
|
||||
- **GIVEN** a meeting's transcription is paused while audio capture remains active
|
||||
- **WHEN** a microphone becomes available and the user accepts its notification
|
||||
- **THEN** the microphone source changes while transcription remains paused
|
||||
@@ -0,0 +1,22 @@
|
||||
## 1. Discovery and prompt lifetime
|
||||
|
||||
- [x] 1.1 Add behavior tests and a monitor that offers newly available endpoints once per capture session without blocking capture.
|
||||
- [x] 1.2 Test and handle rejection, expiration, disappearance, and ended-session responses safely.
|
||||
|
||||
## 2. Capture switching and native UI
|
||||
|
||||
- [x] 2.1 Test and implement accepted microphone switches within the existing capture stream, retaining disconnect recovery.
|
||||
- [x] 2.2 Add native Windows Yes/No notifications and register the monitor and prompt service.
|
||||
- [x] 2.3 Cover cancellation during confirmation lookup and switching or ending the meeting during capture creation; reject stale confirmation and dispose abandoned capture resources.
|
||||
|
||||
## 3. Verification and documentation
|
||||
|
||||
- [x] 3.1 Document discovery, prompt validity, and active switching.
|
||||
- [x] 3.2 Review the implementation, run focused and full solution tests, build Windows, and validate this change strictly.
|
||||
- [x] 3.3 Verify through the safest available operational surface and record any native hardware/UI verification limits.
|
||||
|
||||
Verification: all 11 focused microphone tests and all 534 solution tests pass. The solution build includes the Windows target and succeeds with the four existing NAudio obsolete-API warnings. Strict OpenSpec validation passes. Independent review identified confirmation-cancellation and capture-ownership races; regression tests reproduced them before the fixes, and the follow-up review confirmed both are resolved.
|
||||
|
||||
The capture behavior tests exercise the public audio stream and shared selection with controlled devices and prompts: acceptance replaces samples in the same stream, while decline, expiry, removal, and ended sessions prevent a switch. The running application's recording-status endpoint was checked read-only and reported idle. A physical microphone connection and native toast activation were not tested: no controllable hardware hotplug was available in this session, and the running service was not updated or restarted. Native hardware/UI verification remains a follow-up before operational acceptance or archival.
|
||||
|
||||
Authorized local start (2026-09-25): the current Release build was subsequently published and started using the documented restart helper after another idle check. Health, idle status, tray creation, and model warm-up are successful. The complete solution now has 552 passing tests. Physical microphone hotplug and toast activation remain unverified.
|
||||
@@ -139,7 +139,11 @@ When Azure Speech is still unavailable after recording stops and transcription c
|
||||
|
||||
When a stopped meeting is persisted to the durable transcription backlog, Meeting Assistant SHALL release the active recording slot so another meeting can be recorded while the stopped meeting waits for Azure Speech to become available.
|
||||
|
||||
When Azure Speech becomes available again, Meeting Assistant SHALL retry durable backlog items, rewrite the transcript from the recorded WAV, run the normal post-transcription meeting completion and summary flow, and remove the backlog item after successful completion.
|
||||
When Azure Speech becomes available again, Meeting Assistant SHALL retry durable backlog items, append the recovered transcript from the recorded WAV without replacing the existing transcript body, run the normal post-transcription meeting completion and summary flow, and remove the backlog item after successful completion.
|
||||
|
||||
Before appending recovered lines, Meeting Assistant SHALL insert a visible recovery marker explaining that the following section was recovered by replaying the entire recording and may duplicate earlier transcript passages. The marker SHALL tell the summarizer to treat repeated passages as the same discussion. The marker and appended lines SHALL be available to the summary agent together with the preserved earlier transcript. Existing transcript text, speaker labels, and user edits SHALL remain unchanged.
|
||||
|
||||
An offline replay that returns no speech segments with non-blank text SHALL be treated as an unsuccessful attempt. This includes empty results, blank speech segments, and results containing only status markers. Meeting Assistant SHALL preserve the existing transcript and meeting artifacts unchanged, SHALL skip meeting completion and summarization, and SHALL retain the durable backlog item and its WAV for a later retry.
|
||||
|
||||
When Meeting Assistant starts, it SHALL preserve WAV files that are referenced by durable transcription backlog items instead of deleting them as stale temporary recordings.
|
||||
|
||||
@@ -185,10 +189,28 @@ When Meeting Assistant starts, it SHALL preserve WAV files that are referenced b
|
||||
#### Scenario: Durable Azure backlog resumes after connectivity returns
|
||||
- **GIVEN** a stopped Azure meeting exists in the durable transcription backlog
|
||||
- **WHEN** Azure Speech can transcribe the recorded WAV
|
||||
- **THEN** Meeting Assistant rewrites the transcript from the recorded WAV
|
||||
- **THEN** Meeting Assistant preserves the existing transcript body and appends a recovery marker followed by the transcript from the recorded WAV
|
||||
- **AND** the marker explains the recovery and possible duplicates to the summary agent
|
||||
- **AND** runs meeting completion and summarization
|
||||
- **AND** removes the durable backlog item and its temporary WAV after successful completion
|
||||
|
||||
#### Scenario: Empty offline replay preserves the live transcript and recovery files
|
||||
- **GIVEN** a stopped Azure meeting has an existing live transcript and a durable backlog item referencing its WAV
|
||||
- **WHEN** the offline replay returns no transcript segments
|
||||
- **THEN** Meeting Assistant leaves the transcript and meeting artifacts unchanged
|
||||
- **AND** does not run meeting completion or summarization
|
||||
- **AND** retains the backlog item and WAV for the next retry
|
||||
- **WHEN** a later retry produces a transcript from the WAV
|
||||
- **THEN** Meeting Assistant appends the recovery marker and recovered transcript without replacing earlier content and completes the meeting normally
|
||||
- **AND** removes the backlog item and WAV only after that successful completion
|
||||
|
||||
#### Scenario: Offline status markers or blank speech do not count as successful transcription
|
||||
- **GIVEN** a stopped Azure meeting has an existing live transcript and a durable backlog item referencing its WAV
|
||||
- **WHEN** the offline replay returns only status markers or speech segments with blank text
|
||||
- **THEN** Meeting Assistant preserves the existing transcript and meeting artifacts
|
||||
- **AND** skips meeting completion and summarization
|
||||
- **AND** retains the backlog item and WAV for another retry
|
||||
|
||||
### Requirement: FunASR can provide speaker-attributed streaming transcription
|
||||
Meeting Assistant SHALL provide a FunASR speech recognition pipeline that streams PCM audio to a configured FunASR WebSocket endpoint.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user