Preserve recovery transcripts and add user-approved agent requests

Append successful offline recovery with a duplicate-warning marker and retain failed replay sources. Offer newly available microphones through confirmation notifications. Add project discovery metadata and blocking association approval, plus nonblocking workflow requests that start a separate settings-agent conversation. Include regression tests, OpenSpec changes, and verified local Release startup.
2026-09-25 11:58:34 +02:00
parent e011e36ef7
commit 7b2bcd3631
50 changed files with 2188 additions and 113 deletions
@@ -24,4 +24,8 @@ After configured reconnect attempts are exhausted, Azure should emit a longer di

 If a stopped Azure Speech meeting cannot finish draining before `Recording:StopProcessingTimeout`, the coordinator queues the completed temporary WAV and artifact paths in a persisted offline backlog, releases the active recording slot, and keeps the WAV from startup cleanup. This lets the user start more meetings while Azure or the network is still unavailable.

-The backlog processor retries queued items in the background. Each item creates a fresh speech recognition pipeline for the original launch profile, streams the queued WAV into that pipeline, rewrites the original transcript file with the finished lines, updates meeting-note and transcript metadata, runs transcript-line workflow transformations, transitions the assistant context through summarizing to finished/error, runs the normal summary pipeline, then removes the backlog item and deletes the temporary WAV. Failed processing leaves the backlog item and WAV in place for a later retry.
+The backlog processor retries queued items in the background. Each item creates a fresh speech recognition pipeline for the original launch profile and streams the queued WAV into that pipeline. Results without non-blank speech text leave all artifacts and recovery files unchanged for a later retry.
+
+A usable replay preserves the existing transcript body and appends a visible recovery section before updating meeting metadata and running the normal summary pipeline. The recovery marker explains that the full recording was replayed, earlier passages may be duplicated, and repeated content should be treated as the same discussion when summarizing. Recovered lines still receive transcript-line workflow transformations; the explanatory marker is written directly so it remains visible. The application does not try to deduplicate or reconcile live and recovered recognition results.
+
+After appending, the processor updates meeting-note and transcript metadata, transitions the assistant context through summarizing to finished/error, and runs the normal summary pipeline. Completed processing removes the backlog item and temporary WAV. Failed processing leaves both in place for a later retry; an additional replay may append another clearly marked recovery section without discarding earlier content.
@@ -27,7 +27,11 @@ When Azure Speech is still unavailable after recording stops and transcription c

 When a stopped meeting is persisted to the durable transcription backlog, Meeting Assistant SHALL release the active recording slot so another meeting can be recorded while the stopped meeting waits for Azure Speech to become available.

-When Azure Speech becomes available again, Meeting Assistant SHALL retry durable backlog items, rewrite the transcript from the recorded WAV, run the normal post-transcription meeting completion and summary flow, and remove the backlog item after successful completion.
+When Azure Speech becomes available again, Meeting Assistant SHALL retry durable backlog items, append the recovered transcript from the recorded WAV without replacing the existing transcript body, run the normal post-transcription meeting completion and summary flow, and remove the backlog item after successful completion.
+
+Before appending recovered lines, Meeting Assistant SHALL insert a visible recovery marker explaining that the following section was recovered by replaying the entire recording and may duplicate earlier transcript passages. The marker SHALL tell the summarizer to treat repeated passages as the same discussion. The marker and appended lines SHALL be available to the summary agent together with the preserved earlier transcript. Existing transcript text, speaker labels, and user edits SHALL remain unchanged.
+
+An offline replay that returns no speech segments with non-blank text SHALL be treated as an unsuccessful attempt. This includes empty results, blank speech segments, and results containing only status markers. Meeting Assistant SHALL preserve the existing transcript and meeting artifacts unchanged, SHALL skip meeting completion and summarization, and SHALL retain the durable backlog item and its WAV for a later retry.

 When Meeting Assistant starts, it SHALL preserve WAV files that are referenced by durable transcription backlog items instead of deleting them as stale temporary recordings.

@@ -73,6 +77,24 @@ When Meeting Assistant starts, it SHALL preserve WAV files that are referenced b
 #### Scenario: Durable Azure backlog resumes after connectivity returns
 - **GIVEN** a stopped Azure meeting exists in the durable transcription backlog
 - **WHEN** Azure Speech can transcribe the recorded WAV
- **THEN** Meeting Assistant rewrites the transcript from the recorded WAV
+- **THEN** Meeting Assistant preserves the existing transcript body and appends a recovery marker followed by the transcript from the recorded WAV
+- **AND** the marker explains the recovery and possible duplicates to the summary agent
 - **AND** runs meeting completion and summarization
 - **AND** removes the durable backlog item and its temporary WAV after successful completion
+
+#### Scenario: Empty offline replay preserves the live transcript and recovery files
+- **GIVEN** a stopped Azure meeting has an existing live transcript and a durable backlog item referencing its WAV
+- **WHEN** the offline replay returns no transcript segments
+- **THEN** Meeting Assistant leaves the transcript and meeting artifacts unchanged
+- **AND** does not run meeting completion or summarization
+- **AND** retains the backlog item and WAV for the next retry
+- **WHEN** a later retry produces a transcript from the WAV
+- **THEN** Meeting Assistant appends the recovery marker and recovered transcript without replacing earlier content and completes the meeting normally
+- **AND** removes the backlog item and WAV only after that successful completion
+
+#### Scenario: Offline status markers or blank speech do not count as successful transcription
+- **GIVEN** a stopped Azure meeting has an existing live transcript and a durable backlog item referencing its WAV
+- **WHEN** the offline replay returns only status markers or speech segments with blank text
+- **THEN** Meeting Assistant preserves the existing transcript and meeting artifacts
+- **AND** skips meeting completion and summarization
+- **AND** retains the backlog item and WAV for another retry
@@ -9,3 +9,17 @@
 - [x] Run focused tests, full solution tests, and `openspec validate azure-speech-offline-resilience --strict`.
 - [x] Add a durable offline backlog for stopped Azure meetings across process restarts.
 - [x] Replay queued WAV files through a fresh speech pipeline and finish transcript metadata, meeting context state, summary generation, and backlog cleanup.
+
+## Bug-fix addendum: Preserve transcripts during offline replay
+
+- [x] Specify that replay without speech text preserves the transcript, backlog, and WAV until a successful retry.
+- [x] Reproduce the data-loss bug with real transcript and backlog files, then reject empty, blank-text, and marker-only replay results before any artifact writes.
+- [x] Verify retained files survive repeated retries and a later successful replay still completes and cleans up normally.
+- [x] Run focused regression tests, full solution tests, and strict OpenSpec validation.
+
+- [x] Preserve existing transcript content on successful replay and append a recovery marker explaining possible duplicates before the recovered text.
+- [x] Verify the summary agent receives earlier text, the recovery explanation, and appended text together, then rerun the relevant tests and strict validation.
+
+Verification: the regression cases exercise the backlog processor with real WAV, transcript, meeting-note, and durable JSON files. Each case checks byte-for-byte preservation through two unsuccessful attempts and reloads the backlog from disk. A later usable replay preserves the original body, including a user correction without a trailing newline, and appends the marker and recovered text. A transcript-reading summary stub verifies that both versions, including an intentional duplicate passage, are available with the explanation before backlog/WAV cleanup. All 525 solution tests and strict validation of this change and the accepted transcription spec pass; independent review found no actionable issues. The Azure response is deterministic test input; no live Azure outage was induced and the running workstation service was not restarted.
+
+Authorized local start (2026-09-25): the Release build containing this fix was subsequently published and started after confirming the workstation instance was idle. Health and startup logs are successful; the full solution now has 552 passing tests. No live Azure outage was induced.