Improve meeting audio mixing pipeline

2026-05-29 13:47:21 +02:00
parent 9d0f6a1295
commit 740f93f185
18 changed files with 870 additions and 136 deletions
@@ -35,10 +35,68 @@ When no recording is active, aborting SHALL leave the current recording status u
 ### Requirement: Recording mode captures microphone and computer output
 Meeting Assistant SHALL capture microphone input and computer output and combine them into one audio stream for transcription.

+Meeting Assistant SHALL capture audio as 16 kHz mono PCM chunks for the existing recording and transcription pipeline.
+
+Meeting Assistant SHALL capture microphone and system loopback as separate input streams before producing the final mono chunks.
+
+Meeting Assistant SHALL clean the microphone stream with a local acoustic echo cancellation stage that uses system loopback as the far-end reference.
+
+Meeting Assistant SHALL produce final mono chunks by adding the cleaned microphone samples and system samples.
+
+Meeting Assistant SHALL align microphone and system samples through per-source buffers before mixing and SHALL NOT emit normal live audio chunks that contain only one source while the other source is merely delayed.
+
+When one source stays quiet beyond the alignment timeout, Meeting Assistant SHALL mix the available source with synthetic silence for the missing source instead of blocking transcription.
+
+Meeting Assistant SHALL allow the final microphone/system mono mix to apply configurable microphone and system gain before combining samples.
+
+Meeting Assistant SHALL use the active run or launch profile recording options when configuring capture format and final microphone/system gains.
+
+Meeting Assistant SHALL clamp mixed samples after gain is applied.
+
+Meeting Assistant SHALL write only the mixed stream to the temporary WAV used by transcription and finalization.
+
 #### Scenario: Both sources produce audio
 - **WHEN** microphone and computer output audio chunks are available
 - **THEN** Meeting Assistant mixes them into one PCM stream before transcription

+#### Scenario: Mixed audio uses cleaned microphone and system audio
+- **GIVEN** the echo canceller cleans a microphone chunk to sample `2000`
+- **AND** the matching system chunk has sample `10000`
+- **WHEN** microphone and system chunks are mixed with gains `1` and `1`
+- **THEN** the mixed sample is `12000`
+
+#### Scenario: Temporary recording stores only the mixed stream
+- **GIVEN** Meeting Assistant has mixed microphone and system audio into one PCM chunk
+- **WHEN** Meeting Assistant appends the chunk to the temporary recording
+- **THEN** the main temporary WAV contains the mixed PCM
+- **AND** no microphone or system sidecar WAV is written
+
+#### Scenario: Launch profile recording options configure capture and gains
+- **GIVEN** an active launch profile configures sample format and microphone/system mix gains
+- **WHEN** Meeting Assistant captures and mixes audio for that run
+- **THEN** the microphone and system capture sources receive that launch profile recording configuration
+- **AND** the mixed output uses that launch profile's microphone/system gains
+
+#### Scenario: Delayed sources are buffered before mixing
+- **GIVEN** microphone audio arrives before matching system audio
+- **WHEN** matching system audio arrives after a short delay
+- **THEN** Meeting Assistant emits one mixed chunk for the aligned samples
+- **AND** it does not emit separate microphone-only and system-only chunks for that delayed pair
+
+#### Scenario: Quiet system audio does not block microphone transcription
+- **GIVEN** microphone audio arrives
+- **AND** system loopback audio does not arrive within the alignment timeout
+- **WHEN** Meeting Assistant mixes the available audio
+- **THEN** it emits the microphone audio mixed with silent system audio
+- **AND** live transcription can continue while system loopback is quiet
+
+#### Scenario: Continuous microphone audio does not suppress the alignment timeout
+- **GIVEN** microphone audio keeps arriving
+- **AND** system loopback audio stays unavailable past the alignment timeout
+- **WHEN** Meeting Assistant checks the buffered microphone audio
+- **THEN** it emits the buffered microphone audio mixed with silent system audio
+- **AND** it does not wait indefinitely for a loopback chunk
+
 #### Scenario: Device-level capture cannot be verified in tests
 - **WHEN** automated tests run without live audio devices
 - **THEN** Meeting Assistant verifies the audio mixer through deterministic source abstractions rather than depending on physical microphone or speaker devices