Public Access
Archive meeting workflow and screenshot changes
This commit is contained in:
@@ -0,0 +1,12 @@
|
||||
# Change: Add meeting screenshots
|
||||
|
||||
## Why
|
||||
Meetings often contain visual information that is not captured by audio transcription, such as slides, diagrams, shared screens, and active presenter context. Meeting Assistant should let the user capture the active window during a meeting and store that visual context alongside the assistant context note.
|
||||
|
||||
## What Changes
|
||||
- Add a configurable screenshot hotkey.
|
||||
- Capture the currently active window and save the image under the assistant context attachments folder.
|
||||
- Append a timestamped markdown image link to the assistant context note so the screenshot can be correlated with the transcript timeline.
|
||||
- Optionally send screenshots to a configured OCR/vision model and append the extracted result next to the screenshot entry.
|
||||
- Delay summary generation until pending screenshot OCR jobs finish or time out.
|
||||
|
||||
+125
@@ -0,0 +1,125 @@
|
||||
## MODIFIED Requirements
|
||||
### Requirement: Meeting summary artifacts include assistant context
|
||||
Meeting Assistant SHALL generate a summary, decisions, and next steps from the meeting transcript, metadata, user notes, and assistant-discovered context.
|
||||
|
||||
Meeting Assistant SHALL use a Microsoft Agent Framework pipeline for the first summary implementation. The pipeline SHALL use a configurable OpenAI-compatible endpoint, model, and API key source, including support for a direct key or an environment variable name.
|
||||
|
||||
The summary pipeline SHALL retry transient model endpoint failures according to configurable reconnection attempts and delay settings.
|
||||
|
||||
The summary pipeline SHALL track context-window usage for the configured model using response usage when available and request-size estimates otherwise. The context-window limit, maximum output reserve, compaction enablement, compaction threshold, and Responses compact endpoint path SHALL be configurable.
|
||||
|
||||
When only the configured remaining context ratio is available, the summary pipeline SHALL try to compact the conversation through the configured OpenAI-compatible `POST /v1/responses/compact` endpoint. If that endpoint is unavailable or returns invalid data, it SHALL fall back to Microsoft Agent Framework compaction.
|
||||
|
||||
When summary generation fails after retries, Meeting Assistant SHALL write a markdown failure report to the configured summary note path. The failure report SHALL include a clickable retry link, error details, and the meeting artifact paths needed to diagnose or retry the run.
|
||||
|
||||
After transcript processing finishes for a recording, Meeting Assistant SHALL automatically invoke the summary pipeline for that meeting.
|
||||
|
||||
Before automatic summary generation starts, Meeting Assistant SHALL wait for pending screenshot OCR jobs for that meeting to complete or reach their configured timeout.
|
||||
|
||||
Meeting Assistant SHALL expose an API operation to retry summary generation for a given summary note path. The retry operation SHALL resolve the linked meeting artifacts from the meeting note frontmatter and SHALL overwrite the existing summary note with either the new summary or a new failure report.
|
||||
|
||||
The summary pipeline SHALL expose tools scoped to the current meeting:
|
||||
|
||||
- `read_meetingnote`
|
||||
- `read_transcript`
|
||||
- `read_context`
|
||||
- `read_usernotes`
|
||||
- `read_glossary`
|
||||
- `add_dictation_word`
|
||||
- `write_context`
|
||||
- `write_summary`
|
||||
- `list_projects`
|
||||
- `list_projectfiles`
|
||||
- `read_projectfile`
|
||||
- `write_projectfile`
|
||||
- `search`
|
||||
|
||||
All summary-agent read tools that return file-like content SHALL support either reading the whole file or reading a clamped inclusive 1-based line range when `from` and `to` are supplied.
|
||||
|
||||
The summary pipeline SHALL write the summary note as a markdown artifact linked to the meeting note, transcript, and assistant context.
|
||||
|
||||
The summary instructions SHALL tell the summary agent to include only the most relevant cropped screenshot links from assistant context in the summary, placing them near the related summary text.
|
||||
|
||||
The summary agent SHALL be able to read the meeting note, transcript, assistant context, glossary, and bound project files through tools.
|
||||
|
||||
The summary agent SHALL be able to write the summary and assistant context files as its owned artifacts.
|
||||
|
||||
For summary-agent writes to any file other than the summary file and assistant context file, Meeting Assistant SHALL record an in-memory diff containing removed and added lines.
|
||||
|
||||
When the summary agent finishes, Meeting Assistant SHALL append the collected external write diffs to the assistant context file.
|
||||
|
||||
After writing the meeting summary, the summary agent SHALL update existing project files when the meeting produced project-relevant knowledge.
|
||||
|
||||
The assistant context note SHALL keep `title`, `start_time`, `end_time`, `meeting`, `transcript`, `summary`, `state`, and `agenda` in frontmatter. Meeting Assistant SHALL write `title` and `start_time` when the assistant context note is created, update `title` when later meeting metadata is discovered, and write `end_time` when transcript processing stops. When a Teams appointment is detected at recording start, the assistant context note SHALL also keep `scheduled_end` in frontmatter. The `state` value SHALL be one of `collecting metadata`, `transcribing`, `speaker recognition`, `summarizing`, `finished`, or `error`.
|
||||
|
||||
The summary note SHALL keep frontmatter links to meeting artifacts and SHALL copy `title`, `start_time`, and `end_time` from the meeting note when available. If the meeting note has no title, the summary agent SHALL be able to provide a title when writing the summary.
|
||||
|
||||
The summary agent SHALL be able to read and write the assistant context body as its own notebook using overwrite, replace, and insert line modes.
|
||||
|
||||
#### Scenario: Assistant context note is created
|
||||
- **WHEN** Meeting Assistant creates a meeting session
|
||||
- **THEN** it creates the assistant context note with frontmatter links to the meeting note, transcript note, and summary note
|
||||
- **AND** the assistant context state is `collecting metadata`
|
||||
|
||||
#### Scenario: Assistant context state tracks processing
|
||||
- **WHEN** transcription, final speaker recognition, and summary generation progress
|
||||
- **THEN** Meeting Assistant updates the assistant context state through `transcribing`, `speaker recognition`, `summarizing`, `finished`, or `error`
|
||||
|
||||
#### Scenario: Summary waits for pending screenshot OCR
|
||||
- **GIVEN** screenshot OCR is still running for a meeting
|
||||
- **WHEN** transcript processing is ready to run the automatic summary
|
||||
- **THEN** Meeting Assistant waits for the screenshot OCR to complete before moving the assistant context to `summarizing`
|
||||
- **AND** proceeds after the configured OCR timeout if OCR does not finish
|
||||
|
||||
#### Scenario: Summary can include relevant cropped screenshots
|
||||
- **GIVEN** assistant context contains cropped screenshot markdown links
|
||||
- **WHEN** Meeting Assistant builds summary-agent instructions
|
||||
- **THEN** the instructions tell the summary agent to include only the most relevant cropped screenshots in the summary
|
||||
- **AND** to markdown-link them near the related summary text
|
||||
|
||||
### Requirement: Meeting screenshots are captured into assistant context
|
||||
Meeting Assistant SHALL expose a configurable screenshot hotkey.
|
||||
|
||||
When a meeting is active and the screenshot hotkey is pressed, Meeting Assistant SHALL capture the currently active window.
|
||||
|
||||
The screenshot image SHALL be saved into a configurable attachments folder for the assistant context note. By default, the folder SHALL be `Attachments` beside the assistant context note.
|
||||
|
||||
After the image is saved, Meeting Assistant SHALL append a markdown image link to the assistant context note with a meeting-relative timestamp that correlates to transcript timestamps.
|
||||
|
||||
Meeting Assistant SHALL allow optional screenshot OCR configuration with endpoint URL, API key or key environment variable, model, prompt, and timeout.
|
||||
|
||||
When screenshot OCR is configured, Meeting Assistant SHALL send the screenshot and prompt to the configured OpenAI-compatible Responses endpoint and append the model result after the screenshot link in the assistant context note.
|
||||
|
||||
The screenshot OCR prompt SHALL ask the model to return pixel crop coordinates when it can confidently isolate only the presentation, shared screen, or similarly relevant meeting content.
|
||||
|
||||
When OCR returns valid crop coordinates within the original image bounds, Meeting Assistant SHALL save a cropped PNG beside the original screenshot and SHALL link the cropped image before the OCR result in the assistant context note.
|
||||
|
||||
When OCR returns no crop coordinates or invalid crop coordinates, Meeting Assistant SHALL keep the original screenshot link and OCR result without writing a cropped image.
|
||||
|
||||
When screenshot OCR is not configured, Meeting Assistant SHALL skip OCR and keep the screenshot link.
|
||||
|
||||
The default OCR prompt SHALL explain that the image is from a meeting and ask the model to identify who is talking, who is presenting, what is presented, capture slide text in markdown, convert diagrams to Mermaid when possible, return crop coordinates only for confidently isolated presentation/shared-screen content, and otherwise describe the scene.
|
||||
|
||||
#### Scenario: Screenshot is linked with meeting timestamp
|
||||
- **GIVEN** a meeting started at `10:00:00`
|
||||
- **WHEN** the user captures a screenshot at `10:03:05`
|
||||
- **THEN** Meeting Assistant saves the screenshot under the configured attachments folder
|
||||
- **AND** appends a markdown image link to assistant context with timestamp `[00:03:05]`
|
||||
|
||||
#### Scenario: OCR result is appended after screenshot
|
||||
- **GIVEN** screenshot OCR is configured
|
||||
- **WHEN** the user captures a screenshot
|
||||
- **THEN** Meeting Assistant appends the screenshot link to assistant context
|
||||
- **AND** appends the OCR result for that screenshot after the link when processing completes
|
||||
|
||||
#### Scenario: OCR crop is saved and linked before OCR text
|
||||
- **GIVEN** screenshot OCR is configured
|
||||
- **AND** OCR returns valid crop coordinates for a shared screen
|
||||
- **WHEN** OCR processing completes
|
||||
- **THEN** Meeting Assistant saves a cropped screenshot beside the original image
|
||||
- **AND** links the cropped screenshot before the OCR text in assistant context
|
||||
|
||||
#### Scenario: OCR is skipped when not configured
|
||||
- **GIVEN** screenshot OCR is not configured
|
||||
- **WHEN** the user captures a screenshot
|
||||
- **THEN** Meeting Assistant saves and links the screenshot without calling a model endpoint
|
||||
@@ -0,0 +1,14 @@
|
||||
## 1. Implementation
|
||||
- [x] 1.1 Add screenshot and OCR configuration options.
|
||||
- [x] 1.2 Capture active-window screenshots through a Windows adapter and testable abstraction.
|
||||
- [x] 1.3 Save screenshots to the configured assistant-context attachments folder.
|
||||
- [x] 1.4 Append timestamped screenshot markdown blocks to assistant context.
|
||||
- [x] 1.5 Add optional OCR/vision processing with timeout and default prompt.
|
||||
- [x] 1.6 Wait for pending screenshot OCR before summary generation.
|
||||
- [x] 1.7 Register a configurable screenshot hotkey.
|
||||
|
||||
## 2. Verification
|
||||
- [x] 2.1 Cover screenshot save/link behavior.
|
||||
- [x] 2.2 Cover disabled and configured OCR behavior.
|
||||
- [x] 2.3 Cover summary waiting for pending OCR.
|
||||
- [x] 2.4 Run relevant tests and strict OpenSpec validation.
|
||||
@@ -0,0 +1,14 @@
|
||||
# Change: Add recording abort hotkey
|
||||
|
||||
## Why
|
||||
Users need a fast way to discard an accidental or unwanted active meeting recording without producing a transcript summary or leaving meeting artifacts in the vault.
|
||||
|
||||
## What Changes
|
||||
- Add a configurable abort/discard hotkey, defaulting to `Ctrl+Alt+D`.
|
||||
- Add an abort recording operation that stops active capture/transcription, skips final summary generation, and removes the meeting artifacts created for that run.
|
||||
- Keep normal stop behavior unchanged.
|
||||
|
||||
## Impact
|
||||
- Recording coordinator abort path
|
||||
- Global hotkey registration and launch-profile hotkey handling
|
||||
- Recording control endpoints and local configuration
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
## MODIFIED Requirements
|
||||
### Requirement: Speech recognition pipelines are streamable and configurable
|
||||
Meeting Assistant SHALL route audio through a provider-independent speech recognition pipeline contract.
|
||||
|
||||
The configured pipeline SHALL be selected through DI from the ASR backend configuration and SHALL expose operations to initialize, wait for readiness, write captured audio chunks, read live transcript segments, and read the finished transcript after capture completes.
|
||||
|
||||
Meeting Assistant SHALL expose a configurable abort/discard hotkey for an active recording.
|
||||
|
||||
When aborting an active recording, Meeting Assistant SHALL stop active audio capture and transcription, SHALL remove the meeting note, transcript, assistant context, summary artifact, and linked screenshot attachment files created for that run, and SHALL NOT invoke automatic summary generation.
|
||||
|
||||
When no recording is active, aborting SHALL leave the current recording status unchanged.
|
||||
|
||||
#### Scenario: Active recording is aborted and discarded
|
||||
- **GIVEN** a meeting recording is active and has created meeting artifacts
|
||||
- **WHEN** the user aborts the recording
|
||||
- **THEN** Meeting Assistant stops recording
|
||||
- **AND** deletes the meeting note, transcript, assistant context, summary artifact if present, and linked screenshot attachment files for that run
|
||||
- **AND** does not invoke the summary pipeline
|
||||
|
||||
#### Scenario: Abort does nothing when idle
|
||||
- **GIVEN** no meeting recording is active
|
||||
- **WHEN** the user aborts recording
|
||||
- **THEN** Meeting Assistant remains idle
|
||||
@@ -0,0 +1,10 @@
|
||||
## 1. Implementation
|
||||
- [x] 1.1 Add configurable abort hotkey with launch-profile registration.
|
||||
- [x] 1.2 Add recording abort operation/API that stops active capture and transcription.
|
||||
- [x] 1.3 Delete meeting artifacts and linked screenshot attachments for aborted runs.
|
||||
- [x] 1.4 Ensure abort does not transition to summarizing or invoke the summary pipeline.
|
||||
|
||||
## 2. Verification
|
||||
- [x] 2.1 Cover abort deleting artifacts and skipping summary.
|
||||
- [x] 2.2 Cover abort hotkey registration.
|
||||
- [x] 2.3 Run relevant tests and strict OpenSpec validation.
|
||||
@@ -0,0 +1,13 @@
|
||||
# Change: Limit Outlook metadata attendee import
|
||||
|
||||
## Why
|
||||
Large Outlook meetings often represent presentations or broadcasts. Importing every invited attendee into the meeting note makes speaker matching noisy because most invitees are unlikely to speak.
|
||||
|
||||
## What Changes
|
||||
- Add a configurable maximum attendee count for meeting metadata imports.
|
||||
- Keep applying meeting title, agenda, and scheduled end metadata when the attendee list is too large.
|
||||
- Skip writing metadata attendees when the raw metadata attendee count is above the configured maximum.
|
||||
|
||||
## Impact
|
||||
- Affects meeting-session metadata enrichment behavior.
|
||||
- Reduces false speaker candidates for large presentation-style meetings.
|
||||
+30
@@ -0,0 +1,30 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Windows Outlook enrichment is optional
|
||||
Meeting Assistant SHALL gate Outlook Classic COM enrichment behind the Windows compilation target.
|
||||
|
||||
When the Windows build starts a meeting and Outlook Classic has exactly one current Teams appointment, Meeting Assistant SHALL copy the appointment title to the meeting note and copy the appointment agenda and scheduled end time to the assistant-context frontmatter.
|
||||
|
||||
Meeting Assistant SHALL copy the appointment attendees to the meeting note only when the raw appointment attendee count is less than or equal to the configured `Recording:MaxMetadataAttendeeImportCount`. The default maximum SHALL be 30 attendees.
|
||||
|
||||
The agenda SHALL be extracted from the appointment body content before the Teams join separator or Teams join text.
|
||||
|
||||
#### Scenario: Current Teams appointment enriches meeting artifacts
|
||||
- **WHEN** a Windows build starts a meeting while Outlook Classic exposes exactly one current Teams appointment
|
||||
- **THEN** Meeting Assistant uses the appointment subject as the meeting title
|
||||
- **AND** writes the appointment attendees into meeting note frontmatter when the raw attendee count is within the configured import limit
|
||||
- **AND** writes the appointment agenda into assistant-context frontmatter
|
||||
- **AND** writes the appointment end time as `scheduled_end` into assistant-context frontmatter
|
||||
|
||||
#### Scenario: Oversized attendee list is not imported
|
||||
- **GIVEN** the configured metadata attendee import limit is 30
|
||||
- **WHEN** a Windows build starts a meeting while Outlook Classic exposes exactly one current Teams appointment with 31 attendees
|
||||
- **THEN** Meeting Assistant uses the appointment subject as the meeting title
|
||||
- **AND** does not write the appointment attendees into meeting note frontmatter
|
||||
- **AND** writes the appointment agenda into assistant-context frontmatter
|
||||
- **AND** writes the appointment end time as `scheduled_end` into assistant-context frontmatter
|
||||
|
||||
#### Scenario: Outlook is unavailable or ambiguous
|
||||
- **WHEN** Outlook Classic is unavailable or more than one current Teams appointment is found
|
||||
- **THEN** Meeting Assistant starts the recording with the default generated meeting title and empty agenda
|
||||
- **AND** omits `scheduled_end` from assistant-context frontmatter
|
||||
@@ -0,0 +1,9 @@
|
||||
## 1. Behavior
|
||||
- [x] 1.1 Add meeting-session requirement for capped metadata attendee import.
|
||||
- [x] 1.2 Add behavior test proving oversized metadata attendee lists are skipped.
|
||||
- [x] 1.3 Add configurable default maximum attendee count.
|
||||
- [x] 1.4 Preserve title, agenda, and scheduled end enrichment when attendees are skipped.
|
||||
|
||||
## 2. Verification
|
||||
- [x] 2.1 Run focused recording coordinator tests.
|
||||
- [x] 2.2 Run `openspec validate limit-outlook-metadata-attendees --strict`.
|
||||
Reference in New Issue
Block a user