Manuel/meeting-assistant
Public Access
1
0
Fork 0
Code Issues Pull Requests 4 Actions Packages Projects Releases Wiki Activity Open in Coder

Archive meeting workflow and screenshot changes

Browse Source
This commit is contained in:
codex
2026-05-27 12:55:19 +02:00
parent 2422236ef7
commit 12832dde84
48 changed files with 3041 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
openspec/specs/meeting-summary/spec.md
+84 -3
View File
@@ -2,6 +2,7 @@
## Purpose
TBD - created by archiving change define-meeting-assistant-v1. Update Purpose after archive.
## Requirements
### Requirement: Meeting Assistant generates meeting outputs
Meeting Assistant SHALL generate a summary, decisions, and next steps from the meeting transcript, metadata, user notes, and assistant-discovered context.
@@ -18,7 +19,9 @@ Fallback compaction SHALL become increasingly aggressive: collapse old tool resu
When summary generation fails after retries, Meeting Assistant SHALL write a markdown failure report to the configured summary note path. The failure report SHALL include a clickable retry link, error details, and the meeting artifact paths needed to diagnose or retry the run.
After transcript processing finishes for a recording, Meeting Assistant SHALL automatically invoke the summary pipeline for that meeting.
After transcript processing finishes for a recording, Meeting Assistant SHALL automatically invoke the summary pipeline for the meeting.
Before automatic summary generation starts, Meeting Assistant SHALL wait for pending screenshot OCR jobs for that meeting to complete or reach their configured timeout.
Meeting Assistant SHALL expose an API operation to retry summary generation for a given summary note path. The retry operation SHALL resolve the linked meeting artifacts from the meeting note frontmatter and SHALL overwrite the existing summary note with either the new summary or a new failure report.
@@ -29,6 +32,7 @@ The summary pipeline SHALL expose tools scoped to the current meeting:
- `read_context`
- `read_usernotes`
- `read_glossary`
- `add_dictation_word`
- `write_summary`
- `write_context`
- `list_projects`
@@ -41,6 +45,8 @@ All summary-agent read tools that return file-like content SHALL support either
The summary pipeline SHALL write the summary note as a markdown artifact linked to the meeting note, transcript, and assistant context.
The summary instructions SHALL tell the summary agent to include only the most relevant cropped screenshot links from assistant context in the summary, placing them near the related summary text.
The summary agent SHALL be able to read the meeting note, transcript, assistant context, glossary, and bound project files through tools.
The summary agent SHALL be able to write the summary and assistant context files as its owned artifacts.
@@ -51,11 +57,13 @@ Meeting Assistant SHALL use a diff implementation that can reduce whole-file rew
When the summary agent finishes, Meeting Assistant SHALL append the collected external write diffs to the assistant context file.
After writing the meeting summary, the summary agent SHALL update existing project files when the meeting produced project-relevant knowledge.
After writing the meeting summary, the summary agent SHALL update existing project files when the meeting produced durable project knowledge, decisions, next steps, or context.
The assistant context note SHALL keep `title`, `start_time`, `end_time`, `meeting`, `transcript`, `summary`, `state`, and `agenda` in frontmatter. Meeting Assistant SHALL write `title` and `start_time` when the assistant context note is created, update `title` when later meeting metadata is discovered, and write `end_time` when transcript processing stops. When a Teams appointment is detected at recording start, the assistant context note SHALL also keep `scheduled_end` in frontmatter. The `state` value SHALL be one of `collecting metadata`, `transcribing`, `speaker recognition`, `summarizing`, `finished`, or `error`.
The summary note SHALL keep frontmatter links to meeting artifacts and SHALL copy `title`, `start_time`, and `end_time` from the meeting note when available. If the meeting note has no title, the summary agent SHALL be able to provide a title when writing the summary.
The summary note SHALL keep frontmatter links to meeting artifacts and SHALL copy `title`, `start_time`, `end_time`, and `attendees` from the meeting note when available. If the meeting note has no title, the summary agent SHALL be able to provide a title when writing the summary.
When the summary note already has an `attendees` frontmatter property, Meeting Assistant SHALL preserve the existing summary attendees instead of overwriting them from the meeting note.
The summary agent SHALL be able to read and write the assistant context body as its own notebook using overwrite, replace, and insert line modes.
@@ -71,6 +79,27 @@ The summary agent SHALL be able to read and write the assistant context body as
- **THEN** Meeting Assistant updates assistant context frontmatter state to `collecting metadata`, `transcribing`, `speaker recognition`, `summarizing`, `finished`, or `error` as appropriate
- **AND** preserves existing `agenda` and `scheduled_end` frontmatter values
#### Scenario: Summary copies final meeting attendees
- **WHEN** the summary agent writes a summary note after meeting attendees have changed
- **THEN** the summary note frontmatter includes the final meeting note attendees
#### Scenario: Existing summary attendees are preserved
- **GIVEN** an existing summary note has `attendees` in frontmatter
- **WHEN** the summary agent rewrites the summary
- **THEN** Meeting Assistant preserves the existing summary attendees
#### Scenario: Summary waits for pending screenshot OCR
- **GIVEN** screenshot OCR is still running for a meeting
- **WHEN** transcript processing is ready to run the automatic summary
- **THEN** Meeting Assistant waits for the screenshot OCR to complete before moving the assistant context to `summarizing`
- **AND** proceeds after the configured OCR timeout if OCR does not finish
#### Scenario: Summary can include relevant cropped screenshots
- **GIVEN** assistant context contains cropped screenshot markdown links
- **WHEN** Meeting Assistant builds summary-agent instructions
- **THEN** the instructions tell the summary agent to include only the most relevant cropped screenshots in the summary
- **AND** to markdown-link them near the related summary text
#### Scenario: External project write is audited
- **WHEN** the summary agent writes a bound project file
- **THEN** Meeting Assistant records the changed removed and added lines for that project file
@@ -85,6 +114,58 @@ The summary agent SHALL be able to read and write the assistant context body as
- **WHEN** the summary agent writes the summary or assistant context file
- **THEN** Meeting Assistant does not include those writes in the external write diff audit
#### Scenario: Assistant context body writes do not duplicate frontmatter
- **WHEN** the summary agent writes assistant context content that accidentally includes a markdown frontmatter block
- **THEN** Meeting Assistant preserves the existing assistant context frontmatter
- **AND** writes only the replacement body content after that frontmatter
### Requirement: Meeting screenshots are captured into assistant context
Meeting Assistant SHALL expose a configurable screenshot hotkey.
When a meeting is active and the screenshot hotkey is pressed, Meeting Assistant SHALL capture the currently active window.
The screenshot image SHALL be saved into a configurable attachments folder for the assistant context note. By default, the folder SHALL be `Attachments` beside the assistant context note.
After the image is saved, Meeting Assistant SHALL append a markdown image link to the assistant context note with a meeting-relative timestamp that correlates to transcript timestamps.
Meeting Assistant SHALL allow optional screenshot OCR configuration with endpoint URL, API key or key environment variable, model, prompt, and timeout.
When screenshot OCR is configured, Meeting Assistant SHALL send the screenshot and prompt to the configured OpenAI-compatible Responses endpoint and append the model result after the screenshot link in the assistant context note.
The screenshot OCR prompt SHALL ask the model to return pixel crop coordinates when it can confidently isolate only the presentation, shared screen, or similarly relevant meeting content.
When OCR returns valid crop coordinates within the original image bounds, Meeting Assistant SHALL save a cropped PNG beside the original screenshot and SHALL link the cropped image before the OCR result in the assistant context note.
When OCR returns no crop coordinates or invalid crop coordinates, Meeting Assistant SHALL keep the original screenshot link and OCR result without writing a cropped image.
When screenshot OCR is not configured, Meeting Assistant SHALL skip OCR and keep the screenshot link.
The default OCR prompt SHALL explain that the image is from a meeting and ask the model to identify who is talking, who is presenting, what is presented, capture slide text in markdown, convert diagrams to Mermaid when possible, return crop coordinates only for confidently isolated presentation/shared-screen content, and otherwise describe the scene.
#### Scenario: Screenshot is linked with meeting timestamp
- **GIVEN** a meeting started at `10:00:00`
- **WHEN** the user captures a screenshot at `10:03:05`
- **THEN** Meeting Assistant saves the screenshot under the configured attachments folder
- **AND** appends a markdown image link to assistant context with timestamp `[00:03:05]`
#### Scenario: OCR result is appended after screenshot
- **GIVEN** screenshot OCR is configured
- **WHEN** the user captures a screenshot
- **THEN** Meeting Assistant appends the screenshot link to assistant context
- **AND** appends the OCR result for that screenshot after the link when processing completes
#### Scenario: OCR crop is saved and linked before OCR text
- **GIVEN** screenshot OCR is configured
- **AND** OCR returns valid crop coordinates for a shared screen
- **WHEN** OCR processing completes
- **THEN** Meeting Assistant saves a cropped screenshot beside the original image
- **AND** links the cropped screenshot before the OCR text in assistant context
#### Scenario: OCR is skipped when not configured
- **GIVEN** screenshot OCR is not configured
- **WHEN** the user captures a screenshot
- **THEN** Meeting Assistant saves and links the screenshot without calling a model endpoint
### Requirement: Summary agent instructions are configurable
Meeting Assistant SHALL allow the summary agent initial prompt to be configured through application settings.