meeting-assistant/openspec/specs/meeting-summary/spec.md

meeting-summary Specification

Purpose

TBD - created by archiving change define-meeting-assistant-v1. Update Purpose after archive.

Requirements

Requirement: Meeting Assistant generates meeting outputs

The settings/logs agent SHALL instruct meeting-note correction workflows to keep project knowledge consistent when the corrected meeting note has project references.

When a corrected meeting note adds a new project reference, the settings/logs agent SHALL inspect the corrected note broadly enough to find project-relevant information that should be written to that project.

When a corrected meeting note changes specific content for an existing project reference, the settings/logs agent SHALL check whether corresponding project knowledge needs the same correction.

For each affected project, when an AGENTS.md file exists directly in the project folder, the settings/logs agent SHALL follow those project instructions before updating the project. When no project AGENTS.md exists, the settings/logs agent SHALL use judgment to decide what project file updates are warranted by the note correction.

Scenario: Settings/logs agent syncs corrected meeting notes to projects

• GIVEN the settings/logs agent corrects a meeting note with project references
• WHEN a referenced project has an AGENTS.md
• THEN the settings/logs instructions tell the agent to follow that project's instructions before updating the project
• WHEN a corrected note adds a new project reference
• THEN the instructions tell the agent to review the corrected note for project-relevant information to add to that project
• WHEN a corrected note changes specific information for an existing project reference
• THEN the instructions tell the agent to check whether project knowledge needs a matching correction

Requirement: Meeting screenshots are captured into assistant context

Meeting Assistant SHALL expose a configurable screenshot hotkey.

When a meeting is active and the screenshot hotkey is pressed, Meeting Assistant SHALL capture the currently active window.

The screenshot image SHALL be saved into a configurable attachments folder for the assistant context note. By default, the folder SHALL be Attachments beside the assistant context note.

After the image is saved, Meeting Assistant SHALL append a markdown image link to the assistant context note with a meeting-relative timestamp that correlates to transcript timestamps.

Meeting Assistant SHALL allow optional screenshot OCR configuration with endpoint URL, API key or key environment variable, model, prompt, and timeout.

When screenshot OCR is configured, Meeting Assistant SHALL send the screenshot and prompt to the configured OpenAI-compatible Responses endpoint and append the model result after the screenshot link in the assistant context note.

The screenshot OCR prompt SHALL ask the model to return pixel crop coordinates when it can confidently isolate only the presentation, shared screen, or similarly relevant meeting content.

When OCR returns valid crop coordinates within the original image bounds, Meeting Assistant SHALL save a cropped PNG beside the original screenshot and SHALL link the cropped image before the OCR result in the assistant context note.

When OCR returns no crop coordinates or invalid crop coordinates, Meeting Assistant SHALL keep the original screenshot link and OCR result without writing a cropped image.

After transcription finishes and before summarization starts, Meeting Assistant SHALL scan the meeting note for user-authored Obsidian image embeds and Markdown image embeds.

When configured screenshot OCR is enabled and the meeting note contains image embeds, Meeting Assistant SHALL append each resolvable image to the assistant context note, state that the image came from the meeting note, preserve the original embed text for cross-reference, and run OCR for the linked image.

Meeting-note image OCR SHALL NOT copy the image file, SHALL NOT write crop images, SHALL NOT add attendees from OCR metadata, and SHALL NOT modify the meeting note.

Meeting Assistant SHALL wait for meeting-note image OCR to finish or time out before transitioning the assistant context to summarizing.

When screenshot OCR is not configured, Meeting Assistant SHALL skip OCR and keep the screenshot link.

The default OCR prompt SHALL explain that the image is from a meeting and ask the model to identify who is talking, who is presenting, what is presented, capture slide text in markdown, convert diagrams to Mermaid when possible, indicate whether visible people are clearly the exact meeting participants or only a partial result, return crop coordinates only for confidently isolated presentation/shared-screen content, and otherwise describe the scene.

Scenario: Screenshot is linked with meeting timestamp

• GIVEN a meeting started at 10:00:00
• WHEN the user captures a screenshot at 10:03:05
• THEN Meeting Assistant saves the screenshot under the configured attachments folder
• AND appends a markdown image link to assistant context with timestamp [00:03:05]

Scenario: OCR result is appended after screenshot

• GIVEN screenshot OCR is configured
• WHEN the user captures a screenshot
• THEN Meeting Assistant appends the screenshot link to assistant context
• AND appends the OCR result for that screenshot after the link when processing completes

Scenario: OCR crop is saved and linked before OCR text

• GIVEN screenshot OCR is configured
• AND OCR returns valid crop coordinates for a shared screen
• WHEN OCR processing completes
• THEN Meeting Assistant saves a cropped screenshot beside the original image
• AND links the cropped screenshot before the OCR text in assistant context

Scenario: Meeting note image embeds are OCRed before summarization

• GIVEN screenshot OCR is configured
• AND the meeting note contains ![[whiteboard.png]]
• AND the meeting note contains ![Diagram](attachments/diagram.png)
• WHEN transcription finishes
• THEN Meeting Assistant appends both images to the assistant context as images from the meeting note
• AND includes the original embed text for each image
• AND runs OCR for each image without copying files, writing crop images, adding attendees, or modifying the meeting note
• AND waits for this OCR to finish or time out before transitioning to summarizing

Scenario: OCR is skipped when not configured

• GIVEN screenshot OCR is not configured
• WHEN the user captures a screenshot
• THEN Meeting Assistant saves and links the screenshot without calling a model endpoint

Scenario: OCR reports whether visible people are complete or partial

• WHEN Meeting Assistant uses the built-in screenshot OCR prompt
• THEN the prompt asks the model to state whether the screenshot clearly shows exactly who is in the meeting or only a partial participant result

Requirement: Summary agent instructions are configurable

Meeting Assistant SHALL allow the summary agent initial prompt to be configured through application settings.

When no configured initial prompt is supplied, Meeting Assistant SHALL use the built-in default prompt.

Scenario: Configured initial prompt is used

• WHEN MeetingAssistant:Agent:InitialPrompt is configured with non-empty content
• THEN the summary agent uses that configured prompt as its base instructions

Scenario: Empty initial prompt falls back to built-in default

• WHEN MeetingAssistant:Agent:InitialPrompt is missing or blank
• THEN the summary agent uses the built-in default instructions

Requirement: Summary agent receives bound project instructions

Meeting Assistant SHALL append project instructions to the summary agent instructions for projects bound to the meeting note frontmatter.

For each bound project folder under the configured projects folder, when an AGENTS.md file exists directly in that project folder, Meeting Assistant SHALL append the project name and the AGENTS.md content under a project instructions section.

When no bound projects have AGENTS.md, Meeting Assistant SHALL not append the project instructions section.

Scenario: Project AGENTS files are appended

• GIVEN the meeting note frontmatter lists projects Alpha and Beta
• AND both configured project folders contain AGENTS.md
• WHEN the summary agent is created
• THEN its instructions include ---, projects:, # Alpha, Alpha's AGENTS.md, # Beta, and Beta's AGENTS.md

Scenario: Projects without AGENTS files are skipped

• GIVEN the meeting note frontmatter lists project Alpha
• AND the configured project folder does not contain AGENTS.md
• WHEN the summary agent is created
• THEN no empty project instruction entry is appended for Alpha

Requirement: Summary note includes a one-line blurb

The write_summary tool SHALL require a one-line summary parameter in addition to the markdown summary body.

Meeting Assistant SHALL reject write_summary calls when the one-line summary contains line breaks.

Meeting Assistant SHALL write the one-line summary into summary note frontmatter as oneliner.

The summary-agent instructions SHALL tell the agent to keep the one-line summary very short, blurb-like, and concise, with complete sentences not required.

Scenario: Summary one-liner is written to frontmatter

• WHEN the summary agent calls write_summary with markdown content and one-line summary Architecture risks and next steps
• THEN Meeting Assistant writes oneliner: Architecture risks and next steps to the summary note frontmatter

Scenario: Multiline one-liner is rejected

• WHEN the summary agent calls write_summary with a one-line summary containing a line break
• THEN Meeting Assistant refuses the write
• AND does not mark the summary as written

Requirement: Summary agent preserves meeting-specific working memory

When Meeting Assistant uses the built-in summary-agent instructions, those instructions SHALL identify assistant context as persistent meeting-specific memory.

The built-in instructions SHALL direct the summary agent to append unexpected problems, missing information, and assumptions encountered while summarizing to assistant context.

Scenario: Summarizer encounters uncertainty

• GIVEN the built-in summary-agent instructions are in use
• WHEN the summary agent encounters an unexpected problem, cannot find needed information, or must make an assumption
• THEN its instructions direct it to append a concise record to the meeting's assistant context
• AND later agents can discover that record when working on the same meeting

Requirement: Summary agents tolerate Responses event streams and malformed tool arguments

Meeting Assistant SHALL consume successful summary-agent Responses results through the supported OpenAI Responses and Agent Framework Server-Sent Events adapter.

When a Responses event stream delivers completed output items separately from the final response metadata, Meeting Assistant SHALL assemble those output items into one agent response while preserving final response metadata and usage.

Meeting Assistant SHALL provide an agent setting that selects streaming or non-streaming Responses transport. Streaming SHALL be enabled by default. When streaming is disabled, Meeting Assistant SHALL use the supported non-streaming OpenAI Responses client and adapter.

When a returned function call contains arguments that are not a valid JSON object, Meeting Assistant SHALL NOT invoke the requested function and SHALL return an invalid-tool-arguments result associated with the original call ID to the agent so it can correct the call.

When Meeting Assistant translates chat messages into a Responses request, every message input item SHALL retain the message item discriminator required by the OpenAI SDK and Responses API.

Scenario: Streamed text response is assembled

• WHEN the configured Responses endpoint returns completed message output in Server-Sent Events followed by final response metadata
• THEN the summary agent receives the completed message text, response metadata, and usage without a JSON document parse failure

Scenario: Streamed function call is assembled

• WHEN the configured Responses endpoint returns a completed function-call output item in Server-Sent Events
• THEN the summary agent receives the function call with its call ID, function name, and parsed arguments

Scenario: Streaming transport can be disabled

• WHEN MeetingAssistant:Agent:UseStreaming is false
• THEN the summary agent requests a non-streaming Responses result
• AND converts the response through the supported OpenAI Responses adapter

Scenario: Chat message input remains a Responses message

• WHEN the summary agent sends a user or assistant chat message through the Responses client
• THEN the outbound Responses input item has type message
• AND the request does not contain an unknown input item type

Scenario: Malformed function arguments are returned to the agent

• WHEN the model returns a function call whose arguments are not a valid JSON object
• THEN Meeting Assistant does not invoke the requested function
• AND sends a function result with an invalid-tool-arguments error for the original call ID back to the agent
• AND allows the agent loop to continue