Manuel/meeting-assistant
Public Access
1
0
Fork 0
Code Issues Pull Requests 4 Actions Packages Projects Releases Wiki Activity Open in Coder

Archive meeting workflow and screenshot changes

Browse Source
This commit is contained in:
codex
2026-05-27 12:55:19 +02:00
parent 2422236ef7
commit 12832dde84
48 changed files with 3041 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
openspec/specs/meeting-recording/spec.md
+18 -1
View File
@@ -6,6 +6,12 @@ TBD - created by archiving change define-meeting-assistant-v1. Update Purpose af
### Requirement: Recording mode is controlled by a configurable hotkey
Meeting Assistant SHALL use normal .NET configuration to define the global hotkey that toggles recording/transcription mode.
Meeting Assistant SHALL expose a configurable abort/discard hotkey for an active recording.
When aborting an active recording, Meeting Assistant SHALL stop active audio capture and transcription, SHALL remove the meeting note, transcript, assistant context, summary artifact, and linked screenshot attachment files created for that run, and SHALL NOT invoke automatic summary generation.
When no recording is active, aborting SHALL leave the current recording status unchanged.
#### Scenario: Hotkey starts recording
- **WHEN** the configured hotkey is pressed while recording mode is inactive
- **THEN** Meeting Assistant starts recording/transcription mode
@@ -14,6 +20,18 @@ Meeting Assistant SHALL use normal .NET configuration to define the global hotke
- **WHEN** the configured hotkey is pressed while recording mode is active
- **THEN** Meeting Assistant stops recording/transcription mode
#### Scenario: Active recording is aborted and discarded
- **GIVEN** a meeting recording is active and has created meeting artifacts
- **WHEN** the user aborts the recording
- **THEN** Meeting Assistant stops recording
- **AND** deletes the meeting note, transcript, assistant context, summary artifact if present, and linked screenshot attachment files for that run
- **AND** does not invoke the summary pipeline
#### Scenario: Abort does nothing when idle
- **GIVEN** no meeting recording is active
- **WHEN** the user aborts recording
- **THEN** Meeting Assistant remains idle
### Requirement: Recording mode captures microphone and computer output
Meeting Assistant SHALL capture microphone input and computer output and combine them into one audio stream for transcription.
@@ -89,4 +107,3 @@ Meeting Assistant SHALL use the selected launch profile's summary-agent settings
- **GIVEN** two launch profiles configure the same toggle hotkey
- **WHEN** Meeting Assistant resolves launch profiles
- **THEN** Meeting Assistant rejects the configuration before registering global hotkeys
openspec/specs/meeting-session/spec.md
+100 -2
View File
@@ -68,14 +68,24 @@ Meeting software integrations MAY augment the meeting experience, but they SHALL
### Requirement: Windows Outlook enrichment is optional
Meeting Assistant SHALL gate Outlook Classic COM enrichment behind the Windows compilation target.
When the Windows build starts a meeting and Outlook Classic has exactly one current Teams appointment, Meeting Assistant SHALL copy the appointment title and attendees to the meeting note and copy the appointment agenda and scheduled end time to the assistant-context frontmatter.
When the Windows build starts a meeting and Outlook Classic has exactly one current Teams appointment, Meeting Assistant SHALL copy the appointment title to the meeting note and copy the appointment agenda and scheduled end time to the assistant-context frontmatter.
Meeting Assistant SHALL copy the appointment attendees to the meeting note only when the raw appointment attendee count is less than or equal to the configured `Recording:MaxMetadataAttendeeImportCount`. The default maximum SHALL be 30 attendees.
The agenda SHALL be extracted from the appointment body content before the Teams join separator or Teams join text.
#### Scenario: Current Teams appointment enriches meeting artifacts
- **WHEN** a Windows build starts a meeting while Outlook Classic exposes exactly one current Teams appointment
- **THEN** Meeting Assistant uses the appointment subject as the meeting title
- **AND** writes the appointment attendees into meeting note frontmatter
- **AND** writes the appointment attendees into meeting note frontmatter when the raw attendee count is within the configured import limit
- **AND** writes the appointment agenda into assistant-context frontmatter
- **AND** writes the appointment end time as `scheduled_end` into assistant-context frontmatter
#### Scenario: Oversized attendee list is not imported
- **GIVEN** the configured metadata attendee import limit is 30
- **WHEN** a Windows build starts a meeting while Outlook Classic exposes exactly one current Teams appointment with 31 attendees
- **THEN** Meeting Assistant uses the appointment subject as the meeting title
- **AND** does not write the appointment attendees into meeting note frontmatter
- **AND** writes the appointment agenda into assistant-context frontmatter
- **AND** writes the appointment end time as `scheduled_end` into assistant-context frontmatter
@@ -84,3 +94,91 @@ The agenda SHALL be extracted from the appointment body content before the Teams
- **THEN** Meeting Assistant starts the recording with the default generated meeting title and empty agenda
- **AND** omits `scheduled_end` from assistant-context frontmatter
### Requirement: Meeting automation rules are configurable
Meeting Assistant SHALL allow an optional local YAML rules file path to be configured.
When the configured rules file is missing or blank, Meeting Assistant SHALL continue without applying automation rules.
The local rules file SHALL be ignored by source control.
Rules SHALL be evaluated against the latest meeting note data read from disk for each event.
Meeting Assistant SHALL expose a diagnostic endpoint that reloads application configuration so workflow automation configuration changes can be picked up without restarting the application.
#### Scenario: Missing rules file is ignored
- **WHEN** Meeting Assistant handles a meeting event and no configured rules file exists
- **THEN** it leaves the meeting note and assistant context unchanged
#### Scenario: Created rule adds default attendee
- **GIVEN** a configured rule that triggers on meeting creation when `meeting.attendees.count = 0`
- **WHEN** Meeting Assistant creates a meeting note without attendees
- **THEN** it adds the configured attendee to the meeting note
#### Scenario: Workflow configuration is reloaded diagnostically
- **GIVEN** the workflow automation configuration has changed on disk
- **WHEN** the diagnostic workflow reload endpoint is called
- **THEN** Meeting Assistant reloads application configuration without restarting
- **AND** future workflow events use the reloaded workflow automation configuration
### Requirement: Meeting automation rules support lifecycle triggers
Meeting Assistant SHALL support rule triggers for `created`, `state_transition`, and `speaker_identified`.
A `state_transition` trigger MAY filter by `from`, `to`, or both state values.
A `speaker_identified` trigger MAY filter by speaker name.
#### Scenario: State transition rule matches from and to
- **GIVEN** a configured rule that triggers on a state transition from `collecting metadata` to `transcribing`
- **WHEN** Meeting Assistant transitions that meeting from `collecting metadata` to `transcribing`
- **THEN** it applies the rule steps
#### Scenario: Speaker identified rule filters by name
- **GIVEN** a configured rule that triggers when speaker `Ada` is identified
- **WHEN** Meeting Assistant identifies speaker `Grace`
- **THEN** it does not apply the rule
- **WHEN** Meeting Assistant identifies speaker `Ada`
- **THEN** it applies the rule steps
### Requirement: Meeting automation rules support conditions and steps
Meeting Assistant SHALL support rule conditions using an expression engine.
Rules SHALL support nested `and`, `or`, and `not` condition groups.
Step values SHALL support Razor syntax against the current meeting event model.
Step values SHALL treat `@` characters inside valid email address tokens as literal text rather than Razor transitions.
Meeting Assistant SHALL support these initial rule steps:
- `add_attendee`
- `remove_attendee`
- `set_property`
- `add_context`
- `add_project`
#### Scenario: Nested conditions choose a matching rule
- **GIVEN** a configured rule with nested `and`, `or`, and `not` conditions over meeting title, attendees, and event data
- **WHEN** the condition evaluates to true
- **THEN** Meeting Assistant applies the rule
- **WHEN** the condition evaluates to false
- **THEN** Meeting Assistant skips the rule
#### Scenario: Templated context mentions identified speaker
- **GIVEN** a configured `speaker_identified` rule with an `add_context` step using Razor syntax
- **WHEN** Meeting Assistant identifies matching speaker `Ada`
- **THEN** it appends rendered context text containing `Ada` to the assistant context note
#### Scenario: Email addresses do not trigger Razor templating
- **GIVEN** a configured rule step value containing `Support@contoso.com`
- **WHEN** the rule runs
- **THEN** Meeting Assistant preserves the email address as literal text
#### Scenario: Email addresses can appear beside Razor templating
- **GIVEN** a configured rule step value containing both `Support@contoso.com` and a Razor expression
- **WHEN** the rule runs
- **THEN** Meeting Assistant renders the Razor expression and preserves the email address as literal text
#### Scenario: Rule can clean a meeting title
- **GIVEN** a configured state-transition rule that matches a title containing a configured marker
- **WHEN** the rule runs
- **THEN** Meeting Assistant can update the meeting title through `set_property`
openspec/specs/meeting-summary/spec.md
+84 -3
View File
@@ -2,6 +2,7 @@
## Purpose
TBD - created by archiving change define-meeting-assistant-v1. Update Purpose after archive.
## Requirements
### Requirement: Meeting Assistant generates meeting outputs
Meeting Assistant SHALL generate a summary, decisions, and next steps from the meeting transcript, metadata, user notes, and assistant-discovered context.
@@ -18,7 +19,9 @@ Fallback compaction SHALL become increasingly aggressive: collapse old tool resu
When summary generation fails after retries, Meeting Assistant SHALL write a markdown failure report to the configured summary note path. The failure report SHALL include a clickable retry link, error details, and the meeting artifact paths needed to diagnose or retry the run.
After transcript processing finishes for a recording, Meeting Assistant SHALL automatically invoke the summary pipeline for that meeting.
After transcript processing finishes for a recording, Meeting Assistant SHALL automatically invoke the summary pipeline for the meeting.
Before automatic summary generation starts, Meeting Assistant SHALL wait for pending screenshot OCR jobs for that meeting to complete or reach their configured timeout.
Meeting Assistant SHALL expose an API operation to retry summary generation for a given summary note path. The retry operation SHALL resolve the linked meeting artifacts from the meeting note frontmatter and SHALL overwrite the existing summary note with either the new summary or a new failure report.
@@ -29,6 +32,7 @@ The summary pipeline SHALL expose tools scoped to the current meeting:
- `read_context`
- `read_usernotes`
- `read_glossary`
- `add_dictation_word`
- `write_summary`
- `write_context`
- `list_projects`
@@ -41,6 +45,8 @@ All summary-agent read tools that return file-like content SHALL support either
The summary pipeline SHALL write the summary note as a markdown artifact linked to the meeting note, transcript, and assistant context.
The summary instructions SHALL tell the summary agent to include only the most relevant cropped screenshot links from assistant context in the summary, placing them near the related summary text.
The summary agent SHALL be able to read the meeting note, transcript, assistant context, glossary, and bound project files through tools.
The summary agent SHALL be able to write the summary and assistant context files as its owned artifacts.
@@ -51,11 +57,13 @@ Meeting Assistant SHALL use a diff implementation that can reduce whole-file rew
When the summary agent finishes, Meeting Assistant SHALL append the collected external write diffs to the assistant context file.
After writing the meeting summary, the summary agent SHALL update existing project files when the meeting produced project-relevant knowledge.
After writing the meeting summary, the summary agent SHALL update existing project files when the meeting produced durable project knowledge, decisions, next steps, or context.
The assistant context note SHALL keep `title`, `start_time`, `end_time`, `meeting`, `transcript`, `summary`, `state`, and `agenda` in frontmatter. Meeting Assistant SHALL write `title` and `start_time` when the assistant context note is created, update `title` when later meeting metadata is discovered, and write `end_time` when transcript processing stops. When a Teams appointment is detected at recording start, the assistant context note SHALL also keep `scheduled_end` in frontmatter. The `state` value SHALL be one of `collecting metadata`, `transcribing`, `speaker recognition`, `summarizing`, `finished`, or `error`.
The summary note SHALL keep frontmatter links to meeting artifacts and SHALL copy `title`, `start_time`, and `end_time` from the meeting note when available. If the meeting note has no title, the summary agent SHALL be able to provide a title when writing the summary.
The summary note SHALL keep frontmatter links to meeting artifacts and SHALL copy `title`, `start_time`, `end_time`, and `attendees` from the meeting note when available. If the meeting note has no title, the summary agent SHALL be able to provide a title when writing the summary.
When the summary note already has an `attendees` frontmatter property, Meeting Assistant SHALL preserve the existing summary attendees instead of overwriting them from the meeting note.
The summary agent SHALL be able to read and write the assistant context body as its own notebook using overwrite, replace, and insert line modes.
@@ -71,6 +79,27 @@ The summary agent SHALL be able to read and write the assistant context body as
- **THEN** Meeting Assistant updates assistant context frontmatter state to `collecting metadata`, `transcribing`, `speaker recognition`, `summarizing`, `finished`, or `error` as appropriate
- **AND** preserves existing `agenda` and `scheduled_end` frontmatter values
#### Scenario: Summary copies final meeting attendees
- **WHEN** the summary agent writes a summary note after meeting attendees have changed
- **THEN** the summary note frontmatter includes the final meeting note attendees
#### Scenario: Existing summary attendees are preserved
- **GIVEN** an existing summary note has `attendees` in frontmatter
- **WHEN** the summary agent rewrites the summary
- **THEN** Meeting Assistant preserves the existing summary attendees
#### Scenario: Summary waits for pending screenshot OCR
- **GIVEN** screenshot OCR is still running for a meeting
- **WHEN** transcript processing is ready to run the automatic summary
- **THEN** Meeting Assistant waits for the screenshot OCR to complete before moving the assistant context to `summarizing`
- **AND** proceeds after the configured OCR timeout if OCR does not finish
#### Scenario: Summary can include relevant cropped screenshots
- **GIVEN** assistant context contains cropped screenshot markdown links
- **WHEN** Meeting Assistant builds summary-agent instructions
- **THEN** the instructions tell the summary agent to include only the most relevant cropped screenshots in the summary
- **AND** to markdown-link them near the related summary text
#### Scenario: External project write is audited
- **WHEN** the summary agent writes a bound project file
- **THEN** Meeting Assistant records the changed removed and added lines for that project file
@@ -85,6 +114,58 @@ The summary agent SHALL be able to read and write the assistant context body as
- **WHEN** the summary agent writes the summary or assistant context file
- **THEN** Meeting Assistant does not include those writes in the external write diff audit
#### Scenario: Assistant context body writes do not duplicate frontmatter
- **WHEN** the summary agent writes assistant context content that accidentally includes a markdown frontmatter block
- **THEN** Meeting Assistant preserves the existing assistant context frontmatter
- **AND** writes only the replacement body content after that frontmatter
### Requirement: Meeting screenshots are captured into assistant context
Meeting Assistant SHALL expose a configurable screenshot hotkey.
When a meeting is active and the screenshot hotkey is pressed, Meeting Assistant SHALL capture the currently active window.
The screenshot image SHALL be saved into a configurable attachments folder for the assistant context note. By default, the folder SHALL be `Attachments` beside the assistant context note.
After the image is saved, Meeting Assistant SHALL append a markdown image link to the assistant context note with a meeting-relative timestamp that correlates to transcript timestamps.
Meeting Assistant SHALL allow optional screenshot OCR configuration with endpoint URL, API key or key environment variable, model, prompt, and timeout.
When screenshot OCR is configured, Meeting Assistant SHALL send the screenshot and prompt to the configured OpenAI-compatible Responses endpoint and append the model result after the screenshot link in the assistant context note.
The screenshot OCR prompt SHALL ask the model to return pixel crop coordinates when it can confidently isolate only the presentation, shared screen, or similarly relevant meeting content.
When OCR returns valid crop coordinates within the original image bounds, Meeting Assistant SHALL save a cropped PNG beside the original screenshot and SHALL link the cropped image before the OCR result in the assistant context note.
When OCR returns no crop coordinates or invalid crop coordinates, Meeting Assistant SHALL keep the original screenshot link and OCR result without writing a cropped image.
When screenshot OCR is not configured, Meeting Assistant SHALL skip OCR and keep the screenshot link.
The default OCR prompt SHALL explain that the image is from a meeting and ask the model to identify who is talking, who is presenting, what is presented, capture slide text in markdown, convert diagrams to Mermaid when possible, return crop coordinates only for confidently isolated presentation/shared-screen content, and otherwise describe the scene.
#### Scenario: Screenshot is linked with meeting timestamp
- **GIVEN** a meeting started at `10:00:00`
- **WHEN** the user captures a screenshot at `10:03:05`
- **THEN** Meeting Assistant saves the screenshot under the configured attachments folder
- **AND** appends a markdown image link to assistant context with timestamp `[00:03:05]`
#### Scenario: OCR result is appended after screenshot
- **GIVEN** screenshot OCR is configured
- **WHEN** the user captures a screenshot
- **THEN** Meeting Assistant appends the screenshot link to assistant context
- **AND** appends the OCR result for that screenshot after the link when processing completes
#### Scenario: OCR crop is saved and linked before OCR text
- **GIVEN** screenshot OCR is configured
- **AND** OCR returns valid crop coordinates for a shared screen
- **WHEN** OCR processing completes
- **THEN** Meeting Assistant saves a cropped screenshot beside the original image
- **AND** links the cropped screenshot before the OCR text in assistant context
#### Scenario: OCR is skipped when not configured
- **GIVEN** screenshot OCR is not configured
- **WHEN** the user captures a screenshot
- **THEN** Meeting Assistant saves and links the screenshot without calling a model endpoint
### Requirement: Summary agent instructions are configurable
Meeting Assistant SHALL allow the summary agent initial prompt to be configured through application settings.
openspec/specs/meeting-transcription/spec.md
+9
View File
@@ -283,6 +283,8 @@ When a match is confirmed and the identity has a canonical name, Meeting Assista
When a match is confirmed and the matched speaker is not already listed in meeting note attendees by display name or alias, Meeting Assistant SHALL add the speaker display name to the attendee list.
When a match is confirmed and the meeting note attendees contain both the speaker display name and one or more accepted aliases for that same speaker, Meeting Assistant SHALL remove the alias attendee entries and keep the display name entry.
When Meeting Assistant writes attendees from calendar metadata, it SHALL match attendee display names exactly against known identity canonical names and aliases, replace matches with the identity display name, and deduplicate attendees that map to the same identity.
#### Scenario: Finished transcript is relabeled after a confirmed match
@@ -295,6 +297,13 @@ When Meeting Assistant writes attendees from calendar metadata, it SHALL match a
- **WHEN** a finished transcript has diarized speaker `Guest03` and the matching backend confirms it is `Chris`
- **THEN** Meeting Assistant stores the meeting note and transcript file addresses as a reference for `Chris`
#### Scenario: Confirmed match removes duplicate aliases
- **GIVEN** the speaker identity database contains canonical speaker `Christopher` with alias `Chris`
- **AND** the meeting note attendees contain both `Christopher` and `Chris <chris@example.com>`
- **WHEN** live or final speaker matching confirms a diarized speaker is `Christopher`
- **THEN** Meeting Assistant keeps `Christopher` in the meeting note attendees
- **AND** removes `Chris <chris@example.com>` from the meeting note attendees
### Requirement: Speaker matching runs during active transcription
Meeting Assistant SHALL start speaker identity matching only after the configured initial transcription duration has elapsed.