1 Commits
Author SHA1 Message Date
renovate-bot 6a5689ab95 Update dependency Microsoft.Agents.AI.OpenAI to 1.8.0
PR and Push Build/Test / build-and-test (push) Successful in 10m56s
PR and Push Build/Test / build-and-test (pull_request) Successful in 11m17s
2026-05-30 02:27:33 +00:00
195 changed files with 2879 additions and 18451 deletions
-107
View File
@@ -1,107 +0,0 @@
---
name: tdd
description: Test-driven development with red-green-refactor loop. Use when user wants to build features or fix bugs using TDD, mentions "red-green-refactor", wants integration tests, or asks for test-first development.
---
# Test-Driven Development
## Philosophy
**Core principle**: Tests should verify behavior through public interfaces, not implementation details. Code can change entirely; tests shouldn't.
**Good tests** are integration-style: they exercise real code paths through public APIs. They describe _what_ the system does, not _how_ it does it. A good test reads like a specification - "user can checkout with valid cart" tells you exactly what capability exists. These tests survive refactors because they don't care about internal structure.
**Bad tests** are coupled to implementation. They mock internal collaborators, test private methods, or verify through external means (like querying a database directly instead of using the interface). The warning sign: your test breaks when you refactor, but behavior hasn't changed. If you rename an internal function and tests fail, those tests were testing implementation, not behavior.
See [tests.md](references/tests.md) for examples and [mocking.md](references/mocking.md) for mocking guidelines.
## Anti-Pattern: Horizontal Slices
**DO NOT write all tests first, then all implementation.** This is "horizontal slicing" - treating RED as "write all tests" and GREEN as "write all code."
This produces **crap tests**:
- Tests written in bulk test _imagined_ behavior, not _actual_ behavior
- You end up testing the _shape_ of things (data structures, function signatures) rather than user-facing behavior
- Tests become insensitive to real changes - they pass when behavior breaks, fail when behavior is fine
- You outrun your headlights, committing to test structure before understanding the implementation
**Correct approach**: Vertical slices via tracer bullets. One test → one implementation → repeat. Each test responds to what you learned from the previous cycle. Because you just wrote the code, you know exactly what behavior matters and how to verify it.
```
WRONG (horizontal):
RED: test1, test2, test3, test4, test5
GREEN: impl1, impl2, impl3, impl4, impl5
RIGHT (vertical):
RED→GREEN: test1→impl1
RED→GREEN: test2→impl2
RED→GREEN: test3→impl3
...
```
## Workflow
### 1. Planning
Before writing any code:
- [ ] Confirm with user what interface changes are needed
- [ ] Confirm with user which behaviors to test (prioritize)
- [ ] Identify opportunities for [deep modules](references/deep-modules.md) (small interface, deep implementation)
- [ ] Design interfaces for [testability](references/interface-design.md)
- [ ] List the behaviors to test (not implementation steps)
- [ ] Get user approval on the plan
Ask: "What should the public interface look like? Which behaviors are most important to test?"
**You can't test everything.** Confirm with the user exactly which behaviors matter most. Focus testing effort on critical paths and complex logic, not every possible edge case.
### 2. Tracer Bullet
Write ONE test that confirms ONE thing about the system:
```
RED: Write test for first behavior → test fails
GREEN: Write minimal code to pass → test passes
```
This is your tracer bullet - proves the path works end-to-end.
### 3. Incremental Loop
For each remaining behavior:
```
RED: Write next test → fails
GREEN: Minimal code to pass → passes
```
Rules:
- One test at a time
- Only enough code to pass current test
- Don't anticipate future tests
- Keep tests focused on observable behavior
### 4. Refactor
After all tests pass, look for [refactor candidates](references/refactoring.md):
- [ ] Extract duplication
- [ ] Deepen modules (move complexity behind simple interfaces)
- [ ] Apply SOLID principles where natural
- [ ] Consider what new code reveals about existing code
- [ ] Run tests after each refactor step
**Never refactor while RED.** Get to GREEN first.
## Checklist Per Cycle
```
[ ] Test describes behavior, not implementation
[ ] Test uses public interface only
[ ] Test would survive internal refactor
[ ] Code is minimal for this test
[ ] No speculative features added
```
@@ -1,33 +0,0 @@
# Deep Modules
From "A Philosophy of Software Design":
**Deep module** = small interface + lots of implementation
```
┌─────────────────────┐
│ Small Interface │ ← Few methods, simple params
├─────────────────────┤
│ │
│ │
│ Deep Implementation│ ← Complex logic hidden
│ │
│ │
└─────────────────────┘
```
**Shallow module** = large interface + little implementation (avoid)
```
┌─────────────────────────────────┐
│ Large Interface │ ← Many methods, complex params
├─────────────────────────────────┤
│ Thin Implementation │ ← Just passes through
└─────────────────────────────────┘
```
When designing interfaces, ask:
- Can I reduce the number of methods?
- Can I simplify the parameters?
- Can I hide more complexity inside?
@@ -1,31 +0,0 @@
# Interface Design for Testability
Good interfaces make testing natural:
1. **Accept dependencies, don't create them**
```typescript
// Testable
function processOrder(order, paymentGateway) {}
// Hard to test
function processOrder(order) {
const gateway = new StripeGateway();
}
```
2. **Return results, don't produce side effects**
```typescript
// Testable
function calculateDiscount(cart): Discount {}
// Hard to test
function applyDiscount(cart): void {
cart.total -= discount;
}
```
3. **Small surface area**
- Fewer methods = fewer tests needed
- Fewer params = simpler test setup
-59
View File
@@ -1,59 +0,0 @@
# When to Mock
Mock at **system boundaries** only:
- External APIs (payment, email, etc.)
- Databases (sometimes - prefer test DB)
- Time/randomness
- File system (sometimes)
Don't mock:
- Your own classes/modules
- Internal collaborators
- Anything you control
## Designing for Mockability
At system boundaries, design interfaces that are easy to mock:
**1. Use dependency injection**
Pass external dependencies in rather than creating them internally:
```typescript
// Easy to mock
function processPayment(order, paymentClient) {
return paymentClient.charge(order.total);
}
// Hard to mock
function processPayment(order) {
const client = new StripeClient(process.env.STRIPE_KEY);
return client.charge(order.total);
}
```
**2. Prefer SDK-style interfaces over generic fetchers**
Create specific functions for each external operation instead of one generic function with conditional logic:
```typescript
// GOOD: Each function is independently mockable
const api = {
getUser: (id) => fetch(`/users/${id}`),
getOrders: (userId) => fetch(`/users/${userId}/orders`),
createOrder: (data) => fetch('/orders', { method: 'POST', body: data }),
};
// BAD: Mocking requires conditional logic inside the mock
const api = {
fetch: (endpoint, options) => fetch(endpoint, options),
};
```
The SDK approach means:
- Each mock returns one specific shape
- No conditional logic in test setup
- Easier to see which endpoints a test exercises
- Type safety per endpoint
@@ -1,10 +0,0 @@
# Refactor Candidates
After TDD cycle, look for:
- **Duplication** → Extract function/class
- **Long methods** → Break into private helpers (keep tests on public interface)
- **Shallow modules** → Combine or deepen
- **Feature envy** → Move logic to where data lives
- **Primitive obsession** → Introduce value objects
- **Existing code** the new code reveals as problematic
-61
View File
@@ -1,61 +0,0 @@
# Good and Bad Tests
## Good Tests
**Integration-style**: Test through real interfaces, not mocks of internal parts.
```typescript
// GOOD: Tests observable behavior
test("user can checkout with valid cart", async () => {
const cart = createCart();
cart.add(product);
const result = await checkout(cart, paymentMethod);
expect(result.status).toBe("confirmed");
});
```
Characteristics:
- Tests behavior users/callers care about
- Uses public API only
- Survives internal refactors
- Describes WHAT, not HOW
- One logical assertion per test
## Bad Tests
**Implementation-detail tests**: Coupled to internal structure.
```typescript
// BAD: Tests implementation details
test("checkout calls paymentService.process", async () => {
const mockPayment = jest.mock(paymentService);
await checkout(cart, payment);
expect(mockPayment.process).toHaveBeenCalledWith(cart.total);
});
```
Red flags:
- Mocking internal collaborators
- Testing private methods
- Asserting on call counts/order
- Test breaks when refactoring without behavior change
- Test name describes HOW not WHAT
- Verifying through external means instead of interface
```typescript
// BAD: Bypasses interface to verify
test("createUser saves to database", async () => {
await createUser({ name: "Alice" });
const row = await db.query("SELECT * FROM users WHERE name = ?", ["Alice"]);
expect(row).toBeDefined();
});
// GOOD: Verifies through interface
test("createUser makes user retrievable", async () => {
const user = await createUser({ name: "Alice" });
const retrieved = await getUser(user.id);
expect(retrieved.name).toBe("Alice");
});
```
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v7
uses: actions/checkout@v6
- name: Setup .NET
uses: actions/setup-dotnet@v5
+7 -6
View File
@@ -57,13 +57,18 @@ A spec/change is not considered done merely because code is merged, tests pass,
Before archiving any OpenSpec change, perform a refactoring pass over the code and specs touched by that change. The pass must inspect both the current diff and the surrounding implementation context, because a small diff may reveal repeated patterns or structural problems that only become obvious when compared with nearby code.
Do the refactoring pass in these distinct areas in sequence so they are less likely to converge on the same issues, first start a subagent (`Explorer`) to identify potential improvements, then implement them, then start the repeat for the next area:
Do the refactoring pass in these distinct areas, first start a subagent for each area to identify potential improvements in parallel, then implement them:
1. Check for DRYness. Look for duplication introduced by the change and for existing nearby duplication that the change now makes worth consolidating. A change may be small on its own, but if it is the fifth copy of the same idea, it is a refactoring target.
2. Check for SOLID violations. Look for responsibilities that are mixed together, abstractions that are hard to replace or test, interface shapes that force unrelated dependencies, and code paths that require modifying stable code for each new variant.
3. Check whether the implementation can be made simpler under KISS. Remove accidental abstractions, reduce branching, clarify names, and prefer the smallest structure that still supports the tested behavior and current spec.
Treat these instructions as user instructions, and do not skip or shortcut them. If you find that you cannot follow these instructions, state exactly which part you are having trouble with and why.
Do not say subagents are unavailable unless:
1. tool discovery for "subagent", "delegate", and "agent" found no callable tool, or
2. spawning a subagent returned a concrete tool error.
In either case, report the exact discovery result or error.
Preserve behavior during this pass and run the relevant tests again afterward.
@@ -80,10 +85,6 @@ If production or end-to-end verification is not possible, state exactly why and
## Implementation Notes
Meeting Assistant is Manuel's local meeting capture and knowledge system. It runs as a background .NET service with a Windows tray icon, captures microphone plus system audio, transcribes meetings, maintains Obsidian meeting artifacts, enriches meetings from Outlook metadata when available, applies local workflow rules, identifies speakers, captures screenshots, and runs agentic summarization against a LiteLLM/OpenAI-compatible endpoint.
The app is used during real meetings. Treat active recording, transcription finalization, OCR, and summarization as live user work. Prefer local endpoints, generated note files, and application logs for verification, and avoid restart or cleanup actions that could interrupt an active run unless the user explicitly requested them.
Keep source and deployment ownership separate:
- `Manuel/meeting-assistant` owns application source, tests, build configuration, container image publishing, and OpenSpec source changes.
@@ -42,7 +42,7 @@ public sealed class AsrDiagnosticEndpointTests
configuration.AddInMemoryCollection(new Dictionary<string, string?>
{
["MeetingAssistant:FunAsr:Backend:Enabled"] = "false",
["MeetingAssistant:LaunchProfiles:english:Hotkey:Toggle"] = "Ctrl+Alt+L",
["MeetingAssistant:LaunchProfiles:english:Hotkey:Toggle"] = "Ctrl+Alt+E",
["MeetingAssistant:LaunchProfiles:english:AzureSpeech:Language"] = "en-US"
});
});
@@ -1,405 +0,0 @@
using MeetingAssistant.Calendar;
using MeetingAssistant.MeetingNotes;
using MeetingAssistant.Recording;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
namespace MeetingAssistant.Tests;
public sealed class CalendarRecordingPromptSchedulerTests
{
[Fact]
public async Task DueTeamsMeetingPromptStartsRecordingWhenAccepted()
{
var harness = CreateHarness(isRecording: false);
var meeting = CreateMeeting(harness.Clock);
harness.Provider.Meetings = [meeting];
await SyncThenPromptAsync(harness, meeting);
Assert.Equal(1, harness.Provider.QueryCount);
Assert.Collection(harness.PromptService.PromptedMeetings, prompted => Assert.Same(meeting, prompted));
Assert.Equal(1, harness.Recorder.StartCount);
Assert.Equal(0, harness.Recorder.StopCount);
}
[Fact]
public async Task AcceptedPromptStartsRecordingWithPromptedMeetingMetadata()
{
var harness = CreateHarness(isRecording: false);
var selectedMetadata = new MeetingMetadata(
"Selected planning",
["Ada"],
"Selected agenda",
DateTimeOffset.Parse("2026-06-03T10:30:00+00:00"));
var otherMetadata = new MeetingMetadata(
"Other planning",
["Grace"],
"Other agenda",
DateTimeOffset.Parse("2026-06-03T10:40:00+00:00"));
var selectedMeeting = CreateMeeting(
harness.Clock,
id: "teams-selected",
subject: "Selected planning",
metadata: selectedMetadata);
var otherMeeting = CreateMeeting(
harness.Clock,
id: "teams-other",
subject: "Other planning",
startOffset: TimeSpan.FromMinutes(40),
metadata: otherMetadata);
harness.Provider.Meetings = [selectedMeeting, otherMeeting];
await SyncThenPromptAsync(harness, selectedMeeting);
Assert.Equal(1, harness.Recorder.StartCount);
Assert.Same(selectedMetadata, harness.Recorder.StartMetadata.Single());
Assert.Equal(1, harness.Recorder.PromptStartCount);
}
[Fact]
public async Task AcceptedPromptWithoutMetadataStillUsesPromptStartPath()
{
var harness = CreateHarness(isRecording: false);
var meeting = CreateMeeting(
harness.Clock,
id: "teams-without-metadata",
subject: "No metadata",
metadata: null);
harness.Provider.Meetings = [meeting];
await SyncThenPromptAsync(harness, meeting);
Assert.Equal(1, harness.Recorder.StartCount);
Assert.Null(harness.Recorder.StartMetadata.Single());
Assert.Equal(1, harness.Recorder.PromptStartCount);
}
[Fact]
public async Task AcceptingDisplayedPromptsOutOfOrderUsesAcceptedPromptMetadata()
{
var harness = CreateHarness(isRecording: false, autoAcceptPrompts: false);
var firstMetadata = new MeetingMetadata(
"First planning",
["Ada"],
"First agenda",
DateTimeOffset.Parse("2026-06-03T10:30:00+00:00"));
var secondMetadata = new MeetingMetadata(
"Second planning",
["Grace"],
"Second agenda",
DateTimeOffset.Parse("2026-06-03T10:30:00+00:00"));
var firstMeeting = CreateMeeting(
harness.Clock,
id: "teams-first",
subject: "First planning",
metadata: firstMetadata);
var secondMeeting = CreateMeeting(
harness.Clock,
id: "teams-second",
subject: "Second planning",
metadata: secondMetadata);
harness.Provider.Meetings = [firstMeeting, secondMeeting];
await harness.Scheduler.SyncOnceAsync(CancellationToken.None);
harness.Clock.Now = firstMeeting.Start;
await harness.Scheduler.CheckDuePromptsAsync(CancellationToken.None);
await harness.PromptService.RespondAsync(secondMeeting, MeetingStartPromptResponse.Record);
Assert.Equal([firstMeeting, secondMeeting], harness.PromptService.PromptedMeetings);
Assert.Equal(1, harness.Recorder.StartCount);
Assert.Same(secondMetadata, harness.Recorder.StartMetadata.Single());
}
[Fact]
public async Task AcceptedPromptStopsActiveRecordingBeforeStartingNewRecording()
{
var harness = CreateHarness(isRecording: true);
var meeting = CreateMeeting(harness.Clock);
harness.Provider.Meetings = [meeting];
await SyncThenPromptAsync(harness, meeting);
Assert.Equal(["stop", "start"], harness.Recorder.Commands);
}
[Fact]
public async Task CanceledCachedMeetingDoesNotPromptRecording()
{
var harness = CreateHarness(isRecording: false);
var canceledMeeting = CreateMeeting(
harness.Clock,
id: "teams-canceled",
subject: "Canceled project sync",
isCanceled: true);
harness.Provider.Meetings = [canceledMeeting];
await harness.Scheduler.SyncOnceAsync(CancellationToken.None);
harness.Clock.Now = canceledMeeting.Start;
await harness.Scheduler.CheckDuePromptsAsync(CancellationToken.None);
Assert.Empty(harness.PromptService.PromptedMeetings);
Assert.Equal(0, harness.Recorder.StartCount);
}
[Fact]
public async Task DisabledCalendarPromptsDoNotQueryCalendar()
{
var harness = CreateHarness(enabled: false);
await harness.Scheduler.SyncOnceAsync(CancellationToken.None);
Assert.Equal(0, harness.Provider.QueryCount);
}
[Fact]
public async Task DueMeetingPromptsOnlyOncePerDay()
{
var harness = CreateHarness();
var meeting = CreateMeeting(harness.Clock);
harness.Provider.Meetings = [meeting];
await harness.Scheduler.SyncOnceAsync(CancellationToken.None);
harness.Clock.Now = meeting.Start;
await harness.Scheduler.CheckDuePromptsAsync(CancellationToken.None);
await harness.Scheduler.CheckDuePromptsAsync(CancellationToken.None);
Assert.Single(harness.PromptService.PromptedMeetings);
}
[Fact]
public async Task CachedMeetingPromptDoesNotQueryOutlookAgainAtMeetingStart()
{
var harness = CreateHarness();
var meeting = CreateMeeting(harness.Clock);
harness.Provider.Meetings = [meeting];
await SyncThenPromptAsync(harness, meeting);
Assert.Equal(1, harness.Provider.QueryCount);
Assert.Single(harness.PromptService.PromptedMeetings);
}
[Fact]
public async Task CachedDayMeetingsPromptBackToBackMeetingsWithoutInterveningOutlookSync()
{
var harness = CreateHarness(isRecording: false);
var firstMeeting = CreateMeeting(
harness.Clock,
id: "teams-1",
subject: "First sync",
startOffset: TimeSpan.FromMinutes(30),
duration: TimeSpan.FromMinutes(10));
var secondMeeting = CreateMeeting(
harness.Clock,
id: "teams-2",
subject: "Second sync",
startOffset: TimeSpan.FromMinutes(40),
duration: TimeSpan.FromMinutes(10));
harness.Provider.Meetings = [firstMeeting, secondMeeting];
await harness.Scheduler.SyncOnceAsync(CancellationToken.None);
harness.Clock.Now = firstMeeting.Start;
await harness.Scheduler.CheckDuePromptsAsync(CancellationToken.None);
harness.Clock.Now = secondMeeting.Start;
await harness.Scheduler.CheckDuePromptsAsync(CancellationToken.None);
Assert.Equal(1, harness.Provider.QueryCount);
Assert.Equal([firstMeeting, secondMeeting], harness.PromptService.PromptedMeetings);
Assert.Equal(["start", "stop", "start"], harness.Recorder.Commands);
}
private static async Task SyncThenPromptAsync(
SchedulerHarness harness,
CalendarMeeting meeting)
{
await harness.Scheduler.SyncOnceAsync(CancellationToken.None);
harness.Clock.Now = meeting.Start;
await harness.Scheduler.CheckDuePromptsAsync(CancellationToken.None);
}
private static CalendarMeeting CreateMeeting(
ManualCalendarPromptClock clock,
string id = "teams-1",
string subject = "Project sync",
TimeSpan? startOffset = null,
TimeSpan? duration = null,
MeetingMetadata? metadata = null,
bool isCanceled = false)
{
var start = clock.Now.Add(startOffset ?? TimeSpan.FromMinutes(30));
return new CalendarMeeting(
id,
subject,
start,
start.Add(duration ?? TimeSpan.FromMinutes(30)),
metadata,
isCanceled);
}
private static SchedulerHarness CreateHarness(
bool isRecording = false,
bool enabled = true,
bool autoAcceptPrompts = true)
{
var clock = new ManualCalendarPromptClock(new DateTimeOffset(2026, 6, 3, 9, 30, 0, TimeSpan.Zero));
var provider = new RecordingCalendarMeetingProvider([]);
var promptService = new CapturingMeetingStartPromptService(autoAcceptPrompts);
var recorder = new RecordingPromptRecorder(isRecording);
var scheduler = new CalendarRecordingPromptScheduler(
Options.Create(new MeetingAssistantOptions
{
CalendarRecordingPrompts =
{
Enabled = enabled,
SyncInterval = TimeSpan.FromMinutes(30),
PromptWindow = TimeSpan.FromMinutes(5)
}
}),
provider,
promptService,
recorder,
clock,
NullLogger<CalendarRecordingPromptScheduler>.Instance);
return new SchedulerHarness(scheduler, provider, promptService, recorder, clock);
}
private sealed record SchedulerHarness(
CalendarRecordingPromptScheduler Scheduler,
RecordingCalendarMeetingProvider Provider,
CapturingMeetingStartPromptService PromptService,
RecordingPromptRecorder Recorder,
ManualCalendarPromptClock Clock);
private sealed class RecordingCalendarMeetingProvider : ICalendarMeetingProvider
{
public RecordingCalendarMeetingProvider(IReadOnlyList<CalendarMeeting> meetings)
{
Meetings = meetings;
}
public IReadOnlyList<CalendarMeeting> Meetings { get; set; }
public int QueryCount { get; private set; }
public Task<IReadOnlyList<CalendarMeeting>> GetRecordingPromptCandidatesForDayAsync(
DateOnly day,
CancellationToken cancellationToken)
{
QueryCount++;
return Task.FromResult(Meetings);
}
}
private sealed class CapturingMeetingStartPromptService : IMeetingStartPromptService
{
private readonly bool autoAccept;
private readonly List<PendingPrompt> pendingPrompts = [];
public CapturingMeetingStartPromptService(bool autoAccept)
{
this.autoAccept = autoAccept;
}
public List<CalendarMeeting> PromptedMeetings { get; } = [];
public async Task ShowPromptAsync(
MeetingStartPromptRequest request,
Func<MeetingStartPromptResponse, CancellationToken, Task> handleResponseAsync,
CancellationToken cancellationToken)
{
PromptedMeetings.Add(request.Meeting);
pendingPrompts.Add(new PendingPrompt(request.Meeting, handleResponseAsync));
if (autoAccept)
{
await handleResponseAsync(MeetingStartPromptResponse.Record, cancellationToken);
}
}
public Task RespondAsync(
CalendarMeeting meeting,
MeetingStartPromptResponse response)
{
var prompt = pendingPrompts.Single(pending => ReferenceEquals(pending.Meeting, meeting));
return prompt.HandleResponseAsync(response, CancellationToken.None);
}
private sealed record PendingPrompt(
CalendarMeeting Meeting,
Func<MeetingStartPromptResponse, CancellationToken, Task> HandleResponseAsync);
}
private sealed class RecordingPromptRecorder : IMeetingPromptRecordingController
{
public RecordingPromptRecorder(bool isRecording)
{
CurrentStatus = Status(isRecording);
}
public RecordingStatus CurrentStatus { get; private set; }
public int StartCount { get; private set; }
public int StopCount { get; private set; }
public int PromptStartCount { get; private set; }
public List<string> Commands { get; } = [];
public List<MeetingMetadata?> StartMetadata { get; } = [];
public Task<RecordingStatus> StartAsync(CancellationToken cancellationToken)
{
return StartRecordingAsync(null);
}
private Task<RecordingStatus> StartRecordingAsync(
MeetingMetadata? metadata)
{
StartCount++;
Commands.Add("start");
StartMetadata.Add(metadata);
CurrentStatus = Status(isRecording: true);
return Task.FromResult(CurrentStatus);
}
public Task<RecordingStatus> StartFromPromptAsync(
MeetingMetadata? metadata,
CancellationToken cancellationToken)
{
PromptStartCount++;
return StartRecordingAsync(metadata);
}
public Task<RecordingStatus> StopAsync(CancellationToken cancellationToken)
{
StopCount++;
Commands.Add("stop");
CurrentStatus = Status(isRecording: false);
return Task.FromResult(CurrentStatus);
}
private static RecordingStatus Status(bool isRecording)
{
return new RecordingStatus(
isRecording,
isRecording ? "transcript.md" : null,
isRecording ? "meeting.md" : null,
isRecording ? "context.md" : null,
isRecording ? "summary.md" : null,
isRecording ? RecordingProcessState.Recording : RecordingProcessState.Idle,
isRecording ? "default" : null);
}
}
private sealed class ManualCalendarPromptClock : ICalendarPromptClock
{
public ManualCalendarPromptClock(DateTimeOffset now)
{
Now = now;
}
public DateTimeOffset Now { get; set; }
}
}
@@ -46,7 +46,7 @@ public sealed class ConfiguredSpeechRecognitionPipelineFactoryTests
{
["MeetingAssistant:Recording:TranscriptionProvider"] = "capturing",
["MeetingAssistant:Agent:Model"] = "default-model",
["MeetingAssistant:LaunchProfiles:english:Hotkey:Toggle"] = "Ctrl+Alt+L",
["MeetingAssistant:LaunchProfiles:english:Hotkey:Toggle"] = "Ctrl+Alt+E",
["MeetingAssistant:LaunchProfiles:english:Recording:TranscriptionProvider"] = "capturing",
["MeetingAssistant:LaunchProfiles:english:Agent:Model"] = "english-model"
})
@@ -33,7 +33,7 @@ public sealed class LaunchProfileOptionsProviderTests
["MeetingAssistant:AzureSpeech:Region"] = "germanywestcentral",
["MeetingAssistant:AzureSpeech:Language"] = "de-DE",
["MeetingAssistant:AzureSpeech:AutoDetectLanguages:0"] = "de-DE",
["MeetingAssistant:LaunchProfiles:english:Hotkey:Toggle"] = "Ctrl+Alt+L",
["MeetingAssistant:LaunchProfiles:english:Hotkey:Toggle"] = "Ctrl+Alt+E",
["MeetingAssistant:LaunchProfiles:english:AzureSpeech:Language"] = "en-US",
["MeetingAssistant:LaunchProfiles:english:AzureSpeech:AutoDetectLanguages:0"] = "en-US"
});
@@ -41,23 +41,13 @@ public sealed class LaunchProfileOptionsProviderTests
var profile = provider.GetRequiredProfile("english");
Assert.Equal("english", profile.Name);
Assert.Equal("Ctrl+Alt+L", profile.Options.Hotkey.Toggle);
Assert.Equal("Ctrl+Alt+E", profile.Options.Hotkey.Toggle);
Assert.Equal("azure-speech", profile.Options.Recording.TranscriptionProvider);
Assert.Equal("germanywestcentral", profile.Options.AzureSpeech.Region);
Assert.Equal("en-US", profile.Options.AzureSpeech.Language);
Assert.Equal(["en-US"], profile.Options.AzureSpeech.AutoDetectLanguages);
}
[Fact]
public void CheckedInEnglishProfileUsesLanguageHotkey()
{
var provider = CreateProviderFromAppsettings();
var profile = provider.GetRequiredProfile("english");
Assert.Equal("Ctrl+Alt+L", profile.Options.Hotkey.Toggle);
}
[Fact]
public void DuplicateProfileHotkeysAreRejected()
{
@@ -79,7 +69,7 @@ public sealed class LaunchProfileOptionsProviderTests
["MeetingAssistant:Hotkey:Toggle"] = "Ctrl+Alt+M",
["MeetingAssistant:Hotkey:Abort"] = "Ctrl+Alt+Z",
["MeetingAssistant:Screenshots:Hotkey"] = "Ctrl+Alt+S",
["MeetingAssistant:LaunchProfiles:english:Hotkey:Toggle"] = "Ctrl+Alt+L",
["MeetingAssistant:LaunchProfiles:english:Hotkey:Toggle"] = "Ctrl+Alt+E",
["MeetingAssistant:LaunchProfiles:english:Hotkey:Abort"] = "Ctrl+Alt+Shift+D",
["MeetingAssistant:LaunchProfiles:english:Screenshots:Hotkey"] = "Ctrl+Alt+Shift+S"
});
@@ -146,28 +136,4 @@ public sealed class LaunchProfileOptionsProviderTests
.Build();
return new ConfigurationLaunchProfileOptionsProvider(configuration);
}
private static ConfigurationLaunchProfileOptionsProvider CreateProviderFromAppsettings()
{
var root = FindRepositoryRoot();
var configuration = new ConfigurationBuilder()
.AddJsonFile(Path.Combine(root, "MeetingAssistant", "appsettings.json"), optional: false)
.Build();
return new ConfigurationLaunchProfileOptionsProvider(configuration);
}
private static string FindRepositoryRoot()
{
for (var directory = new DirectoryInfo(AppContext.BaseDirectory);
directory is not null;
directory = directory.Parent)
{
if (File.Exists(Path.Combine(directory.FullName, "MeetingAssistant.slnx")))
{
return directory.FullName;
}
}
throw new InvalidOperationException("Could not locate MeetingAssistant.slnx from the test output folder.");
}
}
@@ -1,253 +1,103 @@
using MeetingAssistant.Summary;
using Microsoft.Extensions.AI;
using System.Net;
using System.Text;
using System.Text.Json;
namespace MeetingAssistant.Tests;
public sealed class LiteLlmResponsesChatClientTests
{
[Fact]
public async Task ClientAssemblesStreamedTextResponseWithMetadataAndUsage()
public void ParserIgnoresReasoningItemsWithNullStatusAndReadsText()
{
var handler = new SequencedHttpMessageHandler(
new HttpResponseMessage(HttpStatusCode.OK)
const string json = """
{
Content = new StringContent(
"""
data: {"type":"response.created","response":{"id":"resp_stream","created_at":1779147100,"model":"gpt-5.5","object":"response","output":[],"parallel_tool_calls":true,"status":"in_progress","store":false},"sequence_number":0}
"id": "resp_test",
"created_at": 1779147100,
"model": "gpt-5.5-2026-04-23",
"output": [
{
"type": "reasoning",
"summary": [],
"status": null
},
{
"type": "message",
"role": "assistant",
"status": "completed",
"content": [
{
"type": "output_text",
"text": "OK"
}
]
}
]
}
""";
data: {"type":"response.output_item.added","output_index":0,"item":{"id":"msg_stream","type":"message","status":"in_progress","content":[],"role":"assistant"},"sequence_number":1}
var response = LiteLlmResponsesChatClient.ParseResponseJson(json);
data: {"type":"response.content_part.added","item_id":"msg_stream","output_index":0,"content_index":0,"part":{"type":"output_text","annotations":[],"text":""},"sequence_number":2}
data: {"type":"response.output_text.delta","item_id":"msg_stream","output_index":0,"content_index":0,"delta":"Streamed OK","sequence_number":3}
data: {"type":"response.output_text.done","item_id":"msg_stream","output_index":0,"content_index":0,"text":"Streamed OK","sequence_number":4}
data: {"type":"response.content_part.done","item_id":"msg_stream","output_index":0,"content_index":0,"part":{"type":"output_text","annotations":[],"text":"Streamed OK"},"sequence_number":5}
data: {"type":"response.output_item.done","output_index":0,"item":{"id":"msg_stream","type":"message","status":"completed","content":[{"type":"output_text","annotations":[],"text":"Streamed OK"}],"role":"assistant"},"sequence_number":6}
data: {"type":"response.completed","response":{"id":"resp_stream","created_at":1779147100,"model":"gpt-5.5","object":"response","output":[],"parallel_tool_calls":true,"status":"completed","store":false,"usage":{"input_tokens":12,"output_tokens":3,"total_tokens":15}},"sequence_number":7}
data: [DONE]
""",
Encoding.UTF8,
"text/event-stream")
});
using var client = CreateClient(handler, reconnectionAttempts: 0);
var response = await client.GetResponseAsync([new ChatMessage(ChatRole.User, "reply")]);
Assert.Equal("Streamed OK", response.Text);
Assert.Equal("resp_stream", response.ResponseId);
Assert.Equal("gpt-5.5", response.ModelId);
Assert.Equal(DateTimeOffset.FromUnixTimeSeconds(1779147100), response.CreatedAt);
Assert.NotNull(response.Usage);
Assert.Equal(12, response.Usage.InputTokenCount);
Assert.Equal(3, response.Usage.OutputTokenCount);
Assert.Equal(15, response.Usage.TotalTokenCount);
Assert.Equal("OK", response.Text);
Assert.Equal("resp_test", response.ResponseId);
Assert.Equal("gpt-5.5-2026-04-23", response.ModelId);
}
[Fact]
public async Task ClientAssemblesStreamedFunctionCall()
public void ParserReadsFunctionCalls()
{
var handler = new SequencedHttpMessageHandler(
new HttpResponseMessage(HttpStatusCode.OK)
const string json = """
{
Content = new StringContent(
"""
data: {"type":"response.output_item.done","output_index":0,"item":{"type":"function_call","call_id":"call_stream","name":"write_summary","arguments":"{\"markdown\":\"# Summary\\nDone\"}","status":"completed"}}
"output": [
{
"type": "function_call",
"call_id": "call_1",
"name": "write_summary",
"arguments": "{\"markdown\":\"# Summary\\nDone\"}",
"status": "completed"
}
]
}
""";
data: {"type":"response.completed","response":{"id":"resp_stream","model":"gpt-5.5","output":[]}}
data: [DONE]
""",
Encoding.UTF8,
"text/event-stream")
});
using var client = CreateClient(handler, reconnectionAttempts: 0);
var response = await client.GetResponseAsync([new ChatMessage(ChatRole.User, "write summary")]);
var response = LiteLlmResponsesChatClient.ParseResponseJson(json);
var call = Assert.IsType<FunctionCallContent>(Assert.Single(response.Messages[0].Contents));
Assert.Equal("call_stream", call.CallId);
Assert.Equal("call_1", call.CallId);
Assert.Equal("write_summary", call.Name);
Assert.NotNull(call.Arguments);
Assert.Equal("# Summary\nDone", call.Arguments["markdown"]?.ToString());
Assert.Equal("# Summary\nDone", call.Arguments["markdown"]);
}
[Fact]
public async Task ClientUsesNonStreamingResponsesWhenConfigured()
public void ParserReadsUsage()
{
var handler = new RecordingHttpMessageHandler(_ => new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(
"""
const string json = """
{
"usage": {
"input_tokens": 123,
"output_tokens": 45,
"total_tokens": 168
},
"output": [
{
"id": "resp_nonstream",
"created_at": 1779147100,
"model": "gpt-5.5",
"object": "response",
"output": [
"type": "message",
"content": [
{
"id": "msg_nonstream",
"type": "message",
"status": "completed",
"content": [
{
"type": "output_text",
"annotations": [],
"text": "Non-streamed OK"
}
],
"role": "assistant"
"type": "output_text",
"text": "OK"
}
],
"parallel_tool_calls": true,
"status": "completed",
"store": false,
"usage": {
"input_tokens": 12,
"output_tokens": 4,
"total_tokens": 16
}
]
}
""",
Encoding.UTF8,
"application/json")
});
using var client = CreateClient(handler, reconnectionAttempts: 0, useStreaming: false);
]
}
""";
var response = await client.GetResponseAsync([new ChatMessage(ChatRole.User, "reply")]);
var response = LiteLlmResponsesChatClient.ParseResponseJson(json);
Assert.Equal("Non-streamed OK", response.Text);
Assert.Contains("\"stream\":false", Assert.Single(handler.RequestBodies));
}
[Fact]
public async Task ClientSendsChatMessagesAsResponsesMessageItems()
{
var handler = new RecordingHttpMessageHandler(_ => CreateStreamedTextResponse("Done."));
using var client = CreateClient(handler, reconnectionAttempts: 0);
await client.GetResponseAsync(
[
new ChatMessage(ChatRole.User, "write summary"),
new ChatMessage(ChatRole.Assistant, "I will inspect the meeting.")
]);
using var request = JsonDocument.Parse(Assert.Single(handler.RequestBodies));
var inputItems = request.RootElement.GetProperty("input").EnumerateArray().ToArray();
Assert.Equal(2, inputItems.Length);
Assert.All(inputItems, item => Assert.Equal("message", item.GetProperty("type").GetString()));
Assert.Equal("user", inputItems[0].GetProperty("role").GetString());
Assert.Equal(
"input_text",
Assert.Single(inputItems[0].GetProperty("content").EnumerateArray())
.GetProperty("type")
.GetString());
Assert.Equal("assistant", inputItems[1].GetProperty("role").GetString());
Assert.Equal(
"output_text",
Assert.Single(inputItems[1].GetProperty("content").EnumerateArray())
.GetProperty("type")
.GetString());
Assert.DoesNotContain("\"type\":\"unknown\"", handler.RequestBodies[0]);
}
[Fact]
public async Task AgentLoopReturnsMalformedFunctionArgumentsWithoutInvokingTool()
{
var responses = new Queue<HttpResponseMessage>(
[
new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(
"""
data: {"type":"response.output_item.done","output_index":0,"item":{"type":"function_call","call_id":"call_invalid","name":"write_summary","arguments":"{not-json","status":"completed"}}
data: {"type":"response.completed","response":{"id":"resp_invalid","model":"gpt-5.5","output":[]}}
data: [DONE]
""",
Encoding.UTF8,
"text/event-stream")
},
CreateStreamedTextResponse("Recovered.")
]);
var handler = new RecordingHttpMessageHandler(_ => responses.Dequeue());
var toolInvoked = false;
var tool = AIFunctionFactory.Create(
(string markdown) =>
{
toolInvoked = true;
return markdown;
},
"write_summary",
"Writes a summary.");
using var innerClient = CreateClient(handler, reconnectionAttempts: 0);
using var functionClient = innerClient
.AsBuilder()
.UseFunctionInvocation(
loggerFactory: null,
client => client.FunctionInvoker = FunctionInvocationGuard.InvokeAsync)
.Build();
var response = await functionClient.GetResponseAsync(
[new ChatMessage(ChatRole.User, "write summary")],
new ChatOptions { Tools = [tool] });
Assert.False(toolInvoked);
Assert.Equal("Recovered.", response.Text);
Assert.Equal(2, handler.RequestBodies.Count);
Assert.Contains("invalid_tool_arguments", handler.RequestBodies[1]);
Assert.Contains("call_invalid", handler.RequestBodies[1]);
}
[Fact]
public async Task ClientReportsVisibleReasoningSummariesSeparatelyFromResponseText()
{
var handler = new SequencedHttpMessageHandler(
CreateStreamedTextResponse(
"Done.",
"Checked the configured rules file.",
"Prepared a targeted update."));
var reasoningSummaries = new List<string>();
using var client = CreateClient(
handler,
reconnectionAttempts: 0,
reasoningSummaryChanged: reasoningSummaries.Add);
var response = await client.GetResponseAsync([new ChatMessage(ChatRole.User, "update rules")]);
Assert.Equal("Done.", response.Text);
Assert.Equal(
[
"Checked the configured rules file.",
"Prepared a targeted update."
],
reasoningSummaries);
}
[Fact]
public async Task ClientDoesNotFailResponseWhenReasoningSummaryCallbackFails()
{
var handler = new SequencedHttpMessageHandler(
CreateStreamedTextResponse("Done.", "Checked the configured rules file."));
using var client = CreateClient(
handler,
reconnectionAttempts: 0,
reasoningSummaryChanged: _ => throw new InvalidOperationException("UI failed"));
var response = await client.GetResponseAsync([new ChatMessage(ChatRole.User, "update rules")]);
Assert.Equal("Done.", response.Text);
Assert.NotNull(response.Usage);
Assert.Equal(123, response.Usage.InputTokenCount);
Assert.Equal(45, response.Usage.OutputTokenCount);
Assert.Equal(168, response.Usage.TotalTokenCount);
}
[Fact]
@@ -258,15 +108,30 @@ public sealed class LiteLlmResponsesChatClientTests
{
Content = new StringContent("Internal Server Error")
},
CreateStreamedTextResponse("Done."));
var retryCount = 0;
using var client = CreateClient(handler, reconnectionAttempts: 1, retrying: () => retryCount++);
new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent("""
{
"output": [
{
"type": "message",
"content": [
{
"type": "output_text",
"text": "Done."
}
]
}
]
}
""")
});
using var client = CreateClient(handler, reconnectionAttempts: 1);
var response = await client.GetResponseAsync([new ChatMessage(ChatRole.User, "write summary")]);
Assert.Equal("Done.", response.Text);
Assert.Equal(2, handler.RequestCount);
Assert.Equal(1, retryCount);
}
[Fact]
@@ -289,7 +154,24 @@ public sealed class LiteLlmResponsesChatClientTests
[Fact]
public async Task ClientSendsUserInitiatorOnceThenAgentInitiator()
{
var handler = new RecordingHttpMessageHandler(_ => CreateStreamedTextResponse("Done."));
var handler = new RecordingHttpMessageHandler(_ => new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent("""
{
"output": [
{
"type": "message",
"content": [
{
"type": "output_text",
"text": "Done."
}
]
}
]
}
""")
});
using var client = CreateClient(handler, reconnectionAttempts: 0);
await client.GetResponseAsync([new ChatMessage(ChatRole.User, "write summary")]);
@@ -321,7 +203,24 @@ public sealed class LiteLlmResponsesChatClientTests
};
}
return CreateStreamedTextResponse("Done.");
return new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent("""
{
"output": [
{
"type": "message",
"content": [
{
"type": "output_text",
"text": "Done."
}
]
}
]
}
""")
};
});
using var client = CreateClient(
handler,
@@ -356,7 +255,24 @@ public sealed class LiteLlmResponsesChatClientTests
};
}
return CreateStreamedTextResponse("Done.");
return new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent("""
{
"output": [
{
"type": "message",
"content": [
{
"type": "output_text",
"text": "Done."
}
]
}
]
}
""")
};
});
using var client = CreateClient(
handler,
@@ -387,10 +303,7 @@ public sealed class LiteLlmResponsesChatClientTests
private static LiteLlmResponsesChatClient CreateClient(
HttpMessageHandler handler,
int reconnectionAttempts,
LiteLlmResponsesCompactionOptions? compactionOptions = null,
Action? retrying = null,
Action<string>? reasoningSummaryChanged = null,
bool useStreaming = true)
LiteLlmResponsesCompactionOptions? compactionOptions = null)
{
return new LiteLlmResponsesChatClient(
new HttpClient(handler)
@@ -403,51 +316,7 @@ public sealed class LiteLlmResponsesChatClientTests
reasoningEffort: "none",
reconnectionAttempts,
TimeSpan.Zero,
compactionOptions,
retrying: retrying,
reasoningSummaryChanged: reasoningSummaryChanged,
useStreaming: useStreaming);
}
private static HttpResponseMessage CreateStreamedTextResponse(
string text,
params string[] reasoningSummaries)
{
var events = new List<string>();
for (var index = 0; index < reasoningSummaries.Length; index++)
{
events.Add("data: " + JsonSerializer.Serialize(new
{
type = "response.reasoning_summary_text.delta",
item_id = "reasoning_stream",
output_index = 0,
summary_index = index,
delta = reasoningSummaries[index]
}));
}
var outputIndex = reasoningSummaries.Length > 0 ? 1 : 0;
events.Add("data: " + JsonSerializer.Serialize(new
{
type = "response.output_text.delta",
item_id = "msg_stream",
output_index = outputIndex,
content_index = 0,
delta = text
}));
events.Add(
"""data: {"type":"response.completed","response":{"id":"resp_stream","created_at":1779147100,"model":"gpt-5.5","object":"response","output":[],"parallel_tool_calls":true,"status":"completed","store":false,"usage":{"input_tokens":12,"output_tokens":3,"total_tokens":15}}}""");
events.Add("data: [DONE]");
return new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(
string.Join($"{Environment.NewLine}{Environment.NewLine}", events)
+ Environment.NewLine
+ Environment.NewLine,
Encoding.UTF8,
"text/event-stream")
};
compactionOptions);
}
private sealed class SequencedHttpMessageHandler : HttpMessageHandler
@@ -16,7 +16,9 @@ public sealed class LiteLlmScreenshotOcrClientTests
[Fact]
public async Task ExtractUsesAgentEndpointAndModelWhenOcrEndpointAndModelAreBlank()
{
var screenshotPath = await CreateScreenshotAsync([1, 2, 3]);
var screenshotPath = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N") + ".png");
Directory.CreateDirectory(Path.GetDirectoryName(screenshotPath)!);
await File.WriteAllBytesAsync(screenshotPath, [1, 2, 3]);
var handler = new RecordingHandler("""
{
"output": [
@@ -73,7 +75,9 @@ public sealed class LiteLlmScreenshotOcrClientTests
[Fact]
public async Task ExtractUsesScreenshotOcrEndpointAndModelWhenConfigured()
{
var screenshotPath = await CreateScreenshotAsync([4, 5, 6]);
var screenshotPath = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N") + ".png");
Directory.CreateDirectory(Path.GetDirectoryName(screenshotPath)!);
await File.WriteAllBytesAsync(screenshotPath, [4, 5, 6]);
var handler = new RecordingHandler("""{ "output_text": "OCR result" }""");
var client = new LiteLlmScreenshotOcrClient(
() => handler,
@@ -112,7 +116,9 @@ public sealed class LiteLlmScreenshotOcrClientTests
[Fact]
public async Task ExtractParsesCropMetadataAndOmitsMetadataFromReturnedText()
{
var screenshotPath = await CreateScreenshotAsync(CreatePngBytes(8, 6));
var screenshotPath = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N") + ".png");
Directory.CreateDirectory(Path.GetDirectoryName(screenshotPath)!);
await File.WriteAllBytesAsync(screenshotPath, CreatePngBytes(8, 6));
var handler = new RecordingHandler("""
{
"output_text": "Slide text\n\n```json\n{ \"crop\": { \"x\": 1, \"y\": 2, \"width\": 3, \"height\": 4 } }\n```"
@@ -144,67 +150,6 @@ public sealed class LiteLlmScreenshotOcrClientTests
StringComparison.Ordinal);
}
[Fact]
public async Task ExtractParsesAttendeeMetadataAndOmitsMetadataFromReturnedText()
{
var screenshotPath = await CreateScreenshotAsync([1, 2, 3]);
var handler = new RecordingHandler("""
{
"output_text": "Visible participant tiles: Ada and Grace.\n\n```json\n{ \"crop\": null, \"attendees\": [\"Ada Lovelace\", \"Grace Hopper\"] }\n```"
}
""");
var client = new LiteLlmScreenshotOcrClient(
() => handler,
NullLogger<LiteLlmScreenshotOcrClient>.Instance);
var options = new MeetingAssistantOptions
{
Agent =
{
Key = "agent-key"
}
};
var result = await client.ExtractAsync(
screenshotPath,
"Extract screenshot.",
options,
CancellationToken.None);
Assert.Equal("Visible participant tiles: Ada and Grace.", result.Text);
Assert.Equal(["Ada Lovelace", "Grace Hopper"], result.Attendees);
}
[Fact]
public async Task ExtractIgnoresMalformedAttendeesMetadataAndStillParsesCrop()
{
var screenshotPath = await CreateScreenshotAsync(CreatePngBytes(8, 6));
var handler = new RecordingHandler("""
{
"output_text": "Slide text\n\n```json\n{ \"crop\": { \"x\": 1, \"y\": 2, \"width\": 3, \"height\": 4 }, \"attendees\": \"Ada\" }\n```"
}
""");
var client = new LiteLlmScreenshotOcrClient(
() => handler,
NullLogger<LiteLlmScreenshotOcrClient>.Instance);
var options = new MeetingAssistantOptions
{
Agent =
{
Key = "agent-key"
}
};
var result = await client.ExtractAsync(
screenshotPath,
"Extract screenshot.",
options,
CancellationToken.None);
Assert.Equal("Slide text", result.Text);
Assert.Equal(new ScreenshotCropCoordinates(1, 2, 3, 4), result.Crop);
Assert.Empty(result.Attendees);
}
private sealed class RecordingHandler : HttpMessageHandler
{
private readonly string responseBody;
@@ -248,14 +193,6 @@ public sealed class LiteLlmScreenshotOcrClientTests
bitmap.Save(stream, ImageFormat.Png);
return stream.ToArray();
}
private static async Task<string> CreateScreenshotAsync(byte[] bytes)
{
var screenshotPath = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N") + ".png");
Directory.CreateDirectory(Path.GetDirectoryName(screenshotPath)!);
await File.WriteAllBytesAsync(screenshotPath, bytes);
return screenshotPath;
}
}
#pragma warning restore CA1416
@@ -1,85 +0,0 @@
using MeetingAssistant.MeetingNotes;
using MeetingAssistant.Recording;
namespace MeetingAssistant.Tests;
public sealed class MeetingArtifactContentDetectorTests
{
[Fact]
public async Task IsDefaultOnlyReturnsFalseWhenTranscriptContainsSegment()
{
using var fixture = await DefaultArtifactFixture.CreateAsync();
await File.AppendAllTextAsync(
fixture.Artifacts.TranscriptPath,
"[00:00:01] Guest-1: hello" + Environment.NewLine);
var result = await MeetingArtifactContentDetector.IsDefaultOnlyAsync(
fixture.Artifacts,
CancellationToken.None);
Assert.False(result);
}
[Fact]
public async Task IsDefaultOnlyReturnsFalseWhenMeetingNoteContainsUserNotes()
{
using var fixture = await DefaultArtifactFixture.CreateAsync();
await File.AppendAllTextAsync(
fixture.Artifacts.MeetingNotePath,
"Manual note" + Environment.NewLine);
var result = await MeetingArtifactContentDetector.IsDefaultOnlyAsync(
fixture.Artifacts,
CancellationToken.None);
Assert.False(result);
}
private sealed class DefaultArtifactFixture : IDisposable
{
private DefaultArtifactFixture(string root, MeetingSessionArtifacts artifacts)
{
Root = root;
Artifacts = artifacts;
}
public string Root { get; }
public MeetingSessionArtifacts Artifacts { get; }
public static async Task<DefaultArtifactFixture> CreateAsync()
{
var root = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N"));
Directory.CreateDirectory(root);
var artifacts = new MeetingSessionArtifacts(
Path.Combine(root, "meeting.md"),
Path.Combine(root, "transcript.md"),
Path.Combine(root, "assistant-context.md"),
Path.Combine(root, "summary.md"));
await File.WriteAllTextAsync(
artifacts.MeetingNotePath,
"""
---
title: Test
---
""");
await File.WriteAllTextAsync(
artifacts.TranscriptPath,
"# Meeting Transcript" + Environment.NewLine + Environment.NewLine);
await File.WriteAllTextAsync(
artifacts.AssistantContextPath,
"# Assistant Context" + Environment.NewLine + Environment.NewLine + "## Live Context" + Environment.NewLine);
return new DefaultArtifactFixture(root, artifacts);
}
public void Dispose()
{
if (Directory.Exists(Root))
{
Directory.Delete(Root, recursive: true);
}
}
}
}
@@ -9,8 +9,8 @@
<ItemGroup>
<PackageReference Include="coverlet.collector" Version="10.0.1" />
<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="10.0.10" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.8.1" />
<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="10.0.8" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.6.0" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.5" />
</ItemGroup>
@@ -57,19 +57,6 @@ public sealed class MeetingNoteStoreTests
Assert.Equal("[[../Summaries/20260519-summary|Summary]]", loaded.Frontmatter.Summary);
}
[Fact]
public async Task StoreNamesGeneratedMeetingNoteWithMinutePrecision()
{
var (_, store) = CreateStore();
var saved = await store.SaveAsync(
MeetingNoteTemplate.Create(
title: "Leadership Sync",
startTime: DateTimeOffset.Parse("2026-05-19T10:03:42+02:00")),
CancellationToken.None);
Assert.Equal("20260519-1003-note.md", Path.GetFileName(saved.Path));
}
[Fact]
public async Task FrontmatterUpdatePreservesExistingUserNotesBody()
{
@@ -105,102 +92,6 @@ public sealed class MeetingNoteStoreTests
Assert.Equal(userNotes, reloaded.UserNotes);
}
[Fact]
public async Task ReadConvertsScalarListFrontmatterToSingleItemLists()
{
var (vaultRoot, store) = CreateStore();
var notePath = Path.Combine(vaultRoot, "Meetings", "Notes", "manual.md");
Directory.CreateDirectory(Path.GetDirectoryName(notePath)!);
await File.WriteAllTextAsync(
notePath,
"""
---
title: Manual frontmatter
start_time: "2026-05-19T10:00:00.0000000+02:00"
end_time: ""
attendees: Ada
projects: Meeting Assistant
transcript: "[[../Transcripts/manual-transcript|Transcript]]"
assistant_context: "[[../Assistant Context/manual-context|Assistant Context]]"
summary: "[[../Summaries/manual-summary|Summary]]"
---
User notes.
""");
var loaded = await store.ReadAsync(notePath, CancellationToken.None);
Assert.Equal(["Ada"], loaded.Frontmatter.Attendees);
Assert.Equal(["Meeting Assistant"], loaded.Frontmatter.Projects);
Assert.Equal("User notes.", loaded.UserNotes);
}
[Fact]
public async Task StoreEscapesApostrophePrefixedAttendeesBeforeWritingFrontmatter()
{
var (store, saved) = await SaveNoteAsync(
title: "Escaped Attendees",
attendees: ["'Ada Lovelace"],
projects: [],
userNotes: "Discuss attendee import.");
var content = await File.ReadAllTextAsync(saved.Path);
var loaded = await store.ReadAsync(saved.Path, CancellationToken.None);
Assert.Contains("- \"'Ada Lovelace\"", content);
Assert.Equal(["'Ada Lovelace"], loaded.Frontmatter.Attendees);
Assert.Equal("Discuss attendee import.", loaded.UserNotes);
}
[Theory]
[InlineData("Ada # platform lead")]
[InlineData("- Ada Lovelace")]
[InlineData("? Ada Lovelace")]
[InlineData("{Ada: Platform}")]
[InlineData("*Ada")]
[InlineData("&Ada")]
[InlineData("!Ada")]
[InlineData("| Ada")]
[InlineData("> Ada")]
[InlineData("@Ada")]
[InlineData("`Ada")]
[InlineData("true")]
[InlineData("null")]
[InlineData("2026-07-08")]
[InlineData("Ada \"The Architect\" Lovelace")]
[InlineData("C:\\People\\Ada")]
public async Task StoreEscapesYamlSensitiveAttendeesBeforeWritingFrontmatter(string attendee)
{
var (store, saved) = await SaveNoteAsync(
title: "Escaped Attendees",
attendees: [attendee],
projects: [],
userNotes: "Discuss attendee import.");
var loaded = await store.ReadAsync(saved.Path, CancellationToken.None);
Assert.Equal([attendee], loaded.Frontmatter.Attendees);
Assert.Equal("Discuss attendee import.", loaded.UserNotes);
}
[Theory]
[InlineData("Planning # Q3")]
[InlineData("- Planning")]
[InlineData("{Planning: Q3}")]
[InlineData("true")]
[InlineData("2026-07-08")]
public async Task StoreEscapesYamlSensitiveScalarAndProjectFrontmatterValues(string value)
{
var (store, saved) = await SaveNoteAsync(
title: value,
attendees: [],
projects: [value],
userNotes: "Discuss YAML escaping.");
var loaded = await store.ReadAsync(saved.Path, CancellationToken.None);
Assert.Equal(value, loaded.Frontmatter.Title);
Assert.Equal([value], loaded.Frontmatter.Projects);
Assert.Equal("Discuss YAML escaping.", loaded.UserNotes);
}
[Fact]
public void ActionLinkEscapesSummaryFileName()
{
@@ -213,26 +104,6 @@ public sealed class MeetingNoteStoreTests
link);
}
private static async Task<(MarkdownMeetingNoteStore Store, MeetingNote Saved)> SaveNoteAsync(
string title,
IReadOnlyList<string> attendees,
IReadOnlyList<string> projects,
string userNotes)
{
var (vaultRoot, store) = CreateStore();
var note = MeetingNoteTemplate.Create(
title: title,
attendees: attendees,
projects: projects,
transcriptPath: Path.Combine(vaultRoot, "Meetings", "Transcripts", "20260519-transcript.md"),
assistantContextPath: Path.Combine(vaultRoot, "Meetings", "Assistant Context", "20260519-context.md"),
summaryPath: Path.Combine(vaultRoot, "Meetings", "Summaries", "20260519-summary.md"),
userNotes: userNotes);
var saved = await store.SaveAsync(note, CancellationToken.None);
return (store, saved);
}
private static (string VaultRoot, MarkdownMeetingNoteStore Store) CreateStore()
{
var vaultRoot = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N"));
@@ -1,9 +1,6 @@
using MeetingAssistant.MeetingNotes;
using MeetingAssistant.Screenshots;
using MeetingAssistant.Speakers;
using MeetingAssistant.Workflow;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using System.Drawing;
using System.Drawing.Imaging;
@@ -133,194 +130,6 @@ public sealed class MeetingScreenshotServiceTests
Assert.Contains("Shared screen text", context);
}
[Fact]
public async Task CaptureAddsOcrAttendeesToMeetingNoteThroughCanonicalizer()
{
var fixture = await ScreenshotFixture.CreateAsync(
options =>
{
options.Screenshots.Ocr.Enabled = true;
},
attendees: ["Ada Lovelace"]);
var ocr = new CapturingScreenshotOcrClient(
"Visible participant tiles: Ada, Grace, and Ada again.",
attendees: ["Ada L.", "Grace Hopper", "Ada Lovelace"]);
var canonicalizer = new MappingAttendeeCanonicalizer(new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
{
["Ada L."] = "Ada Lovelace"
});
var service = fixture.CreateService(
new FixedScreenshotCapture([1, 2, 3]),
ocr,
canonicalizer);
await service.CaptureAsync(
fixture.Artifacts,
DateTimeOffset.Parse("2026-05-26T10:00:00+02:00"),
DateTimeOffset.Parse("2026-05-26T10:00:10+02:00"),
fixture.Options,
CancellationToken.None);
await service.WaitForPendingOcrAsync(fixture.Artifacts, TimeSpan.FromSeconds(5), CancellationToken.None);
var meeting = await fixture.NoteStore.ReadAsync(fixture.Artifacts.MeetingNotePath, CancellationToken.None);
Assert.Equal(["Ada Lovelace", "Grace Hopper"], meeting.Frontmatter.Attendees);
Assert.Contains(canonicalizer.Requests, request =>
request.SequenceEqual(["Ada Lovelace", "Ada L.", "Grace Hopper", "Ada Lovelace"]));
}
[Fact]
public async Task CaptureTransformsOcrAttendeesBeforeWritingMeetingNote()
{
var fixture = await ScreenshotFixture.CreateAsync(
options =>
{
options.Screenshots.Ocr.Enabled = true;
});
var workflowEngine = new TransformingAttendeeWorkflowEngine("Ada Lovelace (Contoso)", "Ada Lovelace");
var service = fixture.CreateService(
new FixedScreenshotCapture([1, 2, 3]),
new CapturingScreenshotOcrClient(
"Visible participant tile: Ada.",
attendees: ["Ada Lovelace (Contoso)"]),
meetingWorkflowEngine: workflowEngine);
await service.CaptureAsync(
fixture.Artifacts,
DateTimeOffset.Parse("2026-05-26T10:00:00+02:00"),
DateTimeOffset.Parse("2026-05-26T10:00:10+02:00"),
fixture.Options,
CancellationToken.None);
await service.WaitForPendingOcrAsync(fixture.Artifacts, TimeSpan.FromSeconds(5), CancellationToken.None);
var meeting = await fixture.NoteStore.ReadAsync(fixture.Artifacts.MeetingNotePath, CancellationToken.None);
Assert.Equal(["Ada Lovelace"], meeting.Frontmatter.Attendees);
Assert.Equal(["Ada Lovelace (Contoso)"], workflowEngine.AttendeeRequests);
}
[Fact]
public async Task CaptureWritesRetryLinkWhenOcrFails()
{
var fixture = await ScreenshotFixture.CreateAsync(options =>
{
options.Api.PublicBaseUrl = "http://localhost:5090";
options.Screenshots.Ocr.Enabled = true;
});
var service = fixture.CreateService(
new FixedScreenshotCapture([1, 2, 3]),
new ThrowingScreenshotOcrClient("vision offline"));
var result = await service.CaptureAsync(
fixture.Artifacts,
DateTimeOffset.Parse("2026-05-26T10:00:00+02:00"),
DateTimeOffset.Parse("2026-05-26T10:00:10+02:00"),
fixture.Options,
CancellationToken.None);
await service.WaitForPendingOcrAsync(fixture.Artifacts, TimeSpan.FromSeconds(5), CancellationToken.None);
var context = await File.ReadAllTextAsync(fixture.Artifacts.AssistantContextPath);
var screenshotId = ExtractScreenshotOcrId(context);
Assert.Contains("_OCR failed: vision offline_", context);
Assert.Contains("<!-- screenshot-ocr:", context);
Assert.Contains("<!-- /screenshot-ocr:", context);
Assert.Contains("[Retry screenshot OCR](http://localhost:5090/meetings/screenshot-ocr/retry?", context);
Assert.Contains($"screenshotId={screenshotId}", context);
Assert.Contains($"screenshotPath={Uri.EscapeDataString(result.ScreenshotPath)}", context);
Assert.Contains($"assistantContextPath={Uri.EscapeDataString(fixture.Artifacts.AssistantContextPath)}", context);
}
[Fact]
public async Task RetryOcrReplacesFailureForSameScreenshot()
{
var fixture = await ScreenshotFixture.CreateAsync(options =>
{
options.Screenshots.Ocr.Enabled = true;
});
var ocr = new SequencedScreenshotOcrClient(
new InvalidOperationException("vision offline"),
new ScreenshotOcrResult("Retried OCR text", null, ["Grace Hopper"]));
var service = fixture.CreateService(
new FixedScreenshotCapture([1, 2, 3]),
ocr);
var result = await service.CaptureAsync(
fixture.Artifacts,
DateTimeOffset.Parse("2026-05-26T10:00:00+02:00"),
DateTimeOffset.Parse("2026-05-26T10:00:10+02:00"),
fixture.Options,
CancellationToken.None);
await service.WaitForPendingOcrAsync(fixture.Artifacts, TimeSpan.FromSeconds(5), CancellationToken.None);
var failedContext = await File.ReadAllTextAsync(fixture.Artifacts.AssistantContextPath);
var screenshotId = ExtractScreenshotOcrId(failedContext);
var retry = await service.TriggerOcrRetryAsync(
fixture.Artifacts,
result.ScreenshotPath,
screenshotId,
fixture.Options,
CancellationToken.None);
await service.WaitForPendingOcrAsync(fixture.Artifacts, TimeSpan.FromSeconds(5), CancellationToken.None);
Assert.NotNull(retry);
Assert.Equal(result.ScreenshotPath, retry.ScreenshotPath);
Assert.Equal(screenshotId, retry.ScreenshotId);
Assert.Equal([result.ScreenshotPath, result.ScreenshotPath], ocr.ScreenshotPaths);
var context = await File.ReadAllTextAsync(fixture.Artifacts.AssistantContextPath);
Assert.Contains("### OCR", context);
Assert.Contains("Retried OCR text", context);
Assert.DoesNotContain("vision offline", context);
Assert.DoesNotContain("Retry screenshot OCR", context);
Assert.DoesNotContain("<!-- screenshot-ocr:", context);
}
[Fact]
public async Task ProcessMeetingNoteImageEmbedsAppendsContextAndRunsOcrWithoutCropOrAttendees()
{
var fixture = await ScreenshotFixture.CreateAsync(
options =>
{
options.Screenshots.Ocr.Enabled = true;
},
attendees: ["Ada Lovelace"],
userNotes: "Discussed ![[whiteboard.png]] and ![Diagram](attachments/diagram.png).");
var noteFolder = Path.GetDirectoryName(fixture.Artifacts.MeetingNotePath)!;
var attachmentsFolder = Path.Combine(noteFolder, "attachments");
Directory.CreateDirectory(attachmentsFolder);
var whiteboardPath = Path.Combine(noteFolder, "whiteboard.png");
var diagramPath = Path.Combine(attachmentsFolder, "diagram.png");
await File.WriteAllBytesAsync(whiteboardPath, [1, 2, 3]);
await File.WriteAllBytesAsync(diagramPath, [4, 5, 6]);
var ocr = new CapturingScreenshotOcrClient(
"Manual image OCR",
new ScreenshotCropCoordinates(1, 1, 2, 2),
["Grace Hopper"]);
var service = fixture.CreateService(
new FixedScreenshotCapture([1, 2, 3]),
ocr);
var originalMeetingNote = await File.ReadAllTextAsync(fixture.Artifacts.MeetingNotePath);
var result = await service.ProcessMeetingNoteImageEmbedsAsync(
fixture.Artifacts,
fixture.Options,
CancellationToken.None);
await service.WaitForPendingOcrAsync(fixture.Artifacts, TimeSpan.FromSeconds(5), CancellationToken.None);
Assert.Equal(2, result.QueuedCount);
Assert.Equal(2, ocr.CallCount);
Assert.Equal([whiteboardPath, diagramPath], ocr.ScreenshotPaths);
var context = await File.ReadAllTextAsync(fixture.Artifacts.AssistantContextPath);
Assert.Contains("## Meeting Note Image", context);
Assert.Contains("Image from meeting note.", context);
Assert.Contains("Original embed: `![[whiteboard.png]]`", context);
Assert.Contains("Original embed: `![Diagram](attachments/diagram.png)`", context);
Assert.Contains("whiteboard.png", context);
Assert.Contains("diagram.png", context);
Assert.Contains("Manual image OCR", context);
Assert.DoesNotContain("Cropped screenshot", context);
Assert.Equal(originalMeetingNote, await File.ReadAllTextAsync(fixture.Artifacts.MeetingNotePath));
var meeting = await fixture.NoteStore.ReadAsync(fixture.Artifacts.MeetingNotePath, CancellationToken.None);
Assert.Equal(["Ada Lovelace"], meeting.Frontmatter.Attendees);
}
[Fact]
public async Task WaitForPendingOcrWaitsForRunningScreenshotOcr()
{
@@ -383,13 +192,11 @@ public sealed class MeetingScreenshotServiceTests
private ScreenshotFixture(
MeetingAssistantOptions options,
MeetingSessionArtifacts artifacts,
MarkdownMeetingArtifactStore artifactStore,
MarkdownMeetingNoteStore noteStore)
MarkdownMeetingArtifactStore artifactStore)
{
Options = options;
Artifacts = artifacts;
ArtifactStore = artifactStore;
NoteStore = noteStore;
}
public MeetingAssistantOptions Options { get; }
@@ -398,12 +205,8 @@ public sealed class MeetingScreenshotServiceTests
public MarkdownMeetingArtifactStore ArtifactStore { get; }
public MarkdownMeetingNoteStore NoteStore { get; }
public static async Task<ScreenshotFixture> CreateAsync(
Action<MeetingAssistantOptions>? configure = null,
IReadOnlyList<string>? attendees = null,
string userNotes = "")
Action<MeetingAssistantOptions>? configure = null)
{
var root = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N"));
var options = new MeetingAssistantOptions
@@ -415,9 +218,6 @@ public sealed class MeetingScreenshotServiceTests
}
};
configure?.Invoke(options);
var noteStore = new MarkdownMeetingNoteStore(
Microsoft.Extensions.Options.Options.Create(options),
NullLogger<MarkdownMeetingNoteStore>.Instance);
var artifacts = new MeetingSessionArtifacts(
Path.Combine(root, "Notes", "meeting.md"),
Path.Combine(root, "Transcripts", "transcript.md"),
@@ -425,7 +225,6 @@ public sealed class MeetingScreenshotServiceTests
Path.Combine(root, "Summaries", "summary.md"));
var artifactStore = new MarkdownMeetingArtifactStore(
NullLogger<MarkdownMeetingArtifactStore>.Instance);
Directory.CreateDirectory(Path.GetDirectoryName(artifacts.MeetingNotePath)!);
var meeting = MeetingNoteTemplate.Create(
"Planning",
DateTimeOffset.Parse("2026-05-26T10:00:00+02:00"),
@@ -433,37 +232,26 @@ public sealed class MeetingScreenshotServiceTests
assistantContextPath: artifacts.AssistantContextPath,
summaryPath: artifacts.SummaryPath) with
{
Path = artifacts.MeetingNotePath,
UserNotes = userNotes
Path = artifacts.MeetingNotePath
};
meeting.Frontmatter.Attendees = attendees?.ToList() ?? [];
var savedMeeting = await noteStore.SaveAsync(
meeting,
options,
CancellationToken.None);
await artifactStore.CreateAssistantContextAsync(
artifacts,
savedMeeting,
meeting,
"",
null,
CancellationToken.None);
return new ScreenshotFixture(options, artifacts, artifactStore, noteStore);
return new ScreenshotFixture(options, artifacts, artifactStore);
}
public MeetingScreenshotService CreateService(
IActiveWindowScreenshotCapture capture,
IScreenshotOcrClient ocrClient,
ISpeakerIdentityAttendeeCanonicalizer? attendeeCanonicalizer = null,
IMeetingWorkflowEngine? meetingWorkflowEngine = null)
IScreenshotOcrClient ocrClient)
{
return new MeetingScreenshotService(
capture,
ArtifactStore,
NoteStore,
attendeeCanonicalizer ?? PassthroughSpeakerIdentityAttendeeCanonicalizer.Instance,
ocrClient,
NullLogger<MeetingScreenshotService>.Instance,
meetingWorkflowEngine);
NullLogger<MeetingScreenshotService>.Instance);
}
}
@@ -488,18 +276,15 @@ public sealed class MeetingScreenshotServiceTests
public CapturingScreenshotOcrClient(
string text = "",
ScreenshotCropCoordinates? crop = null,
IReadOnlyList<string>? attendees = null)
ScreenshotCropCoordinates? crop = null)
{
result = new ScreenshotOcrResult(text, crop, attendees ?? []);
result = new ScreenshotOcrResult(text, crop);
}
public int CallCount { get; private set; }
public string? Prompt { get; private set; }
public List<string> ScreenshotPaths { get; } = [];
public Task<ScreenshotOcrResult> ExtractAsync(
string screenshotPath,
string prompt,
@@ -508,7 +293,6 @@ public sealed class MeetingScreenshotServiceTests
{
CallCount++;
Prompt = prompt;
ScreenshotPaths.Add(screenshotPath);
return Task.FromResult(result);
}
}
@@ -536,79 +320,7 @@ public sealed class MeetingScreenshotServiceTests
public void Release(string value)
{
result.TrySetResult(new ScreenshotOcrResult(value, null, []));
}
}
private sealed class ThrowingScreenshotOcrClient : IScreenshotOcrClient
{
private readonly string message;
public ThrowingScreenshotOcrClient(string message)
{
this.message = message;
}
public Task<ScreenshotOcrResult> ExtractAsync(
string screenshotPath,
string prompt,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
throw new InvalidOperationException(message);
}
}
private sealed class SequencedScreenshotOcrClient : IScreenshotOcrClient
{
private readonly Queue<object> results;
public SequencedScreenshotOcrClient(params object[] results)
{
this.results = new Queue<object>(results);
}
public List<string> ScreenshotPaths { get; } = [];
public Task<ScreenshotOcrResult> ExtractAsync(
string screenshotPath,
string prompt,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
ScreenshotPaths.Add(screenshotPath);
var result = results.Dequeue();
if (result is Exception exception)
{
throw exception;
}
return Task.FromResult((ScreenshotOcrResult)result);
}
}
private sealed class MappingAttendeeCanonicalizer : ISpeakerIdentityAttendeeCanonicalizer
{
private readonly IReadOnlyDictionary<string, string> aliases;
public MappingAttendeeCanonicalizer(IReadOnlyDictionary<string, string> aliases)
{
this.aliases = aliases;
}
public List<IReadOnlyList<string>> Requests { get; } = [];
public Task<IReadOnlyList<string>> CanonicalizeAsync(
IReadOnlyList<string> attendees,
CancellationToken cancellationToken)
{
Requests.Add(attendees.ToList());
var result = attendees
.Select(attendee => aliases.TryGetValue(attendee, out var canonical) ? canonical : attendee)
.Where(attendee => !string.IsNullOrWhiteSpace(attendee))
.Distinct(StringComparer.OrdinalIgnoreCase)
.ToList();
return Task.FromResult<IReadOnlyList<string>>(result);
result.TrySetResult(new ScreenshotOcrResult(value, null));
}
}
@@ -624,17 +336,6 @@ public sealed class MeetingScreenshotServiceTests
bitmap.Save(stream, ImageFormat.Png);
return stream.ToArray();
}
private static string ExtractScreenshotOcrId(string context)
{
const string prefix = "<!-- screenshot-ocr:";
var start = context.IndexOf(prefix, StringComparison.Ordinal);
Assert.True(start >= 0);
start += prefix.Length;
var end = context.IndexOf(" -->", start, StringComparison.Ordinal);
Assert.True(end > start);
return context[start..end];
}
}
#pragma warning restore CA1416
@@ -24,7 +24,6 @@ public sealed class MeetingSummaryInstructionBuilderTests
Assert.Contains("You are the Meeting Assistant summary agent.", instructions);
Assert.Contains("include only the most relevant cropped screenshots", instructions);
Assert.Contains("Do not include every cropped screenshot", instructions);
Assert.Contains("encode spaces in image-link targets as `%20`", instructions);
Assert.Contains("Use add_attendee and remove_attendee", instructions);
Assert.Contains("partial screenshot", instructions);
Assert.Contains("override_speaker", instructions);
@@ -35,30 +34,6 @@ public sealed class MeetingSummaryInstructionBuilderTests
Assert.Contains("oneliner", instructions);
Assert.Contains("very short", instructions);
Assert.Contains("conciseness is more important", instructions);
Assert.Contains("generated default title", instructions);
Assert.Contains("purpose of the meeting is clear", instructions);
Assert.Contains("title parameter", instructions);
}
[Fact]
public async Task DefaultPromptTreatsAssistantContextAsMeetingMemoryForUncertainty()
{
var root = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N"));
var artifacts = CreateArtifacts(root);
await WriteMeetingNoteAsync(artifacts.MeetingNotePath, []);
var builder = new MeetingSummaryInstructionBuilder(Options.Create(new MeetingAssistantOptions
{
Agent = new AgentOptions { InitialPrompt = " " },
Vault = new VaultOptions { ProjectsFolder = Path.Combine(root, "Projects") }
}));
var instructions = await builder.BuildAsync(artifacts, CancellationToken.None);
Assert.Contains("meeting-specific memory", instructions);
Assert.Contains("unexpected problems", instructions);
Assert.Contains("missing information", instructions);
Assert.Contains("assumptions", instructions);
Assert.Contains("write_context", instructions);
}
[Fact]
@@ -50,7 +50,7 @@ public sealed class MeetingSummaryToolTests
Assert.False(tools.SummaryWasWritten);
var result = await tools.WriteSummary("# Summary\n\n- Done.", "Done items");
Assert.Equal(Path.GetFileName(artifacts.SummaryPath), result);
Assert.Equal(artifacts.SummaryPath, result);
Assert.True(tools.SummaryWasWritten);
var summary = await File.ReadAllTextAsync(artifacts.SummaryPath);
Assert.Contains("title: Meeting", summary);
@@ -68,70 +68,6 @@ public sealed class MeetingSummaryToolTests
Assert.Contains("# Summary\n\n- Done.", summary);
}
[Fact]
public async Task WriteSummaryRenamesMeetingWhenTitleIsProvided()
{
var root = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N"));
var artifacts = new MeetingSessionArtifacts(
MeetingNotePath: Path.Combine(root, "Meetings", "Notes", "meeting.md"),
TranscriptPath: Path.Combine(root, "Meetings", "Transcripts", "transcript.md"),
AssistantContextPath: Path.Combine(root, "Meetings", "Assistant Context", "context.md"),
SummaryPath: Path.Combine(root, "Meetings", "Summaries", "summary.md"));
Directory.CreateDirectory(Path.GetDirectoryName(artifacts.MeetingNotePath)!);
await File.WriteAllTextAsync(
artifacts.MeetingNotePath,
"""
---
title: Meeting 2026-06-03 07:26
start_time: "2026-06-03T07:26:00.0000000+02:00"
attendees: []
projects: []
---
User note.
""");
var tools = new MeetingSummaryTools(artifacts);
await tools.WriteSummary("# Summary", "Renamed summary", "Notification Safeguard Test");
var meetingNote = await File.ReadAllTextAsync(artifacts.MeetingNotePath);
var summary = await File.ReadAllTextAsync(artifacts.SummaryPath);
Assert.Contains("title: Notification Safeguard Test", meetingNote);
Assert.Contains("title: Notification Safeguard Test", summary);
Assert.Contains("User note.", meetingNote);
}
[Fact]
public async Task WriteSummaryKeepsMeetingTitleWhenTitleIsOmittedOrBlank()
{
var root = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N"));
var artifacts = new MeetingSessionArtifacts(
MeetingNotePath: Path.Combine(root, "Meetings", "Notes", "meeting.md"),
TranscriptPath: Path.Combine(root, "Meetings", "Transcripts", "transcript.md"),
AssistantContextPath: Path.Combine(root, "Meetings", "Assistant Context", "context.md"),
SummaryPath: Path.Combine(root, "Meetings", "Summaries", "summary.md"));
Directory.CreateDirectory(Path.GetDirectoryName(artifacts.MeetingNotePath)!);
await File.WriteAllTextAsync(
artifacts.MeetingNotePath,
"""
---
title: Architecture Review
attendees: []
projects: []
---
""");
var tools = new MeetingSummaryTools(artifacts);
await tools.WriteSummary("# Summary", "Original title summary");
await tools.WriteSummary("# Summary", "Blank title summary", " ");
var meetingNote = await File.ReadAllTextAsync(artifacts.MeetingNotePath);
var summary = await File.ReadAllTextAsync(artifacts.SummaryPath);
Assert.Contains("title: Architecture Review", meetingNote);
Assert.Contains("title: Architecture Review", summary);
Assert.DoesNotContain("Meeting Summary", summary);
}
[Fact]
public async Task WriteSummaryPreservesExistingSummaryAttendees()
{
@@ -218,40 +154,6 @@ public sealed class MeetingSummaryToolTests
Assert.DoesNotContain("- Stale Project", summary);
}
[Fact]
public async Task WriteSummaryCopiesOnlyDistinctNonBlankMeetingProjects()
{
var root = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N"));
var artifacts = new MeetingSessionArtifacts(
MeetingNotePath: Path.Combine(root, "Meetings", "Notes", "meeting.md"),
TranscriptPath: Path.Combine(root, "Meetings", "Transcripts", "transcript.md"),
AssistantContextPath: Path.Combine(root, "Meetings", "Assistant Context", "context.md"),
SummaryPath: Path.Combine(root, "Meetings", "Summaries", "summary.md"));
Directory.CreateDirectory(Path.GetDirectoryName(artifacts.MeetingNotePath)!);
await File.WriteAllTextAsync(
artifacts.MeetingNotePath,
"""
---
title: Meeting
projects:
- " Current Project "
- current project
- ""
- Second Project
---
""");
var tools = new MeetingSummaryTools(artifacts);
await tools.WriteSummary("# Summary", "Project summary");
var summary = await File.ReadAllTextAsync(artifacts.SummaryPath);
Assert.Contains("projects:", summary);
Assert.Contains("- Current Project", summary);
Assert.Contains("- Second Project", summary);
Assert.DoesNotContain("- current project", summary);
Assert.DoesNotContain("- \"\"", summary);
}
[Fact]
public async Task ToolsCanAddAndRemoveMeetingAttendees()
{
@@ -293,44 +195,6 @@ public sealed class MeetingSummaryToolTests
Assert.Contains("transcript: \"[[../Transcripts/transcript|Transcript]]\"", meetingNote);
}
[Fact]
public async Task ToolsTransformAddedAttendeeBeforeWritingMeetingNote()
{
var root = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N"));
var artifacts = new MeetingSessionArtifacts(
MeetingNotePath: Path.Combine(root, "Meetings", "Notes", "meeting.md"),
TranscriptPath: Path.Combine(root, "Meetings", "Transcripts", "transcript.md"),
AssistantContextPath: Path.Combine(root, "Meetings", "Assistant Context", "context.md"),
SummaryPath: Path.Combine(root, "Meetings", "Summaries", "summary.md"));
Directory.CreateDirectory(Path.GetDirectoryName(artifacts.MeetingNotePath)!);
await File.WriteAllTextAsync(
artifacts.MeetingNotePath,
"""
---
title: Meeting
attendees: []
projects: []
transcript: "[[../Transcripts/transcript|Transcript]]"
assistant_context: "[[../Assistant Context/context|Assistant Context]]"
summary: "[[../Summaries/summary|Summary]]"
---
User note line.
""");
var workflowEngine = new TransformingAttendeeWorkflowEngine("Ada Lovelace (Contoso)", "Ada Lovelace");
var tools = new MeetingSummaryTools(
artifacts,
new MeetingAssistantOptions(),
meetingWorkflowEngine: workflowEngine);
Assert.Equal("Added attendee Ada Lovelace.", await tools.AddAttendee("Ada Lovelace (Contoso)"));
var meetingNote = await File.ReadAllTextAsync(artifacts.MeetingNotePath);
Assert.Contains("- Ada Lovelace", meetingNote);
Assert.DoesNotContain("Ada Lovelace (Contoso)", meetingNote);
Assert.Equal(["Ada Lovelace (Contoso)"], workflowEngine.AttendeeRequests);
}
[Fact]
public async Task ToolsOverrideSpeakerInTranscriptAndRecordOverride()
{
@@ -614,22 +478,6 @@ public sealed class MeetingSummaryToolTests
Assert.Equal("", await tools.ReadTranscript(from: 3, to: 2));
}
[Fact]
public async Task ReadToolsSupportTail()
{
var root = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N"));
var artifacts = new MeetingSessionArtifacts(
MeetingNotePath: Path.Combine(root, "Meetings", "Notes", "meeting.md"),
TranscriptPath: Path.Combine(root, "Meetings", "Transcripts", "transcript.md"),
AssistantContextPath: Path.Combine(root, "Meetings", "Assistant Context", "context.md"),
SummaryPath: Path.Combine(root, "Meetings", "Summaries", "summary.md"));
Directory.CreateDirectory(Path.GetDirectoryName(artifacts.TranscriptPath)!);
await File.WriteAllTextAsync(artifacts.TranscriptPath, "one\ntwo\nthree");
var tools = new MeetingSummaryTools(artifacts);
Assert.Equal("two\nthree", await tools.ReadTranscript(tail: 2));
}
[Fact]
public async Task ToolsWriteAssistantContextBodyWithLineModes()
{
@@ -837,5 +685,4 @@ public sealed class MeetingSummaryToolTests
Assert.False(tools.SummaryWasWritten);
Assert.False(File.Exists(artifacts.SummaryPath));
}
}
@@ -1,6 +1,5 @@
using MeetingAssistant.MeetingNotes;
using MeetingAssistant.Workflow;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
@@ -188,87 +187,6 @@ public sealed class MeetingWorkflowEngineTests
Assert.Contains("Speaker Ada was identified in Architecture Sync.", context);
}
[Fact]
public async Task TranscriptLineRuleCanReplaceMaskedProfanityBeforeWriting()
{
var fixture = await WorkflowFixture.CreateAsync(MeetingWorkflowTestRules.MaskedProfanityRedactionYaml);
var line = await fixture.Engine.TransformTranscriptLineAsync(
MeetingWorkflowEvent.TranscriptLine(
fixture.Artifacts,
"[00:00:04] Guest-1: Azure returned ***** here.",
"Guest-1"),
fixture.Options,
CancellationToken.None);
Assert.Equal("[00:00:04] Guest-1: Azure returned [redacted] here.", line);
}
[Fact]
public async Task TranscriptLineRulePreservesUtf8CharactersWhenRenderingRazor()
{
var fixture = await WorkflowFixture.CreateAsync(MeetingWorkflowTestRules.MaskedProfanityRedactionYaml);
var line = await fixture.Engine.TransformTranscriptLineAsync(
MeetingWorkflowEvent.TranscriptLine(
fixture.Artifacts,
"[00:00:57] Guest-1: Das heißt, Schimpfwörter wie ***** müssen als nächstes richtig bleiben.",
"Guest-1"),
fixture.Options,
CancellationToken.None);
Assert.Equal(
"[00:00:57] Guest-1: Das heißt, Schimpfwörter wie [redacted] müssen als nächstes richtig bleiben.",
line);
}
[Fact]
public async Task TriggeredWorkflowRuleLogsStartAndCompletion()
{
var logger = new ListLogger<MeetingWorkflowEngine>();
var fixture = await WorkflowFixture.CreateAsync(
"""
rules:
- name: logged-rule
on:
- created: {}
steps:
- uses: add_project
value: Diagnostics
""",
logger: logger);
await RunCreatedAsync(fixture);
Assert.Contains(logger.Messages, message =>
message.Contains("Triggered meeting workflow rule logged-rule for event Created", StringComparison.Ordinal));
Assert.Contains(logger.Messages, message =>
message.Contains("Completed meeting workflow rule logged-rule for event Created", StringComparison.Ordinal));
}
[Fact]
public async Task WorkflowRuleFailureLogsRuleNameAndEventType()
{
var logger = new ListLogger<MeetingWorkflowEngine>();
var fixture = await WorkflowFixture.CreateAsync(
"""
rules:
- name: broken-rule
on:
- created: {}
steps:
- uses: set_property
property: unsupported
value: Diagnostics
""",
logger: logger);
await Assert.ThrowsAsync<InvalidOperationException>(() => RunCreatedAsync(fixture));
Assert.Contains(logger.Messages, message =>
message.Contains("Meeting workflow rule broken-rule failed for event Created", StringComparison.Ordinal));
}
[Fact]
public async Task RuleCanRemoveAttendeeWhenNestedConditionMatches()
{
@@ -310,10 +228,6 @@ public sealed class MeetingWorkflowEngineTests
[InlineData("speaker_identified:\n name: ADA", "speaker", null, null, "ada", true)]
[InlineData("speaker_identified:\n name: Ada", "speaker", null, null, "Grace", false)]
[InlineData("speaker_identified: {}", "speaker", null, null, "Grace", true)]
[InlineData("attendee_added:\n equals: Ada Lovelace", "attendee", null, null, "ada lovelace", true)]
[InlineData("attendee_added:\n contains: contoso", "attendee", null, null, "Ada (Contoso)", true)]
[InlineData("attendee_added:\n regex: '^Ada .+Contoso\\)$'", "attendee", null, null, "Ada (Contoso)", true)]
[InlineData("attendee_added:\n regex: '^Grace'", "attendee", null, null, "Ada (Contoso)", false)]
public async Task TriggerMatchingScenarios(
string triggerYaml,
string eventKind,
@@ -322,16 +236,6 @@ public sealed class MeetingWorkflowEngineTests
string? speaker,
bool shouldRun)
{
var stepYaml = eventKind == "attendee"
? """
- uses: set_property
property: attendee.name
value: Triggered
"""
: """
- uses: add_project
value: Triggered
""";
var fixture = await WorkflowFixture.CreateAsync(
$$"""
rules:
@@ -339,21 +243,14 @@ public sealed class MeetingWorkflowEngineTests
on:
- {{triggerYaml}}
steps:
{{stepYaml}}
- uses: add_project
value: Triggered
""");
var workflowEvent = CreateEvent(eventKind, fixture.Artifacts, from, to, speaker);
if (eventKind == "attendee")
{
var transformed = await fixture.Engine.TransformAttendeeAsync(
workflowEvent,
fixture.Options,
CancellationToken.None);
Assert.Equal(shouldRun ? "Triggered" : speaker, transformed);
return;
}
await fixture.Engine.RunAsync(workflowEvent, fixture.Options, CancellationToken.None);
await fixture.Engine.RunAsync(
CreateEvent(eventKind, fixture.Artifacts, from, to, speaker),
fixture.Options,
CancellationToken.None);
var meeting = await fixture.ReadMeetingAsync();
Assert.Equal(shouldRun, meeting.Frontmatter.Projects.Contains("Triggered"));
@@ -624,34 +521,6 @@ public sealed class MeetingWorkflowEngineTests
Assert.Equal(["Ada Lovelace"], meeting.Frontmatter.Attendees);
}
[Fact]
public async Task AttendeeAddedRuleCanTransformWorkflowAddedAttendee()
{
var fixture = await WorkflowFixture.CreateAsync(
"""
rules:
- name: add-raw-attendee
on:
- created: {}
steps:
- uses: add_attendee
value: 'Ada Lovelace (Contoso)'
- name: clean-contoso-attendee
on:
- attendee_added:
contains: 'Contoso'
steps:
- uses: set_property
property: attendee.name
value: '@Model.Attendee.Name.Replace(" (Contoso)", "")'
""");
await RunCreatedAsync(fixture);
var meeting = await fixture.ReadMeetingAsync();
Assert.Equal(["Ada Lovelace"], meeting.Frontmatter.Attendees);
}
[Fact]
public async Task AddAttendeeDoesNotDuplicateExistingDisplayNameFromEmailAddress()
{
@@ -841,7 +710,6 @@ public sealed class MeetingWorkflowEngineTests
{
"created" => MeetingWorkflowEvent.Created(artifacts),
"speaker" => MeetingWorkflowEvent.SpeakerIdentified(artifacts, speaker ?? "Ada"),
"attendee" => MeetingWorkflowEvent.AttendeeAdded(artifacts, speaker ?? "Ada"),
"state" => MeetingWorkflowEvent.StateTransition(
artifacts,
ParseState(from ?? "collecting metadata"),
@@ -891,8 +759,7 @@ public sealed class MeetingWorkflowEngineTests
string title = "Planning Sync",
IReadOnlyList<string>? attendees = null,
IReadOnlyList<string>? projects = null,
string? rulesPath = null,
ILogger<MeetingWorkflowEngine>? logger = null)
string? rulesPath = null)
{
var root = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N"));
Directory.CreateDirectory(root);
@@ -954,7 +821,7 @@ public sealed class MeetingWorkflowEngineTests
NullLogger<FileMeetingWorkflowRulesProvider>.Instance),
noteStore,
artifactStore,
logger ?? NullLogger<MeetingWorkflowEngine>.Instance);
NullLogger<MeetingWorkflowEngine>.Instance);
return new WorkflowFixture(options, noteStore, artifacts, engine);
}
@@ -968,39 +835,4 @@ public sealed class MeetingWorkflowEngineTests
return File.ReadAllTextAsync(Artifacts.AssistantContextPath);
}
}
private sealed class ListLogger<T> : ILogger<T>
{
public List<string> Messages { get; } = [];
public IDisposable BeginScope<TState>(TState state)
where TState : notnull
{
return NullScope.Instance;
}
public bool IsEnabled(LogLevel logLevel)
{
return true;
}
public void Log<TState>(
LogLevel logLevel,
EventId eventId,
TState state,
Exception? exception,
Func<TState, Exception?, string> formatter)
{
Messages.Add(formatter(state, exception));
}
private sealed class NullScope : IDisposable
{
public static NullScope Instance { get; } = new();
public void Dispose()
{
}
}
}
}
@@ -1,18 +0,0 @@
namespace MeetingAssistant.Tests;
internal static class MeetingWorkflowTestRules
{
public const string MaskedProfanityRedactionYaml =
"""
rules:
- name: redact-masked-profanity
on:
- transcript_line: {}
if:
- condition: contains(transcript.line, '*****')
steps:
- uses: set_property
property: transcript.line
value: '@Model.Transcript.Line.Replace("*****", "[redacted]")'
""";
}
@@ -1,54 +0,0 @@
using MeetingAssistant.Recording;
namespace MeetingAssistant.Tests;
public sealed class MicrophoneSelectionTests
{
[Fact]
public void BlankConfiguredMicrophoneUsesDefaultDevice()
{
var selection = new MicrophoneDeviceSelection();
var selected = selection.Resolve(
configuredDeviceId: null,
new MicrophoneDevice("default-id", "default microphone"),
[
new MicrophoneDevice("default-id", "default microphone"),
new MicrophoneDevice("other-id", "other microphone")
]);
Assert.Equal("default-id", selected?.Id);
}
[Fact]
public void ConfiguredMicrophoneUsesMatchingDevice()
{
var selection = new MicrophoneDeviceSelection();
var selected = selection.Resolve(
"other-id",
new MicrophoneDevice("default-id", "default microphone"),
[
new MicrophoneDevice("default-id", "default microphone"),
new MicrophoneDevice("other-id", "other microphone")
]);
Assert.Equal("other-id", selected?.Id);
}
[Fact]
public void RuntimeMicrophoneSelectionOverridesConfiguredDevice()
{
var selection = new MicrophoneDeviceSelection();
selection.Select("runtime-id");
var selected = selection.Resolve(
"configured-id",
new MicrophoneDevice("default-id", "default microphone"),
[
new MicrophoneDevice("configured-id", "configured microphone"),
new MicrophoneDevice("runtime-id", "runtime microphone")
]);
Assert.Equal("runtime-id", selected?.Id);
}
}
@@ -1,28 +0,0 @@
using MeetingAssistant.Recording;
namespace MeetingAssistant.Tests;
public sealed class NotificationActivationArgumentsTests
{
[Fact]
public void ParseAcceptsSemicolonDelimitedToastActivationArguments()
{
var arguments = NotificationActivationArguments.Parse(
"source=meeting-inactivity-safeguard;promptId=abc123;response=stop");
Assert.Equal("meeting-inactivity-safeguard", arguments["source"]);
Assert.Equal("abc123", arguments["promptId"]);
Assert.Equal("stop", arguments["response"]);
}
[Fact]
public void ParseAcceptsAmpersandDelimitedActivationArguments()
{
var arguments = NotificationActivationArguments.Parse(
"source=meeting-inactivity-safeguard&promptId=abc123&response=continue");
Assert.Equal("meeting-inactivity-safeguard", arguments["source"]);
Assert.Equal("abc123", arguments["promptId"]);
Assert.Equal("continue", arguments["response"]);
}
}
@@ -1,26 +0,0 @@
using MeetingAssistant.Notifications;
namespace MeetingAssistant.Tests;
public sealed class NotificationExpirationPolicyTests
{
[Fact]
public void MeetingStopReminderExpiresAfterOneMinute()
{
var now = new DateTimeOffset(2026, 6, 5, 9, 30, 0, TimeSpan.Zero);
var expiration = MeetingToastExpirationPolicy.StopReminderExpiration(now);
Assert.Equal(now.AddMinutes(1), expiration);
}
[Fact]
public void MeetingStartPromptExpiresAfterFiveMinutes()
{
var now = new DateTimeOffset(2026, 6, 5, 9, 30, 0, TimeSpan.Zero);
var expiration = MeetingToastExpirationPolicy.StartRecordingPromptExpiration(now);
Assert.Equal(now.AddMinutes(5), expiration);
}
}
@@ -8,7 +8,7 @@ public sealed class OutlookClassicMeetingMetadataProviderTests
[Fact]
public void ExtractAgendaStopsBeforeTeamsJoinInformation()
{
var agenda = OutlookClassicAppointmentMetadata.ExtractAgenda(
var agenda = OutlookClassicMeetingMetadataProvider.ExtractAgenda(
"""
Review current prototype
Decide next backend
@@ -25,7 +25,7 @@ public sealed class OutlookClassicMeetingMetadataProviderTests
[Fact]
public void NormalizeAttendeesDeduplicatesOrganizerAndRecipientEmail()
{
var attendees = OutlookClassicAppointmentMetadata.NormalizeAttendees([
var attendees = OutlookClassicMeetingMetadataProvider.NormalizeAttendees([
"Marcus.Altmann@sew-eurodrive.de",
"Marcus.Altmann@sew-eurodrive.de <Marcus.Altmann@sew-eurodrive.de>",
"Schweigert, Manuel",
@@ -37,14 +37,5 @@ public sealed class OutlookClassicMeetingMetadataProviderTests
"Schweigert, Manuel"
], attendees);
}
[Theory]
[InlineData("Canceled: Project sync")]
[InlineData("Cancelled: Project sync")]
[InlineData("Abgesagt: Project sync")]
public void SubjectIndicatesCancellationRecognizesOutlookCanceledPrefixes(string subject)
{
Assert.True(OutlookClassicCom.SubjectIndicatesCancellation(subject));
}
}
#endif
@@ -63,34 +63,6 @@ public sealed class ProjectKnowledgeToolTests
await tools.Search("alpha"));
}
[Fact]
public async Task ListProjectsAcceptsScalarProjectFrontmatter()
{
var root = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N"));
var projectsRoot = Path.Combine(root, "Projects");
Directory.CreateDirectory(Path.Combine(projectsRoot, "MeetingAssistant"));
var artifacts = CreateArtifacts(root);
Directory.CreateDirectory(Path.GetDirectoryName(artifacts.MeetingNotePath)!);
await File.WriteAllTextAsync(
artifacts.MeetingNotePath,
"""
---
projects: MeetingAssistant
---
""");
var tools = new MeetingSummaryTools(
artifacts,
new MeetingAssistantOptions
{
Vault =
{
ProjectsFolder = projectsRoot
}
});
Assert.Equal("MeetingAssistant", await tools.ListProjects());
}
[Fact]
public async Task WriteProjectFileSupportsAppendReplaceInsertOverwriteAndCreate()
{
@@ -149,41 +121,6 @@ public sealed class ProjectKnowledgeToolTests
Assert.StartsWith("Refused:", await tools.WriteProjectFile("MeetingAssistant", "notes.md", "content", from: 1, to: 1, replace_file: true));
}
[Fact]
public async Task SearchCanBeLimitedByProjectFilePattern()
{
var root = Path.Combine(Path.GetTempPath(), "meeting-assistant-tests", Guid.NewGuid().ToString("N"));
var projectsRoot = Path.Combine(root, "Projects");
var projectRoot = Path.Combine(projectsRoot, "MeetingAssistant");
Directory.CreateDirectory(projectRoot);
await File.WriteAllTextAsync(Path.Combine(projectRoot, "PROJECT.md"), "durable needle");
await File.WriteAllTextAsync(Path.Combine(projectRoot, "scratch.txt"), "scratch needle");
var artifacts = CreateArtifacts(root);
Directory.CreateDirectory(Path.GetDirectoryName(artifacts.MeetingNotePath)!);
await File.WriteAllTextAsync(
artifacts.MeetingNotePath,
"""
---
projects:
- MeetingAssistant
---
""");
var tools = new MeetingSummaryTools(
artifacts,
new MeetingAssistantOptions
{
Vault =
{
ProjectsFolder = projectsRoot
}
});
var search = await tools.Search("needle", file_pattern: "*.md");
Assert.Contains("PROJECT.md:1 durable needle", search);
Assert.DoesNotContain("scratch.txt", search);
}
private static MeetingSessionArtifacts CreateArtifacts(string root)
{
return new MeetingSessionArtifacts(
File diff suppressed because it is too large Load Diff
@@ -9,6 +9,5 @@ public sealed class ScreenshotOcrOptionsTests
{
Assert.Contains("exactly who is in the meeting", ScreenshotOcrOptions.DefaultPrompt);
Assert.Contains("partial", ScreenshotOcrOptions.DefaultPrompt);
Assert.Contains("\"attendees\"", ScreenshotOcrOptions.DefaultPrompt);
}
}
@@ -1,6 +1,5 @@
using MeetingAssistant.Recording;
using MeetingAssistant.Transcription;
using Microsoft.Extensions.Logging;
using NAudio.Wave;
namespace MeetingAssistant.Tests;
@@ -48,27 +47,6 @@ public sealed class SpeakerAudioSampleCollectorTests
Assert.DoesNotContain(collector.Snapshot(), sample => sample.Speaker == "Guest01");
}
[Fact]
public void CollectorLogsWhenSampleIsDiscardedBecauseSpeechIsTooShort()
{
var logger = new CapturingLogger();
var collector = new SpeakerAudioSampleCollector(
TimeSpan.FromMinutes(2),
maxSamplesPerSpeaker: 3,
minimumUninterruptedSpeechDuration: TimeSpan.FromSeconds(30),
maximumSegmentGap: TimeSpan.FromSeconds(1),
logger: logger);
collector.AppendAudio(CreateAudio(TimeSpan.FromSeconds(35)));
collector.TryAdd(Segment(0, 12, "Guest01", "one two three four five."));
var message = Assert.Single(
logger.Messages,
message => message.Contains("Discarding speaker identity sample for Guest01", StringComparison.Ordinal));
Assert.Contains("duration", message);
Assert.Contains("minimum duration", message);
}
private static TranscriptionSegment Segment(
double start,
double end,
@@ -95,30 +73,4 @@ public sealed class SpeakerAudioSampleCollectorTests
using var reader = new WaveFileReader(new MemoryStream(wavBytes));
return reader.TotalTime;
}
private sealed class CapturingLogger : ILogger
{
public List<string> Messages { get; } = [];
public IDisposable? BeginScope<TState>(TState state)
where TState : notnull
{
return null;
}
public bool IsEnabled(LogLevel logLevel)
{
return true;
}
public void Log<TState>(
LogLevel logLevel,
EventId eventId,
TState state,
Exception? exception,
Func<TState, Exception?, string> formatter)
{
Messages.Add(formatter(state, exception));
}
}
}
@@ -3,7 +3,6 @@ using MeetingAssistant.MeetingNotes;
using MeetingAssistant.Speakers;
using MeetingAssistant.Transcription;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
@@ -150,28 +149,6 @@ public sealed class SpeakerIdentityServiceTests
});
}
[Fact]
public async Task RejectedUnmatchedSpeakerSampleIsLoggedWithSpeakerAndCandidateContext()
{
await using var fixture = await SpeakerIdentityFixture.CreateAsync();
fixture.MatchValidator.SampleIsValid = false;
var logger = new CapturingLogger<SpeakerIdentityService>();
var service = fixture.CreateService(logger);
await service.ProcessFinishedTranscriptAsync(
fixture.CreateRequest(
["John", "Mike"],
[new TranscriptionSegment(TimeSpan.Zero, TimeSpan.FromSeconds(4), "Guest01", "unknown speaker")]),
CancellationToken.None);
var message = Assert.Single(
logger.Messages,
message => message.Contains("Skipping speaker identity candidate for Guest01", StringComparison.Ordinal));
Assert.Contains("Skipping speaker identity candidate for Guest01", message);
Assert.Contains("candidate names John, Mike", message);
Assert.Contains("sample bytes 3", message);
}
[Fact]
public async Task UnmatchedSpeakersExcludeAliasesOfMatchedIdentitiesFromCandidates()
{
@@ -350,64 +327,6 @@ public sealed class SpeakerIdentityServiceTests
Assert.Contains(saved.Snippets, snippet => snippet.WavBytes.SequenceEqual(new byte[] { 8, 8, 8 }));
}
[Fact]
public async Task SpeakerOverrideExtractsOnlyBestContinuousSourceSpanWhenNoLiveSampleExists()
{
await using var fixture = await SpeakerIdentityFixture.CreateAsync();
var service = fixture.CreateService();
var request = fixture.CreateRequest(
["Sabrina"],
[
new TranscriptionSegment(TimeSpan.Zero, TimeSpan.FromSeconds(10), "Guest-01", "short start"),
new TranscriptionSegment(TimeSpan.FromSeconds(10), TimeSpan.FromSeconds(20), "Guest-02", "someone else"),
new TranscriptionSegment(TimeSpan.FromSeconds(60), TimeSpan.FromSeconds(80), "Guest-01", "continuous first part"),
new TranscriptionSegment(TimeSpan.FromSeconds(80.5), TimeSpan.FromSeconds(92), "Guest-01", "continuous second part"),
new TranscriptionSegment(TimeSpan.FromSeconds(180), TimeSpan.FromSeconds(190), "Guest-01", "short end")
]);
await service.ApplySpeakerOverrideAsync(request, "Guest-01", "Sabrina", CancellationToken.None);
Assert.Equal(1, fixture.SnippetExtractor.ExtractCalls);
Assert.Collection(
fixture.SnippetExtractor.LastSpeakerSegments,
segment => Assert.Equal(TimeSpan.FromSeconds(60), segment.Start),
segment => Assert.Equal(TimeSpan.FromSeconds(80.5), segment.Start));
}
[Fact]
public async Task SpeakerOverrideDoesNotCreateNamedIdentityWhenSecondaryValidationRejectsSample()
{
await using var fixture = await SpeakerIdentityFixture.CreateAsync();
fixture.MatchValidator.SampleIsValid = false;
var service = fixture.CreateService();
var request = fixture.CreateRequest(
["Sabrina"],
[
new TranscriptionSegment(TimeSpan.Zero, TimeSpan.FromSeconds(31), "Guest-01", "long enough source sample")
]);
await service.ApplySpeakerOverrideAsync(request, "Guest-01", "Sabrina", CancellationToken.None);
Assert.Equal(1, fixture.MatchValidator.ValidateSampleCalls);
Assert.Empty(await fixture.Context.SpeakerIdentities.ToListAsync());
}
[Fact]
public async Task SpeakerOverrideDoesNotCreateNamedIdentityWithoutSourceSample()
{
await using var fixture = await SpeakerIdentityFixture.CreateAsync();
var service = fixture.CreateService();
var segment = new TranscriptionSegment(TimeSpan.Zero, TimeSpan.FromSeconds(5), "Guest-01", "hello from Sabrina");
var request = fixture.CreateRequest(
["Sabrina"],
[segment],
[new SpeakerAudioSample("Guest-01", segment, [8, 8, 8], 95)]);
await service.ApplySpeakerOverrideAsync(request, "Guest-5", "Sabrina", CancellationToken.None);
Assert.Empty(await fixture.Context.SpeakerIdentities.ToListAsync());
}
[Fact]
public async Task SpeakerIdentityDeletionRemovesAcceptedNameFromDatabase()
{
@@ -713,14 +632,14 @@ public sealed class SpeakerIdentityServiceTests
return new SpeakerIdentityFixture(tempDirectory, dbPath, context, options);
}
public SpeakerIdentityService CreateService(ILogger<SpeakerIdentityService>? logger = null)
public SpeakerIdentityService CreateService()
{
return new SpeakerIdentityService(
new TestSpeakerIdentityDbContextFactory(dbPath),
SnippetExtractor,
Matcher,
Options.Create(new MeetingAssistantOptions { SpeakerIdentification = options }),
logger ?? NullLogger<SpeakerIdentityService>.Instance,
NullLogger<SpeakerIdentityService>.Instance,
MatchValidator);
}
@@ -842,8 +761,6 @@ public sealed class SpeakerIdentityServiceTests
{
public int ExtractCalls { get; private set; }
public IReadOnlyList<TranscriptionSegment> LastSpeakerSegments { get; private set; } = [];
public Task<byte[]> ExtractSnippetAsync(
string audioPath,
IReadOnlyList<TranscriptionSegment> speakerSegments,
@@ -855,7 +772,6 @@ public sealed class SpeakerIdentityServiceTests
}
ExtractCalls++;
LastSpeakerSegments = speakerSegments.ToList();
return Task.FromResult<byte[]>([7, 8, 9]);
}
}
@@ -890,13 +806,10 @@ public sealed class SpeakerIdentityServiceTests
{
public bool SampleIsValid { get; set; } = true;
public int ValidateSampleCalls { get; private set; }
public Task<bool> ValidateSampleAsync(
byte[] wavBytes,
CancellationToken cancellationToken)
{
ValidateSampleCalls++;
return Task.FromResult(SampleIsValid);
}
@@ -907,30 +820,4 @@ public sealed class SpeakerIdentityServiceTests
return Task.FromResult(true);
}
}
private sealed class CapturingLogger<T> : ILogger<T>
{
public List<string> Messages { get; } = [];
public IDisposable? BeginScope<TState>(TState state)
where TState : notnull
{
return null;
}
public bool IsEnabled(LogLevel logLevel)
{
return true;
}
public void Log<TState>(
LogLevel logLevel,
EventId eventId,
TState state,
Exception? exception,
Func<TState, Exception?, string> formatter)
{
Messages.Add(formatter(state, exception));
}
}
}
+5 -124
View File
@@ -1,8 +1,6 @@
using MeetingAssistant.LaunchProfiles;
using MeetingAssistant.Recording;
using MeetingAssistant.Taskbar;
using System.Drawing;
using System.Runtime.Versioning;
namespace MeetingAssistant.Tests;
@@ -13,12 +11,12 @@ public sealed class TaskbarIconTests
{
var menu = MeetingTaskbarMenuBuilder.Build(
Status(),
[Profile("default", "Ctrl+Alt+M"), Profile("english", "Ctrl+Alt+L")]);
[Profile("default", "Ctrl+Alt+M"), Profile("english", "Ctrl+Alt+E")]);
Assert.Equal(RecordingProcessState.Idle, menu.State);
Assert.Contains(menu.Items, item =>
item.Action == MeetingTaskbarAction.EditRules &&
item.Text == "Open agent");
item.Text == "Edit rules and identities");
Assert.Contains(menu.Items, item =>
item.Action == MeetingTaskbarAction.StartRecording &&
item.ProfileName == "default" &&
@@ -26,45 +24,17 @@ public sealed class TaskbarIconTests
Assert.Contains(menu.Items, item =>
item.Action == MeetingTaskbarAction.StartRecording &&
item.ProfileName == "english" &&
item.Text == "Start meeting recording (english)\tCtrl+Alt+L");
item.Text == "Start meeting recording (english)\tCtrl+Alt+E");
Assert.DoesNotContain(menu.Items, item => item.Action == MeetingTaskbarAction.StopRecording);
Assert.DoesNotContain(menu.Items, item => item.Action == MeetingTaskbarAction.AbortRecording);
}
[Fact]
public void MenuOffersMicrophoneSubmenuWithEffectiveDeviceChecked()
{
var menu = MeetingTaskbarMenuBuilder.Build(
Status(),
[Profile("default")],
[
new MicrophoneDevice("integrated", "integrated microphone"),
new MicrophoneDevice("other", "other microphone")
],
"integrated");
var microphoneMenu = Assert.Single(menu.Items, item => item.Text == "Microphone");
Assert.Equal(MeetingTaskbarAction.OpenSubmenu, microphoneMenu.Action);
Assert.NotNull(microphoneMenu.Items);
Assert.Contains(microphoneMenu.Items, item =>
item.Action == MeetingTaskbarAction.SelectMicrophone &&
item.MicrophoneDeviceId == "integrated" &&
item.Text == "integrated microphone" &&
item.IsChecked);
Assert.Contains(microphoneMenu.Items, item =>
item.Action == MeetingTaskbarAction.SelectMicrophone &&
item.MicrophoneDeviceId == "other" &&
item.Text == "other microphone" &&
!item.IsChecked);
}
[Fact]
public void RecordingMenuOffersStopAbortAndOtherProfileSwitches()
{
var menu = MeetingTaskbarMenuBuilder.Build(
Status(isRecording: true, state: RecordingProcessState.Recording, profile: "default"),
[Profile("default", "Ctrl+Alt+M"), Profile("english", "Ctrl+Alt+L"), Profile("french", "Ctrl+Alt+F")]);
[Profile("default", "Ctrl+Alt+M"), Profile("english", "Ctrl+Alt+E"), Profile("french", "Ctrl+Alt+F")]);
Assert.Equal(RecordingProcessState.Recording, menu.State);
Assert.Contains(menu.Items, item => item.Action == MeetingTaskbarAction.StopRecording);
@@ -72,7 +42,7 @@ public sealed class TaskbarIconTests
Assert.Contains(menu.Items, item =>
item.Action == MeetingTaskbarAction.SwitchProfile &&
item.ProfileName == "english" &&
item.Text == "Switch to english\tCtrl+Alt+L");
item.Text == "Switch to english\tCtrl+Alt+E");
Assert.Contains(menu.Items, item =>
item.Action == MeetingTaskbarAction.SwitchProfile &&
item.ProfileName == "french" &&
@@ -111,57 +81,6 @@ public sealed class TaskbarIconTests
Assert.Equal(RecordingProcessState.Recording, menu.State);
}
[Theory]
[InlineData(RecordingProcessState.Idle, false)]
[InlineData(RecordingProcessState.Recording, true)]
[InlineData(RecordingProcessState.Summarizing, false)]
public void MenuAlwaysOffersExit(RecordingProcessState state, bool isRecording)
{
var menu = MeetingTaskbarMenuBuilder.Build(
Status(isRecording: isRecording, state: state, profile: "default"),
[Profile("default"), Profile("english")]);
Assert.Contains(menu.Items, item =>
item.Action == MeetingTaskbarAction.Exit &&
item.Text == "Exit");
}
[Theory]
[InlineData(RecordingProcessState.Idle, false)]
[InlineData(RecordingProcessState.Recording, true)]
[InlineData(RecordingProcessState.Summarizing, true)]
public void ExitRequiresConfirmationWhileRecordingOrProcessing(
RecordingProcessState state,
bool requiresConfirmation)
{
Assert.Equal(
requiresConfirmation,
MeetingTaskbarExitPolicy.RequiresConfirmation(state));
}
[Fact]
[SupportedOSPlatform("windows")]
public void TaskbarIconGlyphsAreVisuallyCentered()
{
foreach (var state in new[]
{
RecordingProcessState.Idle,
RecordingProcessState.Recording,
RecordingProcessState.Summarizing
})
{
using var bitmap = TaskbarIconRenderer.RenderBitmap(state);
var bounds = GetVisibleGlyphBounds(bitmap);
Assert.True(
IsCentered(GetCenter(bounds.Left, bounds.Right)),
$"{state} glyph is horizontally off-center: {bounds}");
Assert.True(
IsCentered(GetCenter(bounds.Top, bounds.Bottom)),
$"{state} glyph is vertically off-center: {bounds}");
}
}
private static LaunchProfile Profile(string name, string hotkey = "")
{
return new LaunchProfile(name, new MeetingAssistantOptions
@@ -187,42 +106,4 @@ public sealed class TaskbarIconTests
state,
profile);
}
[SupportedOSPlatform("windows")]
private static Rectangle GetVisibleGlyphBounds(Bitmap bitmap)
{
var left = bitmap.Width;
var top = bitmap.Height;
var right = -1;
var bottom = -1;
for (var y = 0; y < bitmap.Height; y++)
{
for (var x = 0; x < bitmap.Width; x++)
{
var pixel = bitmap.GetPixel(x, y);
if (pixel.A < 32 || pixel.GetBrightness() < 0.58f)
{
continue;
}
left = Math.Min(left, x);
top = Math.Min(top, y);
right = Math.Max(right, x);
bottom = Math.Max(bottom, y);
}
}
Assert.True(right >= left && bottom >= top, "Expected the icon to contain a visible white glyph.");
return Rectangle.FromLTRB(left, top, right + 1, bottom + 1);
}
private static double GetCenter(int start, int end)
{
return (start + end) / 2.0;
}
private static bool IsCentered(double center)
{
return center is >= 7.25 and <= 8.75;
}
}
@@ -1,44 +0,0 @@
using MeetingAssistant.Workflow;
namespace MeetingAssistant.Tests;
internal sealed class TransformingAttendeeWorkflowEngine : IMeetingWorkflowEngine
{
private readonly string source;
private readonly string replacement;
public TransformingAttendeeWorkflowEngine(string source, string replacement)
{
this.source = source;
this.replacement = replacement;
}
public List<string> AttendeeRequests { get; } = [];
public Task RunAsync(
MeetingWorkflowEvent workflowEvent,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
public Task<string> TransformTranscriptLineAsync(
MeetingWorkflowEvent workflowEvent,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
return Task.FromResult(workflowEvent.TranscriptLineText ?? "");
}
public Task<string> TransformAttendeeAsync(
MeetingWorkflowEvent workflowEvent,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
AttendeeRequests.Add(workflowEvent.AttendeeName ?? "");
return Task.FromResult(string.Equals(workflowEvent.AttendeeName, source, StringComparison.OrdinalIgnoreCase)
? replacement
: workflowEvent.AttendeeName ?? "");
}
}
File diff suppressed because it is too large Load Diff
-279
View File
@@ -1,279 +0,0 @@
using System.Text.RegularExpressions;
namespace MeetingAssistant;
internal static class AgentFileToolContent
{
public static string ReadLines(string content, int? from = null, int? to = null, int? tail = null)
{
if (!from.HasValue && !to.HasValue)
{
return tail.HasValue
? TailLines(content, Math.Clamp(tail.Value, 1, 2000))
: content;
}
var lines = SplitLines(content);
if (lines.Count == 0)
{
return "";
}
var start = Math.Clamp((from ?? 1) - 1, 0, lines.Count - 1);
var end = Math.Clamp((to ?? lines.Count) - 1, 0, lines.Count - 1);
if (end < start)
{
return "";
}
return string.Join('\n', lines.GetRange(start, end - start + 1));
}
public static string TailLines(string content, int count)
{
var lines = SplitLines(content);
if (lines.Count > 0 && lines[^1].Length == 0)
{
lines.RemoveAt(lines.Count - 1);
}
if (lines.Count <= count)
{
return string.Join('\n', lines);
}
return string.Join('\n', lines.GetRange(lines.Count - count, count));
}
public static string ApplyLineEdit(
string existingContent,
string replacementContent,
AgentFileEditMode editMode)
{
if (editMode.Kind == AgentFileEditKind.Overwrite)
{
return replacementContent;
}
if (editMode.Kind == AgentFileEditKind.Append)
{
return AppendContent(existingContent, replacementContent);
}
var lines = SplitLines(existingContent);
var replacementLines = SplitLines(replacementContent);
if (editMode.Kind == AgentFileEditKind.Insert)
{
var insertIndex = Math.Clamp(editMode.Insert!.Value - 1, 0, lines.Count);
lines.InsertRange(insertIndex, replacementLines);
return string.Join('\n', lines);
}
var startIndex = Math.Clamp(editMode.From!.Value - 1, 0, lines.Count);
var endIndexExclusive = Math.Clamp(editMode.To!.Value, 0, lines.Count);
if (endIndexExclusive < startIndex)
{
endIndexExclusive = startIndex;
}
lines.RemoveRange(startIndex, endIndexExclusive - startIndex);
lines.InsertRange(startIndex, replacementLines);
return string.Join('\n', lines);
}
public static async Task WriteFileContentAsync(
string filePath,
string content,
AgentFileEditMode editMode)
{
if (editMode.Kind == AgentFileEditKind.Overwrite)
{
await File.WriteAllTextAsync(filePath, content);
return;
}
if (editMode.Kind == AgentFileEditKind.Append)
{
var existing = File.Exists(filePath)
? await File.ReadAllTextAsync(filePath)
: "";
await File.WriteAllTextAsync(filePath, AppendContent(existing, content));
return;
}
var existingContent = File.Exists(filePath)
? await File.ReadAllTextAsync(filePath)
: "";
await File.WriteAllTextAsync(filePath, ApplyLineEdit(existingContent, content, editMode));
}
public static string AppendContent(string existingContent, string content)
{
if (string.IsNullOrEmpty(existingContent))
{
return content;
}
if (string.IsNullOrEmpty(content))
{
return existingContent;
}
var separator = existingContent.EndsWith('\n') || existingContent.EndsWith('\r')
? ""
: "\n";
return existingContent + separator + content;
}
public static List<string> SplitLines(string content)
{
if (content.Length == 0)
{
return [];
}
return content
.Replace("\r\n", "\n", StringComparison.Ordinal)
.Replace('\r', '\n')
.Split('\n')
.ToList();
}
public static bool IsWithinDirectory(string directory, string path)
{
var normalizedDirectory = Path.GetFullPath(directory)
.TrimEnd(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar) + Path.DirectorySeparatorChar;
var normalizedPath = Path.GetFullPath(path);
return normalizedPath.StartsWith(normalizedDirectory, StringComparison.OrdinalIgnoreCase);
}
public static bool MatchesGlob(string path, string? pattern)
{
if (string.IsNullOrWhiteSpace(pattern))
{
return true;
}
var normalizedPath = ToToolPath(path);
var normalizedPattern = ToToolPath(pattern.Trim());
var placeholder = "\u0000";
var regexPattern = Regex.Escape(normalizedPattern)
.Replace("\\*\\*", placeholder, StringComparison.Ordinal)
.Replace("\\*", "[^/]*", StringComparison.Ordinal)
.Replace("\\?", "[^/]", StringComparison.Ordinal)
.Replace(placeholder, ".*", StringComparison.Ordinal);
return Regex.IsMatch(normalizedPath, $"^{regexPattern}$", RegexOptions.IgnoreCase);
}
public static string ToToolPath(string path)
{
return path.Replace(Path.DirectorySeparatorChar.ToString(), "/", StringComparison.Ordinal)
.Replace(Path.AltDirectorySeparatorChar.ToString(), "/", StringComparison.Ordinal);
}
public static string SearchFiles(
string keywords,
IEnumerable<AgentFileSearchSource> sources,
int maxMatches = 100,
RegexOptions regexOptions = RegexOptions.IgnoreCase)
{
if (string.IsNullOrWhiteSpace(keywords))
{
return "";
}
try
{
var regex = new Regex(keywords, regexOptions);
var matches = new List<string>();
var limit = Math.Clamp(maxMatches, 1, 1000);
foreach (var source in sources)
{
var lines = ReadAllLinesShared(source.Path);
for (var index = 0; index < lines.Length; index++)
{
if (regex.IsMatch(lines[index]))
{
matches.Add($"{ToToolPath(source.DisplayPath)}:{index + 1} {lines[index]}");
}
if (matches.Count >= limit)
{
return string.Join('\n', matches);
}
}
}
return string.Join('\n', matches);
}
catch
{
return "";
}
}
public static async Task<string> ReadAllTextSharedAsync(string path)
{
await using var stream = new FileStream(
path,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite | FileShare.Delete,
bufferSize: 4096,
useAsync: true);
using var reader = new StreamReader(stream);
return await reader.ReadToEndAsync();
}
private static string[] ReadAllLinesShared(string path)
{
using var stream = new FileStream(
path,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite | FileShare.Delete);
using var reader = new StreamReader(stream);
return SplitLines(reader.ReadToEnd()).ToArray();
}
}
internal sealed record AgentFileSearchSource(string Path, string DisplayPath);
internal sealed record AgentFileEditMode(
AgentFileEditKind Kind,
int? From = null,
int? To = null,
int? Insert = null)
{
public static AgentFileEditMode? Create(int? from, int? to, int? insert, bool replaceFile)
{
if (replaceFile && (from.HasValue || to.HasValue || insert.HasValue))
{
return null;
}
if (insert.HasValue)
{
return from.HasValue || to.HasValue
? null
: new AgentFileEditMode(AgentFileEditKind.Insert, Insert: insert);
}
if (from.HasValue || to.HasValue)
{
return from.HasValue && to.HasValue && from.Value <= to.Value
? new AgentFileEditMode(AgentFileEditKind.Replace, From: from, To: to)
: null;
}
return new AgentFileEditMode(replaceFile ? AgentFileEditKind.Overwrite : AgentFileEditKind.Append);
}
}
internal enum AgentFileEditKind
{
Append,
Overwrite,
Replace,
Insert
}
@@ -1,326 +0,0 @@
using MeetingAssistant.Recording;
using MeetingAssistant.MeetingNotes;
using Microsoft.Extensions.Options;
namespace MeetingAssistant.Calendar;
public sealed class CalendarRecordingPromptScheduler : BackgroundService
{
private readonly IOptions<MeetingAssistantOptions> options;
private readonly ICalendarMeetingProvider meetingProvider;
private readonly IMeetingStartPromptService promptService;
private readonly IMeetingPromptRecordingController recordingController;
private readonly ICalendarPromptClock clock;
private readonly ILogger<CalendarRecordingPromptScheduler> logger;
private readonly HashSet<string> promptedMeetingKeys = new(StringComparer.OrdinalIgnoreCase);
private IReadOnlyList<CalendarMeeting> cachedMeetings = [];
private DateOnly promptedMeetingDay;
private DateOnly cachedMeetingDay;
private DateTimeOffset nextSyncAt;
public CalendarRecordingPromptScheduler(
IOptions<MeetingAssistantOptions> options,
ICalendarMeetingProvider meetingProvider,
IMeetingStartPromptService promptService,
IMeetingPromptRecordingController recordingController,
ICalendarPromptClock clock,
ILogger<CalendarRecordingPromptScheduler> logger)
{
this.options = options;
this.meetingProvider = meetingProvider;
this.promptService = promptService;
this.recordingController = recordingController;
this.clock = clock;
this.logger = logger;
}
public async Task SyncOnceAsync(CancellationToken cancellationToken)
{
var promptOptions = options.Value.CalendarRecordingPrompts;
if (!promptOptions.Enabled)
{
return;
}
var now = clock.Now;
var today = DateOnly.FromDateTime(now.LocalDateTime);
ResetPromptedMeetingsIfDayChanged(today);
try
{
cachedMeetings = await meetingProvider.GetRecordingPromptCandidatesForDayAsync(today, cancellationToken);
cachedMeetingDay = today;
nextSyncAt = now.Add(GetSyncInterval(promptOptions));
logger.LogInformation(
"Synced {MeetingCount} Outlook Teams calendar meetings for {Day}; next sync at {NextSyncAt}",
cachedMeetings.Count,
today,
nextSyncAt);
}
catch (Exception exception) when (exception is not OperationCanceledException)
{
logger.LogWarning(exception, "Could not read today's Outlook Teams meetings for recording prompts");
nextSyncAt = now.Add(GetSyncInterval(promptOptions));
return;
}
}
public async Task CheckDuePromptsAsync(CancellationToken cancellationToken)
{
var promptOptions = options.Value.CalendarRecordingPrompts;
if (!promptOptions.Enabled)
{
return;
}
var now = clock.Now;
var today = DateOnly.FromDateTime(now.LocalDateTime);
ResetPromptedMeetingsIfDayChanged(today);
if (cachedMeetingDay != today)
{
cachedMeetings = [];
return;
}
foreach (var meeting in cachedMeetings.OrderBy(meeting => meeting.Start))
{
var promptKey = GetPromptKey(meeting);
if (meeting.IsCanceled ||
promptedMeetingKeys.Contains(promptKey) ||
!IsInPromptWindow(meeting, now, promptOptions))
{
continue;
}
promptedMeetingKeys.Add(promptKey);
logger.LogInformation(
"Prompting to record calendar meeting {Subject} scheduled at {Start}",
meeting.Subject,
meeting.Start);
await promptService.ShowPromptAsync(
new MeetingStartPromptRequest(meeting),
(response, token) => HandlePromptResponseAsync(meeting, response, token),
cancellationToken);
}
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
var promptOptions = options.Value.CalendarRecordingPrompts;
if (!promptOptions.Enabled)
{
await Task.Delay(GetSyncInterval(promptOptions), stoppingToken);
continue;
}
var now = clock.Now;
var today = DateOnly.FromDateTime(now.LocalDateTime);
if (cachedMeetingDay != today || now >= nextSyncAt)
{
await SyncOnceAsync(stoppingToken);
}
await CheckDuePromptsAsync(stoppingToken);
var delay = GetDelayUntilNextWork(clock.Now);
if (delay <= TimeSpan.Zero)
{
continue;
}
await Task.Delay(delay, stoppingToken);
}
}
private async Task HandlePromptResponseAsync(
CalendarMeeting meeting,
MeetingStartPromptResponse response,
CancellationToken cancellationToken)
{
if (response != MeetingStartPromptResponse.Record)
{
return;
}
if (recordingController.CurrentStatus.IsRecording)
{
await recordingController.StopAsync(cancellationToken);
}
await recordingController.StartFromPromptAsync(meeting.Metadata, cancellationToken);
}
private void ResetPromptedMeetingsIfDayChanged(DateOnly today)
{
if (promptedMeetingDay == today)
{
return;
}
promptedMeetingDay = today;
promptedMeetingKeys.Clear();
}
private static bool IsInPromptWindow(
CalendarMeeting meeting,
DateTimeOffset now,
CalendarRecordingPromptOptions options)
{
var promptWindow = options.PromptWindow > TimeSpan.Zero
? options.PromptWindow
: TimeSpan.FromMinutes(5);
return now >= meeting.Start &&
now <= meeting.Start.Add(promptWindow) &&
now < meeting.End;
}
private TimeSpan GetDelayUntilNextWork(DateTimeOffset now)
{
if (nextSyncAt <= DateTimeOffset.MinValue || now >= nextSyncAt)
{
return TimeSpan.Zero;
}
var syncDelay = nextSyncAt - now;
var nextMeetingStart = cachedMeetings
.Where(meeting => !meeting.IsCanceled)
.Where(meeting => !promptedMeetingKeys.Contains(GetPromptKey(meeting)))
.Where(meeting => meeting.End > now)
.Select(meeting => meeting.Start <= now ? now : meeting.Start)
.Order()
.FirstOrDefault();
if (nextMeetingStart == default)
{
return syncDelay;
}
var meetingDelay = nextMeetingStart - now;
return meetingDelay <= TimeSpan.Zero
? TimeSpan.Zero
: meetingDelay < syncDelay ? meetingDelay : syncDelay;
}
private static TimeSpan GetSyncInterval(CalendarRecordingPromptOptions options)
{
return options.SyncInterval > TimeSpan.Zero
? options.SyncInterval
: TimeSpan.FromMinutes(30);
}
private static string GetPromptKey(CalendarMeeting meeting)
{
if (!string.IsNullOrWhiteSpace(meeting.Id))
{
return meeting.Id;
}
return $"{meeting.Start:O}|{meeting.End:O}|{meeting.Subject}";
}
}
public interface ICalendarMeetingProvider
{
Task<IReadOnlyList<CalendarMeeting>> GetRecordingPromptCandidatesForDayAsync(
DateOnly day,
CancellationToken cancellationToken);
}
public sealed record CalendarMeeting(
string Id,
string Subject,
DateTimeOffset Start,
DateTimeOffset End,
MeetingMetadata? Metadata = null,
bool IsCanceled = false);
public interface IMeetingStartPromptService
{
Task ShowPromptAsync(
MeetingStartPromptRequest request,
Func<MeetingStartPromptResponse, CancellationToken, Task> handleResponseAsync,
CancellationToken cancellationToken);
}
public sealed record MeetingStartPromptRequest(CalendarMeeting Meeting);
public enum MeetingStartPromptResponse
{
Record,
Skip
}
public interface IMeetingPromptRecordingController
{
RecordingStatus CurrentStatus { get; }
Task<RecordingStatus> StartAsync(CancellationToken cancellationToken);
Task<RecordingStatus> StartFromPromptAsync(
MeetingMetadata? metadata,
CancellationToken cancellationToken);
Task<RecordingStatus> StopAsync(CancellationToken cancellationToken);
}
public sealed class MeetingPromptRecordingController : IMeetingPromptRecordingController
{
private readonly MeetingRecordingCoordinator coordinator;
public MeetingPromptRecordingController(MeetingRecordingCoordinator coordinator)
{
this.coordinator = coordinator;
}
public RecordingStatus CurrentStatus => coordinator.CurrentStatus;
public Task<RecordingStatus> StartAsync(CancellationToken cancellationToken)
{
return coordinator.StartAsync(cancellationToken);
}
public Task<RecordingStatus> StartFromPromptAsync(
MeetingMetadata? metadata,
CancellationToken cancellationToken)
{
return coordinator.StartFromPromptAsync(metadata, cancellationToken);
}
public Task<RecordingStatus> StopAsync(CancellationToken cancellationToken)
{
return coordinator.StopAsync(cancellationToken);
}
}
public interface ICalendarPromptClock
{
DateTimeOffset Now { get; }
}
public sealed class SystemCalendarPromptClock : ICalendarPromptClock
{
public DateTimeOffset Now => DateTimeOffset.Now;
}
public sealed class NoopCalendarMeetingProvider : ICalendarMeetingProvider
{
public Task<IReadOnlyList<CalendarMeeting>> GetRecordingPromptCandidatesForDayAsync(
DateOnly day,
CancellationToken cancellationToken)
{
return Task.FromResult<IReadOnlyList<CalendarMeeting>>([]);
}
}
public sealed class NoopMeetingStartPromptService : IMeetingStartPromptService
{
public Task ShowPromptAsync(
MeetingStartPromptRequest request,
Func<MeetingStartPromptResponse, CancellationToken, Task> handleResponseAsync,
CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
}
@@ -1,105 +0,0 @@
#if WINDOWS
using MeetingAssistant.MeetingNotes;
namespace MeetingAssistant.Calendar;
public sealed class OutlookClassicCalendarMeetingProvider : ICalendarMeetingProvider
{
private readonly ILogger<OutlookClassicCalendarMeetingProvider> logger;
public OutlookClassicCalendarMeetingProvider(ILogger<OutlookClassicCalendarMeetingProvider> logger)
{
this.logger = logger;
}
public Task<IReadOnlyList<CalendarMeeting>> GetRecordingPromptCandidatesForDayAsync(
DateOnly day,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
try
{
return OutlookClassicCom.RunStaAsync(
() => GetTeamsMeetingsForDay(day),
cancellationToken,
"Meeting Assistant Outlook Calendar Prompt Lookup");
}
catch (Exception exception) when (exception is not OperationCanceledException)
{
logger.LogDebug(exception, "Outlook Classic calendar meetings were not available");
return Task.FromResult<IReadOnlyList<CalendarMeeting>>([]);
}
}
private IReadOnlyList<CalendarMeeting> GetTeamsMeetingsForDay(DateOnly day)
{
using var calendar = OutlookClassicCom.OpenDefaultCalendar(logger, "calendar prompt lookup");
if (calendar is null)
{
return [];
}
var dayStart = day.ToDateTime(TimeOnly.MinValue);
var dayEnd = dayStart.AddDays(1);
var meetings = new List<CalendarMeeting>();
foreach (var item in OutlookClassicCom.EnumerateItems(calendar.Items))
{
try
{
if (item is null || !OutlookClassicCom.IsAppointment(item))
{
continue;
}
var start = Convert.ToDateTime(OutlookClassicCom.GetProperty(item, "Start"));
if (start >= dayEnd)
{
break;
}
var end = Convert.ToDateTime(OutlookClassicCom.GetProperty(item, "End"));
if (end <= dayStart ||
start < dayStart ||
OutlookClassicCom.IsCanceledAppointment(item) ||
!OutlookClassicAppointmentMetadata.IsTeamsAppointment(item))
{
continue;
}
var subject = OutlookClassicAppointmentMetadata.ReadSubject(item);
var localEnd = OutlookClassicCom.ToLocalOffset(end);
meetings.Add(new CalendarMeeting(
GetAppointmentId(item, start, end, subject),
subject,
OutlookClassicCom.ToLocalOffset(start),
localEnd,
OutlookClassicAppointmentMetadata.CreateMetadata(item, end),
IsCanceled: false));
}
finally
{
OutlookClassicCom.ReleaseComObject(item);
}
}
return meetings;
}
private static string GetAppointmentId(
object appointment,
DateTime start,
DateTime end,
string subject)
{
var id = Convert.ToString(OutlookClassicCom.TryGetProperty(appointment, "GlobalAppointmentID"));
if (string.IsNullOrWhiteSpace(id))
{
id = Convert.ToString(OutlookClassicCom.TryGetProperty(appointment, "EntryID"));
}
return string.IsNullOrWhiteSpace(id)
? $"{start:O}|{end:O}|{subject}"
: $"{id}|{start:O}";
}
}
#endif
@@ -1,22 +0,0 @@
namespace MeetingAssistant.Calendar;
internal static class TeamsMeetingMarkerDetector
{
public static bool IsTeamsAppointment(
string subject,
string location,
string body)
{
return ContainsTeamsMarker(subject) ||
ContainsTeamsMarker(location) ||
ContainsTeamsMarker(body);
}
public static bool ContainsTeamsMarker(string value)
{
return value.Contains("teams.microsoft.com", StringComparison.OrdinalIgnoreCase) ||
value.Contains("Microsoft Teams", StringComparison.OrdinalIgnoreCase) ||
value.Contains("Join the meeting now", StringComparison.OrdinalIgnoreCase) ||
value.Contains("Join Microsoft Teams Meeting", StringComparison.OrdinalIgnoreCase);
}
}
@@ -1,207 +0,0 @@
#if WINDOWS
using System.Collections.Concurrent;
using CommunityToolkit.WinUI.Notifications;
using MeetingAssistant.Notifications;
using MeetingAssistant.Recording;
namespace MeetingAssistant.Calendar;
public sealed class WindowsMeetingStartPromptService : IMeetingStartPromptService, IHostedService, IDisposable
{
private const string NotificationSource = "meeting-start-prompt";
private const string NotificationGroup = "meeting-start";
private readonly ConcurrentDictionary<string, Func<MeetingStartPromptResponse, CancellationToken, Task>> pendingPrompts = new(StringComparer.OrdinalIgnoreCase);
private readonly ILogger<WindowsMeetingStartPromptService> logger;
private readonly object registrationGate = new();
private bool registered;
public WindowsMeetingStartPromptService(ILogger<WindowsMeetingStartPromptService> logger)
{
this.logger = logger;
}
public Task ShowPromptAsync(
MeetingStartPromptRequest request,
Func<MeetingStartPromptResponse, CancellationToken, Task> handleResponseAsync,
CancellationToken cancellationToken)
{
if (!OperatingSystem.IsWindowsVersionAtLeast(10, 0, 17763))
{
logger.LogWarning("Windows app notifications require Windows 10 version 1809 or later");
return Task.CompletedTask;
}
if (cancellationToken.IsCancellationRequested)
{
return Task.FromCanceled(cancellationToken);
}
try
{
EnsureRegistered();
var promptId = Guid.NewGuid().ToString("N");
pendingPrompts[promptId] = handleResponseAsync;
BuildNotification(promptId, request).Show(toast =>
{
toast.Group = NotificationGroup;
toast.Tag = promptId;
toast.ExpirationTime = MeetingToastExpirationPolicy.StartRecordingPromptExpiration(DateTimeOffset.Now);
});
logger.LogInformation(
"Displayed native Windows meeting-start notification {PromptId} for {Subject}",
promptId,
request.Meeting.Subject);
}
catch (Exception exception)
{
logger.LogError(exception, "Failed to display native Windows meeting-start notification");
}
return Task.CompletedTask;
}
public Task StartAsync(CancellationToken cancellationToken)
{
if (!OperatingSystem.IsWindowsVersionAtLeast(10, 0, 17763))
{
logger.LogWarning("Windows app notifications require Windows 10 version 1809 or later");
return Task.CompletedTask;
}
EnsureRegistered();
return Task.CompletedTask;
}
public Task StopAsync(CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
public void Dispose()
{
if (!registered)
{
return;
}
try
{
ToastNotificationManagerCompat.OnActivated -= OnNotificationInvoked;
}
catch (Exception exception)
{
logger.LogWarning(exception, "Failed to detach native Windows meeting-start notification activation handler");
}
}
private void EnsureRegistered()
{
if (registered)
{
return;
}
lock (registrationGate)
{
if (registered)
{
return;
}
ToastNotificationManagerCompat.OnActivated += OnNotificationInvoked;
registered = true;
logger.LogInformation("Registered native Windows meeting-start notification activation handler");
}
}
private static ToastContentBuilder BuildNotification(
string promptId,
MeetingStartPromptRequest request)
{
var yesButton = new ToastButton()
.SetContent("Yes")
.AddArgument("source", NotificationSource)
.AddArgument("promptId", promptId)
.AddArgument("response", "record")
.SetBackgroundActivation();
var noButton = new ToastButton()
.SetContent("No")
.AddArgument("source", NotificationSource)
.AddArgument("promptId", promptId)
.AddArgument("response", "skip")
.SetBackgroundActivation();
return new ToastContentBuilder()
.AddArgument("source", NotificationSource)
.AddArgument("promptId", promptId)
.SetToastScenario(ToastScenario.Reminder)
.SetToastDuration(ToastDuration.Long)
.AddText("Record meeting?")
.AddText($"{request.Meeting.Subject} starts at {request.Meeting.Start.LocalDateTime:t}.")
.AddButton(yesButton)
.AddButton(noButton);
}
private void OnNotificationInvoked(ToastNotificationActivatedEventArgsCompat args)
{
var arguments = NotificationActivationArguments.Parse(args.Argument);
if (!TryGetArgument(arguments, "source", out var source) ||
!string.Equals(source, NotificationSource, StringComparison.OrdinalIgnoreCase))
{
return;
}
if (!TryGetArgument(arguments, "promptId", out var promptId))
{
logger.LogWarning("Ignoring native Windows meeting-start notification activation without prompt id");
return;
}
if (!pendingPrompts.TryRemove(promptId, out var handleResponseAsync))
{
logger.LogWarning(
"Ignoring native Windows meeting-start notification {PromptId} activation because no pending callback exists",
promptId);
return;
}
var response = TryGetArgument(arguments, "response", out var responseValue) &&
string.Equals(responseValue, "record", StringComparison.OrdinalIgnoreCase)
? MeetingStartPromptResponse.Record
: MeetingStartPromptResponse.Skip;
_ = Task.Run(async () =>
{
try
{
logger.LogInformation(
"Native Windows meeting-start notification {PromptId} activated with response {Response}",
promptId,
response);
await handleResponseAsync(response, CancellationToken.None);
}
catch (Exception exception)
{
logger.LogError(
exception,
"Failed to handle native Windows meeting-start notification response {Response}",
response);
}
});
}
private static bool TryGetArgument(
IReadOnlyDictionary<string, string> arguments,
string key,
out string value)
{
if (arguments.TryGetValue(key, out value!))
{
return true;
}
value = "";
return false;
}
}
#endif
@@ -1,74 +0,0 @@
The project is split in 3 important files:
PROJECT.md
DECISIONS.md
JOURNAL.md
The idea is a hierarchical summary:
Meeting Summary -> detailed (what was discussed)
then Journal Entry -> compressed (why should I care this event happened?)
then PROJECT.md -> distilled (what do I absolutely have to know about this project EVERY TIME I work on it. This is an onboarding document that every agent should read.)
The journal should always be appended and read using tail or searched.
Example journal entry:
```markdown
## 2026-05-30
### Team Daily
New Blocker: HLS delayed
Impact: Release likely delayed, team implements Z-Levels in the meantime.
[[20260529-093030-6187486-summary|Summary]]
### Refinement
Calendar-Feature is refined, Golem feature needs more work.
[[20260529-103030-3465634-summary|Summary]]
```
PROJECT.md should use this structure:
```markdown
# Executive Summary
<Elevator Pitch>
## Business Goals
## Priorities and High Level Constraints
## Current Phase & Status
## Next Milestones
## Open Risks
## Important Stakeholders
## Key Documents
Links to: Architecture Overview, Technical Onboarding, etc.
[[JOURNAL.md]]
[[DECISIONS.md]]
```
DECISIONS.md logs decisions in the same compressed way that JOURNAL.md logs meetings, with links to ADRs and/or meeting summaries for details.
Only include important decisions: architecture decisions that are expensive to change, and project decisions about timeline, goals, governance, team, or process.
Do not log small implementation details such as how UI elements are aligned.
Example decision entries:
```markdown
## How do we handle project delays?
Communicate to <Stakeholder> first, let them decide further actions.
[[20260529-093030-6187486-summary|Source]]
## How will authentication be handled?
Company SSO with provided UI package.
[[ADR-0027-use-sso-and-ui-package-for-auth|Source]]
```
-2
View File
@@ -1,2 +0,0 @@
global using System.IO;
global using System.Net.Http;
@@ -1,192 +0,0 @@
using System.Collections.Concurrent;
using System.Text;
namespace MeetingAssistant.Logging;
public static class MeetingAssistantLogFiles
{
public const string CurrentLogFileName = "meeting-assistant.log";
public const int RetainedRotatedLogFiles = 4;
public static string DefaultLogDirectory =>
Path.Combine(Path.GetTempPath(), "MeetingAssistant", "Logs");
public static string CurrentLogPath(string? directory = null)
{
return Path.Combine(directory ?? DefaultLogDirectory, CurrentLogFileName);
}
public static string GetRotatedLogFileName(int index)
{
return $"{CurrentLogFileName}.{index}";
}
public static IReadOnlyList<string> EnumerateLogPaths(string? directory = null)
{
var root = directory ?? DefaultLogDirectory;
return Enumerable.Range(0, RetainedRotatedLogFiles + 1)
.Select(index => index == 0
? Path.Combine(root, CurrentLogFileName)
: Path.Combine(root, GetRotatedLogFileName(index)))
.Where(File.Exists)
.ToArray();
}
public static void Rotate(string? directory = null)
{
var root = directory ?? DefaultLogDirectory;
Directory.CreateDirectory(root);
try
{
var oldest = Path.Combine(root, GetRotatedLogFileName(RetainedRotatedLogFiles));
if (File.Exists(oldest))
{
File.Delete(oldest);
}
for (var index = RetainedRotatedLogFiles - 1; index >= 1; index--)
{
var source = Path.Combine(root, GetRotatedLogFileName(index));
if (!File.Exists(source))
{
continue;
}
var target = Path.Combine(root, GetRotatedLogFileName(index + 1));
File.Move(source, target, overwrite: true);
}
var current = Path.Combine(root, CurrentLogFileName);
if (File.Exists(current))
{
File.Move(current, Path.Combine(root, GetRotatedLogFileName(1)), overwrite: true);
}
}
catch (IOException)
{
// Another app instance or test host may still have the log open.
// Logging should keep working, so append to the existing current log.
}
}
}
public sealed class MeetingAssistantFileLoggerProvider : ILoggerProvider
{
private readonly ConcurrentDictionary<string, MeetingAssistantFileLogger> loggers = new(StringComparer.Ordinal);
private readonly object sync = new();
private readonly StreamWriter writer;
private bool disposed;
public MeetingAssistantFileLoggerProvider(string? directory = null)
{
var logDirectory = directory ?? MeetingAssistantLogFiles.DefaultLogDirectory;
MeetingAssistantLogFiles.Rotate(logDirectory);
var stream = new FileStream(
MeetingAssistantLogFiles.CurrentLogPath(logDirectory),
FileMode.Append,
FileAccess.Write,
FileShare.ReadWrite);
writer = new StreamWriter(stream, new UTF8Encoding(encoderShouldEmitUTF8Identifier: false))
{
AutoFlush = true
};
}
public ILogger CreateLogger(string categoryName)
{
return loggers.GetOrAdd(categoryName, name => new MeetingAssistantFileLogger(name, Write));
}
public void Dispose()
{
lock (sync)
{
if (disposed)
{
return;
}
disposed = true;
writer.Dispose();
}
}
private void Write(
string categoryName,
LogLevel logLevel,
EventId eventId,
string message,
Exception? exception)
{
if (disposed)
{
return;
}
lock (sync)
{
if (disposed)
{
return;
}
writer.Write(DateTimeOffset.Now.ToString("O"));
writer.Write(' ');
writer.Write(logLevel);
writer.Write(" [");
writer.Write(categoryName);
writer.Write(']');
if (eventId.Id != 0 || !string.IsNullOrWhiteSpace(eventId.Name))
{
writer.Write(" (");
writer.Write(eventId.Id);
if (!string.IsNullOrWhiteSpace(eventId.Name))
{
writer.Write(':');
writer.Write(eventId.Name);
}
writer.Write(')');
}
writer.Write(' ');
writer.WriteLine(message);
if (exception is not null)
{
writer.WriteLine(exception);
}
}
}
private sealed class MeetingAssistantFileLogger(
string categoryName,
Action<string, LogLevel, EventId, string, Exception?> write) : ILogger
{
public IDisposable? BeginScope<TState>(TState state)
where TState : notnull
{
return null;
}
public bool IsEnabled(LogLevel logLevel)
{
return logLevel != LogLevel.None;
}
public void Log<TState>(
LogLevel logLevel,
EventId eventId,
TState state,
Exception? exception,
Func<TState, Exception?, string> formatter)
{
if (!IsEnabled(logLevel))
{
return;
}
write(categoryName, logLevel, eventId, formatter(state, exception), exception);
}
}
}
+11 -16
View File
@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFrameworks>net10.0;net10.0-windows10.0.19041.0</TargetFrameworks>
<TargetFrameworks>net10.0;net10.0-windows</TargetFrameworks>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<PreserveCompilationContext>true</PreserveCompilationContext>
@@ -12,7 +12,6 @@
<OutputType>WinExe</OutputType>
<ApplicationIcon>Assets\meeting-assistant.ico</ApplicationIcon>
<PlatformTarget>x64</PlatformTarget>
<UseWPF>true</UseWPF>
</PropertyGroup>
<ItemGroup>
@@ -20,33 +19,29 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.OpenAI" Version="1.13.0" />
<PackageReference Include="Microsoft.Agents.AI.OpenAI" Version="1.8.0" />
<PackageReference Include="DiffPlex" Version="1.9.0" />
<PackageReference Include="Microsoft.CognitiveServices.Speech" Version="$(MicrosoftSpeechVersion)" />
<PackageReference Include="Microsoft.CognitiveServices.Speech.Extension.MAS" Version="$(MicrosoftSpeechVersion)" ExcludeAssets="build" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="10.0.10" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="10.0.8" />
<PackageReference Include="NAudio" Version="2.3.0" />
<PackageReference Include="NCalcSync" Version="6.4.0" />
<PackageReference Include="NCalcSync" Version="5.12.0" />
<PackageReference Include="RazorLight" Version="2.3.1" />
<PackageReference Include="SQLitePCLRaw.bundle_e_sqlite3" Version="3.0.5" />
<PackageReference Include="System.Drawing.Common" Version="10.0.10" />
<PackageReference Include="Whisper.net" Version="1.9.1" />
<PackageReference Include="Whisper.net.Runtime" Version="1.9.1" />
<PackageReference Include="YamlDotNet" Version="18.1.0" />
<PackageReference Include="System.Drawing.Common" Version="10.0.8" />
<PackageReference Include="Whisper.net" Version="1.9.0" />
<PackageReference Include="Whisper.net.Runtime" Version="1.9.0" />
<PackageReference Include="YamlDotNet" Version="18.0.0" />
</ItemGroup>
<ItemGroup Condition="$([MSBuild]::GetTargetPlatformIdentifier('$(TargetFramework)')) == 'windows'">
<PackageReference Include="CommunityToolkit.WinUI.Notifications" Version="7.1.2" />
<PackageReference Include="Aprillz.MewUI.Windows" Version="0.15.2" />
<PackageReference Include="H.NotifyIcon.Uno.WinUI" Version="2.4.1" />
</ItemGroup>
<ItemGroup>
<Content Update="appsettings*.json" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" TargetPath="%(Filename)%(Extension)" />
<None Update="Assets\meeting-assistant.ico" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
<None Include="..\docs\meeting-workflow-engine.md" Link="docs\meeting-workflow-engine.md" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
<None Include="..\docs\meeting-assistant-configuration.md" Link="docs\meeting-assistant-configuration.md" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
<None Include="Content\Project-AGENTS.md" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
<None Include="..\openspec\specs\**\*.md" Link="openspec\specs\%(RecursiveDir)%(Filename)%(Extension)" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
<None Include="..\docs\meeting-workflow-engine.md" Link="docs\meeting-workflow-engine.md" CopyToOutputDirectory="PreserveNewest" />
</ItemGroup>
<ItemGroup Condition="$([MSBuild]::GetTargetPlatformIdentifier('$(TargetFramework)')) == 'windows'">
@@ -68,7 +63,7 @@
<Compile Remove="MeetingNotes\OutlookClassicMeetingMetadataProvider.Windows.cs" />
<Compile Remove="Screenshots\ActiveWindowScreenshotCapture.Windows.cs" />
<Compile Remove="Taskbar\UnoTaskbarIconService.Windows.cs" />
<Compile Remove="Workflow\WpfWorkflowRulesEditorWindowService.Windows.cs" />
<Compile Remove="Workflow\MewUiWorkflowRulesEditorWindowService.Windows.cs" />
</ItemGroup>
</Project>
+2 -50
View File
@@ -18,8 +18,6 @@ public sealed class MeetingAssistantOptions
public AutomationOptions Automation { get; set; } = new();
public CalendarRecordingPromptOptions CalendarRecordingPrompts { get; set; } = new();
public ScreenshotOptions Screenshots { get; set; } = new();
public AgentOptions Agent { get; set; } = new();
@@ -34,15 +32,6 @@ public sealed class AutomationOptions
public string? RulesPath { get; set; } = "meeting-rules.local.yaml";
}
public sealed class CalendarRecordingPromptOptions
{
public bool Enabled { get; set; } = true;
public TimeSpan SyncInterval { get; set; } = TimeSpan.FromMinutes(30);
public TimeSpan PromptWindow { get; set; } = TimeSpan.FromMinutes(5);
}
public sealed class ScreenshotOptions
{
public string Hotkey { get; set; } = "Ctrl+Alt+S";
@@ -59,7 +48,6 @@ public sealed class ScreenshotOcrOptions
Identify who appears to be talking, who appears to be presenting, and what is being presented when visible.
If people or participant tiles are visible, state whether it is clear from the screenshot exactly who is in the meeting or whether the visible people are only a partial participant result.
Include any attendee names that are clearly visible or strongly deduced from the screenshot in the meeting-assistant metadata. The attendee list may be partial.
If the screenshot contains slides, capture the visible text in clean markdown.
If the screenshot contains a diagram, convert it to Mermaid when possible and preserve labels accurately.
If it contains another kind of shared screen or scene, describe the scene and the relevant visible information.
@@ -68,9 +56,9 @@ public sealed class ScreenshotOcrOptions
Do not return crop coordinates for a person, webcam tile, whole app chrome, or content you cannot isolate confidently.
End your response with exactly one fenced json block containing meeting-assistant metadata, for example:
```json
{ "crop": { "x": 0, "y": 0, "width": 1920, "height": 1080 }, "attendees": ["Ada Lovelace"] }
{ "crop": { "x": 0, "y": 0, "width": 1920, "height": 1080 } }
```
Use `{ "crop": null, "attendees": [] }` when no confident presentation/shared-screen crop exists and no attendees are visible or deducible.
Use `{ "crop": null }` when no confident presentation/shared-screen crop exists.
Do not invent information that is not visible in the screenshot.
""";
@@ -126,16 +114,12 @@ public sealed class RecordingOptions
public int Channels { get; set; } = 1;
public string? MicrophoneDeviceId { get; set; }
public double MicrophoneMixGain { get; set; } = 1;
public double SystemAudioMixGain { get; set; } = 1;
public TimeSpan StopProcessingTimeout { get; set; } = TimeSpan.FromMinutes(10);
public TimeSpan MinimumCompletedMeetingDuration { get; set; } = TimeSpan.Zero;
public TimeSpan MetadataLookupTimeout { get; set; } = TimeSpan.FromSeconds(10);
public TimeSpan BackgroundMetadataLookupTimeout { get; set; } = TimeSpan.FromMinutes(1);
@@ -144,30 +128,7 @@ public sealed class RecordingOptions
public TimeSpan OpenMeetingNoteDelay { get; set; } = TimeSpan.FromSeconds(1);
public TimeSpan DictationWordsReadTimeout { get; set; } = TimeSpan.FromSeconds(2);
public string TemporaryRecordingsFolder { get; set; } = @"%LOCALAPPDATA%\MeetingAssistant\Recordings";
public RecordingInactivitySafeguardOptions InactivitySafeguard { get; set; } = new();
}
public sealed class RecordingInactivitySafeguardOptions
{
public bool Enabled { get; set; } = true;
public TimeSpan FirstPromptAfter { get; set; } = TimeSpan.FromMinutes(2);
public TimeSpan[] ReminderPromptAfter { get; set; } =
[
TimeSpan.FromMinutes(5),
TimeSpan.FromMinutes(10)
];
public TimeSpan AutoStopAfter { get; set; } = TimeSpan.FromMinutes(30);
public TimeSpan InferredEndPadding { get; set; } = TimeSpan.FromMinutes(1);
public TimeSpan CheckInterval { get; set; } = TimeSpan.FromSeconds(15);
}
public sealed class WhisperLocalOptions
@@ -226,10 +187,6 @@ public sealed class AzureSpeechOptions
public TimeSpan RecognitionStopTimeout { get; set; } = TimeSpan.FromMinutes(3);
public int ReconnectAttemptsBeforeDisconnectedMarker { get; set; } = 5;
public TimeSpan ReconnectDelay { get; set; } = TimeSpan.FromSeconds(5);
public bool DiarizeIntermediateResults { get; set; } = true;
public string? PostProcessingOption { get; set; }
@@ -383,8 +340,6 @@ public sealed class AgentOptions
public string Model { get; set; } = "chatgpt/gpt-5.5";
public bool UseStreaming { get; set; } = true;
public bool EnableThinking { get; set; } = true;
public ReasoningEffortOption ReasoningEffort { get; set; } = ReasoningEffortOption.Medium;
@@ -416,8 +371,6 @@ public sealed class WorkflowRulesEditorOptions
public string? Model { get; set; }
public bool? UseStreaming { get; set; }
public bool? EnableThinking { get; set; }
public ReasoningEffortOption? ReasoningEffort { get; set; }
@@ -446,7 +399,6 @@ public sealed class WorkflowRulesEditorOptions
Key = string.IsNullOrWhiteSpace(Key) ? defaults.Key : Key,
KeyEnv = string.IsNullOrWhiteSpace(KeyEnv) ? defaults.KeyEnv : KeyEnv!,
Model = string.IsNullOrWhiteSpace(Model) ? defaults.Model : Model!,
UseStreaming = UseStreaming ?? defaults.UseStreaming,
EnableThinking = EnableThinking ?? defaults.EnableThinking,
ReasoningEffort = ReasoningEffort ?? defaults.ReasoningEffort,
ReconnectionAttempts = ReconnectionAttempts ?? defaults.ReconnectionAttempts,
@@ -1,24 +0,0 @@
using YamlDotNet.Serialization;
using YamlDotNet.Serialization.NamingConventions;
namespace MeetingAssistant.MeetingNotes;
internal static class FrontmatterYamlDeserializer
{
public static IDeserializer Create()
{
return new DeserializerBuilder()
.WithTypeConverter(new ScalarStringListYamlTypeConverter())
.IgnoreUnmatchedProperties()
.Build();
}
public static IDeserializer CreateWithUnderscoredNaming()
{
return new DeserializerBuilder()
.WithNamingConvention(UnderscoredNamingConvention.Instance)
.WithTypeConverter(new ScalarStringListYamlTypeConverter())
.IgnoreUnmatchedProperties()
.Build();
}
}
@@ -1,6 +1,4 @@
using Microsoft.Extensions.Options;
using System.Globalization;
using System.Text;
using YamlDotNet.Core;
using YamlDotNet.Serialization;
@@ -18,7 +16,9 @@ public sealed class MarkdownMeetingNoteStore : IMeetingNoteStore
{
this.options = options.Value;
this.logger = logger;
yamlDeserializer = FrontmatterYamlDeserializer.Create();
yamlDeserializer = new DeserializerBuilder()
.IgnoreUnmatchedProperties()
.Build();
}
public async Task<MeetingNote> SaveAsync(MeetingNote note, CancellationToken cancellationToken)
@@ -35,7 +35,7 @@ public sealed class MarkdownMeetingNoteStore : IMeetingNoteStore
Directory.CreateDirectory(folder);
var path = string.IsNullOrWhiteSpace(note.Path)
? Path.Combine(folder, MeetingArtifactFileNames.Create(note.Frontmatter.StartTime ?? DateTimeOffset.Now, MeetingArtifactFileNames.Note))
? Path.Combine(folder, $"{DateTimeOffset.Now:yyyyMMdd-HHmmss-fffffff}-{Slugify(note.Frontmatter.Title)}.md")
: note.Path;
var frontmatter = PrepareFrontmatter(note.Frontmatter, path);
var content = Render(frontmatter, note.UserNotes);
@@ -108,6 +108,16 @@ public sealed class MarkdownMeetingNoteStore : IMeetingNoteStore
return string.Join(Environment.NewLine, lines);
}
private static string Slugify(string value)
{
var slug = new string(value
.ToLowerInvariant()
.Select(character => char.IsLetterOrDigit(character) ? character : '-')
.ToArray());
slug = string.Join("-", slug.Split('-', StringSplitOptions.RemoveEmptyEntries));
return string.IsNullOrWhiteSpace(slug) ? "meeting" : slug;
}
private static string EscapeScalar(string value)
{
return string.IsNullOrWhiteSpace(value) ? "\"\"" : EscapeListItem(value);
@@ -115,24 +125,7 @@ public sealed class MarkdownMeetingNoteStore : IMeetingNoteStore
private static string EscapeQuoted(string value)
{
var escaped = new StringBuilder(value.Length + 2);
escaped.Append('"');
foreach (var character in value)
{
escaped.Append(character switch
{
'\\' => "\\\\",
'"' => "\\\"",
'\r' => "\\r",
'\n' => "\\n",
'\t' => "\\t",
< ' ' => "\\x" + ((int)character).ToString("X2", CultureInfo.InvariantCulture),
_ => character.ToString()
});
}
escaped.Append('"');
return escaped.ToString();
return $"\"{value.Replace("\"", "\\\"", StringComparison.Ordinal)}\"";
}
private static string EscapeNullableDateTime(DateTimeOffset? value)
@@ -154,7 +147,7 @@ public sealed class MarkdownMeetingNoteStore : IMeetingNoteStore
return "\"\"";
}
if (RequiresQuotedScalar(value))
if (value.Contains(':', StringComparison.Ordinal) || value.StartsWith("[", StringComparison.Ordinal))
{
return EscapeQuoted(value);
}
@@ -162,49 +155,6 @@ public sealed class MarkdownMeetingNoteStore : IMeetingNoteStore
return value;
}
private static bool RequiresQuotedScalar(string value)
{
if (char.IsWhiteSpace(value[0]) || char.IsWhiteSpace(value[^1]))
{
return true;
}
if (value.Contains(':', StringComparison.Ordinal) ||
value.Contains('\'', StringComparison.Ordinal) ||
value.Contains(" #", StringComparison.Ordinal) ||
value.Any(char.IsControl))
{
return true;
}
if (IsYamlIndicator(value[0]))
{
return true;
}
if (IsYamlCoreSchemaKeyword(value))
{
return true;
}
return DateTimeOffset.TryParse(value, out _) ||
double.TryParse(value, CultureInfo.InvariantCulture, out _);
}
private static bool IsYamlIndicator(char character)
{
return character is '-' or '?' or ':' or ',' or '[' or ']' or '{' or '}' or
'#' or '&' or '*' or '!' or '|' or '>' or '\'' or '"' or '%' or '@' or '`';
}
private static bool IsYamlCoreSchemaKeyword(string value)
{
return string.Equals(value, "true", StringComparison.OrdinalIgnoreCase) ||
string.Equals(value, "false", StringComparison.OrdinalIgnoreCase) ||
string.Equals(value, "null", StringComparison.OrdinalIgnoreCase) ||
value == "~";
}
private sealed class MeetingNoteYaml
{
[YamlMember(Alias = "title")]
@@ -1,14 +0,0 @@
namespace MeetingAssistant.MeetingNotes;
public static class MeetingArtifactFileNames
{
public const string Note = "note";
public const string Context = "context";
public const string Transcript = "transcript";
public const string Summary = "summary";
public static string Create(DateTimeOffset startedAt, string artifactType)
{
return $"{startedAt:yyyyMMdd-HHmm}-{artifactType}.md";
}
}
@@ -8,13 +8,4 @@ internal static class MeetingAttendeeNames
var emailStart = trimmed.IndexOf('<', StringComparison.Ordinal);
return emailStart > 0 ? trimmed[..emailStart].Trim() : trimmed;
}
public static List<string> NormalizeDistinct(IEnumerable<string> attendees)
{
return attendees
.Select(attendee => attendee.Trim())
.Where(attendee => !string.IsNullOrWhiteSpace(attendee))
.Distinct(StringComparer.OrdinalIgnoreCase)
.ToList();
}
}
@@ -14,24 +14,4 @@ public static class MeetingNoteActionLinks
var url = $"{baseUrl}/meetings/summary/retry?summaryPath={Uri.EscapeDataString(summaryFileName)}";
return $"[{label}]({url})";
}
public static string CreateScreenshotOcrRetryLink(
string publicBaseUrl,
MeetingSessionArtifacts artifacts,
string screenshotPath,
string screenshotId,
string label = "Retry screenshot OCR")
{
var baseUrl = string.IsNullOrWhiteSpace(publicBaseUrl)
? "http://localhost:5090"
: publicBaseUrl.TrimEnd('/');
var url = $"{baseUrl}/meetings/screenshot-ocr/retry" +
$"?meetingNotePath={Uri.EscapeDataString(artifacts.MeetingNotePath)}" +
$"&transcriptPath={Uri.EscapeDataString(artifacts.TranscriptPath)}" +
$"&assistantContextPath={Uri.EscapeDataString(artifacts.AssistantContextPath)}" +
$"&summaryPath={Uri.EscapeDataString(artifacts.SummaryPath)}" +
$"&screenshotPath={Uri.EscapeDataString(screenshotPath)}" +
$"&screenshotId={Uri.EscapeDataString(screenshotId)}";
return $"[{label}]({url})";
}
}
@@ -1,176 +0,0 @@
#if WINDOWS
using MeetingAssistant.Calendar;
namespace MeetingAssistant.MeetingNotes;
internal static class OutlookClassicAppointmentMetadata
{
public static MeetingMetadata CreateMetadata(object appointment, DateTime end)
{
var title = ReadSubject(appointment);
return new MeetingMetadata(
title,
ReadAttendees(appointment),
ExtractAgenda(Convert.ToString(OutlookClassicCom.GetProperty(appointment, "Body")) ?? ""),
OutlookClassicCom.ToLocalOffset(end));
}
public static string ReadSubject(object appointment)
{
var subject = Convert.ToString(OutlookClassicCom.GetProperty(appointment, "Subject"))?.Trim();
return string.IsNullOrWhiteSpace(subject) ? "Teams meeting" : subject;
}
public static bool IsTeamsAppointment(object appointment)
{
var subject = Convert.ToString(OutlookClassicCom.GetProperty(appointment, "Subject")) ?? "";
var location = Convert.ToString(OutlookClassicCom.GetProperty(appointment, "Location")) ?? "";
var body = Convert.ToString(OutlookClassicCom.GetProperty(appointment, "Body")) ?? "";
return TeamsMeetingMarkerDetector.IsTeamsAppointment(subject, location, body);
}
public static string ExtractAgenda(string body)
{
if (string.IsNullOrWhiteSpace(body))
{
return "";
}
var lines = body.Replace("\r\n", "\n", StringComparison.Ordinal).Replace('\r', '\n').Split('\n');
var agendaLines = new List<string>();
foreach (var line in lines)
{
if (IsTeamsSeparator(line) || IsTeamsJoinLine(line))
{
break;
}
agendaLines.Add(line);
}
return string.Join(Environment.NewLine, agendaLines).Trim();
}
public static IReadOnlyList<string> ReadAttendees(object appointment)
{
var attendees = new List<string>();
var organizer = Convert.ToString(OutlookClassicCom.GetProperty(appointment, "Organizer"));
if (!string.IsNullOrWhiteSpace(organizer))
{
attendees.Add(organizer.Trim());
}
object? recipients = null;
try
{
recipients = OutlookClassicCom.GetProperty(appointment, "Recipients");
var count = Convert.ToInt32(OutlookClassicCom.GetProperty(recipients!, "Count"));
for (var index = 1; index <= count; index++)
{
object? recipient = null;
try
{
recipient = OutlookClassicCom.Invoke(recipients!, "Item", index);
var formatted = FormatRecipient(recipient!);
if (!string.IsNullOrWhiteSpace(formatted))
{
attendees.Add(formatted);
}
}
finally
{
OutlookClassicCom.ReleaseComObject(recipient);
}
}
}
finally
{
OutlookClassicCom.ReleaseComObject(recipients);
}
return NormalizeAttendees(attendees);
}
public static IReadOnlyList<string> NormalizeAttendees(IEnumerable<string> attendees)
{
return attendees
.Select(NormalizeAttendee)
.Where(attendee => !string.IsNullOrWhiteSpace(attendee))
.GroupBy(GetAttendeeDeduplicationKey, StringComparer.OrdinalIgnoreCase)
.Select(group => group
.OrderByDescending(attendee => attendee.Contains('<', StringComparison.Ordinal))
.ThenBy(attendee => attendee.Length)
.First())
.ToList();
}
private static bool IsTeamsSeparator(string line)
{
var trimmed = line.Trim();
return trimmed.Length >= 8 &&
trimmed.All(character => character is '_' or '-' or '*' or ' ');
}
private static bool IsTeamsJoinLine(string line)
{
return TeamsMeetingMarkerDetector.ContainsTeamsMarker(line);
}
private static string NormalizeAttendee(string attendee)
{
var normalized = attendee.Trim();
var mailtoPrefix = "mailto:";
normalized = normalized.Replace(mailtoPrefix, "", StringComparison.OrdinalIgnoreCase);
while (normalized.Contains(" ", StringComparison.Ordinal))
{
normalized = normalized.Replace(" ", " ", StringComparison.Ordinal);
}
var emailStart = normalized.IndexOf('<', StringComparison.Ordinal);
var emailEnd = normalized.LastIndexOf('>');
if (emailStart > 0 && emailEnd > emailStart)
{
var name = normalized[..emailStart].Trim();
var email = normalized[(emailStart + 1)..emailEnd].Trim();
if (name.Equals(email, StringComparison.OrdinalIgnoreCase))
{
return email;
}
return $"{name} <{email}>";
}
return normalized;
}
private static string GetAttendeeDeduplicationKey(string attendee)
{
var emailStart = attendee.IndexOf('<', StringComparison.Ordinal);
var emailEnd = attendee.LastIndexOf('>');
if (emailStart > 0 && emailEnd > emailStart)
{
return attendee[(emailStart + 1)..emailEnd].Trim();
}
return attendee.Trim();
}
private static string FormatRecipient(object recipient)
{
var name = Convert.ToString(OutlookClassicCom.GetProperty(recipient, "Name"))?.Trim() ?? "";
var email = Convert.ToString(OutlookClassicCom.GetProperty(recipient, "Address"))?.Trim() ?? "";
if (string.IsNullOrWhiteSpace(name))
{
return email;
}
if (string.IsNullOrWhiteSpace(email) ||
email.Contains('/'))
{
return name;
}
return $"{name} <{email}>";
}
}
#endif
@@ -1,248 +0,0 @@
#if WINDOWS
using System.Collections;
using System.Diagnostics;
using System.Runtime.InteropServices;
namespace MeetingAssistant.MeetingNotes;
internal static class OutlookClassicCom
{
private const int OutlookCalendarFolder = 9;
public static OutlookCalendarSession? OpenDefaultCalendar(
ILogger logger,
string logContext)
{
EnsureOutlookRunning(logger, logContext);
object? application = null;
object? session = null;
object? calendar = null;
object? items = null;
try
{
var applicationType = Type.GetTypeFromProgID("Outlook.Application");
if (applicationType is null)
{
return null;
}
application = Activator.CreateInstance(applicationType);
if (application is null)
{
return null;
}
session = GetProperty(application, "Session");
calendar = Invoke(session!, "GetDefaultFolder", OutlookCalendarFolder);
items = GetProperty(calendar!, "Items");
Invoke(items!, "Sort", "[Start]");
SetProperty(items!, "IncludeRecurrences", true);
return new OutlookCalendarSession(application, session, calendar, items!);
}
catch
{
ReleaseComObject(items);
ReleaseComObject(calendar);
ReleaseComObject(session);
ReleaseComObject(application);
throw;
}
}
public static Task<T> RunStaAsync<T>(
Func<T> action,
CancellationToken cancellationToken,
string threadName)
{
var completion = new TaskCompletionSource<T>(TaskCreationOptions.RunContinuationsAsynchronously);
var thread = new Thread(() =>
{
try
{
completion.TrySetResult(action());
}
catch (Exception exception)
{
completion.TrySetException(exception);
}
})
{
IsBackground = true,
Name = threadName
};
thread.SetApartmentState(ApartmentState.STA);
var registration = cancellationToken.Register(() => completion.TrySetCanceled(cancellationToken));
_ = completion.Task.ContinueWith(
_ => registration.Dispose(),
CancellationToken.None,
TaskContinuationOptions.ExecuteSynchronously,
TaskScheduler.Default);
thread.Start();
return completion.Task;
}
public static IEnumerable<object?> EnumerateItems(object items)
{
if (items is IEnumerable enumerable)
{
foreach (var item in enumerable)
{
yield return item;
}
}
}
public static bool IsAppointment(object item)
{
try
{
return Convert.ToInt32(GetProperty(item, "Class")) == 26;
}
catch
{
return false;
}
}
public static bool IsCanceledAppointment(object appointment)
{
var meetingStatus = TryGetProperty(appointment, "MeetingStatus");
if (meetingStatus is not null)
{
try
{
var status = Convert.ToInt32(meetingStatus);
if (status is 5 or 7)
{
return true;
}
}
catch (Exception exception) when (exception is FormatException or InvalidCastException or OverflowException)
{
// Fall back to subject prefixes when Outlook exposes a non-numeric status value.
}
}
var subject = Convert.ToString(TryGetProperty(appointment, "Subject")) ?? "";
return SubjectIndicatesCancellation(subject);
}
internal static bool SubjectIndicatesCancellation(string subject)
{
var normalized = subject.TrimStart();
return normalized.StartsWith("Canceled:", StringComparison.OrdinalIgnoreCase) ||
normalized.StartsWith("Cancelled:", StringComparison.OrdinalIgnoreCase) ||
normalized.StartsWith("Abgesagt:", StringComparison.OrdinalIgnoreCase);
}
public static DateTimeOffset ToLocalOffset(DateTime value)
{
return new DateTimeOffset(DateTime.SpecifyKind(value, DateTimeKind.Local));
}
public static object? TryGetProperty(object target, string name)
{
try
{
return GetProperty(target, name);
}
catch
{
return null;
}
}
public static object? GetProperty(object target, string name)
{
return target.GetType().InvokeMember(
name,
System.Reflection.BindingFlags.GetProperty,
null,
target,
null);
}
public static void SetProperty(object target, string name, object value)
{
target.GetType().InvokeMember(
name,
System.Reflection.BindingFlags.SetProperty,
null,
target,
[value]);
}
public static object? Invoke(object target, string name, params object[] arguments)
{
return target.GetType().InvokeMember(
name,
System.Reflection.BindingFlags.InvokeMethod,
null,
target,
arguments);
}
public static void ReleaseComObject(object? value)
{
if (value is not null && Marshal.IsComObject(value))
{
Marshal.FinalReleaseComObject(value);
}
}
private static void EnsureOutlookRunning(ILogger logger, string logContext)
{
if (Process.GetProcessesByName("OUTLOOK").Length > 0)
{
return;
}
try
{
Process.Start(new ProcessStartInfo
{
FileName = "outlook.exe",
UseShellExecute = true
});
Thread.Sleep(TimeSpan.FromSeconds(5));
}
catch (Exception exception)
{
logger.LogDebug(exception, "Could not start Outlook Classic before {OutlookContext}", logContext);
}
}
}
internal sealed class OutlookCalendarSession : IDisposable
{
private readonly object? application;
private readonly object? session;
private readonly object? calendar;
public OutlookCalendarSession(
object? application,
object? session,
object? calendar,
object items)
{
this.application = application;
this.session = session;
this.calendar = calendar;
Items = items;
}
public object Items { get; }
public void Dispose()
{
OutlookClassicCom.ReleaseComObject(Items);
OutlookClassicCom.ReleaseComObject(calendar);
OutlookClassicCom.ReleaseComObject(session);
OutlookClassicCom.ReleaseComObject(application);
}
}
#endif
@@ -1,10 +1,12 @@
using System.Runtime.InteropServices;
using System.Diagnostics;
using MeetingAssistant.Calendar;
using System.Collections;
namespace MeetingAssistant.MeetingNotes;
public sealed class OutlookClassicMeetingMetadataProvider : IMeetingMetadataProvider, IMeetingMetadataDiagnosticProvider
{
private const int OutlookCalendarFolder = 9;
private readonly ILogger<OutlookClassicMeetingMetadataProvider> logger;
public OutlookClassicMeetingMetadataProvider(ILogger<OutlookClassicMeetingMetadataProvider> logger)
@@ -19,10 +21,7 @@ public sealed class OutlookClassicMeetingMetadataProvider : IMeetingMetadataProv
cancellationToken.ThrowIfCancellationRequested();
try
{
return OutlookClassicCom.RunStaAsync(
() => GetCurrentMeeting(startedAt),
cancellationToken,
"Meeting Assistant Outlook COM Lookup");
return RunStaAsync(() => GetCurrentMeeting(startedAt), cancellationToken);
}
catch (Exception exception) when (exception is not OperationCanceledException)
{
@@ -39,10 +38,7 @@ public sealed class OutlookClassicMeetingMetadataProvider : IMeetingMetadataProv
var stopwatch = Stopwatch.StartNew();
try
{
var lookup = OutlookClassicCom.RunStaAsync(
() => GetCurrentMeeting(startedAt),
cancellationToken,
"Meeting Assistant Outlook COM Lookup");
var lookup = RunStaAsync(() => GetCurrentMeeting(startedAt), cancellationToken);
var metadata = timeout > TimeSpan.Zero
? await lookup.WaitAsync(timeout, cancellationToken)
: await lookup.WaitAsync(cancellationToken);
@@ -88,60 +84,77 @@ public sealed class OutlookClassicMeetingMetadataProvider : IMeetingMetadataProv
private MeetingMetadata? GetCurrentMeeting(DateTimeOffset startedAt)
{
using var calendar = OutlookClassicCom.OpenDefaultCalendar(logger, "metadata lookup");
if (calendar is null)
{
return null;
}
EnsureOutlookRunning();
var startedLocal = startedAt.LocalDateTime;
var windowStart = startedLocal.AddHours(-4);
var windowEnd = startedLocal.AddMinutes(5);
var candidates = new List<OutlookAppointmentCandidate>();
foreach (var item in OutlookClassicCom.EnumerateItems(calendar.Items))
{
if (item is null)
{
continue;
}
var keepAppointment = false;
try
{
if (!OutlookClassicCom.IsAppointment(item))
{
continue;
}
var start = Convert.ToDateTime(OutlookClassicCom.GetProperty(item, "Start"));
if (start > windowEnd)
{
break;
}
var end = Convert.ToDateTime(OutlookClassicCom.GetProperty(item, "End"));
if (end < windowStart ||
OutlookClassicCom.IsCanceledAppointment(item) ||
!OutlookClassicAppointmentMetadata.IsTeamsAppointment(item))
{
continue;
}
candidates.Add(new OutlookAppointmentCandidate(item, start, end));
keepAppointment = true;
}
finally
{
if (!keepAppointment)
{
OutlookClassicCom.ReleaseComObject(item);
}
}
}
object? application = null;
object? session = null;
object? calendar = null;
object? items = null;
try
{
var applicationType = Type.GetTypeFromProgID("Outlook.Application");
if (applicationType is null)
{
return null;
}
application = Activator.CreateInstance(applicationType);
if (application is null)
{
return null;
}
session = GetProperty(application, "Session");
calendar = Invoke(session!, "GetDefaultFolder", OutlookCalendarFolder);
items = GetProperty(calendar!, "Items");
Invoke(items!, "Sort", "[Start]");
SetProperty(items!, "IncludeRecurrences", true);
var startedLocal = startedAt.LocalDateTime;
var windowStart = startedLocal.AddHours(-4);
var windowEnd = startedLocal.AddMinutes(5);
var candidates = new List<OutlookAppointmentCandidate>();
foreach (var item in EnumerateItems(items!))
{
if (item is null)
{
continue;
}
var keepAppointment = false;
try
{
if (!IsAppointment(item))
{
continue;
}
var start = Convert.ToDateTime(GetProperty(item, "Start"));
if (start > windowEnd)
{
break;
}
var end = Convert.ToDateTime(GetProperty(item, "End"));
if (end < windowStart || !IsTeamsAppointment(item))
{
continue;
}
candidates.Add(new OutlookAppointmentCandidate(item, start, end));
keepAppointment = true;
}
finally
{
if (!keepAppointment)
{
ReleaseComObject(item);
}
}
}
var selected = OutlookMeetingCandidateSelector.Select(
candidates,
startedLocal,
@@ -149,20 +162,325 @@ public sealed class OutlookClassicMeetingMetadataProvider : IMeetingMetadataProv
candidate => candidate.End);
if (selected is null)
{
foreach (var candidate in candidates)
{
ReleaseComObject(candidate.Appointment);
}
return null;
}
return OutlookClassicAppointmentMetadata.CreateMetadata(selected.Appointment, selected.End);
try
{
var appointment = selected.Appointment;
var title = Convert.ToString(GetProperty(appointment, "Subject")) ?? "";
var attendees = ReadAttendees(appointment);
var body = Convert.ToString(GetProperty(appointment, "Body")) ?? "";
return new MeetingMetadata(
title.Trim(),
attendees,
ExtractAgenda(body),
ToLocalOffset(selected.End));
}
finally
{
foreach (var candidate in candidates)
{
ReleaseComObject(candidate.Appointment);
}
}
}
finally
{
foreach (var candidate in candidates)
ReleaseComObject(items);
ReleaseComObject(calendar);
ReleaseComObject(session);
ReleaseComObject(application);
}
}
private static IEnumerable<object?> EnumerateItems(object items)
{
if (items is IEnumerable enumerable)
{
foreach (var item in enumerable)
{
OutlookClassicCom.ReleaseComObject(candidate.Appointment);
yield return item;
}
}
}
private static bool IsAppointment(object item)
{
try
{
return Convert.ToInt32(GetProperty(item, "Class")) == 26;
}
catch
{
return false;
}
}
private void EnsureOutlookRunning()
{
if (Process.GetProcessesByName("OUTLOOK").Length > 0)
{
return;
}
try
{
Process.Start(new ProcessStartInfo
{
FileName = "outlook.exe",
UseShellExecute = true
});
Thread.Sleep(TimeSpan.FromSeconds(5));
}
catch (Exception exception)
{
logger.LogDebug(exception, "Could not start Outlook Classic before metadata lookup");
}
}
private static Task<T> RunStaAsync<T>(
Func<T> action,
CancellationToken cancellationToken)
{
var completion = new TaskCompletionSource<T>(TaskCreationOptions.RunContinuationsAsynchronously);
var thread = new Thread(() =>
{
try
{
completion.TrySetResult(action());
}
catch (Exception exception)
{
completion.TrySetException(exception);
}
})
{
IsBackground = true,
Name = "Meeting Assistant Outlook COM Lookup"
};
thread.SetApartmentState(ApartmentState.STA);
var registration = cancellationToken.Register(() => completion.TrySetCanceled(cancellationToken));
_ = completion.Task.ContinueWith(
_ => registration.Dispose(),
CancellationToken.None,
TaskContinuationOptions.ExecuteSynchronously,
TaskScheduler.Default);
thread.Start();
return completion.Task;
}
internal static string ExtractAgenda(string body)
{
if (string.IsNullOrWhiteSpace(body))
{
return "";
}
var lines = body.Replace("\r\n", "\n", StringComparison.Ordinal).Replace('\r', '\n').Split('\n');
var agendaLines = new List<string>();
foreach (var line in lines)
{
if (IsTeamsSeparator(line) || IsTeamsJoinLine(line))
{
break;
}
agendaLines.Add(line);
}
return string.Join(Environment.NewLine, agendaLines).Trim();
}
private static bool IsTeamsAppointment(object appointment)
{
var subject = Convert.ToString(GetProperty(appointment, "Subject")) ?? "";
var location = Convert.ToString(GetProperty(appointment, "Location")) ?? "";
var body = Convert.ToString(GetProperty(appointment, "Body")) ?? "";
return ContainsTeamsMarker(subject) ||
ContainsTeamsMarker(location) ||
ContainsTeamsMarker(body);
}
private static bool ContainsTeamsMarker(string value)
{
return value.Contains("teams.microsoft.com", StringComparison.OrdinalIgnoreCase) ||
value.Contains("Microsoft Teams", StringComparison.OrdinalIgnoreCase) ||
value.Contains("Join the meeting now", StringComparison.OrdinalIgnoreCase) ||
value.Contains("Join Microsoft Teams Meeting", StringComparison.OrdinalIgnoreCase);
}
private static bool IsTeamsSeparator(string line)
{
var trimmed = line.Trim();
return trimmed.Length >= 8 &&
trimmed.All(character => character is '_' or '-' or '*' or ' ');
}
private static bool IsTeamsJoinLine(string line)
{
return ContainsTeamsMarker(line);
}
private static IReadOnlyList<string> ReadAttendees(object appointment)
{
var attendees = new List<string>();
var organizer = Convert.ToString(GetProperty(appointment, "Organizer"));
if (!string.IsNullOrWhiteSpace(organizer))
{
attendees.Add(organizer.Trim());
}
object? recipients = null;
try
{
recipients = GetProperty(appointment, "Recipients");
var count = Convert.ToInt32(GetProperty(recipients!, "Count"));
for (var index = 1; index <= count; index++)
{
object? recipient = null;
try
{
recipient = Invoke(recipients!, "Item", index);
var formatted = FormatRecipient(recipient!);
if (!string.IsNullOrWhiteSpace(formatted))
{
attendees.Add(formatted);
}
}
finally
{
ReleaseComObject(recipient);
}
}
}
finally
{
ReleaseComObject(recipients);
}
return NormalizeAttendees(attendees);
}
private static DateTimeOffset ToLocalOffset(DateTime value)
{
return new DateTimeOffset(DateTime.SpecifyKind(value, DateTimeKind.Local));
}
internal static IReadOnlyList<string> NormalizeAttendees(IEnumerable<string> attendees)
{
return attendees
.Select(NormalizeAttendee)
.Where(attendee => !string.IsNullOrWhiteSpace(attendee))
.GroupBy(GetAttendeeDeduplicationKey, StringComparer.OrdinalIgnoreCase)
.Select(group => group
.OrderByDescending(attendee => attendee.Contains('<', StringComparison.Ordinal))
.ThenBy(attendee => attendee.Length)
.First())
.ToList();
}
private static string NormalizeAttendee(string attendee)
{
var normalized = attendee.Trim();
var mailtoPrefix = "mailto:";
normalized = normalized.Replace(mailtoPrefix, "", StringComparison.OrdinalIgnoreCase);
while (normalized.Contains(" ", StringComparison.Ordinal))
{
normalized = normalized.Replace(" ", " ", StringComparison.Ordinal);
}
var emailStart = normalized.IndexOf('<', StringComparison.Ordinal);
var emailEnd = normalized.LastIndexOf('>');
if (emailStart > 0 && emailEnd > emailStart)
{
var name = normalized[..emailStart].Trim();
var email = normalized[(emailStart + 1)..emailEnd].Trim();
if (name.Equals(email, StringComparison.OrdinalIgnoreCase))
{
return email;
}
return $"{name} <{email}>";
}
return normalized;
}
private static string GetAttendeeDeduplicationKey(string attendee)
{
var emailStart = attendee.IndexOf('<', StringComparison.Ordinal);
var emailEnd = attendee.LastIndexOf('>');
if (emailStart > 0 && emailEnd > emailStart)
{
return attendee[(emailStart + 1)..emailEnd].Trim();
}
return attendee.Trim();
}
private static string FormatRecipient(object recipient)
{
var name = Convert.ToString(GetProperty(recipient, "Name"))?.Trim() ?? "";
var email = Convert.ToString(GetProperty(recipient, "Address"))?.Trim() ?? "";
if (string.IsNullOrWhiteSpace(name))
{
return email;
}
if (string.IsNullOrWhiteSpace(email) ||
email.Contains('/'))
{
return name;
}
return $"{name} <{email}>";
}
private static object? GetProperty(object target, string name)
{
return target.GetType().InvokeMember(
name,
System.Reflection.BindingFlags.GetProperty,
null,
target,
null);
}
private static void SetProperty(object target, string name, object value)
{
target.GetType().InvokeMember(
name,
System.Reflection.BindingFlags.SetProperty,
null,
target,
[value]);
}
private static object? Invoke(object target, string name, params object[] arguments)
{
return target.GetType().InvokeMember(
name,
System.Reflection.BindingFlags.InvokeMethod,
null,
target,
arguments);
}
private static void ReleaseComObject(object? value)
{
if (value is not null && Marshal.IsComObject(value))
{
Marshal.FinalReleaseComObject(value);
}
}
private sealed record OutlookAppointmentCandidate(
object Appointment,
DateTime Start,
@@ -1,38 +0,0 @@
using YamlDotNet.Core;
using YamlDotNet.Core.Events;
using YamlDotNet.Serialization;
namespace MeetingAssistant.MeetingNotes;
internal sealed class ScalarStringListYamlTypeConverter : IYamlTypeConverter
{
public bool Accepts(Type type)
{
return type == typeof(List<string>);
}
public object ReadYaml(IParser parser, Type type, ObjectDeserializer rootDeserializer)
{
if (parser.TryConsume<Scalar>(out var scalar))
{
return string.IsNullOrWhiteSpace(scalar.Value)
? []
: new List<string> { scalar.Value };
}
var values = new List<string>();
parser.Consume<SequenceStart>();
while (!parser.TryConsume<SequenceEnd>(out _))
{
var item = parser.Consume<Scalar>();
values.Add(item.Value);
}
return values;
}
public void WriteYaml(IEmitter emitter, object? value, Type type, ObjectSerializer serializer)
{
serializer(value, typeof(IEnumerable<string>));
}
}
@@ -1,14 +0,0 @@
namespace MeetingAssistant.Notifications;
internal static class MeetingToastExpirationPolicy
{
public static DateTimeOffset StopReminderExpiration(DateTimeOffset now)
{
return now.AddMinutes(1);
}
public static DateTimeOffset StartRecordingPromptExpiration(DateTimeOffset now)
{
return now.AddMinutes(5);
}
}
+2 -173
View File
@@ -1,8 +1,6 @@
using MeetingAssistant;
using MeetingAssistant.Calendar;
using MeetingAssistant.Hotkeys;
using MeetingAssistant.LaunchProfiles;
using MeetingAssistant.Logging;
using MeetingAssistant.MeetingNotes;
using MeetingAssistant.Recording;
using MeetingAssistant.Screenshots;
@@ -15,12 +13,9 @@ using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Options;
var builder = WebApplication.CreateBuilder(args);
builder.Logging.AddProvider(new MeetingAssistantFileLoggerProvider());
builder.Services.Configure<MeetingAssistantOptions>(builder.Configuration.GetSection("MeetingAssistant"));
builder.Services.AddSingleton<ILaunchProfileOptionsProvider, ConfigurationLaunchProfileOptionsProvider>();
#if WINDOWS
builder.Services.AddSingleton<MicrophoneDeviceSelection>();
builder.Services.AddSingleton<IMicrophoneDeviceProvider, WindowsMicrophoneDeviceProvider>();
builder.Services.AddSingleton<MicrophoneAudioSource>();
builder.Services.AddSingleton<SystemAudioSource>();
builder.Services.AddSingleton<IAcousticEchoCancellerFactory, AdaptiveFilterAcousticEchoCancellerFactory>();
@@ -39,43 +34,15 @@ builder.Services.AddSingleton<IMeetingNoteStore, MarkdownMeetingNoteStore>();
builder.Services.AddSingleton<IMeetingNoteOpener, ObsidianMeetingNoteOpener>();
builder.Services.AddSingleton<IMeetingArtifactStore, MarkdownMeetingArtifactStore>();
builder.Services.AddSingleton<IMeetingRunArtifactCleaner, MeetingRunArtifactCleaner>();
builder.Services.AddSingleton<IOfflineTranscriptionBacklog, FileOfflineTranscriptionBacklog>();
builder.Services.AddSingleton<IRecordedAudioStore, TemporaryRecordedAudioStore>();
builder.Services.AddSingleton<IDictationWordStore, MarkdownDictationWordStore>();
builder.Services.AddSingleton<IMeetingInactivityClock, SystemMeetingInactivityClock>();
#if WINDOWS
builder.Services.AddSingleton<WindowsMeetingInactivityPromptService>();
builder.Services.AddSingleton<IMeetingInactivityPromptService>(services =>
services.GetRequiredService<WindowsMeetingInactivityPromptService>());
builder.Services.AddHostedService(services =>
services.GetRequiredService<WindowsMeetingInactivityPromptService>());
#else
builder.Services.AddSingleton<IMeetingInactivityPromptService, NoopMeetingInactivityPromptService>();
#endif
builder.Services.AddSingleton<IRecordingDictationWordProvider, RecordingDictationWordProvider>();
builder.Services.AddSingleton<ICalendarPromptClock, SystemCalendarPromptClock>();
builder.Services.AddSingleton<IMeetingPromptRecordingController, MeetingPromptRecordingController>();
#if WINDOWS
builder.Services.AddSingleton<ICalendarMeetingProvider, OutlookClassicCalendarMeetingProvider>();
builder.Services.AddSingleton<WindowsMeetingStartPromptService>();
builder.Services.AddSingleton<IMeetingStartPromptService>(services =>
services.GetRequiredService<WindowsMeetingStartPromptService>());
builder.Services.AddHostedService(services =>
services.GetRequiredService<WindowsMeetingStartPromptService>());
#else
builder.Services.AddSingleton<ICalendarMeetingProvider, NoopCalendarMeetingProvider>();
builder.Services.AddSingleton<IMeetingStartPromptService, NoopMeetingStartPromptService>();
#endif
#if WINDOWS
builder.Services.AddSingleton<IActiveWindowScreenshotCapture, ActiveWindowScreenshotCapture>();
#else
builder.Services.AddSingleton<IActiveWindowScreenshotCapture, UnavailableActiveWindowScreenshotCapture>();
#endif
builder.Services.AddSingleton<IScreenshotOcrClient, LiteLlmScreenshotOcrClient>();
builder.Services.AddSingleton<MeetingScreenshotService>();
builder.Services.AddSingleton<IMeetingScreenshotService>(sp => sp.GetRequiredService<MeetingScreenshotService>());
builder.Services.AddSingleton<IScreenshotOcrRetryRunner>(sp => sp.GetRequiredService<MeetingScreenshotService>());
builder.Services.AddSingleton<IMeetingNoteImageOcrService>(sp => sp.GetRequiredService<MeetingScreenshotService>());
builder.Services.AddSingleton<IMeetingScreenshotService, MeetingScreenshotService>();
builder.Services.AddDbContextFactory<SpeakerIdentityDbContext>((services, dbOptions) =>
{
var appOptions = services.GetRequiredService<IOptions<MeetingAssistantOptions>>().Value;
@@ -102,9 +69,7 @@ builder.Services.AddSingleton<IWorkflowRulesEditorSamplePlaybackQueue, WorkflowR
builder.Services.AddSingleton<IWorkflowRulesEditorChatPipeline, WorkflowRulesEditorChatPipeline>();
builder.Services.AddTransient<WorkflowRulesEditorChatViewModel>();
#if WINDOWS
builder.Services.AddSingleton(services => WorkflowRulesEditorMarkdownLinkResolver.FromOptions(
services.GetRequiredService<IOptions<MeetingAssistantOptions>>().Value));
builder.Services.AddSingleton<IWorkflowRulesEditorWindowService, WpfWorkflowRulesEditorWindowService>();
builder.Services.AddSingleton<IWorkflowRulesEditorWindowService, MewUiWorkflowRulesEditorWindowService>();
#else
builder.Services.AddSingleton<IWorkflowRulesEditorWindowService, NoopWorkflowRulesEditorWindowService>();
#endif
@@ -122,11 +87,8 @@ builder.Services.AddSingleton<ISpeechRecognitionPipelineBuilder, AzureSpeechReco
builder.Services.AddSingleton<ISpeechRecognitionPipelineBuilder, FunAsrSpeechRecognitionPipelineBuilder>();
builder.Services.AddSingleton<ISpeechRecognitionPipelineBuilder, WhisperLocalSpeechRecognitionPipelineBuilder>();
builder.Services.AddSingleton<ISpeechRecognitionPipelineFactory, ConfiguredSpeechRecognitionPipelineFactory>();
builder.Services.AddSingleton<OfflineTranscriptionBacklogProcessor>();
builder.Services.AddSingleton<MeetingRecordingCoordinator>();
builder.Services.AddHostedService<CalendarRecordingPromptScheduler>();
builder.Services.AddHostedService<TemporaryRecordingCleanupHostedService>();
builder.Services.AddHostedService<OfflineTranscriptionBacklogHostedService>();
builder.Services.AddHostedService<SpeakerIdentityDatabaseInitializer>();
builder.Services.AddHostedService<SpeechRecognitionPipelineHostedService>();
builder.Services.AddHostedService<PyannoteDiarizationWarmupHostedService>();
@@ -270,12 +232,6 @@ app.MapPost("/diagnostics/workflow/rules-editor/show", (
rulesEditorWindow.Show();
return Results.Accepted();
});
app.MapPost("/diagnostics/settings-and-logs/show", (
IWorkflowRulesEditorWindowService rulesEditorWindow) =>
{
rulesEditorWindow.Show();
return Results.Accepted();
});
static WorkflowConfigurationReloadResponse ReloadWorkflowConfiguration(IConfiguration configuration)
{
@@ -415,81 +371,6 @@ app.MapGet("/profiles/{launchProfile}/meetings/summary/retry", async (
launchProfiles,
cancellationToken));
app.MapPost("/meetings/screenshot-ocr/retry", async (
ScreenshotOcrRetryRequest request,
IScreenshotOcrRetryRunner screenshotOcrRetryRunner,
IOptions<MeetingAssistantOptions> options,
CancellationToken cancellationToken) =>
await RetryScreenshotOcrAsync(
null,
request,
screenshotOcrRetryRunner,
options.Value,
null,
cancellationToken));
app.MapPost("/profiles/{launchProfile}/meetings/screenshot-ocr/retry", async (
string launchProfile,
ScreenshotOcrRetryRequest request,
IScreenshotOcrRetryRunner screenshotOcrRetryRunner,
IOptions<MeetingAssistantOptions> options,
ILaunchProfileOptionsProvider launchProfiles,
CancellationToken cancellationToken) =>
await RetryScreenshotOcrAsync(
launchProfile,
request,
screenshotOcrRetryRunner,
options.Value,
launchProfiles,
cancellationToken));
app.MapGet("/meetings/screenshot-ocr/retry", async (
string meetingNotePath,
string transcriptPath,
string assistantContextPath,
string summaryPath,
string screenshotPath,
string screenshotId,
IScreenshotOcrRetryRunner screenshotOcrRetryRunner,
IOptions<MeetingAssistantOptions> options,
CancellationToken cancellationToken) =>
await RetryScreenshotOcrAsync(
null,
new ScreenshotOcrRetryRequest(
meetingNotePath,
transcriptPath,
assistantContextPath,
summaryPath,
screenshotPath,
screenshotId),
screenshotOcrRetryRunner,
options.Value,
null,
cancellationToken));
app.MapGet("/profiles/{launchProfile}/meetings/screenshot-ocr/retry", async (
string launchProfile,
string meetingNotePath,
string transcriptPath,
string assistantContextPath,
string summaryPath,
string screenshotPath,
string screenshotId,
IScreenshotOcrRetryRunner screenshotOcrRetryRunner,
IOptions<MeetingAssistantOptions> options,
ILaunchProfileOptionsProvider launchProfiles,
CancellationToken cancellationToken) =>
await RetryScreenshotOcrAsync(
launchProfile,
new ScreenshotOcrRetryRequest(
meetingNotePath,
transcriptPath,
assistantContextPath,
summaryPath,
screenshotPath,
screenshotId),
screenshotOcrRetryRunner,
options.Value,
launchProfiles,
cancellationToken));
static async Task<IResult> RetrySummaryAsync(
string? launchProfile,
string? summaryPath,
@@ -518,50 +399,6 @@ static async Task<IResult> RetrySummaryAsync(
});
}
static async Task<IResult> RetryScreenshotOcrAsync(
string? launchProfile,
ScreenshotOcrRetryRequest request,
IScreenshotOcrRetryRunner screenshotOcrRetryRunner,
MeetingAssistantOptions defaultOptions,
ILaunchProfileOptionsProvider? launchProfiles,
CancellationToken cancellationToken)
{
if (string.IsNullOrWhiteSpace(request.MeetingNotePath) ||
string.IsNullOrWhiteSpace(request.TranscriptPath) ||
string.IsNullOrWhiteSpace(request.AssistantContextPath) ||
string.IsNullOrWhiteSpace(request.SummaryPath) ||
string.IsNullOrWhiteSpace(request.ScreenshotPath) ||
string.IsNullOrWhiteSpace(request.ScreenshotId))
{
return Results.BadRequest(new { error = "Meeting artifact paths, screenshot path, and screenshot id are required." });
}
var profileOptions = ResolveProfileOptions(launchProfile, defaultOptions, launchProfiles);
var artifacts = new MeetingSessionArtifacts(
request.MeetingNotePath,
request.TranscriptPath,
request.AssistantContextPath,
request.SummaryPath);
var trigger = await screenshotOcrRetryRunner.TriggerOcrRetryAsync(
artifacts,
request.ScreenshotPath,
request.ScreenshotId,
profileOptions,
cancellationToken);
if (trigger is null)
{
return Results.NotFound(new { error = "No retryable screenshot OCR block was found for the requested screenshot." });
}
return Results.Accepted(value: new
{
assistantContextPath = trigger.AssistantContextPath,
screenshotPath = trigger.ScreenshotPath,
screenshotId = trigger.ScreenshotId,
state = "ocr retrying"
});
}
app.MapPost("/asr/transcribe-file", async (
AsrDiagnosticRequest request,
AsrDiagnosticService diagnostics,
@@ -668,14 +505,6 @@ public sealed record AsrDiagnosticRequest(string Path, int? NumSpeakers = null);
public sealed record SummaryRetryRequest(string SummaryPath);
public sealed record ScreenshotOcrRetryRequest(
string MeetingNotePath,
string TranscriptPath,
string AssistantContextPath,
string SummaryPath,
string ScreenshotPath,
string ScreenshotId);
public sealed record SpeakerIdentityMergeDiagnosticRequest(double? RecentDays = null);
public sealed record WorkflowConfigurationReloadResponse(string? RulesPath);
@@ -1,124 +0,0 @@
using System.Text.Json;
using Microsoft.Extensions.Options;
namespace MeetingAssistant.Recording;
public sealed class FileOfflineTranscriptionBacklog : IOfflineTranscriptionBacklog
{
private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web)
{
WriteIndented = true
};
private readonly MeetingAssistantOptions options;
private readonly ILogger<FileOfflineTranscriptionBacklog> logger;
public FileOfflineTranscriptionBacklog(
IOptions<MeetingAssistantOptions> options,
ILogger<FileOfflineTranscriptionBacklog> logger)
{
this.options = options.Value;
this.logger = logger;
}
public async Task EnqueueAsync(OfflineTranscriptionBacklogItem item, CancellationToken cancellationToken)
{
var folder = GetBacklogFolder(options);
Directory.CreateDirectory(folder);
var path = GetItemPath(folder, item.Id);
await File.WriteAllTextAsync(path, JsonSerializer.Serialize(item, JsonOptions), cancellationToken);
logger.LogInformation(
"Queued offline transcription backlog item {BacklogItemId} for recording {RecordingPath}",
item.Id,
item.AudioPath);
}
public async Task<IReadOnlyList<OfflineTranscriptionBacklogItem>> ListAsync(CancellationToken cancellationToken)
{
var folder = GetBacklogFolder(options);
if (!Directory.Exists(folder))
{
return [];
}
var items = new List<OfflineTranscriptionBacklogItem>();
foreach (var path in Directory.EnumerateFiles(folder, "*.json", SearchOption.TopDirectoryOnly))
{
cancellationToken.ThrowIfCancellationRequested();
try
{
var content = await File.ReadAllTextAsync(path, cancellationToken);
if (JsonSerializer.Deserialize<OfflineTranscriptionBacklogItem>(content, JsonOptions) is { } item)
{
items.Add(item);
}
}
catch (JsonException exception)
{
logger.LogWarning(exception, "Could not read offline transcription backlog item {BacklogItemPath}", path);
}
catch (IOException exception)
{
logger.LogWarning(exception, "Could not read offline transcription backlog item {BacklogItemPath}", path);
}
}
return items;
}
public Task CompleteAsync(OfflineTranscriptionBacklogItem item, CancellationToken cancellationToken)
{
var folder = GetBacklogFolder(options);
var path = GetItemPath(folder, item.Id);
if (!File.Exists(path))
{
return Task.CompletedTask;
}
File.Delete(path);
if (File.Exists(item.AudioPath))
{
DeleteCompletedAudio(item.AudioPath);
}
return Task.CompletedTask;
}
private static string GetBacklogFolder(MeetingAssistantOptions options)
{
return Path.Combine(VaultPath.Resolve(options.Recording.TemporaryRecordingsFolder), "offline-transcription-backlog");
}
private static string GetItemPath(string folder, string id)
{
return Path.Combine(folder, $"{id}.json");
}
private void DeleteCompletedAudio(string audioPath)
{
try
{
File.Delete(audioPath);
}
catch (FileNotFoundException)
{
}
catch (DirectoryNotFoundException)
{
}
catch (IOException exception)
{
logger.LogWarning(
exception,
"Could not delete completed offline transcription recording file {RecordingPath}",
audioPath);
}
catch (UnauthorizedAccessException exception)
{
logger.LogWarning(
exception,
"Could not delete completed offline transcription recording file {RecordingPath}",
audioPath);
}
}
}
@@ -1,18 +0,0 @@
namespace MeetingAssistant.Recording;
public interface IMeetingInactivityClock
{
DateTimeOffset Now { get; }
Task DelayAsync(TimeSpan delay, CancellationToken cancellationToken);
}
public sealed class SystemMeetingInactivityClock : IMeetingInactivityClock
{
public DateTimeOffset Now => DateTimeOffset.Now;
public Task DelayAsync(TimeSpan delay, CancellationToken cancellationToken)
{
return Task.Delay(delay, cancellationToken);
}
}
@@ -1,33 +0,0 @@
namespace MeetingAssistant.Recording;
public interface IMeetingInactivityPromptService
{
Task ShowStopPromptAsync(
MeetingInactivityPromptRequest request,
Func<MeetingInactivityPromptResponse, CancellationToken, Task> handleResponseAsync,
CancellationToken cancellationToken);
}
public sealed record MeetingInactivityPromptRequest(
TimeSpan InactivityDuration,
DateTimeOffset SuggestedEndTime,
int PromptIndex,
TimeSpan Threshold);
public enum MeetingInactivityPromptResponse
{
Dismissed,
Continue,
Stop
}
public sealed class NoopMeetingInactivityPromptService : IMeetingInactivityPromptService
{
public Task ShowStopPromptAsync(
MeetingInactivityPromptRequest request,
Func<MeetingInactivityPromptResponse, CancellationToken, Task> handleResponseAsync,
CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
}
@@ -1,12 +0,0 @@
using NAudio.Wave;
namespace MeetingAssistant.Recording;
public interface IMicrophoneDeviceProvider
{
IReadOnlyList<MicrophoneDevice> GetAvailableMicrophones();
MicrophoneDeviceSnapshot GetMicrophoneSnapshot(MeetingAssistantOptions options);
IWaveIn CreateCapture(MeetingAssistantOptions options);
}
+2 -66
View File
@@ -9,50 +9,16 @@ public interface ITranscriptStore
Task<TranscriptSession> CreateSessionAsync(
MeetingAssistantOptions options,
DateTimeOffset startedAt,
CancellationToken cancellationToken);
Task AppendAsync(TranscriptSession session, TranscriptionSegment segment, CancellationToken cancellationToken)
{
return AppendLineAndDiscardReferenceAsync(
session,
TranscriptLineFormatter.Format(segment).Line,
cancellationToken);
}
private async Task AppendLineAndDiscardReferenceAsync(
TranscriptSession session,
string line,
CancellationToken cancellationToken)
{
_ = await AppendLineAsync(session, line, cancellationToken);
return CreateSessionAsync(cancellationToken);
}
Task<TranscriptLineReference> AppendLineAsync(
TranscriptSession session,
string line,
CancellationToken cancellationToken);
Task ReplaceLineAsync(
TranscriptSession session,
TranscriptLineReference lineReference,
string replacementLine,
CancellationToken cancellationToken);
Task AppendAsync(TranscriptSession session, TranscriptionSegment segment, CancellationToken cancellationToken);
Task ReplaceAsync(
TranscriptSession session,
IReadOnlyList<TranscriptionSegment> replacementSegments,
CancellationToken cancellationToken)
{
return ReplaceLinesAsync(
session,
replacementSegments.Select(segment => TranscriptLineFormatter.Format(segment).Line).ToList(),
cancellationToken);
}
Task ReplaceLinesAsync(
TranscriptSession session,
IReadOnlyList<string> replacementLines,
CancellationToken cancellationToken);
Task UpdateMetadataAsync(
@@ -63,33 +29,3 @@ public interface ITranscriptStore
}
public sealed record TranscriptSession(string TranscriptPath);
public sealed record TranscriptLineReference(string TranscriptPath, int BodyLineIndex, string OriginalLine);
public static class TranscriptLineFormatter
{
public static FormattedTranscriptLine Format(TranscriptionSegment segment)
{
var speaker = NormalizeSpeaker(segment.Speaker);
if (segment.Kind == TranscriptionSegmentKind.Marker)
{
return new FormattedTranscriptLine(segment.Text, speaker);
}
return new FormattedTranscriptLine(
$"[{segment.Start:hh\\:mm\\:ss}] {speaker}: {segment.Text}",
speaker);
}
public static string FormatSegmentLine(TranscriptionSegment segment)
{
return Format(segment).Line;
}
public static string NormalizeSpeaker(string? speaker)
{
return string.IsNullOrWhiteSpace(speaker) ? "Unknown" : speaker;
}
}
public sealed record FormattedTranscriptLine(string Line, string Speaker);
@@ -1,59 +0,0 @@
using MeetingAssistant.MeetingNotes;
namespace MeetingAssistant.Recording;
internal static class MeetingArtifactContentDetector
{
private static readonly string[] DefaultAssistantContextLines =
[
"# Assistant Context",
"## Live Context"
];
public static async Task<bool> IsDefaultOnlyAsync(
MeetingSessionArtifacts artifacts,
CancellationToken cancellationToken)
{
if (!File.Exists(artifacts.MeetingNotePath) ||
!File.Exists(artifacts.TranscriptPath) ||
!File.Exists(artifacts.AssistantContextPath))
{
return false;
}
var meetingNote = await File.ReadAllTextAsync(artifacts.MeetingNotePath, cancellationToken);
var transcript = await File.ReadAllTextAsync(artifacts.TranscriptPath, cancellationToken);
var assistantContext = await File.ReadAllTextAsync(artifacts.AssistantContextPath, cancellationToken);
return IsDefaultOnlyMeetingNote(meetingNote) &&
IsDefaultOnlyTranscript(transcript) &&
IsDefaultOnlyAssistantContext(assistantContext);
}
private static bool IsDefaultOnlyMeetingNote(string content)
{
return string.IsNullOrWhiteSpace(MarkdownDocumentParser.SplitOptional(content).Body);
}
private static bool IsDefaultOnlyTranscript(string content)
{
var body = MarkdownDocumentParser.SplitOptional(content).Body;
return GetMeaningfulLines(body)
.All(line => string.Equals(line, "# Meeting Transcript", StringComparison.OrdinalIgnoreCase));
}
private static bool IsDefaultOnlyAssistantContext(string content)
{
var body = MarkdownDocumentParser.SplitOptional(content).Body;
return GetMeaningfulLines(body)
.All(line => DefaultAssistantContextLines.Contains(line, StringComparer.OrdinalIgnoreCase));
}
private static IEnumerable<string> GetMeaningfulLines(string body)
{
return body
.Split(["\r\n", "\n"], StringSplitOptions.None)
.Select(line => line.Trim())
.Where(line => !string.IsNullOrWhiteSpace(line));
}
}
File diff suppressed because it is too large Load Diff
@@ -1,9 +0,0 @@
namespace MeetingAssistant.Recording;
public sealed record MicrophoneDevice(
string Id,
string Name);
public sealed record MicrophoneDeviceSnapshot(
IReadOnlyList<MicrophoneDevice> Available,
MicrophoneDevice? Current);
@@ -1,54 +0,0 @@
namespace MeetingAssistant.Recording;
public sealed class MicrophoneDeviceSelection
{
private readonly object sync = new();
private string? selectedDeviceId;
public string? SelectedDeviceId
{
get
{
lock (sync)
{
return selectedDeviceId;
}
}
}
public void Select(string deviceId)
{
lock (sync)
{
selectedDeviceId = string.IsNullOrWhiteSpace(deviceId)
? null
: deviceId.Trim();
}
}
public MicrophoneDevice? Resolve(
string? configuredDeviceId,
MicrophoneDevice? defaultDevice,
IReadOnlyList<MicrophoneDevice> availableDevices)
{
var selected = SelectedDeviceId;
return FindById(availableDevices, selected) ??
FindById(availableDevices, configuredDeviceId) ??
defaultDevice;
}
private static MicrophoneDevice? FindById(
IReadOnlyList<MicrophoneDevice> devices,
string? id)
{
if (string.IsNullOrWhiteSpace(id))
{
return null;
}
return devices.FirstOrDefault(device => string.Equals(
device.Id,
id.Trim(),
StringComparison.OrdinalIgnoreCase));
}
}
@@ -6,14 +6,10 @@ namespace MeetingAssistant.Recording;
public sealed class MicrophoneAudioSource : IMeetingAudioSource
{
private readonly IMicrophoneDeviceProvider microphones;
private readonly ILogger<MicrophoneAudioSource> logger;
public MicrophoneAudioSource(
IMicrophoneDeviceProvider microphones,
ILogger<MicrophoneAudioSource> logger)
public MicrophoneAudioSource(ILogger<MicrophoneAudioSource> logger)
{
this.microphones = microphones;
this.logger = logger;
}
@@ -26,7 +22,7 @@ public sealed class MicrophoneAudioSource : IMeetingAudioSource
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
return CaptureAsync(microphones.CreateCapture(options), options, cancellationToken);
return CaptureAsync(new WasapiCapture(), options, cancellationToken);
}
private IAsyncEnumerable<AudioChunk> CaptureAsync(
@@ -1,29 +0,0 @@
namespace MeetingAssistant.Recording;
internal static class NotificationActivationArguments
{
public static IReadOnlyDictionary<string, string> Parse(string argumentText)
{
if (string.IsNullOrWhiteSpace(argumentText))
{
return new Dictionary<string, string>();
}
var arguments = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
foreach (var pair in argumentText.Split(['&', ';'], StringSplitOptions.RemoveEmptyEntries))
{
var separatorIndex = pair.IndexOf('=');
if (separatorIndex < 0)
{
arguments[Uri.UnescapeDataString(pair)] = "";
continue;
}
var key = Uri.UnescapeDataString(pair[..separatorIndex]);
var value = Uri.UnescapeDataString(pair[(separatorIndex + 1)..]);
arguments[key] = value;
}
return arguments;
}
}
@@ -1,45 +0,0 @@
namespace MeetingAssistant.Recording;
public interface IOfflineTranscriptionBacklog
{
Task EnqueueAsync(OfflineTranscriptionBacklogItem item, CancellationToken cancellationToken);
Task<IReadOnlyList<OfflineTranscriptionBacklogItem>> ListAsync(CancellationToken cancellationToken);
Task CompleteAsync(OfflineTranscriptionBacklogItem item, CancellationToken cancellationToken);
}
public sealed record OfflineTranscriptionBacklogItem(
string Id,
string AudioPath,
string TranscriptPath,
string MeetingNotePath,
string AssistantContextPath,
string SummaryPath,
DateTimeOffset StartedAt,
DateTimeOffset? InferredEndTime,
string LaunchProfileName);
public sealed class NoopOfflineTranscriptionBacklog : IOfflineTranscriptionBacklog
{
public static readonly NoopOfflineTranscriptionBacklog Instance = new();
private NoopOfflineTranscriptionBacklog()
{
}
public Task EnqueueAsync(OfflineTranscriptionBacklogItem item, CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
public Task<IReadOnlyList<OfflineTranscriptionBacklogItem>> ListAsync(CancellationToken cancellationToken)
{
return Task.FromResult<IReadOnlyList<OfflineTranscriptionBacklogItem>>([]);
}
public Task CompleteAsync(OfflineTranscriptionBacklogItem item, CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
}
@@ -1,53 +0,0 @@
namespace MeetingAssistant.Recording;
public sealed class OfflineTranscriptionBacklogHostedService : BackgroundService
{
private static readonly TimeSpan RetryDelay = TimeSpan.FromMinutes(1);
private readonly OfflineTranscriptionBacklogProcessor processor;
private readonly ILogger<OfflineTranscriptionBacklogHostedService> logger;
public OfflineTranscriptionBacklogHostedService(
OfflineTranscriptionBacklogProcessor processor,
ILogger<OfflineTranscriptionBacklogHostedService> logger)
{
this.processor = processor;
this.logger = logger;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
try
{
var processed = await processor.ProcessPendingAsync(stoppingToken);
if (processed > 0)
{
logger.LogInformation(
"Processed {ProcessedCount} offline transcription backlog item(s)",
processed);
}
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
return;
}
catch (Exception exception)
{
logger.LogWarning(
exception,
"Offline transcription backlog processing failed; it will be retried later");
}
try
{
await Task.Delay(RetryDelay, stoppingToken);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
return;
}
}
}
}
@@ -1,220 +0,0 @@
using MeetingAssistant.LaunchProfiles;
using MeetingAssistant.MeetingNotes;
using MeetingAssistant.Summary;
using MeetingAssistant.Transcription;
using MeetingAssistant.Workflow;
using Microsoft.Extensions.Options;
namespace MeetingAssistant.Recording;
public sealed class OfflineTranscriptionBacklogProcessor
{
private readonly IOfflineTranscriptionBacklog backlog;
private readonly ISpeechRecognitionPipelineFactory pipelineFactory;
private readonly ITranscriptStore transcriptStore;
private readonly IMeetingNoteStore meetingNoteStore;
private readonly IMeetingArtifactStore meetingArtifactStore;
private readonly IMeetingSummaryPipeline summaryPipeline;
private readonly IMeetingWorkflowEngine workflowEngine;
private readonly IRecordingDictationWordProvider dictationWordProvider;
private readonly MeetingAssistantOptions options;
private readonly ILogger<OfflineTranscriptionBacklogProcessor> logger;
private readonly ILaunchProfileOptionsProvider? launchProfiles;
public OfflineTranscriptionBacklogProcessor(
IOfflineTranscriptionBacklog backlog,
ISpeechRecognitionPipelineFactory pipelineFactory,
ITranscriptStore transcriptStore,
IMeetingNoteStore meetingNoteStore,
IMeetingArtifactStore meetingArtifactStore,
IMeetingSummaryPipeline summaryPipeline,
IMeetingWorkflowEngine workflowEngine,
IRecordingDictationWordProvider dictationWordProvider,
IOptions<MeetingAssistantOptions> options,
ILogger<OfflineTranscriptionBacklogProcessor> logger,
ILaunchProfileOptionsProvider? launchProfiles = null)
{
this.backlog = backlog;
this.pipelineFactory = pipelineFactory;
this.transcriptStore = transcriptStore;
this.meetingNoteStore = meetingNoteStore;
this.meetingArtifactStore = meetingArtifactStore;
this.summaryPipeline = summaryPipeline;
this.workflowEngine = workflowEngine;
this.dictationWordProvider = dictationWordProvider;
this.options = options.Value;
this.logger = logger;
this.launchProfiles = launchProfiles;
}
public async Task<int> ProcessPendingAsync(CancellationToken cancellationToken)
{
var processed = 0;
foreach (var item in await backlog.ListAsync(cancellationToken))
{
cancellationToken.ThrowIfCancellationRequested();
if (await TryProcessAsync(item, cancellationToken))
{
processed++;
}
}
return processed;
}
private async Task<bool> TryProcessAsync(
OfflineTranscriptionBacklogItem item,
CancellationToken cancellationToken)
{
try
{
await ProcessAsync(item, cancellationToken);
await backlog.CompleteAsync(item, cancellationToken);
logger.LogInformation(
"Completed offline transcription backlog item {BacklogItemId} for recording {RecordingPath}",
item.Id,
item.AudioPath);
return true;
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception exception)
{
logger.LogWarning(
exception,
"Could not process offline transcription backlog item {BacklogItemId}; it will be retried later",
item.Id);
return false;
}
}
private async Task ProcessAsync(
OfflineTranscriptionBacklogItem item,
CancellationToken cancellationToken)
{
var runOptions = ResolveOptions(item.LaunchProfileName);
var artifacts = new MeetingSessionArtifacts(
item.MeetingNotePath,
item.TranscriptPath,
item.AssistantContextPath,
item.SummaryPath);
var session = new TranscriptSession(item.TranscriptPath);
var meetingNote = await meetingNoteStore.ReadAsync(item.MeetingNotePath, cancellationToken);
var pipelineOptions = await RecordingSpeechRecognitionPipelineOptions.BuildAsync(
dictationWordProvider,
runOptions,
meetingNote,
cancellationToken);
await using var pipeline = pipelineFactory.Create(item.LaunchProfileName);
await pipeline.InitializeAsync(pipelineOptions, cancellationToken);
await pipeline.WaitUntilReadyAsync(cancellationToken);
await WriteRecordingToPipelineAsync(item.AudioPath, pipeline, cancellationToken);
var finishedSegments = await pipeline.ReadFinishedTranscriptAsync(
item.AudioPath,
pipelineOptions,
cancellationToken);
var lines = await TransformTranscriptLinesAsync(artifacts, runOptions, finishedSegments, cancellationToken);
await transcriptStore.ReplaceLinesAsync(session, lines, cancellationToken);
meetingNote.Frontmatter.EndTime = item.InferredEndTime ?? DateTimeOffset.Now;
var completedMeetingNote = await meetingNoteStore.SaveAsync(meetingNote, runOptions, cancellationToken);
await transcriptStore.UpdateMetadataAsync(session, artifacts, completedMeetingNote, cancellationToken);
await meetingArtifactStore.UpdateAssistantContextMeetingAsync(artifacts, completedMeetingNote, cancellationToken);
await TransitionMeetingAsync(
artifacts,
runOptions,
AssistantContextState.Transcribing,
AssistantContextState.Summarizing,
cancellationToken);
var summaryResult = await summaryPipeline.RunAsync(artifacts, runOptions, cancellationToken);
await TransitionMeetingAsync(
artifacts,
runOptions,
AssistantContextState.Summarizing,
summaryResult.Succeeded ? AssistantContextState.Finished : AssistantContextState.Error,
cancellationToken);
}
private MeetingAssistantOptions ResolveOptions(string? launchProfileName)
{
if (launchProfiles is not null)
{
return launchProfiles.GetRequiredProfile(launchProfileName).Options;
}
return options;
}
private async Task<List<string>> TransformTranscriptLinesAsync(
MeetingSessionArtifacts artifacts,
MeetingAssistantOptions runOptions,
IReadOnlyList<TranscriptionSegment> segments,
CancellationToken cancellationToken)
{
var lines = new List<string>(segments.Count);
foreach (var segment in segments)
{
var formatted = TranscriptLineFormatter.Format(segment);
lines.Add(await TransformTranscriptLineAsync(artifacts, runOptions, formatted, cancellationToken));
}
return lines;
}
private async Task<string> TransformTranscriptLineAsync(
MeetingSessionArtifacts artifacts,
MeetingAssistantOptions runOptions,
FormattedTranscriptLine formatted,
CancellationToken cancellationToken)
{
try
{
return await workflowEngine.TransformTranscriptLineAsync(
MeetingWorkflowEvent.TranscriptLine(artifacts, formatted.Line, formatted.Speaker),
runOptions,
cancellationToken);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception exception)
{
logger.LogError(
exception,
"Offline transcript line workflow failed for speaker {Speaker}; keeping original transcript line",
formatted.Speaker);
return formatted.Line;
}
}
private async Task TransitionMeetingAsync(
MeetingSessionArtifacts artifacts,
MeetingAssistantOptions runOptions,
AssistantContextState from,
AssistantContextState to,
CancellationToken cancellationToken)
{
await meetingArtifactStore.UpdateAssistantContextStateAsync(artifacts, to, cancellationToken);
await workflowEngine.RunAsync(
MeetingWorkflowEvent.StateTransition(artifacts, from, to),
runOptions,
cancellationToken);
}
private static async Task WriteRecordingToPipelineAsync(
string audioPath,
ISpeechRecognitionPipeline pipeline,
CancellationToken cancellationToken)
{
await foreach (var chunk in PcmWavAudioChunkReader.ReadChunksAsync(audioPath, cancellationToken: cancellationToken))
{
await pipeline.WriteAsync(chunk, cancellationToken);
}
}
}
@@ -1,48 +0,0 @@
using System.Runtime.CompilerServices;
using NAudio.Wave;
namespace MeetingAssistant.Recording;
public static class PcmWavAudioChunkReader
{
private const int MaxChunkDurationMilliseconds = 1000;
public static async IAsyncEnumerable<AudioChunk> ReadChunksAsync(
string path,
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
var reader = new WaveFileReader(path);
try
{
var format = reader.WaveFormat;
if (format.Encoding != WaveFormatEncoding.Pcm || format.BitsPerSample != 16)
{
throw new InvalidDataException(
$"Only 16-bit PCM WAV files are supported. Found {format.Encoding} with {format.BitsPerSample} bits per sample.");
}
var chunkSize = Math.Max(
format.BlockAlign,
format.AverageBytesPerSecond * MaxChunkDurationMilliseconds / 1000);
chunkSize -= chunkSize % format.BlockAlign;
var buffer = new byte[chunkSize];
int read;
while ((read = await reader.ReadAsync(buffer.AsMemory(0, buffer.Length), cancellationToken)) > 0)
{
yield return new AudioChunk(buffer[..read], format.SampleRate, format.Channels);
}
}
finally
{
try
{
reader.Dispose();
}
catch (NotSupportedException)
{
// NAudio can throw while disposing reader streams from async iterators after all chunks were read.
}
}
}
}
@@ -1,86 +0,0 @@
using MeetingAssistant.Transcription;
namespace MeetingAssistant.Recording;
public interface IRecordingDictationWordProvider
{
Task<IReadOnlyList<string>> ReadOptionalWordsAsync(
MeetingAssistantOptions runOptions,
CancellationToken cancellationToken);
}
public sealed class RecordingDictationWordProvider : IRecordingDictationWordProvider
{
private readonly IDictationWordStore dictationWordStore;
private readonly ILogger<RecordingDictationWordProvider> logger;
public RecordingDictationWordProvider(
IDictationWordStore dictationWordStore,
ILogger<RecordingDictationWordProvider> logger)
{
this.dictationWordStore = dictationWordStore;
this.logger = logger;
}
public async Task<IReadOnlyList<string>> ReadOptionalWordsAsync(
MeetingAssistantOptions runOptions,
CancellationToken cancellationToken)
{
var timeout = runOptions.Recording.DictationWordsReadTimeout;
if (timeout <= TimeSpan.Zero)
{
return await ReadWithoutFallbackTimeoutAsync(runOptions, cancellationToken);
}
try
{
using var readCancellation = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
var readTask = dictationWordStore.ReadWordsAsync(runOptions, readCancellation.Token);
try
{
return await readTask.WaitAsync(timeout, cancellationToken);
}
catch (TimeoutException exception)
{
readCancellation.Cancel();
logger.LogWarning(
exception,
"Reading dictation words timed out after {Timeout}; recording will continue without dictation-word hints",
timeout);
return [];
}
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception exception)
{
logger.LogWarning(
exception,
"Could not read dictation words; recording will continue without dictation-word hints");
return [];
}
}
private async Task<IReadOnlyList<string>> ReadWithoutFallbackTimeoutAsync(
MeetingAssistantOptions runOptions,
CancellationToken cancellationToken)
{
try
{
return await dictationWordStore.ReadWordsAsync(runOptions, cancellationToken);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception exception)
{
logger.LogWarning(
exception,
"Could not read dictation words; recording will continue without dictation-word hints");
return [];
}
}
}
@@ -1,20 +0,0 @@
using MeetingAssistant.MeetingNotes;
using MeetingAssistant.Transcription;
namespace MeetingAssistant.Recording;
public static class RecordingSpeechRecognitionPipelineOptions
{
public static async Task<SpeechRecognitionPipelineOptions> BuildAsync(
IRecordingDictationWordProvider dictationWordProvider,
MeetingAssistantOptions runOptions,
MeetingNote? meetingNote,
CancellationToken cancellationToken)
{
var dictationWords = await dictationWordProvider.ReadOptionalWordsAsync(runOptions, cancellationToken);
var attendeeCount = meetingNote?.Frontmatter.Attendees.Count(attendee => !string.IsNullOrWhiteSpace(attendee)) ?? 0;
return attendeeCount > 1
? new SpeechRecognitionPipelineOptions(attendeeCount, dictationWords)
: new SpeechRecognitionPipelineOptions(DictationWords: dictationWords);
}
}
@@ -1,6 +1,5 @@
using MeetingAssistant.Speakers;
using MeetingAssistant.Transcription;
using Microsoft.Extensions.Logging;
namespace MeetingAssistant.Recording;
@@ -12,7 +11,6 @@ internal sealed class SpeakerAudioSampleCollector
private readonly int maxSamplesPerSpeaker;
private readonly TimeSpan minimumUninterruptedSpeechDuration;
private readonly TimeSpan maximumSegmentGap;
private readonly ILogger? logger;
private PendingSpeakerSpan? pendingSpan;
public SpeakerAudioSampleCollector(TimeSpan bufferDuration, int maxSamplesPerSpeaker)
@@ -20,8 +18,7 @@ internal sealed class SpeakerAudioSampleCollector
bufferDuration,
maxSamplesPerSpeaker,
TimeSpan.FromSeconds(30),
TimeSpan.FromSeconds(1),
logger: null)
TimeSpan.FromSeconds(1))
{
}
@@ -29,8 +26,7 @@ internal sealed class SpeakerAudioSampleCollector
TimeSpan bufferDuration,
int maxSamplesPerSpeaker,
TimeSpan minimumUninterruptedSpeechDuration,
TimeSpan maximumSegmentGap,
ILogger? logger = null)
TimeSpan maximumSegmentGap)
{
audioBuffer = new RollingAudioBuffer(bufferDuration);
this.maxSamplesPerSpeaker = Math.Max(1, maxSamplesPerSpeaker);
@@ -40,7 +36,6 @@ internal sealed class SpeakerAudioSampleCollector
this.maximumSegmentGap = maximumSegmentGap >= TimeSpan.Zero
? maximumSegmentGap
: TimeSpan.Zero;
this.logger = logger;
}
public void AppendAudio(AudioChunk chunk)
@@ -58,61 +53,32 @@ internal sealed class SpeakerAudioSampleCollector
}
}
public SpeakerAudioSample? TryAdd(TranscriptionSegment segment)
public void TryAdd(TranscriptionSegment segment)
{
if (!IsDiarizedSpeaker(segment.Speaker))
{
logger?.LogInformation(
"Discarding speaker identity sample for {Speaker} because the segment has no diarized speaker label",
segment.Speaker);
return null;
return;
}
TranscriptionSegment sampleSegment;
PendingSpanReset? reset;
lock (gate)
{
(sampleSegment, reset) = ExtendPendingSpan(segment);
}
if (reset is not null)
{
logger?.LogInformation(
"Reset speaker identity sample span from {PreviousSpeaker} to {Speaker}: previous end {PreviousEnd}, next start {NextStart}, gap {Gap}, maximum gap {MaximumGap}",
reset.PreviousSpeaker,
segment.Speaker,
reset.PreviousEnd,
segment.Start,
reset.Gap,
maximumSegmentGap);
sampleSegment = ExtendPendingSpan(segment);
}
var score = Score(sampleSegment, minimumUninterruptedSpeechDuration);
if (!score.Accepted)
if (score <= 0)
{
logger?.LogInformation(
"Discarding speaker identity sample for {Speaker} because {Reason}: duration {Duration}, minimum duration {MinimumDuration}, word count {WordCount}",
sampleSegment.Speaker,
score.Reason,
sampleSegment.End - sampleSegment.Start,
minimumUninterruptedSpeechDuration,
score.WordCount);
return null;
return;
}
var wavBytes = audioBuffer.TryExtractWav(sampleSegment.Start, sampleSegment.End);
if (wavBytes.Length == 0)
{
logger?.LogInformation(
"Discarding speaker identity sample for {Speaker} because no audio could be extracted from rolling buffer: start {Start}, end {End}, duration {Duration}",
sampleSegment.Speaker,
sampleSegment.Start,
sampleSegment.End,
sampleSegment.End - sampleSegment.Start);
return null;
return;
}
var sample = new SpeakerAudioSample(sampleSegment.Speaker, sampleSegment, wavBytes, score.Value);
var sample = new SpeakerAudioSample(sampleSegment.Speaker, sampleSegment, wavBytes, score);
lock (gate)
{
if (!samplesBySpeaker.TryGetValue(sampleSegment.Speaker, out var samples))
@@ -122,27 +88,13 @@ internal sealed class SpeakerAudioSampleCollector
}
samples.Add(sample);
var beforeCount = samples.Count;
var bestSamples = samples
.OrderByDescending(candidate => candidate.Score)
.Take(maxSamplesPerSpeaker)
.ToList();
var retained = bestSamples.Contains(sample);
samples.Clear();
samples.AddRange(bestSamples);
if (!retained)
{
logger?.LogInformation(
"Discarding speaker identity sample for {Speaker} because it was not among the best {MaxSamplesPerSpeaker} retained sample(s): score {Score}, candidate count {CandidateCount}",
sample.Speaker,
maxSamplesPerSpeaker,
sample.Score,
beforeCount);
return null;
}
}
return sample;
}
public IReadOnlyList<SpeakerAudioSample> Snapshot()
@@ -163,43 +115,39 @@ internal sealed class SpeakerAudioSampleCollector
!string.Equals(speaker, "Unknown", StringComparison.OrdinalIgnoreCase);
}
private (TranscriptionSegment Segment, PendingSpanReset? Reset) ExtendPendingSpan(TranscriptionSegment segment)
private TranscriptionSegment ExtendPendingSpan(TranscriptionSegment segment)
{
if (pendingSpan is null ||
!SpeakerSampleSpanSelector.CanExtend(pendingSpan.Speaker, pendingSpan.End, segment, maximumSegmentGap))
{
var reset = pendingSpan is null
? null
: new PendingSpanReset(
pendingSpan.Speaker,
pendingSpan.End,
segment.Start - pendingSpan.End);
pendingSpan = new PendingSpeakerSpan(
segment.Speaker,
segment.Start,
segment.End,
[segment.Text]);
return (pendingSpan.ToSegment(), reset);
return pendingSpan.ToSegment();
}
pendingSpan = pendingSpan.Extend(segment);
return (pendingSpan.ToSegment(), null);
return pendingSpan.ToSegment();
}
private static SampleScore Score(
private static double Score(
TranscriptionSegment segment,
TimeSpan minimumUninterruptedSpeechDuration)
{
var durationSeconds = (segment.End - segment.Start).TotalSeconds;
if (durationSeconds < minimumUninterruptedSpeechDuration.TotalSeconds)
{
return new SampleScore(false, 0, "speech duration is below the configured minimum", WordCount(segment.Text));
return 0;
}
var words = WordCount(segment.Text);
var words = segment.Text
.Split(' ', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries)
.Length;
if (words < 3)
{
return new SampleScore(false, 0, "word count is below the minimum useful sample length", words);
return 0;
}
var durationScore = Math.Min(durationSeconds / Math.Max(1, minimumUninterruptedSpeechDuration.TotalSeconds), 2);
@@ -209,20 +157,9 @@ internal sealed class SpeakerAudioSampleCollector
segment.Text.TrimEnd().EndsWith('!')
? 5
: 0;
return new SampleScore(true, durationScore * 70 + wordScore * 30 + sentenceBonus, null, words);
return durationScore * 70 + wordScore * 30 + sentenceBonus;
}
private static int WordCount(string text)
{
return text
.Split(' ', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries)
.Length;
}
private sealed record SampleScore(bool Accepted, double Value, string? Reason, int WordCount);
private sealed record PendingSpanReset(string PreviousSpeaker, TimeSpan PreviousEnd, TimeSpan Gap);
private sealed record PendingSpeakerSpan(
string Speaker,
TimeSpan Start,
@@ -7,16 +7,13 @@ public sealed class TemporaryRecordedAudioStore : IRecordedAudioStore
{
private readonly MeetingAssistantOptions options;
private readonly ILogger<TemporaryRecordedAudioStore> logger;
private readonly IOfflineTranscriptionBacklog offlineTranscriptionBacklog;
public TemporaryRecordedAudioStore(
IOptions<MeetingAssistantOptions> options,
ILogger<TemporaryRecordedAudioStore> logger,
IOfflineTranscriptionBacklog? offlineTranscriptionBacklog = null)
ILogger<TemporaryRecordedAudioStore> logger)
{
this.options = options.Value;
this.logger = logger;
this.offlineTranscriptionBacklog = offlineTranscriptionBacklog ?? NoopOfflineTranscriptionBacklog.Instance;
}
public Task<IRecordedAudioSink> CreateSessionAsync(CancellationToken cancellationToken)
@@ -38,28 +35,21 @@ public sealed class TemporaryRecordedAudioStore : IRecordedAudioStore
return Task.FromResult<IRecordedAudioSink>(new TemporaryRecordedAudioSink(path, format, logger));
}
public async Task DeleteStaleRecordingsAsync(CancellationToken cancellationToken)
public Task DeleteStaleRecordingsAsync(CancellationToken cancellationToken)
{
var folder = GetTemporaryRecordingsFolder(options);
if (!Directory.Exists(folder))
{
return;
return Task.CompletedTask;
}
var queuedAudioPaths = (await offlineTranscriptionBacklog.ListAsync(cancellationToken))
.Select(item => item.AudioPath)
.ToHashSet(StringComparer.OrdinalIgnoreCase);
foreach (var path in Directory.EnumerateFiles(folder, "*.wav", SearchOption.TopDirectoryOnly))
{
cancellationToken.ThrowIfCancellationRequested();
if (queuedAudioPaths.Contains(path))
{
logger.LogInformation("Keeping queued offline transcription recording file {RecordingPath}", path);
continue;
}
DeleteFile(path);
}
return Task.CompletedTask;
}
private static string GetTemporaryRecordingsFolder(MeetingAssistantOptions options)
@@ -1,7 +1,6 @@
using MeetingAssistant.Transcription;
using MeetingAssistant.MeetingNotes;
using Microsoft.Extensions.Options;
using System.Collections.Concurrent;
namespace MeetingAssistant.Recording;
@@ -9,7 +8,6 @@ public sealed class VaultTranscriptStore : ITranscriptStore
{
private readonly MeetingAssistantOptions options;
private readonly ILogger<VaultTranscriptStore> logger;
private readonly ConcurrentDictionary<string, SemaphoreSlim> fileGates = new(StringComparer.OrdinalIgnoreCase);
public VaultTranscriptStore(IOptions<MeetingAssistantOptions> options, ILogger<VaultTranscriptStore> logger)
{
@@ -19,18 +17,17 @@ public sealed class VaultTranscriptStore : ITranscriptStore
public async Task<TranscriptSession> CreateSessionAsync(CancellationToken cancellationToken)
{
return await CreateSessionAsync(options, DateTimeOffset.Now, cancellationToken);
return await CreateSessionAsync(options, cancellationToken);
}
public async Task<TranscriptSession> CreateSessionAsync(
MeetingAssistantOptions options,
DateTimeOffset startedAt,
CancellationToken cancellationToken)
{
var folder = VaultPath.Resolve(options.Vault, options.Vault.TranscriptsFolder);
Directory.CreateDirectory(folder);
var fileName = MeetingArtifactFileNames.Create(startedAt, MeetingArtifactFileNames.Transcript);
var fileName = $"{DateTimeOffset.Now:yyyyMMdd-HHmmss-fffffff}-transcript.md";
var path = Path.Combine(folder, fileName);
await File.WriteAllTextAsync(path, $"# Meeting Transcript{Environment.NewLine}{Environment.NewLine}", cancellationToken);
logger.LogInformation("Created meeting transcript file {TranscriptPath}", path);
@@ -38,72 +35,28 @@ public sealed class VaultTranscriptStore : ITranscriptStore
return new TranscriptSession(path);
}
public async Task<TranscriptLineReference> AppendLineAsync(
TranscriptSession session,
string line,
CancellationToken cancellationToken)
public Task AppendAsync(TranscriptSession session, TranscriptionSegment segment, CancellationToken cancellationToken)
{
return await WithFileGateAsync(
session.TranscriptPath,
() => AppendToBodyAsync(session.TranscriptPath, line, cancellationToken),
cancellationToken);
return AppendToBodyAsync(session.TranscriptPath, FormatSegment(segment), cancellationToken);
}
public async Task ReplaceLineAsync(
public Task ReplaceAsync(
TranscriptSession session,
TranscriptLineReference lineReference,
string replacementLine,
IReadOnlyList<TranscriptionSegment> replacementSegments,
CancellationToken cancellationToken)
{
await WithFileGateAsync(
session.TranscriptPath,
async () =>
{
if (!File.Exists(session.TranscriptPath))
{
return;
}
var content = await File.ReadAllTextAsync(session.TranscriptPath, cancellationToken);
var (frontmatter, body) = MeetingArtifactFrontmatterRenderer.Split(content);
var lines = SplitLines(body);
if (lineReference.BodyLineIndex < 0 ||
lineReference.BodyLineIndex >= lines.Count ||
!string.Equals(lines[lineReference.BodyLineIndex], lineReference.OriginalLine, StringComparison.Ordinal))
{
logger.LogWarning(
"Could not replace transcript line in {TranscriptPath} at body line {BodyLineIndex} because the original line no longer matched",
session.TranscriptPath,
lineReference.BodyLineIndex);
return;
}
lines[lineReference.BodyLineIndex] = replacementLine;
await WriteBodyAsync(session.TranscriptPath, frontmatter, string.Join(Environment.NewLine, lines), cancellationToken);
},
cancellationToken);
}
public Task ReplaceLinesAsync(
TranscriptSession session,
IReadOnlyList<string> replacementLines,
CancellationToken cancellationToken)
{
return WithFileGateAsync(
session.TranscriptPath,
async () =>
{
var existing = File.Exists(session.TranscriptPath)
? File.ReadAllText(session.TranscriptPath)
: "";
var (frontmatter, _) = MeetingArtifactFrontmatterRenderer.Split(existing);
var body = "# Meeting Transcript"
+ Environment.NewLine
+ Environment.NewLine
+ string.Concat(replacementLines.Select(line => line + Environment.NewLine));
await WriteBodyAsync(session.TranscriptPath, frontmatter, body, cancellationToken);
},
cancellationToken);
var existing = File.Exists(session.TranscriptPath)
? File.ReadAllText(session.TranscriptPath)
: "";
var (frontmatter, _) = MeetingArtifactFrontmatterRenderer.Split(existing);
var body = "# Meeting Transcript"
+ Environment.NewLine
+ Environment.NewLine
+ string.Concat(replacementSegments.Select(FormatSegment));
var content = string.IsNullOrWhiteSpace(frontmatter)
? body
: "---" + Environment.NewLine + frontmatter + Environment.NewLine + "---" + Environment.NewLine + Environment.NewLine + body;
return File.WriteAllTextAsync(session.TranscriptPath, content, cancellationToken);
}
public async Task UpdateMetadataAsync(
@@ -112,108 +65,56 @@ public sealed class VaultTranscriptStore : ITranscriptStore
MeetingNote meetingNote,
CancellationToken cancellationToken)
{
await WithFileGateAsync(
session.TranscriptPath,
async () =>
{
var body = File.Exists(session.TranscriptPath)
? MeetingArtifactFrontmatterRenderer.Split(await File.ReadAllTextAsync(session.TranscriptPath, cancellationToken)).Body
: "# Meeting Transcript" + Environment.NewLine + Environment.NewLine;
var frontmatter = MeetingArtifactFrontmatterRenderer.Create(
artifacts,
meetingNote,
MeetingArtifactFrontmatterRenderer.DefaultTitle(meetingNote, "Meeting Transcript"),
session.TranscriptPath);
var body = File.Exists(session.TranscriptPath)
? MeetingArtifactFrontmatterRenderer.Split(await File.ReadAllTextAsync(session.TranscriptPath, cancellationToken)).Body
: "# Meeting Transcript" + Environment.NewLine + Environment.NewLine;
var frontmatter = MeetingArtifactFrontmatterRenderer.Create(
artifacts,
meetingNote,
MeetingArtifactFrontmatterRenderer.DefaultTitle(meetingNote, "Meeting Transcript"),
session.TranscriptPath);
await File.WriteAllTextAsync(
session.TranscriptPath,
MeetingArtifactFrontmatterRenderer.Render(frontmatter, body),
cancellationToken);
},
await File.WriteAllTextAsync(
session.TranscriptPath,
MeetingArtifactFrontmatterRenderer.Render(frontmatter, body),
cancellationToken);
}
private static async Task<TranscriptLineReference> AppendToBodyAsync(
private static string FormatSegment(TranscriptionSegment segment)
{
var speaker = string.IsNullOrWhiteSpace(segment.Speaker) ? "Unknown" : segment.Speaker;
return $"[{segment.Start:hh\\:mm\\:ss}] {speaker}: {segment.Text}{Environment.NewLine}";
}
private static async Task AppendToBodyAsync(
string path,
string line,
string text,
CancellationToken cancellationToken)
{
if (!File.Exists(path))
{
await File.AppendAllTextAsync(path, line + Environment.NewLine, cancellationToken);
return new TranscriptLineReference(path, 0, line);
await File.AppendAllTextAsync(path, text, cancellationToken);
return;
}
var content = await File.ReadAllTextAsync(path, cancellationToken);
var (frontmatter, body) = MeetingArtifactFrontmatterRenderer.Split(content);
var bodyLineIndex = GetAppendBodyLineIndex(body);
if (string.IsNullOrWhiteSpace(frontmatter))
{
await File.AppendAllTextAsync(path, line + Environment.NewLine, cancellationToken);
return new TranscriptLineReference(path, bodyLineIndex, line);
await File.AppendAllTextAsync(path, text, cancellationToken);
return;
}
await WriteBodyAsync(path, frontmatter, body + line + Environment.NewLine, cancellationToken);
return new TranscriptLineReference(path, bodyLineIndex, line);
var updated = "---"
+ Environment.NewLine
+ frontmatter
+ Environment.NewLine
+ "---"
+ Environment.NewLine
+ Environment.NewLine
+ body
+ text;
await File.WriteAllTextAsync(path, updated, cancellationToken);
}
private async Task WithFileGateAsync(
string path,
Func<Task> action,
CancellationToken cancellationToken)
{
await WithFileGateAsync(
path,
async () =>
{
await action();
return true;
},
cancellationToken);
}
private async Task<T> WithFileGateAsync<T>(
string path,
Func<Task<T>> action,
CancellationToken cancellationToken)
{
var gate = fileGates.GetOrAdd(path, static _ => new SemaphoreSlim(1, 1));
await gate.WaitAsync(cancellationToken);
try
{
return await action();
}
finally
{
gate.Release();
}
}
private static async Task WriteBodyAsync(
string path,
string frontmatter,
string body,
CancellationToken cancellationToken)
{
var content = string.IsNullOrWhiteSpace(frontmatter)
? body
: "---" + Environment.NewLine + frontmatter + Environment.NewLine + "---" + Environment.NewLine + Environment.NewLine + body;
await File.WriteAllTextAsync(path, content, cancellationToken);
}
private static int GetAppendBodyLineIndex(string body)
{
var lines = SplitLines(body);
return lines.Count > 0 && lines[^1].Length == 0
? lines.Count - 1
: lines.Count;
}
private static List<string> SplitLines(string text)
{
return text
.Replace("\r\n", "\n", StringComparison.Ordinal)
.Split('\n')
.ToList();
}
}
@@ -1,231 +0,0 @@
#if WINDOWS
using System.Collections.Concurrent;
using CommunityToolkit.WinUI.Notifications;
using MeetingAssistant.Notifications;
namespace MeetingAssistant.Recording;
public sealed class WindowsMeetingInactivityPromptService : IMeetingInactivityPromptService, IHostedService, IDisposable
{
private const string NotificationSource = "meeting-inactivity-safeguard";
private const string NotificationGroup = "meeting-inactivity";
private readonly ConcurrentDictionary<string, PendingPrompt> pendingPrompts = new(StringComparer.OrdinalIgnoreCase);
private readonly ILogger<WindowsMeetingInactivityPromptService> logger;
private readonly object registrationGate = new();
private bool registered;
public WindowsMeetingInactivityPromptService(ILogger<WindowsMeetingInactivityPromptService> logger)
{
this.logger = logger;
}
public Task ShowStopPromptAsync(
MeetingInactivityPromptRequest request,
Func<MeetingInactivityPromptResponse, CancellationToken, Task> handleResponseAsync,
CancellationToken cancellationToken)
{
if (!OperatingSystem.IsWindowsVersionAtLeast(10, 0, 17763))
{
logger.LogWarning("Windows app notifications require Windows 10 version 1809 or later");
return Task.CompletedTask;
}
if (cancellationToken.IsCancellationRequested)
{
return Task.FromCanceled(cancellationToken);
}
try
{
EnsureRegistered();
var promptId = Guid.NewGuid().ToString("N");
pendingPrompts[promptId] = new PendingPrompt(handleResponseAsync);
logger.LogInformation(
"Registered native Windows inactivity notification {PromptId} response callback at threshold {Threshold}",
promptId,
request.Threshold);
var notification = BuildNotification(promptId, request);
notification.Show(toast =>
{
toast.Group = NotificationGroup;
toast.Tag = promptId;
toast.ExpirationTime = MeetingToastExpirationPolicy.StopReminderExpiration(DateTimeOffset.Now);
});
logger.LogInformation(
"Displayed native Windows inactivity notification {PromptId} at threshold {Threshold}",
promptId,
request.Threshold);
}
catch (Exception exception)
{
logger.LogError(exception, "Failed to display native Windows inactivity notification");
}
return Task.CompletedTask;
}
public Task StartAsync(CancellationToken cancellationToken)
{
if (!OperatingSystem.IsWindowsVersionAtLeast(10, 0, 17763))
{
logger.LogWarning("Windows app notifications require Windows 10 version 1809 or later");
return Task.CompletedTask;
}
EnsureRegistered();
return Task.CompletedTask;
}
public Task StopAsync(CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
public void Dispose()
{
if (!registered)
{
return;
}
try
{
ToastNotificationManagerCompat.OnActivated -= OnNotificationInvoked;
}
catch (Exception exception)
{
logger.LogWarning(exception, "Failed to detach native Windows toast notification activation handler");
}
}
private void EnsureRegistered()
{
if (registered)
{
return;
}
lock (registrationGate)
{
if (registered)
{
return;
}
ToastNotificationManagerCompat.OnActivated += OnNotificationInvoked;
registered = true;
logger.LogInformation("Registered native Windows toast notification activation handler");
}
}
private static ToastContentBuilder BuildNotification(
string promptId,
MeetingInactivityPromptRequest request)
{
var yesButton = new ToastButton()
.SetContent("Yes")
.AddArgument("source", NotificationSource)
.AddArgument("promptId", promptId)
.AddArgument("response", "stop")
.SetBackgroundActivation();
var noButton = new ToastButton()
.SetContent("No")
.AddArgument("source", NotificationSource)
.AddArgument("promptId", promptId)
.AddArgument("response", "continue")
.SetBackgroundActivation();
return new ToastContentBuilder()
.AddArgument("source", NotificationSource)
.AddArgument("promptId", promptId)
.SetToastScenario(ToastScenario.Reminder)
.SetToastDuration(ToastDuration.Long)
.AddText("Stop meeting?")
.AddText($"No transcript text has arrived for {FormatDuration(request.InactivityDuration)}.")
.AddButton(yesButton)
.AddButton(noButton);
}
private void OnNotificationInvoked(ToastNotificationActivatedEventArgsCompat args)
{
logger.LogInformation(
"Native Windows inactivity notification activation received with arguments {Arguments}",
args.Argument);
var arguments = NotificationActivationArguments.Parse(args.Argument);
if (!TryGetArgument(arguments, "source", out var source) ||
!string.Equals(source, NotificationSource, StringComparison.OrdinalIgnoreCase))
{
logger.LogDebug(
"Ignoring native Windows notification activation for unrelated source {Source}",
source);
return;
}
if (!TryGetArgument(arguments, "promptId", out var promptId))
{
logger.LogWarning("Ignoring native Windows inactivity notification activation without prompt id");
return;
}
if (!pendingPrompts.TryRemove(promptId, out var pendingPrompt))
{
logger.LogWarning(
"Ignoring native Windows inactivity notification {PromptId} activation because no pending prompt callback exists",
promptId);
return;
}
var response = TryGetArgument(arguments, "response", out var responseValue) &&
string.Equals(responseValue, "stop", StringComparison.OrdinalIgnoreCase)
? MeetingInactivityPromptResponse.Stop
: MeetingInactivityPromptResponse.Continue;
_ = Task.Run(async () =>
{
try
{
logger.LogInformation(
"Native Windows inactivity notification {PromptId} activated with response {Response}",
promptId,
response);
await pendingPrompt.HandleResponseAsync(response, CancellationToken.None);
}
catch (Exception exception)
{
logger.LogError(
exception,
"Failed to handle native Windows inactivity notification response {Response}",
response);
}
});
}
private static bool TryGetArgument(
IReadOnlyDictionary<string, string> arguments,
string key,
out string value)
{
if (arguments.TryGetValue(key, out value!))
{
return true;
}
value = "";
return false;
}
private static string FormatDuration(TimeSpan duration)
{
if (duration.TotalMinutes >= 1)
{
return $"{Math.Round(duration.TotalMinutes):0} minutes";
}
return $"{Math.Max(1, Math.Round(duration.TotalSeconds)):0} seconds";
}
private sealed record PendingPrompt(
Func<MeetingInactivityPromptResponse, CancellationToken, Task> HandleResponseAsync);
}
#endif
@@ -1,79 +0,0 @@
using NAudio.CoreAudioApi;
using NAudio.Wave;
namespace MeetingAssistant.Recording;
public sealed class WindowsMicrophoneDeviceProvider : IMicrophoneDeviceProvider
{
private readonly MicrophoneDeviceSelection selection;
private readonly ILogger<WindowsMicrophoneDeviceProvider> logger;
public WindowsMicrophoneDeviceProvider(
MicrophoneDeviceSelection selection,
ILogger<WindowsMicrophoneDeviceProvider> logger)
{
this.selection = selection;
this.logger = logger;
}
public IReadOnlyList<MicrophoneDevice> GetAvailableMicrophones()
{
try
{
using var enumerator = new MMDeviceEnumerator();
return enumerator
.EnumerateAudioEndPoints(DataFlow.Capture, DeviceState.Active)
.Select(ToMicrophoneDevice)
.OrderBy(device => device.Name, StringComparer.CurrentCultureIgnoreCase)
.ToArray();
}
catch (Exception exception)
{
logger.LogWarning(exception, "Could not enumerate microphone capture endpoints");
return [];
}
}
public MicrophoneDeviceSnapshot GetMicrophoneSnapshot(MeetingAssistantOptions options)
{
var devices = GetAvailableMicrophones();
return new MicrophoneDeviceSnapshot(
devices,
selection.Resolve(options.Recording.MicrophoneDeviceId, GetDefaultMicrophone(), devices));
}
public IWaveIn CreateCapture(MeetingAssistantOptions options)
{
var current = GetMicrophoneSnapshot(options).Current;
if (current is null)
{
logger.LogInformation("Starting microphone capture from Windows default capture endpoint");
return new WasapiCapture();
}
logger.LogInformation(
"Starting microphone capture from {MicrophoneName} ({MicrophoneDeviceId})",
current.Name,
current.Id);
using var enumerator = new MMDeviceEnumerator();
return new WasapiCapture(enumerator.GetDevice(current.Id));
}
private static MicrophoneDevice? GetDefaultMicrophone()
{
try
{
using var enumerator = new MMDeviceEnumerator();
return ToMicrophoneDevice(enumerator.GetDefaultAudioEndpoint(DataFlow.Capture, Role.Console));
}
catch
{
return null;
}
}
private static MicrophoneDevice ToMicrophoneDevice(MMDevice device)
{
return new MicrophoneDevice(device.ID, device.FriendlyName);
}
}
-39
View File
@@ -1,39 +0,0 @@
namespace MeetingAssistant;
internal static class RuntimeContentLocator
{
public static IEnumerable<string> CandidateDocumentationPaths(string fileName)
{
foreach (var directory in EnumerateBaseDirectoryAndParents())
{
yield return Path.Combine(directory, "docs", fileName);
}
}
public static IEnumerable<string> CandidateDirectoryPaths(params string[] segments)
{
foreach (var directory in EnumerateBaseDirectoryAndParents())
{
yield return Path.Combine([directory, .. segments]);
}
}
public static IEnumerable<string> CandidateContentPaths(string fileName)
{
foreach (var directory in EnumerateBaseDirectoryAndParents())
{
yield return Path.Combine(directory, "MeetingAssistant", "Content", fileName);
yield return Path.Combine(directory, "Content", fileName);
}
}
private static IEnumerable<string> EnumerateBaseDirectoryAndParents(int maxParentDepth = 8)
{
var current = new DirectoryInfo(AppContext.BaseDirectory);
for (var i = 0; i < maxParentDepth && current is not null; i++)
{
yield return current.FullName;
current = current.Parent;
}
}
}
@@ -3,7 +3,6 @@ using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
using System.Text.RegularExpressions;
using MeetingAssistant.MeetingNotes;
namespace MeetingAssistant.Screenshots;
@@ -105,69 +104,41 @@ public sealed partial class LiteLlmScreenshotOcrClient : IScreenshotOcrClient
private static ScreenshotOcrResult ParseOcrResult(string text)
{
ScreenshotCropCoordinates? crop = null;
IReadOnlyList<string> attendees = [];
var cleaned = JsonCodeBlockRegex().Replace(text, match =>
{
if (TryParseMetadata(match.Groups["json"].Value, out var parsedCrop, out var parsedAttendees))
if (TryParseCrop(match.Groups["json"].Value, out var parsedCrop))
{
crop = parsedCrop;
attendees = parsedAttendees;
return "";
}
return match.Value;
}).Trim();
return new ScreenshotOcrResult(cleaned, crop, attendees);
return new ScreenshotOcrResult(cleaned, crop);
}
private static bool TryParseMetadata(
string json,
out ScreenshotCropCoordinates? crop,
out IReadOnlyList<string> attendees)
private static bool TryParseCrop(string json, out ScreenshotCropCoordinates? crop)
{
crop = null;
attendees = [];
try
{
using var document = JsonDocument.Parse(json);
if (document.RootElement.ValueKind != JsonValueKind.Object)
{
return false;
}
var hasMetadata = false;
if (document.RootElement.TryGetProperty("attendees", out var attendeesElement))
{
hasMetadata = true;
if (attendeesElement.ValueKind == JsonValueKind.Array)
{
attendees = MeetingAttendeeNames.NormalizeDistinct(attendeesElement
.EnumerateArray()
.Where(element => element.ValueKind == JsonValueKind.String)
.Select(element => element.GetString() ?? ""));
}
}
if (!document.RootElement.TryGetProperty("crop", out var cropElement))
{
return hasMetadata;
}
hasMetadata = true;
if (cropElement.ValueKind == JsonValueKind.Null)
if (!document.RootElement.TryGetProperty("crop", out var cropElement) ||
cropElement.ValueKind == JsonValueKind.Null)
{
return true;
}
if (cropElement.ValueKind == JsonValueKind.Object &&
TryGetInt(cropElement, "x", out var x) &&
TryGetInt(cropElement, "y", out var y) &&
TryGetInt(cropElement, "width", out var width) &&
TryGetInt(cropElement, "height", out var height))
if (cropElement.ValueKind != JsonValueKind.Object ||
!TryGetInt(cropElement, "x", out var x) ||
!TryGetInt(cropElement, "y", out var y) ||
!TryGetInt(cropElement, "width", out var width) ||
!TryGetInt(cropElement, "height", out var height))
{
crop = new ScreenshotCropCoordinates(x, y, width, height);
return false;
}
crop = new ScreenshotCropCoordinates(x, y, width, height);
return true;
}
catch (JsonException)
@@ -2,10 +2,7 @@ using System.Collections.Concurrent;
using System.Drawing;
using System.Drawing.Imaging;
using System.Runtime.InteropServices;
using System.Text.RegularExpressions;
using MeetingAssistant.MeetingNotes;
using MeetingAssistant.Speakers;
using MeetingAssistant.Workflow;
namespace MeetingAssistant.Screenshots;
@@ -25,24 +22,7 @@ public interface IScreenshotOcrClient
CancellationToken cancellationToken);
}
public sealed record ScreenshotOcrResult
{
public ScreenshotOcrResult(
string text,
ScreenshotCropCoordinates? crop,
IReadOnlyList<string>? attendees = null)
{
Text = text;
Crop = crop;
Attendees = attendees ?? [];
}
public string Text { get; init; }
public ScreenshotCropCoordinates? Crop { get; init; }
public IReadOnlyList<string> Attendees { get; init; }
}
public sealed record ScreenshotOcrResult(string Text, ScreenshotCropCoordinates? Crop);
public sealed record ScreenshotCropCoordinates(int X, int Y, int Width, int Height);
@@ -66,36 +46,11 @@ public interface IMeetingScreenshotService
CancellationToken cancellationToken);
}
public interface IMeetingNoteImageOcrService
{
Task<MeetingNoteImageOcrQueueResult> ProcessMeetingNoteImageEmbedsAsync(
MeetingSessionArtifacts artifacts,
MeetingAssistantOptions options,
CancellationToken cancellationToken);
}
public interface IScreenshotOcrRetryRunner
{
Task<MeetingScreenshotOcrRetryTriggerResult?> TriggerOcrRetryAsync(
MeetingSessionArtifacts artifacts,
string screenshotPath,
string screenshotId,
MeetingAssistantOptions options,
CancellationToken cancellationToken);
}
public sealed record MeetingScreenshotCaptureResult(
string ScreenshotPath,
TimeSpan MeetingTimestamp,
bool OcrStarted);
public sealed record MeetingScreenshotOcrRetryTriggerResult(
string AssistantContextPath,
string ScreenshotPath,
string ScreenshotId);
public sealed record MeetingNoteImageOcrQueueResult(int QueuedCount);
public sealed class NoopMeetingScreenshotService : IMeetingScreenshotService
{
public static NoopMeetingScreenshotService Instance { get; } = new();
@@ -125,32 +80,12 @@ public sealed class NoopMeetingScreenshotService : IMeetingScreenshotService
{
return Task.CompletedTask;
}
}
public sealed class NoopMeetingNoteImageOcrService : IMeetingNoteImageOcrService
{
public static NoopMeetingNoteImageOcrService Instance { get; } = new();
public Task<MeetingNoteImageOcrQueueResult> ProcessMeetingNoteImageEmbedsAsync(
MeetingSessionArtifacts artifacts,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
return Task.FromResult(new MeetingNoteImageOcrQueueResult(0));
}
}
public sealed partial class MeetingScreenshotService :
IMeetingScreenshotService,
IScreenshotOcrRetryRunner,
IMeetingNoteImageOcrService
public sealed class MeetingScreenshotService : IMeetingScreenshotService
{
private readonly IActiveWindowScreenshotCapture screenshotCapture;
private readonly IMeetingArtifactStore artifactStore;
private readonly IMeetingNoteStore meetingNoteStore;
private readonly ISpeakerIdentityAttendeeCanonicalizer attendeeCanonicalizer;
private readonly IMeetingWorkflowEngine meetingWorkflowEngine;
private readonly IScreenshotOcrClient ocrClient;
private readonly ILogger<MeetingScreenshotService> logger;
private readonly ConcurrentDictionary<string, List<PendingOcrTask>> pendingOcrByContext = new(StringComparer.OrdinalIgnoreCase);
@@ -159,17 +94,11 @@ public sealed partial class MeetingScreenshotService :
public MeetingScreenshotService(
IActiveWindowScreenshotCapture screenshotCapture,
IMeetingArtifactStore artifactStore,
IMeetingNoteStore meetingNoteStore,
ISpeakerIdentityAttendeeCanonicalizer attendeeCanonicalizer,
IScreenshotOcrClient ocrClient,
ILogger<MeetingScreenshotService> logger,
IMeetingWorkflowEngine? meetingWorkflowEngine = null)
ILogger<MeetingScreenshotService> logger)
{
this.screenshotCapture = screenshotCapture;
this.artifactStore = artifactStore;
this.meetingNoteStore = meetingNoteStore;
this.attendeeCanonicalizer = attendeeCanonicalizer;
this.meetingWorkflowEngine = meetingWorkflowEngine ?? NoopMeetingWorkflowEngine.Instance;
this.ocrClient = ocrClient;
this.logger = logger;
}
@@ -206,13 +135,7 @@ public sealed partial class MeetingScreenshotService :
TrackOcr(
artifacts,
ocrCancellation,
RunOcrAsync(
artifacts,
screenshotPath,
screenshotId,
options,
ScreenshotOcrPolicy.CapturedScreenshot,
ocrCancellation.Token));
RunOcrAsync(artifacts, screenshotPath, screenshotId, options, ocrCancellation.Token));
}
logger.LogInformation(
@@ -289,100 +212,6 @@ public sealed partial class MeetingScreenshotService :
await WaitForPendingOcrAsync(artifacts, timeout, cancellationToken);
}
public async Task<MeetingScreenshotOcrRetryTriggerResult?> TriggerOcrRetryAsync(
MeetingSessionArtifacts artifacts,
string screenshotPath,
string screenshotId,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
if (string.IsNullOrWhiteSpace(screenshotId) ||
string.IsNullOrWhiteSpace(screenshotPath) ||
!File.Exists(screenshotPath) ||
!File.Exists(artifacts.AssistantContextPath))
{
return null;
}
var replaced = await ReplaceOcrPlaceholderAsync(
artifacts.AssistantContextPath,
screenshotId,
"_OCR retry pending..._",
cancellationToken,
preserveMarkers: true,
appendWhenMissing: false);
if (!replaced)
{
return null;
}
var ocrCancellation = new CancellationTokenSource();
TrackOcr(
artifacts,
ocrCancellation,
RunOcrAsync(
artifacts,
screenshotPath,
screenshotId,
options,
ScreenshotOcrPolicy.CapturedScreenshot,
ocrCancellation.Token));
return new MeetingScreenshotOcrRetryTriggerResult(
artifacts.AssistantContextPath,
screenshotPath,
screenshotId);
}
public async Task<MeetingNoteImageOcrQueueResult> ProcessMeetingNoteImageEmbedsAsync(
MeetingSessionArtifacts artifacts,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
if (!options.Screenshots.Ocr.Enabled || !File.Exists(artifacts.MeetingNotePath))
{
return new MeetingNoteImageOcrQueueResult(0);
}
var meetingNote = await meetingNoteStore.ReadAsync(artifacts.MeetingNotePath, cancellationToken);
var embeds = FindMeetingNoteImageEmbeds(
meetingNote.UserNotes,
artifacts.MeetingNotePath,
options);
var queuedEmbeds = new List<(MeetingNoteImageEmbed Embed, string OcrId)>();
foreach (var embed in embeds)
{
cancellationToken.ThrowIfCancellationRequested();
var ocrId = Guid.NewGuid().ToString("N");
await artifactStore.AppendAssistantContextAsync(
artifacts,
CreateMeetingNoteImageMarkdown(
ocrId,
embed.OriginalEmbed,
embed.ImagePath,
artifacts.AssistantContextPath),
cancellationToken);
queuedEmbeds.Add((embed, ocrId));
}
foreach (var (embed, ocrId) in queuedEmbeds)
{
cancellationToken.ThrowIfCancellationRequested();
var ocrCancellation = new CancellationTokenSource();
TrackOcr(
artifacts,
ocrCancellation,
RunOcrAsync(
artifacts,
embed.ImagePath,
ocrId,
options,
ScreenshotOcrPolicy.MeetingNoteImage,
ocrCancellation.Token));
}
return new MeetingNoteImageOcrQueueResult(queuedEmbeds.Count);
}
private async Task<string> SaveScreenshotAsync(
MeetingSessionArtifacts artifacts,
byte[] pngBytes,
@@ -405,7 +234,6 @@ public sealed partial class MeetingScreenshotService :
string screenshotPath,
string screenshotId,
MeetingAssistantOptions options,
ScreenshotOcrPolicy policy,
CancellationToken cancellationToken)
{
var timeout = options.Screenshots.Ocr.GetEffectiveTimeout();
@@ -420,27 +248,17 @@ public sealed partial class MeetingScreenshotService :
: options.Screenshots.Ocr.Prompt,
options,
timeoutSource.Token);
var cropMarkdown = policy.AllowCrop
? await TryCreateCropMarkdownAsync(
artifacts.AssistantContextPath,
screenshotPath,
result.Crop,
timeoutSource.Token)
: "";
var cropMarkdown = await TryCreateCropMarkdownAsync(
artifacts.AssistantContextPath,
screenshotPath,
result.Crop,
timeoutSource.Token);
await ReplaceOcrPlaceholderAsync(
artifacts.AssistantContextPath,
screenshotId,
cropMarkdown +
"### OCR" + Environment.NewLine + Environment.NewLine + result.Text.Trim(),
CancellationToken.None);
if (policy.AddAttendees)
{
await AddOcrAttendeesAsync(
artifacts,
result.Attendees,
options,
CancellationToken.None);
}
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
@@ -451,15 +269,8 @@ public sealed partial class MeetingScreenshotService :
await ReplaceOcrPlaceholderAsync(
artifacts.AssistantContextPath,
screenshotId,
CreateOcrFailureMarkdown(
artifacts,
screenshotPath,
screenshotId,
options,
$"_OCR timed out after {timeout:g}._",
policy.IncludeRetryLink),
CancellationToken.None,
preserveMarkers: true);
$"_OCR timed out after {timeout:g}._",
CancellationToken.None);
}
catch (Exception exception)
{
@@ -467,93 +278,16 @@ public sealed partial class MeetingScreenshotService :
await ReplaceOcrPlaceholderAsync(
artifacts.AssistantContextPath,
screenshotId,
CreateOcrFailureMarkdown(
artifacts,
screenshotPath,
screenshotId,
options,
$"_OCR failed: {exception.Message}_",
policy.IncludeRetryLink),
CancellationToken.None,
preserveMarkers: true);
$"_OCR failed: {exception.Message}_",
CancellationToken.None);
}
}
private async Task AddOcrAttendeesAsync(
MeetingSessionArtifacts artifacts,
IReadOnlyList<string> attendees,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
var additions = MeetingAttendeeNames.NormalizeDistinct(attendees);
if (additions.Count == 0)
{
return;
}
try
{
var meetingNote = await meetingNoteStore.ReadAsync(artifacts.MeetingNotePath, cancellationToken);
var transformedAdditions = await TransformAttendeesAsync(
artifacts,
additions,
options,
cancellationToken);
var canonicalized = await attendeeCanonicalizer.CanonicalizeAsync(
meetingNote.Frontmatter.Attendees.Concat(transformedAdditions).ToList(),
cancellationToken);
if (meetingNote.Frontmatter.Attendees.SequenceEqual(canonicalized, StringComparer.Ordinal))
{
return;
}
meetingNote.Frontmatter.Attendees = canonicalized.ToList();
await meetingNoteStore.SaveAsync(meetingNote, cancellationToken);
logger.LogInformation(
"Added {AttendeeCount} screenshot OCR attendee candidate(s) to meeting note {MeetingNotePath}",
additions.Count,
artifacts.MeetingNotePath);
}
catch (Exception exception) when (exception is not OperationCanceledException)
{
logger.LogWarning(
exception,
"Could not add screenshot OCR attendees to meeting note {MeetingNotePath}",
artifacts.MeetingNotePath);
}
}
private async Task<IReadOnlyList<string>> TransformAttendeesAsync(
MeetingSessionArtifacts artifacts,
IReadOnlyList<string> attendees,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
var transformed = new List<string>();
foreach (var attendee in attendees)
{
var value = await meetingWorkflowEngine.TransformAttendeeAsync(
MeetingWorkflowEvent.AttendeeAdded(artifacts, attendee),
options,
cancellationToken);
var normalized = MeetingAttendeeNames.NormalizeDisplayName(value);
if (!string.IsNullOrWhiteSpace(normalized) &&
!transformed.Contains(normalized, StringComparer.OrdinalIgnoreCase))
{
transformed.Add(normalized);
}
}
return transformed;
}
private async Task<bool> ReplaceOcrPlaceholderAsync(
private async Task ReplaceOcrPlaceholderAsync(
string assistantContextPath,
string screenshotId,
string replacement,
CancellationToken cancellationToken,
bool preserveMarkers = false,
bool appendWhenMissing = true)
CancellationToken cancellationToken)
{
var fileLock = contextFileLocks.GetOrAdd(
NormalizeContextKey(assistantContextPath),
@@ -563,41 +297,28 @@ public sealed partial class MeetingScreenshotService :
{
if (!File.Exists(assistantContextPath))
{
return false;
return;
}
var content = await File.ReadAllTextAsync(assistantContextPath, cancellationToken);
var startMarker = CreateOcrStartMarker(screenshotId);
var endMarker = CreateOcrEndMarker(screenshotId);
var startMarker = $"<!-- screenshot-ocr:{screenshotId} -->";
var endMarker = $"<!-- /screenshot-ocr:{screenshotId} -->";
var startIndex = content.IndexOf(startMarker, StringComparison.Ordinal);
var endIndex = content.IndexOf(endMarker, StringComparison.Ordinal);
if (startIndex < 0 || endIndex < startIndex)
{
if (appendWhenMissing)
{
await File.AppendAllTextAsync(
assistantContextPath,
Environment.NewLine + replacement.TrimEnd() + Environment.NewLine,
cancellationToken);
}
return false;
await File.AppendAllTextAsync(
assistantContextPath,
Environment.NewLine + replacement.TrimEnd() + Environment.NewLine,
cancellationToken);
return;
}
endIndex += endMarker.Length;
var updated = preserveMarkers
? content[..startIndex] +
startMarker +
Environment.NewLine +
replacement.TrimEnd() +
Environment.NewLine +
endMarker +
content[endIndex..]
: content[..startIndex] +
replacement.TrimEnd() +
content[endIndex..];
var updated = content[..startIndex] +
replacement.TrimEnd() +
content[endIndex..];
await File.WriteAllTextAsync(assistantContextPath, updated, cancellationToken);
return true;
}
finally
{
@@ -605,29 +326,6 @@ public sealed partial class MeetingScreenshotService :
}
}
private static string CreateOcrFailureMarkdown(
MeetingSessionArtifacts artifacts,
string screenshotPath,
string screenshotId,
MeetingAssistantOptions options,
string status,
bool includeRetryLink = true)
{
var markdown = status.TrimEnd();
if (includeRetryLink)
{
markdown += Environment.NewLine +
Environment.NewLine +
MeetingNoteActionLinks.CreateScreenshotOcrRetryLink(
options.Api.PublicBaseUrl,
artifacts,
screenshotPath,
screenshotId);
}
return markdown;
}
private async Task<string> TryCreateCropMarkdownAsync(
string assistantContextPath,
string screenshotPath,
@@ -737,189 +435,13 @@ public sealed partial class MeetingScreenshotService :
return content +
Environment.NewLine +
Environment.NewLine +
CreateOcrBlock(screenshotId, "_OCR pending..._");
}
private static string CreateMeetingNoteImageMarkdown(
string ocrId,
string originalEmbed,
string imagePath,
string assistantContextPath)
{
var relativePath = ToMarkdownPath(Path.GetRelativePath(
Path.GetDirectoryName(assistantContextPath)!,
imagePath));
return "## Meeting Note Image" +
$"<!-- screenshot-ocr:{screenshotId} -->" +
Environment.NewLine +
"Image from meeting note." +
"_OCR pending..._" +
Environment.NewLine +
$"Original embed: `{originalEmbed.Replace("`", "\\`", StringComparison.Ordinal)}`" +
Environment.NewLine +
$"![Meeting note image]({relativePath})" +
Environment.NewLine +
Environment.NewLine +
CreateOcrBlock(ocrId, "_OCR pending..._");
$"<!-- /screenshot-ocr:{screenshotId} -->";
}
private static string CreateOcrBlock(string screenshotId, string content)
{
return CreateOcrStartMarker(screenshotId) +
Environment.NewLine +
content.TrimEnd() +
Environment.NewLine +
CreateOcrEndMarker(screenshotId);
}
private static string CreateOcrStartMarker(string screenshotId)
{
return $"<!-- screenshot-ocr:{screenshotId} -->";
}
private static string CreateOcrEndMarker(string screenshotId)
{
return $"<!-- /screenshot-ocr:{screenshotId} -->";
}
private static IReadOnlyList<MeetingNoteImageEmbed> FindMeetingNoteImageEmbeds(
string userNotes,
string meetingNotePath,
MeetingAssistantOptions options)
{
if (string.IsNullOrWhiteSpace(userNotes))
{
return [];
}
var result = new List<MeetingNoteImageEmbed>();
var seen = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
foreach (Match match in ObsidianImageEmbedRegex().Matches(userNotes))
{
AddResolvedEmbed(result, seen, match.Value, match.Groups["target"].Value, meetingNotePath, options);
}
foreach (Match match in MarkdownImageEmbedRegex().Matches(userNotes))
{
AddResolvedEmbed(result, seen, match.Value, match.Groups["target"].Value, meetingNotePath, options);
}
return result;
}
private static void AddResolvedEmbed(
List<MeetingNoteImageEmbed> embeds,
HashSet<string> seen,
string originalEmbed,
string target,
string meetingNotePath,
MeetingAssistantOptions options)
{
var imagePath = ResolveMeetingNoteImagePath(target, meetingNotePath, options);
if (imagePath is null || !seen.Add(imagePath))
{
return;
}
embeds.Add(new MeetingNoteImageEmbed(originalEmbed, imagePath));
}
private static string? ResolveMeetingNoteImagePath(
string target,
string meetingNotePath,
MeetingAssistantOptions options)
{
var cleaned = CleanEmbedTarget(target);
if (string.IsNullOrWhiteSpace(cleaned) ||
cleaned.StartsWith("http://", StringComparison.OrdinalIgnoreCase) ||
cleaned.StartsWith("https://", StringComparison.OrdinalIgnoreCase) ||
cleaned.StartsWith("data:", StringComparison.OrdinalIgnoreCase) ||
!IsSupportedImagePath(cleaned))
{
return null;
}
var candidates = new List<string>();
if (Path.IsPathRooted(cleaned))
{
candidates.Add(VaultPath.Resolve(cleaned));
}
else
{
candidates.Add(Path.GetFullPath(Path.Combine(Path.GetDirectoryName(meetingNotePath)!, cleaned)));
candidates.Add(VaultPath.Resolve(options.Vault, cleaned));
}
foreach (var candidate in candidates)
{
if (File.Exists(candidate))
{
return candidate;
}
}
if (Path.GetFileName(cleaned).Equals(cleaned, StringComparison.Ordinal))
{
try
{
var vaultRoot = VaultPath.Resolve(options.Vault.BaseFolder);
return Directory.EnumerateFiles(vaultRoot, cleaned, SearchOption.AllDirectories)
.FirstOrDefault(IsSupportedImagePath);
}
catch (Exception exception) when (exception is IOException or UnauthorizedAccessException or DirectoryNotFoundException)
{
return null;
}
}
return null;
}
private static string CleanEmbedTarget(string target)
{
var cleaned = target.Trim();
if (cleaned.StartsWith("<", StringComparison.Ordinal) &&
cleaned.EndsWith(">", StringComparison.Ordinal) &&
cleaned.Length > 1)
{
cleaned = cleaned[1..^1].Trim();
}
var pipeIndex = cleaned.IndexOf('|', StringComparison.Ordinal);
if (pipeIndex >= 0)
{
cleaned = cleaned[..pipeIndex].Trim();
}
var hashIndex = cleaned.IndexOf('#', StringComparison.Ordinal);
if (hashIndex >= 0)
{
cleaned = cleaned[..hashIndex].Trim();
}
try
{
return Uri.UnescapeDataString(cleaned.Replace('/', Path.DirectorySeparatorChar));
}
catch (UriFormatException)
{
return cleaned.Replace('/', Path.DirectorySeparatorChar);
}
}
private static bool IsSupportedImagePath(string path)
{
return Path.GetExtension(path).ToLowerInvariant() switch
{
".png" or ".jpg" or ".jpeg" or ".gif" or ".webp" or ".bmp" or ".tif" or ".tiff" => true,
_ => false
};
}
[GeneratedRegex(@"!\[\[(?<target>[^\]\r\n]+)\]\]", RegexOptions.CultureInvariant)]
private static partial Regex ObsidianImageEmbedRegex();
[GeneratedRegex(@"!\[[^\]\r\n]*\]\((?<target>[^)\r\n]+)\)", RegexOptions.CultureInvariant)]
private static partial Regex MarkdownImageEmbedRegex();
private static string ResolveAttachmentsFolder(string assistantContextPath, string configuredFolder)
{
var expanded = Environment.ExpandEnvironmentVariables(
@@ -957,19 +479,4 @@ public sealed partial class MeetingScreenshotService :
}
private sealed record PendingOcrTask(Task Task, CancellationTokenSource Cancellation);
private sealed record ScreenshotOcrPolicy(bool AllowCrop, bool AddAttendees, bool IncludeRetryLink)
{
public static ScreenshotOcrPolicy CapturedScreenshot { get; } = new(
AllowCrop: true,
AddAttendees: true,
IncludeRetryLink: true);
public static ScreenshotOcrPolicy MeetingNoteImage { get; } = new(
AllowCrop: false,
AddAttendees: false,
IncludeRetryLink: false);
}
private sealed record MeetingNoteImageEmbed(string OriginalEmbed, string ImagePath);
}
@@ -63,13 +63,6 @@ public sealed class AzureSpeechSpeakerIdentityMatcher : ISpeakerIdentityMatcher
return null;
}
logger.LogInformation(
"Speaker identity matching {DiarizedSpeaker} composite layout: unknown {UnknownStart}-{UnknownEnd}, {KnownSegmentCount} known segment(s) across identity ids {IdentityIds}",
request.DiarizedSpeaker,
layout.UnknownSegment.Value.Start,
layout.UnknownSegment.Value.End,
layout.KnownSegments.Count,
string.Join(", ", layout.KnownSegments.Select(segment => segment.IdentityId).Distinct().Order()));
var segments = await DiarizeWithTimeoutAsync(tempPath, cancellationToken);
if (segments.Count == 0)
{
@@ -83,16 +76,6 @@ public sealed class AzureSpeechSpeakerIdentityMatcher : ISpeakerIdentityMatcher
"Speaker identity matching {DiarizedSpeaker} received {SegmentCount} diarized segment(s)",
request.DiarizedSpeaker,
segments.Count);
foreach (var segment in segments.OrderBy(segment => segment.Start))
{
logger.LogInformation(
"Speaker identity matching {DiarizedSpeaker} diarized turn {Start}-{End} as {Speaker}",
request.DiarizedSpeaker,
segment.Start,
segment.End,
segment.Speaker);
}
var unknownSpeaker = FindBestSpeaker(layout.UnknownSegment.Value, segments);
if (string.IsNullOrWhiteSpace(unknownSpeaker))
{
@@ -110,14 +93,6 @@ public sealed class AzureSpeechSpeakerIdentityMatcher : ISpeakerIdentityMatcher
unknownSpeaker,
StringComparison.Ordinal));
var requiredMatches = known.Count() > 1 ? 2 : 1;
logger.LogInformation(
"Speaker identity matching {DiarizedSpeaker} compared unknown speaker {UnknownSpeaker} with identity {IdentityId}: {MatchingSnippetCount}/{KnownSnippetCount} matching known snippet(s), required {RequiredMatches}",
request.DiarizedSpeaker,
unknownSpeaker,
known.Key,
matchingSnippetCount,
known.Count(),
requiredMatches);
if (matchingSnippetCount >= requiredMatches)
{
var validationRequest = new SpeakerIdentityMatchValidationRequest(
@@ -30,29 +30,14 @@ public sealed class PyannoteSpeakerIdentityMatchValidator : ISpeakerIdentityMatc
}
var segments = await DiarizeBytesAsync(wavBytes, cancellationToken);
var result = AnalyzeSingleSpeakerCoverage(segments);
if (!result.Accepted)
var result = HasSingleSpeaker(segments);
if (!result)
{
logger.LogInformation(
"pyannote rejected speaker identity sample because it did not contain one dominant speaker: segments {SegmentCount}, speakers {SpeakerCount}, dominant speaker {DominantSpeaker}, coverage {Coverage:P2}, required coverage {RequiredCoverage:P2}, duration {Duration}",
segments.Count,
result.SpeakerCount,
result.DominantSpeaker,
result.Coverage,
Math.Clamp(options.MinimumSingleSpeakerCoverage, 0, 1),
result.TotalDuration);
return false;
"pyannote rejected speaker identity sample because it did not contain one dominant speaker");
}
logger.LogInformation(
"pyannote accepted speaker identity sample: segments {SegmentCount}, speakers {SpeakerCount}, dominant speaker {DominantSpeaker}, coverage {Coverage:P2}, duration {Duration}, sample bytes {SampleBytes}",
segments.Count,
result.SpeakerCount,
result.DominantSpeaker,
result.Coverage,
result.TotalDuration,
wavBytes.Length);
return true;
return result;
}
public async Task<bool> ValidateMatchAsync(
@@ -78,20 +63,10 @@ public sealed class PyannoteSpeakerIdentityMatchValidator : ISpeakerIdentityMatc
TimeSpan.FromSeconds(1));
if (layout.UnknownSegment is null || layout.KnownSegments.Count == 0)
{
logger.LogInformation(
"pyannote rejected speaker identity match for {DiarizedSpeaker} and identity {IdentityId} because no readable composite snippets were available",
request.DiarizedSpeaker,
request.IdentityId);
return false;
}
var segments = await DiarizePathAsync(tempPath, cancellationToken);
logger.LogInformation(
"pyannote diarized speaker identity match for {DiarizedSpeaker} and identity {IdentityId}: {SegmentCount} segment(s), {KnownSnippetCount} known snippet(s)",
request.DiarizedSpeaker,
request.IdentityId,
segments.Count,
layout.KnownSegments.Count);
var unknownSpeaker = FindBestSpeaker(layout.UnknownSegment.Value, segments);
if (string.IsNullOrWhiteSpace(unknownSpeaker))
{
@@ -123,22 +98,11 @@ public sealed class PyannoteSpeakerIdentityMatchValidator : ISpeakerIdentityMatc
if (!accepted)
{
logger.LogInformation(
"pyannote rejected speaker identity match for {DiarizedSpeaker} and identity {IdentityId}: {Matching}/{Compared} known sample(s) matched, required ratio {RequiredRatio:P2}",
"pyannote rejected speaker identity match for {DiarizedSpeaker} and identity {IdentityId}: {Matching}/{Compared} known sample(s) matched",
request.DiarizedSpeaker,
request.IdentityId,
matching,
compared,
minimumRatio);
}
else
{
logger.LogInformation(
"pyannote accepted speaker identity match for {DiarizedSpeaker} and identity {IdentityId}: {Matching}/{Compared} known sample(s) matched, required ratio {RequiredRatio:P2}",
request.DiarizedSpeaker,
request.IdentityId,
matching,
compared,
minimumRatio);
compared);
}
return accepted;
@@ -198,7 +162,7 @@ public sealed class PyannoteSpeakerIdentityMatchValidator : ISpeakerIdentityMatc
cancellationToken);
}
private SingleSpeakerCoverage AnalyzeSingleSpeakerCoverage(IReadOnlyList<TranscriptionSegment> segments)
private bool HasSingleSpeaker(IReadOnlyList<TranscriptionSegment> segments)
{
var durationsBySpeaker = segments
.Where(segment => !string.IsNullOrWhiteSpace(segment.Speaker) && segment.End > segment.Start)
@@ -212,29 +176,17 @@ public sealed class PyannoteSpeakerIdentityMatchValidator : ISpeakerIdentityMatc
.ToList();
if (durationsBySpeaker.Count == 0)
{
return new SingleSpeakerCoverage(false, 0, null, 0, TimeSpan.Zero);
return false;
}
if (durationsBySpeaker.Count == 1)
{
return true;
}
var total = durationsBySpeaker.Sum(group => group.Duration);
var dominant = durationsBySpeaker.OrderByDescending(group => group.Duration).First();
var coverage = total > 0 ? dominant.Duration / total : 0;
if (durationsBySpeaker.Count == 1)
{
return new SingleSpeakerCoverage(
true,
durationsBySpeaker.Count,
dominant.Speaker,
coverage,
TimeSpan.FromSeconds(total));
}
var accepted = total > 0 && coverage >= Math.Clamp(options.MinimumSingleSpeakerCoverage, 0, 1);
return new SingleSpeakerCoverage(
accepted,
durationsBySpeaker.Count,
dominant.Speaker,
coverage,
TimeSpan.FromSeconds(total));
var dominant = durationsBySpeaker.Max(group => group.Duration);
return total > 0 && dominant / total >= Math.Clamp(options.MinimumSingleSpeakerCoverage, 0, 1);
}
private CompositeLayout WriteCompositeWav(
@@ -282,11 +234,4 @@ public sealed class PyannoteSpeakerIdentityMatchValidator : ISpeakerIdentityMatc
}
private sealed record CompositeLayout(TimeSegment? UnknownSegment, IReadOnlyList<TimeSegment> KnownSegments);
private sealed record SingleSpeakerCoverage(
bool Accepted,
int SpeakerCount,
string? DominantSpeaker,
double Coverage,
TimeSpan TotalDuration);
}
@@ -114,7 +114,11 @@ public sealed class PassthroughSpeakerIdentityAttendeeCanonicalizer : ISpeakerId
IReadOnlyList<string> attendees,
CancellationToken cancellationToken)
{
var distinctAttendees = MeetingAttendeeNames.NormalizeDistinct(attendees);
var distinctAttendees = attendees
.Select(attendee => attendee.Trim())
.Where(name => !string.IsNullOrWhiteSpace(name))
.Distinct(StringComparer.OrdinalIgnoreCase)
.ToList();
return Task.FromResult<IReadOnlyList<string>>(distinctAttendees);
}
}
@@ -58,97 +58,53 @@ public sealed class SpeakerIdentityMergeService : ISpeakerIdentityMergeService
.ToHashSet();
var mergedPairs = 0;
var attempts = 0;
logger.LogInformation(
"Speaker identity merge diagnostics started: cutoff {Cutoff}, recent identity ids {RecentIdentityIds}, candidate identity count {CandidateIdentityCount}",
cutoff,
FormatIds(recentIds),
identities.Count);
foreach (var sourceId in recentIds.ToList())
{
var source = identities.SingleOrDefault(identity => identity.Id == sourceId);
if (source is null || source.Snippets.Count == 0)
{
logger.LogInformation(
"Speaker identity merge diagnostics skipped source identity {SourceIdentityId} because it was missing or had no snippets",
sourceId);
continue;
}
var targets = identities
.Where(identity => identity.Id != source.Id && identity.Snippets.Count > 0)
.ToList();
logger.LogInformation(
"Speaker identity merge diagnostics evaluating source identity {SourceIdentityId} ({SourceName}) with {SnippetCount} snippet(s) against target ids {TargetIdentityIds}",
source.Id,
source.GetDisplayName(),
source.Snippets.Count,
FormatIds(targets.Select(target => target.Id)));
foreach (var batch in targets.Chunk(Math.Max(1, options.MatchBatchSize)))
{
var firstSnippet = SelectSnippet(source, excludedSnippet: null);
if (firstSnippet is null)
{
logger.LogInformation(
"Speaker identity merge diagnostics stopped evaluating source identity {SourceIdentityId} because no first snippet was available",
source.Id);
break;
}
attempts++;
logger.LogInformation(
"Speaker identity merge diagnostics round 1 attempt {Attempt} for source identity {SourceIdentityId}: target ids {TargetIdentityIds}, source snippet {SnippetId}",
attempts,
source.Id,
FormatIds(batch.Select(target => target.Id)),
firstSnippet.Id);
var firstMatch = await matcher.MatchAsync(
CreateRequest(source, firstSnippet, batch),
cancellationToken);
if (firstMatch is null || firstMatch.IdentityId == source.Id)
{
logger.LogInformation(
"Speaker identity merge diagnostics round 1 found no usable target for source identity {SourceIdentityId}",
source.Id);
continue;
}
var target = batch.SingleOrDefault(identity => identity.Id == firstMatch.IdentityId);
if (target is null)
{
logger.LogInformation(
"Speaker identity merge diagnostics round 1 matched identity {IdentityId}, but it was not in the current target batch",
firstMatch.IdentityId);
continue;
}
var secondSnippet = SelectSnippet(source, excludedSnippet: firstSnippet);
if (secondSnippet is null)
{
logger.LogInformation(
"Speaker identity merge diagnostics could not validate source identity {SourceIdentityId} against target identity {TargetIdentityId} because no second source snippet was available",
source.Id,
target.Id);
continue;
}
attempts++;
logger.LogInformation(
"Speaker identity merge diagnostics round 2 attempt {Attempt} for source identity {SourceIdentityId}: target identity {TargetIdentityId}, source snippet {SnippetId}",
attempts,
source.Id,
target.Id,
secondSnippet.Id);
var secondMatch = await matcher.MatchAsync(
CreateRequest(source, secondSnippet, batch),
cancellationToken);
if (secondMatch?.IdentityId != target.Id)
{
logger.LogInformation(
"Speaker identity merge diagnostics rejected merge source identity {SourceIdentityId} into target identity {TargetIdentityId}: second match was {SecondMatchIdentityId}",
source.Id,
target.Id,
secondMatch?.IdentityId);
continue;
}
@@ -158,12 +114,6 @@ public sealed class SpeakerIdentityMergeService : ISpeakerIdentityMergeService
target,
source,
options.MaxSnippetsPerSpeaker);
logger.LogInformation(
"Speaker identity merge diagnostics merging source identity {SourceIdentityId} ({SourceName}) into target identity {TargetIdentityId} ({TargetName})",
source.Id,
sourceName,
target.Id,
targetName);
await SpeakerIdentityTranscriptAudit.AppendMergedAsync(
target.References,
targetName,
@@ -214,13 +164,4 @@ public sealed class SpeakerIdentityMergeService : ISpeakerIdentityMergeService
.FirstOrDefault(snippet => excludedSnippet is null || snippet.Id != excludedSnippet.Id);
}
private static string FormatIds(IEnumerable<int> ids)
{
var formatted = ids
.Distinct()
.Order()
.Select(id => id.ToString(System.Globalization.CultureInfo.InvariantCulture))
.ToList();
return formatted.Count == 0 ? "<none>" : string.Join(", ", formatted);
}
}
@@ -67,55 +67,21 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
var sourceLabel = sourceSpeaker.Trim();
var targetName = targetSpeaker.Trim();
logger.LogInformation(
"Applying speaker override from {SourceSpeaker} to {TargetSpeaker} for meeting {MeetingNotePath}",
sourceLabel,
targetName,
request.MeetingNote.Path);
await using var context = await dbContextFactory.CreateDbContextAsync(cancellationToken);
await SpeakerIdentitySchema.EnsureCreatedOrUpdatedAsync(context, cancellationToken);
var now = DateTimeOffset.UtcNow;
var meetingReference = CreateReference(request.MeetingNote, now);
var snippetResolution = await ResolveOverrideSnippetAsync(request, sourceLabel, cancellationToken);
var snippet = snippetResolution.WavBytes;
if (snippet.Length > 0 && !await matchValidator.ValidateSampleAsync(snippet, cancellationToken))
{
logger.LogInformation(
"Speaker override from {SourceSpeaker} to {TargetSpeaker} rejected source sample after secondary validation: sample source {SampleSource}, sample bytes {SampleBytes}",
sourceLabel,
targetName,
snippetResolution.Source,
snippet.Length);
snippet = [];
}
var snippet = await ResolveOverrideSnippetAsync(request, sourceLabel, cancellationToken);
var target = await FindIdentityByAcceptedNameAsync(context, targetName, cancellationToken);
var sourceCandidate = await FindCurrentRunCandidateAsync(
context,
meetingReference,
targetName,
cancellationToken);
logger.LogInformation(
"Speaker override from {SourceSpeaker} to {TargetSpeaker} resolved sample source {SampleSource}, sample bytes {SampleBytes}, target identity {TargetIdentityId}, source candidate identity {SourceCandidateIdentityId}",
sourceLabel,
targetName,
snippetResolution.Source,
snippet.Length,
target?.Id,
sourceCandidate?.Id);
if (target is null)
{
if (sourceCandidate is null && snippet.Length == 0)
{
logger.LogWarning(
"Skipping speaker override from {SourceSpeaker} to {TargetSpeaker} because no source sample could be resolved",
sourceLabel,
targetName);
return;
}
target = sourceCandidate ?? new SpeakerIdentity
{
CreatedAt = now,
@@ -124,19 +90,10 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
if (target.Id == 0)
{
context.SpeakerIdentities.Add(target);
logger.LogInformation(
"Creating speaker identity from override for {TargetSpeaker} using source {SourceSpeaker}",
targetName,
sourceLabel);
}
}
else if (sourceCandidate is not null && sourceCandidate.Id != target.Id)
{
logger.LogInformation(
"Merging override source candidate identity {SourceCandidateIdentityId} into target identity {TargetIdentityId} for {TargetSpeaker}",
sourceCandidate.Id,
target.Id,
targetName);
MergeOverrideCandidate(target, sourceCandidate);
context.SpeakerIdentities.Remove(sourceCandidate);
}
@@ -145,16 +102,8 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
target.UpdatedAt = now;
ResetCandidates(target, [targetName]);
AddMeetingReference(target, meetingReference);
var snippetAdded = AddSnippetIfNeeded(target, snippet);
AddSnippetIfNeeded(target, snippet);
await context.SaveChangesAsync(cancellationToken);
logger.LogInformation(
"Applied speaker override from {SourceSpeaker} to {TargetSpeaker}: identity {IdentityId}, snippet added {SnippetAdded}, snippet count {SnippetCount}, reference count {ReferenceCount}",
sourceLabel,
targetName,
target.Id,
snippetAdded,
target.Snippets.Count,
target.References.Count);
await SpeakerIdentityTranscriptAudit.AppendIdentifiedAsync(
target.References,
sourceLabel,
@@ -176,20 +125,9 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
var target = await FindIdentityByAcceptedNameAsync(context, identity.Trim(), cancellationToken);
if (target is null)
{
logger.LogInformation(
"Speaker identity deletion skipped because {IdentityName} was not found",
identity.Trim());
return;
}
logger.LogInformation(
"Deleting speaker identity {IdentityId} ({IdentityName}) with {SnippetCount} snippet(s), {CandidateCount} candidate(s), {AliasCount} alias(es), {ReferenceCount} reference(s)",
target.Id,
target.GetDisplayName() ?? identity.Trim(),
target.Snippets.Count,
target.CandidateNames.Count,
target.Aliases.Count,
target.References.Count);
context.SpeakerIdentities.Remove(target);
await context.SaveChangesAsync(cancellationToken);
}
@@ -233,15 +171,6 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
matchedAcceptedNames.Add(name);
}
logger.LogInformation(
"Speaker identity processing started in {Mode} mode for meeting {MeetingNotePath}: {SegmentCount} segment(s), {SampleCount} supplied sample(s), {KnownMappingCount} known mapping(s), attendees {Attendees}",
mode,
request.MeetingNote.Path,
request.Segments.Count,
request.Samples?.Count ?? 0,
knownSpeakerMappings.Count,
FormatNames(attendees));
var unmatchedSpeakers = new List<(string Speaker, byte[] Snippet)>();
var samplesBySpeaker = request.Samples?
.Where(sample => !string.IsNullOrWhiteSpace(sample.Speaker) && sample.WavBytes.Length > 0)
@@ -251,12 +180,6 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
group => group.OrderByDescending(sample => sample.Score).Select(sample => sample.WavBytes).First(),
StringComparer.OrdinalIgnoreCase)
?? new Dictionary<string, byte[]>(StringComparer.OrdinalIgnoreCase);
if (samplesBySpeaker.Count > 0)
{
logger.LogInformation(
"Speaker identity processing has supplied samples for {SampleSpeakers}",
string.Join(", ", samplesBySpeaker.Select(pair => $"{pair.Key}:{pair.Value.Length} bytes")));
}
foreach (var group in request.Segments
.Where(segment => !string.IsNullOrWhiteSpace(segment.Speaker))
@@ -266,45 +189,26 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
var speaker = group.Key;
if (knownDiarizedSpeakers.Contains(speaker) || alreadyIdentifiedNames.Contains(speaker))
{
logger.LogInformation(
"Speaker identity processing skipped {Speaker} because it is already known or identified",
speaker);
continue;
}
if (speakerMappings.ContainsKey(speaker))
{
logger.LogInformation(
"Speaker identity processing skipped {Speaker} because a mapping already exists to {MappedSpeaker}",
speaker,
speakerMappings[speaker]);
continue;
}
var snippetSource = "supplied";
var snippet = samplesBySpeaker.TryGetValue(speaker, out var suppliedSnippet)
? suppliedSnippet
: [];
if (snippet.Length == 0)
{
snippetSource = "extracted";
(snippet, _) = await ExtractBestContinuousSampleAsync(
request,
speaker,
"Speaker identity processing",
: await snippetExtractor.ExtractSnippetAsync(
request.AudioPath,
SpeakerSampleSpanSelector.SelectBestContinuousSpan(
request.Segments,
speaker,
options.MaximumSampleSegmentGap,
options.MinimumSampleSpeechDuration),
cancellationToken);
}
logger.LogInformation(
"Speaker identity processing resolved {SampleSource} sample for {Speaker}: {SampleBytes} byte(s)",
snippetSource,
speaker,
snippet.Length);
if (snippet.Length == 0)
{
logger.LogInformation(
"Speaker identity processing cannot match or learn {Speaker} because no usable sample was available",
speaker);
unmatchedSpeakers.Add((speaker, snippet));
continue;
}
@@ -318,9 +222,6 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
cancellationToken);
if (match is null)
{
logger.LogInformation(
"Speaker identity processing found no existing identity match for {Speaker}",
speaker);
unmatchedSpeakers.Add((speaker, snippet));
continue;
}
@@ -328,10 +229,6 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
var identity = await LoadIdentityAsync(context, match.IdentityId, cancellationToken);
if (identity is null)
{
logger.LogWarning(
"Speaker identity processing matched {Speaker} to identity {IdentityId}, but the identity could not be loaded",
speaker,
match.IdentityId);
unmatchedSpeakers.Add((speaker, snippet));
continue;
}
@@ -341,15 +238,6 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
var previousCanonicalName = identity.CanonicalName;
AddMeetingReference(identity, meetingReference);
UpdateMatchedIdentity(identity, attendees, snippet);
logger.LogInformation(
"Speaker identity processing updated matched identity {IdentityId} for {Speaker}: previous canonical {PreviousCanonicalName}, canonical {CanonicalName}, candidates {CandidateNames}, snippets {SnippetCount}, references {ReferenceCount}",
identity.Id,
speaker,
previousCanonicalName,
identity.CanonicalName,
FormatNames(identity.CandidateNames.Select(candidate => candidate.Name)),
identity.Snippets.Count,
identity.References.Count);
if (string.IsNullOrWhiteSpace(previousCanonicalName) &&
!string.IsNullOrWhiteSpace(identity.CanonicalName))
{
@@ -382,34 +270,18 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
}
}
IReadOnlyList<SpeakerIdentity> learnedIdentities = [];
if (mode == SpeakerIdentityProcessingMode.Final)
{
learnedIdentities = await LearnUnmatchedSpeakersAsync(
await LearnUnmatchedSpeakersAsync(
context,
attendees,
matchedAcceptedNames,
unmatchedSpeakers,
meetingReference,
cancellationToken);
logger.LogInformation(
"Speaker identity processing queued {LearnedIdentityCount} new unmatched speaker identity candidate(s)",
learnedIdentities.Count);
}
await context.SaveChangesAsync(cancellationToken);
if (learnedIdentities.Count > 0)
{
logger.LogInformation(
"Speaker identity processing saved learned identity candidate ids {IdentityIds}",
FormatIds(learnedIdentities.Select(identity => identity.Id)));
}
logger.LogInformation(
"Speaker identity processing completed in {Mode} mode for meeting {MeetingNotePath}: {MappingCount} mapping(s), {AttendeeMatchCount} attendee match(es)",
mode,
request.MeetingNote.Path,
speakerMappings.Count,
attendeeMatches.Count);
var relabeledSegments = request.Segments
.Select(segment => speakerMappings.TryGetValue(segment.Speaker, out var speakerName)
@@ -452,16 +324,8 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
.Select(candidate => candidate.Identity)
.ToList();
logger.LogInformation(
"Speaker identity match candidate selection for {Speaker}: {CandidateCount} candidate(s) after active/attendee/name filters, ids {IdentityIds}",
speaker,
identities.Count,
FormatIds(identities.Select(identity => identity.Id)));
var round = 0;
foreach (var batch in identities.Chunk(Math.Max(1, options.MatchBatchSize)))
{
round++;
var request = new SpeakerIdentityMatchRequest(
speaker,
snippet,
@@ -471,31 +335,17 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
identity.ReferenceCount,
identity.Snippets.Select(storedSnippet => storedSnippet.WavBytes).ToList()))
.ToList());
logger.LogInformation(
"Speaker identity match round {Round} for {Speaker}: candidate identity ids {IdentityIds}",
round,
speaker,
FormatIds(request.Candidates.Select(candidate => candidate.IdentityId)));
var match = await matcher.MatchAsync(request, cancellationToken);
if (match is not null)
{
logger.LogInformation(
"Speaker identity match round {Round} for {Speaker} matched identity {IdentityId}",
round,
speaker,
match.IdentityId);
return match;
}
}
logger.LogInformation(
"Speaker identity match candidate selection for {Speaker} completed with no match after {RoundCount} round(s)",
speaker,
round);
return null;
}
private async Task<SpeakerSnippetResolution> ResolveOverrideSnippetAsync(
private async Task<byte[]> ResolveOverrideSnippetAsync(
SpeakerIdentificationRequest request,
string sourceSpeaker,
CancellationToken cancellationToken)
@@ -506,55 +356,15 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
.FirstOrDefault();
if (sample is not null)
{
return new SpeakerSnippetResolution(
sample.WavBytes,
"supplied-sample",
SegmentCount: 1,
sample.Score,
sample.Segment.End - sample.Segment.Start);
return sample.WavBytes;
}
var segments = request.Segments
.Where(segment => string.Equals(segment.Speaker, sourceSpeaker, StringComparison.OrdinalIgnoreCase))
.ToList();
if (segments.Count == 0)
{
return new SpeakerSnippetResolution([], "missing-segments", SegmentCount: 0, Score: null, Duration: TimeSpan.Zero);
}
var (snippet, span) = await ExtractBestContinuousSampleAsync(
request,
sourceSpeaker,
"Speaker override",
cancellationToken);
return new SpeakerSnippetResolution(
snippet,
"extracted-from-recording",
span.Count,
Score: null,
SpeakerSampleSpanSelector.SpanDuration(span));
}
private async Task<(byte[] Snippet, IReadOnlyList<TranscriptionSegment> Span)> ExtractBestContinuousSampleAsync(
SpeakerIdentificationRequest request,
string speaker,
string operation,
CancellationToken cancellationToken)
{
var span = SpeakerSampleSpanSelector.SelectBestContinuousSpan(
request.Segments,
speaker,
options.MaximumSampleSegmentGap,
options.MinimumSampleSpeechDuration);
logger.LogInformation(
"{Operation} extracting fallback sample for {Speaker}: selected {SegmentCount} segment(s), span {SpanDuration}, minimum {MinimumDuration}",
operation,
speaker,
span.Count,
SpeakerSampleSpanSelector.SpanDuration(span),
options.MinimumSampleSpeechDuration);
var snippet = await snippetExtractor.ExtractSnippetAsync(request.AudioPath, span, cancellationToken);
return (snippet, span);
return segments.Count == 0
? []
: await snippetExtractor.ExtractSnippetAsync(request.AudioPath, segments, cancellationToken);
}
private static async Task<SpeakerIdentity?> FindIdentityByAcceptedNameAsync(
@@ -602,10 +412,6 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
SpeakerIdentity target,
SpeakerIdentity source)
{
logger.LogInformation(
"Speaker override candidate merge started: source identity {SourceIdentityId} into target identity {TargetIdentityId}",
source.Id,
target.Id);
AddAlias(target, source.CanonicalName);
foreach (var alias in source.Aliases)
{
@@ -626,12 +432,6 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
{
AddSnippetIfNeeded(target, snippet.WavBytes);
}
logger.LogInformation(
"Speaker override candidate merge completed: target identity {TargetIdentityId} now has {AliasCount} alias(es), {SnippetCount} snippet(s), {ReferenceCount} reference(s)",
target.Id,
target.Aliases.Count,
target.Snippets.Count,
target.References.Count);
}
private static void AddAlias(SpeakerIdentity identity, string? alias)
@@ -717,36 +517,22 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
if (intersection.Count == 0)
{
logger.LogInformation(
"Speaker identity candidate elimination for identity {IdentityId} had empty intersection; resetting candidates to attendees {Attendees} and replacing oldest snippet",
identity.Id,
FormatNames(attendees));
ResetCandidates(identity, attendees);
ReplaceOldestSnippet(identity, snippet);
return;
}
logger.LogInformation(
"Speaker identity candidate elimination for identity {IdentityId}: candidates {CurrentCandidates}, attendees {Attendees}, intersection {Intersection}",
identity.Id,
FormatNames(currentCandidates),
FormatNames(attendees),
FormatNames(intersection));
ResetCandidates(identity, intersection);
if (intersection.Count == 1)
{
identity.CanonicalName = intersection[0];
logger.LogInformation(
"Speaker identity candidate elimination promoted identity {IdentityId} to canonical name {CanonicalName}",
identity.Id,
identity.CanonicalName);
}
}
AddSnippetIfNeeded(identity, snippet);
}
private async Task<IReadOnlyList<SpeakerIdentity>> LearnUnmatchedSpeakersAsync(
private async Task LearnUnmatchedSpeakersAsync(
SpeakerIdentityDbContext context,
IReadOnlyList<string> attendees,
IEnumerable<string> matchedCanonicalNames,
@@ -760,26 +546,16 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
.ToList();
if (remainingCandidates.Count == 0)
{
logger.LogInformation(
"Speaker identity candidate learning skipped because no attendee candidates remain. Matched names {MatchedNames}",
FormatNames(matchedCanonicalNames));
return [];
return;
}
logger.LogInformation(
"Speaker identity candidate learning started for {UnmatchedSpeakerCount} unmatched speaker(s) with remaining candidate names {CandidateNames}",
unmatchedSpeakers.Count,
FormatNames(remainingCandidates));
var learned = new List<SpeakerIdentity>();
foreach (var (speaker, snippet) in unmatchedSpeakers.Where(speaker => speaker.Snippet.Length > 0))
{
if (!await matchValidator.ValidateSampleAsync(snippet, cancellationToken))
{
logger.LogInformation(
"Skipping speaker identity candidate for {Speaker} because secondary validation rejected the sample: candidate names {CandidateNames}, sample bytes {SampleBytes}",
speaker,
FormatNames(remainingCandidates),
snippet.Length);
"Skipping speaker identity candidate for {Speaker} because secondary validation rejected the sample",
speaker);
continue;
}
@@ -810,14 +586,6 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
]
};
context.SpeakerIdentities.Add(identity);
learned.Add(identity);
logger.LogInformation(
"Created speaker identity candidate for {Speaker}: canonical {CanonicalName}, candidate names {CandidateNames}, sample bytes {SampleBytes}, meeting reference {MeetingNotePath}",
speaker,
canonicalName,
FormatNames(remainingCandidates),
snippet.Length,
meetingReference.MeetingNotePath);
if (!string.IsNullOrWhiteSpace(canonicalName))
{
await SpeakerIdentityTranscriptAudit.AppendIdentifiedAsync(
@@ -828,22 +596,14 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
}
}
foreach (var (speaker, _) in unmatchedSpeakers.Where(speaker => speaker.Snippet.Length == 0))
{
logger.LogInformation(
"Skipping speaker identity candidate for {Speaker} because no sample was available: candidate names {CandidateNames}",
speaker,
FormatNames(remainingCandidates));
}
return learned;
await Task.CompletedTask;
}
private bool AddSnippetIfNeeded(SpeakerIdentity identity, byte[] snippet)
private void AddSnippetIfNeeded(SpeakerIdentity identity, byte[] snippet)
{
if (snippet.Length == 0 || identity.Snippets.Count >= options.MaxSnippetsPerSpeaker)
{
return false;
return;
}
identity.Snippets.Add(new SpeakerSnippet
@@ -851,7 +611,6 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
WavBytes = snippet,
CreatedAt = DateTimeOffset.UtcNow
});
return true;
}
private static void ReplaceOldestSnippet(SpeakerIdentity identity, byte[] snippet)
@@ -932,34 +691,6 @@ public sealed class SpeakerIdentityService : ISpeakerIdentificationService
SpeakerIdentityReferences.AddIfMissing(identity, reference, DateTimeOffset.UtcNow);
}
private static string FormatNames(IEnumerable<string?> names)
{
var formatted = names
.Where(name => !string.IsNullOrWhiteSpace(name))
.Select(name => name!.Trim())
.Distinct(StringComparer.OrdinalIgnoreCase)
.Order(StringComparer.OrdinalIgnoreCase)
.ToList();
return formatted.Count == 0 ? "<none>" : string.Join(", ", formatted);
}
private static string FormatIds(IEnumerable<int> ids)
{
var formatted = ids
.Distinct()
.Order()
.Select(id => id.ToString(System.Globalization.CultureInfo.InvariantCulture))
.ToList();
return formatted.Count == 0 ? "<none>" : string.Join(", ", formatted);
}
private sealed record SpeakerSnippetResolution(
byte[] WavBytes,
string Source,
int SegmentCount,
double? Score,
TimeSpan Duration);
private enum SpeakerIdentityProcessingMode
{
LiveReadOnly,
@@ -7,7 +7,9 @@ public sealed record BoundMeetingProject(string Name, string Path);
public sealed class BoundMeetingProjectResolver
{
private static readonly IDeserializer YamlDeserializer = FrontmatterYamlDeserializer.Create();
private static readonly IDeserializer YamlDeserializer = new DeserializerBuilder()
.IgnoreUnmatchedProperties()
.Build();
private readonly MeetingAssistantOptions options;
@@ -1,26 +0,0 @@
using System.Text.Json.Nodes;
using Microsoft.Extensions.AI;
namespace MeetingAssistant.Summary;
internal static class FunctionInvocationGuard
{
public static ValueTask<object?> InvokeAsync(
FunctionInvocationContext context,
CancellationToken cancellationToken)
{
if (context.CallContent.Exception is not null)
{
return ValueTask.FromResult<object?>(new JsonObject
{
["error"] = new JsonObject
{
["code"] = "invalid_tool_arguments",
["message"] = "Tool arguments must be a valid JSON object."
}
});
}
return context.Function.InvokeAsync(context.Arguments, cancellationToken);
}
}
@@ -1,36 +1,26 @@
using System.Net.Http.Headers;
using System.ClientModel;
using System.ClientModel.Primitives;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
using Microsoft.Agents.AI.Compaction;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Logging;
using OpenAI;
using OpenAI.Responses;
namespace MeetingAssistant.Summary;
#pragma warning disable MAAI001
#pragma warning disable OPENAI001
public sealed class LiteLlmResponsesChatClient : IChatClient
{
private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web);
private readonly HttpClient httpClient;
private readonly ResponsesClient responsesClient;
private readonly AsyncLocal<string?> requestInitiator = new();
private readonly string model;
private readonly bool useStreaming;
private readonly bool enableThinking;
private readonly string reasoningEffort;
private readonly int reconnectionAttempts;
private readonly TimeSpan reconnectionDelay;
private readonly LiteLlmResponsesCompactionOptions? compactionOptions;
private readonly ILogger? logger;
private readonly Action? retrying;
private readonly Action<string>? reasoningSummaryChanged;
private int responsesRequestCount;
public LiteLlmResponsesChatClient(
@@ -43,10 +33,7 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
TimeSpan reconnectionDelay,
LiteLlmResponsesCompactionOptions? compactionOptions = null,
ILogger? logger = null,
bool firstRequestIsUser = true,
Action? retrying = null,
Action<string>? reasoningSummaryChanged = null,
bool useStreaming = true)
bool firstRequestIsUser = true)
: this(
new HttpClient { BaseAddress = NormalizeEndpoint(endpoint) },
apiKey,
@@ -57,10 +44,7 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
reconnectionDelay,
compactionOptions,
logger,
firstRequestIsUser,
retrying,
reasoningSummaryChanged,
useStreaming)
firstRequestIsUser)
{
}
@@ -74,24 +58,17 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
TimeSpan reconnectionDelay,
LiteLlmResponsesCompactionOptions? compactionOptions = null,
ILogger? logger = null,
bool firstRequestIsUser = true,
Action? retrying = null,
Action<string>? reasoningSummaryChanged = null,
bool useStreaming = true)
bool firstRequestIsUser = true)
{
this.httpClient = httpClient;
this.httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", apiKey);
responsesClient = CreateResponsesClient(httpClient, apiKey, requestInitiator);
this.model = model;
this.useStreaming = useStreaming;
this.enableThinking = enableThinking;
this.reasoningEffort = reasoningEffort;
this.reconnectionAttempts = Math.Max(0, reconnectionAttempts);
this.reconnectionDelay = reconnectionDelay > TimeSpan.Zero ? reconnectionDelay : TimeSpan.Zero;
this.compactionOptions = compactionOptions;
this.logger = logger;
this.retrying = retrying;
this.reasoningSummaryChanged = reasoningSummaryChanged;
responsesRequestCount = firstRequestIsUser ? 0 : 1;
}
@@ -100,50 +77,19 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
httpClient.Dispose();
}
internal static LiteLlmResponsesChatClient Create(
AgentOptions options,
string apiKey,
LiteLlmResponsesCompactionOptions? compactionOptions,
ILogger? logger,
bool firstRequestIsUser,
Action? retrying = null,
Action<string>? reasoningSummaryChanged = null)
{
return new LiteLlmResponsesChatClient(
new Uri(options.Endpoint),
apiKey,
options.Model,
options.EnableThinking,
options.ReasoningEffort switch
{
ReasoningEffortOption.None => "none",
ReasoningEffortOption.Low => "low",
ReasoningEffortOption.High => "high",
ReasoningEffortOption.ExtraHigh => "xhigh",
_ => "medium"
},
options.ReconnectionAttempts,
options.ReconnectionDelay,
compactionOptions,
logger,
firstRequestIsUser,
retrying,
reasoningSummaryChanged,
options.UseStreaming);
}
public async Task<ChatResponse> GetResponseAsync(
IEnumerable<ChatMessage> messages,
ChatOptions? options = null,
CancellationToken cancellationToken = default)
{
var payload = await CreateCompactedPayloadAsync(messages.ToList(), options, cancellationToken).ConfigureAwait(false);
var result = await PostWithRetryAsync(payload, cancellationToken).ConfigureAwait(false);
ReportVisibleReasoningSummaries(result.ReasoningSummaries);
LogResponseDiagnostics(result.Response);
ThrowIfResponseHasNoContent(result.Response);
LogResponseUsage(result.Response.Usage);
return result.Response;
var responseJson = await PostWithRetryAsync(payload, cancellationToken).ConfigureAwait(false);
var response = ParseResponseJson(responseJson);
LogResponseDiagnostics(responseJson, response);
ThrowIfResponseHasNoContent(responseJson, response);
LogResponseUsage(response.Usage);
return response;
}
public async IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
@@ -170,36 +116,45 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
: null;
}
private static IReadOnlyList<string> ExtractVisibleReasoningSummaries(ChatResponse response)
public static ChatResponse ParseResponseJson(string responseJson)
{
return response.Messages
.SelectMany(message => message.Contents)
.OfType<TextReasoningContent>()
.Select(content => content.Text)
.Where(text => !string.IsNullOrWhiteSpace(text))
.ToArray();
}
using var document = JsonDocument.Parse(responseJson);
var root = document.RootElement;
var contents = new List<AIContent>();
private void ReportVisibleReasoningSummaries(IReadOnlyList<string> summaries)
{
if (reasoningSummaryChanged is null)
if (root.TryGetProperty("output", out var output) && output.ValueKind == JsonValueKind.Array)
{
return;
}
foreach (var summary in summaries)
{
try
foreach (var item in output.EnumerateArray())
{
reasoningSummaryChanged(summary);
}
catch (Exception exception)
{
logger?.LogWarning(
exception,
"Reasoning summary callback failed; continuing with the LiteLLM response.");
var type = GetString(item, "type");
if (type == "message")
{
AddMessageContent(contents, item);
}
else if (type == "function_call")
{
contents.Add(new FunctionCallContent(
GetRequiredString(item, "call_id"),
GetRequiredString(item, "name"),
ParseArguments(GetString(item, "arguments"))));
}
}
}
var message = new ChatMessage
{
Role = ChatRole.Assistant,
Contents = contents
};
return new ChatResponse(message)
{
ResponseId = GetString(root, "id"),
ModelId = GetString(root, "model"),
CreatedAt = GetUnixTimestamp(root, "created_at"),
Usage = ParseUsage(root),
RawRepresentation = responseJson
};
}
private async Task<JsonObject> CreateCompactedPayloadAsync(
@@ -385,9 +340,7 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
return payload;
}
private async Task<ResponsesChatResult> PostWithRetryAsync(
JsonObject payload,
CancellationToken cancellationToken)
private async Task<string> PostWithRetryAsync(JsonObject payload, CancellationToken cancellationToken)
{
var payloadJson = payload.ToJsonString(JsonOptions);
Exception? lastException = null;
@@ -398,91 +351,38 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
{
var initiator = NextInitiator();
LogRequestDiagnostics(payload, initiator, attempt);
using var request = CreateJsonRequest(
"responses",
payloadJson,
initiator);
using var response = await httpClient.SendAsync(request, cancellationToken).ConfigureAwait(false);
var responseJson = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
var createOptions = ModelReaderWriter.Read<CreateResponseOptions>(
BinaryData.FromString(payloadJson),
ModelReaderWriterOptions.Json)
?? throw new InvalidOperationException("Unable to create OpenAI Responses request options.");
createOptions.StreamingEnabled = useStreaming;
if (response.IsSuccessStatusCode)
{
return responseJson;
}
var previousInitiator = requestInitiator.Value;
requestInitiator.Value = initiator;
try
var exception = new InvalidOperationException(
$"LiteLLM Responses request failed with {(int)response.StatusCode} {response.ReasonPhrase}: {responseJson}");
if (!IsRetryableStatusCode((int)response.StatusCode) || attempt == reconnectionAttempts)
{
return useStreaming
? await GetStreamingResponseAsync(createOptions, cancellationToken).ConfigureAwait(false)
: CreateNonStreamingResult((await responsesClient
.CreateResponseAsync(createOptions, cancellationToken)
.ConfigureAwait(false))
.Value
.AsChatResponse(createOptions));
throw exception;
}
finally
{
requestInitiator.Value = previousInitiator;
}
}
catch (ClientResultException exception)
when (!IsRetryableStatusCode(exception.Status) || attempt == reconnectionAttempts)
{
throw new InvalidOperationException(
$"LiteLLM Responses request failed with {exception.Status}: {exception.Message}",
exception);
lastException = exception;
}
catch (Exception exception) when (IsRetryableException(exception) && attempt < reconnectionAttempts)
{
lastException = exception;
}
retrying?.Invoke();
await DelayBeforeRetryAsync(cancellationToken).ConfigureAwait(false);
}
throw lastException ?? new InvalidOperationException("LiteLLM Responses request failed.");
}
private async Task<ResponsesChatResult> GetStreamingResponseAsync(
CreateResponseOptions createOptions,
CancellationToken cancellationToken)
{
var updates = new List<ChatResponseUpdate>();
var reasoningSummaries = new Dictionary<(string ItemId, int SummaryIndex), StringBuilder>();
await foreach (var update in responsesClient
.CreateResponseStreamingAsync(createOptions, cancellationToken)
.AsChatResponseUpdatesAsync(createOptions, cancellationToken)
.ConfigureAwait(false))
{
updates.Add(update);
var reasoningUpdate = update.RawRepresentation
as StreamingResponseReasoningSummaryTextDeltaUpdate;
if (reasoningUpdate is null)
{
continue;
}
var key = (reasoningUpdate.ItemId, reasoningUpdate.SummaryIndex);
if (!reasoningSummaries.TryGetValue(key, out var summary))
{
summary = new StringBuilder();
reasoningSummaries[key] = summary;
}
summary.Append(reasoningUpdate.Delta);
}
return new ResponsesChatResult(
updates.ToChatResponse(),
reasoningSummaries.Values
.Select(summary => summary.ToString())
.Where(summary => !string.IsNullOrWhiteSpace(summary))
.ToArray());
}
private static ResponsesChatResult CreateNonStreamingResult(ChatResponse response)
{
return new ResponsesChatResult(response, ExtractVisibleReasoningSummaries(response));
}
private HttpRequestMessage CreateJsonRequest(
string requestUri,
string payloadJson,
@@ -521,9 +421,7 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
private static bool IsRetryableException(Exception exception)
{
return exception is ClientResultException clientException
&& IsRetryableStatusCode(clientException.Status)
|| exception is HttpRequestException or TaskCanceledException;
return exception is HttpRequestException or TaskCanceledException;
}
private void LogContextWindow(int estimatedTokens, bool compacted, string source)
@@ -595,18 +493,18 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
instructionsPreview);
}
private void LogResponseDiagnostics(ChatResponse response)
private void LogResponseDiagnostics(string responseJson, ChatResponse response)
{
if (logger is null)
{
return;
}
var outputTypes = GetOutputItemTypes(responseJson);
var parsedTypes = response.Messages
.SelectMany(message => message.Contents)
.Select(content => content.GetType().Name)
.ToArray();
var outputTypes = GetOutputItemTypes(response);
var functionCalls = response.Messages
.SelectMany(message => message.Contents)
.OfType<FunctionCallContent>()
@@ -624,29 +522,40 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
if (parsedTypes.Length == 0)
{
logger.LogWarning(
"LiteLLM Responses response contained no parseable message text or function calls.");
"LiteLLM Responses response contained no parseable message text or function calls. Raw response preview: {ResponsePreview}",
Truncate(responseJson, maxLength: 6000));
}
}
private static void ThrowIfResponseHasNoContent(ChatResponse response)
private static void ThrowIfResponseHasNoContent(string responseJson, ChatResponse response)
{
if (response.Messages.SelectMany(message => message.Contents).Any())
{
return;
}
var outputTypes = GetOutputItemTypes(response);
var outputTypes = GetOutputItemTypes(responseJson);
throw new InvalidOperationException(
"LiteLLM Responses returned no parseable message text or function calls. " +
$"ResponseId={response.ResponseId ?? "<none>"}, outputTypes=[{string.Join(", ", outputTypes)}].");
}
private static string[] GetOutputItemTypes(ChatResponse response)
private static string[] GetOutputItemTypes(string responseJson)
{
return response.Messages
.SelectMany(message => message.Contents)
.Select(content => content.RawRepresentation?.GetType().Name ?? content.GetType().Name)
.ToArray();
try
{
using var document = JsonDocument.Parse(responseJson);
return document.RootElement.TryGetProperty("output", out var output) && output.ValueKind == JsonValueKind.Array
? output
.EnumerateArray()
.Select(item => GetString(item, "type") ?? item.ValueKind.ToString())
.ToArray()
: [];
}
catch (JsonException)
{
return ["<invalid-json>"];
}
}
private static string Truncate(string value, int maxLength)
@@ -672,19 +581,10 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
if (!string.IsNullOrWhiteSpace(text))
{
var isAssistant = role == ChatRole.Assistant.Value;
input.Add(new JsonObject
{
["type"] = "message",
["role"] = isAssistant ? "assistant" : "user",
["content"] = new JsonArray
{
new JsonObject
{
["type"] = isAssistant ? "output_text" : "input_text",
["text"] = text
}
}
["role"] = role == ChatRole.Assistant.Value ? "assistant" : "user",
["content"] = text
});
}
@@ -697,7 +597,7 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
["type"] = "function_call",
["call_id"] = call.CallId,
["name"] = call.Name,
["arguments"] = SerializeFunctionArguments(call)
["arguments"] = JsonSerializer.Serialize(call.Arguments, JsonOptions)
});
}
else if (content is FunctionResultContent result)
@@ -734,15 +634,50 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
return result;
}
private static string SerializeFunctionArguments(FunctionCallContent call)
private static void AddMessageContent(List<AIContent> contents, JsonElement item)
{
if (call.RawRepresentation is FunctionCallResponseItem responseItem
&& responseItem.FunctionArguments is not null)
if (!item.TryGetProperty("content", out var messageContent) || messageContent.ValueKind != JsonValueKind.Array)
{
return responseItem.FunctionArguments.ToString();
return;
}
return JsonSerializer.Serialize(call.Arguments, JsonOptions);
foreach (var content in messageContent.EnumerateArray())
{
if (GetString(content, "type") == "output_text")
{
var text = GetString(content, "text");
if (!string.IsNullOrEmpty(text))
{
contents.Add(new TextContent(text));
}
}
}
}
private static Dictionary<string, object?> ParseArguments(string? arguments)
{
if (string.IsNullOrWhiteSpace(arguments))
{
return [];
}
using var document = JsonDocument.Parse(arguments);
return document.RootElement.EnumerateObject()
.ToDictionary(property => property.Name, property => ConvertJsonValue(property.Value));
}
private static object? ConvertJsonValue(JsonElement element)
{
return element.ValueKind switch
{
JsonValueKind.String => element.GetString(),
JsonValueKind.Number when element.TryGetInt64(out var integer) => integer,
JsonValueKind.Number when element.TryGetDouble(out var number) => number,
JsonValueKind.True => true,
JsonValueKind.False => false,
JsonValueKind.Null => null,
_ => JsonSerializer.Deserialize<object>(element.GetRawText(), JsonOptions)
};
}
private static string ResultToString(object? result)
@@ -755,12 +690,57 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
};
}
private static UsageDetails? ParseUsage(JsonElement root)
{
if (!root.TryGetProperty("usage", out var usage) || usage.ValueKind != JsonValueKind.Object)
{
return null;
}
var inputTokens = GetInt(usage, "input_tokens") ?? GetInt(usage, "prompt_tokens");
var outputTokens = GetInt(usage, "output_tokens") ?? GetInt(usage, "completion_tokens");
var totalTokens = GetInt(usage, "total_tokens");
return new UsageDetails
{
InputTokenCount = inputTokens,
OutputTokenCount = outputTokens,
TotalTokenCount = totalTokens
};
}
private static int EstimateTokens(JsonObject payload)
{
var json = payload.ToJsonString(JsonOptions);
return Math.Max(1, (int)Math.Ceiling(json.Length / 4.0));
}
private static int? GetInt(JsonElement element, string propertyName)
{
return element.TryGetProperty(propertyName, out var property) && property.TryGetInt32(out var value)
? value
: null;
}
private static string GetRequiredString(JsonElement element, string propertyName)
{
return GetString(element, propertyName)
?? throw new InvalidOperationException($"LiteLLM response item did not include '{propertyName}'.");
}
private static string? GetString(JsonElement element, string propertyName)
{
return element.TryGetProperty(propertyName, out var property) && property.ValueKind == JsonValueKind.String
? property.GetString()
: null;
}
private static DateTimeOffset? GetUnixTimestamp(JsonElement element, string propertyName)
{
return element.TryGetProperty(propertyName, out var property) && property.TryGetInt64(out var value)
? DateTimeOffset.FromUnixTimeSeconds(value)
: null;
}
private static Uri NormalizeEndpoint(Uri endpoint)
{
var value = endpoint.ToString().TrimEnd('/');
@@ -776,60 +756,5 @@ public sealed class LiteLlmResponsesChatClient : IChatClient
{
return value.TrimStart('/');
}
private static ResponsesClient CreateResponsesClient(
HttpClient httpClient,
string apiKey,
AsyncLocal<string?> requestInitiator)
{
var options = new OpenAIClientOptions
{
Endpoint = httpClient.BaseAddress
?? throw new InvalidOperationException("LiteLLM HTTP client requires a base address."),
Transport = new HttpClientPipelineTransport(httpClient),
RetryPolicy = new ClientRetryPolicy(0)
};
options.AddPolicy(
new InitiatorHeaderPolicy(requestInitiator),
PipelinePosition.PerCall);
return new ResponsesClient(
new ApiKeyCredential(apiKey),
options);
}
private sealed record ResponsesChatResult(
ChatResponse Response,
IReadOnlyList<string> ReasoningSummaries);
private sealed class InitiatorHeaderPolicy(AsyncLocal<string?> requestInitiator) : PipelinePolicy
{
public override void Process(
PipelineMessage message,
IReadOnlyList<PipelinePolicy> pipeline,
int currentIndex)
{
SetHeader(message);
ProcessNext(message, pipeline, currentIndex);
}
public override ValueTask ProcessAsync(
PipelineMessage message,
IReadOnlyList<PipelinePolicy> pipeline,
int currentIndex)
{
SetHeader(message);
return ProcessNextAsync(message, pipeline, currentIndex);
}
private void SetHeader(PipelineMessage message)
{
if (!string.IsNullOrWhiteSpace(requestInitiator.Value))
{
message.Request.Headers.Set("X-Initiator", requestInitiator.Value);
}
}
}
}
#pragma warning restore OPENAI001
#pragma warning restore MAAI001
@@ -5,7 +5,9 @@ namespace MeetingAssistant.Summary;
internal static class MeetingSummaryFrontmatterFactory
{
private static readonly IDeserializer YamlDeserializer = FrontmatterYamlDeserializer.Create();
private static readonly IDeserializer YamlDeserializer = new DeserializerBuilder()
.IgnoreUnmatchedProperties()
.Build();
public static async Task<MeetingArtifactFrontmatter> CreateAsync(
MeetingSessionArtifacts artifacts,
@@ -22,7 +24,7 @@ internal static class MeetingSummaryFrontmatterFactory
artifacts.SummaryPath,
meetingNote,
cancellationToken);
frontmatter.Projects = CopyNonEmptyProjectList(meetingNote.Frontmatter.Projects);
frontmatter.Projects = CopyNonEmptyList(meetingNote.Frontmatter.Projects);
return frontmatter;
}
@@ -40,16 +42,6 @@ internal static class MeetingSummaryFrontmatterFactory
return values.Count == 0 ? null : values.ToList();
}
private static List<string>? CopyNonEmptyProjectList(IEnumerable<string> values)
{
var projects = values
.Where(value => !string.IsNullOrWhiteSpace(value))
.Select(value => value.Trim())
.Distinct(StringComparer.OrdinalIgnoreCase)
.ToList();
return projects.Count == 0 ? null : projects;
}
private static async Task<List<string>?> ReadExistingSummaryAttendeesAsync(
string summaryPath,
CancellationToken cancellationToken)
@@ -10,11 +10,11 @@ public sealed class MeetingSummaryInstructionBuilder : IMeetingSummaryInstructio
Use the provided tools to read the meeting transcript, assistant context, user notes, glossary, project files, and scoped past project meeting summaries.
All read tools can return the whole file or a clamped inclusive line range when from and to are supplied; use ranges for large inputs before asking for more lines.
Then write the finished meeting summary as markdown by calling write_summary. The write_summary call requires an oneliner parameter: keep it very short, blurb-like, and concise. No line breaks. Complete sentences are not necessary; conciseness is more important. write_summary returns the written summary filename so project journal entries can link to it.
Then write the finished meeting summary as markdown by calling write_summary. The write_summary call requires an oneliner parameter: keep it very short, blurb-like, and concise. No line breaks. Complete sentences are not necessary; conciseness is more important.
write_summary is only for the current meeting's summary file. Past project meeting summaries are read-only historical context; use list_past_project_meetings and read_past_project_meeting_summary to inspect them, and never try to mutate them.
If the meeting note has no title, or still has a generated default title like `Meeting yyyy-MM-dd HH:mm`, provide a concise title parameter to write_summary when the purpose of the meeting is clear from transcript, user notes, or assistant context. If the purpose is not clear, omit the title parameter.
If the meeting note has no title, provide a concise title parameter to write_summary.
Use read_meetingnote to inspect frontmatter such as title, attendees, projects, start_time, and end_time.
Treat the assistant context as persistent meeting-specific memory and use read_context and write_context as your own meeting notebook. Its frontmatter may include agenda from the calendar appointment. Whenever you encounter unexpected problems, discover missing information, or make assumptions while summarizing, use write_context to append a concise note so later agents working on this meeting can understand them. Also record useful internal notes, requests for future tools, suggested improvements, and relevant context discovered from other sources. write_context appends by default; use replace_file=true only when you are intentionally replacing the whole assistant context body. Keep user-facing summary content in the summary note.
Use read_context and write_context as your own meeting notebook. Its frontmatter may include agenda from the calendar appointment. Record useful internal notes, missing context, requests for future tools, suggested improvements, and relevant context discovered from other sources. write_context appends by default; use replace_file=true only when you are intentionally replacing the whole assistant context body. Keep user-facing summary content in the summary note.
Use add_dictation_word when project context, user notes, or transcript evidence show that a domain term, acronym, name, or unusual word is likely to be repeatedly mistranscribed. Add only the canonical spelling, one term at a time.
Use add_attendee and remove_attendee to sharpen the meeting attendees list from clear transcript evidence and screenshot OCR participant evidence. Treat OCR that says visible people are a partial screenshot result as incomplete evidence; do not remove attendees solely because they are absent from a partial screenshot.
Use override_speaker only when you are very certain that a transcript speaker label belongs to a named person, for example from user notes, OCR evidence with a matching timestamp, or very clear context cues. Provide the exact speaker label from the transcript and the replacement speaker name. If the replacement speaker already exists in the transcript, use merge=true only when you are certain both speaker labels are the same identity; otherwise do not merge them.
@@ -22,7 +22,7 @@ public sealed class MeetingSummaryInstructionBuilder : IMeetingSummaryInstructio
After writing the summary, update existing project files when the meeting produced durable project knowledge, decisions, next steps, or context.
Use list_projects first to see which projects are bound to this meeting. Use search, list_past_project_meetings, read_past_project_meeting_summary, and read_projectfile before changing existing project files. search includes both project files and past meeting summaries for the requested current-meeting project scope.
The summary note should contain concise sections for summary, decisions, open questions, and next steps.
If the assistant context contains cropped screenshot markdown links, include only the most relevant cropped screenshots in the summary by markdown-linking them near the related summary text. When embedding them, encode spaces in image-link targets as `%20`; never leave literal spaces in the link target. Do not include every cropped screenshot by default, and do not link uncropped screenshots unless no cropped version exists and the image is important.
If the assistant context contains cropped screenshot markdown links, include only the most relevant cropped screenshots in the summary by markdown-linking them near the related summary text. Do not include every cropped screenshot by default, and do not link uncropped screenshots unless no cropped version exists and the image is important.
Keep the output grounded in the source material and explicitly say when a section has no known items.
""";
+232 -61
View File
@@ -1,20 +1,20 @@
using MeetingAssistant.MeetingNotes;
using MeetingAssistant.Speakers;
using MeetingAssistant.Transcription;
using MeetingAssistant.Workflow;
using YamlDotNet.Serialization;
namespace MeetingAssistant.Summary;
public sealed class MeetingSummaryTools
{
private static readonly IDeserializer YamlDeserializer = FrontmatterYamlDeserializer.Create();
private static readonly IDeserializer YamlDeserializer = new DeserializerBuilder()
.IgnoreUnmatchedProperties()
.Build();
private readonly MeetingSessionArtifacts artifacts;
private readonly MeetingAssistantOptions options;
private readonly IDictationWordStore? dictationWordStore;
private readonly SummaryAgentWriteAudit? writeAudit;
private readonly IMeetingWorkflowEngine meetingWorkflowEngine;
private readonly BoundMeetingProjectResolver projectResolver;
public MeetingSummaryTools(MeetingSessionArtifacts artifacts)
@@ -26,33 +26,31 @@ public sealed class MeetingSummaryTools
MeetingSessionArtifacts artifacts,
MeetingAssistantOptions options,
IDictationWordStore? dictationWordStore = null,
SummaryAgentWriteAudit? writeAudit = null,
IMeetingWorkflowEngine? meetingWorkflowEngine = null)
SummaryAgentWriteAudit? writeAudit = null)
{
this.artifacts = artifacts;
this.options = options;
this.dictationWordStore = dictationWordStore;
this.writeAudit = writeAudit;
this.meetingWorkflowEngine = meetingWorkflowEngine ?? NoopMeetingWorkflowEngine.Instance;
projectResolver = new BoundMeetingProjectResolver(options);
}
public Task<string> ReadTranscript(int? @from = null, int? to = null, int? tail = null)
public Task<string> ReadTranscript(int? @from = null, int? to = null)
{
return ReadIfExistsAsync(artifacts.TranscriptPath, @from, to, tail);
return ReadIfExistsAsync(artifacts.TranscriptPath, @from, to);
}
public Task<string> ReadMeetingNote(int? @from = null, int? to = null, int? tail = null)
public Task<string> ReadMeetingNote(int? @from = null, int? to = null)
{
return ReadIfExistsAsync(artifacts.MeetingNotePath, @from, to, tail);
return ReadIfExistsAsync(artifacts.MeetingNotePath, @from, to);
}
public Task<string> ReadContext(int? @from = null, int? to = null, int? tail = null)
public Task<string> ReadContext(int? @from = null, int? to = null)
{
return ReadIfExistsAsync(artifacts.AssistantContextPath, @from, to, tail);
return ReadIfExistsAsync(artifacts.AssistantContextPath, @from, to);
}
public async Task<string> ReadUserNotes(int? @from = null, int? to = null, int? tail = null)
public async Task<string> ReadUserNotes(int? @from = null, int? to = null)
{
if (!File.Exists(artifacts.MeetingNotePath))
{
@@ -62,12 +60,12 @@ public sealed class MeetingSummaryTools
var content = await File.ReadAllTextAsync(artifacts.MeetingNotePath);
var document = MarkdownDocumentParser.SplitOptional(content);
var body = document.HasFrontmatter ? document.Body.Trim() : content;
return AgentFileToolContent.ReadLines(body, @from, to, tail);
return ReadLines(body, @from, to);
}
public Task<string> ReadGlossary(int? @from = null, int? to = null, int? tail = null)
public Task<string> ReadGlossary(int? @from = null, int? to = null)
{
return ReadIfExistsAsync(VaultPath.Resolve(options.Vault, options.Vault.DictationWordsPath), @from, to, tail);
return ReadIfExistsAsync(VaultPath.Resolve(options.Vault, options.Vault.DictationWordsPath), @from, to);
}
public async Task<string> AddDictationWord(string word)
@@ -102,11 +100,7 @@ public sealed class MeetingSummaryTools
return "Refused: attendee must not be empty.";
}
var displayName = await meetingWorkflowEngine.TransformAttendeeAsync(
MeetingWorkflowEvent.AttendeeAdded(artifacts, attendee.Trim()),
options,
CancellationToken.None);
displayName = displayName.Trim();
var displayName = attendee.Trim();
var normalized = MeetingAttendeeNames.NormalizeDisplayName(displayName);
if (string.IsNullOrWhiteSpace(normalized))
{
@@ -225,14 +219,9 @@ public sealed class MeetingSummaryTools
Directory.CreateDirectory(Path.GetDirectoryName(artifacts.SummaryPath)!);
var meetingNote = await ReadMeetingNoteAsync();
var trimmedTitle = title?.Trim();
if (!string.IsNullOrWhiteSpace(trimmedTitle))
{
meetingNote.Frontmatter.Title = trimmedTitle;
await WriteMeetingNoteAsync(meetingNote);
}
var summaryTitle = meetingNote.Frontmatter.Title;
var summaryTitle = string.IsNullOrWhiteSpace(meetingNote.Frontmatter.Title)
? title
: meetingNote.Frontmatter.Title;
var frontmatter = await MeetingSummaryFrontmatterFactory.CreateAsync(
artifacts,
meetingNote,
@@ -243,7 +232,7 @@ public sealed class MeetingSummaryTools
artifacts.SummaryPath,
MeetingArtifactFrontmatterRenderer.Render(frontmatter, markdown));
SummaryWasWritten = true;
return Path.GetFileName(artifacts.SummaryPath);
return artifacts.SummaryPath;
}
private static bool ContainsLineBreak(string value)
@@ -261,7 +250,7 @@ public sealed class MeetingSummaryTools
int? insert = null,
bool replace_file = false)
{
var editMode = AgentFileEditMode.Create(@from, to, insert, replace_file);
var editMode = FileLineEditMode.Create(@from, to, insert, replace_file);
if (editMode is null)
{
return "Refused: supply either both from and to for replacement, insert for insertion, or no line arguments for append; set replace_file=true only for whole-file replacement.";
@@ -274,7 +263,7 @@ public sealed class MeetingSummaryTools
var existing = await File.ReadAllTextAsync(artifacts.AssistantContextPath);
var (frontmatter, body) = MeetingArtifactFrontmatterRenderer.Split(existing);
var updatedBody = AgentFileToolContent.ApplyLineEdit(body, StripAccidentalFrontmatter(content), editMode);
var updatedBody = ApplyLineEdit(body, StripAccidentalFrontmatter(content), editMode);
var updated = string.IsNullOrWhiteSpace(frontmatter)
? updatedBody
: "---" + Environment.NewLine + frontmatter + Environment.NewLine + "---" + Environment.NewLine + Environment.NewLine + updatedBody;
@@ -304,12 +293,12 @@ public sealed class MeetingSummaryTools
}
var files = Directory.EnumerateFiles(projectRoot, "*", SearchOption.AllDirectories)
.Select(path => AgentFileToolContent.ToToolPath(Path.GetRelativePath(projectRoot, path)))
.Select(path => ToToolPath(Path.GetRelativePath(projectRoot, path)))
.Order(StringComparer.Ordinal);
return string.Join('\n', files);
}
public async Task<string> ReadProjectFile(string project, string path, int? @from = null, int? to = null, int? tail = null)
public async Task<string> ReadProjectFile(string project, string path, int? @from = null, int? to = null)
{
var filePath = await ResolveBoundProjectFilePathAsync(project, path);
if (filePath is null || !File.Exists(filePath))
@@ -317,7 +306,7 @@ public sealed class MeetingSummaryTools
return "";
}
return AgentFileToolContent.ReadLines(await File.ReadAllTextAsync(filePath), @from, to, tail);
return ReadLines(await File.ReadAllTextAsync(filePath), @from, to);
}
public async Task<string> ListPastProjectMeetings(
@@ -349,7 +338,7 @@ public sealed class MeetingSummaryTools
return string.Join('\n', lines);
}
public async Task<string> ReadPastProjectMeetingSummary(string path, int? @from = null, int? to = null, int? tail = null)
public async Task<string> ReadPastProjectMeetingSummary(string path, int? @from = null, int? to = null)
{
var summary = await ResolvePastSummaryAsync(path);
if (summary.Refusal is not null)
@@ -357,7 +346,7 @@ public sealed class MeetingSummaryTools
return summary.Refusal;
}
return AgentFileToolContent.ReadLines(await File.ReadAllTextAsync(summary.Summary!.Path), @from, to, tail);
return ReadLines(await File.ReadAllTextAsync(summary.Summary!.Path), @from, to);
}
public async Task<string> WriteProjectFile(
@@ -369,7 +358,7 @@ public sealed class MeetingSummaryTools
int? insert = null,
bool replace_file = false)
{
var editMode = AgentFileEditMode.Create(@from, to, insert, replace_file);
var editMode = FileLineEditMode.Create(@from, to, insert, replace_file);
if (editMode is null)
{
return "Refused: supply either both from and to for replacement, insert for insertion, or no line arguments for append; set replace_file=true only for whole-file replacement.";
@@ -386,17 +375,17 @@ public sealed class MeetingSummaryTools
{
await writeAudit.CaptureFileWriteAsync(
target.Path,
() => AgentFileToolContent.WriteFileContentAsync(target.Path, content, editMode));
() => WriteProjectFileContentAsync(target.Path, content, editMode));
}
else
{
await AgentFileToolContent.WriteFileContentAsync(target.Path, content, editMode);
await WriteProjectFileContentAsync(target.Path, content, editMode);
}
return $"{target.Project.Name}/{AgentFileToolContent.ToToolPath(path)}";
return $"{target.Project.Name}/{ToToolPath(path)}";
}
public async Task<string> Search(string keywords, string[]? projects = null, string? file_pattern = null)
public async Task<string> Search(string keywords, string[]? projects = null)
{
if (string.IsNullOrWhiteSpace(keywords))
{
@@ -411,10 +400,10 @@ public sealed class MeetingSummaryTools
var selectedProjects = await GetExistingProjectsInScopeAsync(scope.ProjectNames);
var summaries = await GetScopedPastSummariesAsync(scope.ProjectNames);
return SearchSources(selectedProjects, summaries, keywords, selectedProjects.Count == 1, file_pattern);
return SearchSources(selectedProjects, summaries, keywords, selectedProjects.Count == 1);
}
private static async Task<string> ReadIfExistsAsync(string path, int? from = null, int? to = null, int? tail = null)
private static async Task<string> ReadIfExistsAsync(string path, int? from = null, int? to = null)
{
if (!File.Exists(path))
{
@@ -422,7 +411,7 @@ public sealed class MeetingSummaryTools
}
var content = await File.ReadAllTextAsync(path);
return AgentFileToolContent.ReadLines(content, from, to, tail);
return ReadLines(content, from, to);
}
private async Task<MeetingNote> ReadMeetingNoteAsync()
@@ -586,7 +575,7 @@ public sealed class MeetingSummaryTools
}
var fullPath = Path.GetFullPath(Path.Combine(projectRoot, path));
return AgentFileToolContent.IsWithinDirectory(projectRoot, fullPath) ? fullPath : null;
return IsWithinDirectory(projectRoot, fullPath) ? fullPath : null;
}
private ProjectFileTarget? ResolveExistingProjectFilePath(string project, string path)
@@ -613,11 +602,147 @@ public sealed class MeetingSummaryTools
}
var fullPath = Path.GetFullPath(Path.Combine(projectFolder.Path, path));
return AgentFileToolContent.IsWithinDirectory(projectFolder.Path, fullPath)
return IsWithinDirectory(projectFolder.Path, fullPath)
? new ProjectFileTarget(projectFolder, fullPath)
: null;
}
private static async Task WriteProjectFileContentAsync(
string filePath,
string content,
FileLineEditMode editMode)
{
if (editMode.Kind == FileLineEditKind.Overwrite)
{
await File.WriteAllTextAsync(filePath, content);
return;
}
if (editMode.Kind == FileLineEditKind.Append)
{
var existing = File.Exists(filePath)
? await File.ReadAllTextAsync(filePath)
: "";
await File.WriteAllTextAsync(filePath, AppendContent(existing, content));
return;
}
var lines = File.Exists(filePath)
? (await File.ReadAllLinesAsync(filePath)).ToList()
: [];
var replacementLines = SplitLines(content);
if (editMode.Kind == FileLineEditKind.Insert)
{
var insertIndex = Math.Clamp(editMode.Insert!.Value - 1, 0, lines.Count);
lines.InsertRange(insertIndex, replacementLines);
await File.WriteAllTextAsync(filePath, string.Join('\n', lines));
return;
}
var startIndex = Math.Clamp(editMode.From!.Value - 1, 0, lines.Count);
var endIndexExclusive = Math.Clamp(editMode.To!.Value, 0, lines.Count);
if (endIndexExclusive < startIndex)
{
endIndexExclusive = startIndex;
}
lines.RemoveRange(startIndex, endIndexExclusive - startIndex);
lines.InsertRange(startIndex, replacementLines);
await File.WriteAllTextAsync(filePath, string.Join('\n', lines));
}
private static List<string> SplitLines(string content)
{
if (content.Length == 0)
{
return [];
}
return content
.Replace("\r\n", "\n", StringComparison.Ordinal)
.Replace('\r', '\n')
.Split('\n')
.ToList();
}
private static string ReadLines(string content, int? from = null, int? to = null)
{
if (!from.HasValue && !to.HasValue)
{
return content;
}
var lines = SplitLines(content);
if (lines.Count == 0)
{
return "";
}
var start = Math.Clamp((from ?? 1) - 1, 0, lines.Count - 1);
var end = Math.Clamp((to ?? lines.Count) - 1, 0, lines.Count - 1);
if (end < start)
{
return "";
}
return string.Join('\n', lines.GetRange(start, end - start + 1));
}
private static string ApplyLineEdit(
string existingContent,
string replacementContent,
FileLineEditMode editMode)
{
if (editMode.Kind == FileLineEditKind.Overwrite)
{
return replacementContent;
}
if (editMode.Kind == FileLineEditKind.Append)
{
return AppendContent(existingContent, replacementContent);
}
var lines = SplitLines(existingContent);
var replacementLines = SplitLines(replacementContent);
if (editMode.Kind == FileLineEditKind.Insert)
{
var insertIndex = Math.Clamp(editMode.Insert!.Value - 1, 0, lines.Count);
lines.InsertRange(insertIndex, replacementLines);
return string.Join('\n', lines);
}
var startIndex = Math.Clamp(editMode.From!.Value - 1, 0, lines.Count);
var endIndexExclusive = Math.Clamp(editMode.To!.Value, 0, lines.Count);
if (endIndexExclusive < startIndex)
{
endIndexExclusive = startIndex;
}
lines.RemoveRange(startIndex, endIndexExclusive - startIndex);
lines.InsertRange(startIndex, replacementLines);
return string.Join('\n', lines);
}
private static string AppendContent(string existingContent, string content)
{
if (string.IsNullOrEmpty(existingContent))
{
return content;
}
if (string.IsNullOrEmpty(content))
{
return existingContent;
}
var separator = existingContent.EndsWith('\n') || existingContent.EndsWith('\r')
? ""
: "\n";
return existingContent + separator + content;
}
private Task<List<BoundMeetingProject>> GetBoundProjectsAsync()
{
return projectResolver.GetBoundProjectsAsync(artifacts);
@@ -632,8 +757,7 @@ public sealed class MeetingSummaryTools
IReadOnlyList<BoundMeetingProject> projects,
IReadOnlyList<PastMeetingSummary> summaries,
string keywords,
bool singleProject,
string? filePattern)
bool singleProject)
{
System.Text.RegularExpressions.Regex regex;
try
@@ -652,13 +776,7 @@ public sealed class MeetingSummaryTools
{
foreach (var file in Directory.EnumerateFiles(project.Path, "*", SearchOption.AllDirectories))
{
var projectRelative = AgentFileToolContent.ToToolPath(Path.GetRelativePath(project.Path, file));
if (!AgentFileToolContent.MatchesGlob(projectRelative, filePattern))
{
continue;
}
var relative = Path.Combine(project.Name, projectRelative);
var relative = Path.Combine(project.Name, Path.GetRelativePath(project.Path, file));
AddFileMatches(matches, regex, file, FormatSearchPath(relative, singleProject));
}
}
@@ -687,14 +805,14 @@ public sealed class MeetingSummaryTools
{
if (regex.IsMatch(lines[index]))
{
matches.Add($"{AgentFileToolContent.ToToolPath(formattedPath)}:{index + 1} {lines[index]}");
matches.Add($"{ToToolPath(formattedPath)}:{index + 1} {lines[index]}");
}
}
}
private static string FormatSearchPath(string path, bool singleProject)
{
var formatted = AgentFileToolContent.ToToolPath(path);
var formatted = ToToolPath(path);
if (!singleProject)
{
return formatted;
@@ -704,6 +822,20 @@ public sealed class MeetingSummaryTools
return slashIndex < 0 ? formatted : formatted[(slashIndex + 1)..];
}
private static bool IsWithinDirectory(string directory, string path)
{
var normalizedDirectory = Path.GetFullPath(directory)
.TrimEnd(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar) + Path.DirectorySeparatorChar;
var normalizedPath = Path.GetFullPath(path);
return normalizedPath.StartsWith(normalizedDirectory, StringComparison.OrdinalIgnoreCase);
}
private static string ToToolPath(string path)
{
return path.Replace(Path.DirectorySeparatorChar.ToString(), "/", StringComparison.Ordinal)
.Replace(Path.AltDirectorySeparatorChar.ToString(), "/", StringComparison.Ordinal);
}
private async Task<List<PastMeetingSummary>> GetScopedPastSummariesAsync(IReadOnlySet<string> projectNames)
{
if (projectNames.Count == 0)
@@ -753,7 +885,7 @@ public sealed class MeetingSummaryTools
var summariesRoot = GetSummariesRoot();
var fullPath = Path.GetFullPath(Path.Combine(summariesRoot, path));
if (!AgentFileToolContent.IsWithinDirectory(summariesRoot, fullPath) || !File.Exists(fullPath))
if (!IsWithinDirectory(summariesRoot, fullPath) || !File.Exists(fullPath))
{
return PastMeetingSummaryResolution.Refused("Refused: summary file does not exist under the configured summaries folder.");
}
@@ -779,7 +911,7 @@ public sealed class MeetingSummaryTools
private static string NormalizePastMeetingSummaryToolPath(string path)
{
var toolPath = AgentFileToolContent.ToToolPath(path.Trim());
var toolPath = ToToolPath(path.Trim());
const string pastMeetingPrefix = "past-meetings/";
return toolPath.StartsWith(pastMeetingPrefix, StringComparison.OrdinalIgnoreCase)
? toolPath[pastMeetingPrefix.Length..]
@@ -818,7 +950,7 @@ public sealed class MeetingSummaryTools
return new PastMeetingSummary(
path,
AgentFileToolContent.ToToolPath(Path.GetRelativePath(summariesRoot, path)),
ToToolPath(Path.GetRelativePath(summariesRoot, path)),
string.IsNullOrWhiteSpace(frontmatter.Title) ? Path.GetFileNameWithoutExtension(path) : frontmatter.Title.Trim(),
ParseDateTime(frontmatter.StartTime),
projects);
@@ -853,6 +985,45 @@ public sealed class MeetingSummaryTools
}
}
private sealed record FileLineEditMode(
FileLineEditKind Kind,
int? From = null,
int? To = null,
int? Insert = null)
{
public static FileLineEditMode? Create(int? from, int? to, int? insert, bool replaceFile)
{
if (replaceFile && (from.HasValue || to.HasValue || insert.HasValue))
{
return null;
}
if (insert.HasValue)
{
return from.HasValue || to.HasValue
? null
: new FileLineEditMode(FileLineEditKind.Insert, Insert: insert);
}
if (from.HasValue || to.HasValue)
{
return from.HasValue && to.HasValue && from.Value <= to.Value
? new FileLineEditMode(FileLineEditKind.Replace, From: from, To: to)
: null;
}
return new FileLineEditMode(replaceFile ? FileLineEditKind.Overwrite : FileLineEditKind.Append);
}
}
private enum FileLineEditKind
{
Append,
Overwrite,
Replace,
Insert
}
private sealed class MeetingNoteFrontmatterYaml
{
[YamlDotNet.Serialization.YamlMember(Alias = "title")]
@@ -1,6 +1,5 @@
using MeetingAssistant.MeetingNotes;
using MeetingAssistant.Transcription;
using MeetingAssistant.Workflow;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Compaction;
using Microsoft.Extensions.AI;
@@ -18,7 +17,6 @@ public sealed class OpenAiMeetingSummaryAgentPipeline : IMeetingSummaryPipeline
private readonly IMeetingSummaryFailureWriter failureWriter;
private readonly IMeetingSummaryInstructionBuilder instructionBuilder;
private readonly IDictationWordStore dictationWordStore;
private readonly IMeetingWorkflowEngine meetingWorkflowEngine;
public OpenAiMeetingSummaryAgentPipeline(
IOptions<MeetingAssistantOptions> options,
@@ -27,8 +25,7 @@ public sealed class OpenAiMeetingSummaryAgentPipeline : IMeetingSummaryPipeline
IServiceProvider services,
IMeetingSummaryFailureWriter failureWriter,
IMeetingSummaryInstructionBuilder instructionBuilder,
IDictationWordStore dictationWordStore,
IMeetingWorkflowEngine meetingWorkflowEngine)
IDictationWordStore dictationWordStore)
{
this.options = options.Value;
this.loggerFactory = loggerFactory;
@@ -37,7 +34,6 @@ public sealed class OpenAiMeetingSummaryAgentPipeline : IMeetingSummaryPipeline
this.failureWriter = failureWriter;
this.instructionBuilder = instructionBuilder;
this.dictationWordStore = dictationWordStore;
this.meetingWorkflowEngine = meetingWorkflowEngine;
}
public async Task<MeetingSummaryRunResult> RunAsync(
@@ -55,32 +51,35 @@ public sealed class OpenAiMeetingSummaryAgentPipeline : IMeetingSummaryPipeline
var agentOptions = options.Agent;
var key = ResolveApiKey(agentOptions);
var writeAudit = new SummaryAgentWriteAudit(artifacts);
var meetingTools = new MeetingSummaryTools(
artifacts,
options,
dictationWordStore,
writeAudit,
meetingWorkflowEngine);
var meetingTools = new MeetingSummaryTools(artifacts, options, dictationWordStore, writeAudit);
var tools = CreateTools(meetingTools);
var instructions = await instructionBuilder.BuildAsync(artifacts, options, cancellationToken);
using var compactionSummaryClient = LiteLlmResponsesChatClient.Create(
agentOptions,
using var compactionSummaryClient = new LiteLlmResponsesChatClient(
new Uri(agentOptions.Endpoint),
key,
agentOptions.Model,
agentOptions.EnableThinking,
ToReasoningEffortValue(agentOptions.ReasoningEffort),
agentOptions.ReconnectionAttempts,
agentOptions.ReconnectionDelay,
compactionOptions: null,
logger,
firstRequestIsUser: false);
var compactionOptions = CreateCompactionOptions(agentOptions, compactionSummaryClient);
using var chatClient = LiteLlmResponsesChatClient.Create(
agentOptions,
using var chatClient = new LiteLlmResponsesChatClient(
new Uri(agentOptions.Endpoint),
key,
agentOptions.Model,
agentOptions.EnableThinking,
ToReasoningEffortValue(agentOptions.ReasoningEffort),
agentOptions.ReconnectionAttempts,
agentOptions.ReconnectionDelay,
compactionOptions,
logger,
firstRequestIsUser: false);
var agent = chatClient
.AsBuilder()
.UseFunctionInvocation(
loggerFactory,
client => client.FunctionInvoker = FunctionInvocationGuard.InvokeAsync)
.UseFunctionInvocation()
.Build()
.AsAIAgent(
instructions,
@@ -128,19 +127,19 @@ public sealed class OpenAiMeetingSummaryAgentPipeline : IMeetingSummaryPipeline
AIFunctionFactory.Create(
tools.ReadMeetingNote,
"read_meetingnote",
"Read the meeting note, including YAML frontmatter and user-authored body notes. With no line arguments, read the whole file. Supports from/to or tail."),
"Read the meeting note, including YAML frontmatter and user-authored body notes. With no line arguments, read the whole file. With from and to, read that clamped inclusive 1-based line range."),
AIFunctionFactory.Create(
tools.ReadTranscript,
"read_transcript",
"Read the transcript markdown for the current meeting. With no line arguments, read the whole file. Supports from/to or tail."),
"Read the transcript markdown for the current meeting. With no line arguments, read the whole file. With from and to, read that clamped inclusive 1-based line range."),
AIFunctionFactory.Create(
tools.ReadContext,
"read_context",
"Read the assistant context markdown for the current meeting. With no line arguments, read the whole file. Supports from/to or tail."),
"Read the assistant context markdown for the current meeting. With no line arguments, read the whole file. With from and to, read that clamped inclusive 1-based line range."),
AIFunctionFactory.Create(
tools.ReadUserNotes,
"read_usernotes",
"Read the user-authored body notes from the meeting note. With no line arguments, read the whole body. Supports from/to or tail."),
"Read the user-authored body notes from the meeting note. With no line arguments, read the whole body. With from and to, read that clamped inclusive 1-based body line range."),
AIFunctionFactory.Create(
tools.WriteContext,
"write_context",
@@ -148,7 +147,7 @@ public sealed class OpenAiMeetingSummaryAgentPipeline : IMeetingSummaryPipeline
AIFunctionFactory.Create(
tools.ReadGlossary,
"read_glossary",
"Read the glossary or vocabulary context for this meeting. With no line arguments, read the whole file. Supports from/to or tail. May be empty."),
"Read the glossary or vocabulary context for this meeting. With no line arguments, read the whole file. With from and to, read that clamped inclusive 1-based line range. May be empty."),
AIFunctionFactory.Create(
tools.AddDictationWord,
"add_dictation_word",
@@ -172,7 +171,7 @@ public sealed class OpenAiMeetingSummaryAgentPipeline : IMeetingSummaryPipeline
AIFunctionFactory.Create(
tools.WriteSummary,
"write_summary",
"Write the finished summary markdown to the configured summary note and return the written summary filename. Provide title to rename the meeting when the current meeting title is missing or is still a generated default and the meeting purpose is clear."),
"Write the finished summary markdown to the configured summary note. Provide title when the meeting note has no title."),
AIFunctionFactory.Create(
tools.ListProjects,
"list_projects",
@@ -184,7 +183,7 @@ public sealed class OpenAiMeetingSummaryAgentPipeline : IMeetingSummaryPipeline
AIFunctionFactory.Create(
tools.ReadProjectFile,
"read_projectfile",
"Read a bound project file. Supports from/to or tail."),
"Read a bound project file, optionally clamping to from and to inclusive line numbers."),
AIFunctionFactory.Create(
tools.WriteProjectFile,
"write_projectfile",
@@ -196,11 +195,11 @@ public sealed class OpenAiMeetingSummaryAgentPipeline : IMeetingSummaryPipeline
AIFunctionFactory.Create(
tools.ReadPastProjectMeetingSummary,
"read_past_project_meeting_summary",
"Read a past project meeting summary note as read-only historical context. The path is relative to the configured summaries folder. Supports from/to or tail. Cannot read or write the current meeting summary."),
"Read a past project meeting summary note as read-only historical context. The path is relative to the configured summaries folder. With from and to, read that clamped inclusive 1-based line range. Cannot read or write the current meeting summary."),
AIFunctionFactory.Create(
tools.Search,
"search",
"Search project files and past meeting summaries for the requested current-meeting project scope, or all projects assigned to the current meeting when projects is omitted. Use file_pattern to limit project files by glob.")
"Search project files and past meeting summaries for the requested current-meeting project scope, or all projects assigned to the current meeting when projects is omitted.")
];
}
@@ -304,6 +303,18 @@ public sealed class OpenAiMeetingSummaryAgentPipeline : IMeetingSummaryPipeline
return new PipelineCompactionStrategy(strategies);
}
private static string ToReasoningEffortValue(ReasoningEffortOption effort)
{
return effort switch
{
ReasoningEffortOption.None => "none",
ReasoningEffortOption.Low => "low",
ReasoningEffortOption.High => "high",
ReasoningEffortOption.ExtraHigh => "xhigh",
_ => "medium"
};
}
private static ReasoningEffort ToReasoningEffort(ReasoningEffortOption effort)
{
return effort switch
+4 -47
View File
@@ -6,13 +6,10 @@ namespace MeetingAssistant.Taskbar;
public enum MeetingTaskbarAction
{
EditRules,
OpenSubmenu,
StartRecording,
StopRecording,
AbortRecording,
SwitchProfile,
SelectMicrophone,
Exit
SwitchProfile
}
public sealed record MeetingTaskbarMenu(
@@ -23,29 +20,19 @@ public sealed record MeetingTaskbarMenu(
public sealed record MeetingTaskbarMenuItem(
string Text,
MeetingTaskbarAction Action,
string? ProfileName = null,
string? MicrophoneDeviceId = null,
bool IsChecked = false,
IReadOnlyList<MeetingTaskbarMenuItem>? Items = null);
string? ProfileName = null);
public static class MeetingTaskbarMenuBuilder
{
public static MeetingTaskbarMenu Build(
RecordingStatus status,
IReadOnlyList<LaunchProfile> launchProfiles,
IReadOnlyList<MicrophoneDevice>? microphones = null,
string? currentMicrophoneDeviceId = null)
IReadOnlyList<LaunchProfile> launchProfiles)
{
var items = new List<MeetingTaskbarMenuItem>
{
new("Open agent", MeetingTaskbarAction.EditRules)
new("Edit rules and identities", MeetingTaskbarAction.EditRules)
};
if (microphones is { Count: > 0 })
{
items.Add(BuildMicrophoneMenu(microphones, currentMicrophoneDeviceId));
}
if (status.IsRecording)
{
items.Add(new MeetingTaskbarMenuItem(
@@ -74,34 +61,12 @@ public static class MeetingTaskbarMenuBuilder
}
}
items.Add(new MeetingTaskbarMenuItem(
"Exit",
MeetingTaskbarAction.Exit));
return new MeetingTaskbarMenu(
status.State,
BuildTooltip(status),
items);
}
private static MeetingTaskbarMenuItem BuildMicrophoneMenu(
IReadOnlyList<MicrophoneDevice> microphones,
string? currentMicrophoneDeviceId)
{
var microphoneItems = microphones
.Select(microphone => new MeetingTaskbarMenuItem(
microphone.Name,
MeetingTaskbarAction.SelectMicrophone,
MicrophoneDeviceId: microphone.Id,
IsChecked: string.Equals(microphone.Id, currentMicrophoneDeviceId, StringComparison.Ordinal)))
.ToArray();
return new MeetingTaskbarMenuItem(
"Microphone",
MeetingTaskbarAction.OpenSubmenu,
Items: microphoneItems);
}
private static string BuildTooltip(RecordingStatus status)
{
return status.State switch
@@ -131,11 +96,3 @@ public static class MeetingTaskbarMenuBuilder
: $"{text}\t{hotkey.Trim()}";
}
}
public static class MeetingTaskbarExitPolicy
{
public static bool RequiresConfirmation(RecordingProcessState state)
{
return state != RecordingProcessState.Idle;
}
}
@@ -1,134 +0,0 @@
using System.Drawing;
using System.Drawing.Drawing2D;
using System.Drawing.Imaging;
using System.Drawing.Text;
using System.Runtime.InteropServices;
using System.Runtime.Versioning;
using MeetingAssistant.Recording;
namespace MeetingAssistant.Taskbar;
[SupportedOSPlatform("windows")]
internal static class TaskbarIconRenderer
{
private const int IconSize = 16;
private const float GlyphPixelGridOffsetX = -0.5f;
public static Icon CreateIcon(RecordingProcessState state)
{
using var bitmap = RenderBitmap(state);
var handle = bitmap.GetHicon();
try
{
return (Icon)Icon.FromHandle(handle).Clone();
}
finally
{
DestroyIcon(handle);
}
}
internal static Bitmap RenderBitmap(RecordingProcessState state)
{
var (color, text) = state switch
{
RecordingProcessState.Recording => (Color.FromArgb(220, 38, 38), "R"),
RecordingProcessState.Summarizing => (Color.FromArgb(37, 99, 235), "S"),
_ => (Color.FromArgb(75, 85, 99), "I")
};
var bitmap = new Bitmap(IconSize, IconSize, PixelFormat.Format32bppArgb);
using var graphics = Graphics.FromImage(bitmap);
graphics.Clear(Color.Transparent);
graphics.SmoothingMode = SmoothingMode.AntiAlias;
graphics.TextRenderingHint = TextRenderingHint.AntiAliasGridFit;
using var brush = new SolidBrush(color);
graphics.FillEllipse(brush, 1, 1, 14, 14);
DrawCenteredGlyph(graphics, text);
return bitmap;
}
private static void DrawCenteredGlyph(Graphics graphics, string text)
{
using var path = new GraphicsPath();
using var format = StringFormat.GenericTypographic;
using var fontFamily = CreateGlyphFontFamily(out var fontStyle);
path.AddString(
text,
fontFamily,
(int)fontStyle,
9,
Point.Empty,
format);
var bounds = path.GetBounds();
using var transform = new Matrix();
transform.Translate(
(IconSize - bounds.Width) / 2 - bounds.Left + GlyphPixelGridOffsetX,
(IconSize - bounds.Height) / 2 - bounds.Top);
path.Transform(transform);
using var textBrush = new SolidBrush(Color.White);
graphics.FillPath(textBrush, path);
}
private static FontFamily CreateGlyphFontFamily(out FontStyle fontStyle)
{
foreach (var name in new[] { "Segoe UI", "Arial", "DejaVu Sans", "Liberation Sans", "Noto Sans" })
{
try
{
var family = new FontFamily(name);
if (TryGetUsableStyle(family, out fontStyle))
{
return family;
}
family.Dispose();
}
catch (ArgumentException)
{
}
}
using var installedFonts = new InstalledFontCollection();
foreach (var installedFamily in installedFonts.Families)
{
try
{
var family = new FontFamily(installedFamily.Name);
if (TryGetUsableStyle(family, out fontStyle))
{
return family;
}
family.Dispose();
}
catch (ArgumentException)
{
}
}
fontStyle = FontStyle.Bold;
return FontFamily.GenericSansSerif;
}
private static bool TryGetUsableStyle(FontFamily family, out FontStyle fontStyle)
{
if (family.IsStyleAvailable(FontStyle.Bold))
{
fontStyle = FontStyle.Bold;
return true;
}
if (family.IsStyleAvailable(FontStyle.Regular))
{
fontStyle = FontStyle.Regular;
return true;
}
fontStyle = FontStyle.Regular;
return false;
}
[DllImport("user32.dll", SetLastError = true)]
private static extern bool DestroyIcon(IntPtr hIcon);
}
@@ -1,6 +1,6 @@
#if WINDOWS
using System.Drawing;
using System.Windows;
using System.Runtime.InteropServices;
using H.NotifyIcon.Core;
using MeetingAssistant.LaunchProfiles;
using MeetingAssistant.Recording;
@@ -14,10 +14,7 @@ public sealed class UnoTaskbarIconService : IHostedService, IDisposable
private readonly MeetingRecordingCoordinator coordinator;
private readonly ILaunchProfileOptionsProvider launchProfiles;
private readonly IMicrophoneDeviceProvider microphones;
private readonly MicrophoneDeviceSelection microphoneSelection;
private readonly IWorkflowRulesEditorWindowService rulesEditorWindow;
private readonly IHostApplicationLifetime applicationLifetime;
private readonly ILogger<UnoTaskbarIconService> logger;
private readonly object sync = new();
private CancellationTokenSource? refreshCancellation;
@@ -31,18 +28,12 @@ public sealed class UnoTaskbarIconService : IHostedService, IDisposable
public UnoTaskbarIconService(
MeetingRecordingCoordinator coordinator,
ILaunchProfileOptionsProvider launchProfiles,
IMicrophoneDeviceProvider microphones,
MicrophoneDeviceSelection microphoneSelection,
IWorkflowRulesEditorWindowService rulesEditorWindow,
IHostApplicationLifetime applicationLifetime,
ILogger<UnoTaskbarIconService> logger)
{
this.coordinator = coordinator;
this.launchProfiles = launchProfiles;
this.microphones = microphones;
this.microphoneSelection = microphoneSelection;
this.rulesEditorWindow = rulesEditorWindow;
this.applicationLifetime = applicationLifetime;
this.logger = logger;
}
@@ -56,7 +47,7 @@ public sealed class UnoTaskbarIconService : IHostedService, IDisposable
var menu = BuildMenu();
currentMenuSignature = BuildMenuSignature(menu);
currentIcon = TaskbarIconRenderer.CreateIcon(menu.State);
currentIcon = CreateIcon(menu.State);
currentState = menu.State;
trayIcon = new TrayIconWithContextMenu(TrayIconId)
{
@@ -135,7 +126,7 @@ public sealed class UnoTaskbarIconService : IHostedService, IDisposable
if (currentState != menu.State)
{
var previousIcon = currentIcon;
var nextIcon = TaskbarIconRenderer.CreateIcon(menu.State);
var nextIcon = CreateIcon(menu.State);
if (!trayIcon.UpdateIcon(nextIcon.Handle))
{
logger.LogWarning("Taskbar icon update returned false for state {State}", menu.State);
@@ -173,8 +164,6 @@ public sealed class UnoTaskbarIconService : IHostedService, IDisposable
private MeetingTaskbarMenu BuildMenu()
{
var profiles = launchProfiles.GetProfiles();
var activeOptions = GetActiveProfileOptions(profiles);
var microphoneSnapshot = microphones.GetMicrophoneSnapshot(activeOptions);
var profilesSignature = string.Join(
", ",
profiles.Select(profile => $"{profile.Name}:{profile.Options.Hotkey.Toggle}"));
@@ -186,9 +175,7 @@ public sealed class UnoTaskbarIconService : IHostedService, IDisposable
return MeetingTaskbarMenuBuilder.Build(
coordinator.CurrentStatus,
profiles,
microphoneSnapshot.Available,
microphoneSnapshot.Current?.Id);
profiles);
}
private PopupMenu BuildContextMenu(MeetingTaskbarMenu menu)
@@ -196,41 +183,20 @@ public sealed class UnoTaskbarIconService : IHostedService, IDisposable
var popupMenu = new PopupMenu();
for (var index = 0; index < menu.Items.Count; index++)
{
if (index == 1 ||
(menu.Items[index].Action == MeetingTaskbarAction.Exit &&
menu.Items[index - 1].Action != MeetingTaskbarAction.EditRules))
if (index == 1)
{
popupMenu.Items.Add(new PopupMenuSeparator());
}
var menuItem = menu.Items[index];
popupMenu.Items.Add(BuildPopupItem(menuItem));
popupMenu.Items.Add(new PopupMenuItem(
menuItem.Text,
(_, _) => _ = ExecuteMenuItemAsync(menuItem)));
}
return popupMenu;
}
private PopupItem BuildPopupItem(MeetingTaskbarMenuItem menuItem)
{
if (menuItem.Items is { Count: > 0 })
{
var submenu = new PopupSubMenu(menuItem.Text);
foreach (var child in menuItem.Items)
{
submenu.Items.Add(BuildPopupItem(child));
}
return submenu;
}
return new PopupMenuItem(
menuItem.Text,
(_, _) => _ = ExecuteMenuItemAsync(menuItem))
{
Checked = menuItem.IsChecked
};
}
private async Task ExecuteMenuItemAsync(MeetingTaskbarMenuItem item)
{
try
@@ -257,16 +223,6 @@ public sealed class UnoTaskbarIconService : IHostedService, IDisposable
case MeetingTaskbarAction.SwitchProfile:
await coordinator.ToggleAsync(item.ProfileName, CancellationToken.None);
break;
case MeetingTaskbarAction.SelectMicrophone:
if (!string.IsNullOrWhiteSpace(item.MicrophoneDeviceId))
{
microphoneSelection.Select(item.MicrophoneDeviceId);
}
break;
case MeetingTaskbarAction.Exit:
ExitApplication();
break;
}
RefreshVisualState();
@@ -289,69 +245,48 @@ public sealed class UnoTaskbarIconService : IHostedService, IDisposable
{
return string.Join(
"|",
FlattenMenuItems(menu.Items).Select(item =>
$"{item.Action}:{item.ProfileName}:{item.MicrophoneDeviceId}:{item.IsChecked}:{item.Text}"));
menu.Items.Select(item => $"{item.Action}:{item.ProfileName}:{item.Text}"));
}
private static IEnumerable<MeetingTaskbarMenuItem> FlattenMenuItems(
IEnumerable<MeetingTaskbarMenuItem> items)
private static Icon CreateIcon(RecordingProcessState state)
{
foreach (var item in items)
var (color, text) = state switch
{
yield return item;
if (item.Items is null)
{
continue;
}
RecordingProcessState.Recording => (Color.FromArgb(220, 38, 38), "R"),
RecordingProcessState.Summarizing => (Color.FromArgb(37, 99, 235), "S"),
_ => (Color.FromArgb(75, 85, 99), "I")
};
foreach (var child in FlattenMenuItems(item.Items))
{
yield return child;
}
using var bitmap = new Bitmap(16, 16);
using (var graphics = Graphics.FromImage(bitmap))
{
graphics.Clear(Color.Transparent);
graphics.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.AntiAlias;
using var brush = new SolidBrush(color);
graphics.FillEllipse(brush, 1, 1, 14, 14);
using var font = new Font(FontFamily.GenericSansSerif, 8, FontStyle.Bold, GraphicsUnit.Pixel);
using var textBrush = new SolidBrush(Color.White);
var size = graphics.MeasureString(text, font);
graphics.DrawString(
text,
font,
textBrush,
(16 - size.Width) / 2,
(16 - size.Height) / 2);
}
var handle = bitmap.GetHicon();
try
{
return (Icon)Icon.FromHandle(handle).Clone();
}
finally
{
DestroyIcon(handle);
}
}
private MeetingAssistantOptions GetActiveProfileOptions(IReadOnlyList<LaunchProfile> profiles)
{
var activeProfile = coordinator.CurrentStatus.LaunchProfile;
return profiles.FirstOrDefault(profile => string.Equals(
profile.Name,
activeProfile,
StringComparison.OrdinalIgnoreCase))?.Options ??
profiles.FirstOrDefault(profile => string.Equals(
profile.Name,
"default",
StringComparison.OrdinalIgnoreCase))?.Options ??
profiles.FirstOrDefault()?.Options ??
new MeetingAssistantOptions();
}
private void ExitApplication()
{
var status = coordinator.CurrentStatus;
if (MeetingTaskbarExitPolicy.RequiresConfirmation(status.State) && !ConfirmExitDuringProcessing(status.State))
{
logger.LogInformation("Taskbar exit canceled while meeting state was {State}", status.State);
return;
}
logger.LogInformation("Taskbar exit requested while meeting state was {State}", status.State);
applicationLifetime.StopApplication();
}
private static bool ConfirmExitDuringProcessing(RecordingProcessState state)
{
var activity = state == RecordingProcessState.Recording
? "A meeting is still recording and transcribing."
: "A stopped meeting is still transcribing, recognizing speakers, or summarizing.";
var result = MessageBox.Show(
$"{activity}\n\nExit Meeting Assistant anyway?",
"Exit Meeting Assistant",
MessageBoxButton.YesNo,
MessageBoxImage.Warning,
MessageBoxResult.No);
return result == MessageBoxResult.Yes;
}
[DllImport("user32.dll", SetLastError = true)]
private static extern bool DestroyIcon(IntPtr hIcon);
}
#endif
@@ -1,4 +1,5 @@
using MeetingAssistant.Recording;
using NAudio.Wave;
namespace MeetingAssistant.Transcription;
@@ -76,7 +77,7 @@ public sealed class AsrDiagnosticService
bool paceAudio,
CancellationToken cancellationToken)
{
await foreach (var chunk in PcmWavAudioChunkReader.ReadChunksAsync(fullPath, cancellationToken: cancellationToken))
await foreach (var chunk in ReadPcmWavChunks(fullPath, cancellationToken))
{
await pipeline.WriteAsync(chunk, cancellationToken);
if (paceAudio && chunk.Duration > TimeSpan.Zero)
@@ -101,6 +102,40 @@ public sealed class AsrDiagnosticService
return fullPath;
}
private static async IAsyncEnumerable<AudioChunk> ReadPcmWavChunks(
string path,
[System.Runtime.CompilerServices.EnumeratorCancellation] CancellationToken cancellationToken = default)
{
var reader = new WaveFileReader(path);
try
{
var format = reader.WaveFormat;
if (format.Encoding != WaveFormatEncoding.Pcm || format.BitsPerSample != 16)
{
throw new InvalidDataException(
$"Only 16-bit PCM WAV files are supported. Found {format.Encoding} with {format.BitsPerSample} bits per sample.");
}
var buffer = new byte[format.AverageBytesPerSecond];
int read;
while ((read = await reader.ReadAsync(buffer.AsMemory(0, buffer.Length), cancellationToken)) > 0)
{
yield return new AudioChunk(buffer[..read], format.SampleRate, format.Channels);
}
}
finally
{
try
{
reader.Dispose();
}
catch (NotSupportedException)
{
// NAudio can throw while disposing reader streams from async iterators; diagnostics should not fail after reading all chunks.
}
}
}
}
public sealed record AsrDiagnosticResult(string Path, IReadOnlyList<TranscriptionSegment> Segments);
@@ -33,78 +33,69 @@ public sealed class AzureSpeechStreamingTranscriptionProvider : IStreamingTransc
yield break;
}
var nextChunk = enumerator.Current;
var reconnectAttempt = 0;
var markerId = "";
var disconnectedMarkerWritten = false;
var firstChunk = enumerator.Current;
using var pushStream = AudioInputStream.CreatePushStream(
AudioStreamFormat.GetWaveFormatPCM(
(uint)firstChunk.SampleRate,
16,
(byte)firstChunk.Channels));
using var audioConfig = AudioConfig.FromStreamInput(pushStream);
var speechConfig = CreateSpeechConfig();
using var recognizer = CreateTranscriber(speechConfig, audioConfig);
AddPhraseList(recognizer, pipelineOptions.DictationWords);
var segments = Channel.CreateUnbounded<TranscriptionSegment>();
while (true)
recognizer.Transcribed += (_, args) =>
{
var session = StartSession(
nextChunk,
if (args.Result.Reason != ResultReason.RecognizedSpeech
|| string.IsNullOrWhiteSpace(args.Result.Text))
{
return;
}
var start = TimeSpan.FromTicks(args.Result.OffsetInTicks);
var segment = new TranscriptionSegment(
start,
start + args.Result.Duration,
NormalizeSpeakerId(args.Result.SpeakerId),
args.Result.Text.Trim());
logger.LogInformation(
"Azure Speech emitted segment at {Start} from {Speaker}: {Text}",
segment.Start,
segment.Speaker,
segment.Text);
segments.Writer.TryWrite(segment);
};
recognizer.Canceled += (_, args) =>
{
if (args.Reason == CancellationReason.Error)
{
segments.Writer.TryComplete(new InvalidOperationException(
$"Azure Speech recognition canceled: {args.ErrorCode} {args.ErrorDetails}"));
return;
}
segments.Writer.TryComplete();
};
await recognizer.StartTranscribingAsync();
var pumpTask = Task.Run(
() => PumpAudioAsync(
firstChunk,
enumerator,
pipelineOptions,
markerId,
cancellationToken);
pushStream,
recognizer,
segments.Writer,
options.AzureSpeech.RecognitionStopTimeout,
cancellationToken),
CancellationToken.None);
await foreach (var segment in session.Segments.ReadAllAsync(cancellationToken))
{
if (segment.ReplacesMarkerId is not null)
{
reconnectAttempt = 0;
disconnectedMarkerWritten = false;
}
yield return segment;
}
var result = await session.Completion.WaitAsync(cancellationToken);
if (result.IsCompleted)
{
yield break;
}
reconnectAttempt++;
markerId = string.IsNullOrWhiteSpace(markerId)
? $"azure-reconnect-{Guid.NewGuid():N}"
: markerId;
nextChunk = result.NextChunk ?? nextChunk;
if (reconnectAttempt <= Math.Max(1, options.AzureSpeech.ReconnectAttemptsBeforeDisconnectedMarker))
{
logger.LogWarning(
"Azure Speech reconnecting after connection interruption, attempt {Attempt}/{MaxAttempts}: {ErrorDetails}",
reconnectAttempt,
Math.Max(1, options.AzureSpeech.ReconnectAttemptsBeforeDisconnectedMarker),
result.ErrorDetails);
yield return new TranscriptionSegment(
TimeSpan.Zero,
TimeSpan.Zero,
"System",
$"<Reconnecting... {reconnectAttempt}/{Math.Max(1, options.AzureSpeech.ReconnectAttemptsBeforeDisconnectedMarker)}>",
TranscriptionSegmentKind.Marker,
MarkerId: markerId);
}
else if (!disconnectedMarkerWritten)
{
disconnectedMarkerWritten = true;
logger.LogWarning(
"Azure Speech remains disconnected after {AttemptCount} reconnect attempt(s); recording continues and transcription will retry: {ErrorDetails}",
reconnectAttempt - 1,
result.ErrorDetails);
yield return new TranscriptionSegment(
TimeSpan.Zero,
TimeSpan.Zero,
"System",
"<Azure Speech disconnected. The meeting can continue; transcription and summarization will continue once Azure Speech reconnects.>",
TranscriptionSegmentKind.Marker,
MarkerId: markerId);
}
if (options.AzureSpeech.ReconnectDelay > TimeSpan.Zero)
{
await Task.Delay(options.AzureSpeech.ReconnectDelay, cancellationToken);
}
await foreach (var segment in segments.Reader.ReadAllAsync(cancellationToken))
{
yield return segment;
}
await pumpTask.WaitAsync(cancellationToken);
}
finally
{
@@ -119,117 +110,6 @@ public sealed class AzureSpeechStreamingTranscriptionProvider : IStreamingTransc
}
}
private AzureSpeechSession StartSession(
AudioChunk firstChunk,
IAsyncEnumerator<AudioChunk> audio,
SpeechRecognitionPipelineOptions pipelineOptions,
string reconnectMarkerId,
CancellationToken cancellationToken)
{
var segments = Channel.CreateUnbounded<TranscriptionSegment>();
var completion = Task.Run(
() => RunSessionAsync(
firstChunk,
audio,
pipelineOptions,
reconnectMarkerId,
segments.Writer,
cancellationToken),
CancellationToken.None);
return new AzureSpeechSession(segments.Reader, completion);
}
private async Task<AzureSpeechSessionResult> RunSessionAsync(
AudioChunk firstChunk,
IAsyncEnumerator<AudioChunk> audio,
SpeechRecognitionPipelineOptions pipelineOptions,
string reconnectMarkerId,
ChannelWriter<TranscriptionSegment> segments,
CancellationToken cancellationToken)
{
try
{
using var pushStream = AudioInputStream.CreatePushStream(
AudioStreamFormat.GetWaveFormatPCM(
(uint)firstChunk.SampleRate,
16,
(byte)firstChunk.Channels));
using var audioConfig = AudioConfig.FromStreamInput(pushStream);
var speechConfig = CreateSpeechConfig();
using var recognizer = CreateTranscriber(speechConfig, audioConfig);
AddPhraseList(recognizer, pipelineOptions.DictationWords);
var sessionStop = new TaskCompletionSource<AzureSpeechSessionResult>(
TaskCreationOptions.RunContinuationsAsynchronously);
var pendingReconnectMarkerId = reconnectMarkerId;
recognizer.Transcribed += (_, args) =>
{
if (args.Result.Reason != ResultReason.RecognizedSpeech
|| string.IsNullOrWhiteSpace(args.Result.Text))
{
return;
}
var start = TimeSpan.FromTicks(args.Result.OffsetInTicks);
var replacesMarkerId = string.IsNullOrWhiteSpace(pendingReconnectMarkerId)
? null
: Interlocked.Exchange(ref pendingReconnectMarkerId, null);
var segment = new TranscriptionSegment(
start,
start + args.Result.Duration,
NormalizeSpeakerId(args.Result.SpeakerId),
args.Result.Text.Trim(),
ReplacesMarkerId: replacesMarkerId);
logger.LogInformation(
"Azure Speech emitted segment at {Start} from {Speaker}: {Text}",
segment.Start,
segment.Speaker,
segment.Text);
segments.TryWrite(segment);
};
recognizer.Canceled += (_, args) =>
{
if (args.Reason == CancellationReason.Error)
{
logger.LogWarning(
"Azure Speech recognition canceled with {ErrorCode}: {ErrorDetails}",
args.ErrorCode,
args.ErrorDetails);
sessionStop.TrySetResult(AzureSpeechSessionResult.Reconnect(
$"Azure Speech recognition canceled: {args.ErrorCode} {args.ErrorDetails}"));
return;
}
sessionStop.TrySetResult(AzureSpeechSessionResult.Completed());
};
recognizer.SessionStopped += (_, _) => sessionStop.TrySetResult(AzureSpeechSessionResult.Completed());
await recognizer.StartTranscribingAsync();
return await PumpAudioAsync(
firstChunk,
audio,
pushStream,
recognizer,
sessionStop.Task,
options.AzureSpeech.RecognitionStopTimeout,
cancellationToken);
}
catch (Exception exception) when (IsTransientConnectionException(exception))
{
logger.LogWarning(exception, "Azure Speech session failed transiently; reconnecting.");
return AzureSpeechSessionResult.Reconnect(firstChunk, exception.Message);
}
catch (Exception exception)
{
segments.TryComplete(exception);
throw;
}
finally
{
segments.TryComplete();
}
}
internal ConversationTranscriber CreateTranscriber(SpeechConfig speechConfig, AudioConfig audioConfig)
{
var languages = GetAutoDetectLanguages(options.AzureSpeech);
@@ -354,118 +234,47 @@ public sealed class AzureSpeechStreamingTranscriptionProvider : IStreamingTransc
: "Continuous";
}
private static async Task<AzureSpeechSessionResult> PumpAudioAsync(
private static async Task PumpAudioAsync(
AudioChunk firstChunk,
IAsyncEnumerator<AudioChunk> audio,
PushAudioInputStream pushStream,
ConversationTranscriber recognizer,
Task<AzureSpeechSessionResult> sessionStop,
ChannelWriter<TranscriptionSegment> segments,
TimeSpan recognitionStopTimeout,
CancellationToken cancellationToken)
{
WriteChunk(pushStream, firstChunk, firstChunk);
while (true)
{
var moveNextTask = audio.MoveNextAsync().AsTask();
var completedTask = await Task.WhenAny(moveNextTask, sessionStop);
if (completedTask == sessionStop)
{
var stop = await sessionStop;
if (stop.IsCompleted)
{
return AzureSpeechSessionResult.Completed();
}
if (!await moveNextTask)
{
return AzureSpeechSessionResult.Completed();
}
return AzureSpeechSessionResult.Reconnect(audio.Current, stop.ErrorDetails);
}
if (!await moveNextTask)
{
break;
}
if (sessionStop.IsCompleted)
{
var stop = await sessionStop;
if (!stop.IsCompleted)
{
return AzureSpeechSessionResult.Reconnect(audio.Current, stop.ErrorDetails);
}
}
cancellationToken.ThrowIfCancellationRequested();
WriteChunk(pushStream, firstChunk, audio.Current);
}
pushStream.Close();
try
{
var stopTask = recognizer.StopTranscribingAsync();
if (recognitionStopTimeout > TimeSpan.Zero)
WriteChunk(pushStream, firstChunk, firstChunk);
while (await audio.MoveNextAsync())
{
await stopTask.WaitAsync(recognitionStopTimeout, cancellationToken);
cancellationToken.ThrowIfCancellationRequested();
WriteChunk(pushStream, firstChunk, audio.Current);
}
else
pushStream.Close();
try
{
await stopTask.WaitAsync(cancellationToken);
var stopTask = recognizer.StopTranscribingAsync();
if (recognitionStopTimeout > TimeSpan.Zero)
{
await stopTask.WaitAsync(recognitionStopTimeout, cancellationToken);
}
else
{
await stopTask.WaitAsync(cancellationToken);
}
}
catch (TimeoutException)
{
// The SDK can outlive short diagnostic streams while waiting for final service events.
}
segments.TryComplete();
}
catch (TimeoutException)
catch (Exception exception)
{
// The SDK can outlive short diagnostic streams while waiting for final service events.
}
catch (Exception exception) when (IsTransientConnectionException(exception))
{
return AzureSpeechSessionResult.Reconnect(firstChunk, exception.Message);
}
return AzureSpeechSessionResult.Completed();
}
private static bool IsTransientConnectionException(Exception exception)
{
if (exception is OperationCanceledException)
{
return false;
}
return exception is TimeoutException
or IOException
or HttpRequestException
or System.Net.Sockets.SocketException
|| exception.Message.Contains("connection", StringComparison.OrdinalIgnoreCase)
|| exception.Message.Contains("websocket", StringComparison.OrdinalIgnoreCase)
|| exception.Message.Contains("network", StringComparison.OrdinalIgnoreCase)
|| exception.Message.Contains("transport", StringComparison.OrdinalIgnoreCase);
}
private sealed record AzureSpeechSession(
ChannelReader<TranscriptionSegment> Segments,
Task<AzureSpeechSessionResult> Completion);
private sealed record AzureSpeechSessionResult(
bool IsCompleted,
AudioChunk? NextChunk,
string? ErrorDetails)
{
public static AzureSpeechSessionResult Completed()
{
return new AzureSpeechSessionResult(true, null, null);
}
public static AzureSpeechSessionResult Reconnect(AudioChunk nextChunk, string? errorDetails)
{
return new AzureSpeechSessionResult(false, nextChunk, errorDetails);
}
public static AzureSpeechSessionResult Reconnect(string? errorDetails)
{
return new AzureSpeechSessionResult(false, null, errorDetails);
segments.TryComplete(exception);
}
}
@@ -1,16 +1,3 @@
namespace MeetingAssistant.Transcription;
public sealed record TranscriptionSegment(
TimeSpan Start,
TimeSpan End,
string Speaker,
string Text,
TranscriptionSegmentKind Kind = TranscriptionSegmentKind.Speech,
string? MarkerId = null,
string? ReplacesMarkerId = null);
public enum TranscriptionSegmentKind
{
Speech,
Marker
}
public sealed record TranscriptionSegment(TimeSpan Start, TimeSpan End, string Speaker, string Text);
+112 -297
View File
@@ -3,6 +3,7 @@ using System.Globalization;
using System.Text.RegularExpressions;
using MeetingAssistant.MeetingNotes;
using NCalc;
using RazorLight;
namespace MeetingAssistant.Workflow;
@@ -12,19 +13,6 @@ public interface IMeetingWorkflowEngine
MeetingWorkflowEvent workflowEvent,
MeetingAssistantOptions options,
CancellationToken cancellationToken);
Task<string> TransformTranscriptLineAsync(
MeetingWorkflowEvent workflowEvent,
MeetingAssistantOptions options,
CancellationToken cancellationToken);
Task<string> TransformAttendeeAsync(
MeetingWorkflowEvent workflowEvent,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
return Task.FromResult(workflowEvent.AttendeeName ?? "");
}
}
public sealed class NoopMeetingWorkflowEngine : IMeetingWorkflowEngine
@@ -38,26 +26,14 @@ public sealed class NoopMeetingWorkflowEngine : IMeetingWorkflowEngine
{
return Task.CompletedTask;
}
public Task<string> TransformTranscriptLineAsync(
MeetingWorkflowEvent workflowEvent,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
return Task.FromResult(workflowEvent.TranscriptLineText ?? "");
}
public Task<string> TransformAttendeeAsync(
MeetingWorkflowEvent workflowEvent,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
return Task.FromResult(workflowEvent.AttendeeName ?? "");
}
}
public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
{
private static readonly Regex EmailAddressPattern = new(
@"(?<![A-Za-z0-9.!#$%&'*+/=?^_`{|}~-])(?<local>[A-Za-z0-9.!#$%&'*+/=?^_`{|}~-]+)@(?<domain>(?:[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?\.)+[A-Za-z]{2,63})(?![A-Za-z0-9-])",
RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.IgnoreCase);
private static readonly string[] ParameterNames =
[
"meeting.attendees.count",
@@ -68,17 +44,16 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
"event.type",
"state.from",
"state.to",
"speaker.name",
MeetingWorkflowRuleSchema.PropertyTranscriptLine,
MeetingWorkflowRuleSchema.ParameterTranscriptSpeaker,
MeetingWorkflowRuleSchema.PropertyAttendeeName
"speaker.name"
];
private readonly IMeetingWorkflowRulesProvider rulesProvider;
private readonly IMeetingNoteStore meetingNoteStore;
private readonly IMeetingArtifactStore meetingArtifactStore;
private readonly ILogger<MeetingWorkflowEngine> logger;
private readonly MeetingWorkflowTemplateRenderer templateRenderer = new();
private readonly RazorLightEngine razorEngine = new RazorLightEngineBuilder()
.UseNoProject()
.Build();
public MeetingWorkflowEngine(
IMeetingWorkflowRulesProvider rulesProvider,
@@ -96,50 +71,11 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
MeetingWorkflowEvent workflowEvent,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
_ = await RunCoreAsync(workflowEvent, options, cancellationToken);
}
public async Task<string> TransformTranscriptLineAsync(
MeetingWorkflowEvent workflowEvent,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
if (workflowEvent.Type != MeetingWorkflowEventType.TranscriptLine)
{
throw new ArgumentException("Workflow event must be a transcript line event.", nameof(workflowEvent));
}
return await RunCoreAsync(workflowEvent, options, cancellationToken)
?? workflowEvent.TranscriptLineText
?? "";
}
public async Task<string> TransformAttendeeAsync(
MeetingWorkflowEvent workflowEvent,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
if (workflowEvent.Type != MeetingWorkflowEventType.AttendeeAdded)
{
throw new ArgumentException("Workflow event must be an attendee added event.", nameof(workflowEvent));
}
return await RunCoreAsync(workflowEvent, options, cancellationToken)
?? workflowEvent.AttendeeName
?? "";
}
private async Task<string?> RunCoreAsync(
MeetingWorkflowEvent workflowEvent,
MeetingAssistantOptions options,
CancellationToken cancellationToken)
{
var rules = await rulesProvider.GetRulesAsync(options, cancellationToken);
var context = new MeetingWorkflowExecutionContext(workflowEvent);
if (rules.Count == 0)
{
return context.ResultValue;
return;
}
var meeting = await meetingNoteStore.ReadAsync(
@@ -149,61 +85,26 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
foreach (var rule in rules)
{
try
if (!MatchesTrigger(rule, workflowEvent) ||
!EvaluateConditions(rule.If, meeting, workflowEvent))
{
if (!MatchesTrigger(rule, workflowEvent) ||
!EvaluateConditions(rule.If, meeting, context))
{
continue;
}
logger.LogInformation(
"Triggered meeting workflow rule {RuleName} for event {EventType}",
rule.Name,
workflowEvent.Type);
var ruleNoteChanged = false;
var originalTranscriptLine = context.TranscriptLine;
var model = CreateTemplateModel(meeting, context);
foreach (var step in rule.Steps)
{
if (workflowEvent.Type == MeetingWorkflowEventType.AttendeeAdded &&
!IsAttendeeTransformStep(step))
{
throw new InvalidOperationException(
"attendee_added workflow rules can only transform attendee.name with set_property.");
}
var stepChanged = await ApplyStepAsync(
step,
meeting,
context,
options,
model,
cancellationToken);
ruleNoteChanged |= stepChanged;
noteChanged |= stepChanged;
model = CreateTemplateModel(meeting, context);
}
logger.LogInformation(
"Completed meeting workflow rule {RuleName} for event {EventType}; noteChanged={NoteChanged}, transcriptLineChanged={TranscriptLineChanged}",
rule.Name,
workflowEvent.Type,
ruleNoteChanged,
!string.Equals(originalTranscriptLine, context.TranscriptLine, StringComparison.Ordinal));
continue;
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
logger.LogInformation(
"Applying meeting workflow rule {RuleName} for event {EventType}",
rule.Name,
workflowEvent.Type);
var model = CreateTemplateModel(meeting, workflowEvent);
foreach (var step in rule.Steps)
{
throw;
}
catch (Exception exception)
{
logger.LogError(
exception,
"Meeting workflow rule {RuleName} failed for event {EventType}",
rule.Name,
workflowEvent.Type);
throw;
noteChanged |= await ApplyStepAsync(
step,
meeting,
workflowEvent,
model,
cancellationToken);
model = CreateTemplateModel(meeting, workflowEvent);
}
}
@@ -215,8 +116,6 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
saved,
cancellationToken);
}
return context.ResultValue;
}
private static bool MatchesTrigger(
@@ -259,83 +158,41 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
StringComparison.OrdinalIgnoreCase));
}
if (trigger.TranscriptLine is not null)
{
return workflowEvent.Type == MeetingWorkflowEventType.TranscriptLine &&
(string.IsNullOrWhiteSpace(trigger.TranscriptLine.Speaker) ||
string.Equals(
trigger.TranscriptLine.Speaker.Trim(),
workflowEvent.SpeakerName,
StringComparison.OrdinalIgnoreCase));
}
if (trigger.AttendeeAdded is not null)
{
return workflowEvent.Type == MeetingWorkflowEventType.AttendeeAdded &&
MatchesAttendeeAddedTrigger(trigger.AttendeeAdded, workflowEvent.AttendeeName);
}
return false;
}
private static bool MatchesAttendeeAddedTrigger(
MeetingWorkflowAttendeeAddedTrigger trigger,
string? attendeeName)
{
var attendee = attendeeName ?? "";
if (!string.IsNullOrWhiteSpace(trigger.EqualsValue) &&
!string.Equals(attendee, trigger.EqualsValue.Trim(), StringComparison.OrdinalIgnoreCase))
{
return false;
}
if (!string.IsNullOrWhiteSpace(trigger.Contains) &&
!attendee.Contains(trigger.Contains.Trim(), StringComparison.OrdinalIgnoreCase))
{
return false;
}
if (!string.IsNullOrWhiteSpace(trigger.Regex) &&
!Regex.IsMatch(attendee, trigger.Regex, RegexOptions.IgnoreCase))
{
return false;
}
return true;
}
private bool EvaluateConditions(
IReadOnlyList<MeetingWorkflowCondition> conditions,
MeetingNote meeting,
MeetingWorkflowExecutionContext context)
MeetingWorkflowEvent workflowEvent)
{
return conditions.Count == 0 ||
conditions.All(condition => EvaluateCondition(condition, meeting, context));
conditions.All(condition => EvaluateCondition(condition, meeting, workflowEvent));
}
private bool EvaluateCondition(
MeetingWorkflowCondition condition,
MeetingNote meeting,
MeetingWorkflowExecutionContext context)
MeetingWorkflowEvent workflowEvent)
{
if (!string.IsNullOrWhiteSpace(condition.Condition))
{
return EvaluateExpression(condition.Condition, meeting, context);
return EvaluateExpression(condition.Condition, meeting, workflowEvent);
}
if (condition.And is { Count: > 0 })
{
return condition.And.All(child => EvaluateCondition(child, meeting, context));
return condition.And.All(child => EvaluateCondition(child, meeting, workflowEvent));
}
if (condition.Or is { Count: > 0 })
{
return condition.Or.Any(child => EvaluateCondition(child, meeting, context));
return condition.Or.Any(child => EvaluateCondition(child, meeting, workflowEvent));
}
if (condition.Not is not null)
{
return !EvaluateCondition(condition.Not, meeting, context);
return !EvaluateCondition(condition.Not, meeting, workflowEvent);
}
return true;
@@ -344,23 +201,23 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
private bool EvaluateExpression(
string expressionText,
MeetingNote meeting,
MeetingWorkflowExecutionContext context)
MeetingWorkflowEvent workflowEvent)
{
var expression = new Expression(PrepareExpression(expressionText));
foreach (var (name, value) in BuildParameters(meeting, context))
foreach (var (name, value) in BuildParameters(meeting, workflowEvent))
{
expression.Parameters[name] = value;
}
expression.Functions["contains"] = args =>
Contains(args.Evaluate(0), Convert.ToString(args.Evaluate(1), CultureInfo.InvariantCulture));
Contains(args[0].Evaluate(), Convert.ToString(args[1].Evaluate(), CultureInfo.InvariantCulture));
expression.Functions["starts_with"] = args =>
Convert.ToString(args.Evaluate(0), CultureInfo.InvariantCulture)?.StartsWith(
Convert.ToString(args.Evaluate(1), CultureInfo.InvariantCulture) ?? "",
Convert.ToString(args[0].Evaluate(), CultureInfo.InvariantCulture)?.StartsWith(
Convert.ToString(args[1].Evaluate(), CultureInfo.InvariantCulture) ?? "",
StringComparison.OrdinalIgnoreCase) == true;
expression.Functions["ends_with"] = args =>
Convert.ToString(args.Evaluate(0), CultureInfo.InvariantCulture)?.EndsWith(
Convert.ToString(args.Evaluate(1), CultureInfo.InvariantCulture) ?? "",
Convert.ToString(args[0].Evaluate(), CultureInfo.InvariantCulture)?.EndsWith(
Convert.ToString(args[1].Evaluate(), CultureInfo.InvariantCulture) ?? "",
StringComparison.OrdinalIgnoreCase) == true;
return Convert.ToBoolean(expression.Evaluate(), CultureInfo.InvariantCulture);
@@ -369,38 +226,27 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
private async Task<bool> ApplyStepAsync(
MeetingWorkflowStep step,
MeetingNote meeting,
MeetingWorkflowExecutionContext context,
MeetingAssistantOptions options,
MeetingWorkflowEvent workflowEvent,
MeetingWorkflowTemplateModel model,
CancellationToken cancellationToken)
{
var value = await RenderValueAsync(step.Value ?? "", model);
switch (step.Uses.Trim().ToLowerInvariant())
{
case MeetingWorkflowRuleSchema.StepAddAttendee:
var attendee = await TransformAttendeeAsync(
MeetingWorkflowEvent.AttendeeAdded(context.Event.Artifacts, value),
options,
cancellationToken);
return AddUnique(meeting.Frontmatter.Attendees, attendee, MeetingAttendeeNames.NormalizeDisplayName);
case MeetingWorkflowRuleSchema.StepRemoveAttendee:
case "add_attendee":
return AddUnique(meeting.Frontmatter.Attendees, value, MeetingAttendeeNames.NormalizeDisplayName);
case "remove_attendee":
return RemoveValue(meeting.Frontmatter.Attendees, value);
case MeetingWorkflowRuleSchema.StepAddProject:
case "add_project":
return AddUnique(meeting.Frontmatter.Projects, value, static project => project.Trim());
case MeetingWorkflowRuleSchema.StepAddContext:
case "set_property":
return SetProperty(meeting, step.Property ?? step.Name, value);
case "add_context":
await meetingArtifactStore.AppendAssistantContextAsync(
context.Event.Artifacts,
workflowEvent.Artifacts,
value,
cancellationToken);
return false;
case MeetingWorkflowRuleSchema.StepSetProperty when IsTranscriptLineProperty(step.Property ?? step.Name):
SetTranscriptLine(context, value);
return false;
case MeetingWorkflowRuleSchema.StepSetProperty when IsAttendeeNameProperty(step.Property ?? step.Name):
SetAttendeeName(context, value);
return false;
case MeetingWorkflowRuleSchema.StepSetProperty:
return SetProperty(meeting, step.Property ?? step.Name, value);
default:
throw new InvalidOperationException($"Unknown meeting workflow step '{step.Uses}'.");
}
@@ -410,7 +256,63 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
string template,
MeetingWorkflowTemplateModel model)
{
return await templateRenderer.RenderAsync(template, model);
if (!ContainsRazorTemplateSyntax(template))
{
return template;
}
return await razorEngine.CompileRenderStringAsync(
Guid.NewGuid().ToString("N"),
EscapeEmailAddressAtSigns(template),
model);
}
private static bool ContainsRazorTemplateSyntax(string value)
{
if (!value.Contains('@', StringComparison.Ordinal))
{
return false;
}
var emailAtSigns = EmailAddressPattern
.Matches(value)
.Select(match => match.Index + match.Value.IndexOf('@', StringComparison.Ordinal))
.ToHashSet();
for (var index = 0; index < value.Length; index++)
{
if (value[index] != '@')
{
continue;
}
if (index + 1 < value.Length && value[index + 1] == '@')
{
index++;
continue;
}
if (emailAtSigns.Contains(index))
{
continue;
}
return true;
}
return false;
}
private static string EscapeEmailAddressAtSigns(string value)
{
if (!value.Contains('@', StringComparison.Ordinal))
{
return value;
}
return EmailAddressPattern.Replace(
value,
match => match.Value.Replace("@", "@@", StringComparison.Ordinal));
}
private static bool SetProperty(
@@ -419,7 +321,7 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
string value)
{
var normalized = property?.Trim().ToLowerInvariant();
if (normalized is MeetingWorkflowRuleSchema.PropertyTitle or MeetingWorkflowRuleSchema.PropertyMeetingTitle)
if (normalized is "title" or "meeting.title")
{
if (string.Equals(meeting.Frontmatter.Title, value, StringComparison.Ordinal))
{
@@ -433,56 +335,6 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
throw new InvalidOperationException($"Unsupported meeting workflow property '{property}'.");
}
private static void SetTranscriptLine(
MeetingWorkflowExecutionContext context,
string value)
{
if (context.Event.Type != MeetingWorkflowEventType.TranscriptLine)
{
throw new InvalidOperationException("The transcript.line property can only be set during transcript_line events.");
}
if (string.Equals(context.TranscriptLine, value, StringComparison.Ordinal))
{
return;
}
context.TranscriptLine = value;
}
private static void SetAttendeeName(
MeetingWorkflowExecutionContext context,
string value)
{
if (context.Event.Type != MeetingWorkflowEventType.AttendeeAdded)
{
throw new InvalidOperationException("The attendee.name property can only be set during attendee_added events.");
}
if (string.Equals(context.AttendeeName, value, StringComparison.Ordinal))
{
return;
}
context.AttendeeName = value;
}
private static bool IsTranscriptLineProperty(string? property)
{
return MeetingWorkflowRuleSchema.IsProperty(property, MeetingWorkflowRuleSchema.PropertyTranscriptLine);
}
private static bool IsAttendeeNameProperty(string? property)
{
return MeetingWorkflowRuleSchema.IsProperty(property, MeetingWorkflowRuleSchema.PropertyAttendeeName);
}
private static bool IsAttendeeTransformStep(MeetingWorkflowStep step)
{
return MeetingWorkflowRuleSchema.IsStep(step.Uses, MeetingWorkflowRuleSchema.StepSetProperty) &&
IsAttendeeNameProperty(step.Property ?? step.Name);
}
private static bool AddUnique(
List<string> values,
string value,
@@ -549,9 +401,8 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
private static IReadOnlyDictionary<string, object?> BuildParameters(
MeetingNote meeting,
MeetingWorkflowExecutionContext context)
MeetingWorkflowEvent workflowEvent)
{
var workflowEvent = context.Event;
return new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
{
["meeting.attendees.count"] = meeting.Frontmatter.Attendees.Count,
@@ -562,18 +413,14 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
["event.type"] = workflowEvent.Type.ToString(),
["state.from"] = workflowEvent.FromState is { } from ? MeetingWorkflowStateNames.ToRuleName(from) : "",
["state.to"] = workflowEvent.ToState is { } to ? MeetingWorkflowStateNames.ToRuleName(to) : "",
["speaker.name"] = workflowEvent.SpeakerName,
[MeetingWorkflowRuleSchema.PropertyTranscriptLine] = context.TranscriptLine ?? "",
[MeetingWorkflowRuleSchema.ParameterTranscriptSpeaker] = workflowEvent.SpeakerName ?? "",
[MeetingWorkflowRuleSchema.PropertyAttendeeName] = context.AttendeeName ?? ""
["speaker.name"] = workflowEvent.SpeakerName
};
}
private static MeetingWorkflowTemplateModel CreateTemplateModel(
MeetingNote meeting,
MeetingWorkflowExecutionContext context)
MeetingWorkflowEvent workflowEvent)
{
var workflowEvent = context.Event;
return new MeetingWorkflowTemplateModel(
new MeetingWorkflowMeetingModel(
meeting.Frontmatter.Title,
@@ -586,38 +433,6 @@ public sealed class MeetingWorkflowEngine : IMeetingWorkflowEngine
workflowEvent.ToState is { } to ? MeetingWorkflowStateNames.ToRuleName(to) : null),
string.IsNullOrWhiteSpace(workflowEvent.SpeakerName)
? null
: new MeetingWorkflowSpeakerModel(workflowEvent.SpeakerName),
new MeetingWorkflowTranscriptModel(
context.TranscriptLine ?? "",
workflowEvent.SpeakerName ?? ""),
string.IsNullOrWhiteSpace(context.AttendeeName)
? null
: new MeetingWorkflowAttendeeModel(context.AttendeeName));
}
private sealed class MeetingWorkflowExecutionContext
{
public MeetingWorkflowExecutionContext(MeetingWorkflowEvent workflowEvent)
{
Event = workflowEvent;
TranscriptLine = workflowEvent.TranscriptLineText;
AttendeeName = workflowEvent.AttendeeName;
}
public MeetingWorkflowEvent Event { get; }
public string? TranscriptLine { get; set; }
public string? AttendeeName { get; set; }
public string? ResultValue
{
get
{
return Event.Type == MeetingWorkflowEventType.AttendeeAdded
? AttendeeName
: TranscriptLine;
}
}
: new MeetingWorkflowSpeakerModel(workflowEvent.SpeakerName));
}
}
@@ -6,9 +6,7 @@ public enum MeetingWorkflowEventType
{
Created,
StateTransition,
SpeakerIdentified,
TranscriptLine,
AttendeeAdded
SpeakerIdentified
}
public sealed record MeetingWorkflowEvent(
@@ -16,9 +14,7 @@ public sealed record MeetingWorkflowEvent(
MeetingSessionArtifacts Artifacts,
AssistantContextState? FromState = null,
AssistantContextState? ToState = null,
string? SpeakerName = null,
string? TranscriptLineText = null,
string? AttendeeName = null)
string? SpeakerName = null)
{
public static MeetingWorkflowEvent Created(MeetingSessionArtifacts artifacts)
{
@@ -39,26 +35,4 @@ public sealed record MeetingWorkflowEvent(
{
return new MeetingWorkflowEvent(MeetingWorkflowEventType.SpeakerIdentified, artifacts, SpeakerName: speakerName);
}
public static MeetingWorkflowEvent TranscriptLine(
MeetingSessionArtifacts artifacts,
string line,
string speakerName)
{
return new MeetingWorkflowEvent(
MeetingWorkflowEventType.TranscriptLine,
artifacts,
SpeakerName: speakerName,
TranscriptLineText: line);
}
public static MeetingWorkflowEvent AttendeeAdded(
MeetingSessionArtifacts artifacts,
string attendeeName)
{
return new MeetingWorkflowEvent(
MeetingWorkflowEventType.AttendeeAdded,
artifacts,
AttendeeName: attendeeName);
}
}

Some files were not shown because too many files have changed in this diff Show More