meeting-assistant/openspec/changes/archive/2026-07-27-handle-responses-event-streams/design.md

Context

LiteLlmResponsesChatClient exposes a non-streaming IChatClient interface and currently buffers every successful /v1/responses body before parsing it as one JSON document. The deployed LiteLLM chatgpt/gpt-5.5 route instead returns Responses Server-Sent Events. Microsoft.Extensions.AI.OpenAI already provides a Responses streaming adapter that maps the OpenAI SDK's typed SSE updates into ChatResponseUpdate values, including text, reasoning, function calls, response metadata, and usage.

That adapter uses FunctionCallContent.CreateFromParsedArguments, which records argument-mapping failures on the function call instead of throwing them from the response parser. The default function invoker does not itself prevent invocation when that property is populated, so Meeting Assistant must turn the recorded parse failure into a matching function result before calling the tool.

Goals / Non-Goals

Goals:

• Preserve the existing non-streaming IChatClient contract while selecting the supported Responses streaming or non-streaming path through configuration.
• Delegate SSE framing, event deserialization, streamed function-call assembly, response metadata, and usage mapping to the OpenAI SDK and Microsoft.Extensions.AI.OpenAI adapter.
• Preserve the original encoded function arguments when forwarding conversation history.
• Return a safe, structured invalid-arguments result for malformed argument JSON without invoking the requested tool.
• Apply the guarded invocation behavior to both summary and workflow-editor agent pipelines that use this Responses client.

Non-Goals:

• Expose token-by-token upstream streaming to the UI.
• Replace Microsoft.Extensions.AI function invocation or implement general JSON Schema validation.
• Recover from an incomplete or malformed Responses event stream that has no usable completed output.

Decisions

1. Add Agent:UseStreaming, defaulting to true, and send summary requests through the matching OpenAI SDK method. The streaming path uses ResponsesClient.CreateResponseStreamingAsync and collects the Microsoft.Extensions.AI.OpenAI typed updates into one ChatResponse; the non-streaming path uses ResponsesClient.CreateResponseAsync and its supported AsChatResponse adapter.

2. Keep the existing JSON payload builder because it owns Meeting Assistant-specific compaction input, tool serialization, reasoning settings, retry diagnostics, and initiator behavior. Deserialize that payload into the OpenAI SDK's CreateResponseOptions, set its streaming flag from configuration, and let the selected SDK method own the HTTP response protocol.

   The SDK deserializes ResponseItem values by their type discriminator. Message items must therefore include "type": "message" before the JSON payload is converted to CreateResponseOptions; otherwise the SDK reserializes them as "type": "unknown" and the provider rejects the request.

3. Rely on the framework adapter's FunctionCallContent.CreateFromParsedArguments mapping for function-call argument decoding and raw response preservation. A shared function-invocation callback checks the recorded parse exception before invocation. For invalid JSON it returns a structured invalid_tool_arguments value that the framework associates with the original call ID; otherwise it delegates to the actual function.

4. Configure both agent pipelines with the shared guarded invoker. This avoids duplicating the safety decision and prevents zero-argument tools from accidentally running when malformed JSON would otherwise map to an empty argument set.

Risks / Trade-offs

• [Provider event variants] A provider could emit events outside the OpenAI Responses schema. → Use the maintained OpenAI SDK parser and fail diagnostically for genuinely incompatible streams instead of maintaining local event variants.
• [Buffered result] The summary pipeline does not expose token-by-token updates to its caller. → Collect the framework updates only at the existing non-streaming boundary; the HTTP response itself remains streamed and incrementally parsed.
• [Experimental API] The OpenAI Responses adapter is marked experimental in the currently referenced package. → Keep it behind LiteLlmResponsesChatClient, which isolates future package API changes from the rest of Meeting Assistant.
• [Error disclosure] Raw parser exceptions may contain implementation details. → Return a stable error code and concise validation message rather than exception text or the malformed arguments.