mirror of
https://github.com/microsoft/semantic-kernel.git
synced 2026-10-12 02:19:06 +00:00
30aec16700
### Motivation and Context Bumps core AI SDK dependencies to their latest versions to pick up new features and fixes. ### Description Updates the following packages in `Directory.Packages.props`: - **OpenAI** 2.8.0 → 2.9.1 - **Azure.AI.OpenAI** 2.8.0-beta.1 → 2.9.0-beta.1 - **Azure.AI.Projects** 1.1.0 → 2.0.0-beta.2 - **Azure.Identity** 1.17.1 → 1.19.0 (cascading dependency from Azure.AI.Projects 2.0.0-beta.2) - **Microsoft.Extensions.AI\*** 10.3.0 → 10.4.0 - **System.Text.Json, System.Numerics.Tensors, Microsoft.Extensions.Logging.Abstractions, Microsoft.Bcl.AsyncInterfaces, System.Threading.Channels** → 10.0.4 Addresses the following breaking changes introduced by OpenAI 2.9.1: - `CreateInputImagePart` BinaryData overload removed (replaced with data URI approach) - Realtime API type renames (`ConversationTool` → `RealtimeTool`, `ConversationSessionOptions` → `RealtimeConversationSessionOptions`, etc.) - `ResponsesClient` constructor no longer accepts a model parameter; model is now specified on `CreateResponseOptions.Model` - `ResponseResult.Instructions` is now read-only Code changes to adapt: - Added `ModelId` property to `OpenAIResponseAgent` and propagated it through `ResponseCreationOptionsFactory` - Updated unit tests (`BaseOpenAIResponseClientTest`, `OpenAIResponseAgentExtensionsTests`, `OpenAIResponseExtensionsTests`) - Updated integration tests (`OpenAIResponseAgentTests`, `OpenAIResponseAgentFixture`) - Updated `BaseResponsesAgentTest` and `GettingStartedWithAgents` samples - Updated `AgentFrameworkMigration` samples (`GetResponsesClient`, `AsIChatClient`, `CreateAIAgent`, modelId propagation) - Rewrote `OpenAIRealtime` demo for the restructured Realtime API event model ### Contribution Checklist - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 <!-- START COPILOT CODING AGENT TIPS --> --- 📍 Connect Copilot coding agent with [Jira](https://gh.io/cca-jira-docs), [Azure Boards](https://gh.io/cca-azure-boards-docs) or [Linear](https://gh.io/cca-linear-docs) to delegate work to Copilot in one click without leaving your project management tool. --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: stephentoub <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Roger Barreto <[email protected]>
OpenAI Realtime API
This console application demonstrates the use of the OpenAI Realtime API with function calling and Semantic Kernel.
For conversational experiences, it is recommended to use RealtimeConversationClient from the Azure/OpenAI SDK.
Since the OpenAI Realtime API supports function calling, the example shows how to combine it with Semantic Kernel plugins and functions.
Configuring Secrets
The example requires credentials to access OpenAI or Azure OpenAI.
If you have set up those credentials as secrets within Secret Manager or through environment variables for other samples from the solution in which this project is found, they will be re-used.
To set your secrets with Secret Manager:
cd dotnet/samples/Demos/OpenAIRuntime
dotnet user-secrets init
dotnet user-secrets set "OpenAI:ApiKey" "..."
dotnet user-secrets set "AzureOpenAI:DeploymentName" "..."
dotnet user-secrets set "AzureOpenAI:Endpoint" "https://... .openai.azure.com/"
dotnet user-secrets set "AzureOpenAI:ApiKey" "..."
To set your secrets with environment variables
Use these names:
# OpenAI
OpenAI__ApiKey
# Azure OpenAI
AzureOpenAI__DeploymentName
AzureOpenAI__Endpoint
AzureOpenAI__ApiKey