mirror of
https://github.com/microsoft/semantic-kernel.git
synced 2026-10-12 02:19:06 +00:00
dotnet-1.80.1
210
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
b39d95a344 |
Python: [Breaking] Remove unsupported service auth mode from Copilot Studio agent (#14306)
### Motivation and Context The Copilot Studio agent exposed a `SERVICE` authentication mode that was never reachable — it was guarded to always raise before its implementation ran. Its dormant credential handling also triggered certificate-related static analysis alerts. ### Description Removes the service authentication path along with its settings, parameters, tests, and documentation. `CopilotStudioAgentAuthMode` is kept with its `INTERACTIVE` member, which is the only supported mode. Interactive authentication is unchanged. Service authentication can be reintroduced later as a complete, tested feature. ### Contribution Checklist - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Copilot-Session: 25dd6e2a-f759-4148-a630-40110e90eff2 |
||
|
|
3da48a3d01 |
Python: [Breaking] Add MCP tool approval callback for Azure AI Agent (#14210)
## Summary - Add an Azure AI Agent callback for MCP tool approval requests. - Route MCP tool approval submissions through a shared helper in thread actions. - Add unit coverage for approval, denial, callback failures, and per-call decisions. ## Testing - python -m pytest tests/unit/agents/azure_ai_agent --------- Copilot-Session: 3e190979-228b-4a82-8e63-5ad14bd4879f |
||
|
|
3df33f7862 |
Python: [Breaking] Update runtime handling (#14135)
This PR updates Python runtime handling and related tests. --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Evan Mattson <[email protected]> |
||
|
|
0f322bcc48 |
Python: Improve prompt-template msg serialize and sample usage (#13738)
### Motivation and Context This PR updates the Jinja2 and Handlebars prompt-template helpers to serialize chat messages through the existing XML/message serializer instead of assembling message XML manually. It also aligns the prompt-template samples with the serializer-backed helper and adds regression coverage for common message content. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Prompt template message serialize improvements. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 --------- Co-authored-by: MAF Dashboard Bot <[email protected]> Co-authored-by: Copilot <[email protected]> |
||
|
|
3f6d72725f |
Python: Fix image integration tests that are now blocked due to bot policies (#13680)
### Motivation and Context Fix image integration tests that are now blocked due to bot policies <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Fix image integration tests that are now blocked due to bot policies <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
|
5c891d1769 |
Python: Add support to new openai text to image model (#13651)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Dall-e-3 and Dall-e-2 have been deprecated. The newer gpt-1-image model returns base64 strings. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Return url if set, otherwise return the base64 string. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 --------- Co-authored-by: Evan Mattson <[email protected]> |
||
|
|
d5c2b9d889 |
Python: Improves the robustness of filename handling (#13643)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Improves the robustness of filename handling in the Bedrock agent integration. Ensures that filenames sourced from external responses are properly sanitized before use in file operations, and adds safer defaults for file write behavior. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> 1. Sanitize filenames received from Bedrock agent responses. 2. Add overwrite parameter to BinaryContent.write_to_file (defaults to False). 3. Add tests. 4. Update samples. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
|
429dd1c4c0 |
Python: support (Azure) OpenAI realtime audio models (#13291)
### Motivation and Context As of 2/27/2026, revived the PR, and the latest code is working for both Azure OpenAI realtime models and OpenAI realtime models. SK has support for realtime-preview models; however, since they've gone to GA we have not added support for the latest library abstractions. This PR brings in the changes to support running models like `gpt-realtime-1.5`, `gpt-realtime`, `gpt-realtime-mini` or `gpt-audio`. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description - Closes #13267 - Code now relies on `openai` >= 2.0 - Considered a breaking change due some new config added to the execution settings. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
|
d84363f477 |
Python: Migrate to new Google GenAI SDK (#13371)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Closes #12970, #13352 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> 1. Migrate Google AI connector to use the new Google AI SDK. 2. This PR also marks the VertexAI connector as deprecated in favor of the Google AI connector. With this update, if users wants to connect to a VertexAI project instead of a Google AI endpoint, they can do so with the Google AI connector. 3. This PR also adds the ability for users to provide a custom client to the Google AI connector (#13352). 4. This PR also marks `AzureTextCompletion` deprecated. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
|
145ad9bfd2 |
Python: Upgrade Onnx Connector to use 0.9.0 (#13162)
### Motivation and Context Fixes : #13001 ### Description Package version 0.9.0 introduced very helpful methods like applying chat templates automatically from the tokenizer and also support for multi audio & image files. This PR adds the following functionalities: - [x] Add Inference with multiple Images - [x] Add Inference with multiple Audios - [x] Chat Templates for non Multimodal Models are read via onnx ### Samples Text Sample with ONNX: <img width="1728" height="90" alt="image" src="https://github.com/user-attachments/assets/b93d9fc1-4e38-4fa9-b535-cc316e0900ed" /> Image Sample with ONNX : <img width="1721" height="91" alt="image" src="https://github.com/user-attachments/assets/47761beb-b728-4936-a6b6-fd253e041689" /> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Evan Mattson <[email protected]> |
||
|
|
1c11258a32 |
Python: added af_tool bridge for kernel_functions (#13227)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Adds a new method to KernelFunction that creates a Agent Framework Tool from that function. Does require the user to install agent-framework-core themselves. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
61d3e6ee8d |
Python: Introduce NvidiaChatCompletion AI Connector (#12952)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? Enables developers to use NVIDIA's chat models through Semantic Kernel with structured output support. 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This change adds chat completion support with structured output to the existing NVIDIA connector, enabling developers to use NVIDIA's chat models through Semantic Kernel while building conversational AI applications that require structured, validated responses. The connector will support more model types iteratively, with future PRs introducing VLM models and additional capabilities. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> - New Chat Completion Service: `NvidiaChatCompletion`: Chat completion service following same structure as other connectors - Extends existing NVIDIA embedding connector patterns and architecture - Custom logic for handling structured output specific to NVIDIA models - Enhanced Configuration: `NvidiaChatPromptExecutionSettings`: Chat-specific settings following existing connector patterns ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
d7db35dc47 |
Python: Fix Azure AI Inference connector model_extras duplication (#13066)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Addresses #13005 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> 1. Remove the `extra_parameters` key from `AzureAIInferenceChatPromptExecutionSettings` before calling the chat completion API to avoid unknown parameter error. 2. Allow setting the API version on the `AzureAIInferenceXXX` connectors. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
f7fdfcd9a9 |
Python: Add AzureAIAgent Deep Research Tool support (#13034)
### Motivation and Context The deep research tool was introduced for the AzureAIAgent; however, we didn't have support yet in SK. Adding support now for using the deep research tool for both streaming and non-streaming. Samples added as well. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description - Add support for deep research tool. - Add samples with notes on prerequisites to configure the sample to run properly. - Added support for `AZURE_AI_AGENT_DEEP_RESEARCH_MODEL` env var within `AzureAIAgentSettings` class. - Bumped min allowed `azure-ai-agents` package ver to 1.2.0b3. - Closes #12978 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
86b10b95d2 |
Python: provide AzureAIAgent thread msg id during streaming (#13032)
### Motivation and Context As part of invoking an AzureAIAgent via streaming, we weren't providing the `thread_id` via the `AgentStreamEvent.THREAD_MESSAGE_CREATED` event. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Supply the caller with the `thread_msg_id` by sticking it into the returned SCMC's `metadata` dictionary. - Closes #12959 Sample output looks like: ``` Sample Output: # User: 'Hello' # AuthorRole.ASSISTANT: (thread message id: msg_HZ2h4Wzbj7GEcnVCjnyEuYWT) Hello! How can I assist you with the menu today? # User: 'What is the special soup?' # AuthorRole.ASSISTANT: (thread message id: msg_TSjkJK6hHJojIkPvF6uUofHD) The special soup today is Clam Chowder. Would you like to know more about it or anything else from the menu? # User: 'How much does that cost?' # AuthorRole.ASSISTANT: (thread message id: msg_liwTpBFrB9JpCM1oM9EXKiwq) The Clam Chowder costs $9.99. Is there anything else you'd like to know? # User: 'Thank you' # AuthorRole.ASSISTANT: (thread message id: msg_K6lpR3gYIHethXq17T6gJcxi) You're welcome! If you have any more questions or need assistance, feel free to ask. Enjoy your meal! ``` <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
5e50e190c4 |
Python: Add reasoning support for OpenAI Responses Agents (GPT-5, o4-mini, o3) (#12881)
## Summary Adds reasoning capabilities to Python OpenAI ResponsesAgent, bringing parity with the C# implementation. This enables fine-grained control over reasoning effort for O-series models (o1, o3-mini, o4-mini) and gpt-5 that support the reasoning parameter. Fixes #12843 ## Changes ### Core Implementation - **`openai_responses_agent.py`**: Added reasoning configuration support - Constructor-level reasoning effort setting - Per-invocation reasoning effort override capability - Proper parameter validation and model compatibility checks - **`responses_agent_thread_actions.py`**: Extended thread actions to support reasoning parameters - Reasoning effort propagation through thread operations - Metadata preservation for reasoning tokens and summaries ### Sample and Tests - **`responses_agent_reasoning.py`**: Comprehensive demonstration sample - Constructor vs per-invocation reasoning configuration - Function calling integration with reasoning - Reasoning comparison scenarios (low/medium/high effort) - Error handling and troubleshooting guidance - **`test_openai_responses_agent_reasoning.py`**: Full unit test coverage - Parameter validation tests - Integration scenarios with function calling - Edge cases and error conditions ## Features ✅ **Constructor-Level Reasoning**: Set default reasoning effort when creating agents ✅ **Per-Invocation Override**: Override reasoning effort per request ✅ **Priority Hierarchy**: per-invocation > constructor > model default ✅ **Function Calling Compatible**: Works seamlessly with existing plugin system ✅ **Azure OpenAI & OpenAI Support**: Compatible with both service providers ✅ **Model Validation**: Automatic compatibility checks for O-series models nad GÜT-5 ✅ **Metadata Access**: Reasoning tokens and summaries available in response metadata ## Usage Example ```python # Constructor-level reasoning # Constructor-level reasoning configuration agent = AzureResponsesAgent( ai_model_id="gpt-5", client=client, reasoning={"effort": "low"} # Default reasoning for all requests ) # Per-invocation override response = await agent.invoke( "Solve this complex problem step by step", reasoning={"effort": "high"} ) # Invoke with reasoning callback to capture intermediate thoughts response = await agent.invoke( "Analyze this data step by step", reasoning={"effort": "high", "summary": "detailed"}, on_intermediate_message=handle_reasoning_message ) # Streaming with reasoning async for response in agent.invoke_stream( "Explain quantum computing in detail", reasoning={"effort": "high", "summary": "detailed"}, on_intermediate_message=handle_reasoning_message ): print(response.content, end="", flush=True) --------- Co-authored-by: Evan Mattson <[email protected]> Co-authored-by: Evan Mattson <[email protected]> |
||
|
|
18d9a8e8b2 |
Python: .Net: Updated encoding logic in prompt templates (#12983)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Resolves: https://github.com/microsoft/semantic-kernel/issues/11821 Today, the encoding of template arguments is performed only if argument type is `string`. In case of custom type, anonymous type or collection - the encoding is not performed. This PR contains changes to throw an exception in case if encoding is enabled but complex type is used. In case of complex type, the encoding should be performed manually according to business logic and automatic encoding should be explicitly disabled. This enforces stricter, but more secure template rendering rules. **Note**: this is a breaking change for customers who use Handlebars or Liquid template with complex type arguments. Code changes are required when initializing template arguments: ```diff var arguments = new KernelArguments() { { "customer", new { - firstName = userInput.FirstName, - lastName = userInput.LastName, + firstName = HttpUtility.HtmlEncode(userInput.FirstName), + lastName = HttpUtility.HtmlEncode(userInput.LastName), } } }; var templateFactory = new LiquidPromptTemplateFactory(); var promptTemplateConfig = new PromptTemplateConfig() { TemplateFormat = "liquid" + InputVariables = new() + { + // We set AllowDangerouslySetContent to 'true' because each property of this argument is encoded manually. + new() { Name = "customer", AllowDangerouslySetContent = true }, + } }; var promptTemplate = templateFactory.Create(promptTemplateConfig); var renderedPrompt = await promptTemplate.RenderAsync(kernel, arguments); ``` ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
|
2a47ba9b7d |
Python: Removed usage of DefaultAzureCredential (#12964)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> The usage of `DefaultAzureCredential` by default is unsafe and raises security issues. Instead, users should provide their own credential types such as `AzureCliCredential`, `ManagedIdentityCredential` and so on. More information here: https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.aio?view=azure-python In this PR: 1. Removed the usage of `DefaultAzureCredential` from authentication logic. 2. Replaced `DefaultAzureCredential` with `AzureCliCredential` in all examples. **Note**: this is breaking change for Azure users, who relies on `DefaultAzureCredential`. Users will need to update the code and pass the credential type of their choice by using `credential` parameter when initializing one of the Azure clients. Before: ```python chat_completion = AzureChatCompletion() ``` After: ```python chat_completion = AzureChatCompletion(credential=AzureCliCredential()) ``` ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
|
96bad5ceac |
Python: Update MCP streamable sample (#12950)
### Motivation and Context The MCPStreamableHttpPlugin concept sample pointing to GitHub isn't working. Updating it to use the Microsoft Learn site MCP server instead. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Update sample. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
8336c320d9 |
Python: Add bedrock model provider parameter (#12853)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Fixes #12833 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Add an optional setting to the Bedrock AI connectors to allow users to specify the Bedrock model providers (i.e. amazon, anthropic, etc) so that they can use application inference profiles. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Dmytro Struk <[email protected]> |
||
|
|
c16db27622 |
Python: Update README.md (#12737)
Update links due to renamed files ### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [ ] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [ ] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Evan Mattson <[email protected]> |
||
|
|
c79f5a5c63 |
Python: improved MCP connect and additional samples (#12696)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Adding a sample using StreamableHTTPServer towards Github. Added a demo showing how to connect to a OAuth protected MCP server. Also some improvements in the connect logic for MCP. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
48f71b1f2e |
Python: Support AzureAI agent MCP tools for streaming and non-streaming invocations (#12736)
### Motivation and Context MCP tool support was released for the AzureAI Agent and it has yet to be added to the SK Python AzureAIAgent. This PR adds functionality to handle the MCP tools for both streaming and non-streaming invocations. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Adds MCP support for the Python AzureAI Agent. - Upgrades dependent packages to be able to use the mcp tool from the Azure SDK. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
9e3cdffc6b |
Python: Improve AzureAIAgent get thread msg handling. (#12535)
### Motivation and Context Currently, when fetching thread messages for an AzureAIAgent, it assumes that the agent exists, so we can pull it down and get its id. This may not always be the case - an agent could have been deleted, but the thread still exists. We need to account for this. When creating user messages that we add to the thread, there's no `name` or `id` parameter allowed. We can, however, add metadata to the message. Going forward we are adding the `agent_id` as metadata to the message, so upon retrieval later, we can attempt to get it and return the id with the user's message. This allows one to see which agent_id has responded to the user's message: ``` # Messages in the thread (asc order): # AuthorRole.USER for name=asst_mXwZOwyJLxXGtaYKHizRH6Ip: Hello # AuthorRole.ASSISTANT for name=asst_mXwZOwyJLxXGtaYKHizRH6Ip: Hello! How can I assist you with the menu today? # AuthorRole.USER for name=asst_mXwZOwyJLxXGtaYKHizRH6Ip: What is the special soup? # AuthorRole.ASSISTANT for name=asst_mXwZOwyJLxXGtaYKHizRH6Ip: The special soup today is Clam Chowder. Would you like to know more about it or anything else on the menu? # AuthorRole.USER for name=asst_mXwZOwyJLxXGtaYKHizRH6Ip: How much does that cost? # AuthorRole.ASSISTANT for name=asst_mXwZOwyJLxXGtaYKHizRH6Ip: The Clam Chowder costs $9.99. Would you like to order it or need information on other items? # AuthorRole.USER for name=asst_mXwZOwyJLxXGtaYKHizRH6Ip: Thank you # AuthorRole.ASSISTANT for name=asst_mXwZOwyJLxXGtaYKHizRH6Ip: You're welcome! If you have any more questions or need assistance with the menu, feel free to ask. Enjoy your meal! ``` <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description - Closes #12435 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
003cc4c010 |
Python: Feature python vector stores preb (#12271)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Overhaul of the VectorStores to get it matching with dotnet and the latest development there. Closes: - #11938 - #11598 - #11597 - #11517 - #10561 - #10391 - #9911 - #9892 - #10867 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Many changes, some highlights: - Moved from different vector fields types, to a single with a enum first VectorStoreField() - Renamed several items - Made embeddinggenerrator a part of the vector field spec and do automatic vectorization from there or from the same param on a collection - Adds hybrid search using the same setup - Removed the intermediate TextSeachVectorSearch object need, by allowing `create_search_function` directly on a collection Will write out full changes in a migration guide in the docs. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
|
bd4168316d |
Python: add Foundry local sample (#12214)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Sample on how to use Foundry Local with SK. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
9b8f8e22c3 |
Python: Correct chat completion agent history truncation samples (#12474)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> While addressing the developer in this [discussion](https://github.com/microsoft/semantic-kernel/discussions/12450), I notice that there are a few changes needed to the improve the truncation history reducer samples of the chat completion agent. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Update the samples. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [ ] The code builds clean without any errors or warnings - [ ] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [ ] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
|
4b23389f71 |
Python: Emit token usage with streaming chat completion agent. (#12416)
### Motivation and Context
The chat completion agent was not emitting token using during streaming
invocation because we were only allowing through `response.items`. In
the case of token usage, `response.items` is [] and the usage is
contained as part of the message's `metadata` dict. This PR fixes that
bug and allows for `response.items or response.metadata.get("usage")`.
Two new samples are added to the concepts/agents/chat_completion dir to
show how one can track token use for streaming and non-streaming agent
invocation. Token usage handling is also added to the chat completion
agent integration tests.
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
- Fixes a bug where we weren't emitting the streaming token usage for
the chat completion agent. Also now includes the `prompt_tokens_details`
and `completion_tokens_details` models that are returned, but not
previously handled.
- Adds new samples
- Updates integration tests to track token usage and make sure they're
non-zero.
- Closes #12411
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
|
||
|
|
e0221e58ce |
Python: Fix OpenAPI concept sample (#12418)
### Motivation and Context An OpenAPI concept sample that we have is broken. This PR fixes it to run properly based on the current OpenAPI plugin handling. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Fixes the broken OpenAPI sample. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
0e7556e112 |
Python: Add file handling support to BinaryContent for OpenAI Responses API (#12258)
## Summary
Enhances `BinaryContent` to support file handling for OpenAI Responses
API, enabling file uploads through the responses agent while maintaining
a provider-agnostic design.
## Changes
### BinaryContent Enhancements
- **Add `can_read` property**: Indicates whether content has readable
data available
- **Add `from_file()` class method**: Creates BinaryContent instances
from file paths with automatic base64 encoding
- **Fix Unicode handling**: Prevents decode errors when processing
binary files (PDFs, images, etc.)
### OpenAI Responses Agent Integration
- **Add BinaryContent support**: Pattern matching case for file handling
in `responses_agent_thread_actions.py`
- **Correct OpenAI API format**: Uses proper `filename` and `file_data`
structure with data URI format
- **UUID-based filenames**: Generates appropriate filenames with
mime-type extensions
- **Provider-specific mapping**: File format conversion happens only in
OpenAI agent code
### Testing
- **Test coverage**: New tests for `can_read`, `from_file()`, and binary
data handling
- **Unicode error prevention**: Specific test for binary PDF-like
content
- **Base64 encoding verification**: Ensures proper data format for API
compatibility
## Design Principles
- **Provider-agnostic**: BinaryContent remains completely generic with
no OpenAI-specific dependencies
- **Clean separation**: OpenAI format mapping isolated to OpenAI agent
files
- **No FileContent class**: Enhances existing BinaryContent instead of
introducing new types
- **Follows existing patterns**: Similar approach to ImageContent and
TextContent handling
## Usage Example
```python
response = await self.agent.get_response(
messages=ChatMessageContent(
content="Analyse PDF",
role=AuthorRole("user"),
items=[BinaryContent.from_file(file_path=c":/test.pdf")],
)
)
Co-authored-by: Evan Mattson <[email protected]>
|
||
|
|
2f84e5d1bf |
Python: Add chat completion agent code interpreter sample (#12393)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Closing #10962 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> 1. Add a sample to demonstrate how to create a chat completion agent with code interpreter capabilities using the Azure Container Apps session pool service. 2. Fix a bug that cause the session tool to throw because the upload API doesn't return the file uploaded. 3. Modify unit tests accordingly. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
c822511f2c |
Python: Add special notes to samples that are still using AgentGroupchat (#12377)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> As we move away from `AgentGroupChat` to the new orchestrations, we want to make sure it's clear to developers getting started with SK are aware of the changes. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> We are including a special note to all the samples that use `AgentGroupChat` to inform developers that this feature is no longer maintained and they should consider using `GroupChatOrchestration` instead. We are keeping the samples for now so that developers can still refer to them and we don't break any external links to these samples, but we want to make sure they are aware of the changes and can migrate to the new orchestrations. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
fb0e418a7c |
Python: Fix for AzureAIAgent streaming func call invocations. Other fixes/improvements for tool call content types, and logging. (#12335)
### Motivation and Context
The AzureAIAgent streaming function call invocation is broken when
handling more than one required function call from the service. There is
nested logic to handle passing the tool calls back to the service
because it requires one to pass in a handler.
In this PR, we're now controlling everything from the main loop and are
passing back the handler as the event stream for further processing.
This does require some logic to determine if we have an initial async
context manager (based on the first call to create the run stream) or if
we're now using the handler from submitting tool calls. We can now see
that when we have multiple functions involved, we are able to get the
results as we expect:
```
Sample Output:
# User: 'What is the price of the special drink and then special food item added together?'
Function Call:> MenuPlugin-get_specials with arguments: {}
Function Result:>
Special Soup: Clam Chowder
Special Salad: Cobb Salad
Special Drink: Chai Tea
for function: MenuPlugin-get_specials
Function Call:> MenuPlugin-get_item_price with arguments: {"menu_item": "Chai Tea"}
Function Call:> MenuPlugin-get_item_price with arguments: {"menu_item": "Clam Chowder"}
Function Result:> $9.99 for function: MenuPlugin-get_item_price
Function Result:> $9.99 for function: MenuPlugin-get_item_price
Function Call:> MathPlugin-Add with arguments: {"input":9.99,"amount":9.99}
Function Result:> 19.98 for function: MathPlugin-Add
# AuthorRole.ASSISTANT: The price of the special drink, Chai Tea, is $9.99 and the price of the special food
item, Clam Chowder, is $9.99. Added together, the total price is $19.98.
```
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
Fix handling multiple function calls for the streaming AzureAIAgent.
- Fixes a missing `quote` from the url citation for streaming annotation
content
- Shows the correct way to retrieve Bing Grounding tool calls in the two
concept samples.
- Adds handling for tool content content for OpenAPI plugins.
- Adds logging for streaming agent invocation tool calls.
- Relaxes the handling of the `FunctionResultContent` id attribute to be
`str | None` similar to that of `FunctionCallContent`. The type hints
for `id` have always been `str | None`, but the actual type hint on the
attribute has been `str`. We don't always get a `FunctionResultContent`
id from the Azure Agent Service during streaming invocations - message
addition via the overide `__add__` method still work properly.
- Move the AzureAIAgent pkg dependencies to be installed by default,
instead of requiring the `azure` extra.
- Closes #12312
- Closes #12328
- Closes #12331
- Closes #12324
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
|
||
|
|
82229cecc3 |
Python: Improve Python README (#12289)
### Motivation and Context The Python README file is a bit stale, and lacking current content related to agent framework and directly calling the AI service (not always needing to invoke via the kernel). Additionally, there are links to samples/notebooks that are relative and thus not resolving from pypi. The [`ALL_SETTINGS.md` guide](https://github.com/microsoft/semantic-kernel/blob/main/python/samples/concepts/setup/ALL_SETTINGS.md#semantic-kernel-settings) is also missing the Agent Framework settings used. Adding those in this PR. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Improve the README to be more relevant. - Closes #12286 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
ff1997047d |
Python: Support Declarative Spec for OpenAIAssistantAgent & OpenAIResponsesAgent (#12247)
### Motivation and Context This PR builds on the initial work/abstractions from #11982. The `AzureAssistantAgent` / `OpenAIAssistantAgent` as well as the `AzureResponsesAgent` / `OpenAIResponsesAgent` now support declarative specs. Several examples on how to declare the spec are shown in the respective dirs in `samples/concepts/agents`, as well as a simple getting started example for each agent type. To further simplify working with these two agents, the `setup_resources()` methods for each agent type have been marked with a deprecation tag. The new method to use is `OpenAIAssistantAgent.create_client()` or `OpenAIResponsesAgent.create_client()`. Further, when one needs to access the underlying setting's, it's simple to do so with a settings object and accessing the attribute. In the previous PR for the AzureAIAgent, it lacked support for templating - this is included in this current PR. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Bring in support for the declarative single agent spec for these agent types. - Add samples - Add unit tests - Add support for prompt templates for all current agent types - Closes #11979 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
c084b067de |
Python: Support structured outputs with Azure AI inference chat completion. (#12183)
### Motivation and Context The Azure AI inference SDK released support for structured outputs for chat completions (for models that support it). We had not brought this into SK. This PR adds functionality and the ability to run the structured output samples that exist today. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Adds functionality to handle structured outputs for the Azure AI Inference Chat Completion calls. - Fixes a bug within AzureAIInferenceChatCompletion where the wrong isinstance check type is used. This prevented adding tool calls during streaming invocation. Also properly add the `ai_model_id` from the connector during message construction during streaming invocation, - Adds unit tests for AzureAIInferenceChatCompletion structured outputs - Closes #11952 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
c64e51c40e |
Python: Update the AzureAIAgent packages, types, and APIs based on Azure SDK's new code (#12084)
### Motivation and Context The Azure AI Agent Service is moving to GA soon. As part of that, they've made several changes in the underlying APIs, which require the use of new packages, updated type imports and some new underlying APIs. This PR updates the AzureAIAgent to use the latest Azure SDK's new code. - The client's connection string has been replaced with an endpoint. - The `AzureAIAgentSettings`'s `model_deployment_name` was updated to `deployment_name`. - Tools are now imported from `from azure.ai.agents.models` instead of `from azure.ai.projects.models`. - Declarative spec handling for AzureAIAgent was updated based on the new APIs. - Unit tests were updated. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Update the AzureAIAgent related code, samples and tests based on the Azure SDK's latest APIs. - Closes #12177 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 --------- Co-authored-by: Tao Chen <[email protected]> |
||
|
|
03f0aba633 |
Python: added WebRTC support for Azure OpenAI Realtime (#12078)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Added support for WebRTC for Azure OpenAI Realtime models. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
d7c2489a24 |
Python: Add MultiAgent Orchestration to README. (#12073)
### Motivation and Context The new multi-agent orchestration samples are missing from the getting started with agents README. Add them in with their descriptions. Also, remove two unused intermediate message callback lists from two samples that are no longer used. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Add samples to README. Clean up some samples. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
6115e8d7dc |
Python: fixes and updates for MCP (#12039)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Removes the two surpressed calls for extra info out of a MCP server, since they seemed to hang every now and then. Updated the package version spec Added one of the samples as a integration test Added: - load_tools flag on the constructor, default = true - load_prompt flag on the constructor, default = true - request_timeout param, optional, default None ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
ea3215ebb7 |
Python: Support Declarative Agent Spec for ChatCompletionAgent & AzureAIAgent (#11982)
### Motivation and Context This is the first installment of a series of PRs related to adding single agent declarative spec support. SK .NET has had support for single agent declarative specs for a bit now, and Python needs the functionality as well. This PR adds the core abstractions and functionality to be able to define agents via a declarative yaml spec. Declarative specs require the use of a kernel, and if in the case of function calling, require that any plugins defined in the spec already be present in the kernel. For naming, there are placeholder names that are camel case. This needs to be the case so it can match .NET. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Provide declarative spec support for ChatCompletionAgent and AzureAIAgent - Adds many samples to show the use of declarative specs to handle various AzureAIAgent + ChatCompletionAgent tools/plugins. - Closes #11978 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
904c10a11c |
Python: Add support for MCP Streamable HTTP transport (#12001)
### Motivation and Context Added support for the new Streamable HTTP transport introduced in the `2025-03-26` revision of MCP, which replaces SSE. ### Description 1. Bump the `mcp` library version to `1.8` 2. Introduced a new `MCPStreamableHttpPlugin` class ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
4724dbc8f3 |
Python: Fix AzureAIAgent intermediate message ordering for invoke and invoke_stream (#12005)
### Motivation and Context This PR fixes a core issue in `AzureAIAgent` where `FunctionCallContent` and `FunctionResultContent` messages were not being emitted in the correct order during `invoke()` and `invoke_stream()` execution. - `invoke_stream`: tool messages are now drained and emitted after each streamed assistant chunk, preserving causal ordering (tool call -> result -> assistant reply). - `invoke`: tool messages are emitted via callback immediately after the non-streaming response is received, ensuring they precede the final assistant message if applicable. ### Description Updated samples to better show how the callbacks are used to receive the intermediate messages in the proper order. - Added unit tests to ensure proper coverage and message ordering. - Closes #11999 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
b07454aff8 |
Python: Fix OpenAIAssistantAgent/OpenAIResponsesAgent intermediate message ordering for invoke and invoke_stream (#12008)
### Motivation and Context This PR fixes a core issue in `OpenAIAssistantAgent` and `OpenAIResponsesAgent` where `FunctionCallContent` and `FunctionResultContent` messages were not being emitted in the correct order during `invoke()` and `invoke_stream()` execution. - `invoke_stream`: tool messages are now drained and emitted after each streamed assistant chunk, preserving causal ordering (tool call -> result -> assistant reply). - `invoke`: tool messages are emitted via callback immediately after the non-streaming response is received, ensuring they precede the final assistant message if applicable. ### Description Updated samples to better show how the callbacks are used to receive the intermediate messages in the proper order. - Added unit tests to ensure proper coverage and message ordering. - Closes #12007 ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [ ] The code builds clean without any errors or warnings - [ ] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [ ] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
|
fb2fc6b30c |
Python: Fix ChatCompletionAgent intermediate message ordering for invoke and invoke_stream (#12000)
### Motivation and Context When using the `on_intermediate_message` callback, we are returning messages to the caller in an incorrect order. While using function calling, the order is: 1. Provide user input to agent that requires a tool call 2. FunctionCallContent chunks returned via callback 3. FunctionResultContent returned via callback 4. Final output (StreamingTextContent/TextContent) from agent returned via invoke_stream/invoke What we are really seeing is: 1. Provide user input to agent that requires a tool call 2. Final output (StreamingTextContent/TextContent) from agent returned via invoke_stream/invoke 3. FunctionCallContent chunks returned via callback 4. FunctionResultContent returned via callback The callback needs a bit of adjusting -- for the streaming case it must emit messages as part of handling the AsyncGen response_list loop. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Fix the ordering of the intermediate callback so messages aren't received out of order. - Adds unit tests to check message order - Adjusts some samples - Closes #11998 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
d266ac5db0 |
Python: Properly surface streaming code responses and handle Bing Grounding for AzureAIAgent (#11850)
### Motivation and Context
The AzureAIAgent streaming event path for handling streaming code output
as well as both streaming and non-streaming Bing Grounding tool calls
were missing in the current implementation. This PR adds handling for
those two scenarios.
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
Updates to allow for proper handling of streaming code using the
AzureAIAgent
- code_interpreter: note that the code interpreter tool call is handled
as returning `StreamingTextContent` with the `metadata` dict configured
with `{"code": True}`.
- bing_grounding: support for the Bing Grounding Tool is added. There is
a current issue with streaming events and the Bing Grounding tool - the
`requesturl` as part of the tool call is coming back empty. This issue
has been raised with the appropriate team. Both streaming and
non-streaming do properly return the `StreamingAnnotationContent` or
`AnnotationContent` respectively.
- Samples were added to show interaction with the AzureAIAgent and these
tools, some also showing how to use the `on_intermediate_message`
callback.
- Fixes #11847
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
|
||
|
|
30d2b0802a |
Python: Support Process State Management (#11637)
### Motivation and Context
SK Python processes have been missing support to be able to serialize
and deserialize JSON state for a process and its steps. This PR brings
in the functionality to allow the developer to do so. The
`getting_started_with_processes` step03 has been update to reflect this
latest functionality. It is possible to dump a JSON state to a file, and
reload the state to continue running the process.
State metadata that handles the version for steps is managed via a
decorator:
```python
@kernel_process_step_metadata("CutFoodStep.V1")
class CutFoodStep(KernelProcessStep):
class Functions(Enum):
ChopFood = "ChopFood"
SliceFood = "SliceFood"
```
If no decorator/state is supplied the step will be built with a default
state version of "v1" which aligns with .Net.
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
Support state and versioning management for Python processes.
- Update samples to reflect changes.
- Closes #9584
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
|
||
|
|
9febe95fbb |
Python: updated mixed chat sample (#11624)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Updated the mixed_chat sample to use Azure AI Agents instead of assistant api and used the newest syntax for the chat completion agent ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
8edcdb3ab0 |
Python: Added Brave search capability in SK(python) based on PR #9632 (#11531)
### Motivation and Context Added BRAVE SEARCH CAPABILITIES in Semantic Kernel Python based on PR #11308 . ### Description - Introduce BraveConnector & BraveSearch for Brave Search API integration, - BraveSettings and response models were added, respectively. - Add unit Tests ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Evan Mattson <[email protected]> |
||
|
|
5cbd980f55 |
Python: create MCP server from agent (#11506)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This PR adds a second create_mcp_server function that takes in a list of functions/plugins/agents, and can thus easily be used to create a MCP server based on a Agent Adds a sample with a Azure AI Agent. Partial for #11463 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |