mirror of
https://github.com/microsoft/semantic-kernel.git
synced 2026-10-12 02:19:06 +00:00
d3404ccc28943eb64edbd2cb385aed6be56f6a46
440
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Evan Mattson
|
d3404ccc28 |
Python: Support callers passing in max supersteps for processes (#11798)
### Motivation and Context Today in Python processes the `max_supersteps` config is hidden from the caller. The internal execute method defaults the value to 100, and this doesn't allow for developers to leverage custom values depending on their use cases. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Provide a way for callers to pass in `max_supersteps` via the process runtime `start` method. - Update README - Update unit tests accordingly. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Evan Mattson
|
30d2b0802a |
Python: Support Process State Management (#11637)
### Motivation and Context
SK Python processes have been missing support to be able to serialize
and deserialize JSON state for a process and its steps. This PR brings
in the functionality to allow the developer to do so. The
`getting_started_with_processes` step03 has been update to reflect this
latest functionality. It is possible to dump a JSON state to a file, and
reload the state to continue running the process.
State metadata that handles the version for steps is managed via a
decorator:
```python
@kernel_process_step_metadata("CutFoodStep.V1")
class CutFoodStep(KernelProcessStep):
class Functions(Enum):
ChopFood = "ChopFood"
SliceFood = "SliceFood"
```
If no decorator/state is supplied the step will be built with a default
state version of "v1" which aligns with .Net.
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
Support state and versioning management for Python processes.
- Update samples to reflect changes.
- Closes #9584
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
|
||
|
gaudybandGaudy Blanco
|
96962c94e8 |
Python: New tests for azure_cosmos_db_mongodb_collection and local_step (#11518)
### Motivation and Context Improve coverage for azure_cosmos_db_mongodb_collection and local_step classes. ### Description New test files: - test_azure_cosmos_db_mongodb_collection.py - test_local_step.py ### Contribution Checklist - [ ] The code builds clean without any errors or warnings - [ ] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [ ] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 --------- Co-authored-by: Gaudy Blanco <[email protected]> |
||
|
Tao Chen
|
ab53559399 |
Python: Return thread id and run id in agent response metadata (#11744)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> We are currently not returning the thread id and run id from the Azure AI and Assistant agents. These properties will be useful for evaluation purposes. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Return the thread id and run id in the agent response metadata. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Evan Mattson
|
d7e4e3f12a |
Python: Support polling options at run-level for AzureAIAgent, OpenAIAssistantAgent, and OpenAIResponsesAgent (#11705)
### Motivation and Context Based on feedback, there are customer scenarios where it can be helpful to be able to pass in run-level polling options, instead of only at the agent definition/constructor level. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Add run-level run polling options parameter. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Tao Chen
|
2a5fa441e8 |
Python: Add ability to clone a kernel (#11680)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Currently, we don't have a dedicated method to create a clone of a kernel instance. Using `model_copy(deep=True)` will result in an error because some service clients aren't serializable. Being able to clone a kernel will be critical in many scenarios where developers don't want to mutate the original one that is attached to an agent or some other objects. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> 1. Create a clone method in the `Kernel` class that will return a new instance of the kernel with the same configuration, including the list of filters, plugins and the AI selector. It will also include the same list of services references. 2. Add unit tests and integration tests to ensure the new method works as expected. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
NEWTON MALLICKandEvan Mattson
|
8edcdb3ab0 |
Python: Added Brave search capability in SK(python) based on PR #9632 (#11531)
### Motivation and Context Added BRAVE SEARCH CAPABILITIES in Semantic Kernel Python based on PR #11308 . ### Description - Introduce BraveConnector & BraveSearch for Brave Search API integration, - BraveSettings and response models were added, respectively. - Add unit Tests ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Evan Mattson <[email protected]> |
||
|
Tao Chen
|
71caf53a9e |
Python: Add samples to show how to get completion token usage (#11575)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> We have had questions on how to get the token usage from a chat completion response. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Add samples to show how to get the token usage from a chat completion (streaming) response. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Evan Mattson
|
da1e1e5b63 |
Python: Fix Agent StreamingTextContent Yielding and Proper Handling of FunctionCallContent/FunctionResultContent (#11563)
### Motivation and Context This PR addresses an issue where responses during the `invoke_stream` call were not being properly filtered by content type. The bug caused the system to yield responses containing non-text content (i.e., FunctionCallContent or FunctionResultContent) directly, rather than routing them through the on_intermediate_message callback for specialized handling. The fix ensures that only responses exclusively composed of StreamingTextContent are yielded, while other function-related content types are passed to the appropriate callback. Additionally, the `ChatHistoryAgentThread` id property wasn't set - it was configuring a `_thread_id`, which the base class doesn't use. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description - Properly yield expected content type `StreamingTextContent` from an agent `invoke_stream(...)` call. Other FCC or FRC types are yielded via the `on_intermediate_message` callback. - Fix `ChatHistoryAgentThread` id bug that was setting `_thread_id` instead of `_id`. - Update integration tests to make sure we receive the expected content types during function call invocations. - Closes #11561 - Closes #11535 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Eduard van Valkenburg
|
12851923bc |
Python: allow settings to be created directly (#11468)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> I've always hated having to add the .create to a settings object, this removes that, you can just use the regular init, added benefit is that it has the proper fields in the docstrings for each implemented settings object! ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Adds a __new__ method to the base settings class, which takes the prefix and if supplied the env_file and encoding, then calls model_rebuild and then the init. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Eduard van ValkenburgandNico Möller
|
78dd3d5e9c |
Python: Introducing SK as a MCP Server (#11362)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This PR adds the ability to stand up your Kernel as a MCP Server, it returns the Server object, which you can then run as either a Stdio or Sse server. Includes a sample to demonstrate. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Nico Möller <[email protected]> |
||
|
Eduard van Valkenburg
|
97fb2c38f8 |
Python: Allow Kernel Functions from Prompt for image and audio content (#11403)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> I noticed that even though the input and prompt rendering match what you want to use for image and audio generation, we didn't support that. This introduces just that, with two samples. This unlocks the following scenario's: - Running text to speech pipelines with set intro/outro statements - Creating function calls for image generation with limited scope and a lot of set pieces. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> - Adds a `get_image_content` method to the TextToImageClientBase class - Adds the option to select a TextToImage or TextToAudio client in the service selector (only for non-streaming) - Adds branches in the KernelFunctionFromPrompt _invoke_internal for those types. - Adds handling the output as a FunctionResult - Adds samples for both ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Evan Mattson
|
81e89f0c22 |
Python: Improve agent integration tests (#11475)
### Motivation and Context The current integration tests for agents work fine; however, if any errors are encountered due to server-side hiccups, the instant failure cause the whole test run to fail. This PR creates an `agent_test_base` that provides some retry logic, as well as a standardized way for agent test classes to use the underlying methods/protocol. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Improve the agent integration tests. - The OpenAI Responses API is throwing 500s for file search right now. Adding `xfail` for the time being. Filed an [issue](https://github.com/openai/openai-python/issues/2298) on them. - Handle errors when calling for a response for the OpenAI Responses API -- throw a specific agent except or content exception, if received from Azure OpenAI. - Removed the `OpenAIResponsesAgent` `invoke` and `invoke_stream` web_search tests. We have coverage for `get_responses`. The web search is flaky sometimes. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Evan Mattson
|
6f35b627f4 |
Python: Support Auto Function Invocation Filter for AzureAIAgent and OpenAIAssistantAgent (#11460)
### Motivation and Context The server-side agents like AzureAIAgent and OpenAIAssistantAgent do not support the Auto function invocation filter. Although we return intermediate step results, like FunctionResultContent, as part of the `on_intermediate_steps` callback, this didn't allow for developers to configure a filter to return the result as-is, or modify the result as part of the filter. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Add support for the auto function invocation filter for the `AzureAIAgent` and `OpenAIAssistantAgent`. - Closes #11054 - Closes #11456 - Adds samples on how to configure the filter for both agent types, and how to configure the callback to show the various content types. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Evan Mattson
|
6551a860c7 |
Python: Add ChatCompletionAgent integration tests (#11430)
### Motivation and Context Add ChatCompletionAgent integration tests. The last set of agent integration tests that aren't available right now. Additionally, as part of creating the tests, it was discovered that we're yielding FunctionCallContent and FunctionResultContent, when we shouldn't be. To handle those content types, one needs to use the `on_intermediate_message` callback. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Adds ChatCompletionAgent integration tests. - Fix Responses Agent thread get_messages method to properly return input messages from the API. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Evan Mattson
|
0ff97c7fe7 |
Python: add AzureAIAgent, OpenAIAssistant and OpenAIResponsesAgent Integration tests, AzureAIAgent FCC gen bug fix (#11394)
### Motivation and Context AzureAIAgents, OpenAIAssistantAgents, and OpenAIResponsesAgents previously were not covered by integration tests. Adding a set of integration tests to cover high-level usage of an AzureAIAgent and OpenAIAssistantAgents for `get_response`, `invoke`, and `invoke_stream`. As part of this work, a previous fix to typing caused a regression while generating FunctionCallContent for an AzureAIAgent. Fixing to use the correct instance check. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Adds integration tests for AzureAIAgent and OpenAIAssistantAgent. - Fixes #11393 - Bump version to 1.27.2 for bug fix release. - Remove unnecessary yield for function result content during streaming invocation - these types will be added if the `on_intermediate_message` callback is provided. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Evan Mattson
|
ff498b4ca9 |
Python: Updates (#11371)
### Motivation and Context Some code cleanup. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Some code cleanup. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
gaudybandGaudy Blanco
|
fa96214dbc |
Python: Unit tests for OpenAI realtime and Ollama chat completion (#11343)
### Motivation and Context Improve overall unit test coverage. ### Description adding unit tests for ollama chat completion adding unit tests for openAI realtime ### Contribution Checklist - [ ] The code builds clean without any errors or warnings - [ ] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [ ] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 --------- Co-authored-by: Gaudy Blanco <[email protected]> |
||
|
Tao Chen
|
94a78d15e2 |
Python: Add on_intermediate_message callback to the Agent abstraction (#11321)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Currently, our agents' `invoke` and `invoke_stream` have inconsistent behavior where the responses may contain different contents. For example, the chat completion agent (`invoke`) may return all content types generated by the agent including function call/result contents that are meant to be internal only, while the assistant agent doesn't return those contents. This creates confusion both for users and developers of SK of what an agent should return when working with agents. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> This PR standardizes the behavior of agents by adding an `on_intermediate_message ` parameter to the `invoke` and `invoke_stream` methods in the `Agent` abstraction. This parameter accepts a callback function that will be called whenever a new message is generated by the agent, except the last one that gets returned to the caller. This allows the caller to inspect what the agent did after a response is returned. In addition, this PR also converges on the behavior of the `invoke` and `invoke_stream` methods (respectively) to return the same content types from all agents, making it easier for users to work with agents ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
Eduard van ValkenburgandNico Möller
|
f00dc80267 |
Python: Introducing support for using a MCP server as a plugin (#11334)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Adds MCP server configs and a function that turns the server into a plugin, with each tools of the server represented as a function. Adds a sample showing how to use that with a Github MCP server. Closes: #10785 and #11190 With special thanks to @nmoeller ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Nico Möller <[email protected]> |
||
|
Eduard van Valkenburg
|
b1f64eea5f |
Python: Introducing the SQL Connector (#10981)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This PR adds the Azure SQL Store and Collection Closes #9882 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Evan Mattson
|
329a35edea |
Python: Introduce the OpenAI Responses Agent (#11240)
### Motivation and Context OpenAI recently released their Responses API, which is their newest core API and an agentic API primitive, combining the simplicity of Chat Completions with the ability to do more agentic tasks. Azure OpenAI also recently released the Responses API, with close-to feature parity with OpenAI's SDK. It gives us enough to introduce an `AzureResponsesAgent` and an `OpenAIResponsesAgent` to Semantic Kernel. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description This PR introduces: - The `AzureResponsesAgent` and `OpenAIResponsesAgent` - Allows use of the web search tool with `OpenAI` (`Azure OpenAI` currently doesn't support this). - Allows use of the file search tool with both `OpenAI` and `Azure OpenAI`. - Provides getting started samples for both agent types. **TODO:** - Add unit test coverage for the `ResponsesAgentThreadActions` class. - Add support for the computer user agent tool. - Improve typing in the `ResponsesAgentThreadActions` class. - Closes #11026 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Eduard van ValkenburgandEvan Mattson
|
a52d47e22f |
Python: restructed data folder and multiple improvements to vector stores (#11302)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This PR: - restructures the data folder, flattening the hierarchy - it adds overloads to the Get, Upsert, Delete methods, to allow batch as well, marks the batch versions as deprecated, closes #11301 - restructured the inheritance of VectorStoreRecordCollection, which now has a base that does record handling, which is shared with the VectorSearchBase for the serialization, and the search method mixins now subclass the VectorSearchBase so that there is one less parent direct parent, a full Collection will now inherit from VectorStoreRecordCollection and zero or more VectorSearch mixins, this is done to clean things up and in preparation of adding HybridSearch. - Also adds a convenience method to each search mixin to create a VectorStoreTextSearch directly, allowing folks to import one less concept, closes #11111 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Most of the code has been moved out of many files and folders, into a few files which make purpose and collaboration between the classes easier to understand. This is a breaking change: If you have implemented your own VectorStoreRecordCollection, make sure to: - Add `VectorStoreRecordCollection[TKey, TModel]` as superclass to the implementation - Remove `VectorSearchBase` as superclass from implementation - Set the SearchMixins to `[TKey, TModel]` instead of just `[TModel]` ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 --------- Co-authored-by: Evan Mattson <[email protected]> |
||
|
Tao Chen
|
84f4841465 |
Python: Fix agent samples (#11278)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This [sample](https://github.com/microsoft/semantic-kernel/blob/main/python/samples/concepts/agents/chat_completion_agent/chat_completion_agent_function_termination.py) was throwing an error due to duplicating tool messages in the chat history. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Fix a few agent samples. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Evan Mattson
|
5fc8381177 |
Python: Pass kernel args through to kernel invoke function call for AzureAIAgent and AssistantAgent (#11250)
### Motivation and Context In our AzureAIAgent and AssistantAgent thread actions, we're not passing the supplied args from the caller through to kernel function call invoke methods. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description This PR fixes that, and makes sure we pass kernel args all the way through the process, so any function calls can leverage the args. - Adds a concept sample for AzureAIAgent showing how to prompt with different prompt templates. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Evan Mattson
|
d7172a5ae4 |
Python: Common agent invocation API updates (#11224)
### Motivation and Context After this week's release of the common agent invocation API, there are some things that we can do as further improvements. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description This PR includes updates for: - Right now we force any agent invocation to provide some type of input, whether a str, a CMC, or a list of str | CMC. At times, there could be a reason, based on using an existing thread or just providing instructions to the agent, that one doesn't need to provide a message to invoke the agent. Updating to make messages optional. Updating the ABC contracts as well. - For `invoke_stream` calls on agents, there's no need to call `thread.on_new_message` that contains a streaming chunks -- once we move to support memory, this is where the "hook" will be. Removing this call. - The `get_messages(...)` methods on the `AutoGenConversableAgentThread` and the `ChatHistoryAgentThread` returned concrete `ChatHistory` objects, whereas the `AssistantAgentThread` and `AzureAIAgentThread` returned `AsyncIterable[ChatMessageContent]`. To align to a common API, the `AutoGenConversableAgentThread` and `ChatHistoryAgentThread`'s `get_messages(...)` methods were moved to return `AsyncIterable[ChatMessageContent]`. - Removing a public facing `output_messages` for streaming invoke, and replacing it with a callback to get a chat history back of "full" messages. Two samples are added in `samples/concepts/agents`: - `azure_ai_agent/azure_ai_agent_streaming_chat_history_callback.py` - `openai_assistant_streaming_chat_history_callback.py` - Update the README for OpenAI Assistants to showcase new thread abstraction. - Include unit tests for chat history (`on_complete`) callback. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
Evan Mattson
|
858644a8d1 |
Python: Allow for Agents as Kernel Functions (#11184)
### Motivation and Context We're moving towards a pattern where Agents can be called as tools. We currently don't have this in Semantic Kernel Python. This work builds off of the previous PR (#10828) but extends it to work with any SK Agent now that we have a bit more of a common invocation pattern (all agents take in `messages` kwarg). This also works now because each agent creates its own thread internally, if one isn't provided - so we don't need to worry about handling chat history for an OpenAI Assistant or an Azure AI Agent. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Allow agents to be configured as Kernel Functions, which can then be used during Function Calling. We will advertise the Agent's name and either its description or instructions (if description is blank). - Provide a sample where we show a triage agent (ChatCompletionAgent) calling either a billing agent (ChatCompletionAgent) or a refund agent (ChatCompletionAgent ) as part of a tool call to satisfy the user's query. - Provide a sample where we show a triage agent (ChatCompletionAgent) calling either a billing agent (AzureAIAgent) or a refund agent (ChatCompletionAgent) to satisfy the user's query. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Tao Chen
|
ad5026fb12 |
Python: Update Agent invoke abstraction parameters (#11192)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Now that our agents have a more aligned interface, we can move the common interface parameters to the abstract class. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Move the messages and thread parameters to the abstract class `Agent` for the `get_response`, `invoke`, and `invoke_stream` methods. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
17e63a782b |
Python/.NET: Add a common agent invoke api. (#11116)
### Motivation and Context Being able to different invoke agents using a common API is useful to build higher level code on top of. ### Description Adding a common Invoke method with a common Thread object. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [ ] The code builds clean without any errors or warnings - [ ] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [ ] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 --------- Co-authored-by: Ben Thomas <[email protected]> Co-authored-by: Evan Mattson <[email protected]> Co-authored-by: Tao Chen <[email protected]> Co-authored-by: Ben Thomas <[email protected]> Co-authored-by: Mark Wallace <[email protected]> Co-authored-by: Evan Mattson <[email protected]> |
||
|
Eduard van Valkenburg
|
00afbdfc78 |
Python: removed old search plugins (#11035)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Removed some of the old search samples. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Evan Mattson
|
ec89fe4a5e |
Python: Exclude certain fields from KernelFunction model dump (#11081)
### Motivation and Context When trying to do a model dump for `KernelFunction` models, they are failing with a message related to: ``` pydantic_core._pydantic_core.PydanticSerializationError: Unable to serialize unknown type: <class 'type'> ``` There are some class-level attributes that cannot be properly serialized. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Exclude dumping certain model attributes from the model dump, which allows the main KernelFunction metadata and its underlying parameters to be serialized. - Closes #11078 - Adds unit tests to ensure we have coverage and can dump Pydantic models. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Evan Mattson
|
5cf856830d |
Python: Fix agent_id kwarg in AzureAIAgent retrieval sample (#11057)
### Motivation and Context The current AzureAIAgent retrieval sample uses the old `assistant_id` kwarg to retrieve the agent. We need to fix this to use the new `agent_id` kwarg in AzureAIAgent retrieval sample <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Fix the kwarg to use `agent_id`. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Rodrigo Martins RacanicciandEvan Mattson
|
74714cb774 |
Python: Unit tests for ollama services utils and dapr_actor_registration (#10849)
### Motivation and Context Improve overall unit test coverage. ### Description - adding unit tests for ollama services utils - adding unit tests for dapr_actor_registration ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [ ] The code builds clean without any errors or warnings - [ ] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [ ] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 --------- Co-authored-by: Evan Mattson <[email protected]> |
||
|
Eduard van Valkenburg
|
1874eebaba |
Python: small fix for Pinecone integrated embeddings (#10953)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Pinecone offers integrated embeddings, expended the way you can set up for that Made some fixes in the DeSer process of the vector stores to deal with non-returned vectors. Now (finally) uses local_embeddings field to allow the field to be present but not used. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Rashmi PawarandLakshmi Ramesh
|
28e9d11a98 |
Python: Nvidia Embedding Connector (#10410)
### Motivation and Context This PR adds an Nvidia Embedding Connector. This connector enables integration with NVIDIA NIM API for text embeddings. It allows you to use NVIDIA's embedding models within the Semantic Kernel framework. ### Description ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 --------- Co-authored-by: Lakshmi Ramesh <[email protected]> |
||
|
Evan Mattson
|
b3b20b7028 |
Python: Model dump json step info for Dapr step actor (#10946)
### Motivation and Context Dapr is throwing an error while trying to dump the `step_info` model to the state store. To avoid this, perform a `model_dump_json()` first and then store. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Fixes Dapr step info error. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Eduard van Valkenburg
|
acf5156848 |
Python: Introducing the Pinecone Connector (#10886)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Introducing the Pinecone Vector Store and Collection This also upgrades the dependencies to v6 Adding the ability to run with asyncio or GRPC TODO: - [x] Filtering - [x] Tests ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Adds PineconeStore and PineconeCollection Closes: #9928 ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Rodrigo Martins RacanicciandEvan Mattson
|
63f140e704 |
Python: Improve test coverage (#10681)
### Motivation and Context Adding more tests to improve overall coverage. ### Description - adding tests for bedrock agent channel - adding tests for bedrock model provider utils - adding tests for mistral ai chat completion - tests for bing search - tests for google search ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [ ] The code builds clean without any errors or warnings - [ ] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [ ] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 --------- Co-authored-by: Evan Mattson <[email protected]> |
||
|
Evan Mattson
|
87bc3d6bc1 |
Python: Support AzureAIAgent agent_id instead of assistant_id (#10908)
### Motivation and Context The Azure Python SDK made an underlying change to move away from `assistant_id` keyword arguments to now support `agent_id` keyword arguments. Additionally, they now support a `MessageTextUrlCitationAnnotation` and `MessageDeltaTextUrlCitationAnnotation` for annotations. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Update the `AzureAIAgent` to pass in `agent_id` in place of `assistant_id`, and bring in support for the new annotation types. - Updates samples to simpler patterns where needed. - Updates unit tests. - Closes #10852 Validated that our [AzureAIAgent + Azure AI Search sample](https://github.com/microsoft/semantic-kernel/blob/main/python/samples/concepts/agents/azure_ai_agent/azure_ai_agent_azure_ai_search.py) works properly: --- ```markdown # User: 'Which hotels are available with full-sized kitchens in Nashville, TN?' # Agent: In Nashville, TN, here are some hotel options that feature full-sized kitchens: 1. **Extended Stay Suites** - All suites include full-sized kitchens stocked with cookware, separate living and sleeping areas, and sofa beds. Some larger rooms have fireplaces and patios or balconies【3:0†source】. 2. **New Premium Rooms** - These newly furnished rooms are fully equipped with premium amenities, including a full kitchen, washer & dryer, and access to a fitness center. Units also include fireplaces and small outdoor balconies【3:1†source】. 3. **Conveniently Located Extended Stay** - This option offers a full kitchen along with a washer & dryer, and amenities such as free WiFi and a fitness center, located near downtown Nashville【3:3†source】. 4. **Inn with Functional Kitchens** - This inn provides living room suites and functional kitchens, situated conveniently near a lake and welcomes pets【3:4†source】. These options provide a range of amenities for travelers looking for the convenience of a full-sized kitchen during their stay. # User: 'Fun hotels with free WiFi.' # Agent: Here are some fun hotels that offer free WiFi: 1. **Hotel with Rooftop Plaza** - Offers complimentary continental breakfast in the lobby and free Wi-Fi throughout the hotel. Its penthouse suites provide stunning views【7:0†source】. 2. **Lakeside Hotel** - Features newly redesigned rooms, an airport shuttle, and lakeside amenities including a resort-style pool. The hotel provides stylish new guestrooms with Internet TVs and free WiFi【7:3†source】. 3. **Downtown Hotel** - Located near the vibrant downtown area and the business district. This hotel offers free WiFi, local transportation, and a special treat of milk and cookies for guests【7:4†source】. These hotels combine fun experiences with the convenience of free internet access for guests. ``` <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Eduard van Valkenburg
|
45f4234827 |
Python: Introducing the Faiss Connector (#10810)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This PR adds a FaissVectorStore and FaissCollection Faiss is a really fast similarity search engine, which runs in memory. You can also use GPU's with it, but you will have to build the package yourself. ### Description Closes #4130 TODO: - [x] other index kinds - [x] deal with distance functions - [x] validate GPU setup - [x] maybe subclass InMemoryCollection - [x] tests <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Eduard van Valkenburg
|
b4456aac2b |
Python: restructured the client classes, agent verbs (#10834)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Added some extra ways to provide functions to the models, and allow more things to be set in the client constructor, rather then the __call__ that get's called by the async context manager. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> - This also splits the backend classes into a new file called _open_ai_realtime.py keeping only the public classes available in the open_ai_realtime.py file. - Adds additional ways of setting up - Adds usage of model_extra to set things up in the constructor ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Tao Chen
|
191f595f7d |
Python: Remove custom separator in Bedrock tool call fully qualified name (#10826)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Closes #10619 Bedrock updated its rules on tool names. Previously, Bedrock disallowed "-" in tool names, which is the default plugin name and function name separator in SK. The solution was to use "_" as the separator. However, this introduced a bug where the translation between the SK fully qualified function names and the Bedrock tool names would fail if the function name had an underscore. Bedrock now allows "-" in tool names, so we can remove the custom logic and just use the default. Gemini also has rules on how tools are named. It doesn't recommend using "-". The solution was to use "_" just like Bedrock. To address the bug mentioned above, the solution is to use double underscores. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> 1. Remove Bedrock custom fully qualified function names. 2. Use double underscores for Gemini fully qualified function names. TODO: Add a guide for SK function/plugin naming conventions. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
Tao Chen
|
1c9b1667b1 |
Python: Fix issues of Gemini parallel function call (#10813)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Closing #10597 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> 1. Collapse consecutive tool call messages into a single tool message with multiple function result items before sending the chat history to the API. 2. Add unit test. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Tao Chen
|
e1184aea74 |
Python: Add embedding and text completion samples (#10797)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Closing https://github.com/microsoft/semantic-kernel/issues/9963 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> 1. Add text embedding and text completion samples. 2. Move the embedding generator base to a `semantic_kernel.connectors.ai`. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Evan Mattson
|
e7beb6a143 |
Python: Update Cross Language Test to get raw request and now raw response (#10799)
### Motivation and Context In our cross language tests we were getting the raw request using a custom logging transport class. This didn't show how to get the raw HTTP response back after making the call to the OpenAI SDK. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description This PR updates the cross language test to get the raw request and now the raw response. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Evan Mattson
|
fb70825df7 |
Python: Improve handling the Dapr process context (#10725)
### Motivation and Context There is a bug in the current Dapr runtime code that prohibits trying to get the Dapr process context out from a process once it's finished. This is related to incorrect handling of serializing/deserializing the Dapr process context. Additionally, in a state that handles `KernelProcessStepState`, due to Dapr serialization constraints, the step step injected to a step is not the actual object. We use Pydantic's `model_validate` to get the actual step object, but and we need to make sure we re-assign that object back to the underlying step state so the objects don't diverge (`model_validate` creates a new object). <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Improve the handling for the Dapr process context so we're properly serializing data through the Dapr process. - Fixes the issue brought up in the discussion: #10234 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
Eduard van ValkenburgandEvan Mattson
|
de54193a0f |
Python: Introducing Realtime Clients for OpenAI and Azure OpenAI (#10127)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Implements the OpenAI Realtime API with Semantic Kernel ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Adds a RealtimeClientBase Adds RealtimeEvents Adds OpenAI and Azure Realtime clients Closes: #10108 ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Evan Mattson <[email protected]> |
||
|
Tao Chen
|
8fd6da2bd8 |
Python: Improve Bedrock service setup docs (#10772)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Closing: https://github.com/microsoft/semantic-kernel/issues/9910 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> 1. Add detailed instructions on how to setup one's environment to use Bedrock services, including models and agents. 2. Re-enable bedrock integration tests. 3. Some other cleanups. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
Tao Chen
|
75453a1219 |
Python: Move to a thinner wrap for Bedrock Agent in Python (#10740)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Though the current Bedrock agent implementation uses a "thin wrap" approach, it still has some abstractions that we don't want to maintain. Just like what we have done for the other agents, we remove those abstractions. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> 1. Update the Bedrock agent abstraction to be thinner. 2. Update samples and tests. Note: The .Net implementation doesn't need to be updated as it was already using a thinner wrap. Note: This breaks the current usage pattern for Bedrock agents in Python. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
Evan Mattson
|
0153073aa3 |
Python: Bump Python version to 1.22.1 for a release (#10751)
### Motivation and Context Bump Python version to 1.22.1 for a release <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Bump Python version to 1.22.1 for a release <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |