mirror of
https://github.com/microsoft/semantic-kernel.git
synced 2026-10-12 11:19:56 +00:00
ffeb099658d71fbee83e255e85574cdf106a5a09
17
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
6b01468efb |
Python: fix for kernel function type_object when not using Annotations. Add more unit tests. (#7338)
### Motivation and Context The current kernel function decorator class parses function params, but had a miss related to how the underlying param's type is parsed if not using Annotations. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description This PR properly handles the type_object for params that don't using typing.Annotations. - Adds unit tests to exercise the code. - Adds more kernel.py unit tests for near 100% test coverage. - Adds some planner tests to improve the test coverage. - Closes #7300 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
904d1f514f |
Python: Increase unit test coverage (#7211)
### Motivation and Context We've been working on increasing the test coverage. These tests aren't specific to the CQ spike, but I noticed that there was covering lacking for them, and they were "low hanging fruit." <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Increasing the test coverage for some classes that have current test coverage but were missing some line coverage here and there. At this point in time, the test coverage report is showing 95% covered now. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 --------- Co-authored-by: Gil LaHaye <[email protected]> |
||
|
|
4508e8d966 |
Python: properly handle types while building the schema (#6469)
### Motivation and Context While parsing the kernel function's arguments, we recurse to the type's base origin and then set that as the `type_object`. For a type like `list[MyCustomClass]` only setting the `type_object` as `list` has us lose out on the other valuable information of the list's type, which is `MyCustomClass` and what attributes it contains within. This then leads us to build an incomplete schema when creating the kernel function parameters and ultimately when we send the JSON schema to the model during tool calls. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description This PR: - Closes #6466 - Refactors the kernel function decorator code to parse arguments/parameters. We now grab the `type_object` of the parameter/argument in question, which is used to build the JSON schema. - Fixes the schema building for the types we build. We were missing `required` parameters as well. - Attempts to handle as many edge cases as possible, although there are probably more that we aren't handling right now -- for example properly handling all schemas for Unions is tricky. - Adds unit tests to make sure we are building the JSON schema as expected. All unit, integration and sample tests are passing. - Handles a `TypeError` exception during auto function calling where the model sends back an argument like `2`, which doesn't cause JSON parsing to fail, but will fail when trying to update the argument dictionary. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
9e99933a27 |
Python: new pre-commit actions and pre-commit as GHA (#6376)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> - Added static code checking to pre-commit (check-ast and nbqa-check-ast) - Added Bandit security checking to pre-commit - Added pre-commit step to python-lint workflow, if it works, can delete seperate mypy, ruff and black tests ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
0c9517359a |
Python: Fix schema handling. Fix function result return for type list. (#6370)
### Motivation and Context Building the tools json payload from the kernel parameter metadata wasn't properly including an object of type `array`. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Correctly include the object type `array` so that the tool call doesn't return a bad request. Add unit tests. - Closes #6367 - Closes #6360 - Fixes the FunctionResult return for a type string -- if the FunctionResult is of type KernelContent then return the first element of the list, otherwise return the complete list. - Fix the kernel function from method to include the proper type_object for the return parameter so that the schema can be created properly. - Add retry logic for a sometimes flaky function calling stepwise planner integration test. - Add a check during function calling that makes sure the model is returning the proper number of arguments based on how many function arguments are required. <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
81cdde24fc |
Python: upgraded all files to 310 plus format and removed from future (#6353)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Now that we only support 310 and up, we could unify all files, to use the new style typing and annotations. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
517a0f8ed2 |
Python: Add json schema handling. Add experimental tag to OpenAPI and Memory Connectors. (#6335)
### Motivation and Context The Python code base could handle some primitive types for the schema for tool call objects and kernel parameter metadata. However, it couldn't properly handle the more complex JSON schemas for tool call objects. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description This PR introduces: - JSON schema handling for KernelParameterMetadata and tool call objects. - Updates to the tool call utils method to properly recurse on the KernelParameterMetadata's stucture. - Adds unit tests for this code. - Add experimental_class/experimental_functions to various parts of the code base like Memory Connectors and OpenAPI plugin. - Fixes #6310 - Fixes #6280 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 |
||
|
|
a894aebaa9 |
Python: implement filters (#5681)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This pull request includes significant changes across multiple files, mainly related to the addition of hooks and the modification of function invocations in the `semantic_kernel` module. The changes also include the addition of a new sample and a YAML file, and modifications to the `__init__.py` files. Removals: * [`python/semantic_kernel/events`](diffhunk://#diff-ebda9504832b19ab83239a92c9a6d5f8c744deff9fef86071c13956ec92bb010L1-L11): Removed the previously used events. New Exceptions: * [`python/semantic_kernel/exceptions/kernel_exceptions.py`](diffhunk://#diff-450aaa5595a8b22cd6ee212eb79b7d6b0d4e9c1072063ef32018a3e7d3fdf21dR41-R48): Added new exception classes `OperationCancelledException` and `HookInvalidSignatureError`. [[1]](diffhunk://#diff-450aaa5595a8b22cd6ee212eb79b7d6b0d4e9c1072063ef32018a3e7d3fdf21dR41-R48) [[2]](diffhunk://#diff-450aaa5595a8b22cd6ee212eb79b7d6b0d4e9c1072063ef32018a3e7d3fdf21dR57-R58) Fixes: #3038 Fixes: #6276 ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
|
142aef82b1 |
Python: OpenAPI plugin enhance (#6279)
### Motivation and Context Python's OpenAPI manager `run_openapi_operation` was hard-coded to use certain parameters that would ultimately be built up to make an API request. This didn't allow the model to know which parameters were actually defined as required in the OpenAPI spec and would cause errors during function calling. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description This PR does a quite major overhaul on the openapi manager, with the caveat that the code is going to be refactored / cleaned up in a next iteration (we're pressed for time right now). In the `_create_function_from_operation` method, the `rest_operation_params` are now built up from the operation which means we included the required parameters and will have them during function calling. This allows us to properly build up the url, headers, request body and paths to make the API call. - The concept samples were updated and are functioning with this latest code. - function calling was tested with the AzureKeyVault OpenAPI example and the model was able to automatically create a secret in a test key vault. - Old unit tests were removed. Note: in the next iteration new unit tests for all of the new functionality will be added. - In the next iteration, the entire `openapi_manager.py` file will be broken apart to separate files for classes/moels to clean it up. - Closes #6261 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 |
||
|
|
dc54c6f203 |
Python: mypy coverage for functions (#5883)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> extend mypy typing coverage to all classes in 'functions' folder. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
75089cb711 |
Python: Enhance Chat Message Content creation and parsing (#5520)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Introduces a polymorphic approach to creating and rendering Chat Message Contents and it's subclasses, leverages Pydantic Rootmodel for that. Fixes #5496 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Adds ChatMessageContentBase that can create all types of CMC's, based on a type field (str) Redoes some of the rendering and parsing also retooled the hierarchy structure for Contents, including introducing a StreamingContentMixin instead of the StreamingKernelContent (which was never used), now all content classes (TextContent, CMC, etc) determine the fields used, while the Mixin adds one additional field and two additional methods for streaming messages, especially __add__ everything else runs the same way. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
2bc3de72da |
Python: Add extras and minor package updates (#5440)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Poetry groups are not the same as pip extras, so I removed the individual groups per service, replaced with the ones needed for unit tests and for integration tests (all). Then added Extras, for each connector, as well as a `all` extra. Also added a notebooks extra, this can be used instead of the first cell in the notebooks as `pip install -U semantic-kernel[notebooks]` closing #4713 Made all dependencies part of the main, tagged with optional. Closes #5322 ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
275db86fd1 |
Python: add py3.12 to unit tests (#5191)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Since Python 3.12 is now released we should test with it. In the fall 3.8 will be out of support so then we should remove support for that. Since Numpy no longer support py 3.8 as of [1.25](https://numpy.org/doc/stable/release/1.25.0-notes.html) we pin that for py3.8, including other packages that depend on older version of numpy for py3.8, and allow the new versions to work against newer version of python. Closes #5144 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
e884e81776 |
Python: update usearch to 2.9 (#5420)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Updates the usearch package to 2.9+, and fixes the tests for that. Closes #2676 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
fabd1a8478 |
Python: Complex Type support for Method functions (#5260)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Added support for complex types (classes) for KernelFunctionFromMethod. Implements #4635 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> This PR adds a field to the KernelParameterMetadata class for type_object When that field is filled and supplied by a KernelFunctionFromMethod decorator, it is used during invoke to cast the type. It checks if it is a pydantic type and calls `model_validate_json` in that case, otherwise invokes it directly as a init (like with a str), this then also casts strs to ints for example. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
739e7f0e2a |
Python: rebuilt exceptions structure; pythonic version (#5231)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> The existing Exceptions structure was very much inspired by dotnet, this is now replaced with a pythonic implementations. This means all Exceptions derive from KernelException and then specialise for different purposes. Closes #2194 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Added folder for all exception types, all can be imported through `from semantic_kernel.exceptions import ...` no need for a user to know which file the relevant one is in, but keeps things tidy for developers. Removed old KernelException, added back subtypes for the errorcodes. Went through everything to make sure the `raise ... from exc` pattern is used as much as possible as that returns a better stacktrace. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 |
||
|
|
a6f66ec098 |
Python: split kernelfunction into fromprompt and frommethod classes (#5159)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Closes #4633 and brings this part of the python sdk in sync with dotnet, also makes for a much more readable set of classes. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Added: - KernelFunctionFromPrompt class - KernelFunctionFromMethod class - tests for both. Changed: - removed parts of KernelFunction that are specific to one of the two subtypes - removed expose attribute of KernelParameterMetadata as that is no longer needed ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 --------- Co-authored-by: Evan Mattson <[email protected]> Co-authored-by: Evan Mattson <[email protected]> |