### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
This PR graduates the filters.
This includes some updates to the docstrings.
And adds a specific Exception for errors during adding or removing of
filters.
Closes#9838Fixes#9641
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
### Motivation and Context
OpenAI has released in their August 6th model the ability to specific a
json_schema response format. This allows the model to use the specified
Pydantic model or the provided JSON schema representing a "strict"
model/class to make sure that it responds with the exact schema that was
provided.
Note that the strict tool calling will be handled via a separate
ADR/work item once agreed upon by all three SK languages.
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
This PR introduces the ability to specific either a Pydantic model or a
non-Pydantic model for the json_schema response format
OpenAIChatPromptExecutionSettings.
- There are two new code sample concept files showing how to do this
with both a Pydantic and a non-Pydantic model. The other code sample
shows how to configure a json_schema response format while using
function calling.
- If the model is unable to respond based on the provided schema, it
will contain a `refusal` attribute as part of the chat completion
message contents. This will be included in the response as a TextContent
item explaining why it couldn't fulfill the request.
- Unit tests added for this new functionality.
- Handles #7946
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
### Motivation and Context
We have some gaps in how we build the JSON schema for function calling
related to Union and Optional types. There is also some work to do
related to handling `Optional` versus `param | None = None`.
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
This PR improves the JSON schema building for the Union and Optional
types. It adds additional tests and improves some existing tests.
- Closes#7297
- Closes#7298
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Introduces a ImageContent class
Allows that type to be part of ChatMessageContent
Adds a sample for both local and online images
Adds many tests with data uri edge cases, and binary parsing edge cases.
Closes#6536
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
### Motivation and Context
While parsing the kernel function's arguments, we recurse to the type's
base origin and then set that as the `type_object`. For a type like
`list[MyCustomClass]` only setting the `type_object` as `list` has us
lose out on the other valuable information of the list's type, which is
`MyCustomClass` and what attributes it contains within. This then leads
us to build an incomplete schema when creating the kernel function
parameters and ultimately when we send the JSON schema to the model
during tool calls.
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
This PR:
- Closes#6466
- Refactors the kernel function decorator code to parse
arguments/parameters. We now grab the `type_object` of the
parameter/argument in question, which is used to build the JSON schema.
- Fixes the schema building for the types we build. We were missing
`required` parameters as well.
- Attempts to handle as many edge cases as possible, although there are
probably more that we aren't handling right now -- for example properly
handling all schemas for Unions is tricky.
- Adds unit tests to make sure we are building the JSON schema as
expected. All unit, integration and sample tests are passing.
- Handles a `TypeError` exception during auto function calling where the
model sends back an argument like `2`, which doesn't cause JSON parsing
to fail, but will fail when trying to update the argument dictionary.
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
### Motivation and Context
The schema building for complex types worked in some regard, but it
didn't work for all cases. There needs to be better checks on complex
types, if they have __args__ and if so, recurse to the base class type,
and also grab the type that is specified in the list like
list[CustomClass], list[str] or even Union[int, bool].
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
This PR improves the parsing/building of complex types. Added
integration tests to cover more types in the hope that this doesn't lead
to any more issues.
- Closes#6388
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Working through improving test coverage
Small changes introduced:
- ai_service_selector: select_ai_service method, now has extra optional
variable for type_, when not set the behavior is the same as before,
otherwise allows any type of AI service present to be selected,
including embeddings (which was not possible)
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Evan Mattson <[email protected]>
### Motivation and Context
The Python code base could handle some primitive types for the schema
for tool call objects and kernel parameter metadata. However, it
couldn't properly handle the more complex JSON schemas for tool call
objects.
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
This PR introduces:
- JSON schema handling for KernelParameterMetadata and tool call
objects.
- Updates to the tool call utils method to properly recurse on the
KernelParameterMetadata's stucture.
- Adds unit tests for this code.
- Add experimental_class/experimental_functions to various parts of the
code base like Memory Connectors and OpenAPI plugin.
- Fixes#6310
- Fixes#6280
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
Eduard van Valkenburg