microsoft/semantic-kernel
1
0
Fork 0
mirror of https://github.com/microsoft/semantic-kernel.git synced 2026-10-12 02:19:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,033 Commits 63 Branches 1,077 Tags
e4025c3446479bff5de0906161620033d6e15d7a
Commit Graph
87 Commits
Author SHA1 Message Date
SergeyMenshykhCopilotEvan Mattson
 		3df33f7862 Python: [Breaking] Update runtime handling (#14135) 
This PR updates Python runtime handling and related tests.

---------

Co-authored-by: Copilot <[email protected]>
Co-authored-by: Evan Mattson <[email protected]>
 2026-07-07 03:22:20 +00:00
Eduard van Valkenburg 1c11258a32 Python: added af_tool bridge for kernel_functions (#13227) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Adds a new method to KernelFunction that creates a Agent Framework Tool
from that function.

Does require the user to install agent-framework-core themselves.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2025-10-30 15:24:16 +00:00
Dmytro Struk 18d9a8e8b2 Python: .Net: Updated encoding logic in prompt templates (#12983) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Resolves: https://github.com/microsoft/semantic-kernel/issues/11821

Today, the encoding of template arguments is performed only if argument
type is `string`. In case of custom type, anonymous type or collection -
the encoding is not performed.

This PR contains changes to throw an exception in case if encoding is
enabled but complex type is used. In case of complex type, the encoding
should be performed manually according to business logic and automatic
encoding should be explicitly disabled.

This enforces stricter, but more secure template rendering rules.

**Note**: this is a breaking change for customers who use Handlebars or
Liquid template with complex type arguments. Code changes are required
when initializing template arguments:
```diff
var arguments = new KernelArguments()
{
    { "customer", new
        {
-             firstName = userInput.FirstName,
-             lastName = userInput.LastName,
+             firstName = HttpUtility.HtmlEncode(userInput.FirstName),
+             lastName = HttpUtility.HtmlEncode(userInput.LastName),
        }
    }
};

var templateFactory = new LiquidPromptTemplateFactory();
var promptTemplateConfig = new PromptTemplateConfig()
{
    TemplateFormat = "liquid"
+   InputVariables = new()
+   {
+       // We set AllowDangerouslySetContent to 'true' because each property of this argument is encoded manually. 
+       new() { Name = "customer", AllowDangerouslySetContent = true },
+   }
};

var promptTemplate = templateFactory.Create(promptTemplateConfig);
var renderedPrompt = await promptTemplate.RenderAsync(kernel, arguments); 
```

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
 2025-08-26 22:09:00 -07:00
Tao Chen 6416c58b90 Python: Add arguments and results attributes to execute tool span (#12940) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
We are adding two new attributes to the execute tool span according to
this proposal to Otel Semantic Conventions:
https://github.com/open-telemetry/semantic-conventions/pull/2528

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
The following two attributes are added to the `execute_tool` span:
1. `gen_ai.tool.call.arguments`
2. `gen_ai.tool.call.result`

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2025-08-13 23:35:42 +00:00
Mehdi MoshtaghiandEvan Mattson 73ad6e8923 Python: Fix: Implement __deepcopy__ on KernelFunction to handle non-serializable OTEL metrics (#12803) 
### Description

This PR fixes TypeError: cannot pickle '_thread.RLock' object which
occurs when kernel.clone() is called while OpenTelemetry metrics are
enabled.

-   Fixes: #12802

### Motivation and Context

When using features like `HandoffOrchestration`, the agent's `Kernel` is
cloned. This process uses `copy.deepcopy`, which fails on the
`KernelFunction`'s OpenTelemetry `Histogram` fields
(`invocation_duration_histogram` and `streaming_duration_histogram`).
These histogram objects contain thread locks, making them
non-serializable.

Previous solutions, like the one in **PR #9292**, used
`Field(exclude=True, default_factory=...)`. While that was a necessary
step, it was insufficient because `deepcopy` does not respect the
`exclude` flag in the same way that Pydantic's serialization does. This
bug blocks the use of certain agentic features in any application that
has metrics-based observability enabled.

### Description of the change

This solution introduces a custom __deepcopy__ method to the
KernelFunction Pydantic model. This method correctly handles the
deep-copying process for KernelFunction instances:

1.  It intercepts the deepcopy call for KernelFunction objects.
2. It uses self.model_copy(deep=False) to create a new instance of the
Pydantic model. Using deep=False allows the default_factory for the
histogram fields to create new, clean instances on the copied object,
while other fields are shallow-copied.
3. It then manually deep-copies all other attributes from the original
object to the new one, ensuring that any other mutable objects (like
metadata`) are correctly duplicated. The histogram fields are explicitly
skipped in this step, as they have already been recreated by
`model_copy.
4. Standard deepcopy memoization is used to prevent infinite recursion
in the case of cyclic object graphs.

This approach ensures that the non-serializable fields are
re-initialized instead of copied, resolving the TypeError while
maintaining the integrity of the cloned object.

### How has this been tested?

The fix was validated using the reproduction steps outlined in the
associated issue. Running an agent orchestration that triggers
kernel.clone() with OpenTelemetry metrics enabled now completes
successfully without any errors.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
 2025-08-07 22:48:43 +00:00
Evan Mattson 1198b573d1 Python: Add ability to specify encoding when adding a plugin. Add tests. (#12797) 
### Motivation and Context

Right now when adding a plugin, we don't allow one to specify the type
of encoding used.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Allow configuration for the type of encoding used.
- Adds unit tests
- Closes #12440

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
 2025-08-04 22:00:30 +00:00
Eduard van Valkenburg 003cc4c010 Python: Feature python vector stores preb (#12271) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Overhaul of the VectorStores to get it matching with dotnet and the
latest development there.

Closes: 
- #11938 
- #11598 
- #11597
- #11517 
- #10561 
- #10391 
- #9911 
- #9892 
- #10867 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

Many changes, some highlights:
- Moved from different vector fields types, to a single with a enum
first VectorStoreField()
- Renamed several items
- Made embeddinggenerrator a part of the vector field spec and do
automatic vectorization from there or from the same param on a
collection
- Adds hybrid search using the same setup
- Removed the intermediate TextSeachVectorSearch object need, by
allowing `create_search_function` directly on a collection

Will write out full changes in a migration guide in the docs.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
 2025-06-19 18:34:59 +00:00
kanchi cef5ae7402 Python: Support | and |= operators for KernelArgument (#12499) 
### Motivation and Context

`KernelArguments` extends the built-in `dict` by adding an
`execution_settings` attribute.
However, when using the `|` and `|=` operators, only the `dict` part
(excluding `execution_settings`) is merged, and the result becomes a
plain `dict`.
This causes the `execution_settings` attribute to be lost and not
updated.

### Description

- Implemented support for the `|` and `|=` operators to make it easier
for users to merge `KernelArgument` objects.
- The right-hand or left-hand side of the operator can be either a
`KernelArgument` or a `dict` (or any subclass of dict).
- When merging, both the `execution_settings` attribute and the rest of
the `dict` data are merged individually, and the result is returned as a
`KernelArgument`.
- As long as either side of the operator is a `KernelArgument`, the
result will also be a `KernelArgument`.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2025-06-19 07:24:16 +00:00
Tao Chen c0605cc4e7 Python: Update kernel function span (#12285) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Closing #12262

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
This PR updates tracing support for function execution across both .NET
and Python according to the latest OTel GenAI semantic conventions:
https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/#execute-tool-span

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
 2025-05-27 23:41:44 +00:00
kanchi 966fa8ca04 Python: Replace mutable default arguments and prevent unintended side effects (#11849) 
### Motivation and Context

Some functions and methods were using mutable objects (typically `dict`)
as default argument values.
In Python, default arguments are evaluated only once at the time of
function definition.
This can lead to unexpected behavior when the default value is modified
within the function, as subsequent calls to the function may reuse the
modified object.

As a result, calling such functions or methods multiple times can lead
to unintended behavior.

Reference:  
[8.7. Function definitions (Official Python
Documentation)](https://docs.python.org/3.12/reference/compound_stmts.html#function-definitions)
> Default parameter values are evaluated from left to right when the
function definition is executed. This means that the expression is
evaluated once, when the function is defined, and that the same
“pre-computed” value is used for each call. This is especially important
to understand when a default parameter value is a mutable object, such
as a list or a dictionary: if the function modifies the object (e.g. by
appending an item to a list), the default parameter value is in effect
modified. This is generally not what was intended. A way around this is
to use None as the default, and explicitly test for it in the body of
the function, e.g.:


### Description

- Replaced mutable types (e.g., `dict`, `list`) used as default argument
values with `None`, and properly initialized them within the function
body.
- Reviewed and updated the parts of the code that performed destructive
modifications on the arguments, to avoid unintentionally modifying
variables from the caller’s scope.
- ~Added the `flake8-bugbear` rule in Ruff to detect this kind of
issue.~
- ~For now, errors in `tests` and `samples` directories are ignored due
to the large number of violations.~

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
 2025-05-06 19:03:05 +00:00
Evan Mattson 30d2b0802a Python: Support Process State Management (#11637) 
### Motivation and Context

SK Python processes have been missing support to be able to serialize
and deserialize JSON state for a process and its steps. This PR brings
in the functionality to allow the developer to do so. The
`getting_started_with_processes` step03 has been update to reflect this
latest functionality. It is possible to dump a JSON state to a file, and
reload the state to continue running the process.

State metadata that handles the version for steps is managed via a
decorator:

```python
@kernel_process_step_metadata("CutFoodStep.V1")
class CutFoodStep(KernelProcessStep):
    class Functions(Enum):
        ChopFood = "ChopFood"
        SliceFood = "SliceFood"
```

If no decorator/state is supplied the step will be built with a default
state version of "v1" which aligns with .Net.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Support state and versioning management for Python processes.
- Update samples to reflect changes.
- Closes #9584

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
 2025-04-28 22:51:11 +00:00
Eduard van Valkenburg 5f37e464df Python: add support for prompts and sample (#11508) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Allow a list of prompts to be added and exposed with the MCP server
setup.

This uses the built-in rendering types (all of them) to be used and
exposed as prompts.

Closes: #11143

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2025-04-15 08:25:35 +00:00
Eduard van ValkenburgandNico Möller 78dd3d5e9c Python: Introducing SK as a MCP Server (#11362) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

This PR adds the ability to stand up your Kernel as a MCP Server, it
returns the Server object, which you can then run as either a Stdio or
Sse server.
Includes a sample to demonstrate.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Nico Möller <[email protected]>
 2025-04-11 05:22:07 +00:00
Eduard van Valkenburg 97fb2c38f8 Python: Allow Kernel Functions from Prompt for image and audio content (#11403) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
I noticed that even though the input and prompt rendering match what you
want to use for image and audio generation, we didn't support that.

This introduces just that, with two samples. This unlocks the following
scenario's:

- Running text to speech pipelines with set intro/outro statements
- Creating function calls for image generation with limited scope and a
lot of set pieces.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

- Adds a `get_image_content` method to the TextToImageClientBase class
- Adds the option to select a TextToImage or TextToAudio client in the
service selector (only for non-streaming)
- Adds branches in the KernelFunctionFromPrompt _invoke_internal for
those types.
- Adds handling the output as a FunctionResult
- Adds samples for both


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2025-04-10 12:51:53 +00:00
Eduard van ValkenburgandNico Möller f00dc80267 Python: Introducing support for using a MCP server as a plugin (#11334) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Adds MCP server configs and a function that turns the server into a
plugin, with each tools of the server represented as a function.
Adds a sample showing how to use that with a Github MCP server.

Closes: #10785 and #11190

With special thanks to @nmoeller 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Nico Möller <[email protected]>
 2025-04-03 18:08:45 +00:00
Eduard van ValkenburgandEvan Mattson a52d47e22f Python: restructed data folder and multiple improvements to vector stores (#11302) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
This PR:
- restructures the data folder, flattening the hierarchy
- it adds overloads to the Get, Upsert, Delete methods, to allow batch
as well, marks the batch versions as deprecated, closes #11301
- restructured the inheritance of VectorStoreRecordCollection, which now
has a base that does record handling, which is shared with the
VectorSearchBase for the serialization, and the search method mixins now
subclass the VectorSearchBase so that there is one less parent direct
parent, a full Collection will now inherit from
VectorStoreRecordCollection and zero or more VectorSearch mixins, this
is done to clean things up and in preparation of adding HybridSearch.
- Also adds a convenience method to each search mixin to create a
VectorStoreTextSearch directly, allowing folks to import one less
concept, closes #11111

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Most of the code has been moved out of many files and folders, into a
few files which make purpose and collaboration between the classes
easier to understand.

This is a breaking change:
If you have implemented your own VectorStoreRecordCollection, make sure
to:

- Add `VectorStoreRecordCollection[TKey, TModel]` as superclass to the
implementation
- Remove `VectorSearchBase` as superclass from implementation
- Set the SearchMixins to `[TKey, TModel]` instead of just `[TModel]`

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
 2025-04-01 23:51:29 +00:00
Evan Mattson 858644a8d1 Python: Allow for Agents as Kernel Functions (#11184) 
### Motivation and Context

We're moving towards a pattern where Agents can be called as tools. We
currently don't have this in Semantic Kernel Python.

This work builds off of the previous PR (#10828) but extends it to work
with any SK Agent now that we have a bit more of a common invocation
pattern (all agents take in `messages` kwarg). This also works now
because each agent creates its own thread internally, if one isn't
provided - so we don't need to worry about handling chat history for an
OpenAI Assistant or an Azure AI Agent.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Allow agents to be configured as Kernel Functions, which can then be
used during Function Calling. We will advertise the Agent's name and
either its description or instructions (if description is blank).
- Provide a sample where we show a triage agent (ChatCompletionAgent)
calling either a billing agent (ChatCompletionAgent) or a refund agent
(ChatCompletionAgent ) as part of a tool call to satisfy the user's
query.
- Provide a sample where we show a triage agent (ChatCompletionAgent)
calling either a billing agent (AzureAIAgent) or a refund agent
(ChatCompletionAgent) to satisfy the user's query.


<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
 2025-03-26 02:29:12 +00:00
Evan Mattson ec89fe4a5e Python: Exclude certain fields from KernelFunction model dump (#11081) 
### Motivation and Context

When trying to do a model dump for `KernelFunction` models, they are
failing with a message related to:

```
pydantic_core._pydantic_core.PydanticSerializationError: Unable to serialize unknown type: <class 'type'>
```

There are some class-level attributes that cannot be properly
serialized.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Exclude dumping certain model attributes from the model dump, which
allows the main KernelFunction metadata and its underlying parameters to
be serialized.
- Closes #11078 
- Adds unit tests to ensure we have coverage and can dump Pydantic
models.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
 2025-03-20 02:16:30 +00:00
Eduard van ValkenburgandEvan Mattson 374d8f91ce Python: filter improvements (#10588) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Fixes some inconsistency in the filter setup between dotnet and python
Adds a sample for how to use a filter to retry a prompt with a different
model.
Adds a sample for semantic caching

Closes: #10572 #5924 #10595

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
 2025-02-19 00:40:51 +00:00
Eduard van Valkenburg cbfd7e9330 Python: improved content inits, added ndarray support for binary content and small fixes to defaults (#10469) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
This PR adds support for ndarray's as the content carrier for all binary
content types (binary, image, audio) as that is more optimized for
larger content.

It also does some fixes to the initialization of those content types and
the underlying data uri type.

Also some fixes for unspecified default param in pydantic Field,
language servers do not recognize `Field("default value")` as having a
default, so changed those occurances to `Field(default="default value")`

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2025-02-12 16:59:08 +00:00
Evan Mattson 1a84cb64b2 Python: prompt template config and kernel args support for agents (#10298) 
### Motivation and Context

To bring the SK Python framework closer to .Net parity, we need to
support the ability to specify a prompt template config and kernel
arguments for ChatCompletion Agents as well as for Assistant Agents.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR adds support for:
- being able to specify a prompt template config for SK agents.
- allows kernel arguments to be passed to the agent as well as during
invoke - invoke args, similar to other kwargs, take precedence over
agent level config.
- adds test coverage
- creates agent templating samples for both ChatCompletionAgents and
OpenAI/AzureAssistantAgents
- closes #8022 
- closes #8699

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
 2025-01-28 04:41:16 +00:00
Evan Mattson 5098ba39a8 Python: Improve handling for kernel plugin from file. (#10286) 
### Motivation and Context

This PR addresses an issue in the `KernelPlugin.from_directory()` method
where classes without any `@kernel_function` decorated methods are still
being initialized during plugin loading.

The fix makes sure that only classes with at least one
`@kernel_function` decorated method are instantiated and included as
plugins. Classes without `@kernel_function` methods are now skipped
entirely.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR:
- Updates the `from_python_file` to inspect each class for
`@kernel_function` decorated methods before instantiation.
- Updates the `add_plugin` method `plugin_name` to make sure it is
indeed a string, per the type hint.
   - Adds a unit test to exercise the new behavior
- Closes #10280

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
 2025-01-24 18:00:28 +00:00
Eduard van Valkenburg e44f70d9d6 Python: bug fixes from graphrag insights (#10270) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Based on insights from the graphrag team, several bug fixes related to
data and model validators.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
For all cases of model_validator in before mode, there is no guarantee
that the data passed is a dict,
especially when a model is used as a field in another model, then a
string is passed to the before validator.
So in those cases we check before doing logic, otherwise just pass it on
and let Pydantic handle the invalid data.

Some small logic fixes in the vector store model decorator:
- updated docstring with the right exceptions
- additional checks on attributes
- additional checks on VectorFields with tests

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2025-01-23 20:22:29 +00:00
Evan Mattson 0d8e3fd3af Python: Remove OpenAI plugin. Update OpenAPI exec params to use direct predicate reference (#10095) 
### Motivation and Context

Dotnet had long removed the OpenAI plugin from its code base, and Python
did not follow at the time. We had previously provided deprecation
messages around using the OpenAI plugin and to switch to the OpenAPI
plugin. We are now removing the OpenAI plugin from the SK Python SDK.

Additionally, there was a forward reference created without specifying
the direct predicate context reference in the
OpenAPIFunctionExecutionParameters. This put the burden on the dev to
have to perform a model rebuild once the import was added. This PR
cleans up the `OperationSelectionPredicateContext` import, which is now
direct and via the sub-module and not via the parent module, which
causes the __init__.py to load all dependencies (which is unnecessary).

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR:
- removes the unsupported OpenAI plugin (previously marked as
deprecated)
- fixes the `OperationSelectionPredicateContext` import to allow a user
to create the func exec params without receiving a pydantic error.
- updates unit tests to remove references to OpenAI plugin.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
 2025-01-07 21:53:31 +00:00
Evan Mattson 16690ed857 Python: Yield FunctionResultContent in streaming chat completion path. Update tests. (#9974) 
### Motivation and Context

Currently, if using SK's Python streaming chat completion path, we only
yield the following content types: `StreamingChatMessageContent` and
`FunctionCallContent`. We are not yielding `FunctionResultContent` which
is valuable for some use cases.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR updates the code to yield `FunctionResultContent` if it exists
in the streaming chat completion path. When we merge the function call
result content together into a `StreamingChatMessageContent` type, we
check if that message has items (which are of type
`FunctionResultContent`) and if so, we yield them. The filter path still
works because once they're yielded, we break out of the function calling
loop.

We need to include the `ai_model_id` if it exists for the current
PromptExecutionSettings because if performing a `reduce` operation to
add two streaming chat message chunks together, the
`StreamingChatMessageContent` that has the function results will break
if the `ai_model_id` is not set (the error is thrown in the `__add__`
override for the `StreamingChatMessageContent`.

Some unit tests that cover function calling were also updated -- during
the test, the test JSON function args were breaking in the `json.loads`
call because they contained single quotes and not double. We're now
sanitizing the args, just in case, so we don't break there.

This PR fixes:
- #9408
- #9968 

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
 2024-12-18 01:09:25 +00:00
Evan MattsonandEduard van Valkenburg b42720890d Python: FunctionResultContent hash fix to handle lists/sets (#9978) 
### Motivation and Context

Currently, the hash function in `FunctionResultContent` does not
properly handle types of list or set. Adding the proper handling to turn
these into a tuple, and then the hash function works properly.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR:
- Makes sure the FunctionResultContent `__hash__` method can properly
handle mutable types like lists or sets
- Adds tests to ensure the new behavior works as expected
- Ensures the create message content method can handle lists or sets
when generating the content and the model can only accept a string.
- Fixes #9977

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Eduard van Valkenburg <[email protected]>
 2024-12-16 22:14:22 +00:00
Eduard van Valkenburg b1f4769ba5 Python: updated pydantic 2.10.3 works again (#9923) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Manual update to pydantic requirement, they fixed the incompatibility in
10.3 (10.<3 was broken).

We need one more fix in Pydantic 2.10, which should come in 2.10.4:
https://github.com/pydantic/pydantic/issues/10910
this is the issue: https://github.com/pydantic/pydantic/issues/11024
Comes into play with some of the vector stores.

So excluded 2.10 smaller then 2.10.4.

Closes #9788

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2024-12-12 21:08:46 +00:00
Evan Mattson 51675d2607 Python: OpenAPI plugin updates to promote plugin to preview (#9670) 
### Motivation and Context

We currently handle OpenAPI plugins via the kernel, but in a pretty
simple way. The changes in this PR are allowing one to define multiple
server paths for an operation as well as configure security on the
operations either at the operation level or at the "global" spec level
(these are included as metadata in the kernel function parameter
`additional_parameters` dict).

The OpenAPI plugin is graduating from an experimental state to a preview
state. This, in turn, removes the `experimental` tag from some public
facing methods. In the near future, all `experimental` tags will be
removed once we're confident there are no other underlying changes
required.

Closes #9719 

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Adds support for OpenAPI plugin:
- allow for multiple servers for an API operation
- allows one to define security configuration at the spec level as well
as per operation level
- fixes a bug where the `excluded_operations` on the execution settings
were never checked during REST API operation building
- adds the deprecated tag to the `OpenAI plugin` code as it was
deprecated by OpenAI and we never handled it in SK Python.
- renames some model class to remove the `operation` portion of the
name.
- adds ability to specify a callback to be able to skip certain api
operations based on name, method, or description
- removes the experimental tag from public facing methods.
- allows one to pass in a parsed OpenAPI spec to the plugin add method
- adds unit tests 

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
 2024-11-21 15:27:32 +00:00
Eduard van ValkenburgandEvan Mattson 93bf06ed3c Python: updated pre-commits (#9739) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Put some of the pre-commit steps to their latest versions.
Improve the setup of the uv lock hook.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
 2024-11-19 16:36:01 +00:00
Eduard van ValkenburgandTao Chen c8b4094835 Python: Introducing vector and text search (#9345) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

This PR does the following things:
- Introduces TextSearch abstractions, including implementation for Bing
- This consists of the TextSearch class, which implements three public
search methods, and handles the internals, the search methods are:
'search' returns a string, 'get_text_search_results' returns a
TextSearchResult object and 'get_search_results' returns a object native
to the search service (i.e. BingWebPages for Bing)
- This also has a method called "create_{search_method}' which returns a
KernelFunction based on the search method. This function can be adapted
by setting the parameters and has several adaptability options and
allows you to create a RAG pipeline easily with custom names and
descriptions of both the functions and the parameters!
- Introduces VectorSearch abstractions, including implementation for
Azure AI Search
- This consists of a VectorStoreBase class which handles all the
internal and three public interfaces, vectorized_search (supply a
vector), vectorizable_text_search (supply a string that get's vectorized
downstream), vector_text_search (supply a string), each vector store
record collection can pick and choose which ones they need to support by
importing one or more next to the VectorSearchBase class.
- Introduces VectorStoreTextSearch as a way to leverage text search
against vector stores
- Since this builds on TextSearch this is now the best way to create a
super powerfull RAG setup with your own data model!
- Adds all the related classes, samples and tests for the above.
- Also reorders the data folder, which might cause some slight breaking
changes for the few stores that have the new vector store model.
- Adds additional IndexKinds and DistanceFunctions to stay in sync with
dotnet.
- Renames VolatileStore and VolatileCollection to InMemoryVectorStore
and InMemoryVectorCollection.

Closes #6832 #6833


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Tao Chen <[email protected]>
 2024-11-06 13:22:50 +00:00
Jerome TremblayandEvan Mattson 4c8e81e399 Python: Fix issue with default opentelemetry metric in pydantic base model (#9292) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
- This is fixing an issue with the default opentelemery metric in
KernelFunction. This issue is with SemanticKernel version >= 1.8
causing 
`TypeError: cannot pickle '_thread.RLock' object`
when importing SemanticKernel

- This is blocking us from updating to the latest SemanticKernel
version.

It is difficult to reproduce outside my application code. But is seems
to be the same issue mentioned in
#9057 and #9012 

changing to a Field with a default factory solve the issue.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

- Updating the default metrics value to use a Field with a
default_factory

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
 2024-10-17 18:00:42 +00:00
Tao Chen 481d16a749 Python: Bedrock connector (#9100) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Add Amazon Bedrock AI to Semantic Kernel.

Amazon Bedrock is a platform on AWS that offers access to a wide range
to LLMs, including proprietary Amazon Titan models.

We are integrating it into Semantic Kernel as one of our AI connectors
to provide more options to developers.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
1. All interfaces implemented: chat completion (streaming +
non-streaming), text completion (streaming + non-streaming), and text
embedding.
2. Supports for function calling.
3. Supports for image content.
4. Supports for all models available on Bedrock.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2024-10-10 22:33:42 +00:00
Evan Mattson 3d0a890e71 Python: Support OpenAI json_schema response format (#8958) 
### Motivation and Context

OpenAI has released in their August 6th model the ability to specific a
json_schema response format. This allows the model to use the specified
Pydantic model or the provided JSON schema representing a "strict"
model/class to make sure that it responds with the exact schema that was
provided.

Note that the strict tool calling will be handled via a separate
ADR/work item once agreed upon by all three SK languages.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR introduces the ability to specific either a Pydantic model or a
non-Pydantic model for the json_schema response format
OpenAIChatPromptExecutionSettings.
- There are two new code sample concept files showing how to do this
with both a Pydantic and a non-Pydantic model. The other code sample
shows how to configure a json_schema response format while using
function calling.
- If the model is unable to respond based on the provided schema, it
will contain a `refusal` attribute as part of the chat completion
message contents. This will be included in the response as a TextContent
item explaining why it couldn't fulfill the request.
- Unit tests added for this new functionality.
- Handles #7946 

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
 2024-09-25 19:12:32 +00:00
Tao Chen 2205df8b88 Python: Fix KernelArgument bug in invoke_prompt functions (#8414) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
In `invoke_prompt(...)` and `invoke_prompt_async(...)`, the `arguments`
parameter is tested to see if it's empty before proceeding. The type of
the `arguments` parameter is `KernelArgument`, which derives from a
`dict` type, with an additional field for execution settings. When
`arguments` contains only the execution settings, it will be considered
as an empty dictionary. This will result in the `arguments` parameter
being overridden completely, invalidating all execution settings passed
to the APIs.

To reproduce (assuming we have a plugin/function that will return the
weather):
```
result = await kernel.invoke_prompt(
        "What is the weather like in my location?",
        arguments=KernelArguments(
            settings={
                service_id: PromptExecutionSettings(
                    service_id=service_id,
                    function_choice_behavior=FunctionChoiceBehavior.Auto(),
                ),
            }
        ),
    )
```
In the example above, the settings will be overridden to None.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
1. Fix the bug.
2. Fix mypy in the pre-commit hook in order to commit.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2024-08-29 15:40:53 +00:00
Tao Chen 6680b2e356 Python: Add metrics instrumentation (#8317) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
We very recently added instrumentation to kernel functions:
https://github.com/microsoft/semantic-kernel/pull/8280. That PR added
logs and traces for observing a kernel function. This PR adds metrics
and modifies the telemetry sample app to show how to export metric
instruments to Application Insights.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
1. Kernel function now produces metrics data for observability.
2. Add code to export metric instruments to the telemetry sample.
3. Add Google AI as a second service to the telemetry sample to show
case nested spans.
4. Mics optimizations.


![image](https://github.com/user-attachments/assets/141e9bd2-06b2-4399-a913-f8aeb12aa8af)


![image](https://github.com/user-attachments/assets/fd35ad84-c59f-418c-b9e6-84beb4b22eb1)


![image](https://github.com/user-attachments/assets/1f40fac0-23fa-4b3e-b6c9-644e654c0cbd)


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2024-08-22 15:34:28 +00:00
Tao Chen 93729f653b Python: Instrument kernel function (#8280) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
It is essential to provide observability in Semantic Kernel so that
developers can monitor their applications built using SK.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
1. Add instrumentation to kernel function (log and traces, metrics will
follow in the next PR).
2. Add a sample application that shows how to configure an application
to be monitored using Application Insights.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2024-08-21 16:10:26 +00:00
Eduard van Valkenburg 51f5554b39 Python: removed D101 and fixes for it (#7853) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Added comments about each of the rules to pyproject.toml
Removed D101 from ignore list and fixed that, now docstrings at the
class level are required.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2024-08-05 16:33:49 +00:00
Gil LaHayeandTao Chen ca78ff73e0 Python: Add OpenTelemetry to Python SK (#6914) 
### Motivation and Context
We want observability into usage of SK

### Description
Add OpenTelemetry to Python SK

### Contribution Checklist
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations

---------

Co-authored-by: Tao Chen <[email protected]>
 2024-07-25 20:18:41 +00:00
Evan Mattson 5f49880d52 Python: Json schema improvements for Union and Optional (#7367) 
### Motivation and Context

We have some gaps in how we build the JSON schema for function calling
related to Union and Optional types. There is also some work to do
related to handling `Optional` versus `param | None = None`.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR improves the JSON schema building for the Union and Optional
types. It adds additional tests and improves some existing tests.
- Closes #7297 
- Closes #7298

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
 2024-07-23 04:38:55 +00:00
Evan Mattson 6b01468efb Python: fix for kernel function type_object when not using Annotations. Add more unit tests. (#7338) 
### Motivation and Context

The current kernel function decorator class parses function params, but
had a miss related to how the underlying param's type is parsed if not
using Annotations.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR properly handles the type_object for params that don't using
typing.Annotations.
- Adds unit tests to exercise the code.
- Adds more kernel.py unit tests for near 100% test coverage.
- Adds some planner tests to improve the test coverage.
- Closes #7300 

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
 2024-07-18 05:12:08 +00:00
Evan MattsonandGil LaHaye 904d1f514f Python: Increase unit test coverage (#7211) 
### Motivation and Context

We've been working on increasing the test coverage. These tests aren't
specific to the CQ spike, but I noticed that there was covering lacking
for them, and they were "low hanging fruit."

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Increasing the test coverage for some classes that have current test
coverage but were missing some line coverage here and there. At this
point in time, the test coverage report is showing 95% covered now.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Gil LaHaye <[email protected]>
 2024-07-11 20:05:06 +00:00
qowlsdn8007andEvan Mattson 90e2237296 Python: Small docstring fix (#7187) 
Docstring in add_plugin_from_openai method explain that it's a plugin
from the openapi manifest, not open ai.


https://github.com/microsoft/semantic-kernel/blob/ed463329e7463b068a6669ecf3b7d03d040d1699/python/semantic_kernel/functions/kernel_function_extension.py#L211

### Motivation and Context
I found the add_plugin_from_openapi method because I needed a function
related to the semantic function for calling the API. While reading the
docstring, I saw an open AI description that had nothing to do with the
method, so I am reporting this.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description
See changes, it's a one-liner 😁

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
 2024-07-10 16:33:56 +00:00
Eduard van Valkenburg fe38a5f5d6 Python: slight improvements in kernel and function result content, sample of local function calling with Nexus Raven (#6982) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
This introduces a sample that uses function calling with a local model,
Nexus Raven V2, in this case using the API, but the same model can be
loaded using the Hugging Face Connection, the class here is highly
adapted and specific to Nexus including parsing the returned function
call value, into FunctionCallContent and then running that.
It then returns one or more ChatMessageContents with the function call
content and function result content which can then be shared with a
OpenAI call.

Error handling improvements:

*
[`python/semantic_kernel/contents/chat_history.py`](diffhunk://#diff-b57a30f97fec1a91bbf6f7d15584473a8f0cf409797bdcd90d8c80bedc670892L266-R267):
Improved error logging when parsing prompts as XML fails.
*
[`python/semantic_kernel/contents/function_result_content.py`](diffhunk://#diff-3b69e6aedf4c29038cdff0ac99602b70583ab6c85f7bd777b08ad96b7fa48d87L7-R10):
Removed field validator for `result` and allow result to be any value,
instead of casting everything to string.
[[1]](diffhunk://#diff-3b69e6aedf4c29038cdff0ac99602b70583ab6c85f7bd777b08ad96b7fa48d87L7-R10)
[[2]](diffhunk://#diff-3b69e6aedf4c29038cdff0ac99602b70583ab6c85f7bd777b08ad96b7fa48d87L50-R51)
[[3]](diffhunk://#diff-3b69e6aedf4c29038cdff0ac99602b70583ab6c85f7bd777b08ad96b7fa48d87L63-L69)
[[4]](diffhunk://#diff-3b69e6aedf4c29038cdff0ac99602b70583ab6c85f7bd777b08ad96b7fa48d87R92-R115)

Minor changes:

*
[`python/semantic_kernel/core_plugins/math_plugin.py`](diffhunk://#diff-632444474bd3ea991b4409dfd0ab194e0789c73617bef4a3dcae4db5c4b0b6c0L32-R38):
Removed some docstrings for clarity.
[[1]](diffhunk://#diff-632444474bd3ea991b4409dfd0ab194e0789c73617bef4a3dcae4db5c4b0b6c0L32-R38)
[[2]](diffhunk://#diff-632444474bd3ea991b4409dfd0ab194e0789c73617bef4a3dcae4db5c4b0b6c0L55-R47)
*
[`python/semantic_kernel/functions/kernel_function_from_prompt.py`](diffhunk://#diff-eabd53da2ec241cce6f92de85283065e7b478d62cefb52817c5b29ff07e28189L173-R174):
Changed the order of condition checks for better readability.
*
[`python/semantic_kernel/kernel.py`](diffhunk://#diff-3db6fe98818eda0b824788901c718d35eac8b13d0ff428d3dfd417866d922d3aR9-R10):
Added functionality to update kernel arguments in `invoke_stream` method
and made `function_name` and `plugin_name` optional in `invoke_prompt`
method.
[[1]](diffhunk://#diff-3db6fe98818eda0b824788901c718d35eac8b13d0ff428d3dfd417866d922d3aR9-R10)
[[2]](diffhunk://#diff-3db6fe98818eda0b824788901c718d35eac8b13d0ff428d3dfd417866d922d3aR29)
[[3]](diffhunk://#diff-3db6fe98818eda0b824788901c718d35eac8b13d0ff428d3dfd417866d922d3aR119-R120)
[[4]](diffhunk://#diff-3db6fe98818eda0b824788901c718d35eac8b13d0ff428d3dfd417866d922d3aL195-R202)
[[5]](diffhunk://#diff-3db6fe98818eda0b824788901c718d35eac8b13d0ff428d3dfd417866d922d3aL209-R216)

Closes #6804
Closes #6944

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2024-06-28 20:17:05 +00:00
Evan Mattson 8381e50e8b Python: Introduce the new function calling abstraction, FunctionChoiceBehavior (#6910) 
### Motivation and Context

The current `FunctionCallBehavior` has allowed us to utilize auto
function calling for OpenAI type models. As we proceed to support more
AI connectors, that differ from OpenAI models, we need to be able to
handle functions for all models that support function calling.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

In this PR:
- We introduce a new function calling abstraction called
`FunctionChoiceBehavior` which has three types: Auto, Required, and
None.
- One is able to configure the `function_choice_behavior` along with
fully qualified function names (e.g., plugin1.function1),
`maximum_auto_invoke_attempts` or `auto_invoke_kernel_functions` in yaml
and JSON prompts. We have a new concept example showing how to do this
for for yaml and JSON prompts.
- If the fully qualified names are specified in the config file, they
take precedence over the filters, if specified at a later point.
- To make sure this isn't a breaking change, we still handle the
previous `FunctionCallBehavior`, but so we can make decisions on the new
`FunctionChoiceBehavior` we map the `FunctionCallBehavior` to
`FunctionChoiceBehavior`. Each time `FunctionCallBehavior` is updated,
`FunctionChoiceBehavior` will be updated, too.
- The `_process_tool_call()` method in the
`open_ai_chat_completion_base` needs to maintain the argument name as
`function_call_behavior` as we cannot introduce a breaking change.
- All concept samples have been converted to use
`FunctionChoiceBehavior`.
- New unit tests have been added for `FunctionChoiceBehavior` and we
still currently support `FunctionCallBehavior` tests to make sure we
haven't broken the backwards compatibility.
- Added `deprecated` typing decorators to classes/methods to alert users
that it would be best to transition to `FunctionChoiceBehavior` even
though `FunctionCallBehavior` is still supported.
- The `FunctionCallingStepwisePlanner` was updated to use the new
`FunctionChoiceBehavior`.
- Closes #6496, #6458
- Addresses #6626 

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2024-06-27 20:34:59 +00:00
Evan Mattson a2720a605d Python: Fix broken concept samples. Run concept examples as tests. (#6660) 
### Motivation and Context

We had many broken concept examples. These samples are also not covered
by tests so it's hard to know when things are out of sync.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Fix the Python concept samples, and also cover them with tests so we
aren't checking in code that is going to break them.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
 2024-06-11 15:28:12 +00:00
Eduard van Valkenburg ba9af0e07b Python: mypy coverage enhancement (#6250) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
extend mypy coverage for prompt_templates, template_engine and some
other smaller folders.

still todo: connectors, memory and planner folders.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2024-05-31 16:37:19 +00:00
Evan Mattson 4508e8d966 Python: properly handle types while building the schema (#6469) 
### Motivation and Context

While parsing the kernel function's arguments, we recurse to the type's
base origin and then set that as the `type_object`. For a type like
`list[MyCustomClass]` only setting the `type_object` as `list` has us
lose out on the other valuable information of the list's type, which is
`MyCustomClass` and what attributes it contains within. This then leads
us to build an incomplete schema when creating the kernel function
parameters and ultimately when we send the JSON schema to the model
during tool calls.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR:
- Closes #6466 
- Refactors the kernel function decorator code to parse
arguments/parameters. We now grab the `type_object` of the
parameter/argument in question, which is used to build the JSON schema.
- Fixes the schema building for the types we build. We were missing
`required` parameters as well.
- Attempts to handle as many edge cases as possible, although there are
probably more that we aren't handling right now -- for example properly
handling all schemas for Unions is tricky.
- Adds unit tests to make sure we are building the JSON schema as
expected. All unit, integration and sample tests are passing.
- Handles a `TypeError` exception during auto function calling where the
model sends back an argument like `2`, which doesn't cause JSON parsing
to fail, but will fail when trying to update the argument dictionary.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
 2024-05-31 16:04:08 +00:00
Eduard van Valkenburg fbb26b5cf6 Python: further improvements to checks and docs in dev_setup (#6445) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Added a section on documentation to the DEV_SETUP
Added a check for the copyright row to pre-commit
Closes #5629

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2024-05-31 13:06:41 +00:00
Eduard van ValkenburgandEvan Mattson 7cb651a809 Python: fix for add_function with conflicts (#6437) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Small change in the logic and detection and log.warning for when the
prompt and template and template_format are not the same, when using
kernel.add_function

Fixes #6412 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
 2024-05-29 18:21:14 +00:00
Eduard van Valkenburg 9e99933a27 Python: new pre-commit actions and pre-commit as GHA (#6376) 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

- Added static code checking to pre-commit (check-ast and
nbqa-check-ast)
- Added Bandit security checking to pre-commit
- Added pre-commit step to python-lint workflow, if it works, can delete
seperate mypy, ruff and black tests

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
 2024-05-28 15:28:04 +00:00