Commit Graph
270 Commits
Author SHA1 Message Date
Aditya NageshandEduard van Valkenburg 8fd1f6a22e Python: Replace 'import_plugin_from_object' with 'add_plugin' (#6025)
'import_plugin_from_object' is removed from Kernel. Replace it's usage
with add_plugin in function docstring

### Motivation and Context

Fix docstrings of core_plugins to replace 'import_plugin_from_object'
with 'add_plugin'.

### Description

'import_plugin_from_object' is removed from Kernel. Replace it's usage
with 'add_plugin' in docstring


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Eduard van Valkenburg <[email protected]>
2024-04-29 17:30:59 +00:00
Eduard van Valkenburg 65e94b3454 Python: updated Chat Message Content with Function Call and Result Content (#5946)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Introducing FunctionCallContent and FunctionResultContent
Changed ChatRole to AuthorRole
Adapting ChatMessageContent to have 1 or more other contents within
(currently TextContent or one of the above)
Closes #5890 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Changed OpenAI classes, to remove OpenAIChatMessageContent and
AzureChatMessageContent
Changed OpenAI classes to create and parse FunctionCallContent and other
new things.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-04-25 12:35:59 +00:00
373083b7d8 .Net: Azure CosmosDB Mongo vCore Memory Store Integration (#5228)
## Motivation and Context (Why the change? What's the scenario?)
I have added Azure Cosmos DB Mongo vCore as Memory to be used by
customers using Kernel Memory and Cosmos DB for their LLM applications.


[https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/vcore/vector-search](url)

---------

Co-authored-by: Mark Wallace <[email protected]>
Co-authored-by: Dmytro Struk <[email protected]>
Co-authored-by: Eduard van Valkenburg <[email protected]>
2024-04-24 16:57:19 +00:00
Eduard van Valkenburg 7d2ac133bc Python: move Hugging Face tests to integration (#5983)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Even though the completion itself does not callout, loading the model is
a HTTP call, hence moving the Hugging Face tests to Integration tests.

Plus small fix to make them run.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-04-24 07:25:43 +00:00
Stefano LottiniandEduard van Valkenburg 57b55af578 Python: (Astra DB) Include caller identity as user-agent when issuing HTTP request to Astra DB's Data API (#5921)
### Motivation and Context

This PR enriches the HTTP requests to the Astra DB Data API (occurring
in the AstraClient) with a User-Agent header specifying the identity of
the caller (specifically, "semantic_kernel"). While not mandatory, this
is good practice with Astra DB and helps collecting aggregate usage data
on the API side.

### Description

Using the `importlib.metadata` library the package version is extracted
(in a fail-safe way) and used, if present, to enrich the caller
information (variable `ASTRA_CALLER_IDENTITY`, later set as the
user-agent header for all HTTP requests to the Data API).

### Contribution Checklist

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Eduard van Valkenburg <[email protected]>
2024-04-22 17:40:39 +00:00
Eduard van Valkenburg 26e6c065fe Python: reduce the amount of complex classes in the serialization to xml (#5907)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Small fix for errors regarding serialization of complex objects to xml
and back when going from input ChatHistory to prompt string to
ChatHistory.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-04-17 15:37:05 +00:00
Eduard van Valkenburg 15691eb931 Python: mypy coverage for contents folder (#5904)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Adds mypy typing coverage for the contents folder, includes some small
changes to the types in it.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-04-17 13:15:38 +00:00
Eduard van Valkenburg dc54c6f203 Python: mypy coverage for functions (#5883)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
extend mypy typing coverage to all classes in 'functions' folder.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-04-16 14:21:29 +00:00
Eduard van Valkenburg 00b75ed7dd Python: added kwargs to embedding and text memory (#5885)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Added kwargs handling throughout embeddings connectors and as a
additional field for the TextMemoryPlugin.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-04-16 14:12:29 +00:00
Eduard van Valkenburg 66a3b5bfea Python: init cleanup (#5872)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

When the root init has to load a lot of stuff, loading the whole thing
becomes slower, only one concept there Kernel, all other things must now
be loaded from a sub package.

The following is not the guidance:
- imports within SK use the full path
- init files are created as much as possible on the root+1 level (like
semantic_kernel.functions) with the pieces that common developers need,
except in connectors and utils, there a developer needs to go deeper
- within the connectors folder this is further detailed first to ai,
memory, search_engine, within ai it is further spread into the different
connectors, for instance everything for openai and azure openai can be
imported using `from semantic_kernel.connectors.ai.open_ai import ...`,
the same within memory
- imports in samples use the abbreviated path as much as possible.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-04-16 13:21:20 +00:00
Eduard van Valkenburg 6751f9d819 Python: Updated plugins (#5827)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

The loading of plugins and functions into Kernel was not fully
consistent and clear, this is now simplified.

Adding a plugin can now be done in two ways:
- multiple: `kernel.add_plugins[plugin1, plugin2]`
- single plugin: `kernel.add_plugin(plugin)`
- if the plugin here is a class, with methods that have the
kernel_function decorator then that is parsed into a plugin

Adding a function can now be done in three ways:
- multiple: `kernel.add_functions([func1, func2]` or
`kernel.add_function({'func1': func1})`
- single: `kernel.add_function('plugin_name', func1)`
- from a prompt: `kernel.add_function(function_name='func1',
prompt='test prompt', ...)`

In other words, all the different methods that were available have been
simplified, these methods also return the created plugin (or updated
plugin for the `add_functions` method. The `add_function` (singular)
method has a parameter `return_plugin` to control whether you get the
created or updated plugin, instead of the created function.

**An important callout:**

One big internal change that this introduces is that a function that
gets added to a plugin (new or existing) is automatically copied, the
metadata is deep-copied and the plugin_name in the metadata is updated,
so this sequence is valid now:

```python
@kernel_function(name='test')
def f(...):
   etc

func = KernelFunctionFromMethod(method=f)
func2 = kernel.add_function('plugin', func)
assert func != func2
```

Also closes: #5855

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Removed KernelPluginCollection
Updated KernelPlugin
   added from_... method to load from different sources
Updated KernelFunctionFromPrompt
   added from_yaml and from_directory methods.

Added and updated tests to 99% coverage of KernelPlugin and
KernelFunctionFromPrompt

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-04-15 13:51:59 +00:00
Evan Mattson ed1770c40b Python: Allow args to pass to func calling stepwise planner (#5830)
### Motivation and Context

A recent feature was added to dotnet to allow for a chat_history object
to be passed into invoke. This only allows the user to affect the
`chatHistoryForSteps`, but doesn't account for generating a plan with
provided context (via the chat history).

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR allows the user to pass in optional kernel arguments or kwargs
(that get turned into kernel arguments). If the user then overrides the
prompt template/step prompt.txt, they can utilize any new arguments that
are supplied.
- Closes #5824 

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-04-10 20:15:06 +00:00
Eduard van ValkenburgandEvan Mattson 629624cee3 Python: removed pipeline (#5779)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Remove the ability to run multiple functions in go in kernel.invoke.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2024-04-04 15:54:19 +00:00
Eduard van Valkenburg 4edaa13d4a Python: updated decorator to allow no brackets (#5776)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Small change to allow a kernel function decorator without name and
description, to be defined without brackets.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-04-04 15:14:15 +00:00
ca2ad19130 Python: Update to AOAI OYD 2024-02-15-preview API version (#5684)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

* Updated models to support the new preview API version
2024-02-15-preview, with some backward compatibility with previous API
shape through the use of aliases
* Updated samples, unit tests and integration tests
* Added experimental settings management using pydantic-settings -- not
strictly required here, but I thought it was useful for testing to be
able to set all values through the .env file, and validation comes for
free using Pydantic. Looking forward to feedback on this. :)

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Abby Hartman <[email protected]>
Co-authored-by: Evan Mattson <[email protected]>
2024-04-03 12:52:25 +00:00
Nick Walker 3437683270 Python: Small docstring fix (#5741)
### Motivation and Context

This is a single-line fix to a Python docstring to get it to conform to
Sphinx standards and convention used
[elsewhere](https://github.com/microsoft/semantic-kernel/blob/290f44d593daa21f5a4f88eb5bb4e49468ebb608/python/semantic_kernel/planners/plan.py#L137)
in the project.

(My team owns the reference docs generation process for this and other
libraries. We forced a build the other day from our side for test
purposes and noticed it stumbled over this line. Feel free to ping me if
any questions.)

### Description

See changes, it's a one-liner 😁

### Contribution Checklist

I ashamedly admit I have done *none* of the below; hopefully this is a
gimme, apologies if it creates noise.

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-04-03 04:40:39 +00:00
Eduard van Valkenburg 290f44d593 Python: small fix for CH serialization (#5738)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Noticed a small issue when ChatHistory is created with different types
of messages, and then serialized.
This fixed that and adds a test for that case.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-04-02 20:20:22 +00:00
Evan Mattson e563920b70 Python: Honor the function calling options if configured. (#5701)
### Motivation and Context

In the function calling stepwise planner, some settings like max_tokens
(based on max_completion_tokens from the planner settings) and the max
auto invoke attempts were not used.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR fixes the issue where the prompt execution settings for chat
completion wasn't using the function calling stepwise planner options,
if configured. Closes #5661

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-04-01 16:32:04 +00:00
Evan Mattson c4c7464f07 Python: Introduce operations to handle OpenAI plugins, improve OpenAPI plugins, and allow for auth (#5695)
### Motivation and Context

Python had the ability to handle operations for OpenAPI plugins, but at
a basic level. It did not have the ability to handle OpenAI plugins, nor
did it allow the user to configure an authentication callback. This PR
brings in that functionality.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR:
- Allows a user to handle OpenAI plugins with/without auth. Closes #5355
- Improves the functionality around handling OpenAPI specs.
- Introduces an auth callback mechanism to allow for REST API calls to
use auth
- Introduces OpenAI/OpenAPI function execution settings.
- Provides two new kernel examples that show how to handle a simple
OpenAI plugin (Klarna) as well as a more complex plugin (Azure Key
Vault) that requires auth.
- Updates unit tests.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-04-01 15:01:08 +00:00
Eduard van Valkenburg 14a0881989 Python: mypy clears kernel.py (#5689)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Fixed all mypy errors for kernel.py
In preparation of extending that to all files in semantic_kernel
Also got the pre-commit hook working on a commit, with mypy checks

fixes #5630

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Moved pre-commit-config to base folder, can now be extended to have
specific hooks for other languages
fixed things in kernel.py to satisfy mypy
used assert for some things, so that they can be skipped when running in
-o mode.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-28 15:10:28 +00:00
Eduard van Valkenburg 75089cb711 Python: Enhance Chat Message Content creation and parsing (#5520)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Introduces a polymorphic approach to creating and rendering Chat Message
Contents and it's subclasses, leverages Pydantic Rootmodel for that.

Fixes #5496 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Adds ChatMessageContentBase that can create all types of CMC's, based on
a type field (str)
Redoes some of the rendering and parsing
also retooled the hierarchy structure for Contents, including
introducing a StreamingContentMixin instead of the
StreamingKernelContent (which was never used), now all content classes
(TextContent, CMC, etc) determine the fields used, while the Mixin adds
one additional field and two additional methods for streaming messages,
especially __add__
everything else runs the same way.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-28 12:45:56 +00:00
Eduard van Valkenburg 192414180e Python: small fix for running prompt without any execution settings (#5687)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
I noticed that when running a piece without any execution settings, it
errored out, while it should use 'default' service with the default
settings for that service, that is now fixed.
Small updates to the rag example.
Added TODO: to replace the embedding_generator service in
SemanticTextMemory with Kernel and the same service_selector pattern as
for KernelFunctions.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-28 11:33:29 +00:00
c110327249 Python: Fix - Action Planner breaking when parameters are not explicitly given in the ask (#5590)
Solution to the issue: Python: Action Planner breaking when parameters
are not explicitly given in the ask
https://github.com/microsoft/semantic-kernel/issues/5583

### Motivation and Context
The action planner breaks when the user does not provide specific
parameters in the ask/goal. Users might have generic questions like
"what is the date today?". Fix for the issue
https://github.com/microsoft/semantic-kernel/issues/5583

### Description

Just adding an IF condition before iterating through the generated_plan
variable to check if parameters are present or not does the job. Added
unit test for a case like the example give above.

### Contribution Checklist

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: arfsyed <[email protected]>
Co-authored-by: Evan Mattson <[email protected]>
2024-03-25 18:23:02 +00:00
Evan Mattson 84a75eca86 Python: Allow function calling stepwise planner to use AzureOpenAI chat service (#5618)
### Motivation and Context

The first iteration of the function calling stepwise planner only used
OpenAI. Now allow AzureChatService to use the planner.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR:
- Allows the use of AzureOpenAI for the FC Stepwise Planner
- Fixes some formatting issues when generating the names/metadata of
Plugins. .NET gets the function metadata as JSON, and the Azure Model
performs much better with it formatted like this.
- Fix some missing spaces in the constant message used in the planner --
Azure is treating the new lines in that const message incorrectly and it
causes content filtering results (reaching out to the team to understand
why this is).

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-03-25 13:33:13 +00:00
Evan Mattson 79620c727e Python: Remove jinja2 built in helpers from custom helpers. Introduce messages custom func helper. (#5617)
### Motivation and Context

Jinja2 has a lot of built-in global functions, filters, and test
methods. There's no need to have the overlap between those built-ins and
the custom helpers we had defined.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR:
- Removes the overlap custom methods we had added so that one can use
the built-in jinja2 filters, tests, and others.
- Add some unit tests to exercise those built-ins, mainly to show how
they can be used.
- Introduces a `messages` custom helper function for handlebars and
jinja2 which pretty prints the current chat_history.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-03-25 08:53:00 +00:00
Evan Mattson 9924aff39c Python: preprend 'Semantic-Kernel' to User-Agent, and add version key-value pair to headers (#5565)
### Motivation and Context

The SK Python custom headers to track the USER_AGENT and the Python
version were getting stuck into the default headers under the USER_AGENT
key. Instead, they need to be represented like:

```
    {
        "UserAgent": HTTP_USER_AGENT,
        "Semantic-Kernel-Version": f"python-{version_info}",
    }
```

Trying to extract the Python version from telemetry data was not working
as expected.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Use the APP_INFO dict as-is, and stick the key-value pairs into the
default_headers.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-03-20 16:46:24 +00:00
Justin D. HarrisandEvan Mattson 03f713ccd6 Python: Ollama: Correct API parameters (#5564)
### Motivation and Context
The inputs to some Ollama APIs were wrong as we were not always passing
"model"
The /embeddings API does not take in a list of texts, it just takes 1
"prompt".

This might resolve: #5124

### Description
Correct Ollama API params for /embeddings.
Ensure that the model set on the Ollama classes is passed in API calls
even if it's not in the settings.

### Contribution Checklist

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄 (I don't think so, also, things were
already not working for me).

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-03-20 14:47:13 +00:00
Eduard van Valkenburg ccbeb7b285 Python: rebuilt xml creation and parsing (#5550)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
We got a report stating that a empty chat_history between two other
params, didn't work, so had to change some things in the way
chat_history xml is done, and subsequent improved the parsing, now using
xml fully.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
- Added to_element methods to CMC classes, to_prompt uses that.
- Redid the chat_history from_prompt method to fully utilize xml
parsing.
- Added chat_history tags to detect an empty chatHistory in a prompt.
- Made sure when only 1 message is put into the ChatHistory after
rendering that that is a USER message, instead of SYSTEM, if multiple
the first one is still SYSTEM.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-19 20:22:25 +00:00
Eduard van Valkenburg 1cf984cc99 Python: Enhanced pre commit and tasks (#5512)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
To further make the SK python robust and make it easier to get
high-quality PRs, some work on the pre-commit-config.yaml and adding a
mypy settings ini.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Added two steps to the pre-commit-config, the first for pypy, the second
for tests with coverage.
Over time we want to mandate these checks to run against a PR before it
goes to github, that will also reduce the number of ruff and black fix
commits, and non-passing unit tests.

Added a mypy.ini, currently all but the root folder is excluded, so that
we can gradually introduce mypy coverage, did the first pieces in
kernel.py, including switch to new style annotations (using from
`__future__ import annotations`) in Kernel.


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-19 20:20:09 +00:00
Evan Mattson 5b127badbf Python: Add support for yaml prompt template (#5527)
### Motivation and Context

SK Python doesn't have a way to create a prompt template config from a
yaml file. This is neeed to further achieve parity with dotnet and java.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR introduces:
- some kernel methods to load yaml prompts from a directory as well as
create a kernel function from a string that contains yaml. Closes #4640
- Unit tests and a syntax example showing how to create a yaml prompt
and load it

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-03-19 19:59:38 +00:00
Evan Mattson 0a1a7cabe8 Python: support Jinja2 templating (#5483)
### Motivation and Context

Jinja2 is a heavily supported Python templating language and we're
introducing it to SK Python.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR brings in the ability to specify Jinja2 templates for use within
SK Python:
- Closes #5343 

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-03-15 13:23:25 +00:00
Eduard van Valkenburg 59c3a9fc9a Python: planner love (#5477)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Based on #5474 went into the `using-the-planner` notebook and made sure
it works with the latest code.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Moved all the planners to use '-' in line with the current work on new
templates and planners, once that is in several extension classes can be
changed.
Encountered several small bugs left over from changes over the last
months.
Got through the whole notebook with results.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-14 14:59:10 +00:00
Eduard van Valkenburg 3af69df7ea Python: Handlebars template support (#5466)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Introducing Handlebars templating to the Python Semantic Kernel!

Closes: #5446
Prework for: #4641

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Adds HandlebarsPromptTemplate as a subclass of PromptTemplateBase
Adds second literal to the PromptTemplateConfig definition
Adds helpers for Handlebars (system_helpers) and
`create_helper_from_function` function
Moves fully_qualified_name into KernelFunctionMetadata instead of in
different extensions.
Defaults are still 'semantic-kernel' templates, and without making
explicit changes that still is the norm.
Adds a azure_chat_gpt_handlebars.py sample

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-14 14:00:05 +00:00
ab65b21be1 Python: convert null descriptions to empty str in basic planner (#5406)
### Motivation and Context

1. Why is this change required? Currently, null descriptions block
running completion of notebook step 5.
  2. What problem does it solve? This fixes null description error.
3. What scenario does it contribute to? Unblocks notebook tutorial step
5
4. If it fixes an open issue, please link to the issue here.
https://github.com/microsoft/semantic-kernel/issues/5309
https://github.com/microsoft/semantic-kernel/issues/5380


### Description

Fixed https://github.com/microsoft/semantic-kernel/issues/5309 and
https://github.com/microsoft/semantic-kernel/issues/5380 by converting
null descriptions to empty strings.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Tony Mu <[email protected]>
Co-authored-by: Evan Mattson <[email protected]>
2024-03-13 17:34:46 +00:00
Evan Mattson 8fbb55b99c Python: Pass kernel args to tool calls (#5455)
### Motivation and Context

After introducing auto tool calls, they mostly work well for primitive
type arguments that are handled by the LLM; however, it may be that the
user would like to include other kernel arguments to be used as part of
their custom functions, that are of type like KernelArguments.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR combines the kernel arguments plus the required arguments for
tool calls. It also includes the kernel function's metadata, which has
the kernel arguments while storing the chat message after performing
tool calling.
- This closes #5454 

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-03-13 13:55:11 +00:00
e7bc4c9c1e Python: Fixed unnecessary slashes in URLs (#5237)
### Motivation and Context

Extra slashes are inserted in endpoint. openai-python receives endpoint
as a string and adds one slash.
Due to the omission of string formatting, processing using rstrip was
added.

```
OK: endpoint ... `http://127.0.0.1/openai/deployments/~`
NG: endpoint ... `http://127.0.0.1//openai/deployments/~`
```

### Description
https://github.com/microsoft/semantic-kernel/issues/5236

### Contribution Checklist

- [*] The code builds clean without any errors or warnings
- [*] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [*] All unit tests pass, and I have added new tests where possible
- [*] I didn't break anyone 😄

Co-authored-by: Eduard van Valkenburg <[email protected]>
Co-authored-by: Evan Mattson <[email protected]>
2024-03-12 20:03:32 +00:00
Evan Mattson 0636b17061 Python: Introduce the function calling stepwise planner (#5350)
### Motivation and Context

To further align with the SK dotnet SDK and its functionality, the
Python SDK needs a function calling stepwise planner. Now that we have
auto tool calling available, it makes sense to introduce this planner.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR:
- Introduces the function calling stepwise planner, its planner options,
and planner result. Closes #4642
- Introduces a kernel example `function_calling_stepwise_planner.py` to
exercise the planner and show how it works.
- Adds unit tests and an integration test.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-03-12 15:52:12 +00:00
Eduard van Valkenburg 2bc3de72da Python: Add extras and minor package updates (#5440)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).

Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713

Made all dependencies part of the main, tagged with optional.

Closes #5322

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-12 15:25:24 +00:00
Eduard van Valkenburg 275db86fd1 Python: add py3.12 to unit tests (#5191)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Since Python 3.12 is now released we should test with it. In the fall
3.8 will be out of support so then we should remove support for that.

Since Numpy no longer support py 3.8 as of
[1.25](https://numpy.org/doc/stable/release/1.25.0-notes.html) we pin
that for py3.8, including other packages that depend on older version of
numpy for py3.8, and allow the new versions to work against newer
version of python.

Closes #5144 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-12 09:27:55 +00:00
Eduard van Valkenburg e884e81776 Python: update usearch to 2.9 (#5420)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Updates the usearch package to 2.9+, and fixes the tests for that.
Closes #2676 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-11 21:10:53 +00:00
Eduard van ValkenburgandEvan Mattson b562d026e3 Python: add missing init from prompt_template folder (#5413)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
adds a __init__ file to the prompt_template folder which exposes the
most important classes from the files.

fixes #5172 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2024-03-11 17:58:09 +00:00
Evan Mattson d7c1febc7b Python: Use the correct exception key for handling streaming exceptions in the kernel (#5417)
### Motivation and Context

In a previous fix, the error -> exception key change was missed in a
line in the kernel. This fixes that.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR:
- fixes the incorrect key (error) and now pulls the exception from the
metadata for streaming invoke for the key `exception`. Closes #5385
- Adds a unit test

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-03-11 14:23:11 +00:00
Eduard van Valkenburg 555a7c8436 Python: fix for streaming openai responses, and first parts of fixes for Chat With Your Data (#5387)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-03-08 21:10:43 +00:00
Evan Mattson bf5d21c69a Python: Properly handle exceptions in the kernel using the exception metadata (#5391)
### Motivation and Context

Two things: 1) Add the exception to the on function invoked args if an
exception exists in the function result. 2) Set the stepwise planner's
goal to the plan's description, not the name.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

The PR addresses:
- Properly handling exceptions to raise them from the kernel if a
function result contains an exception in the metadata. Closes #5385
- In the stepwise planner, the goal was getting set as the plan's name,
instead of the description. Closes #5316

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-03-08 20:21:19 +00:00
Eduard van Valkenburg 9005c7cc2c Python: improved chat history parsing from xml (#5315)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
I noticed that there was a slight bug in the way the chat_history is
parsed from a prompt when you have a message without content (like a
tool_call), fixed that and added a test for that case.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-07 16:26:45 +00:00
d956799df7 Python: fix: qdrant upsert on same id (#5090)
### Motivation and Context
This bug prevent us from using `QdrantMemoryStore` as memory.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
Co-authored-by: Eduard van Valkenburg <[email protected]>
2024-03-07 13:01:21 +00:00
Eduard van Valkenburg 2493e293fb Python: Update kernel tests (#5341)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Adding a whole bunch of tests to better cover Kernel.py.
Also reviewed and updated the conftest and tweaked the scope, now runs
faster!

Skipped efforts for handlers since work is being done there. 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-07 07:38:04 +00:00
Eduard van Valkenburg 973ffc5367 Python: Improve web search (#5317)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Update the web search plugin for the new structures, where types can be
passed, instead of str everywhere.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-06 17:04:06 +00:00
Eduard van Valkenburg 9789608761 Python: Fix action planner (#5312)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Got a report of a bug in the action planner creation, fixed here.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-03-06 16:38:08 +00:00
Evan Mattson d7820df9c2 Python: Add function name and plugin name to invoke prompt. Add unit tests. (#5318)
### Motivation and Context

The kernel.invoke_prompt was missing the function name and plugin name
args and was failing.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR:
- Adds the required arguments. Closes #5313 
- Adds unit tests. 

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-03-06 08:38:47 +00:00