Commit Graph
336 Commits
Author SHA1 Message Date
Mihir PatilandEvan Mattson ccc6ed0467 Python: Catch correct error for does_collection_exist (#6950)
### Motivation and Context

Fixes #6949. 

### Description

Catches the correct error for does_collection_exist. 

### Contribution Checklist

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-06-28 13:35:04 +00:00
Evan Mattson 8381e50e8b Python: Introduce the new function calling abstraction, FunctionChoiceBehavior (#6910)
### Motivation and Context

The current `FunctionCallBehavior` has allowed us to utilize auto
function calling for OpenAI type models. As we proceed to support more
AI connectors, that differ from OpenAI models, we need to be able to
handle functions for all models that support function calling.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

In this PR:
- We introduce a new function calling abstraction called
`FunctionChoiceBehavior` which has three types: Auto, Required, and
None.
- One is able to configure the `function_choice_behavior` along with
fully qualified function names (e.g., plugin1.function1),
`maximum_auto_invoke_attempts` or `auto_invoke_kernel_functions` in yaml
and JSON prompts. We have a new concept example showing how to do this
for for yaml and JSON prompts.
- If the fully qualified names are specified in the config file, they
take precedence over the filters, if specified at a later point.
- To make sure this isn't a breaking change, we still handle the
previous `FunctionCallBehavior`, but so we can make decisions on the new
`FunctionChoiceBehavior` we map the `FunctionCallBehavior` to
`FunctionChoiceBehavior`. Each time `FunctionCallBehavior` is updated,
`FunctionChoiceBehavior` will be updated, too.
- The `_process_tool_call()` method in the
`open_ai_chat_completion_base` needs to maintain the argument name as
`function_call_behavior` as we cannot introduce a breaking change.
- All concept samples have been converted to use
`FunctionChoiceBehavior`.
- New unit tests have been added for `FunctionChoiceBehavior` and we
still currently support `FunctionCallBehavior` tests to make sure we
haven't broken the backwards compatibility.
- Added `deprecated` typing decorators to classes/methods to alert users
that it would be best to transition to `FunctionChoiceBehavior` even
though `FunctionCallBehavior` is still supported.
- The `FunctionCallingStepwisePlanner` was updated to use the new
`FunctionChoiceBehavior`.
- Closes #6496, #6458
- Addresses #6626 

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-06-27 20:34:59 +00:00
Tao ChenandEvan Mattson d22ee5af55 Python: Azure Model-as-a-Service Python connector (#6742)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Related to: https://github.com/microsoft/semantic-kernel/pull/6693

Azure Model-as-a-Service allows users to deploy certain models from the
Azure AI Studio model catalog as an API. This option also provides
pay-as-you-go access to the models hosted. Below are some of the models
that are supported:
- Microsoft Phi-3 family
- Meta Llama family (Llama 2 chat & Llama 3 instruct)
- Mistral-Small & Mistral-Large
- and more

We'd like to provide an AI connector for users of SK to use Azure
Model-as-a-Service.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
A new AI connector named `azure_ai_inference` is added to support Azure
Model-as-a-Service. This connector takes a new dependency on the Python
`azure.ai.inference` SDK.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-06-24 22:00:57 +00:00
Evan Mattson 148e8de701 Python: Updates to user agent. Add unit tests. (#6824)
### Motivation and Context

Updates to the User-Agent are required so that we can align with all
three SK languages. Examples:

User agent
- semantic-kernel-csharp/1.1.1 [existing user agent]
- semantic-kernel-java/1.1.1 [existing user agent]
- semantic-kernel-python/1.1.1 [existing user agent]

semantic-kernel-version header
- csharp/1.1.1 
- java/1.1.1
- python/1.1.1

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Updates to the user agent to reflect the desired changes/standardization
across all SK languages. Adding unit tests.
- Closes #6827 
- Moves the `USER_AGENT` const from its previous home in the openai
const file to a more generic const file.
- Updates unit tests to be able to run properly when a valid `.env` file
exists in the current working directory. The unit tests patch the env
vars and our current implementation of Pydantic Settings loads the
`.env` file if not specified.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-06-20 15:02:01 +00:00
Eduard van Valkenburg 9f205c3245 Python: fix for Python: parameters mentioned in a SK prompt are not synced wi… (#6807)
fixes #6806

### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-06-19 06:35:05 +00:00
Evan Mattson e244dad208 Python: If no env_file is configured then default to relative .env file (#6754)
### Motivation and Context

We'd like to improve the handling of the .env file related to Pydantic
Settings. If Environment variables aren't found, it's possible to fall
back on reading the values from a .env file.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Previous to this PR, that is done by specifying the .env file path. Now,
if the env_file is not specified, we will default to reading from the
relative ".env" file in the cwd.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-06-18 15:29:01 +00:00
Eduard van Valkenburg 1cc6464860 Python: Env sample (#6711)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Clarify the names and uses of the environment variables and pydantic
settings.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-06-14 19:04:44 +00:00
Milad Mobarhan a9d361a82f Python: fix: Preserve class type in experimental_class decorator to avoid type inference issues (#6709)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Closes https://github.com/microsoft/semantic-kernel/issues/6707 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

- Introduce `TypeVar` `T` bounded to `type` to ensure the decorator
maintains the class type.
- Modify the `experimental_class` decorator to use the bounded `TypeVar`
for accurate type hinting.


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-06-14 16:51:25 +00:00
Evan Mattson 68ffd1b1de Python: Azure AI Search api key should not be required. (#6698)
### Motivation and Context

We have a regression that makes the AI Search Setting's API key
required.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Make the AI Search setting a non-required attribute. Fixes #6697

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-06-13 15:12:27 +00:00
Eduard van Valkenburg 05dcb29dea Python: Support for image content and chat message content with image content (#6520)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Introduces a ImageContent class
Allows that type to be part of ChatMessageContent
Adds a sample for both local and online images
Adds many tests with data uri edge cases, and binary parsing edge cases.

Closes #6536 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-06-13 15:09:25 +00:00
Evan Mattson a2720a605d Python: Fix broken concept samples. Run concept examples as tests. (#6660)
### Motivation and Context

We had many broken concept examples. These samples are also not covered
by tests so it's hard to know when things are out of sync.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Fix the Python concept samples, and also cover them with tests so we
aren't checking in code that is going to break them.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-06-11 15:28:12 +00:00
Eduard van Valkenburg 032fa2cfbb Python: import fix for azure ai search settings in chat (#6562)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Fix for an issue where Azure AI search packages where needed for core
behavior.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-06-05 15:06:01 +00:00
Evan Mattson b85bca1919 Python: Handle streaming tool call attempts. Fix func calling concept example. (#6560)
### Motivation and Context

If auto_invoke was False and the function used streaming, the max func
attempts were being set to 0 which Python's range function doesn't like.
The chat gpt auto function calling example was also broken due to wrong
checks on the returned result for FunctionCallResult objects.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Fixing handling streaming tool calls when auto invoke is false and
fixing the chat gpt auto function calling concept example.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-06-05 14:35:01 +00:00
8828b86fd8 Python: Refactoring and fix bug. AuthorRole instead of string literals. (#6391)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

There's an enum called `AuthorRole`. I replaced string literals with the
enum values. PyCharm highlights this in some cases: Expected type
'AuthorRole', got 'str' instead

I also fixed a bug in _function_invocation_filters_stream.py_ where the
argument name was wrong. `author="assistant"` instead of
`role="assistant"`

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
Co-authored-by: Eduard van Valkenburg <[email protected]>
2024-06-04 07:09:00 +00:00
Evan Mattson 2a7763aa3b Python: Updates to fix changes made to ACA Sessions Plugin (#6507)
### Motivation and Context

There were changes made to the ACA upload file API that weren't made in
the Python SDK. This caused the file upload to error out. The ACA team
also passed along their docs and advice to update to a certain API
version, and what changes that brings.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Fixes #6494
Closes #6509 
- Properly handle the file upload's response body as it now contains a
`$values` attribute and then a list of key-value pairs.
- Also properly construct the `remote_file_path` by checking if it
includes `/mnt/data/` and if not, appending it to the front of the
`remote_file_path`.
- Update to use ACA API version 2024-02-02. This means the API version
needs to be included with all requests. Adding API version to requests,
and adding session ID to all requests.
- Removed the ability to pass in a BufferedReader to the upload file
function since the LLM doesn't know how to handle that. As is done in
dotnet, allow the user to specify a local_file_path which the model can
handle, to upload a file.
- Update unit tests

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-06-04 07:00:26 +00:00
Chris BoandEvan Mattson e9b140721b Python: Fix Bing Connector Setting Usage (#6493)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

[PR 6278](https://github.com/microsoft/semantic-kernel/pull/6278) added
support for Bing Custom Search, but had a bug introduced when merging a
change to how settings worked.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

This PR fixes the bug in how settings are referenced.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2024-06-03 18:12:49 +00:00
Eduard van Valkenburg 780aa63e76 Python: implement async support for templates (#6486)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Jinja2 templates now use async functions instead of nested ones!
With full testing

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-06-03 17:12:57 +00:00
Eduard van Valkenburg e6ec0eff4a Python: added tests for core_plugins (#6470)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Added tests to completely cover the core_plugins.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-06-03 15:41:56 +00:00
Chris BoandEvan Mattson 5f40f57897 Python: Support Bing Custom Search (#6278)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Update BingConnector to add support for Bing Custom Search, which can be
used to specify domains and webpages to search and allows custom ranking
adjustments.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Note that API keys for Bing are separate from Bing Custom Search.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-05-31 23:46:06 +00:00
Evan Mattson bf42feda6d Python: Fix function calling concept example. (#6477)
### Motivation and Context

A function calling plugin example is failing due to an incorrect chat
completion base method. Additionally, if one specifies to enable
function calling with auto invoke set to False, and they don't provide a
kernel, then there is no error thrown.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR:
- Fixes the concept example that shows three different ways to handle
function calling.
- Fixes #6453
- Adjusts the error checks for the kernel and arguments in the
open_ai_chat_completion_base: manual function calling doesn't require
arguments, so don't throw if they aren't provided; however, it does
require a kernel, which is used to configure the tools for the settings,
so this check has to be by itself.
- Add a unit test to exercise slightly different behavior.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-05-31 20:50:26 +00:00
Eduard van Valkenburg ba9af0e07b Python: mypy coverage enhancement (#6250)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
extend mypy coverage for prompt_templates, template_engine and some
other smaller folders.

still todo: connectors, memory and planner folders.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-05-31 16:37:19 +00:00
Evan Mattson 4508e8d966 Python: properly handle types while building the schema (#6469)
### Motivation and Context

While parsing the kernel function's arguments, we recurse to the type's
base origin and then set that as the `type_object`. For a type like
`list[MyCustomClass]` only setting the `type_object` as `list` has us
lose out on the other valuable information of the list's type, which is
`MyCustomClass` and what attributes it contains within. This then leads
us to build an incomplete schema when creating the kernel function
parameters and ultimately when we send the JSON schema to the model
during tool calls.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR:
- Closes #6466 
- Refactors the kernel function decorator code to parse
arguments/parameters. We now grab the `type_object` of the
parameter/argument in question, which is used to build the JSON schema.
- Fixes the schema building for the types we build. We were missing
`required` parameters as well.
- Attempts to handle as many edge cases as possible, although there are
probably more that we aren't handling right now -- for example properly
handling all schemas for Unions is tricky.
- Adds unit tests to make sure we are building the JSON schema as
expected. All unit, integration and sample tests are passing.
- Handles a `TypeError` exception during auto function calling where the
model sends back an argument like `2`, which doesn't cause JSON parsing
to fail, but will fail when trying to update the argument dictionary.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-05-31 16:04:08 +00:00
Eduard van Valkenburg fbb26b5cf6 Python: further improvements to checks and docs in dev_setup (#6445)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Added a section on documentation to the DEV_SETUP
Added a check for the copyright row to pre-commit
Closes #5629

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-05-31 13:06:41 +00:00
Eduard van Valkenburg 7ce352b9e3 Python: Pydantic settings config (#6392)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
The initial implementation of Pydantic Settings was not optimal yet,
more optimized now.

### Description

Added a KernelBaseSettings class in kernel_pydantic
Rebased all settings off that
Added single create method to KernelBaseSettings:
- sets model_config.env_prefix to cls.end_prefix allowing us to set the
env_prefix as a classvar on each subclass and it works
- if env_file_path is set, set that in model_config
- strip all None values out of inputted values, then instantiate

This also allows us to pass the settings that are supplied to the
settings class, for overriding env settings.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-05-30 09:38:28 +00:00
Eduard van ValkenburgandEvan Mattson 7cb651a809 Python: fix for add_function with conflicts (#6437)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Small change in the logic and detection and log.warning for when the
prompt and template and template_format are not the same, when using
kernel.add_function

Fixes #6412 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2024-05-29 18:21:14 +00:00
Daniele Antonio MaggioandEvan Mattson 5d25f6a8dd Python: fix: Remove wrong response_format override in AzureChatPromptExecutionSettings class (#6424)
### Motivation and Context

This change is required to fix #5997 


### Description

The change is removing the wrong `response_format` override in
`AzureChatPromptExecutionSettings` class. The PR is also adding a unit
test covering the case where `response format` is defined.

The changes were successfully tested against an `Azure OpenAI` instance
with a `gpt-4o` deployment by specifying an
`AzureChatPromptExecutionSettings` like follows:

```python 
execution_settings =  AzureChatPromptExecutionSettings(
        service_id=service_id,
        ai_model_id=ai_model_id,
        max_tokens=1000,
        temperature=0.2,
        response_format={"type": "json_object"},
    )
```

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2024-05-29 15:52:50 +00:00
Evan Mattson 94c89ed7f8 Python: Fix doc string for allow_dangerously_set_content (#6431)
### Motivation and Context

Fix doc string for allow_dangerously_set_content

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Fix doc string for allow_dangerously_set_content

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-05-28 16:00:00 -04:00
Stefano LottiniandEvan Mattson f7f5abca30 Python: (Astra DB) Explicit projection when reading from Astra DB (#6246)
### Motivation and Context

In view of upcoming changes in the Astra DB Data API, this PR explicitly
sets a projection every time a `find` command is executed, in order to
ensure all necessary fields of the document are returned by the API.

### Description

Added an else branch in the `astra_client.py` to ensure there always is
a "projection" field in the body for the `find` command. This ensures
that `$vector` is returned by the API even in case of Astra DB rolling
out an exclude-it-by-default policy in the future.

### Contribution Checklist

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2024-05-28 19:38:42 +00:00
Evan Mattson 895a580303 Python: Fix schema building for complex types (#6394)
### Motivation and Context

The schema building for complex types worked in some regard, but it
didn't work for all cases. There needs to be better checks on complex
types, if they have __args__ and if so, recurse to the base class type,
and also grab the type that is specified in the list like
list[CustomClass], list[str] or even Union[int, bool].

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR improves the parsing/building of complex types. Added
integration tests to cover more types in the hope that this doesn't lead
to any more issues.
- Closes #6388 

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-05-28 18:03:39 +00:00
Eduard van Valkenburg 9e99933a27 Python: new pre-commit actions and pre-commit as GHA (#6376)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

- Added static code checking to pre-commit (check-ast and
nbqa-check-ast)
- Added Bandit security checking to pre-commit
- Added pre-commit step to python-lint workflow, if it works, can delete
seperate mypy, ruff and black tests

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-05-28 15:28:04 +00:00
StefanandEvan Mattson 65515468f8 Python: Fix typos. (#6381)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

Fixed a bunch of typos.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2024-05-24 16:01:51 +00:00
Evan Mattson 5a050990fa Python: Remove assert on non-required api_key (#6384)
### Motivation and Context

The ACS admin key isn't required as the user can pass in either azure
credentials or token credentials. Right now there is an assert on the
api_key not being null that is blocking.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Remove the assert on the api_key. Closes #6369

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-05-24 14:51:52 +00:00
StefanandEvan Mattson a9d7d5d90f Python: Refactoring. Use get_function and get_plugin. (#6382)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Refactoring some of the code following [this
advice](https://github.com/microsoft/semantic-kernel/pull/6371#discussion_r1611138252)
from the maintainers.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

Look up kernel functions and plugins with the `get_function` and
`get_plugin` method. Removed the unused `DEFAULT_CHAT_SYSTEM_PROMPT`

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-05-24 06:02:18 +00:00
Stefan 76039ed4bd Python: Log exception in planner. (#6371)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

The planner code catches an exception and does nothing with it. I added
an error log so it's not silently failing.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-05-23 20:19:38 +00:00
Evan Mattson 0c9517359a Python: Fix schema handling. Fix function result return for type list. (#6370)
### Motivation and Context

Building the tools json payload from the kernel parameter metadata
wasn't properly including an object of type `array`.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Correctly include the object type `array` so that the tool call doesn't
return a bad request. Add unit tests.
- Closes #6367 
- Closes #6360
- Fixes the FunctionResult return for a type string -- if the
FunctionResult is of type KernelContent then return the first element of
the list, otherwise return the complete list.
- Fix the kernel function from method to include the proper type_object
for the return parameter so that the schema can be created properly.
- Add retry logic for a sometimes flaky function calling stepwise
planner integration test.
- Add a check during function calling that makes sure the model is
returning the proper number of arguments based on how many function
arguments are required.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-05-23 18:00:29 +00:00
Evan Mattson e98cd182fe Python: Fix doc strings (#6378)
### Motivation and Context

Fix docstrings so the docs tool can pass.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Fix docstrings so the docs tool can pass.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-05-23 12:29:23 -04:00
Evan Mattson d66fdcfe5c Python: Add cross language tests (#6318)
### Motivation and Context

As we move towards v1 and beyond it's essential that we have a way to
make sure we are staying in line with the other SK SDKs. Previous to
this we didn't have a way to capture request bodies and payloads and
make sure they confirm to the proper SK standards.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR introduces a number of integration tests that exercise various
aspects of the SK SDK like prompts, prompt templates, functions, and the
kernel.

*TODO*: update the OpenAPI tests with a more JSON specific response that
we can check against.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-05-22 15:42:41 +00:00
Eduard van ValkenburgandEvan Mattson 9d56f4276d Python: Improve test coverage (#6366)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Working through improving test coverage

Small changes introduced:

- ai_service_selector: select_ai_service method, now has extra optional
variable for type_, when not set the behavior is the same as before,
otherwise allows any type of AI service present to be selected,
including embeddings (which was not possible)

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-05-22 15:26:29 +00:00
Eduard van Valkenburg eccad335ae Python: split kernel into kernel extensions for relevant pieces (#6361)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
In order to keep better track of related pieces within the Kernel,
including easier unit testing, this PR splits the kernel into pieces
that are self-contained, but abstract.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
New:
- KernelServicesExtension
- KernelFunctionExtension
- KernelReliabilityExtension

Changed:
- Kernel, now imports the new ones.
- KernelFiltersExtension, moved to filters folder, in line with the
rest.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-05-22 15:03:34 +00:00
Eduard van Valkenburg c35651f242 Python: update for kernel function decorator defaults (#6351)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
There was an issue when you create a function with multiple arguments,
some with a default.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Improved the parsing of the function, using py3.10 specific features.
Closes #6311 

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-05-22 13:22:51 +00:00
Evan Mattson 6158c5b432 Python: Fix FC stepwise planner. (#6357)
### Motivation and Context

The FC stepwise planner was breaking when trying to process the function
call result because if a filter is not configured, then the context will
be None. We need to only try to extract the FC result if the context is
not none, otherwise it will already be present in the chat history.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Check that the FC result context is not None, if so, then extract the FC
result, otherwise proceed as the FC result is in the chat history. Fixes
#6350.
- Removes some extranous logging from json schema addition.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-05-21 22:56:10 +00:00
Evan Mattson 8a5fc1c0ac Python: Separate set and __setitem__ strings (#6356)
### Motivation and Context

Separate set and __setitem__ doc strings

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Separate set and __setitem__ doc strings

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-05-21 16:31:01 -04:00
Evan Mattson 82aede879d Python: Try to fix a doc building issue. (#6354)
### Motivation and Context

The docs tool is breaking because two methods share the same signatures
and the beginnings of the docstring. 
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Try to differentiate the docstrings by a char to see if it fixes the
docs generation.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-05-21 19:21:10 +00:00
Eduard van Valkenburg 81cdde24fc Python: upgraded all files to 310 plus format and removed from future (#6353)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Now that we only support 310 and up, we could unify all files, to use
the new style typing and annotations.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-05-21 16:33:08 +00:00
Eduard van ValkenburgandEvan Mattson 25813502a6 Python: fix for fc stepwise (#6337)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
fixes #6333 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2024-05-20 22:09:54 +00:00
Evan Mattson 517a0f8ed2 Python: Add json schema handling. Add experimental tag to OpenAPI and Memory Connectors. (#6335)
### Motivation and Context

The Python code base could handle some primitive types for the schema
for tool call objects and kernel parameter metadata. However, it
couldn't properly handle the more complex JSON schemas for tool call
objects.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR introduces:
- JSON schema handling for KernelParameterMetadata and tool call
objects.
- Updates to the tool call utils method to properly recurse on the
KernelParameterMetadata's stucture.
- Adds unit tests for this code.
- Add experimental_class/experimental_functions to various parts of the
code base like Memory Connectors and OpenAPI plugin.
- Fixes #6310 
- Fixes #6280

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-05-20 21:56:30 +00:00
ec93cb45eb Python: Adds a memory connector for Azure Cosmos DB for NoSQL (#6195)
### Motivation and Context

Azure Cosmos DB is adding Vector Similarity APIs to the NoSQL project,
and would like Semantic Kernel users to be able to leverage them.

This adds a Memory Connector implementation for Azure Cosmos DB's,
including support for the new vector search functionality coming soon in
Cosmos DB.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Eduard van Valkenburg <[email protected]>
Co-authored-by: Evan Mattson <[email protected]>
2024-05-18 12:32:30 +00:00
Eduard van Valkenburg a894aebaa9 Python: implement filters (#5681)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
This pull request includes significant changes across multiple files,
mainly related to the addition of hooks and the modification of function
invocations in the `semantic_kernel` module. The changes also include
the addition of a new sample and a YAML file, and modifications to the
`__init__.py` files.

Removals:

*
[`python/semantic_kernel/events`](diffhunk://#diff-ebda9504832b19ab83239a92c9a6d5f8c744deff9fef86071c13956ec92bb010L1-L11):
Removed the previously used events.

New Exceptions:

*
[`python/semantic_kernel/exceptions/kernel_exceptions.py`](diffhunk://#diff-450aaa5595a8b22cd6ee212eb79b7d6b0d4e9c1072063ef32018a3e7d3fdf21dR41-R48):
Added new exception classes `OperationCancelledException` and
`HookInvalidSignatureError`.
[[1]](diffhunk://#diff-450aaa5595a8b22cd6ee212eb79b7d6b0d4e9c1072063ef32018a3e7d3fdf21dR41-R48)
[[2]](diffhunk://#diff-450aaa5595a8b22cd6ee212eb79b7d6b0d4e9c1072063ef32018a3e7d3fdf21dR57-R58)

Fixes: #3038 
Fixes: #6276

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-05-17 21:05:56 +00:00
Evan Mattson d9aa617ec8 Python: Create an experimental class and function decorator. (#6298)
### Motivation and Context

There may be classes or functions inside of SK Python that should be
deemed as experimental. Currently in Python there is no out-of-the-box
way to get the decorator.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

As there is a name clash with using `experimental` as the decorator
name, we are introducing two decorators: one `experimental_class` and
one `experimental_function`. The note that "This {function | class} is
experimental and may change in the future" is included in the docstring
and a bool `is_experimental` on the class is added.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-05-16 18:16:12 +00:00
Eduard van ValkenburgandEvan Mattson 2530367c17 Python: improved plugins docstrings (#6287)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Fixes: #6254

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2024-05-16 15:41:23 +00:00