Commit Graph
193 Commits
Author SHA1 Message Date
Evan MattsonandEvan Mattson 2b6bd03b03 Python: Add kernel plugin collection and remove old plugin collection classes (#4764)
### Motivation and Context

This PR solves two works items: #4560 and #4667. The Python SDK had
different ways of handling plugins and collections. The work in the PR
helps align the SDK with dotnet nomenclature and structure.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

A `KernelPluginCollection` is introduced with an underlying
`DefaultKernelPlugin` and a `KernelPlugin` base class. Other code is
refactored to use this new `KernelPluginCollection` model which should
simplify how plugins are handled. Unit tests are added to test the new
functionality. Existing unit tests and integration tests, along with
kernel examples and notebooks were updated based on these changes.

Unlike before, we're requiring plugins to be defined with a name, which
removes the GLOBAL_PLUGIN constant name of "__GLOBAL_PLUGIN__". In some
senses, while creating a plugin or a function, a name may not be
provided and so a random name in the form of a plugin
(p_<random_ascii_chars>) or function (f_<random_ascii_chars>) is used.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-02-01 18:57:00 +00:00
Evan Mattson 7170f0d6a1 Python: Remove sync function invoke. Update other code to use async invoke. (#4796)
### Motivation and Context

Completes #4794. 

Most of SK Python is written with async code; however, there were some
code paths that masked the underlying async code and made it look like a
sync function invoke. In reality, asyncio spun up event loops and
attempted to manage them properly. This becomes tricky when running a
Jupyter notebook as they have their own event loop.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR removes the ability to call a kernel function synchronously.
Since the majority of the code base is written with async methods, and
to match dotnet, which is async, we're removing the sync function invoke
and its dependent code. Some Planners were also run synchronously --
these were updated to run asynchronously. Kernel examples, notebooks,
docs, and tests were updated accordingly.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-01-31 20:29:22 +00:00
Julio Carlos Menendez 5303126c0d Python: Renames request settings to prompt execution settings (#4782)
### Motivation and Context

Following the C# implementation, renames all classes and variables of
`*RequestSettings` to `*PromptExecutionSettings`.

Fixes #4676 

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-01-31 15:58:09 +00:00
9dbbf56c96 Python: Add astra memory (#4374)
### Motivation and Context

We are going to add
[Astra](https://docs.datastax.com/en/home/docs/index.html) as a memory
store.

DataStax Astra DB is a serverless vector database that’s perfect for
managing mission-critical AI workloads. It’s built on Apache Cassandra®,
making Astra DB a highly scalable, reliable database technology. It’s a
powerful all-in-one data storage solution, ideal for Generative AI
projects.

### Description

- Add astra into python/semantic_kernel/connectors/memory/astradb
- Add test into
python/tests/integration/connectors/memory/test_astradb.py

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: jsbilgi <[email protected]>
Co-authored-by: Obioma Anomnachi <[email protected]>
2024-01-30 21:51:04 +00:00
Eduard van Valkenburg 11cf8eacd4 Python: implement KernelContent and all related classes (#4491)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
As we are preparing to move towards v1 where the context no longer is
used, we need a better way of handling the responses from models and
capture them in a more structured way, also in preparation of
multi-modal models and the like.

Closes #4598 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Added KernelContent and StreamingKernelContent base classs, consistent
with the dotnet versions of those.
Subclasses of this class for specific services, for instance, the OpenAI
class has parsing for the content, function_calls, tool_calls in it, and
the Azure OpenAI one has parsing for the context in it.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-01-30 21:15:32 +00:00
Julio Carlos MenendezandEvan Mattson d97f9536c4 Python: Removes the _async suffix where is not needed (#4735)
### Motivation and Context
Delete all *_async where *_sync doesn't exist to make the library more
Pythonic.

Fixes #4629 

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-01-26 17:15:35 +00:00
Eduard van Valkenburg e0ccee0e15 Python: updates to chat_prompt and related (#4720)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Fixes #4711 and #4705 and addresses some inconsistencies in the way the
chat_prompt_template is created, now has more control.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Added parse flags to chatPromptTemplate to allow the user to control is
something is loaded into messages or not.
Added better docstrings and tests for this as well.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-01-25 16:09:10 +00:00
Julio Carlos Menendez 08710373a4 Python: Renames SK prefix to Kernel (#4716)
### Motivation and Context
The .NET SDK uses the `Kernel` prefix instead of `SK` for the context,
function, model, etc. classes. This PR aligns the Python package with
that policy.

Fixes #4561 
### Contribution Checklist

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2024-01-24 15:51:56 +00:00
Julio Carlos MenendezandEvan Mattson a5ce301656 Python: Sets a valid default content filter code to ResponsibleAIPolicyViolation (#4715)
### Motivation and Context

When the RAI response doesn't contain a `code` the class
`ContentFilterAIException` would fail to initialize because it was
setting `""` as content_filter_code. This sets
`"ResponsibleAIPolicyViolation"` instead in that case.

Fixes #4709 

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2024-01-23 19:24:29 +00:00
Evan MattsonandEvan Mattson 674af5468d Python: Rename skills to plugins. Update prompt template config to use execution settings to align with dotnet. (#4595)
### Motivation and Context

To better align with SK dotnet v1, Python skills have been renamed to
plugins. This removes the root samples skills directory as dotnet
supports the plugins and their configs. This the beginning of more work
to consolidate other files/names, per the backlog. I've run several
greps for files or file names containing "skill" (case insensitive), and
no results appear. Do alert if you find anything else that includes
"skill."

Fixes #3319 

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This is a large, breaking change PR as the replacement of "skills" to
"plugins" affects many files. It was difficult to break this up into
smaller chunks without leaving the SK Python repo in a bad state for
some time. Other updates include:

- Prompt template has been updated to use `execution_settings`, which
aligns with dotnet, from the previous `completion` that Python used.
- AzureOpenAI/OpenAI function calling has been updated to match the
current versions of tools/tool_choice. Parsing messages in the open_ai
utils respects this as it now looks for tool_calls and it configures the
function_call with the id (new), name, and argument).
- File renames, where the filename used to contain "skill" will look
like new code, but it's the same code, just with "plugin" in the name.
- Kernel examples and notebooks have been tested and are working. Unit
tests and integration tests are passing.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-01-19 16:47:30 +00:00
Julio Carlos MenendezandEvan Mattson f357b45659 Python: Fixes Plan using an empty function in describe and incorrectly setting context variables for next steps. (#4673)
### Motivation and Context
This PR fixes #4658 and fixes #2933.

### Description
The first one basically needed a None check before using
`self._function` in the `Plan.describe` method.
The second one was incorrectly setting the values for variables for the
next step in the plan. The solution for this was suggested by
@amritpal-singh-98, thank you!

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-01-19 16:32:41 +00:00
Sarah WidderandSarah Widder 9b45e609eb Python: Fix OYD tool message parsing (#4599)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
A recent OYD service deployment changed the schema for the "tool"
message containing citations, this cause the OYD response parsing to
fail.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Updated OYD response parser to accept both schemas for citations and
pass gracefully on to the assistant message deltas if no tool message
can be extracted.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Sarah Widder <[email protected]>
2024-01-12 22:27:44 +00:00
Eduard van Valkenburg cdc99d81f6 Python: added ollama connector (#4477)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Ollama is a great way to run several smaller models locally, this
connector allows that to be used with SK.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Added services and request settings for Ollama.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-01-11 16:24:39 +00:00
Sarah WidderandSarah Widder 4778b0fd30 Python: Fix issues with chat OYD, update samples and tests (#4570)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Fixes #4538 - data source parameters in ExtraBody were not serialized
correctly by Pydantic, so the call to the chat completions extensions
API was failing.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

- Add Pydantic annotation `SerializeAsAny` to `AzureDataSources` to
ensure all parameters of subclasses are serialized, not just the ones in
the base class.
- Update with-data samples to use `AzureChatRequestSettings`
- Update unit tests and unskip integration test

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Sarah Widder <[email protected]>
2024-01-11 02:48:04 +00:00
Evan MattsonandEvan Mattson eaec3d349d Python: Fix positional argument mismatch for invoking native functions. (#4495)
### Motivation and Context

Fixes #4483. Invoking native functions doesn't require 3 params, but
rather 2.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Specify the coroutine for a native function to only take the context,
unlike the semantic invoke that takes context and settings. Black
formatting Jupyter notebooks.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-01-09 17:46:05 +00:00
kanchiandEvan Mattson b17e0641df Python: Debug logging for StepwisePlanner's next step thought (#4333)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

The next_step.thought of StepwisePlanner is not being output in the
debug logs.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2024-01-08 19:35:41 +00:00
Eduard van Valkenburg 111a550e45 Python: moved cspell to own settings, small fixes in typing (#4507)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
As we button down the python version, making some improvements in the
settings and with that come some updates.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
- Moved cspell config from settings to .cspell.json file and added
missing words.
- Some fixes in typing: final -> Final[str]
- Some other small fixes coming from turning on pyright.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-01-08 16:25:26 +00:00
Eduard van Valkenburg cdec3b96da Python: constr removed and replaced by annotated stringconstraints (#4509)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Noticed that constr was depricated, removed and replaced with Annotated
and StringConstraints

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-01-08 16:17:47 +00:00
Evan MattsonandEvan Mattson 8618cd32ef Python: Bump Python Package version for release. Update notebook refs to use AIRequestSettings. (#4478)
### Motivation and Context

Bump Python Package version for release. Update notebook refs and
request settings to use the new AIRequestSettings classes. Fix hugging
face bug that requires generator input as text_inputs instead of prompt.
Add a hugging face request settings unit test.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Bump Python Package version for release. Update notebook refs to point
to new package version.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-01-05 16:35:49 +00:00
Evan MattsonandEvan Mattson 756937fb2c Python: Handle errors if no event loop running in current thread. (#4485)
### Motivation and Context

If invoking multiple functions, there may not be an event loop running
in the current thread. The check for the event loop can throw an error,
which is seen when trying to run functions in a loop, as an example.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Adding a try catch to handle an errors when checking for this event
loop.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2024-01-05 15:31:57 +00:00
Eduard van Valkenburg 833a2d7b86 Python: response format fix (#4479)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-01-04 15:50:55 +00:00
Julio Carlos MenendezandEvan Mattson 7f68a88c25 Python: Supports better exceptions when Azure OpenAI content filtering is triggered (#4428)
### Motivation and Context

When an Azure OpenAI content filter error is triggered the current
`AIException` hides the details of that specific error. This makes it
hard for the consumer to see a more detailed description of the error in
their prompt or input.
Solves #4197

### Description

Detects when a content filter error is raised, then parses the
meaningful information about it and raises a new
`ContentFilterAIException` with that information ready for the consumer
to use.

### Contribution Checklist

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2024-01-04 14:52:50 +00:00
Eduard van Valkenburg fa3e32bb16 Python: set line-length for black in sync with Ruff, run black. (#4396)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
As we are tightening the formatting and linting setups, this was
missing, now the line-length for black and ruff are the same, thereby no
longer running into black issues as often.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2024-01-04 14:28:20 +00:00
Eduard van Valkenburg b91529a62e Python: Implement AI request settings (#4097)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
This PR implements the new approach to AI Request settings as described
here:
https://github.com/microsoft/semantic-kernel/blob/main/docs/decisions/0008-support-generic-llm-request-settings.md
and completes #3312

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Does this:
- Creates AI Request Settings with service_id (referring to the
registered AI services in a kernel) and extension_data, a dict that can
hold any number of settings
- has several methods:
- `from_ai_request_settings` -> constructor to create a new
request_settings object based on another, supports creating specific
ai_request_settings from generic ones.
- `update_from_ai_request_settings` -> updating the fields of a request
setting from another ones.
- `prepare_settings_dict` -> method that is used when actually creating
the dict that get's passed to the handler of the AI service.
- Created subclasses like OpenAIChatRequestSettings which has all the
required fields of a OpenAI Chat completion, same for OpenAIText,
AzureChat, GooglePalm, HuggingFace, etc.
- added `get_request_settings_class` to ai_services to return the class
used for the settings for that service.

The goal is that when running, this settings object get's updated and a
single operations (called `prepare_settings_dict`) is used to return a
dict that can be passed to the api call directly, thereby reducing the
need for custom logic everywhere, this will also negate the need for
things like `complete_chat_with_functions_async` and others. So also
removed complete_chat_..._async methods from openai and azure openai.

It has also changed the prompt_template_config with the generic type for
the AIRequestSettings or subclasses, and some other changes.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2024-01-03 19:57:27 +00:00
Julio Carlos Menendez 09d850b5c5 Python: Prevents LLMs from stripping complex function names (#4366)
### Motivation and Context

When using the `StepwisePlanner` planner along with core skills, for
example `TextMemorySkill`, the LLMs tend to plans with function names
stripped down to the name without the skill name. This makes the plan
fail with an exception like:

`Error invoking action recall: (<ErrorCodes.UnknownError: -1>, "The
function 'recall' was not found.", None)`

This error is because the completion returned by the planner prompt does
not include the fully structured skill function name. Ex.: instead of
returning `_GLOBAL_FUNCTIONS_.recall` it returns `recall` or
`GLOBAL_FUNCTIONS.recall`.

This is somehow relates #3237

### Description


This change instructs the LLM to keep the skill function name unchanged
in its response by adding an example that includes `_` in the name. This
approach mimics how the `SequentialPlanner` solves this same issue.

### Contribution Checklist

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
2023-12-20 11:52:34 +00:00
Julio Carlos MenendezandEduard van Valkenburg d3dd5c8e7e Python: Uses Python standard logging (#4115)
### Motivation and Context

This replaces passing Logger instances as arguments around with the
preferred Python way of having each module define a module-wide `logger`
instance at the top of the module. Fixes #3711

### Description

Removes any `log` argument and argument documentation. Replaces usage of
those `log` arguments with the module-wide `logger` instance.

### Contribution Checklist

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Eduard van Valkenburg <[email protected]>
2023-12-16 18:41:41 +00:00
Evan MattsonandEvan Mattson 67b121515d Python: remove unnecessary class attributes from SKFunctionBase. (#4283)
### Motivation and Context

Fixes #4280. Currently, the str representation of SK functions look
verbose:

'uppercase': SKFunction(FUNCTION_PARAM_NAME_REGEX='^[0-9A-Za-z_]*$',
FUNCTION_NAME_REGEX='^[0-9A-Za-z_]*$',
SKILL_NAME_REGEX='^[0-9A-Za-z_]*$')

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

This PR removes unnecessary class attributes from SKFunctionBase and
exposes some constants in utils/validation.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2023-12-16 14:41:35 +00:00
49a1cb8e9e Python: pysdk issue 3897 - update notebooks (#4177)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

https://github.com/microsoft/semantic-kernel/issues/3897

In this PR, I updated the notebooks to use the latest version of the
python SDK.

To make the notebooks work, I need to:
1. Use keyword arguments
2. Change the default value in the `ContextVariables` from `int/float`
to `str`
3. Update `TextMemorySkill`
4. Update some inputs, such as `ChatMessage`

I have noted some inconsistency issues in the
https://github.com/microsoft/semantic-kernel/issues/3897

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Huijing Huang <[email protected]>
Co-authored-by: Evan Mattson <[email protected]>
2023-12-14 16:38:36 +00:00
Evan MattsonandEvan Mattson be89d6bc92 Python: Resolve hanging issue in Jupyter notebooks when re-running async functions (#4237)
### Motivation and Context

Resolves #4137. The issue was that if you try to re-run the sample joke
in the 00-getting-started notebook, the subsequent call to the joke
hangs.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

There were issues when re-running asynchronous functions
(`_invoke_semantic_async` and `_invoke_native_async`) due to improper
handling of asyncio event loops in a multi-threaded env. Specifically,
the code had an issue when an event loop was already running, leading to
hanging or blocking behavior on subsequent runs.

The change replaced the old code Python's ThreadPoolExecutor, which
simplifies thread management. It handles the creation, execution, and
cleanup of threads more efficiently and safely.

The new `run_async_in_executor` method ensures that each async function
runs in its own new event loop, avoiding conflicts with the main
thread's event loop or other threads.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2023-12-13 18:31:53 +00:00
Sarah WidderandSarah Widder a11c9951c0 Python: Add python connector for Azure OpenAI on your data (#3919)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
This change adds a python connector for Azure OpenAI chat completions
with data, which was already supported in the .NET SDK.


### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

Add connector for Azure OpenAI chat completions with data and associated
sample code for how to use it. Some minor changes were made to the SK
function to handle the "tool" message that contains the citations and
other information returned from the chat completions extensions API.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Sarah Widder <[email protected]>
2023-12-12 21:15:48 +00:00
John Liu 37f2760fdd Python: Fix SKFunction.invoke_stream_async (#3986)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

The `invoke_stream_async` method on a native `SKFunction` is not
functional. The following sample code should work:
```python
import asyncio

import semantic_kernel as sk
from semantic_kernel.skill_definition import sk_function


class StreamNativeSkill:
    @sk_function(name="stream")
    async def stream(self, input: str) -> None: # This should also work: -> AsyncGenerator
        for i in range(5):
            await asyncio.sleep(1)
            yield input + str(i)


async def main():
    kernel = sk.Kernel()

    skill = kernel.import_skill(StreamNativeSkill(), "stream_skill")
    native_stream_func = skill["stream"]

    async for chunk in native_stream_func.invoke_stream_async("HELLO!"):
        print(chunk)


if __name__ == "__main__":
    asyncio.run(main())
```

And should output:
```
HELLO!0
HELLO!1
HELLO!2
HELLO!3
HELLO!4
```

But now it only shows:
```
HELLO!
```

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

Add four new `DelegateTypes` for native asynchronous generator functions
for different types of parameters:
- OutAsyncGenerator: No parameter, has `yield`
- InStringOutAsyncGenerator: One `str` parameter, has `yield`
- InContextOutAsyncGenerator: One `SKContext` parameter, has `yield`
- InStringAndContextOutAsyncGenerator: Both `str` and `SKContext`
parameters, has `yield`

Along with these four new types, add four new `DelegateInference`
functions to detect the types:
- infer_out_async_generator
- infer_in_string_out_async_generator
- infer_in_context_out_async_generator
- infer_in_string_and_context_out_async_generator

And because the async generator functions needs to use
`inspect.isasyncgenfunction` function to check instead of
`inspect.iscoroutinefunction`, add another parameter `is_asyncgenfunc`
to all the infer functions.

And finally, fix `SKFunction._invoke_native_stream_async` so that it can
correctly iterating from an async generator function.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2023-12-08 14:48:11 +00:00
Eduard van ValkenburgandEvan Mattson 61b385defd Python: moved telemetry from pkg_resource to importlib (#4098)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Noticed a message saying pkg_resources.get_distribution was deprecated
so moved getting the version to importlib metadata
### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

Co-authored-by: Evan Mattson <[email protected]>
2023-12-08 14:36:41 +00:00
Evan MattsonandEvan Mattson a204be8faf Python: Bump Python version to 0.4.1. Make TextMemorySkill attributes ClassVar. (#4001)
Bump Python version to 0.4.1. Make TextMemorySkill attributes ClassVar.

### Motivation and Context

Bumping Python version to 0.4.1 for next release. Fix TextMemorySkill
attribute error to make it be a ClassVar after upgrading to Pydantic
2.0.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2023-12-05 18:28:02 +00:00
Evan MattsonandEvan Mattson 15f13b285d Python: allow default_headers configuration and users to pass in custom AzureOpenAI/OpenAI clients. (#3903)
### Motivation and Context

Currently, users in Python have no ability to configure default_headers
for the AzureOpenAI/OpenAI clients we use. Additionally, if a user wants
to set up their own AzureOpenAI/OpenAI client, this isn't possible. This
PR addresses #2895.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Allow users to set default_headers for Chat/Text/Embedding Completion
classes for both OpenAI and AzureOpenAI. Additionally, allow users to
pass in a customer OpenAI/AzureOpenAI client, if desired. Update tests
accordingly.

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2023-12-05 17:05:13 +00:00
Eduard van Valkenburg 0982e3f271 Python: extra tests and a fix for the base_url (#3967)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Fixes the issue flagged by @sarah-widder in #3919, 
and added tests for base_url scenario's.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2023-12-05 15:01:04 +00:00
Eduard van Valkenburg e0259c9e2c Python: fix deployment name parameter when using base_url (#3880)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Small fix, deployment_name is always needed, also when using base_url
for openai.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2023-12-01 15:56:49 +00:00
Eduard van Valkenburg fdd16a7c7c Python: readded user agent telemetry to openai classes (#3881)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
During the reworking of the OpenAI classes for openai sdk v1, the
user-agent telemetry was not ported over, readding that here.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
2023-12-01 15:56:39 +00:00
Eduard van Valkenburg 0868068f9b Python: Upgrade to Pydantic 2+ (#3723)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Upgraded pydantic to 2.0+, much cleaner, less complexity in different
types of base models.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2023-11-30 19:34:08 +00:00
+17 c765cb1109 Python: Make SK compatible with OpenAI 1.0 (#3555)
Need to update SK Python to integrate the breaking changes from
openai-python 1.0

https://github.com/microsoft/semantic-kernel/issues/3330

This PR replaces the old one that was merging to main
https://github.com/microsoft/semantic-kernel/pull/3417

---------

### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Joowon <[email protected]>
Co-authored-by: Roger Barreto <[email protected]>
Co-authored-by: Dmytro Struk <[email protected]>
Co-authored-by: Mark Wallace <[email protected]>
Co-authored-by: Ben Thomas <[email protected]>
Co-authored-by: Ben Thomas <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
Co-authored-by: Aayush Kataria <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Stephen Toub <[email protected]>
Co-authored-by: Teresa Hoang <[email protected]>
Co-authored-by: Chris <[email protected]>
Co-authored-by: Gil LaHaye <[email protected]>
Co-authored-by: Gina Triolo <[email protected]>
Co-authored-by: Tao Chen <[email protected]>
Co-authored-by: Lee Miller <[email protected]>
Co-authored-by: Jennifer Marsman <[email protected]>
Co-authored-by: Weihan Li <[email protected]>
Co-authored-by: SergeyMenshykh <[email protected]>
Co-authored-by: Diego Colombo <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alex Chao <achao@achao>
Co-authored-by: Eduard van Valkenburg <[email protected]>
Co-authored-by: Evan Mattson <[email protected]>
Co-authored-by: Evan Mattson <[email protected]>
2023-11-29 06:44:10 +00:00
rom212andEvan Mattson 712cbafa10 Python: extended notebook 06 to showcase Azure AI Search for semantic memory (#3573)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Sample notebook 06-memory-and-embeddings demonstrated how semantic
kernel lets users add memory to their kernels and also work around
ever-growing contexts that eventually exceed the maximum token limit. It
did so by using the in RAM VolatileMemoryStore and teased the
possibility of offloading this to an external vector store. With this
pull request, we want to provide guidance and repeatable steps so that
users can actually perform semantic search by leveraging an external
vector store, Azure AI Search (formerly known as Azure Cognitive Search

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

The changes consist in new cells added at the tail end of the existing
notebook. We register a new memory store, external this time and show
case how the rest of the code can be reused as-is. We add the capability
of providing access credentials to Azure AI Search in environment
variables as well.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <[email protected]>
2023-11-27 17:18:16 +00:00
dcad9f7ba5 Python: Adding Azure CosmosDB Mongo vCore as a datastore. (#2990)
### Motivation and Context

I have added Azure CosmosDB MongoDB vCore as a data store. MongoDB vCore
now supports vector search on embeddings, and it could be used to
seamlessly integrate your AI-based applications with your data stored in
the Azure CosmosDB. More details about Mongo vCore can be found here:
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/vcore/vector-search.

Issue https://github.com/microsoft/semantic-kernel/issues/2375

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, by providing the following
information:
-->
1. Why is this change required? **MongoDB vCore now supports vector
search on embeddings, and it could be used to seamlessly integrate your
AI-based applications with your data stored in the Azure CosmosDB.**
2. What problem does it solve? This adds a new memory store MongoDB
vCore(Azure CosmosDB)


### Description

<!-- Describe your changes, the overall approach, and the underlying
design.
These notes will help you understand how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Mark Wallace <[email protected]>
2023-11-14 22:55:55 +00:00
Mark WallaceandDevis Lucato 6f7cd6a9f7 Python: Add user agent to OpenAI and OpenAPI headers (#3488)
### Motivation and Context

Replaces: https://github.com/microsoft/semantic-kernel/pull/3074

This PR adds the User-Agent header "Semantic Kernel" to calls made to
OpenAI/Azure OpenAI and OpenAPI calls.

### Description

* Addition of telemetry.py file containing telemetry constants
* ability to disable Azure/OpenAI telemetry with an environment variable
called AZURE_TELEMETRY_DISABLED

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Devis Lucato <[email protected]>
2023-11-14 18:50:09 +00:00
194f2401eb Python: sync C# for Pre/Post RunAsync Handlers (#3063)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
#3038 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Roger Barreto <[email protected]>
Co-authored-by: Dmytro Struk <[email protected]>
2023-11-13 21:45:04 +00:00
Adarsh AcharyaandDmytro Struk 965c23d374 Python: Fixed type error in plan.run_next_step_async method (#3231)
### Description

This PR fixes #3219 

### Contribution Checklist

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Dmytro Struk <[email protected]>
2023-10-19 15:01:30 +00:00
d8a2254685 Python: strings don't have contains method in python (#3229)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
- A substring in string check was being done using a '.contains' method
which doesn't exist in python.
  2. What problem does it solve?
- avoids the error: Error occurred while running plan step: 'str' object
has no attribute 'contains'", AttributeError("'str' object has no
attribute 'contains'"))
  3. What scenario does it contribute to?
  - stepwise planner execution
  7. If it fixes an open issue, please link to the issue here.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

I changed the substring in string check to the proper method for python.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X ] The code builds clean without any errors or warnings
- [ X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ X] All unit tests pass, and I have added new tests where possible
- [ X] I didn't break anyone 😄

---------

Co-authored-by: Kevin Kent <[email protected]>
Co-authored-by: Dmytro Struk <[email protected]>
2023-10-19 14:37:23 +00:00
7d31513d3f Python: Context Variables like dict (#3039)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
It's shame sometimes it shines through that some of the earliest code in
the python code base was a direct port from C#, like with the .get
method on the context variables, made the change here to have the exact
some signature as the default python dict.
Also replaces contains_key with __contains__ so that 'in' keyword can be
used, just like a normal dict.
Fixes #2962 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Changed the .get method
Replaced the contains_key method with __contains__
Made changes in samples, tests and planner to deal with that.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Devis Lucato <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
2023-10-18 17:22:55 +00:00
Adarsh AcharyaandDmytro Struk 988f65e41e Python: Prevent "device" override on "device_map" in HuggingFace Pipeline (#3111)
### Description

Fixes #3110 

### Contribution Checklist

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Dmytro Struk <[email protected]>
2023-10-13 17:07:28 +00:00
JoowonandDmytro Struk b45c35c687 Python: improve ai service usability for text/chat completion (#3172)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
fix #3171

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

Co-authored-by: Dmytro Struk <[email protected]>
2023-10-13 16:54:32 +00:00
Eduard van ValkenburgandAbby Harrison 85ee34b924 Python: Implement Function calling for Chat (#2356)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
I built support for function calling into SK!

Related to/fixes: 
- #2315 
- #2175 
- #1450 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
This implementation builds on top of all the existing pieces, but did
require some major work, so feel free to comment on where that is or is
not appropriate.

- Added a `ChatMessage` class to capture the relevant pieces of function
calling (name and content)
- Added a `complete_chat_with_functions_async` into
OpenAIChatCompletions class
- Added a `function_call` field to ChatRequestSettings class
- Added several helper functions and smaller changes
- Added a sample with updated core_skill that uses function calling to
demonstrate
- Added a second sample that shows how to use function_calling with
non-sk functions.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Abby Harrison <[email protected]>
2023-10-12 17:34:22 +00:00
John LiuandDmytro Struk 98797a9978 Python: Allow action planner to exclude skills or functions (#3096)
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Allow ActionPlanner to exclude skills and functions when generating the
plan. This is very useful and other planner types all have this
functionality.

Solve #3072

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

- Add `ActionPlannerConfig` class for three settings:
  - `excluded_skills`
  - `excluded_functions`
  - `max_tokens`
- Filter out available skills and functions in
`ActionPlanner.list_of_functions` method.
- Rename class attribute "ActionPlanner._skill_name" to
"ActionPlanner.RESTRICTED_SKILL_NAME" just like other planners and
update its value to "ActionPlanner". The `this.*` skills defined in
`skprompt.txt` must also be updated.
- Add the missing `__init__.py`:
`semantic_kernel/planning/action_planner/__init__.py`

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Dmytro Struk <[email protected]>
2023-10-10 17:25:34 +00:00