### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
- A substring in string check was being done using a '.contains' method
which doesn't exist in python.
2. What problem does it solve?
- avoids the error: Error occurred while running plan step: 'str' object
has no attribute 'contains'", AttributeError("'str' object has no
attribute 'contains'"))
3. What scenario does it contribute to?
- stepwise planner execution
7. If it fixes an open issue, please link to the issue here.
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
I changed the substring in string check to the proper method for python.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X ] The code builds clean without any errors or warnings
- [ X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ X] All unit tests pass, and I have added new tests where possible
- [ X] I didn't break anyone 😄
---------
Co-authored-by: Kevin Kent <[email protected]>
Co-authored-by: Dmytro Struk <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
It's shame sometimes it shines through that some of the earliest code in
the python code base was a direct port from C#, like with the .get
method on the context variables, made the change here to have the exact
some signature as the default python dict.
Also replaces contains_key with __contains__ so that 'in' keyword can be
used, just like a normal dict.
Fixes#2962
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Changed the .get method
Replaced the contains_key method with __contains__
Made changes in samples, tests and planner to deal with that.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Devis Lucato <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
I built support for function calling into SK!
Related to/fixes:
- #2315
- #2175
- #1450
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
This implementation builds on top of all the existing pieces, but did
require some major work, so feel free to comment on where that is or is
not appropriate.
- Added a `ChatMessage` class to capture the relevant pieces of function
calling (name and content)
- Added a `complete_chat_with_functions_async` into
OpenAIChatCompletions class
- Added a `function_call` field to ChatRequestSettings class
- Added several helper functions and smaller changes
- Added a sample with updated core_skill that uses function calling to
demonstrate
- Added a second sample that shows how to use function_calling with
non-sk functions.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Allow ActionPlanner to exclude skills and functions when generating the
plan. This is very useful and other planner types all have this
functionality.
Solve #3072
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
- Add `ActionPlannerConfig` class for three settings:
- `excluded_skills`
- `excluded_functions`
- `max_tokens`
- Filter out available skills and functions in
`ActionPlanner.list_of_functions` method.
- Rename class attribute "ActionPlanner._skill_name" to
"ActionPlanner.RESTRICTED_SKILL_NAME" just like other planners and
update its value to "ActionPlanner". The `this.*` skills defined in
`skprompt.txt` must also be updated.
- Add the missing `__init__.py`:
`semantic_kernel/planning/action_planner/__init__.py`
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
---------
Co-authored-by: Dmytro Struk <[email protected]>
### Motivation and Context
Resolves: #2591
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
* Added in the `MongoDBAtlasMemoryStore`. A memory store abstraction
that facilitates connections between a MongoDB Atlas cluster to conduct
[Atlas Vector
Search](https://www.mongodb.com/products/platform/atlas-vector-search)
on the Microsoft Semantic Kernel.
* Leverages our async python driver
[motor](https://motor.readthedocs.io/en/stable/) to keep with the
heuristic of asynchronous code
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
### Callout
* When writing my test, I noticed there's a habit of other tests
conducting a `try` on imports. Grepping through those implementations, I
see some of them [encapsulate their respective libraries in function
calls rather than doing direct imports in the
module.](https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/connectors/memory/chroma/chroma_memory_store.py#L55-L63)
Is this the general code pattern we should follow? Or is it fine to have
our imports at the top of the module fine?
---------
Co-authored-by: Steven Silvester <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
This solves [this
issue](https://github.com/microsoft/semantic-kernel/issues/1069) where
native functions are not returning the expected result. In python,
return typehints are optional. However, the semantic kernel requires a
return typehint to determine how to handle the function. For that
reason, we're now enforcing return typehints and raising an exception
when one is not provided. This may break existing void functions but
none in our repo are affected.
Fixes#1069
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [] I didn't break anyone 😄
## Summary
This pull request includes two sets of changes. First, it fixes the
return type annotations in three core skills: file_io_skill,
text_memory_skill, and wait_skill. Specifically, the write_async method
in file_io_skill and the save_async method in text_memory_skill now have
a return type of None, and the wait method in wait_skill now has a
return type of None. Additionally, the
import_native_skill_from_directory method in kernel.py has been
refactored to simplify the code. Second, this pull request adds error
handling for functions with no return type in the delegate inference
module. Previously, if a function did not specify a return type, the
DelegateType could not be inferred, leading to errors. This pull request
adds a check for functions with no return type and raises a
KernelException with an appropriate error message. Additionally, this
pull request includes some minor changes to unit tests for native
functions.
## Changes
- Fixed return type annotation for write_async method in file_io_skill
- Fixed return type annotation for save_async method in
text_memory_skill
- Fixed return type annotation for wait method in wait_skill
- Refactored import_native_skill_from_directory method in kernel.py
- Added error handling for functions with no return type in the delegate
inference module
- Added a check for functions with no return type and raised a
KernelException with an appropriate error message
- Made minor changes to unit tests for native functions
---
*Powered by [Microsoft Semantic
Kernel](https://github.com/microsoft/semantic-kernel)*
### Motivation and Context
This PR parallels work done in .NET for
https://github.com/microsoft/semantic-kernel/pull/2671 and closes
https://github.com/microsoft/semantic-kernel/issues/2722.
This PR allows users to set the chat system message using the semantic
function prompt template.
### Description
* added new `chat_system_message` parameter to
`complete_request_setttings` (defaults to None) and
`chat_request_settings `(defults to "Assistant is a large language
model.")
* added handling of this new parameter to the `chat_prompt_template`
* added corresponding tests
* added a small helper function for deserializing prompt_configs - token
selection biases need to be deserialized to ints, otherwise json loads
deserializes the keys in the set to be strings.
Unrelated
* added missing __init__ file to the weaviate connector's directory
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
### Motivation and Context
The python integration tests use the public redis docker image. This
image is not available to our macOS and windows test agents. This PR
ensures that the tests are skipped on non-linux platforms (CI only).
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Noticed that another commonly edited environment setting for api_version
was missing from the settings util. And want to pass the env to the
constructor.
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Added api_version to the settings util so that that can be controlled
through .env as well.
Added convenience function that outputs a dict that can be passed
directly into Completion constructors.
Updated sample to use that.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#dev-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Lee Miller <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
1. .NET supports Redis memory while Python does not
2. Match up Python features w/ .NET
3. Users can utilize a Redis backend for memories
4. Closes#1981
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Implemented a memory connector for Redis and made unit tests for it.
Also added .env configuration for a connection string.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#dev-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
---------
Co-authored-by: Dmytro Struk <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
I was trying to use this library to load and use [HF Falcon
model](https://huggingface.co/tiiuae/falcon-7b-instruct). Unfortunately
in order to make it work I needed to pass in some additional arguments
such as `torch_dtype`, `cache_dir` or `tokenizer`. It was not possible
with current implementation so I decided to add `model_kwargs` and
`pipeline_kwargs` to the `HuggingFaceTextCompletion` class init function
so now, with the changes I introduced I can successfully load any
`falcon` models by using this code:
```python
import semantic_kernel as sk
import semantic_kernel.connectors.ai.hugging_face as sk_hf
import torch
from transformers import AutoTokenizer
cache_dir = "/mnt/tmp"
model_name = "tiiuae/falcon-7b-instruct"
tokenizer = AutoTokenizer.from_pretrained(
pretrained_model_name_or_path=model_name,
cache_dir=cache_dir,
trust_remote_code=True)
hf_model = sk_hf.HuggingFaceTextCompletion(
model_name,
task="text-generation",
model_kwargs={
"cache_dir": cache_dir,
"torch_dtype": torch.bfloat16
},
pipeline_kwargs={
"tokenizer": tokenizer,
"trust_remote_code": True,
"device": 0,
}
)
```
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
The changes I introduced add `model_kwargs` and `pipeline_kwargs` to
`HuggingFaceTextCompletion` class so that these arguments are then
passed to the underlying `transformers.pipeline()` function and thus
allow more customization options.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
SKFunction has been excluded for now.
Handles issues arising due to many circular imports Update
DelegateInference to handle some cases where annotation is given as a
string to avoid circular imports like in FileIOSkill.
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Mark Karle <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Lee Miller <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
### Motivation and Context
The integration of [USearch](https://github.com/unum-cloud/usearch) as a
memory connector to Semantic Kernel (SK).
### Description
The USearch `Index` does not natively have the ability to store
different collections, and it only stores embeddings without other
attributes like `MemoryRecord`.
The `USearchMemoryStore` class encapsulates these capabilities. It uses
the USearch `Index` to store a collection of embeddings under unique
IDs, with original collection names mapped to those IDs. Other
`MemoryRecord ` attributes are stored in a `pyarrow.Table`, which is
mapped to each collection.
It's important to note the current behavior when a user removes a record
or upserts a new one with an existing ID: the old row is not removed
from the `pyarrow.Table`. This is done for performance reasons but could
lead to the table growing in size.
By default, `USearchMemoryStore` operates as an in-memory store. To
enable persistence, you must set the persist mode with calling
appropriate `__init__ `, supplying a path to the directory for the
persist files. For each collection, two files will be created:
`{collection_name}.usearch` and `{collection_name}.parquet`. Changes
will only be dumped to the disk when `close_async` is called. Due to the
interface provided by the base class `MemoryStoreBase`, this happens
implicitly when using a context manager, or it may be called explicitly.
Since collection names are used to store files on disk, all names are
converted to lowercase.
To ensure efficient use of memory, you should call `close_async`.
---------
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
solve #2321
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Lee Miller <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
implement stepwise planner #2221
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Lee Miller <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Fix failing integration tests introduced in #2474 (May be filing follow
up issue on that)
- Update WriterSkillFake to include a context parameter for the end
marker
- Update test_sequential_planner to use the new end marker parameter in
WriterSkillFake
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
### Motivation and Context
Implementation of Google PaLM connector with chat completion, text
embedding and three example files to demonstrate their functionality.
Closes#2098
I also opened a PR for text completion #2076
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
**What's new:**
1. Implemented Google Palm connector with chat completion and text
embedding
2. Added 3 example files to `python/samples/kernel-syntax-examples`
3. The example files show PaLM chat in action with skills, memory, and
just with user messages. The memory example uses PaLM embedding as well
5. Added integration tests to test chat with skills and embedding with
kernel.memory functions
6. Added unit tests to verify successful class initialization and
correct behavior from class functions
**There are some important differences between google palm chat and open
ai chat:**
1. PaLM has two functions for chatting, `chat` and `reply`. The chat
function in google's genai library starts a new conversation, and reply
continues the conversation. Reply is an attribute of the response object
returned by chat. So an instance of the `GooglePalmChatCompletion` class
needs a way to determine which function to use, which is why I
introduced a private attribute to store the response object. See
https://developers.generativeai.google/api/python/google/generativeai/types/ChatResponse
2. PaLM does not use system messages. Instead, the chat function takes a
parameter called context. It serves the same purpose, to prime the
assistant with certain behaviors and information. So when the user
passes a system message to `complete_chat_async`, it is passed to `chat`
as the context parameter.
3. Semantic memory works with the chat service as long as the user
creates a chat prompt template. The prompt containing the memory needs
to be added to the chat prompt template as a system message. See
`python\samples\kernel-syntax-examples\google_palm_chat_with_memory.py`
for more details. If the only purpose of `complete_async` in
`GooglePalmChatCompletion` is to send memories + user messages to the
chat service as a text prompt, then `complete_async` is not fulfilling
its intended purpose. A possible solution would be to send the text
prompt as a request to the text service within `complete_async`.
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
Currently no warnings, there was 1 warning when first installing genai
with `poetry add google.generativeai==v0.1.0rc2` from within poetry
shell: "The locked version 0.1.0rc2 for google-generativeai is a yanked
version. Reason for being yanked: Release is marked as supporting Py3.8,
but in practice it requires 3.9". We would need to require later
versions of python to fix it.
---------
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
solve #2473 and add unit test to prevent this from happening again.
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Implementation of Google PaLM connector with text completion and an
example file to demonstrate its functionality.
Closes#1979
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
1. Implemented Google Palm connector with text completion
2. Added example file to ```python/samples/kernel-syntax-examples```
3. Added integration tests with different inputs to kernel.run_async
4. Added unit tests to ensure successful initialization of the class and
successful API calls
5. 3 optional arguments (top_k, safety_settings, client) for
google.generativeai.generate_text were not included. See more
information about the function and its arguments:
https://developers.generativeai.google/api/python/google/generativeai/generate_text
I also opened a PR for text embedding and chat completion #2258
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#dev-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
Currently no warnings, there was 1 warning when first installing genai
with `poetry add google.generativeai==v0.1.0rc2` from within poetry
shell: "The locked version 0.1.0rc2 for google-generativeai is a yanked
version. Reason for being yanked: Release is marked as supporting Py3.8,
but in practice it requires 3.9". We would need to require later
versions of python to fix it.
---------
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
All (unit) tests are passing.
### Motivation and Context
This change reduces the number of user facing changes and aligns the
type of
ReadOnlySkillCollection with the usage in SK. There is only one type of
`SkillCollection` in SK, so it doesn't make sense to make
`ReadOnlySkillCollection`
a generic. If, in the future, another SkillCollection type is added,
then we might
want to change this to a generic type.
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Adding optional batch size parameter for when Azure OpenAI limits number
of inputs for embeddings requests. Fixes
https://github.com/microsoft/semantic-kernel/issues/2099
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
This is part 4 of the Serialization work for semantic kernel.
Corresponding issue:
https://github.com/microsoft/semantic-kernel/issues/131
### Motivation and Context
While there are no user facing changes, the ReadOnlySkillCollection has
been modified to remove
circular dependencies that were causing issues with serialization.
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
This allows us to import OpenAPI documents, including ChatGPT plugins,
into the Semantic Kernel.
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
- The interface reads the operationIds of the openapi spec into a skill:
```python
from semantic_kernel.connectors.openapi import register_openapi_skill
skill = register_openapi_skill(kernel=kernel, skill_name="test", openapi_document="url/or/path/to/openapi.yamlorjson")
skill['operationId'].invoke_async()
```
- Parse an OpenAPI document
- For each operation in the document, create a function that will
execute the operation
- Add all those operations to a skill in the kernel
- Modified `import_skill` to accept a dictionary of functions instead of
just class so that we can import dynamically created functions
- Created unit tests
TESTING:
I've been testing this with the following ChatGPT plugins:
- [Semantic Kernel Starter's Python Flask
plugin](https://github.com/microsoft/semantic-kernel-starters/tree/main/sk-python-flask)
- [ChatGPT's example retrieval
plugin](https://github.com/openai/chatgpt-retrieval-plugin/blob/main/docs/providers/azuresearch/setup.md)
- This one was annoying to setup. I didn't get the plugin functioning,
but I was able to send the right API requests
- Also, their openapi file was invalid. The "servers" attribute is
misindented
- [Google ChatGPT
plugin](https://github.com/Sogody/google-chatgpt-plugin)
- [Chat TODO plugin](https://github.com/lencx/chat-todo-plugin)
- This openapi file is also invalid. I checked with an online validator.
I had to remove"required" from the referenced request objects'
properties:
https://github.com/lencx/chat-todo-plugin/blob/main/openapi.yaml#L85
Then I used this python file to test the examples:
```python
import asyncio
import logging
import semantic_kernel as sk
from semantic_kernel import ContextVariables, Kernel
from semantic_kernel.connectors.ai.open_ai import AzureTextCompletion
from semantic_kernel.connectors.openapi.sk_openapi import register_openapi_skill
# Example usage
chatgpt_retrieval_plugin = {
"openapi": # location of the plugin's openapi.yaml file,
"payload": {
"queries": [
{
"query": "string",
"filter": {
"document_id": "string",
"source": "email",
"source_id": "string",
"author": "string",
"start_date": "string",
"end_date": "string",
},
"top_k": 3,
}
]
},
"operation_id": "query_query_post",
}
sk_python_flask = {
"openapi": # location of the plugin's openapi.yaml file,
"path_params": {"skill_name": "FunSkill", "function_name": "Joke"},
"payload": {"input": "dinosaurs"},
"operation_id": "executeFunction",
}
google_chatgpt_plugin = {
"openapi": # location of the plugin's openapi.yaml file,
"query_params": {"q": "dinosaurs"},
"operation_id": "searchGet",
}
todo_plugin_add = {
"openapi": # location of the plugin's openapi.yaml file,
"path_params": {"username": "markkarle"},
"payload": {"todo": "finish this"},
"operation_id": "addTodo",
}
todo_plugin_get = {
"openapi": # location of the plugin's openapi.yaml file,
"path_params": {"username": "markkarle"},
"operation_id": "getTodos",
}
todo_plugin_delete = {
"openapi": # location of the plugin's openapi.yaml file,
"path_params": {"username": "markkarle"},
"payload": {"todo_idx": 0},
"operation_id": "deleteTodo",
}
plugin = todo_plugin_get # set this to the plugin you want to try
logger = logging.getLogger(__name__)
logger.addHandler(logging.StreamHandler())
logger.setLevel(logging.DEBUG)
kernel = Kernel(log=logger)
deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()
kernel.add_text_completion_service(
"dv", AzureTextCompletion(deployment, endpoint, api_key)
)
skill = register_openapi_skill(
kernel=kernel, skill_name="test", openapi_document=plugin["openapi"]
)
context_variables = ContextVariables(variables=plugin)
result = asyncio.run(
skill[plugin["operation_id"]].invoke_async(variables=context_variables)
)
print(result)
```
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
This is part 3 of the Serialization work for semantic kernel.
Corresponding issue:
https://github.com/microsoft/semantic-kernel/issues/131
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Contains one breaking change:
`TextBlock(...)` becomes `TextBlock.from_text(...)`
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
### Summary
This is part 1 of the Serialization work for semantic kernel.
Corresponding issue:
https://github.com/microsoft/semantic-kernel/issues/131
### Motivation and Context
This PR allows core skills (all except 2), retry mechanisms and template
engine protocols to be used
as valid pydantic fields. This allows me to convert the classes that
depend on these classes to pydantic
models, and thus be able to serialize them.
No user facing code changes are expected from this PR.
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
### Motivation and Context
This PR addresses multiple issues with the integration tests. Fixes
integration test breaks related to qdrant and chroma package updates.
Labels known flaky tests as xfail. Adds merge queue gates. Adds push
notifications to teams for test results.
### Description
- integration tests are now 1x daily down from 2x daily
- adds a run condition for `merge_group`
- if run for merge_group, a subset of the tests will gate the merge
- ends the results of each test matrix to a dedicated teams channel
- update qdrant client requirement in the pyproject.toml
- remove `persist_directory` from chroma, as the default is now
persistent
- default with_embedding to false in qdrant_memory_store
- modify chroma test setup and teardown to create the database in a
known location and clear the database of entries at the end of each test
(I would have just deleted the database at the end of each test, but
there are test agent permissions issues with this in CI)
- marked known flaky tests (pinecone and sequential_planner) as allowed
to fail
Fixes#2284
---------
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Dmytro Struk <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
This PR is a first draft for implementation of ActionPlanner in Python.
I try to follow the C# implementation as much as possible.
Fixes: #1917
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
- I have implemented ActionPlanner in the
`python/semantic_kernel/planning/action_planner` directory. The
implementation is in a file called `action_planner.py` and the prompt
used by the planner (a replica of the prompt used in C#) is found in
`skprompt.txt` in the same directory.
- The functions `good_examples`, `edge_case_examples` and
`list_of_functions` are used by the prompt to populate examples and list
of available functions in the kernel.
- After creating the plan, I parse the JSON generated by the planner by
using regex matching to extract out pure JSON as implemented in #1496
- Based on whether or not the planner was able to identify a function, I
create a Plan instance and return it.
### Here is some example code for using the planner
```python
import logging
import semantic_kernel as sk
from semantic_kernel.planning import ActionPlanner
from semantic_kernel.connectors.ai.open_ai import (
OpenAIChatCompletion,
OpenAITextCompletion
)
from semantic_kernel.core_skills import (
MathSkill,
FileIOSkill,
TimeSkill,
TextSkill
)
async def main():
kernel = sk.Kernel()
api_key, org_id = sk.openai_settings_from_dot_env()
kernel.add_text_completion_service("dv", OpenAITextCompletion("text-davinci-003", api_key, org_id))
kernel.import_skill(MathSkill(), "math")
kernel.import_skill(FileIOSkill(), "fileIO")
kernel.import_skill(TimeSkill(), "time")
kernel.import_skill(TextSkill(), "text")
planner = ActionPlanner(kernel)
ask = "What is the sum of 110 and 990?"
plan = await planner.create_plan_async(goal=ask)
result = await plan.invoke_async()
print(result.result)
if __name__ == "__main__":
import asyncio
asyncio.run(main())
```
### Queries
- In the C# implementation, when populating the function parameters in
the plan object, the `plan.parameters` variable is updated with the
function parameters. When I did the same in python, these parameters
weren't being passed onto the function in `invoke_async` method of the
plan instance. The parameters are only being passed when I update them
in the `plan.state` variable. This code is in `line 121` of
`action_planner.py`. What is the right way to do this?
- Some skills / plugins require one of the inputs to be passed as the
`input` variable and not in the context. How should this be handled in
the planner?
**OR**
### Issues that I faced
- The `text-davinci-002` text-completion model regularly output
incorrectly formatted JSON with commas missing. For example
```json
{""plan"":{
""rationale"": ""The list contains a function that allows for writing to
files, so the plan is to use that function to write the desired content
to the file 'output.txt'."",
""function"": ""fileIO.writeAsync"",
""parameters"": {
""input"": null,
""content"": ""A brief history of computer science""
""path"": ""output.txt""
}}}
```
Pending Items
- [x] Create unit tests
- [x] Send the function parameters to the planner in list of available
functions with/without input parameter.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [x] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Shawn Callegari <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
fixes#1915
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [ ] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Shawn Callegari <[email protected]>
### Motivation and Context
This PR adds Qdrant support for SK memory to the Python SK
### Description
- added `qdrant_memory_store.py`
- added `test_qdrant_memory_store.py`
- added dependency group `qdrant` to the pyproject.toml
- added dependency group `qdrant` to the `--without` list for python
unit tests
- updated poetry
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#dev-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Tara W <[email protected]>
Co-authored-by: Dmytro Struk <[email protected]>
### Motivation and Context
The milvus integration tests are failing on windows systems only - get
calls seem to be timing out. The integration tests rely on milvus-lite,
a local, lightweight milvus application for python. Milvus is not
officially supported for windows, so this PR disables the tests for any
windows platform.
See milvus requirements documentation at:
https://github.com/milvus-io/milvus-lite/blob/main/README.md#requirements
### Description
- mark the milvus tests as _skip_ if the local platform is windows.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#dev-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
### Motivation and Context
The service was renamed to Azure Cognitive Search in 2019, so we should
follow the new convention. The only thing still named "Azure Search" is
the old Azure SDK, out of scope.
### Description
* Revert nuget package name to the previous name (we'll delete the new
package, which has been out only for few days)
* Rename C# and Python classes and namespaces
This change brings in Milvus to the semantic kernel connectors.
---------
Signed-off-by: Filip Haltmayer <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
* Move Azure Search dependencies to own group
* Improve Azure Search integration tests: try to delete collections when
tests fail
* Wait 1 sec after upsert to reduce random failures
---------
Co-authored-by: Abby Harrison <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Temporal solution for #1606
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [ ] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Shawn Callegari <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
* .NET, Python: Add new Memory Storage based on Azure Cognitive Search
with custom embeddings
* .NET: Delete previous Azure Search semantic memory component based on old
Azure Semantic Search w/ Lucene
### Motivation and Context
This PR ports the plan object model found in the .NET SK to Python. This
fundamentally changes the concept of a planner from a `semantic skill`
to a unique type of `SKFunction`. With this model, we can begin
implementing sequential planner and offer much better plan
customization.
**Notes:**
- This PR does not implement plan serialization/deserialization to/from
JSON. There is a huge existing effort to make the Python SK completely
Pydantic enabled, which will bring serialization and deserialization
capability with it.
- This PR does not include ports of `Action Planner` and `Sequential
Planner` to the Plan Object Model (POM). Following PRs will address
this.
- I also added some missing copywrite headers. A future PR will add
headers to any still missing them.
- added a VSCode launch configuration to run the current file with
pytest
- closes#1486
### Description
- Added file called `plan.py` which is a python port of the Plan.cs file
found in the .NET SK. Note that it does not contain serialization logic
at this time.
- Added new kernel error `FunctionInvokeError` - mirrors the error list
of the .NET SK
- Added tests for Plan creation
- Added tests for Plan execution - both sync and async
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [ ] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Shawn Callegari <[email protected]>
### Motivation and Context
The openai API expects that logit bias is always a set. Passing None
will not be successful.
### Description
- if logit-bias is None, pass {} to the openai request create call
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#dev-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Dmytro Struk <[email protected]>
Co-authored-by: Shawn Callegari <[email protected]>
### Motivation and Context
This PR addresses a number of failing integration tests. Chroma,
ConversationSummary Skill, and Pinecone.
- Chroma memory store tests broke due to a chroma dependency update
where the metadata checks for booleans (which are not allowed).
Previously, boolean metadata was not supported, but failed to validate
correctly. This PR gets around this by converting booleans to strings
when storing metadata, and converting string "True" to a boolean upon
metadata retrieval.
- ConversationSummarySkill added a callback parameter to all calls to
estimate the token count for the LLM call. This PR ensures that all
these calls have a default callback function to ensure backwards
compatibility with applications.
- This PR introduces retry logic to the Pinecone integration tests. The
tests are backed by the free tier of Pinecone which is a shared
instance. Traffic is high and unreliable causing the integration tests
to fail consistently. Retry logic addresses a lot of the unreliability,
but it is not a perfect fix. **We should address the backing service in
parallel to this change.**
- Add new dependencies to postgres to account for different install
needs for linux/macos/windows
### Description
- converts chroma boolean params to strings upon metadata save
- converts chroma boolean params from strings upon metadata retrieve
- all conversationsummaryskill methods have a default callback for token
count
- Adds a 1 second delay between each pinecone test
- Adds retry logic with progressive increases in waits between calls
should a pinecone request fail on known errors caused by service
availability
Action:
https://github.com/microsoft/semantic-kernel/actions/runs/5547243155
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [ ] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
---------
Co-authored-by: Dmytro Struk <[email protected]>
Co-authored-by: Shawn Callegari <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
1. Adds the Logit-Bias feature that is in the C# into Python.
2. It adds the token bias configuration to the text and chat completion
settings.
3. It contributes to flexibility in prompt engineering by allowing the
user to influence the likelihood of specified tokens appearing in the
text and chat completion.
4. Closes#1854
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
We added a dictionary field called token_selection_biases to the classes
CompleteRequestSettings and ChatRequestSettings to implement logit bias
and included an example to demonstrate functionality.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [X] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [x] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [x] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
---------
Co-authored-by: sneha <[email protected]>
Co-authored-by: Alisha Maddy <[email protected]>
Co-authored-by: Alisha Maddy <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Sneha <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
Brings parity to python implementation
2. What problem does it solve?
With the pgvector extension, Postgres is a viable option for the storing
of embeddings for memory in Semantic Kernel.
The C# implementation is already capable of this, but the feature has
yet to be added to python.
3. What scenario does it contribute to?
Additional options for memory storage
4. If it fixes an open issue, please link to the issue here.
-->
### Description
Implemention of the MemoryStore to use a Postgresql (with pgvector
extension) to resolve#1270
Taking a different approach from the .net implementation:
Collections as postgres schemas as opposed to collection being another
column in the table, willing to change to match but would love to have a
discussion comparing the options.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [x] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
---------
Co-authored-by: Dmytro Struk <[email protected]>
Co-authored-by: Shawn Callegari <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
addressing issue #1387
chunking text should allow use of an external tokenizer
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
added pass-through of an token counting function, defaulting to the
existing _token_counter() method
while fixing a type hint bug I got sucked into making a few changes to
clean up the code.
future work would be nice to add chunk overlap functionality similar to
langchain's [TextSplitter](
https://github.com/hwchase17/langchain/blob/a6ebffb69504576a805f3b9f09732ad344751b89/langchain/text_splitter.py#L58)
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [X] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
---------
Co-authored-by: Scott Graham <[email protected]>
Co-authored-by: Lee Miller <[email protected]>
Co-authored-by: Mark Karle <[email protected]>
Co-authored-by: Shawn Callegari <[email protected]>
### Motivation and Context
The pinecone integration tests can run locally, but not in CI. CI has no
way of picking up the pinecone environment.
Pinecone tests rely on the following environment variables in CI:
- Pinecone__ApiKey
- Pinecone__Environment
`get_pinecone_config` in `test_pinecone.py` currently only accounts for
Pinecone__ApiKey
### Description
This PR changes the method `get_pinecone_config` in `test_pinecone.py`
to return the pinecone environment from an environment variable if the
tests are being run in CI.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [ ] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
Co-authored-by: Devis Lucato <[email protected]>
### Motivation and Context
Adding a Pinecone Memory store using the Pinecone vector database client
for SK python
Add Pinecone vector db memory to SK Python
Memory storage for SK and vectors
### Description
- Using the existing pinecone python module/client
- Reviewing with console exceptions responses that come back from
pinecone, will update if any were omitted
- Currently investigating and would like feedback if I should respond to
throttling errors in a different manner
- Currently testing each API response, slight delay may occur due to
throttling of my starter account in Pinecone.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [X] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [ ] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [ ] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
---------
Co-authored-by: Lee Miller <[email protected]>
Co-authored-by: Casey Schadewitz <[email protected]>
Co-authored-by: cschadewitz <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
Co-authored-by: Dmytro Struk <[email protected]>
Co-authored-by: Shawn Callegari <[email protected]>
Rebrand end-to-end folder to kernel-syntax-examples because these aren't
actually tests for the repo. They are console apps.
Addresses #1397
Co-authored-by: Alex Chao <achao@achao>
Co-authored-by: Lee Miller <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required? Improved parity between .Net and Python
implementations
2. What problem does it solve? Allow for waiting in Python
implementation
3. What scenario does it contribute to? Parity
4. If it fixes an open issue, please link to the issue here.
-->
### Description
Pretty simple allows for waiting X seconds in a thread
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [x] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
### Motivation and Context
Replacing flake8 and isort with ruff in the pre-commit hook.
Adding "I" to ruff rules so that it checks for import errors.
Co-authored-by: Abby Harrison <[email protected]>
### Motivation and Context
The addition of the field `additional metadata` to `MemoryRecord` was
not propagated to the weaviate integration tests. This PR ensures that
the fields are properly filled and aligned.
### Description
- Added an entry for `"additional info"` to the memory record test
fixtures for weaviate.
