* Fix lint errors
* Bump version number
* Add retry loop to integration tests
* Enable all integration tests: some tests will fail if OpenAI/Azure are
throttling.
### Motivation and Context
KernelConfig isn't needed as it's less pythonic and is not used
independently from Kernel. This moves all its methods and members into
Kernel.
### Description
- Removed KernelConfig
- Put KernelConfig's methods and variables into Kernel
- Ran unit and end-to-end tests
- Did not run notebooks because they are out of date with the current
repo.
---------
Co-authored-by: Shawn Callegari <[email protected]>
### Motivation and Context
Run python integration tests on every push to main, and every 12 hours.
These integration tests will run on every push to main as well as twice
a day. At the time of this PR, we are seeing ~3 PRs to the python SK per
day. Each Integration test run will make ~32 AI requests. For each PR,
12 runs will be made: 3 each for python 3.8, 3.9, 3.10, and 3.11 for a
total of ~400 calls per push to main.
We should re-evaluate the trigger on pushes to main if PRs to the python
SK significantly increase.
### Description
- add workflow definition for integration tests
- Example run:
https://github.com/microsoft/semantic-kernel/actions/runs/4874609026/jobs/8695771345?pr=763
- For push triggers, add path checks for dotnet/** or python/**
depending on the type of workflow
- pinned requirements.txt dependencies to the versions pinned in the
pyproject.toml file to ensure parity between pip package dependencies
and local code
**Note**: Integration tests are failing on windows with python 3.8 -
already captured a task to address the problem
### Motivation and Context
New release version 0.2.6
Fix bug in 0.2.5.dev0 pip package
Update notebooks to the latest pip package (0.2.6)
### Description of Changes
- fixed a param name error in the creation of a reference memory_record
- renamed text_service -> text_completion_service (consistency with .NET)
- renamed text_embedding_service -> text_embedding_generation_service
(consistency with .NET)
- Updated notebooks accordingly
- bumped pip versions to 0.2.6.dev0
### Motivation and Context
This PR introduces the ability to run formal integration tests for
python using pytest - separated from unit tests for easier automation.
### Description
- moved `python/tests/end-to-end` directory to
`python/tests/integration`
- renamed integration tests to test_* pattern so pytest will detect them
- import `GenerationConfig` for hugging face completions - according to
logs, this is the recommended practice
### Motivation and Context
This PR introduces native python support for Hugging Face models that
can: complete text, generate new text, summarize, and that generate
embeddings. Currently only supports downloading the models locally from
the HF model hub. Future plans include supporting the HF inference API
as well.
### Description
- Added 2 services: `hf_text_completion` and `hf_text_embedding`
- `hf_text_completion` supports the following tasks: _text-generation_,
_text2text-generation_, and _summarization_
- `hf_text_embedding` supports any model supported by the
sentence-transformers pip package
- Added dependencies: pytorch, transformers, sentence-transformers to
`requirements.txt` and `poetry.lock`
- fixed typo: `get_embedding_service_service_id` ->
`get_embedding_service_id`
- Added a number of integration tests for supported HF models
### Motivation and Context
Porting the text partitioning module to python
(Reopening of PR #427 )
### Description
- This is adding the Text partitioning module in
`semantic_kernel/semantic_functions/semantic_text_partitioner.py`
and the function_extention in
`semantic_kernel/semantic_functions/function_extension.py`
- Compared to the C# version the files were added directly into the
`semantic_functions` directory instead of the
`semantic_functions/partitioning` to not have too many nested
directories.
### Motivation and Context
Users want to use ChatGPT as the service for their semantic functions.
This feature exists with the C# SK. ChatGPT is perfectly capable of
doing this. Additionally added more end-to-end tests with the intention
of being able to run them as integration tests for the Python SK.
### Description
- `OpenAIChatCompletion` now additionally inherits from
`TextCompletionClientBase`. This extends to `AzureOpenAIChatCompletion`.
- If a chat service is also a `TextCompletionClientBase`, it will be
added to both `text_services` and `chat_services`.
- Created 4 end-to-end tests that target the combination of Azure,
OpenAI, text service, and chat service. Abstracted common behavior for
these tests.
- Added asserts to these end-to-end tests. These should be run as
integration tests, but that will be handled in a separate PR.
### Motivation and Context
This PR is a consistency update between the Python and .NET Semantic
Kernels.
**This PR does not contain any feature enhancements or improvements!**
### Description
- Replace terminology 'backend' with 'service'
- Move semantic_kernel.ai (semantic_kernel/ai) to
semantic_kernel.connectors.ai (semantic_kernel/connectors/ai)
### Motivation and Context
HttpSkill python version
### Description
- Porting the HttpSkill from C# to Python.
- Adding corresponding unit tests.
Co-authored-by: Abby Harrison <[email protected]>
### Motivation and Context
To make the Python SK more pythonic, combined all kernel.run_* methods
into a single method with optional parameters.
### Description
- Removed `kernel.run_on_vars_async()`
- Removed `kernel.run_on_str_async()`
- Added `SKContext`, `ContextVariables`, and string params to
`kernel.run_async()`
- Moved `run_on_vars_async()` logic to `run_async()` method
- Arranged method params so that required params come before optional
params.
- Updated tests and notebooks where applicable.
### Motivation and Context
This PR makes creating a kernel instance more pythonic. The builder
pattern is removed and the kernel is constructed using Kernel(...).
Work originally started with #137
### Description
- Removed kernel_builder.py.
- Instance creation pattern `kernel = sk.create_kernel() -> kernel =
sk.Kernel()`
- For notebook 6, there is no need to create a separate kernel for the
GitHub files. Changed to use a separate collection.
Co-authored-by: Devis Lucato <[email protected]>
AAD tokens offer greater authentication security and is used by several products.
Add support for Azure Active Directory auth for the `Azure*` backends.
### Motivation and Context
The `python-preview` branch is under active development and many
docs/tests/examples are getting a bit out of sync. There are also a few
small cleanup chores/refactors to make _usage_ easier (e.g., to make
examples cleaner). I've grouped these small fixes (plus a few bug fixes)
into this "cleanup" PR.
### Description
This PR does the following:
1. Updates the out-of-date example in `README.md`
2. Fixes all 5 of the notebooks we support right now (1,2,3,4, and 6;
notebook 5 is planner, I've blanked out most of the code there so we
don't confuse people, especially as the planner API is changing)
3. Fixes a bug w/ `stop_sequences` in `(Azure)OpenAITextCompletion`
4. Fixes a bug introduced in our upgrades to the embeddings cosine-sim
check (we should add more thorough tests here)
5. Cleans up tests
6. Ensures end-to-end tests are all runnable
7. Fixes up the `kernel_extensions` to work more naturally (so that end
users can use them directly on the `kernel` object and get good
type-hinting)
### Motivation and Context
Right now, there are lots of Config classes in SK. This doesn't feel
very "pythonic" and can be a bit confusing at times. To reduce the
amount of indirection, this PR removes several config classes from the
Python port.
NOTE: this PR is also critical preparation for a large change to re-sync
with how C# SK handles multi-modality. I have changes ready for adding a
`./connectors` dir and syncing with the Text/Chat/Image/Embedding
support, but there's a few "prep" PRs I need to get through first.
### Description
This PR makes the following changes:
1. We remove the `(Azure)OpenAIConfig` classes (simplifying the code
related to creating/managing backends)
2. We remove the `./configuration` sub-directory and move the
`KernelConfig` class to the module's root dir (this mirrors the C#
version)
3. We re-tool the `KernelConfig` class to behave similarly to the
current C# version (again, this is prep for later changes RE
multi-modality)
4. We make corresponding updates across the code/tests now that we've
removed some config classes
In future PRs, it'd be great to also simplify:
1. `(Chat|Completion)RequestSettings`
2. `(ReadOnly)SkillsCollection` and related classes
3. Take a hard look at `ContextVariables` and `SKContext` (maybe
`ContextVariables` is just a `dict` in SK python)
### Motivation and Context
This PR fixes#235 and adds a test for the functionality. This is ahead
of the planning skill.
### Description
1. Fix to fix the typo in the `from_dict()` method in
`prompt_template_config.py`
2. Add assignment back to skill config so results from JSON aren't
discarded in `import_semantic_skill_from_directory.py`
3. Add tests
### Motivation and Context
We recently merged an upgrade to the `PromptTemplateEngine`, let's make
the rest of the tests consistent with the directory structure used in
that PR.
### Description
This PR does three (small) things:
1. Makes the `./tests` directory have consistent structure
2. Re-names some of the "tests" that were at the top-level of the
`./tests` dir to `./tests/end-to-end` (which better describes their
purpose: things like `basics.py` and `memory.py` are end-to-end examples
of using SK and ways to verify nothing is horribly broken)
3. Applies `isort` (which we plan to add to our linting workflow here on
GitHub soon)
### Motivation and Context
The C# Semantic Kernel has recently undergone an upgrade to its
`PromptTemplateEngine`. This brings Python back in line with the
semantics of Prompt Templates in C#.
### Description
Here, unlike the original port, I've tried to make things more
pythonic/idiomatic (instead of trying to directly mirror the C#
codebase). I've also brought over the corresponding unit tests (and
added some of my own as I was building/validating).
### Motivation and Context
Port of the core TimeSkill
### Description
This PR adds the core TimeSkill with unit tests.
```
kernel = sk.create_kernel()
kernel.import_skill(TimeSkill(), "time")
```
```
sk_prompt = """
{{time.now}}
"""
```
This PR provides a path to using the ChatGPT API in the Python Preview
of Semantic Kernel. In addition, this provides `Azure*` versions of many
existing models (so Python users can leverage Azure OpenAI).
I think that, in general, it may be worth considering how best to work
with models that have different modalities: now we have text
completions, embeddings, chat completions (and I'd imagine images/etc.
may be nice to support someday too).
Regardless, this PR provides a fast path to using the exciting new Chat
APIs from OpenAI with SK!
See the new `python/tests/chat_gpt_api.py` for a usage example.
### Motivation and Context
Porting the FileIOSkill to python.
### Description
This is the port of the FileIOSkill to python with unit tests.
```
kernel = sk.create_kernel()
kernel.import_skill(FileIOSkill(), "file")
context = kernel.create_new_context()
context["path"] = "test_file_io_skill.txt"
context["content"] = "Hello, world!"
```
Using the same function names than the C# version
```
{{file.readAsync $path}}
{{file.writeAsync}}
```
modification to the depencies :
- Adding the package : aiofiles
- Adding the dev package : pytest-asyncio
### Motivation and Context
This PR simplifies `@sk_*` decorators while porting the core TextSkill
### Description
This PR is a first step at adapting the python codebase to be more
*pythonic*, it contains the following modifications :
1. Merged the decorators `@sk_function_context_parameter`,
`@sk_function_input` , `@sk_function_name` with `@sk_function`.
The decorators were replaced with new kwargs on `sk_function` : `name`,
`input_description`, `input_default_value`
2. The `name` kwarg is optional, the name of the method will be used if
none is provided.
3. Ported core skill - TextSkill
4. Added some pytest unit test for SK decorators and TextSkill
5. Changed how skills are imported in the kernel by using instance of
the class, not relying on static method anymore for the discovery.
e.g.
```
kernel.import_skill(TextSkill())
```
Mark Karle
