Commit Graph
20 Commits
Author SHA1 Message Date
Abby Harrison 00ff74e2e7 Feature python integration tests (#1134)
### Motivation and Context
Python integration tests were failing consistently for 2 reasons. First,
all ubuntu system tests were failing due to missing shared object files
related to cuda. Second, macOS + python 3.11 tests were failing due to
hsnwlib unable to install on native hardware.

See https://github.com/microsoft/semantic-kernel/pull/1066 for
description of changes to integration test code to condense test code,
make better use of pytest fixtures, and reduce the number of calls to
external APIs (OpenAI, AOAI)

### Description
- Integration tests were failing on Linux due to a regression in python
torch=2.0.1. Downgrading the version to 2.0.0 resolved these failures.
- For MacOS + 3.11 specifically, setting the environment variable
`HNSWLIB_NO_NATIVE=1` ensures that the hnswlib wheel can be built and
installed properly on M1 hardware.
- Python integration tests reliably and consistently passing:
https://github.com/microsoft/semantic-kernel/actions/runs/5035748596
2023-05-21 06:04:09 +00:00
Abby Harrison 493dfc1b61 Python: Merge Kernel, KernelBase, KernelExtensions (#1015)
### Motivation and Context
The separation of Kernel, KernelBase, and KernelExtensions is several
layers of unnecessary abstraction. This PR does not change any API or
kernel behavior - it just reduces the number of files and namespaces to
accomplish the same functionality.


### Description
- Removed the `semantic_kernel/kernel_extensions` namespace. Methods
that existed in this namespace now exist in kernel.py.
- Removed KernelBase - this abstract class implies that there are
multiple ways to instantiate SK. For Python, we expect only a single
type of Kernel to instantiated for all instances.
- Updated tests and imports to accommodate the above changes.
2023-05-17 15:38:02 -07:00
Devis Lucato fd1e014618 Spell checking automation + fix typos (#977)
Spell checking automation:
* Add typos checker to PR flow using
https://github.com/marketplace/actions/typos-action
* Fix existing typos
* Add config file, with exclusions and exceptions: `.github/_typos.toml`

The checker can be run locally, e.g.

* Install: `brew install typos-cli`
* Install: `conda install typos`

Run: `typos -c .github/_typos.toml`
2023-05-12 20:04:16 +00:00
Mark KarleandDevis Lucato 0237be49aa Fix python bug preventing use of Azure OpenAI and OpenAI at the same … (#931)
### Motivation and Context

Using both OpenAI and Azure OpenAI services would not work due to the
way they were configured. The `openai` module would be configured
globally with keys and endpoints for one type of service and then
overwritten when adding another service. Instead, we can configure the
settings when making the API calls.

Also, some integration tests are failing because of throttling by GPT4.

### Description

- Use the API settings when calling the openai API instead of
configuring gloabally
- Increased delay to 3 seconds for some integration tests to avoid rate
limit errors
- Relaxed the assertion strings for some integration tests
- Ran pre-commit linting
- Switch some tests to GPT Turbo 3.5

---------

Co-authored-by: Devis Lucato <[email protected]>
2023-05-11 10:32:21 -07:00
7cd0feacea [Python] Port MathSkill from C# to python3 (#841)
### Motivation and Context

Port MathSkill from C# to python3. This is essential for advanced skill
like plan


### Description

The PR includes two part. The first part is an implementation of
MathSkill. The implementation follows C# version.
It does several things:
1. Parse initial value
2. Get value from context
3. Sum or subtract

The unit test covers these scenarios:
1. Add/subtract value
2. Error should throw if string is not correct. Not able to parse.

Last part is the init.py and feature matrix.
Unit test pass locally.


Co-authored-by: Po-Wei Huang <[email protected]>
Co-authored-by: Mark Karle <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
2023-05-11 06:20:46 +00:00
Mark KarleandShawn Callegari 3e1d13e945 Merge KernelConfig with Kernel (#731)
### Motivation and Context
KernelConfig isn't needed as it's less pythonic and is not used
independently from Kernel. This moves all its methods and members into
Kernel.

### Description
- Removed KernelConfig
- Put KernelConfig's methods and variables into Kernel
- Ran unit and end-to-end tests
- Did not run notebooks because they are out of date with the current
repo.

---------
Co-authored-by: Shawn Callegari <[email protected]>
2023-05-05 22:27:49 -07:00
Abby Harrison 04206c67dc Python: Update Notebooks to version 0.2.6.dev0; Fix bug in 0.2.5.dev0 (#761)
### Motivation and Context
New release version 0.2.6
Fix bug in 0.2.5.dev0 pip package
Update notebooks to the latest pip package (0.2.6)

### Description of Changes
- fixed a param name error in the creation of a reference memory_record
- renamed text_service -> text_completion_service (consistency with .NET)
- renamed text_embedding_service -> text_embedding_generation_service
(consistency with .NET)
- Updated notebooks accordingly
- bumped pip versions to 0.2.6.dev0
2023-05-01 17:16:01 -07:00
Jerome Tremblay 971d2b668b Python: Text partitioning module (#450)
### Motivation and Context
Porting the text partitioning module to python
(Reopening of PR #427 )

### Description

- This is adding the Text partitioning module in
`semantic_kernel/semantic_functions/semantic_text_partitioner.py`
and the function_extention in
`semantic_kernel/semantic_functions/function_extension.py`
- Compared to the C# version the files were added directly into the
`semantic_functions` directory instead of the
`semantic_functions/partitioning` to not have too many nested
directories.
2023-04-26 22:50:24 +00:00
Abby Harrison c9b9220333 Refactor 'AI Backend' -> 'AI Service'; Move AI Namespace to semantic_kernel.connectors.ai (#595)
### Motivation and Context
This PR is a consistency update between the Python and .NET Semantic
Kernels.

**This PR does not contain any feature enhancements or improvements!**

### Description
- Replace terminology 'backend' with 'service'
- Move semantic_kernel.ai (semantic_kernel/ai) to
semantic_kernel.connectors.ai (semantic_kernel/connectors/ai)
2023-04-25 00:01:16 +00:00
Jerome TremblayandAbby Harrison cef3fc34e4 Python: Adding core HttpSkill (#504)
### Motivation and Context
HttpSkill  python version


### Description
- Porting the HttpSkill from C# to Python.
- Adding corresponding unit tests.

Co-authored-by: Abby Harrison <[email protected]>
2023-04-21 14:19:16 -07:00
JoowonandAbby Harrison 7d11dac74a Python: Add native skill import from directory (#506)
### Motivation and Context
#505 

### Description
for semantics skills, original code provide
`import_semantic_skill_from_directory`.
I hope native functions can be handled like that.
```
    kernel = sk.Kernel()

    # import skills
    skills_directory = os.path.join(
        os.path.dirname(__file__), "../..", "test_native_skills"
    )
    # path to skills directory
    skill_config_dict = kernel.import_native_skill_from_directory(
        skills_directory, "FileIOSkill"
    )
```

<img width="1277" alt="image"
src="https://user-images.githubusercontent.com/69464062/232785972-6a882609-b3cd-41d5-8ab7-0bb2fc0a1475.png">

Co-authored-by: Abby Harrison <[email protected]>
2023-04-21 12:57:28 -07:00
JoowonandAbby Harrison ba40c2b628 Python: linter fix (#578)
### Motivation and Context
lint fix for pre commit checks


Co-authored-by: Abby Harrison <[email protected]>
2023-04-21 12:47:37 -07:00
Abby Harrison e8c299a30d Python: Use overloads for run_async method (#511)
### Motivation and Context
To make the Python SK more pythonic, combined all kernel.run_* methods
into a single method with optional parameters.

### Description
- Removed `kernel.run_on_vars_async()`
- Removed `kernel.run_on_str_async()`
- Added `SKContext`, `ContextVariables`, and string params to
`kernel.run_async()`
- Moved `run_on_vars_async()` logic to `run_async()` method
- Arranged method params so that required params come before optional
params.
- Updated tests and notebooks where applicable.
2023-04-19 19:18:10 -07:00
Abby HarrisonandDevis Lucato d9323b573f Remove kernel builder pattern for Python: sk.kernel_create() -> sk.Kernel() (#454)
### Motivation and Context

This PR makes creating a kernel instance more pythonic. The builder
pattern is removed and the kernel is constructed using Kernel(...).

Work originally started with #137

### Description

- Removed kernel_builder.py.
- Instance creation pattern `kernel = sk.create_kernel() -> kernel =
sk.Kernel()`
- For notebook 6, there is no need to create a separate kernel for the
GitHub files. Changed to use a separate collection.


Co-authored-by: Devis Lucato <[email protected]>
2023-04-17 11:07:46 -07:00
Devis Lucato e2e5e2dad3 Python: docs update + ruff config + fix tests 2023-04-13 16:12:36 -07:00
Jordan Henkel 7d9c40f08a Python: support Azure AD auth (#340)
AAD tokens offer greater authentication security and is used by several products.

Add support for Azure Active Directory auth for the `Azure*` backends.
2023-04-13 16:12:36 -07:00
Jordan Henkel 407fb36e59 Python: cleanup/sync/fixes (for examples, docs, tests, notebooks) (#317)
### Motivation and Context

The `python-preview` branch is under active development and many
docs/tests/examples are getting a bit out of sync. There are also a few
small cleanup chores/refactors to make _usage_ easier (e.g., to make
examples cleaner). I've grouped these small fixes (plus a few bug fixes)
into this "cleanup" PR.

### Description

This PR does the following:

1. Updates the out-of-date example in `README.md`
2. Fixes all 5 of the notebooks we support right now (1,2,3,4, and 6;
notebook 5 is planner, I've blanked out most of the code there so we
don't confuse people, especially as the planner API is changing)
3. Fixes a bug w/ `stop_sequences` in `(Azure)OpenAITextCompletion`
4. Fixes a bug introduced in our upgrades to the embeddings cosine-sim
check (we should add more thorough tests here)
5. Cleans up tests
6. Ensures end-to-end tests are all runnable 
7. Fixes up the `kernel_extensions` to work more naturally (so that end
users can use them directly on the `kernel` object and get good
type-hinting)
2023-04-13 16:12:36 -07:00
vivekpandit-msft 12d28ca9f0 added logger warning and error for cosine similarity computation for … (#203)
Added logger warning and error for cosine similarity computation for
zero vectors
2023-04-13 16:12:36 -07:00
Jordan Henkel 1d1e71971e Python: normalize ./tests directory structure (#270)
### Motivation and Context

We recently merged an upgrade to the `PromptTemplateEngine`, let's make
the rest of the tests consistent with the directory structure used in
that PR.


### Description

This PR does three (small) things:

1. Makes the `./tests` directory have consistent structure
2. Re-names some of the "tests" that were at the top-level of the
`./tests` dir to `./tests/end-to-end` (which better describes their
purpose: things like `basics.py` and `memory.py` are end-to-end examples
of using SK and ways to verify nothing is horribly broken)
3. Applies `isort` (which we plan to add to our linting workflow here on
GitHub soon)
2023-04-13 16:12:36 -07:00
Jordan Henkel 016cd3334f Python: Upgrade PromptTemplateEngine (bring back to parity w/ C# version) (#200)
### Motivation and Context

The C# Semantic Kernel has recently undergone an upgrade to its
`PromptTemplateEngine`. This brings Python back in line with the
semantics of Prompt Templates in C#.

### Description

Here, unlike the original port, I've tried to make things more
pythonic/idiomatic (instead of trying to directly mirror the C#
codebase). I've also brought over the corresponding unit tests (and
added some of my own as I was building/validating).
2023-04-13 16:12:36 -07:00