Commit Graph
51 Commits
Author SHA1 Message Date
Abby HarrisonandDevis Lucato db040e2e45 Python: Enable pinecone tests to run in CI (#1864)
### Motivation and Context
The pinecone integration tests can run locally, but not in CI. CI has no
way of picking up the pinecone environment.

Pinecone tests rely on the following environment variables in CI:

- Pinecone__ApiKey
- Pinecone__Environment

`get_pinecone_config` in `test_pinecone.py` currently only accounts for
Pinecone__ApiKey


### Description
This PR changes the method `get_pinecone_config` in `test_pinecone.py`
to return the pinecone environment from an environment variable if the
tests are being run in CI.


### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [ ] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄

Co-authored-by: Devis Lucato <[email protected]>
2023-07-05 23:13:46 +00:00
3f092e1487 Creating Pinecone memory store for Python (#769)
### Motivation and Context
Adding a Pinecone Memory store using the Pinecone vector database client
for SK python

  Add Pinecone vector db memory to SK Python

  Memory storage for SK and vectors

### Description
- Using the existing pinecone python module/client
- Reviewing with console exceptions responses that come back from
pinecone, will update if any were omitted
- Currently investigating and would like feedback if I should respond to
throttling errors in a different manner
- Currently testing each API response, slight delay may occur due to
throttling of my starter account in Pinecone.

### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [X] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [ ] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [ ] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Lee Miller <[email protected]>
Co-authored-by: Casey Schadewitz <[email protected]>
Co-authored-by: cschadewitz <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
Co-authored-by: Dmytro Struk <[email protected]>
Co-authored-by: Shawn Callegari <[email protected]>
2023-06-29 20:09:58 +00:00
6299040bd2 Rebrand end-to-end folder under python tests (#1503)
Rebrand end-to-end folder to kernel-syntax-examples because these aren't
actually tests for the repo. They are console apps.

Addresses #1397

Co-authored-by: Alex Chao <achao@achao>
Co-authored-by: Lee Miller <[email protected]>
2023-06-20 17:29:49 +00:00
cschadewitz 40517f2c03 Python wait core skill (#1373)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required? Improved parity between .Net and Python
implementations
2. What problem does it solve? Allow for waiting in Python
implementation
  3. What scenario does it contribute to? Parity
  4. If it fixes an open issue, please link to the issue here.
-->


### Description
Pretty simple allows for waiting X seconds in a thread


### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [x] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
2023-06-17 00:15:15 +00:00
Mark KarleandAbby Harrison 4994f37643 Adding import checking to ruff (#1294)
### Motivation and Context

Replacing flake8 and isort with ruff in the pre-commit hook.
Adding "I" to ruff rules so that it checks for import errors.

Co-authored-by: Abby Harrison <[email protected]>
2023-06-13 16:42:49 -07:00
Abby Harrison 67aa7325d8 Python: Fix weaviate integration tests (#1422)
### Motivation and Context
The addition of the field `additional metadata` to `MemoryRecord` was
not propagated to the weaviate integration tests. This PR ensures that
the fields are properly filled and aligned.


### Description
- Added an entry for `"additional info"` to the memory record test
fixtures for weaviate.
2023-06-12 09:51:15 -07:00
f60d7ba0d7 Fix functions_view.py (#1213)
### Motivation and Context
#1212 

### Description
Fixed a bug in the process of checking native function.


---------

Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
2023-06-09 14:21:34 -07:00
Abby Harrison b2e1548379 Python: Multiple results per prompt (incl. streaming) (#1316)
### Motivation and Context
To take advantage of APIs offered by most LLMs and to be in sync with
.NET SK, this PR introduces the ability to generate multiple text
completions or chat completions from a single prompt.


![MultiChatCompletionStreamMulti](https://github.com/microsoft/semantic-kernel/assets/54643756/7bec03ec-0be2-40b0-b938-6ff71beac209)


### Description
- Return type hint for `complete_async` and `complete_chat_async`
changed from `str -> Union[str, List[str]]`. The use of `Union` is the
proper way to indicate multiple return types prior to Python 3.10+. 3.10
supports the use of the `|` symbol, but since the Python SK is supported
on 3.8 and 3.9, I did not adopt the newer standard.
- `complete_async`, `complete_stream_async`, `complete_chat_async`, and
`complete_chat_stream_async` now support settings field
`number_of_responses` greater than 1. Previously only a value of 1 was
supported.
- **Note: hf_text_completion does not support streaming multiple
responses due to a limitation of TextIteratorStreamer. This feature
requires the ability to parse distinct responses from
TextIteratorStreamer.**
- Fixed a bug where `complete_async` was streaming single responses as
1D arrays, content now is simply a string.

### Example Usage
#### Setup
```
    kernel = sk.Kernel()

    # Configure OpenAI service
    api_key, org_id = sk.openai_settings_from_dot_env()
    oai_text_service = OpenAITextCompletion("text-davinci-003", api_key, org_id)
    oai_chat_service = OpenAIChatCompletion("gpt-3.5-turbo", api_key, org_id)

    # Configure Hugging Face service
    hf_text_service = HuggingFaceTextCompletion("gpt2", task="text-generation")
  
    # Configure Prompt
    prompt = "what is the purpose of a rubber duck?"

    # Configure Request Settings
    text_request_settings_multi = CompleteRequestSettings(
        max_tokens=100,
        temperature=0.7,
        top_p=1,
        frequency_penalty=0.5,
        presence_penalty=0.8,
        number_of_responses=4
    )

    chat_request_settings_multi = ChatRequestSettings(
        max_tokens=100,
        temperature=0.7,
        top_p=1,
        frequency_penalty=0.5,
        presence_penalty=0.8,
        number_of_responses=4
    )
```

#### Text Completion (Standard)
```
    texts = await oai_text_service.complete_async(prompt, text_request_settings_multi)
    i = 0
    for text in texts:
        print("Option " + str(i) + ": " + text)
        i += 1
```

#### Streaming Text Completion
```
    multi_stream = oai_text_service.complete_stream_async(prompt, text_request_settings_multi)
    texts = [''] * text_request_settings_multi.number_of_responses
    async for text in multi_stream:
        i = 0
        os.system('cls' if os.name == 'nt' else 'clear') # clear the screen for a better experience
        print("PROMPT: " + prompt)
        for option in text:
            texts[i] = texts[i] + option
            print("{0}: {1}".format(i, texts[i]))
            i += 1
```

#### Chat Completion (Standard)
```
    texts = await oai_chat_service.complete_chat_async([("user",prompt)], chat_request_settings_multi)
    i = 0
    for text in texts:
        print("Option " + str(i) + ": " + text)
        i += 1
```

#### Streaming Chat Completion
```
    multi_stream = oai_chat_service.complete_chat_stream_async([("user",prompt)], chat_request_settings_multi)
    texts = [''] * chat_request_settings_multi.number_of_responses
    async for text in multi_stream:
        i = 0
        os.system('cls' if os.name == 'nt' else 'clear') # clear the screen for a better experience
        print("PROMPT: " + prompt)
        for option in text:
            texts[i] = texts[i] + option
            print("{0}: {1}".format(i, texts[i]))
            i += 1
```

#### HuggingFace Standard Completion
```
    texts = await hf_text_service.complete_async(prompt, request_settings_multi)
    i = 0
    for text in texts:
        print("-----------------------------------")
        print("Option " + str(i) + ": " + text)
        i += 1
```
2023-06-09 14:13:45 -07:00
Abby Harrison 6cbea85fd8 Python: Add additional_metadata field to MemoryRecord and address TODOs in ChromaMemoryStore (#1323)
### Motivation and Context
Closes #1271 and addresses several TODOs in ChromaMemoryStore


### Description
- added field `additional_metadata` to `MemoryRecord` and all downstream
methods/classes (see `SemanticTextMemoryBase`, `SemanticTextMemory`,
`MemoryQueryResult`, and `ChromaMemoryStore`)
- addressed ChromaMemoryStore TODO 'what is key used for?' For chromadb,
key and id are effectively the same value, but the two fields are
differentiated for memory stores like Qdrant which require that key be a
uuid. I updated the logic so that this is more apparent. Note that this
results in no change in end-user functionality.
- Updated ChromaMemoryStore TODO 'get_async say embedding is optional
but Record constructor requires it.` MemoryRecord constructor no longer
requires an embedding. Logic is updated and a test added to show that
embedding retrieval is functionally optional.
2023-06-07 15:33:51 -07:00
Shukri 8947e6845c Weaviate: Fix to be compatible with python 3.8 (#1349)
use run_in_executor instead of to_thread to run the weaviate client ops
in async because to_thread is not available in python 3.8
2023-06-06 14:49:14 -07:00
18cd0cd042 Add support for weaviate as a memory store (Python) (#804)
Add support for weaviate as a memory store
see #803

Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
2023-06-05 09:37:41 -07:00
tim-gillandDevis Lucato d9028ee44f Add new relative time skills for ' ago' and 'last ' (#927)
### Motivation and Context

The change enables the planner to make use of time references like "last
wednesday" or "three days ago" which currently cause issues. This
enables better use of SequentialPlanner when servicing user intent.
Issue: https://github.com/microsoft/semantic-kernel/issues/922

### Description

This adds two new functions to the core TimeSkill class in both .net and
python.
1. DaysAgo(str days) - takes an argument representing a number of days
and offsets it from the current date to return the resultant date as the
answer.
2. LastMatchingDay(str dayName) - takes the name of a day of the week
and works backwards from yesterday to give it the first matching date of
that day (e.g. "last wednesday"). Currently it only works with English
day names, and this is reflected in the prompt description.

Simple unit tests are added for each function.

Co-authored-by: Devis Lucato <[email protected]>
2023-05-25 15:47:21 -07:00
Abby Harrison cfcb4636f9 Python/text completion streaming (#1115)
### Motivation and Context
This PR introduces streaming methods to TextCompletionBase and
ChatCompletionBase. With this pr, you can stream LLM output in the
following ways:

```
import semantic_kernel as sk
from semantic_kernel.connectors.ai import ChatCompletionClientBase, TextCompletionClientBase, ChatRequestSettings, CompleteRequestSettings
from semantic_kernel.connectors.ai.open_ai import AzureTextCompletion, AzureChatCompletion, OpenAITextCompletion, OpenAIChatCompletion
from semantic_kernel.connectors.ai.hugging_face import HuggingFaceTextCompletion

kernel = sk.Kernel()

# Configure Azure LLM service
deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()
text_service = AzureTextCompletion("text-davinci-003", endpoint, api_key)
chat_service = AzureChatCompletion("gpt-35-turbo", endpoint, api_key)

# Configure OpenAI service
api_key, org_id = sk.openai_settings_from_dot_env()
oai_text_service = OpenAITextCompletion("text-davinci-003", api_key, org_id)
oai_chat_service = OpenAIChatCompletion("gpt-3.5-turbo", api_key, org_id)

# Configure Hugging Face service
hf_text_service = HuggingFaceTextCompletion("gpt2", task="text-generation")

request_settings = CompleteRequestSettings(
    max_tokens=1000,
    temperature=0.7,
    top_p=1,
    frequency_penalty=0.5,
    presence_penalty=0.5
)

stream = oai_text_service.complete_stream_async("Write an essay on why AI is awesome:", request_settings)
async for text in stream:
    print(text, end = "") # end = "" to avoid newlines

chat_request_settings = ChatRequestSettings(
    max_tokens=1000,
    temperature=0.7,
    top_p=1,
    frequency_penalty=0.5,
    presence_penalty=0.5,
)

stream = oai_chat_service.complete_chat_stream_async([("user","Write an essay on why AI is awesome:")], chat_request_settings)
async for text in stream:
    print(text, end = "") # end = "" to avoid newlines

request_settings = CompleteRequestSettings(
    max_tokens=256,
    temperature=0.7,
    top_p=1,
    frequency_penalty=0.5,
    presence_penalty=0.5
)

stream = hf_text_service.complete_stream_async("Hi my name is ", request_settings)
async for text in stream:
    print(text, end = "") # end = "" to avoid newlines
```
**Out of Scope**: Improving the chat history interface with come in a
future PR

### Description
- added the method complete_stream_async to TextCompletionBase
- added the method complete_chat_stream_async to ChatCompletionBase
- Updated OpenAI and Hugging Face text completion service classes to
support new streaming methods
- Added new __init__.py to make importing service classes
2023-05-23 18:02:50 +00:00
Abby Harrison 00ff74e2e7 Feature python integration tests (#1134)
### Motivation and Context
Python integration tests were failing consistently for 2 reasons. First,
all ubuntu system tests were failing due to missing shared object files
related to cuda. Second, macOS + python 3.11 tests were failing due to
hsnwlib unable to install on native hardware.

See https://github.com/microsoft/semantic-kernel/pull/1066 for
description of changes to integration test code to condense test code,
make better use of pytest fixtures, and reduce the number of calls to
external APIs (OpenAI, AOAI)

### Description
- Integration tests were failing on Linux due to a regression in python
torch=2.0.1. Downgrading the version to 2.0.0 resolved these failures.
- For MacOS + 3.11 specifically, setting the environment variable
`HNSWLIB_NO_NATIVE=1` ensures that the hnswlib wheel can be built and
installed properly on M1 hardware.
- Python integration tests reliably and consistently passing:
https://github.com/microsoft/semantic-kernel/actions/runs/5035748596
2023-05-21 06:04:09 +00:00
Abby Harrison 493dfc1b61 Python: Merge Kernel, KernelBase, KernelExtensions (#1015)
### Motivation and Context
The separation of Kernel, KernelBase, and KernelExtensions is several
layers of unnecessary abstraction. This PR does not change any API or
kernel behavior - it just reduces the number of files and namespaces to
accomplish the same functionality.


### Description
- Removed the `semantic_kernel/kernel_extensions` namespace. Methods
that existed in this namespace now exist in kernel.py.
- Removed KernelBase - this abstract class implies that there are
multiple ways to instantiate SK. For Python, we expect only a single
type of Kernel to instantiated for all instances.
- Updated tests and imports to accommodate the above changes.
2023-05-17 15:38:02 -07:00
7066a4c037 python: add chroma memory store (#957)
Add support for Chroma https://docs.trychroma.com/

> Chroma is the open-source embedding database. Chroma makes it easy to
build LLM apps by making knowledge, facts, and skills pluggable for
LLMs.

### Motivation and Context

* #403 Support for Chroma embedding database
* #426 Python: feat: add chroma memory store
* #449 Python: feat: add chroma memory store

---------

Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
2023-05-17 20:32:08 +00:00
Devis Lucato b453936f81 Bump python release number - 0.2.8.dev (#1000)
* New version number for next SK Python release
* Change integration tests retry logic to progressively back off from 2 to 7 secs
2023-05-15 11:37:12 -07:00
Lee Miller 6d148f5205 Fix typo in poem generated by chatbot (#995)
This commit fixes a typo in the e2e_text_completion.py test file, where
the word "signularity" was misspelled as "singularity". This could cause
confusion or errors in the test output. The typo was corrected to
"singularity" to match the intended meaning of the poem. No other
changes were made to the test file or the functionality of the chatbot.
2023-05-15 16:11:54 +01:00
9fccbe4410 [Python] Conversation summary skill porting part I (#853)
### Motivation and Context
Porting conversation summary skill. Integration test is added as well.
Action item and topic will be ported later.

---------

Co-authored-by: Po-Wei Huang <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Abby Harrison <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
2023-05-13 01:02:31 +00:00
Devis Lucato fd1e014618 Spell checking automation + fix typos (#977)
Spell checking automation:
* Add typos checker to PR flow using
https://github.com/marketplace/actions/typos-action
* Fix existing typos
* Add config file, with exclusions and exceptions: `.github/_typos.toml`

The checker can be run locally, e.g.

* Install: `brew install typos-cli`
* Install: `conda install typos`

Run: `typos -c .github/_typos.toml`
2023-05-12 20:04:16 +00:00
Mark KarleandDevis Lucato 0237be49aa Fix python bug preventing use of Azure OpenAI and OpenAI at the same … (#931)
### Motivation and Context

Using both OpenAI and Azure OpenAI services would not work due to the
way they were configured. The `openai` module would be configured
globally with keys and endpoints for one type of service and then
overwritten when adding another service. Instead, we can configure the
settings when making the API calls.

Also, some integration tests are failing because of throttling by GPT4.

### Description

- Use the API settings when calling the openai API instead of
configuring gloabally
- Increased delay to 3 seconds for some integration tests to avoid rate
limit errors
- Relaxed the assertion strings for some integration tests
- Ran pre-commit linting
- Switch some tests to GPT Turbo 3.5

---------

Co-authored-by: Devis Lucato <[email protected]>
2023-05-11 10:32:21 -07:00
7cd0feacea [Python] Port MathSkill from C# to python3 (#841)
### Motivation and Context

Port MathSkill from C# to python3. This is essential for advanced skill
like plan


### Description

The PR includes two part. The first part is an implementation of
MathSkill. The implementation follows C# version.
It does several things:
1. Parse initial value
2. Get value from context
3. Sum or subtract

The unit test covers these scenarios:
1. Add/subtract value
2. Error should throw if string is not correct. Not able to parse.

Last part is the init.py and feature matrix.
Unit test pass locally.


Co-authored-by: Po-Wei Huang <[email protected]>
Co-authored-by: Mark Karle <[email protected]>
Co-authored-by: Devis Lucato <[email protected]>
2023-05-11 06:20:46 +00:00
Mark Karle 36d31aa3b0 Removing end-to-end tests that were moved to integration tests (#858)
These end-to-end tests were converted into integration tests and
deleted. These files were added back in
https://github.com/microsoft/semantic-kernel/pull/731/files when fixing
merge conflicts. They were originally removed in this commit
https://github.com/microsoft/semantic-kernel/commit/4e2d886eff4b4aedb15f5032ffb4f06ed9d3e8ab
2023-05-08 17:02:20 -07:00
Devis Lucato de3436d02f Python: fixes and release (#855)
* Fix lint errors
* Bump version number
* Add retry loop to integration tests
* Enable all integration tests: some tests will fail if OpenAI/Azure are
throttling.
2023-05-08 15:52:48 -07:00
Mark KarleandShawn Callegari 3e1d13e945 Merge KernelConfig with Kernel (#731)
### Motivation and Context
KernelConfig isn't needed as it's less pythonic and is not used
independently from Kernel. This moves all its methods and members into
Kernel.

### Description
- Removed KernelConfig
- Put KernelConfig's methods and variables into Kernel
- Ran unit and end-to-end tests
- Did not run notebooks because they are out of date with the current
repo.

---------
Co-authored-by: Shawn Callegari <[email protected]>
2023-05-05 22:27:49 -07:00
Abby Harrison 05c586b9d8 Create python-integration-tests.yml (#763)
### Motivation and Context
Run python integration tests on every push to main, and every 12 hours. 

These integration tests will run on every push to main as well as twice
a day. At the time of this PR, we are seeing ~3 PRs to the python SK per
day. Each Integration test run will make ~32 AI requests. For each PR,
12 runs will be made: 3 each for python 3.8, 3.9, 3.10, and 3.11 for a
total of ~400 calls per push to main.

We should re-evaluate the trigger on pushes to main if PRs to the python
SK significantly increase.

### Description
- add workflow definition for integration tests
- Example run:
https://github.com/microsoft/semantic-kernel/actions/runs/4874609026/jobs/8695771345?pr=763
- For push triggers, add path checks for dotnet/** or python/**
depending on the type of workflow
- pinned requirements.txt dependencies to the versions pinned in the
pyproject.toml file to ensure parity between pip package dependencies
and local code

**Note**: Integration tests are failing on windows with python 3.8 -
already captured a task to address the problem
2023-05-03 17:05:01 -07:00
Abby Harrison 04206c67dc Python: Update Notebooks to version 0.2.6.dev0; Fix bug in 0.2.5.dev0 (#761)
### Motivation and Context
New release version 0.2.6
Fix bug in 0.2.5.dev0 pip package
Update notebooks to the latest pip package (0.2.6)

### Description of Changes
- fixed a param name error in the creation of a reference memory_record
- renamed text_service -> text_completion_service (consistency with .NET)
- renamed text_embedding_service -> text_embedding_generation_service
(consistency with .NET)
- Updated notebooks accordingly
- bumped pip versions to 0.2.6.dev0
2023-05-01 17:16:01 -07:00
Abby Harrison 4e2d886eff Python: Proper separation of unit and integration tests (#732)
### Motivation and Context
This PR introduces the ability to run formal integration tests for
python using pytest - separated from unit tests for easier automation.

### Description
- moved `python/tests/end-to-end` directory to
`python/tests/integration`
- renamed integration tests to test_* pattern so pytest will detect them
- import `GenerationConfig` for hugging face completions - according to
logs, this is the recommended practice
2023-05-01 16:16:36 -07:00
Abby Harrison f6059cdf94 Python/Local Hugging Face Inference for Completions and Embeddings (#658)
### Motivation and Context
This PR introduces native python support for Hugging Face models that
can: complete text, generate new text, summarize, and that generate
embeddings. Currently only supports downloading the models locally from
the HF model hub. Future plans include supporting the HF inference API
as well.

### Description
- Added 2 services: `hf_text_completion` and `hf_text_embedding`
- `hf_text_completion` supports the following tasks: _text-generation_,
_text2text-generation_, and _summarization_
- `hf_text_embedding` supports any model supported by the
sentence-transformers pip package
- Added dependencies: pytorch, transformers, sentence-transformers to
`requirements.txt` and `poetry.lock`
- fixed typo: `get_embedding_service_service_id` ->
`get_embedding_service_id`
- Added a number of integration tests for supported HF models
2023-04-26 17:07:29 -07:00
Jerome Tremblay 971d2b668b Python: Text partitioning module (#450)
### Motivation and Context
Porting the text partitioning module to python
(Reopening of PR #427 )

### Description

- This is adding the Text partitioning module in
`semantic_kernel/semantic_functions/semantic_text_partitioner.py`
and the function_extention in
`semantic_kernel/semantic_functions/function_extension.py`
- Compared to the C# version the files were added directly into the
`semantic_functions` directory instead of the
`semantic_functions/partitioning` to not have too many nested
directories.
2023-04-26 22:50:24 +00:00
Abby Harrison 548429dd78 Python SK: Use ChatGPT for Semantic Functions (#643)
### Motivation and Context
Users want to use ChatGPT as the service for their semantic functions.
This feature exists with the C# SK. ChatGPT is perfectly capable of
doing this. Additionally added more end-to-end tests with the intention
of being able to run them as integration tests for the Python SK.

### Description
- `OpenAIChatCompletion` now additionally inherits from
`TextCompletionClientBase`. This extends to `AzureOpenAIChatCompletion`.
- If a chat service is also a `TextCompletionClientBase`, it will be
added to both `text_services` and `chat_services`.
- Created 4 end-to-end tests that target the combination of Azure,
OpenAI, text service, and chat service. Abstracted common behavior for
these tests.
- Added asserts to these end-to-end tests. These should be run as
integration tests, but that will be handled in a separate PR.
2023-04-25 11:42:46 -07:00
Abby Harrison c9b9220333 Refactor 'AI Backend' -> 'AI Service'; Move AI Namespace to semantic_kernel.connectors.ai (#595)
### Motivation and Context
This PR is a consistency update between the Python and .NET Semantic
Kernels.

**This PR does not contain any feature enhancements or improvements!**

### Description
- Replace terminology 'backend' with 'service'
- Move semantic_kernel.ai (semantic_kernel/ai) to
semantic_kernel.connectors.ai (semantic_kernel/connectors/ai)
2023-04-25 00:01:16 +00:00
Jerome TremblayandAbby Harrison cef3fc34e4 Python: Adding core HttpSkill (#504)
### Motivation and Context
HttpSkill  python version


### Description
- Porting the HttpSkill from C# to Python.
- Adding corresponding unit tests.

Co-authored-by: Abby Harrison <[email protected]>
2023-04-21 14:19:16 -07:00
JoowonandAbby Harrison 7d11dac74a Python: Add native skill import from directory (#506)
### Motivation and Context
#505 

### Description
for semantics skills, original code provide
`import_semantic_skill_from_directory`.
I hope native functions can be handled like that.
```
    kernel = sk.Kernel()

    # import skills
    skills_directory = os.path.join(
        os.path.dirname(__file__), "../..", "test_native_skills"
    )
    # path to skills directory
    skill_config_dict = kernel.import_native_skill_from_directory(
        skills_directory, "FileIOSkill"
    )
```

<img width="1277" alt="image"
src="https://user-images.githubusercontent.com/69464062/232785972-6a882609-b3cd-41d5-8ab7-0bb2fc0a1475.png">

Co-authored-by: Abby Harrison <[email protected]>
2023-04-21 12:57:28 -07:00
JoowonandAbby Harrison ba40c2b628 Python: linter fix (#578)
### Motivation and Context
lint fix for pre commit checks


Co-authored-by: Abby Harrison <[email protected]>
2023-04-21 12:47:37 -07:00
Abby Harrison e8c299a30d Python: Use overloads for run_async method (#511)
### Motivation and Context
To make the Python SK more pythonic, combined all kernel.run_* methods
into a single method with optional parameters.

### Description
- Removed `kernel.run_on_vars_async()`
- Removed `kernel.run_on_str_async()`
- Added `SKContext`, `ContextVariables`, and string params to
`kernel.run_async()`
- Moved `run_on_vars_async()` logic to `run_async()` method
- Arranged method params so that required params come before optional
params.
- Updated tests and notebooks where applicable.
2023-04-19 19:18:10 -07:00
Abby HarrisonandDevis Lucato d9323b573f Remove kernel builder pattern for Python: sk.kernel_create() -> sk.Kernel() (#454)
### Motivation and Context

This PR makes creating a kernel instance more pythonic. The builder
pattern is removed and the kernel is constructed using Kernel(...).

Work originally started with #137

### Description

- Removed kernel_builder.py.
- Instance creation pattern `kernel = sk.create_kernel() -> kernel =
sk.Kernel()`
- For notebook 6, there is no need to create a separate kernel for the
GitHub files. Changed to use a separate collection.


Co-authored-by: Devis Lucato <[email protected]>
2023-04-17 11:07:46 -07:00
Devis Lucato e2e5e2dad3 Python: docs update + ruff config + fix tests 2023-04-13 16:12:36 -07:00
Jordan Henkel 7d9c40f08a Python: support Azure AD auth (#340)
AAD tokens offer greater authentication security and is used by several products.

Add support for Azure Active Directory auth for the `Azure*` backends.
2023-04-13 16:12:36 -07:00
Jordan Henkel 407fb36e59 Python: cleanup/sync/fixes (for examples, docs, tests, notebooks) (#317)
### Motivation and Context

The `python-preview` branch is under active development and many
docs/tests/examples are getting a bit out of sync. There are also a few
small cleanup chores/refactors to make _usage_ easier (e.g., to make
examples cleaner). I've grouped these small fixes (plus a few bug fixes)
into this "cleanup" PR.

### Description

This PR does the following:

1. Updates the out-of-date example in `README.md`
2. Fixes all 5 of the notebooks we support right now (1,2,3,4, and 6;
notebook 5 is planner, I've blanked out most of the code there so we
don't confuse people, especially as the planner API is changing)
3. Fixes a bug w/ `stop_sequences` in `(Azure)OpenAITextCompletion`
4. Fixes a bug introduced in our upgrades to the embeddings cosine-sim
check (we should add more thorough tests here)
5. Cleans up tests
6. Ensures end-to-end tests are all runnable 
7. Fixes up the `kernel_extensions` to work more naturally (so that end
users can use them directly on the `kernel` object and get good
type-hinting)
2023-04-13 16:12:36 -07:00
Jordan Henkel cf7a4a731d Python: reduce the number of config classes (#296)
### Motivation and Context

Right now, there are lots of Config classes in SK. This doesn't feel
very "pythonic" and can be a bit confusing at times. To reduce the
amount of indirection, this PR removes several config classes from the
Python port.

NOTE: this PR is also critical preparation for a large change to re-sync
with how C# SK handles multi-modality. I have changes ready for adding a
`./connectors` dir and syncing with the Text/Chat/Image/Embedding
support, but there's a few "prep" PRs I need to get through first.

### Description

This PR makes the following changes:

1. We remove the `(Azure)OpenAIConfig` classes (simplifying the code
related to creating/managing backends)
2. We remove the `./configuration` sub-directory and move the
`KernelConfig` class to the module's root dir (this mirrors the C#
version)
3. We re-tool the `KernelConfig` class to behave similarly to the
current C# version (again, this is prep for later changes RE
multi-modality)
4. We make corresponding updates across the code/tests now that we've
removed some config classes

In future PRs, it'd be great to also simplify:

1. `(Chat|Completion)RequestSettings` 
2. `(ReadOnly)SkillsCollection` and related classes
3. Take a hard look at `ContextVariables` and `SKContext` (maybe
`ContextVariables` is just a `dict` in SK python)
2023-04-13 16:12:36 -07:00
vivekpandit-msft 12d28ca9f0 added logger warning and error for cosine similarity computation for … (#203)
Added logger warning and error for cosine similarity computation for
zero vectors
2023-04-13 16:12:36 -07:00
Turner Strayhorn db1fae3f18 Fix import_semantic_skill_from_directory (#236)
### Motivation and Context
This PR fixes #235 and adds a test for the functionality. This is ahead
of the planning skill.

### Description
1. Fix to fix the typo in the `from_dict()` method in
`prompt_template_config.py`
2. Add assignment back to skill config so results from JSON aren't
discarded in `import_semantic_skill_from_directory.py`
3. Add tests
2023-04-13 16:12:36 -07:00
Jordan Henkel 1d1e71971e Python: normalize ./tests directory structure (#270)
### Motivation and Context

We recently merged an upgrade to the `PromptTemplateEngine`, let's make
the rest of the tests consistent with the directory structure used in
that PR.


### Description

This PR does three (small) things:

1. Makes the `./tests` directory have consistent structure
2. Re-names some of the "tests" that were at the top-level of the
`./tests` dir to `./tests/end-to-end` (which better describes their
purpose: things like `basics.py` and `memory.py` are end-to-end examples
of using SK and ways to verify nothing is horribly broken)
3. Applies `isort` (which we plan to add to our linting workflow here on
GitHub soon)
2023-04-13 16:12:36 -07:00
Jordan Henkel 016cd3334f Python: Upgrade PromptTemplateEngine (bring back to parity w/ C# version) (#200)
### Motivation and Context

The C# Semantic Kernel has recently undergone an upgrade to its
`PromptTemplateEngine`. This brings Python back in line with the
semantics of Prompt Templates in C#.

### Description

Here, unlike the original port, I've tried to make things more
pythonic/idiomatic (instead of trying to directly mirror the C#
codebase). I've also brought over the corresponding unit tests (and
added some of my own as I was building/validating).
2023-04-13 16:12:36 -07:00
Jerome Tremblay 3e3745a8d7 python: Porting TimeSkill (#209)
### Motivation and Context
Port of the core TimeSkill

### Description
This PR adds the core TimeSkill with unit tests.

```
kernel = sk.create_kernel()
kernel.import_skill(TimeSkill(), "time")
```

```
sk_prompt = """
{{time.now}}
"""
```
2023-04-13 16:12:36 -07:00
Jordan Henkel 0ce2acbf6d ChatGPT API Python Support (#121)
This PR provides a path to using the ChatGPT API in the Python Preview
of Semantic Kernel. In addition, this provides `Azure*` versions of many
existing models (so Python users can leverage Azure OpenAI).

I think that, in general, it may be worth considering how best to work
with models that have different modalities: now we have text
completions, embeddings, chat completions (and I'd imagine images/etc.
may be nice to support someday too).

Regardless, this PR provides a fast path to using the exciting new Chat
APIs from OpenAI with SK!

See the new `python/tests/chat_gpt_api.py` for a usage example.
2023-04-13 16:12:36 -07:00
Jerome Tremblay 5ea0242c58 python: core skill FileIOSkill (#187)
### Motivation and Context
Porting the FileIOSkill to python.

### Description
This is the port of the FileIOSkill to python with unit tests.  
```
kernel = sk.create_kernel()
kernel.import_skill(FileIOSkill(), "file")
context = kernel.create_new_context()
context["path"] = "test_file_io_skill.txt"
context["content"] = "Hello, world!"
```

Using the same function names than the C# version 

```
{{file.readAsync $path}}

{{file.writeAsync}}
```
modification to the depencies :
- Adding the package : aiofiles 
- Adding the dev package : pytest-asyncio
2023-04-13 16:12:36 -07:00
Jerome Tremblay 043890e2c6 Python: Github action fixes - formatting (#151)
### Motivation and Context
Fixes formatting issue to make Lint github workflow passes.

### Description
Contains only reformatting changes.
2023-04-13 16:12:36 -07:00
Jerome Tremblay 9c16007985 Python : Simplification for the SK decorators and core TextSkill (#144)
### Motivation and Context
This PR simplifies `@sk_*`  decorators while porting the core TextSkill

### Description
This PR is a first step at adapting the python codebase to be more
*pythonic*, it contains the following modifications :

1. Merged the decorators `@sk_function_context_parameter`,
`@sk_function_input` , `@sk_function_name` with `@sk_function`.
The decorators were replaced with new kwargs on `sk_function` : `name`,
`input_description`, `input_default_value`
2. The `name` kwarg is optional, the name of the method will be used if
none is provided.
3. Ported core skill - TextSkill
4. Added some pytest unit test for  SK decorators and TextSkill
5. Changed how skills are imported in the kernel by using instance of
the class, not relying on static method anymore for the discovery.
e.g.
```
kernel.import_skill(TextSkill())
```
2023-04-13 16:12:36 -07:00