### Motivation and Context
Reduce the responsibility surface of Semantic Kernel regarding
Resiliency implementations, keeping only the abstractions that can be
delegated and handling by well know resiliency libraries and custom
implementations.
Extract existing logic from Kernel Core to an extension project that can
handle retry logic similar to the prior using Polly Policies.
Resolves#2271Closes#2271⚠️ Small breaking changes.
### Description
This change removes and deprecates the usage of Retry implementations in
our Resiliency namespace with `NullHttpHandlerFactory` implementations.
### Contribution Checklist
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
### Motivation and Context
1. Why is this change required? This is part of the work to extract
functionality from Semantic Kernel core prior to 1.0 release.
2. What problem does it solve? Provides a pattern to follow for
additional template engine implementations.
3. What scenario does it contribute to? Contributes to the goal of
reducing the Semantic Kernel core to focus on AI orchestration.
4. If it fixes an open issue, please link to the issue here.
https://github.com/microsoft/semantic-kernel/issues/2527
### Description
- Create new template engine project in the extensions folder
- Move template engine code from semantic kernel abstractions and core
- Refactor `IPromptTemplateEngine` so that it doesn't depend on details
of the default implementation
- Refactor `PromptTemplateEngine` to match the new interface definition
- Move template engine unit tests to the extensions unit tests project
### Contribution Checklist
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
### Description
Adding Kusto as an external memory.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
---------
Co-authored-by: Dmytro Struk <[email protected]>
Co-authored-by: Shawn Callegari <[email protected]>
### Motivation and Context
The service was renamed to Azure Cognitive Search in 2019, so we should
follow the new convention. The only thing still named "Azure Search" is
the old Azure SDK, out of scope.
### Description
* Revert nuget package name to the previous name (we'll delete the new
package, which has been out only for few days)
* Rename C# and Python classes and namespaces
### Description
Updating KernelSyntaxExamples to use the same environment variable keys
as the Integration tests. This is a step toward making these tests
simpler to run, and even running them as automation in CI tests.
This is a first pass -- more cleanup to be done.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [X] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
---------
Co-authored-by: name <email>
Co-authored-by: Dmytro Struk <[email protected]>
* .NET, Python: Add new Memory Storage based on Azure Cognitive Search
with custom embeddings
* .NET: Delete previous Azure Search semantic memory component based on old
Azure Semantic Search w/ Lucene
This pains me, as it's a technical step backwards. But there are two
problems today:
1) The .NET support policy today unfortunately applies not only to the
.NET SDK and runtime but also to nuget packages released in the same
wave, which means that if someone is relying on eg System.Text.Json and
is running on .NET 6, having SK depend on System.Text.Json with a
minimum version of 7.0 forces the consumer to also use the 7.0 version
of System.Text.Json from the nuget, which then means that one assembly
falls under the STS support policy instead of LTS support policy, which
means that one assembly will have support end for it a few months
earlier than it otherwise would. I've raised this issue for further
evaluation on the .NET side of things, but in the meantime, this lowers
the version number to remove the perceived problem and possible barrier
to adoption.
2) Azure Functions has two deployment models: in-process and isolated.
Ideally functions use isolated, which gives them the freedom to
reference whatever they need. The in-process model is exactly what it
sounds like: the function runs in the same process as the host, and the
host pins several dependencies at 6.0 versions. That means that if a
function references a 7.0 version, it'll fail to load in the in-process
model. While we'd like for folks to be using the isolated model, we
can't force it, and we don't want a need for the in-process model to
block SK usage.
This commit downgrades back to the 6.0 versions, at least where
possible. Some of the connectors reference libraries (e.g. NRedisStack,
pgvector, etc.) that themselves have a 7.0 dependency.
Closes https://github.com/microsoft/semantic-kernel/issues/1793
---------
Co-authored-by: Shawn Callegari <[email protected]>
### Motivation and Context
This pull request adds a new extension for semantic planning using a
stepwise approach. The StepwisePlanner extension allows users to create
and execute plans that consist of a sequence of semantic and native
functions, each with a goal and a set of inputs and outputs. The
extension uses a semantic search engine to find relevant functions for
each step, and a plan creation service to generate a plan that satisfies
the user's ask. The extension also provides a system step function that
executes the plan and returns the final answer and intermediate
observations. The extension can be configured with various parameters,
such as the relevancy threshold, the maximum number of relevant
functions, the excluded and included functions and skills, and the
maximum number of tokens, iterations, and time for the plan.
Regarding #1472
### Description
- Add StepwisePlanner.cs, which registers the planner native functions
and the system step function
- Add StepwisePlannerConfig.cs, which defines the configuration options
for the StepwisePlanner extension
- Add SystemStep.cs, which represents a step in a Stepwise plan, with
properties for the thought, action, action variables, observation, final
answer, and original response
- Add helper methods for formatting and validating function views,
generating plan requests, and invoking the plan.
- Add logging and error handling for the planner extension
- Add unit tests for the planner extension and the native functions
### Related
- Majority of work initially started from @kaza in #992
### Changes in other PRs to merge separately
- #1464
- #1465
- #1466
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [x] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
---------
Co-authored-by: Almir Kazazic <[email protected]>
---------
Co-authored-by: Lee Miller <[email protected]>
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
This PR contains changes for Chroma C# memory store.
For more information about Chroma, see:
https://docs.trychroma.com/getting-started.
To run integration tests, local Chroma server should be up and running.
To start local Chroma server for testing (this information is also
included in
[README](https://github.com/microsoft/semantic-kernel/blob/0fb77e0c0f347e3f22d79b3c99e7f84d35340462/dotnet/src/Connectors/Connectors.Memory.Chroma/README.md)):
1. Clone Chroma:
```bash
git clone https://github.com/chroma-core/chroma.git
cd chroma
```
2. Run local Chroma server with Docker within Chroma repository root:
```bash
docker-compose up -d --build
```
3. Use Semantic Kernel with Chroma, using server local endpoint
`http://localhost:8000`:
```csharp
const string endpoint = "http://localhost:8000";
ChromaMemoryStore memoryStore = new(endpoint);
IKernel kernel = Kernel.Builder
.WithLogger(logger)
.WithOpenAITextEmbeddingGenerationService("text-embedding-ada-002", "OPENAI_API_KEY")
.WithMemoryStorage(memoryStore)
//.WithChromaMemoryStore(endpoint) // This method offers an alternative approach to registering Chroma memory store.
.Build();
```
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Main public classes:
1. `ChromaMemoryStore` - implementation of `IMemoryStore` interface.
Uses `ChromaClient` to communicate with Chroma API. Throws
`ChromaMemoryStoreException` in case of issues.
2. `ChromaClient` - responsible for communication with Chroma API.
Throws `ChromaClientException` in case of issues.
Chroma entity models:
1. `ChromaCollectionModel` - defines Chroma collection.
2. `ChromaEmbeddingsModel` - defines Chroma embeddings, constructed from
`MemoryRecord`.
3. `ChromaQueryResultModel` - defines Chroma search result set.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [x] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
### Motivation and Context
Slimming down the kernel core project, nuget package.
### Description
These "Core Skills" are good ones to have, but are not fundamental to
the operation of SK. They will continue to be included in the
"Microsoft.SemanticKernel" nuget metapackage, but if a developer chooses
to just install the minimal "Microsoft.SemanticKernel.Core", they may
not want these skills. This packages them so that they are now optional.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [X] The code builds clean without any errors or warnings
- [X] The PR follows SK Contribution Guidelines
(https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
- [X] The code follows the .NET coding conventions
(https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions)
verified with `dotnet format`
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
---------
Co-authored-by: name <email>
Co-authored-by: Lee Miller <[email protected]>
"KernelSyntaxExamples" is the only outstanding project under /samples
that still uses project references to access the kernel libraries.
This change consolidates all the building, linked dotnet SDK code in
SK-dotnet.sln into a single top-level folder. This will simplify build
jobs, automatic code owner assignment, and other dev ops tasks.
Coming next: other projects in samples will get removed from the
dotnet/SK-dotnet.sln solution, and a new solution with dedicated builds
created for the /samples folder.
TBD:
- Language-specific code "samples" will move into a similar subfolder
for each language.
- The current "/samples" folder (at the root) will likely become
something like "apps", with the demo apps promoted to the top-level
folder for better discoverability.
- notebooks may also move to this folder in the future.
Co-authored-by: name <email>
Roger Barreto