microsoft/semantic-kernel
1
0
Fork 0
mirror of https://github.com/microsoft/semantic-kernel.git synced 2026-10-12 02:19:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into java-development

Browse Source
This commit is contained in:
markwallace-microsoft
2023-11-01 15:29:25 +00:00
parent d63d094b81 f3df736d7b
commit 0f3f79eb32
124 changed files with 2275 additions and 2311 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/codeql-analysis.yml
+1 -1
View File
@@ -22,7 +22,7 @@ jobs:
strategy:
fail-fast: false
matrix:
language: ["csharp", "python", "java"]
language: ["csharp", "python"]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Use only 'java' to analyze code written in Java, Kotlin or both
# Use only 'javascript' to analyze code written in JavaScript, TypeScript or both
README.md
+1 -1
View File
@@ -144,7 +144,7 @@ If you like Semantic Kernel, you may also be interested in other repos the Seman
| [Chat Copilot](https://github.com/microsoft/chat-copilot) | A reference application that demonstrates how to build a chatbot with Semantic Kernel. |
| [Semantic Kernel Docs](https://github.com/MicrosoftDocs/semantic-kernel-docs) | The home for Semantic Kernel documentation that appears on the Microsoft learn site. |
| [Semantic Kernel Starters](https://github.com/microsoft/semantic-kernel-starters) | Starter projects for Semantic Kernel to make it easier to get started. |
| [Semantic Memory](https://github.com/microsoft/semantic-memory) | A service that allows you to create pipelines for ingesting, storing, and querying knowledge. |
| [Kernel Memory](https://github.com/microsoft/kernel-memory) | A scalable Memory service to store information and ask questions using the RAG pattern. |
## Join the community
docs/decisions/0001-madr-architecture-decisions.md
+19 -17
View File
@@ -1,11 +1,12 @@
---
# These are optional elements. Feel free to remove any of them.
status: proposed
status: accepted
date: 2023-05-29
deciders: dluc,shawncal,hathind,alliscode
consulted:
informed:
deciders: dluc, shawncal, hathind, alliscode
consulted:
informed:
---
# Use Markdown Any Decision Records to track Semantic Kernel Architecture Decisions
## Context and Problem Statement
@@ -18,14 +19,15 @@ MADR is a lean template to capture any decisions in a structured way. The templa
For more information [see](https://adr.github.io/)
<!-- This is an optional element. Feel free to remove. -->
## Decision Drivers
* Architecture changes and the associated decision making process should be transparent to the community.
* Decision records are stored in the repository and are easily discoverable for teams involved in the various language ports.
- Architecture changes and the associated decision making process should be transparent to the community.
- Decision records are stored in the repository and are easily discoverable for teams involved in the various language ports.
## Considered Options
* Use MADR format and store decision documents in the repository.
- Use MADR format and store decision documents in the repository.
## Decision Outcome
@@ -38,21 +40,21 @@ Chosen option:
How would we use ADR's to track technical decisions?
1. Copy docs/decisions/adr-template.md to docs/decisions/NNNN-title-with-dashes.md, where NNNN indicates the next number in sequence.
1. Check for existing PR's to make sure you use the correct sequence number.
2. There is also a short form template docs/decisions/adr-short-template.md
1. Check for existing PR's to make sure you use the correct sequence number.
2. There is also a short form template docs/decisions/adr-short-template.md
2. Edit NNNN-title-with-dashes.md.
1. Status must initially be `proposed`
2. List of `deciders` must include the aliases of the people who will sign off on the decision.
3. The relevant EM and `dluc` must be listed as deciders or informed of all decisions.
4. You should list the aliases of all partners who were consulted as part of the decision.
1. Status must initially be `proposed`
2. List of `deciders` must include the aliases of the people who will sign off on the decision.
3. The relevant EM and `dluc` must be listed as deciders or informed of all decisions.
4. You should list the aliases of all partners who were consulted as part of the decision.
3. For each option list the good, neutral and bad aspects of each considered alternative.
1. Detailed investigations can be included in the `More Information` section inline or as links to external documents.
1. Detailed investigations can be included in the `More Information` section inline or as links to external documents.
4. Share your PR with the deciders and other interested parties.
1. Deciders must be listed as required reviewers.
2. The status must be updated to `accepted` once a decision is agreed and the date must also be updated.
3. Approval of the decision is captured using PR approval.
5. Decisions can be changed later and superseded by a new ADR. In this case it is useful to record any negative outcomes in the original ADR.
* Good, because lightweight format which is easy to edit
* Good, because this uses the standard Git review process for commenting and approval
* Good, because decisions and review process are transparent to the community
- Good, because lightweight format which is easy to edit
- Good, because this uses the standard Git review process for commenting and approval
- Good, because decisions and review process are transparent to the community
docs/decisions/0002-java-folder-structure.md
+1 -1
View File
@@ -1,6 +1,6 @@
---
# These are optional elements. Feel free to remove any of them.
status: {proposed}
status: accepted
date: 2013-06-19
deciders: shawncal,johnoliver
consulted:
docs/decisions/0003-support-multiple-native-function-args.md
+1
View File
@@ -1,6 +1,7 @@
---
# These are optional elements. Feel free to remove any of them.
status: accepted
contact: markwallace-microsoft
date: 2023-06-16
deciders: shawncal,dluc
consulted:
docs/decisions/0004-error-handling.md
+2 -1
View File
@@ -1,6 +1,7 @@
---
# These are optional elements. Feel free to remove any of them.
status: proposed
status: accepted
contact: semenshi
date: 2023-06-23
deciders: shawncal
consulted: stephentoub
docs/decisions/0005-function-execution-handlers.md
-133
View File
@@ -1,133 +0,0 @@
---
# These are optional elements. Feel free to remove any of them.
status: proposed
date: 2023-05-29
deciders: @rogerbarreto, @shawncal, @stephentoub
consulted:
informed:
---
# Kernel/Function Handlers
## Context and Problem Statement
A Kernel function caller needs to be able to handle/intercept any function execution in the Kernel before and after it was attempted. Allowing it to modify the prompt, abort the execution, or modify the output and many other scenarios as follows:
- Pre-Execution / Running
- Get: Original prompt template
- Get: Prompt generated by the current Kernel `TemplateEngine` before calling the LLM
- Get: Current settings used
- Get: Parameters used
- Get: SKContext
- Set: Modify a prompt content before sending it to LLM
- Set: Abort/Cancel function execution
- In-Execution / Stream Processing
- Get: Text Prompt generated per stream block `IAsyncEnumerable<string>`
- Set: Filter/Change a prompt stream block
- Set: Skip a prompt stream block
- Post-Execution / Ran
- Get: LLM Model Result (Tokens Usage, Stop Sequence, ...)
- Get: Generated Prompt
- Get: SKContext
- Get: Output parameters
- Set: Modify a prompt content after getting it from LLM
- Set: Modify output parameters content (before returning the output)
## Decision Drivers
- Architecture changes and the associated decision making process should be transparent to the community.
- Decision records are stored in the repository and are easily discoverable for teams involved in the various language ports.
## Considered Options
- Callback Registration + Recursive
- Single Callback
- Event Based Registration
- Middleware
## Pros and Cons of the Options
### Callback Registration Recursive Delegate (Kernel, Plan, Function)
- Specified on plan and function level as a configuration be able to specify what are the callback Handlers that will be triggered.
Pros:
- Common pattern for observing and also changing data exposed as parameter into the delegate signature for (Get/Set) scenarios
- Registering a callback gives back the registration object that can be used to cancel the execution of the function in the future.
- Recursive approach, allows to register multiple callbacks for the same event, and also allows to register callbacks on top of pre existing callbacks.
Cons:
- Registrations may use more memory and might not be garbage collected in the recursive approach, only when the function or the plan is disposed.
### Single Callback Delegate (Kernel, Plan, Function)
- Specified on kernel level as a configuration be able to specify what are the callback Handlers that will be triggered.
- Specified on function creation: As part of the function constructor be able to specify what are the callback Handlers that will be triggered.
- Specified on function invocation: As part of the function invoke be able to specify what are the callback Handlers as a parameter that will be triggered.
Pros:
- Common pattern for observing and also changing data exposed as parameter into the delegate signature for (Get/Set) scenarios
Cons:
- Limited to only one method observing a specific event (Pre Post and InExecution). - Function When used as parameter, three new parameters would be needed as part of the function. (Specified on function invocation) - Extra Cons on
### Event Base Registration (Kernel only)
Expose events on both IKernel and ISKFunction that the call can can be observing to interact.
Pros:
- Multiple Listeners can registered for the same event
- Listeners can be registered and unregistered at will
- Common pattern (EventArgs) for observing and also changing data exposed as parameter into the event signature for (Get/Set) scenarios
Cons:
- Event handlers are void, making the EventArgs by reference the only way to modify the data.
- Not clear how supportive is this approach for asynchronous pattern/multi threading
- Won't support `ISKFunction.InvokeAsync`
### Middleware (Kernel Only)
Specified on Kernel level, and would only be used using IKernel.RunAsync operation, this pattern would be similar to asp.net core middlewares, running the pipelines with a context and a requestdelegate next for controlling (Pre/Post conditions)
Pros:
- Common pattern for handling Pre/Post Setting/Filtering data
Cons:
- Functions can run on their own instance, middlewares suggest more complexity and the existence of an external container/manager (Kernel) to intercept/observe function calls.
## Main Questions
- Q: Post Execution Handlers should execute right after the LLM result or before the end of the function execution itself?
A: Currently post execution Handlers are executed after function execution.
- Q: Should Pre/Post Handlers be many (pub/sub) allowing registration/deregistration?
A: By using the standard .NET event implementation, this already supports multiple registrations as well as deregistrations managed by the caller.
- Q: Setting Handlers on top of pre existing Handlers should be allowed or throw an error?
A: By using the standard .NET event implementation, the stander behavior will not throw an error and will execute all the registered handlers.
- Q: Setting Handlers on Plans should automatically cascade this Handlers for all the inner steps + overriding existing ones in the process?
A: Handlers will be triggered before and after each step is executed the same way the Kernel RunAsync pipeline works.
- Q: When a pre execution handler cancel the execution of the function,
- Q: When a pre function execution handler intents to cancel the execution, should further handlers in the chain be called or not?
A: Currently the standard .net behavior is to call all the registered handlers. This way function execution will solely depends on the final boolean state of the Cancel property after all handlers were called.
## Decision Outcome
TBD.
docs/decisions/0005-kernel-hooks-phase1.md
+244
View File
@@ -0,0 +1,244 @@
---
# These are optional elements. Feel free to remove any of them.
status: accepted
contact: rogerbarreto
date: 2023-05-29
deciders: rogerbarreto, shawncal, stephentoub
consulted:
informed:
---
# Kernel/Function Handlers - Phase 1
## Context and Problem Statement
A Kernel function caller needs to be able to handle/intercept any function execution in the Kernel before and after it was attempted. Allowing it to modify the prompt, abort the execution, or modify the output and many other scenarios as follows:
- Pre-Execution / Function Invoking
- Get: SKContext
- Set: Modify input parameters sending to the function
- Set: Abort/Cancel pipeline execution
- Set: Skip function execution
- Post-Execution / Function Invoked
- Get: LLM Model Result (Tokens Usage, Stop Sequence, ...)
- Get: SKContext
- Get: Output parameters
- Set: Modify output parameters content (before returning the output)
- Set: Cancel pipeline execution
- Set: Repeat function execution
## Out of Scope (Will be in phase 2)
- Pre-Execution / Function Invoking
- Get: Rendered Prompt
- Get: Current settings used
- Set: Modify the Rendered Prompt
- Post-Execution / Function Invoked
- Get: Rendered Prompt
- Get: Current settings used
## Decision Drivers
- Architecture changes and the associated decision making process should be transparent to the community.
- Decision records are stored in the repository and are easily discoverable for teams involved in the various language ports.
- Simple, Extensible and easy to understand.
## Considered Options
1. Callback Registration + Recursive
2. Single Callback
3. Event Based Registration
4. Middleware
5. ISKFunction Event Support Interfaces
## Pros and Cons of the Options
### 1. Callback Registration Recursive Delegate (Kernel, Plan, Function)
- Specified on plan and function level as a configuration be able to specify what are the callback Handlers that will be triggered.
Pros:
- Common pattern for observing and also changing data exposed as parameter into the delegate signature for (Get/Set) scenarios
- Registering a callback gives back the registration object that can be used to cancel the execution of the function in the future.
- Recursive approach, allows to register multiple callbacks for the same event, and also allows to register callbacks on top of pre existing callbacks.
Cons:
- Registrations may use more memory and might not be garbage collected in the recursive approach, only when the function or the plan is disposed.
### 2. Single Callback Delegate (Kernel, Plan, Function)
- Specified on kernel level as a configuration be able to specify what are the callback Handlers that will be triggered.
- Specified on function creation: As part of the function constructor be able to specify what are the callback Handlers that will be triggered.
- Specified on function invocation: As part of the function invoke be able to specify what are the callback Handlers as a parameter that will be triggered.
Pros:
- Common pattern for observing and also changing data exposed as parameter into the delegate signature for (Get/Set) scenarios
Cons:
- Limited to only one method observing a specific event (Pre Post and InExecution). - Function When used as parameter, three new parameters would be needed as part of the function. (Specified on function invocation) - Extra Cons on
### 3. Event Base Registration (Kernel only)
Expose events on both IKernel and ISKFunction that the call can can be observing to interact.
Pros:
- Multiple Listeners can registered for the same event
- Listeners can be registered and unregistered at will
- Common pattern (EventArgs) for observing and also changing data exposed as parameter into the event signature for (Get/Set) scenarios
Cons:
- Event handlers are void, making the EventArgs by reference the only way to modify the data.
- Not clear how supportive is this approach for asynchronous pattern/multi threading
- Won't support `ISKFunction.InvokeAsync`
### 4. Middleware (Kernel Only)
Specified on Kernel level, and would only be used using IKernel.RunAsync operation, this pattern would be similar to asp.net core middlewares, running the pipelines with a context and a requestdelegate next for controlling (Pre/Post conditions)
Pros:
- Common pattern for handling Pre/Post Setting/Filtering data
Cons:
- Functions can run on their own instance, middlewares suggest more complexity and the existence of an external container/manager (Kernel) to intercept/observe function calls.
### 5. ISKFunction Event Support Interfaces
```csharp
class Kernel : IKernel
{
RunAsync() {
var functionInvokingArgs = await this.TriggerEvent<FunctionInvokingEventArgs>(this.FunctionInvoking, skFunction, context);
var functionResult = await skFunction.InvokeAsync(context, cancellationToken: cancellationToken);
var functionInvokedArgs = await this.TriggerEvent<FunctionInvokedEventArgs>(
this.FunctionInvoked,
skFunction,
context);
}
private TEventArgs? TriggerEvent<TEventArgs>(EventHandler<TEventArgs>? eventHandler, ISKFunction function, SKContext context) where TEventArgs : SKEventArgs
{
if (eventHandler is null)
{
return null;
}
if (function is ISKFunctionEventSupport<TEventArgs> supportedFunction)
{
var eventArgs = await supportedFunction.PrepareEventArgsAsync(context);
eventHandler.Invoke(this, eventArgs);
return eventArgs;
}
// Think about allowing to add data with the extra interface.
// If a function don't support the specific event we can:
return null; // Ignore or Throw.
throw new NotSupportedException($"The provided function \"{function.Name}\" does not supports and implements ISKFunctionHandles<{typeof(TEventArgs).Name}>");
}
}
public interface ISKFunctionEventSupport<TEventArgs> where TEventArgs : SKEventArgs
{
Task<TEventArgs> PrepareEventArgsAsync(SKContext context, TEventArgs? eventArgs = null);
}
class SemanticFunction : ISKFunction,
ISKFunctionEventSupport<FunctionInvokingEventArgs>,
ISKFunctionEventSupport<FunctionInvokedEventArgs>
{
public FunctionInvokingEventArgs PrepareEventArgsAsync(SKContext context, FunctionInvokingEventArgs? eventArgs = null)
{
var renderedPrompt = await this.RenderPromptTemplateAsync(context);
context.Variables.Set(SemanticFunction.RenderedPromptKey, renderedPrompt);
return new SemanticFunctionInvokingEventArgs(this.Describe(), context);
// OR Metadata Dictionary<string, object>
return new FunctionInvokingEventArgs(this.Describe(), context, new Dictionary<string, object>() { { RenderedPrompt, renderedPrompt } });
}
public FunctionInvokedEventArgs PrepareEventArgsAsync(SKContext context, FunctionInvokedEventArgs? eventArgs = null)
{
return Task.FromResult<FunctionInvokedEventArgs>(new SemanticFunctionInvokedEventArgs(this.Describe(), context));
}
}
public sealed class SemanticFunctionInvokedEventArgs : FunctionInvokedEventArgs
{
public SemanticFunctionInvokedEventArgs(FunctionDescription functionDescription, SKContext context)
: base(functionDescription, context)
{
_context = context;
Metadata[RenderedPromptKey] = this._context.Variables[RenderedPromptKey];
}
public string? RenderedPrompt => this.Metadata[RenderedPromptKey];
}
public sealed class SemanticFunctionInvokingEventArgs : FunctionInvokingEventArgs
{
public SemanticFunctionInvokingEventArgs(FunctionDescription functionDescription, SKContext context)
: base(functionDescription, context)
{
_context = context;
}
public string? RenderedPrompt => this._context.Variables[RenderedPromptKey];
}
```
### Pros and Cons
Pros:
- `Kernel` is not aware of `SemanticFunction` implementation details or any other `ISKFunction` implementation
- Extensible to show dedicated EventArgs per custom `ISKFunctions` implementation, including prompts for semantic functions
- Extensible to support future events on the Kernel thru the `ISKFunctionEventSupport<NewEvent>` interface
- Functions can have their own EventArgs specialization.
- Interface is optional, so custom `ISKFunctions` can choose to implement it or not
Cons:
- Any custom functions now will have to responsibility implement the `ISKFunctionEventSupport` interface if they want to support events.
- `Kernel` will have to check if the function implements the interface or not, and if not, it will have to throw an exception or ignore the event.
- Functions implementations that once were limited to InvokeAsync now need to be scattered across multiple places and handle the state of the execution related to content that needs to be get at the beginning or at the end of the invocation.
## Main Questions
- Q: Post Execution Handlers should execute right after the LLM result or before the end of the function execution itself?
A: Currently post execution Handlers are executed after function execution.
- Q: Should Pre/Post Handlers be many (pub/sub) allowing registration/deregistration?
A: By using the standard .NET event implementation, this already supports multiple registrations as well as deregistrations managed by the caller.
- Q: Setting Handlers on top of pre existing Handlers should be allowed or throw an error?
A: By using the standard .NET event implementation, the standard behavior will not throw an error and will execute all the registered handlers.
- Q: Setting Handlers on Plans should automatically cascade this Handlers for all the inner steps + overriding existing ones in the process?
A: Handlers will be triggered before and after each step is executed the same way the Kernel RunAsync pipeline works.
- Q: When a pre function execution handler intents to cancel the execution, should further handlers in the chain be called or not?
A: Currently the standard .net behavior is to call all the registered handlers. This way function execution will solely depends on the final state of the Cancellation Request after all handlers were called.
## Decision Outcome
Chosen option: **3. Event Base Registration (Kernel only)**
This approach is the simplest and take the benefits of the standard .NET event implementation.
Further changes will be implemented to fully support all the scenarios in phase 2.
docs/decisions/0006-open-api-dynamic-payload-and-namespaces.md
+2 -2
View File
@@ -1,6 +1,6 @@
---
# These are optional elements. Feel free to remove any of them.
status: proposed
status: accepted
contact: semenshi
date: 2023-08-15
deciders: shawncal
consulted:
docs/decisions/0007-prompt-extract-template-engine.md
+1
View File
@@ -1,6 +1,7 @@
---
# These are optional elements. Feel free to remove any of them.
status: accepted
contact: markwallace-microsoft
date: 2023-08-25
deciders: shawncal
consulted:
docs/decisions/0008-support-generic-llm-request-settings.md
+2 -1
View File
@@ -1,6 +1,7 @@
---
# These are optional elements. Feel free to remove any of them.
status: proposed
status: accepted
contact: markwallace-microsoft
date: 2023-=9-15
deciders: shawncal
consulted: stoub, lemiller, dmytrostruk
docs/decisions/0009-support-multiple-named-args-in-template-function-calls.md
+3 -2
View File
@@ -1,7 +1,8 @@
---
# These are optional elements. Feel free to remove any of them.
status: proposed
date: 6/16/2023
status: accepted
contact: dmytrostruk
date: 2013-06-16
deciders: shawncal, hario90
consulted: dmytrostruk, matthewbolanos
informed: lemillermicrosoft
docs/decisions/0010-dotnet-project-structure.md
+1
View File
@@ -3,6 +3,7 @@
# These are optional elements. Feel free to remove any of them
status: accepted
contact: markwallace-microsoft
date: 2023-09-29
deciders: semenshi, dmytrostruk, rbarreto
consulted: shawncal, stoub, lemiller
docs/decisions/0010-openai-function-calling.md
+2 -1
View File
@@ -1,5 +1,6 @@
---
status: proposed
status: accepted
contact: gitri-ms
date: 2023-09-21
deciders: gitri-ms, shawncal
consulted: lemillermicrosoft, awharrison-28, dmytrostruk, nacharya1
docs/decisions/0011-function-and-kernel-result-types.md
+1
View File
@@ -1,6 +1,7 @@
---
# These are optional elements. Feel free to remove any of them.
status: accepted
contact: dmytrostruk
date: 2023-09-21
deciders: shawncal, dmytrostruk
consulted:
docs/decisions/0012-kernel-service-registration.md
+3 -2
View File
@@ -1,9 +1,10 @@
---
# These are optional elements. Feel free to remove any of them.
status: proposed
status: accepted
contact: dmytrostruk
date: 2023-10-03
deciders: dmytrostruk
consulted: semenshi, rbarreto, markwallace
consulted: semenshi, rbarreto, markwallace-microsoft
informed:
---
# Kernel Service Registration
docs/decisions/0013-memory-as-plugin.md
+2 -1
View File
@@ -1,6 +1,7 @@
---
# These are optional elements. Feel free to remove any of them.
status: proposed
status: accepted
contact: dmytrostruk
date: 2023-09-21
deciders: shawncal, dmytrostruk
consulted:
docs/decisions/0014-chat-completion-roles-in-prompt.md
+146
View File
@@ -0,0 +1,146 @@
---
# These are optional elements. Feel free to remove any of them.
status: accepted
contact: semenshi
date: 2023-10-23
deciders: markwallace-microsoft, mabolan
consulted:
informed:
---
# SK prompt syntax for chat completion roles
## Context and Problem Statement
Today, SK does not have the ability to mark a block of text in a prompt as a message with a specific role, such as assistant, system, or user. As a result, SK can't chunk the prompt into the list of messages required by chat completion connectors.
Additionally, prompts can be defined using a range of template syntaxes supported by various template engines, such as Handlebars, Jinja, and others. Each of these syntaxes may represent chat messages or roles in a distinct way. Consequently, the template engine syntax may leak into SK's domain if no proper abstraction is put in place, coupling SK with the template engines and making it impossible to support new ones.
<!-- This is an optional element. Feel free to remove. -->
## Decision Drivers
* It should be possible to mark a block of text in a prompt as a message with a role so that it can be converted into a list of chat messages for use by chat completion connectors.
* The syntax specific to the template engine message/role should be mapped to the SK message/role syntax to abstract SK from a specific template engine syntax.
## Considered Options
**1. Message/role tags are generated by functions specified in a prompt.** This option relies on the fact that many template engines can invoke functions specified in the template. Therefore, an internal function can be registered with a template engine, and the function will create a message/model tag based on the provided arguments. The prompt template engine will execute the function and emit the function result into the prompt template, and the rendered prompt will have a section for each message/role decorated with these tags. Here's an example of how this can be done using the SK basic template engine and Handlebars:
Function:
```csharp
internal class SystemFunctions
{
public string Message(string role)
{
return $"<message role=\"{role}\">";
}
}
```
Prompt:
```bash
{{message role="system"}}
You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
{{message role="system"}}
{{message role="user"}}
I want to {{$input}}
{{message role="user"}}
```
Rendered prompt:
```xml
<message role="system">
You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
</message>
<message role="user">
I want to buy a house.
</message>
```
**2. Message/role tags are generated by a prompt-specific mechanism.** This option utilizes template engine syntax constructions, helpers, and handlers other than functions to inject SK message/role tags into the final prompt.
In the example below, to parse the prompt that uses the handlebars syntax we need to register a block helper (a callback that is invoked when the Handlebars engine encounters it) to emit the SK message/role tags in the resulting prompt.
Block helpers:
```csharp
this.handlebarsEngine.RegisterHelper("system", (EncodedTextWriter output, Context context, Arguments arguments) => {
//Emit the <message role="system"> tags
});
this.handlebarsEngine.RegisterHelper("user", (EncodedTextWriter output, Context context, Arguments arguments) => {
//Emit the <message role="user"> tags
});
```
Prompt:
```bash
{{#system~}}
You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
{{~/system}}
{{#user~}}
I want to {{$input}}
{{~/user}}
```
Rendered prompt:
```xml
<message role="system">
You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
</message>
<message role="user">
I want to buy a house.
</message>
```
**3. Message/role tags are applied on top of prompt template engine**. This option presumes specifying the SK message/role tags directly in a prompt to denote message/role blocks in way that template engine does not parse/handle them and considers them as a regular text.
In the example below, the prompt the `<message role="*">` tags are marking boundaries of the system and user messages and SK basic template engine consider them as regular text without processing them.
Prompt:
```xml
<message role="system">
You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
</message>
<message role="user">
I want to {{$input}}
</message>
```
Rendered prompt:
```xml
<message role="system">
You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
</message>
<message role="user">
I want to buy a house.
</message>
```
## Pros and Cons
**1. Message/role tags are generated by functions specified in a prompt**
Pros:
* Functions can be defined once and reused in prompt templates that support function calling.
Cons:
* Functions might not be supported by some template engines.
* The system/internal functions should be pre-registered by SK so users don't need to import them.
* Each prompt template engine will have how to discover and call the system/internal functions.
**2. Message/role tags are generated by prompt specific mechanism**
Pros:
* Enables message/role representation with the optimal template engine syntax constructions, aligning with other constructions for that specific engine.
Cons:
* Each prompt template engine will have to register callbacks/handlers to handle template syntax constructions rendering to emit SK message/role tags.
**3. Message/role tags are applied on top of prompt template engine**
Pros:
* No changes are required to prompt template engines.
Cons:
* The message/role tag syntax may not align with other syntax constructions for that template engine.
* Syntax errors in message/role tags will be detected by components parsing the prompt and not by prompt template engines.
## Decision Outcome
It was agreed not to limit ourselves to only one possible option because it may not be feasible to apply that option to new template engines we might need to support in the future. Instead, each time a new template engine is added, every option should be considered, and the optimal one should be preferred for that particular template engine.
It was also agreed that, at the moment, we will go with the "3. Message/role tags are applied on top of the prompt template engine" option to support the message/role prompt syntax in SK, which currently uses the `BasicPromptTemplateEngine` engine.
docs/decisions/0015-completion-service-selection.md
+117
View File
@@ -0,0 +1,117 @@
---
# These are optional elements. Feel free to remove any of them.
status: accepted
date: 2023-10-25
deciders: markwallace, mabolan
consulted:
informed:
---
# Completion service type selection strategy
## Context and Problem Statement
Today, SK runs all text prompts using the text completion service. With the addition of a new chat completion prompts and potentially other prompt types, such as image, on the horizon, we need a way to select a completion service type to run these prompts.
<!-- This is an optional element. Feel free to remove. -->
## Decision Drivers
* Semantic function should be able to identify a completion service type to use when processing text, chat, or image prompts.
## Considered Options
**1. Completion service type identified by the "prompt_type" property.** This option presumes adding the 'prompt_type' property to the prompt template config model class, 'PromptTemplateConfig.' The property will be specified once by a prompt developer and will be used by the 'SemanticFunction' class to decide which completion service type (not instance) to use when resolving an instance of that particular completion service type.
**Prompt template**
```json
{
"schema": "1",
"description": "Hello AI, what can you do for me?",
"prompt_type": "<text|chat|image>",
"models": [...]
}
```
**Semantic function pseudocode**
```csharp
if(string.IsNullOrEmpty(promptTemplateConfig.PromptType) || promptTemplateConfig.PromptType == "text")
{
var service = this._serviceSelector.SelectAIService<ITextCompletion>(context.ServiceProvider, this._modelSettings);
//render the prompt, call the service, process and return result
}
else (promptTemplateConfig.PromptType == "chat")
{
var service = this._serviceSelector.SelectAIService<IChatCompletion>(context.ServiceProvider, this._modelSettings);
//render the prompt, call the service, process and return result
},
else (promptTemplateConfig.PromptType == "image")
{
var service = this._serviceSelector.SelectAIService<IImageGeneration>(context.ServiceProvider, this._modelSettings);
//render the prompt, call the service, process and return result
}
```
**Example**
```json
name: ComicStrip.Create
prompt: "Generate ideas for a comic strip based on {{$input}}. Design characters, develop the plot, ..."
config: {
"schema": 1,
"prompt_type": "text",
...
}
name: ComicStrip.Draw
prompt: "Draw the comic strip - {{$comicStrip.Create $input}}"
config: {
"schema": 1,
"prompt_type": "image",
...
}
```
Pros:
- Deterministically specifies which completion service **type** to use, so image prompts won't be rendered by a text completion service, and vice versa.
Cons:
- Another property to specify by a prompt developer.
**2. Completion service type identified by prompt content.** The idea behind this option is to analyze the rendered prompt by using regex to check for the presence of specific markers associated with the prompt type. For example, the presence of the `<message role="*"></message>` tag in the rendered prompt might indicate that the prompt is a chat prompt and should be handled by the chat completion service. This approach may work reliably when we have two completion service types - text and chat - since the logic would be straightforward: if the message tag is found in the rendered prompt, handle it with the chat completion service; otherwise, use the text completion service. However, this logic becomes unreliable when we start adding new prompt types, and those prompts lack markers specific to their prompt type. For example, if we add an image prompt, we won't be able to distinguish between a text prompt and an image prompt unless the image prompt has a unique marker identifying it as such.
```csharp
if (Regex.IsMatch(renderedPrompt, @"<message>.*?</message>"))
{
var service = this._serviceSelector.SelectAIService<IChatCompletion>(context.ServiceProvider, this._modelSettings);
//render the prompt, call the service, process and return result
},
else
{
var service = this._serviceSelector.SelectAIService<ITextCompletion>(context.ServiceProvider, this._modelSettings);
//render the prompt, call the service, process and return result
}
```
**Example**
```json
name: ComicStrip.Create
prompt: "Generate ideas for a comic strip based on {{$input}}. Design characters, develop the plot, ..."
config: {
"schema": 1,
...
}
name: ComicStrip.Draw
prompt: "Draw the comic strip - {{$comicStrip.Create $input}}"
config: {
"schema": 1,
...
}
```
Pros:
- No need for a new property to identify the prompt type.
Cons:
- Unreliable unless the prompt contains unique markers specifically identifying the prompt type.
## Decision Outcome
We decided to choose the '2. Completion service type identified by prompt content' option and will reconsider it when we encounter another completion service type that cannot be supported by this option or when we have a solid set of requirements for using a different mechanism for selecting the completion service type.
docs/decisions/0016-semantic-function-multiple-model-support.md
+173
View File
@@ -0,0 +1,173 @@
---
# These are optional elements. Feel free to remove any of them.
status: approved
contact: markwallace-microsoft
date: 2023-10-26
deciders: markwallace-microsoft, semenshi, rogerbarreto
consulted: mabolan, dmytrostruk
informed:
---
# Multiple Model Support for Semantic Functions
## Context and Problem Statement
Developers need to be able to use multiple models simultaneously e.g., using GPT4 for certain prompts and GPT3.5 for others to reduce cost.
## Use Cases
In scope for Semantic Kernel V1.0 is the ability to select AI Service and Model Request Settings:
1. By service id.
* A Service id uniquely identifies a registered AI Service and is typically defined in the scope of an application.
1. By developer defined strategy.
* A _developer defined strategy_ is a code first approach where a developer provides the logic.
1. By model id.
* A model id uniquely identifies a Large Language Model. Multiple AI service providers can support the same LLM.
1. By arbitrary AI service attributes
* E.g. an AI service can define a provider id which uniquely identifies an AI provider e.g. "Azure OpenAI", "OpenAI", "Hugging Face"
**This ADR focuses on items 1 & 2 in the above list. To implement 3 & 4 we need to provide the ability to store `AIService` metadata.**
## Decision Outcome
Support use cases 1 & 2 listed in this ADR and create separate ADR to add support for AI service metadata.
## Descriptions of the Use Cases
**Note: All code is pseudo code and does not accurately reflect what the final implementations will look like.**
### Select Model Request Settings by Service Id
_As a developer using the Semantic Kernel I can configure multiple request settings for a semantic function and associate each one with a service id so that the correct request settings are used when different services are used to execute my semantic function._
The semantic function template configuration allows multiple model request settings to be configured. In this case the developer configures different settings based on the service id that is used to execute the semantic function.
In the example below the semantic function is executed with "AzureText" using `max_tokens=60` because "AzureText" is the first service id in the list of models configured for the prompt.
```csharp
// Configure a Kernel with multiple LLM's
IKernel kernel = new KernelBuilder()
.WithLoggerFactory(ConsoleLogger.LoggerFactory)
.WithAzureTextCompletionService(deploymentName: aoai.DeploymentName,
endpoint: aoai.Endpoint, serviceId: "AzureText", apiKey: aoai.ApiKey)
.WithAzureChatCompletionService(deploymentName: aoai.ChatDeploymentName,
endpoint: aoai.Endpoint, serviceId: "AzureChat", apiKey: aoai.ApiKey)
.WithOpenAITextCompletionService(modelId: oai.ModelId,
serviceId: "OpenAIText", apiKey: oai.ApiKey, setAsDefault: true)
.WithOpenAIChatCompletionService(modelId: oai.ChatModelId,
serviceId: "OpenAIChat", apiKey: oai.ApiKey, setAsDefault: true)
.Build();
// Configure semantic function with multiple LLM request settings
var modelSettings = new List<AIRequestSettings>
{
new OpenAIRequestSettings() { ServiceId = "AzureText", MaxTokens = 60 },
new OpenAIRequestSettings() { ServiceId = "AzureChat", MaxTokens = 120 },
new OpenAIRequestSettings() { ServiceId = "OpenAIText", MaxTokens = 180 },
new OpenAIRequestSettings() { ServiceId = "OpenAIChat", MaxTokens = 240 }
};
var prompt = "Hello AI, what can you do for me?";
var promptTemplateConfig = new PromptTemplateConfig() { ModelSettings = modelSettings };
var func = kernel.CreateSemanticFunction(prompt, config: promptTemplateConfig, "HelloAI");
// Semantic function is executed with AzureText using max_tokens=60
result = await kernel.RunAsync(func);
```
This works by using the `IAIServiceSelector` interface as the strategy for selecting the AI service and request settings to user when invoking a semantic function.
The interface is defined as follows:
```csharp
public interface IAIServiceSelector
{
(T?, AIRequestSettings?) SelectAIService<T>(
string renderedPrompt,
IAIServiceProvider serviceProvider,
IReadOnlyList<AIRequestSettings>? modelSettings) where T : IAIService;
}
```
A default `OrderedIAIServiceSelector` implementation is provided which selects the AI service based on the order of the model request settings defined for the semantic function.
* The implementation checks if a service exists which the corresponding service id and if it does it and the associated model request settings will be used.
* In no model request settings are defined then the default text completion service is used.
* A default set of request settings can be specified by leaving the service id undefined or empty, the first such default will be used.
* If no default if specified and none of the specified services are available the operation will fail.
### Select AI Service and Model Request Settings By Developer Defined Strategy
_As a developer using the Semantic Kernel I can provide an implementation which selects the AI service and request settings used to execute my function so that I can dynamically control which AI service and settings are used to execute my semantic function._
In this case the developer configures different settings based on the service id and provides an AI Service Selector which determines which AI Service will be used when the semantic function is executed.
In the example below the semantic function is executed with whatever AI Service and AI Request Settings `MyAIServiceSelector` returns e.g. it will be possible to create an AI Service Selector that computes the token count of the rendered prompt and uses that to determine which service to use.
```csharp
// Configure a Kernel with multiple LLM's
IKernel kernel = new KernelBuilder()
.WithLoggerFactory(ConsoleLogger.LoggerFactory)
.WithAzureTextCompletionService(deploymentName: aoai.DeploymentName,
endpoint: aoai.Endpoint, serviceId: "AzureText", apiKey: aoai.ApiKey)
.WithAzureChatCompletionService(deploymentName: aoai.ChatDeploymentName,
endpoint: aoai.Endpoint, serviceId: "AzureChat", apiKey: aoai.ApiKey)
.WithOpenAITextCompletionService(modelId: oai.ModelId,
serviceId: "OpenAIText", apiKey: oai.ApiKey, setAsDefault: true)
.WithOpenAIChatCompletionService(modelId: oai.ChatModelId,
serviceId: "OpenAIChat", apiKey: oai.ApiKey, setAsDefault: true)
.WithAIServiceSelector(new MyAIServiceSelector())
.Build();
// Configure semantic function with multiple LLM request settings
var modelSettings = new List<AIRequestSettings>
{
new OpenAIRequestSettings() { ServiceId = "AzureText", MaxTokens = 60 },
new OpenAIRequestSettings() { ServiceId = "AzureChat", MaxTokens = 120 },
new OpenAIRequestSettings() { ServiceId = "OpenAIText", MaxTokens = 180 },
new OpenAIRequestSettings() { ServiceId = "OpenAIChat", MaxTokens = 240 }
};
var prompt = "Hello AI, what can you do for me?";
var promptTemplateConfig = new PromptTemplateConfig() { ModelSettings = modelSettings };
var func = kernel.CreateSemanticFunction(prompt, config: promptTemplateConfig, "HelloAI");
// Semantic function is executed with AI Service and AI request Settings dynamically determined
result = await kernel.RunAsync(func, funcVariables);
```
## More Information
### Select AI Service by Service Id
The following use case is supported. Developers can create a `Kernel`` instance with multiple named AI services. When invoking a semantic function the service id (and optionally request settings to be used) can be specified. The named AI service will be used to execute the prompt.
```csharp
var aoai = TestConfiguration.AzureOpenAI;
var oai = TestConfiguration.OpenAI;
// Configure a Kernel with multiple LLM's
IKernel kernel = Kernel.Builder
.WithLoggerFactory(ConsoleLogger.LoggerFactory)
.WithAzureTextCompletionService(deploymentName: aoai.DeploymentName,
endpoint: aoai.Endpoint, serviceId: "AzureText", apiKey: aoai.ApiKey)
.WithAzureChatCompletionService(deploymentName: aoai.ChatDeploymentName,
endpoint: aoai.Endpoint, serviceId: "AzureChat", apiKey: aoai.ApiKey)
.WithOpenAITextCompletionService(modelId: oai.ModelId,
serviceId: "OpenAIText", apiKey: oai.ApiKey)
.WithOpenAIChatCompletionService(modelId: oai.ChatModelId,
serviceId: "OpenAIChat", apiKey: oai.ApiKey)
.Build();
// Invoke the semantic function and service and request settings to use
result = await kernel.InvokeSemanticFunctionAsync(prompt,
requestSettings: new OpenAIRequestSettings()
{ ServiceId = "AzureText", MaxTokens = 60 });
result = await kernel.InvokeSemanticFunctionAsync(prompt,
requestSettings: new OpenAIRequestSettings()
{ ServiceId = "AzureChat", MaxTokens = 120 });
result = await kernel.InvokeSemanticFunctionAsync(prompt,
requestSettings: new OpenAIRequestSettings()
{ ServiceId = "OpenAIText", MaxTokens = 180 });
result = await kernel.InvokeSemanticFunctionAsync(prompt,
requestSettings: new OpenAIRequestSettings()
{ ServiceId = "OpenAIChat", MaxTokens = 240 });
```
docs/decisions/0017-custom-prompt-template-formats.md
+284
View File
@@ -0,0 +1,284 @@
---
status: approved
contact: markwallace-microsoft
date: 2023-10-26
deciders: mabolan, markwallace-microsoft, semenshi, rbarreto
consulted: dmytrostruk
informed:
---
# Custom Prompt Template Formats
## Context and Problem Statement
Semantic Kernel currently supports a custom prompt template language that allows for variable interpolation and function execution.
Semantic Kernel allows for custom prompt template formats to be integrated e.g., prompt templates using [Handlebars](https://handlebarsjs.com/) syntax.
The purpose of this ADR is to describe how a custom prompt template formats will be supported in the Semantic Kernel.
### Current Design
By default the `Kernel` uses the `BasicPromptTemplateEngine` which supports the Semantic Kernel specific template format.
#### Code Patterns
Below is an expanded example of how to create a semantic function from a prompt template string which uses the built-in Semantic Kernel format:
```csharp
IKernel kernel = Kernel.Builder
.WithPromptTemplateEngine(new BasicPromptTemplateEngine())
.WithOpenAIChatCompletionService(
modelId: openAIModelId,
apiKey: openAIApiKey)
.Build();
kernel.ImportFunctions(new TimePlugin(), "time");
string templateString = "Today is: {{time.Date}} Is it weekend time (weekend/not weekend)?";
var promptTemplateConfig = new PromptTemplateConfig();
var promptTemplate = new PromptTemplate(templateString, promptTemplateConfig, kernel.PromptTemplateEngine);
var kindOfDay = kernel.RegisterSemanticFunction("KindOfDay", promptTemplateConfig, promptTemplate);
var result = await kernel.RunAsync(kindOfDay);
Console.WriteLine(result.GetValue<string>());
```
We have an extension method `var kindOfDay = kernel.CreateSemanticFunction(promptTemplate);` to simplify the process to create and register a semantic function but the expanded format is shown above to highlight the dependency on `kernel.PromptTemplateEngine`.
Also the `BasicPromptTemplateEngine` is the default prompt template engine and will be loaded automatically if the package is available and no other prompt template engine is specified.
Some issues with this:
1. You need to have a `Kernel` instance to create a semantic function, which is contrary to one of the goals of allow semantic functions to be created once and reused across multiple `Kernel` instances.
1. `Kernel` only supports a single `IPromptTemplateEngine` so we cannot support using multiple prompt templates at the same time.
1. `IPromptTemplateEngine` is stateless and must perform a parse of the template for each render
1. Our semantic function extension methods rely on our implementation of `IPromptTemplate` (i.e., `PromptTemplate`) which stores the template string and uses the `IPromptTemplateEngine` to render it every time. Note implementations of `IPromptTemplate` are currently stateful as they also store the parameters.
#### Performance
The `BasicPromptTemplateEngine` uses the `TemplateTokenizer` to parse the template i.e. extract the blocks.
Then it renders the template i.e. inserts variables and executes functions. Some sample timings for these operations:
| Operation | Ticks | Milliseconds |
|------------------|---------|--------------|
| Extract blocks | 1044427 | 103 |
| Render variables | 168 | 0 |
Sample template used was: `"{{variable1}} {{variable2}} {{variable3}} {{variable4}} {{variable5}}"`
**Note: We will use the sample implementation to support the f-string template format.**
Using `HandlebarsDotNet` for the same use case results in the following timings:
| Operation | Ticks | Milliseconds |
|------------------|---------|--------------|
| Compile template | 66277 | 6 |
| Render variables | 4173 | 0 |
**By separating the extract blocks/compile from the render variables operation it will be possible to optimise performance by compiling templates just once.**
#### Implementing a Custom Prompt Template Engine
There are two interfaces provided:
```csharp
public interface IPromptTemplateEngine
{
Task<string> RenderAsync(string templateText, SKContext context, CancellationToken cancellationToken = default);
}
public interface IPromptTemplate
{
IReadOnlyList<ParameterView> Parameters { get; }
public Task<string> RenderAsync(SKContext executionContext, CancellationToken cancellationToken = default);
}
```
A prototype implementation of a handlebars prompt template engine could look something like this:
```csharp
public class HandlebarsTemplateEngine : IPromptTemplateEngine
{
private readonly ILoggerFactory _loggerFactory;
public HandlebarsTemplateEngine(ILoggerFactory? loggerFactory = null)
{
this._loggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
}
public async Task<string> RenderAsync(string templateText, SKContext context, CancellationToken cancellationToken = default)
{
var handlebars = HandlebarsDotNet.Handlebars.Create();
var functionViews = context.Functions.GetFunctionViews();
foreach (FunctionView functionView in functionViews)
{
var skfunction = context.Functions.GetFunction(functionView.PluginName, functionView.Name);
handlebars.RegisterHelper($"{functionView.PluginName}_{functionView.Name}", async (writer, hcontext, parameters) =>
{
var result = await skfunction.InvokeAsync(context).ConfigureAwait(true);
writer.WriteSafeString(result.GetValue<string>());
});
}
var template = handlebars.Compile(templateText);
var prompt = template(context.Variables);
return await Task.FromResult(prompt).ConfigureAwait(true);
}
}
```
**Note: This is just a prototype implementation for illustration purposes only.**
Some issues:
1. The `IPromptTemplate` interface is not used and causes confusion.
1. There is no way to allow developers to support multiple prompt template formats at the same time.
There is one implementation of `IPromptTemplate` provided in the Semantic Kernel core package.
The `RenderAsync` implementation just delegates to the `IPromptTemplateEngine`.
The `Parameters` list get's populated with the parameters defined in the `PromptTemplateConfig` and any missing variables defined in the template.
#### Handlebars Considerations
Handlebars does not support dynamic binding of helpers. Consider the following snippet:
```csharp
HandlebarsHelper link_to = (writer, context, parameters) =>
{
writer.WriteSafeString($"<a href='{context["url"]}'>{context["text"]}</a>");
};
string source = @"Click here: {{link_to}}";
var data = new
{
url = "https://github.com/rexm/handlebars.net",
text = "Handlebars.Net"
};
// Act
var handlebars = HandlebarsDotNet.Handlebars.Create();
handlebars.RegisterHelper("link_to", link_to);
var template = handlebars1.Compile(source);
// handlebars.RegisterHelper("link_to", link_to); This also works
var result = template1(data);
```
Handlebars allows the helpers to be registered with the `Handlebars` instance either before or after a template is compiled.
The optimum would be to have a shared `Handlebars` instance for a specific collection of functions and register the helpers just once.
For use cases where the Kernel function collection may have been mutated we will be forced to create a `Handlebars` instance at render time
and then register the helpers. This means we cannot take advantage of the performance improvement provided by compiling the template.
## Decision Drivers
In no particular order:
* Support creating a semantic function without a `IKernel`instance.
* Support late binding of functions i.e., having functions resolved when the prompt is rendered.
* Support allowing the prompt template to be parsed (compiled) just once to optimize performance if needed.
* Support using multiple prompt template formats with a single `Kernel` instance.
* Provide simple abstractions which allow third parties to implement support for custom prompt template formats.
## Considered Options
* Obsolete `IPromptTemplateEngine` and replace with `IPromptTemplateFactory`.
### Obsolete `IPromptTemplateEngine` and replace with `IPromptTemplateFactory`
<img src="./diagrams/prompt-template-factory.png" alt="ISKFunction class relationships"/>
Below is an expanded example of how to create a semantic function from a prompt template string which uses the built-in Semantic Kernel format:
```csharp
// Semantic function can be created once
var promptTemplateFactory = new BasicPromptTemplateFactory();
string templateString = "Today is: {{time.Date}} Is it weekend time (weekend/not weekend)?";
var promptTemplate = promptTemplateFactory.CreatePromptTemplate(templateString, new PromptTemplateConfig());
var kindOfDay = ISKFunction.CreateSemanticFunction("KindOfDay", promptTemplateConfig, promptTemplate)
// Create Kernel after creating the semantic function
// Later we will support passing a function collection to the KernelBuilder
IKernel kernel = Kernel.Builder
.WithOpenAIChatCompletionService(
modelId: openAIModelId,
apiKey: openAIApiKey)
.Build();
kernel.ImportFunctions(new TimePlugin(), "time");
// Optionally register the semantic function with the Kernel
// kernel.RegisterCustomFunction(kindOfDay);
var result = await kernel.RunAsync(kindOfDay);
Console.WriteLine(result.GetValue<string>());
```
**Notes:**
* `BasicPromptTemplateFactory` will be the default implementation and will be automatically provided in `KernelSemanticFunctionExtensions`. Developers will also be able to provide their own implementation.
* The factory uses the new `PromptTemplateConfig.TemplateFormat` to create the appropriate `IPromptTemplate` instance.
* We should look to remove `promptTemplateConfig` as a parameter to `CreateSemanticFunction`. That change is outside of the scope of this ADR.
The `BasicPromptTemplateFactory` and `BasicPromptTemplate` implementations look as follows:
```csharp
public sealed class BasicPromptTemplateFactory : IPromptTemplateFactory
{
private readonly IPromptTemplateFactory _promptTemplateFactory;
private readonly ILoggerFactory _loggerFactory;
public BasicPromptTemplateFactory(IPromptTemplateFactory promptTemplateFactory, ILoggerFactory? loggerFactory = null)
{
this._promptTemplateFactory = promptTemplateFactory;
this._loggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
}
public IPromptTemplate? CreatePromptTemplate(string templateString, PromptTemplateConfig promptTemplateConfig)
{
if (promptTemplateConfig.TemplateFormat.Equals(PromptTemplateConfig.SEMANTICKERNEL, System.StringComparison.Ordinal))
{
return new BasicPromptTemplate(templateString, promptTemplateConfig, this._loggerFactory);
}
else if (this._promptTemplateFactory is not null)
{
return this._promptTemplateFactory.CreatePromptTemplate(templateString, promptTemplateConfig);
}
throw new SKException($"Invalid prompt template format {promptTemplateConfig.TemplateFormat}");
}
}
public sealed class BasicPromptTemplate : IPromptTemplate
{
public BasicPromptTemplate(string templateString, PromptTemplateConfig promptTemplateConfig, ILoggerFactory? loggerFactory = null)
{
this._loggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
this._logger = this._loggerFactory.CreateLogger(typeof(BasicPromptTemplate));
this._templateString = templateString;
this._promptTemplateConfig = promptTemplateConfig;
this._parameters = new(() => this.InitParameters());
this._blocks = new(() => this.ExtractBlocks(this._templateString));
this._tokenizer = new TemplateTokenizer(this._loggerFactory);
}
public IReadOnlyList<ParameterView> Parameters => this._parameters.Value;
public async Task<string> RenderAsync(SKContext executionContext, CancellationToken cancellationToken = default)
{
return await this.RenderAsync(this._blocks.Value, executionContext, cancellationToken).ConfigureAwait(false);
}
// Not showing the implementation details
}
```
**Note:**
* The call to `ExtractBlocks` is called lazily once for each prompt template
* The `RenderAsync` doesn't need to extract the blocks every time
## Decision Outcome
Chosen option: "Obsolete `IPromptTemplateEngine` and replace with `IPromptTemplateFactory`", because
addresses the requirements and provides good flexibility for the future.
docs/decisions/diagrams/prompt-template-factory.mmd
+36
View File
@@ -0,0 +1,36 @@
---
title: IPromptTemplateFactory
---
classDiagram
%% Use https://mermaid.live/ to preview this diagram. The VS Code extension does not handle namespaces.
direction BT
namespace SemanticKernel {
class IPromptTemplate {
<<interface>>
IReadOnlyList<ParameterView> Parameters;
public Task<string> RenderAsync(SKContext executionContext, CancellationToken cancellationToken = default);
}
class IPromptTemplateFactory {
<<interface>>
IPromptTemplate CreatePromptTemplate(string template, PromptTemplateConfig config)
}
}
namespace TemplateEngine_Basic {
class BasicPromptTemplate
class FStringPromptTemplate
class BasicPromptTemplateFactory
}
namespace TemplateEngine_Handlebars {
class HandlebarsPromptTemplate
class HandlebarsPromptTemplateFactory
}
IPromptTemplateFactory <|.. BasicPromptTemplateFactory
IPromptTemplateFactory <|.. HandlebarsPromptTemplateFactory
IPromptTemplate <|.. BasicPromptTemplate
IPromptTemplate <|.. FStringPromptTemplate
IPromptTemplate <|.. HandlebarsPromptTemplate
docs/decisions/diagrams/prompt-template-factory.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 76 KiB

dotnet/Directory.Packages.props
+2 -2
View File
@@ -6,7 +6,7 @@
</PropertyGroup>
<ItemGroup>
<PackageVersion Include="Azure.AI.OpenAI" Version="1.0.0-beta.8" />
<PackageVersion Include="Azure.Identity" Version="1.10.2" />
<PackageVersion Include="Azure.Identity" Version="1.10.3" />
<PackageVersion Include="Azure.Search.Documents" Version="11.5.0-beta.5" />
<PackageVersion Include="Microsoft.ApplicationInsights.WorkerService" Version="2.21.0" />
<PackageVersion Include="Microsoft.Azure.Kusto.Data" Version="11.3.4" />
@@ -55,7 +55,7 @@
<PackageVersion Include="Microsoft.OpenApi.Readers" Version="[1.6.3, )" />
<PackageVersion Include="Newtonsoft.Json" Version="[13.0.3, )" />
<PackageVersion Include="Google.Apis.CustomSearchAPI.v1" Version="[1.60.0.3001, )" />
<PackageVersion Include="Grpc.Net.Client" Version="2.57.0" />
<PackageVersion Include="Grpc.Net.Client" Version="2.58.0" />
<PackageVersion Include="protobuf-net" Version="3.2.26" />
<PackageVersion Include="protobuf-net.Reflection" Version="3.2.12" />
<PackageVersion Include="CoreCLR-NCalc" Version="2.2.113" />
dotnet/SK-dotnet.sln
-9
View File
@@ -140,8 +140,6 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Plugins.Core", "src\Plugins
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "NCalcPlugins", "samples\NCalcPlugins\NCalcPlugins.csproj", "{E6EDAB8F-3406-4DBF-9AAB-DF40DC2CA0FA}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Connectors.AI.Oobabooga", "src\Connectors\Connectors.AI.Oobabooga\Connectors.AI.Oobabooga.csproj", "{677F1381-7830-4115-9C1A-58B282629DC6}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "ApplicationInsightsExample", "samples\ApplicationInsightsExample\ApplicationInsightsExample.csproj", "{C754950A-E16C-4F96-9CC7-9328E361B5AF}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Connectors.Memory.Kusto", "src\Connectors\Connectors.Memory.Kusto\Connectors.Memory.Kusto.csproj", "{E07608CC-D710-4655-BB9E-D22CF3CDD193}"
@@ -348,12 +346,6 @@ Global
{E6EDAB8F-3406-4DBF-9AAB-DF40DC2CA0FA}.Publish|Any CPU.ActiveCfg = Release|Any CPU
{E6EDAB8F-3406-4DBF-9AAB-DF40DC2CA0FA}.Release|Any CPU.ActiveCfg = Release|Any CPU
{E6EDAB8F-3406-4DBF-9AAB-DF40DC2CA0FA}.Release|Any CPU.Build.0 = Release|Any CPU
{677F1381-7830-4115-9C1A-58B282629DC6}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{677F1381-7830-4115-9C1A-58B282629DC6}.Debug|Any CPU.Build.0 = Debug|Any CPU
{677F1381-7830-4115-9C1A-58B282629DC6}.Publish|Any CPU.ActiveCfg = Publish|Any CPU
{677F1381-7830-4115-9C1A-58B282629DC6}.Publish|Any CPU.Build.0 = Publish|Any CPU
{677F1381-7830-4115-9C1A-58B282629DC6}.Release|Any CPU.ActiveCfg = Release|Any CPU
{677F1381-7830-4115-9C1A-58B282629DC6}.Release|Any CPU.Build.0 = Release|Any CPU
{C754950A-E16C-4F96-9CC7-9328E361B5AF}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C754950A-E16C-4F96-9CC7-9328E361B5AF}.Debug|Any CPU.Build.0 = Debug|Any CPU
{C754950A-E16C-4F96-9CC7-9328E361B5AF}.Publish|Any CPU.ActiveCfg = Release|Any CPU
@@ -460,7 +452,6 @@ Global
{3CDE10B2-AE8F-4FC4-8D55-92D4AD32E144} = {958AD708-F048-4FAF-94ED-D2F2B92748B9}
{0D0C4DAD-E6BC-4504-AE3A-EEA4E35920C1} = {D6D598DF-C17C-46F4-B2B9-CDE82E2DE132}
{E6EDAB8F-3406-4DBF-9AAB-DF40DC2CA0FA} = {FA3720F1-C99A-49B2-9577-A940257098BF}
{677F1381-7830-4115-9C1A-58B282629DC6} = {0247C2C9-86C3-45BA-8873-28B0948EDC0C}
{C754950A-E16C-4F96-9CC7-9328E361B5AF} = {FA3720F1-C99A-49B2-9577-A940257098BF}
{E07608CC-D710-4655-BB9E-D22CF3CDD193} = {0247C2C9-86C3-45BA-8873-28B0948EDC0C}
{10E4B697-D4E8-468D-872D-49670FD150FB} = {078F96B4-09E1-4E0E-B214-F71A4F4BF633}
dotnet/nuget/nuget-package.props
+1 -1
View File
@@ -1,7 +1,7 @@
<Project>
<PropertyGroup>
<!-- Central version prefix - applies to all nuget packages. -->
<Version>1.0.0-beta3</Version>
<Version>1.0.0-beta4</Version>
<Configurations>Debug;Release;Publish</Configurations>
<IsPackable>true</IsPackable>
dotnet/samples/KernelSyntaxExamples/Example06_TemplateLanguage.cs
+1 -1
View File
@@ -28,7 +28,7 @@ public static class Example06_TemplateLanguage
return;
}
IKernel kernel = Kernel.Builder
IKernel kernel = new KernelBuilder()
.WithLoggerFactory(ConsoleLogger.LoggerFactory)
.WithOpenAIChatCompletionService(
modelId: openAIModelId,
dotnet/samples/KernelSyntaxExamples/Example08_RetryHandler.cs
+1 -1
View File
@@ -75,7 +75,7 @@ public static class Example08_RetryHandler
private static KernelBuilder InitializeKernelBuilder()
{
return Kernel.Builder
return new KernelBuilder()
.WithLoggerFactory(InfoLogger.LoggerFactory)
// OpenAI settings - you can set the OpenAI.ApiKey to an invalid value to see the retry policy in play
.WithOpenAIChatCompletionService(TestConfiguration.OpenAI.ChatModelId, "BAD_KEY");
dotnet/samples/KernelSyntaxExamples/Example09_FunctionTypes.cs
+1 -1
View File
@@ -15,7 +15,7 @@ public static class Example09_FunctionTypes
{
Console.WriteLine("======== Native function types ========");
var kernel = Kernel.Builder
var kernel = new KernelBuilder()
.WithLoggerFactory(ConsoleLogger.LoggerFactory)
.WithOpenAIChatCompletionService(TestConfiguration.OpenAI.ChatModelId, TestConfiguration.OpenAI.ApiKey)
.Build();
dotnet/samples/KernelSyntaxExamples/Example10_DescribeAllPluginsAndFunctions.cs
+1 -1
View File
@@ -20,7 +20,7 @@ public static class Example10_DescribeAllPluginsAndFunctions
{
Console.WriteLine("======== Describe all plugins and functions ========");
var kernel = Kernel.Builder
var kernel = new KernelBuilder()
.WithOpenAIChatCompletionService(
modelId: TestConfiguration.OpenAI.ChatModelId,
apiKey: TestConfiguration.OpenAI.ApiKey)
dotnet/samples/KernelSyntaxExamples/Example11_WebSearchQueries.cs
+1 -1
View File
@@ -13,7 +13,7 @@ public static class Example11_WebSearchQueries
{
Console.WriteLine("======== WebSearchQueries ========");
IKernel kernel = Kernel.Builder.WithLoggerFactory(ConsoleLogger.LoggerFactory).Build();
IKernel kernel = new KernelBuilder().WithLoggerFactory(ConsoleLogger.LoggerFactory).Build();
// Load native plugins
var plugin = new SearchUrlPlugin();
dotnet/samples/KernelSyntaxExamples/Example13_ConversationSummaryPlugin.cs
+1 -1
View File
@@ -177,7 +177,7 @@ Jane: Goodbye!
private static IKernel InitializeKernel()
{
IKernel kernel = Kernel.Builder
IKernel kernel = new KernelBuilder()
.WithLoggerFactory(ConsoleLogger.LoggerFactory)
.WithAzureChatCompletionService(
TestConfiguration.AzureOpenAI.ChatDeploymentName,
dotnet/samples/KernelSyntaxExamples/Example15_TextMemoryPlugin.cs
+1 -1
View File
@@ -141,7 +141,7 @@ public static class Example15_TextMemoryPlugin
private static async Task RunWithStoreAsync(IMemoryStore memoryStore, CancellationToken cancellationToken)
{
var kernel = Kernel.Builder
var kernel = new KernelBuilder()
.WithLoggerFactory(ConsoleLogger.LoggerFactory)
.WithOpenAIChatCompletionService(TestConfiguration.OpenAI.ChatModelId, TestConfiguration.OpenAI.ApiKey)
.WithOpenAITextEmbeddingGenerationService(TestConfiguration.OpenAI.EmbeddingModelId, TestConfiguration.OpenAI.ApiKey)
dotnet/samples/KernelSyntaxExamples/Example40_DIContainer.cs
+1 -1
View File
@@ -44,7 +44,7 @@ public static class Example40_DIContainer
//Registering Kernel
collection.AddTransient<IKernel>((serviceProvider) =>
{
return Kernel.Builder
return new KernelBuilder()
.WithLoggerFactory(serviceProvider.GetRequiredService<ILoggerFactory>())
.WithOpenAITextCompletionService(TestConfiguration.OpenAI.ModelId, TestConfiguration.OpenAI.ApiKey)
.Build();
dotnet/samples/KernelSyntaxExamples/Example41_HttpClientUsage.cs
+4 -4
View File
@@ -32,7 +32,7 @@ public static class Example41_HttpClientUsage
/// </summary>
private static void UseDefaultHttpClient()
{
var kernel = Kernel.Builder
var kernel = new KernelBuilder()
.WithOpenAIChatCompletionService(
modelId: TestConfiguration.OpenAI.ChatModelId,
apiKey: TestConfiguration.OpenAI.ApiKey) // If you need to use the default HttpClient from the SK SDK, simply omit the argument for the httpMessageInvoker parameter.
@@ -47,7 +47,7 @@ public static class Example41_HttpClientUsage
using var httpClient = new HttpClient();
// If you need to use a custom HttpClient, simply pass it as an argument for the httpClient parameter.
var kernel = Kernel.Builder
var kernel = new KernelBuilder()
.WithOpenAIChatCompletionService(
modelId: TestConfiguration.OpenAI.ModelId,
apiKey: TestConfiguration.OpenAI.ApiKey,
@@ -68,7 +68,7 @@ public static class Example41_HttpClientUsage
{
var factory = sp.GetRequiredService<IHttpClientFactory>();
var kernel = Kernel.Builder
var kernel = new KernelBuilder()
.WithOpenAIChatCompletionService(
modelId: TestConfiguration.OpenAI.ChatModelId,
apiKey: TestConfiguration.OpenAI.ApiKey,
@@ -99,7 +99,7 @@ public static class Example41_HttpClientUsage
{
var factory = sp.GetRequiredService<IHttpClientFactory>();
var kernel = Kernel.Builder
var kernel = new KernelBuilder()
.WithOpenAIChatCompletionService(
modelId: TestConfiguration.OpenAI.ChatModelId,
apiKey: TestConfiguration.OpenAI.ApiKey,
dotnet/samples/KernelSyntaxExamples/Example42_KernelBuilder.cs
+11 -11
View File
@@ -2,7 +2,7 @@
// ==========================================================================================================
// The easier way to instantiate the Semantic Kernel is to use KernelBuilder.
// You can access the builder using either Kernel.Builder or KernelBuilder.
// You can access the builder using new KernelBuilder().
#pragma warning disable CA1852
@@ -39,20 +39,20 @@ public static class Example42_KernelBuilder
string azureOpenAIEmbeddingDeployment = TestConfiguration.AzureOpenAIEmbeddings.DeploymentName;
#pragma warning disable CA1852 // Seal internal types
IKernel kernel1 = Kernel.Builder.Build();
IKernel kernel1 = new KernelBuilder().Build();
#pragma warning restore CA1852 // Seal internal types
IKernel kernel2 = Kernel.Builder.Build();
IKernel kernel2 = new KernelBuilder().Build();
// ==========================================================================================================
// Kernel.Builder returns a new builder instance, in case you want to configure the builder differently.
// new KernelBuilder() returns a new builder instance, in case you want to configure the builder differently.
// The following are 3 distinct builder instances.
var builder1 = new KernelBuilder();
var builder2 = Kernel.Builder;
var builder2 = new KernelBuilder();
var builder3 = Kernel.Builder;
var builder3 = new KernelBuilder();
// ==========================================================================================================
// A builder instance can create multiple kernel instances, e.g. in case you need
@@ -108,7 +108,7 @@ public static class Example42_KernelBuilder
// ==========================================================================================================
// The AI services are defined with the builder
var kernel7 = Kernel.Builder
var kernel7 = new KernelBuilder()
.WithAzureChatCompletionService(
deploymentName: azureOpenAIChatCompletionDeployment,
endpoint: azureOpenAIEndpoint,
@@ -121,7 +121,7 @@ public static class Example42_KernelBuilder
// The default behavior can be configured or a custom retry handler can be injected that will apply to all
// AI requests (when using the kernel).
var kernel8 = Kernel.Builder.WithRetryBasic(
var kernel8 = new KernelBuilder().WithRetryBasic(
new BasicRetryConfig
{
MaxRetryCount = 3,
@@ -147,11 +147,11 @@ public static class Example42_KernelBuilder
(ex, timespan, retryCount, _)
=> logger?.LogWarning(ex, "Error executing action [attempt {RetryCount} of 3], pausing {PausingMilliseconds}ms", retryCount, timespan.TotalMilliseconds));
var kernel9 = Kernel.Builder.WithHttpHandlerFactory(new PollyHttpRetryHandlerFactory(retryThreeTimesPolicy)).Build();
var kernel9 = new KernelBuilder().WithHttpHandlerFactory(new PollyHttpRetryHandlerFactory(retryThreeTimesPolicy)).Build();
var kernel10 = Kernel.Builder.WithHttpHandlerFactory(new PollyRetryThreeTimesFactory()).Build();
var kernel10 = new KernelBuilder().WithHttpHandlerFactory(new PollyRetryThreeTimesFactory()).Build();
var kernel11 = Kernel.Builder.WithHttpHandlerFactory(new MyCustomHandlerFactory()).Build();
var kernel11 = new KernelBuilder().WithHttpHandlerFactory(new MyCustomHandlerFactory()).Build();
return Task.CompletedTask;
}
dotnet/samples/KernelSyntaxExamples/Example52_ApimAuth.cs
+1 -1
View File
@@ -59,7 +59,7 @@ public static class Example52_ApimAuth
.AddConsole();
});
var kernel = Kernel.Builder
var kernel = new KernelBuilder()
.WithLoggerFactory(loggerFactory)
.WithAIService<IChatCompletion>(TestConfiguration.AzureOpenAI.ChatDeploymentName, (loggerFactory) =>
new AzureChatCompletion(TestConfiguration.AzureOpenAI.ChatDeploymentName, openAIClient, loggerFactory))
dotnet/samples/KernelSyntaxExamples/Example56_TemplateNativeFunctionsWithMultipleArguments.cs
+1 -1
View File
@@ -30,7 +30,7 @@ public static class Example56_TemplateNativeFunctionsWithMultipleArguments
return;
}
IKernel kernel = Kernel.Builder
IKernel kernel = new KernelBuilder()
.WithLoggerFactory(ConsoleLogger.LoggerFactory)
.WithAzureChatCompletionService(
deploymentName: deploymentName,
dotnet/samples/KernelSyntaxExamples/Example58_ConfigureRequestSettings.cs
+1 -1
View File
@@ -29,7 +29,7 @@ public static class Example58_ConfigureRequestSettings
return;
}
IKernel kernel = Kernel.Builder
IKernel kernel = new KernelBuilder()
.WithLoggerFactory(ConsoleLogger.LoggerFactory)
.WithAzureChatCompletionService(
deploymentName: chatDeploymentName,
dotnet/samples/KernelSyntaxExamples/Example61_MultipleLLMs.cs
+69
View File
@@ -0,0 +1,69 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Threading.Tasks;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.AI;
using RepoUtils;
// ReSharper disable once InconsistentNaming
public static class Example61_MultipleLLMs
{
/// <summary>
/// Show how to run a semantic function and specify a specific service to use.
/// </summary>
public static async Task RunAsync()
{
Console.WriteLine("======== Example61_MultipleLLMs ========");
string apiKey = TestConfiguration.AzureOpenAI.ApiKey;
string chatDeploymentName = TestConfiguration.AzureOpenAI.ChatDeploymentName;
string endpoint = TestConfiguration.AzureOpenAI.Endpoint;
if (apiKey == null || chatDeploymentName == null || endpoint == null)
{
Console.WriteLine("Azure endpoint, apiKey, or deploymentName not found. Skipping example.");
return;
}
string openAIModelId = TestConfiguration.OpenAI.ChatModelId;
string openAIApiKey = TestConfiguration.OpenAI.ApiKey;
if (openAIModelId == null || openAIApiKey == null)
{
Console.WriteLine("OpenAI credentials not found. Skipping example.");
return;
}
IKernel kernel = new KernelBuilder()
.WithLoggerFactory(ConsoleLogger.LoggerFactory)
.WithAzureChatCompletionService(
deploymentName: chatDeploymentName,
endpoint: endpoint,
serviceId: "AzureOpenAIChat",
apiKey: apiKey)
.WithOpenAIChatCompletionService(
modelId: openAIModelId,
serviceId: "OpenAIChat",
apiKey: openAIApiKey)
.Build();
await RunSemanticFunctionAsync(kernel, "AzureOpenAIChat");
await RunSemanticFunctionAsync(kernel, "OpenAIChat");
}
public static async Task RunSemanticFunctionAsync(IKernel kernel, string serviceId)
{
Console.WriteLine($"======== {serviceId} ========");
var prompt = "Hello AI, what can you do for me?";
var result = await kernel.InvokeSemanticFunctionAsync(
prompt,
requestSettings: new AIRequestSettings()
{
ServiceId = serviceId
});
Console.WriteLine(result.GetValue<string>());
}
}
dotnet/samples/KernelSyntaxExamples/Example62_CustomAIServiceSelector.cs
+164
View File
@@ -0,0 +1,164 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Microsoft.ML.Tokenizers;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.AI;
using Microsoft.SemanticKernel.Connectors.AI.OpenAI;
using Microsoft.SemanticKernel.Diagnostics;
using Microsoft.SemanticKernel.Services;
using Microsoft.SemanticKernel.TemplateEngine;
using RepoUtils;
// ReSharper disable once InconsistentNaming
public static class Example62_CustomAIServiceSelector
{
/// <summary>
/// Show how to configure model request settings
/// </summary>
public static async Task RunAsync()
{
Console.WriteLine("======== Example61_CustomAIServiceSelector ========");
string apiKey = TestConfiguration.AzureOpenAI.ApiKey;
string chatDeploymentName = TestConfiguration.AzureOpenAI.ChatDeploymentName;
string endpoint = TestConfiguration.AzureOpenAI.Endpoint;
if (apiKey == null || chatDeploymentName == null || endpoint == null)
{
Console.WriteLine("Azure endpoint, apiKey, or deploymentName not found. Skipping example.");
return;
}
string openAIModelId = TestConfiguration.OpenAI.ChatModelId;
string openAIApiKey = TestConfiguration.OpenAI.ApiKey;
if (openAIModelId == null || openAIApiKey == null)
{
Console.WriteLine("OpenAI credentials not found. Skipping example.");
return;
}
IKernel kernel = new KernelBuilder()
.WithLoggerFactory(ConsoleLogger.LoggerFactory)
.WithAzureChatCompletionService(
deploymentName: chatDeploymentName,
endpoint: endpoint,
serviceId: "AzureOpenAIChat",
apiKey: apiKey)
.WithOpenAIChatCompletionService(
modelId: openAIModelId,
serviceId: "OpenAIChat",
apiKey: openAIApiKey)
.WithAIServiceSelector(new MyAIServiceSelector())
.Build();
var modelSettings = new List<AIRequestSettings>
{
new OpenAIRequestSettings() { ServiceId = "AzureOpenAIChat", MaxTokens = 400 },
new OpenAIRequestSettings() { ServiceId = "OpenAIChat", MaxTokens = 200 }
};
await RunSemanticFunctionAsync(kernel, "Hello AI, what can you do for me?", modelSettings);
await RunSemanticFunctionAsync(kernel, "Hello AI, provide an indepth description of what can you do for me as a bulleted list?", modelSettings);
}
public static async Task RunSemanticFunctionAsync(IKernel kernel, string prompt, List<AIRequestSettings> modelSettings)
{
Console.WriteLine($"======== {prompt} ========");
var promptTemplateConfig = new PromptTemplateConfig() { ModelSettings = modelSettings };
var promptTemplate = new PromptTemplate(prompt, promptTemplateConfig, kernel);
var skfunction = kernel.RegisterSemanticFunction(
"MyFunction",
promptTemplateConfig,
promptTemplate);
var result = await kernel.RunAsync(skfunction);
Console.WriteLine(result.GetValue<string>());
}
}
public class MyAIServiceSelector : IAIServiceSelector
{
private readonly int _defaultMaxTokens = 300;
private readonly int _minResponseTokens = 150;
public (T?, AIRequestSettings?) SelectAIService<T>(string renderedPrompt, IAIServiceProvider serviceProvider, IReadOnlyList<AIRequestSettings>? modelSettings) where T : IAIService
{
if (modelSettings is null || modelSettings.Count == 0)
{
var service = serviceProvider.GetService<T>(null);
if (service is not null)
{
return (service, null);
}
}
else
{
var tokens = this.CountTokens(renderedPrompt);
string? serviceId = null;
int fewestTokens = 0;
AIRequestSettings? requestSettings = null;
AIRequestSettings? defaultRequestSettings = null;
foreach (var model in modelSettings)
{
if (!string.IsNullOrEmpty(model.ServiceId))
{
if (model is OpenAIRequestSettings openAIModel)
{
var responseTokens = (openAIModel.MaxTokens ?? this._defaultMaxTokens) - tokens;
if (serviceId is null || (responseTokens > this._minResponseTokens && responseTokens < fewestTokens))
{
fewestTokens = responseTokens;
serviceId = model.ServiceId;
requestSettings = model;
}
}
}
else
{
// First request settings with empty or null service id is the default
defaultRequestSettings ??= model;
}
}
Console.WriteLine($"Prompt tokens: {tokens}, Response tokens: {fewestTokens}");
if (serviceId is not null)
{
Console.WriteLine($"Selected service: {serviceId}");
var service = serviceProvider.GetService<T>(serviceId);
if (service is not null)
{
return (service, requestSettings);
}
}
if (defaultRequestSettings is not null)
{
var service = serviceProvider.GetService<T>(null);
if (service is not null)
{
return (service, defaultRequestSettings);
}
}
}
throw new SKException("Unable to find AI service to handled request.");
}
/// <summary>
/// MicrosoftML token counter implementation.
/// </summary>
private int CountTokens(string input)
{
Tokenizer tokenizer = new(new Bpe());
var tokens = tokenizer.Encode(input).Tokens;
return tokens.Count;
}
}
dotnet/src/Connectors/Connectors.AI.Oobabooga/Connectors.AI.Oobabooga.csproj
-28
View File
@@ -1,28 +0,0 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<!-- THIS PROPERTY GROUP MUST COME FIRST -->
<AssemblyName>Microsoft.SemanticKernel.Connectors.AI.Oobabooga</AssemblyName>
<RootNamespace>$(AssemblyName)</RootNamespace>
<TargetFramework>netstandard2.0</TargetFramework>
</PropertyGroup>
<!-- IMPORT NUGET PACKAGE SHARED PROPERTIES -->
<Import Project="$(RepoRoot)/dotnet/nuget/nuget-package.props" />
<Import Project="$(RepoRoot)/dotnet/src/InternalUtilities/src/InternalUtilities.props" />
<PropertyGroup>
<!-- NuGet Package Settings -->
<Title>Semantic Kernel - Oobabooga Connector</Title>
<Description>Semantic Kernel connector for the oobabooga text-generation-webui open source project. Contains a client for text completion.</Description>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="System.Threading.Channels" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\SemanticKernel.Core\SemanticKernel.Core.csproj" />
</ItemGroup>
</Project>
dotnet/src/Connectors/Connectors.AI.Oobabooga/README.md
+3
View File
@@ -0,0 +1,3 @@
# Semantic Kernel Oobabooga AI Connector
This connector have moved, please go [here](https://github.com/MyIntelligenceAgency/semantic-fleet) for more information.
dotnet/src/Connectors/Connectors.AI.Oobabooga/TextCompletion/OobaboogaTextCompletion.cs
-481
View File
@@ -1,481 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net.Http;
using System.Net.WebSockets;
using System.Runtime.CompilerServices;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel.AI;
using Microsoft.SemanticKernel.AI.TextCompletion;
using Microsoft.SemanticKernel.Diagnostics;
namespace Microsoft.SemanticKernel.Connectors.AI.Oobabooga.TextCompletion;
/// <summary>
/// Oobabooga text completion service API.
/// Adapted from <see href="https://github.com/oobabooga/text-generation-webui/tree/main/api-examples"/>
/// </summary>
[Obsolete("This functionality is available as part of new NuGet package: https://www.nuget.org/packages/MyIA.SemanticKernel.Connectors.AI.Oobabooga/. This will be removed in a future release.")]
public sealed class OobaboogaTextCompletion : ITextCompletion
{
/// <summary>
/// The URI path for blocking API requests.
/// </summary>
public const string BlockingUriPath = "/api/v1/generate";
private const string StreamingUriPath = "/api/v1/stream";
private readonly UriBuilder _blockingUri;
private readonly UriBuilder _streamingUri;
private readonly HttpClient _httpClient;
private readonly Func<ClientWebSocket> _webSocketFactory;
private readonly bool _useWebSocketsPooling;
private readonly int _maxNbConcurrentWebSockets;
private readonly SemaphoreSlim? _concurrentSemaphore;
private readonly ConcurrentBag<bool>? _activeConnections;
private readonly ConcurrentBag<ClientWebSocket> _webSocketPool = new();
private readonly int _keepAliveWebSocketsDuration;
private readonly ILogger? _logger;
private Task? _cleanupTask = null;
private long _lastCallTicks = long.MaxValue;
/// <summary>
/// Controls the size of the buffer used to receive websocket packets.
/// </summary>
public int WebSocketBufferSize { get; set; } = 2048;
/// <summary>
/// Initializes a new instance of the <see cref="OobaboogaTextCompletion"/> class.
/// </summary>
/// <param name="endpoint">The service API endpoint to which requests should be sent.</param>
/// <param name="blockingPort">The port used for handling blocking requests. Default value is 5000</param>
/// <param name="streamingPort">The port used for handling streaming requests. Default value is 5005</param>
/// <param name="concurrentSemaphore">You can optionally set a hard limit on the max number of concurrent calls to the either of the completion methods by providing a <see cref="SemaphoreSlim"/>. Calls in excess will wait for existing consumers to release the semaphore</param>
/// <param name="httpClient">Optional. The HTTP client used for making blocking API requests. If not specified, a default client will be used.</param>
/// <param name="useWebSocketsPooling">If true, websocket clients will be recycled in a reusable pool as long as concurrent calls are detected</param>
/// <param name="webSocketsCleanUpCancellationToken">if websocket pooling is enabled, you can provide an optional CancellationToken to properly dispose of the clean up tasks when disposing of the connector</param>
/// <param name="keepAliveWebSocketsDuration">When pooling is enabled, pooled websockets are flushed on a regular basis when no more connections are made. This is the time to keep them in pool before flushing</param>
/// <param name="webSocketFactory">The WebSocket factory used for making streaming API requests. Note that only when pooling is enabled will websocket be recycled and reused for the specified duration. Otherwise, a new websocket is created for each call and closed and disposed afterwards, to prevent data corruption from concurrent calls.</param>
/// <param name="logger">Application logger</param>
public OobaboogaTextCompletion(Uri endpoint,
int blockingPort = 5000,
int streamingPort = 5005,
SemaphoreSlim? concurrentSemaphore = null,
HttpClient? httpClient = null,
bool useWebSocketsPooling = true,
CancellationToken? webSocketsCleanUpCancellationToken = default,
int keepAliveWebSocketsDuration = 100,
Func<ClientWebSocket>? webSocketFactory = null,
ILogger? logger = null)
{
Verify.NotNull(endpoint);
this._blockingUri = new UriBuilder(endpoint)
{
Port = blockingPort,
Path = BlockingUriPath
};
this._streamingUri = new(endpoint)
{
Port = streamingPort,
Path = StreamingUriPath
};
if (this._streamingUri.Uri.Scheme.StartsWith("http", StringComparison.OrdinalIgnoreCase))
{
this._streamingUri.Scheme = (this._streamingUri.Scheme == "https") ? "wss" : "ws";
}
this._httpClient = httpClient ?? new HttpClient(NonDisposableHttpClientHandler.Instance, disposeHandler: false);
this._useWebSocketsPooling = useWebSocketsPooling;
this._keepAliveWebSocketsDuration = keepAliveWebSocketsDuration;
this._logger = logger;
if (webSocketFactory != null)
{
this._webSocketFactory = () =>
{
var webSocket = webSocketFactory();
this.SetWebSocketOptions(webSocket);
return webSocket;
};
}
else
{
this._webSocketFactory = () =>
{
ClientWebSocket webSocket = new();
this.SetWebSocketOptions(webSocket);
return webSocket;
};
}
// if a hard limit is defined, we use a semaphore to limit the number of concurrent calls, otherwise, we use a stack to track active connections
if (concurrentSemaphore != null)
{
this._concurrentSemaphore = concurrentSemaphore;
this._maxNbConcurrentWebSockets = concurrentSemaphore.CurrentCount;
}
else
{
this._activeConnections = new();
this._maxNbConcurrentWebSockets = 0;
}
if (this._useWebSocketsPooling)
{
this.StartCleanupTask(webSocketsCleanUpCancellationToken ?? CancellationToken.None);
}
}
/// <inheritdoc/>
public async IAsyncEnumerable<ITextStreamingResult> GetStreamingCompletionsAsync(
string text,
AIRequestSettings? requestSettings = null,
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
await this.StartConcurrentCallAsync(cancellationToken).ConfigureAwait(false);
var completionRequest = this.CreateOobaboogaRequest(text, requestSettings);
var requestJson = JsonSerializer.Serialize(completionRequest);
var requestBytes = Encoding.UTF8.GetBytes(requestJson);
ClientWebSocket? clientWebSocket = null;
try
{
// if pooling is enabled, web socket is going to be recycled for reuse, if not it will be properly disposed of after the call
#pragma warning disable CA2000 // Dispose objects before losing scope
if (!this._useWebSocketsPooling || !this._webSocketPool.TryTake(out clientWebSocket))
{
clientWebSocket = this._webSocketFactory();
}
#pragma warning restore CA2000 // Dispose objects before losing scope
if (clientWebSocket.State == WebSocketState.None)
{
await clientWebSocket.ConnectAsync(this._streamingUri.Uri, cancellationToken).ConfigureAwait(false);
}
var sendSegment = new ArraySegment<byte>(requestBytes);
await clientWebSocket.SendAsync(sendSegment, WebSocketMessageType.Text, true, cancellationToken).ConfigureAwait(false);
TextCompletionStreamingResult streamingResult = new();
var processingTask = this.ProcessWebSocketMessagesAsync(clientWebSocket, streamingResult, cancellationToken);
yield return streamingResult;
// Await the processing task to make sure it's finished before continuing
await processingTask.ConfigureAwait(false);
}
finally
{
if (clientWebSocket != null)
{
if (this._useWebSocketsPooling && clientWebSocket.State == WebSocketState.Open)
{
this._webSocketPool.Add(clientWebSocket);
}
else
{
await this.DisposeClientGracefullyAsync(clientWebSocket).ConfigureAwait(false);
}
}
this.FinishConcurrentCall();
}
}
/// <inheritdoc/>
public async Task<IReadOnlyList<ITextResult>> GetCompletionsAsync(
string text,
AIRequestSettings? requestSettings = null,
CancellationToken cancellationToken = default)
{
try
{
await this.StartConcurrentCallAsync(cancellationToken).ConfigureAwait(false);
var completionRequest = this.CreateOobaboogaRequest(text, requestSettings);
using var stringContent = new StringContent(
JsonSerializer.Serialize(completionRequest),
Encoding.UTF8,
"application/json");
using var httpRequestMessage = new HttpRequestMessage()
{
Method = HttpMethod.Post,
RequestUri = this._blockingUri.Uri,
Content = stringContent
};
httpRequestMessage.Headers.Add("User-Agent", Telemetry.HttpUserAgent);
using var response = await this._httpClient.SendWithSuccessCheckAsync(httpRequestMessage, cancellationToken).ConfigureAwait(false);
var body = await response.Content.ReadAsStringWithExceptionMappingAsync().ConfigureAwait(false);
TextCompletionResponse? completionResponse = JsonSerializer.Deserialize<TextCompletionResponse>(body);
if (completionResponse is null)
{
throw new SKException($"Unexpected response from Oobabooga API: {body}");
}
return completionResponse.Results.Select(completionText => new TextCompletionResult(completionText)).ToList();
}
finally
{
this.FinishConcurrentCall();
}
}
#region private ================================================================================
/// <summary>
/// Creates an Oobabooga request, mapping dynamic request setting fields to their Oobabooga API counter parts
/// </summary>
/// <param name="text">The text to complete.</param>
/// <param name="requestSettings">The request settings.</param>
/// <returns>An Oobabooga TextCompletionRequest object with the text and completion parameters.</returns>
private TextCompletionRequest CreateOobaboogaRequest(string text, AIRequestSettings? requestSettings)
{
if (string.IsNullOrWhiteSpace(text))
{
throw new ArgumentNullException(nameof(text));
}
if (requestSettings is null)
{
return new TextCompletionRequest()
{
Prompt = text
};
}
if (requestSettings is TextCompletionRequest requestSettingsTextCompletionRequest)
{
requestSettingsTextCompletionRequest.Prompt = text;
return requestSettingsTextCompletionRequest;
}
var json = JsonSerializer.Serialize(requestSettings);
var textCompletionRequest = JsonSerializer.Deserialize<TextCompletionRequest>(json);
if (textCompletionRequest is not null)
{
textCompletionRequest.Prompt = text;
return textCompletionRequest;
}
throw new ArgumentException("Invalid request settings, cannot convert to TextCompletionRequest", nameof(requestSettings));
}
/// <summary>
/// Sets the options for the <paramref name="clientWebSocket"/>, either persistent and provided by the ctor, or transient if none provided.
/// </summary>
private void SetWebSocketOptions(ClientWebSocket clientWebSocket)
{
clientWebSocket.Options.SetRequestHeader("User-Agent", Telemetry.HttpUserAgent);
}
/// <summary>
/// That method is responsible for processing the websocket messages that build a streaming response object. It is crucial that it is run asynchronously to prevent a deadlock with results iteration
/// </summary>
private async Task ProcessWebSocketMessagesAsync(ClientWebSocket clientWebSocket, TextCompletionStreamingResult streamingResult, CancellationToken cancellationToken)
{
var buffer = new byte[this.WebSocketBufferSize];
var finishedProcessing = false;
while (!finishedProcessing && !cancellationToken.IsCancellationRequested)
{
MemoryStream messageStream = new();
WebSocketReceiveResult result;
do
{
var segment = new ArraySegment<byte>(buffer);
result = await clientWebSocket.ReceiveAsync(segment, cancellationToken).ConfigureAwait(false);
await messageStream.WriteAsync(buffer, 0, result.Count, cancellationToken).ConfigureAwait(false);
} while (!result.EndOfMessage);
messageStream.Seek(0, SeekOrigin.Begin);
if (result.MessageType == WebSocketMessageType.Text)
{
string messageText;
using (var reader = new StreamReader(messageStream, Encoding.UTF8))
{
messageText = await reader.ReadToEndAsync().ConfigureAwait(false);
}
var responseObject = JsonSerializer.Deserialize<TextCompletionStreamingResponse>(messageText);
if (responseObject is null)
{
throw new SKException($"Unexpected response from Oobabooga API: {messageText}");
}
switch (responseObject.Event)
{
case TextCompletionStreamingResponse.ResponseObjectTextStreamEvent:
streamingResult.AppendResponse(responseObject);
break;
case TextCompletionStreamingResponse.ResponseObjectStreamEndEvent:
streamingResult.SignalStreamEnd();
if (!this._useWebSocketsPooling)
{
await clientWebSocket.CloseAsync(WebSocketCloseStatus.NormalClosure, "Acknowledge stream-end oobabooga message", CancellationToken.None).ConfigureAwait(false);
}
finishedProcessing = true;
break;
default:
break;
}
}
else if (result.MessageType == WebSocketMessageType.Close)
{
await clientWebSocket.CloseOutputAsync(WebSocketCloseStatus.NormalClosure, "Acknowledge Close frame", CancellationToken.None).ConfigureAwait(false);
finishedProcessing = true;
}
if (clientWebSocket.State != WebSocketState.Open)
{
finishedProcessing = true;
}
}
}
/// <summary>
/// Starts a concurrent call, either by taking a semaphore slot or by pushing a value on the active connections stack
/// </summary>
/// <param name="cancellationToken"></param>
private async Task StartConcurrentCallAsync(CancellationToken cancellationToken)
{
if (this._concurrentSemaphore != null)
{
await this._concurrentSemaphore!.WaitAsync(cancellationToken).ConfigureAwait(false);
}
else
{
this._activeConnections!.Add(true);
}
}
/// <summary>
/// Gets the number of concurrent calls, either by reading the semaphore count or by reading the active connections stack count
/// </summary>
/// <returns></returns>
private int GetCurrentConcurrentCallsNb()
{
if (this._concurrentSemaphore != null)
{
return this._maxNbConcurrentWebSockets - this._concurrentSemaphore!.CurrentCount;
}
return this._activeConnections!.Count;
}
/// <summary>
/// Ends a concurrent call, either by releasing a semaphore slot or by popping a value from the active connections stack
/// </summary>
private void FinishConcurrentCall()
{
if (this._concurrentSemaphore != null)
{
this._concurrentSemaphore!.Release();
}
else
{
this._activeConnections!.TryTake(out _);
}
Interlocked.Exchange(ref this._lastCallTicks, DateTime.UtcNow.Ticks);
}
private void StartCleanupTask(CancellationToken cancellationToken)
{
if (this._cleanupTask == null || this._cleanupTask.IsCompleted)
{
this._cleanupTask = Task.Factory.StartNew(
async () =>
{
while (!cancellationToken.IsCancellationRequested)
{
await this.FlushWebSocketClientsAsync(cancellationToken).ConfigureAwait(false);
}
},
cancellationToken,
TaskCreationOptions.LongRunning,
TaskScheduler.Default);
}
}
/// <summary>
/// Flushes the web socket clients that have been idle for too long
/// </summary>
/// <returns></returns>
private async Task FlushWebSocketClientsAsync(CancellationToken cancellationToken)
{
// In the cleanup task, make sure you handle OperationCanceledException appropriately
// and make frequent checks on whether cancellation is requested.
try
{
if (!cancellationToken.IsCancellationRequested)
{
await Task.Delay(this._keepAliveWebSocketsDuration, cancellationToken).ConfigureAwait(false);
// If another call was made during the delay, do not proceed with flushing
if (DateTime.UtcNow.Ticks - Interlocked.Read(ref this._lastCallTicks) < TimeSpan.FromMilliseconds(this._keepAliveWebSocketsDuration).Ticks)
{
return;
}
while (this.GetCurrentConcurrentCallsNb() == 0 && this._webSocketPool.TryTake(out ClientWebSocket clientToDispose))
{
await this.DisposeClientGracefullyAsync(clientToDispose).ConfigureAwait(false);
}
}
}
catch (OperationCanceledException exception)
{
this._logger?.LogTrace(message: "FlushWebSocketClientsAsync cleaning task was cancelled", exception: exception);
while (this._webSocketPool.TryTake(out ClientWebSocket clientToDispose))
{
await this.DisposeClientGracefullyAsync(clientToDispose).ConfigureAwait(false);
}
}
}
/// <summary>
/// Closes and disposes of a client web socket after use
/// </summary>
private async Task DisposeClientGracefullyAsync(ClientWebSocket clientWebSocket)
{
try
{
if (clientWebSocket.State == WebSocketState.Open)
{
await clientWebSocket.CloseAsync(WebSocketCloseStatus.NormalClosure, "Closing client before disposal", CancellationToken.None).ConfigureAwait(false);
}
}
catch (OperationCanceledException exception)
{
this._logger?.LogTrace(message: "Closing client web socket before disposal was cancelled", exception: exception);
}
catch (WebSocketException exception)
{
this._logger?.LogTrace(message: "Closing client web socket before disposal raised web socket exception", exception: exception);
}
finally
{
clientWebSocket.Dispose();
}
}
#endregion
}
dotnet/src/Connectors/Connectors.AI.Oobabooga/TextCompletion/TextCompletionRequest.cs
-179
View File
@@ -1,179 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Text.Json.Serialization;
using Microsoft.SemanticKernel.AI;
namespace Microsoft.SemanticKernel.Connectors.AI.Oobabooga.TextCompletion;
/// <summary>
/// HTTP schema to perform oobabooga completion request. Contains many parameters, some of which are specific to certain kinds of models.
/// See <see href="https://github.com/oobabooga/text-generation-webui/blob/main/docs/Generation-parameters.md"/> and subsequent links for additional information.
/// </summary>
[Serializable]
[Obsolete("This functionality is available as part of new NuGet package: https://www.nuget.org/packages/MyIA.SemanticKernel.Connectors.AI.Oobabooga/. This will be removed in a future release.")]
public sealed class TextCompletionRequest : AIRequestSettings
{
/// <summary>
/// The prompt text to complete.
/// </summary>
[JsonPropertyName("prompt")]
public string Prompt { get; set; } = string.Empty;
/// <summary>
/// The maximum number of tokens to generate, ignoring the number of tokens in the prompt.
/// </summary>
[JsonPropertyName("max_new_tokens")]
public int? MaxNewTokens { get; set; }
/// <summary>
/// Determines whether or not to use sampling; use greedy decoding if false.
/// </summary>
[JsonPropertyName("do_sample")]
public bool DoSample { get; set; } = true;
/// <summary>
/// Modulates the next token probabilities. A value of 0 implies deterministic output (only the most likely token is used). Higher values increase randomness.
/// </summary>
[JsonPropertyName("temperature")]
public double Temperature { get; set; }
/// <summary>
/// If set to a value less than 1, only the most probable tokens with cumulative probability less than this value are kept for generation.
/// </summary>
[JsonPropertyName("top_p")]
public double TopP { get; set; }
/// <summary>
/// Measures how similar the conditional probability of predicting a target token is to the expected conditional probability of predicting a random token, given the generated text.
/// </summary>
[JsonPropertyName("typical_p")]
public double TypicalP { get; set; } = 1;
/// <summary>
/// Sets a probability floor below which tokens are excluded from being sampled.
/// </summary>
[JsonPropertyName("epsilon_cutoff")]
public double EpsilonCutoff { get; set; }
/// <summary>
/// Used with top_p, top_k, and epsilon_cutoff set to 0. This parameter hybridizes locally typical sampling and epsilon sampling.
/// </summary>
[JsonPropertyName("eta_cutoff")]
public double EtaCutoff { get; set; }
/// <summary>
/// Controls Tail Free Sampling (value between 0 and 1)
/// </summary>
[JsonPropertyName("tfs")]
public double Tfs { get; set; } = 1;
/// <summary>
/// Top A Sampling is a way to pick the next word in a sentence based on how important it is in the context. Top-A considers the probability of the most likely token, and sets a limit based on its percentage. After this, remaining tokens are compared to this limit. If their probability is too low, they are removed from the pool​.
/// </summary>
[JsonPropertyName("top_a")]
public double TopA { get; set; }
/// <summary>
/// Exponential penalty factor for repeating prior tokens. 1 means no penalty, higher value = less repetition.
/// </summary>
[JsonPropertyName("repetition_penalty")]
public double RepetitionPenalty { get; set; } = 1.18;
/// <summary>
///When using "top k", you select the top k most likely words to come next based on their probability of occurring, where k is a fixed number that you specify. You can use Top_K to control the amount of diversity in the model output​
/// </summary>
[JsonPropertyName("top_k")]
public int TopK { get; set; }
/// <summary>
/// Minimum length of the sequence to be generated.
/// </summary>
[JsonPropertyName("min_length")]
public int MinLength { get; set; }
/// <summary>
/// If set to a value greater than 0, all ngrams of that size can only occur once.
/// </summary>
[JsonPropertyName("no_repeat_ngram_size")]
public int NoRepeatNgramSize { get; set; }
/// <summary>
/// Number of beams for beam search. 1 means no beam search.
/// </summary>
[JsonPropertyName("num_beams")]
public int NumBeams { get; set; } = 1;
/// <summary>
/// The values balance the model confidence and the degeneration penalty in contrastive search decoding.
/// </summary>
[JsonPropertyName("penalty_alpha")]
public int PenaltyAlpha { get; set; }
/// <summary>
/// Exponential penalty to the length that is used with beam-based generation
/// </summary>
[JsonPropertyName("length_penalty")]
public double LengthPenalty { get; set; } = 1;
/// <summary>
/// Controls the stopping condition for beam-based methods, like beam-search. It accepts the following values: True, where the generation stops as soon as there are num_beams complete candidates; False, where an heuristic is applied and the generation stops when is it very unlikely to find better candidates.
/// </summary>
[JsonPropertyName("early_stopping")]
public bool EarlyStopping { get; set; }
/// <summary>
/// Parameter used for mirostat sampling in Llama.cpp, controlling perplexity during text (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
/// </summary>
[JsonPropertyName("mirostat_mode")]
public int MirostatMode { get; set; }
/// <summary>
/// Set the Mirostat target entropy, parameter tau (default: 5.0)
/// </summary>
[JsonPropertyName("mirostat_tau")]
public int MirostatTau { get; set; } = 5;
/// <summary>
/// Set the Mirostat learning rate, parameter eta (default: 0.1)
/// </summary>
[JsonPropertyName("mirostat_eta")]
public double MirostatEta { get; set; } = 0.1;
/// <summary>
/// Random seed to control sampling, used when DoSample is True.
/// </summary>
[JsonPropertyName("seed")]
public int Seed { get; set; } = -1;
/// <summary>
/// Controls whether to add beginning of a sentence token
/// </summary>
[JsonPropertyName("add_bos_token")]
public bool AddBosToken { get; set; } = true;
/// <summary>
/// The leftmost tokens are removed if the prompt exceeds this length. Most models require this to be at most 2048.
/// </summary>
[JsonPropertyName("truncation_length")]
public int TruncationLength { get; set; } = 2048;
/// <summary>
/// Forces the model to never end the generation prematurely.
/// </summary>
[JsonPropertyName("ban_eos_token")]
public bool BanEosToken { get; set; } = true;
/// <summary>
/// Some specific models need this unset.
/// </summary>
[JsonPropertyName("skip_special_tokens")]
public bool SkipSpecialTokens { get; set; } = true;
/// <summary>
/// In addition to the defaults. Written between "" and separated by commas. For instance: "\nYour Assistant:", "\nThe assistant:"
/// </summary>
[JsonPropertyName("stopping_strings")]
public IList<string> StoppingStrings { get; set; } = new List<string>();
}
dotnet/src/Connectors/Connectors.AI.Oobabooga/TextCompletion/TextCompletionResponse.cs
-33
View File
@@ -1,33 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Text.Json.Serialization;
namespace Microsoft.SemanticKernel.Connectors.AI.Oobabooga.TextCompletion;
/// <summary>
/// HTTP Schema for Oobabooga completion response. Contains a list of results. Adapted from <see href="https://github.com/oobabooga/text-generation-webui/blob/main/extensions/api/blocking_api.py"/>
/// </summary>
[Obsolete("This functionality is available as part of new NuGet package: https://www.nuget.org/packages/MyIA.SemanticKernel.Connectors.AI.Oobabooga/. This will be removed in a future release.")]
public sealed class TextCompletionResponse
{
/// <summary>
/// A field used by Oobabooga to return results from the blocking API.
/// </summary>
[JsonPropertyName("results")]
public List<TextCompletionResponseText> Results { get; set; } = new();
}
/// <summary>
/// HTTP Schema for an single Oobabooga result as part of a completion response.
/// </summary>
[Obsolete("This functionality is available as part of new NuGet package: https://www.nuget.org/packages/MyIA.SemanticKernel.Connectors.AI.Oobabooga/. This will be removed in a future release.")]
public sealed class TextCompletionResponseText
{
/// <summary>
/// Completed text.
/// </summary>
[JsonPropertyName("text")]
public string? Text { get; set; } = string.Empty;
}
dotnet/src/Connectors/Connectors.AI.Oobabooga/TextCompletion/TextCompletionResult.cs
-30
View File
@@ -1,30 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.SemanticKernel.AI.TextCompletion;
using Microsoft.SemanticKernel.Orchestration;
namespace Microsoft.SemanticKernel.Connectors.AI.Oobabooga.TextCompletion;
/// <summary>
/// Oobabooga implementation of <see cref="ITextResult"/>. Actual response object is stored in a ModelResult instance, and completion text is simply passed forward.
/// </summary>
[Obsolete("This functionality is available as part of new NuGet package: https://www.nuget.org/packages/MyIA.SemanticKernel.Connectors.AI.Oobabooga/. This will be removed in a future release.")]
internal sealed class TextCompletionResult : ITextResult
{
private readonly ModelResult _responseData;
public TextCompletionResult(TextCompletionResponseText responseData)
{
this._responseData = new ModelResult(responseData);
}
public ModelResult ModelResult => this._responseData;
public Task<string> GetCompletionAsync(CancellationToken cancellationToken = default)
{
return Task.FromResult(this._responseData.GetResult<TextCompletionResponseText>().Text ?? string.Empty);
}
}
dotnet/src/Connectors/Connectors.AI.Oobabooga/TextCompletion/TextCompletionStreamingResponse.cs
-41
View File
@@ -1,41 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Text.Json.Serialization;
namespace Microsoft.SemanticKernel.Connectors.AI.Oobabooga.TextCompletion;
/// <summary>
/// Represents the HTTP schema for streaming completion response. Adapted from <see href="https://github.com/oobabooga/text-generation-webui/blob/main/extensions/api/streaming_api.py"/>.
/// </summary>
[Obsolete("This functionality is available as part of new NuGet package: https://www.nuget.org/packages/MyIA.SemanticKernel.Connectors.AI.Oobabooga/. This will be removed in a future release.")]
public sealed class TextCompletionStreamingResponse
{
/// <summary>
/// Constant string representing the event that is fired when text is received from a websocket.
/// </summary>
public const string ResponseObjectTextStreamEvent = "text_stream";
/// <summary>
/// Constant string representing the event that is fired when streaming from a websocket ends.
/// </summary>
public const string ResponseObjectStreamEndEvent = "stream_end";
/// <summary>
/// A field used by Oobabooga to signal the type of websocket message sent, e.g. "text_stream" or "stream_end".
/// </summary>
[JsonPropertyName("event")]
public string Event { get; set; } = string.Empty;
/// <summary>
/// A field used by Oobabooga to signal the number of messages sent, starting with 0 and incremented on each message.
/// </summary>
[JsonPropertyName("message_num")]
public int MessageNum { get; set; }
/// <summary>
/// A field used by Oobabooga with the text chunk sent in the websocket message.
/// </summary>
[JsonPropertyName("text")]
public string Text { get; set; } = string.Empty;
}
dotnet/src/Connectors/Connectors.AI.Oobabooga/TextCompletion/TextCompletionStreamingResult.cs
-68
View File
@@ -1,68 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices;
using System.Text;
using System.Threading;
using System.Threading.Channels;
using System.Threading.Tasks;
using Microsoft.SemanticKernel.AI.TextCompletion;
using Microsoft.SemanticKernel.Orchestration;
namespace Microsoft.SemanticKernel.Connectors.AI.Oobabooga.TextCompletion;
[Obsolete("This functionality is available as part of new NuGet package: https://www.nuget.org/packages/MyIA.SemanticKernel.Connectors.AI.Oobabooga/. This will be removed in a future release.")]
internal sealed class TextCompletionStreamingResult : ITextStreamingResult, ITextResult
{
private readonly List<TextCompletionStreamingResponse> _modelResponses;
private readonly Channel<string> _responseChannel;
public ModelResult ModelResult { get; }
public TextCompletionStreamingResult()
{
this._modelResponses = new();
this.ModelResult = new ModelResult(this._modelResponses);
this._responseChannel = Channel.CreateUnbounded<string>(new UnboundedChannelOptions()
{
SingleReader = true,
SingleWriter = true,
AllowSynchronousContinuations = false
});
}
public void AppendResponse(TextCompletionStreamingResponse response)
{
this._modelResponses.Add(response);
this._responseChannel.Writer.TryWrite(response.Text);
}
public void SignalStreamEnd()
{
this._responseChannel.Writer.Complete();
}
public async Task<string> GetCompletionAsync(CancellationToken cancellationToken = default)
{
StringBuilder resultBuilder = new();
await foreach (var chunk in this.GetCompletionStreamingAsync(cancellationToken))
{
resultBuilder.Append(chunk);
}
return resultBuilder.ToString();
}
public async IAsyncEnumerable<string> GetCompletionStreamingAsync([EnumeratorCancellation] CancellationToken cancellationToken = default)
{
while (await this._responseChannel.Reader.WaitToReadAsync(cancellationToken).ConfigureAwait(false))
{
while (this._responseChannel.Reader.TryRead(out string? chunk))
{
yield return chunk;
}
}
}
}
dotnet/src/Connectors/Connectors.Memory.Chroma/README.md
+1 -1
View File
@@ -25,7 +25,7 @@ const string endpoint = "http://localhost:8000";
ChromaMemoryStore memoryStore = new(endpoint);
IKernel kernel = Kernel.Builder
IKernel kernel = new KernelBuilder()
.WithLogger(logger)
.WithOpenAITextEmbeddingGenerationService("text-embedding-ada-002", "OPENAI_API_KEY")
.WithMemoryStorage(memoryStore)
dotnet/src/Connectors/Connectors.Memory.Kusto/README.md
+1 -1
View File
@@ -14,7 +14,7 @@ using Kusto.Data;
var connectionString = new KustoConnectionStringBuilder("https://kvc123.eastus.kusto.windows.net").WithAadUserPromptAuthentication();
KustoMemoryStore memoryStore = new(connectionString, "MyDatabase");
IKernel kernel = Kernel.Builder
IKernel kernel = new KernelBuilder()
.WithLogger(ConsoleLogger.Log)
.WithOpenAITextCompletionService(modelId: TestConfiguration.OpenAI.ModelId, apiKey: TestConfiguration.OpenAI.ApiKey)
.WithOpenAITextEmbeddingGenerationService(modelId: TestConfiguration.OpenAI.EmbeddingModelId,apiKey: TestConfiguration.OpenAI.ApiKey)
dotnet/src/Connectors/Connectors.Memory.Milvus/README.md
+1 -1
View File
@@ -23,7 +23,7 @@ docker-compose up -d
```csharp
using MilvusMemoryStore memoryStore = new("localhost");
IKernel kernel = Kernel.Builder
IKernel kernel = new KernelBuilder()
.WithLogger(logger)
.WithOpenAITextEmbeddingGenerationService("text-embedding-ada-002", "OPENAI_API_KEY")
.WithMemoryStorage(memoryStore)
dotnet/src/Connectors/Connectors.Memory.Postgres/README.md
+1 -1
View File
@@ -41,7 +41,7 @@ NpgsqlDataSource dataSource = dataSourceBuilder.Build();
PostgresMemoryStore memoryStore = new PostgresMemoryStore(dataSource, vectorSize: 1536/*, schema: "public" */);
IKernel kernel = Kernel.Builder
IKernel kernel = new KernelBuilder()
.WithLogger(ConsoleLogger.Logger)
.WithOpenAITextEmbeddingGenerationService("text-embedding-ada-002", Env.Var("OPENAI_API_KEY"))
.WithMemoryStorage(memoryStore)
dotnet/src/Connectors/Connectors.Memory.Redis/README.md
+1 -1
View File
@@ -30,7 +30,7 @@ ConnectionMultiplexer connectionMultiplexer = await ConnectionMultiplexer.Connec
IDatabase database = connectionMultiplexer.GetDatabase();
RedisMemoryStore memoryStore = new RedisMemoryStore(database, vectorSize: 1536);
IKernel kernel = Kernel.Builder
IKernel kernel = new KernelBuilder()
.WithLogger(ConsoleLogger.Logger)
.WithOpenAITextEmbeddingGenerationService("text-embedding-ada-002", Env.Var("OPENAI_API_KEY"))
.WithMemoryStorage(memoryStore)
dotnet/src/Connectors/Connectors.UnitTests/ConnectedClient.cs
-25
View File
@@ -1,25 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Net;
using System.Net.WebSockets;
namespace SemanticKernel.Connectors.UnitTests;
internal sealed class ConnectedClient
{
public Guid Id { get; }
public HttpListenerContext Context { get; }
public WebSocket? Socket { get; private set; }
public ConnectedClient(Guid id, HttpListenerContext context)
{
this.Id = id;
this.Context = context;
}
public void SetSocket(WebSocket socket)
{
this.Socket = socket;
}
}
dotnet/src/Connectors/Connectors.UnitTests/Connectors.UnitTests.csproj
-7
View File
@@ -32,7 +32,6 @@
<ItemGroup>
<ProjectReference Include="..\..\SemanticKernel.Core\SemanticKernel.Core.csproj" />
<ProjectReference Include="..\Connectors.AI.HuggingFace\Connectors.AI.HuggingFace.csproj" />
<ProjectReference Include="..\Connectors.AI.Oobabooga\Connectors.AI.Oobabooga.csproj" />
<ProjectReference Include="..\Connectors.AI.OpenAI\Connectors.AI.OpenAI.csproj" />
<ProjectReference Include="..\Connectors.Memory.AzureCognitiveSearch\Connectors.Memory.AzureCognitiveSearch.csproj" />
<ProjectReference Include="..\Connectors.Memory.Chroma\Connectors.Memory.Chroma.csproj" />
@@ -59,12 +58,6 @@
<None Update="OpenAI\TestData\image_generation_test_response.json">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
<None Update="Oobabooga\TestData\completion_test_streaming_response.json">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
<None Update="Oobabooga\TestData\completion_test_response.json">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
</ItemGroup>
</Project>
dotnet/src/Connectors/Connectors.UnitTests/Oobabooga/OobaboogaTestHelper.cs
-44
View File
@@ -1,44 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System.IO;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using Moq;
using Moq.Protected;
namespace SemanticKernel.Connectors.UnitTests.Oobabooga;
/// <summary>
/// Helper for Oobabooga test purposes.
/// </summary>
internal static class OobaboogaTestHelper
{
/// <summary>
/// Reads test response from file for mocking purposes.
/// </summary>
/// <param name="fileName">Name of the file with test response.</param>
internal static string GetTestResponse(string fileName)
{
return File.ReadAllText($"./Oobabooga/TestData/{fileName}");
}
/// <summary>
/// Returns mocked instance of <see cref="HttpClientHandler"/>.
/// </summary>
/// <param name="httpResponseMessage">Message to return for mocked <see cref="HttpClientHandler"/>.</param>
internal static HttpClientHandler GetHttpClientHandlerMock(HttpResponseMessage httpResponseMessage)
{
var httpClientHandler = new Mock<HttpClientHandler>();
httpClientHandler
.Protected()
.Setup<Task<HttpResponseMessage>>(
"SendAsync",
ItExpr.IsAny<HttpRequestMessage>(),
ItExpr.IsAny<CancellationToken>())
.ReturnsAsync(httpResponseMessage);
return httpClientHandler.Object;
}
}
dotnet/src/Connectors/Connectors.UnitTests/Oobabooga/OobaboogaWebSocketTestServer.cs
-63
View File
@@ -1,63 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Text;
using System.Text.Json;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel.Connectors.AI.Oobabooga.TextCompletion;
namespace SemanticKernel.Connectors.UnitTests.Oobabooga;
/// <summary>
/// Represents a WebSocket test server specifically designed for the Oobabooga text completion service.
/// It inherits from the base WebSocketTestServer class and handles Oobabooga-specific request and response classes.
/// The server accepts WebSocket connections, receives requests, and generates responses based on the Oobabooga text completion logic.
/// The OobaboogaWebSocketTestServer class uses a delegate to handle the request and response logic, allowing customization of the behavior.
/// </summary>
[Obsolete("This functionality is available as part of new NuGet package: https://www.nuget.org/packages/MyIA.SemanticKernel.Connectors.AI.Oobabooga/. This will be removed in a future release.")]
internal sealed class OobaboogaWebSocketTestServer : WebSocketTestServer
{
public OobaboogaWebSocketTestServer(string url, Func<string, List<string>> stringHandler, ILogger? logger = null)
: base(url, bytes => HandleRequest(bytes, stringHandler), logger: logger)
{
}
private static List<ArraySegment<byte>> HandleRequest(ArraySegment<byte> request, Func<string, List<string>> stringHandler)
{
var requestString = Encoding.UTF8.GetString(request.ToArray());
var requestObj = JsonSerializer.Deserialize<TextCompletionRequest>(requestString);
var responseList = stringHandler(requestObj?.Prompt ?? string.Empty);
var responseSegments = new List<ArraySegment<byte>>();
int messageNum = 0;
foreach (var responseChunk in responseList)
{
var responseObj = new TextCompletionStreamingResponse
{
Event = "text_stream",
MessageNum = messageNum,
Text = responseChunk
};
var responseJson = JsonSerializer.Serialize(responseObj);
var responseBytes = Encoding.UTF8.GetBytes(responseJson);
responseSegments.Add(new ArraySegment<byte>(responseBytes));
messageNum++;
}
var streamEndObj = new TextCompletionStreamingResponse
{
Event = "stream_end",
MessageNum = messageNum
};
var streamEndJson = JsonSerializer.Serialize(streamEndObj);
var streamEndBytes = Encoding.UTF8.GetBytes(streamEndJson);
responseSegments.Add(new ArraySegment<byte>(streamEndBytes));
return responseSegments;
}
}
dotnet/src/Connectors/Connectors.UnitTests/Oobabooga/TestData/completion_test_response.json
-9
View File
@@ -1,9 +0,0 @@
{
"results": [
{
"text": "This is test completion response"
}
]
}
dotnet/src/Connectors/Connectors.UnitTests/Oobabooga/TestData/completion_test_streaming_response.json
-5
View File
@@ -1,5 +0,0 @@
{
"event": "text_stream",
"message_num": 0,
"text": "This is test completion response"
}
dotnet/src/Connectors/Connectors.UnitTests/Oobabooga/TextCompletion/OobaboogaTextCompletionTests.cs
-401
View File
@@ -1,401 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Globalization;
using System.Linq;
using System.Net.Http;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel.AI.TextCompletion;
using Microsoft.SemanticKernel.Connectors.AI.Oobabooga.TextCompletion;
using Xunit;
using Xunit.Abstractions;
namespace SemanticKernel.Connectors.UnitTests.Oobabooga.TextCompletion;
/// <summary>
/// Unit tests for <see cref="OobaboogaTextCompletion"/> class.
/// </summary>
[Obsolete("This functionality is available as part of new NuGet package: https://www.nuget.org/packages/MyIA.SemanticKernel.Connectors.AI.Oobabooga/. This will be removed in a future release.")]
public sealed class OobaboogaTextCompletionTests : IDisposable
{
private readonly XunitLogger<OobaboogaTextCompletion> _logger;
private const string EndPoint = "https://fake-random-test-host";
private const int BlockingPort = 1234;
private const int StreamingPort = 2345;
private const string CompletionText = "fake-test";
private const string CompletionMultiText = "Hello, my name is";
private readonly HttpMessageHandlerStub _messageHandlerStub;
private readonly HttpClient _httpClient;
private readonly Uri _endPointUri;
private readonly string _streamCompletionResponseStub;
public OobaboogaTextCompletionTests(ITestOutputHelper output)
{
this._logger = new XunitLogger<OobaboogaTextCompletion>(output);
this._messageHandlerStub = new HttpMessageHandlerStub();
this._messageHandlerStub.ResponseToReturn.Content = new StringContent(OobaboogaTestHelper.GetTestResponse("completion_test_response.json"));
this._streamCompletionResponseStub = OobaboogaTestHelper.GetTestResponse("completion_test_streaming_response.json");
this._httpClient = new HttpClient(this._messageHandlerStub, false);
this._endPointUri = new Uri(EndPoint);
}
[Fact]
public async Task UserAgentHeaderShouldBeUsedAsync()
{
//Arrange
var sut = new OobaboogaTextCompletion(endpoint: this._endPointUri,
blockingPort: BlockingPort,
httpClient: this._httpClient,
logger: this._logger);
//Act
await sut.GetCompletionsAsync(CompletionText, null);
//Assert
Assert.True(this._messageHandlerStub.RequestHeaders?.Contains("User-Agent"));
var values = this._messageHandlerStub.RequestHeaders!.GetValues("User-Agent");
var value = values.SingleOrDefault();
Assert.Equal("Semantic-Kernel", value);
}
[Fact]
public async Task ProvidedEndpointShouldBeUsedAsync()
{
//Arrange
var sut = new OobaboogaTextCompletion(endpoint: this._endPointUri,
blockingPort: BlockingPort,
httpClient: this._httpClient,
logger: this._logger);
//Act
await sut.GetCompletionsAsync(CompletionText, null);
//Assert
Assert.StartsWith(EndPoint, this._messageHandlerStub.RequestUri?.AbsoluteUri, StringComparison.OrdinalIgnoreCase);
}
[Fact]
public async Task BlockingUrlShouldBeBuiltSuccessfullyAsync()
{
//Arrange
var sut = new OobaboogaTextCompletion(endpoint: this._endPointUri,
blockingPort: BlockingPort,
httpClient: this._httpClient,
logger: this._logger);
//Act
await sut.GetCompletionsAsync(CompletionText);
var expectedUri = new UriBuilder(this._endPointUri)
{
Path = OobaboogaTextCompletion.BlockingUriPath,
Port = BlockingPort
};
//Assert
Assert.Equal(expectedUri.Uri, this._messageHandlerStub.RequestUri);
}
[Fact]
public async Task ShouldSendPromptToServiceAsync()
{
//Arrange
var sut = new OobaboogaTextCompletion(endpoint: this._endPointUri,
blockingPort: BlockingPort,
httpClient: this._httpClient,
logger: this._logger);
//Act
await sut.GetCompletionsAsync(CompletionText, null);
//Assert
var requestPayload = JsonSerializer.Deserialize<TextCompletionRequest>(this._messageHandlerStub.RequestContent);
Assert.NotNull(requestPayload);
Assert.Equal(CompletionText, requestPayload.Prompt);
}
[Fact]
public async Task ShouldHandleServiceResponseAsync()
{
//Arrange
var sut = new OobaboogaTextCompletion(endpoint: this._endPointUri,
blockingPort: BlockingPort,
httpClient: this._httpClient,
logger: this._logger);
//Act
var result = await sut.GetCompletionsAsync(CompletionText, null);
//Assert
Assert.NotNull(result);
var completions = result.SingleOrDefault();
Assert.NotNull(completions);
var completion = await completions.GetCompletionAsync();
Assert.Equal("This is test completion response", completion);
}
[Fact]
public async Task ShouldHandleStreamingServicePersistentWebSocketResponseAsync()
{
var requestMessage = CompletionText;
var expectedResponse = new List<string> { this._streamCompletionResponseStub };
await this.RunWebSocketMultiPacketStreamingTestAsync(
requestMessage: requestMessage,
expectedResponse: expectedResponse,
isPersistent: true);
}
[Fact]
public async Task ShouldHandleStreamingServiceTransientWebSocketResponseAsync()
{
var requestMessage = CompletionText;
var expectedResponse = new List<string> { this._streamCompletionResponseStub };
await this.RunWebSocketMultiPacketStreamingTestAsync(
requestMessage: requestMessage,
expectedResponse: expectedResponse);
}
[Fact]
public async Task ShouldHandleConcurrentWebSocketConnectionsAsync()
{
var serverUrl = $"http://localhost:{StreamingPort}/";
var clientUrl = $"ws://localhost:{StreamingPort}/";
var expectedResponses = new List<string>
{
"Response 1",
"Response 2",
"Response 3",
"Response 4",
"Response 5"
};
await using var server = new WebSocketTestServer(serverUrl, request =>
{
// Simulate different responses for each request
var responseIndex = int.Parse(Encoding.UTF8.GetString(request.ToArray()), CultureInfo.InvariantCulture);
byte[] bytes = Encoding.UTF8.GetBytes(expectedResponses[responseIndex]);
var toReturn = new List<ArraySegment<byte>> { new(bytes) };
return toReturn;
});
var tasks = new List<Task<string>>();
// Simulate multiple concurrent WebSocket connections
for (int i = 0; i < expectedResponses.Count; i++)
{
var currentIndex = i;
tasks.Add(Task.Run(async () =>
{
using var client = new ClientWebSocket();
await client.ConnectAsync(new Uri(clientUrl), CancellationToken.None);
// Send a request to the server
var requestBytes = Encoding.UTF8.GetBytes(currentIndex.ToString(CultureInfo.InvariantCulture));
await client.SendAsync(new ArraySegment<byte>(requestBytes), WebSocketMessageType.Text, true, CancellationToken.None);
// Receive the response from the server
var responseBytes = new byte[1024];
var responseResult = await client.ReceiveAsync(new ArraySegment<byte>(responseBytes), CancellationToken.None);
await client.CloseAsync(WebSocketCloseStatus.NormalClosure, "Close connection after message received", CancellationToken.None);
var response = Encoding.UTF8.GetString(responseBytes, 0, responseResult.Count);
return response;
}));
}
// Assert
for (int i = 0; i < expectedResponses.Count; i++)
{
var response = await tasks[i];
Assert.Equal(expectedResponses[i], response);
}
}
[Fact]
public async Task ShouldHandleMultiPacketStreamingServiceTransientWebSocketResponseAsync()
{
await this.RunWebSocketMultiPacketStreamingTestAsync();
}
[Fact]
public async Task ShouldHandleMultiPacketStreamingServicePersistentWebSocketResponseBroadcastBlockAsync()
{
await this.RunWebSocketMultiPacketStreamingTestAsync(isPersistent: true);
}
[Fact]
public async Task ShouldHandleConcurrentMultiPacketStreamingServiceTransientWebSocketResponseAsync()
{
await this.RunWebSocketMultiPacketStreamingTestAsync(nbConcurrentCalls: 10);
}
[Fact]
public async Task ShouldHandleConcurrentMultiPacketStreamingServicePersistentWebSocketResponseAsync()
{
await this.RunWebSocketMultiPacketStreamingTestAsync(nbConcurrentCalls: 10, isPersistent: true);
}
/// <summary>
/// This test will assess concurrent enumeration of the same long multi message (500 websocket messages) streaming result.
/// </summary>
[Fact]
public async Task ShouldHandleConcurrentEnumerationOfLongStreamingServiceResponseAsync()
{
var expectedResponse = Enumerable.Range(0, 500).Select(i => i.ToString(CultureInfo.InvariantCulture)).ToList();
using SemaphoreSlim enforcedConcurrentCallSemaphore = new(20);
await this.RunWebSocketMultiPacketStreamingTestAsync(
expectedResponse: expectedResponse,
nbConcurrentCalls: 1,
nbConcurrentEnumeration: 100,
isPersistent: true,
keepAliveWebSocketsDuration: 100,
concurrentCallsTicksDelay: 0,
enforcedConcurrentCallSemaphore: enforcedConcurrentCallSemaphore,
maxExpectedNbClients: 20);
}
private async Task RunWebSocketMultiPacketStreamingTestAsync(
string requestMessage = CompletionMultiText,
List<string>? expectedResponse = null,
int nbConcurrentCalls = 1,
int nbConcurrentEnumeration = 1,
bool isPersistent = false,
int requestProcessingDuration = 0,
int segmentMessageDelay = 0,
int keepAliveWebSocketsDuration = 100,
int concurrentCallsTicksDelay = 0,
SemaphoreSlim? enforcedConcurrentCallSemaphore = null,
int maxExpectedNbClients = 0,
int maxTestDuration = 0)
{
if (expectedResponse == null)
{
expectedResponse = new List<string> { " John", ". I", "'m a", " writer" };
}
Func<ClientWebSocket>? webSocketFactory = null;
// Counter to track the number of WebSocket clients created
int clientCount = 0;
var delayTimeSpan = new TimeSpan(concurrentCallsTicksDelay);
if (isPersistent)
{
ClientWebSocket ExternalWebSocketFactory()
{
this._logger?.LogInformation(message: "Creating new client web socket");
var toReturn = new ClientWebSocket();
return toReturn;
}
if (maxExpectedNbClients > 0)
{
ClientWebSocket IncrementFactory()
{
var toReturn = ExternalWebSocketFactory();
Interlocked.Increment(ref clientCount);
return toReturn;
}
webSocketFactory = IncrementFactory;
}
else
{
webSocketFactory = ExternalWebSocketFactory;
}
}
using var cleanupToken = new CancellationTokenSource();
var sut = new OobaboogaTextCompletion(
endpoint: new Uri("http://localhost/"),
streamingPort: StreamingPort,
httpClient: this._httpClient,
webSocketsCleanUpCancellationToken: cleanupToken.Token,
webSocketFactory: webSocketFactory,
keepAliveWebSocketsDuration: keepAliveWebSocketsDuration,
concurrentSemaphore: enforcedConcurrentCallSemaphore,
logger: this._logger);
await using var server = new OobaboogaWebSocketTestServer($"http://localhost:{StreamingPort}/", request => expectedResponse, logger: this._logger)
{
RequestProcessingDelay = TimeSpan.FromMilliseconds(requestProcessingDuration),
SegmentMessageDelay = TimeSpan.FromMilliseconds(segmentMessageDelay)
};
var sw = Stopwatch.StartNew();
var tasks = new List<Task<IAsyncEnumerable<string>>>();
for (int i = 0; i < nbConcurrentCalls; i++)
{
tasks.Add(Task.FromResult(sut.CompleteStreamAsync(requestMessage, new TextCompletionRequest()
{
Temperature = 0.01,
MaxNewTokens = 7,
TopP = 0.1,
}, cancellationToken: cleanupToken.Token)));
}
var callEnumerationTasks = new List<Task<List<string>>>();
var results = await Task.WhenAll(tasks);
foreach (var completion in results)
{
callEnumerationTasks.AddRange(Enumerable.Range(0, nbConcurrentEnumeration).Select(_ => Task.Run(async () =>
{
var result = new List<string>();
await foreach (var chunk in completion)
{
result.Add(chunk);
}
return result;
})));
// Introduce a delay between creating each WebSocket client
await Task.Delay(delayTimeSpan);
}
var allResults = await Task.WhenAll(callEnumerationTasks);
var elapsed = sw.ElapsedMilliseconds;
if (maxExpectedNbClients > 0)
{
Assert.InRange(clientCount, 1, maxExpectedNbClients);
}
// Validate all results
foreach (var result in allResults)
{
Assert.Equal(expectedResponse.Count, result.Count);
for (int i = 0; i < expectedResponse.Count; i++)
{
Assert.Equal(expectedResponse[i], result[i]);
}
}
if (maxTestDuration > 0)
{
Assert.InRange(elapsed, 0, maxTestDuration);
}
}
public void Dispose()
{
this._httpClient.Dispose();
this._messageHandlerStub.Dispose();
this._logger.Dispose();
}
}
dotnet/src/Connectors/Connectors.UnitTests/OpenAI/AIServicesOpenAIExtensionsTests.cs
+3 -3
View File
@@ -16,7 +16,7 @@ public class AIServicesOpenAIExtensionsTests
[Fact]
public void ItSucceedsWhenAddingDifferentServiceTypeWithSameId()
{
KernelBuilder targetBuilder = Kernel.Builder;
KernelBuilder targetBuilder = new();
targetBuilder.WithAzureTextCompletionService("depl", "https://url", "key", "azure");
targetBuilder.WithAzureTextEmbeddingGenerationService("depl2", "https://url", "key", "azure");
@@ -28,7 +28,7 @@ public class AIServicesOpenAIExtensionsTests
[Fact]
public void ItTellsIfAServiceIsAvailable()
{
KernelBuilder targetBuilder = Kernel.Builder;
KernelBuilder targetBuilder = new();
targetBuilder.WithAzureTextCompletionService("depl", "https://url", "key", serviceId: "azure");
targetBuilder.WithOpenAITextCompletionService("model", "apikey", serviceId: "oai");
targetBuilder.WithAzureTextEmbeddingGenerationService("depl2", "https://url2", "key", serviceId: "azure");
@@ -46,7 +46,7 @@ public class AIServicesOpenAIExtensionsTests
public void ItCanOverwriteServices()
{
// Arrange
KernelBuilder targetBuilder = Kernel.Builder;
KernelBuilder targetBuilder = new();
// Act - Assert no exception occurs
targetBuilder.WithAzureTextCompletionService("dep", "https://localhost", "key", serviceId: "one");
dotnet/src/Connectors/Connectors.UnitTests/WebSocketTestServer.cs
-229
View File
@@ -1,229 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.WebSockets;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
namespace SemanticKernel.Connectors.UnitTests;
internal class WebSocketTestServer : IDisposable
{
private readonly ILogger? _logger;
private readonly HttpListener _httpListener;
private readonly CancellationTokenSource _mainCancellationTokenSource;
private readonly CancellationTokenSource _socketCancellationTokenSource;
private bool _serverIsRunning;
private readonly Func<ArraySegment<byte>, List<ArraySegment<byte>>> _arraySegmentHandler;
private readonly ConcurrentDictionary<Guid, ConcurrentQueue<byte[]>> _requestContentQueues;
private readonly ConcurrentBag<Task> _runningTasks = new();
private readonly ConcurrentDictionary<Guid, ConnectedClient> _clients = new();
private readonly Task? _handleRequestTask = null;
public TimeSpan RequestProcessingDelay { get; set; } = TimeSpan.Zero;
public TimeSpan SegmentMessageDelay { get; set; } = TimeSpan.Zero;
public ConcurrentDictionary<Guid, byte[]> RequestContents
{
get
{
return new ConcurrentDictionary<Guid, byte[]>(
this._requestContentQueues
.ToDictionary(kvp => kvp.Key, kvp => kvp.Value.ToList().SelectMany(bytes => bytes).ToArray()));
}
}
public WebSocketTestServer(string url, Func<ArraySegment<byte>, List<ArraySegment<byte>>> arraySegmentHandler, ILogger? logger = null)
{
this._logger = logger;
this._arraySegmentHandler = arraySegmentHandler;
this._requestContentQueues = new ConcurrentDictionary<Guid, ConcurrentQueue<byte[]>>();
this._mainCancellationTokenSource = new();
this._socketCancellationTokenSource = new();
this._httpListener = new HttpListener();
this._httpListener.Prefixes.Add(url);
this._httpListener.Start();
this._serverIsRunning = true;
if (this._handleRequestTask is null || this._handleRequestTask.IsCompleted)
{
this._handleRequestTask = Task.Run((Func<Task>)this.HandleRequestsAsync, this._mainCancellationTokenSource.Token);
}
}
private async Task HandleRequestsAsync()
{
while (!this._mainCancellationTokenSource.IsCancellationRequested)
{
var context = await this._httpListener.GetContextAsync().ConfigureAwait(false);
if (this._serverIsRunning)
{
if (context.Request.IsWebSocketRequest)
{
var connectedClient = new ConnectedClient(Guid.NewGuid(), context);
this._clients[connectedClient.Id] = connectedClient;
try
{
var socketContext = await context.AcceptWebSocketAsync(subProtocol: null);
connectedClient.SetSocket(socketContext.WebSocket);
this._runningTasks.Add(this.HandleSingleWebSocketRequestAsync(connectedClient));
}
catch
{
// server error if upgrade from HTTP to WebSocket fails
context.Response.StatusCode = 500;
context.Response.StatusDescription = "WebSocket upgrade failed";
context.Response.Close();
throw;
}
}
}
else
{
// HTTP 409 Conflict (with server's current state)
context.Response.StatusCode = 409;
context.Response.StatusDescription = "Server is shutting down";
context.Response.Close();
return;
}
}
await Task.WhenAll(this._runningTasks).ConfigureAwait(false);
}
private async Task HandleSingleWebSocketRequestAsync(ConnectedClient connectedClient)
{
var buffer = WebSocket.CreateServerBuffer(4096);
Guid requestId = connectedClient.Id;
this._requestContentQueues[requestId] = new ConcurrentQueue<byte[]>();
try
{
while (!this._socketCancellationTokenSource.IsCancellationRequested && connectedClient.Socket != null && connectedClient.Socket.State != WebSocketState.Closed && connectedClient.Socket.State != WebSocketState.Aborted)
{
WebSocketReceiveResult result = await connectedClient.Socket.ReceiveAsync(buffer, this._socketCancellationTokenSource.Token).ConfigureAwait(false);
if (!this._socketCancellationTokenSource.IsCancellationRequested && connectedClient.Socket.State != WebSocketState.Closed && connectedClient.Socket.State != WebSocketState.Aborted)
{
if (connectedClient.Socket.State == WebSocketState.CloseReceived && result.MessageType == WebSocketMessageType.Close)
{
await connectedClient.Socket.CloseOutputAsync(WebSocketCloseStatus.NormalClosure, "Acknowledge Close frame", CancellationToken.None);
break;
}
var receivedBytes = buffer.Slice(0, result.Count);
this._requestContentQueues[requestId].Enqueue(receivedBytes.ToArray());
if (result.EndOfMessage)
{
var responseSegments = this._arraySegmentHandler(receivedBytes);
if (this.RequestProcessingDelay.Ticks > 0)
{
await Task.Delay(this.RequestProcessingDelay).ConfigureAwait(false);
}
foreach (var responseSegment in responseSegments)
{
if (connectedClient.Socket.State != WebSocketState.Open)
{
break;
}
if (this.SegmentMessageDelay.Ticks > 0)
{
await Task.Delay(this.SegmentMessageDelay).ConfigureAwait(false);
}
await connectedClient.Socket.SendAsync(responseSegment, WebSocketMessageType.Text, true, this._socketCancellationTokenSource.Token).ConfigureAwait(false);
}
}
}
}
if (connectedClient.Socket?.State == WebSocketState.Open)
{
await connectedClient.Socket.CloseAsync(WebSocketCloseStatus.NormalClosure, "Closing waiting for acknowledgement", CancellationToken.None).ConfigureAwait(false);
}
else if (connectedClient.Socket?.State == WebSocketState.CloseReceived)
{
await connectedClient.Socket.CloseOutputAsync(WebSocketCloseStatus.NormalClosure, "Closing without waiting for acknowledgment", CancellationToken.None).ConfigureAwait(false);
}
}
catch (OperationCanceledException exception)
{
this._logger?.LogTrace(message: "Closing server web socket before disposal was cancelled", exception: exception);
}
catch (WebSocketException exception)
{
this._logger?.LogTrace(message: "Closing server web socket before disposal raised web socket exception", exception: exception);
}
finally
{
if (connectedClient.Socket?.State != WebSocketState.Closed)
{
connectedClient.Socket?.Abort();
}
connectedClient.Socket?.Dispose();
// Remove client from dictionary when done
this._clients.TryRemove(requestId, out _);
}
}
private async Task CloseAllSocketsAsync()
{
// Close all active sockets before disposing
foreach (var client in this._clients.Values)
{
if (client.Socket?.State == WebSocketState.Open)
{
await client.Socket.CloseAsync(WebSocketCloseStatus.NormalClosure, "Closing", this._mainCancellationTokenSource.Token);
}
}
}
public async ValueTask DisposeAsync()
{
try
{
this._serverIsRunning = false;
await this.CloseAllSocketsAsync(); // Close all sockets before finishing the tasks
await Task.WhenAll(this._runningTasks).ConfigureAwait(false);
this._socketCancellationTokenSource.Cancel();
this._mainCancellationTokenSource.Cancel();
}
catch (OperationCanceledException exception)
{
this._logger?.LogTrace(message: "\"Disposing web socket test server raised operation cancel exception", exception: exception);
}
finally
{
this._httpListener.Stop();
this._httpListener.Close();
this._socketCancellationTokenSource.Dispose();
this._mainCancellationTokenSource.Dispose();
}
}
#pragma warning disable VSTHRD002 // Avoid problematic synchronous waits
public void Dispose()
{
this.DisposeAsync().AsTask().GetAwaiter().GetResult();
}
}
dotnet/src/Connectors/Connectors.UnitTests/XunitLogger.cs
-40
View File
@@ -1,40 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using Microsoft.Extensions.Logging;
using Xunit.Abstractions;
namespace SemanticKernel.Connectors.UnitTests;
/// <summary>
/// A logger that writes to the Xunit test output
/// </summary>
internal sealed class XunitLogger<T> : ILogger, IDisposable
{
private readonly ITestOutputHelper _output;
public XunitLogger(ITestOutputHelper output)
{
this._output = output;
}
/// <inheritdoc/>
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func<TState, Exception?, string> formatter)
{
this._output.WriteLine(state?.ToString());
}
/// <inheritdoc/>
public bool IsEnabled(LogLevel logLevel) => true;
/// <inheritdoc/>
public IDisposable BeginScope<TState>(TState state)
=> this;
/// <inheritdoc/>
public void Dispose()
{
// This class is marked as disposable to support the BeginScope method.
// However, there is no need to dispose anything.
}
}
dotnet/src/Extensions/Extensions.UnitTests/TemplateEngine/Prompt/Blocks/CodeBlockTests.cs
+14 -11
View File
@@ -10,6 +10,7 @@ using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.AI;
using Microsoft.SemanticKernel.Diagnostics;
using Microsoft.SemanticKernel.Orchestration;
using Microsoft.SemanticKernel.Services;
using Microsoft.SemanticKernel.TemplateEngine.Basic.Blocks;
using Moq;
using Xunit;
@@ -21,6 +22,8 @@ public class CodeBlockTests
private readonly Mock<IReadOnlyFunctionCollection> _functions;
private readonly ILoggerFactory _logger = NullLoggerFactory.Instance;
private readonly Mock<IFunctionRunner> _functionRunner = new();
private readonly Mock<IAIServiceProvider> _serviceProvider = new();
private readonly Mock<IAIServiceSelector> _serviceSelector = new();
public CodeBlockTests()
{
@@ -32,7 +35,7 @@ public class CodeBlockTests
{
// Arrange
var functionRunner = new Mock<IFunctionRunner>();
var context = new SKContext(this._functionRunner.Object);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object);
var target = new CodeBlock("functionName", this._logger);
this._functionRunner.Setup(r => r.RunAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<ContextVariables>(), It.IsAny<CancellationToken>()))
@@ -49,7 +52,7 @@ public class CodeBlockTests
public async Task ItThrowsIfAFunctionCallThrowsAsync()
{
// Arrange
var context = new SKContext(this._functionRunner.Object, functions: this._functions.Object);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object, functions: this._functions.Object);
var function = new Mock<ISKFunction>();
function
.Setup(x => x.InvokeAsync(It.IsAny<SKContext>(), It.IsAny<AIRequestSettings?>(), It.IsAny<CancellationToken>()))
@@ -145,7 +148,7 @@ public class CodeBlockTests
{
// Arrange
var variables = new ContextVariables { ["varName"] = "foo" };
var context = new SKContext(this._functionRunner.Object, variables, functions: this._functions.Object);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object, variables, functions: this._functions.Object);
// Act
var codeBlock = new CodeBlock("$varName", NullLoggerFactory.Instance);
@@ -160,7 +163,7 @@ public class CodeBlockTests
{
// Arrange
var variables = new ContextVariables { ["varName"] = "bar" };
var context = new SKContext(this._functionRunner.Object, variables, functions: this._functions.Object);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object, variables, functions: this._functions.Object);
var varBlock = new VarBlock("$varName");
// Act
@@ -175,7 +178,7 @@ public class CodeBlockTests
public async Task ItRendersCodeBlockConsistingOfJustAValBlock1Async()
{
// Arrange
var context = new SKContext(this._functionRunner.Object);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object);
// Act
var codeBlock = new CodeBlock("'ciao'", NullLoggerFactory.Instance);
@@ -190,7 +193,7 @@ public class CodeBlockTests
{
// Arrange
var kernel = new Mock<IKernel>();
var context = new SKContext(this._functionRunner.Object);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object);
var valBlock = new ValBlock("'arrivederci'");
// Act
@@ -209,7 +212,7 @@ public class CodeBlockTests
const string Plugin = "pluginName";
var variables = new ContextVariables { ["input"] = "zero", ["var1"] = "uno", ["var2"] = "due" };
var context = new SKContext(this._functionRunner.Object, variables, functions: this._functions.Object);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object, variables, functions: this._functions.Object);
var funcId = new FunctionIdBlock(Func);
var canary0 = string.Empty;
@@ -257,7 +260,7 @@ public class CodeBlockTests
const string VarValue = "varValue";
var variables = new ContextVariables { [Var] = VarValue };
var context = new SKContext(this._functionRunner.Object, variables, functions: this._functions.Object);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object, variables, functions: this._functions.Object);
var funcId = new FunctionIdBlock(Func);
var varBlock = new VarBlock($"${Var}");
@@ -290,7 +293,7 @@ public class CodeBlockTests
const string Plugin = "pluginName";
const string Value = "value";
var context = new SKContext(this._functionRunner.Object, variables: null, functions: this._functions.Object);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object, variables: null, functions: this._functions.Object);
var funcId = new FunctionIdBlock(Func);
var valBlock = new ValBlock($"'{Value}'");
@@ -328,7 +331,7 @@ public class CodeBlockTests
var variables = new ContextVariables();
variables.Set("bob", BobValue);
variables.Set("input", Value);
var context = new SKContext(this._functionRunner.Object, variables: variables, functions: this._functions.Object);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object, variables: variables, functions: this._functions.Object);
var funcId = new FunctionIdBlock(Func);
var namedArgBlock1 = new NamedArgBlock($"foo='{FooValue}'");
var namedArgBlock2 = new NamedArgBlock("baz=$bob");
@@ -362,7 +365,7 @@ public class CodeBlockTests
this._functionRunner.Setup(r => r.RunAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<ContextVariables>(), It.IsAny<CancellationToken>()))
.Returns<string, string, ContextVariables, CancellationToken>((pluginName, functionName, variables, cancellationToken) =>
{
var context = new SKContext(this._functionRunner.Object, variables);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object, variables);
return function.InvokeAsync(context, null, cancellationToken);
});
}
dotnet/src/Extensions/Extensions.UnitTests/TemplateEngine/Prompt/PromptTemplateEngineTests.cs
+9 -2
View File
@@ -11,6 +11,7 @@ using System.Threading.Tasks;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Diagnostics;
using Microsoft.SemanticKernel.Orchestration;
using Microsoft.SemanticKernel.Services;
using Microsoft.SemanticKernel.TemplateEngine.Basic;
using Microsoft.SemanticKernel.TemplateEngine.Basic.Blocks;
using Moq;
@@ -29,6 +30,8 @@ public sealed class PromptTemplateEngineTests
private readonly ITestOutputHelper _logger;
private readonly Mock<IKernel> _kernel;
private readonly Mock<IFunctionRunner> _functionRunner;
private readonly Mock<IAIServiceProvider> _serviceProvider;
private readonly Mock<IAIServiceSelector> _serviceSelector;
public PromptTemplateEngineTests(ITestOutputHelper testOutputHelper)
{
@@ -38,6 +41,8 @@ public sealed class PromptTemplateEngineTests
this._functions = new Mock<IReadOnlyFunctionCollection>();
this._kernel = new Mock<IKernel>();
this._functionRunner = new Mock<IFunctionRunner>();
this._serviceProvider = new Mock<IAIServiceProvider>();
this._serviceSelector = new Mock<IAIServiceSelector>();
}
[Fact]
@@ -375,7 +380,7 @@ public sealed class PromptTemplateEngineTests
this._functionRunner.Setup(r => r.RunAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<ContextVariables>(), It.IsAny<CancellationToken>()))
.Returns<string, string, ContextVariables, CancellationToken>((pluginName, functionName, variables, cancellationToken) =>
{
var context = new SKContext(this._functionRunner.Object, variables);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object, variables);
return function.InvokeAsync(context, null, cancellationToken);
});
}
@@ -385,7 +390,7 @@ public sealed class PromptTemplateEngineTests
this._functionRunner.Setup(r => r.RunAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<ContextVariables>(), It.IsAny<CancellationToken>()))
.Returns<string, string, ContextVariables, CancellationToken>((pluginName, functionName, variables, cancellationToken) =>
{
var context = new SKContext(this._functionRunner.Object, variables);
var context = new SKContext(this._functionRunner.Object, this._serviceProvider.Object, this._serviceSelector.Object, variables);
var function = functions.First(f => f.PluginName == functionName);
return function.InvokeAsync(context, null, cancellationToken);
@@ -396,6 +401,8 @@ public sealed class PromptTemplateEngineTests
{
return new SKContext(
this._functionRunner.Object,
this._serviceProvider.Object,
this._serviceSelector.Object,
this._variables,
this._functions.Object);
}
dotnet/src/IntegrationTests/Connectors/Oobabooga/OobaboogaTextCompletionTests.cs
-111
View File
@@ -1,111 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Net.WebSockets;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Microsoft.SemanticKernel.AI.TextCompletion;
using Microsoft.SemanticKernel.Connectors.AI.Oobabooga.TextCompletion;
using Xunit;
namespace SemanticKernel.IntegrationTests.Connectors.Oobabooga;
/// <summary>
/// Integration tests for <see cref=" OobaboogaTextCompletion"/>.
/// </summary>
[Obsolete("This functionality is available as part of new NuGet package: https://www.nuget.org/packages/MyIA.SemanticKernel.Connectors.AI.Oobabooga/. This will be removed in a future release.")]
public sealed class OobaboogaTextCompletionTests : IDisposable
{
private const string Endpoint = "http://localhost";
private const int BlockingPort = 5000;
private const int StreamingPort = 5005;
private readonly IConfigurationRoot _configuration;
private readonly List<ClientWebSocket> _webSockets = new();
private readonly Func<ClientWebSocket> _webSocketFactory;
public OobaboogaTextCompletionTests()
{
// Load configuration
this._configuration = new ConfigurationBuilder()
.AddJsonFile(path: "testsettings.json", optional: false, reloadOnChange: true)
.AddJsonFile(path: "testsettings.development.json", optional: true, reloadOnChange: true)
.AddEnvironmentVariables()
.Build();
this._webSocketFactory = () =>
{
var toReturn = new ClientWebSocket();
this._webSockets.Add(toReturn);
return toReturn;
};
}
private const string Input = " My name is";
[Fact(Skip = "This test is for manual verification.")]
public async Task OobaboogaLocalTextCompletionAsync()
{
var oobaboogaLocal = new OobaboogaTextCompletion(
endpoint: new Uri(Endpoint),
blockingPort: BlockingPort);
// Act
var localResponse = await oobaboogaLocal.CompleteAsync(Input, requestSettings: new TextCompletionRequest()
{
Temperature = 0.01,
MaxNewTokens = 7,
TopP = 0.1,
});
AssertAcceptableResponse(localResponse);
}
[Fact(Skip = "This test is for manual verification.")]
public async Task OobaboogaLocalTextCompletionStreamingAsync()
{
var oobaboogaLocal = new OobaboogaTextCompletion(
endpoint: new Uri(Endpoint),
streamingPort: StreamingPort,
webSocketFactory: this._webSocketFactory);
// Act
var localResponse = oobaboogaLocal.CompleteStreamAsync(Input, requestSettings: new TextCompletionRequest()
{
Temperature = 0.01,
MaxNewTokens = 7,
TopP = 0.1,
});
StringBuilder stringBuilder = new();
await foreach (var result in localResponse)
{
stringBuilder.Append(result);
}
var resultsMerged = stringBuilder.ToString();
AssertAcceptableResponse(resultsMerged);
}
private static void AssertAcceptableResponse(string localResponse)
{
// Assert
Assert.NotNull(localResponse);
// Depends on the target LLM obviously, but most LLMs should propose an arbitrary surname preceded by a white space, including the start prompt or not
// ie " My name is" => " John (...)" or " My name is" => " My name is John (...)".
// Here are a couple LLMs that were tested successfully: gpt2, aisquared_dlite-v1-355m, bigscience_bloomz-560m, eachadea_vicuna-7b-1.1, TheBloke_WizardLM-30B-GPTQ etc.
// A few will return an empty string, but well those shouldn't be used for integration tests.
var expectedRegex = new Regex(@"\s\w+.*");
Assert.Matches(expectedRegex, localResponse);
}
public void Dispose()
{
foreach (ClientWebSocket clientWebSocket in this._webSockets)
{
clientWebSocket.Dispose();
}
}
}
dotnet/src/IntegrationTests/Extensions/KernelSemanticFunctionExtensionsTests.cs
+2 -2
View File
@@ -27,7 +27,7 @@ public sealed class KernelSemanticFunctionExtensionsTests : IDisposable
[Fact]
public async Task ItSupportsFunctionCallsAsync()
{
var builder = Kernel.Builder
var builder = new KernelBuilder()
.WithAIService<ITextCompletion>(null, new RedirectTextCompletion(), true)
.WithLoggerFactory(this._logger);
IKernel target = builder.Build();
@@ -46,7 +46,7 @@ public sealed class KernelSemanticFunctionExtensionsTests : IDisposable
[Fact]
public async Task ItSupportsFunctionCallsWithInputAsync()
{
var builder = Kernel.Builder
var builder = new KernelBuilder()
.WithAIService<ITextCompletion>(null, new RedirectTextCompletion(), true)
.WithLoggerFactory(this._logger);
IKernel target = builder.Build();
dotnet/src/IntegrationTests/IntegrationTests.csproj
-1
View File
@@ -31,7 +31,6 @@
<PackageVersion Update="System.Text.Json" Version="7.0.3" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\Connectors\Connectors.AI.Oobabooga\Connectors.AI.Oobabooga.csproj" />
<ProjectReference Include="..\Connectors\Connectors.AI.OpenAI\Connectors.AI.OpenAI.csproj" />
<ProjectReference Include="..\Connectors\Connectors.AI.HuggingFace\Connectors.AI.HuggingFace.csproj" />
<ProjectReference Include="..\Connectors\Connectors.Memory.Chroma\Connectors.Memory.Chroma.csproj" />
dotnet/src/IntegrationTests/Planners/PlanTests.cs
+1 -1
View File
@@ -507,7 +507,7 @@ public sealed class PlanTests : IDisposable
AzureOpenAIConfiguration? azureOpenAIEmbeddingsConfiguration = this._configuration.GetSection("AzureOpenAIEmbeddings").Get<AzureOpenAIConfiguration>();
Assert.NotNull(azureOpenAIEmbeddingsConfiguration);
var builder = Kernel.Builder
var builder = new KernelBuilder()
.WithLoggerFactory(this._loggerFactory)
.WithRetryBasic();
dotnet/src/IntegrationTests/Planners/SequentialPlanner/SequentialPlanParserTests.cs
+1 -1
View File
@@ -32,7 +32,7 @@ public class SequentialPlanParserTests
AzureOpenAIConfiguration? azureOpenAIConfiguration = this._configuration.GetSection("AzureOpenAI").Get<AzureOpenAIConfiguration>();
Assert.NotNull(azureOpenAIConfiguration);
IKernel kernel = Kernel.Builder
IKernel kernel = new KernelBuilder()
.WithRetryBasic()
.WithAzureTextCompletionService(
deploymentName: azureOpenAIConfiguration.DeploymentName,
dotnet/src/IntegrationTests/Planners/SequentialPlanner/SequentialPlannerTests.cs
+1 -1
View File
@@ -117,7 +117,7 @@ public sealed class SequentialPlannerTests : IDisposable
AzureOpenAIConfiguration? azureOpenAIEmbeddingsConfiguration = this._configuration.GetSection("AzureOpenAIEmbeddings").Get<AzureOpenAIConfiguration>();
Assert.NotNull(azureOpenAIEmbeddingsConfiguration);
var builder = Kernel.Builder.WithLoggerFactory(this._logger);
var builder = new KernelBuilder().WithLoggerFactory(this._logger);
builder.WithRetryBasic();
if (useChatModel)
dotnet/src/IntegrationTests/Planners/StepwisePlanner/StepwisePlannerTests.cs
+1 -1
View File
@@ -147,7 +147,7 @@ public sealed class StepwisePlannerTests : IDisposable
AzureOpenAIConfiguration? azureOpenAIEmbeddingsConfiguration = this._configuration.GetSection("AzureOpenAIEmbeddings").Get<AzureOpenAIConfiguration>();
Assert.NotNull(azureOpenAIEmbeddingsConfiguration);
var builder = Kernel.Builder
var builder = new KernelBuilder()
.WithLoggerFactory(this._loggerFactory)
.WithRetryBasic();
dotnet/src/IntegrationTests/Plugins/SamplePluginsTests.cs
-4
View File
@@ -25,8 +25,6 @@ public class SamplePluginsTests
{
var function = kernel.Functions.GetFunction(view.PluginName, view.Name);
Assert.NotNull(function);
Assert.NotNull(function.RequestSettings);
Assert.True(function.RequestSettings.ExtensionData.ContainsKey("max_tokens"));
});
}
@@ -49,8 +47,6 @@ public class SamplePluginsTests
{
var function = kernel.Functions.GetFunction(view.PluginName, view.Name);
Assert.NotNull(function);
Assert.NotNull(function.RequestSettings);
Assert.True(function.RequestSettings.ExtensionData.ContainsKey("max_tokens"));
});
}
}
dotnet/src/IntegrationTests/TemplateLanguage/PromptTemplateEngineTests.cs
+8 -8
View File
@@ -32,7 +32,7 @@ public sealed class PromptTemplateEngineTests : IDisposable
const string Winner = "SK";
const string Template = "And the winner\n of {{$input}} \nis: {{ $winner }}!";
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
var context = kernel.CreateNewContext();
context.Variables["input"] = Input;
context.Variables["winner"] = Winner;
@@ -54,7 +54,7 @@ public sealed class PromptTemplateEngineTests : IDisposable
const string Template = "And the winner\n of {{'template\ntests'}} \nis: {{ \"SK\" }}!";
const string Expected = "And the winner\n of template\ntests \nis: SK!";
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
var context = kernel.CreateNewContext();
// Act
@@ -69,7 +69,7 @@ public sealed class PromptTemplateEngineTests : IDisposable
{
// Arrange
const string Template = "== {{my.check123 $call}} ==";
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
kernel.ImportFunctions(new MyPlugin(), "my");
var context = kernel.CreateNewContext();
context.Variables["call"] = "123";
@@ -86,7 +86,7 @@ public sealed class PromptTemplateEngineTests : IDisposable
{
// Arrange
const string Template = "== {{my.check123 '234'}} ==";
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
kernel.ImportFunctions(new MyPlugin(), "my");
var context = kernel.CreateNewContext();
@@ -103,7 +103,7 @@ public sealed class PromptTemplateEngineTests : IDisposable
// Arrange
const char Esc = '\\';
string template = "== {{my.check123 'a" + Esc + "'b'}} ==";
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
kernel.ImportFunctions(new MyPlugin(), "my");
var context = kernel.CreateNewContext();
@@ -120,7 +120,7 @@ public sealed class PromptTemplateEngineTests : IDisposable
// Arrange
const char Esc = '\\';
string template = "== {{my.check123 \"a" + Esc + "\"b\"}} ==";
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
kernel.ImportFunctions(new MyPlugin(), "my");
var context = kernel.CreateNewContext();
@@ -136,7 +136,7 @@ public sealed class PromptTemplateEngineTests : IDisposable
{
// Arrange
string template = "Output: {{my.sayAge name=\"Mario\" birthdate=$birthdate exclamation='Wow, that\\'s surprising'}}";
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
kernel.ImportFunctions(new MyPlugin(), "my");
var context = kernel.CreateNewContext();
context.Variables["birthdate"] = "1981-08-20T00:00:00";
@@ -153,7 +153,7 @@ public sealed class PromptTemplateEngineTests : IDisposable
public async Task ItHandleEdgeCasesAsync(string template, string expectedResult)
{
// Arrange
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
kernel.ImportFunctions(new MyPlugin());
// Act
dotnet/src/IntegrationTests/WebPlugin/WebPluginTests.cs
+2 -2
View File
@@ -43,7 +43,7 @@ public sealed class WebPluginTests : IDisposable
public async Task BingPluginTestAsync(string prompt, string expectedAnswerContains)
{
// Arrange
IKernel kernel = Kernel.Builder.WithLoggerFactory(this._logger).Build();
IKernel kernel = new KernelBuilder().WithLoggerFactory(this._logger).Build();
using XunitLogger<BingConnector> connectorLogger = new(this._output);
BingConnector connector = new(this._bingApiKey, connectorLogger);
@@ -66,7 +66,7 @@ public sealed class WebPluginTests : IDisposable
public async Task WebFileDownloadPluginFileTestAsync()
{
// Arrange
IKernel kernel = Kernel.Builder.WithLoggerFactory(this._logger).Build();
IKernel kernel = new KernelBuilder().WithLoggerFactory(this._logger).Build();
using XunitLogger<WebFileDownloadPlugin> pluginLogger = new(this._output);
var plugin = new WebFileDownloadPlugin(pluginLogger);
var downloadFunctions = kernel.ImportFunctions(plugin, "WebFileDownload");
dotnet/src/InternalUtilities/test/FunctionHelpers.cs
+1 -1
View File
@@ -18,7 +18,7 @@ internal static class FunctionHelpers
string methodName,
params (string Name, object Value)[] variables)
{
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
IDictionary<string, ISKFunction> functions = kernel.ImportFunctions(pluginInstance);
dotnet/src/Planners/Planners.Core.UnitTests/Action/ActionPlannerTests.cs
+5 -2
View File
@@ -6,6 +6,7 @@ using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.SemanticKernel.AI;
using Microsoft.SemanticKernel.Diagnostics;
using Microsoft.SemanticKernel.Orchestration;
using Microsoft.SemanticKernel.Services;
using Moq;
using Xunit;
@@ -163,11 +164,13 @@ This plan uses the `GitHubPlugin.PullsList` function to list the open pull reque
functions.Setup(x => x.GetFunctionViews()).Returns(new List<FunctionView>());
}
var functionRunner = new Mock<IFunctionRunner>();
var serviceProvider = new Mock<IAIServiceProvider>();
var serviceSelector = new Mock<IAIServiceSelector>();
var kernel = new Mock<IKernel>();
var returnContext = new SKContext(functionRunner.Object, new ContextVariables(testPlanString), functions.Object);
var returnContext = new SKContext(functionRunner.Object, serviceProvider.Object, serviceSelector.Object, new ContextVariables(testPlanString), functions.Object);
var context = new SKContext(functionRunner.Object, functions: functions.Object);
var context = new SKContext(functionRunner.Object, serviceProvider.Object, serviceSelector.Object, functions: functions.Object);
var mockFunctionFlowFunction = new Mock<ISKFunction>();
mockFunctionFlowFunction.Setup(x => x.InvokeAsync(
dotnet/src/Planners/Planners.Core.UnitTests/Extensions/ReadOnlyFunctionCollectionExtensionsTests.cs
+13 -4
View File
@@ -2,6 +2,7 @@
using Microsoft.SemanticKernel.Memory;
using Microsoft.SemanticKernel.Orchestration;
using Microsoft.SemanticKernel.Services;
using Moq;
using Xunit;
@@ -57,9 +58,11 @@ public class ReadOnlyFunctionCollectionExtensionsTests
.Returns(asyncEnumerable);
var functionRunner = new Mock<IFunctionRunner>();
var serviceProvider = new Mock<IAIServiceProvider>();
var serviceSelector = new Mock<IAIServiceSelector>();
// Arrange GetAvailableFunctionsAsync parameters
var context = new SKContext(functionRunner.Object, variables);
var context = new SKContext(functionRunner.Object, serviceProvider.Object, serviceSelector.Object, variables);
var config = InitializeConfig(t);
var semanticQuery = "test";
@@ -136,9 +139,11 @@ public class ReadOnlyFunctionCollectionExtensionsTests
functions.Setup(x => x.GetFunctionViews()).Returns(functionsView);
var functionRunner = new Mock<IFunctionRunner>();
var serviceProvider = new Mock<IAIServiceProvider>();
var serviceSelector = new Mock<IAIServiceSelector>();
// Arrange GetAvailableFunctionsAsync parameters
var context = new SKContext(functionRunner.Object, variables, functions.Object);
var context = new SKContext(functionRunner.Object, serviceProvider.Object, serviceSelector.Object, variables, functions.Object);
var config = InitializeConfig(t);
var semanticQuery = "test";
@@ -206,9 +211,11 @@ public class ReadOnlyFunctionCollectionExtensionsTests
functions.Setup(x => x.GetFunctionViews()).Returns(functionsView);
var functionRunner = new Mock<IFunctionRunner>();
var serviceProvider = new Mock<IAIServiceProvider>();
var serviceSelector = new Mock<IAIServiceSelector>();
// Arrange GetAvailableFunctionsAsync parameters
var context = new SKContext(functionRunner.Object, variables, functions.Object);
var context = new SKContext(functionRunner.Object, serviceProvider.Object, serviceSelector.Object, variables, functions.Object);
var config = InitializeConfig(t);
config.SemanticMemoryConfig = new() { RelevancyThreshold = 0.78, Memory = memory.Object };
var semanticQuery = "test";
@@ -243,6 +250,8 @@ public class ReadOnlyFunctionCollectionExtensionsTests
// Arrange
var kernel = new Mock<IKernel>();
var functionRunner = new Mock<IFunctionRunner>();
var serviceProvider = new Mock<IAIServiceProvider>();
var serviceSelector = new Mock<IAIServiceSelector>();
var variables = new ContextVariables();
var functions = new FunctionCollection();
@@ -267,7 +276,7 @@ public class ReadOnlyFunctionCollectionExtensionsTests
.Returns(asyncEnumerable);
// Arrange GetAvailableFunctionsAsync parameters
var context = new SKContext(functionRunner.Object, variables);
var context = new SKContext(functionRunner.Object, serviceProvider.Object, serviceSelector.Object, variables);
var config = InitializeConfig(t);
config.SemanticMemoryConfig = new() { RelevancyThreshold = 0.78, Memory = memory.Object };
var semanticQuery = "test";
dotnet/src/Planners/Planners.Core.UnitTests/Sequential/SequentialPlanParserTests.cs
+8 -3
View File
@@ -4,6 +4,7 @@ using System.Globalization;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel.Diagnostics;
using Microsoft.SemanticKernel.Orchestration;
using Microsoft.SemanticKernel.Services;
using Microsoft.SemanticKernel.TemplateEngine;
using Moq;
using Xunit;
@@ -38,9 +39,11 @@ public class SequentialPlanParserTests
private SKContext CreateSKContext(
IFunctionRunner functionRunner,
IAIServiceProvider serviceProvider,
IAIServiceSelector serviceSelector,
ContextVariables? variables = null)
{
return new SKContext(functionRunner, variables);
return new SKContext(functionRunner, serviceProvider, serviceSelector, variables);
}
private static Mock<ISKFunction> CreateMockFunction(FunctionView functionView, string result = "")
@@ -59,12 +62,14 @@ public class SequentialPlanParserTests
kernel = kernelMock.Object;
var functionRunnerMock = new Mock<IFunctionRunner>();
var serviceProviderMock = new Mock<IAIServiceProvider>();
var serviceSelector = new Mock<IAIServiceSelector>();
// For Create
kernelMock.Setup(k => k.CreateNewContext(It.IsAny<ContextVariables>(), It.IsAny<IReadOnlyFunctionCollection>(), It.IsAny<ILoggerFactory>(), It.IsAny<CultureInfo>()))
.Returns<ContextVariables, IReadOnlyFunctionCollection, ILoggerFactory, CultureInfo>((contextVariables, skills, loggerFactory, culture) =>
{
return this.CreateSKContext(functionRunnerMock.Object, contextVariables);
return this.CreateSKContext(functionRunnerMock.Object, serviceProviderMock.Object, serviceSelector.Object, contextVariables);
});
var functionsView = new List<FunctionView>();
@@ -77,7 +82,7 @@ public class SequentialPlanParserTests
var mockFunction = CreateMockFunction(functionView);
functionsView.Add(functionView);
var result = this.CreateSKContext(functionRunnerMock.Object);
var result = this.CreateSKContext(functionRunnerMock.Object, serviceProviderMock.Object, serviceSelector.Object);
result.Variables.Update(resultString);
mockFunction.Setup(x => x.InvokeAsync(It.IsAny<SKContext>(), null, It.IsAny<CancellationToken>()))
.ReturnsAsync(new FunctionResult(name, pluginName, result));
dotnet/src/Planners/Planners.Core.UnitTests/Sequential/SequentialPlannerTests.cs
+11 -2
View File
@@ -5,6 +5,7 @@ using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel.AI;
using Microsoft.SemanticKernel.Diagnostics;
using Microsoft.SemanticKernel.Orchestration;
using Microsoft.SemanticKernel.Services;
using Moq;
using Xunit;
@@ -60,6 +61,8 @@ public sealed class SequentialPlannerTests
functions.Setup(x => x.GetFunctionViews()).Returns(functionsView);
var functionRunner = new Mock<IFunctionRunner>();
var serviceProvider = new Mock<IAIServiceProvider>();
var serviceSelector = new Mock<IAIServiceSelector>();
kernel.Setup(x => x.LoggerFactory).Returns(new Mock<ILoggerFactory>().Object);
var expectedFunctions = input.Select(x => x.name).ToList();
@@ -67,10 +70,14 @@ public sealed class SequentialPlannerTests
var context = new SKContext(
functionRunner.Object,
serviceProvider.Object,
serviceSelector.Object,
new ContextVariables());
var returnContext = new SKContext(
functionRunner.Object,
serviceProvider.Object,
serviceSelector.Object,
new ContextVariables());
var planString =
@@ -147,15 +154,17 @@ public sealed class SequentialPlannerTests
{
// Arrange
var functionRunner = new Mock<IFunctionRunner>();
var serviceProvider = new Mock<IAIServiceProvider>();
var serviceSelector = new Mock<IAIServiceSelector>();
var kernel = new Mock<IKernel>();
var functions = new Mock<IFunctionCollection>();
functions.Setup(x => x.GetFunctionViews()).Returns(new List<FunctionView>());
var planString = "<plan>notvalid<</plan>";
var returnContext = new SKContext(functionRunner.Object, new ContextVariables(planString));
var returnContext = new SKContext(functionRunner.Object, serviceProvider.Object, serviceSelector.Object, new ContextVariables(planString));
var context = new SKContext(functionRunner.Object, new ContextVariables());
var context = new SKContext(functionRunner.Object, serviceProvider.Object, serviceSelector.Object, new ContextVariables());
var mockFunctionFlowFunction = new Mock<ISKFunction>();
mockFunctionFlowFunction.Setup(x => x.InvokeAsync(
dotnet/src/Plugins/Plugins.Core/ConversationSummaryPlugin.cs
+1 -1
View File
@@ -14,7 +14,7 @@ namespace Microsoft.SemanticKernel.Plugins.Core;
/// </summary>
/// <example>
/// <code>
/// var kernel Kernel.Builder.Build();
/// var kernel = new KernelBuilder().Build();
/// kernel.ImportFunctions(new ConversationSummaryPlugin(kernel));
/// </code>
/// </example>
dotnet/src/Plugins/Plugins.UnitTests/Core/FileIOPluginTests.cs
+1 -1
View File
@@ -22,7 +22,7 @@ public class FileIOPluginTests
public void ItCanBeImported()
{
// Arrange
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
// Act
var functions = kernel.ImportFunctions(new FileIOPlugin(), "fileIO");
dotnet/src/Plugins/Plugins.UnitTests/Core/MathPluginTests.cs
+1 -1
View File
@@ -22,7 +22,7 @@ public class MathPluginTests
public void ItCanBeImported()
{
// Arrange
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
// Act - Assert no exception occurs e.g. due to reflection
kernel.ImportFunctions(new MathPlugin(), "math");
dotnet/src/Plugins/Plugins.UnitTests/Core/TextPluginTests.cs
+1 -1
View File
@@ -19,7 +19,7 @@ public class TextPluginTests
public void ItCanBeImported()
{
// Arrange
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
// Act - Assert no exception occurs e.g. due to reflection
kernel.ImportFunctions(new TextPlugin(), "text");
dotnet/src/Plugins/Plugins.UnitTests/Core/TimePluginTests.cs
+1 -1
View File
@@ -25,7 +25,7 @@ public class TimePluginTests
public void ItCanBeImported()
{
// Arrange
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
// Act - Assert no exception occurs e.g. due to reflection
kernel.ImportFunctions(new TimePlugin(), "time");
dotnet/src/Plugins/Plugins.UnitTests/Core/WaitPluginTests.cs
+1 -1
View File
@@ -24,7 +24,7 @@ public class WaitPluginTests
public void ItCanBeImported()
{
// Arrange
var kernel = Kernel.Builder.Build();
var kernel = new KernelBuilder().Build();
// Act - Assert no exception occurs e.g. due to reflection
kernel.ImportFunctions(new WaitPlugin(), "wait");
dotnet/src/Plugins/Plugins.UnitTests/Web/SearchUrlSkillTests.cs
+1 -1
View File
@@ -23,7 +23,7 @@ public class SearchUrlPluginTests
public void ItCanBeImported()
{
// Arrange
IKernel kernel = Kernel.Builder.Build();
IKernel kernel = new KernelBuilder().Build();
// Act - Assert no exception occurs e.g. due to reflection
kernel.ImportFunctions(new SearchUrlPlugin(), "search");
dotnet/src/SemanticKernel.Abstractions/Events/SKCancelEventArgs.cs
-8
View File
@@ -29,12 +29,4 @@ public abstract class SKCancelEventArgs : SKEventArgs
{
this._cancelTokenSource.Cancel();
}
/// <summary>
/// Dispose resources.
/// </summary>
~SKCancelEventArgs()
{
this._cancelTokenSource.Dispose();
}
}
dotnet/src/SemanticKernel.Abstractions/Functions/ISKFunction.cs
+10 -7
View File
@@ -33,11 +33,6 @@ public interface ISKFunction
/// </summary>
string Description { get; }
/// <summary>
/// AI service settings
/// </summary>
AIRequestSettings? RequestSettings { get; }
/// <summary>
/// Returns a description of the function, including parameters.
/// </summary>
@@ -56,12 +51,21 @@ public interface ISKFunction
AIRequestSettings? requestSettings = null,
CancellationToken cancellationToken = default);
#region Obsolete
/// <summary>
/// AI service settings
/// </summary>
[Obsolete("Use PromptTemplateConfig.ModelSettings instead. This will be removed in a future release.")]
AIRequestSettings? RequestSettings { get; }
/// <summary>
/// Set the AI service used by the semantic function, passing a factory method.
/// The factory allows to lazily instantiate the client and to properly handle its disposal.
/// </summary>
/// <param name="serviceFactory">AI service factory</param>
/// <returns>Self instance</returns>
[Obsolete("Use implementation of IAIServiceConfigurationProvider instead. This will be removed in a future release.")]
ISKFunction SetAIService(Func<ITextCompletion> serviceFactory);
/// <summary>
@@ -69,10 +73,9 @@ public interface ISKFunction
/// </summary>
/// <param name="requestSettings">LLM completion settings</param>
/// <returns>Self instance</returns>
[Obsolete("Use implementation of IAIServiceConfigurationProvider instead. This will be removed in a future release.")]
ISKFunction SetAIConfiguration(AIRequestSettings? requestSettings);
#region Obsolete
/// <summary>
/// Name of the plugin containing the function. The name is used by the function collection and in prompt templates e.g. {{skillName.functionName}}
/// </summary>
dotnet/src/SemanticKernel.Abstractions/Orchestration/SKContext.cs
+19
View File
@@ -7,6 +7,7 @@ using System.Globalization;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.SemanticKernel.Diagnostics;
using Microsoft.SemanticKernel.Services;
namespace Microsoft.SemanticKernel.Orchestration;
@@ -58,16 +59,30 @@ public sealed class SKContext
/// </summary>
public IFunctionRunner Runner { get; }
/// <summary>
/// AI service provider
/// </summary>
internal IAIServiceProvider ServiceProvider { get; }
/// <summary>
/// AIService selector implementation
/// </summary>
internal IAIServiceSelector ServiceSelector { get; }
/// <summary>
/// Constructor for the context.
/// </summary>
/// <param name="functionRunner">Function runner reference</param>
/// <param name="serviceProvider">AI service provider</param>
/// <param name="serviceSelector">AI service selector</param>
/// <param name="variables">Context variables to include in context.</param>
/// <param name="functions">Functions to include in context.</param>
/// <param name="loggerFactory">Logger factory to be used in context</param>
/// <param name="culture">Culture related to the context</param>
internal SKContext(
IFunctionRunner functionRunner,
IAIServiceProvider serviceProvider,
IAIServiceSelector serviceSelector,
ContextVariables? variables = null,
IReadOnlyFunctionCollection? functions = null,
ILoggerFactory? loggerFactory = null,
@@ -76,6 +91,8 @@ public sealed class SKContext
Verify.NotNull(functionRunner, nameof(functionRunner));
this.Runner = functionRunner;
this.ServiceProvider = serviceProvider;
this.ServiceSelector = serviceSelector;
this.Variables = variables ?? new();
this.Functions = functions ?? NullReadOnlyFunctionCollection.Instance;
this.LoggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
@@ -110,6 +127,8 @@ public sealed class SKContext
{
return new SKContext(
this.Runner,
this.ServiceProvider,
this.ServiceSelector,
variables ?? this.Variables.Clone(),
functions ?? this.Functions,
this.LoggerFactory,
dotnet/src/SemanticKernel.Abstractions/Services/IAIServiceSelector.cs
+26
View File
@@ -0,0 +1,26 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using Microsoft.SemanticKernel.AI;
using Microsoft.SemanticKernel.Services;
#pragma warning disable IDE0130
// ReSharper disable once CheckNamespace - Using the main namespace
namespace Microsoft.SemanticKernel;
#pragma warning restore IDE0130
/// <summary>
/// Selector which will return a tuple containing instances of <see cref="IAIService"/> and <see cref="AIRequestSettings"/> from the specified provider based on the model settings.
/// </summary>
public interface IAIServiceSelector
{
/// <summary>
/// Return the AI service and requesting settings from the specified provider based on the model settings.
/// The returned value is a tuple containing instances of <see cref="IAIService"/> and <see cref="AIRequestSettings"/>
/// </summary>
/// <typeparam name="T">Type of AI service to return</typeparam>
/// <param name="renderedPrompt">Rendered prompt</param>
/// <param name="serviceProvider">AI service provider</param>
/// <param name="modelSettings">Collection of model settings</param>
/// <returns></returns>
(T?, AIRequestSettings?) SelectAIService<T>(string renderedPrompt, IAIServiceProvider serviceProvider, IReadOnlyList<AIRequestSettings>? modelSettings) where T : IAIService;
}
dotnet/src/SemanticKernel.Core/Extensions/KernelSemanticFunctionExtensions.cs
+1 -9
View File
@@ -7,7 +7,6 @@ using System.IO;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel.AI;
using Microsoft.SemanticKernel.AI.TextCompletion;
using Microsoft.SemanticKernel.Diagnostics;
using Microsoft.SemanticKernel.Orchestration;
using Microsoft.SemanticKernel.TemplateEngine;
@@ -278,19 +277,12 @@ public static class KernelSemanticFunctionExtensions
PromptTemplateConfig promptTemplateConfig,
IPromptTemplate promptTemplate)
{
ISKFunction func = SemanticFunction.FromSemanticConfig(
return SemanticFunction.FromSemanticConfig(
pluginName,
functionName,
promptTemplateConfig,
promptTemplate,
kernel.LoggerFactory
);
func.SetAIConfiguration(promptTemplateConfig.GetDefaultRequestSettings());
// Note: the service is instantiated using the kernel configuration state when the function is invoked
func.SetAIService(() => kernel.GetService<ITextCompletion>(promptTemplateConfig.GetDefaultRequestSettings()?.ServiceId ?? null));
return func;
}
}
dotnet/src/SemanticKernel.Core/Extensions/SKFunctionExtensions.cs
+3 -1
View File
@@ -1,5 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Globalization;
using System.Threading;
using System.Threading.Tasks;
@@ -22,6 +23,7 @@ public static class SKFunctionExtensions
/// <param name="skFunction">Semantic function</param>
/// <param name="requestSettings">Request settings</param>
/// <returns>Self instance</returns>
[Obsolete("Use implementation of IAIServiceConfigurationProvider instead. This will be removed in a future release.")]
public static ISKFunction UseCompletionSettings(this ISKFunction skFunction, AIRequestSettings requestSettings)
{
return skFunction.SetAIConfiguration(requestSettings);
@@ -49,7 +51,7 @@ public static class SKFunctionExtensions
CancellationToken cancellationToken = default)
{
var context = kernel.CreateNewContext(variables, functions, loggerFactory, culture);
return function.InvokeAsync(context, requestSettings ?? function.RequestSettings, cancellationToken);
return function.InvokeAsync(context, requestSettings, cancellationToken);
}
/// <summary>
dotnet/src/SemanticKernel.Core/Functions/DelegatingAIServiceSelector.cs
+25
View File
@@ -0,0 +1,25 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using Microsoft.SemanticKernel.AI;
using Microsoft.SemanticKernel.AI.TextCompletion;
using Microsoft.SemanticKernel.Services;
namespace Microsoft.SemanticKernel.Functions;
/// <summary>
/// Added for backward compatibility only, this will be removed when ISKFunction.SetAIService and ISKFunction.SetAIConfiguration are removed.
/// </summary>
[Obsolete("Remove this when ISKFunction.SetAIService and ISKFunction.SetAIConfiguration are removed.")]
internal class DelegatingAIServiceSelector : IAIServiceSelector
{
internal Func<ITextCompletion>? ServiceFactory { get; set; }
internal AIRequestSettings? RequestSettings { get; set; }
/// <inheritdoc/>
public (T?, AIRequestSettings?) SelectAIService<T>(string renderedPrompt, IAIServiceProvider serviceProvider, IReadOnlyList<AIRequestSettings>? modelSettings) where T : IAIService
{
return ((T?)this.ServiceFactory?.Invoke() ?? serviceProvider.GetService<T>(null), this.RequestSettings ?? modelSettings?[0]);
}
}
dotnet/src/SemanticKernel.Core/Functions/InstrumentedSKFunction.cs
+14 -11
View File
@@ -31,9 +31,6 @@ internal sealed class InstrumentedSKFunction : ISKFunction
/// <inheritdoc/>
public string Description => this._function.Description;
/// <inheritdoc/>
public AIRequestSettings? RequestSettings => this._function.RequestSettings;
/// <summary>
/// Initialize a new instance of the <see cref="InstrumentedSKFunction"/> class.
/// </summary>
@@ -78,14 +75,6 @@ internal sealed class InstrumentedSKFunction : ISKFunction
this._function.InvokeAsync(context, requestSettings, cancellationToken)).ConfigureAwait(false);
}
/// <inheritdoc/>
public ISKFunction SetAIConfiguration(AIRequestSettings? requestSettings) =>
this._function.SetAIConfiguration(requestSettings);
/// <inheritdoc/>
public ISKFunction SetAIService(Func<ITextCompletion> serviceFactory) =>
this._function.SetAIService(serviceFactory);
#region private ================================================================================
private readonly ISKFunction _function;
@@ -174,6 +163,20 @@ internal sealed class InstrumentedSKFunction : ISKFunction
#region Obsolete =======================================================================
/// <inheritdoc/>
[Obsolete("Use ISKFunction.RequestSettingsFactory instead. This will be removed in a future release.")]
public AIRequestSettings? RequestSettings => this._function.RequestSettings;
/// <inheritdoc/>
[Obsolete("Use ISKFunction.SetAIRequestSettingsFactory instead. This will be removed in a future release.")]
public ISKFunction SetAIConfiguration(AIRequestSettings? requestSettings) =>
this._function.SetAIConfiguration(requestSettings);
/// <inheritdoc/>
[Obsolete("Use ISKFunction.SetAIServiceFactory instead. This will be removed in a future release.")]
public ISKFunction SetAIService(Func<ITextCompletion> serviceFactory) =>
this._function.SetAIService(serviceFactory);
/// <inheritdoc/>
[Obsolete("Methods, properties and classes which include Skill in the name have been renamed. Use ISKFunction.PluginName instead. This will be removed in a future release.")]
[EditorBrowsable(EditorBrowsableState.Never)]
dotnet/src/SemanticKernel.Core/Functions/NativeFunction.cs
+20 -17
View File
@@ -44,9 +44,6 @@ internal sealed class NativeFunction : ISKFunction, IDisposable
/// <inheritdoc/>
public string Description { get; }
/// <inheritdoc/>
public AIRequestSettings? RequestSettings { get; }
/// <summary>
/// List of function parameters
/// </summary>
@@ -150,20 +147,6 @@ internal sealed class NativeFunction : ISKFunction, IDisposable
}
}
/// <inheritdoc/>
public ISKFunction SetAIService(Func<ITextCompletion> serviceFactory)
{
this.ThrowNotSemantic();
return this;
}
/// <inheritdoc/>
public ISKFunction SetAIConfiguration(AIRequestSettings? requestSettings)
{
this.ThrowNotSemantic();
return this;
}
/// <summary>
/// Dispose of resources.
/// </summary>
@@ -873,6 +856,26 @@ internal sealed class NativeFunction : ISKFunction, IDisposable
#region Obsolete
/// <inheritdoc/>
[Obsolete("Use ISKFunction.RequestSettingsFactory instead. This will be removed in a future release.")]
public AIRequestSettings? RequestSettings { get; }
/// <inheritdoc/>
[Obsolete("Use ISKFunction.SetAIServiceFactory instead. This will be removed in a future release.")]
public ISKFunction SetAIService(Func<ITextCompletion> serviceFactory)
{
this.ThrowNotSemantic();
return this;
}
/// <inheritdoc/>
[Obsolete("Use ISKFunction.SetAIRequestSettingsFactory instead. This will be removed in a future release.")]
public ISKFunction SetAIConfiguration(AIRequestSettings? requestSettings)
{
this.ThrowNotSemantic();
return this;
}
/// <inheritdoc/>
[Obsolete("Methods, properties and classes which include Skill in the name have been renamed. Use ISKFunction.PluginName instead. This will be removed in a future release.")]
[EditorBrowsable(EditorBrowsableState.Never)]
dotnet/src/SemanticKernel.Core/Functions/OrderedIAIServiceSelector.cs
+61
View File
@@ -0,0 +1,61 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Linq;
using Microsoft.SemanticKernel.AI;
using Microsoft.SemanticKernel.Diagnostics;
using Microsoft.SemanticKernel.Services;
namespace Microsoft.SemanticKernel.Functions;
/// <summary>
/// Implementation of <see cref="IAIServiceSelector"/> that selects the AI service based on the order of the model settings.
/// Uses the service id to select the preferred service provider and then returns the service and associated model settings.
/// </summary>
internal class OrderedIAIServiceSelector : IAIServiceSelector
{
/// <inheritdoc/>
public (T?, AIRequestSettings?) SelectAIService<T>(string renderedPrompt, IAIServiceProvider serviceProvider, IReadOnlyList<AIRequestSettings>? modelSettings) where T : IAIService
{
if (modelSettings is null || modelSettings.Count == 0)
{
var service = serviceProvider.GetService<T>(null);
if (service is not null)
{
return (service, null);
}
}
else
{
AIRequestSettings? defaultRequestSettings = null;
foreach (var model in modelSettings)
{
if (!string.IsNullOrEmpty(model.ServiceId))
{
var service = serviceProvider.GetService<T>(model.ServiceId);
if (service is not null)
{
return (service, model);
}
}
else
{
// First request settings with empty or null service id is the default
defaultRequestSettings ??= model;
}
}
if (defaultRequestSettings is not null)
{
var service = serviceProvider.GetService<T>(null);
if (service is not null)
{
return (service, defaultRequestSettings);
}
}
}
var names = string.Join("|", modelSettings.Select(model => model.ServiceId).ToArray());
throw new SKException($"Service of type {typeof(T)} and name {names ?? "<NONE>"} not registered.");
}
}
dotnet/src/SemanticKernel.Core/Functions/SemanticFunction.cs
+54 -31
View File
@@ -13,6 +13,7 @@ using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.SemanticKernel.AI;
using Microsoft.SemanticKernel.AI.TextCompletion;
using Microsoft.SemanticKernel.Diagnostics;
using Microsoft.SemanticKernel.Functions;
using Microsoft.SemanticKernel.Orchestration;
using Microsoft.SemanticKernel.TemplateEngine;
@@ -38,9 +39,6 @@ internal sealed class SemanticFunction : ISKFunction, IDisposable
/// <inheritdoc/>
public string Description { get; }
/// <inheritdoc/>
public AIRequestSettings? RequestSettings { get; private set; }
/// <summary>
/// List of function parameters
/// </summary>
@@ -73,8 +71,10 @@ internal sealed class SemanticFunction : ISKFunction, IDisposable
pluginName: pluginName,
functionName: functionName,
loggerFactory: loggerFactory
);
func.SetAIConfiguration(promptTemplateConfig.GetDefaultRequestSettings());
)
{
_modelSettings = promptTemplateConfig.ModelSettings
};
return func;
}
@@ -93,22 +93,7 @@ internal sealed class SemanticFunction : ISKFunction, IDisposable
{
this.AddDefaultValues(context.Variables);
return await this.RunPromptAsync(this._aiService?.Value, requestSettings ?? this.RequestSettings, context, cancellationToken).ConfigureAwait(false);
}
/// <inheritdoc/>
public ISKFunction SetAIService(Func<ITextCompletion> serviceFactory)
{
Verify.NotNull(serviceFactory);
this._aiService = new Lazy<ITextCompletion>(serviceFactory);
return this;
}
/// <inheritdoc/>
public ISKFunction SetAIConfiguration(AIRequestSettings? requestSettings)
{
this.RequestSettings = requestSettings;
return this;
return await this.RunPromptAsync(requestSettings, context, cancellationToken).ConfigureAwait(false);
}
/// <summary>
@@ -116,10 +101,6 @@ internal sealed class SemanticFunction : ISKFunction, IDisposable
/// </summary>
public void Dispose()
{
if (this._aiService is { IsValueCreated: true } aiService)
{
(aiService.Value as IDisposable)?.Dispose();
}
}
/// <summary>
@@ -162,9 +143,10 @@ internal sealed class SemanticFunction : ISKFunction, IDisposable
private static readonly JsonSerializerOptions s_toStringStandardSerialization = new();
private static readonly JsonSerializerOptions s_toStringIndentedSerialization = new() { WriteIndented = true };
private readonly ILogger _logger;
private Lazy<ITextCompletion>? _aiService;
private IAIServiceSelector? _serviceSelector;
private List<AIRequestSettings>? _modelSettings;
private readonly Lazy<FunctionView> _view;
public IPromptTemplate _promptTemplate { get; }
private readonly IPromptTemplate _promptTemplate;
private static async Task<string> GetCompletionsResultContentAsync(IReadOnlyList<ITextResult> completions, CancellationToken cancellationToken = default)
{
@@ -188,19 +170,20 @@ internal sealed class SemanticFunction : ISKFunction, IDisposable
}
private async Task<FunctionResult> RunPromptAsync(
ITextCompletion? client,
AIRequestSettings? requestSettings,
SKContext context,
CancellationToken cancellationToken)
{
Verify.NotNull(client);
FunctionResult result;
try
{
string renderedPrompt = await this._promptTemplate.RenderAsync(context, cancellationToken).ConfigureAwait(false);
IReadOnlyList<ITextResult> completionResults = await client.GetCompletionsAsync(renderedPrompt, requestSettings, cancellationToken).ConfigureAwait(false);
// For backward compatibility, use the service selector from the class if it exists, otherwise use the one from the context
var serviceSelector = this._serviceSelector ?? context.ServiceSelector;
(var textCompletion, var defaultRequestSettings) = serviceSelector.SelectAIService<ITextCompletion>(renderedPrompt, context.ServiceProvider, this._modelSettings);
Verify.NotNull(textCompletion);
IReadOnlyList<ITextResult> completionResults = await textCompletion.GetCompletionsAsync(renderedPrompt, requestSettings ?? defaultRequestSettings, cancellationToken).ConfigureAwait(false);
string completion = await GetCompletionsResultContentAsync(completionResults, cancellationToken).ConfigureAwait(false);
// Update the result with the completion
@@ -225,6 +208,46 @@ internal sealed class SemanticFunction : ISKFunction, IDisposable
#region Obsolete
/// <inheritdoc/>
[Obsolete("Use ISKFunction.ModelSettings instead. This will be removed in a future release.")]
public AIRequestSettings? RequestSettings => this._modelSettings?.FirstOrDefault<AIRequestSettings>();
/// <inheritdoc/>
[Obsolete("Use ISKFunction.SetAIServiceFactory instead. This will be removed in a future release.")]
public ISKFunction SetAIService(Func<ITextCompletion> serviceFactory)
{
Verify.NotNull(serviceFactory);
if (this._serviceSelector is DelegatingAIServiceSelector delegatingProvider)
{
delegatingProvider.ServiceFactory = serviceFactory;
}
else
{
var serviceSelector = new DelegatingAIServiceSelector();
serviceSelector.ServiceFactory = serviceFactory;
this._serviceSelector = serviceSelector;
}
return this;
}
/// <inheritdoc/>
[Obsolete("Use ISKFunction.SetAIRequestSettingsFactory instead. This will be removed in a future release.")]
public ISKFunction SetAIConfiguration(AIRequestSettings? requestSettings)
{
if (this._serviceSelector is DelegatingAIServiceSelector delegatingProvider)
{
delegatingProvider.RequestSettings = requestSettings;
}
else
{
var configurationProvider = new DelegatingAIServiceSelector();
configurationProvider.RequestSettings = requestSettings;
this._serviceSelector = configurationProvider;
}
return this;
}
/// <inheritdoc/>
[Obsolete("Methods, properties and classes which include Skill in the name have been renamed. Use ISKFunction.PluginName instead. This will be removed in a future release.")]
[EditorBrowsable(EditorBrowsableState.Never)]
dotnet/src/SemanticKernel.Core/Kernel.cs
+11 -2
View File
@@ -10,6 +10,7 @@ using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.SemanticKernel.Diagnostics;
using Microsoft.SemanticKernel.Events;
using Microsoft.SemanticKernel.Functions;
using Microsoft.SemanticKernel.Http;
using Microsoft.SemanticKernel.Memory;
using Microsoft.SemanticKernel.Orchestration;
@@ -44,6 +45,7 @@ public sealed class Kernel : IKernel, IDisposable
/// <summary>
/// Return a new instance of the kernel builder, used to build and configure kernel instances.
/// </summary>
[Obsolete("This field will be removed in a future release. Initialize KernelBuilder through constructor instead (new KernelBuilder()).")]
public static KernelBuilder Builder => new();
/// <inheritdoc/>
@@ -62,15 +64,18 @@ public sealed class Kernel : IKernel, IDisposable
/// <param name="aiServiceProvider">AI Service Provider</param>
/// <param name="promptTemplateEngine">Prompt template engine</param>
/// <param name="memory">Semantic text Memory</param>
/// <param name="httpHandlerFactory"></param>
/// <param name="httpHandlerFactory">HTTP handler factory</param>
/// <param name="loggerFactory">The <see cref="ILoggerFactory"/> to use for logging. If null, no logging will be performed.</param>
/// <param name="serviceSelector">AI service selector</param>
public Kernel(
IFunctionCollection functionCollection,
IAIServiceProvider aiServiceProvider,
IPromptTemplateEngine promptTemplateEngine,
ISemanticTextMemory memory,
IDelegatingHandlerFactory httpHandlerFactory,
ILoggerFactory? loggerFactory)
ILoggerFactory? loggerFactory,
IAIServiceSelector? serviceSelector = null
)
{
loggerFactory ??= NullLoggerFactory.Instance;
@@ -81,6 +86,7 @@ public sealed class Kernel : IKernel, IDisposable
this._aiServiceProvider = aiServiceProvider;
this._promptTemplateEngine = promptTemplateEngine;
this._functionCollection = functionCollection;
this._aiServiceSelector = serviceSelector ?? new OrderedIAIServiceSelector();
this._logger = loggerFactory.CreateLogger(typeof(Kernel));
}
@@ -174,6 +180,8 @@ repeat:
{
return new SKContext(
new FunctionRunner(this),
this._aiServiceProvider,
this._aiServiceSelector,
variables,
functions ?? this.Functions,
loggerFactory ?? this.LoggerFactory,
@@ -210,6 +218,7 @@ repeat:
private ISemanticTextMemory _memory;
private readonly IPromptTemplateEngine _promptTemplateEngine;
private readonly IAIServiceProvider _aiServiceProvider;
private readonly IAIServiceSelector _aiServiceSelector;
private readonly ILogger _logger;
/// <summary>

Some files were not shown because too many files have changed in this diff Show More