mirror of
https://github.com/microsoft/semantic-kernel.git
synced 2026-10-12 02:19:06 +00:00
.Net: New API for Filters (#5936)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Resolves: https://github.com/microsoft/semantic-kernel/issues/4896 Resolves: https://github.com/microsoft/semantic-kernel/issues/5104 Resolves: https://github.com/microsoft/semantic-kernel/issues/5476 Resolves: https://github.com/microsoft/semantic-kernel/issues/5470 Merging following PRs from feature branch to `main`: 1. https://github.com/microsoft/semantic-kernel/pull/5787 2. https://github.com/microsoft/semantic-kernel/pull/5788 3. https://github.com/microsoft/semantic-kernel/pull/5865 4. https://github.com/microsoft/semantic-kernel/pull/5900 5. https://github.com/microsoft/semantic-kernel/pull/5935 `IFunctionFilter` was renamed to `IFunctionInvocationFilter` with new signature: ```csharp public class MyFilter : IFunctionInvocationFilter { public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next) { // Perform some actions before function invocation await next(context); // Perform some actions after function invocation } } ``` `IPromptFilter` was renamed to `IPromptRenderFilter` with new signature: ```csharp public class PromptFilter: IPromptRenderFilter { public async Task OnPromptRenderAsync(PromptRenderContext context, Func<PromptRenderContext, Task> next) { // Perform some actions before prompt rendering await next(context); // Perform some actions after prompt rendering } } ``` New type of filter was added `IAutoFunctionInvocationFilter`: ```csharp public class AutoFunctionInvocationFilter(ILogger logger) : IAutoFunctionInvocationFilter { private readonly ILogger _logger = logger; public async Task OnAutoFunctionInvocationAsync(AutoFunctionInvocationContext context, Func<AutoFunctionInvocationContext, Task> next) { // Example: get function information var functionName = context.Function.Name; // Example: get chat history var chatHistory = context.ChatHistory; // Example: get request sequence number this._logger.LogDebug($"Request sequence number: {context.RequestSequenceNumber}"); // Example: get function sequence number this._logger.LogDebug($"Function sequence number: {context.FunctionSequenceNumber}"); // Calling next filter in pipeline or function itself. // By skipping this call, next filters and function won't be invoked, and function call loop will proceed to the next function. await next(context); // Example: get function result var result = context.Result; // Example: override function result value context.Result = new FunctionResult(context.Result, "Result from auto function invocation filter"); // Example: cancel function invocation context.Cancel = true; } } ``` ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄
This commit is contained in:
Dmytro Struk
@@ -0,0 +1,198 @@
|
||||
---
|
||||
# These are optional elements. Feel free to remove any of them.
|
||||
status: accepted
|
||||
contact: dmytrostruk
|
||||
date: 2024-04-24
|
||||
deciders: sergeymenshykh, markwallace, rbarreto, dmytrostruk, stoub
|
||||
---
|
||||
|
||||
# Exception handling in filters
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
In .NET version of Semantic Kernel, when kernel function throws an exception, it will be propagated through execution stack until some code will catch it. To handle exception for `kernel.InvokeAsync(function)`, this code should be wrapped in `try/catch` block, which is intuitive approach how to deal with exceptions.
|
||||
|
||||
Unfortunately, `try/catch` block is not useful for auto function calling scenario, when a function is called based on some prompt. In this case, when function throws an exception, message `Error: Exception while invoking function.` will be added to chat history with `tool` author role, which should provide some context to LLM that something went wrong.
|
||||
|
||||
There is a requirement to have the ability to override function result - instead of throwing an exception and sending error message to AI, it should be possible to set some custom result, which should allow to control LLM behavior.
|
||||
|
||||
## Considered Options
|
||||
|
||||
### [Option 1] Add new method to existing `IFunctionFilter` interface
|
||||
|
||||
Abstraction:
|
||||
|
||||
```csharp
|
||||
public interface IFunctionFilter
|
||||
{
|
||||
void OnFunctionInvoking(FunctionInvokingContext context);
|
||||
|
||||
void OnFunctionInvoked(FunctionInvokedContext context);
|
||||
|
||||
// New method
|
||||
void OnFunctionException(FunctionExceptionContext context);
|
||||
}
|
||||
```
|
||||
|
||||
Disadvantages:
|
||||
|
||||
- Adding new method to existing interface will be a breaking change, as it will force current filter users to implement new method.
|
||||
- This method will be always required to implement when using function filters, even when exception handling is not needed. On the other hand, this method won't return anything, so it could remain always empty, or with .NET multitargeting, it should be possible to define default implementation for C# 8 and above.
|
||||
|
||||
### [Option 2] Introduce new `IExceptionFilter` interface
|
||||
|
||||
New interface will allow to receive exception objects, cancel exception or rethrowing new type of exception. This option can be also added later as filter on a higher level for global exception handling.
|
||||
|
||||
Abstraction:
|
||||
|
||||
```csharp
|
||||
public interface IExceptionFilter
|
||||
{
|
||||
// ExceptionContext class will contain information about actual exception, kernel function etc.
|
||||
void OnException(ExceptionContext context);
|
||||
}
|
||||
```
|
||||
|
||||
Usage:
|
||||
|
||||
```csharp
|
||||
public class MyFilter : IFunctionFilter, IExceptionFilter
|
||||
{
|
||||
public void OnFunctionInvoking(FunctionInvokingContext context) { }
|
||||
|
||||
public void OnFunctionInvoked(FunctionInvokedContext context) { }
|
||||
|
||||
public void OnException(ExceptionContext context) {}
|
||||
}
|
||||
```
|
||||
|
||||
Advantages:
|
||||
|
||||
- It's not a breaking change, and all exception handling logic should be added on top of existing filter mechanism.
|
||||
- Similar to `IExceptionFilter` API in ASP.NET.
|
||||
|
||||
Disadvantages:
|
||||
|
||||
- It may be not intuitive and hard to remember, that for exception handling, separate interface should be implemented.
|
||||
|
||||
### [Option 3] Extend Context model in existing `IFunctionFilter` interface
|
||||
|
||||
In `IFunctionFilter.OnFunctionInvoked` method, it's possible to extend `FunctionInvokedContext` model by adding `Exception` property. In this case, as soon as `OnFunctionInvoked` is triggered, it will be possible to observe whether there was an exception during function execution.
|
||||
|
||||
If there was an exception, users could do nothing and the exception will be thrown as usual, which means that in order to handle it, function invocation should be wrapped with `try/catch` block. But it will be also possible to cancel that exception and override function result, which should provide more control over function execution and what is passed to LLM.
|
||||
|
||||
Abstraction:
|
||||
|
||||
```csharp
|
||||
public sealed class FunctionInvokedContext : FunctionFilterContext
|
||||
{
|
||||
// other properties...
|
||||
|
||||
public Exception? Exception { get; private set; }
|
||||
}
|
||||
```
|
||||
|
||||
Usage:
|
||||
|
||||
```csharp
|
||||
public class MyFilter : IFunctionFilter
|
||||
{
|
||||
public void OnFunctionInvoking(FunctionInvokingContext context) { }
|
||||
|
||||
public void OnFunctionInvoked(FunctionInvokedContext context)
|
||||
{
|
||||
// This means that exception occurred during function execution.
|
||||
// If we ignore it, the exception will be thrown as usual.
|
||||
if (context.Exception is not null)
|
||||
{
|
||||
// Possible options to handle it:
|
||||
|
||||
// 1. Do not throw an exception that occurred during function execution
|
||||
context.Exception = null;
|
||||
|
||||
// 2. Override the result with some value, that is meaningful to LLM
|
||||
context.Result = new FunctionResult(context.Function, "Friendly message instead of exception");
|
||||
|
||||
// 3. Rethrow another type of exception if needed - Option 1.
|
||||
context.Exception = new Exception("New exception");
|
||||
|
||||
// 3. Rethrow another type of exception if needed - Option 2.
|
||||
throw new Exception("New exception");
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Advantages:
|
||||
|
||||
- Requires minimum changes to existing implementation and also it won't break existing filter users.
|
||||
- Similar to `IActionFilter` API in ASP.NET.
|
||||
- Scalable, because it will be possible to extend similar Context models for other type of filters when needed (prompt or function calling filters).
|
||||
|
||||
Disadvantages:
|
||||
|
||||
- Not .NET-friendly way of exception handling with `context.Exception = null` or `context.Exception = new AnotherException()`, instead of using native `try/catch` approach.
|
||||
|
||||
### [Option 4] Change `IFunctionFilter` signature by adding `next` delegate.
|
||||
|
||||
This approach changes the way how filters work at the moment. Instead of having two `Invoking` and `Invoked` methods in filter, there will be only one method that will be invoked during function execution with `next` delegate, which will be responsible to call next registered filter in pipeline or function itself, in case there are no remaining filters.
|
||||
|
||||
Abstraction:
|
||||
|
||||
```csharp
|
||||
public interface IFunctionFilter
|
||||
{
|
||||
Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next);
|
||||
}
|
||||
```
|
||||
|
||||
Usage:
|
||||
|
||||
```csharp
|
||||
public class MyFilter : IFunctionFilter
|
||||
{
|
||||
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
|
||||
{
|
||||
// Perform some actions before function invocation
|
||||
await next(context);
|
||||
// Perform some actions after function invocation
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Exception handling with native `try/catch` approach:
|
||||
|
||||
```csharp
|
||||
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
|
||||
{
|
||||
try
|
||||
{
|
||||
await next(context);
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
this._logger.LogError(exception, "Something went wrong during function invocation");
|
||||
|
||||
// Example: override function result value
|
||||
context.Result = new FunctionResult(context.Function, "Friendly message instead of exception");
|
||||
|
||||
// Example: Rethrow another type of exception if needed
|
||||
throw new InvalidOperationException("New exception");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Advantages:
|
||||
|
||||
- Native way how to handle and rethrow exceptions.
|
||||
- Similar to `IAsyncActionFilter` and `IEndpointFilter` API in ASP.NET.
|
||||
- One filter method to implement instead of two (`Invoking/Invoked`) - this allows to keep invocation context information in one method instead of storing it on class level. For example, to measure function execution time, `Stopwatch` can be created and started before `await next(context)` call and used after the call, while in approach with `Invoking/Invoked` methods the data should be passed between filter actions in other way, for example setting it on class level, which is harder to maintain.
|
||||
- No need in cancellation logic (e.g. `context.Cancel = true`). To cancel the operation, simply don't call `await next(context)`.
|
||||
|
||||
Disadvantages:
|
||||
|
||||
- Remember to call `await next(context)` manually in all filters. If it's not called, next filter in pipeline and/or function itself won't be called.
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
Proceed with Option 4 and apply this approach to function, prompt and function calling filters.
|
||||
@@ -1,5 +1,6 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System;
|
||||
using System.Threading.Tasks;
|
||||
using Examples;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
@@ -26,7 +27,7 @@ public sealed class Step6_Responsible_AI(ITestOutputHelper output) : BaseTest(ou
|
||||
builder.Services.AddSingleton<ITestOutputHelper>(this.Output);
|
||||
|
||||
// Add prompt filter to the kernel
|
||||
builder.Services.AddSingleton<IPromptFilter, PromptFilter>();
|
||||
builder.Services.AddSingleton<IPromptRenderFilter, PromptFilter>();
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
@@ -35,31 +36,31 @@ public sealed class Step6_Responsible_AI(ITestOutputHelper output) : BaseTest(ou
|
||||
var result = await kernel.InvokePromptAsync("Tell me some useful information about this credit card number {{$card_number}}?", arguments);
|
||||
|
||||
WriteLine(result);
|
||||
|
||||
// Output: Sorry, but I can't assist with that.
|
||||
}
|
||||
|
||||
private sealed class PromptFilter(ITestOutputHelper output) : IPromptFilter
|
||||
private sealed class PromptFilter(ITestOutputHelper output) : IPromptRenderFilter
|
||||
{
|
||||
private readonly ITestOutputHelper _output = output;
|
||||
|
||||
/// <summary>
|
||||
/// Method which is called after a prompt is rendered.
|
||||
/// Method which is called asynchronously before prompt rendering.
|
||||
/// </summary>
|
||||
public void OnPromptRendered(PromptRenderedContext context)
|
||||
{
|
||||
context.RenderedPrompt += " NO SEXISM, RACISM OR OTHER BIAS/BIGOTRY";
|
||||
|
||||
this._output.WriteLine(context.RenderedPrompt);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Method which is called before a prompt is rendered.
|
||||
/// </summary>
|
||||
public void OnPromptRendering(PromptRenderingContext context)
|
||||
/// <param name="context">Instance of <see cref="PromptRenderContext"/> with prompt rendering details.</param>
|
||||
/// <param name="next">Delegate to the next filter in pipeline or prompt rendering operation itself. If it's not invoked, next filter or prompt rendering won't be invoked.</param>
|
||||
public async Task OnPromptRenderAsync(PromptRenderContext context, Func<PromptRenderContext, Task> next)
|
||||
{
|
||||
if (context.Arguments.ContainsName("card_number"))
|
||||
{
|
||||
context.Arguments["card_number"] = "**** **** **** ****";
|
||||
}
|
||||
|
||||
await next(context);
|
||||
|
||||
context.RenderedPrompt += " NO SEXISM, RACISM OR OTHER BIAS/BIGOTRY";
|
||||
|
||||
this._output.WriteLine(context.RenderedPrompt);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -31,12 +31,12 @@ public sealed class Step7_Observability(ITestOutputHelper output) : BaseTest(out
|
||||
|
||||
// Add filter using DI
|
||||
kernelBuilder.Services.AddSingleton<ITestOutputHelper>(this.Output);
|
||||
kernelBuilder.Services.AddSingleton<IFunctionFilter, MyFunctionFilter>();
|
||||
kernelBuilder.Services.AddSingleton<IFunctionInvocationFilter, MyFunctionFilter>();
|
||||
|
||||
Kernel kernel = kernelBuilder.Build();
|
||||
|
||||
// Add filter without DI
|
||||
kernel.PromptFilters.Add(new MyPromptFilter(this.Output));
|
||||
kernel.PromptRenderFilters.Add(new MyPromptFilter(this.Output));
|
||||
|
||||
// Invoke the kernel with a prompt and allow the AI to automatically invoke functions
|
||||
OpenAIPromptExecutionSettings settings = new() { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };
|
||||
@@ -111,41 +111,39 @@ public sealed class Step7_Observability(ITestOutputHelper output) : BaseTest(out
|
||||
/// <summary>
|
||||
/// Function filter for observability.
|
||||
/// </summary>
|
||||
private sealed class MyFunctionFilter(ITestOutputHelper output) : IFunctionFilter
|
||||
private sealed class MyFunctionFilter(ITestOutputHelper output) : IFunctionInvocationFilter
|
||||
{
|
||||
private readonly ITestOutputHelper _output = output;
|
||||
|
||||
public void OnFunctionInvoked(FunctionInvokedContext context)
|
||||
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
|
||||
{
|
||||
var metadata = context.Result.Metadata;
|
||||
this._output.WriteLine($"Invoking {context.Function.Name}");
|
||||
|
||||
await next(context);
|
||||
|
||||
var metadata = context.Result?.Metadata;
|
||||
|
||||
if (metadata is not null && metadata.ContainsKey("Usage"))
|
||||
{
|
||||
this._output.WriteLine($"Token usage: {metadata["Usage"]?.AsJson()}");
|
||||
}
|
||||
}
|
||||
|
||||
public void OnFunctionInvoking(FunctionInvokingContext context)
|
||||
{
|
||||
this._output.WriteLine($"Invoking {context.Function.Name}");
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Prompt filter for observability.
|
||||
/// </summary>
|
||||
private sealed class MyPromptFilter(ITestOutputHelper output) : IPromptFilter
|
||||
private sealed class MyPromptFilter(ITestOutputHelper output) : IPromptRenderFilter
|
||||
{
|
||||
private readonly ITestOutputHelper _output = output;
|
||||
|
||||
public void OnPromptRendered(PromptRenderedContext context)
|
||||
{
|
||||
this._output.WriteLine($"Rendered prompt: {context.RenderedPrompt}");
|
||||
}
|
||||
|
||||
public void OnPromptRendering(PromptRenderingContext context)
|
||||
public async Task OnPromptRenderAsync(PromptRenderContext context, Func<PromptRenderContext, Task> next)
|
||||
{
|
||||
this._output.WriteLine($"Rendering prompt for {context.Function.Name}");
|
||||
|
||||
await next(context);
|
||||
|
||||
this._output.WriteLine($"Rendered prompt: {context.RenderedPrompt}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,8 +1,14 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Threading.Tasks;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Microsoft.SemanticKernel;
|
||||
using Microsoft.SemanticKernel.Connectors.OpenAI;
|
||||
using Xunit;
|
||||
using Xunit.Abstractions;
|
||||
|
||||
@@ -26,13 +32,13 @@ public class Example76_Filters(ITestOutputHelper output) : BaseTest(output)
|
||||
builder.Services.AddSingleton<ITestOutputHelper>(this.Output);
|
||||
|
||||
// Add filters with DI
|
||||
builder.Services.AddSingleton<IFunctionFilter, FirstFunctionFilter>();
|
||||
builder.Services.AddSingleton<IFunctionFilter, SecondFunctionFilter>();
|
||||
builder.Services.AddSingleton<IFunctionInvocationFilter, FirstFunctionFilter>();
|
||||
builder.Services.AddSingleton<IFunctionInvocationFilter, SecondFunctionFilter>();
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
// Add filter without DI
|
||||
kernel.PromptFilters.Add(new FirstPromptFilter(this.Output));
|
||||
kernel.PromptRenderFilters.Add(new FirstPromptFilter(this.Output));
|
||||
|
||||
var function = kernel.CreateFunctionFromPrompt("What is Seattle", functionName: "MyFunction");
|
||||
kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("MyPlugin", functions: [function]));
|
||||
@@ -41,81 +47,370 @@ public class Example76_Filters(ITestOutputHelper output) : BaseTest(output)
|
||||
WriteLine(result);
|
||||
}
|
||||
|
||||
#region Filters
|
||||
|
||||
private sealed class FirstFunctionFilter(ITestOutputHelper output) : IFunctionFilter
|
||||
[Fact]
|
||||
public async Task PromptFilterRenderedPromptOverrideAsync()
|
||||
{
|
||||
private readonly ITestOutputHelper _output = output;
|
||||
var builder = Kernel.CreateBuilder();
|
||||
|
||||
public void OnFunctionInvoking(FunctionInvokingContext context) =>
|
||||
this._output.WriteLine($"{nameof(FirstFunctionFilter)}.{nameof(OnFunctionInvoking)} - {context.Function.PluginName}.{context.Function.Name}");
|
||||
builder.AddAzureOpenAIChatCompletion(
|
||||
deploymentName: TestConfiguration.AzureOpenAI.ChatDeploymentName,
|
||||
endpoint: TestConfiguration.AzureOpenAI.Endpoint,
|
||||
apiKey: TestConfiguration.AzureOpenAI.ApiKey);
|
||||
|
||||
public void OnFunctionInvoked(FunctionInvokedContext context) =>
|
||||
this._output.WriteLine($"{nameof(FirstFunctionFilter)}.{nameof(OnFunctionInvoked)} - {context.Function.PluginName}.{context.Function.Name}");
|
||||
builder.Services.AddSingleton<IPromptRenderFilter, PromptFilterExample>();
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
var result = await kernel.InvokePromptAsync("Hi, how can you help me?");
|
||||
|
||||
WriteLine(result);
|
||||
|
||||
// Output:
|
||||
// Prompt from filter
|
||||
}
|
||||
|
||||
private sealed class SecondFunctionFilter(ITestOutputHelper output) : IFunctionFilter
|
||||
[Fact]
|
||||
public async Task FunctionFilterResultOverrideAsync()
|
||||
{
|
||||
private readonly ITestOutputHelper _output = output;
|
||||
var builder = Kernel.CreateBuilder();
|
||||
|
||||
public void OnFunctionInvoking(FunctionInvokingContext context) =>
|
||||
this._output.WriteLine($"{nameof(SecondFunctionFilter)}.{nameof(OnFunctionInvoking)} - {context.Function.PluginName}.{context.Function.Name}");
|
||||
// This filter overrides result with "Result from filter" value.
|
||||
builder.Services.AddSingleton<IFunctionInvocationFilter, FunctionFilterExample>();
|
||||
|
||||
public void OnFunctionInvoked(FunctionInvokedContext context) =>
|
||||
this._output.WriteLine($"{nameof(SecondFunctionFilter)}.{nameof(OnFunctionInvoked)} - {context.Function.PluginName}.{context.Function.Name}");
|
||||
var kernel = builder.Build();
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => "Result from method");
|
||||
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
WriteLine(result);
|
||||
WriteLine($"Metadata: {string.Join(",", result.Metadata!.Select(kv => $"{kv.Key}: {kv.Value}"))}");
|
||||
|
||||
// Output:
|
||||
// Result from filter.
|
||||
// Metadata: metadata_key: metadata_value
|
||||
}
|
||||
|
||||
private sealed class FirstPromptFilter(ITestOutputHelper output) : IPromptFilter
|
||||
[Fact]
|
||||
public async Task FunctionFilterResultOverrideOnStreamingAsync()
|
||||
{
|
||||
private readonly ITestOutputHelper _output = output;
|
||||
var builder = Kernel.CreateBuilder();
|
||||
|
||||
public void OnPromptRendering(PromptRenderingContext context) =>
|
||||
this._output.WriteLine($"{nameof(FirstPromptFilter)}.{nameof(OnPromptRendering)} - {context.Function.PluginName}.{context.Function.Name}");
|
||||
// This filter overrides streaming results with "item * 2" logic.
|
||||
builder.Services.AddSingleton<IFunctionInvocationFilter, StreamingFunctionFilterExample>();
|
||||
|
||||
public void OnPromptRendered(PromptRenderedContext context) =>
|
||||
this._output.WriteLine($"{nameof(FirstPromptFilter)}.{nameof(OnPromptRendered)} - {context.Function.PluginName}.{context.Function.Name}");
|
||||
var kernel = builder.Build();
|
||||
|
||||
static async IAsyncEnumerable<int> GetData()
|
||||
{
|
||||
yield return 1;
|
||||
yield return 2;
|
||||
yield return 3;
|
||||
}
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromMethod(GetData);
|
||||
|
||||
await foreach (var item in kernel.InvokeStreamingAsync<int>(function))
|
||||
{
|
||||
WriteLine(item);
|
||||
}
|
||||
|
||||
// Output: 2, 4, 6.
|
||||
}
|
||||
|
||||
#endregion
|
||||
[Fact]
|
||||
public async Task FunctionFilterExceptionHandlingAsync()
|
||||
{
|
||||
var builder = Kernel.CreateBuilder();
|
||||
|
||||
// This filter handles an exception and returns overridden result.
|
||||
builder.Services.AddSingleton<IFunctionInvocationFilter>(new ExceptionHandlingFilterExample(NullLogger.Instance));
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
// Simulation of exception during function invocation.
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => { throw new KernelException("Exception in function"); });
|
||||
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
WriteLine(result);
|
||||
|
||||
// Output: Friendly message instead of exception.
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task FunctionFilterExceptionHandlingOnStreamingAsync()
|
||||
{
|
||||
var builder = Kernel.CreateBuilder();
|
||||
|
||||
// This filter handles an exception and returns overridden streaming result.
|
||||
builder.Services.AddSingleton<IFunctionInvocationFilter>(new StreamingExceptionHandlingFilterExample(NullLogger.Instance));
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
static async IAsyncEnumerable<string> GetData()
|
||||
{
|
||||
yield return "first chunk";
|
||||
// Simulation of exception during function invocation.
|
||||
throw new KernelException("Exception in function");
|
||||
}
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromMethod(GetData);
|
||||
|
||||
await foreach (var item in kernel.InvokeStreamingAsync<string>(function))
|
||||
{
|
||||
WriteLine(item);
|
||||
}
|
||||
|
||||
// Output: first chunk, chunk instead of exception.
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task AutoFunctionInvocationFilterAsync()
|
||||
{
|
||||
var builder = Kernel.CreateBuilder();
|
||||
|
||||
builder.AddOpenAIChatCompletion("gpt-4", TestConfiguration.OpenAI.ApiKey);
|
||||
|
||||
// This filter outputs information about auto function invocation and returns overridden result.
|
||||
builder.Services.AddSingleton<IAutoFunctionInvocationFilter>(new AutoFunctionInvocationFilterExample(this.Output));
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => "Result from function", "MyFunction");
|
||||
|
||||
kernel.ImportPluginFromFunctions("MyPlugin", [function]);
|
||||
|
||||
var executionSettings = new OpenAIPromptExecutionSettings
|
||||
{
|
||||
ToolCallBehavior = ToolCallBehavior.RequireFunction(function.Metadata.ToOpenAIFunction(), autoInvoke: true)
|
||||
};
|
||||
|
||||
var result = await kernel.InvokePromptAsync("Invoke provided function and return result", new(executionSettings));
|
||||
|
||||
WriteLine(result);
|
||||
|
||||
// Output:
|
||||
// Request sequence number: 0
|
||||
// Function sequence number: 0
|
||||
// Total number of functions: 1
|
||||
// Result from auto function invocation filter.
|
||||
}
|
||||
|
||||
#region Filter capabilities
|
||||
|
||||
private sealed class FunctionFilterExample : IFunctionFilter
|
||||
/// <summary>Shows syntax for function filter in non-streaming scenario.</summary>
|
||||
private sealed class FunctionFilterExample : IFunctionInvocationFilter
|
||||
{
|
||||
public void OnFunctionInvoked(FunctionInvokedContext context)
|
||||
{
|
||||
// Example: get function result value
|
||||
var value = context.Result.GetValue<object>();
|
||||
|
||||
// Example: override function result value
|
||||
context.SetResultValue("new result value");
|
||||
|
||||
// Example: get token usage from metadata
|
||||
var usage = context.Result.Metadata?["Usage"];
|
||||
}
|
||||
|
||||
public void OnFunctionInvoking(FunctionInvokingContext context)
|
||||
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
|
||||
{
|
||||
// Example: override kernel arguments
|
||||
context.Arguments["input"] = "new input";
|
||||
|
||||
// Example: cancel function execution
|
||||
context.Cancel = true;
|
||||
// This call is required to proceed with next filters in pipeline and actual function.
|
||||
// Without this call next filters and function won't be invoked.
|
||||
await next(context);
|
||||
|
||||
// Example: get function result value
|
||||
var value = context.Result!.GetValue<object>();
|
||||
|
||||
// Example: get token usage from metadata
|
||||
var usage = context.Result.Metadata?["Usage"];
|
||||
|
||||
// Example: override function result value and metadata
|
||||
Dictionary<string, object?> metadata = context.Result.Metadata is not null ? new(context.Result.Metadata) : [];
|
||||
metadata["metadata_key"] = "metadata_value";
|
||||
|
||||
context.Result = new FunctionResult(context.Result, "Result from filter")
|
||||
{
|
||||
Metadata = metadata
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class PromptFilterExample : IPromptFilter
|
||||
/// <summary>Shows syntax for prompt filter.</summary>
|
||||
private sealed class PromptFilterExample : IPromptRenderFilter
|
||||
{
|
||||
public void OnPromptRendered(PromptRenderedContext context)
|
||||
{
|
||||
// Example: override rendered prompt before sending it to AI
|
||||
context.RenderedPrompt = "Safe prompt";
|
||||
}
|
||||
|
||||
public void OnPromptRendering(PromptRenderingContext context)
|
||||
public async Task OnPromptRenderAsync(PromptRenderContext context, Func<PromptRenderContext, Task> next)
|
||||
{
|
||||
// Example: get function information
|
||||
var functionName = context.Function.Name;
|
||||
|
||||
await next(context);
|
||||
|
||||
// Example: override rendered prompt before sending it to AI
|
||||
context.RenderedPrompt = "Respond with following text: Prompt from filter.";
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Shows syntax for auto function invocation filter.</summary>
|
||||
private sealed class AutoFunctionInvocationFilterExample(ITestOutputHelper output) : IAutoFunctionInvocationFilter
|
||||
{
|
||||
private readonly ITestOutputHelper _output = output;
|
||||
|
||||
public async Task OnAutoFunctionInvocationAsync(AutoFunctionInvocationContext context, Func<AutoFunctionInvocationContext, Task> next)
|
||||
{
|
||||
// Example: get function information
|
||||
var functionName = context.Function.Name;
|
||||
|
||||
// Example: get chat history
|
||||
var chatHistory = context.ChatHistory;
|
||||
|
||||
// Example: get information about all functions which will be invoked
|
||||
var functionCalls = FunctionCallContent.GetFunctionCalls(context.ChatHistory.Last());
|
||||
|
||||
// Example: get request sequence index
|
||||
this._output.WriteLine($"Request sequence index: {context.RequestSequenceIndex}");
|
||||
|
||||
// Example: get function sequence index
|
||||
this._output.WriteLine($"Function sequence index: {context.FunctionSequenceIndex}");
|
||||
|
||||
// Example: get total number of functions which will be called
|
||||
this._output.WriteLine($"Total number of functions: {context.FunctionCount}");
|
||||
|
||||
// Calling next filter in pipeline or function itself.
|
||||
// By skipping this call, next filters and function won't be invoked, and function call loop will proceed to the next function.
|
||||
await next(context);
|
||||
|
||||
// Example: get function result
|
||||
var result = context.Result;
|
||||
|
||||
// Example: override function result value
|
||||
context.Result = new FunctionResult(context.Result, "Result from auto function invocation filter");
|
||||
|
||||
// Example: Terminate function invocation
|
||||
context.Terminate = true;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Shows syntax for function filter in streaming scenario.</summary>
|
||||
private sealed class StreamingFunctionFilterExample : IFunctionInvocationFilter
|
||||
{
|
||||
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
|
||||
{
|
||||
await next(context);
|
||||
|
||||
// In streaming scenario, async enumerable is available in context result object.
|
||||
// To override data: get async enumerable from function result, override data and set new async enumerable in context result:
|
||||
var enumerable = context.Result.GetValue<IAsyncEnumerable<int>>();
|
||||
context.Result = new FunctionResult(context.Result, OverrideStreamingDataAsync(enumerable!));
|
||||
}
|
||||
|
||||
private async IAsyncEnumerable<int> OverrideStreamingDataAsync(IAsyncEnumerable<int> data)
|
||||
{
|
||||
await foreach (var item in data)
|
||||
{
|
||||
// Example: override streaming data
|
||||
yield return item * 2;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Shows syntax for exception handling in function filter in non-streaming scenario.</summary>
|
||||
private sealed class ExceptionHandlingFilterExample(ILogger logger) : IFunctionInvocationFilter
|
||||
{
|
||||
private readonly ILogger _logger = logger;
|
||||
|
||||
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
|
||||
{
|
||||
try
|
||||
{
|
||||
await next(context);
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
this._logger.LogError(exception, "Something went wrong during function invocation");
|
||||
|
||||
// Example: override function result value
|
||||
context.Result = new FunctionResult(context.Result, "Friendly message instead of exception");
|
||||
|
||||
// Example: Rethrow another type of exception if needed
|
||||
// throw new InvalidOperationException("New exception");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Shows syntax for exception handling in function filter in streaming scenario.</summary>
|
||||
private sealed class StreamingExceptionHandlingFilterExample(ILogger logger) : IFunctionInvocationFilter
|
||||
{
|
||||
private readonly ILogger _logger = logger;
|
||||
|
||||
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
|
||||
{
|
||||
await next(context);
|
||||
|
||||
var enumerable = context.Result.GetValue<IAsyncEnumerable<string>>();
|
||||
context.Result = new FunctionResult(context.Result, StreamingWithExceptionHandlingAsync(enumerable!));
|
||||
}
|
||||
|
||||
private async IAsyncEnumerable<string> StreamingWithExceptionHandlingAsync(IAsyncEnumerable<string> data)
|
||||
{
|
||||
var enumerator = data.GetAsyncEnumerator();
|
||||
|
||||
await using (enumerator.ConfigureAwait(false))
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
string result;
|
||||
|
||||
try
|
||||
{
|
||||
if (!await enumerator.MoveNextAsync().ConfigureAwait(false))
|
||||
{
|
||||
break;
|
||||
}
|
||||
|
||||
result = enumerator.Current;
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
this._logger.LogError(exception, "Something went wrong during function invocation");
|
||||
|
||||
result = "chunk instead of exception";
|
||||
}
|
||||
|
||||
yield return result;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#endregion
|
||||
|
||||
#region Filters
|
||||
|
||||
private sealed class FirstFunctionFilter(ITestOutputHelper output) : IFunctionInvocationFilter
|
||||
{
|
||||
private readonly ITestOutputHelper _output = output;
|
||||
|
||||
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
|
||||
{
|
||||
this._output.WriteLine($"{nameof(FirstFunctionFilter)}.FunctionInvoking - {context.Function.PluginName}.{context.Function.Name}");
|
||||
await next(context);
|
||||
this._output.WriteLine($"{nameof(FirstFunctionFilter)}.FunctionInvoked - {context.Function.PluginName}.{context.Function.Name}");
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class SecondFunctionFilter(ITestOutputHelper output) : IFunctionInvocationFilter
|
||||
{
|
||||
private readonly ITestOutputHelper _output = output;
|
||||
|
||||
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
|
||||
{
|
||||
this._output.WriteLine($"{nameof(SecondFunctionFilter)}.FunctionInvoking - {context.Function.PluginName}.{context.Function.Name}");
|
||||
await next(context);
|
||||
this._output.WriteLine($"{nameof(SecondFunctionFilter)}.FunctionInvoked - {context.Function.PluginName}.{context.Function.Name}");
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class FirstPromptFilter(ITestOutputHelper output) : IPromptRenderFilter
|
||||
{
|
||||
private readonly ITestOutputHelper _output = output;
|
||||
|
||||
public async Task OnPromptRenderAsync(PromptRenderContext context, Func<PromptRenderContext, Task> next)
|
||||
{
|
||||
this._output.WriteLine($"{nameof(FirstPromptFilter)}.PromptRendering - {context.Function.PluginName}.{context.Function.Name}");
|
||||
await next(context);
|
||||
this._output.WriteLine($"{nameof(FirstPromptFilter)}.PromptRendered - {context.Function.PluginName}.{context.Function.Name}");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -313,7 +313,7 @@ internal abstract class ClientCore
|
||||
// Create the Azure SDK ChatCompletionOptions instance from all available information.
|
||||
var chatOptions = CreateChatCompletionsOptions(chatExecutionSettings, chat, kernel, this.DeploymentOrModelName);
|
||||
|
||||
for (int iteration = 1; ; iteration++)
|
||||
for (int requestIndex = 1; ; requestIndex++)
|
||||
{
|
||||
// Make the request.
|
||||
var responseData = (await RunRequestAsync(() => this.Client.GetChatCompletionsAsync(chatOptions, cancellationToken)).ConfigureAwait(false)).Value;
|
||||
@@ -362,9 +362,9 @@ internal abstract class ClientCore
|
||||
|
||||
// We must send back a response for every tool call, regardless of whether we successfully executed it or not.
|
||||
// If we successfully execute it, we'll add the result. If we don't, we'll add an error.
|
||||
for (int i = 0; i < result.ToolCalls.Count; i++)
|
||||
for (int toolCallIndex = 0; toolCallIndex < result.ToolCalls.Count; toolCallIndex++)
|
||||
{
|
||||
ChatCompletionsToolCall toolCall = result.ToolCalls[i];
|
||||
ChatCompletionsToolCall toolCall = result.ToolCalls[toolCallIndex];
|
||||
|
||||
// We currently only know about function tool calls. If it's anything else, we'll respond with an error.
|
||||
if (toolCall is not ChatCompletionsFunctionToolCall functionToolCall)
|
||||
@@ -403,14 +403,31 @@ internal abstract class ClientCore
|
||||
}
|
||||
|
||||
// Now, invoke the function, and add the resulting tool call message to the chat options.
|
||||
FunctionResult functionResult = new(function) { Culture = kernel.Culture };
|
||||
AutoFunctionInvocationContext invocationContext = new(kernel, function, functionResult, chat)
|
||||
{
|
||||
Arguments = functionArgs,
|
||||
RequestSequenceIndex = requestIndex - 1,
|
||||
FunctionSequenceIndex = toolCallIndex,
|
||||
FunctionCount = result.ToolCalls.Count
|
||||
};
|
||||
|
||||
s_inflightAutoInvokes.Value++;
|
||||
object? functionResult;
|
||||
try
|
||||
{
|
||||
// Note that we explicitly do not use executionSettings here; those pertain to the all-up operation and not necessarily to any
|
||||
// further calls made as part of this function invocation. In particular, we must not use function calling settings naively here,
|
||||
// as the called function could in turn telling the model about itself as a possible candidate for invocation.
|
||||
functionResult = (await function.InvokeAsync(kernel, functionArgs, cancellationToken: cancellationToken).ConfigureAwait(false)).GetValue<object>() ?? string.Empty;
|
||||
invocationContext = await OnAutoFunctionInvocationAsync(kernel, invocationContext, async (context) =>
|
||||
{
|
||||
// Check if filter requested termination.
|
||||
if (context.Terminate)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// Note that we explicitly do not use executionSettings here; those pertain to the all-up operation and not necessarily to any
|
||||
// further calls made as part of this function invocation. In particular, we must not use function calling settings naively here,
|
||||
// as the called function could in turn telling the model about itself as a possible candidate for invocation.
|
||||
context.Result = await function.InvokeAsync(kernel, invocationContext.Arguments, cancellationToken: cancellationToken).ConfigureAwait(false);
|
||||
}).ConfigureAwait(false);
|
||||
}
|
||||
#pragma warning disable CA1031 // Do not catch general exception types
|
||||
catch (Exception e)
|
||||
@@ -424,10 +441,25 @@ internal abstract class ClientCore
|
||||
s_inflightAutoInvokes.Value--;
|
||||
}
|
||||
|
||||
var stringResult = ProcessFunctionResult(functionResult, chatExecutionSettings.ToolCallBehavior);
|
||||
// Apply any changes from the auto function invocation filters context to final result.
|
||||
functionResult = invocationContext.Result;
|
||||
|
||||
object functionResultValue = functionResult.GetValue<object>() ?? string.Empty;
|
||||
var stringResult = ProcessFunctionResult(functionResultValue, chatExecutionSettings.ToolCallBehavior);
|
||||
|
||||
AddResponseMessage(chatOptions, chat, stringResult, errorMessage: null, functionToolCall, this.Logger);
|
||||
|
||||
// If filter requested termination, returning latest function result.
|
||||
if (invocationContext.Terminate)
|
||||
{
|
||||
if (this.Logger.IsEnabled(LogLevel.Debug))
|
||||
{
|
||||
this.Logger.LogDebug("Filter requested termination of automatic function invocation.");
|
||||
}
|
||||
|
||||
return [chat.Last()];
|
||||
}
|
||||
|
||||
static void AddResponseMessage(ChatCompletionsOptions chatOptions, ChatHistory chat, string? result, string? errorMessage, ChatCompletionsToolCall toolCall, ILogger logger)
|
||||
{
|
||||
// Log any error
|
||||
@@ -463,7 +495,7 @@ internal abstract class ClientCore
|
||||
chatOptions.ToolChoice = ChatCompletionsToolChoice.None;
|
||||
chatOptions.Tools.Clear();
|
||||
|
||||
if (iteration >= chatExecutionSettings.ToolCallBehavior!.MaximumUseAttempts)
|
||||
if (requestIndex >= chatExecutionSettings.ToolCallBehavior!.MaximumUseAttempts)
|
||||
{
|
||||
// Don't add any tools as we've reached the maximum attempts limit.
|
||||
if (this.Logger.IsEnabled(LogLevel.Debug))
|
||||
@@ -487,7 +519,7 @@ internal abstract class ClientCore
|
||||
}
|
||||
|
||||
// Disable auto invocation if we've exceeded the allowed limit.
|
||||
if (iteration >= chatExecutionSettings.ToolCallBehavior!.MaximumAutoInvokeAttempts)
|
||||
if (requestIndex >= chatExecutionSettings.ToolCallBehavior!.MaximumAutoInvokeAttempts)
|
||||
{
|
||||
autoInvoke = false;
|
||||
if (this.Logger.IsEnabled(LogLevel.Debug))
|
||||
@@ -519,7 +551,8 @@ internal abstract class ClientCore
|
||||
Dictionary<int, string>? toolCallIdsByIndex = null;
|
||||
Dictionary<int, string>? functionNamesByIndex = null;
|
||||
Dictionary<int, StringBuilder>? functionArgumentBuildersByIndex = null;
|
||||
for (int iteration = 1; ; iteration++)
|
||||
|
||||
for (int requestIndex = 1; ; requestIndex++)
|
||||
{
|
||||
// Make the request.
|
||||
var response = await RunRequestAsync(() => this.Client.GetChatCompletionsStreamingAsync(chatOptions, cancellationToken)).ConfigureAwait(false);
|
||||
@@ -589,8 +622,10 @@ internal abstract class ClientCore
|
||||
chat.Add(new OpenAIChatMessageContent(streamedRole ?? default, content, this.DeploymentOrModelName, toolCalls, metadata) { AuthorName = streamedName });
|
||||
|
||||
// Respond to each tooling request.
|
||||
foreach (ChatCompletionsFunctionToolCall toolCall in toolCalls)
|
||||
for (int toolCallIndex = 0; toolCallIndex < toolCalls.Length; toolCallIndex++)
|
||||
{
|
||||
ChatCompletionsFunctionToolCall toolCall = toolCalls[toolCallIndex];
|
||||
|
||||
// We currently only know about function tool calls. If it's anything else, we'll respond with an error.
|
||||
if (string.IsNullOrEmpty(toolCall.Name))
|
||||
{
|
||||
@@ -628,14 +663,31 @@ internal abstract class ClientCore
|
||||
}
|
||||
|
||||
// Now, invoke the function, and add the resulting tool call message to the chat options.
|
||||
FunctionResult functionResult = new(function) { Culture = kernel.Culture };
|
||||
AutoFunctionInvocationContext invocationContext = new(kernel, function, functionResult, chat)
|
||||
{
|
||||
Arguments = functionArgs,
|
||||
RequestSequenceIndex = requestIndex - 1,
|
||||
FunctionSequenceIndex = toolCallIndex,
|
||||
FunctionCount = toolCalls.Length
|
||||
};
|
||||
|
||||
s_inflightAutoInvokes.Value++;
|
||||
object? functionResult;
|
||||
try
|
||||
{
|
||||
// Note that we explicitly do not use executionSettings here; those pertain to the all-up operation and not necessarily to any
|
||||
// further calls made as part of this function invocation. In particular, we must not use function calling settings naively here,
|
||||
// as the called function could in turn telling the model about itself as a possible candidate for invocation.
|
||||
functionResult = (await function.InvokeAsync(kernel, functionArgs, cancellationToken: cancellationToken).ConfigureAwait(false)).GetValue<object>() ?? string.Empty;
|
||||
invocationContext = await OnAutoFunctionInvocationAsync(kernel, invocationContext, async (context) =>
|
||||
{
|
||||
// Check if filter requested termination.
|
||||
if (context.Terminate)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// Note that we explicitly do not use executionSettings here; those pertain to the all-up operation and not necessarily to any
|
||||
// further calls made as part of this function invocation. In particular, we must not use function calling settings naively here,
|
||||
// as the called function could in turn telling the model about itself as a possible candidate for invocation.
|
||||
context.Result = await function.InvokeAsync(kernel, invocationContext.Arguments, cancellationToken: cancellationToken).ConfigureAwait(false);
|
||||
}).ConfigureAwait(false);
|
||||
}
|
||||
#pragma warning disable CA1031 // Do not catch general exception types
|
||||
catch (Exception e)
|
||||
@@ -649,10 +701,25 @@ internal abstract class ClientCore
|
||||
s_inflightAutoInvokes.Value--;
|
||||
}
|
||||
|
||||
var stringResult = ProcessFunctionResult(functionResult, chatExecutionSettings.ToolCallBehavior);
|
||||
// Apply any changes from the auto function invocation filters context to final result.
|
||||
functionResult = invocationContext.Result;
|
||||
|
||||
object functionResultValue = functionResult.GetValue<object>() ?? string.Empty;
|
||||
var stringResult = ProcessFunctionResult(functionResultValue, chatExecutionSettings.ToolCallBehavior);
|
||||
|
||||
AddResponseMessage(chatOptions, chat, streamedRole, toolCall, metadata, stringResult, errorMessage: null, this.Logger);
|
||||
|
||||
// If filter requested termination, breaking request iteration loop.
|
||||
if (invocationContext.Terminate)
|
||||
{
|
||||
if (this.Logger.IsEnabled(LogLevel.Debug))
|
||||
{
|
||||
this.Logger.LogDebug("Filter requested termination of automatic function invocation.");
|
||||
}
|
||||
|
||||
yield break;
|
||||
}
|
||||
|
||||
static void AddResponseMessage(
|
||||
ChatCompletionsOptions chatOptions, ChatHistory chat, ChatRole? streamedRole, ChatCompletionsToolCall tool, IReadOnlyDictionary<string, object?>? metadata,
|
||||
string? result, string? errorMessage, ILogger logger)
|
||||
@@ -677,7 +744,7 @@ internal abstract class ClientCore
|
||||
chatOptions.ToolChoice = ChatCompletionsToolChoice.None;
|
||||
chatOptions.Tools.Clear();
|
||||
|
||||
if (iteration >= chatExecutionSettings.ToolCallBehavior!.MaximumUseAttempts)
|
||||
if (requestIndex >= chatExecutionSettings.ToolCallBehavior!.MaximumUseAttempts)
|
||||
{
|
||||
// Don't add any tools as we've reached the maximum attempts limit.
|
||||
if (this.Logger.IsEnabled(LogLevel.Debug))
|
||||
@@ -701,7 +768,7 @@ internal abstract class ClientCore
|
||||
}
|
||||
|
||||
// Disable auto invocation if we've exceeded the allowed limit.
|
||||
if (iteration >= chatExecutionSettings.ToolCallBehavior!.MaximumAutoInvokeAttempts)
|
||||
if (requestIndex >= chatExecutionSettings.ToolCallBehavior!.MaximumAutoInvokeAttempts)
|
||||
{
|
||||
autoInvoke = false;
|
||||
if (this.Logger.IsEnabled(LogLevel.Debug))
|
||||
@@ -1275,4 +1342,42 @@ internal abstract class ClientCore
|
||||
return JsonSerializer.Serialize(functionResult, toolCallBehavior?.ToolCallResultSerializerOptions);
|
||||
#pragma warning restore CS0618 // Type or member is obsolete
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Executes auto function invocation filters and/or function itself.
|
||||
/// This method can be moved to <see cref="Kernel"/> when auto function invocation logic will be extracted to common place.
|
||||
/// </summary>
|
||||
private static async Task<AutoFunctionInvocationContext> OnAutoFunctionInvocationAsync(
|
||||
Kernel kernel,
|
||||
AutoFunctionInvocationContext context,
|
||||
Func<AutoFunctionInvocationContext, Task> functionCallCallback)
|
||||
{
|
||||
await InvokeFilterOrFunctionAsync(kernel.AutoFunctionInvocationFilters, functionCallCallback, context).ConfigureAwait(false);
|
||||
|
||||
return context;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// This method will execute auto function invocation filters and function recursively.
|
||||
/// If there are no registered filters, just function will be executed.
|
||||
/// If there are registered filters, filter on <paramref name="index"/> position will be executed.
|
||||
/// Second parameter of filter is callback. It can be either filter on <paramref name="index"/> + 1 position or function if there are no remaining filters to execute.
|
||||
/// Function will be always executed as last step after all filters.
|
||||
/// </summary>
|
||||
private static async Task InvokeFilterOrFunctionAsync(
|
||||
IList<IAutoFunctionInvocationFilter>? autoFunctionInvocationFilters,
|
||||
Func<AutoFunctionInvocationContext, Task> functionCallCallback,
|
||||
AutoFunctionInvocationContext context,
|
||||
int index = 0)
|
||||
{
|
||||
if (autoFunctionInvocationFilters is { Count: > 0 } && index < autoFunctionInvocationFilters.Count)
|
||||
{
|
||||
await autoFunctionInvocationFilters[index].OnAutoFunctionInvocationAsync(context,
|
||||
(context) => InvokeFilterOrFunctionAsync(autoFunctionInvocationFilters, functionCallCallback, context, index + 1)).ConfigureAwait(false);
|
||||
}
|
||||
else
|
||||
{
|
||||
await functionCallCallback(context).ConfigureAwait(false);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+608
@@ -0,0 +1,608 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Net;
|
||||
using System.Net.Http;
|
||||
using System.Threading.Tasks;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.SemanticKernel;
|
||||
using Microsoft.SemanticKernel.ChatCompletion;
|
||||
using Microsoft.SemanticKernel.Connectors.OpenAI;
|
||||
using Xunit;
|
||||
|
||||
namespace SemanticKernel.Connectors.UnitTests.OpenAI.FunctionCalling;
|
||||
|
||||
public sealed class AutoFunctionInvocationFilterTests : IDisposable
|
||||
{
|
||||
private readonly MultipleHttpMessageHandlerStub _messageHandlerStub;
|
||||
private readonly HttpClient _httpClient;
|
||||
|
||||
public AutoFunctionInvocationFilterTests()
|
||||
{
|
||||
this._messageHandlerStub = new MultipleHttpMessageHandlerStub();
|
||||
|
||||
this._httpClient = new HttpClient(this._messageHandlerStub, false);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task FiltersAreExecutedCorrectlyAsync()
|
||||
{
|
||||
// Arrange
|
||||
int filterInvocations = 0;
|
||||
int functionInvocations = 0;
|
||||
int[] expectedRequestSequenceNumbers = [0, 0, 1, 1];
|
||||
int[] expectedFunctionSequenceNumbers = [0, 1, 0, 1];
|
||||
List<int> requestSequenceNumbers = [];
|
||||
List<int> functionSequenceNumbers = [];
|
||||
Kernel? contextKernel = null;
|
||||
|
||||
var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => { functionInvocations++; return parameter; }, "Function1");
|
||||
var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => { functionInvocations++; return parameter; }, "Function2");
|
||||
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]);
|
||||
|
||||
var kernel = this.GetKernelWithFilter(plugin, async (context, next) =>
|
||||
{
|
||||
contextKernel = context.Kernel;
|
||||
|
||||
if (context.ChatHistory.Last() is OpenAIChatMessageContent content)
|
||||
{
|
||||
Assert.Equal(2, content.ToolCalls.Count);
|
||||
}
|
||||
|
||||
requestSequenceNumbers.Add(context.RequestSequenceIndex);
|
||||
functionSequenceNumbers.Add(context.FunctionSequenceIndex);
|
||||
|
||||
await next(context);
|
||||
|
||||
filterInvocations++;
|
||||
});
|
||||
|
||||
this._messageHandlerStub.ResponsesToReturn = GetFunctionCallingResponses();
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokePromptAsync("Test prompt", new(new OpenAIPromptExecutionSettings
|
||||
{
|
||||
ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
|
||||
}));
|
||||
|
||||
// Assert
|
||||
Assert.Equal(4, filterInvocations);
|
||||
Assert.Equal(4, functionInvocations);
|
||||
Assert.Equal(expectedRequestSequenceNumbers, requestSequenceNumbers);
|
||||
Assert.Equal(expectedFunctionSequenceNumbers, functionSequenceNumbers);
|
||||
Assert.Same(kernel, contextKernel);
|
||||
Assert.Equal("Test chat response", result.ToString());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task FiltersAreExecutedCorrectlyOnStreamingAsync()
|
||||
{
|
||||
// Arrange
|
||||
int filterInvocations = 0;
|
||||
int functionInvocations = 0;
|
||||
List<int> requestSequenceNumbers = [];
|
||||
List<int> functionSequenceNumbers = [];
|
||||
|
||||
var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => { functionInvocations++; return parameter; }, "Function1");
|
||||
var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => { functionInvocations++; return parameter; }, "Function2");
|
||||
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]);
|
||||
|
||||
var kernel = this.GetKernelWithFilter(plugin, async (context, next) =>
|
||||
{
|
||||
if (context.ChatHistory.Last() is OpenAIChatMessageContent content)
|
||||
{
|
||||
Assert.Equal(2, content.ToolCalls.Count);
|
||||
}
|
||||
|
||||
requestSequenceNumbers.Add(context.RequestSequenceIndex);
|
||||
functionSequenceNumbers.Add(context.FunctionSequenceIndex);
|
||||
|
||||
await next(context);
|
||||
|
||||
filterInvocations++;
|
||||
});
|
||||
|
||||
this._messageHandlerStub.ResponsesToReturn = GetFunctionCallingStreamingResponses();
|
||||
|
||||
var executionSettings = new OpenAIPromptExecutionSettings { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };
|
||||
|
||||
// Act
|
||||
await foreach (var item in kernel.InvokePromptStreamingAsync("Test prompt", new(executionSettings)))
|
||||
{ }
|
||||
|
||||
// Assert
|
||||
Assert.Equal(4, filterInvocations);
|
||||
Assert.Equal(4, functionInvocations);
|
||||
Assert.Equal([0, 0, 1, 1], requestSequenceNumbers);
|
||||
Assert.Equal([0, 1, 0, 1], functionSequenceNumbers);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task DifferentWaysOfAddingFiltersWorkCorrectlyAsync()
|
||||
{
|
||||
// Arrange
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => "Result");
|
||||
var executionOrder = new List<string>();
|
||||
|
||||
var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => parameter, "Function1");
|
||||
var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => parameter, "Function2");
|
||||
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]);
|
||||
|
||||
var filter1 = new AutoFunctionInvocationFilter(async (context, next) =>
|
||||
{
|
||||
executionOrder.Add("Filter1-Invoking");
|
||||
await next(context);
|
||||
});
|
||||
|
||||
var filter2 = new AutoFunctionInvocationFilter(async (context, next) =>
|
||||
{
|
||||
executionOrder.Add("Filter2-Invoking");
|
||||
await next(context);
|
||||
});
|
||||
|
||||
var builder = Kernel.CreateBuilder();
|
||||
|
||||
builder.Plugins.Add(plugin);
|
||||
|
||||
builder.AddOpenAIChatCompletion(
|
||||
modelId: "test-model-id",
|
||||
apiKey: "test-api-key",
|
||||
httpClient: this._httpClient);
|
||||
|
||||
this._messageHandlerStub.ResponsesToReturn = GetFunctionCallingResponses();
|
||||
|
||||
// Act
|
||||
|
||||
// Case #1 - Add filter to services
|
||||
builder.Services.AddSingleton<IAutoFunctionInvocationFilter>(filter1);
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
// Case #2 - Add filter to kernel
|
||||
kernel.AutoFunctionInvocationFilters.Add(filter2);
|
||||
|
||||
var result = await kernel.InvokePromptAsync("Test prompt", new(new OpenAIPromptExecutionSettings
|
||||
{
|
||||
ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
|
||||
}));
|
||||
|
||||
// Assert
|
||||
Assert.Equal("Filter1-Invoking", executionOrder[0]);
|
||||
Assert.Equal("Filter2-Invoking", executionOrder[1]);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(true)]
|
||||
[InlineData(false)]
|
||||
public async Task MultipleFiltersAreExecutedInOrderAsync(bool isStreaming)
|
||||
{
|
||||
// Arrange
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => "Result");
|
||||
var executionOrder = new List<string>();
|
||||
|
||||
var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => parameter, "Function1");
|
||||
var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => parameter, "Function2");
|
||||
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]);
|
||||
|
||||
var filter1 = new AutoFunctionInvocationFilter(async (context, next) =>
|
||||
{
|
||||
executionOrder.Add("Filter1-Invoking");
|
||||
await next(context);
|
||||
executionOrder.Add("Filter1-Invoked");
|
||||
});
|
||||
|
||||
var filter2 = new AutoFunctionInvocationFilter(async (context, next) =>
|
||||
{
|
||||
executionOrder.Add("Filter2-Invoking");
|
||||
await next(context);
|
||||
executionOrder.Add("Filter2-Invoked");
|
||||
});
|
||||
|
||||
var filter3 = new AutoFunctionInvocationFilter(async (context, next) =>
|
||||
{
|
||||
executionOrder.Add("Filter3-Invoking");
|
||||
await next(context);
|
||||
executionOrder.Add("Filter3-Invoked");
|
||||
});
|
||||
|
||||
var builder = Kernel.CreateBuilder();
|
||||
|
||||
builder.Plugins.Add(plugin);
|
||||
|
||||
builder.AddOpenAIChatCompletion(
|
||||
modelId: "test-model-id",
|
||||
apiKey: "test-api-key",
|
||||
httpClient: this._httpClient);
|
||||
|
||||
builder.Services.AddSingleton<IAutoFunctionInvocationFilter>(filter1);
|
||||
builder.Services.AddSingleton<IAutoFunctionInvocationFilter>(filter2);
|
||||
builder.Services.AddSingleton<IAutoFunctionInvocationFilter>(filter3);
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
var arguments = new KernelArguments(new OpenAIPromptExecutionSettings
|
||||
{
|
||||
ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
|
||||
});
|
||||
|
||||
// Act
|
||||
if (isStreaming)
|
||||
{
|
||||
this._messageHandlerStub.ResponsesToReturn = GetFunctionCallingStreamingResponses();
|
||||
|
||||
await foreach (var item in kernel.InvokePromptStreamingAsync("Test prompt", arguments))
|
||||
{ }
|
||||
}
|
||||
else
|
||||
{
|
||||
this._messageHandlerStub.ResponsesToReturn = GetFunctionCallingResponses();
|
||||
|
||||
await kernel.InvokePromptAsync("Test prompt", arguments);
|
||||
}
|
||||
|
||||
// Assert
|
||||
Assert.Equal("Filter1-Invoking", executionOrder[0]);
|
||||
Assert.Equal("Filter2-Invoking", executionOrder[1]);
|
||||
Assert.Equal("Filter3-Invoking", executionOrder[2]);
|
||||
Assert.Equal("Filter3-Invoked", executionOrder[3]);
|
||||
Assert.Equal("Filter2-Invoked", executionOrder[4]);
|
||||
Assert.Equal("Filter1-Invoked", executionOrder[5]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task FilterCanOverrideArgumentsAsync()
|
||||
{
|
||||
// Arrange
|
||||
const string NewValue = "NewValue";
|
||||
|
||||
var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => { return parameter; }, "Function1");
|
||||
var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => { return parameter; }, "Function2");
|
||||
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]);
|
||||
|
||||
var kernel = this.GetKernelWithFilter(plugin, async (context, next) =>
|
||||
{
|
||||
context.Arguments!["parameter"] = NewValue;
|
||||
await next(context);
|
||||
context.Terminate = true;
|
||||
});
|
||||
|
||||
this._messageHandlerStub.ResponsesToReturn = GetFunctionCallingResponses();
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokePromptAsync("Test prompt", new(new OpenAIPromptExecutionSettings
|
||||
{
|
||||
ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
|
||||
}));
|
||||
|
||||
// Assert
|
||||
Assert.Equal("NewValue", result.ToString());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task FilterCanHandleExceptionAsync()
|
||||
{
|
||||
// Arrange
|
||||
var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => { throw new KernelException("Exception from Function1"); }, "Function1");
|
||||
var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => "Result from Function2", "Function2");
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]);
|
||||
|
||||
var kernel = this.GetKernelWithFilter(plugin, async (context, next) =>
|
||||
{
|
||||
try
|
||||
{
|
||||
await next(context);
|
||||
}
|
||||
catch (KernelException exception)
|
||||
{
|
||||
Assert.Equal("Exception from Function1", exception.Message);
|
||||
context.Result = new FunctionResult(context.Result, "Result from filter");
|
||||
}
|
||||
});
|
||||
|
||||
this._messageHandlerStub.ResponsesToReturn = GetFunctionCallingResponses();
|
||||
|
||||
var chatCompletion = new OpenAIChatCompletionService(modelId: "test-model-id", apiKey: "test-api-key", httpClient: this._httpClient);
|
||||
var executionSettings = new OpenAIPromptExecutionSettings { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };
|
||||
|
||||
var chatHistory = new ChatHistory();
|
||||
|
||||
// Act
|
||||
var result = await chatCompletion.GetChatMessageContentsAsync(chatHistory, executionSettings, kernel);
|
||||
|
||||
var firstFunctionResult = chatHistory[^2].Content;
|
||||
var secondFunctionResult = chatHistory[^1].Content;
|
||||
|
||||
// Assert
|
||||
Assert.Equal("Result from filter", firstFunctionResult);
|
||||
Assert.Equal("Result from Function2", secondFunctionResult);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task FilterCanHandleExceptionOnStreamingAsync()
|
||||
{
|
||||
// Arrange
|
||||
var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => { throw new KernelException("Exception from Function1"); }, "Function1");
|
||||
var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => "Result from Function2", "Function2");
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]);
|
||||
|
||||
var kernel = this.GetKernelWithFilter(plugin, async (context, next) =>
|
||||
{
|
||||
try
|
||||
{
|
||||
await next(context);
|
||||
}
|
||||
catch (KernelException)
|
||||
{
|
||||
context.Result = new FunctionResult(context.Result, "Result from filter");
|
||||
}
|
||||
});
|
||||
|
||||
this._messageHandlerStub.ResponsesToReturn = GetFunctionCallingStreamingResponses();
|
||||
|
||||
var chatCompletion = new OpenAIChatCompletionService(modelId: "test-model-id", apiKey: "test-api-key", httpClient: this._httpClient);
|
||||
var chatHistory = new ChatHistory();
|
||||
var executionSettings = new OpenAIPromptExecutionSettings { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };
|
||||
|
||||
// Act
|
||||
await foreach (var item in chatCompletion.GetStreamingChatMessageContentsAsync(chatHistory, executionSettings, kernel))
|
||||
{ }
|
||||
|
||||
var firstFunctionResult = chatHistory[^2].Content;
|
||||
var secondFunctionResult = chatHistory[^1].Content;
|
||||
|
||||
// Assert
|
||||
Assert.Equal("Result from filter", firstFunctionResult);
|
||||
Assert.Equal("Result from Function2", secondFunctionResult);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task FiltersCanSkipFunctionExecutionAsync()
|
||||
{
|
||||
// Arrange
|
||||
int filterInvocations = 0;
|
||||
int firstFunctionInvocations = 0;
|
||||
int secondFunctionInvocations = 0;
|
||||
|
||||
var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => { firstFunctionInvocations++; return parameter; }, "Function1");
|
||||
var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => { secondFunctionInvocations++; return parameter; }, "Function2");
|
||||
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]);
|
||||
|
||||
var kernel = this.GetKernelWithFilter(plugin, async (context, next) =>
|
||||
{
|
||||
// Filter delegate is invoked only for second function, the first one should be skipped.
|
||||
if (context.Function.Name == "Function2")
|
||||
{
|
||||
await next(context);
|
||||
}
|
||||
|
||||
filterInvocations++;
|
||||
});
|
||||
|
||||
using var response1 = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OpenAITestHelper.GetTestResponse("filters_multiple_function_calls_test_response.json")) };
|
||||
using var response2 = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OpenAITestHelper.GetTestResponse("chat_completion_test_response.json")) };
|
||||
|
||||
this._messageHandlerStub.ResponsesToReturn = [response1, response2];
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokePromptAsync("Test prompt", new(new OpenAIPromptExecutionSettings
|
||||
{
|
||||
ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
|
||||
}));
|
||||
|
||||
// Assert
|
||||
Assert.Equal(2, filterInvocations);
|
||||
Assert.Equal(0, firstFunctionInvocations);
|
||||
Assert.Equal(1, secondFunctionInvocations);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PreFilterCanTerminateOperationAsync()
|
||||
{
|
||||
// Arrange
|
||||
int firstFunctionInvocations = 0;
|
||||
int secondFunctionInvocations = 0;
|
||||
|
||||
var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => { firstFunctionInvocations++; return parameter; }, "Function1");
|
||||
var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => { secondFunctionInvocations++; return parameter; }, "Function2");
|
||||
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]);
|
||||
|
||||
var kernel = this.GetKernelWithFilter(plugin, async (context, next) =>
|
||||
{
|
||||
// Terminating before first function, so all functions won't be invoked.
|
||||
context.Terminate = true;
|
||||
|
||||
await next(context);
|
||||
});
|
||||
|
||||
this._messageHandlerStub.ResponsesToReturn = GetFunctionCallingResponses();
|
||||
|
||||
// Act
|
||||
await kernel.InvokePromptAsync("Test prompt", new(new OpenAIPromptExecutionSettings
|
||||
{
|
||||
ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
|
||||
}));
|
||||
|
||||
// Assert
|
||||
Assert.Equal(0, firstFunctionInvocations);
|
||||
Assert.Equal(0, secondFunctionInvocations);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PreFilterCanTerminateOperationOnStreamingAsync()
|
||||
{
|
||||
// Arrange
|
||||
int firstFunctionInvocations = 0;
|
||||
int secondFunctionInvocations = 0;
|
||||
|
||||
var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => { firstFunctionInvocations++; return parameter; }, "Function1");
|
||||
var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => { secondFunctionInvocations++; return parameter; }, "Function2");
|
||||
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]);
|
||||
|
||||
var kernel = this.GetKernelWithFilter(plugin, async (context, next) =>
|
||||
{
|
||||
// Terminating before first function, so all functions won't be invoked.
|
||||
context.Terminate = true;
|
||||
|
||||
await next(context);
|
||||
});
|
||||
|
||||
this._messageHandlerStub.ResponsesToReturn = GetFunctionCallingStreamingResponses();
|
||||
|
||||
var executionSettings = new OpenAIPromptExecutionSettings { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };
|
||||
|
||||
// Act
|
||||
await foreach (var item in kernel.InvokePromptStreamingAsync("Test prompt", new(executionSettings)))
|
||||
{ }
|
||||
|
||||
// Assert
|
||||
Assert.Equal(0, firstFunctionInvocations);
|
||||
Assert.Equal(0, secondFunctionInvocations);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PostFilterCanTerminateOperationAsync()
|
||||
{
|
||||
// Arrange
|
||||
int firstFunctionInvocations = 0;
|
||||
int secondFunctionInvocations = 0;
|
||||
List<int> requestSequenceNumbers = [];
|
||||
List<int> functionSequenceNumbers = [];
|
||||
|
||||
var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => { firstFunctionInvocations++; return parameter; }, "Function1");
|
||||
var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => { secondFunctionInvocations++; return parameter; }, "Function2");
|
||||
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]);
|
||||
|
||||
var kernel = this.GetKernelWithFilter(plugin, async (context, next) =>
|
||||
{
|
||||
requestSequenceNumbers.Add(context.RequestSequenceIndex);
|
||||
functionSequenceNumbers.Add(context.FunctionSequenceIndex);
|
||||
|
||||
await next(context);
|
||||
|
||||
// Terminating after first function, so second function won't be invoked.
|
||||
context.Terminate = true;
|
||||
});
|
||||
|
||||
this._messageHandlerStub.ResponsesToReturn = GetFunctionCallingResponses();
|
||||
|
||||
// Act
|
||||
await kernel.InvokePromptAsync("Test prompt", new(new OpenAIPromptExecutionSettings
|
||||
{
|
||||
ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
|
||||
}));
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, firstFunctionInvocations);
|
||||
Assert.Equal(0, secondFunctionInvocations);
|
||||
Assert.Equal([0], requestSequenceNumbers);
|
||||
Assert.Equal([0], functionSequenceNumbers);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PostFilterCanTerminateOperationOnStreamingAsync()
|
||||
{
|
||||
// Arrange
|
||||
int firstFunctionInvocations = 0;
|
||||
int secondFunctionInvocations = 0;
|
||||
List<int> requestSequenceNumbers = [];
|
||||
List<int> functionSequenceNumbers = [];
|
||||
|
||||
var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => { firstFunctionInvocations++; return parameter; }, "Function1");
|
||||
var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => { secondFunctionInvocations++; return parameter; }, "Function2");
|
||||
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]);
|
||||
|
||||
var kernel = this.GetKernelWithFilter(plugin, async (context, next) =>
|
||||
{
|
||||
requestSequenceNumbers.Add(context.RequestSequenceIndex);
|
||||
functionSequenceNumbers.Add(context.FunctionSequenceIndex);
|
||||
|
||||
await next(context);
|
||||
|
||||
// Terminating after first function, so second function won't be invoked.
|
||||
context.Terminate = true;
|
||||
});
|
||||
|
||||
this._messageHandlerStub.ResponsesToReturn = GetFunctionCallingStreamingResponses();
|
||||
|
||||
var executionSettings = new OpenAIPromptExecutionSettings { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };
|
||||
|
||||
// Act
|
||||
await foreach (var item in kernel.InvokePromptStreamingAsync("Test prompt", new(executionSettings)))
|
||||
{ }
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, firstFunctionInvocations);
|
||||
Assert.Equal(0, secondFunctionInvocations);
|
||||
Assert.Equal([0], requestSequenceNumbers);
|
||||
Assert.Equal([0], functionSequenceNumbers);
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
this._httpClient.Dispose();
|
||||
this._messageHandlerStub.Dispose();
|
||||
}
|
||||
|
||||
#region private
|
||||
|
||||
#pragma warning disable CA2000 // Dispose objects before losing scope
|
||||
private static List<HttpResponseMessage> GetFunctionCallingResponses()
|
||||
{
|
||||
return [
|
||||
new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OpenAITestHelper.GetTestResponse("filters_multiple_function_calls_test_response.json")) },
|
||||
new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OpenAITestHelper.GetTestResponse("filters_multiple_function_calls_test_response.json")) },
|
||||
new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OpenAITestHelper.GetTestResponse("chat_completion_test_response.json")) }
|
||||
];
|
||||
}
|
||||
|
||||
private static List<HttpResponseMessage> GetFunctionCallingStreamingResponses()
|
||||
{
|
||||
return [
|
||||
new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OpenAITestHelper.GetTestResponse("filters_streaming_multiple_function_calls_test_response.txt")) },
|
||||
new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OpenAITestHelper.GetTestResponse("filters_streaming_multiple_function_calls_test_response.txt")) },
|
||||
new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OpenAITestHelper.GetTestResponse("chat_completion_streaming_test_response.txt")) }
|
||||
];
|
||||
}
|
||||
#pragma warning restore CA2000
|
||||
|
||||
private Kernel GetKernelWithFilter(
|
||||
KernelPlugin plugin,
|
||||
Func<AutoFunctionInvocationContext, Func<AutoFunctionInvocationContext, Task>, Task>? onAutoFunctionInvocation)
|
||||
{
|
||||
var builder = Kernel.CreateBuilder();
|
||||
var filter = new AutoFunctionInvocationFilter(onAutoFunctionInvocation);
|
||||
|
||||
builder.Plugins.Add(plugin);
|
||||
builder.Services.AddSingleton<IAutoFunctionInvocationFilter>(filter);
|
||||
|
||||
builder.AddOpenAIChatCompletion(
|
||||
modelId: "test-model-id",
|
||||
apiKey: "test-api-key",
|
||||
httpClient: this._httpClient);
|
||||
|
||||
return builder.Build();
|
||||
}
|
||||
|
||||
private sealed class AutoFunctionInvocationFilter(
|
||||
Func<AutoFunctionInvocationContext, Func<AutoFunctionInvocationContext, Task>, Task>? onAutoFunctionInvocation) : IAutoFunctionInvocationFilter
|
||||
{
|
||||
private readonly Func<AutoFunctionInvocationContext, Func<AutoFunctionInvocationContext, Task>, Task>? _onAutoFunctionInvocation = onAutoFunctionInvocation;
|
||||
|
||||
public Task OnAutoFunctionInvocationAsync(AutoFunctionInvocationContext context, Func<AutoFunctionInvocationContext, Task> next) =>
|
||||
this._onAutoFunctionInvocation?.Invoke(context, next) ?? Task.CompletedTask;
|
||||
}
|
||||
|
||||
#endregion
|
||||
}
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
{
|
||||
"id": "response-id",
|
||||
"object": "chat.completion",
|
||||
"created": 1699896916,
|
||||
"model": "gpt-3.5-turbo-0613",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": null,
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "1",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "MyPlugin-Function1",
|
||||
"arguments": "{\n\"parameter\": \"function1-value\"\n}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "2",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "MyPlugin-Function2",
|
||||
"arguments": "{\n\"parameter\": \"function2-value\"\n}"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"logprobs": null,
|
||||
"finish_reason": "tool_calls"
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
"prompt_tokens": 82,
|
||||
"completion_tokens": 17,
|
||||
"total_tokens": 99
|
||||
}
|
||||
}
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
data: {"id":"response-id","object":"chat.completion.chunk","created":1704212243,"model":"gpt-4","system_fingerprint":null,"choices":[{"index":0,"delta":{"role":"assistant","content":"Test chat streaming response","tool_calls":[{"index":0,"id":"1","type":"function","function":{"name":"MyPlugin-Function1","arguments":"{\n\"parameter\": \"function1-value\"\n}"}}]},"finish_reason":"tool_calls"}]}
|
||||
|
||||
data: {"id":"response-id","object":"chat.completion.chunk","created":1704212243,"model":"gpt-4","system_fingerprint":null,"choices":[{"index":0,"delta":{"role":"assistant","content":"Test chat streaming response","tool_calls":[{"index":1,"id":"2","type":"function","function":{"name":"MyPlugin-Function2","arguments":"{\n\"parameter\": \"function2-value\"\n}"}}]},"finish_reason":"tool_calls"}]}
|
||||
|
||||
data: [DONE]
|
||||
@@ -29,14 +29,13 @@ public sealed class OpenAIToolsTests : BaseIntegrationTest
|
||||
|
||||
var invokedFunctions = new List<string>();
|
||||
|
||||
#pragma warning disable CS0618 // Events are deprecated
|
||||
void MyInvokingHandler(object? sender, FunctionInvokingEventArgs e)
|
||||
var filter = new FakeFunctionFilter(async (context, next) =>
|
||||
{
|
||||
invokedFunctions.Add(e.Function.Name);
|
||||
}
|
||||
invokedFunctions.Add(context.Function.Name);
|
||||
await next(context);
|
||||
});
|
||||
|
||||
kernel.FunctionInvoking += MyInvokingHandler;
|
||||
#pragma warning restore CS0618 // Events are deprecated
|
||||
kernel.FunctionInvocationFilters.Add(filter);
|
||||
|
||||
// Act
|
||||
OpenAIPromptExecutionSettings settings = new() { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };
|
||||
@@ -56,14 +55,13 @@ public sealed class OpenAIToolsTests : BaseIntegrationTest
|
||||
|
||||
var invokedFunctions = new List<string>();
|
||||
|
||||
#pragma warning disable CS0618 // Events are deprecated
|
||||
void MyInvokingHandler(object? sender, FunctionInvokingEventArgs e)
|
||||
var filter = new FakeFunctionFilter(async (context, next) =>
|
||||
{
|
||||
invokedFunctions.Add($"{e.Function.Name}({string.Join(", ", e.Arguments)})");
|
||||
}
|
||||
invokedFunctions.Add($"{context.Function.Name}({string.Join(", ", context.Arguments)})");
|
||||
await next(context);
|
||||
});
|
||||
|
||||
kernel.FunctionInvoking += MyInvokingHandler;
|
||||
#pragma warning restore CS0618 // Events are deprecated
|
||||
kernel.FunctionInvocationFilters.Add(filter);
|
||||
|
||||
// Act
|
||||
OpenAIPromptExecutionSettings settings = new() { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };
|
||||
@@ -534,4 +532,22 @@ public sealed class OpenAIToolsTests : BaseIntegrationTest
|
||||
public string Name { get; set; } = string.Empty;
|
||||
public string Country { get; set; } = string.Empty;
|
||||
}
|
||||
|
||||
#region private
|
||||
|
||||
private sealed class FakeFunctionFilter : IFunctionInvocationFilter
|
||||
{
|
||||
private readonly Func<FunctionInvocationContext, Func<FunctionInvocationContext, Task>, Task>? _onFunctionInvocation;
|
||||
|
||||
public FakeFunctionFilter(
|
||||
Func<FunctionInvocationContext, Func<FunctionInvocationContext, Task>, Task>? onFunctionInvocation = null)
|
||||
{
|
||||
this._onFunctionInvocation = onFunctionInvocation;
|
||||
}
|
||||
|
||||
public Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next) =>
|
||||
this._onFunctionInvocation?.Invoke(context, next) ?? Task.CompletedTask;
|
||||
}
|
||||
|
||||
#endregion
|
||||
}
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- https://learn.microsoft.com/en-us/dotnet/fundamentals/package-validation/diagnostic-ids -->
|
||||
<Suppressions xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
|
||||
<Suppression>
|
||||
<DiagnosticId>CP0001</DiagnosticId>
|
||||
<Target>T:Microsoft.SemanticKernel.IFunctionFilter</Target>
|
||||
<Left>lib/netstandard2.0/Microsoft.SemanticKernel.Abstractions.dll</Left>
|
||||
<Right>lib/netstandard2.0/Microsoft.SemanticKernel.Abstractions.dll</Right>
|
||||
<IsBaselineSuppression>true</IsBaselineSuppression>
|
||||
</Suppression>
|
||||
<Suppression>
|
||||
<DiagnosticId>CP0001</DiagnosticId>
|
||||
<Target>T:Microsoft.SemanticKernel.IPromptFilter</Target>
|
||||
<Left>lib/netstandard2.0/Microsoft.SemanticKernel.Abstractions.dll</Left>
|
||||
<Right>lib/netstandard2.0/Microsoft.SemanticKernel.Abstractions.dll</Right>
|
||||
<IsBaselineSuppression>true</IsBaselineSuppression>
|
||||
</Suppression>
|
||||
<Suppression>
|
||||
<DiagnosticId>CP0002</DiagnosticId>
|
||||
<Target>M:Microsoft.SemanticKernel.Kernel.get_FunctionFilters</Target>
|
||||
<Left>lib/netstandard2.0/Microsoft.SemanticKernel.Abstractions.dll</Left>
|
||||
<Right>lib/netstandard2.0/Microsoft.SemanticKernel.Abstractions.dll</Right>
|
||||
<IsBaselineSuppression>true</IsBaselineSuppression>
|
||||
</Suppression>
|
||||
<Suppression>
|
||||
<DiagnosticId>CP0002</DiagnosticId>
|
||||
<Target>M:Microsoft.SemanticKernel.Kernel.get_PromptFilters</Target>
|
||||
<Left>lib/netstandard2.0/Microsoft.SemanticKernel.Abstractions.dll</Left>
|
||||
<Right>lib/netstandard2.0/Microsoft.SemanticKernel.Abstractions.dll</Right>
|
||||
<IsBaselineSuppression>true</IsBaselineSuppression>
|
||||
</Suppression>
|
||||
</Suppressions>
|
||||
+85
@@ -0,0 +1,85 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
using Microsoft.SemanticKernel.ChatCompletion;
|
||||
|
||||
namespace Microsoft.SemanticKernel;
|
||||
|
||||
/// <summary>
|
||||
/// Class with data related to automatic function invocation.
|
||||
/// </summary>
|
||||
[Experimental("SKEXP0001")]
|
||||
public class AutoFunctionInvocationContext
|
||||
{
|
||||
/// <summary>
|
||||
/// Initializes a new instance of the <see cref="AutoFunctionInvocationContext"/> class.
|
||||
/// </summary>
|
||||
/// <param name="kernel">The <see cref="Microsoft.SemanticKernel.Kernel"/> containing services, plugins, and other state for use throughout the operation.</param>
|
||||
/// <param name="function">The <see cref="KernelFunction"/> with which this filter is associated.</param>
|
||||
/// <param name="result">The result of the function's invocation.</param>
|
||||
/// <param name="chatHistory">The chat history associated with automatic function invocation.</param>
|
||||
public AutoFunctionInvocationContext(
|
||||
Kernel kernel,
|
||||
KernelFunction function,
|
||||
FunctionResult result,
|
||||
ChatHistory chatHistory)
|
||||
{
|
||||
Verify.NotNull(kernel);
|
||||
Verify.NotNull(function);
|
||||
Verify.NotNull(result);
|
||||
Verify.NotNull(chatHistory);
|
||||
|
||||
this.Kernel = kernel;
|
||||
this.Function = function;
|
||||
this.Result = result;
|
||||
this.ChatHistory = chatHistory;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets the arguments associated with the operation.
|
||||
/// </summary>
|
||||
public KernelArguments? Arguments { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Request sequence index of automatic function invocation process. Starts from 0.
|
||||
/// </summary>
|
||||
public int RequestSequenceIndex { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Function sequence index. Starts from 0.
|
||||
/// </summary>
|
||||
public int FunctionSequenceIndex { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Number of functions that will be invoked during auto function invocation request.
|
||||
/// </summary>
|
||||
public int FunctionCount { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets the <see cref="Microsoft.SemanticKernel.ChatCompletion.ChatHistory"/> associated with automatic function invocation.
|
||||
/// </summary>
|
||||
public ChatHistory ChatHistory { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets the <see cref="KernelFunction"/> with which this filter is associated.
|
||||
/// </summary>
|
||||
public KernelFunction Function { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets the <see cref="Microsoft.SemanticKernel.Kernel"/> containing services, plugins, and other state for use throughout the operation.
|
||||
/// </summary>
|
||||
public Kernel Kernel { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets or sets the result of the function's invocation.
|
||||
/// </summary>
|
||||
public FunctionResult Result { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets or sets a value indicating whether the operation associated with the filter should be terminated.
|
||||
/// By default it's <see langword="false"/>, in this case all functions will be executed.
|
||||
/// As soon as it's set to <see langword="true"/>, the remaining functions won't be executed and last request to LLM won't be performed.
|
||||
/// Automatic function invocation process will be terminated and result of last executed function will be returned to the caller.
|
||||
/// </summary>
|
||||
public bool Terminate { get; set; }
|
||||
}
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System;
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Microsoft.SemanticKernel;
|
||||
|
||||
#pragma warning disable CA1716 // Identifiers should not match keywords (Func<FunctionCallInvocationContext, Task> next)
|
||||
|
||||
/// <summary>
|
||||
/// Interface for filtering actions during automatic function invocation.
|
||||
/// </summary>
|
||||
[Experimental("SKEXP0001")]
|
||||
public interface IAutoFunctionInvocationFilter
|
||||
{
|
||||
/// <summary>
|
||||
/// Method which is called asynchronously before automatic function invocation.
|
||||
/// </summary>
|
||||
/// <param name="context">Instance of <see cref="AutoFunctionInvocationContext"/> with automatic function invocation details.</param>
|
||||
/// <param name="next">Delegate to the next filter in pipeline or function invocation itself. If it's not invoked, next filter won't be invoked and function invocation will be skipped.</param>
|
||||
Task OnAutoFunctionInvocationAsync(AutoFunctionInvocationContext context, Func<AutoFunctionInvocationContext, Task> next);
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
|
||||
namespace Microsoft.SemanticKernel;
|
||||
|
||||
/// <summary>
|
||||
/// Class with data related to function invocation.
|
||||
/// </summary>
|
||||
[Experimental("SKEXP0001")]
|
||||
public class FunctionInvocationContext
|
||||
{
|
||||
/// <summary>
|
||||
/// Initializes a new instance of the <see cref="FunctionInvocationContext"/> class.
|
||||
/// </summary>
|
||||
/// <param name="kernel">The <see cref="Microsoft.SemanticKernel.Kernel"/> containing services, plugins, and other state for use throughout the operation.</param>
|
||||
/// <param name="function">The <see cref="KernelFunction"/> with which this filter is associated.</param>
|
||||
/// <param name="arguments">The arguments associated with the operation.</param>
|
||||
/// <param name="result">The result of the function's invocation.</param>
|
||||
internal FunctionInvocationContext(Kernel kernel, KernelFunction function, KernelArguments arguments, FunctionResult result)
|
||||
{
|
||||
Verify.NotNull(kernel);
|
||||
Verify.NotNull(function);
|
||||
Verify.NotNull(arguments);
|
||||
|
||||
this.Kernel = kernel;
|
||||
this.Function = function;
|
||||
this.Arguments = arguments;
|
||||
this.Result = result;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets the <see cref="Microsoft.SemanticKernel.Kernel"/> containing services, plugins, and other state for use throughout the operation.
|
||||
/// </summary>
|
||||
public Kernel Kernel { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets the <see cref="KernelFunction"/> with which this filter is associated.
|
||||
/// </summary>
|
||||
public KernelFunction Function { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets the arguments associated with the operation.
|
||||
/// </summary>
|
||||
public KernelArguments Arguments { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets or sets the result of the function's invocation.
|
||||
/// </summary>
|
||||
public FunctionResult Result { get; set; }
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
|
||||
namespace Microsoft.SemanticKernel;
|
||||
|
||||
/// <summary>
|
||||
/// Interface for filtering actions during function invocation.
|
||||
/// </summary>
|
||||
[Experimental("SKEXP0001")]
|
||||
public interface IFunctionFilter
|
||||
{
|
||||
/// <summary>
|
||||
/// Method which is executed before function invocation.
|
||||
/// </summary>
|
||||
/// <param name="context">Data related to function before invocation.</param>
|
||||
void OnFunctionInvoking(FunctionInvokingContext context);
|
||||
|
||||
/// <summary>
|
||||
/// Method which is executed after function invocation.
|
||||
/// </summary>
|
||||
/// <param name="context">Data related to function after invocation.</param>
|
||||
void OnFunctionInvoked(FunctionInvokedContext context);
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System;
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Microsoft.SemanticKernel;
|
||||
|
||||
#pragma warning disable CA1716 // Identifiers should not match keywords (Func<FunctionInvocationContext, Task> next)
|
||||
|
||||
/// <summary>
|
||||
/// Interface for filtering actions during function invocation.
|
||||
/// </summary>
|
||||
[Experimental("SKEXP0001")]
|
||||
public interface IFunctionInvocationFilter
|
||||
{
|
||||
/// <summary>
|
||||
/// Method which is called asynchronously before function invocation.
|
||||
/// </summary>
|
||||
/// <param name="context">Instance of <see cref="FunctionInvocationContext"/> with function invocation details.</param>
|
||||
/// <param name="next">Delegate to the next filter in pipeline or function itself. If it's not invoked, next filter or function won't be invoked.</param>
|
||||
Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next);
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
|
||||
namespace Microsoft.SemanticKernel;
|
||||
|
||||
/// <summary>
|
||||
/// Interface for filtering actions during prompt rendering.
|
||||
/// </summary>
|
||||
[Experimental("SKEXP0001")]
|
||||
public interface IPromptFilter
|
||||
{
|
||||
/// <summary>
|
||||
/// Method which is executed before prompt rendering.
|
||||
/// </summary>
|
||||
/// <param name="context">Data related to prompt before rendering.</param>
|
||||
void OnPromptRendering(PromptRenderingContext context);
|
||||
|
||||
/// <summary>
|
||||
/// Method which is executed after prompt rendering.
|
||||
/// </summary>
|
||||
/// <param name="context">Data related to prompt after rendering.</param>
|
||||
void OnPromptRendered(PromptRenderedContext context);
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System;
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Microsoft.SemanticKernel;
|
||||
|
||||
#pragma warning disable CA1716 // Identifiers should not match keywords (Func<PromptRenderContext, Task> next)
|
||||
|
||||
/// <summary>
|
||||
/// Interface for filtering actions during prompt rendering.
|
||||
/// </summary>
|
||||
[Experimental("SKEXP0001")]
|
||||
public interface IPromptRenderFilter
|
||||
{
|
||||
/// <summary>
|
||||
/// Method which is called asynchronously before prompt rendering.
|
||||
/// </summary>
|
||||
/// <param name="context">Instance of <see cref="PromptRenderContext"/> with prompt rendering details.</param>
|
||||
/// <param name="next">Delegate to the next filter in pipeline or prompt rendering operation itself. If it's not invoked, next filter or prompt rendering won't be invoked.</param>
|
||||
Task OnPromptRenderAsync(PromptRenderContext context, Func<PromptRenderContext, Task> next);
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
|
||||
namespace Microsoft.SemanticKernel;
|
||||
|
||||
/// <summary>
|
||||
/// Class with data related to prompt rendering.
|
||||
/// </summary>
|
||||
[Experimental("SKEXP0001")]
|
||||
public sealed class PromptRenderContext
|
||||
{
|
||||
private string? _renderedPrompt;
|
||||
|
||||
/// <summary>
|
||||
/// Initializes a new instance of the <see cref="PromptRenderContext"/> class.
|
||||
/// </summary>
|
||||
/// <param name="kernel">The <see cref="Microsoft.SemanticKernel.Kernel"/> containing services, plugins, and other state for use throughout the operation.</param>
|
||||
/// <param name="function">The <see cref="KernelFunction"/> with which this filter is associated.</param>
|
||||
/// <param name="arguments">The arguments associated with the operation.</param>
|
||||
internal PromptRenderContext(Kernel kernel, KernelFunction function, KernelArguments arguments)
|
||||
{
|
||||
Verify.NotNull(kernel);
|
||||
Verify.NotNull(function);
|
||||
Verify.NotNull(arguments);
|
||||
|
||||
this.Kernel = kernel;
|
||||
this.Function = function;
|
||||
this.Arguments = arguments;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets the <see cref="Microsoft.SemanticKernel.Kernel"/> containing services, plugins, and other state for use throughout the operation.
|
||||
/// </summary>
|
||||
public Kernel Kernel { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets the <see cref="KernelFunction"/> with which this filter is associated.
|
||||
/// </summary>
|
||||
public KernelFunction Function { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets the arguments associated with the operation.
|
||||
/// </summary>
|
||||
public KernelArguments Arguments { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets or sets the rendered prompt.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// The filter may view the rendered prompt and change it, if desired.
|
||||
/// If there are multiple filters registered, subsequent filters may
|
||||
/// overwrite a value set by a previous filter. The final result is what will
|
||||
/// be the prompt used by the system.
|
||||
/// </remarks>
|
||||
public string? RenderedPrompt
|
||||
{
|
||||
get => this._renderedPrompt;
|
||||
set
|
||||
{
|
||||
Verify.NotNullOrWhiteSpace(value);
|
||||
this._renderedPrompt = value;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -28,15 +28,35 @@ public sealed class FunctionResult
|
||||
this.Metadata = metadata;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Initializes a new instance of the <see cref="FunctionResult"/> class.
|
||||
/// </summary>
|
||||
/// <param name="result">Instance of <see cref="FunctionResult"/> with result data to copy.</param>
|
||||
/// <param name="value">The resulting object of the function's invocation.</param>
|
||||
public FunctionResult(FunctionResult result, object? value = null)
|
||||
{
|
||||
Verify.NotNull(result);
|
||||
|
||||
this.Function = result.Function;
|
||||
this.Value = value ?? result.Value;
|
||||
this.Culture = result.Culture;
|
||||
this.Metadata = result.Metadata;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets the <see cref="KernelFunction"/> whose result is represented by this instance.
|
||||
/// </summary>
|
||||
public KernelFunction Function { get; }
|
||||
public KernelFunction Function { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets any metadata associated with the function's execution.
|
||||
/// </summary>
|
||||
public IReadOnlyDictionary<string, object?>? Metadata { get; }
|
||||
public IReadOnlyDictionary<string, object?>? Metadata { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// The culture configured on the Kernel that executed the function.
|
||||
/// </summary>
|
||||
public CultureInfo Culture { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets the <see cref="Type"/> of the function's result.
|
||||
@@ -88,9 +108,4 @@ public sealed class FunctionResult
|
||||
/// Function result object.
|
||||
/// </summary>
|
||||
internal object? Value { get; }
|
||||
|
||||
/// <summary>
|
||||
/// The culture configured on the Kernel that executed the function.
|
||||
/// </summary>
|
||||
internal CultureInfo Culture { get; }
|
||||
}
|
||||
|
||||
@@ -148,7 +148,6 @@ public abstract class KernelFunction
|
||||
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
|
||||
/// <returns>The result of the function's execution.</returns>
|
||||
/// <exception cref="ArgumentNullException"><paramref name="kernel"/> is null.</exception>
|
||||
/// <exception cref="KernelFunctionCanceledException">The <see cref="KernelFunction"/>'s invocation was canceled.</exception>
|
||||
public async Task<FunctionResult> InvokeAsync(
|
||||
Kernel kernel,
|
||||
KernelArguments? arguments = null,
|
||||
@@ -166,7 +165,7 @@ public abstract class KernelFunction
|
||||
|
||||
TagList tags = new() { { MeasurementFunctionTagName, this.Name } };
|
||||
long startingTimestamp = Stopwatch.GetTimestamp();
|
||||
FunctionResult? functionResult = null;
|
||||
FunctionResult functionResult = new(this, culture: kernel.Culture);
|
||||
try
|
||||
{
|
||||
// Quick check for cancellation after logging about function start but before doing any real work.
|
||||
@@ -177,52 +176,36 @@ public abstract class KernelFunction
|
||||
var invokingEventArgs = kernel.OnFunctionInvoking(this, arguments);
|
||||
#pragma warning restore CS0618 // Events are deprecated
|
||||
|
||||
// Invoke pre-invocation filter. If it requests cancellation, throw.
|
||||
var invokingContext = kernel.OnFunctionInvokingFilter(this, arguments);
|
||||
|
||||
if (invokingEventArgs?.Cancel is true)
|
||||
{
|
||||
throw new OperationCanceledException($"A {nameof(Kernel)}.{nameof(Kernel.FunctionInvoking)} event handler requested cancellation before function invocation.");
|
||||
}
|
||||
|
||||
if (invokingContext?.Cancel is true)
|
||||
var invocationContext = await kernel.OnFunctionInvocationAsync(this, arguments, functionResult, async (context) =>
|
||||
{
|
||||
throw new OperationCanceledException("A function filter requested cancellation before function invocation.");
|
||||
}
|
||||
// Invoking the function and updating context with result.
|
||||
context.Result = functionResult = await this.InvokeCoreAsync(kernel, context.Arguments, cancellationToken).ConfigureAwait(false);
|
||||
}).ConfigureAwait(false);
|
||||
|
||||
// Invoke the function.
|
||||
functionResult = await this.InvokeCoreAsync(kernel, arguments, cancellationToken).ConfigureAwait(false);
|
||||
// Apply any changes from the function filters context to final result.
|
||||
functionResult = invocationContext.Result;
|
||||
|
||||
// Invoke the post-invocation event handler. If it requests cancellation, throw.
|
||||
#pragma warning disable CS0618 // Events are deprecated
|
||||
var invokedEventArgs = kernel.OnFunctionInvoked(this, arguments, functionResult);
|
||||
#pragma warning restore CS0618 // Events are deprecated
|
||||
|
||||
// Invoke the post-invocation filter. If it requests cancellation, throw.
|
||||
var invokedContext = kernel.OnFunctionInvokedFilter(arguments, functionResult);
|
||||
|
||||
if (invokedEventArgs is not null)
|
||||
{
|
||||
// Apply any changes from the event handlers to final result.
|
||||
functionResult = new FunctionResult(this, invokedEventArgs.ResultValue, functionResult.Culture, invokedEventArgs.Metadata ?? functionResult.Metadata);
|
||||
}
|
||||
|
||||
if (invokedContext is not null)
|
||||
{
|
||||
// Apply any changes from the function filters to final result.
|
||||
functionResult = new FunctionResult(this, invokedContext.ResultValue, functionResult.Culture, invokedContext.Metadata ?? functionResult.Metadata);
|
||||
}
|
||||
|
||||
if (invokedEventArgs?.Cancel is true)
|
||||
{
|
||||
throw new OperationCanceledException($"A {nameof(Kernel)}.{nameof(Kernel.FunctionInvoked)} event handler requested cancellation after function invocation.");
|
||||
}
|
||||
|
||||
if (invokedContext?.Cancel is true)
|
||||
{
|
||||
throw new OperationCanceledException("A function filter requested cancellation after function invocation.");
|
||||
}
|
||||
|
||||
logger.LogFunctionInvokedSuccess(this.Name);
|
||||
logger.LogFunctionResultValue(functionResult);
|
||||
|
||||
@@ -251,7 +234,6 @@ public abstract class KernelFunction
|
||||
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
|
||||
/// <returns>The result of the function's execution, cast to <typeparamref name="TResult"/>.</returns>
|
||||
/// <exception cref="ArgumentNullException"><paramref name="kernel"/> is null.</exception>
|
||||
/// <exception cref="KernelFunctionCanceledException">The <see cref="KernelFunction"/>'s invocation was canceled.</exception>
|
||||
/// <exception cref="InvalidCastException">The function's result could not be cast to <typeparamref name="TResult"/>.</exception>
|
||||
public async Task<TResult?> InvokeAsync<TResult>(
|
||||
Kernel kernel,
|
||||
@@ -308,6 +290,7 @@ public abstract class KernelFunction
|
||||
|
||||
TagList tags = new() { { MeasurementFunctionTagName, this.Name } };
|
||||
long startingTimestamp = Stopwatch.GetTimestamp();
|
||||
|
||||
try
|
||||
{
|
||||
IAsyncEnumerator<TResult> enumerator;
|
||||
@@ -321,21 +304,27 @@ public abstract class KernelFunction
|
||||
var invokingEventArgs = kernel.OnFunctionInvoking(this, arguments);
|
||||
#pragma warning restore CS0618 // Events are deprecated
|
||||
|
||||
// Invoke pre-invocation filter. If it requests cancellation, throw.
|
||||
var invokingContext = kernel.OnFunctionInvokingFilter(this, arguments);
|
||||
|
||||
if (invokingEventArgs?.Cancel is true)
|
||||
{
|
||||
throw new OperationCanceledException($"A {nameof(Kernel)}.{nameof(Kernel.FunctionInvoking)} event handler requested cancellation before function invocation.");
|
||||
}
|
||||
|
||||
if (invokingContext?.Cancel is true)
|
||||
{
|
||||
throw new OperationCanceledException("A function filter requested cancellation before function invocation.");
|
||||
}
|
||||
FunctionResult functionResult = new(this, culture: kernel.Culture);
|
||||
|
||||
// Invoke the function and get its streaming enumerator.
|
||||
enumerator = this.InvokeStreamingCoreAsync<TResult>(kernel, arguments, cancellationToken).GetAsyncEnumerator(cancellationToken);
|
||||
var invocationContext = await kernel.OnFunctionInvocationAsync(this, arguments, functionResult, (context) =>
|
||||
{
|
||||
// Invoke the function and get its streaming enumerable.
|
||||
var enumerable = this.InvokeStreamingCoreAsync<TResult>(kernel, context.Arguments, cancellationToken);
|
||||
|
||||
// Update context with enumerable as result value.
|
||||
context.Result = new FunctionResult(this, enumerable, kernel.Culture);
|
||||
|
||||
return Task.CompletedTask;
|
||||
}).ConfigureAwait(false);
|
||||
|
||||
// Apply changes from the function filters to final result.
|
||||
var enumerable = invocationContext.Result.GetValue<IAsyncEnumerable<TResult>>() ?? AsyncEnumerable.Empty<TResult>();
|
||||
enumerator = enumerable.GetAsyncEnumerator(cancellationToken);
|
||||
|
||||
// yielding within a try/catch isn't currently supported, so we break out of the try block
|
||||
// in order to then wrap the actual MoveNextAsync in its own try/catch and allow the yielding
|
||||
@@ -370,8 +359,6 @@ public abstract class KernelFunction
|
||||
yield return enumerator.Current;
|
||||
}
|
||||
}
|
||||
|
||||
// The FunctionInvoked hook and filter are not used when streaming.
|
||||
}
|
||||
finally
|
||||
{
|
||||
|
||||
@@ -9,7 +9,7 @@ namespace Microsoft.SemanticKernel;
|
||||
/// <summary>
|
||||
/// Provides an <see cref="OperationCanceledException"/>-derived exception type
|
||||
/// that's thrown from a <see cref="KernelFunction"/> invocation when a <see cref="Kernel"/>
|
||||
/// event handler (e.g. <see cref="Kernel.FunctionInvoked"/>) requests cancellation.
|
||||
/// function filter (e.g. <see cref="Kernel.FunctionInvocationFilters"/>) requests cancellation.
|
||||
/// </summary>
|
||||
public sealed class KernelFunctionCanceledException : OperationCanceledException
|
||||
{
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.ComponentModel;
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
using System.Globalization;
|
||||
using System.Linq;
|
||||
@@ -33,9 +34,11 @@ public sealed class Kernel
|
||||
/// <summary>The collection of plugins, initialized via the constructor or lazily-initialized on first access via <see cref="Plugins"/>.</summary>
|
||||
private KernelPluginCollection? _plugins;
|
||||
/// <summary>The collection of function filters, initialized via the constructor or lazily-initialized on first access via <see cref="Plugins"/>.</summary>
|
||||
private NonNullCollection<IFunctionFilter>? _functionFilters;
|
||||
private NonNullCollection<IFunctionInvocationFilter>? _functionInvocationFilters;
|
||||
/// <summary>The collection of prompt filters, initialized via the constructor or lazily-initialized on first access via <see cref="Plugins"/>.</summary>
|
||||
private NonNullCollection<IPromptFilter>? _promptFilters;
|
||||
private NonNullCollection<IPromptRenderFilter>? _promptRenderFilters;
|
||||
/// <summary>The collection of automatic function invocation filters, initialized via the constructor or lazily-initialized on first access via <see cref="Plugins"/>.</summary>
|
||||
private NonNullCollection<IAutoFunctionInvocationFilter>? _autoFunctionInvocationFilters;
|
||||
|
||||
/// <summary>
|
||||
/// Initializes a new instance of <see cref="Kernel"/>.
|
||||
@@ -62,32 +65,18 @@ public sealed class Kernel
|
||||
if (this._plugins is null)
|
||||
{
|
||||
// Otherwise, enumerate any plugins that may have been registered directly.
|
||||
IEnumerable<KernelPlugin> e = this.Services.GetServices<KernelPlugin>();
|
||||
IEnumerable<KernelPlugin> registeredPlugins = this.Services.GetServices<KernelPlugin>();
|
||||
|
||||
// It'll be common not to have any plugins directly registered as a service.
|
||||
// If we can efficiently tell there aren't any, avoid proactively allocating
|
||||
// the plugins collection.
|
||||
if (e is not ICollection<KernelPlugin> c || c.Count != 0)
|
||||
if (IsNotEmpty(registeredPlugins))
|
||||
{
|
||||
this._plugins = new(e);
|
||||
this._plugins = new(registeredPlugins);
|
||||
}
|
||||
}
|
||||
|
||||
// Enumerate any function filters that may have been registered.
|
||||
IEnumerable<IFunctionFilter> functionFilters = this.Services.GetServices<IFunctionFilter>();
|
||||
|
||||
if (functionFilters is not ICollection<IFunctionFilter> functionFilterCollection || functionFilterCollection.Count != 0)
|
||||
{
|
||||
this._functionFilters = new(functionFilters);
|
||||
}
|
||||
|
||||
// Enumerate any prompt filters that may have been registered.
|
||||
IEnumerable<IPromptFilter> promptFilters = this.Services.GetServices<IPromptFilter>();
|
||||
|
||||
if (promptFilters is not ICollection<IPromptFilter> promptFilterCollection || promptFilterCollection.Count != 0)
|
||||
{
|
||||
this._promptFilters = new(promptFilters);
|
||||
}
|
||||
this.AddFilters();
|
||||
}
|
||||
|
||||
/// <summary>Creates a builder for constructing <see cref="Kernel"/> instances.</summary>
|
||||
@@ -141,19 +130,28 @@ public sealed class Kernel
|
||||
/// Gets the collection of function filters available through the kernel.
|
||||
/// </summary>
|
||||
[Experimental("SKEXP0001")]
|
||||
public IList<IFunctionFilter> FunctionFilters =>
|
||||
this._functionFilters ??
|
||||
Interlocked.CompareExchange(ref this._functionFilters, [], null) ??
|
||||
this._functionFilters;
|
||||
public IList<IFunctionInvocationFilter> FunctionInvocationFilters =>
|
||||
this._functionInvocationFilters ??
|
||||
Interlocked.CompareExchange(ref this._functionInvocationFilters, [], null) ??
|
||||
this._functionInvocationFilters;
|
||||
|
||||
/// <summary>
|
||||
/// Gets the collection of function filters available through the kernel.
|
||||
/// </summary>
|
||||
[Experimental("SKEXP0001")]
|
||||
public IList<IPromptFilter> PromptFilters =>
|
||||
this._promptFilters ??
|
||||
Interlocked.CompareExchange(ref this._promptFilters, [], null) ??
|
||||
this._promptFilters;
|
||||
public IList<IPromptRenderFilter> PromptRenderFilters =>
|
||||
this._promptRenderFilters ??
|
||||
Interlocked.CompareExchange(ref this._promptRenderFilters, [], null) ??
|
||||
this._promptRenderFilters;
|
||||
|
||||
/// <summary>
|
||||
/// Gets the collection of auto function invocation filters available through the kernel.
|
||||
/// </summary>
|
||||
[Experimental("SKEXP0001")]
|
||||
public IList<IAutoFunctionInvocationFilter> AutoFunctionInvocationFilters =>
|
||||
this._autoFunctionInvocationFilters ??
|
||||
Interlocked.CompareExchange(ref this._autoFunctionInvocationFilters, [], null) ??
|
||||
this._autoFunctionInvocationFilters;
|
||||
|
||||
/// <summary>
|
||||
/// Gets the service provider used to query for services available through the kernel.
|
||||
@@ -279,78 +277,108 @@ public sealed class Kernel
|
||||
|
||||
#endregion
|
||||
|
||||
#region Internal Filtering
|
||||
#region Filters
|
||||
|
||||
private void AddFilters()
|
||||
{
|
||||
// Enumerate any function filters that may have been registered.
|
||||
IEnumerable<IFunctionInvocationFilter> functionInvocationFilters = this.Services.GetServices<IFunctionInvocationFilter>();
|
||||
|
||||
if (IsNotEmpty(functionInvocationFilters))
|
||||
{
|
||||
this._functionInvocationFilters = new(functionInvocationFilters);
|
||||
}
|
||||
|
||||
// Enumerate any prompt filters that may have been registered.
|
||||
IEnumerable<IPromptRenderFilter> promptRenderFilters = this.Services.GetServices<IPromptRenderFilter>();
|
||||
|
||||
if (IsNotEmpty(promptRenderFilters))
|
||||
{
|
||||
this._promptRenderFilters = new(promptRenderFilters);
|
||||
}
|
||||
|
||||
// Enumerate any automatic function invocation filters that may have been registered.
|
||||
IEnumerable<IAutoFunctionInvocationFilter> autoFunctionInvocationFilters = this.Services.GetServices<IAutoFunctionInvocationFilter>();
|
||||
|
||||
if (IsNotEmpty(autoFunctionInvocationFilters))
|
||||
{
|
||||
this._autoFunctionInvocationFilters = new(autoFunctionInvocationFilters);
|
||||
}
|
||||
}
|
||||
|
||||
[Experimental("SKEXP0001")]
|
||||
internal FunctionInvokingContext? OnFunctionInvokingFilter(KernelFunction function, KernelArguments arguments)
|
||||
internal async Task<FunctionInvocationContext> OnFunctionInvocationAsync(
|
||||
KernelFunction function,
|
||||
KernelArguments arguments,
|
||||
FunctionResult functionResult,
|
||||
Func<FunctionInvocationContext, Task> functionCallback)
|
||||
{
|
||||
FunctionInvokingContext? context = null;
|
||||
FunctionInvocationContext context = new(this, function, arguments, functionResult);
|
||||
|
||||
if (this._functionFilters is { Count: > 0 })
|
||||
{
|
||||
context = new(function, arguments);
|
||||
|
||||
for (int i = 0; i < this._functionFilters.Count; i++)
|
||||
{
|
||||
this._functionFilters[i].OnFunctionInvoking(context);
|
||||
}
|
||||
}
|
||||
await InvokeFilterOrFunctionAsync(this._functionInvocationFilters, functionCallback, context).ConfigureAwait(false);
|
||||
|
||||
return context;
|
||||
}
|
||||
|
||||
[Experimental("SKEXP0001")]
|
||||
internal FunctionInvokedContext? OnFunctionInvokedFilter(KernelArguments arguments, FunctionResult result)
|
||||
/// <summary>
|
||||
/// This method will execute filters and kernel function recursively.
|
||||
/// If there are no registered filters, just kernel function will be executed.
|
||||
/// If there are registered filters, filter on <paramref name="index"/> position will be executed.
|
||||
/// Second parameter of filter is callback. It can be either filter on <paramref name="index"/> + 1 position or kernel function if there are no remaining filters to execute.
|
||||
/// Kernel function will be always executed as last step after all filters.
|
||||
/// </summary>
|
||||
private static async Task InvokeFilterOrFunctionAsync(
|
||||
NonNullCollection<IFunctionInvocationFilter>? functionFilters,
|
||||
Func<FunctionInvocationContext, Task> functionCallback,
|
||||
FunctionInvocationContext context,
|
||||
int index = 0)
|
||||
{
|
||||
FunctionInvokedContext? context = null;
|
||||
|
||||
if (this._functionFilters is { Count: > 0 })
|
||||
if (functionFilters is { Count: > 0 } && index < functionFilters.Count)
|
||||
{
|
||||
context = new(arguments, result);
|
||||
|
||||
for (int i = 0; i < this._functionFilters.Count; i++)
|
||||
{
|
||||
this._functionFilters[i].OnFunctionInvoked(context);
|
||||
}
|
||||
await functionFilters[index].OnFunctionInvocationAsync(context,
|
||||
(context) => InvokeFilterOrFunctionAsync(functionFilters, functionCallback, context, index + 1)).ConfigureAwait(false);
|
||||
}
|
||||
else
|
||||
{
|
||||
await functionCallback(context).ConfigureAwait(false);
|
||||
}
|
||||
}
|
||||
|
||||
[Experimental("SKEXP0001")]
|
||||
internal async Task<PromptRenderContext> OnPromptRenderAsync(
|
||||
KernelFunction function,
|
||||
KernelArguments arguments,
|
||||
Func<PromptRenderContext, Task> renderCallback)
|
||||
{
|
||||
PromptRenderContext context = new(this, function, arguments);
|
||||
|
||||
await InvokeFilterOrPromptRenderAsync(this._promptRenderFilters, renderCallback, context).ConfigureAwait(false);
|
||||
|
||||
return context;
|
||||
}
|
||||
|
||||
[Experimental("SKEXP0001")]
|
||||
internal PromptRenderingContext? OnPromptRenderingFilter(KernelFunction function, KernelArguments arguments)
|
||||
/// <summary>
|
||||
/// This method will execute prompt filters and prompt rendering recursively.
|
||||
/// If there are no registered filters, just prompt rendering will be executed.
|
||||
/// If there are registered filters, filter on <paramref name="index"/> position will be executed.
|
||||
/// Second parameter of filter is callback. It can be either filter on <paramref name="index"/> + 1 position or prompt rendering if there are no remaining filters to execute.
|
||||
/// Prompt rendering will be always executed as last step after all filters.
|
||||
/// </summary>
|
||||
private static async Task InvokeFilterOrPromptRenderAsync(
|
||||
NonNullCollection<IPromptRenderFilter>? promptFilters,
|
||||
Func<PromptRenderContext, Task> renderCallback,
|
||||
PromptRenderContext context,
|
||||
int index = 0)
|
||||
{
|
||||
PromptRenderingContext? context = null;
|
||||
|
||||
if (this._promptFilters is { Count: > 0 })
|
||||
if (promptFilters is { Count: > 0 } && index < promptFilters.Count)
|
||||
{
|
||||
context = new(function, arguments);
|
||||
|
||||
for (int i = 0; i < this._promptFilters.Count; i++)
|
||||
{
|
||||
this._promptFilters[i].OnPromptRendering(context);
|
||||
}
|
||||
await promptFilters[index].OnPromptRenderAsync(context,
|
||||
(context) => InvokeFilterOrPromptRenderAsync(promptFilters, renderCallback, context, index + 1)).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
return context;
|
||||
}
|
||||
|
||||
[Experimental("SKEXP0001")]
|
||||
internal PromptRenderedContext? OnPromptRenderedFilter(KernelFunction function, KernelArguments arguments, string renderedPrompt)
|
||||
{
|
||||
PromptRenderedContext? context = null;
|
||||
|
||||
if (this._promptFilters is { Count: > 0 })
|
||||
else
|
||||
{
|
||||
context = new(function, arguments, renderedPrompt);
|
||||
|
||||
for (int i = 0; i < this._promptFilters.Count; i++)
|
||||
{
|
||||
this._promptFilters[i].OnPromptRendered(context);
|
||||
}
|
||||
await renderCallback(context).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
return context;
|
||||
}
|
||||
|
||||
#endregion
|
||||
@@ -561,29 +589,40 @@ public sealed class Kernel
|
||||
}
|
||||
#endregion
|
||||
|
||||
#region Private
|
||||
|
||||
private static bool IsNotEmpty<T>(IEnumerable<T> enumerable) =>
|
||||
enumerable is not ICollection<T> collection || collection.Count != 0;
|
||||
|
||||
#endregion
|
||||
|
||||
#region Obsolete
|
||||
|
||||
/// <summary>
|
||||
/// Provides an event that's raised prior to a function's invocation.
|
||||
/// </summary>
|
||||
[EditorBrowsable(EditorBrowsableState.Never)]
|
||||
[Obsolete("Events are deprecated in favor of filters. Example in dotnet/samples/KernelSyntaxExamples/Getting_Started/Step7_Observability.cs of Semantic Kernel repository.")]
|
||||
public event EventHandler<FunctionInvokingEventArgs>? FunctionInvoking;
|
||||
|
||||
/// <summary>
|
||||
/// Provides an event that's raised after a function's invocation.
|
||||
/// </summary>
|
||||
[EditorBrowsable(EditorBrowsableState.Never)]
|
||||
[Obsolete("Events are deprecated in favor of filters. Example in dotnet/samples/KernelSyntaxExamples/Getting_Started/Step7_Observability.cs of Semantic Kernel repository.")]
|
||||
public event EventHandler<FunctionInvokedEventArgs>? FunctionInvoked;
|
||||
|
||||
/// <summary>
|
||||
/// Provides an event that's raised prior to a prompt being rendered.
|
||||
/// </summary>
|
||||
[EditorBrowsable(EditorBrowsableState.Never)]
|
||||
[Obsolete("Events are deprecated in favor of filters. Example in dotnet/samples/KernelSyntaxExamples/Getting_Started/Step7_Observability.cs of Semantic Kernel repository.")]
|
||||
public event EventHandler<PromptRenderingEventArgs>? PromptRendering;
|
||||
|
||||
/// <summary>
|
||||
/// Provides an event that's raised after a prompt is rendered.
|
||||
/// </summary>
|
||||
[EditorBrowsable(EditorBrowsableState.Never)]
|
||||
[Obsolete("Events are deprecated in favor of filters. Example in dotnet/samples/KernelSyntaxExamples/Getting_Started/Step7_Observability.cs of Semantic Kernel repository.")]
|
||||
public event EventHandler<PromptRenderedEventArgs>? PromptRendered;
|
||||
|
||||
|
||||
@@ -149,8 +149,6 @@ internal sealed class KernelFunctionFromMethod : KernelFunction
|
||||
}
|
||||
|
||||
throw new NotSupportedException($"Streaming function {this.Name} does not support type {typeof(TResult)}");
|
||||
|
||||
// We don't invoke the hook here as the InvokeCoreAsync will do that for us
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
|
||||
@@ -132,11 +132,6 @@ internal sealed class KernelFunctionFromPrompt : KernelFunction
|
||||
}
|
||||
#pragma warning restore CS0612 // Events are deprecated
|
||||
|
||||
if (result.RenderedContext?.Cancel is true)
|
||||
{
|
||||
throw new OperationCanceledException("A prompt filter requested cancellation after prompt rendering.");
|
||||
}
|
||||
|
||||
if (result.AIService is IChatCompletionService chatCompletion)
|
||||
{
|
||||
var chatContent = await chatCompletion.GetChatMessageContentAsync(result.RenderedPrompt, result.ExecutionSettings, kernel, cancellationToken).ConfigureAwait(false);
|
||||
@@ -172,11 +167,6 @@ internal sealed class KernelFunctionFromPrompt : KernelFunction
|
||||
}
|
||||
#pragma warning restore CS0612 // Events are deprecated
|
||||
|
||||
if (result.RenderedContext?.Cancel is true)
|
||||
{
|
||||
yield break;
|
||||
}
|
||||
|
||||
IAsyncEnumerable<StreamingKernelContent>? asyncReference = null;
|
||||
|
||||
if (result.AIService is IChatCompletionService chatCompletion)
|
||||
@@ -318,7 +308,9 @@ internal sealed class KernelFunctionFromPrompt : KernelFunction
|
||||
private async Task<PromptRenderingResult> RenderPromptAsync(Kernel kernel, KernelArguments arguments, CancellationToken cancellationToken)
|
||||
{
|
||||
var serviceSelector = kernel.ServiceSelector;
|
||||
|
||||
IAIService? aiService;
|
||||
string renderedPrompt = string.Empty;
|
||||
|
||||
// Try to use IChatCompletionService.
|
||||
if (serviceSelector.TrySelectAIService<IChatCompletionService>(
|
||||
@@ -340,13 +332,27 @@ internal sealed class KernelFunctionFromPrompt : KernelFunction
|
||||
kernel.OnPromptRendering(this, arguments);
|
||||
#pragma warning restore CS0618 // Events are deprecated
|
||||
|
||||
kernel.OnPromptRenderingFilter(this, arguments);
|
||||
|
||||
var renderedPrompt = await this._promptTemplate.RenderAsync(kernel, arguments, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
if (this._logger.IsEnabled(LogLevel.Trace))
|
||||
var renderingContext = await kernel.OnPromptRenderAsync(this, arguments, async (context) =>
|
||||
{
|
||||
this._logger.LogTrace("Rendered prompt: {Prompt}", renderedPrompt);
|
||||
renderedPrompt = await this._promptTemplate.RenderAsync(kernel, context.Arguments, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
if (this._logger.IsEnabled(LogLevel.Trace))
|
||||
{
|
||||
this._logger.LogTrace("Rendered prompt: {Prompt}", renderedPrompt);
|
||||
}
|
||||
|
||||
context.RenderedPrompt = renderedPrompt;
|
||||
}).ConfigureAwait(false);
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(renderingContext.RenderedPrompt) &&
|
||||
!string.Equals(renderingContext.RenderedPrompt, renderedPrompt, StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
renderedPrompt = renderingContext.RenderedPrompt!;
|
||||
|
||||
if (this._logger.IsEnabled(LogLevel.Trace))
|
||||
{
|
||||
this._logger.LogTrace("Rendered prompt changed by prompt filter: {Prompt}", renderingContext.RenderedPrompt);
|
||||
}
|
||||
}
|
||||
|
||||
#pragma warning disable CS0618 // Events are deprecated
|
||||
@@ -365,25 +371,10 @@ internal sealed class KernelFunctionFromPrompt : KernelFunction
|
||||
}
|
||||
#pragma warning restore CS0618 // Events are deprecated
|
||||
|
||||
var renderedContext = kernel.OnPromptRenderedFilter(this, arguments, renderedPrompt);
|
||||
|
||||
if (renderedContext is not null &&
|
||||
!renderedContext.Cancel &&
|
||||
renderedContext.RenderedPrompt != renderedPrompt)
|
||||
{
|
||||
renderedPrompt = renderedContext.RenderedPrompt;
|
||||
|
||||
if (this._logger.IsEnabled(LogLevel.Trace))
|
||||
{
|
||||
this._logger.LogTrace("Rendered prompt changed by prompt filter: {Prompt}", renderedContext.RenderedPrompt);
|
||||
}
|
||||
}
|
||||
|
||||
return new(aiService, renderedPrompt)
|
||||
{
|
||||
ExecutionSettings = executionSettings,
|
||||
RenderedEventArgs = renderedEventArgs,
|
||||
RenderedContext = renderedContext
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -19,8 +19,6 @@ internal sealed class PromptRenderingResult
|
||||
public PromptRenderedEventArgs? RenderedEventArgs { get; set; }
|
||||
#pragma warning restore CS0618 // Events are deprecated
|
||||
|
||||
public PromptRenderedContext? RenderedContext { get; set; }
|
||||
|
||||
public PromptRenderingResult(IAIService aiService, string renderedPrompt)
|
||||
{
|
||||
this.AIService = aiService;
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.SemanticKernel;
|
||||
using Microsoft.SemanticKernel.TextGeneration;
|
||||
using Moq;
|
||||
|
||||
namespace SemanticKernel.UnitTests.Filters;
|
||||
|
||||
public abstract class FilterBaseTest
|
||||
{
|
||||
protected Kernel GetKernelWithFilters(
|
||||
Func<FunctionInvocationContext, Func<FunctionInvocationContext, Task>, Task>? onFunctionInvocation = null,
|
||||
Func<PromptRenderContext, Func<PromptRenderContext, Task>, Task>? onPromptRender = null,
|
||||
ITextGenerationService? textGenerationService = null)
|
||||
{
|
||||
var builder = Kernel.CreateBuilder();
|
||||
|
||||
// Add function filter before kernel construction
|
||||
if (onFunctionInvocation is not null)
|
||||
{
|
||||
var functionFilter = new FakeFunctionFilter(onFunctionInvocation);
|
||||
builder.Services.AddSingleton<IFunctionInvocationFilter>(functionFilter);
|
||||
}
|
||||
|
||||
if (textGenerationService is not null)
|
||||
{
|
||||
builder.Services.AddSingleton<ITextGenerationService>(textGenerationService);
|
||||
}
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
if (onPromptRender is not null)
|
||||
{
|
||||
// Add prompt filter after kernel construction
|
||||
kernel.PromptRenderFilters.Add(new FakePromptFilter(onPromptRender));
|
||||
}
|
||||
|
||||
return kernel;
|
||||
}
|
||||
|
||||
protected Mock<ITextGenerationService> GetMockTextGeneration(string? textResult = null, IReadOnlyDictionary<string, object?>? metadata = null)
|
||||
{
|
||||
var mockTextGeneration = new Mock<ITextGenerationService>();
|
||||
mockTextGeneration
|
||||
.Setup(m => m.GetTextContentsAsync(It.IsAny<string>(), It.IsAny<PromptExecutionSettings>(), It.IsAny<Kernel>(), It.IsAny<CancellationToken>()))
|
||||
.ReturnsAsync([new(textResult ?? "result text", metadata: metadata)]);
|
||||
|
||||
mockTextGeneration
|
||||
.Setup(s => s.GetStreamingTextContentsAsync(It.IsAny<string>(), It.IsAny<PromptExecutionSettings>(), It.IsAny<Kernel>(), It.IsAny<CancellationToken>()))
|
||||
.Returns(new List<StreamingTextContent>() { new(textResult ?? "result chunk", metadata: metadata) }.ToAsyncEnumerable());
|
||||
|
||||
return mockTextGeneration;
|
||||
}
|
||||
|
||||
protected sealed class FakeFunctionFilter(
|
||||
Func<FunctionInvocationContext, Func<FunctionInvocationContext, Task>, Task>? onFunctionInvocation) : IFunctionInvocationFilter
|
||||
{
|
||||
private readonly Func<FunctionInvocationContext, Func<FunctionInvocationContext, Task>, Task>? _onFunctionInvocation = onFunctionInvocation;
|
||||
|
||||
public Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next) =>
|
||||
this._onFunctionInvocation?.Invoke(context, next) ?? Task.CompletedTask;
|
||||
}
|
||||
|
||||
protected sealed class FakePromptFilter(
|
||||
Func<PromptRenderContext, Func<PromptRenderContext, Task>, Task>? onPromptRender) : IPromptRenderFilter
|
||||
{
|
||||
private readonly Func<PromptRenderContext, Func<PromptRenderContext, Task>, Task>? _onPromptRender = onPromptRender;
|
||||
|
||||
public Task OnPromptRenderAsync(PromptRenderContext context, Func<PromptRenderContext, Task> next) =>
|
||||
this._onPromptRender?.Invoke(context, next) ?? Task.CompletedTask;
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,657 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.SemanticKernel;
|
||||
using Microsoft.SemanticKernel.TextGeneration;
|
||||
using Moq;
|
||||
using Xunit;
|
||||
|
||||
namespace SemanticKernel.UnitTests.Filters;
|
||||
|
||||
public class KernelFilterTests
|
||||
{
|
||||
[Fact]
|
||||
public async Task PreInvocationFunctionFilterIsTriggeredAsync()
|
||||
{
|
||||
// Arrange
|
||||
var functionInvocations = 0;
|
||||
var filterInvocations = 0;
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => functionInvocations++);
|
||||
|
||||
var kernel = this.GetKernelWithFilters(onFunctionInvoking: (context) =>
|
||||
{
|
||||
filterInvocations++;
|
||||
});
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, functionInvocations);
|
||||
Assert.Equal(1, filterInvocations);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PreInvocationFunctionFilterChangesArgumentAsync()
|
||||
{
|
||||
// Arrange
|
||||
const string OriginalInput = "OriginalInput";
|
||||
const string NewInput = "NewInput";
|
||||
|
||||
var kernel = this.GetKernelWithFilters(onFunctionInvoking: (context) =>
|
||||
{
|
||||
context.Arguments["originalInput"] = NewInput;
|
||||
});
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromMethod((string originalInput) => originalInput);
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function, new() { ["originalInput"] = OriginalInput });
|
||||
|
||||
// Assert
|
||||
Assert.Equal(NewInput, result.GetValue<string>());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PreInvocationFunctionFilterCancellationWorksCorrectlyAsync()
|
||||
{
|
||||
// Arrange
|
||||
var functionInvocations = 0;
|
||||
var preFilterInvocations = 0;
|
||||
var postFilterInvocations = 0;
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => functionInvocations++);
|
||||
|
||||
var kernel = this.GetKernelWithFilters(
|
||||
onFunctionInvoking: (context) =>
|
||||
{
|
||||
preFilterInvocations++;
|
||||
context.Cancel = true;
|
||||
},
|
||||
onFunctionInvoked: (context) =>
|
||||
{
|
||||
postFilterInvocations++;
|
||||
});
|
||||
|
||||
// Act
|
||||
var exception = await Assert.ThrowsAsync<KernelFunctionCanceledException>(() => kernel.InvokeAsync(function));
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, preFilterInvocations);
|
||||
Assert.Equal(0, functionInvocations);
|
||||
Assert.Equal(0, postFilterInvocations);
|
||||
Assert.Same(function, exception.Function);
|
||||
Assert.Null(exception.FunctionResult);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PreInvocationFunctionFilterCancellationWorksCorrectlyOnStreamingAsync()
|
||||
{
|
||||
// Arrange
|
||||
var functionInvocations = 0;
|
||||
var filterInvocations = 0;
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => functionInvocations++);
|
||||
|
||||
var kernel = this.GetKernelWithFilters(onFunctionInvoking: (context) =>
|
||||
{
|
||||
filterInvocations++;
|
||||
context.Cancel = true;
|
||||
});
|
||||
|
||||
// Act
|
||||
IAsyncEnumerable<StreamingKernelContent> enumerable = function.InvokeStreamingAsync<StreamingKernelContent>(kernel);
|
||||
IAsyncEnumerator<StreamingKernelContent> enumerator = enumerable.GetAsyncEnumerator();
|
||||
|
||||
Assert.Equal(0, filterInvocations);
|
||||
|
||||
var exception = await Assert.ThrowsAsync<KernelFunctionCanceledException>(async () => await enumerator.MoveNextAsync());
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, filterInvocations);
|
||||
Assert.Equal(0, functionInvocations);
|
||||
Assert.Same(function, exception.Function);
|
||||
Assert.Same(kernel, exception.Kernel);
|
||||
Assert.Null(exception.FunctionResult);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PostInvocationFunctionFilterIsTriggeredAsync()
|
||||
{
|
||||
// Arrange
|
||||
var functionInvocations = 0;
|
||||
var filterInvocations = 0;
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => functionInvocations++);
|
||||
|
||||
var kernel = this.GetKernelWithFilters(onFunctionInvoked: (context) =>
|
||||
{
|
||||
filterInvocations++;
|
||||
});
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, functionInvocations);
|
||||
Assert.Equal(1, filterInvocations);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PostInvocationFunctionFilterReturnsModifiedResultAsync()
|
||||
{
|
||||
// Arrange
|
||||
const int OriginalResult = 42;
|
||||
const int NewResult = 84;
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => OriginalResult);
|
||||
|
||||
var kernel = this.GetKernelWithFilters(onFunctionInvoked: (context) =>
|
||||
{
|
||||
context.SetResultValue(NewResult);
|
||||
});
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal(NewResult, result.GetValue<int>());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PostInvocationFunctionFilterCancellationWorksCorrectlyAsync()
|
||||
{
|
||||
// Arrange
|
||||
const int Result = 42;
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => Result);
|
||||
var args = new KernelArguments() { { "a", "b" } };
|
||||
|
||||
var kernel = this.GetKernelWithFilters(onFunctionInvoked: (context) =>
|
||||
{
|
||||
context.Cancel = true;
|
||||
});
|
||||
|
||||
// Act
|
||||
var exception = await Assert.ThrowsAsync<KernelFunctionCanceledException>(() => kernel.InvokeAsync(function, args));
|
||||
|
||||
// Assert
|
||||
Assert.Same(kernel, exception.Kernel);
|
||||
Assert.Same(function, exception.Function);
|
||||
Assert.Same(args, exception.Arguments);
|
||||
Assert.NotNull(exception.FunctionResult);
|
||||
Assert.Equal(Result, exception.FunctionResult.GetValue<int>());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PostInvocationFunctionFilterCancellationWithModifiedResultAsync()
|
||||
{
|
||||
// Arrange
|
||||
const int OriginalResult = 42;
|
||||
const int NewResult = 84;
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => OriginalResult);
|
||||
var args = new KernelArguments() { { "a", "b" } };
|
||||
|
||||
var kernel = this.GetKernelWithFilters(onFunctionInvoked: (context) =>
|
||||
{
|
||||
context.SetResultValue(NewResult);
|
||||
context.Cancel = true;
|
||||
});
|
||||
|
||||
// Act
|
||||
var exception = await Assert.ThrowsAsync<KernelFunctionCanceledException>(() => kernel.InvokeAsync(function, args));
|
||||
|
||||
// Assert
|
||||
Assert.Same(kernel, exception.Kernel);
|
||||
Assert.Same(function, exception.Function);
|
||||
Assert.Same(args, exception.Arguments);
|
||||
Assert.NotNull(exception.FunctionResult);
|
||||
Assert.Equal(NewResult, exception.FunctionResult.GetValue<int>());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PostInvocationFunctionFilterIsNotTriggeredOnStreamingAsync()
|
||||
{
|
||||
// Arrange
|
||||
var functionInvocations = 0;
|
||||
var preFilterInvocations = 0;
|
||||
var postFilterInvocations = 0;
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => functionInvocations++);
|
||||
|
||||
var kernel = this.GetKernelWithFilters(
|
||||
onFunctionInvoking: (context) =>
|
||||
{
|
||||
preFilterInvocations++;
|
||||
},
|
||||
onFunctionInvoked: (context) =>
|
||||
{
|
||||
postFilterInvocations++;
|
||||
});
|
||||
|
||||
// Act
|
||||
await foreach (var chunk in kernel.InvokeStreamingAsync(function))
|
||||
{
|
||||
}
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, functionInvocations);
|
||||
Assert.Equal(1, preFilterInvocations);
|
||||
Assert.Equal(0, postFilterInvocations);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task FunctionFiltersWithPromptsWorkCorrectlyAsync()
|
||||
{
|
||||
// Arrange
|
||||
var preFilterInvocations = 0;
|
||||
var postFilterInvocations = 0;
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
|
||||
var kernel = this.GetKernelWithFilters(textGenerationService: mockTextGeneration.Object,
|
||||
onFunctionInvoking: (context) =>
|
||||
{
|
||||
preFilterInvocations++;
|
||||
},
|
||||
onFunctionInvoked: (context) =>
|
||||
{
|
||||
postFilterInvocations++;
|
||||
});
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Write a simple phrase about UnitTests");
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, preFilterInvocations);
|
||||
Assert.Equal(1, postFilterInvocations);
|
||||
mockTextGeneration.Verify(m => m.GetTextContentsAsync(It.IsAny<string>(), It.IsAny<PromptExecutionSettings>(), It.IsAny<Kernel>(), It.IsAny<CancellationToken>()), Times.Exactly(1));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PromptFiltersAreNotTriggeredForMethodsAsync()
|
||||
{
|
||||
// Arrange
|
||||
var functionInvocations = 0;
|
||||
var preFilterInvocations = 0;
|
||||
var postFilterInvocations = 0;
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => functionInvocations++);
|
||||
|
||||
var kernel = this.GetKernelWithFilters(
|
||||
onPromptRendering: (context) =>
|
||||
{
|
||||
preFilterInvocations++;
|
||||
},
|
||||
onPromptRendered: (context) =>
|
||||
{
|
||||
postFilterInvocations++;
|
||||
});
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, functionInvocations);
|
||||
Assert.Equal(0, preFilterInvocations);
|
||||
Assert.Equal(0, postFilterInvocations);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PromptFiltersAreTriggeredForPromptsAsync()
|
||||
{
|
||||
// Arrange
|
||||
var preFilterInvocations = 0;
|
||||
var postFilterInvocations = 0;
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Prompt");
|
||||
|
||||
var kernel = this.GetKernelWithFilters(textGenerationService: mockTextGeneration.Object,
|
||||
onPromptRendering: (context) =>
|
||||
{
|
||||
preFilterInvocations++;
|
||||
},
|
||||
onPromptRendered: (context) =>
|
||||
{
|
||||
postFilterInvocations++;
|
||||
});
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, preFilterInvocations);
|
||||
Assert.Equal(1, postFilterInvocations);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PromptFiltersAreTriggeredForPromptsStreamingAsync()
|
||||
{
|
||||
// Arrange
|
||||
var preFilterInvocations = 0;
|
||||
var postFilterInvocations = 0;
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Prompt");
|
||||
|
||||
var kernel = this.GetKernelWithFilters(textGenerationService: mockTextGeneration.Object,
|
||||
onPromptRendering: (context) =>
|
||||
{
|
||||
preFilterInvocations++;
|
||||
},
|
||||
onPromptRendered: (context) =>
|
||||
{
|
||||
postFilterInvocations++;
|
||||
});
|
||||
|
||||
// Act
|
||||
await foreach (var chunk in kernel.InvokeStreamingAsync(function))
|
||||
{
|
||||
}
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, preFilterInvocations);
|
||||
Assert.Equal(1, postFilterInvocations);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PostInvocationPromptFilterChangesRenderedPromptAsync()
|
||||
{
|
||||
// Arrange
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Prompt");
|
||||
var kernel = this.GetKernelWithFilters(textGenerationService: mockTextGeneration.Object,
|
||||
onPromptRendered: (context) =>
|
||||
{
|
||||
context.RenderedPrompt += " - updated from filter";
|
||||
});
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
mockTextGeneration.Verify(m => m.GetTextContentsAsync("Prompt - updated from filter", It.IsAny<PromptExecutionSettings>(), It.IsAny<Kernel>(), It.IsAny<CancellationToken>()), Times.Once());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PostInvocationPromptFilterCancellationWorksCorrectlyAsync()
|
||||
{
|
||||
// Arrange
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Prompt");
|
||||
var kernel = this.GetKernelWithFilters(textGenerationService: mockTextGeneration.Object,
|
||||
onPromptRendered: (context) =>
|
||||
{
|
||||
context.Cancel = true;
|
||||
});
|
||||
|
||||
// Act
|
||||
var exception = await Assert.ThrowsAsync<KernelFunctionCanceledException>(() => kernel.InvokeAsync(function));
|
||||
|
||||
// Assert
|
||||
Assert.Same(function, exception.Function);
|
||||
Assert.Same(kernel, exception.Kernel);
|
||||
Assert.Null(exception.FunctionResult);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task FunctionAndPromptFiltersAreExecutedInCorrectOrderAsync()
|
||||
{
|
||||
// Arrange
|
||||
var builder = Kernel.CreateBuilder();
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Prompt");
|
||||
|
||||
var executionOrder = new List<string>();
|
||||
|
||||
var functionFilter1 = new FakeFunctionFilter(
|
||||
(context) => executionOrder.Add("FunctionFilter1-Invoking"),
|
||||
(context) => executionOrder.Add("FunctionFilter1-Invoked"));
|
||||
|
||||
var functionFilter2 = new FakeFunctionFilter(
|
||||
(context) => executionOrder.Add("FunctionFilter2-Invoking"),
|
||||
(context) => executionOrder.Add("FunctionFilter2-Invoked"));
|
||||
|
||||
var promptFilter1 = new FakePromptFilter(
|
||||
(context) => executionOrder.Add("PromptFilter1-Rendering"),
|
||||
(context) => executionOrder.Add("PromptFilter1-Rendered"));
|
||||
|
||||
var promptFilter2 = new FakePromptFilter(
|
||||
(context) => executionOrder.Add("PromptFilter2-Rendering"),
|
||||
(context) => executionOrder.Add("PromptFilter2-Rendered"));
|
||||
|
||||
builder.Services.AddSingleton<IFunctionFilter>(functionFilter1);
|
||||
builder.Services.AddSingleton<IFunctionFilter>(functionFilter2);
|
||||
|
||||
builder.Services.AddSingleton<IPromptFilter>(promptFilter1);
|
||||
builder.Services.AddSingleton<IPromptFilter>(promptFilter2);
|
||||
|
||||
builder.Services.AddSingleton<ITextGenerationService>(mockTextGeneration.Object);
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal("FunctionFilter1-Invoking", executionOrder[0]);
|
||||
Assert.Equal("FunctionFilter2-Invoking", executionOrder[1]);
|
||||
Assert.Equal("PromptFilter1-Rendering", executionOrder[2]);
|
||||
Assert.Equal("PromptFilter2-Rendering", executionOrder[3]);
|
||||
Assert.Equal("PromptFilter1-Rendered", executionOrder[4]);
|
||||
Assert.Equal("PromptFilter2-Rendered", executionOrder[5]);
|
||||
Assert.Equal("FunctionFilter1-Invoked", executionOrder[6]);
|
||||
Assert.Equal("FunctionFilter2-Invoked", executionOrder[7]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task MultipleFunctionFiltersCancellationWorksCorrectlyAsync()
|
||||
{
|
||||
// Arrange
|
||||
var functionInvocations = 0;
|
||||
var filterInvocations = 0;
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => functionInvocations++);
|
||||
|
||||
var functionFilter1 = new FakeFunctionFilter(onFunctionInvoking: (context) =>
|
||||
{
|
||||
filterInvocations++;
|
||||
context.Cancel = true;
|
||||
});
|
||||
|
||||
var functionFilter2 = new FakeFunctionFilter(onFunctionInvoking: (context) =>
|
||||
{
|
||||
Assert.True(context.Cancel);
|
||||
|
||||
filterInvocations++;
|
||||
context.Cancel = false;
|
||||
});
|
||||
|
||||
var builder = Kernel.CreateBuilder();
|
||||
|
||||
builder.Services.AddSingleton<IFunctionFilter>(functionFilter1);
|
||||
builder.Services.AddSingleton<IFunctionFilter>(functionFilter2);
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, functionInvocations);
|
||||
Assert.Equal(2, filterInvocations);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task DifferentWaysOfAddingFunctionFiltersWorkCorrectlyAsync()
|
||||
{
|
||||
// Arrange
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => "Result");
|
||||
var executionOrder = new List<string>();
|
||||
|
||||
var functionFilter1 = new FakeFunctionFilter((context) => executionOrder.Add("FunctionFilter1-Invoking"));
|
||||
var functionFilter2 = new FakeFunctionFilter((context) => executionOrder.Add("FunctionFilter2-Invoking"));
|
||||
|
||||
var builder = Kernel.CreateBuilder();
|
||||
|
||||
// Act
|
||||
|
||||
// Case #1 - Add filter to services
|
||||
builder.Services.AddSingleton<IFunctionFilter>(functionFilter1);
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
// Case #2 - Add filter to kernel
|
||||
kernel.FunctionFilters.Add(functionFilter2);
|
||||
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal("FunctionFilter1-Invoking", executionOrder[0]);
|
||||
Assert.Equal("FunctionFilter2-Invoking", executionOrder[1]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task DifferentWaysOfAddingPromptFiltersWorkCorrectlyAsync()
|
||||
{
|
||||
// Arrange
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Prompt");
|
||||
var executionOrder = new List<string>();
|
||||
|
||||
var promptFilter1 = new FakePromptFilter((context) => executionOrder.Add("PromptFilter1-Rendering"));
|
||||
var promptFilter2 = new FakePromptFilter((context) => executionOrder.Add("PromptFilter2-Rendering"));
|
||||
|
||||
var builder = Kernel.CreateBuilder();
|
||||
builder.Services.AddSingleton<ITextGenerationService>(mockTextGeneration.Object);
|
||||
|
||||
// Act
|
||||
// Case #1 - Add filter to services
|
||||
builder.Services.AddSingleton<IPromptFilter>(promptFilter1);
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
// Case #2 - Add filter to kernel
|
||||
kernel.PromptFilters.Add(promptFilter2);
|
||||
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal("PromptFilter1-Rendering", executionOrder[0]);
|
||||
Assert.Equal("PromptFilter2-Rendering", executionOrder[1]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task InsertFilterInMiddleOfPipelineTriggersFiltersInCorrectOrderAsync()
|
||||
{
|
||||
// Arrange
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => "Result");
|
||||
var executionOrder = new List<string>();
|
||||
|
||||
var functionFilter1 = new FakeFunctionFilter(
|
||||
(context) => executionOrder.Add("FunctionFilter1-Invoking"),
|
||||
(context) => executionOrder.Add("FunctionFilter1-Invoked"));
|
||||
|
||||
var functionFilter2 = new FakeFunctionFilter(
|
||||
(context) => executionOrder.Add("FunctionFilter2-Invoking"),
|
||||
(context) => executionOrder.Add("FunctionFilter2-Invoked"));
|
||||
|
||||
var functionFilter3 = new FakeFunctionFilter(
|
||||
(context) => executionOrder.Add("FunctionFilter3-Invoking"),
|
||||
(context) => executionOrder.Add("FunctionFilter3-Invoked"));
|
||||
|
||||
var builder = Kernel.CreateBuilder();
|
||||
|
||||
builder.Services.AddSingleton<IFunctionFilter>(functionFilter1);
|
||||
builder.Services.AddSingleton<IFunctionFilter>(functionFilter2);
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
kernel.FunctionFilters.Insert(1, functionFilter3);
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal("FunctionFilter1-Invoking", executionOrder[0]);
|
||||
Assert.Equal("FunctionFilter3-Invoking", executionOrder[1]);
|
||||
Assert.Equal("FunctionFilter2-Invoking", executionOrder[2]);
|
||||
Assert.Equal("FunctionFilter1-Invoked", executionOrder[3]);
|
||||
Assert.Equal("FunctionFilter3-Invoked", executionOrder[4]);
|
||||
Assert.Equal("FunctionFilter2-Invoked", executionOrder[5]);
|
||||
}
|
||||
|
||||
private Kernel GetKernelWithFilters(
|
||||
Action<FunctionInvokingContext>? onFunctionInvoking = null,
|
||||
Action<FunctionInvokedContext>? onFunctionInvoked = null,
|
||||
Action<PromptRenderingContext>? onPromptRendering = null,
|
||||
Action<PromptRenderedContext>? onPromptRendered = null,
|
||||
ITextGenerationService? textGenerationService = null)
|
||||
{
|
||||
var builder = Kernel.CreateBuilder();
|
||||
var functionFilter = new FakeFunctionFilter(onFunctionInvoking, onFunctionInvoked);
|
||||
var promptFilter = new FakePromptFilter(onPromptRendering, onPromptRendered);
|
||||
|
||||
// Add function filter before kernel construction
|
||||
builder.Services.AddSingleton<IFunctionFilter>(functionFilter);
|
||||
|
||||
if (textGenerationService is not null)
|
||||
{
|
||||
builder.Services.AddSingleton<ITextGenerationService>(textGenerationService);
|
||||
}
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
// Add prompt filter after kernel construction
|
||||
kernel.PromptFilters.Add(promptFilter);
|
||||
|
||||
return kernel;
|
||||
}
|
||||
|
||||
private Mock<ITextGenerationService> GetMockTextGeneration()
|
||||
{
|
||||
var mockTextGeneration = new Mock<ITextGenerationService>();
|
||||
mockTextGeneration
|
||||
.Setup(m => m.GetTextContentsAsync(It.IsAny<string>(), It.IsAny<PromptExecutionSettings>(), It.IsAny<Kernel>(), It.IsAny<CancellationToken>()))
|
||||
.ReturnsAsync([new("result text")]);
|
||||
|
||||
mockTextGeneration
|
||||
.Setup(s => s.GetStreamingTextContentsAsync(It.IsAny<string>(), It.IsAny<PromptExecutionSettings>(), It.IsAny<Kernel>(), It.IsAny<CancellationToken>()))
|
||||
.Returns(new List<StreamingTextContent>() { new("result chunk") }.ToAsyncEnumerable());
|
||||
|
||||
return mockTextGeneration;
|
||||
}
|
||||
|
||||
private sealed class FakeFunctionFilter(
|
||||
Action<FunctionInvokingContext>? onFunctionInvoking = null,
|
||||
Action<FunctionInvokedContext>? onFunctionInvoked = null) : IFunctionFilter
|
||||
{
|
||||
private readonly Action<FunctionInvokingContext>? _onFunctionInvoking = onFunctionInvoking;
|
||||
private readonly Action<FunctionInvokedContext>? _onFunctionInvoked = onFunctionInvoked;
|
||||
|
||||
public void OnFunctionInvoked(FunctionInvokedContext context) =>
|
||||
this._onFunctionInvoked?.Invoke(context);
|
||||
|
||||
public void OnFunctionInvoking(FunctionInvokingContext context) =>
|
||||
this._onFunctionInvoking?.Invoke(context);
|
||||
}
|
||||
|
||||
private sealed class FakePromptFilter(
|
||||
Action<PromptRenderingContext>? onPromptRendering = null,
|
||||
Action<PromptRenderedContext>? onPromptRendered = null) : IPromptFilter
|
||||
{
|
||||
private readonly Action<PromptRenderingContext>? _onPromptRendering = onPromptRendering;
|
||||
private readonly Action<PromptRenderedContext>? _onPromptRendered = onPromptRendered;
|
||||
|
||||
public void OnPromptRendered(PromptRenderedContext context) =>
|
||||
this._onPromptRendered?.Invoke(context);
|
||||
|
||||
public void OnPromptRendering(PromptRenderingContext context) =>
|
||||
this._onPromptRendering?.Invoke(context);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,239 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System.Collections.Generic;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.SemanticKernel;
|
||||
using Microsoft.SemanticKernel.TextGeneration;
|
||||
using Moq;
|
||||
using Xunit;
|
||||
|
||||
namespace SemanticKernel.UnitTests.Filters;
|
||||
|
||||
public class PromptRenderFilterTests : FilterBaseTest
|
||||
{
|
||||
[Fact]
|
||||
public async Task PromptFiltersAreNotTriggeredForMethodsAsync()
|
||||
{
|
||||
// Arrange
|
||||
var functionInvocations = 0;
|
||||
var filterInvocations = 0;
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromMethod(() => functionInvocations++);
|
||||
|
||||
var kernel = this.GetKernelWithFilters(onPromptRender: async (context, next) =>
|
||||
{
|
||||
filterInvocations++;
|
||||
await next(context);
|
||||
filterInvocations++;
|
||||
});
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal(1, functionInvocations);
|
||||
Assert.Equal(0, filterInvocations);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(true)]
|
||||
[InlineData(false)]
|
||||
public async Task PromptFiltersAreTriggeredForPromptsAsync(bool isStreaming)
|
||||
{
|
||||
// Arrange
|
||||
Kernel? contextKernel = null;
|
||||
|
||||
var filterInvocations = 0;
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
|
||||
var arguments = new KernelArguments() { ["key1"] = "value1" };
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Prompt");
|
||||
|
||||
var kernel = this.GetKernelWithFilters(textGenerationService: mockTextGeneration.Object,
|
||||
onPromptRender: async (context, next) =>
|
||||
{
|
||||
Assert.Same(arguments, context.Arguments);
|
||||
Assert.Same(function, context.Function);
|
||||
|
||||
contextKernel = context.Kernel;
|
||||
|
||||
filterInvocations++;
|
||||
await next(context);
|
||||
filterInvocations++;
|
||||
|
||||
Assert.Equal("Prompt", context.RenderedPrompt);
|
||||
});
|
||||
|
||||
// Act
|
||||
if (isStreaming)
|
||||
{
|
||||
await foreach (var item in kernel.InvokeStreamingAsync(function, arguments))
|
||||
{ }
|
||||
}
|
||||
else
|
||||
{
|
||||
await kernel.InvokeAsync(function, arguments);
|
||||
}
|
||||
|
||||
// Assert
|
||||
Assert.Equal(2, filterInvocations);
|
||||
Assert.Same(contextKernel, kernel);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task DifferentWaysOfAddingPromptFiltersWorkCorrectlyAsync()
|
||||
{
|
||||
// Arrange
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Prompt");
|
||||
var executionOrder = new List<string>();
|
||||
|
||||
var promptFilter1 = new FakePromptFilter(onPromptRender: async (context, next) =>
|
||||
{
|
||||
executionOrder.Add("PromptFilter1-Rendering");
|
||||
await next(context);
|
||||
});
|
||||
|
||||
var promptFilter2 = new FakePromptFilter(onPromptRender: async (context, next) =>
|
||||
{
|
||||
executionOrder.Add("PromptFilter2-Rendering");
|
||||
await next(context);
|
||||
});
|
||||
|
||||
var builder = Kernel.CreateBuilder();
|
||||
builder.Services.AddSingleton<ITextGenerationService>(mockTextGeneration.Object);
|
||||
|
||||
// Act
|
||||
// Case #1 - Add filter to services
|
||||
builder.Services.AddSingleton<IPromptRenderFilter>(promptFilter1);
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
// Case #2 - Add filter to kernel
|
||||
kernel.PromptRenderFilters.Add(promptFilter2);
|
||||
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal("PromptFilter1-Rendering", executionOrder[0]);
|
||||
Assert.Equal("PromptFilter2-Rendering", executionOrder[1]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task MultipleFiltersAreExecutedInOrderAsync()
|
||||
{
|
||||
// Arrange
|
||||
var builder = Kernel.CreateBuilder();
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Prompt");
|
||||
|
||||
var executionOrder = new List<string>();
|
||||
|
||||
var promptFilter1 = new FakePromptFilter(onPromptRender: async (context, next) =>
|
||||
{
|
||||
executionOrder.Add("PromptFilter1-Rendering");
|
||||
await next(context);
|
||||
executionOrder.Add("PromptFilter1-Rendered");
|
||||
});
|
||||
|
||||
var promptFilter2 = new FakePromptFilter(onPromptRender: async (context, next) =>
|
||||
{
|
||||
executionOrder.Add("PromptFilter2-Rendering");
|
||||
await next(context);
|
||||
executionOrder.Add("PromptFilter2-Rendered");
|
||||
});
|
||||
|
||||
var promptFilter3 = new FakePromptFilter(onPromptRender: async (context, next) =>
|
||||
{
|
||||
executionOrder.Add("PromptFilter3-Rendering");
|
||||
await next(context);
|
||||
executionOrder.Add("PromptFilter3-Rendered");
|
||||
});
|
||||
|
||||
builder.Services.AddSingleton<IPromptRenderFilter>(promptFilter1);
|
||||
builder.Services.AddSingleton<IPromptRenderFilter>(promptFilter2);
|
||||
builder.Services.AddSingleton<IPromptRenderFilter>(promptFilter3);
|
||||
|
||||
builder.Services.AddSingleton<ITextGenerationService>(mockTextGeneration.Object);
|
||||
|
||||
var kernel = builder.Build();
|
||||
|
||||
// Act
|
||||
await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
Assert.Equal("PromptFilter1-Rendering", executionOrder[0]);
|
||||
Assert.Equal("PromptFilter2-Rendering", executionOrder[1]);
|
||||
Assert.Equal("PromptFilter3-Rendering", executionOrder[2]);
|
||||
Assert.Equal("PromptFilter3-Rendered", executionOrder[3]);
|
||||
Assert.Equal("PromptFilter2-Rendered", executionOrder[4]);
|
||||
Assert.Equal("PromptFilter1-Rendered", executionOrder[5]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PromptFilterCanOverrideArgumentsAsync()
|
||||
{
|
||||
// Arrange
|
||||
const string OriginalInput = "OriginalInput";
|
||||
const string NewInput = "NewInput";
|
||||
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
|
||||
var kernel = this.GetKernelWithFilters(textGenerationService: mockTextGeneration.Object,
|
||||
onPromptRender: async (context, next) =>
|
||||
{
|
||||
context.Arguments["originalInput"] = NewInput;
|
||||
await next(context);
|
||||
});
|
||||
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Prompt: {{$originalInput}}");
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function, new() { ["originalInput"] = OriginalInput });
|
||||
|
||||
// Assert
|
||||
mockTextGeneration.Verify(m => m.GetTextContentsAsync("Prompt: NewInput", It.IsAny<PromptExecutionSettings>(), It.IsAny<Kernel>(), It.IsAny<CancellationToken>()), Times.Once());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PostInvocationPromptFilterCanOverrideRenderedPromptAsync()
|
||||
{
|
||||
// Arrange
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Prompt");
|
||||
var kernel = this.GetKernelWithFilters(textGenerationService: mockTextGeneration.Object,
|
||||
onPromptRender: async (context, next) =>
|
||||
{
|
||||
await next(context);
|
||||
context.RenderedPrompt += " - updated from filter";
|
||||
});
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
mockTextGeneration.Verify(m => m.GetTextContentsAsync("Prompt - updated from filter", It.IsAny<PromptExecutionSettings>(), It.IsAny<Kernel>(), It.IsAny<CancellationToken>()), Times.Once());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PostInvocationPromptFilterSkippingWorksCorrectlyAsync()
|
||||
{
|
||||
// Arrange
|
||||
var mockTextGeneration = this.GetMockTextGeneration();
|
||||
var function = KernelFunctionFactory.CreateFromPrompt("Prompt");
|
||||
var kernel = this.GetKernelWithFilters(textGenerationService: mockTextGeneration.Object,
|
||||
onPromptRender: (context, next) =>
|
||||
{
|
||||
// next(context) is not called here, prompt rendering is cancelled.
|
||||
return Task.CompletedTask;
|
||||
});
|
||||
|
||||
// Act
|
||||
var result = await kernel.InvokeAsync(function);
|
||||
|
||||
// Assert
|
||||
mockTextGeneration.Verify(m => m.GetTextContentsAsync("", It.IsAny<PromptExecutionSettings>(), It.IsAny<Kernel>(), It.IsAny<CancellationToken>()), Times.Once());
|
||||
}
|
||||
}
|
||||
@@ -587,10 +587,10 @@ public class KernelFunctionFromPromptTests
|
||||
// Arrange
|
||||
var mockTextContent = new TextContent("Result");
|
||||
var mockTextCompletion = new Mock<ITextGenerationService>();
|
||||
mockTextCompletion.Setup(m => m.GetTextContentsAsync(It.IsAny<string>(), It.IsAny<PromptExecutionSettings>(), It.IsAny<Kernel>(), It.IsAny<CancellationToken>())).ReturnsAsync([mockTextContent]);
|
||||
mockTextCompletion.Setup(m => m.GetTextContentsAsync(It.IsAny<string>(), It.IsAny<PromptExecutionSettings>(), It.IsAny<Kernel>(), It.IsAny<CancellationToken>())).ReturnsAsync(new List<TextContent> { mockTextContent });
|
||||
|
||||
#pragma warning disable CS0618 // Events are deprecated
|
||||
static void MyRenderedHandler(object? sender, PromptRenderedEventArgs e)
|
||||
void MyRenderedHandler(object? sender, PromptRenderedEventArgs e)
|
||||
{
|
||||
e.RenderedPrompt += " USE SHORT, CLEAR, COMPLETE SENTENCES.";
|
||||
}
|
||||
|
||||
@@ -241,7 +241,7 @@ public class KernelTests
|
||||
Assert.Equal(1, handlerInvocations);
|
||||
Assert.Equal(0, functionInvocations);
|
||||
Assert.Same(function, ex.Function);
|
||||
Assert.Null(ex.FunctionResult);
|
||||
Assert.Null(ex.FunctionResult?.Value);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -266,7 +266,7 @@ public class KernelTests
|
||||
Assert.Equal(1, handlerInvocations);
|
||||
Assert.Equal(0, functionInvocations);
|
||||
Assert.Same(function, ex.Function);
|
||||
Assert.Null(ex.FunctionResult);
|
||||
Assert.Null(ex.FunctionResult?.Value);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -293,7 +293,7 @@ public class KernelTests
|
||||
// Assert
|
||||
Assert.Equal(0, invoked);
|
||||
Assert.Same(functions["GetAnyValue"], ex.Function);
|
||||
Assert.Null(ex.FunctionResult);
|
||||
Assert.Null(ex.FunctionResult?.Value);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -585,8 +585,8 @@ public class KernelTests
|
||||
.AddSingleton(new HttpClient())
|
||||
#pragma warning restore CA2000
|
||||
.AddSingleton(loggerFactory.Object)
|
||||
.AddSingleton<IFunctionFilter>(new MyFunctionFilter())
|
||||
.AddSingleton<IPromptFilter>(new MyPromptFilter())
|
||||
.AddSingleton<IFunctionInvocationFilter>(new MyFunctionFilter())
|
||||
.AddSingleton<IPromptRenderFilter>(new MyPromptFilter())
|
||||
.BuildServiceProvider();
|
||||
var plugin = KernelPluginFactory.CreateFromFunctions("plugin1");
|
||||
var plugins = new KernelPluginCollection() { plugin };
|
||||
@@ -712,11 +712,11 @@ public class KernelTests
|
||||
|
||||
private void AssertFilters(Kernel kernel1, Kernel kernel2)
|
||||
{
|
||||
var functionFilters1 = kernel1.GetAllServices<IFunctionFilter>().ToArray();
|
||||
var promptFilters1 = kernel1.GetAllServices<IPromptFilter>().ToArray();
|
||||
var functionFilters1 = kernel1.GetAllServices<IFunctionInvocationFilter>().ToArray();
|
||||
var promptFilters1 = kernel1.GetAllServices<IPromptRenderFilter>().ToArray();
|
||||
|
||||
var functionFilters2 = kernel2.GetAllServices<IFunctionFilter>().ToArray();
|
||||
var promptFilters2 = kernel2.GetAllServices<IPromptFilter>().ToArray();
|
||||
var functionFilters2 = kernel2.GetAllServices<IFunctionInvocationFilter>().ToArray();
|
||||
var promptFilters2 = kernel2.GetAllServices<IPromptRenderFilter>().ToArray();
|
||||
|
||||
Assert.Equal(functionFilters1.Length, functionFilters2.Length);
|
||||
|
||||
@@ -755,21 +755,19 @@ public class KernelTests
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class MyFunctionFilter : IFunctionFilter
|
||||
private sealed class MyFunctionFilter : IFunctionInvocationFilter
|
||||
{
|
||||
public void OnFunctionInvoked(FunctionInvokedContext context)
|
||||
{ }
|
||||
|
||||
public void OnFunctionInvoking(FunctionInvokingContext context)
|
||||
{ }
|
||||
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
|
||||
{
|
||||
await next(context);
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class MyPromptFilter : IPromptFilter
|
||||
private sealed class MyPromptFilter : IPromptRenderFilter
|
||||
{
|
||||
public void OnPromptRendered(PromptRenderedContext context)
|
||||
{ }
|
||||
|
||||
public void OnPromptRendering(PromptRenderingContext context)
|
||||
{ }
|
||||
public async Task OnPromptRenderAsync(PromptRenderContext context, Func<PromptRenderContext, Task> next)
|
||||
{
|
||||
await next(context);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -406,17 +406,20 @@ public class CodeBlockTests
|
||||
]
|
||||
);
|
||||
|
||||
#pragma warning disable CS0618 // Events are deprecated
|
||||
kernel.PromptRendering += (object? sender, PromptRenderingEventArgs e) =>
|
||||
var promptFilter = new FakePromptFilter(onPromptRender: async (context, next) =>
|
||||
{
|
||||
Assert.Equal(FooValue, e.Arguments[parameterName]);
|
||||
};
|
||||
Assert.Equal(FooValue, context.Arguments[parameterName]);
|
||||
await next(context);
|
||||
});
|
||||
|
||||
kernel.FunctionInvoking += (object? sender, FunctionInvokingEventArgs e) =>
|
||||
var functionFilter = new FakeFunctionFilter(async (context, next) =>
|
||||
{
|
||||
Assert.Equal(FooValue, e.Arguments[parameterName]);
|
||||
};
|
||||
#pragma warning restore CS0618 // Events are deprecated
|
||||
Assert.Equal(FooValue, context.Arguments[parameterName]);
|
||||
await next(context);
|
||||
});
|
||||
|
||||
kernel.PromptRenderFilters.Add(promptFilter);
|
||||
kernel.FunctionInvocationFilters.Add(functionFilter);
|
||||
|
||||
var codeBlock = new CodeBlock(blockList, "");
|
||||
await codeBlock.RenderCodeAsync(kernel);
|
||||
@@ -454,19 +457,22 @@ public class CodeBlockTests
|
||||
]
|
||||
);
|
||||
|
||||
#pragma warning disable CS0618 // Events are deprecated
|
||||
kernel.PromptRendering += (object? sender, PromptRenderingEventArgs e) =>
|
||||
var promptFilter = new FakePromptFilter(onPromptRender: async (context, next) =>
|
||||
{
|
||||
Assert.Equal(FooValue, e.Arguments["foo"]);
|
||||
Assert.Equal(FooValue, e.Arguments["x11"]);
|
||||
};
|
||||
Assert.Equal(FooValue, context.Arguments["foo"]);
|
||||
Assert.Equal(FooValue, context.Arguments["x11"]);
|
||||
await next(context);
|
||||
});
|
||||
|
||||
kernel.FunctionInvoking += (object? sender, FunctionInvokingEventArgs e) =>
|
||||
var functionFilter = new FakeFunctionFilter(async (context, next) =>
|
||||
{
|
||||
Assert.Equal(FooValue, e.Arguments["foo"]);
|
||||
Assert.Equal(FooValue, e.Arguments["x11"]);
|
||||
};
|
||||
#pragma warning restore CS0618 // Events are deprecated
|
||||
Assert.Equal(FooValue, context.Arguments["foo"]);
|
||||
Assert.Equal(FooValue, context.Arguments["x11"]);
|
||||
await next(context);
|
||||
});
|
||||
|
||||
kernel.PromptRenderFilters.Add(promptFilter);
|
||||
kernel.FunctionInvocationFilters.Add(functionFilter);
|
||||
|
||||
var codeBlock = new CodeBlock(blockList, "");
|
||||
await codeBlock.RenderCodeAsync(kernel, arguments);
|
||||
@@ -507,4 +513,26 @@ public class CodeBlockTests
|
||||
var exception = await Assert.ThrowsAsync<ArgumentException>(async () => await codeBlock.RenderCodeAsync(this._kernel, arguments));
|
||||
Assert.Contains(FooValue, exception.Message, StringComparison.OrdinalIgnoreCase);
|
||||
}
|
||||
|
||||
#region private
|
||||
|
||||
private sealed class FakeFunctionFilter(
|
||||
Func<FunctionInvocationContext, Func<FunctionInvocationContext, Task>, Task>? onFunctionInvocation) : IFunctionInvocationFilter
|
||||
{
|
||||
private readonly Func<FunctionInvocationContext, Func<FunctionInvocationContext, Task>, Task>? _onFunctionInvocation = onFunctionInvocation;
|
||||
|
||||
public Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next) =>
|
||||
this._onFunctionInvocation?.Invoke(context, next) ?? Task.CompletedTask;
|
||||
}
|
||||
|
||||
private sealed class FakePromptFilter(
|
||||
Func<PromptRenderContext, Func<PromptRenderContext, Task>, Task>? onPromptRender = null) : IPromptRenderFilter
|
||||
{
|
||||
private readonly Func<PromptRenderContext, Func<PromptRenderContext, Task>, Task>? _onPromptRender = onPromptRender;
|
||||
|
||||
public Task OnPromptRenderAsync(PromptRenderContext context, Func<PromptRenderContext, Task> next) =>
|
||||
this._onPromptRender?.Invoke(context, next) ?? Task.CompletedTask;
|
||||
}
|
||||
|
||||
#endregion
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user