microsoft/semantic-kernel
1
0
Fork 0
mirror of https://github.com/microsoft/semantic-kernel.git synced 2026-10-12 02:19:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

.Net: Add AIContextProvider support to Semantic Kernel (#11689)

Browse Source 
### Motivation and Context

Adding support for AI context behaviors to Semantic Kernel.
These allow creating plugins that are able to listen to messages being
added to the chat history and contribute to the AI context on each
invocation. This PR also integrates support for these with Agents and
AgentThreads.

#10100 
#10712

---------

Co-authored-by: SergeyMenshykh <[email protected]>
Co-authored-by: Roger Barreto <[email protected]>
This commit is contained in:
westey
2025-05-22 10:56:51 +00:00
committed by GitHub
co-authored by SergeyMenshykh Roger Barreto
parent 7243188e92
commit 9ada3aa701
68 changed files with 4821 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/decisions/0072-agents-with-memory.md
+209
View File
@@ -0,0 +1,209 @@
---
# These are optional elements. Feel free to remove any of them.
status: accepted
contact: westey-m
date: 2025-04-17
deciders: westey-m, markwallace-microsoft, alliscode, TaoChenOSU, moonbox3, crickman
consulted: westey-m, markwallace-microsoft, alliscode, TaoChenOSU, moonbox3, crickman
informed: westey-m, markwallace-microsoft, alliscode, TaoChenOSU, moonbox3, crickman
---
# Agents with Memory
## What do we mean by Memory?
By memory we mean the capability to remember information and skills that are learned during
a conversation and re-use those later in the same conversation or later in a subsequent conversation.
## Context and Problem Statement
Today we support multiple agent types with different characteristics:
1. In process vs remote.
2. Remote agents that store and maintain conversation state in the service vs those that require the caller to provide conversation state on each invocation.
We need to support advanced memory capabilities across this range of agent types.
### Memory Scope
Another aspect of memory that is important to consider is the scope of different memory types.
Most agent implementations have instructions and skills but the agent is not tied to a single conversation.
On each invocation of the agent, the agent is told which conversation to participate in, during that invocation.
Memories about a user or about a conversation with a user is therefore extracted from one of these conversation and recalled
during the same or another conversation with the same user.
These memories will typically contain information that the user would not like to share with other users of the system.
Other types of memories also exist which are not tied to a specific user or conversation.
E.g. an Agent may learn how to do something and be able to do that in many conversations with different users.
With these type of memories there is of cousrse risk in leaking personal information between different users which is important to guard against.
### Packaging memory capabilities
All of the above memory types can be supported for any agent by attaching software components to conversation threads.
This is achieved via a simple mechanism of:
1. Inspecting and using messages as they are passed to and from the agent.
2. Passing additional context to the agent per invocation.
With our current `AgentThread` implementation, when an agent is invoked, all input and output messages are already passed to the `AgentThread`
and can be made available to any components attached to the `AgentThread`.
Where agents are remote/external and manage conversation state in the service, passing the messages to the `AgentThread` may not have any
affect on the thread in the service. This is OK, since the service will have already updated the thread during the remote invocation.
It does however, still allow us to subscribe to messages in any attached components.
For the second requirement of getting additional context per invocation, the agent may ask the thread passed to it, to in turn ask
each of the components attached to it, to provide context to pass to the Agent.
This enables the component to provide memories that it contains to the Agent as needed.
Different memory capabilities can be built using separate components. Each component would have the following characteristics:
1. May store some context that can be provided to the agent per invocation.
2. May inspect messages from the conversation to learn from the conversation and build its context.
3. May register plugins to allow the agent to directly store, retrieve, update or clear memories.
### Suspend / Resume
Building a service to host an agent comes with challenges.
It's hard to build a stateful service, but service consumers expect an experience that looks stateful from the outside.
E.g. on each invocation, the user expects that the service can continue a conversation they are having.
This means that where the the service is exposing a local agent with local conversation state management (e.g. via `ChatHistory`)
that conversation state needs to be loaded and persisted for each invocation of the service.
It also means that any memory components that may have some in-memory state will need to be loaded and persisted too.
For cases like this, the `OnSuspend` and `OnResume` methods allow notification of the components that they need to save or reload their state.
It is up to each of these components to decide how and where to save state to or load state from.
## Proposed interface for Memory Components
The types of events that Memory Components require are not unique to memory, and can be used to package up other capabilities too.
The suggestion is therefore to create a more generally named type that can be used for other scenarios as well and can even
be used for non-agent scenarios too.
This type should live in the `Microsoft.SemanticKernel.Abstractions` nuget, since these components can be used by systems other than just agents.
```csharp
namespace Microsoft.SemanticKernel;
public abstract class AIContextBehavior
{
public virtual IReadOnlyCollection<AIFunction> AIFunctions => Array.Empty<AIFunction>();
public virtual Task OnThreadCreatedAsync(string? threadId, CancellationToken cancellationToken = default);
public virtual Task OnThreadDeleteAsync(string? threadId, CancellationToken cancellationToken = default);
// OnThreadCheckpointAsync not included in initial release, maybe in future.
public virtual Task OnThreadCheckpointAsync(string? threadId, CancellationToken cancellationToken = default);
public virtual Task OnNewMessageAsync(string? threadId, ChatMessage newMessage, CancellationToken cancellationToken = default);
public abstract Task<string> OnModelInvokeAsync(ICollection<ChatMessage> newMessages, CancellationToken cancellationToken = default);
public virtual Task OnSuspendAsync(string? threadId, CancellationToken cancellationToken = default);
public virtual Task OnResumeAsync(string? threadId, CancellationToken cancellationToken = default);
}
```
## Managing multiple components
To manage multiple components I propose that we have a `AIContextBehavior`.
This class allows registering components and delegating new message notifications, ai invocation calls, etc. to the contained components.
## Integrating with agents
I propose to add a `AIContextBehaviorManager` to the `AgentThread` class, allowing us to attach components to any `AgentThread`.
When an `Agent` is invoked, we will call `OnModelInvokeAsync` on each component via the `AIContextBehaviorManager` to get
a combined set of context to pass to the agent for this invocation. This will be internal to the `Agent` class and transparent to the user.
```csharp
var additionalInstructions = await currentAgentThread.OnModelInvokeAsync(messages, cancellationToken).ConfigureAwait(false);
```
## Usage examples
### Multiple threads using the same memory component
```csharp
// Create a vector store for storing memories.
var vectorStore = new InMemoryVectorStore();
// Create a memory store that is tired to a "Memories" collection in the vector store and stores memories under the "user/12345" namespace.
using var textMemoryStore = new VectorDataTextMemoryStore<string>(vectorStore, textEmbeddingService, "Memories", "user/12345", 1536);
// Create a memory component to will pull user facts from the conversation, store them in the vector store
// and pass them to the agent as additional instructions.
var userFacts = new UserFactsMemoryComponent(this.Fixture.Agent.Kernel, textMemoryStore);
// Create a thread and attach a Memory Component.
var agentThread1 = new ChatHistoryAgentThread();
agentThread1.ThreadExtensionsManager.Add(userFacts);
var asyncResults1 = agent.InvokeAsync("Hello, my name is Caoimhe.", agentThread1);
// Create a second thread and attach a Memory Component.
var agentThread2 = new ChatHistoryAgentThread();
agentThread2.ThreadExtensionsManager.Add(userFacts);
var asyncResults2 = agent.InvokeAsync("What is my name?.", agentThread2);
// Expected response contains Caoimhe.
```
### Using a RAG component
```csharp
// Create Vector Store and Rag Store/Component
var vectorStore = new InMemoryVectorStore();
using var ragStore = new TextRagStore<string>(vectorStore, textEmbeddingService, "Memories", 1536, "group/g2");
var ragComponent = new TextRagComponent(ragStore, new TextRagComponentOptions());
// Upsert docs into vector store.
await ragStore.UpsertDocumentsAsync(
[
new TextRagDocument("The financial results of Contoso Corp for 2023 is as follows:\nIncome EUR 174 000 000\nExpenses EUR 152 000 000")
{
SourceName = "Contoso 2023 Financial Report",
SourceReference = "https://www.consoso.com/reports/2023.pdf",
Namespaces = ["group/g2"]
}
]);
// Create a new agent thread and register the Rag component
var agentThread = new ChatHistoryAgentThread();
agentThread.ThreadExtensionsManager.RegisterThreadExtension(ragComponent);
// Inovke the agent.
var asyncResults1 = agent.InvokeAsync("What was the income of Contoso for 2023", agentThread);
// Expected response contains the 174M income from the document.
```
## Decisions to make
### Extension base class name
1. ConversationStateExtension
1.1. Long
2. MemoryComponent
2.1. Too specific
3. AIContextBehavior
Decided 3. AIContextBehavior.
### Location for abstractions
1. Microsoft.SemanticKernel.<baseclass>
2. Microsoft.SemanticKernel.Memory.<baseclass>
3. Microsoft.SemanticKernel.Memory.<baseclass> (in separate nuget)
Decided: 1. Microsoft.SemanticKernel.<baseclass>.
### Location for memory components
1. A nuget for each component
2. Microsoft.SemanticKernel.Core nuget
3. Microsoft.SemanticKernel.Memory nuget
4. Microsoft.SemanticKernel.ConversationStateExtensions nuget
Decided: 2. Microsoft.SemanticKernel.Core nuget
dotnet/docs/EXPERIMENTS.md
+1
View File
@@ -25,6 +25,7 @@ You can use the following diagnostic IDs to ignore warnings or errors for a part
| SKEXP0100 | Advanced Semantic Kernel features |
| SKEXP0110 | Semantic Kernel Agents |
| SKEXP0120 | Native-AOT |
| SKEXP0130 | AI Context Providers |
| MEVD9000 | Microsoft.Extensions.VectorData experimental user-facing APIs |
| MEVD9001 | Microsoft.Extensions.VectorData experimental connector-facing APIs |
dotnet/samples/Concepts/Agents/ChatCompletion_Mem0.cs
+119
View File
@@ -0,0 +1,119 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Net.Http.Headers;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.Memory;
namespace Agents;
#pragma warning disable SKEXP0130 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
/// <summary>
/// Demonstrate creation of <see cref="ChatCompletionAgent"/> and
/// adding memory capabilities to it using https://mem0.ai/.
/// </summary>
public class ChatCompletion_Mem0(ITestOutputHelper output) : BaseTest(output)
{
private const string AgentName = "FriendlyAssistant";
private const string AgentInstructions = "You are a friendly assistant";
/// <summary>
/// Shows how to allow an agent to remember user preferences across multiple threads.
/// </summary>
[Fact]
private async Task UseMemoryAsync()
{
// Create a new HttpClient with the base address of the mem0 service and a token for authentication.
using var httpClient = new HttpClient()
{
BaseAddress = new Uri(TestConfiguration.Mem0.BaseAddress ?? "https://api.mem0.ai")
};
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Token", TestConfiguration.Mem0.ApiKey);
// Create a mem0 component with the current user's id, so that it stores memories for that user.
var mem0Provider = new Mem0Provider(httpClient, new()
{
UserId = "U1"
});
// Clear out any memories from previous runs, if any, to demonstrate a first run experience.
await mem0Provider.ClearStoredMemoriesAsync();
// Create our agent and add our finance plugin with auto function invocation.
Kernel kernel = this.CreateKernelWithChatCompletion();
kernel.Plugins.AddFromType<FinancePlugin>();
ChatCompletionAgent agent =
new()
{
Name = AgentName,
Instructions = AgentInstructions,
Kernel = kernel,
Arguments = new KernelArguments(new PromptExecutionSettings { FunctionChoiceBehavior = FunctionChoiceBehavior.Auto() })
};
Console.WriteLine("----- First Conversation -----");
// Create a thread for the agent and add the mem0 component to it.
ChatHistoryAgentThread agentThread = new();
agentThread.AIContextProviders.Add(mem0Provider);
// First ask the agent to retrieve a company report with no previous context.
// The agent will not be able to invoke the plugin, since it doesn't know
// the company code or the report format, so it should ask for clarification.
string userMessage = "Please retrieve my company report";
Console.WriteLine($"User: {userMessage}");
ChatMessageContent message = await agent.InvokeAsync(userMessage, agentThread).FirstAsync();
Console.WriteLine($"Assistant:\n{message.Content}");
// Now tell the agent the company code and the report format that you want to use
// and it should be able to invoke the plugin and return the report.
userMessage = "I always work with CNTS and I always want a detailed report format";
Console.WriteLine($"User: {userMessage}");
message = await agent.InvokeAsync(userMessage, agentThread).FirstAsync();
Console.WriteLine($"Assistant:\n{message.Content}");
Console.WriteLine("----- Second Conversation -----");
// Create a new thread for the agent and add our mem0 component to it again
// The new thread has no context of the previous conversation.
agentThread = new();
agentThread.AIContextProviders.Add(mem0Provider);
// Since we have the mem0 component in the thread, the agent should be able to
// retrieve the company report without asking for clarification, as it will
// be able to remember the user preferences from the last thread.
userMessage = "Please retrieve my company report";
Console.WriteLine($"User: {userMessage}");
message = await agent.InvokeAsync(userMessage, agentThread).FirstAsync();
Console.WriteLine($"Assistant:\n{message.Content}");
}
private sealed class FinancePlugin
{
[KernelFunction]
public string RetrieveCompanyReport(string companyCode, ReportFormat reportFormat)
{
if (companyCode != "CNTS")
{
throw new ArgumentException("Company code not found");
}
return reportFormat switch
{
ReportFormat.Brief => "CNTS is a company that specializes in technology.",
ReportFormat.Detailed => "CNTS is a company that specializes in technology. It had a revenue of $10 million in 2022. It has 100 employees.",
_ => throw new ArgumentException("Report format not found")
};
}
}
private enum ReportFormat
{
Brief,
Detailed
}
}
dotnet/samples/Concepts/Agents/ChatCompletion_Rag.cs
+183
View File
@@ -0,0 +1,183 @@
// Copyright (c) Microsoft. All rights reserved.
using Azure.AI.OpenAI;
using Azure.Identity;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.VectorData;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.Connectors.InMemory;
using Microsoft.SemanticKernel.Data;
namespace Agents;
#pragma warning disable SKEXP0130 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
/// <summary>
/// Demonstrate creation of <see cref="ChatCompletionAgent"/> and
/// adding simple retrieval augmented generation (RAG) capabilities to it.
/// </summary>
/// <remarks>
/// This example shows how to use the <see cref="TextSearchStore{TKey}"/> class which is designed
/// to simplify the process of storing and searching text documents by having a built in schema.
/// If you want to control the schema yourself, you can use an implementation of <see cref="VectorStoreCollection{TKey, TRecord}"/>
/// with the <see cref="VectorStoreTextSearch{TRecord}"/> class instead.
/// </remarks>
public class ChatCompletion_Rag(ITestOutputHelper output) : BaseTest(output)
{
private const string AgentName = "FriendlyAssistant";
private const string AgentInstructions = "You are a friendly assistant";
/// <summary>
/// Shows how to do Retrieval Augmented Generation (RAG) with some basic text strings.
/// </summary>
[Fact]
private async Task UseChatCompletionAgentWithBasicRag()
{
var embeddingGenerator = new AzureOpenAIClient(new Uri(TestConfiguration.AzureOpenAIEmbeddings.Endpoint), new AzureCliCredential())
.GetEmbeddingClient(TestConfiguration.AzureOpenAIEmbeddings.DeploymentName)
.AsIEmbeddingGenerator(1536);
// Create a vector store to store our documents.
// Note that the embedding generator provided here must be able to generate embeddings matching the
// number of dimensions configured for the TextSearchStore below.
var vectorStore = new InMemoryVectorStore(new() { EmbeddingGenerator = embeddingGenerator });
// Create a store that uses a built in schema for storing text documents
// and provides easy upload and search capabilities.
// The data is stored in the `FinancialData` collection and embeddings have 1536 dimensions.
// When searching results will be limited to those with the `group/g2` namespace.
using var textSearchStore = new TextSearchStore<string>(vectorStore, collectionName: "FinancialData", vectorDimensions: 1536);
// Upsert documents into the store.
await textSearchStore.UpsertTextAsync(
[
"The financial results of Contoso Corp for 2024 is as follows:\nIncome EUR 154 000 000\nExpenses EUR 142 000 000",
"The financial results of Contoso Corp for 2023 is as follows:\nIncome EUR 174 000 000\nExpenses EUR 152 000 000",
"The financial results of Contoso Corp for 2022 is as follows:\nIncome EUR 184 000 000\nExpenses EUR 162 000 000",
"The Contoso Corporation is a multinational business with its headquarters in Paris. The company is a manufacturing, sales, and support organization with more than 100,000 products.",
"The financial results of AdventureWorks for 2021 is as follows:\nIncome USD 223 000 000\nExpenses USD 210 000 000",
"AdventureWorks is a large American business that specializes in adventure parks and family entertainment.",
]);
// Create our agent.
Kernel kernel = this.CreateKernelWithChatCompletion();
ChatCompletionAgent agent =
new()
{
Name = AgentName,
Instructions = AgentInstructions,
Kernel = kernel,
};
// Create a thread for the agent.
ChatHistoryAgentThread agentThread = new();
// Create a text search provider that can automatically search the vector store
// for documents that match the user's query and inject them into the agent's prompt.
var textSearchProvider = new TextSearchProvider(textSearchStore);
agentThread.AIContextProviders.Add(textSearchProvider);
// Invoke and display assistant response
ChatMessageContent message = await agent.InvokeAsync("Where is Contoso based?", agentThread).FirstAsync();
Console.WriteLine(message.Content);
message = await agent.InvokeAsync("What was its expenses for 2022?", agentThread).FirstAsync();
Console.WriteLine(message.Content);
}
/// <summary>
/// Shows how to do Retrieval Augmented Generation (RAG) with citations and filtering.
/// </summary>
[Fact]
private async Task RagWithCitationsAndFiltering()
{
var embeddingGenerator = new AzureOpenAIClient(new Uri(TestConfiguration.AzureOpenAIEmbeddings.Endpoint), new AzureCliCredential())
.GetEmbeddingClient(TestConfiguration.AzureOpenAIEmbeddings.DeploymentName)
.AsIEmbeddingGenerator(1536);
// Create a vector store to store our documents.
// Note that the embedding generator provided here must be able to generate embeddings matching the
// number of dimensions configured for the TextSearchStore below.
var vectorStore = new InMemoryVectorStore(new() { EmbeddingGenerator = embeddingGenerator });
// Create a store that uses a built in schema for storing text documents
// and provides easy upload and search capabilities.
// The data is stored in the `FinancialData` collection and embeddings have 1536 dimensions.
// When searching results will be limited to those with the `group/g2` namespace.
using var textSearchStore = new TextSearchStore<string>(vectorStore, collectionName: "FinancialData", vectorDimensions: 1536, new() { SearchNamespace = "group/g2" });
// Upsert documents into the store.
// Not that documents have different namespaces, and only the ones
// with the `group/g2` namespace will be matched.
await textSearchStore.UpsertDocumentsAsync(GetSampleDocuments());
// Create our agent.
Kernel kernel = this.CreateKernelWithChatCompletion();
ChatCompletionAgent agent =
new()
{
Name = AgentName,
Instructions = AgentInstructions,
Kernel = kernel,
};
// Create a thread for the agent.
ChatHistoryAgentThread agentThread = new();
// Create a text search provider that can automatically search the vector store
// for documents that match the user's query and inject them into the agent's prompt.
var textSearchProvider = new TextSearchProvider(textSearchStore);
agentThread.AIContextProviders.Add(textSearchProvider);
// Invoke and display assistant response
ChatMessageContent message = await agent.InvokeAsync("What was the income of Contoso for 2023", agentThread).FirstAsync();
Console.WriteLine(message.Content);
}
private static IEnumerable<TextSearchDocument> GetSampleDocuments()
{
yield return new TextSearchDocument
{
Text = "The financial results of Contoso Corp for 2024 is as follows:\nIncome EUR 154 000 000\nExpenses EUR 142 000 000",
SourceName = "Contoso 2024 Financial Report",
SourceLink = "https://www.consoso.com/reports/2024.pdf",
Namespaces = ["group/g1"]
};
yield return new TextSearchDocument
{
Text = "The financial results of Contoso Corp for 2023 is as follows:\nIncome EUR 174 000 000\nExpenses EUR 152 000 000",
SourceName = "Contoso 2023 Financial Report",
SourceLink = "https://www.consoso.com/reports/2023.pdf",
Namespaces = ["group/g2"]
};
yield return new TextSearchDocument
{
Text = "The financial results of Contoso Corp for 2022 is as follows:\nIncome EUR 184 000 000\nExpenses EUR 162 000 000",
SourceName = "Contoso 2022 Financial Report",
SourceLink = "https://www.consoso.com/reports/2022.pdf",
Namespaces = ["group/g2"]
};
yield return new TextSearchDocument
{
Text = "The Contoso Corporation is a multinational business with its headquarters in Paris. The company is a manufacturing, sales, and support organization with more than 100,000 products.",
SourceName = "About Contoso",
SourceLink = "https://www.consoso.com/about-us",
Namespaces = ["group/g2"]
};
yield return new TextSearchDocument
{
Text = "The financial results of AdventureWorks for 2021 is as follows:\nIncome USD 223 000 000\nExpenses USD 210 000 000",
SourceName = "AdventureWorks 2021 Financial Report",
SourceLink = "https://www.adventure-works.com/reports/2021.pdf",
Namespaces = ["group/g1", "group/g2"]
};
yield return new TextSearchDocument
{
Text = "AdventureWorks is a large American business that specializes in adventure parks and family entertainment.",
SourceName = "About AdventureWorks",
SourceLink = "https://www.adventure-works.com/about-us",
Namespaces = ["group/g1", "group/g2"]
};
}
}
dotnet/samples/Concepts/Agents/ChatCompletion_Whiteboard.cs
+137
View File
@@ -0,0 +1,137 @@
// Copyright (c) Microsoft. All rights reserved.
using Azure.AI.OpenAI;
using Azure.Identity;
using Microsoft.Extensions.AI;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Memory;
namespace Agents;
#pragma warning disable SKEXP0130 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
/// <summary>
/// Demonstrate creation of <see cref="ChatCompletionAgent"/> and
/// adding whiteboarding capabilities, where the most relevant information from the conversation is captured on a whiteboard.
/// This is useful for long running conversations where the conversation history may need to be truncated
/// over time, but you do not want to agent to lose context.
/// </summary>
public class ChatCompletion_Whiteboard(ITestOutputHelper output) : BaseTest(output)
{
private const string AgentName = "FriendlyAssistant";
private const string AgentInstructions = "You are a friendly assistant";
/// <summary>
/// Shows how to allow an agent to use a whiteboard for storing the most important information
/// from a long running, truncated conversation.
/// </summary>
[Fact]
private async Task UseWhiteboardForShortTermMemory()
{
var chatClient = new AzureOpenAIClient(new Uri(TestConfiguration.AzureOpenAI.Endpoint), new AzureCliCredential())
.GetChatClient(TestConfiguration.AzureOpenAI.ChatDeploymentName)
.AsIChatClient();
// Create the whiteboard.
var whiteboardProvider = new WhiteboardProvider(chatClient);
// Create our agent and add our finance plugin with auto function invocation.
Kernel kernel = this.CreateKernelWithChatCompletion();
// Create the agent with our sample plugin.
kernel.Plugins.AddFromType<VMPlugin>();
ChatCompletionAgent agent =
new()
{
Name = AgentName,
Instructions = AgentInstructions,
Kernel = kernel,
Arguments = new KernelArguments(new PromptExecutionSettings { FunctionChoiceBehavior = FunctionChoiceBehavior.Auto() })
};
// Create a chat history reducer that we can use to truncate the chat history
// when it goes over 3 items.
var chatHistoryReducer = new ChatHistoryTruncationReducer(3, 3);
// Create a thread for the agent and add the whiteboard to it.
ChatHistoryAgentThread agentThread = new();
agentThread.AIContextProviders.Add(whiteboardProvider);
// Simulate a conversation with the agent.
// We will also truncate the conversation once it goes over a few items.
await InvokeWithConsoleWriteLine("Hello");
await InvokeWithConsoleWriteLine("I'd like to create a VM?");
await InvokeWithConsoleWriteLine("I want it to have 3 cores.");
await InvokeWithConsoleWriteLine("I want it to have 48GB of RAM.");
await InvokeWithConsoleWriteLine("I want it to have a 500GB Harddrive.");
await InvokeWithConsoleWriteLine("I want it in Europe.");
await InvokeWithConsoleWriteLine("Can you make it Linux and call it 'ContosoVM'.");
await InvokeWithConsoleWriteLine("OK, let's call it `ContosoFinanceVM_Europe` instead.");
await InvokeWithConsoleWriteLine("Thanks, now I want to create another VM.");
await InvokeWithConsoleWriteLine("Make all the options the same as the last one, except for the region, which should be North America, and the name, which should be 'ContosoFinanceVM_NorthAmerica'.");
async Task InvokeWithConsoleWriteLine(string message)
{
// Print the user input.
Console.WriteLine($"User: {message}");
// Invoke the agent.
ChatMessageContent response = await agent.InvokeAsync(message, agentThread).FirstAsync();
// Print the response.
Console.WriteLine($"Assistant:\n{response.Content}\n");
// Make sure any async whiteboard processing is complete before we print out its contents.
await whiteboardProvider.WhenProcessingCompleteAsync();
// Print out the whiteboard contents.
Console.WriteLine("Whiteboard contents:");
foreach (var item in whiteboardProvider.CurrentWhiteboardContent)
{
Console.WriteLine($"- {item}");
}
Console.WriteLine();
// Truncate the chat history if it gets too big.
await agentThread.ChatHistory.ReduceInPlaceAsync(chatHistoryReducer, CancellationToken.None);
}
}
private sealed class VMPlugin
{
[KernelFunction]
public Task<VMCreateResult> CreateVM(Region region, OperatingSystem os, string name, int numberOfCores, int memorySizeInGB, int hddSizeInGB)
{
if (name == "ContosoVM")
{
throw new Exception("VM name already exists");
}
return Task.FromResult(new VMCreateResult { VMId = Guid.NewGuid().ToString() });
}
}
public class VMCreateResult
{
public string VMId { get; set; } = string.Empty;
}
private enum Region
{
NorthAmerica,
SouthAmerica,
Europe,
Asia,
Africa,
Australia
}
private enum OperatingSystem
{
Windows,
Linux,
MacOS
}
}
dotnet/samples/Concepts/README.md
+3
View File
@@ -23,6 +23,9 @@ dotnet test -l "console;verbosity=detailed" --filter "FullyQualifiedName=ChatCom
- [ComplexChat_NestedShopper](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Agents/ComplexChat_NestedShopper.cs)
- [MixedChat_Agents](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Agents/MixedChat_Agents.cs)
- [OpenAIAssistant_ChartMaker](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Agents/OpenAIAssistant_ChartMaker.cs)
- [ChatCompletion_Rag: Shows how to easily add RAG to an agent](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Agents/ChatCompletion_Rag.cs)
- [ChatCompletion_Mem0: Shows how to add memory to an agent using mem0](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Agents/ChatCompletion_Mem0.cs)
- [ChatCompletion_Whiteboard: Shows how to add short term Whiteboarding memory to an agent](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Agents/ChatCompletion_Whiteboard.cs)
### AudioToText - Different ways of using [`AudioToText`](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel.Abstractions/AI/AudioToText/IAudioToTextService.cs) services to extract text from audio
dotnet/src/Agents/Abstractions/AgentThread.cs
+59
View File
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Diagnostics.CodeAnalysis;
using System.Threading;
using System.Threading.Tasks;
@@ -26,6 +27,52 @@ public abstract class AgentThread
/// </summary>
public virtual bool IsDeleted { get; protected set; } = false;
/// <summary>
/// Gets or sets the container for <see cref="AIContextProvider"/> objects that manages their lifecycle and interactions.
/// </summary>
[Experimental("SKEXP0110")]
public virtual AggregateAIContextProvider AIContextProviders { get; init; } = new AggregateAIContextProvider();
/// <summary>
/// Called when the current conversion is temporarily suspended and any state should be saved.
/// </summary>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>An async task.</returns>
/// <remarks>
/// In a service that hosts an agent, that is invoked via calls to the service, this might be at the end of each service call.
/// In a client application, this might be when the user closes the chat window or the application.
/// </remarks>
[Experimental("SKEXP0110")]
public virtual Task OnSuspendAsync(CancellationToken cancellationToken = default)
{
return this.AIContextProviders.SuspendingAsync(this.Id, cancellationToken);
}
/// <summary>
/// Called when the current conversion is resumed and any state should be restored.
/// </summary>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>An async task.</returns>
/// <remarks>
/// In a service that hosts an agent, that is invoked via calls to the service, this might be at the start of each service call where a previous conversation is being continued.
/// In a client application, this might be when the user re-opens the chat window to resume a conversation after having previously closed it.
/// </remarks>
[Experimental("SKEXP0110")]
public virtual Task OnResumeAsync(CancellationToken cancellationToken = default)
{
if (this.IsDeleted)
{
throw new InvalidOperationException("This thread has been deleted and cannot be used anymore.");
}
if (this.Id is null)
{
throw new InvalidOperationException("This thread cannot be resumed, since it has not been created.");
}
return this.AIContextProviders.ResumingAsync(this.Id, cancellationToken);
}
/// <summary>
/// Creates the thread and returns the thread id.
/// </summary>
@@ -45,6 +92,10 @@ public abstract class AgentThread
}
this.Id = await this.CreateInternalAsync(cancellationToken: cancellationToken).ConfigureAwait(false);
#pragma warning disable SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
await this.AIContextProviders.ConversationCreatedAsync(this.Id, cancellationToken).ConfigureAwait(false);
#pragma warning restore SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
}
/// <summary>
@@ -65,6 +116,10 @@ public abstract class AgentThread
throw new InvalidOperationException("This thread cannot be deleted, since it has not been created.");
}
#pragma warning disable SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
await this.AIContextProviders.ConversationDeletingAsync(this.Id, cancellationToken).ConfigureAwait(false);
#pragma warning restore SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
await this.DeleteInternalAsync(cancellationToken).ConfigureAwait(false);
this.IsDeleted = true;
@@ -92,6 +147,10 @@ public abstract class AgentThread
await this.CreateAsync(cancellationToken).ConfigureAwait(false);
}
#pragma warning disable SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
await this.AIContextProviders.MessageAddingAsync(this.Id, newMessage, cancellationToken).ConfigureAwait(false);
#pragma warning restore SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
await this.OnNewMessageInternalAsync(newMessage, cancellationToken).ConfigureAwait(false);
}
dotnet/src/Agents/AzureAI/Agents.AzureAI.csproj
+1
View File
@@ -34,6 +34,7 @@
<ItemGroup>
<ProjectReference Include="..\Abstractions\Agents.Abstractions.csproj" />
<ProjectReference Include="..\..\SemanticKernel.Core\SemanticKernel.Core.csproj" />
</ItemGroup>
<ItemGroup>
dotnet/src/Agents/AzureAI/AzureAIAgent.cs
+45 -4
View File
@@ -185,6 +185,19 @@ public sealed partial class AzureAIAgent : Agent
() => new AzureAIAgentThread(this.Client),
cancellationToken).ConfigureAwait(false);
var kernel = (options?.Kernel ?? this.Kernel).Clone();
// Get the context contributions from the AIContextProviders.
#pragma warning disable SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var providersContext = await azureAIAgentThread.AIContextProviders.ModelInvokingAsync(messages, cancellationToken).ConfigureAwait(false);
kernel.Plugins.AddFromAIContext(providersContext, "Tools");
#pragma warning restore SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var mergedAdditionalInstructions = MergeAdditionalInstructions(options?.AdditionalInstructions, providersContext.Instructions);
var extensionsContextOptions = options is null ?
new AzureAIAgentInvokeOptions() { AdditionalInstructions = mergedAdditionalInstructions } :
new AzureAIAgentInvokeOptions(options) { AdditionalInstructions = mergedAdditionalInstructions };
var invokeResults = ActivityExtensions.RunWithActivityAsync(
() => ModelDiagnostics.StartAgentInvocationActivity(this.Id, this.GetDisplayName(), this.Description),
() => InternalInvokeAsync(),
@@ -196,9 +209,9 @@ public sealed partial class AzureAIAgent : Agent
this,
this.Client,
azureAIAgentThread.Id!,
options?.ToAzureAIInvocationOptions(),
extensionsContextOptions?.ToAzureAIInvocationOptions(),
this.Logger,
options?.Kernel ?? this.Kernel,
kernel,
options?.KernelArguments,
cancellationToken).ConfigureAwait(false))
{
@@ -302,14 +315,27 @@ public sealed partial class AzureAIAgent : Agent
() => new AzureAIAgentThread(this.Client),
cancellationToken).ConfigureAwait(false);
var kernel = (options?.Kernel ?? this.Kernel).Clone();
// Get the context contributions from the AIContextProviders.
#pragma warning disable SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var providersContext = await azureAIAgentThread.AIContextProviders.ModelInvokingAsync(messages, cancellationToken).ConfigureAwait(false);
kernel.Plugins.AddFromAIContext(providersContext, "Tools");
#pragma warning restore SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var mergedAdditionalInstructions = MergeAdditionalInstructions(options?.AdditionalInstructions, providersContext.Instructions);
var extensionsContextOptions = options is null ?
new AzureAIAgentInvokeOptions() { AdditionalInstructions = mergedAdditionalInstructions } :
new AzureAIAgentInvokeOptions(options) { AdditionalInstructions = mergedAdditionalInstructions };
#pragma warning disable CS0618 // Type or member is obsolete
// Invoke the Agent with the thread that we already added our message to.
var newMessagesReceiver = new ChatHistory();
var invokeResults = this.InvokeStreamingAsync(
azureAIAgentThread.Id!,
options?.ToAzureAIInvocationOptions(),
extensionsContextOptions.ToAzureAIInvocationOptions(),
options?.KernelArguments,
options?.Kernel ?? this.Kernel,
kernel,
newMessagesReceiver,
cancellationToken);
#pragma warning restore CS0618 // Type or member is obsolete
@@ -436,4 +462,19 @@ public sealed partial class AzureAIAgent : Agent
return new AzureAIChannel(this.Client, thread.Id);
}
private static string MergeAdditionalInstructions(string? optionsAdditionalInstructions, string? extensionsContext) =>
(optionsAdditionalInstructions, extensionsContext) switch
{
(string ai, string ec) when !string.IsNullOrWhiteSpace(ai) && !string.IsNullOrWhiteSpace(ec) => string.Concat(
ai,
Environment.NewLine,
Environment.NewLine,
ec),
(string ai, string ec) when string.IsNullOrWhiteSpace(ai) => ec,
(string ai, string ec) when string.IsNullOrWhiteSpace(ec) => ai,
(null, string ec) => ec,
(string ai, null) => ai,
_ => string.Empty
};
}
dotnet/src/Agents/Bedrock/Agents.Bedrock.csproj
+1
View File
@@ -33,6 +33,7 @@
<ItemGroup>
<ProjectReference Include="..\Abstractions\Agents.Abstractions.csproj" />
<ProjectReference Include="..\..\SemanticKernel.Core\SemanticKernel.Core.csproj" />
</ItemGroup>
<ItemGroup>
dotnet/src/Agents/Bedrock/BedrockAgent.cs
+51 -4
View File
@@ -4,6 +4,7 @@ using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Amazon.BedrockAgent;
@@ -20,6 +21,8 @@ namespace Microsoft.SemanticKernel.Agents.Bedrock;
/// </summary>
public sealed class BedrockAgent : Agent
{
private const string AdditionalInstructionsSessionAttributeName = "AdditionalInstructions";
/// <summary>
/// The client used to interact with the Bedrock Agent service.
/// </summary>
@@ -117,11 +120,18 @@ public sealed class BedrockAgent : Agent
() => new BedrockAgentThread(this.RuntimeClient),
cancellationToken).ConfigureAwait(false);
// Get the context contributions from the AIContextProviders.
#pragma warning disable SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var providersContext = await bedrockThread.AIContextProviders.ModelInvokingAsync(messages, cancellationToken).ConfigureAwait(false);
#pragma warning restore SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
// Ensure that the last message provided is a user message
string? message = this.ExtractUserMessage(messages.Last());
// Build session state with conversation history if needed
// Build session state with conversation history and override instructions if needed
SessionState sessionState = this.ExtractSessionState(messages);
var mergedAdditionalInstructions = MergeAdditionalInstructions(options?.AdditionalInstructions, providersContext.Instructions);
sessionState.PromptSessionAttributes = new() { [AdditionalInstructionsSessionAttributeName] = mergedAdditionalInstructions };
// Configure the agent request with the provided options
var invokeAgentRequest = this.ConfigureAgentRequest(options, () =>
@@ -346,11 +356,18 @@ public sealed class BedrockAgent : Agent
() => new BedrockAgentThread(this.RuntimeClient),
cancellationToken).ConfigureAwait(false);
// Get the context contributions from the AIContextProviders.
#pragma warning disable SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var providersContext = await bedrockThread.AIContextProviders.ModelInvokingAsync(messages, cancellationToken).ConfigureAwait(false);
#pragma warning restore SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
// Ensure that the last message provided is a user message
string? message = this.ExtractUserMessage(messages.Last());
// Build session state with conversation history if needed
// Build session state with conversation history and override instructions if needed
SessionState sessionState = this.ExtractSessionState(messages);
var mergedAdditionalInstructions = MergeAdditionalInstructions(options?.AdditionalInstructions, providersContext.Instructions);
sessionState.PromptSessionAttributes = new() { [AdditionalInstructionsSessionAttributeName] = mergedAdditionalInstructions };
// Configure the agent request with the provided options
var invokeAgentRequest = this.ConfigureAgentRequest(options, () =>
@@ -639,20 +656,35 @@ public sealed class BedrockAgent : Agent
async IAsyncEnumerable<StreamingChatMessageContent> InvokeInternal()
{
var combinedResponseMessageBuilder = new StringBuilder();
StreamingChatMessageContent? lastMessage = null;
// The Bedrock agent service has the same API for both streaming and non-streaming responses.
// We are invoking the same method as the non-streaming response with the streaming configuration set,
// and converting the chat message content to streaming chat message content.
await foreach (var chatMessageContent in this.InternalInvokeAsync(invokeAgentRequest, arguments, cancellationToken).ConfigureAwait(false))
{
await this.NotifyThreadOfNewMessage(thread, chatMessageContent, cancellationToken).ConfigureAwait(false);
yield return new StreamingChatMessageContent(chatMessageContent.Role, chatMessageContent.Content)
lastMessage = new StreamingChatMessageContent(chatMessageContent.Role, chatMessageContent.Content)
{
AuthorName = chatMessageContent.AuthorName,
ModelId = chatMessageContent.ModelId,
InnerContent = chatMessageContent.InnerContent,
Metadata = chatMessageContent.Metadata,
};
yield return lastMessage;
combinedResponseMessageBuilder.Append(chatMessageContent.Content);
}
// Build a combined message containing the text from all response parts
// to send to the thread.
var combinedMessage = new ChatMessageContent(AuthorRole.Assistant, combinedResponseMessageBuilder.ToString())
{
AuthorName = lastMessage?.AuthorName,
ModelId = lastMessage?.ModelId,
Metadata = lastMessage?.Metadata,
};
await this.NotifyThreadOfNewMessage(thread, combinedMessage, cancellationToken).ConfigureAwait(false);
}
}
@@ -726,4 +758,19 @@ public sealed class BedrockAgent : Agent
}
#endregion
private static string MergeAdditionalInstructions(string? optionsAdditionalInstructions, string? extensionsContext) =>
(optionsAdditionalInstructions, extensionsContext) switch
{
(string ai, string ec) when !string.IsNullOrWhiteSpace(ai) && !string.IsNullOrWhiteSpace(ec) => string.Concat(
ai,
Environment.NewLine,
Environment.NewLine,
ec),
(string ai, string ec) when string.IsNullOrWhiteSpace(ai) => ec,
(string ai, string ec) when string.IsNullOrWhiteSpace(ec) => ai,
(null, string ec) => ec,
(string ai, null) => ai,
_ => string.Empty
};
}
dotnet/src/Agents/Core/ChatCompletionAgent.cs
+24 -4
View File
@@ -73,6 +73,14 @@ public sealed class ChatCompletionAgent : ChatHistoryAgent
() => new ChatHistoryAgentThread(),
cancellationToken).ConfigureAwait(false);
var kernel = (options?.Kernel ?? this.Kernel).Clone();
// Get the context contributions from the AIContextProviders.
#pragma warning disable SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var providersContext = await chatHistoryAgentThread.AIContextProviders.ModelInvokingAsync(messages, cancellationToken).ConfigureAwait(false);
kernel.Plugins.AddFromAIContext(providersContext, "Tools");
#pragma warning restore SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
// Invoke Chat Completion with the updated chat history.
var chatHistory = new ChatHistory();
await foreach (var existingMessage in chatHistoryAgentThread.GetMessagesAsync(cancellationToken).ConfigureAwait(false))
@@ -91,8 +99,10 @@ public sealed class ChatCompletionAgent : ChatHistoryAgent
}
},
options?.KernelArguments,
options?.Kernel,
options?.AdditionalInstructions,
kernel,
options?.AdditionalInstructions == null ?
providersContext.Instructions :
string.Concat(options.AdditionalInstructions, Environment.NewLine, Environment.NewLine, providersContext.Instructions),
cancellationToken);
// Notify the thread of new messages and return them to the caller.
@@ -156,6 +166,14 @@ public sealed class ChatCompletionAgent : ChatHistoryAgent
() => new ChatHistoryAgentThread(),
cancellationToken).ConfigureAwait(false);
var kernel = (options?.Kernel ?? this.Kernel).Clone();
// Get the context contributions from the AIContextProviders.
#pragma warning disable SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var providersContext = await chatHistoryAgentThread.AIContextProviders.ModelInvokingAsync(messages, cancellationToken).ConfigureAwait(false);
kernel.Plugins.AddFromAIContext(providersContext, "Tools");
#pragma warning restore SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
// Invoke Chat Completion with the updated chat history.
var chatHistory = new ChatHistory();
await foreach (var existingMessage in chatHistoryAgentThread.GetMessagesAsync(cancellationToken).ConfigureAwait(false))
@@ -175,8 +193,10 @@ public sealed class ChatCompletionAgent : ChatHistoryAgent
}
},
options?.KernelArguments,
options?.Kernel,
options?.AdditionalInstructions,
kernel,
options?.AdditionalInstructions == null ?
providersContext.Instructions :
string.Concat(options.AdditionalInstructions, Environment.NewLine, Environment.NewLine, providersContext),
cancellationToken);
await foreach (var result in invokeResults.ConfigureAwait(false))
dotnet/src/Agents/OpenAI/Internal/AssistantRunOptionsFactory.cs
+7 -2
View File
@@ -1,5 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using OpenAI.Assistants;
@@ -10,12 +11,16 @@ namespace Microsoft.SemanticKernel.Agents.OpenAI.Internal;
/// </summary>
internal static class AssistantRunOptionsFactory
{
public static RunCreationOptions GenerateOptions(RunCreationOptions? defaultOptions, string? agentInstructions, RunCreationOptions? invocationOptions)
public static RunCreationOptions GenerateOptions(RunCreationOptions? defaultOptions, string? agentInstructions, RunCreationOptions? invocationOptions, string? threadExtensionsContext)
{
var additionalInstructions = string.Concat(
(invocationOptions?.AdditionalInstructions ?? defaultOptions?.AdditionalInstructions),
string.IsNullOrWhiteSpace(threadExtensionsContext) ? string.Empty : string.Concat(Environment.NewLine, Environment.NewLine, threadExtensionsContext));
RunCreationOptions runOptions =
new()
{
AdditionalInstructions = invocationOptions?.AdditionalInstructions ?? defaultOptions?.AdditionalInstructions,
AdditionalInstructions = additionalInstructions,
InstructionsOverride = invocationOptions?.InstructionsOverride ?? agentInstructions,
MaxOutputTokenCount = invocationOptions?.MaxOutputTokenCount ?? defaultOptions?.MaxOutputTokenCount,
MaxInputTokenCount = invocationOptions?.MaxInputTokenCount ?? defaultOptions?.MaxInputTokenCount,
dotnet/src/Agents/OpenAI/Internal/AssistantThreadActions.cs
+6 -2
View File
@@ -105,6 +105,7 @@ internal static class AssistantThreadActions
/// <param name="client">The assistant client</param>
/// <param name="threadId">The thread identifier</param>
/// <param name="invocationOptions">Options to utilize for the invocation</param>
/// <param name="providersAdditionalInstructions">Additional instructions from <see cref="AIContextProvider"/> instances to pass to the invoke method.</param>
/// <param name="logger">The logger to utilize (might be agent or channel scoped)</param>
/// <param name="kernel">The <see cref="Kernel"/> plugins and other state.</param>
/// <param name="arguments">Optional arguments to pass to the agents's invocation, including any <see cref="PromptExecutionSettings"/>.</param>
@@ -115,6 +116,7 @@ internal static class AssistantThreadActions
AssistantClient client,
string threadId,
RunCreationOptions? invocationOptions,
string? providersAdditionalInstructions,
ILogger logger,
Kernel kernel,
KernelArguments? arguments,
@@ -133,7 +135,7 @@ internal static class AssistantThreadActions
string? instructions = await agent.GetInstructionsAsync(kernel, arguments, cancellationToken).ConfigureAwait(false);
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(agent.RunOptions, instructions, invocationOptions);
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(agent.RunOptions, instructions, invocationOptions, providersAdditionalInstructions);
options.ToolsOverride.AddRange(tools);
@@ -335,6 +337,7 @@ internal static class AssistantThreadActions
/// <param name="threadId">The thread identifier</param>
/// <param name="messages">The receiver for the completed messages generated</param>
/// <param name="invocationOptions">Options to utilize for the invocation</param>
/// <param name="providersAdditionalInstructions">Additional instructions from <see cref="AIContextProvider"/> instances to pass to the invoke method.</param>
/// <param name="logger">The logger to utilize (might be agent or channel scoped)</param>
/// <param name="kernel">The <see cref="Kernel"/> plugins and other state.</param>
/// <param name="arguments">Optional arguments to pass to the agents's invocation, including any <see cref="PromptExecutionSettings"/>.</param>
@@ -350,6 +353,7 @@ internal static class AssistantThreadActions
string threadId,
IList<ChatMessageContent>? messages,
RunCreationOptions? invocationOptions,
string? providersAdditionalInstructions,
ILogger logger,
Kernel kernel,
KernelArguments? arguments,
@@ -361,7 +365,7 @@ internal static class AssistantThreadActions
string? instructions = await agent.GetInstructionsAsync(kernel, arguments, cancellationToken).ConfigureAwait(false);
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(agent.RunOptions, instructions, invocationOptions);
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(agent.RunOptions, instructions, invocationOptions, providersAdditionalInstructions);
options.ToolsOverride.AddRange(tools);
dotnet/src/Agents/OpenAI/OpenAIAssistantAgent.cs
+40 -12
View File
@@ -411,6 +411,14 @@ public sealed partial class OpenAIAssistantAgent : Agent
AdditionalInstructions = options?.AdditionalInstructions,
});
var kernel = (options?.Kernel ?? this.Kernel).Clone();
// Get the context contributions from the AIContextProviders.
#pragma warning disable SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var providersContext = await openAIAssistantAgentThread.AIContextProviders.ModelInvokingAsync(messages, cancellationToken).ConfigureAwait(false);
kernel.Plugins.AddFromAIContext(providersContext, "Tools");
#pragma warning restore SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var invokeResults = ActivityExtensions.RunWithActivityAsync(
() => ModelDiagnostics.StartAgentInvocationActivity(this.Id, this.GetDisplayName(), this.Description),
() => InternalInvokeAsync(),
@@ -423,8 +431,9 @@ public sealed partial class OpenAIAssistantAgent : Agent
this.Client,
openAIAssistantAgentThread.Id!,
internalOptions,
providersContext.Instructions,
this.Logger,
options?.Kernel ?? this.Kernel,
kernel,
options?.KernelArguments,
cancellationToken).ConfigureAwait(false))
{
@@ -496,7 +505,7 @@ public sealed partial class OpenAIAssistantAgent : Agent
async IAsyncEnumerable<ChatMessageContent> InternalInvokeAsync()
{
kernel ??= this.Kernel;
await foreach ((bool isVisible, ChatMessageContent message) in AssistantThreadActions.InvokeAsync(this, this.Client, threadId, options, this.Logger, kernel, arguments, cancellationToken).ConfigureAwait(false))
await foreach ((bool isVisible, ChatMessageContent message) in AssistantThreadActions.InvokeAsync(this, this.Client, threadId, options, null, this.Logger, kernel, arguments, cancellationToken).ConfigureAwait(false))
{
if (isVisible)
{
@@ -547,6 +556,14 @@ public sealed partial class OpenAIAssistantAgent : Agent
() => new OpenAIAssistantAgentThread(this.Client),
cancellationToken).ConfigureAwait(false);
var kernel = (options?.Kernel ?? this.Kernel).Clone();
// Get the context contributions from the AIContextProviders.
#pragma warning disable SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var providersContext = await openAIAssistantAgentThread.AIContextProviders.ModelInvokingAsync(messages, cancellationToken).ConfigureAwait(false);
kernel.Plugins.AddFromAIContext(providersContext, "Tools");
#pragma warning restore SKEXP0110 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
// Create options that use the RunCreationOptions from the options param if provided or
// falls back to creating a new RunCreationOptions if additional instructions is provided
// separately.
@@ -555,17 +572,28 @@ public sealed partial class OpenAIAssistantAgent : Agent
AdditionalInstructions = options?.AdditionalInstructions,
});
#pragma warning disable CS0618 // Type or member is obsolete
// Invoke the Agent with the thread that we already added our message to.
#pragma warning disable SKEXP0001 // ModelDiagnostics is marked experimental.
var newMessagesReceiver = new ChatHistory();
var invokeResults = this.InvokeStreamingAsync(
openAIAssistantAgentThread.Id!,
internalOptions,
options?.KernelArguments,
options?.Kernel ?? this.Kernel,
newMessagesReceiver,
var invokeResults = ActivityExtensions.RunWithActivityAsync(
() => ModelDiagnostics.StartAgentInvocationActivity(this.Id, this.GetDisplayName(), this.Description),
() => InternalInvokeStreamingAsync(),
cancellationToken);
#pragma warning restore CS0618 // Type or member is obsolete
#pragma warning restore SKEXP0001 // ModelDiagnostics is marked experimental.
IAsyncEnumerable<StreamingChatMessageContent> InternalInvokeStreamingAsync()
{
return AssistantThreadActions.InvokeStreamingAsync(
this,
this.Client,
openAIAssistantAgentThread.Id!,
newMessagesReceiver,
internalOptions,
providersContext.Instructions,
this.Logger,
kernel,
options?.KernelArguments,
cancellationToken);
}
// Return the chunks to the caller.
await foreach (var result in invokeResults.ConfigureAwait(false))
@@ -638,7 +666,7 @@ public sealed partial class OpenAIAssistantAgent : Agent
IAsyncEnumerable<StreamingChatMessageContent> InternalInvokeStreamingAsync()
{
kernel ??= this.Kernel;
return AssistantThreadActions.InvokeStreamingAsync(this, this.Client, threadId, messages, options, this.Logger, kernel, arguments, cancellationToken);
return AssistantThreadActions.InvokeStreamingAsync(this, this.Client, threadId, messages, options, providersAdditionalInstructions: null, this.Logger, kernel, arguments, cancellationToken);
}
}
dotnet/src/Agents/OpenAI/OpenAIAssistantChannel.cs
+2 -2
View File
@@ -36,7 +36,7 @@ internal sealed class OpenAIAssistantChannel(AssistantClient client, string thre
{
return ActivityExtensions.RunWithActivityAsync(
() => ModelDiagnostics.StartAgentInvocationActivity(agent.Id, agent.GetDisplayName(), agent.Description),
() => AssistantThreadActions.InvokeAsync(agent, this._client, this._threadId, invocationOptions: null, this.Logger, agent.Kernel, agent.Arguments, cancellationToken),
() => AssistantThreadActions.InvokeAsync(agent, this._client, this._threadId, invocationOptions: null, providersAdditionalInstructions: null, this.Logger, agent.Kernel, agent.Arguments, cancellationToken),
cancellationToken);
}
@@ -45,7 +45,7 @@ internal sealed class OpenAIAssistantChannel(AssistantClient client, string thre
{
return ActivityExtensions.RunWithActivityAsync(
() => ModelDiagnostics.StartAgentInvocationActivity(agent.Id, agent.GetDisplayName(), agent.Description),
() => AssistantThreadActions.InvokeStreamingAsync(agent, this._client, this._threadId, messages, invocationOptions: null, this.Logger, agent.Kernel, agent.Arguments, cancellationToken),
() => AssistantThreadActions.InvokeStreamingAsync(agent, this._client, this._threadId, messages, invocationOptions: null, providersAdditionalInstructions: null, this.Logger, agent.Kernel, agent.Arguments, cancellationToken),
cancellationToken);
}
dotnet/src/Agents/UnitTests/Agents.UnitTests.csproj
+1 -1
View File
@@ -8,7 +8,7 @@
<IsTestProject>true</IsTestProject>
<IsPackable>false</IsPackable>
<LangVersion>12</LangVersion>
<NoWarn>$(NoWarn);CA2007,CA1812,CA1861,CA1707,CA1063,CS0618,CS1591,VSTHRD111,SKEXP0001,SKEXP0050,SKEXP0110;OPENAI001</NoWarn>
<NoWarn>$(NoWarn);CA2007,CA1812,CA1861,CA1707,CA1063,CS0618,CS1591,VSTHRD111,SKEXP0001,SKEXP0050,SKEXP0110,SKEXP0130;OPENAI001</NoWarn>
</PropertyGroup>
<ItemGroup>
dotnet/src/Agents/UnitTests/Core/AgentThreadTests.cs
+132
View File
@@ -3,8 +3,11 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.ChatCompletion;
using Moq;
using Xunit;
namespace SemanticKernel.Agents.UnitTests.Core;
@@ -141,6 +144,135 @@ public class AgentThreadTests
Assert.Equal(0, thread.OnNewMessageInternalAsyncCount);
}
/// <summary>
/// Tests that the <see cref="AgentThread.OnResumeAsync(CancellationToken)"/> method throws an InvalidOperationException if the thread is not yet created.
/// </summary>
[Fact]
public async Task OnResumeShouldThrowIfThreadNotCreatedAsync()
{
// Arrange
var thread = new TestAgentThread();
// Act & Assert
await Assert.ThrowsAsync<InvalidOperationException>(() => thread.OnResumeAsync());
}
/// <summary>
/// Tests that the <see cref="AgentThread.OnResumeAsync(CancellationToken)"/> method throws an InvalidOperationException if the thread is deleted.
/// </summary>
[Fact]
public async Task OnResumeShouldThrowIfThreadDeletedAsync()
{
// Arrange
var thread = new TestAgentThread();
await thread.CreateAsync();
await thread.DeleteAsync();
// Act & Assert
await Assert.ThrowsAsync<InvalidOperationException>(() => thread.OnResumeAsync());
}
/// <summary>
/// Tests that the <see cref="AgentThread.OnSuspendAsync(CancellationToken)"/> method
/// calls each registered state part in turn.
/// </summary>
[Fact]
public async Task OnSuspendShouldCallOnSuspendOnRegisteredPartsAsync()
{
// Arrange.
var thread = new TestAgentThread();
var mockProvider = new Mock<AIContextProvider>();
thread.AIContextProviders.Add(mockProvider.Object);
await thread.CreateAsync();
// Act.
await thread.OnSuspendAsync();
// Assert.
mockProvider.Verify(x => x.SuspendingAsync("test-thread-id", It.IsAny<CancellationToken>()), Times.Once);
}
/// <summary>
/// Tests that the <see cref="AgentThread.OnResumeAsync(CancellationToken)"/> method
/// calls each registered state part in turn.
/// </summary>
[Fact]
public async Task OnResumeShouldCallOnResumeOnRegisteredPartsAsync()
{
// Arrange.
var thread = new TestAgentThread();
var mockProvider = new Mock<AIContextProvider>();
thread.AIContextProviders.Add(mockProvider.Object);
await thread.CreateAsync();
// Act.
await thread.OnResumeAsync();
// Assert.
mockProvider.Verify(x => x.ResumingAsync("test-thread-id", It.IsAny<CancellationToken>()), Times.Once);
}
/// <summary>
/// Tests that the <see cref="AgentThread.CreateAsync(CancellationToken)"/> method
/// calls each registered state parts in turn.
/// </summary>
[Fact]
public async Task CreateShouldCallOnThreadCreatedOnRegisteredPartsAsync()
{
// Arrange.
var thread = new TestAgentThread();
var mockProvider = new Mock<AIContextProvider>();
thread.AIContextProviders.Add(mockProvider.Object);
// Act.
await thread.CreateAsync();
// Assert.
mockProvider.Verify(x => x.ConversationCreatedAsync("test-thread-id", It.IsAny<CancellationToken>()), Times.Once);
}
/// <summary>
/// Tests that the <see cref="AgentThread.DeleteAsync(CancellationToken)"/> method
/// calls each registered state parts in turn.
/// </summary>
[Fact]
public async Task DeleteShouldCallOnThreadDeleteOnRegisteredPartsAsync()
{
// Arrange.
var thread = new TestAgentThread();
var mockProvider = new Mock<AIContextProvider>();
thread.AIContextProviders.Add(mockProvider.Object);
await thread.CreateAsync();
// Act.
await thread.DeleteAsync();
// Assert.
mockProvider.Verify(x => x.ConversationDeletingAsync("test-thread-id", It.IsAny<CancellationToken>()), Times.Once);
}
/// <summary>
/// Tests that the <see cref="AgentThread.OnNewMessageAsync(ChatMessageContent, CancellationToken)"/> method
/// calls each registered state part in turn.
/// </summary>
[Fact]
public async Task OnNewMessageShouldCallOnNewMessageOnRegisteredPartsAsync()
{
// Arrange.
var thread = new TestAgentThread();
var mockProvider = new Mock<AIContextProvider>();
thread.AIContextProviders.Add(mockProvider.Object);
var message = new ChatMessageContent(AuthorRole.User, "Test Message.");
await thread.CreateAsync();
// Act.
await thread.OnNewMessageAsync(message);
// Assert.
mockProvider.Verify(x => x.MessageAddingAsync("test-thread-id", It.Is<ChatMessage>(x => x.Text == "Test Message." && x.Role == ChatRole.User), It.IsAny<CancellationToken>()), Times.Once);
}
private sealed class TestAgentThread : AgentThread
{
public int CreateInternalAsyncCount { get; private set; }
dotnet/src/Agents/UnitTests/OpenAI/Internal/AssistantRunOptionsFactoryTests.cs
+39 -6
View File
@@ -1,4 +1,5 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents.OpenAI;
using Microsoft.SemanticKernel.Agents.OpenAI.Internal;
@@ -29,7 +30,7 @@ public class AssistantRunOptionsFactoryTests
};
// Act
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, null, null);
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, null, null, threadExtensionsContext: null);
// Assert
Assert.NotNull(options);
@@ -62,7 +63,7 @@ public class AssistantRunOptionsFactoryTests
};
// Act
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, "test", invocationOptions);
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, "test", invocationOptions, threadExtensionsContext: null);
// Assert
Assert.NotNull(options);
@@ -97,7 +98,7 @@ public class AssistantRunOptionsFactoryTests
};
// Act
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, null, invocationOptions);
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, null, invocationOptions, threadExtensionsContext: null);
// Assert
Assert.NotNull(options);
@@ -134,7 +135,7 @@ public class AssistantRunOptionsFactoryTests
};
// Act
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, null, invocationOptions);
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, null, invocationOptions, threadExtensionsContext: null);
// Assert
Assert.Equal(2, options.Metadata.Count);
@@ -163,7 +164,7 @@ public class AssistantRunOptionsFactoryTests
};
// Act
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, null, invocationOptions);
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, null, invocationOptions, threadExtensionsContext: null);
// Assert
Assert.Single(options.AdditionalMessages);
@@ -186,10 +187,42 @@ public class AssistantRunOptionsFactoryTests
};
// Act
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, null, null);
RunCreationOptions options = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, null, null, threadExtensionsContext: null);
// Assert
Assert.Equal(1024, options.MaxInputTokenCount);
Assert.Equal(4096, options.MaxOutputTokenCount);
}
/// <summary>
/// Verify run options generation with <see cref="OpenAIAssistantInvocationOptions"/> metadata.
/// </summary>
[Fact]
public void AssistantRunOptionsFactoryAdditionalInstructionsTest()
{
// Arrange
RunCreationOptions defaultOptions =
new()
{
ModelOverride = "gpt-anything",
Temperature = 0.5F,
MaxOutputTokenCount = 4096,
MaxInputTokenCount = 1024,
AdditionalInstructions = "DefaultInstructions"
};
RunCreationOptions invocationOptions =
new()
{
AdditionalInstructions = "OverrideInstructions",
};
// Act
RunCreationOptions optionsWithOverride = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, null, invocationOptions, threadExtensionsContext: "Context");
RunCreationOptions optionsWithoutOverride = AssistantRunOptionsFactory.GenerateOptions(defaultOptions, null, null, threadExtensionsContext: "Context");
// Assert
Assert.Equal($"OverrideInstructions{Environment.NewLine}{Environment.NewLine}Context", optionsWithOverride.AdditionalInstructions);
Assert.Equal($"DefaultInstructions{Environment.NewLine}{Environment.NewLine}Context", optionsWithoutOverride.AdditionalInstructions);
}
}
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/AgentFixture.cs
+2
View File
@@ -23,6 +23,8 @@ public abstract class AgentFixture : IAsyncLifetime
public abstract AgentThread CreatedServiceFailingAgentThread { get; }
public abstract AgentThread GetNewThread();
public abstract Task<ChatHistory> GetChatHistory();
public abstract Task DeleteThread(AgentThread thread);
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/AgentThreadConformance/BedrockAgentThreadTests.cs
+8 -6
View File
@@ -7,38 +7,40 @@ namespace SemanticKernel.IntegrationTests.Agents.CommonInterfaceConformance.Agen
public class BedrockAgentThreadTests() : AgentThreadTests(() => new BedrockAgentFixture())
{
[Fact(Skip = "Manual verification only")]
private const string ManualVerificationSkipReason = "This test is for manual verification.";
[Fact(Skip = ManualVerificationSkipReason)]
public override Task OnNewMessageWithServiceFailureThrowsAgentOperationExceptionAsync()
{
// The Bedrock agent does not support writing to a thread with OnNewMessage.
return Task.CompletedTask;
}
[Fact(Skip = "Manual verification only")]
[Fact(Skip = ManualVerificationSkipReason)]
public override Task DeletingThreadTwiceDoesNotThrowAsync()
{
return base.DeletingThreadTwiceDoesNotThrowAsync();
}
[Fact(Skip = "Manual verification only")]
[Fact(Skip = ManualVerificationSkipReason)]
public override Task UsingThreadAfterDeleteThrowsAsync()
{
return base.UsingThreadAfterDeleteThrowsAsync();
}
[Fact(Skip = "Manual verification only")]
[Fact(Skip = ManualVerificationSkipReason)]
public override Task DeleteThreadBeforeCreateThrowsAsync()
{
return base.DeleteThreadBeforeCreateThrowsAsync();
}
[Fact(Skip = "Manual verification only")]
[Fact(Skip = ManualVerificationSkipReason)]
public override Task UsingThreadBeforeCreateCreatesAsync()
{
return base.UsingThreadBeforeCreateCreatesAsync();
}
[Fact(Skip = "Manual verification only")]
[Fact(Skip = ManualVerificationSkipReason)]
public override Task DeleteThreadWithServiceFailureThrowsAgentOperationExceptionAsync()
{
return base.DeleteThreadWithServiceFailureThrowsAgentOperationExceptionAsync();
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/AgentWithAIContextProviderConformance/AgentWithAIContextProviderTests.cs
+162
View File
@@ -0,0 +1,162 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.SemanticKernel;
using Moq;
using Xunit;
namespace SemanticKernel.IntegrationTests.Agents.CommonInterfaceConformance.AgentWithStatePartConformance;
public abstract class AgentWithAIContextProviderTests<TFixture>(Func<TFixture> createAgentFixture) : IAsyncLifetime
where TFixture : AgentFixture
{
#pragma warning disable CS8618 // Non-nullable field must contain a non-null value when exiting constructor. Consider adding the 'required' modifier or declaring as nullable.
private TFixture _agentFixture;
#pragma warning restore CS8618 // Non-nullable field must contain a non-null value when exiting constructor. Consider adding the 'required' modifier or declaring as nullable.
protected TFixture Fixture => this._agentFixture;
[Fact]
public virtual async Task StatePartReceivesMessagesFromAgentAsync()
{
// Arrange
var mockStatePart = new Mock<AIContextProvider>() { CallBase = true };
mockStatePart.Setup(x => x.MessageAddingAsync(It.IsAny<string>(), It.IsAny<ChatMessage>(), It.IsAny<CancellationToken>()));
mockStatePart.Setup(x => x.ModelInvokingAsync(It.IsAny<ICollection<ChatMessage>>(), It.IsAny<CancellationToken>())).ReturnsAsync(new AIContext());
var agent = this.Fixture.Agent;
var agentThread = this.Fixture.GetNewThread();
try
{
agentThread.AIContextProviders.Add(mockStatePart.Object);
// Act
var inputMessage = "What is the capital of France?";
var asyncResults1 = agent.InvokeAsync(inputMessage, agentThread);
var result = await asyncResults1.FirstAsync();
// Assert
Assert.Contains("Paris", result.Message.Content);
mockStatePart.Verify(x => x.MessageAddingAsync(It.IsAny<string>(), It.Is<ChatMessage>(cm => cm.Text == inputMessage), It.IsAny<CancellationToken>()), Times.Once);
mockStatePart.Verify(x => x.MessageAddingAsync(It.IsAny<string>(), It.Is<ChatMessage>(cm => cm.Text == result.Message.Content), It.IsAny<CancellationToken>()), Times.Once);
}
finally
{
// Cleanup
await this.Fixture.DeleteThread(agentThread);
}
}
[Fact]
public virtual async Task StatePartReceivesMessagesFromAgentWhenStreamingAsync()
{
// Arrange
var mockStatePart = new Mock<AIContextProvider>() { CallBase = true };
mockStatePart.Setup(x => x.MessageAddingAsync(It.IsAny<string>(), It.IsAny<ChatMessage>(), It.IsAny<CancellationToken>()));
mockStatePart.Setup(x => x.ModelInvokingAsync(It.IsAny<ICollection<ChatMessage>>(), It.IsAny<CancellationToken>())).ReturnsAsync(new AIContext());
var agent = this.Fixture.Agent;
var agentThread = this.Fixture.GetNewThread();
try
{
agentThread.AIContextProviders.Add(mockStatePart.Object);
// Act
var inputMessage = "What is the capital of France?";
var asyncResults1 = agent.InvokeStreamingAsync(inputMessage, agentThread);
var results = await asyncResults1.ToListAsync();
// Assert
var responseMessage = string.Concat(results.Select(x => x.Message.Content));
Assert.Contains("Paris", responseMessage);
mockStatePart.Verify(x => x.MessageAddingAsync(It.IsAny<string>(), It.Is<ChatMessage>(cm => cm.Text == inputMessage), It.IsAny<CancellationToken>()), Times.Once);
mockStatePart.Verify(x => x.MessageAddingAsync(It.IsAny<string>(), It.Is<ChatMessage>(cm => cm.Text == responseMessage), It.IsAny<CancellationToken>()), Times.Once);
}
finally
{
// Cleanup
await this.Fixture.DeleteThread(agentThread);
}
}
[Fact]
public virtual async Task StatePartPreInvokeStateIsUsedByAgentAsync()
{
// Arrange
var mockStatePart = new Mock<AIContextProvider>() { CallBase = true };
mockStatePart.Setup(x => x.ModelInvokingAsync(It.IsAny<ICollection<ChatMessage>>(), It.IsAny<CancellationToken>())).ReturnsAsync(new AIContext { Instructions = "User name is Caoimhe" });
var agent = this.Fixture.Agent;
var agentThread = this.Fixture.GetNewThread();
try
{
agentThread.AIContextProviders.Add(mockStatePart.Object);
// Act
var inputMessage = "What is my name?.";
var asyncResults1 = agent.InvokeAsync(inputMessage, agentThread);
var result = await asyncResults1.FirstAsync();
// Assert
Assert.Contains("Caoimhe", result.Message.Content);
}
finally
{
// Cleanup
await this.Fixture.DeleteThread(agentThread);
}
}
[Fact]
public virtual async Task StatePartPreInvokeStateIsUsedByAgentWhenStreamingAsync()
{
// Arrange
var mockStatePart = new Mock<AIContextProvider>() { CallBase = true };
mockStatePart.Setup(x => x.ModelInvokingAsync(It.IsAny<ICollection<ChatMessage>>(), It.IsAny<CancellationToken>())).ReturnsAsync(new AIContext { Instructions = "User name is Caoimhe" });
var agent = this.Fixture.Agent;
var agentThread = this.Fixture.GetNewThread();
try
{
agentThread.AIContextProviders.Add(mockStatePart.Object);
// Act
var inputMessage = "What is my name?.";
var asyncResults1 = agent.InvokeStreamingAsync(inputMessage, agentThread);
var results = await asyncResults1.ToListAsync();
// Assert
var responseMessage = string.Concat(results.Select(x => x.Message.Content));
Assert.Contains("Caoimhe", responseMessage);
}
finally
{
// Cleanup
await this.Fixture.DeleteThread(agentThread);
}
}
public Task InitializeAsync()
{
this._agentFixture = createAgentFixture();
return this._agentFixture.InitializeAsync();
}
public Task DisposeAsync()
{
return this._agentFixture.DisposeAsync();
}
}
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/AgentWithAIContextProviderConformance/AzureAIAgentWithAIContextProviderTests.cs
+7
View File
@@ -0,0 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
namespace SemanticKernel.IntegrationTests.Agents.CommonInterfaceConformance.AgentWithStatePartConformance;
public class AzureAIAgentWithAIContextProviderTests() : AgentWithAIContextProviderTests<AzureAIAgentFixture>(() => new AzureAIAgentFixture())
{
}
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/AgentWithAIContextProviderConformance/BedrockAgentWithAIContextProviderTests.cs
+35
View File
@@ -0,0 +1,35 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Threading.Tasks;
using Xunit;
namespace SemanticKernel.IntegrationTests.Agents.CommonInterfaceConformance.AgentWithStatePartConformance;
public class BedrockAgentWithAIContextProviderTests() : AgentWithAIContextProviderTests<BedrockAgentFixture>(() => new BedrockAgentFixture())
{
private const string ManualVerificationSkipReason = "This test is for manual verification.";
[Fact(Skip = ManualVerificationSkipReason)]
public override Task StatePartReceivesMessagesFromAgentAsync()
{
return base.StatePartReceivesMessagesFromAgentAsync();
}
[Fact(Skip = ManualVerificationSkipReason)]
public override Task StatePartReceivesMessagesFromAgentWhenStreamingAsync()
{
return base.StatePartReceivesMessagesFromAgentWhenStreamingAsync();
}
[Fact(Skip = ManualVerificationSkipReason)]
public override Task StatePartPreInvokeStateIsUsedByAgentAsync()
{
return base.StatePartPreInvokeStateIsUsedByAgentAsync();
}
[Fact(Skip = ManualVerificationSkipReason)]
public override Task StatePartPreInvokeStateIsUsedByAgentWhenStreamingAsync()
{
return base.StatePartPreInvokeStateIsUsedByAgentWhenStreamingAsync();
}
}
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/AgentWithAIContextProviderConformance/ChatCompletionAgentWithAIContextProviderTests.cs
+7
View File
@@ -0,0 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
namespace SemanticKernel.IntegrationTests.Agents.CommonInterfaceConformance.AgentWithStatePartConformance;
public class ChatCompletionAgentWithAIContextProviderTests() : AgentWithAIContextProviderTests<ChatCompletionAgentFixture>(() => new ChatCompletionAgentFixture())
{
}
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/AgentWithAIContextProviderConformance/OpenAIAssistantAgentWithAIContextProviderTests.cs
+7
View File
@@ -0,0 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
namespace SemanticKernel.IntegrationTests.Agents.CommonInterfaceConformance.AgentWithStatePartConformance;
public class OpenAIAssistantAgentWithAIContextProviderTests() : AgentWithAIContextProviderTests<OpenAIAssistantAgentFixture>(() => new OpenAIAssistantAgentFixture())
{
}
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/AgentWithTextSearchProviderConformance/AgentWithTextSearchProvider.cs
+89
View File
@@ -0,0 +1,89 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.SemanticKernel.Data;
using Moq;
using Xunit;
using Xunit.Abstractions;
namespace SemanticKernel.IntegrationTests.Agents.CommonInterfaceConformance.AgentWithTextSearchBehaviorConformance;
public abstract class AgentWithTextSearchProvider<TFixture>(Func<TFixture> createAgentFixture, ITestOutputHelper output) : IAsyncLifetime
where TFixture : AgentFixture
{
#pragma warning disable CS8618 // Non-nullable field must contain a non-null value when exiting constructor. Consider adding the 'required' modifier or declaring as nullable.
private TFixture _agentFixture;
#pragma warning restore CS8618 // Non-nullable field must contain a non-null value when exiting constructor. Consider adding the 'required' modifier or declaring as nullable.
protected TFixture Fixture => this._agentFixture;
[Theory]
// Private data.
[InlineData("What was Acme Corp's revenue in Q1 2025?", "12.5", "Acme Corp reported a revenue of $12.5 million in Q1 2025.", "This represents a 15% increase from Q4 2024.")]
[InlineData("What was BetaTech Inc.'s stock price on May 1, 2025?", "45.67", "The stock price of BetaTech Inc. closed at $45.67 on May 1, 2025.", "It saw a 3% increase from the previous day.")]
[InlineData("What was Gamma Solutions' profit margin in FY 2024?", "22", "Gamma Solutions had a profit margin of 22% in FY 2024.", "This was an improvement from 18% in FY 2023.")]
[InlineData("What is DeltaSoft's market share in the enterprise software sector?", "35", "DeltaSoft holds a 35% market share in the enterprise software sector.", "Its closest competitor holds 25%.")]
// Generate knowledge.
[InlineData("When was the Eiffel Tower completed?", "1889", "The Eiffel Tower was completed in 1889.", "It is located in Paris, France.")]
[InlineData("At what temperature in Celsius does water boil?", "100", "Water boils at 212 degrees fahrenheit.", "Water boils at 100 degrees Celsius.")]
[InlineData("What did Einstein say about imagination?", "Imagination is more important than knowledge", "Albert Einstein said, 'Imagination is more important than knowledge.'")]
// Data contradicting Generate knowledge.
[InlineData("When was the Eiffel Tower completed?", "2005", "The Eiffel Tower was completed in 2005.", "It is located in Paris, France.")]
[InlineData("At what temperature in Celsius does water boil?", "150", "Water boils at 300 degrees fahrenheit.", "Water boils at 150 degrees Celsius.")]
// Check for citations
[InlineData("When was the Eiffel Tower completed?", "http://mydata.mycompany.com/dataset2", "It is located in Paris, France.", "The Eiffel Tower was completed in 1889.")]
[InlineData("What was Gamma Solutions' profit margin in FY 2024?", "http://mydata.mycompany.com/dataset1", "Gamma Solutions had a profit margin of 22% in FY 2024.", "This was an improvement from 18% in FY 2023.")]
[InlineData("When was the Eiffel Tower completed?", "DataSet2", "It is located in Paris, France.", "The Eiffel Tower was completed in 1889.")]
[InlineData("What was Gamma Solutions' profit margin in FY 2024?", "DataSet1", "Gamma Solutions had a profit margin of 22% in FY 2024.", "This was an improvement from 18% in FY 2023.")]
public async Task TextSearchBehaviorStateIsUsedByAgentInternalAsync(string question, string expectedResult, params string[] ragResults)
{
// Arrange
ragResults.Select(x => new TextSearchResult(x)).ToAsyncEnumerable();
var mockTextSearch = new Mock<ITextSearch>();
mockTextSearch.Setup(x => x.GetTextSearchResultsAsync(
It.IsAny<string>(),
It.IsAny<TextSearchOptions>(),
It.IsAny<CancellationToken>()))
.ReturnsAsync(
new KernelSearchResults<TextSearchResult>(ragResults.Select((x, i) => new TextSearchResult(x) { Name = $"DataSet{i + 1}", Link = $"http://mydata.mycompany.com/dataset{i + 1}" }).ToAsyncEnumerable()));
var textSearchBehavior = new TextSearchProvider(mockTextSearch.Object);
var agent = this.Fixture.Agent;
var agentThread = this.Fixture.GetNewThread();
try
{
agentThread.AIContextProviders.Add(textSearchBehavior);
// Act
var inputMessage = question;
var asyncResults1 = agent.InvokeAsync(inputMessage, agentThread);
var result = await asyncResults1.FirstAsync();
// Assert
output.WriteLine(result.Message.Content);
Assert.Contains(expectedResult, result.Message.Content, StringComparison.OrdinalIgnoreCase);
}
finally
{
// Cleanup
await this.Fixture.DeleteThread(agentThread);
}
}
public Task InitializeAsync()
{
this._agentFixture = createAgentFixture();
return this._agentFixture.InitializeAsync();
}
public Task DisposeAsync()
{
return this._agentFixture.DisposeAsync();
}
}
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/AgentWithTextSearchProviderConformance/AzureAIAgentWithTextSearchProviderTests.cs
+9
View File
@@ -0,0 +1,9 @@
// Copyright (c) Microsoft. All rights reserved.
using Xunit.Abstractions;
namespace SemanticKernel.IntegrationTests.Agents.CommonInterfaceConformance.AgentWithTextSearchBehaviorConformance;
public class AzureAIAgentWithTextSearchProviderTests(ITestOutputHelper output) : AgentWithTextSearchProvider<AzureAIAgentFixture>(() => new AzureAIAgentFixture(), output)
{
}
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/AgentWithTextSearchProviderConformance/ChatCompletionAgentWithTextSearchProviderTests.cs
+9
View File
@@ -0,0 +1,9 @@
// Copyright (c) Microsoft. All rights reserved.
using Xunit.Abstractions;
namespace SemanticKernel.IntegrationTests.Agents.CommonInterfaceConformance.AgentWithTextSearchBehaviorConformance;
public class ChatCompletionAgentWithTextSearchProviderTests(ITestOutputHelper output) : AgentWithTextSearchProvider<ChatCompletionAgentFixture>(() => new ChatCompletionAgentFixture(), output)
{
}
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/AgentWithTextSearchProviderConformance/OpenAIAssistantAgentWithTextSearchProviderTests.cs
+9
View File
@@ -0,0 +1,9 @@
// Copyright (c) Microsoft. All rights reserved.
using Xunit.Abstractions;
namespace SemanticKernel.IntegrationTests.Agents.CommonInterfaceConformance.AgentWithTextSearchBehaviorConformance;
public class OpenAIAssistantAgentWithTextSearchProviderTests(ITestOutputHelper output) : AgentWithTextSearchProvider<OpenAIAssistantAgentFixture>(() => new OpenAIAssistantAgentFixture(), output)
{
}
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/AzureAIAgentFixture.cs
+7
View File
@@ -30,6 +30,8 @@ public class AzureAIAgentFixture : AgentFixture
private AzureAIAgentThread? _serviceFailingAgentThread;
private AzureAIAgentThread? _createdServiceFailingAgentThread;
public PersistentAgentsClient AgentsClient => this._agentsClient!;
public override Agent Agent => this._agent!;
public override AgentThread AgentThread => this._thread!;
@@ -40,6 +42,11 @@ public class AzureAIAgentFixture : AgentFixture
public override AgentThread CreatedServiceFailingAgentThread => this._createdServiceFailingAgentThread!;
public override AgentThread GetNewThread()
{
return new AzureAIAgentThread(this._agentsClient!);
}
public override async Task<ChatHistory> GetChatHistory()
{
var chatHistory = new ChatHistory();
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/BedrockAgentFixture.cs
+6 -1
View File
@@ -18,7 +18,7 @@ using Xunit;
namespace SemanticKernel.IntegrationTests.Agents.CommonInterfaceConformance;
internal sealed class BedrockAgentFixture : AgentFixture, IAsyncDisposable
public sealed class BedrockAgentFixture : AgentFixture, IAsyncDisposable
{
private readonly IConfigurationRoot _configuration = new ConfigurationBuilder()
.AddJsonFile(path: "testsettings.json", optional: true, reloadOnChange: true)
@@ -47,6 +47,11 @@ internal sealed class BedrockAgentFixture : AgentFixture, IAsyncDisposable
public override AgentThread CreatedServiceFailingAgentThread => this._createdServiceFailingAgentThread!;
public override AgentThread GetNewThread()
{
return new BedrockAgentThread(this._runtimeClient);
}
public override async Task DeleteThread(AgentThread thread)
{
await this._runtimeClient!.EndSessionAsync(new EndSessionRequest() { SessionIdentifier = thread.Id });
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/ChatCompletionAgentFixture.cs
+5
View File
@@ -36,6 +36,11 @@ public class ChatCompletionAgentFixture : AgentFixture
public override AgentThread CreatedServiceFailingAgentThread => null!;
public override AgentThread GetNewThread()
{
return new ChatHistoryAgentThread();
}
public override async Task<ChatHistory> GetChatHistory()
{
var chatHistory = new ChatHistory();
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/InvokeConformance/BedrockAgentInvokeTests.cs
+9 -9
View File
@@ -1,6 +1,5 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.SemanticKernel;
@@ -11,7 +10,9 @@ namespace SemanticKernel.IntegrationTests.Agents.CommonInterfaceConformance.Invo
public class BedrockAgentInvokeTests() : InvokeTests(() => new BedrockAgentFixture())
{
[Fact(Skip = "This test is for manual verification.")]
private const string ManualVerificationSkipReason = "This test is for manual verification.";
[Fact(Skip = ManualVerificationSkipReason)]
public override async Task ConversationMaintainsHistoryAsync()
{
var q1 = "What is the capital of France.";
@@ -32,7 +33,7 @@ public class BedrockAgentInvokeTests() : InvokeTests(() => new BedrockAgentFixtu
//Assert.Contains("Eiffel", result2.Message.Content);
}
[Fact(Skip = "This test is for manual verification.")]
[Fact(Skip = ManualVerificationSkipReason)]
public override async Task InvokeReturnsResultAsync()
{
var agent = this.Fixture.Agent;
@@ -48,7 +49,7 @@ public class BedrockAgentInvokeTests() : InvokeTests(() => new BedrockAgentFixtu
//Assert.Contains("Paris", firstResult.Message.Content);
}
[Fact(Skip = "This test is for manual verification.")]
[Fact(Skip = ManualVerificationSkipReason)]
public override async Task InvokeWithoutThreadCreatesThreadAsync()
{
var agent = this.Fixture.Agent;
@@ -66,20 +67,19 @@ public class BedrockAgentInvokeTests() : InvokeTests(() => new BedrockAgentFixtu
await this.Fixture.DeleteThread(firstResult.Thread);
}
[Fact(Skip = "This test is for manual verification.")]
[Fact(Skip = "The BedrockAgent does not support invoking without a message.")]
public override Task InvokeWithoutMessageCreatesThreadAsync()
{
// The Bedrock agent does not support invoking without a message.
return Assert.ThrowsAsync<InvalidOperationException>(async () => await base.InvokeWithoutThreadCreatesThreadAsync());
return base.InvokeWithoutMessageCreatesThreadAsync();
}
[Fact(Skip = "This test is for manual verification.")]
[Fact(Skip = "The BedrockAgent does not yet support plugins")]
public override Task MultiStepInvokeWithPluginAndArgOverridesAsync()
{
return base.MultiStepInvokeWithPluginAndArgOverridesAsync();
}
[Fact(Skip = "This test is for manual verification.")]
[Fact(Skip = "The BedrockAgent does not yet support plugins")]
public override Task InvokeWithPluginNotifiesForAllMessagesAsync()
{
return base.InvokeWithPluginNotifiesForAllMessagesAsync();
dotnet/src/IntegrationTests/Agents/CommonInterfaceConformance/OpenAIAssistantAgentFixture.cs
+7
View File
@@ -34,6 +34,8 @@ public class OpenAIAssistantAgentFixture : AgentFixture
private OpenAIAssistantAgentThread? _serviceFailingAgentThread;
private OpenAIAssistantAgentThread? _createdServiceFailingAgentThread;
public AssistantClient AssistantClient => this._assistantClient!;
public override Agent Agent => this._agent!;
public override AgentThread AgentThread => this._thread!;
@@ -44,6 +46,11 @@ public class OpenAIAssistantAgentFixture : AgentFixture
public override AgentThread CreatedServiceFailingAgentThread => this._createdServiceFailingAgentThread!;
public override AgentThread GetNewThread()
{
return new OpenAIAssistantAgentThread(this._assistantClient!);
}
public override async Task<ChatHistory> GetChatHistory()
{
var chatHistory = new ChatHistory();
dotnet/src/IntegrationTests/IntegrationTests.csproj
+1 -1
View File
@@ -5,7 +5,7 @@
<TargetFramework>net8.0</TargetFramework>
<IsTestProject>true</IsTestProject>
<IsPackable>false</IsPackable>
<NoWarn>$(NoWarn);CA2007,CA1861,VSTHRD111,SKEXP0001,SKEXP0010,SKEXP0040,SKEXP0050,SKEXP0060,SKEXP0070,SKEXP0080,SKEXP0110,OPENAI001,MEVD9000</NoWarn>
<NoWarn>$(NoWarn);CA2007,CA1861,VSTHRD111,SKEXP0001,SKEXP0010,SKEXP0040,SKEXP0050,SKEXP0060,SKEXP0070,SKEXP0080,SKEXP0110,SKEXP0130,OPENAI001,MEVD9000</NoWarn>
<UserSecretsId>b7762d10-e29b-4bb1-8b74-b6d69a667dd4</UserSecretsId>
</PropertyGroup>
<ItemGroup>
dotnet/src/IntegrationTests/Memory/Mem0ProviderTests.cs
+169
View File
@@ -0,0 +1,169 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Configuration;
using Microsoft.SemanticKernel.Memory;
using SemanticKernel.IntegrationTests.TestSettings;
using Xunit;
namespace SemanticKernel.IntegrationTests.Memory;
/// <summary>
/// Contains tests for the <see cref="Mem0Provider"/> class.
/// </summary>
public class Mem0ProviderTests : IDisposable
{
// If null, all tests will be enabled
private const string SkipReason = "Requires a Mem0 service configured";
private readonly HttpClient _httpClient;
private bool _disposedValue;
public Mem0ProviderTests()
{
IConfigurationRoot configuration = new ConfigurationBuilder()
.AddJsonFile(path: "testsettings.json", optional: false, reloadOnChange: true)
.AddJsonFile(path: "testsettings.development.json", optional: true, reloadOnChange: true)
.AddEnvironmentVariables()
.AddUserSecrets<Mem0ProviderTests>()
.Build();
var mem0Settings = configuration.GetRequiredSection("Mem0").Get<Mem0Configuration>()!;
this._httpClient = new HttpClient();
this._httpClient.BaseAddress = new Uri(mem0Settings.ServiceUri);
this._httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Token", mem0Settings.ApiKey);
}
[Fact(Skip = SkipReason)]
public async Task CanAddAndRetrieveMemoriesAsync()
{
// Arrange
var question = new ChatMessage(ChatRole.User, "What is my name?");
var input = new ChatMessage(ChatRole.User, "Hello, my name is Caoimhe.");
var sut = new Mem0Provider(this._httpClient, new() { ThreadId = "test-thread-id", UserId = "test-user-id", ScopeToPerOperationThreadId = true });
await sut.ClearStoredMemoriesAsync();
var answerBeforeAdding = await sut.ModelInvokingAsync([question]);
Assert.DoesNotContain("Caoimhe", answerBeforeAdding.Instructions);
// Act
await sut.MessageAddingAsync("test-thread-id", input);
await sut.MessageAddingAsync("test-thread-id", question);
var answerAfterAdding = await sut.ModelInvokingAsync([question]);
await sut.ClearStoredMemoriesAsync();
var answerAfterClearing = await sut.ModelInvokingAsync([question]);
// Assert
Assert.Contains("Caoimhe", answerAfterAdding.Instructions);
Assert.DoesNotContain("Caoimhe", answerAfterClearing.Instructions);
}
[Fact(Skip = SkipReason)]
public async Task CanAddAndRetrieveAgentMemoriesAsync()
{
// Arrange
var question = new ChatMessage(ChatRole.User, "What is your name?");
var input = new ChatMessage(ChatRole.Assistant, "Hello, I'm a friendly assistant and my name is Caoimhe.");
var sut = new Mem0Provider(this._httpClient, new() { AgentId = "test-agent-id" });
await sut.ClearStoredMemoriesAsync();
var answerBeforeAdding = await sut.ModelInvokingAsync([question]);
Assert.DoesNotContain("Caoimhe", answerBeforeAdding.Instructions);
// Act
await sut.MessageAddingAsync("test-thread-id", input);
await sut.MessageAddingAsync("test-thread-id", question);
var answerAfterAdding = await sut.ModelInvokingAsync([question]);
await sut.ClearStoredMemoriesAsync();
var answerAfterClearing = await sut.ModelInvokingAsync([question]);
// Assert
Assert.Contains("Caoimhe", answerAfterAdding.Instructions);
Assert.DoesNotContain("Caoimhe", answerAfterClearing.Instructions);
}
[Fact(Skip = SkipReason)]
public async Task DoesNotLeakMessagesAcrossScopesAsync()
{
// Arrange
var question = new ChatMessage(ChatRole.User, "What is your name?");
var input = new ChatMessage(ChatRole.Assistant, "I'm an AI tutor with a personality. My name is Caoimhe.");
var sut1 = new Mem0Provider(this._httpClient, new() { AgentId = "test-agent-id-1" });
var sut2 = new Mem0Provider(this._httpClient, new() { AgentId = "test-agent-id-2" });
await sut1.ClearStoredMemoriesAsync();
await sut2.ClearStoredMemoriesAsync();
var answerBeforeAdding1 = await sut1.ModelInvokingAsync([question]);
var answerBeforeAdding2 = await sut2.ModelInvokingAsync([question]);
Assert.DoesNotContain("Caoimhe", answerBeforeAdding1.Instructions);
Assert.DoesNotContain("Caoimhe", answerBeforeAdding2.Instructions);
// Act
await sut1.MessageAddingAsync("test-thread-id-1", input);
var answerAfterAdding = await sut1.ModelInvokingAsync([question]);
await sut2.MessageAddingAsync("test-thread-id-2", question);
var answerAfterAddingOnOtherScope = await sut2.ModelInvokingAsync([question]);
// Assert
Assert.Contains("Caoimhe", answerAfterAdding.Instructions);
Assert.DoesNotContain("Caoimhe", answerAfterAddingOnOtherScope.Instructions);
// Cleanup.
await sut1.ClearStoredMemoriesAsync();
await sut2.ClearStoredMemoriesAsync();
}
[Fact(Skip = SkipReason)]
public async Task DoesNotWorkWithMultiplePerOperationThreadsAsync()
{
// Arrange
var input = new ChatMessage(ChatRole.User, "Hello, my name is Caoimhe.");
var sut = new Mem0Provider(this._httpClient, new() { UserId = "test-user-id", ScopeToPerOperationThreadId = true });
await sut.ClearStoredMemoriesAsync();
// Act & Assert
await sut.ConversationCreatedAsync("test-thread-id-1");
await Assert.ThrowsAsync<InvalidOperationException>(() => sut.ConversationCreatedAsync("test-thread-id-2"));
await sut.MessageAddingAsync("test-thread-id-1", input);
await Assert.ThrowsAsync<InvalidOperationException>(() => sut.MessageAddingAsync("test-thread-id-2", input));
// Cleanup
await sut.ClearStoredMemoriesAsync();
}
protected virtual void Dispose(bool disposing)
{
if (!this._disposedValue)
{
if (disposing)
{
this._httpClient.Dispose();
}
this._disposedValue = true;
}
}
public void Dispose()
{
this.Dispose(disposing: true);
GC.SuppressFinalize(this);
}
}
dotnet/src/IntegrationTests/Memory/WhiteboardProviderTests.cs
+166
View File
@@ -0,0 +1,166 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Azure.AI.OpenAI;
using Azure.Identity;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Configuration;
using Microsoft.SemanticKernel.Memory;
using SemanticKernel.IntegrationTests.TestSettings;
using Xunit;
using Xunit.Abstractions;
namespace SemanticKernel.IntegrationTests.Memory;
/// <summary>
/// Contains tests for the <see cref="WhiteboardProvider"/> class.
/// </summary>
public class WhiteboardProviderTests
{
private readonly IConfigurationRoot _configuration = new ConfigurationBuilder()
.AddJsonFile(path: "testsettings.json", optional: true, reloadOnChange: true)
.AddJsonFile(path: "testsettings.development.json", optional: true, reloadOnChange: true)
.AddEnvironmentVariables()
.AddUserSecrets<WhiteboardProviderTests>()
.Build();
private readonly ITestOutputHelper _output;
private readonly IChatClient _chatClient;
public WhiteboardProviderTests(ITestOutputHelper output)
{
this._output = output;
AzureOpenAIConfiguration configuration = this._configuration.GetSection("AzureOpenAI").Get<AzureOpenAIConfiguration>()!;
this._chatClient = new AzureOpenAIClient(new Uri(configuration.Endpoint), new AzureCliCredential())
.GetChatClient(configuration.ChatDeploymentName)
.AsIChatClient();
}
[Fact]
public Task AddsRequirementToWhiteboardAsync()
{
return this.CanAddMessagesToWhiteboardAsync(
new[]
{
new ChatMessage(ChatRole.User, "Hello") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.Assistant, "Hello, how can I help you?") { AuthorName = "Copilot" },
new ChatMessage(ChatRole.User, "I want to create a presentation") { AuthorName = "Siobhan" },
},
new string[][] { new string[] { "REQUIREMENT", "presentation" } });
}
[Fact]
public Task AddsRequirementsAndProposalToWhiteboardAsync()
{
return this.CanAddMessagesToWhiteboardAsync(
new[]
{
new ChatMessage(ChatRole.User, "I want to create a presentation") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.Assistant, "What would you like it to be about?") { AuthorName = "Copilot" },
new ChatMessage(ChatRole.User, "I want to feature our top 3 customers.") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.User, "I want it to be professional looking.") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.User, "I want a grey colour scheme.") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.Assistant, "Would you like me to create a presentation with Contoso, Northwind and Adventureworks? I'll make it professional with a grey colour scheme.") { AuthorName = "Copilot" },
},
new string[][]
{
new string[] { "REQUIREMENT", "presentation" },
new string[] { "REQUIREMENT", "customers" },
new string[] { "PROPOSAL", "Contoso", "Northwind" }
});
}
[Fact]
public Task AddsDecisionToWhiteboardAsync()
{
return this.CanAddMessagesToWhiteboardAsync(
new[]
{
new ChatMessage(ChatRole.User, "I want to create a presentation") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.Assistant, "What would you like it to be about?") { AuthorName = "Copilot" },
new ChatMessage(ChatRole.User, "I want to feature our top 3 customers.") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.User, "I want it to be professional looking.") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.User, "I want a grey colour scheme.") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.Assistant, "Would you like me to create a presentation with Contoso, Northwind and Adventureworks? I'll make it professional with a grey colour scheme.") { AuthorName = "Copilot" },
new ChatMessage(ChatRole.User, "That sounds good, let's to that.") { AuthorName = "Siobhan" },
},
new string[][]
{
new string[] { "DECISION", "presentation", "Contoso", "professional" }
});
}
[Fact]
public Task AddsProposalToWhiteboardAsync()
{
return this.CanAddMessagesToWhiteboardAsync(
new[]
{
new ChatMessage(ChatRole.User, "I am looking to create a VM") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.User, "It should be in Europe") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.User, "It should have 16GB or RAM") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.User, "It should have 4 cores") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.Assistant, "OK, shall I create a VM for you in Europe with 16GB of RAM, 4 cores and with the name `VM-Europe`?") { AuthorName = "Copilot" },
},
new string[][]
{
new string[] { "PROPOSAL", "Europe", "16GB", "4", "VM", "VM-Europe" }
});
}
[Fact]
public Task AddsActionToWhiteboardAsync()
{
return this.CanAddMessagesToWhiteboardAsync(
new[]
{
new ChatMessage(ChatRole.User, "I am looking to create a VM") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.Assistant, "I need you to give me the required location, amount of RAM you need, and number of cores required.") { AuthorName = "Copilot" },
new ChatMessage(ChatRole.User, "It should be in Europe") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.User, "It should have 16GB or RAM") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.User, "It should have 4 cores") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.Assistant, "OK, shall I create a VM for you in Europe with 16GB of RAM, 4 cores and with the name `VM-Europe`?") { AuthorName = "Copilot" },
new ChatMessage(ChatRole.User, "Yes, please go ahead and create that.") { AuthorName = "Siobhan" },
new ChatMessage(ChatRole.Assistant, "OK, I've created the VM for you.") { AuthorName = "Copilot" },
},
new string[][]
{
new string[] { "ACTION", "Europe", "16GB", "4", "VM" }
});
}
private async Task CanAddMessagesToWhiteboardAsync(ChatMessage[] chatMessages, string[][] expectedWhiteboardContent)
{
// Arrange
var WhiteboardProvider = new WhiteboardProvider(this._chatClient);
// Act
foreach (var chatMessage in chatMessages)
{
await WhiteboardProvider.MessageAddingAsync(null, chatMessage);
}
// Assert
await WhiteboardProvider.WhenProcessingCompleteAsync();
var aiContextAdditions = await WhiteboardProvider.ModelInvokingAsync(new List<ChatMessage> { new(ChatRole.User, string.Empty) });
var whiteboardContent = aiContextAdditions.Instructions!;
this._output.WriteLine(string.Join(Environment.NewLine, whiteboardContent));
var whiteboardLines = whiteboardContent.Split('\n');
foreach (var expectedContent in expectedWhiteboardContent)
{
bool foundAllInOneLine = false;
foreach (var line in whiteboardLines)
{
bool foundAll = expectedContent.All(line.Contains);
foundAllInOneLine = foundAllInOneLine || foundAll;
}
Assert.True(foundAllInOneLine, $"Expected content '{string.Join(", ", expectedContent)}' not found in whiteboard content.");
}
}
}
dotnet/src/IntegrationTests/TestSettings/Mem0Configuration.cs
+13
View File
@@ -0,0 +1,13 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Diagnostics.CodeAnalysis;
namespace SemanticKernel.IntegrationTests.TestSettings;
[SuppressMessage("Performance", "CA1812:Internal class that is apparently never instantiated",
Justification = "Configuration classes are instantiated through IConfiguration.")]
internal sealed class Mem0Configuration
{
public string ServiceUri { get; init; } = string.Empty;
public string ApiKey { get; init; } = string.Empty;
}
dotnet/src/IntegrationTests/testsettings.json
+4
View File
@@ -131,5 +131,9 @@
"BedrockAgent": {
"AgentResourceRoleArn": "",
"FoundationModel": "anthropic.claude-3-haiku-20240307-v1:0"
},
"Mem0": {
"ServiceUri": "https://api.mem0.ai",
"ApiKey": ""
}
}
dotnet/src/InternalUtilities/samples/InternalUtilities/TestConfiguration.cs
+7
View File
@@ -54,6 +54,7 @@ public sealed class TestConfiguration
public static CrewAIConfig CrewAI => LoadSection<CrewAIConfig>();
public static BedrockConfig Bedrock => LoadSection<BedrockConfig>();
public static BedrockAgentConfig BedrockAgent => LoadSection<BedrockAgentConfig>();
public static Mem0Config Mem0 => LoadSection<Mem0Config>();
public static IConfiguration GetSection(string caller)
{
@@ -357,4 +358,10 @@ public sealed class TestConfiguration
public string FoundationModel { get; set; }
public string? KnowledgeBaseId { get; set; }
}
public class Mem0Config
{
public string? BaseAddress { get; set; }
public string ApiKey { get; set; }
}
}
dotnet/src/SemanticKernel.Abstractions/Memory/AIContext.cs
+33
View File
@@ -0,0 +1,33 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using Microsoft.Extensions.AI;
namespace Microsoft.SemanticKernel;
/// <summary>
/// A class containing any context that should be provided to the AI model
/// as supplied by an <see cref="AIContextProvider"/>.
/// </summary>
/// <remarks>
/// Each <see cref="AIContextProvider"/> has the ability to provide its own context for each invocation.
/// The <see cref="AIContext"/> class contains the additional context supplied by the <see cref="AIContextProvider"/>.
/// This context will be combined with context supplied by other providers before being passed to the AI model.
/// This context is per invocation, and will not be stored as part of the chat history.
/// </remarks>
[Experimental("SKEXP0130")]
public sealed class AIContext
{
/// <summary>
/// Any instructions to pass to the AI model in addition to any other prompts
/// that it may already have (in the case of an agent), or chat history that may
/// already exist.
/// </summary>
public string? Instructions { get; set; }
/// <summary>
/// A list of functions/tools to make available to the AI model for the current invocation.
/// </summary>
public IList<AIFunction> AIFunctions { get; set; } = [];
}
dotnet/src/SemanticKernel.Abstractions/Memory/AIContextProvider.cs
+106
View File
@@ -0,0 +1,106 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
namespace Microsoft.SemanticKernel;
/// <summary>
/// Base class for all AI context providers.
/// </summary>
/// <remarks>
/// An AI context provider is a component that can be used to enhance the AI's context management.
/// It can listen to changes in the conversation, provide additional context to
/// the AI model just before invocation and supply additional tools for function invocation.
/// </remarks>
[Experimental("SKEXP0130")]
public abstract class AIContextProvider
{
/// <summary>
/// Called just after a new conversation/thread is created.
/// </summary>
/// <remarks>
/// Implementers can use this method to do any operations required at the creation of a new conversation/thread.
/// For example, checking long term storage for any data that is relevant to the current session based on the input text.
/// </remarks>
/// <param name="conversationId">The ID of the new conversation/thread, if the conversation/thread has an ID.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>A task that completes when the context has been loaded.</returns>
public virtual Task ConversationCreatedAsync(string? conversationId, CancellationToken cancellationToken = default)
{
return Task.CompletedTask;
}
/// <summary>
/// Called just before a message is added to the chat by any participant.
/// </summary>
/// <remarks>
/// Inheritors can use this method to update their context based on the new message.
/// </remarks>
/// <param name="conversationId">The ID of the conversation/thread for the new message, if the conversation/thread has an ID.</param>
/// <param name="newMessage">The new message.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>A task that completes when the context has been updated.</returns>
public virtual Task MessageAddingAsync(string? conversationId, ChatMessage newMessage, CancellationToken cancellationToken = default)
{
return Task.CompletedTask;
}
/// <summary>
/// Called just before a conversation/thread is deleted.
/// </summary>
/// <remarks>
/// Implementers can use this method to do any operations required before a conversation/thread is deleted.
/// For example, storing the context to long term storage.
/// </remarks>
/// <param name="conversationId">The ID of the conversation/thread that will be deleted, if the conversation/thread has an ID.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>A task that completes when the context has been saved.</returns>
public virtual Task ConversationDeletingAsync(string? conversationId, CancellationToken cancellationToken = default)
{
return Task.CompletedTask;
}
/// <summary>
/// Called just before the Model/Agent/etc. is invoked
/// Implementers can load any additional context required at this time,
/// and they should return any context that should be passed to the Model/Agent/etc.
/// </summary>
/// <param name="newMessages">The most recent messages that the Model/Agent/etc. is being invoked with.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>A task that completes when the context has been rendered and returned.</returns>
public abstract Task<AIContext> ModelInvokingAsync(ICollection<ChatMessage> newMessages, CancellationToken cancellationToken = default);
/// <summary>
/// Called when the current conversion is temporarily suspended and any state should be saved.
/// </summary>
/// <param name="conversationId">The ID of the current conversation/thread, if the conversation/thread has an ID.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>An async task.</returns>
/// <remarks>
/// In a service that hosts an agent, that is invoked via calls to the service, this might be at the end of each service call.
/// In a client application, this might be when the user closes the chat window or the application.
/// </remarks>
public virtual Task SuspendingAsync(string? conversationId, CancellationToken cancellationToken = default)
{
return Task.CompletedTask;
}
/// <summary>
/// Called when the current conversion is resumed and any state should be restored.
/// </summary>
/// <param name="conversationId">The ID of the current conversation/thread, if the conversation/thread has an ID.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>An async task.</returns>
/// <remarks>
/// In a service that hosts an agent, that is invoked via calls to the service, this might be at the start of each service call where a previous conversation is being continued.
/// In a client application, this might be when the user re-opens the chat window to resume a conversation after having previously closed it.
/// </remarks>
public virtual Task ResumingAsync(string? conversationId, CancellationToken cancellationToken = default)
{
return Task.CompletedTask;
}
}
dotnet/src/SemanticKernel.Abstractions/Memory/AggregateAIContextProvider.cs
+98
View File
@@ -0,0 +1,98 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.DependencyInjection;
namespace Microsoft.SemanticKernel;
/// <summary>
/// An <see cref="AIContextProvider"/> that contains context providers inside, delegates events to them, and aggregates responses from those events before returning.
/// </summary>
[Experimental("SKEXP0130")]
public sealed class AggregateAIContextProvider : AIContextProvider
{
private readonly List<AIContextProvider> _providers = new();
/// <summary>
/// Gets the list of registered <see cref="AIContextProvider"/> objects.
/// </summary>
public IReadOnlyList<AIContextProvider> Providers => this._providers;
/// <summary>
/// Initializes a new instance of the <see cref="AggregateAIContextProvider"/> class with the specified <see cref="AIContextProvider"/> objects.
/// </summary>
/// <param name="aiContextProviders">The <see cref="AIContextProvider"/> objects to add to the manager.</param>
public AggregateAIContextProvider(IEnumerable<AIContextProvider>? aiContextProviders = null)
{
this._providers.AddRange(aiContextProviders ?? []);
}
/// <summary>
/// Adds a new <see cref="AIContextProvider"/> objects.
/// </summary>
/// <param name="aiContextProvider">The <see cref="AIContextProvider"/> object to register.</param>
public void Add(AIContextProvider aiContextProvider)
{
this._providers.Add(aiContextProvider);
}
/// <summary>
/// Adds all <see cref="AIContextProvider"/> objects registered on the provided dependency injection service provider.
/// </summary>
/// <param name="serviceProvider">The dependency injection service provider to read <see cref="AIContextProvider"/> objects from.</param>
public void AddFromServiceProvider(IServiceProvider serviceProvider)
{
foreach (var aiContextProvider in serviceProvider.GetServices<AIContextProvider>())
{
this.Add(aiContextProvider);
}
}
/// <inheritdoc />
public override async Task ConversationCreatedAsync(string? conversationId, CancellationToken cancellationToken = default)
{
await Task.WhenAll(this.Providers.Select(x => x.ConversationCreatedAsync(conversationId, cancellationToken)).ToList()).ConfigureAwait(false);
}
/// <inheritdoc />
public override async Task ConversationDeletingAsync(string? conversationId, CancellationToken cancellationToken = default)
{
await Task.WhenAll(this.Providers.Select(x => x.ConversationDeletingAsync(conversationId, cancellationToken)).ToList()).ConfigureAwait(false);
}
/// <inheritdoc />
public override async Task MessageAddingAsync(string? conversationId, ChatMessage newMessage, CancellationToken cancellationToken = default)
{
await Task.WhenAll(this.Providers.Select(x => x.MessageAddingAsync(conversationId, newMessage, cancellationToken)).ToList()).ConfigureAwait(false);
}
/// <inheritdoc />
public override async Task<AIContext> ModelInvokingAsync(ICollection<ChatMessage> newMessages, CancellationToken cancellationToken = default)
{
var subContexts = await Task.WhenAll(this.Providers.Select(x => x.ModelInvokingAsync(newMessages, cancellationToken)).ToList()).ConfigureAwait(false);
subContexts = subContexts.Where(x => x != null).ToArray();
var combinedContext = new AIContext();
combinedContext.AIFunctions = subContexts.Where(x => x.AIFunctions != null).SelectMany(x => x.AIFunctions).ToList();
combinedContext.Instructions = string.Join("\n", subContexts.Where(x => !string.IsNullOrWhiteSpace(x.Instructions)).Select(x => x.Instructions));
return combinedContext;
}
/// <inheritdoc />
public override async Task SuspendingAsync(string? conversationId, CancellationToken cancellationToken = default)
{
await Task.WhenAll(this.Providers.Select(x => x.SuspendingAsync(conversationId, cancellationToken))).ConfigureAwait(false);
}
/// <inheritdoc />
public override async Task ResumingAsync(string? conversationId, CancellationToken cancellationToken = default)
{
await Task.WhenAll(this.Providers.Select(x => x.ResumingAsync(conversationId, cancellationToken))).ConfigureAwait(false);
}
}
dotnet/src/SemanticKernel.Abstractions/Memory/AggregateAIContextProviderExtensions.cs
+41
View File
@@ -0,0 +1,41 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
namespace Microsoft.SemanticKernel;
/// <summary>
/// Extension methods for <see cref="AggregateAIContextProvider"/>.
/// </summary>
[Experimental("SKEXP0130")]
public static class AggregateAIContextProviderExtensions
{
/// <summary>
/// This method is called when a new message has been contributed to the chat by any participant.
/// </summary>
/// <param name="aggregateAIContextProvider">The <see cref="AggregateAIContextProvider"/> to pass the new message to.</param>
/// <param name="conversationId">The ID of the conversation/thread for the new message, if the conversation/thread has an ID.</param>
/// <param name="newMessage">The new message.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
public static Task MessageAddingAsync(this AggregateAIContextProvider aggregateAIContextProvider, string? conversationId, ChatMessageContent newMessage, CancellationToken cancellationToken = default)
{
return aggregateAIContextProvider.MessageAddingAsync(conversationId, newMessage.ToChatMessage(), cancellationToken);
}
/// <summary>
/// Called just before the Model/Agent/etc. is invoked
/// </summary>
/// <param name="aggregateAIContextProvider">The <see cref="AggregateAIContextProvider"/> to call.</param>
/// <param name="newMessages">The most recent messages that the Model/Agent/etc. is being invoked with.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>A task that represents the asynchronous operation, containing the combined context from all <see cref="AIContextProvider"/> objects.</returns>
public static Task<AIContext> ModelInvokingAsync(this AggregateAIContextProvider aggregateAIContextProvider, ICollection<ChatMessageContent> newMessages, CancellationToken cancellationToken = default)
{
return aggregateAIContextProvider.ModelInvokingAsync(newMessages.Select(m => m.ToChatMessage()).ToList(), cancellationToken);
}
}
dotnet/src/SemanticKernel.Core/Data/TextSearchBehavior/TextSearchProvider.cs
+148
View File
@@ -0,0 +1,148 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
namespace Microsoft.SemanticKernel.Data;
/// <summary>
/// A component that does a search based on any messages that the AI model is invoked with and injects the results into the AI model invocation context.
/// </summary>
[Experimental("SKEXP0130")]
public sealed class TextSearchProvider : AIContextProvider
{
private const string DefaultPluginSearchFunctionName = "Search";
private const string DefaultPluginSearchFunctionDescription = "Allows searching for additional information to help answer the user question.";
private const string DefaultContextPrompt = "## Additional Context\nConsider the following information from source documents when responding to the user:";
private const string DefaultIncludeCitationsPrompt = "Include citations to the source document with document name and link if document name and link is available.";
private readonly ITextSearch _textSearch;
private readonly AIFunction[] _aIFunctions;
/// <summary>
/// Initializes a new instance of the <see cref="TextSearchProvider"/> class.
/// </summary>
/// <param name="textSearch">The text search component to retrieve results from.</param>
/// <param name="options">Options that configure the behavior of the component.</param>
/// <exception cref="ArgumentNullException"></exception>
public TextSearchProvider(ITextSearch textSearch, TextSearchProviderOptions? options = default)
{
Verify.NotNull(textSearch);
this._textSearch = textSearch;
this.Options = options ?? new();
this._aIFunctions =
[
AIFunctionFactory.Create(
this.SearchAsync,
name: this.Options.PluginFunctionName ?? DefaultPluginSearchFunctionName,
description: this.Options.PluginFunctionDescription ?? DefaultPluginSearchFunctionDescription)
];
}
/// <summary>
/// Gets the options that have been configured for this component.
/// </summary>
public TextSearchProviderOptions Options { get; }
/// <inheritdoc/>
public override async Task<AIContext> ModelInvokingAsync(ICollection<ChatMessage> newMessages, CancellationToken cancellationToken = default)
{
if (this.Options.SearchTime != TextSearchProviderOptions.RagBehavior.BeforeAIInvoke)
{
return new()
{
AIFunctions = this._aIFunctions.ToArray(),
};
}
Verify.NotNull(newMessages);
string input = string.Join("\n", newMessages.Where(m => m is not null).Select(m => m.Text));
var searchResults = await this._textSearch.GetTextSearchResultsAsync(
input,
new() { Top = this.Options.Top },
cancellationToken: cancellationToken).ConfigureAwait(false);
var results = await searchResults.Results.ToListAsync(cancellationToken).ConfigureAwait(false);
return new() { Instructions = this.FormatResults(results) };
}
/// <summary>
/// Plugin method to search the database on demand.
/// </summary>
[KernelFunction]
internal async Task<string> SearchAsync(string userQuestion, CancellationToken cancellationToken = default)
{
var searchResults = await this._textSearch.GetTextSearchResultsAsync(
userQuestion,
new() { Top = this.Options.Top },
cancellationToken: cancellationToken).ConfigureAwait(false);
var results = await searchResults.Results.ToListAsync(cancellationToken).ConfigureAwait(false);
return this.FormatResults(results);
}
/// <summary>
/// Format the results showing the content with source link and name for each result.
/// </summary>
/// <param name="results">The results to format.</param>
/// <returns>The formatted results.</returns>
private string FormatResults(List<TextSearchResult> results)
{
if (this.Options.ContextFormatter is not null)
{
return this.Options.ContextFormatter(results);
}
if (results.Count == 0)
{
return string.Empty;
}
var sb = new StringBuilder();
sb.AppendLine(this.Options.ContextPrompt ?? DefaultContextPrompt);
for (int i = 0; i < results.Count; i++)
{
var result = results[i];
// Only output the doc name and link lines
// if they are not empty to save tokens.
if (!string.IsNullOrWhiteSpace(result.Name))
{
sb.AppendLine($"SourceDocName: {result.Name}");
}
if (!string.IsNullOrWhiteSpace(result.Link))
{
sb.AppendLine($"SourceDocLink: {result.Link}");
}
sb.AppendLine($"Contents: {result.Value}");
sb.AppendLine("----");
}
sb.AppendLine(this.Options.IncludeCitationsPrompt ?? DefaultIncludeCitationsPrompt);
sb.AppendLine();
return sb.ToString();
}
}
[JsonSourceGenerationOptions(JsonSerializerDefaults.General,
UseStringEnumConverter = false,
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
WriteIndented = false)]
[JsonSerializable(typeof(List<TextSearchResult>))]
internal partial class TextRagSourceGenerationContext : JsonSerializerContext
{
}
dotnet/src/SemanticKernel.Core/Data/TextSearchBehavior/TextSearchProviderOptions.cs
+111
View File
@@ -0,0 +1,111 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
namespace Microsoft.SemanticKernel.Data;
/// <summary>
/// Contains options for the <see cref="TextSearchProvider"/>.
/// </summary>
[Experimental("SKEXP0130")]
public sealed class TextSearchProviderOptions
{
private int _top = 3;
/// <summary>
/// Maximum number of results to return from the similarity search.
/// </summary>
/// <remarks>The value must be greater than 0.</remarks>
/// <value>The default value is 3 if not set.</value>
public int Top
{
get => this._top;
init
{
if (value < 1)
{
throw new ArgumentOutOfRangeException(nameof(value), "Top must be greater than 0.");
}
this._top = value;
}
}
/// <summary>
/// Gets or sets the time at which the text search is performed.
/// </summary>
public RagBehavior SearchTime { get; init; } = RagBehavior.BeforeAIInvoke;
/// <summary>
/// Gets or sets the name of the plugin method that will be made available for searching
/// if the <see cref="SearchTime"/> option is set to <see cref="RagBehavior.OnDemandFunctionCalling"/>.
/// </summary>
/// <value>
/// Defaults to &quot;Search&quot; if not set.
/// </value>
public string? PluginFunctionName { get; init; }
/// <summary>
/// Gets or sets the description of the plugin method that will be made available for searching
/// if the <see cref="SearchTime"/> option is set to <see cref="RagBehavior.OnDemandFunctionCalling"/>.
/// </summary>
/// <value>
/// Defaults to &quot;Allows searching for additional information to help answer the user question.&quot; if not set.
/// </value>
public string? PluginFunctionDescription { get; init; }
/// <summary>
/// When providing the text chunks to the AI model on invocation, this string is prefixed
/// to those chunks, in order to provide some context to the model.
/// </summary>
/// <value>
/// Defaults to &quot;Consider the following information from source documents when responding to the user:&quot;
/// </value>
public string? ContextPrompt { get; init; }
/// <summary>
/// When providing the text chunks to the AI model on invocation, this string is postfixed
/// to those chunks, in order to instruct the model to include citations.
/// </summary>
/// <value>
/// Defaults to &quot;## Additional Context\nInclude citations to the source document with document name and link if document name and link is available.:&quot;
/// </value>
public string? IncludeCitationsPrompt { get; init; }
/// <summary>
/// Optional delegate to override the default context creation implementation.
/// </summary>
/// <remarks>
/// <para>
/// If provided, this delegate will be used to do the following:
/// 1. Create the output context provided by the <see cref="TextSearchProvider"/> when invoking the AI model.
/// 2. Create the response text when invoking the component via a plugin.
/// </para>
/// <para>
/// Note that the delegate should include the context prompt and the
/// include citations prompt if they are required in the output.
/// The <see cref="ContextPrompt"/> and <see cref="IncludeCitationsPrompt"/> settings
/// will not be used if providing this delegate.
/// </para>
/// </remarks>
public Func<List<TextSearchResult>, string>? ContextFormatter { get; init; }
/// <summary>
/// Choices for controlling the behavior of the <see cref="TextSearchProvider"/>.
/// </summary>
public enum RagBehavior
{
/// <summary>
/// A search is performed each time that the model/agent is invoked just before invocation
/// and the results are provided to the model/agent via the invocation context.
/// </summary>
BeforeAIInvoke,
/// <summary>
/// A search may be performed by the model/agent on demand via function calling.
/// </summary>
OnDemandFunctionCalling
}
}
dotnet/src/SemanticKernel.Core/Data/TextSearchStore/TextSearchDocument.cs
+55
View File
@@ -0,0 +1,55 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
namespace Microsoft.SemanticKernel.Data;
/// <summary>
/// Represents a document that can be used for Retrieval Augmented Generation (RAG).
/// </summary>
[Experimental("SKEXP0130")]
public sealed class TextSearchDocument
{
/// <summary>
/// Gets or sets an optional list of namespaces that the document should belong to.
/// </summary>
/// <remarks>
/// A namespace is a logical grouping of documents, e.g. may include a group id to scope the document to a specific group of users.
/// </remarks>
public IList<string> Namespaces { get; set; } = [];
/// <summary>
/// Gets or sets the content as text.
/// </summary>
public string? Text { get; set; }
/// <summary>
/// Gets or sets an optional source ID for the document.
/// </summary>
/// <remarks>
/// This ID should be unique within the collection that the document is stored in, and can
/// be used to map back to the source artifact for this document.
/// If updates need to be made later or the source document was deleted and this document
/// also needs to be deleted, this id can be used to find the document again.
/// </remarks>
public string? SourceId { get; set; }
/// <summary>
/// Gets or sets an optional name for the source document.
/// </summary>
/// <remarks>
/// This can be used to provide display names for citation links when the document is referenced as
/// part of a response to a query.
/// </remarks>
public string? SourceName { get; set; }
/// <summary>
/// Gets or sets an optional link back to the source of the document.
/// </summary>
/// <remarks>
/// This can be used to provide citation links when the document is referenced as
/// part of a response to a query.
/// </remarks>
public string? SourceLink { get; set; }
}
dotnet/src/SemanticKernel.Core/Data/TextSearchStore/TextSearchStore.cs
+414
View File
@@ -0,0 +1,414 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Linq.Expressions;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.VectorData;
namespace Microsoft.SemanticKernel.Data;
/// <summary>
/// A class that allows for easy storage and retrieval of documents in a Vector Store for Retrieval Augmented Generation (RAG).
/// </summary>
/// <remarks>
/// <para>
/// This class provides an opinionated schema for storing documents in a vector store. It is valuable for simple scenarios
/// where you want to store text + embedding, or a reference to an external document + embedding without needing to customize the schema.
/// If you want to control the schema yourself, you can use an implementation of <see cref="VectorStoreCollection{TKey, TRecord}"/>
/// with the <see cref="VectorStoreTextSearch{TRecord}"/> class instead.
/// </para>
/// <para>
/// This class can also be used with the <see cref="TextSearchProvider"/> to easily add RAG capabilities to an Agent.
/// </para>
/// </remarks>
/// <typeparam name="TKey">The key type to use with the vector store. Choose a key type supported by your chosen vector store type. Currently this class only supports string or Guid.</typeparam>
[Experimental("SKEXP0130")]
[RequiresDynamicCode("This API is not compatible with NativeAOT.")]
[RequiresUnreferencedCode("This API is not compatible with trimming.")]
public sealed class TextSearchStore<TKey> : ITextSearch, IDisposable
where TKey : notnull
{
private readonly VectorStore _vectorStore;
private readonly int _vectorDimensions;
private readonly TextSearchStoreOptions _options;
private readonly Lazy<VectorStoreCollection<TKey, TextRagStorageDocument<TKey>>> _vectorStoreRecordCollection;
private readonly SemaphoreSlim _collectionInitializationLock = new(1, 1);
private bool _collectionInitialized = false;
private bool _disposedValue;
/// <summary>
/// Initializes a new instance of the <see cref="TextSearchStore{TKey}"/> class.
/// </summary>
/// <param name="vectorStore">The vector store to store and read the memories from.</param>
/// <param name="collectionName">The name of the collection in the vector store to store and read the memories from.</param>
/// <param name="vectorDimensions">The number of dimensions to use for the memory embeddings.</param>
/// <param name="options">Options to configure the behavior of this class.</param>
/// <exception cref="NotSupportedException">Thrown if the key type provided is not supported.</exception>
public TextSearchStore(
VectorStore vectorStore,
string collectionName,
int vectorDimensions,
TextSearchStoreOptions? options = default)
{
// Verify
Verify.NotNull(vectorStore);
Verify.NotNullOrWhiteSpace(collectionName);
Verify.True(vectorDimensions > 0, "Vector dimensions must be greater than 0");
if (typeof(TKey) != typeof(string) && typeof(TKey) != typeof(Guid))
{
throw new NotSupportedException($"Unsupported key of type '{typeof(TKey).Name}'");
}
if (typeof(TKey) != typeof(string) && options?.UseSourceIdAsPrimaryKey is true)
{
throw new NotSupportedException($"The {nameof(TextSearchStoreOptions.UseSourceIdAsPrimaryKey)} option can only be used when the key type is 'string'.");
}
// Assign
this._vectorStore = vectorStore;
this._vectorDimensions = vectorDimensions;
this._options = options ?? new TextSearchStoreOptions();
// Create a definition so that we can use the dimensions provided at runtime.
VectorStoreCollectionDefinition ragDocumentDefinition = new()
{
Properties = new List<VectorStoreProperty>()
{
new VectorStoreKeyProperty("Key", typeof(TKey)),
new VectorStoreDataProperty("Namespaces", typeof(List<string>)) { IsIndexed = true },
new VectorStoreDataProperty("SourceId", typeof(string)) { IsIndexed = true },
new VectorStoreDataProperty("Text", typeof(string)),
new VectorStoreDataProperty("SourceName", typeof(string)),
new VectorStoreDataProperty("SourceLink", typeof(string)),
new VectorStoreVectorProperty("TextEmbedding", typeof(string), vectorDimensions),
}
};
this._vectorStoreRecordCollection = new Lazy<VectorStoreCollection<TKey, TextRagStorageDocument<TKey>>>(() =>
this._vectorStore.GetCollection<TKey, TextRagStorageDocument<TKey>>(collectionName, ragDocumentDefinition));
}
/// <summary>
/// Upserts a batch of text chunks into the vector store.
/// </summary>
/// <param name="textChunks">The text chunks to upload.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>A task that completes when the documents have been upserted.</returns>
public async Task UpsertTextAsync(IEnumerable<string> textChunks, CancellationToken cancellationToken = default)
{
Verify.NotNull(textChunks);
var vectorStoreRecordCollection = await this.EnsureCollectionExistsAsync(cancellationToken).ConfigureAwait(false);
var storageDocuments = textChunks.Select(textChunk =>
{
// Without text we cannot generate a vector.
if (string.IsNullOrWhiteSpace(textChunk))
{
throw new ArgumentException("One of the provided text chunks is null.", nameof(textChunks));
}
var key = GenerateUniqueKey<TKey>(null);
return new TextRagStorageDocument<TKey>
{
Key = key,
Text = textChunk,
TextEmbedding = textChunk,
};
});
await vectorStoreRecordCollection.UpsertAsync(storageDocuments, cancellationToken).ConfigureAwait(false);
}
/// <summary>
/// Upserts a batch of documents into the vector store.
/// </summary>
/// <param name="documents">The documents to upload.</param>
/// <param name="options">Optional options to control the upsert behavior.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>A task that completes when the documents have been upserted.</returns>
public async Task UpsertDocumentsAsync(IEnumerable<TextSearchDocument> documents, TextSearchStoreUpsertOptions? options = null, CancellationToken cancellationToken = default)
{
Verify.NotNull(documents);
var vectorStoreRecordCollection = await this.EnsureCollectionExistsAsync(cancellationToken).ConfigureAwait(false);
var storageDocuments = documents.Select(document =>
{
if (document is null)
{
throw new ArgumentNullException(nameof(documents), "One of the provided documents is null.");
}
// Without text we cannot generate a vector.
if (string.IsNullOrWhiteSpace(document.Text))
{
throw new ArgumentException($"The {nameof(TextSearchDocument.Text)} property must be set.", nameof(document));
}
// If we aren't persisting the text, we need a source id or link to refer back to the original document.
if (options?.PersistSourceText is false && string.IsNullOrWhiteSpace(document.SourceId) && string.IsNullOrWhiteSpace(document.SourceLink))
{
throw new ArgumentException($"Either the {nameof(TextSearchDocument.SourceId)} or {nameof(TextSearchDocument.SourceLink)} properties must be set when the {nameof(TextSearchStoreUpsertOptions.PersistSourceText)} setting is false.", nameof(document));
}
var key = GenerateUniqueKey<TKey>(this._options.UseSourceIdAsPrimaryKey ?? false ? document.SourceId : null);
return new TextRagStorageDocument<TKey>
{
Key = key,
Namespaces = document.Namespaces.ToList(),
SourceId = document.SourceId,
Text = options?.PersistSourceText is false ? null : document.Text,
SourceName = document.SourceName,
SourceLink = document.SourceLink,
TextEmbedding = document.Text,
};
});
await vectorStoreRecordCollection.UpsertAsync(storageDocuments, cancellationToken).ConfigureAwait(false);
}
/// <inheritdoc/>
public async Task<KernelSearchResults<string>> SearchAsync(string query, TextSearchOptions? searchOptions = null, CancellationToken cancellationToken = default)
{
var searchResult = await this.SearchInternalAsync(query, searchOptions, cancellationToken).ConfigureAwait(false);
return new(searchResult.Select(x => x.Text ?? string.Empty).ToAsyncEnumerable());
}
/// <inheritdoc/>
public async Task<KernelSearchResults<TextSearchResult>> GetTextSearchResultsAsync(string query, TextSearchOptions? searchOptions = null, CancellationToken cancellationToken = default)
{
var searchResult = await this.SearchInternalAsync(query, searchOptions, cancellationToken).ConfigureAwait(false);
var results = searchResult.Select(x => new TextSearchResult(x.Text ?? string.Empty) { Name = x.SourceName, Link = x.SourceLink });
return new(searchResult.Select(x =>
new TextSearchResult(x.Text ?? string.Empty)
{
Name = x.SourceName,
Link = x.SourceLink
}).ToAsyncEnumerable());
}
/// <inheritdoc/>
public async Task<KernelSearchResults<object>> GetSearchResultsAsync(string query, TextSearchOptions? searchOptions = null, CancellationToken cancellationToken = default)
{
var searchResult = await this.SearchInternalAsync(query, searchOptions, cancellationToken).ConfigureAwait(false);
return new(searchResult.Select(x => (object)x).ToAsyncEnumerable());
}
/// <summary>
/// Internal search implementation with hydration of id / link only storage.
/// </summary>
/// <param name="query">The text query to find similar documents to.</param>
/// <param name="searchOptions">Search options.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>The search results.</returns>
private async Task<IEnumerable<TextRagStorageDocument<TKey>>> SearchInternalAsync(string query, TextSearchOptions? searchOptions = null, CancellationToken cancellationToken = default)
{
var vectorStoreRecordCollection = await this.EnsureCollectionExistsAsync(cancellationToken).ConfigureAwait(false);
// Optional filter to limit the search to a specific namespace.
Expression<Func<TextRagStorageDocument<TKey>, bool>>? filter = string.IsNullOrWhiteSpace(this._options.SearchNamespace) ? null : x => x.Namespaces.Contains(this._options.SearchNamespace);
// Generate the vector for the query and search.
var searchResult = vectorStoreRecordCollection.SearchAsync(
query,
searchOptions?.Top ?? 3,
options: new()
{
Filter = filter,
},
cancellationToken: cancellationToken);
// Retrieve the documents from the search results.
var searchResponseDocs = await searchResult
.SelectAsync(x => x.Record, cancellationToken)
.ToListAsync(cancellationToken)
.ConfigureAwait(false);
// Find any source ids and links for which the text needs to be retrieved.
var sourceIdsToRetrieve = searchResponseDocs
.Where(x => string.IsNullOrWhiteSpace(x.Text))
.Select(x => new TextSearchStoreSourceRetrievalRequest(x.SourceId, x.SourceLink))
.ToList();
if (sourceIdsToRetrieve.Count > 0)
{
if (this._options.SourceRetrievalCallback is null)
{
throw new InvalidOperationException($"The {nameof(TextSearchStoreOptions.SourceRetrievalCallback)} option must be set if retrieving documents without stored text.");
}
var retrievalResponses = await this._options.SourceRetrievalCallback(sourceIdsToRetrieve).ConfigureAwait(false);
if (retrievalResponses is null)
{
throw new InvalidOperationException($"The {nameof(TextSearchStoreOptions.SourceRetrievalCallback)} must return a non-null value.");
}
// Update the retrieved documents with the retrieved text.
searchResponseDocs = searchResponseDocs.GroupJoin(
retrievalResponses,
searchResponseDoc => (searchResponseDoc.SourceId, searchResponseDoc.SourceLink),
retrievalResponse => (retrievalResponse.SourceId, retrievalResponse.SourceLink),
(searchResponseDoc, textRetrievalResponse) => (searchResponseDoc, textRetrievalResponse))
.SelectMany(
joinedSet => joinedSet.textRetrievalResponse.DefaultIfEmpty(),
(combined, textRetrievalResponse) =>
{
combined.searchResponseDoc.Text = textRetrievalResponse?.Text ?? combined.searchResponseDoc.Text;
return combined.searchResponseDoc;
})
.ToList();
}
return searchResponseDocs;
}
/// <summary>
/// Thread safe method to get the collection and ensure that it is created at least once.
/// </summary>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>The created collection.</returns>
private async Task<VectorStoreCollection<TKey, TextRagStorageDocument<TKey>>> EnsureCollectionExistsAsync(CancellationToken cancellationToken)
{
var vectorStoreRecordCollection = this._vectorStoreRecordCollection.Value;
// Return immediately if the collection is already created, no need to do any locking in this case.
if (this._collectionInitialized)
{
return vectorStoreRecordCollection;
}
// Wait on a lock to ensure that only one thread can create the collection.
await this._collectionInitializationLock.WaitAsync(cancellationToken).ConfigureAwait(false);
// If multiple threads waited on the lock, and the first already created the collection,
// we can return immediately without doing any work in subsequent threads.
if (this._collectionInitialized)
{
this._collectionInitializationLock.Release();
return vectorStoreRecordCollection;
}
// Only the winning thread should reach this point and create the collection.
try
{
await vectorStoreRecordCollection.EnsureCollectionExistsAsync(cancellationToken).ConfigureAwait(false);
this._collectionInitialized = true;
}
finally
{
this._collectionInitializationLock.Release();
}
return vectorStoreRecordCollection;
}
/// <summary>
/// Generates a unique key for the RAG document.
/// </summary>
/// <param name="sourceId">Source id of the source document for this RAG document.</param>
/// <typeparam name="TDocumentKey">The type of the key to use, since different databases require/support different keys.</typeparam>
/// <returns>A new unique key.</returns>
/// <exception cref="NotSupportedException">Thrown if the requested key type is not supported.</exception>
private static TDocumentKey GenerateUniqueKey<TDocumentKey>(string? sourceId)
=> typeof(TDocumentKey) switch
{
_ when typeof(TDocumentKey) == typeof(string) && !string.IsNullOrWhiteSpace(sourceId) => (TDocumentKey)(object)sourceId!,
_ when typeof(TDocumentKey) == typeof(string) => (TDocumentKey)(object)Guid.NewGuid().ToString(),
_ when typeof(TDocumentKey) == typeof(Guid) => (TDocumentKey)(object)Guid.NewGuid(),
_ => throw new NotSupportedException($"Unsupported key of type '{typeof(TDocumentKey).Name}'")
};
/// <inheritdoc/>
private void Dispose(bool disposing)
{
if (!this._disposedValue)
{
if (disposing)
{
this._collectionInitializationLock.Dispose();
}
this._disposedValue = true;
}
}
/// <inheritdoc/>
public void Dispose()
{
// Do not change this code. Put cleanup code in 'Dispose(bool disposing)' method
this.Dispose(disposing: true);
GC.SuppressFinalize(this);
}
/// <summary>
/// The data model to use for storing RAG documents in the vector store.
/// </summary>
/// <typeparam name="TDocumentKey">The type of the key to use, since different databases require/support different keys.</typeparam>
internal sealed class TextRagStorageDocument<TDocumentKey>
{
/// <summary>
/// Gets or sets a unique identifier for the memory document.
/// </summary>
public TDocumentKey Key { get; set; } = default!;
/// <summary>
/// Gets or sets an optional list of namespaces that the document should belong to.
/// </summary>
/// <remarks>
/// A namespace is a logical grouping of documents, e.g. may include a group id to scope the document to a specific group of users.
/// </remarks>
public List<string> Namespaces { get; set; } = [];
/// <summary>
/// Gets or sets the content as text.
/// </summary>
public string? Text { get; set; }
/// <summary>
/// Gets or sets an optional source ID for the document.
/// </summary>
/// <remarks>
/// This ID should be unique within the collection that the document is stored in, and can
/// be used to map back to the source artifact for this document.
/// If updates need to be made later or the source document was deleted and this document
/// also needs to be deleted, this id can be used to find the document again.
/// </remarks>
public string? SourceId { get; set; }
/// <summary>
/// Gets or sets an optional name for the source document.
/// </summary>
/// <remarks>
/// This can be used to provide display names for citation links when the document is referenced as
/// part of a response to a query.
/// </remarks>
public string? SourceName { get; set; }
/// <summary>
/// Gets or sets an optional link back to the source of the document.
/// </summary>
/// <remarks>
/// This can be used to provide citation links when the document is referenced as
/// part of a response to a query.
/// </remarks>
public string? SourceLink { get; set; }
/// <summary>
/// Gets or sets the text that will be used to generate the embedding for the document.
/// </summary>
public string? TextEmbedding { get; set; }
}
}
dotnet/src/SemanticKernel.Core/Data/TextSearchStore/TextSearchStoreOptions.cs
+49
View File
@@ -0,0 +1,49 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Threading.Tasks;
namespace Microsoft.SemanticKernel.Data;
/// <summary>
/// Contains options for the <see cref="TextSearchStore{TKey}"/>.
/// </summary>
[Experimental("SKEXP0130")]
public sealed class TextSearchStoreOptions
{
/// <summary>
/// Gets or sets an optional namespace to pre-filter the possible
/// records with when doing a vector search.
/// </summary>
public string? SearchNamespace { get; init; }
/// <summary>
/// Gets or sets a value indicating whether to use the source ID as the primary key for records.
/// </summary>
/// <remarks>
/// <para>
/// Using the source ID as the primary key allows for easy updates from the source for any changed
/// records, since those records can just be upserted again, and will overwrite the previous version
/// of the same record.
/// </para>
/// <para>
/// This setting can only be used when the chosen key type is a string.
/// </para>
/// </remarks>
/// <value>
/// Defaults to <c>false</c> if not set.
/// </value>
public bool? UseSourceIdAsPrimaryKey { get; init; }
/// <summary>
/// Gets or sets an optional callback to load the source text using the source id or source link
/// if the source text is not persisted in the database.
/// </summary>
/// <remarks>
/// The response should include the source id or source link, as provided in the request,
/// plus the source text loaded from the source.
/// </remarks>
public Func<List<TextSearchStoreSourceRetrievalRequest>, Task<IEnumerable<TextSearchStoreSourceRetrievalResponse>>>? SourceRetrievalCallback { get; init; }
}
dotnet/src/SemanticKernel.Core/Data/TextSearchStore/TextSearchStoreSourceRetrievalRequest.cs
+33
View File
@@ -0,0 +1,33 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Diagnostics.CodeAnalysis;
namespace Microsoft.SemanticKernel.Data;
/// <summary>
/// Represents a request to the <see cref="TextSearchStoreOptions.SourceRetrievalCallback"/>.
/// </summary>
[Experimental("SKEXP0130")]
public sealed class TextSearchStoreSourceRetrievalRequest
{
/// <summary>
/// Initializes a new instance of the <see cref="TextSearchStoreSourceRetrievalRequest"/> class.
/// </summary>
/// <param name="sourceId">The source ID of the document to retrieve.</param>
/// <param name="sourceLink">The source link of the document to retrieve.</param>
public TextSearchStoreSourceRetrievalRequest(string? sourceId, string? sourceLink)
{
this.SourceId = sourceId;
this.SourceLink = sourceLink;
}
/// <summary>
/// Gets or sets the source ID of the document to retrieve.
/// </summary>
public string? SourceId { get; set; }
/// <summary>
/// Gets or sets the source link of the document to retrieve.
/// </summary>
public string? SourceLink { get; set; }
}
dotnet/src/SemanticKernel.Core/Data/TextSearchStore/TextSearchStoreSourceRetrievalResponse.cs
+42
View File
@@ -0,0 +1,42 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Diagnostics.CodeAnalysis;
namespace Microsoft.SemanticKernel.Data;
/// <summary>
/// Represents a response from the <see cref="TextSearchStoreOptions.SourceRetrievalCallback"/>.
/// </summary>
[Experimental("SKEXP0130")]
public sealed class TextSearchStoreSourceRetrievalResponse
{
/// <summary>
/// Initializes a new instance of the <see cref="TextSearchStoreSourceRetrievalResponse"/> class.
/// </summary>
/// <param name="request">The request matching this response.</param>
/// <param name="text">The source text that was retrieved.</param>
public TextSearchStoreSourceRetrievalResponse(TextSearchStoreSourceRetrievalRequest request, string text)
{
Verify.NotNull(request);
Verify.NotNull(text);
this.SourceId = request.SourceId;
this.SourceLink = request.SourceLink;
this.Text = text;
}
/// <summary>
/// Gets or sets the source ID of the document that was retrieved.
/// </summary>
public string? SourceId { get; set; }
/// <summary>
/// Gets or sets the source link of the document that was retrieved.
/// </summary>
public string? SourceLink { get; set; }
/// <summary>
/// Gets or sets the source text of the document that was retrieved.
/// </summary>
public string Text { get; set; }
}
dotnet/src/SemanticKernel.Core/Data/TextSearchStore/TextSearchStoreUpsertOptions.cs
+22
View File
@@ -0,0 +1,22 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Threading;
namespace Microsoft.SemanticKernel.Data;
/// <summary>
/// Contains options for <see cref="TextSearchStore{TKey}.UpsertDocumentsAsync(IEnumerable{TextSearchDocument}, TextSearchStoreUpsertOptions?, CancellationToken)"/>.
/// </summary>
[Experimental("SKEXP0130")]
public sealed class TextSearchStoreUpsertOptions
{
/// <summary>
/// Gets or sets a value indicating whether the source text should be persisted in the database.
/// </summary>
/// <value>
/// Defaults to <see langword="true"/> if not set.
/// </value>
public bool? PersistSourceText { get; init; }
}
dotnet/src/SemanticKernel.Core/Memory/AIContextExtensions.cs
+26
View File
@@ -0,0 +1,26 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using Microsoft.Extensions.AI;
namespace Microsoft.SemanticKernel;
/// <summary>
/// Extension methods for <see cref="ICollection{KernelPlugin}"/> to add <see cref="AIContext"/> plugins.
/// </summary>
[Experimental("SKEXP0130")]
public static class AIContextExtensions
{
/// <summary>
/// Registers the <see cref="AIFunction"/> objects available on the provided <see cref="AIContext"/> as a plugin.
/// </summary>
/// <param name="plugins">The plugins collection to register the <see cref="AIFunction"/> objects on.</param>
/// <param name="aiContext">The <see cref="AIContext"/> to get plugins from.</param>
/// <param name="pluginName">The name to give to the plugin.</param>
public static void AddFromAIContext(this ICollection<KernelPlugin> plugins, AIContext aiContext, string pluginName)
{
plugins.AddFromFunctions(pluginName, aiContext.AIFunctions.Select(x => x.AsKernelFunction()));
}
}
dotnet/src/SemanticKernel.Core/Memory/Mem0/Mem0Client.cs
+182
View File
@@ -0,0 +1,182 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net.Http;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Threading.Tasks;
namespace Microsoft.SemanticKernel.Memory;
/// <summary>
/// Client for the Mem0 memory service.
/// </summary>
internal sealed class Mem0Client
{
private static readonly Uri s_searchUri = new("/v1/memories/search/", UriKind.Relative);
private static readonly Uri s_createMemoryUri = new("/v1/memories/", UriKind.Relative);
private readonly HttpClient _httpClient;
public Mem0Client(HttpClient httpClient)
{
Verify.NotNull(httpClient);
this._httpClient = httpClient;
}
public async Task<IEnumerable<string>> SearchAsync(string? applicationId, string? agentId, string? threadId, string? userId, string? inputText)
{
if (string.IsNullOrWhiteSpace(applicationId)
&& string.IsNullOrWhiteSpace(agentId)
&& string.IsNullOrWhiteSpace(threadId)
&& string.IsNullOrWhiteSpace(userId))
{
throw new ArgumentException("At least one of applicationId, agentId, threadId, or userId must be provided.");
}
var searchRequest = new SearchRequest
{
AppId = applicationId,
AgentId = agentId,
RunId = threadId,
UserId = userId,
Query = inputText ?? string.Empty
};
// Search.
using var content = new StringContent(JsonSerializer.Serialize(searchRequest, Mem0SourceGenerationContext.Default.SearchRequest), Encoding.UTF8, "application/json");
var responseMessage = await this._httpClient.PostAsync(s_searchUri, content).ConfigureAwait(false);
responseMessage.EnsureSuccessStatusCode();
// Process response.
var response = await responseMessage.Content.ReadAsStringAsync().ConfigureAwait(false);
var searchResponseItems = JsonSerializer.Deserialize<SearchResponseItem[]>(response, Mem0SourceGenerationContext.Default.SearchResponseItemArray);
return searchResponseItems?.Select(item => item.Memory) ?? [];
}
public async Task CreateMemoryAsync(string? applicationId, string? agentId, string? threadId, string? userId, string messageContent, string messageRole)
{
if (string.IsNullOrWhiteSpace(applicationId)
&& string.IsNullOrWhiteSpace(agentId)
&& string.IsNullOrWhiteSpace(threadId)
&& string.IsNullOrWhiteSpace(userId))
{
throw new ArgumentException("At least one of applicationId, agentId, threadId, or userId must be provided.");
}
#pragma warning disable CA1308 // Normalize strings to uppercase: mem0 requires lowercase values
var createMemoryRequest = new CreateMemoryRequest()
{
AppId = applicationId,
AgentId = agentId,
RunId = threadId,
UserId = userId,
Messages = new[]
{
new CreateMemoryMessage
{
Content = messageContent,
Role = messageRole.ToLowerInvariant()
}
}
};
#pragma warning restore CA1308 // Normalize strings to uppercase
using var content = new StringContent(JsonSerializer.Serialize(createMemoryRequest, Mem0SourceGenerationContext.Default.CreateMemoryRequest), Encoding.UTF8, "application/json");
var responseMessage = await this._httpClient.PostAsync(s_createMemoryUri, content).ConfigureAwait(false);
responseMessage.EnsureSuccessStatusCode();
}
public async Task ClearMemoryAsync(string? applicationId, string? agentId, string? threadId, string? userId)
{
string[] paramNames = ["app_id", "agent_id", "run_id", "user_id"];
// Build query string.
var querystringParams = new string?[4] { applicationId, agentId, threadId, userId }
.Select((param, index) => string.IsNullOrWhiteSpace(param) ? null : $"{paramNames[index]}={param}")
.Where(x => x != null);
var queryString = string.Join("&", querystringParams);
var clearMemoryUrl = new Uri($"/v1/memories/?{queryString}", UriKind.Relative);
// Delete.
var responseMessage = await this._httpClient.DeleteAsync(clearMemoryUrl).ConfigureAwait(false);
responseMessage.EnsureSuccessStatusCode();
}
internal sealed class CreateMemoryRequest
{
[JsonPropertyName("app_id")]
public string? AppId { get; set; }
[JsonPropertyName("agent_id")]
public string? AgentId { get; set; }
[JsonPropertyName("run_id")]
public string? RunId { get; set; }
[JsonPropertyName("user_id")]
public string? UserId { get; set; }
[JsonPropertyName("messages")]
public CreateMemoryMessage[] Messages { get; set; } = [];
}
internal sealed class CreateMemoryMessage
{
[JsonPropertyName("content")]
public string Content { get; set; } = string.Empty;
[JsonPropertyName("role")]
public string Role { get; set; } = string.Empty;
}
internal sealed class SearchRequest
{
[JsonPropertyName("app_id")]
public string? AppId { get; set; }
[JsonPropertyName("agent_id")]
public string? AgentId { get; set; } = null;
[JsonPropertyName("run_id")]
public string? RunId { get; set; } = null;
[JsonPropertyName("user_id")]
public string? UserId { get; set; } = null;
[JsonPropertyName("query")]
public string Query { get; set; } = string.Empty;
}
internal sealed class SearchResponseItem
{
[JsonPropertyName("id")]
public string Id { get; set; } = string.Empty;
[JsonPropertyName("memory")]
public string Memory { get; set; } = string.Empty;
[JsonPropertyName("hash")]
public string Hash { get; set; } = string.Empty;
[JsonPropertyName("metadata")]
public object? Metadata { get; set; }
[JsonPropertyName("score")]
public double Score { get; set; }
[JsonPropertyName("created_at")]
public DateTime CreatedAt { get; set; }
[JsonPropertyName("updated_at")]
public DateTime? UpdatedAt { get; set; }
[JsonPropertyName("user_id")]
public string UserId { get; set; } = string.Empty;
[JsonPropertyName("app_id")]
public string? AppId { get; set; }
[JsonPropertyName("agent_id")]
public string AgentId { get; set; } = string.Empty;
[JsonPropertyName("session_id")]
public string RunId { get; set; } = string.Empty;
}
}
[JsonSourceGenerationOptions(JsonSerializerDefaults.General,
UseStringEnumConverter = false,
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
WriteIndented = false)]
[JsonSerializable(typeof(Mem0Client.CreateMemoryRequest))]
[JsonSerializable(typeof(Mem0Client.SearchRequest))]
[JsonSerializable(typeof(Mem0Client.SearchResponseItem[]))]
internal partial class Mem0SourceGenerationContext : JsonSerializerContext
{
}
dotnet/src/SemanticKernel.Core/Memory/Mem0/Mem0Provider.cs
+178
View File
@@ -0,0 +1,178 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
namespace Microsoft.SemanticKernel.Memory;
/// <summary>
/// A component that listens to messages added to the conversation thread, and automatically captures
/// information about the user. It is also able to retrieve this information and add it to the AI invocation context.
/// </summary>
/// <remarks>
/// <para>
/// Mem0 allows memories to be stored under one or more optional scopes: application, agent, thread, and user.
/// At least one scope must always be provided.
/// </para>
/// <para>
/// There are some special considerations when using thread as a scope.
/// A thread id may not be available at the time that this component is instantiated.
/// It is therefore possible to provide no thread id when instantiating this class and instead set
/// <see cref="Mem0ProviderOptions.ScopeToPerOperationThreadId"/> to <see langword="true"/>.
/// The component will then capture a thread id when a thread is created or when messages are received
/// and use this thread id to scope the memories in mem0.
/// </para>
/// <para>
/// Note that this component will keep the current thread id in a private field for the duration of
/// the component's lifetime, and therefore using the component with multiple threads, with
/// <see cref="Mem0ProviderOptions.ScopeToPerOperationThreadId"/> set to <see langword="true"/> is not supported.
/// </para>
/// </remarks>
[Experimental("SKEXP0130")]
public sealed class Mem0Provider : AIContextProvider
{
private const string DefaultContextPrompt = "## Memories\nConsider the following memories when answering user questions:";
private readonly string? _applicationId;
private readonly string? _agentId;
private readonly string? _threadId;
private string? _perOperationThreadId;
private readonly string? _userId;
private readonly bool _scopeToPerOperationThreadId;
private readonly string _contextPrompt;
private readonly Mem0Client _mem0Client;
/// <summary>
/// Initializes a new instance of the <see cref="Mem0Provider"/> class.
/// </summary>
/// <param name="httpClient">The HTTP client used for making requests.</param>
/// <param name="options">Options for configuring the component.</param>
/// <remarks>
/// The base address of the required mem0 service, and any authentication headers, should be set on the <paramref name="httpClient"/>
/// already, when passed as a parameter here. E.g.:
/// <code>
/// using var httpClient = new HttpClient();
/// httpClient.BaseAddress = new Uri("https://api.mem0.ai");
/// httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Token", "&lt;Your APIKey&gt;");
/// new Mem0Client(httpClient);
/// </code>
/// </remarks>
public Mem0Provider(HttpClient httpClient, Mem0ProviderOptions? options = default)
{
Verify.NotNull(httpClient);
if (string.IsNullOrWhiteSpace(httpClient.BaseAddress?.AbsolutePath))
{
throw new ArgumentException("The BaseAddress of the provided httpClient parameter must be set.", nameof(httpClient));
}
this._applicationId = options?.ApplicationId;
this._agentId = options?.AgentId;
this._threadId = options?.ThreadId;
this._userId = options?.UserId;
this._scopeToPerOperationThreadId = options?.ScopeToPerOperationThreadId ?? false;
this._contextPrompt = options?.ContextPrompt ?? DefaultContextPrompt;
this._mem0Client = new(httpClient);
}
/// <inheritdoc/>
public override Task ConversationCreatedAsync(string? conversationId, CancellationToken cancellationToken = default)
{
this.ValidatePerOperationThreadId(conversationId);
this._perOperationThreadId ??= conversationId;
return Task.CompletedTask;
}
/// <inheritdoc/>
public override async Task MessageAddingAsync(string? conversationId, ChatMessage newMessage, CancellationToken cancellationToken = default)
{
Verify.NotNull(newMessage);
this.ValidatePerOperationThreadId(conversationId);
switch (newMessage.Role)
{
case ChatRole u when u == ChatRole.User:
case ChatRole a when a == ChatRole.Assistant:
case ChatRole s when s == ChatRole.System:
break;
default:
return;
}
this._perOperationThreadId ??= conversationId;
if (!string.IsNullOrWhiteSpace(newMessage.Text))
{
await this._mem0Client.CreateMemoryAsync(
this._applicationId,
this._agentId,
this._scopeToPerOperationThreadId ? this._perOperationThreadId : this._threadId,
this._userId,
newMessage.Text,
newMessage.Role.Value).ConfigureAwait(false);
}
}
/// <inheritdoc/>
public override async Task<AIContext> ModelInvokingAsync(ICollection<ChatMessage> newMessages, CancellationToken cancellationToken = default)
{
Verify.NotNull(newMessages);
string inputText = string.Join(
Environment.NewLine,
newMessages.
Where(m => m is not null && !string.IsNullOrWhiteSpace(m.Text)).
Select(m => m.Text));
var memories = await this._mem0Client.SearchAsync(
this._applicationId,
this._agentId,
this._scopeToPerOperationThreadId ? this._perOperationThreadId : this._threadId,
this._userId,
inputText).ConfigureAwait(false);
var lineSeparatedMemories = string.Join(Environment.NewLine, memories);
return new()
{
Instructions =
$"""
{this._contextPrompt}
{lineSeparatedMemories}
"""
};
}
/// <summary>
/// Plugin method to clear memories for the current agent/thread/user.
/// </summary>
/// <returns>A task that completes when the memory is cleared.</returns>
public Task ClearStoredMemoriesAsync()
{
return this._mem0Client.ClearMemoryAsync(
this._applicationId,
this._agentId,
this._scopeToPerOperationThreadId ? this._perOperationThreadId : this._threadId,
this._userId);
}
/// <summary>
/// Validate that we are not receiving a new thread id when the component has already received one before.
/// </summary>
/// <param name="threadId">The new thread id.</param>
private void ValidatePerOperationThreadId(string? threadId)
{
if (this._scopeToPerOperationThreadId && !string.IsNullOrWhiteSpace(threadId) && this._perOperationThreadId != null && threadId != this._perOperationThreadId)
{
throw new InvalidOperationException($"The {nameof(Mem0Provider)} can only be used with one thread at a time when {nameof(Mem0ProviderOptions.ScopeToPerOperationThreadId)} is set to true.");
}
}
}
dotnet/src/SemanticKernel.Core/Memory/Mem0/Mem0ProviderOptions.cs
+65
View File
@@ -0,0 +1,65 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Diagnostics.CodeAnalysis;
namespace Microsoft.SemanticKernel.Memory;
/// <summary>
/// Options for the <see cref="Mem0Provider"/>.
/// </summary>
[Experimental("SKEXP0130")]
public sealed class Mem0ProviderOptions
{
/// <summary>
/// Gets or sets an optional ID for the application to scope memories to.
/// </summary>
/// <remarks>
/// If not set, the scope of the memories will span all applications.
/// </remarks>
public string? ApplicationId { get; init; }
/// <summary>
/// Gets or sets an optional ID for the agent to scope memories to.
/// </summary>
/// <remarks>
/// If not set, the scope of the memories will span all agents.
/// </remarks>
public string? AgentId { get; init; }
/// <summary>
/// Gets or sets an optional ID for the thread to scope memories to.
/// </summary>
/// <remarks>
/// This value will be overridden by any thread id provided to the methods of the <see cref="Mem0Provider"/>.
/// </remarks>
public string? ThreadId { get; init; }
/// <summary>
/// Gets or sets an optional ID for the user to scope memories to.
/// </summary>
/// <remarks>
/// If not set, the scope of the memories will span all users.
/// </remarks>
public string? UserId { get; init; }
/// <summary>
/// Gets or sets a value indicating whether memories should be scoped to the thread id provided on a per operation basis.
/// </summary>
/// <remarks>
/// This setting is useful if the thread id is not known when the <see cref="Mem0Provider"/> is instantiated, but
/// per thread scoping is desired.
/// If <see langword="false"/>, and <see cref="ThreadId"/> is not set, there will be no per thread scoping.
/// if <see langword="false"/>, and <see cref="ThreadId"/> is set, <see cref="ThreadId"/> will be used for scoping.
/// If <see langword="true"/>, the thread id will be set to the thread id of the current operation, regardless of the value of <see cref="ThreadId"/>.
/// </remarks>
public bool ScopeToPerOperationThreadId { get; init; } = false;
/// <summary>
/// When providing the memories found in Mem0 to the AI model on invocation, this string is prefixed
/// to those memories, in order to provide some context to the model.
/// </summary>
/// <value>
/// Defaults to &quot;## Memories\nConsider the following memories when answering user questions:&quot;
/// </value>
public string? ContextPrompt { get; init; }
}
dotnet/src/SemanticKernel.Core/Memory/Whiteboard/WhiteboardProvider.cs
+317
View File
@@ -0,0 +1,317 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
namespace Microsoft.SemanticKernel.Memory;
/// <summary>
/// An <see cref="AIContextProvider"/> that maintains a whiteboard during a conversation.
/// </summary>
[Experimental("SKEXP0130")]
public sealed class WhiteboardProvider : AIContextProvider
{
private readonly static JsonDocument s_structuredOutputSchema = JsonDocument.Parse("""{"type":"object","properties":{"newWhiteboard":{"type":"array","items":{"type":"string"}}}}""");
private const string DefaultContextPrompt = "## Whiteboard\nThe following list of messages are currently on the whiteboard:";
private const string DefaultWhiteboardEmptyPrompt = "## Whiteboard\nThe whiteboard is currently empty.";
private const int MaxQueueSize = 3;
private readonly int _maxWhiteboardMessages;
private readonly string _contextPrompt;
private readonly string _whiteboardEmptyPrompt;
private readonly string _maintenancePrompt;
private readonly IChatClient _chatClient;
private List<string> _currentWhiteboardContent = [];
private readonly ConcurrentQueue<ChatMessage> _recentMessages = new();
private ChatMessage? _messageBeingProcessed = null;
private Task _updateWhiteboardTask = Task.CompletedTask;
/// <summary>
/// Initializes a new instance of the <see cref="WhiteboardProvider"/> class.
/// </summary>
/// <param name="chatClient">A <see cref="IChatClient"/> to use for making chat completion calls.</param>
/// <param name="options">Options for configuring the provider.</param>
public WhiteboardProvider(IChatClient chatClient, WhiteboardProviderOptions? options = default)
{
Verify.NotNull(chatClient);
this._chatClient = chatClient;
this._maxWhiteboardMessages = options?.MaxWhiteboardMessages ?? 10;
this._contextPrompt = options?.ContextPrompt ?? DefaultContextPrompt;
this._whiteboardEmptyPrompt = options?.WhiteboardEmptyPrompt ?? DefaultWhiteboardEmptyPrompt;
this._maintenancePrompt = options?.MaintenancePromptTemplate ?? MaintenancePromptTemplate;
}
/// <summary>
/// Gets the current whiteboard content.
/// </summary>
public IReadOnlyList<string> CurrentWhiteboardContent => this._currentWhiteboardContent;
/// <inheritdoc/>
public override async Task MessageAddingAsync(string? conversationId, ChatMessage newMessage, CancellationToken cancellationToken = default)
{
if (string.IsNullOrWhiteSpace(newMessage.Text))
{
return;
}
if (newMessage.Role == ChatRole.User || newMessage.Role == ChatRole.Assistant)
{
this._recentMessages.Enqueue(newMessage);
// Store the last message that we got as a way to avoid doing multiple updates at the same time.
Interlocked.CompareExchange(ref this._messageBeingProcessed, newMessage, this._messageBeingProcessed);
// If a whiteboard update task is already running, wait for it to finish before starting a new one.
if (!this._updateWhiteboardTask.IsCompleted)
{
await this._updateWhiteboardTask.ConfigureAwait(false);
}
// If our message isn't the last one we got, we don't need to do anything since
// we will run the update, on the thread that received the last message.
if (this._messageBeingProcessed != newMessage)
{
return;
}
this._updateWhiteboardTask = this.UpdateWhiteboardAsync(newMessage, cancellationToken);
}
}
/// <inheritdoc/>
public override Task<AIContext> ModelInvokingAsync(ICollection<ChatMessage> newMessages, CancellationToken cancellationToken = default)
{
if (this._currentWhiteboardContent.Count == 0)
{
return Task.FromResult(new AIContext() { Instructions = this._whiteboardEmptyPrompt });
}
var numberedMessages = this._currentWhiteboardContent.Select((x, i) => $"{i} {x}");
var joinedMessages = string.Join(Environment.NewLine, numberedMessages);
return Task.FromResult(new AIContext()
{
Instructions = $"{this._contextPrompt}\n{joinedMessages}"
});
}
/// <summary>
/// Wait for all messages to be processed and the whiteboard to be up to date.
/// </summary>
/// <returns>A task that completes when all messages are processed.</returns>
public async Task WhenProcessingCompleteAsync()
{
if (!this._updateWhiteboardTask.IsCompleted)
{
await this._updateWhiteboardTask.ConfigureAwait(false);
}
}
private async Task UpdateWhiteboardAsync(ChatMessage newMessage, CancellationToken cancellationToken = default)
{
var recentMessagesList = this._recentMessages.ToList();
// If there are more than MaxQueueSize messages in the queue, remove the oldest ones
for (int i = MaxQueueSize; i < recentMessagesList.Count; i++)
{
this._recentMessages.TryDequeue(out _);
}
// Extract the most important information from the input messages.
var basicMessages = recentMessagesList
.Select(m => new BasicMessage
{
AuthorName = m.AuthorName,
Role = m.Role.ToString(),
Text = m.Text
});
// Serialize the input messages and the current whiteboard content to JSON.
var inputMessagesJson = JsonSerializer.Serialize(basicMessages, WhiteboardProviderSourceGenerationContext.Default.IEnumerableBasicMessage);
var currentWhiteboardJson = JsonSerializer.Serialize(this._currentWhiteboardContent, WhiteboardProviderSourceGenerationContext.Default.ListString);
// Inovke the LLM to extract the latest information from the input messages and update the whiteboard.
var result = await this._chatClient.GetResponseAsync(
this.FormatPromptTemplate(inputMessagesJson, currentWhiteboardJson, this._maxWhiteboardMessages),
new()
{
Temperature = 0,
ResponseFormat = new ChatResponseFormatJson(s_structuredOutputSchema.RootElement),
},
cancellationToken).ConfigureAwait(false);
// Update the current whiteboard content with the LLM result.
var newWhiteboardResponse = JsonSerializer.Deserialize(result.ToString(), WhiteboardProviderSourceGenerationContext.Default.NewWhiteboardResponse);
this._currentWhiteboardContent = newWhiteboardResponse?.NewWhiteboard ?? [];
}
private string FormatPromptTemplate(string inputMessagesJson, string currentWhiteboardJson, int maxWhiteboardMessages)
{
var sb = new StringBuilder(this._maintenancePrompt);
sb.Replace("{{$inputMessages}}", inputMessagesJson);
sb.Replace("{{$currentWhiteboard}}", currentWhiteboardJson);
sb.Replace("{{$maxWhiteboardMessages}}", maxWhiteboardMessages.ToString());
return sb.ToString();
}
/// <summary>
/// Gets the prompt template to use for maintaining the whiteboard.
/// </summary>
private const string MaintenancePromptTemplate =
"""
You are an expert in maintaining a whiteboard during a conversation.The whiteboard should capture:
- **Requirements**: Goals or needs expressed by the user.
- **Proposals**: Suggested solutions to the requirements, provided by the assistant.
- **Decisions**: Decisions made by the user, including all relevant details.
- **Actions**: Actions that had been taken to implement a proposal or decision, including all relevant details.
## Transitions:
- **Requirements -> Proposal**: When a proposal is made to satisfy one or more requirements.
- **Proposal -> Decision**: When a proposal is accepted by the user.
- **Proposal -> Actions**: When an action has been taken to execute a proposal.
- **Decision -> Actions**: When an action has been taken to implement a decision.
## Guidelines:
1. **Update Existing Entries**: Modify whiteboard entries as requirements change or new proposals and decisions are made.
2. **User is decision maker**: Only users can make decisions. The assistant can only make proposals and execute them.
3. **Remove Redundant Information**: When a decision is made or an action is taken, remove the requirements and proposals that led to it.
4. **Keep Requirements Concise**: Ensure requirements are clear and to the point.
5. **Keep Decisions, Proposals and Actions Detailed**: Ensure decisions, proposals and actions are comprehensive and include all requirements that went into the decision, proposal or action.
6. **Keep Decisions, Proposals and Actions Self Contained**: Ensure decisions, proposals and actions are self-contained and do not reference other entries, e.g. output "ACTION - The agent booked flight going out, COA 1133 DUB to CDG, 14 April 2025 and return, COA 1134 CDG to DUB, 16 April 2025", instead of "ACTION - The agent booked the flights as defined in requirements.".
7. **Categorize Entries**: Prefix each entry with `REQUIREMENT`, `PROPOSAL`, `DECISION` or `ACTION`.
8. **Prioritize Decisions and Actions**: Retain detailed decisions and actions longer than requirements or proposals.
9. **Limit Entries**: Maintain a maximum of {{$maxWhiteboardMessages}} entries. If the limit is exceeded, combine or remove the least important entries, prioritize keeping decisions and actions.
## Examples:
### Example 1:
New Message:
[{"AuthorName":"Mary","Role":"user","Text":"I want the colour scheme to be green and brown."}]
Current Whiteboard:
["REQUIREMENT - Mary wants to create a presentation."]
New Whiteboard:
{"newWhiteboard":["REQUIREMENT - Mary wants to create a presentation.", "REQUIREMENT - The presentation colour schema should be green and brown."]}
### Example 2:
New Message:
[{"AuthorName":"John","Role":"user","Text":"I need you to help me with my homework."}]
Current Whiteboard:
[]
New Whiteboard:
{"newWhiteboard":["REQUIREMENT - John wants help with homework."]}
### Example 3:
New Message:
[{"AuthorName":"John","Role":"user","Text":"Hello"}]
Current Whiteboard:
[]
New Whiteboard:
{"newWhiteboard":[]}
### Example 4:
New Message:
[{"AuthorName":"Mary","Role":"user","Text":"I've changed my mind, I want to go to London instead."}]
Current Whiteboard:
["REQUIREMENT - Mary wants to book a flight.", "REQUIREMENT - The flight should be to Paris."]
New Whiteboard:
{"newWhiteboard":["REQUIREMENT - Mary wants to book a flight.", "REQUIREMENT - The flight should be to London."]}
### Example 5:
New Message:
[{"AuthorName":"TravelAgent","Role":"assistant","Text":"Here is an itinerary for your trip to Paris. Departing on the 17th of June at 10:00 AM and returning on the 20th of June at 5:00 PM with direct flights to Paris Charles de Gaul airport on NotsocheapoAir. The cost of the flights are EUR 243."}]
Current Whiteboard:
["REQUIREMENT - Mary wants to book a flight.", "REQUIREMENT - The flight should be to Paris during the week of 16th of June 2025."]
New Whiteboard:
{"newWhiteboard":["REQUIREMENT - Mary wants to book a flight.", "REQUIREMENT - The flight should be to Paris during the week of 16th of June 2025.", "PROPOSAL - The current proposed itinerary by the TravelAgent Assistant is to depart on the 17th of June at 10:00 AM and return on the 20th of June at 5:00 PM with direct flights to Paris Charles de Gaul airport on NotsocheapoAir. The cost of the flights are EUR 243."]}
### Example 6:
New Message:
[{"AuthorName":"Mary","Role":"user","Text":"That sounds good, let's book that."}]
Current Whiteboard:
["REQUIREMENT - Mary wants to book a flight.", "REQUIREMENT - The flight should be to Paris during the week of 16th of June 2025.", "PROPOSAL - The current proposed itinerary by the TravelAgent Assistant is to depart on the 17th of June at 10:00 AM and return on the 20th of June at 5:00 PM with direct flights to Paris Charles de Gaul airport on NotsocheapoAir. The cost of the flights are EUR 243."]
New Whiteboard:
{"newWhiteboard":["DECISION - Mary decided to book the flight departing on the 17th of June at 10:00 AM and returning on the 20th of June at 5:00 PM with direct flights to Paris Charles de Gaul airport on NotsocheapoAir. The cost of the flights are EUR 243."]}
### Example 7:
New Message:
[{"AuthorName":"TravelAgent","Role":"assistant","Text":"OK, I've booked that for you."}]
Current Whiteboard:
[""DECISION - Mary decided to book the flight departing on the 17th of June at 10:00 AM and returning on the 20th of June at 5:00 PM with direct flights to Paris Charles de Gaul airport on NotsocheapoAir. The cost of the flights are EUR 243."]
New Whiteboard:
{"newWhiteboard":["ACTION - TravelAgent booked a flight for Mary departing on the 17th of June at 10:00 AM and returning on the 20th of June at 5:00 PM with direct flights to Paris Charles de Gaul airport on NotsocheapoAir for EUR 243."]}
### Example 8:
New Message:
[{"AuthorName":"Mary","Role":"user","Text":"I don't like the suggested option. Can I leave a day earlier and fly with anyone but NotsocheapoAir?"}]
Current Whiteboard:
["REQUIREMENT - Mary wants to book a flight.", "REQUIREMENT - The flight should be to Paris during the week of 16th of June 2025.", "PROPOSAL - The current proposed itinerary by the TravelAgent Assistant is to depart on the 17th of June at 10:00 AM and return on the 20th of June at 5:00 PM with direct flights to Paris Charles de Gaul airport on NotsocheapoAir. The cost of the flights are EUR 243."]
New Whiteboard:
{"newWhiteboard":["REQUIREMENT - Mary wants to book a flight.", "REQUIREMENT - The flight should be to Paris during the week of 16th of June 2025.", "REQUIREMENT - Mary does not want to fly with NotsocheapoAir."]}
## Action
Now return a new whiteboard for the following inputs like shown in the examples above and using the previously mentioned instructions:
New Message:
{{$inputMessages}}
Current Whiteboard:
{{$currentWhiteboard}}
New Whiteboard:
""";
/// <summary>
/// A simple message class that contains just the most basic msessage information
/// that is required to pass to the LLM.
/// </summary>
internal class BasicMessage
{
public string? AuthorName { get; set; }
public string Role { get; set; } = string.Empty;
public string Text { get; set; } = string.Empty;
}
/// <summary>
/// Represents the response from the LLM when updating the whiteboard.
/// </summary>
internal class NewWhiteboardResponse
{
[JsonPropertyName("newWhiteboard")]
public List<string> NewWhiteboard { get; set; } = [];
}
}
/// <summary>
/// Source generated json serializer for <see cref="WhiteboardProvider"/>.
/// </summary>
[Experimental("SKEXP0130")]
[JsonSourceGenerationOptions(JsonSerializerDefaults.General,
UseStringEnumConverter = false,
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
WriteIndented = false)]
[JsonSerializable(typeof(IEnumerable<WhiteboardProvider.BasicMessage>))]
[JsonSerializable(typeof(WhiteboardProvider.BasicMessage))]
[JsonSerializable(typeof(List<string>))]
[JsonSerializable(typeof(WhiteboardProvider.NewWhiteboardResponse))]
internal partial class WhiteboardProviderSourceGenerationContext : JsonSerializerContext
{
}
dotnet/src/SemanticKernel.Core/Memory/Whiteboard/WhiteboardProviderOptions.cs
+49
View File
@@ -0,0 +1,49 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Diagnostics.CodeAnalysis;
namespace Microsoft.SemanticKernel.Memory;
/// <summary>
/// Options for configuring the <see cref="WhiteboardProvider"/>.
/// </summary>
[Experimental("SKEXP0130")]
public sealed class WhiteboardProviderOptions
{
/// <summary>
/// Gets or sets the maximum number of messages to keep on the whiteboard.
/// </summary>
/// <value>
/// Defaults to 10 if not specified.
/// </value>
public int? MaxWhiteboardMessages { get; set; }
/// <summary>
/// Gets or sets a string that is prefixed to the messages on the whiteboard,
/// when providing them as context to the AI model.
/// </summary>
/// <value>
/// Defaults to &quot;## Whiteboard\nThe following list of messages are currently on the whiteboard:&quot;
/// </value>
public string? ContextPrompt { get; init; }
/// <summary>
/// Gets or sets the message to provide to the AI model when there are no messages on the whiteboard.
/// </summary>
/// <value>
/// Defaults to &quot;## Whiteboard\nThe whiteboard is currently empty.&quot;
/// </value>
public string? WhiteboardEmptyPrompt { get; init; }
/// <summary>
/// Gets or sets a prompt template to use to update the whiteboard with the latest messages
/// if the built-in prompt needs to be customized.
/// </summary>
/// <remarks>
/// The following parameters can be used in the prompt:
/// {{$maxWhiteboardMessages}}
/// {{$inputMessages}}
/// {{$currentWhiteboard}}
/// </remarks>
public string? MaintenancePromptTemplate { get; init; }
}
dotnet/src/SemanticKernel.Core/SemanticKernel.Core.csproj
+1
View File
@@ -44,6 +44,7 @@
<ItemGroup>
<InternalsVisibleTo Include="Microsoft.SemanticKernel.Experimental.Assistants" />
<InternalsVisibleTo Include="DynamicProxyGenAssembly2" />
</ItemGroup>
<ItemGroup>
dotnet/src/SemanticKernel.UnitTests/Data/TextSearchProviderTests.cs
+224
View File
@@ -0,0 +1,224 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.SemanticKernel.Data;
using Moq;
using Xunit;
namespace SemanticKernel.UnitTests.Data;
/// <summary>
/// Contains tests for <see cref="TextSearchProvider"/>
/// </summary>
public class TextSearchProviderTests
{
[Theory]
[InlineData(null, null, "Consider the following information from source documents when responding to the user:", "Include citations to the source document with document name and link if document name and link is available.")]
[InlineData("Custom context prompt", "Custom citations prompt", "Custom context prompt", "Custom citations prompt")]
public async Task ModelInvokingShouldIncludeSearchResultsInOutputAsync(
string? overrideContextPrompt,
string? overrideCitationsPrompt,
string expectedContextPrompt,
string expectedCitationsPrompt)
{
// Arrange
var mockTextSearch = new Mock<ITextSearch>();
var searchResults = new Mock<IAsyncEnumerable<TextSearchResult>>();
var mockEnumerator = new Mock<IAsyncEnumerator<TextSearchResult>>();
// Mock search results
var results = new List<TextSearchResult>
{
new("Content of Doc1") { Name = "Doc1", Link = "http://example.com/doc1" },
new("Content of Doc2") { Name = "Doc2", Link = "http://example.com/doc2" }
};
mockEnumerator.SetupSequence(e => e.MoveNextAsync())
.ReturnsAsync(true)
.ReturnsAsync(true)
.ReturnsAsync(false);
mockEnumerator.SetupSequence(e => e.Current)
.Returns(results[0])
.Returns(results[1]);
searchResults.Setup(r => r.GetAsyncEnumerator(It.IsAny<CancellationToken>()))
.Returns(mockEnumerator.Object);
mockTextSearch.Setup(ts => ts.GetTextSearchResultsAsync(
It.IsAny<string>(),
It.IsAny<TextSearchOptions>(),
It.IsAny<CancellationToken>()))
.ReturnsAsync(new KernelSearchResults<TextSearchResult>(searchResults.Object));
var options = new TextSearchProviderOptions
{
SearchTime = TextSearchProviderOptions.RagBehavior.BeforeAIInvoke,
Top = 2,
ContextPrompt = overrideContextPrompt,
IncludeCitationsPrompt = overrideCitationsPrompt
};
var component = new TextSearchProvider(mockTextSearch.Object, options);
// Act
var result = await component.ModelInvokingAsync([new ChatMessage(ChatRole.User, "Sample user question?")], CancellationToken.None);
// Assert
Assert.Contains(expectedContextPrompt, result.Instructions);
Assert.Contains("SourceDocName: Doc1", result.Instructions);
Assert.Contains("SourceDocLink: http://example.com/doc1", result.Instructions);
Assert.Contains("Contents: Content of Doc1", result.Instructions);
Assert.Contains("SourceDocName: Doc2", result.Instructions);
Assert.Contains("SourceDocLink: http://example.com/doc2", result.Instructions);
Assert.Contains("Contents: Content of Doc2", result.Instructions);
Assert.Contains(expectedCitationsPrompt, result.Instructions);
}
[Theory]
[InlineData(null, null, "Search", "Allows searching for additional information to help answer the user question.")]
[InlineData("CustomSearch", "CustomDescription", "CustomSearch", "CustomDescription")]
public async Task AIFunctionsShouldBeRegisteredCorrectly(
string? overridePluginFunctionName,
string? overridePluginFunctionDescription,
string expectedPluginFunctionName,
string expectedPluginFunctionDescription)
{
// Arrange
var mockTextSearch = new Mock<ITextSearch>();
var options = new TextSearchProviderOptions
{
SearchTime = TextSearchProviderOptions.RagBehavior.OnDemandFunctionCalling,
PluginFunctionName = overridePluginFunctionName,
PluginFunctionDescription = overridePluginFunctionDescription
};
var component = new TextSearchProvider(mockTextSearch.Object, options);
// Act
var aiContextAdditions = await component.ModelInvokingAsync([new ChatMessage(ChatRole.User, "Sample user question?")], CancellationToken.None);
// Assert
var aiFunctions = aiContextAdditions.AIFunctions;
Assert.NotNull(aiFunctions);
Assert.Single(aiFunctions);
var aiFunction = aiFunctions.First();
Assert.Equal(expectedPluginFunctionName, aiFunction.Name);
Assert.Equal(expectedPluginFunctionDescription, aiFunction.Description);
}
[Theory]
[InlineData(null, null, "Consider the following information from source documents when responding to the user:", "Include citations to the source document with document name and link if document name and link is available.")]
[InlineData("Custom context prompt", "Custom citations prompt", "Custom context prompt", "Custom citations prompt")]
public async Task SearchAsyncShouldIncludeSearchResultsInOutputAsync(
string? overrideContextPrompt,
string? overrideCitationsPrompt,
string expectedContextPrompt,
string expectedCitationsPrompt)
{
// Arrange
var mockTextSearch = new Mock<ITextSearch>();
var searchResults = new Mock<IAsyncEnumerable<TextSearchResult>>();
var mockEnumerator = new Mock<IAsyncEnumerator<TextSearchResult>>();
// Mock search results
var results = new List<TextSearchResult>
{
new("Content of Doc1") { Name = "Doc1", Link = "http://example.com/doc1" },
new("Content of Doc2") { Name = "Doc2", Link = "http://example.com/doc2" }
};
mockEnumerator.SetupSequence(e => e.MoveNextAsync())
.ReturnsAsync(true)
.ReturnsAsync(true)
.ReturnsAsync(false);
mockEnumerator.SetupSequence(e => e.Current)
.Returns(results[0])
.Returns(results[1]);
searchResults.Setup(r => r.GetAsyncEnumerator(It.IsAny<CancellationToken>()))
.Returns(mockEnumerator.Object);
mockTextSearch.Setup(ts => ts.GetTextSearchResultsAsync(
It.IsAny<string>(),
It.IsAny<TextSearchOptions>(),
It.IsAny<CancellationToken>()))
.ReturnsAsync(new KernelSearchResults<TextSearchResult>(searchResults.Object));
var options = new TextSearchProviderOptions
{
ContextPrompt = overrideContextPrompt,
IncludeCitationsPrompt = overrideCitationsPrompt
};
var component = new TextSearchProvider(mockTextSearch.Object, options);
// Act
var result = await component.SearchAsync("Sample user question?", CancellationToken.None);
// Assert
Assert.Contains(expectedContextPrompt, result);
Assert.Contains("SourceDocName: Doc1", result);
Assert.Contains("SourceDocLink: http://example.com/doc1", result);
Assert.Contains("Contents: Content of Doc1", result);
Assert.Contains("SourceDocName: Doc2", result);
Assert.Contains("SourceDocLink: http://example.com/doc2", result);
Assert.Contains("Contents: Content of Doc2", result);
Assert.Contains(expectedCitationsPrompt, result);
}
[Fact]
public async Task ModelInvokingShouldUseOverrideContextFormatterIfProvidedAsync()
{
// Arrange
var mockTextSearch = new Mock<ITextSearch>();
var searchResults = new Mock<IAsyncEnumerable<TextSearchResult>>();
var mockEnumerator = new Mock<IAsyncEnumerator<TextSearchResult>>();
// Mock search results
var results = new List<TextSearchResult>
{
new("Content of Doc1") { Name = "Doc1", Link = "http://example.com/doc1" },
new("Content of Doc2") { Name = "Doc2", Link = "http://example.com/doc2" }
};
mockEnumerator.SetupSequence(e => e.MoveNextAsync())
.ReturnsAsync(true)
.ReturnsAsync(true)
.ReturnsAsync(false);
mockEnumerator.SetupSequence(e => e.Current)
.Returns(results[0])
.Returns(results[1]);
searchResults.Setup(r => r.GetAsyncEnumerator(It.IsAny<CancellationToken>()))
.Returns(mockEnumerator.Object);
mockTextSearch.Setup(ts => ts.GetTextSearchResultsAsync(
It.IsAny<string>(),
It.IsAny<TextSearchOptions>(),
It.IsAny<CancellationToken>()))
.ReturnsAsync(new KernelSearchResults<TextSearchResult>(searchResults.Object));
var options = new TextSearchProviderOptions
{
SearchTime = TextSearchProviderOptions.RagBehavior.BeforeAIInvoke,
Top = 2,
ContextFormatter = results => $"Custom formatted context with {results.Count} results."
};
var component = new TextSearchProvider(mockTextSearch.Object, options);
// Act
var result = await component.ModelInvokingAsync([new ChatMessage(ChatRole.User, "Sample user question?")], CancellationToken.None);
// Assert
Assert.Equal("Custom formatted context with 2 results.", result.Instructions);
}
}
dotnet/src/SemanticKernel.UnitTests/Data/TextSearchStoreTests.cs
+287
View File
@@ -0,0 +1,287 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.VectorData;
using Microsoft.SemanticKernel.Data;
using Moq;
using Xunit;
namespace SemanticKernel.UnitTests.Data;
public class TextSearchStoreTests
{
private readonly Mock<VectorStore> _vectorStoreMock;
private readonly Mock<VectorStoreCollection<string, TextSearchStore<string>.TextRagStorageDocument<string>>> _recordCollectionMock;
public TextSearchStoreTests()
{
this._vectorStoreMock = new Mock<VectorStore>();
this._recordCollectionMock = new Mock<VectorStoreCollection<string, TextSearchStore<string>.TextRagStorageDocument<string>>>();
this._vectorStoreMock
.Setup(v => v.GetCollection<string, TextSearchStore<string>.TextRagStorageDocument<string>>("testCollection", It.IsAny<VectorStoreCollectionDefinition>()))
.Returns(this._recordCollectionMock.Object);
}
[Fact]
public async Task UpsertDocumentsAsyncThrowsWhenDocumentsAreNull()
{
// Arrange
using var store = new TextSearchStore<string>(this._vectorStoreMock.Object, "testCollection", 128);
// Act & Assert
await Assert.ThrowsAsync<ArgumentNullException>(() => store.UpsertDocumentsAsync(null!));
}
[Fact]
public async Task UpsertTextAsyncThrowsWhenDocumentsAreNull()
{
// Arrange
using var store = new TextSearchStore<string>(this._vectorStoreMock.Object, "testCollection", 128);
// Act & Assert
await Assert.ThrowsAsync<ArgumentNullException>(() => store.UpsertTextAsync(null!));
}
[Theory]
[InlineData(null)]
[InlineData(" ")]
public async Task UpsertDocumentsAsyncThrowsDocumentTextIsNullOrWhiteSpace(string? text)
{
// Arrange
using var store = new TextSearchStore<string>(this._vectorStoreMock.Object, "testCollection", 128);
this._recordCollectionMock
.Setup(r => r.UpsertAsync(It.IsAny<IEnumerable<TextSearchStore<string>.TextRagStorageDocument<string>>>(), It.IsAny<CancellationToken>()))
.Callback(
(IEnumerable<TextSearchStore<string>.TextRagStorageDocument<string>> documents, CancellationToken token) =>
{
// Enumerate upserted documents to trigger the exception.
var a = documents.ToList();
})
.Returns(Task.CompletedTask);
var documents = new List<TextSearchDocument>
{
new() { Text = text }
};
// Act & Assert
await Assert.ThrowsAsync<ArgumentException>(() => store.UpsertDocumentsAsync(documents));
}
[Theory]
[InlineData(null)]
[InlineData(" ")]
public async Task UpsertTextAsyncThrowsDocumentTextIsNullOrWhiteSpace(string? text)
{
// Arrange
using var store = new TextSearchStore<string>(this._vectorStoreMock.Object, "testCollection", 128);
this._recordCollectionMock
.Setup(r => r.UpsertAsync(It.IsAny<IEnumerable<TextSearchStore<string>.TextRagStorageDocument<string>>>(), It.IsAny<CancellationToken>()))
.Callback(
(IEnumerable<TextSearchStore<string>.TextRagStorageDocument<string>> documents, CancellationToken token) =>
{
// Enumerate upserted documents to trigger the exception.
var a = documents.ToList();
})
.Returns(Task.CompletedTask);
// Act & Assert
await Assert.ThrowsAsync<ArgumentException>(() => store.UpsertTextAsync([text!]));
}
[Fact]
public async Task UpsertDocumentsAsyncCreatesCollectionUpsertsDocument()
{
// Arrange
this._recordCollectionMock
.Setup(r => r.UpsertAsync(It.IsAny<IEnumerable<TextSearchStore<string>.TextRagStorageDocument<string>>>(), It.IsAny<CancellationToken>()))
.Returns(Task.CompletedTask);
using var store = new TextSearchStore<string>(this._vectorStoreMock.Object, "testCollection", 128);
var documents = new List<TextSearchDocument>
{
new() { Text = "Sample text", Namespaces = ["ns1"], SourceId = "sid", SourceLink = "sl", SourceName = "sn" }
};
// Act
await store.UpsertDocumentsAsync(documents);
// Assert
this._recordCollectionMock.Verify(r => r.EnsureCollectionExistsAsync(It.IsAny<CancellationToken>()), Times.Once);
this._recordCollectionMock.Verify(r => r.UpsertAsync(
It.Is<IEnumerable<TextSearchStore<string>.TextRagStorageDocument<string>>>(doc =>
doc.Count() == 1 &&
doc.First().Text == "Sample text" &&
doc.First().Namespaces.Count == 1 &&
doc.First().Namespaces[0] == "ns1" &&
doc.First().SourceId == "sid" &&
doc.First().SourceLink == "sl" &&
doc.First().SourceName == "sn" &&
doc.First().TextEmbedding == "Sample text"),
It.IsAny<CancellationToken>()), Times.Once);
}
[Fact]
public async Task UpsertDocumentsAsyncCreatesCollectionUpsertsDocumentWithSourceIdAsId()
{
// Arrange
this._recordCollectionMock
.Setup(r => r.UpsertAsync(It.IsAny<IEnumerable<TextSearchStore<string>.TextRagStorageDocument<string>>>(), It.IsAny<CancellationToken>()))
.Returns(Task.CompletedTask);
using var store = new TextSearchStore<string>(this._vectorStoreMock.Object, "testCollection", 128, new() { UseSourceIdAsPrimaryKey = true });
var documents = new List<TextSearchDocument>
{
new() { Text = "Sample text", Namespaces = ["ns1"], SourceId = "sid", SourceLink = "sl", SourceName = "sn" }
};
// Act
await store.UpsertDocumentsAsync(documents);
// Assert
this._recordCollectionMock.Verify(r => r.UpsertAsync(
It.Is<IEnumerable<TextSearchStore<string>.TextRagStorageDocument<string>>>(doc =>
doc.Count() == 1 &&
doc.First().Key == "sid" &&
doc.First().Text == "Sample text" &&
doc.First().Namespaces.Count == 1 &&
doc.First().Namespaces[0] == "ns1" &&
doc.First().SourceId == "sid" &&
doc.First().SourceLink == "sl" &&
doc.First().SourceName == "sn" &&
doc.First().TextEmbedding == "Sample text"),
It.IsAny<CancellationToken>()), Times.Once);
}
[Fact]
public async Task UpsertDocumentsAsyncCreatesCollectionUpsertsDocumentWithoutSourceText()
{
// Arrange
this._recordCollectionMock
.Setup(r => r.UpsertAsync(It.IsAny<IEnumerable<TextSearchStore<string>.TextRagStorageDocument<string>>>(), It.IsAny<CancellationToken>()))
.Returns(Task.CompletedTask);
using var store = new TextSearchStore<string>(this._vectorStoreMock.Object, "testCollection", 128);
var documents = new List<TextSearchDocument>
{
new() { Text = "Sample text", Namespaces = ["ns1"], SourceId = "sid", SourceLink = "sl", SourceName = "sn" }
};
// Act
await store.UpsertDocumentsAsync(documents, new() { PersistSourceText = false });
// Assert
this._recordCollectionMock.Verify(r => r.UpsertAsync(
It.Is<IEnumerable<TextSearchStore<string>.TextRagStorageDocument<string>>>(doc =>
doc.Count() == 1 &&
doc.First().Text == null &&
doc.First().Namespaces.Count == 1 &&
doc.First().Namespaces[0] == "ns1" &&
doc.First().SourceId == "sid" &&
doc.First().SourceLink == "sl" &&
doc.First().SourceName == "sn" &&
doc.First().TextEmbedding == "Sample text"),
It.IsAny<CancellationToken>()), Times.Once);
}
[Fact]
public async Task UpsertTextAsyncCreatesCollectionUpsertsDocument()
{
// Arrange
this._recordCollectionMock
.Setup(r => r.UpsertAsync(It.IsAny<IEnumerable<TextSearchStore<string>.TextRagStorageDocument<string>>>(), It.IsAny<CancellationToken>()))
.Returns(Task.CompletedTask);
using var store = new TextSearchStore<string>(this._vectorStoreMock.Object, "testCollection", 128);
// Act
await store.UpsertTextAsync(["Sample text"]);
// Assert
this._recordCollectionMock.Verify(r => r.EnsureCollectionExistsAsync(It.IsAny<CancellationToken>()), Times.Once);
this._recordCollectionMock.Verify(r => r.UpsertAsync(
It.Is<IEnumerable<TextSearchStore<string>.TextRagStorageDocument<string>>>(doc =>
doc.Count() == 1 &&
doc.First().Text == "Sample text" &&
doc.First().Namespaces.Count == 0 &&
doc.First().SourceId == null &&
doc.First().SourceLink == null &&
doc.First().SourceName == null &&
doc.First().TextEmbedding == "Sample text"),
It.IsAny<CancellationToken>()), Times.Once);
}
[Fact]
public async Task SearchAsyncReturnsSearchResults()
{
// Arrange
var mockResults = new List<VectorSearchResult<TextSearchStore<string>.TextRagStorageDocument<string>>>
{
new(new TextSearchStore<string>.TextRagStorageDocument<string> { Text = "Sample text" }, 0.9f)
};
this._recordCollectionMock
.Setup(r => r.SearchAsync("query", 3, It.IsAny<VectorSearchOptions<TextSearchStore<string>.TextRagStorageDocument<string>>>(), It.IsAny<CancellationToken>()))
.Returns(mockResults.ToAsyncEnumerable());
using var store = new TextSearchStore<string>(this._vectorStoreMock.Object, "testCollection", 128);
// Act
var actualResults = await store.SearchAsync("query");
// Assert
var actualResultsList = await actualResults.Results.ToListAsync();
Assert.Single(actualResultsList);
Assert.Equal("Sample text", actualResultsList[0]);
}
[Fact]
public async Task SearchAsyncWithHydrationCallsCallbackAndReturnsSearchResults()
{
// Arrange
var mockResults = new List<VectorSearchResult<TextSearchStore<string>.TextRagStorageDocument<string>>>
{
new(new TextSearchStore<string>.TextRagStorageDocument<string> { SourceId = "sid1", SourceLink = "sl1", Text = "Sample text 1" }, 0.9f),
new(new TextSearchStore<string>.TextRagStorageDocument<string> { SourceId = "sid2", SourceLink = "sl2" }, 0.9f),
new(new TextSearchStore<string>.TextRagStorageDocument<string> { SourceId = "sid3", SourceLink = "sl3", Text = "Sample text 3" }, 0.9f),
};
this._recordCollectionMock
.Setup(r => r.SearchAsync("query", 3, It.IsAny<VectorSearchOptions<TextSearchStore<string>.TextRagStorageDocument<string>>>(), It.IsAny<CancellationToken>()))
.Returns(mockResults.ToAsyncEnumerable());
using var store = new TextSearchStore<string>(
this._vectorStoreMock.Object,
"testCollection",
128,
new()
{
SourceRetrievalCallback = sourceIds =>
{
Assert.Single(sourceIds);
Assert.Equal("sid2", sourceIds[0].SourceId);
Assert.Equal("sl2", sourceIds[0].SourceLink);
return Task.FromResult<IEnumerable<TextSearchStoreSourceRetrievalResponse>>([new TextSearchStoreSourceRetrievalResponse(new TextSearchStoreSourceRetrievalRequest("sid2", "sl2"), "Sample text 2")]);
}
});
// Act
var actualResults = await store.SearchAsync("query");
// Assert
var actualResultsList = await actualResults.Results.ToListAsync();
Assert.Equal(3, actualResultsList.Count);
Assert.Equal("Sample text 1", actualResultsList[0]);
Assert.Equal("Sample text 2", actualResultsList[1]);
Assert.Equal("Sample text 3", actualResultsList[2]);
}
}
dotnet/src/SemanticKernel.UnitTests/Memory/AIContextProviderTests.cs
+67
View File
@@ -0,0 +1,67 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.SemanticKernel;
using Moq;
using Xunit;
namespace SemanticKernel.UnitTests.Memory;
/// <summary>
/// Contains tests for the <see cref="AIContextProvider"/> class.
/// </summary>
public class AIContextProviderTests
{
[Fact]
public async Task ConversationCreatedAsyncBaseImplementationSucceeds()
{
// Arrange.
var mockPart = new Mock<AIContextProvider>() { CallBase = true };
// Act & Assert.
await mockPart.Object.ConversationCreatedAsync("threadId", CancellationToken.None);
}
[Fact]
public async Task MessageAddingAsyncBaseImplementationSucceeds()
{
// Arrange.
var mockPart = new Mock<AIContextProvider>() { CallBase = true };
var newMessage = new ChatMessage(ChatRole.User, "Hello");
// Act & Assert.
await mockPart.Object.MessageAddingAsync("threadId", newMessage, CancellationToken.None);
}
[Fact]
public async Task ConversationDeletingAsyncBaseImplementationSucceeds()
{
// Arrange.
var mockPart = new Mock<AIContextProvider>() { CallBase = true };
// Act & Assert.
await mockPart.Object.ConversationDeletingAsync("threadId", CancellationToken.None);
}
[Fact]
public async Task SuspendingAsyncBaseImplementationSucceeds()
{
// Arrange.
var mockPart = new Mock<AIContextProvider>() { CallBase = true };
// Act & Assert.
await mockPart.Object.SuspendingAsync("threadId", CancellationToken.None);
}
[Fact]
public async Task ResumingAsyncBaseImplementationSucceeds()
{
// Arrange.
var mockPart = new Mock<AIContextProvider>() { CallBase = true };
// Act & Assert.
await mockPart.Object.ResumingAsync("threadId", CancellationToken.None);
}
}
dotnet/src/SemanticKernel.UnitTests/Memory/AggregateAIContextProviderExtensionsTests.cs
+61
View File
@@ -0,0 +1,61 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Moq;
using Xunit;
namespace SemanticKernel.UnitTests.Memory;
/// <summary>
/// Tests for the AggregateAIContextProviderExtensions class.
/// </summary>
public class AggregateAIContextProviderExtensionsTests
{
[Fact]
public async Task OnNewMessageShouldConvertMessageAndInvokeRegisteredPartsAsync()
{
// Arrange
var manager = new AggregateAIContextProvider();
var partMock = new Mock<AIContextProvider>();
manager.Add(partMock.Object);
var newMessage = new ChatMessageContent(AuthorRole.User, "Test Message");
// Act
await manager.MessageAddingAsync("test-thread-id", newMessage);
// Assert
partMock.Verify(x => x.MessageAddingAsync("test-thread-id", It.Is<ChatMessage>(m => m.Text == "Test Message" && m.Role == ChatRole.User), It.IsAny<CancellationToken>()), Times.Once);
}
[Fact]
public async Task OnAIInvocationShouldConvertMessagesInvokeRegisteredPartsAsync()
{
// Arrange
var manager = new AggregateAIContextProvider();
var partMock = new Mock<AIContextProvider>();
manager.Add(partMock.Object);
var messages = new List<ChatMessageContent>
{
new(AuthorRole.User, "Message 1"),
new(AuthorRole.Assistant, "Message 2")
};
partMock
.Setup(x => x.ModelInvokingAsync(It.IsAny<ICollection<ChatMessage>>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(new AIContext { Instructions = "Combined Context" });
// Act
var result = await manager.ModelInvokingAsync(messages);
// Assert
Assert.Equal("Combined Context", result.Instructions);
partMock.Verify(x => x.ModelInvokingAsync(It.Is<ICollection<ChatMessage>>(m => m.Count == 2), It.IsAny<CancellationToken>()), Times.Once);
}
}
dotnet/src/SemanticKernel.UnitTests/Memory/AggregateAIContextProviderTests.cs
+176
View File
@@ -0,0 +1,176 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.SemanticKernel;
using Moq;
using Xunit;
namespace SemanticKernel.UnitTests.Memory;
/// <summary>
/// Contains tests for the <see cref="AggregateAIContextProvider"/> class.
/// </summary>
public class AggregateAIContextProviderTests
{
[Fact]
public void ConstructorShouldInitializeEmptyPartsList()
{
// Act
var sut = new AggregateAIContextProvider();
// Assert
Assert.NotNull(sut.Providers);
Assert.Empty(sut.Providers);
}
[Fact]
public void ConstructorShouldInitializeWithProvidedParts()
{
// Arrange
var mockPart = new Mock<AIContextProvider>();
// Act
var sut = new AggregateAIContextProvider(new[] { mockPart.Object });
// Assert
Assert.Single(sut.Providers);
Assert.Contains(mockPart.Object, sut.Providers);
}
[Fact]
public void AddShouldRegisterNewPart()
{
// Arrange
var sut = new AggregateAIContextProvider();
var mockPart = new Mock<AIContextProvider>();
// Act
sut.Add(mockPart.Object);
// Assert
Assert.Single(sut.Providers);
Assert.Contains(mockPart.Object, sut.Providers);
}
[Fact]
public void AddFromServiceProviderShouldRegisterPartsFromServiceProvider()
{
// Arrange
var serviceCollection = new ServiceCollection();
var mockPart = new Mock<AIContextProvider>();
serviceCollection.AddSingleton(mockPart.Object);
var serviceProvider = serviceCollection.BuildServiceProvider();
var sut = new AggregateAIContextProvider();
// Act
sut.AddFromServiceProvider(serviceProvider);
// Assert
Assert.Single(sut.Providers);
Assert.Contains(mockPart.Object, sut.Providers);
}
[Fact]
public async Task OnThreadCreatedAsyncShouldCallOnThreadCreatedOnAllParts()
{
// Arrange
var sut = new AggregateAIContextProvider();
var mockPart = new Mock<AIContextProvider>();
sut.Add(mockPart.Object);
// Act
await sut.ConversationCreatedAsync("test-thread-id");
// Assert
mockPart.Verify(x => x.ConversationCreatedAsync("test-thread-id", It.IsAny<CancellationToken>()), Times.Once);
}
[Fact]
public async Task OnThreadDeleteAsyncShouldCallOnThreadDeleteOnAllParts()
{
// Arrange
var sut = new AggregateAIContextProvider();
var mockPart = new Mock<AIContextProvider>();
sut.Add(mockPart.Object);
// Act
await sut.ConversationDeletingAsync("test-thread-id");
// Assert
mockPart.Verify(x => x.ConversationDeletingAsync("test-thread-id", It.IsAny<CancellationToken>()), Times.Once);
}
[Fact]
public async Task OnNewMessageAsyncShouldCallOnNewMessageOnAllParts()
{
// Arrange
var sut = new AggregateAIContextProvider();
var mockPart = new Mock<AIContextProvider>();
var message = new ChatMessage(ChatRole.User, "Hello");
sut.Add(mockPart.Object);
// Act
await sut.MessageAddingAsync("test-thread-id", message);
// Assert
mockPart.Verify(x => x.MessageAddingAsync("test-thread-id", message, It.IsAny<CancellationToken>()), Times.Once);
}
[Fact]
public async Task OnAIInvocationAsyncShouldAggregateContextsFromAllParts()
{
// Arrange
var sut = new AggregateAIContextProvider();
var mockPart1 = new Mock<AIContextProvider>();
var mockPart2 = new Mock<AIContextProvider>();
mockPart1.Setup(x => x.ModelInvokingAsync(It.IsAny<ICollection<ChatMessage>>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(new AIContext { Instructions = "Context1" });
mockPart2.Setup(x => x.ModelInvokingAsync(It.IsAny<ICollection<ChatMessage>>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(new AIContext { Instructions = "Context2" });
sut.Add(mockPart1.Object);
sut.Add(mockPart2.Object);
var messages = new List<ChatMessage>();
// Act
var result = await sut.ModelInvokingAsync(messages);
// Assert
Assert.Equal("Context1\nContext2", result.Instructions);
}
[Fact]
public async Task OnSuspendAsyncShouldCallOnSuspendOnAllParts()
{
// Arrange
var sut = new AggregateAIContextProvider();
var mockPart = new Mock<AIContextProvider>();
sut.Add(mockPart.Object);
// Act
await sut.SuspendingAsync("test-thread-id");
// Assert
mockPart.Verify(x => x.SuspendingAsync("test-thread-id", It.IsAny<CancellationToken>()), Times.Once);
}
[Fact]
public async Task OnResumeAsyncShouldCallOnResumeOnAllParts()
{
// Arrange
var sut = new AggregateAIContextProvider();
var mockPart = new Mock<AIContextProvider>();
sut.Add(mockPart.Object);
// Act
await sut.ResumingAsync("test-thread-id");
// Assert
mockPart.Verify(x => x.ResumingAsync("test-thread-id", It.IsAny<CancellationToken>()), Times.Once);
}
}
dotnet/src/SemanticKernel.UnitTests/Memory/Mem0ProviderTests.cs
+187
View File
@@ -0,0 +1,187 @@
// Copyright (c) Microsoft. All rights reserved.
#pragma warning disable CA1054 // URI-like parameters should not be strings
using System;
using System.Net.Http;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.SemanticKernel.Memory;
using Moq;
using Xunit;
namespace SemanticKernel.UnitTests.Memory;
/// <summary>
/// Contains tests for the <see cref="Mem0Provider"/> class.
/// </summary>
public class Mem0ProviderTests : IDisposable
{
private readonly HttpClient _httpClient;
private readonly Mock<MockableMessageHandler> _mockMessageHandler;
private bool _disposedValue;
public Mem0ProviderTests()
{
this._mockMessageHandler = new Mock<MockableMessageHandler>() { CallBase = true };
this._httpClient = new HttpClient(this._mockMessageHandler.Object)
{
BaseAddress = new Uri("https://localhost/fakepath")
};
}
[Fact]
public void ValidatesHttpClientBaseAddress()
{
// Arrange
using var httpClientWithoutBaseAddress = new HttpClient();
// Act & Assert
var exception = Assert.Throws<ArgumentException>(() =>
{
new Mem0Provider(httpClientWithoutBaseAddress);
});
Assert.Equal("The BaseAddress of the provided httpClient parameter must be set. (Parameter 'httpClient')", exception.Message);
}
[Theory]
[InlineData(false, "test-thread-id")]
[InlineData(true, "test-thread-id-1")]
public async Task PostsMemoriesOnNewMessage(bool scopePerOperationThread, string expectedThreadId)
{
// Arrange
using var httpResponse = new HttpResponseMessage() { StatusCode = System.Net.HttpStatusCode.OK };
this._mockMessageHandler
.Setup(x => x.MockableSendAsync(It.IsAny<HttpMethod>(), It.IsAny<string>(), It.IsAny<string>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(httpResponse);
var sut = new Mem0Provider(this._httpClient, new() { ApplicationId = "test-app-id", AgentId = "test-agent-id", ThreadId = "test-thread-id", UserId = "test-user-id", ScopeToPerOperationThreadId = scopePerOperationThread });
// Act
await sut.MessageAddingAsync("test-thread-id-1", new ChatMessage(ChatRole.User, "Hello, my name is Caoimhe."));
// Assert
var expectedPayload = $$"""
{"app_id":"test-app-id","agent_id":"test-agent-id","run_id":"{{expectedThreadId}}","user_id":"test-user-id","messages":[{"content":"Hello, my name is Caoimhe.","role":"user"}]}
""";
this._mockMessageHandler.Verify(x => x.MockableSendAsync(HttpMethod.Post, "https://localhost/v1/memories/", expectedPayload, It.IsAny<CancellationToken>()), Times.Once);
}
[Theory]
[InlineData(false, "test-thread-id", null, "## Memories\nConsider the following memories when answering user questions:\nName is Caoimhe")]
[InlineData(true, "test-thread-id-1", "Custom Prompt:", "Custom Prompt:\nName is Caoimhe")]
public async Task SearchesForMemoriesOnModelInvoke(bool scopePerOperationThread, string expectedThreadId, string? customContextPrompt, string expectedAdditionalInstructions)
{
// Arrange
var expectedResponseString = """
[{"id":"1","memory":"Name is Caoimhe","hash":"abc123","metadata":null,"score":0.9,"created_at":"2023-01-01T00:00:00Z","updated_at":null,"user_id":"test-user-id","app_id":null,"agent_id":"test-agent-id","session_id":"test-thread-id-1"}]
""";
using var httpResponse = new HttpResponseMessage()
{
StatusCode = System.Net.HttpStatusCode.OK,
Content = new StringContent(expectedResponseString, Encoding.UTF8, "application/json")
};
this._mockMessageHandler
.Setup(x => x.MockableSendAsync(It.IsAny<HttpMethod>(), It.IsAny<string>(), It.IsAny<string>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(httpResponse);
var sut = new Mem0Provider(this._httpClient, new()
{
ApplicationId = "test-app-id",
AgentId = "test-agent-id",
ThreadId = "test-thread-id",
UserId = "test-user-id",
ScopeToPerOperationThreadId = scopePerOperationThread,
ContextPrompt = customContextPrompt
});
await sut.ConversationCreatedAsync("test-thread-id-1");
// Act
var actual = await sut.ModelInvokingAsync(new[] { new ChatMessage(ChatRole.User, "What is my name?") });
// Assert
var expectedPayload = $$"""
{"app_id":"test-app-id","agent_id":"test-agent-id","run_id":"{{expectedThreadId}}","user_id":"test-user-id","query":"What is my name?"}
""";
this._mockMessageHandler.Verify(x => x.MockableSendAsync(HttpMethod.Post, "https://localhost/v1/memories/search/", expectedPayload, It.IsAny<CancellationToken>()), Times.Once);
Assert.Equal(expectedAdditionalInstructions, actual.Instructions);
}
[Theory]
[InlineData(false, "test-thread-id")]
[InlineData(true, "test-thread-id-1")]
public async Task ClearsStoredUserFacts(bool scopePerOperationThread, string expectedThreadId)
{
// Arrange
using var httpResponse = new HttpResponseMessage() { StatusCode = System.Net.HttpStatusCode.OK };
this._mockMessageHandler
.Setup(x => x.MockableSendAsync(It.IsAny<HttpMethod>(), It.IsAny<string>(), null, It.IsAny<CancellationToken>()))
.ReturnsAsync(httpResponse);
var sut = new Mem0Provider(this._httpClient, new() { ApplicationId = "test-app-id", AgentId = "test-agent-id", ThreadId = "test-thread-id", UserId = "test-user-id", ScopeToPerOperationThreadId = scopePerOperationThread });
await sut.ConversationCreatedAsync("test-thread-id-1");
// Act
await sut.ClearStoredMemoriesAsync();
// Assert
var expectedUrl = $"https://localhost/v1/memories/?app_id=test-app-id&agent_id=test-agent-id&run_id={expectedThreadId}&user_id=test-user-id";
this._mockMessageHandler.Verify(x => x.MockableSendAsync(HttpMethod.Delete, expectedUrl, null, It.IsAny<CancellationToken>()), Times.Once);
}
[Fact]
public async Task ThrowsExceptionWhenThreadIdChangesAfterBeingSet()
{
// Arrange
var sut = new Mem0Provider(this._httpClient, new() { ScopeToPerOperationThreadId = true });
// Act
await sut.ConversationCreatedAsync("initial-thread-id");
// Assert
var exception = await Assert.ThrowsAsync<InvalidOperationException>(async () =>
{
await sut.ConversationCreatedAsync("new-thread-id");
});
Assert.Equal("The Mem0Provider can only be used with one thread at a time when ScopeToPerOperationThreadId is set to true.", exception.Message);
}
protected virtual void Dispose(bool disposing)
{
if (!this._disposedValue)
{
if (disposing)
{
this._httpClient.Dispose();
}
this._disposedValue = true;
}
}
public void Dispose()
{
this.Dispose(disposing: true);
GC.SuppressFinalize(this);
}
public class MockableMessageHandler : DelegatingHandler
{
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
string? contentString = request.Content is null ? null : await request.Content.ReadAsStringAsync(cancellationToken);
return await this.MockableSendAsync(request.Method, request.RequestUri?.AbsoluteUri, contentString, cancellationToken);
}
public virtual Task<HttpResponseMessage> MockableSendAsync(HttpMethod method, string? absoluteUri, string? content, CancellationToken cancellationToken)
{
throw new NotImplementedException();
}
}
}
dotnet/src/SemanticKernel.UnitTests/Memory/WhiteboardProviderTests.cs
+50
View File
@@ -0,0 +1,50 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.SemanticKernel.Memory;
using Moq;
using Xunit;
namespace SemanticKernel.UnitTests.Memory;
public class WhiteboardProviderTests
{
private readonly Mock<IChatClient> _mockChatClient;
public WhiteboardProviderTests()
{
this._mockChatClient = new Mock<IChatClient>();
}
[Fact]
public async Task ShouldInvokeAIAndReturnWhiteboardMessagesInContextAsync()
{
// Arrange
var sut = new WhiteboardProvider(this._mockChatClient.Object);
var chatMessage = new ChatMessage(ChatRole.User, "I want to create a presentation");
this._mockChatClient
.Setup(service => service.GetResponseAsync(
It.IsAny<IEnumerable<ChatMessage>>(),
It.IsAny<ChatOptions?>(),
It.IsAny<CancellationToken>()))
.ReturnsAsync(new ChatResponse(new ChatMessage(ChatRole.Assistant, """{"newWhiteboard":["REQUIREMENT - User wants to create a presentation."]}""")));
// Act
await sut.MessageAddingAsync(null, chatMessage);
// Assert
this._mockChatClient.Verify(service => service.GetResponseAsync(
It.Is<IEnumerable<ChatMessage>>(x => x.Count() == 1 && x.First().Text.Contains("I want to create a presentation")),
It.IsAny<ChatOptions?>(),
It.IsAny<CancellationToken>()), Times.Once);
await sut.WhenProcessingCompleteAsync();
var actualContext = await sut.ModelInvokingAsync([chatMessage]);
Assert.Contains("REQUIREMENT - User wants to create a presentation", actualContext.Instructions);
}
}
dotnet/src/SemanticKernel.UnitTests/SemanticKernel.UnitTests.csproj
+1 -1
View File
@@ -6,7 +6,7 @@
<TargetFrameworks>net8.0</TargetFrameworks> <!-- ;net48 -->
<IsTestProject>true</IsTestProject>
<IsPackable>false</IsPackable>
<NoWarn>$(NoWarn);CA2007,CA1861,IDE1006,VSTHRD111,SKEXP0001,SKEXP0010,SKEXP0050,SKEXP0110,SKEXP0120,MEVD9000</NoWarn>
<NoWarn>$(NoWarn);CA2007,CA1861,IDE1006,VSTHRD111,SKEXP0001,SKEXP0010,SKEXP0050,SKEXP0110,SKEXP0120,SKEXP0130,MEVD9000</NoWarn>
</PropertyGroup>
<ItemGroup>