microsoft/semantic-kernel
1
0
Fork 0
mirror of https://github.com/microsoft/semantic-kernel.git synced 2026-10-12 02:19:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'microsoft:main' into update-modelid

Browse Source
This commit is contained in:
Sandra Ahlgrimm
2024-01-16 15:40:51 +01:00
committed by GitHub
parent 4277528ea0 f9bef6375c
commit 355eb68765
111 changed files with 3744 additions and 1460 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/dotnet-format.yml
+2 -2
View File
@@ -26,8 +26,8 @@ jobs:
matrix:
include:
#- { dotnet: '6.0', configuration: Release, os: ubuntu-latest }
- { dotnet: '7.0', configuration: Release, os: ubuntu-latest }
#- { dotnet: '8.0', configuration: Release, os: ubuntu-latest }
#- { dotnet: '7.0', configuration: Release, os: ubuntu-latest }
- { dotnet: '8.0', configuration: Release, os: ubuntu-latest }
runs-on: ${{ matrix.os }}
env:
docs/FAQS.md
+55
View File
@@ -0,0 +1,55 @@
# Frequently Asked Questions
### How do I get access to nightly builds?
Nightly builds of the Semantic Kernel are available [here](https://github.com/orgs/microsoft/packages?repo_name=semantic-kernel).
To download nightly builds follow the following steps:
1. You will need a GitHub account to complete these steps.
1. Create a GitHub Personal Access Token with the `read:packages` scope using these [instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic).
1. If you account is part of the Microsoft organization then you must authorize the `Microsoft` organization as a single sign-on organization.
1. Click the "Configure SSO" next to the Person Access Token you just created and then authorize `Microsoft`.
1. Use the following command to add the Microsoft GitHub Packages source to your NuGet configuration:
```powershell
dotnet nuget add source --username GITHUBUSERNAME --password GITHUBPERSONALACCESSTOKEN --store-password-in-clear-text --name GitHubMicrosoft "https://nuget.pkg.github.com/microsoft/index.json"
```
1. Or you can manually create a `NuGet.Config` file.
```xml
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
<add key="github" value="https://nuget.pkg.github.com/microsoft/index.json" />
</packageSources>
<packageSourceMapping>
<packageSource key="nuget.org">
<package pattern="*" />
</packageSource>
<packageSource key="github">
<package pattern="*nightly"/>
</packageSource>
</packageSourceMapping>
<packageSourceCredentials>
<github>
<add key="Username" value="<Your GitHub Id>" />
<add key="ClearTextPassword" value="<Your Personal Access Token>" />
</github>
</packageSourceCredentials>
</configuration>
```
* If you place this file in your project folder make sure to have Git (or whatever source control you use) ignore it.
* For more information on where to store this file go [here](https://learn.microsoft.com/en-us/nuget/reference/nuget-config-file).
* You can also use the following command `he Microsoft GitHub Packages source can be added easier to NuGet:`
1. You can now add packages from the nightly build to your project.
* E.g. use this command `dotnet add package Microsoft.SemanticKernel.Core --version 0.26.231003.1-nightly`
1. And the latest package release can be referenced in the project like this:
* `<PackageReference Include="Microsoft.SemanticKernel" Version="*-*" />`
For more information see: <https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-nuget-registry>
docs/decisions/0030-branching-strategy.md
+130
View File
@@ -0,0 +1,130 @@
---
# These are optional elements. Feel free to remove any of them.
status: proposed
contact: SergeyMenshykh
date: 2024-01-04
deciders: markwallace-microsoft
consulted: rogerbarreto, dmytrostruk
informed:
---
# SK Branching Strategy
## Industry-adopted branching strategies
There are several industry-adopted branching strategies for Git, such as GitHub Flow, Git-Flow, and GitLab Flow. However, we will only focus on the two most widely-used ones: GitHub Flow and Git-Flow.
### GitHub Flow
GitHub Flow is a straightforward branching strategy that centres around the 'main' branch. Developers create a new branch for each feature or bugfix, make changes, submit a pull request, and merge the changes back to the 'main' branch. Releases are done directly from the 'main' branch, making this model ideal for projects with continuous integration/deployment. Learn more about [GitHub Flow](https://docs.github.com/en/get-started/quickstart/github-flow).
<img src="./diagrams/git-hub-flow.png" alt="GitFlow" width="500"/>
[Image source](https://www.abtasty.com/blog/git-branching-strategies/)
Pros:
- Straightforward with fewer branches to manage and less merge conflicts.
- No long running development branches.
Cons:
- Not as well organized as Git-Flow.
- The 'main' branch can get cluttered more easily since it functions as both the production and development branch.
### Git-Flow
Git-Flow is a branching strategy that organizes software development around two long-lived main branches, 'main' and 'develop', along with short-lived feature, release, and hotfix branches. Developers work on new features in feature branches, which are then merged into the 'develop' branch. When preparing for a release, to avoid blocking future release features, a release branch is created, and once finalized (testing & bug fixing), it is merged into both 'main' and 'develop'. Hotfix branches in Git Flow are created from the 'main' branch to address critical bug fixes and are subsequently merged back into both the 'main' and 'develop' branches. The actual release(deployable artifact) is done from the 'main' branch that is reflects actual production worthy official releases. Learn more about [Git-Flow](https://nvie.com/posts/a-successful-git-branching-model/).
<img src="./diagrams/git-flow.png" alt="GitFlow" width="700"/>
Pros:
- Clear separation between code under development and production-ready code.
- Efficient release management.
Cons:
- More complex than GitHub Flow, which may be overwhelming for smaller teams or projects that do not require as much structure.
- Less suited for projects that prioritize continuous deployment, as it emphasizes a more controlled release process.
- Not ideal for projects with continuous deployment due to the overhead of managing multiple branches.
- Spaghetti history in Git - [GitFlow considered harmful](https://www.endoflineblog.com/gitflow-considered-harmful)
# SK branching strategies
Today, the SK SDK is available in three languages: .NET, Java and Python. All of them coexist in the same Git repository, organized under corresponding folders. However, the branching strategies for those differ.
For both .NET and Python versions, development takes place in short-lived topic branches that branch off the 'main' branch. These topic branches are merged back into the 'main' branch when features are considered production-ready through PR reviews, unit tests, and integration test runs. Releases are carried out directly from the 'main' branch. This approach aligns with the GitHub Flow branching strategy, with a minor deviation where releases are conducted weekly rather than being continuously deployed.
The Java version of SK adheres to the Git-Flow strategy by being developed in a dedicated development branch. Topic branches are created from the development branch and merged back through pull requests after unit tests and integration test runs. Release branches are also created from the development branch and merged to both the development branch and the 'main' one when a release is considered production-ready. This strategy deviates slightly from vanilla Git-Flow in that release artifacts are generated from release branches rather than from the 'main' branch.
## Decision Drivers
- The strategy should be easy to implement and maintain without requiring significant investments.
- The strategy should allow for maintaining several releases in parallel if required.
- Ideally, the strategy is intuitive and simple so that everyone familiar with Git can adopt and follow it.
- Ideally, all SK languages are able to adopt and use the same branching strategy.
- Ability to continually deploy new release with minimal overhead.
- Ability to release language versions independently and on different schedules.
- Allow the .Net, Java and Python teams to be able to operate independently.
- Ability to patch a release (for all languages).
- Consolidation of PR's and Issues to simplify the triage and review process.
Another aspect to consider when deciding on a branching strategy for SK is access permissions and action scopes. GitHub does not allow enforcing access restrictions on just a part of a repository, such as a folder. This means that it is not possible to restrict SK .NET contributors from pushing Python PRs, which ideally should be done by the corresponding team. However, GitHub does allow assigning access permissions to a branch, which can be successfully leveraged if the appropriate strategy option is chosen. The similar issue occurs with GitHub's required actions/status checks, which can only be set at the branch level. Considering that development for .NET and Python takes place in the 'main' branch, and status checks are configured per branch rather than per folder, it is not possible to configure separate status checks for .NET and Python PRs. As a result, the same status check runs for both .NET and Python PRs, even though it may not be relevant to a specific language.
!["Net PR status checks"](./diagrams/net-pr-status-checks.png)
Regardless of the chosen strategy, it should be possible to support multiple versions of SK. For example, applying a bug fix or a security patch to released SK v1.1.0 and v2.4.0 should be feasible while working on v3.0.0. One way to achieve this would be to create a release branch for each SK release. So that the required patch/fix can be pushed to the branch and released from it. However, marking released commits with tags should suffice, as it is always possible to create a new branch from a tag retrospectively when needed, if at all. Existing release pipelines should accept a source branch as a parameter, enabling releases from any branch and not only from the 'main' one.
## Considered Options
### Repository per SK language
This option suggests having a separate GitHub repository for each SK language. These repositories can be created under a corresponding organization. Development and releases will follow the GitHub flow, with new features and fixes being developed in topic branches that created from the 'main' branch and eventually merged back.
Pros:
- Each repository will have only language-specific status checks and actions.
- Branch commits and release history will not contain irrelevant commits or releases.
- Utilizes the familiar GitHub Flow without Git-Flow overhead, resulting in a shorter learning curve.
- Access permissions are limited to the specific owning team.
Cons:
- There is an initial overhead in setting up the three repositories.
- There may be potential ongoing maintenance overhead for the three repositories.
- Secrets must be managed across three repositories instead of just one.
- Each repo will have a backlog that will have to be managed separately.
### Branch per SK language
This option involves having a dedicated, language-specific development branch for each SDK language: 'net-development', 'java-development', and 'python-development'. SDK Java is already using this option. Development and releases will follow the GitHub Flow, with new features and fixes being developed in topic branches that are branched off the corresponding language branch and eventually merged back.
Pros:
- Simple, language specific, status checks, actions and rules configured per language branch.
- Allow only teams that own language-specific branches to push or merge to them, rather than just approving PRs.
- Branch commits history does not contain irrelevant commits.
Cons:
- GitHub release history contains releases for all languages.
- Language-specific branches may not be straightforward to discover/use.
This option has two sub-options that define the way the 'main' branch is used:
1. The 'main' branch will contain general/common artifacts such as documentation, GitHub actions, and samples. All language folders will be removed from the 'main' branch, and it can be locked to prevent accidental merges.
2. The 'main' branch will include everything that dev branches have for discoverability purposes. A job/action will be implemented to merge commits from dev branches to the 'main' branch. The number of common artifacts between SK languages should be minimized to reduce the potential for merge conflicts. A solution for the squash merge problem that SK Java is experiencing today should be found before deciding on the sub-option.
The second sub-option is preferred over the first one due to its discoverability benefits. There is no need to select a development branch in the GitHub UI when searching for something in the repository. The 'main' branch is selected by default, and as soon as the latest bits are in the branch, they can be found easily. This intuitive approach is familiar to many, and changing it by requiring the selection of a branch before searching would complicate the search experience and introduce frustration.
### All SK languages in the 'main'
This option assumes maintaining the code for all SK languages - .NET, Java, and Python in the 'main' branch. Development would occur using typical topic branches, while releases would also be made from the 'main' branch. This is the strategy currently adopted by .NET and Python, and corresponds to the GitHub Flow.
Pros:
- All code in one place - the 'main' branch.
- Familiar GitHub Flow, no Git-Flow overhead - shorter learning curve.
Cons:
- Branch commits/release history contains irrelevant commits/releases.
- Complex and irrelevant GitHub status checks/actions.
- PRs can be pushed by non-owner teams.
### Current 'Hybrid' approach
This choice keeps the existing method used by SK. .NET and Python development is done in the 'main' branch using GitHub Flow, while Java development happens in the java-development branch following Git-Flow.
Pros:
- No changes required.
- Each SK language uses a strategy that is convenient for it.
Cons:
- Branch commits/release history contains irrelevant commits/releases.
- Complex and irrelevant GitHub status checks/actions.
- PRs can be pushed by non-owner teams.
## Decision Outcome
Chosen option: "Current 'Hybrid' approach" because it works with minor inefficiencies (such as cluttered release history and multi-language complex actions) and requires no investments now. Later, depending on the team size and the problems the team encounters with the "Current 'Hybrid' approach," we may consider either the 'Repository per SK language' option or the 'Branch per SK language' one.
docs/decisions/diagrams/git-flow.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 191 KiB

docs/decisions/diagrams/git-hub-flow.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 185 KiB

docs/decisions/diagrams/net-pr-status-checks.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 69 KiB

dotnet/Directory.Packages.props
+1 -1
View File
@@ -63,7 +63,7 @@
<PackageVersion Include="Microsoft.OpenApi" Version="1.6.11" />
<PackageVersion Include="Microsoft.OpenApi.Readers" Version="1.6.11" />
<PackageVersion Include="Google.Apis.CustomSearchAPI.v1" Version="[1.60.0.3001, )" />
<PackageVersion Include="Grpc.Net.Client" Version="2.59.0" />
<PackageVersion Include="Grpc.Net.Client" Version="2.60.0" />
<PackageVersion Include="protobuf-net" Version="3.2.30" />
<PackageVersion Include="protobuf-net.Reflection" Version="3.2.12" />
<PackageVersion Include="CoreCLR-NCalc" Version="2.2.113" />
dotnet/SK-dotnet.sln
+1 -5
View File
@@ -8,11 +8,6 @@ EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{831DDCA2-7D2C-4C31-80DB-6BDB3E1F7AE0}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "samples", "samples", "{FA3720F1-C99A-49B2-9577-A940257098BF}"
ProjectSection(SolutionItems) = preProject
..\samples\dotnet\Directory.Build.props = ..\samples\dotnet\Directory.Build.props
..\samples\dotnet\Directory.Build.targets = ..\samples\dotnet\Directory.Build.targets
..\samples\dotnet\Directory.Packages.props = ..\samples\dotnet\Directory.Packages.props
EndProjectSection
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "KernelSyntaxExamples", "samples\KernelSyntaxExamples\KernelSyntaxExamples.csproj", "{47C6F821-5103-431F-B3B8-A2868A68BB78}"
EndProject
@@ -29,6 +24,7 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution
..\.editorconfig = ..\.editorconfig
..\.gitignore = ..\.gitignore
Directory.Build.props = Directory.Build.props
Directory.Build.targets = Directory.Build.targets
Directory.Packages.props = Directory.Packages.props
..\.github\workflows\dotnet-format.yml = ..\.github\workflows\dotnet-format.yml
..\nuget.config = ..\nuget.config
dotnet/nuget/nuget-package.props
+8 -1
View File
@@ -1,7 +1,7 @@
<Project>
<PropertyGroup>
<!-- Central version prefix - applies to all nuget packages. -->
<VersionPrefix>1.0.1</VersionPrefix>
<VersionPrefix>1.1.0</VersionPrefix>
<PackageVersion Condition="'$(VersionSuffix)' != ''">$(VersionPrefix)-$(VersionSuffix)</PackageVersion>
<PackageVersion Condition="'$(VersionSuffix)' == ''">$(VersionPrefix)</PackageVersion>
@@ -9,6 +9,13 @@
<Configurations>Debug;Release;Publish</Configurations>
<IsPackable>true</IsPackable>
<!-- Package validation. Baseline Version should be lower than current version. -->
<PackageValidationBaselineVersion>1.0.1</PackageValidationBaselineVersion>
<!-- Validate assembly attributes only for Publish builds -->
<NoWarn Condition="'$(Configuration)' != 'Publish'">$(NoWarn);CP0003</NoWarn>
<!-- Do not validate reference assemblies -->
<NoWarn>$(NoWarn);CP1002</NoWarn>
<!-- Default description and tags. Packages can override. -->
<Authors>Microsoft</Authors>
<Company>Microsoft</Company>
dotnet/samples/KernelSyntaxExamples/Example09_FunctionTypes.cs
+3 -2
View File
@@ -52,8 +52,8 @@ public static class Example09_FunctionTypes
await kernel.InvokeAsync(plugin[nameof(LocalExamplePlugin.TaskInjectingKernelWithInputTextAndStringResult)],
new()
{
["textToSummarize"] = @"C# is a modern, versatile language by Microsoft, blending the efficiency of C++
with Visual Basic's simplicity. It's ideal for a wide range of applications,
["textToSummarize"] = @"C# is a modern, versatile language by Microsoft, blending the efficiency of C++
with Visual Basic's simplicity. It's ideal for a wide range of applications,
emphasizing type safety, modularity, and modern programming paradigms."
});
@@ -238,6 +238,7 @@ public class LocalExamplePlugin
/// <summary>
/// Example how to inject CultureInfo or IFormatProvider in your function
/// </summary>
[KernelFunction]
public async Task<string> TaskInjectingCultureInfoOrIFormatProviderWithStringResult(CultureInfo cultureInfo, IFormatProvider formatProvider)
{
var result = $"Culture Name: {cultureInfo.Name}, FormatProvider Equals CultureInfo?: {formatProvider.Equals(cultureInfo)}";
dotnet/samples/KernelSyntaxExamples/Example59_OpenAIFunctionCalling.cs
+2 -2
View File
@@ -29,7 +29,7 @@ public static class Example59_OpenAIFunctionCalling
Kernel kernel = builder.Build();
// Add a plugin with some helper functions we want to allow the model to utilize.
kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("HelperFunctions", new[]
kernel.ImportPluginFromFunctions("HelperFunctions", new[]
{
kernel.CreateFunctionFromMethod(() => DateTime.UtcNow.ToString("R"), "GetCurrentUtcTime", "Retrieves the current time in UTC."),
kernel.CreateFunctionFromMethod((string cityName) =>
@@ -44,7 +44,7 @@ public static class Example59_OpenAIFunctionCalling
"Tel Aviv" => "80 and sunny",
_ => "31 and snowing",
}, "Get_Weather_For_City", "Gets the current weather for the specified city"),
}));
});
Console.WriteLine("======== Example 1: Use automated function calling with a non-streaming prompt ========");
{
dotnet/samples/KernelSyntaxExamples/Example65_HandlebarsPlanner.cs
+36 -32
View File
@@ -23,22 +23,22 @@ public static class Example65_HandlebarsPlanner
{
s_sampleIndex = 1;
// Plugin with Complex Types as inputs and outputs
await RunLocalDictionaryWithComplexTypesSampleAsync(shouldPrintPrompt: true);
// Plugin with primitive types as inputs and outputs
await PlanNotPossibleSampleAsync();
await RunDictionaryWithBasicTypesSampleAsync();
await RunPoetrySampleAsync();
await RunBookSampleAsync();
await PlanNotPossibleSampleAsync();
// Plugin with Complex Types as inputs and outputs
await RunLocalDictionaryWithComplexTypesSampleAsync();
// OpenAPI plugin
await RunCourseraSampleAsync(true);
await RunCourseraSampleAsync(shouldPrintPrompt: true);
}
private static void WriteSampleHeadingToConsole(string name)
{
Console.WriteLine($"======== [Handlebars Planner] Sample {s_sampleIndex++} - Create and Execute {name} Plan ========");
Console.WriteLine($"======== [Handlebars Planner] Sample {s_sampleIndex++} - Create and Execute Plan with: {name} ========");
}
private static async Task RunSampleAsync(string goal, bool shouldPrintPrompt = false, params string[] pluginDirectoryNames)
@@ -63,28 +63,31 @@ public static class Example65_HandlebarsPlanner
modelId: chatModelId)
.Build();
if (pluginDirectoryNames[0] == StringParamsDictionaryPlugin.PluginName)
if (pluginDirectoryNames.Length > 0)
{
kernel.ImportPluginFromType<StringParamsDictionaryPlugin>(StringParamsDictionaryPlugin.PluginName);
}
else if (pluginDirectoryNames[0] == ComplexParamsDictionaryPlugin.PluginName)
{
kernel.ImportPluginFromType<ComplexParamsDictionaryPlugin>(ComplexParamsDictionaryPlugin.PluginName);
}
else if (pluginDirectoryNames[0] == CourseraPluginName)
{
await kernel.ImportPluginFromOpenApiAsync(
CourseraPluginName,
new Uri("https://www.coursera.org/api/rest/v1/search/openapi.yaml")
);
}
else
{
string folder = RepoFiles.SamplePluginsPath();
foreach (var pluginDirectoryName in pluginDirectoryNames)
if (pluginDirectoryNames[0] == StringParamsDictionaryPlugin.PluginName)
{
kernel.ImportPluginFromPromptDirectory(Path.Combine(folder, pluginDirectoryName));
kernel.ImportPluginFromType<StringParamsDictionaryPlugin>(StringParamsDictionaryPlugin.PluginName);
}
else if (pluginDirectoryNames[0] == ComplexParamsDictionaryPlugin.PluginName)
{
kernel.ImportPluginFromType<ComplexParamsDictionaryPlugin>(ComplexParamsDictionaryPlugin.PluginName);
}
else if (pluginDirectoryNames[0] == CourseraPluginName)
{
await kernel.ImportPluginFromOpenApiAsync(
CourseraPluginName,
new Uri("https://www.coursera.org/api/rest/v1/search/openapi.yaml")
);
}
else
{
string folder = RepoFiles.SamplePluginsPath();
foreach (var pluginDirectoryName in pluginDirectoryNames)
{
kernel.ImportPluginFromPromptDirectory(Path.Combine(folder, pluginDirectoryName));
}
}
}
@@ -127,7 +130,8 @@ public static class Example65_HandlebarsPlanner
}
catch (KernelException ex) when (
ex.Message.Contains(nameof(HandlebarsPlannerErrorCodes.InsufficientFunctionsForGoal), StringComparison.CurrentCultureIgnoreCase)
|| ex.Message.Contains(nameof(HandlebarsPlannerErrorCodes.HallucinatedHelpers), StringComparison.CurrentCultureIgnoreCase))
|| ex.Message.Contains(nameof(HandlebarsPlannerErrorCodes.HallucinatedHelpers), StringComparison.CurrentCultureIgnoreCase)
|| ex.Message.Contains(nameof(HandlebarsPlannerErrorCodes.InvalidTemplate), StringComparison.CurrentCultureIgnoreCase))
{
/*
Unable to create plan for goal with available functions.
@@ -144,13 +148,13 @@ public static class Example65_HandlebarsPlanner
private static async Task RunCourseraSampleAsync(bool shouldPrintPrompt = false)
{
WriteSampleHeadingToConsole("Coursera");
WriteSampleHeadingToConsole("Coursera OpenAPI Plugin");
await RunSampleAsync("Show me courses about Artificial Intelligence.", shouldPrintPrompt, CourseraPluginName);
}
private static async Task RunDictionaryWithBasicTypesSampleAsync(bool shouldPrintPrompt = false)
{
WriteSampleHeadingToConsole("Dictionary");
WriteSampleHeadingToConsole("Basic Types using Local Dictionary Plugin");
await RunSampleAsync("Get a random word and its definition.", shouldPrintPrompt, StringParamsDictionaryPlugin.PluginName);
/*
Original plan:
@@ -170,7 +174,7 @@ public static class Example65_HandlebarsPlanner
private static async Task RunLocalDictionaryWithComplexTypesSampleAsync(bool shouldPrintPrompt = false)
{
WriteSampleHeadingToConsole("Complex Types with Local Dictionary Plugin");
WriteSampleHeadingToConsole("Complex Types using Local Dictionary Plugin");
await RunSampleAsync("Teach me two random words and their definition.", shouldPrintPrompt, ComplexParamsDictionaryPlugin.PluginName);
/*
Original Plan:
@@ -204,7 +208,7 @@ public static class Example65_HandlebarsPlanner
private static async Task RunPoetrySampleAsync(bool shouldPrintPrompt = false)
{
WriteSampleHeadingToConsole("Poetry");
WriteSampleHeadingToConsole("Multiple Plugins");
await RunSampleAsync("Write a poem about John Doe, then translate it into Italian.", shouldPrintPrompt, "SummarizePlugin", "WriterPlugin");
/*
Original plan:
@@ -231,7 +235,7 @@ public static class Example65_HandlebarsPlanner
private static async Task RunBookSampleAsync(bool shouldPrintPrompt = false)
{
WriteSampleHeadingToConsole("Book Creation");
WriteSampleHeadingToConsole("Loops and Conditionals");
await RunSampleAsync("Create a book with 3 chapters about a group of kids in a club called 'The Thinking Caps.'", shouldPrintPrompt, "WriterPlugin", "MiscPlugin");
/*
Original plan:
dotnet/samples/KernelSyntaxExamples/Example70_Agents.cs
+8 -4
View File
@@ -58,6 +58,7 @@ public static class Example70_Agent
await ChatAsync(
"Agents.ParrotAgent.yaml", // Defined under ./Resources/Agents
plugin: null, // No plugin
arguments: new KernelArguments { { "count", 3 } },
"Fortune favors the bold.",
"I came, I saw, I conquered.",
"Practice makes perfect.");
@@ -77,6 +78,7 @@ public static class Example70_Agent
await ChatAsync(
"Agents.ToolAgent.yaml", // Defined under ./Resources/Agents
plugin,
arguments: null,
"Hello",
"What is the special soup?",
"What is the special drink?",
@@ -95,14 +97,15 @@ public static class Example70_Agent
var function = KernelFunctionFactory.CreateFromPrompt(
"Correct any misspelling or gramatical errors provided in input: {{$input}}",
functionName: "spellChecker",
description: "Correct the spelling for the user input."
);
description: "Correct the spelling for the user input.");
var plugin = KernelPluginFactory.CreateFromFunctions("spelling", "Spelling functions", new[] { function });
// Call the common chat-loop
await ChatAsync(
"Agents.ToolAgent.yaml", // Defined under ./Resources/Agents
plugin,
arguments: null,
"Hello",
"Is this spelled correctly: exercize",
"What is the special soup?",
@@ -126,7 +129,7 @@ public static class Example70_Agent
try
{
// Invoke agent plugin.
var response = await agent.AsPlugin().InvokeAsync("Practice makes perfect.");
var response = await agent.AsPlugin().InvokeAsync("Practice makes perfect.", new KernelArguments { { "count", 2 } });
// Display result.
Console.WriteLine(response ?? $"No response from agent: {agent.Id}");
@@ -149,6 +152,7 @@ public static class Example70_Agent
private static async Task ChatAsync(
string resourcePath,
KernelPlugin? plugin = null,
KernelArguments? arguments = null,
params string[] messages)
{
// Read agent resource
@@ -170,7 +174,7 @@ public static class Example70_Agent
Console.WriteLine($"[{agent.Id}]");
// Process each user message and agent response.
foreach (var response in messages.Select(m => thread.InvokeAsync(agent, m)))
foreach (var response in messages.Select(m => thread.InvokeAsync(agent, m, arguments)))
{
await foreach (var message in response)
{
dotnet/samples/KernelSyntaxExamples/Example75_AgentTools.cs
+168
View File
@@ -0,0 +1,168 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.SemanticKernel.Experimental.Agents;
// ReSharper disable once InconsistentNaming
/// <summary>
/// Showcase usage of code_interpreter and retrieval tools.
/// </summary>
public static class Example75_AgentTools
{
/// <summary>
/// Specific model is required that supports agents and parallel function calling.
/// Currently this is limited to Open AI hosted services.
/// </summary>
private const string OpenAIFunctionEnabledModel = "gpt-4-1106-preview";
// Track agents for clean-up
private static readonly List<IAgent> s_agents = new();
/// <summary>
/// Show how to utilize code_interpreter and retrieval tools.
/// </summary>
public static async Task RunAsync()
{
Console.WriteLine("======== Example75_AgentTools ========");
if (TestConfiguration.OpenAI.ApiKey == null)
{
Console.WriteLine("OpenAI apiKey not found. Skipping example.");
return;
}
// Run agent with 'code_interpreter' tool
await RunCodeInterpreterToolAsync();
// Run agent with 'retrieval' tool
await RunRetrievalToolAsync();
}
private static async Task RunCodeInterpreterToolAsync()
{
Console.WriteLine("======== Run:CodeInterpreterTool ========");
var builder =
new AgentBuilder()
.WithOpenAIChatCompletion(OpenAIFunctionEnabledModel, TestConfiguration.OpenAI.ApiKey)
.WithInstructions("Write only code to solve the given problem without comment.");
try
{
var defaultAgent =
Track(
await builder.BuildAsync());
var codeInterpreterAgent =
Track(
await builder.WithCodeInterpreter().BuildAsync());
await ChatAsync(
defaultAgent,
codeInterpreterAgent,
"What is the solution to `3x + 2 = 14`?",
"What is the fibinacci sequence until 101?");
}
finally
{
await Task.WhenAll(s_agents.Select(a => a.DeleteAsync()));
}
}
private static async Task RunRetrievalToolAsync()
{
Console.WriteLine("======== Run:RunRetrievalTool ========");
// REQUIRED:
//
// Use `curl` to upload document prior to running example and assign the
// identifier to `fileId`.
//
// Powershell:
// curl https://api.openai.com/v1/files `
// -H "Authorization: Bearer $Env:OPENAI_APIKEY" `
// -F purpose="assistants" `
// -F file="@Resources/travelinfo.txt"
var fileId = "<see comment>";
var defaultAgent =
await new AgentBuilder()
.WithOpenAIChatCompletion(OpenAIFunctionEnabledModel, TestConfiguration.OpenAI.ApiKey)
.BuildAsync();
var retrievalAgent =
await new AgentBuilder()
.WithOpenAIChatCompletion(OpenAIFunctionEnabledModel, TestConfiguration.OpenAI.ApiKey)
.WithRetrieval(fileId)
.BuildAsync();
try
{
await ChatAsync(
defaultAgent,
retrievalAgent,
"Where did sam go?",
"When does the flight leave Seattle?",
"What is the hotel contact info at the destination?");
}
finally
{
await Task.WhenAll(s_agents.Select(a => a.DeleteAsync()));
}
}
/// <summary>
/// Common chat loop used for: RunCodeInterpreterToolAsync and RunRetrievalToolAsync.
/// Processes each question for both "default" and "enabled" agents.
/// </summary>
private static async Task ChatAsync(
IAgent defaultAgent,
IAgent enabledAgent,
params string[] questions)
{
foreach (var question in questions)
{
Console.WriteLine("\nDEFAULT AGENT:");
await InvokeAgentAsync(defaultAgent, question);
Console.WriteLine("\nTOOL ENABLED AGENT:");
await InvokeAgentAsync(enabledAgent, question);
}
static async Task InvokeAgentAsync(IAgent agent, string question)
{
await foreach (var message in agent.InvokeAsync(question))
{
string content = message.Content;
foreach (var annotation in message.Annotations)
{
content = content.Replace(annotation.Label, string.Empty, StringComparison.Ordinal);
}
Console.WriteLine($"# {message.Role}: {content}");
if (message.Annotations.Count > 0)
{
Console.WriteLine("\n# files:");
foreach (var annotation in message.Annotations)
{
Console.WriteLine($"* {annotation.FileId}");
}
}
}
Console.WriteLine();
}
}
private static IAgent Track(IAgent agent)
{
s_agents.Add(agent);
return agent;
}
}
dotnet/samples/KernelSyntaxExamples/KernelSyntaxExamples.csproj
+2
View File
@@ -21,6 +21,7 @@
<ItemGroup>
<None Remove="Resources\GenerateStory.yaml" />
<None Remove="Resources\GenerateStoryHandlebars.yaml" />
<None Remove="Resources\travelinfo.txt" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Azure.Identity" />
@@ -93,5 +94,6 @@
<EmbeddedResource Include="Resources\GenerateStoryHandlebars.yaml">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</EmbeddedResource>
<EmbeddedResource Include="Resources\travelinfo.txt" />
</ItemGroup>
</Project>
dotnet/samples/KernelSyntaxExamples/Resources/Agents/ParrotAgent.yaml
+7 -2
View File
@@ -1,4 +1,9 @@
name: Parrot
instructions: |
Repeat the user message in the voice of a pirate and then end with a parrot sound.
template_format: semantic-kernel
template: |
Repeat the user message in the voice of a pirate and then end with {{$count}} parrot sounds.
description: A fun chat bot that repeats the user message in the voice of a pirate.
input_variables:
- name: count
description: The number of parrot sounds.
is_required: true
dotnet/samples/KernelSyntaxExamples/Resources/Agents/ToolAgent.yaml
+2 -1
View File
@@ -1,5 +1,6 @@
name: ToolRunner
instructions: |
template_format: semantic-kernel
template: |
Respond to the user using the single best tool.
If no tool is appropriate, let the user know you only provide responses using tools.
When reporting a tool result, start with, "The tool I used informed me that"
dotnet/samples/KernelSyntaxExamples/Resources/travelinfo.txt
+217
View File
@@ -0,0 +1,217 @@
Invoice Booking Reference LMNOPQ Trip ID - 11110011111
Passenger Name(s)
MARKS/SAM ALBERT Agent W2
MICROSOFT CORPORATION 14820 NE 36TH STREET REDMOND WA US 98052
American Express Global Business Travel Microsoft Travel
14711 NE 29th Place, Suite 215
Bellevue, WA 98007
Phone: +1 (669) 210-8041
BILLING CODE : 1010-10010110
Invoice Information
Invoice Details
Ticket Number
0277993883295
Charges
Ticket Base Fare
306.29
Airline Name
ALASKA AIRLINES
Ticket Tax Fare 62.01
Passenger Name Flight Details
MARKS/SAM ALBERT
11 Sep 2023 ALASKA AIRLINES
0572 H Class
SEATTLE-TACOMA,WA/RALEIGH DURHAM,NC
13 Sep 2023 ALASKA AIRLINES
0491 M Class
RALEIGH DURHAM,NC/SEATTLE- TACOMA,WA
Total (USD) Ticket Amount
368.30
Credit Card Information
Charged to Card
AX XXXXXXXXXXX4321
368.30
Payment Details
Charged by Airline
Total Invoice Charge
USD
368.30
368.30
Monday 11 September 2023
10:05 AM
Seattle (SEA) to Durham (RDU)
Airline Booking Ref: ABCXYZ
Carrier: ALASKA AIRLINES
Flight: AS 572
Status: Confirmed
Operated By: ALASKA AIRLINES
Origin: Seattle, WA, Seattle-Tacoma International Apt (SEA)
Departing: Monday 11 September 2023 at 10:05 AM Destination: Durham, Raleigh, Raleigh (RDU) Arriving: Monday 11 September 2023 at 06:15 PM
Additional Information
Departure Terminal: Not Applicable
Arrival Terminal: TERMINAL 2
Class: ECONOMY
Aircraft Type: Boeing 737-900
Meal Service: Not Applicable
Frequent Flyer Number: Not Applicable
Number of Stops: 0
Greenhouse Gas Emissions: 560 kg CO2e / person
Distance: 2354 Miles Estimated Time: 05 hours 10 minutes
Seat: 24A
THE WESTIN RALEIGH DURHAM AP
Address: 3931 Macaw Street, Raleigh, NC, 27617, US
Phone: (1) 919-224-1400 Fax: (1) 919-224-1401
Check In Date: Monday 11 September 2023 Check Out Date: Wednesday 13 September 2023 Number Of Nights: 2
Rate: USD 280.00 per night may be subject to local taxes and service charges
Guaranteed to: AX XXXXXXXXXXX4321
Reference Number: 987654
Additional Information
Membership ID: 123456789
CANCEL PERMITTED UP TO 1 DAYS BEFORE CHECKIN
Status: Confirmed
Corporate Id: Not Applicable
Number Of Rooms: 1
Wednesday 13 September 2023
07:15 PM
Durham (RDU) to Seattle (SEA)
Airline Booking Ref: ABCXYZ
Carrier: ALASKA AIRLINES
Flight: AS 491
Status: Confirmed
Operated By: ALASKA AIRLINES
Origin: Durham, Raleigh, Raleigh (RDU)
Departing: Wednesday 13 September 2023 at 07:15 PM
Departure Terminal: TERMINAL 2
Destination: Seattle, WA, Seattle-Tacoma International Apt (SEA)
Arriving: Wednesday 13 September 2023 at 09:59 PM Arrival Terminal: Not Applicable
Additional Information
Class: ECONOMY
Aircraft Type: Boeing 737-900
Meal Service: Not Applicable
Frequent Flyer Number: Not Applicable
Number of Stops: 0
Greenhouse Gas Emissions: 560 kg CO2e / person
Distance: 2354 Miles Estimated Time: 05 hours 44 minutes
Seat: 16A
Greenhouse Gas Emissions
Total Greenhouse Gas Emissions for this trip is: 1120 kg CO2e / person
Air Fare Information
Routing : ONLINE RESERVATION
Total Fare : USD 368.30
Additional Messages
FOR 24X7 Travel Reservations Please Call 1-669-210-8041 Unable To Use Requested As Frequent Flyer Program Invalid Use Of Frequent Flyer Number 0123XYZ Please Contact Corresponding Frequent Travel Program Support Desk For Assistance
Trip Name-Trip From Seattle To Raleigh/Durham
This Ticket Is Nonrefundable. Changes Or Cancellations Must Be Made Prior To Scheduled Flight Departure
All Changes Must Be Made On Same Carrier And Will Be Subject To Service Fee And Difference In Airfare
*******************************************************
Please Be Advised That Certain Mandatory Hotel-Imposed Charges Including But Not Limited To Daily Resort Or Facility Fees May Be Applicable To Your Stay And Payable To The Hotel Operator At Check-Out From The Property. You May Wish To Inquire With The Hotel Before Your Trip Regarding The Existence And Amount Of Such Charges.
*******************************************************
Hotel Cancel Policies Vary Depending On The Property And Date. If You Have Questions Regarding Cancellation Fees Please Call The Travel Office.
Important Information
COVID-19 Updates: Click here to access Travel Vitals https://travelvitals.amexgbt.com for the latest information and advisories compiled by American Express Global Business Travel.
Carbon Emissions: The total emissions value for this itinerary includes air travel only. Emissions for each individual flight are displayed in the flight details section. For more information on carbon emissions please refer to https://www.amexglobalbusinesstravel.com/sustainable-products-and-platforms.
For important information regarding your booking in relation to the conditions applying to your booking, managing your booking and travel advisory, please refer to www.amexglobalbusinesstravel.com/booking-info.
GBT Travel Services UK Limited (GBT UK) and its authorized sublicensees (including Ovation Travel Group and Egencia) use certain trademarks and service marks of American Express Company or its subsidiaries (American Express) in the American Express Global Business Travel and American Express Meetings & Events brands and in connection with its business for permitted uses only under a limited license from American Express (Licensed Marks). The Licensed Marks are trademarks or service marks of, and the property of, American Express. GBT UK is a subsidiary of Global Business Travel Group, Inc. (NYSE: GBTG). American Express holds a minority interest in GBTG, which operates as a separate company from American Express.
dotnet/src/Connectors/Connectors.OpenAI/AzureSdk/ClientCore.cs
+15 -5
View File
@@ -274,9 +274,12 @@ internal abstract class ClientCore
// Get our single result and extract the function call information. If this isn't a function call, or if it is
// but we're unable to find the function or extract the relevant information, just return the single result.
// Note that we don't check the FinishReason and instead check whether there are any tool calls, as the service
// may return a FinishReason of "stop" even if there are tool calls to be made, in particular if a required tool
// is specified.
ChatChoice resultChoice = responseData.Choices[0];
OpenAIChatMessageContent result = new(resultChoice.Message, this.DeploymentOrModelName, metadata);
if (resultChoice.FinishReason != CompletionsFinishReason.ToolCalls)
if (result.ToolCalls.Count == 0)
{
return new[] { result };
}
@@ -383,7 +386,7 @@ internal abstract class ClientCore
if (iteration >= chatExecutionSettings.ToolCallBehavior!.MaximumUseAttempts)
{
chatOptions.Tools.Clear();
// Set the tool choice to none. We'd also like to clear the tools, but doing so can make the service unhappy ("[] is too short - 'tools'").
chatOptions.ToolChoice = ChatCompletionsToolChoice.None;
if (this.Logger.IsEnabled(LogLevel.Debug))
{
@@ -458,9 +461,11 @@ internal abstract class ClientCore
yield return new OpenAIStreamingChatMessageContent(update, update.ChoiceIndex ?? 0, this.DeploymentOrModelName, metadata);
}
// If we don't have a function call to invoke, we're done.
// If we don't have a function to invoke, we're done.
// Note that we don't check the FinishReason and instead check whether there are any tool calls, as the service
// may return a FinishReason of "stop" even if there are tool calls to be made, in particular if a required tool
// is specified.
if (!autoInvoke ||
finishReason != CompletionsFinishReason.ToolCalls ||
toolCallIdsByIndex is not { Count: > 0 })
{
yield break;
@@ -572,7 +577,7 @@ internal abstract class ClientCore
if (iteration >= chatExecutionSettings.ToolCallBehavior!.MaximumUseAttempts)
{
chatOptions.Tools.Clear();
// Set the tool choice to none. We'd also like to clear the tools, but doing so can make the service unhappy ("[] is too short - 'tools'").
chatOptions.ToolChoice = ChatCompletionsToolChoice.None;
if (this.Logger.IsEnabled(LogLevel.Debug))
{
@@ -794,6 +799,11 @@ internal abstract class ClientCore
}
}
if (!string.IsNullOrWhiteSpace(executionSettings?.ChatSystemPrompt) && !chatHistory.Any(m => m.Role == AuthorRole.System))
{
options.Messages.Add(GetRequestMessage(new ChatMessageContent(AuthorRole.System, executionSettings!.ChatSystemPrompt)));
}
foreach (var message in chatHistory)
{
options.Messages.Add(GetRequestMessage(message));
dotnet/src/Connectors/Connectors.OpenAI/Connectors.OpenAI.csproj
+1
View File
@@ -7,6 +7,7 @@
<TargetFramework>netstandard2.0</TargetFramework>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<NoWarn>$(NoWarn);NU5104;SKEXP0013,SKEXP0014</NoWarn>
<EnablePackageValidation>true</EnablePackageValidation>
</PropertyGroup>
<!-- IMPORT NUGET PACKAGE SHARED PROPERTIES -->
dotnet/src/Connectors/Connectors.OpenAI/OpenAIServiceCollectionExtensions.cs
+36 -48
View File
@@ -160,25 +160,24 @@ public static class OpenAIServiceCollectionExtensions
/// </summary>
/// <param name="builder">The <see cref="IKernelBuilder"/> instance to augment.</param>
/// <param name="deploymentName">Azure OpenAI deployment name, see https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource</param>
/// <param name="openAIClient">Custom <see cref="OpenAIClient"/>.</param>
/// <param name="openAIClient"><see cref="OpenAIClient"/> to use for the service. If null, one must be available in the service provider when this service is resolved.</param>
/// <param name="serviceId">A local identifier for the given AI service</param>
/// <param name="modelId">Model identifier, see https://learn.microsoft.com/azure/cognitive-services/openai/quickstart</param>
/// <returns>The same instance as <paramref name="builder"/>.</returns>
public static IKernelBuilder AddAzureOpenAITextGeneration(
this IKernelBuilder builder,
string deploymentName,
OpenAIClient openAIClient,
OpenAIClient? openAIClient = null,
string? serviceId = null,
string? modelId = null)
{
Verify.NotNull(builder);
Verify.NotNullOrWhiteSpace(deploymentName);
Verify.NotNull(openAIClient);
builder.Services.AddKeyedSingleton<ITextGenerationService>(serviceId, (serviceProvider, _) =>
new AzureOpenAITextGenerationService(
deploymentName,
openAIClient,
openAIClient ?? serviceProvider.GetRequiredService<OpenAIClient>(),
modelId,
serviceProvider.GetService<ILoggerFactory>()));
@@ -190,25 +189,24 @@ public static class OpenAIServiceCollectionExtensions
/// </summary>
/// <param name="services">The <see cref="IServiceCollection"/> instance to augment.</param>
/// <param name="deploymentName">Azure OpenAI deployment name, see https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource</param>
/// <param name="openAIClient">Custom <see cref="OpenAIClient"/>.</param>
/// <param name="openAIClient"><see cref="OpenAIClient"/> to use for the service. If null, one must be available in the service provider when this service is resolved.</param>
/// <param name="serviceId">A local identifier for the given AI service</param>
/// <param name="modelId">Model identifier, see https://learn.microsoft.com/azure/cognitive-services/openai/quickstart</param>
/// <returns>The same instance as <paramref name="services"/>.</returns>
public static IServiceCollection AddAzureOpenAITextGeneration(
this IServiceCollection services,
string deploymentName,
OpenAIClient openAIClient,
OpenAIClient? openAIClient = null,
string? serviceId = null,
string? modelId = null)
{
Verify.NotNull(services);
Verify.NotNullOrWhiteSpace(deploymentName);
Verify.NotNull(openAIClient);
return services.AddKeyedSingleton<ITextGenerationService>(serviceId, (serviceProvider, _) =>
new AzureOpenAITextGenerationService(
deploymentName,
openAIClient,
openAIClient ?? serviceProvider.GetRequiredService<OpenAIClient>(),
modelId,
serviceProvider.GetService<ILoggerFactory>()));
}
@@ -280,23 +278,22 @@ public static class OpenAIServiceCollectionExtensions
/// </summary>
/// <param name="builder">The <see cref="IKernelBuilder"/> instance to augment.</param>
/// <param name="modelId">OpenAI model name, see https://platform.openai.com/docs/models</param>
/// <param name="openAIClient">Custom <see cref="OpenAIClient"/>.</param>
/// <param name="openAIClient"><see cref="OpenAIClient"/> to use for the service. If null, one must be available in the service provider when this service is resolved.</param>
/// <param name="serviceId">A local identifier for the given AI service</param>
/// <returns>The same instance as <paramref name="builder"/>.</returns>
public static IKernelBuilder AddOpenAITextGeneration(
this IKernelBuilder builder,
string modelId,
OpenAIClient openAIClient,
OpenAIClient? openAIClient = null,
string? serviceId = null)
{
Verify.NotNull(builder);
Verify.NotNullOrWhiteSpace(modelId);
Verify.NotNull(openAIClient);
builder.Services.AddKeyedSingleton<ITextGenerationService>(serviceId, (serviceProvider, _) =>
new OpenAITextGenerationService(
modelId,
openAIClient,
openAIClient ?? serviceProvider.GetRequiredService<OpenAIClient>(),
serviceProvider.GetService<ILoggerFactory>()));
return builder;
@@ -307,22 +304,21 @@ public static class OpenAIServiceCollectionExtensions
/// </summary>
/// <param name="services">The <see cref="IServiceCollection"/> instance to augment.</param>
/// <param name="modelId">OpenAI model name, see https://platform.openai.com/docs/models</param>
/// <param name="openAIClient">Custom <see cref="OpenAIClient"/>.</param>
/// <param name="openAIClient"><see cref="OpenAIClient"/> to use for the service. If null, one must be available in the service provider when this service is resolved.</param>
/// <param name="serviceId">A local identifier for the given AI service</param>
/// <returns>The same instance as <paramref name="services"/>.</returns>
public static IServiceCollection AddOpenAITextGeneration(this IServiceCollection services,
string modelId,
OpenAIClient openAIClient,
OpenAIClient? openAIClient = null,
string? serviceId = null)
{
Verify.NotNull(services);
Verify.NotNullOrWhiteSpace(modelId);
Verify.NotNull(openAIClient);
return services.AddKeyedSingleton<ITextGenerationService>(serviceId, (serviceProvider, _) =>
new OpenAITextGenerationService(
modelId,
openAIClient,
openAIClient ?? serviceProvider.GetRequiredService<OpenAIClient>(),
serviceProvider.GetService<ILoggerFactory>()));
}
@@ -479,7 +475,7 @@ public static class OpenAIServiceCollectionExtensions
/// </summary>
/// <param name="builder">The <see cref="IKernelBuilder"/> instance to augment.</param>
/// <param name="deploymentName">Azure OpenAI deployment name, see https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource</param>
/// <param name="openAIClient">Custom <see cref="OpenAIClient"/>.</param>
/// <param name="openAIClient"><see cref="OpenAIClient"/> to use for the service. If null, one must be available in the service provider when this service is resolved.</param>
/// <param name="serviceId">A local identifier for the given AI service</param>
/// <param name="modelId">Model identifier, see https://learn.microsoft.com/azure/cognitive-services/openai/quickstart</param>
/// <returns>The same instance as <paramref name="builder"/>.</returns>
@@ -487,18 +483,17 @@ public static class OpenAIServiceCollectionExtensions
public static IKernelBuilder AddAzureOpenAITextEmbeddingGeneration(
this IKernelBuilder builder,
string deploymentName,
OpenAIClient openAIClient,
OpenAIClient? openAIClient = null,
string? serviceId = null,
string? modelId = null)
{
Verify.NotNull(builder);
Verify.NotNullOrWhiteSpace(deploymentName);
Verify.NotNull(openAIClient);
builder.Services.AddKeyedSingleton<ITextEmbeddingGenerationService>(serviceId, (serviceProvider, _) =>
new AzureOpenAITextEmbeddingGenerationService(
deploymentName,
openAIClient,
openAIClient ?? serviceProvider.GetRequiredService<OpenAIClient>(),
modelId,
serviceProvider.GetService<ILoggerFactory>()));
@@ -510,7 +505,7 @@ public static class OpenAIServiceCollectionExtensions
/// </summary>
/// <param name="services">The <see cref="IServiceCollection"/> instance to augment.</param>
/// <param name="deploymentName">Azure OpenAI deployment name, see https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource</param>
/// <param name="openAIClient">Custom <see cref="OpenAIClient"/>.</param>
/// <param name="openAIClient"><see cref="OpenAIClient"/> to use for the service. If null, one must be available in the service provider when this service is resolved.</param>
/// <param name="serviceId">A local identifier for the given AI service</param>
/// <param name="modelId">Model identifier, see https://learn.microsoft.com/azure/cognitive-services/openai/quickstart</param>
/// <returns>The same instance as <paramref name="services"/>.</returns>
@@ -518,18 +513,17 @@ public static class OpenAIServiceCollectionExtensions
public static IServiceCollection AddAzureOpenAITextEmbeddingGeneration(
this IServiceCollection services,
string deploymentName,
OpenAIClient openAIClient,
OpenAIClient? openAIClient = null,
string? serviceId = null,
string? modelId = null)
{
Verify.NotNull(services);
Verify.NotNullOrWhiteSpace(deploymentName);
Verify.NotNull(openAIClient);
return services.AddKeyedSingleton<ITextEmbeddingGenerationService>(serviceId, (serviceProvider, _) =>
new AzureOpenAITextEmbeddingGenerationService(
deploymentName,
openAIClient,
openAIClient ?? serviceProvider.GetRequiredService<OpenAIClient>(),
modelId,
serviceProvider.GetService<ILoggerFactory>()));
}
@@ -603,24 +597,23 @@ public static class OpenAIServiceCollectionExtensions
/// </summary>
/// <param name="builder">The <see cref="IServiceCollection"/> instance to augment.</param>
/// <param name="modelId">OpenAI model name, see https://platform.openai.com/docs/models</param>
/// <param name="openAIClient">Custom <see cref="OpenAIClient"/>.</param>
/// <param name="openAIClient"><see cref="OpenAIClient"/> to use for the service. If null, one must be available in the service provider when this service is resolved.</param>
/// <param name="serviceId">A local identifier for the given AI service</param>
/// <returns>The same instance as <paramref name="builder"/>.</returns>
[Experimental("SKEXP0011")]
public static IKernelBuilder AddOpenAITextEmbeddingGeneration(
this IKernelBuilder builder,
string modelId,
OpenAIClient openAIClient,
OpenAIClient? openAIClient = null,
string? serviceId = null)
{
Verify.NotNull(builder);
Verify.NotNullOrWhiteSpace(modelId);
Verify.NotNull(openAIClient);
builder.Services.AddKeyedSingleton<ITextEmbeddingGenerationService>(serviceId, (serviceProvider, _) =>
new OpenAITextEmbeddingGenerationService(
modelId,
openAIClient,
openAIClient ?? serviceProvider.GetRequiredService<OpenAIClient>(),
serviceProvider.GetService<ILoggerFactory>()));
return builder;
@@ -631,23 +624,22 @@ public static class OpenAIServiceCollectionExtensions
/// </summary>
/// <param name="services">The <see cref="IServiceCollection"/> instance to augment.</param>
/// <param name="modelId">The OpenAI model id.</param>
/// <param name="openAIClient">Custom <see cref="OpenAIClient"/>.</param>
/// <param name="openAIClient"><see cref="OpenAIClient"/> to use for the service. If null, one must be available in the service provider when this service is resolved.</param>
/// <param name="serviceId">A local identifier for the given AI service</param>
/// <returns>The same instance as <paramref name="services"/>.</returns>
[Experimental("SKEXP0011")]
public static IServiceCollection AddOpenAITextEmbeddingGeneration(this IServiceCollection services,
string modelId,
OpenAIClient openAIClient,
OpenAIClient? openAIClient = null,
string? serviceId = null)
{
Verify.NotNull(services);
Verify.NotNullOrWhiteSpace(modelId);
Verify.NotNull(openAIClient);
return services.AddKeyedSingleton<ITextEmbeddingGenerationService>(serviceId, (serviceProvider, _) =>
new OpenAITextEmbeddingGenerationService(
modelId,
openAIClient,
openAIClient ?? serviceProvider.GetRequiredService<OpenAIClient>(),
serviceProvider.GetService<ILoggerFactory>()));
}
@@ -820,23 +812,22 @@ public static class OpenAIServiceCollectionExtensions
/// </summary>
/// <param name="builder">The <see cref="IKernelBuilder"/> instance to augment.</param>
/// <param name="deploymentName">Azure OpenAI deployment name, see https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource</param>
/// <param name="openAIClient">Custom <see cref="OpenAIClient"/>.</param>
/// <param name="openAIClient"><see cref="OpenAIClient"/> to use for the service. If null, one must be available in the service provider when this service is resolved.</param>
/// <param name="serviceId">A local identifier for the given AI service</param>
/// <param name="modelId">Model identifier, see https://learn.microsoft.com/azure/cognitive-services/openai/quickstart</param>
/// <returns>The same instance as <paramref name="builder"/>.</returns>
public static IKernelBuilder AddAzureOpenAIChatCompletion(
this IKernelBuilder builder,
string deploymentName,
OpenAIClient openAIClient,
OpenAIClient? openAIClient = null,
string? serviceId = null,
string? modelId = null)
{
Verify.NotNull(builder);
Verify.NotNullOrWhiteSpace(deploymentName);
Verify.NotNull(openAIClient);
Func<IServiceProvider, object?, AzureOpenAIChatCompletionService> factory = (serviceProvider, _) =>
new(deploymentName, openAIClient, modelId, serviceProvider.GetService<ILoggerFactory>());
new(deploymentName, openAIClient ?? serviceProvider.GetRequiredService<OpenAIClient>(), modelId, serviceProvider.GetService<ILoggerFactory>());
builder.Services.AddKeyedSingleton<IChatCompletionService>(serviceId, factory);
builder.Services.AddKeyedSingleton<ITextGenerationService>(serviceId, factory);
@@ -849,23 +840,22 @@ public static class OpenAIServiceCollectionExtensions
/// </summary>
/// <param name="services">The <see cref="IServiceCollection"/> instance to augment.</param>
/// <param name="deploymentName">Azure OpenAI deployment name, see https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource</param>
/// <param name="openAIClient">Custom <see cref="OpenAIClient"/>.</param>
/// <param name="openAIClient"><see cref="OpenAIClient"/> to use for the service. If null, one must be available in the service provider when this service is resolved.</param>
/// <param name="serviceId">A local identifier for the given AI service</param>
/// <param name="modelId">Model identifier, see https://learn.microsoft.com/azure/cognitive-services/openai/quickstart</param>
/// <returns>The same instance as <paramref name="services"/>.</returns>
public static IServiceCollection AddAzureOpenAIChatCompletion(
this IServiceCollection services,
string deploymentName,
OpenAIClient openAIClient,
OpenAIClient? openAIClient = null,
string? serviceId = null,
string? modelId = null)
{
Verify.NotNull(services);
Verify.NotNullOrWhiteSpace(deploymentName);
Verify.NotNull(openAIClient);
Func<IServiceProvider, object?, AzureOpenAIChatCompletionService> factory = (serviceProvider, _) =>
new(deploymentName, openAIClient, modelId, serviceProvider.GetService<ILoggerFactory>());
new(deploymentName, openAIClient ?? serviceProvider.GetRequiredService<OpenAIClient>(), modelId, serviceProvider.GetService<ILoggerFactory>());
services.AddKeyedSingleton<IChatCompletionService>(serviceId, factory);
services.AddKeyedSingleton<ITextGenerationService>(serviceId, factory);
@@ -1006,21 +996,20 @@ public static class OpenAIServiceCollectionExtensions
/// </summary>
/// <param name="builder">The <see cref="IKernelBuilder"/> instance to augment.</param>
/// <param name="modelId">OpenAI model id</param>
/// <param name="openAIClient">Custom <see cref="OpenAIClient"/> for HTTP requests.</param>
/// <param name="openAIClient"><see cref="OpenAIClient"/> to use for the service. If null, one must be available in the service provider when this service is resolved.</param>
/// <param name="serviceId">A local identifier for the given AI service</param>
/// <returns>The same instance as <paramref name="builder"/>.</returns>
public static IKernelBuilder AddOpenAIChatCompletion(
this IKernelBuilder builder,
string modelId,
OpenAIClient openAIClient,
OpenAIClient? openAIClient = null,
string? serviceId = null)
{
Verify.NotNull(builder);
Verify.NotNullOrWhiteSpace(modelId);
Verify.NotNull(openAIClient);
Func<IServiceProvider, object?, OpenAIChatCompletionService> factory = (serviceProvider, _) =>
new(modelId, openAIClient, serviceProvider.GetService<ILoggerFactory>());
new(modelId, openAIClient ?? serviceProvider.GetRequiredService<OpenAIClient>(), serviceProvider.GetService<ILoggerFactory>());
builder.Services.AddKeyedSingleton<IChatCompletionService>(serviceId, factory);
builder.Services.AddKeyedSingleton<ITextGenerationService>(serviceId, factory);
@@ -1033,20 +1022,19 @@ public static class OpenAIServiceCollectionExtensions
/// </summary>
/// <param name="services">The <see cref="IServiceCollection"/> instance to augment.</param>
/// <param name="modelId">OpenAI model id</param>
/// <param name="openAIClient">Custom <see cref="OpenAIClient"/> for HTTP requests.</param>
/// <param name="openAIClient"><see cref="OpenAIClient"/> to use for the service. If null, one must be available in the service provider when this service is resolved.</param>
/// <param name="serviceId">A local identifier for the given AI service</param>
/// <returns>The same instance as <paramref name="services"/>.</returns>
public static IServiceCollection AddOpenAIChatCompletion(this IServiceCollection services,
string modelId,
OpenAIClient openAIClient,
OpenAIClient? openAIClient = null,
string? serviceId = null)
{
Verify.NotNull(services);
Verify.NotNullOrWhiteSpace(modelId);
Verify.NotNull(openAIClient);
Func<IServiceProvider, object?, OpenAIChatCompletionService> factory = (serviceProvider, _) =>
new(modelId, openAIClient, serviceProvider.GetService<ILoggerFactory>());
new(modelId, openAIClient ?? serviceProvider.GetRequiredService<OpenAIClient>(), serviceProvider.GetService<ILoggerFactory>());
services.AddKeyedSingleton<IChatCompletionService>(serviceId, factory);
services.AddKeyedSingleton<ITextGenerationService>(serviceId, factory);
dotnet/src/Connectors/Connectors.OpenAI/ToolCallBehavior.cs
-2
View File
@@ -2,7 +2,6 @@
using System.Collections.Generic;
using System.Diagnostics;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using Azure.AI.OpenAI;
@@ -76,7 +75,6 @@ public abstract class ToolCallBehavior
/// The <see cref="ToolCallBehavior"/> that may be set into <see cref="OpenAIPromptExecutionSettings.ToolCallBehavior"/>
/// to indicate that the specified function should be requested by the model.
/// </returns>
[Experimental("SKEXP0013")]
public static ToolCallBehavior RequireFunction(OpenAIFunction function, bool autoInvoke = false)
{
Verify.NotNull(function);
dotnet/src/Connectors/Connectors.UnitTests/OpenAI/ChatCompletion/OpenAIChatCompletionTests.cs
+26 -2
View File
@@ -121,8 +121,8 @@ public sealed class OpenAIChatCompletionTests : IDisposable
var actualRequestContent = Encoding.UTF8.GetString(this._messageHandlerStub.RequestContent!);
Assert.NotNull(actualRequestContent);
var optionsJson = JsonSerializer.Deserialize<JsonElement>(actualRequestContent);
Assert.Equal(1, optionsJson.GetProperty("messages").GetArrayLength());
Assert.Equal("John Doe", optionsJson.GetProperty("messages")[0].GetProperty("tool_call_id").GetString());
Assert.Equal(2, optionsJson.GetProperty("messages").GetArrayLength());
Assert.Equal("John Doe", optionsJson.GetProperty("messages")[1].GetProperty("tool_call_id").GetString());
}
[Fact]
@@ -163,6 +163,30 @@ public sealed class OpenAIChatCompletionTests : IDisposable
Assert.Equal("gpt-3.5-turbo", textContent.ModelId);
}
[Fact]
public async Task ItAddsSystemMessageAsync()
{
// Arrange
var chatCompletion = new OpenAIChatCompletionService(modelId: "gpt-3.5-turbo", apiKey: "NOKEY", httpClient: this._httpClient);
this._messageHandlerStub.ResponseToReturn = new HttpResponseMessage(System.Net.HttpStatusCode.OK)
{ Content = new StringContent(ChatCompletionResponse) };
var chatHistory = new ChatHistory();
chatHistory.AddMessage(AuthorRole.User, "Hello");
// Act
await chatCompletion.GetChatMessageContentsAsync(chatHistory, this._executionSettings);
// Assert
var actualRequestContent = Encoding.UTF8.GetString(this._messageHandlerStub.RequestContent!);
Assert.NotNull(actualRequestContent);
var optionsJson = JsonSerializer.Deserialize<JsonElement>(actualRequestContent);
Assert.Equal(2, optionsJson.GetProperty("messages").GetArrayLength());
Assert.Equal("Assistant is a large language model.", optionsJson.GetProperty("messages")[0].GetProperty("content").GetString());
Assert.Equal("system", optionsJson.GetProperty("messages")[0].GetProperty("role").GetString());
Assert.Equal("Hello", optionsJson.GetProperty("messages")[1].GetProperty("content").GetString());
Assert.Equal("user", optionsJson.GetProperty("messages")[1].GetProperty("role").GetString());
}
public void Dispose()
{
this._httpClient.Dispose();
dotnet/src/Experimental/Agents.UnitTests/Extensions/KernelExtensionTests.cs
+1 -1
View File
@@ -20,7 +20,7 @@ public sealed class KernelExtensionTests
var function = KernelFunctionFactory.CreateFromMethod(() => { }, functionName: "Bogus");
var kernel = new Kernel();
kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("Fake", "Fake functions", new[] { function }));
kernel.ImportPluginFromFunctions("Fake", new[] { function });
//Act
var tool = kernel.GetAssistantTool(TwoPartToolName);
dotnet/src/Experimental/Agents/AgentBuilder.Static.cs
+1 -1
View File
@@ -54,6 +54,6 @@ public partial class AgentBuilder
var restContext = new OpenAIRestContext(apiKey);
var resultModel = await restContext.GetAssistantModelAsync(agentId, cancellationToken).ConfigureAwait(false);
return new Agent(resultModel, restContext, plugins);
return new Agent(resultModel, null, restContext, plugins);
}
}
dotnet/src/Experimental/Agents/AgentBuilder.cs
+76 -9
View File
@@ -3,13 +3,13 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.SemanticKernel.Experimental.Agents.Exceptions;
using Microsoft.SemanticKernel.Experimental.Agents.Internal;
using Microsoft.SemanticKernel.Experimental.Agents.Models;
using YamlDotNet.Serialization;
namespace Microsoft.SemanticKernel.Experimental.Agents;
@@ -20,9 +20,11 @@ public partial class AgentBuilder
{
private readonly AssistantModel _model;
private readonly KernelPluginCollection _plugins;
private readonly HashSet<string> _tools;
private readonly List<string> _fileIds;
private string? _apiKey;
private Func<HttpClient>? _httpClientProvider;
private PromptTemplateConfig? _config;
/// <summary>
/// Initializes a new instance of the <see cref="AgentBuilder"/> class.
@@ -31,6 +33,8 @@ public partial class AgentBuilder
{
this._model = new AssistantModel();
this._plugins = new KernelPluginCollection();
this._tools = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
this._fileIds = new List<string>();
}
/// <summary>
@@ -50,10 +54,14 @@ public partial class AgentBuilder
throw new AgentException("ApiKey must be provided for agent.");
}
this._model.Tools.AddRange(this._tools.Select(t => new ToolModel { Type = t }));
this._model.FileIds.AddRange(this._fileIds.Distinct(StringComparer.OrdinalIgnoreCase));
return
await Agent.CreateAsync(
new OpenAIRestContext(this._apiKey!, this._httpClientProvider),
this._model,
this._config,
this._plugins,
cancellationToken).ConfigureAwait(false);
}
@@ -77,15 +85,21 @@ public partial class AgentBuilder
/// <returns><see cref="AgentBuilder"/> instance for fluid expression.</returns>
public AgentBuilder FromTemplate(string template)
{
var deserializer = new DeserializerBuilder().Build();
this._config = KernelFunctionYaml.ToPromptTemplateConfig(template);
var agentKernelModel = deserializer.Deserialize<AgentConfigurationModel>(template);
this.WithInstructions(this._config.Template.Trim());
return
this
.WithInstructions(agentKernelModel.Instructions.Trim())
.WithName(agentKernelModel.Name.Trim())
.WithDescription(agentKernelModel.Description.Trim());
if (!string.IsNullOrWhiteSpace(this._config.Name))
{
this.WithName(this._config.Name?.Trim());
}
if (!string.IsNullOrWhiteSpace(this._config.Description))
{
this.WithDescription(this._config.Description?.Trim());
}
return this;
}
/// <summary>
@@ -169,6 +183,29 @@ public partial class AgentBuilder
return this;
}
/// <summary>
/// Enable the code-interpreter tool with this agent.
/// </summary>
/// <returns><see cref="AgentBuilder"/> instance for fluid expression.</returns>
public AgentBuilder WithCodeInterpreter()
{
this._tools.Add(Agent.ToolCodeInterpreter);
return this;
}
/// <summary>
/// Enable the retrieval tool with this agent.
/// </summary>
/// <param name="fileIds">Optional set of uploaded file identifiers.</param>
/// <returns><see cref="AgentBuilder"/> instance for fluid expression.</returns>
public AgentBuilder WithRetrieval(params string[] fileIds)
{
this._tools.Add(Agent.ToolRetrieval);
return this.WithFiles(fileIds);
}
/// <summary>
/// Define functions associated with agent instance (optional).
/// </summary>
@@ -193,4 +230,34 @@ public partial class AgentBuilder
return this;
}
/// <summary>
/// Associate an uploaded file with the agent, by identifier.
/// </summary>
/// <param name="fileId">The uploaded file identifier.</param>
/// <returns><see cref="AgentBuilder"/> instance for fluid expression.</returns>
public AgentBuilder WithFile(string fileId)
{
if (!string.IsNullOrWhiteSpace(fileId))
{
this._fileIds.Add(fileId);
}
return this;
}
/// <summary>
/// Associate uploaded files with the agent, by identifier.
/// </summary>
/// <param name="fileIds">The uploaded file identifiers.</param>
/// <returns><see cref="AgentBuilder"/> instance for fluid expression.</returns>
public AgentBuilder WithFiles(params string[] fileIds)
{
if (fileIds.Length > 0)
{
this._fileIds.AddRange(fileIds);
}
return this;
}
}
dotnet/src/Experimental/Agents/AgentCapability.cs
+32
View File
@@ -0,0 +1,32 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
namespace Microsoft.SemanticKernel.Experimental.Agents;
/// <summary>
/// Description of agent capabilities.
/// </summary>
[Flags]
public enum AgentCapability
{
/// <summary>
/// No additional capabilities.
/// </summary>
None = 0,
/// <summary>
/// Has function / plugin capability.
/// </summary>
Functions,
/// <summary>
/// Has document / data retrieval capability.
/// </summary>
Retrieval,
/// <summary>
/// Has code-interpereter capability.
/// </summary>
CodeInterpreter,
}
dotnet/src/Experimental/Agents/AgentPlugin.cs
+17 -2
View File
@@ -29,8 +29,23 @@ public abstract class AgentPlugin : KernelPlugin
/// <returns>The agent response</returns>
public async Task<string> InvokeAsync(string input, CancellationToken cancellationToken = default)
{
var args = new KernelArguments { { "input", input } };
var result = await this.First().InvokeAsync(this.Agent.Kernel, args, cancellationToken).ConfigureAwait(false);
return await this.InvokeAsync(input, arguments: null, cancellationToken).ConfigureAwait(false);
}
/// <summary>
/// Invoke plugin with user input
/// </summary>
/// <param name="input">The user input</param>
/// <param name="arguments">The arguments</param>
/// <param name="cancellationToken">A cancel token</param>
/// <returns>The agent response</returns>
public async Task<string> InvokeAsync(string input, KernelArguments? arguments, CancellationToken cancellationToken = default)
{
arguments ??= new KernelArguments();
arguments["input"] = input;
var result = await this.First().InvokeAsync(this.Agent.Kernel, arguments, cancellationToken).ConfigureAwait(false);
var response = result.GetValue<AgentResponse>()!;
return response.Message;
dotnet/src/Experimental/Agents/Experimental.Agents.csproj
+2 -1
View File
@@ -19,10 +19,11 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="System.Linq.Async" />
<PackageReference Include="YamlDotNet" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\Connectors\Connectors.OpenAI\Connectors.OpenAI.csproj" />
<ProjectReference Include="..\..\Extensions\PromptTemplates.Handlebars\PromptTemplates.Handlebars.csproj" />
<ProjectReference Include="..\..\Functions\Functions.Yaml\Functions.Yaml.csproj" />
<ProjectReference Include="..\..\SemanticKernel.Core\SemanticKernel.Core.csproj" />
</ItemGroup>
<ItemGroup>
dotnet/src/Experimental/Agents/Extensions/OpenAIRestExtensions.AssistantFiles.cs
+69
View File
@@ -0,0 +1,69 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Threading;
using System.Threading.Tasks;
using Microsoft.SemanticKernel.Experimental.Agents.Internal;
using Microsoft.SemanticKernel.Experimental.Agents.Models;
namespace Microsoft.SemanticKernel.Experimental.Agents;
/// <summary>
/// Supported OpenAI REST API actions for managing assistant files.
/// </summary>
internal static partial class OpenAIRestExtensions
{
/// <summary>
/// Associate uploaded file with the assistant, by identifier.
/// </summary>
/// <param name="context">A context for accessing OpenAI REST endpoint</param>
/// <param name="assistantId">The assistant identifier</param>
/// <param name="fileId">The identifier of the uploaded file.</param>
/// <param name="cancellationToken">A cancellation token</param>
/// <returns>An assistant definition</returns>
public static async Task<string> AddAssistantFileAsync(
this OpenAIRestContext context,
string assistantId,
string fileId,
CancellationToken cancellationToken = default)
{
var payload =
new
{
file_id = fileId
};
var result =
await context.ExecutePostAsync<AssistantModel.FileModel>(
GetAssistantFileUrl(assistantId),
payload,
cancellationToken).ConfigureAwait(false);
return result.Id;
}
/// <summary>
/// Disassociate uploaded file with from assistant, by identifier.
/// </summary>
/// <param name="context">A context for accessing OpenAI REST endpoint</param>
/// <param name="assistantId">The assistant identifier</param>
/// <param name="fileId">The identifier of the uploaded file.</param>
/// <param name="cancellationToken">A cancellation token</param>
public static Task RemoveAssistantFileAsync(
this OpenAIRestContext context,
string assistantId,
string fileId,
CancellationToken cancellationToken = default)
{
return context.ExecuteDeleteAsync(GetAssistantFileUrl(assistantId, fileId), cancellationToken);
}
internal static string GetAssistantFileUrl(string assistantId)
{
return $"{BaseAssistantUrl}/{assistantId}/files";
}
internal static string GetAssistantFileUrl(string assistantId, string fileId)
{
return $"{BaseAssistantUrl}/{assistantId}/files/{fileId}";
}
}
dotnet/src/Experimental/Agents/IAgent.cs
+39 -3
View File
@@ -1,7 +1,9 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.SemanticKernel.Experimental.Agents.Models;
namespace Microsoft.SemanticKernel.Experimental.Agents;
@@ -24,6 +26,11 @@ public interface IAgent
#pragma warning restore CA1716 // Identifiers should not match keywords
#pragma warning restore CA1720 // Identifier contains type name
/// <summary>
/// Identifies additional agent capabilities.
/// </summary>
AgentCapability Capabilities { get; }
/// <summary>
/// Unix timestamp (in seconds) for when the agent was created
/// </summary>
@@ -49,20 +56,35 @@ public interface IAgent
/// </summary>
string Instructions { get; }
/// <summary>
/// Identifiers of files associated with agent.
/// </summary>
IEnumerable<string> FileIds { get; }
/// <summary>
/// Tools defined for run execution.
/// </summary>
KernelPluginCollection Plugins { get; }
/// <summary>
/// A semantic-kernel <see cref="Kernel"/> instance associated with the agent.
/// </summary>
internal Kernel Kernel { get; }
/// <summary>
/// Tools defined for run execution.
/// Internal tools model.
/// </summary>
public KernelPluginCollection Plugins { get; }
internal IEnumerable<ToolModel> Tools { get; }
/// <summary>
/// Expose the agent as a plugin.
/// </summary>
public AgentPlugin AsPlugin();
AgentPlugin AsPlugin();
/// <summary>
/// Expose the agent internally as a prompt-template
/// </summary>
internal IPromptTemplate AsPromptTemplate();
/// <summary>
/// Creates a new agent chat thread.
@@ -84,6 +106,20 @@ public interface IAgent
/// <param name="cancellationToken">A cancellation token</param>
Task DeleteThreadAsync(string? id, CancellationToken cancellationToken = default);
/// <summary>
/// Associate uploaded file with the agent, by identifier.
/// </summary>
/// <param name="fileId">The identifier of the uploaded file.</param>
/// <param name="cancellationToken">A cancellation token</param>
Task AddFileAsync(string fileId, CancellationToken cancellationToken = default);
/// <summary>
/// Remove association of uploaded file with the agent, by identifier.
/// </summary>
/// <param name="fileId">The identifier of the uploaded file.</param>
/// <param name="cancellationToken">A cancellation token</param>
Task RemoveFileAsync(string fileId, CancellationToken cancellationToken = default);
/// <summary>
/// Delete current agent. Terminal state - Unable to perform any
/// subsequent actions.
dotnet/src/Experimental/Agents/IAgentExtensions.cs
+5 -3
View File
@@ -16,17 +16,19 @@ public static class IAgentExtensions
/// </summary>
/// <param name="agent">the agent</param>
/// <param name="input">the user input</param>
/// <param name="cancellationToken">a cancel token</param>
/// <returns>chat messages</returns>
/// <param name="arguments">Optional arguments for parameterized instructions</param>
/// <param name="cancellationToken">Optional cancellation token</param>
/// <returns>Chat messages</returns>
public static async IAsyncEnumerable<IChatMessage> InvokeAsync(
this IAgent agent,
string input,
KernelArguments? arguments = null,
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
IAgentThread thread = await agent.NewThreadAsync(cancellationToken).ConfigureAwait(false);
try
{
await foreach (var message in thread.InvokeAsync(agent, input, cancellationToken))
await foreach (var message in thread.InvokeAsync(agent, input, arguments, cancellationToken))
{
yield return message;
}
dotnet/src/Experimental/Agents/IAgentThread.cs
+4 -2
View File
@@ -28,18 +28,20 @@ public interface IAgentThread
/// Advance the thread with the specified agent.
/// </summary>
/// <param name="agent">An agent instance.</param>
/// <param name="arguments">Optional arguments for parameterized instructions</param>
/// <param name="cancellationToken">A cancellation token</param>
/// <returns>The resulting agent message(s)</returns>
IAsyncEnumerable<IChatMessage> InvokeAsync(IAgent agent, CancellationToken cancellationToken = default);
IAsyncEnumerable<IChatMessage> InvokeAsync(IAgent agent, KernelArguments? arguments = null, CancellationToken cancellationToken = default);
/// <summary>
/// Advance the thread with the specified agent.
/// </summary>
/// <param name="agent">An agent instance.</param>
/// <param name="userMessage">The user message</param>
/// <param name="arguments">Optional arguments for parameterized instructions</param>
/// <param name="cancellationToken">A cancellation token</param>
/// <returns>The resulting agent message(s)</returns>
IAsyncEnumerable<IChatMessage> InvokeAsync(IAgent agent, string userMessage, CancellationToken cancellationToken = default);
IAsyncEnumerable<IChatMessage> InvokeAsync(IAgent agent, string userMessage, KernelArguments? arguments = null, CancellationToken cancellationToken = default);
/// <summary>
/// Delete current thread. Terminal state - Unable to perform any
dotnet/src/Experimental/Agents/IChatMessage.cs
+37
View File
@@ -1,5 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Collections.ObjectModel;
namespace Microsoft.SemanticKernel.Experimental.Agents;
@@ -29,8 +30,44 @@ public interface IChatMessage
/// </summary>
string Role { get; }
/// <summary>
/// Annotations associated with the message.
/// </summary>
IList<IAnnotation> Annotations { get; }
/// <summary>
/// Properties associated with the message.
/// </summary>
ReadOnlyDictionary<string, object> Properties { get; }
/// <summary>
/// Defines message annotation.
/// </summary>
interface IAnnotation
{
/// <summary>
/// The file identifier.
/// </summary>
string FileId { get; }
/// <summary>
/// The text in the message content that needs to be replaced.
/// </summary>
string Label { get; }
/// <summary>
/// The citation.
/// </summary>
string? Quote { get; }
/// <summary>
/// Start index of the citation.
/// </summary>
int StartIndex { get; }
/// <summary>
/// End index of the citation.
/// </summary>
int EndIndex { get; }
}
}
dotnet/src/Experimental/Agents/Internal/Agent.cs
+95 -3
View File
@@ -10,6 +10,7 @@ using System.Threading;
using System.Threading.Tasks;
using Microsoft.SemanticKernel.Experimental.Agents.Exceptions;
using Microsoft.SemanticKernel.Experimental.Agents.Models;
using Microsoft.SemanticKernel.PromptTemplates.Handlebars;
namespace Microsoft.SemanticKernel.Experimental.Agents.Internal;
@@ -18,6 +19,9 @@ namespace Microsoft.SemanticKernel.Experimental.Agents.Internal;
/// </summary>
internal sealed class Agent : IAgent
{
public const string ToolCodeInterpreter = "code_interpreter";
public const string ToolRetrieval = "retrieval";
/// <inheritdoc/>
public string Id => this._model.Id;
@@ -34,6 +38,9 @@ internal sealed class Agent : IAgent
#pragma warning restore CA1720 // Identifier contains type name - We don't control the schema
#pragma warning restore CA1716 // Identifiers should not match keywords
/// <inheritdoc/>
public AgentCapability Capabilities { get; }
/// <inheritdoc/>
public long CreatedAt => this._model.CreatedAt;
@@ -49,10 +56,25 @@ internal sealed class Agent : IAgent
/// <inheritdoc/>
public string Instructions => this._model.Instructions;
/// <inheritdoc/>
public IEnumerable<ToolModel> Tools => this._tools;
/// <inheritdoc/>
public IEnumerable<string> FileIds => this._fileIds.AsEnumerable();
private static readonly Regex s_removeInvalidCharsRegex = new("[^0-9A-Za-z-]");
private static readonly Dictionary<string, IPromptTemplateFactory> s_templateFactories =
new(StringComparer.OrdinalIgnoreCase)
{
{ PromptTemplateConfig.SemanticKernelTemplateFormat, new KernelPromptTemplateFactory() },
{ HandlebarsPromptTemplateFactory.HandlebarsTemplateFormat, new HandlebarsPromptTemplateFactory() },
};
private readonly OpenAIRestContext _restContext;
private readonly AssistantModel _model;
private readonly IPromptTemplate _promptTemplate;
private readonly ToolModel[] _tools;
private readonly HashSet<string> _fileIds;
private AgentPlugin? _agentPlugin;
private bool _isDeleted;
@@ -62,18 +84,20 @@ internal sealed class Agent : IAgent
/// </summary>
/// <param name="restContext">A context for accessing OpenAI REST endpoint</param>
/// <param name="assistantModel">The assistant definition</param>
/// <param name="config">The template config</param>
/// <param name="plugins">Plugins to initialize as agent tools</param>
/// <param name="cancellationToken">A cancellation token</param>
/// <returns>An initialized <see cref="Agent"> instance.</see></returns>
public static async Task<IAgent> CreateAsync(
OpenAIRestContext restContext,
AssistantModel assistantModel,
PromptTemplateConfig? config,
IEnumerable<KernelPlugin>? plugins = null,
CancellationToken cancellationToken = default)
{
var resultModel = await restContext.CreateAssistantModelAsync(assistantModel, cancellationToken).ConfigureAwait(false);
return new Agent(resultModel, restContext, plugins);
return new Agent(resultModel, config, restContext, plugins);
}
/// <summary>
@@ -81,14 +105,25 @@ internal sealed class Agent : IAgent
/// </summary>
internal Agent(
AssistantModel assistantModel,
PromptTemplateConfig? config,
OpenAIRestContext restContext,
IEnumerable<KernelPlugin>? plugins = null)
{
config ??=
new PromptTemplateConfig
{
Name = assistantModel.Name,
Description = assistantModel.Description,
Template = assistantModel.Instructions,
};
this._model = assistantModel;
this._restContext = restContext;
this._promptTemplate = this.DefinePromptTemplate(config);
this._fileIds = new HashSet<string>(assistantModel.FileIds, StringComparer.OrdinalIgnoreCase);
IKernelBuilder builder = Kernel.CreateBuilder();
;
this.Kernel =
Kernel
.CreateBuilder()
@@ -99,10 +134,19 @@ internal sealed class Agent : IAgent
{
this.Kernel.Plugins.AddRange(plugins);
}
this.Capabilities =
(this.Kernel.Plugins.Count > 0 ? AgentCapability.Functions : AgentCapability.None) |
(this._model.Tools.Any(t => string.Equals(t.Type, ToolRetrieval, StringComparison.OrdinalIgnoreCase)) ? AgentCapability.Retrieval : AgentCapability.None) |
(this._model.Tools.Any(t => string.Equals(t.Type, ToolCodeInterpreter, StringComparison.OrdinalIgnoreCase)) ? AgentCapability.CodeInterpreter : AgentCapability.None);
this._tools = this._model.Tools.Concat(this.Kernel.Plugins.SelectMany(p => p.Select(f => f.ToToolModel(p.Name)))).ToArray();
}
public AgentPlugin AsPlugin() => this._agentPlugin ??= this.DefinePlugin();
public IPromptTemplate AsPromptTemplate() => this._promptTemplate;
/// <inheritdoc/>
public Task<IAgentThread> NewThreadAsync(CancellationToken cancellationToken = default)
{
@@ -130,6 +174,42 @@ internal sealed class Agent : IAgent
await this._restContext.DeleteThreadModelAsync(id!, cancellationToken).ConfigureAwait(false);
}
/// <inheritdoc/>
public async Task AddFileAsync(string fileId, CancellationToken cancellationToken = default)
{
if (this._isDeleted)
{
return;
}
if (this._fileIds.Contains(fileId))
{
return;
}
await this._restContext.AddAssistantFileAsync(this.Id, fileId, cancellationToken).ConfigureAwait(false);
this._fileIds.Add(fileId);
}
/// <inheritdoc/>
public async Task RemoveFileAsync(string fileId, CancellationToken cancellationToken = default)
{
if (this._isDeleted)
{
return;
}
if (!this._fileIds.Contains(fileId))
{
return;
}
await this._restContext.RemoveAssistantFileAsync(this.Id, fileId, cancellationToken).ConfigureAwait(false);
this._fileIds.Remove(fileId);
}
/// <inheritdoc/>
public async Task DeleteAsync(CancellationToken cancellationToken = default)
{
@@ -146,11 +226,13 @@ internal sealed class Agent : IAgent
/// Marshal thread run through <see cref="KernelFunction"/> interface.
/// </summary>
/// <param name="input">The user input</param>
/// <param name="arguments">Arguments for parameterized instructions</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>An agent response (<see cref="AgentResponse"/></returns>
private async Task<AgentResponse> AskAsync(
[Description("The user message provided to the agent.")]
string input,
KernelArguments arguments,
CancellationToken cancellationToken = default)
{
var thread = await this.NewThreadAsync(cancellationToken).ConfigureAwait(false);
@@ -158,7 +240,7 @@ internal sealed class Agent : IAgent
{
await thread.AddUserMessageAsync(input, cancellationToken).ConfigureAwait(false);
var messages = await thread.InvokeAsync(this, cancellationToken).ToArrayAsync(cancellationToken).ConfigureAwait(false);
var messages = await thread.InvokeAsync(this, input, arguments, cancellationToken).ToArrayAsync(cancellationToken).ConfigureAwait(false);
var response =
new AgentResponse
{
@@ -181,6 +263,16 @@ internal sealed class Agent : IAgent
return new AgentPluginImpl(this, functionAsk);
}
private IPromptTemplate DefinePromptTemplate(PromptTemplateConfig config)
{
if (!s_templateFactories.TryGetValue(config.TemplateFormat, out var factory))
{
factory = new KernelPromptTemplateFactory();
}
return factory.Create(config);
}
private void ThrowIfDeleted()
{
if (this._isDeleted)
dotnet/src/Experimental/Agents/Internal/ChatMessage.cs
+33 -2
View File
@@ -4,6 +4,7 @@ using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Linq;
using Microsoft.SemanticKernel.Experimental.Agents.Models;
using static Microsoft.SemanticKernel.Experimental.Agents.IChatMessage;
namespace Microsoft.SemanticKernel.Experimental.Agents.Internal;
@@ -27,13 +28,16 @@ internal sealed class ChatMessage : IChatMessage
/// <inheritdoc/>
public ReadOnlyDictionary<string, object> Properties { get; }
public IList<IAnnotation> Annotations { get; }
/// <summary>
/// Initializes a new instance of the <see cref="ChatMessage"/> class.
/// </summary>
internal ChatMessage(ThreadMessageModel model)
{
var content = (IEnumerable<ThreadMessageModel.ContentModel>)model.Content;
var text = content.First().Text?.Value ?? string.Empty;
var content = model.Content.First();
var text = content.Text?.Value ?? string.Empty;
this.Annotations = content.Text!.Annotations.Select(a => new Annotation(a.Text, a.StartIndex, a.EndIndex, a.FileCitation?.FileId ?? a.FilePath!.FileId, a.FileCitation?.Quote)).ToArray();
this.Id = model.Id;
this.AgentId = string.IsNullOrWhiteSpace(model.AgentId) ? null : model.AgentId;
@@ -41,4 +45,31 @@ internal sealed class ChatMessage : IChatMessage
this.Content = text;
this.Properties = new ReadOnlyDictionary<string, object>(model.Metadata);
}
private class Annotation : IAnnotation
{
public Annotation(string label, int startIndex, int endIndex, string fileId, string? quote)
{
this.FileId = fileId;
this.Label = label;
this.Quote = quote;
this.StartIndex = startIndex;
this.EndIndex = endIndex;
}
/// <inheritdoc/>
public string FileId { get; }
/// <inheritdoc/>
public string Label { get; }
/// <inheritdoc/>
public string? Quote { get; }
/// <inheritdoc/>
public int StartIndex { get; }
/// <inheritdoc/>
public int EndIndex { get; }
}
}
dotnet/src/Experimental/Agents/Internal/ChatThread.cs
+7 -6
View File
@@ -1,7 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Threading;
using System.Threading.Tasks;
@@ -65,13 +64,13 @@ internal sealed class ChatThread : IAgentThread
}
/// <inheritdoc/>
public IAsyncEnumerable<IChatMessage> InvokeAsync(IAgent agent, CancellationToken cancellationToken)
public IAsyncEnumerable<IChatMessage> InvokeAsync(IAgent agent, KernelArguments? arguments = null, CancellationToken cancellationToken = default)
{
return this.InvokeAsync(agent, string.Empty, cancellationToken);
return this.InvokeAsync(agent, string.Empty, arguments, cancellationToken);
}
/// <inheritdoc/>
public async IAsyncEnumerable<IChatMessage> InvokeAsync(IAgent agent, string userMessage, [EnumeratorCancellation] CancellationToken cancellationToken)
public async IAsyncEnumerable<IChatMessage> InvokeAsync(IAgent agent, string userMessage, KernelArguments? arguments = null, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
this.ThrowIfDeleted();
@@ -80,9 +79,11 @@ internal sealed class ChatThread : IAgentThread
yield return await this.AddUserMessageAsync(userMessage, cancellationToken).ConfigureAwait(false);
}
var tools = agent.Plugins.SelectMany(p => p.Select(f => f.ToToolModel(p.Name)));
var runModel = await this._restContext.CreateRunAsync(this.Id, agent.Id, agent.Instructions, tools, cancellationToken).ConfigureAwait(false);
// Finalize prompt / agent instructions using provided parameters.
var instructions = await agent.AsPromptTemplate().RenderAsync(agent.Kernel, arguments, cancellationToken).ConfigureAwait(false);
// Create run using templated prompt
var runModel = await this._restContext.CreateRunAsync(this.Id, agent.Id, instructions, agent.Tools, cancellationToken).ConfigureAwait(false);
var run = new ChatRun(runModel, agent.Kernel, this._restContext);
var results = await run.GetResultAsync(cancellationToken).ConfigureAwait(false);
dotnet/src/Experimental/Agents/Models/AgentConfigurationModel.cs
-30
View File
@@ -1,30 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
#pragma warning disable CA1812
using YamlDotNet.Serialization;
namespace Microsoft.SemanticKernel.Experimental.Agents.Models;
/// <summary>
/// Represents a yaml configuration file for an agent.
/// </summary>
internal sealed class AgentConfigurationModel
{
/// <summary>
/// The agent name
/// </summary>
[YamlMember(Alias = "name")]
public string Name { get; set; } = string.Empty;
/// <summary>
/// The agent description
/// </summary>
[YamlMember(Alias = "description")]
public string Description { get; set; } = string.Empty;
/// <summary>
/// The agent instructions template
/// </summary>
[YamlMember(Alias = "instructions")]
public string Instructions { get; set; } = string.Empty;
}
dotnet/src/Experimental/Agents/Models/AssistantModel.cs
+32
View File
@@ -88,4 +88,36 @@ internal sealed record AssistantModel
/// </summary>
[JsonPropertyName("metadata")]
public Dictionary<string, object> Metadata { get; init; } = new Dictionary<string, object>();
/// <summary>
/// Assistant file model.
/// </summary>
public sealed class FileModel
{
/// <summary>
/// ID of the assistant.
/// </summary>
[JsonPropertyName("assistant_id")]
public string AssistantId { get; set; } = string.Empty;
/// <summary>
/// ID of the uploaded file.
/// </summary>
[JsonPropertyName("id")]
public string Id { get; set; } = string.Empty;
/// <summary>
/// Unix timestamp (in seconds) for when the assistant was created
/// </summary>
[JsonPropertyName("created_at")]
public long CreatedAt { get; init; }
/// <summary>
/// Always "assistant.file"
/// </summary>
[JsonPropertyName("object")]
#pragma warning disable CA1720 // Identifier contains type name - We don't control the schema
public string Object { get; init; } = "assistant.file";
#pragma warning restore CA1720 // Identifier contains type name
}
}
dotnet/src/Experimental/Agents/Models/ThreadMessageModel.cs
+64 -1
View File
@@ -118,6 +118,69 @@ internal sealed class ThreadMessageModel
/// Any annotations on the text.
/// </summary>
[JsonPropertyName("annotations")]
public List<object> Annotations { get; set; } = new List<object>();
public List<TextAnnotationModel> Annotations { get; set; } = new List<TextAnnotationModel>();
}
public sealed class TextAnnotationModel
{
/// <summary>
/// Type of content.
/// </summary>
[JsonPropertyName("type")]
public string Type { get; set; } = string.Empty;
/// <summary>
/// The text of the citation-label text in the message content that can be replaced/reformatted.
/// </summary>
[JsonPropertyName("text")]
public string Text { get; set; } = string.Empty;
/// <summary>
/// Annotation when type == "file_citation"
/// </summary>
[JsonPropertyName("file_citation")]
public TextFileCitationModel? FileCitation { get; set; }
/// <summary>
/// Annotation when type == "file_path"
/// </summary>
[JsonPropertyName("file_path")]
public TextFilePathModel? FilePath { get; set; }
/// <summary>
/// Start index of the citation.
/// </summary>
[JsonPropertyName("start_index")]
public int StartIndex { get; set; }
/// <summary>
/// End index of the citation.
/// </summary>
[JsonPropertyName("end_index")]
public int EndIndex { get; set; }
}
public sealed class TextFileCitationModel
{
/// <summary>
/// The file identifier.
/// </summary>
[JsonPropertyName("file_id")]
public string FileId { get; set; } = string.Empty;
/// <summary>
/// The citation.
/// </summary>
[JsonPropertyName("quote")]
public string Quote { get; set; } = string.Empty;
}
public sealed class TextFilePathModel
{
/// <summary>
/// The file identifier.
/// </summary>
[JsonPropertyName("file_id")]
public string FileId { get; set; } = string.Empty;
}
}
dotnet/src/Extensions/PromptTemplates.Handlebars/Helpers/KernelHelperUtils.cs
+35
View File
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Linq;
using System.Text.Json;
using System.Text.Json.Nodes;
using HandlebarsDotNet;
@@ -28,6 +29,40 @@ internal static class KernelHelpersUtils
handlebarsInstance.RegisterHelper(helperName, helper);
}
/// <summary>
/// Returns value if defined, else, tries to resolve value from given KernelArguments dictionary.
/// </summary>
/// <param name="argument">Argument to process.</param>
/// <param name="kernelArguments">Dictionary of variables maintained by the Handlebars context.</param>
internal static object? GetArgumentValue(object argument, KernelArguments kernelArguments)
{
// If the argument is of type UndefinedBindingResult, it means that Handlebars attempted to retrieve the value for a binding
// but was unable to do so because the variable was not defined or not passed to the template context at the time of render.
// Thus, we try to get the value from the kernel arguments dictionary.
if (argument is UndefinedBindingResult result)
{
return kernelArguments.TryGetValue(result.Value, out var variable) ? variable : result.Value;
}
return argument;
}
/// <summary>
/// Processes arguments to resolve unbinded values. If argument was not bound to the Handlebars template at render time, get the value from the KernelArguments dictionary.
/// </summary>
/// <param name="arguments">Arguments to process.</param>
/// <param name="kernelArguments">Dictionary of variables maintained by the Handlebars context.</param>
/// <returns>Arguments with processed values.</returns>
internal static Arguments ProcessArguments(Arguments arguments, KernelArguments kernelArguments)
{
var processedArguments = arguments.Select(arg =>
{
return GetArgumentValue(arg, kernelArguments);
});
return new Arguments(processedArguments.ToArray());
}
/// <summary>
/// Determines whether the specified type is a numeric type.
/// </summary>
dotnet/src/Extensions/PromptTemplates.Handlebars/Helpers/KernelHelpers/KernelFunctionHelpers.cs
+9 -7
View File
@@ -126,13 +126,14 @@ internal static class KernelFunctionHelpers
var fullyQualifiedParamName = functionMetadata.Name + nameDelimiter + param.Name;
if (handlebarsArguments is not null && (handlebarsArguments.TryGetValue(fullyQualifiedParamName, out var value) || handlebarsArguments.TryGetValue(param.Name, out value)))
{
if (IsExpectedParameterType(param, value))
value = KernelHelpersUtils.GetArgumentValue(value, executionContext);
if (value is not null && IsExpectedParameterType(param, value))
{
executionContext[param.Name] = value;
}
else
{
throw new KernelException($"Invalid argument type for function {functionMetadata.Name}. Parameter {param.Name} expects type {param.ParameterType ?? (object?)param.Schema} but received {value.GetType()}.");
throw new KernelException($"Invalid argument type for function {functionMetadata.Name}. Parameter {param.Name} expects type {param.ParameterType ?? (object?)param.Schema} but received {value?.GetType()}.");
}
}
else if (param.IsRequired)
@@ -152,20 +153,21 @@ internal static class KernelFunctionHelpers
private static void ProcessPositionalArguments(KernelFunctionMetadata functionMetadata, KernelArguments executionContext, Arguments handlebarsArguments)
{
var requiredParameters = functionMetadata.Parameters.Where(p => p.IsRequired).ToList();
if (requiredParameters.Count <= handlebarsArguments.Length && handlebarsArguments.Length <= functionMetadata.Parameters.Count)
{
var argIndex = 0;
foreach (var arg in handlebarsArguments)
var arguments = KernelHelpersUtils.ProcessArguments(handlebarsArguments, executionContext);
foreach (var arg in arguments)
{
var param = functionMetadata.Parameters[argIndex];
var param = functionMetadata.Parameters[argIndex++];
if (IsExpectedParameterType(param, arg))
{
executionContext[param.Name] = handlebarsArguments[argIndex];
argIndex++;
executionContext[param.Name] = arg;
}
else
{
throw new KernelException($"Invalid parameter type for function {functionMetadata.Name}. Parameter {param.Name} expects type {param.ParameterType ?? (object?)param.Schema} but received {handlebarsArguments[argIndex].GetType()}.");
throw new KernelException($"Invalid parameter type for function {functionMetadata.Name}. Parameter {param.Name} expects type {param.ParameterType ?? (object?)param.Schema} but received {arg.GetType()}.");
}
}
}
dotnet/src/Extensions/PromptTemplates.Handlebars/Helpers/KernelHelpers/KernelSystemHelpers.cs
+36 -25
View File
@@ -6,6 +6,7 @@ using System.Text.Json;
using System.Text.Json.Nodes;
using HandlebarsDotNet;
using HandlebarsDotNet.Compiler;
using static Microsoft.SemanticKernel.PromptTemplates.Handlebars.Helpers.KernelHelpersUtils;
namespace Microsoft.SemanticKernel.PromptTemplates.Handlebars.Helpers;
@@ -62,32 +63,34 @@ internal static class KernelSystemHelpers
handlebarsInstance.RegisterHelper("set", (writer, context, arguments) =>
{
var name = string.Empty;
object value = string.Empty;
object? value = string.Empty;
if (arguments[0].GetType() == typeof(HashParameterDictionary))
{
// Get the parameters from the template arguments
var parameters = (IDictionary<string, object>)arguments[0];
name = (string)parameters!["name"];
value = parameters!["value"];
value = GetArgumentValue(parameters!["value"], variables);
}
else
{
name = arguments[0].ToString();
value = arguments[1];
var args = ProcessArguments(arguments, variables);
name = args[0].ToString();
value = args[1];
}
// Set the variable in the Handlebars context
variables[name] = value;
});
handlebarsInstance.RegisterHelper("json", static (in HelperOptions options, in Context context, in Arguments arguments) =>
handlebarsInstance.RegisterHelper("json", (in HelperOptions options, in Context context, in Arguments arguments) =>
{
if (arguments.Length == 0 || arguments[0] is null)
{
throw new HandlebarsRuntimeException("`json` helper requires a value to be passed in.");
}
object objectToSerialize = arguments[0];
var args = ProcessArguments(arguments, variables);
object objectToSerialize = args[0];
var type = objectToSerialize.GetType();
return type == typeof(string) ? objectToSerialize
@@ -97,22 +100,14 @@ internal static class KernelSystemHelpers
handlebarsInstance.RegisterHelper("concat", (in HelperOptions options, in Context context, in Arguments arguments) =>
{
var args = arguments.ToList().Select(arg =>
{
if (arg is UndefinedBindingResult result)
{
return variables.TryGetValue(result.Value, out var variable) ? variable : result.Value;
}
return arg;
});
return string.Concat(arguments);
var args = ProcessArguments(arguments, variables);
return string.Concat(args);
});
handlebarsInstance.RegisterHelper("array", (in HelperOptions options, in Context context, in Arguments arguments) =>
{
return arguments.ToArray();
var args = ProcessArguments(arguments, variables);
return args.ToArray();
});
handlebarsInstance.RegisterHelper("raw", static (writer, options, context, arguments) =>
@@ -122,28 +117,44 @@ internal static class KernelSystemHelpers
handlebarsInstance.RegisterHelper("range", (in HelperOptions options, in Context context, in Arguments arguments) =>
{
var args = ProcessArguments(arguments, variables);
// Create list with numbers from start to end (inclusive)
var start = int.Parse(arguments[0].ToString(), kernel.Culture);
var end = int.Parse(arguments[1].ToString(), kernel.Culture) + 1;
var start = int.Parse(args[0].ToString(), kernel.Culture);
var end = int.Parse(args[1].ToString(), kernel.Culture) + 1;
var count = end - start;
return Enumerable.Range(start, count);
});
handlebarsInstance.RegisterHelper("or", static (in HelperOptions options, in Context context, in Arguments arguments) =>
handlebarsInstance.RegisterHelper("or", (in HelperOptions options, in Context context, in Arguments arguments) =>
{
return arguments.Any(arg => arg != null && arg is not false);
var args = ProcessArguments(arguments, variables);
return args.Any(arg => arg != null && arg is not false);
});
handlebarsInstance.RegisterHelper("equals", static (in HelperOptions options, in Context context, in Arguments arguments) =>
handlebarsInstance.RegisterHelper("add", (in HelperOptions options, in Context context, in Arguments arguments) =>
{
var args = ProcessArguments(arguments, variables);
return args.Sum(arg => decimal.Parse(arg.ToString(), kernel.Culture));
});
handlebarsInstance.RegisterHelper("subtract", (in HelperOptions options, in Context context, in Arguments arguments) =>
{
var args = ProcessArguments(arguments, variables);
return args.Aggregate((a, b) => decimal.Parse(a.ToString(), kernel.Culture) - decimal.Parse(b.ToString(), kernel.Culture));
});
handlebarsInstance.RegisterHelper("equals", (in HelperOptions options, in Context context, in Arguments arguments) =>
{
if (arguments.Length < 2)
{
return false;
}
object? left = arguments[0];
object? right = arguments[1];
var args = ProcessArguments(arguments, variables);
object? left = args[0];
object? right = args[1];
return left == right || (left is not null && left.Equals(right));
});
dotnet/src/Extensions/PromptTemplates.Handlebars/PromptTemplates.Handlebars.csproj
+1
View File
@@ -5,6 +5,7 @@
<AssemblyName>Microsoft.SemanticKernel.PromptTemplates.Handlebars</AssemblyName>
<RootNamespace>Microsoft.SemanticKernel.PromptTemplates.Handlebars</RootNamespace>
<TargetFramework>netstandard2.0</TargetFramework>
<EnablePackageValidation>true</EnablePackageValidation>
</PropertyGroup>
<Import Project="$(RepoRoot)/dotnet/nuget/nuget-package.props" />
dotnet/src/Functions/Functions.OpenApi/Extensions/OpenApiSchemaExtensions.cs
+2 -1
View File
@@ -1,5 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Globalization;
using System.IO;
using System.Text;
using Microsoft.OpenApi.Models;
@@ -17,7 +18,7 @@ internal static class OpenApiSchemaExtensions
internal static KernelJsonSchema ToJsonSchema(this OpenApiSchema schema)
{
var schemaBuilder = new StringBuilder();
var jsonWriter = new OpenApiJsonWriter(new StringWriter(schemaBuilder));
var jsonWriter = new OpenApiJsonWriter(new StringWriter(schemaBuilder, CultureInfo.InvariantCulture));
jsonWriter.Settings.InlineLocalReferences = true;
schema.SerializeAsV3(jsonWriter);
return KernelJsonSchema.Parse(schemaBuilder.ToString());
dotnet/src/Functions/Functions.UnitTests/OpenApi/Extensions/OpenApiSchemaExtensionsTests.cs
+49
View File
@@ -0,0 +1,49 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Globalization;
using Microsoft.OpenApi.Any;
using Microsoft.OpenApi.Models;
using Microsoft.SemanticKernel.Plugins.OpenApi;
using Xunit;
namespace SemanticKernel.Functions.UnitTests.OpenApi.Extensions;
public class OpenApiSchemaExtensionsTests
{
[Fact]
public void ItShouldConvertOpenApiSchemaUsingInvariantCulture()
{
// Arrange
var schema = new OpenApiSchema
{
Type = "object",
Properties = new Dictionary<string, OpenApiSchema>
{
["property1"] = new OpenApiSchema
{
Type = "number",
Format = "double",
Default = new OpenApiDouble(12.01)
}
}
};
var currentCulture = CultureInfo.CurrentCulture; // Backup current culture
// Act & Assert
try
{
CultureInfo.CurrentCulture = new CultureInfo("fr-FR"); // French culture uses comma as decimal separator
var result = OpenApiSchemaExtensions.ToJsonSchema(schema); // Should use invariant culture
Assert.True(result.RootElement.TryGetProperty("properties", out var properties));
Assert.True(properties.TryGetProperty("property1", out var property2));
Assert.Equal(12.01, property2.GetProperty("default").GetDouble());
}
finally
{
CultureInfo.CurrentCulture = currentCulture; // Restore current culture
}
}
}
dotnet/src/Functions/Functions.Yaml/Functions.Yaml.csproj
+1
View File
@@ -5,6 +5,7 @@
<AssemblyName>Microsoft.SemanticKernel.Yaml</AssemblyName>
<RootNamespace>$(AssemblyName)</RootNamespace>
<TargetFramework>netstandard2.0</TargetFramework>
<EnablePackageValidation>true</EnablePackageValidation>
</PropertyGroup>
<Import Project="$(RepoRoot)/dotnet/nuget/nuget-package.props" />
dotnet/src/Functions/Functions.Yaml/KernelFunctionYaml.cs
+15 -6
View File
@@ -27,12 +27,7 @@ public static class KernelFunctionYaml
IPromptTemplateFactory? promptTemplateFactory = null,
ILoggerFactory? loggerFactory = null)
{
var deserializer = new DeserializerBuilder()
.WithNamingConvention(UnderscoredNamingConvention.Instance)
.WithNodeDeserializer(new PromptExecutionSettingsNodeDeserializer())
.Build();
var promptTemplateConfig = deserializer.Deserialize<PromptTemplateConfig>(text);
PromptTemplateConfig promptTemplateConfig = ToPromptTemplateConfig(text);
// Prevent the default value from being any type other than a string.
// It's a temporary limitation that helps shape the public API surface
@@ -53,4 +48,18 @@ public static class KernelFunctionYaml
promptTemplateFactory,
loggerFactory);
}
/// <summary>
/// Convert the given YAML text to a <see cref="PromptTemplateConfig"/> model.
/// </summary>
/// <param name="text">YAML representation of the <see cref="PromptTemplateConfig"/> to use to create the prompt function.</param>
public static PromptTemplateConfig ToPromptTemplateConfig(string text)
{
var deserializer = new DeserializerBuilder()
.WithNamingConvention(UnderscoredNamingConvention.Instance)
.WithNodeDeserializer(new PromptExecutionSettingsNodeDeserializer())
.Build();
return deserializer.Deserialize<PromptTemplateConfig>(text);
}
}
dotnet/src/IntegrationTests/Connectors/OpenAI/ChatHistoryTests.cs
+48
View File
@@ -70,6 +70,54 @@ public sealed class ChatHistoryTests : IDisposable
Assert.Null(exception);
}
[Fact]
public async Task ItUsesChatSystemPromptFromSettingsAsync()
{
// Arrange
this._kernelBuilder.Services.AddSingleton<ILoggerFactory>(this._logger);
var builder = this._kernelBuilder;
this.ConfigureAzureOpenAIChatAsText(builder);
builder.Plugins.AddFromType<FakePlugin>();
var kernel = builder.Build();
string systemPrompt = "You are batman. If asked who you are, say 'I am Batman!'";
OpenAIPromptExecutionSettings settings = new() { ChatSystemPrompt = systemPrompt };
ChatHistory history = new();
// Act
history.AddUserMessage("Who are you?");
var service = kernel.GetRequiredService<IChatCompletionService>();
ChatMessageContent result = await service.GetChatMessageContentAsync(history, settings, kernel);
// Assert
Assert.Contains("Batman", result.ToString(), StringComparison.OrdinalIgnoreCase);
}
[Fact]
public async Task ItUsesChatSystemPromptFromChatHistoryAsync()
{
// Arrange
this._kernelBuilder.Services.AddSingleton<ILoggerFactory>(this._logger);
var builder = this._kernelBuilder;
this.ConfigureAzureOpenAIChatAsText(builder);
builder.Plugins.AddFromType<FakePlugin>();
var kernel = builder.Build();
string systemPrompt = "You are batman. If asked who you are, say 'I am Batman!'";
OpenAIPromptExecutionSettings settings = new();
ChatHistory history = new(systemPrompt);
// Act
history.AddUserMessage("Who are you?");
var service = kernel.GetRequiredService<IChatCompletionService>();
ChatMessageContent result = await service.GetChatMessageContentAsync(history, settings, kernel);
// Assert
Assert.Contains("Batman", result.ToString(), StringComparison.OrdinalIgnoreCase);
}
private void ConfigureAzureOpenAIChatAsText(IKernelBuilder kernelBuilder)
{
var azureOpenAIConfiguration = this._configuration.GetSection("Planners:AzureOpenAI").Get<AzureOpenAIConfiguration>();
dotnet/src/IntegrationTests/Connectors/OpenAI/OpenAIToolsTests.cs
+1 -1
View File
@@ -77,7 +77,7 @@ public sealed class OpenAIToolsTests : IDisposable
}
// Assert
Assert.Equal("6", result);
Assert.Contains("6", result, StringComparison.InvariantCulture);
Assert.Contains("GetAge([personName, John])", invokedFunctions);
Assert.Contains("GetAge([personName, Jim])", invokedFunctions);
Assert.Contains("InterpretValue([value, 3])", invokedFunctions);
dotnet/src/IntegrationTests/Fakes/ThrowingEmailPluginFake.cs
+7 -9
View File
@@ -21,19 +21,17 @@ internal sealed class ThrowingEmailPluginFake
throw new ArgumentException($"Failed to send email to {email_address}");
}
[KernelFunction, Description("Lookup an email address for a person given a name")]
public Task<string> GetEmailAddressAsync(
ILogger logger,
[Description("The name of the person to email.")] string? input = null)
{
// Throw a critical exception for testing
throw new InvalidProgramException();
}
[KernelFunction, Description("Write a short poem for an e-mail")]
public Task<string> WritePoemAsync(
[Description("The topic of the poem.")] string input)
{
return Task.FromResult($"Roses are red, violets are blue, {input} is hard, so is this test.");
}
[KernelFunction, Description("Write a joke for an e-mail")]
public Task<string> WriteJokeAsync()
{
// Throw a critical exception for testing
throw new InvalidProgramException();
}
}
dotnet/src/IntegrationTests/Planners/Stepwise/FunctionCallingStepwisePlannerTests.cs
+31 -6
View File
@@ -4,6 +4,8 @@ using System;
using System.Text.Json;
using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Planning;
using Microsoft.SemanticKernel.Plugins.Core;
@@ -21,7 +23,9 @@ public sealed class FunctionCallingStepwisePlannerTests : IDisposable
public FunctionCallingStepwisePlannerTests(ITestOutputHelper output)
{
this._logger = new XunitLogger<Kernel>(output);
this._testOutputHelper = new RedirectOutput(output);
Console.SetOut(this._testOutputHelper);
// Load configuration
this._configuration = new ConfigurationBuilder()
@@ -75,8 +79,17 @@ public sealed class FunctionCallingStepwisePlannerTests : IDisposable
[Fact]
public async Task DoesNotThrowWhenPluginFunctionThrowsNonCriticalExceptionAsync()
{
// Arrange
Kernel kernel = this.InitializeKernel();
kernel.ImportPluginFromType<ThrowingEmailPluginFake>("Email");
var emailPluginFake = new ThrowingEmailPluginFake();
kernel.Plugins.Add(
KernelPluginFactory.CreateFromFunctions(
"Email",
new[] {
KernelFunctionFactory.CreateFromMethod(emailPluginFake.WritePoemAsync),
KernelFunctionFactory.CreateFromMethod(emailPluginFake.SendEmailAsync),
}));
var planner = new FunctionCallingStepwisePlanner(
new FunctionCallingStepwisePlannerConfig() { MaxIterations = 5 });
@@ -97,15 +110,24 @@ public sealed class FunctionCallingStepwisePlannerTests : IDisposable
[Fact]
public async Task ThrowsWhenPluginFunctionThrowsCriticalExceptionAsync()
{
// Arrange
Kernel kernel = this.InitializeKernel();
kernel.ImportPluginFromType<ThrowingEmailPluginFake>("Email");
var emailPluginFake = new ThrowingEmailPluginFake();
kernel.Plugins.Add(
KernelPluginFactory.CreateFromFunctions(
"Email",
new[] {
KernelFunctionFactory.CreateFromMethod(emailPluginFake.WriteJokeAsync),
KernelFunctionFactory.CreateFromMethod(emailPluginFake.SendEmailAsync),
}));
var planner = new FunctionCallingStepwisePlanner(
new FunctionCallingStepwisePlannerConfig() { MaxIterations = 5 });
// Act & Assert
// Planner should call ThrowingEmailPluginFake.GetEmailAddressAsync, which throws InvalidProgramException
await Assert.ThrowsAsync<InvalidProgramException>(async () => await planner.ExecuteAsync(kernel, "What is Kelly's email address?"));
// Planner should call ThrowingEmailPluginFake.WriteJokeAsync, which throws InvalidProgramException
await Assert.ThrowsAsync<InvalidProgramException>(async () => await planner.ExecuteAsync(kernel, "Email a joke to [email protected]"));
}
private Kernel InitializeKernel()
@@ -113,8 +135,9 @@ public sealed class FunctionCallingStepwisePlannerTests : IDisposable
OpenAIConfiguration? openAIConfiguration = this._configuration.GetSection("Planners:OpenAI").Get<OpenAIConfiguration>();
Assert.NotNull(openAIConfiguration);
IKernelBuilder builder = Kernel.CreateBuilder()
.AddOpenAIChatCompletion(
IKernelBuilder builder = Kernel.CreateBuilder();
builder.Services.AddSingleton<ILoggerFactory>(this._logger);
builder.AddOpenAIChatCompletion(
modelId: openAIConfiguration.ModelId,
apiKey: openAIConfiguration.ApiKey);
@@ -125,6 +148,7 @@ public sealed class FunctionCallingStepwisePlannerTests : IDisposable
private readonly RedirectOutput _testOutputHelper;
private readonly IConfigurationRoot _configuration;
private readonly XunitLogger<Kernel> _logger;
public void Dispose()
{
@@ -141,6 +165,7 @@ public sealed class FunctionCallingStepwisePlannerTests : IDisposable
{
if (disposing)
{
this._logger.Dispose();
this._testOutputHelper.Dispose();
}
}
dotnet/src/IntegrationTests/README.md
+8
View File
@@ -48,6 +48,14 @@ dotnet user-secrets set "AzureOpenAIEmbeddings:ApiKey" "..."
dotnet user-secrets set "HuggingFace:ApiKey" "..."
dotnet user-secrets set "Bing:ApiKey" "..."
dotnet user-secrets set "Postgres:ConnectionString" "..."
dotnet user-secrets set "Planners:AzureOpenAI:Endpoint" "https://contoso.openai.azure.com/"
dotnet user-secrets set "Planners:AzureOpenAI:ChatDeploymentName" "gpt-4-1106-preview"
dotnet user-secrets set "Planners:AzureOpenAI:ServiceId" "gpt-4-1106-preview"
dotnet user-secrets set "Planners:AzureOpenAI:ApiKey" "..."
dotnet user-secrets set "Planners:OpenAI:ModelId" "gpt-3.5-turbo-1106"
dotnet user-secrets set "Planners:OpenAI:ApiKey" "..."
```
### Option 2: Use Configuration File
dotnet/src/Planners/Planners.Handlebars/Handlebars/CreatePlanPrompt.handlebars
+55 -32
View File
@@ -13,7 +13,7 @@ What's the sum of 5+10+15?
{{#message role="assistant"}}Here's the Handlebars template that achieves the goal:
```handlebars
{{#if allowLoops}}
\{{!-- Step 1: Initialize the count --}}
\{{!-- Step 1: Identify key values --}}
\{{set
"count"
10
@@ -55,7 +55,7 @@ What's the sum of 5+10+15?
"num3"
15
}}
\{{!-- Step 2: Call the Example-AddNums helper with the variables and store the result --}}
\{{!-- Step 2: Call the Example helper with the variables and store the result --}}
\{{set
"sum"
(Example{{nameDelimiter}}AddNums
@@ -69,13 +69,10 @@ What's the sum of 5+10+15?
{{/if}}
```{{/message}}
{{#message role="system"}}Now let's try the real thing.{{/message}}
{{#message role="user"}}Please show me how to write a Handlebars template that achieves the following goal with the available helpers.
## Goal
{{goal}}
{{#message role="user"}}
The following helpers are available to you:
## Out-of-the-box helpers
The following helpers are available to you:
- `\{{#if}}\{{/if}}`
- `\{{#unless}}\{{/unless}}`{{#if allowLoops}}
- `\{{#each}}\{{/each}}`{{/if}}
@@ -87,12 +84,12 @@ If you need to loop through a list of values with `\{{#each}}`, you can use the
- `\{{range}}` – Generates a list of integral numbers within a specified range, inclusive of the first and last value.
- `\{{array}}` – Generates an array of values from the given values (zero-indexed).
IMPORTANT: `range` and `array` are the only supported data structures. Others like `hash` are not supported. Also, you cannot use any methods or properties on the built-in data structures, such as `array.push` or `range.length`.
IMPORTANT: `range` and `array` are the only supported data structures. Others like `hash` are not supported. Also, you cannot use any methods or properties on the built-in data structures.
## Math helpers
If you need to do basic operations, you can use these two helpers with numerical values:
- `\{{Add}}` – Adds two values together.
- `\{{Subtract}}` – Subtracts the second value from the first.
- `\{{add}}` – Adds two values together.
- `\{{subtract}}` – Subtracts the second value from the first.
{{/if}}
## Comparison helpers
@@ -102,8 +99,8 @@ To use the {{#if allowLoops}}math and {{/if}}comparison helpers, you must pass i
## Variable helpers
If you need to create or retrieve a variable, you can use the following helpers:
- `\{{set}}` – Creates a variable with the given name and value. It does not print anything to the template, so you must use `\{{json}}` to print the value.
- `\{{json}}` – Generates a JSON string from the given value; no need to use on strings.
- `\{{concat}}` – Concatenates the given values into a string.
- `\{{json}}` – Generates and prints a JSON string from the given value.
- `\{{concat}}` – Concatenates the given values into one string.
{{#if (or complexTypeDefinitions complexSchemaDefinitions)}}
## Complex types
@@ -112,7 +109,7 @@ Some helpers require arguments that are complex objects. The JSON schemas for th
{{#each complexTypeDefinitions}}
### {{Name}}:
{
"type": "object",
"type": "Object",
"properties": {
{{#each Properties}}
"{{Name}}": {
@@ -130,7 +127,7 @@ Some helpers require arguments that are complex objects. The JSON schemas for th
{{/each}}
{{/if}}
## Custom helpers
Lastly, you also have the following helpers you can use:
Lastly, you have the following custom helpers to use.
{{#each functions}}
### `{{doubleOpen}}{{PluginName}}{{../nameDelimiter}}{{Name}}{{doubleClose}}`
@@ -156,27 +153,53 @@ Output:
{{/if}}
{{/each}}
IMPORTANT: You can only use the helpers that are listed above. Do not use any other helpers that are not listed here. For example, do not use `\{{log}}` or any `\{{Example}}` helpers, as they are not supported.{{/message}}
IMPORTANT: You can only use the helpers that are listed above. Do not use any other helpers that are not explicitly listed here. For example, do not use `\{{log}}` or any `\{{Example}}` helpers, as they are not supported.{{/message}}
{{#message role="user"}}Please show me how to write a Handlebars template that achieves the following goal with the available helpers.
{{#if lastError}}
{{#message role="system"}}
## Tips and tricks
## Previous attempt
This previous plan failed to achieve the goal:
```handlebars
{{lastPlan}}
```
The error was :
```
{{lastError}}
```
Try again to achieve the goal while fixing the error.{{/message}}
{{/if}}
## Goal
{{goal}}{{/message}}
{{#message role="system"}}
## Tips and reminders
- Add a comment above each step to describe what the step does.
- Use the `\{{set}}` helper to save and retrieve the results of another helper so you can use it later in the template without wasting resources.
- There are no initial variables available to you. You must create them yourself using the `\{{set}}` helper.
- Do not make up values. Use the helpers to generate the data you need or extract it from the goal.
- Keep data well-defined. Each variable should have a unique name. Create and assign each variable only once, and always use the full name when calling helpers.
- Do not pass raw data to helpers. Use variables instead.
- Each variable should have a well-defined name.
- Be extremely careful about types. For example, if you pass an array to a helper that expects a number, the template will error out.
{{#if allowLoops}}- Avoid using loops. Try a solution without before you deploy a loop.{{/if}}
- There is no need to check your results in the template.
- Do not nest sub-expressions or helpers because it will cause the template to error out.
- Each step should contain only one helper call.
## Start
Now take a deep breath and accomplish the task:
1. Keep the template short and sweet. Be as efficient as possible.
2. Do not make up helpers or functions that were not provided to you, and be especially careful to NOT assume or use any helpers or operations that were not explicitly defined already.
3. If you can't fully accomplish the goal with the available helpers, just print "{{insufficientFunctionsErrorMessage}}".
4. Always start by identifying any important values in the goal. Then, use the `\{{set}}` helper to create variables for each of these values.
5. The template should use the \{{json}} helper at least once to output the result of the final step.
6. Don't forget to use the tips and tricks otherwise the template will not work.
7. Don't close the ``` handlebars block until you're done with all the steps.{{/message}}
Follow these steps to create one Handlebars template to achieve the goal:
1. Identify Key Values:
- Read the goal carefully and note any numbers, strings, or conditions that you'll need. Do not truncate or modify any data.
- Use the `\{{set}}` helper to create a variable for each key value.
2. Choose the Right Helpers:
- Use the provided helpers to manipulate the variables you've created. Start with the basic helpers and only use custom helpers if necessary to accomplish the goal.
- Always reference a custom helper by its full name.
3. Don't Create or Assume Unlisted Helpers:
- Only use the helpers provided. Any helper not listed is considered hallucinated and must not be used.
- Do not invent or assume the existence of any functions not explicitly defined above.
4. What if I Need More Helpers?
- If the goal cannot be fully achieved with the provided helpers or you reference a hallucinated helper, print the following message: "{{insufficientFunctionsErrorMessage}}".
5. Keep It Simple:
- Your template should be intelligent and efficient, avoiding unnecessary complexity or redundant steps.
{{#if allowLoops}} - Avoid using loops or block expressions. They are allowed but not always necessary, so try to find a solution that does not use them.{{/if}}
6. No Nested Helpers:
- Do not nest helpers or conditionals inside other helpers. This can cause errors in the template.
7. Output the Result:
- Once you have completed the necessary steps to reach the goal, use the `\{{json}}` helper to output the final result.
- Ensure your template and all steps are enclosed in a ``` handlebars block.
Remember, the objective is not to use all the helpers available, but to use the correct ones to achieve the desired outcome with a clear and concise template.{{/message}}
dotnet/src/Planners/Planners.Handlebars/Handlebars/HandlebarsPlanner.cs
+3 -3
View File
@@ -25,7 +25,7 @@ public sealed class HandlebarsPlanner
public static readonly HandlebarsPromptTemplateOptions PromptTemplateOptions = new()
{
// Options for built-in Handlebars helpers
Categories = new Category[] { Category.Math },
Categories = new Category[] { Category.DateTime },
UseCategoryPrefix = false,
// Custom helpers
@@ -85,7 +85,7 @@ public sealed class HandlebarsPlanner
var completionResults = await chatCompletionService.GetChatMessageContentAsync(chatMessages, cancellationToken: cancellationToken).ConfigureAwait(false);
// Check if plan could not be created due to insufficient functions
if (completionResults.Content is not null && completionResults.Content.Contains(InsufficientFunctionsError))
if (completionResults.Content is not null && completionResults.Content.IndexOf(InsufficientFunctionsError, StringComparison.OrdinalIgnoreCase) >= 0)
{
var functionNames = availableFunctions.ToList().Select(func => $"{func.PluginName}{this._templateFactory.NameDelimiter}{func.Name}");
throw new KernelException($"[{HandlebarsPlannerErrorCodes.InsufficientFunctionsForGoal}] Unable to create plan for goal with available functions.\nGoal: {goal}\nAvailable Functions: {string.Join(", ", functionNames)}\nPlanner output:\n{completionResults}");
@@ -94,7 +94,7 @@ public sealed class HandlebarsPlanner
Match match = Regex.Match(completionResults.Content, @"```\s*(handlebars)?\s*(.*)\s*```", RegexOptions.Singleline);
if (!match.Success)
{
throw new KernelException($"[{HandlebarsPlannerErrorCodes.InvalidTemplate}] Could not find the plan in the results");
throw new KernelException($"[{HandlebarsPlannerErrorCodes.InvalidTemplate}] Could not find the plan in the results\nPlanner output:\n{completionResults}");
}
var planTemplate = match.Groups[2].Value.Trim();
dotnet/src/SemanticKernel.Abstractions/SemanticKernel.Abstractions.csproj
+1
View File
@@ -5,6 +5,7 @@
<RootNamespace>Microsoft.SemanticKernel</RootNamespace>
<TargetFramework>netstandard2.0</TargetFramework>
<NoWarn>$(NoWarn);SKEXP0004</NoWarn>
<EnablePackageValidation>true</EnablePackageValidation>
</PropertyGroup>
<Import Project="$(RepoRoot)/dotnet/nuget/nuget-package.props" />
dotnet/src/SemanticKernel.Core/KernelExtensions.cs
+162 -21
View File
@@ -169,6 +169,37 @@ public static class KernelExtensions
}
#endregion
#region CreatePluginFromFunctions
/// <summary>Creates a plugin that contains the specified functions.</summary>
/// <param name="kernel">The <see cref="Kernel"/> containing services, plugins, and other state for use throughout the operation.</param>
/// <param name="pluginName">The name for the plugin.</param>
/// <param name="functions">The initial functions to be available as part of the plugin.</param>
/// <returns>A <see cref="KernelPlugin"/> containing the functions provided in <paramref name="functions"/>.</returns>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is null.</exception>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is an invalid plugin name.</exception>
/// <exception cref="ArgumentNullException"><paramref name="functions"/> contains a null function.</exception>
/// <exception cref="ArgumentException"><paramref name="functions"/> contains two functions with the same name.</exception>
public static KernelPlugin CreatePluginFromFunctions(this Kernel kernel, string pluginName, IEnumerable<KernelFunction>? functions) =>
CreatePluginFromFunctions(kernel, pluginName, description: null, functions);
/// <summary>Creates a plugin that contains the specified functions.</summary>
/// <param name="kernel">The <see cref="Kernel"/> containing services, plugins, and other state for use throughout the operation.</param>
/// <param name="pluginName">The name for the plugin.</param>
/// <param name="description">A description of the plugin.</param>
/// <param name="functions">The initial functions to be available as part of the plugin.</param>
/// <returns>A <see cref="KernelPlugin"/> containing the functions provided in <paramref name="functions"/>.</returns>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is null.</exception>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is an invalid plugin name.</exception>
/// <exception cref="ArgumentNullException"><paramref name="functions"/> contains a null function.</exception>
/// <exception cref="ArgumentException"><paramref name="functions"/> contains two functions with the same name.</exception>
public static KernelPlugin CreatePluginFromFunctions(this Kernel kernel, string pluginName, string? description = null, IEnumerable<KernelFunction>? functions = null)
{
Verify.NotNull(kernel);
return KernelPluginFactory.CreateFromFunctions(pluginName, description, functions);
}
#endregion
#region ImportPlugin/AddFromType
/// <summary>Creates a plugin that wraps a new instance of the specified type <typeparamref name="T"/> and imports it into the <paramref name="kernel"/>'s plugin collection.</summary>
/// <typeparam name="T">Specifies the type of the object to wrap.</typeparam>
@@ -279,25 +310,6 @@ public static class KernelExtensions
return plugin;
}
/// <summary>Creates a plugin that contains the specified functions and adds it into the plugin collection.</summary>
/// <param name="plugins">The plugin collection to which the new plugin should be added.</param>
/// <param name="pluginName">The name for the plugin.</param>
/// <param name="description">A description of the plugin.</param>
/// <param name="functions">The initial functions to be available as part of the plugin.</param>
/// <returns>A <see cref="KernelPlugin"/> containing the functions provided in <paramref name="functions"/>.</returns>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is null.</exception>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is an invalid plugin name.</exception>
/// <exception cref="ArgumentNullException"><paramref name="functions"/> contains a null function.</exception>
/// <exception cref="ArgumentException"><paramref name="functions"/> contains two functions with the same name.</exception>
public static KernelPlugin AddFromFunctions(this ICollection<KernelPlugin> plugins, string pluginName, string? description, IEnumerable<KernelFunction>? functions = null)
{
Verify.NotNull(plugins);
var plugin = new DefaultKernelPlugin(pluginName, description, functions);
plugins.Add(plugin);
return plugin;
}
/// <summary>Creates a plugin that wraps the specified target object and adds it into the plugin collection.</summary>
/// <param name="plugins">The plugin collection to which the new plugin should be added.</param>
/// <param name="target">The instance of the class to be wrapped.</param>
@@ -318,6 +330,99 @@ public static class KernelExtensions
}
#endregion
#region ImportPlugin/AddFromFunctions
/// <summary>Creates a plugin that contains the specified functions and imports it into the <paramref name="kernel"/>'s plugin collection.</summary>
/// <param name="kernel">The <see cref="Kernel"/> containing services, plugins, and other state for use throughout the operation.</param>
/// <param name="pluginName">The name for the plugin.</param>
/// <param name="functions">The initial functions to be available as part of the plugin.</param>
/// <returns>A <see cref="KernelPlugin"/> containing the functions provided in <paramref name="functions"/>.</returns>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is null.</exception>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is an invalid plugin name.</exception>
/// <exception cref="ArgumentNullException"><paramref name="functions"/> contains a null function.</exception>
/// <exception cref="ArgumentException"><paramref name="functions"/> contains two functions with the same name.</exception>
public static KernelPlugin ImportPluginFromFunctions(this Kernel kernel, string pluginName, IEnumerable<KernelFunction>? functions) =>
ImportPluginFromFunctions(kernel, pluginName, description: null, functions);
/// <summary>Creates a plugin that contains the specified functions and imports it into the <paramref name="kernel"/>'s plugin collection.</summary>
/// <param name="kernel">The <see cref="Kernel"/> containing services, plugins, and other state for use throughout the operation.</param>
/// <param name="pluginName">The name for the plugin.</param>
/// <param name="description">A description of the plugin.</param>
/// <param name="functions">The initial functions to be available as part of the plugin.</param>
/// <returns>A <see cref="KernelPlugin"/> containing the functions provided in <paramref name="functions"/>.</returns>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is null.</exception>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is an invalid plugin name.</exception>
/// <exception cref="ArgumentNullException"><paramref name="functions"/> contains a null function.</exception>
/// <exception cref="ArgumentException"><paramref name="functions"/> contains two functions with the same name.</exception>
public static KernelPlugin ImportPluginFromFunctions(this Kernel kernel, string pluginName, string? description = null, IEnumerable<KernelFunction>? functions = null)
{
KernelPlugin plugin = CreatePluginFromFunctions(kernel, pluginName, description, functions);
kernel.Plugins.Add(plugin);
return plugin;
}
/// <summary>Creates a plugin that contains the specified functions and adds it into the plugin collection.</summary>
/// <param name="plugins">The plugin collection to which the new plugin should be added.</param>
/// <param name="pluginName">The name for the plugin.</param>
/// <param name="functions">The initial functions to be available as part of the plugin.</param>
/// <returns>A <see cref="KernelPlugin"/> containing the functions provided in <paramref name="functions"/>.</returns>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is null.</exception>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is an invalid plugin name.</exception>
/// <exception cref="ArgumentNullException"><paramref name="functions"/> contains a null function.</exception>
/// <exception cref="ArgumentException"><paramref name="functions"/> contains two functions with the same name.</exception>
public static KernelPlugin AddFromFunctions(this ICollection<KernelPlugin> plugins, string pluginName, IEnumerable<KernelFunction>? functions) =>
AddFromFunctions(plugins, pluginName, description: null, functions);
/// <summary>Creates a plugin that contains the specified functions and adds it into the plugin collection.</summary>
/// <param name="plugins">The plugin collection to which the new plugin should be added.</param>
/// <param name="pluginName">The name for the plugin.</param>
/// <param name="description">A description of the plugin.</param>
/// <param name="functions">The initial functions to be available as part of the plugin.</param>
/// <returns>A <see cref="KernelPlugin"/> containing the functions provided in <paramref name="functions"/>.</returns>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is null.</exception>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is an invalid plugin name.</exception>
/// <exception cref="ArgumentNullException"><paramref name="functions"/> contains a null function.</exception>
/// <exception cref="ArgumentException"><paramref name="functions"/> contains two functions with the same name.</exception>
public static KernelPlugin AddFromFunctions(this ICollection<KernelPlugin> plugins, string pluginName, string? description = null, IEnumerable<KernelFunction>? functions = null)
{
Verify.NotNull(plugins);
var plugin = new DefaultKernelPlugin(pluginName, description, functions);
plugins.Add(plugin);
return plugin;
}
/// <summary>Creates a plugin that wraps the specified target object and adds it into the plugin collection.</summary>
/// <param name="plugins">The plugin collection to which the new plugin should be added.</param>
/// <param name="pluginName">The name for the plugin.</param>
/// <param name="functions">The initial functions to be available as part of the plugin.</param>
/// <returns>The same instance as <paramref name="plugins"/>.</returns>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is null.</exception>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is an invalid plugin name.</exception>
/// <exception cref="ArgumentNullException"><paramref name="functions"/> contains a null function.</exception>
/// <exception cref="ArgumentException"><paramref name="functions"/> contains two functions with the same name.</exception>
public static IKernelBuilderPlugins AddFromFunctions(this IKernelBuilderPlugins plugins, string pluginName, IEnumerable<KernelFunction>? functions) =>
AddFromFunctions(plugins, pluginName, description: null, functions);
/// <summary>Creates a plugin that wraps the specified target object and adds it into the plugin collection.</summary>
/// <param name="plugins">The plugin collection to which the new plugin should be added.</param>
/// <param name="pluginName">The name for the plugin.</param>
/// <param name="description">A description of the plugin.</param>
/// <param name="functions">The initial functions to be available as part of the plugin.</param>
/// <returns>The same instance as <paramref name="plugins"/>.</returns>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is null.</exception>
/// <exception cref="ArgumentException"><paramref name="pluginName"/> is an invalid plugin name.</exception>
/// <exception cref="ArgumentNullException"><paramref name="functions"/> contains a null function.</exception>
/// <exception cref="ArgumentException"><paramref name="functions"/> contains two functions with the same name.</exception>
public static IKernelBuilderPlugins AddFromFunctions(this IKernelBuilderPlugins plugins, string pluginName, string? description = null, IEnumerable<KernelFunction>? functions = null)
{
Verify.NotNull(plugins);
plugins.Services.AddSingleton(KernelPluginFactory.CreateFromFunctions(pluginName, description, functions));
return plugins;
}
#endregion
#region CreatePluginFromDirectory
/// <summary>Creates a plugin containing one function per child directory of the specified <paramref name="pluginDirectory"/>.</summary>
/// <remarks>
@@ -574,6 +679,7 @@ public static class KernelExtensions
/// The <see cref="IPromptTemplateFactory"/> to use when interpreting the <paramref name="promptTemplate"/> into a <see cref="IPromptTemplate"/>.
/// If null, a default factory will be used.
/// </param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests. The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>The <typeparamref name="T"/> of the function result value.</returns>
/// <exception cref="ArgumentNullException"><paramref name="kernel"/> is null.</exception>
/// <exception cref="ArgumentNullException"><paramref name="promptTemplate"/> is null.</exception>
@@ -585,7 +691,8 @@ public static class KernelExtensions
string promptTemplate,
KernelArguments? arguments = null,
string? templateFormat = null,
IPromptTemplateFactory? promptTemplateFactory = null)
IPromptTemplateFactory? promptTemplateFactory = null,
CancellationToken cancellationToken = default)
{
Verify.NotNull(kernel);
Verify.NotNullOrWhiteSpace(promptTemplate);
@@ -596,7 +703,41 @@ public static class KernelExtensions
promptTemplateFactory: promptTemplateFactory,
loggerFactory: kernel.LoggerFactory);
return kernel.InvokeAsync<T>(function, arguments);
return kernel.InvokeAsync<T>(function, arguments, cancellationToken);
}
/// <summary>
/// Invokes a prompt specified via a prompt template and returns the results of type <typeparamref name="T"/>.
/// </summary>
/// <param name="kernel">The <see cref="Kernel"/> containing services, plugins, and other state for use throughout the operation.</param>
/// <param name="promptTemplate">Prompt template for the function.</param>
/// <param name="arguments">The arguments to pass to the function's invocation, including any <see cref="PromptExecutionSettings"/>.</param>
/// <param name="templateFormat">The template format of <paramref name="promptTemplate"/>. This must be provided if <paramref name="promptTemplateFactory"/> is not null.</param>
/// <param name="promptTemplateFactory">
/// The <see cref="IPromptTemplateFactory"/> to use when interpreting the <paramref name="promptTemplate"/> into a <see cref="IPromptTemplate"/>.
/// If null, a default factory will be used.
/// </param>
/// <returns>The <typeparamref name="T"/> of the function result value.</returns>
/// <exception cref="ArgumentNullException"><paramref name="kernel"/> is null.</exception>
/// <exception cref="ArgumentNullException"><paramref name="promptTemplate"/> is null.</exception>
/// <exception cref="ArgumentException"><paramref name="promptTemplate"/> is empty or composed entirely of whitespace.</exception>
/// <exception cref="KernelFunction">The function failed to invoke successfully.</exception>
/// <exception cref="KernelFunctionCanceledException">The <see cref="KernelFunction"/>'s invocation was canceled.</exception>
[EditorBrowsable(EditorBrowsableState.Never)]
public static Task<T?> InvokePromptAsync<T>(
this Kernel kernel,
string promptTemplate,
KernelArguments? arguments,
string? templateFormat,
IPromptTemplateFactory? promptTemplateFactory)
{
return InvokePromptAsync<T>(
kernel,
promptTemplate,
arguments,
templateFormat,
promptTemplateFactory,
CancellationToken.None);
}
#endregion
dotnet/src/SemanticKernel.Core/SemanticKernel.Core.csproj
+1
View File
@@ -8,6 +8,7 @@
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
<NoWarn>$(NoWarn);SKEXP0004</NoWarn>
<EnablePackageValidation>true</EnablePackageValidation>
</PropertyGroup>
<Import Project="$(RepoRoot)/dotnet/nuget/nuget-package.props" />
dotnet/src/SemanticKernel.UnitTests/Functions/KernelExtensionsTests.cs
+159
View File
@@ -0,0 +1,159 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.SemanticKernel;
using Xunit;
namespace SemanticKernel.UnitTests.Functions;
public class KernelExtensionsTests
{
[Fact]
public void CreatePluginFromFunctions()
{
Kernel kernel = new();
KernelPlugin plugin = kernel.CreatePluginFromFunctions("coolplugin", new[]
{
kernel.CreateFunctionFromMethod(() => { }, "Function1"),
kernel.CreateFunctionFromMethod(() => { }, "Function2"),
});
Assert.NotNull(plugin);
Assert.Empty(kernel.Plugins);
Assert.Equal("coolplugin", plugin.Name);
Assert.Empty(plugin.Description);
Assert.Equal(2, plugin.FunctionCount);
Assert.True(plugin.Contains("Function1"));
Assert.True(plugin.Contains("Function2"));
}
[Fact]
public void CreateEmptyPluginFromFunctions()
{
Kernel kernel = new();
KernelPlugin plugin = kernel.CreatePluginFromFunctions("coolplugin");
Assert.NotNull(plugin);
Assert.Empty(kernel.Plugins);
Assert.Equal("coolplugin", plugin.Name);
Assert.Empty(plugin.Description);
Assert.Empty(plugin);
Assert.Equal(0, plugin.FunctionCount);
}
[Fact]
public void CreatePluginFromDescriptionAndFunctions()
{
Kernel kernel = new();
KernelPlugin plugin = kernel.CreatePluginFromFunctions("coolplugin", "the description", new[]
{
kernel.CreateFunctionFromMethod(() => { }, "Function1"),
kernel.CreateFunctionFromMethod(() => { }, "Function2"),
});
Assert.NotNull(plugin);
Assert.Empty(kernel.Plugins);
Assert.Equal("coolplugin", plugin.Name);
Assert.Equal("the description", plugin.Description);
Assert.Equal(2, plugin.FunctionCount);
Assert.True(plugin.Contains("Function1"));
Assert.True(plugin.Contains("Function2"));
}
[Fact]
public void ImportPluginFromFunctions()
{
Kernel kernel = new();
kernel.ImportPluginFromFunctions("coolplugin", new[]
{
kernel.CreateFunctionFromMethod(() => { }, "Function1"),
kernel.CreateFunctionFromMethod(() => { }, "Function2"),
});
Assert.Single(kernel.Plugins);
KernelPlugin plugin = kernel.Plugins["coolplugin"];
Assert.Equal("coolplugin", plugin.Name);
Assert.Empty(plugin.Description);
Assert.NotNull(plugin);
Assert.Equal(2, plugin.FunctionCount);
Assert.True(plugin.Contains("Function1"));
Assert.True(plugin.Contains("Function2"));
}
[Fact]
public void ImportPluginFromDescriptionAndFunctions()
{
Kernel kernel = new();
kernel.ImportPluginFromFunctions("coolplugin", "the description", new[]
{
kernel.CreateFunctionFromMethod(() => { }, "Function1"),
kernel.CreateFunctionFromMethod(() => { }, "Function2"),
});
Assert.Single(kernel.Plugins);
KernelPlugin plugin = kernel.Plugins["coolplugin"];
Assert.Equal("coolplugin", plugin.Name);
Assert.Equal("the description", plugin.Description);
Assert.NotNull(plugin);
Assert.Equal(2, plugin.FunctionCount);
Assert.True(plugin.Contains("Function1"));
Assert.True(plugin.Contains("Function2"));
}
[Fact]
public void AddFromFunctions()
{
Kernel kernel = new();
kernel.Plugins.AddFromFunctions("coolplugin", new[]
{
kernel.CreateFunctionFromMethod(() => { }, "Function1"),
kernel.CreateFunctionFromMethod(() => { }, "Function2"),
});
Assert.Single(kernel.Plugins);
KernelPlugin plugin = kernel.Plugins["coolplugin"];
Assert.Equal("coolplugin", plugin.Name);
Assert.Empty(plugin.Description);
Assert.NotNull(plugin);
Assert.Equal(2, plugin.FunctionCount);
Assert.True(plugin.Contains("Function1"));
Assert.True(plugin.Contains("Function2"));
}
[Fact]
public void AddFromDescriptionAndFunctions()
{
Kernel kernel = new();
kernel.Plugins.AddFromFunctions("coolplugin", "the description", new[]
{
kernel.CreateFunctionFromMethod(() => { }, "Function1"),
kernel.CreateFunctionFromMethod(() => { }, "Function2"),
});
Assert.Single(kernel.Plugins);
KernelPlugin plugin = kernel.Plugins["coolplugin"];
Assert.Equal("coolplugin", plugin.Name);
Assert.Equal("the description", plugin.Description);
Assert.NotNull(plugin);
Assert.Equal(2, plugin.FunctionCount);
Assert.True(plugin.Contains("Function1"));
Assert.True(plugin.Contains("Function2"));
}
}
dotnet/src/SemanticKernel.UnitTests/Functions/KernelFunctionFromPromptTests.cs
+2 -2
View File
@@ -48,7 +48,7 @@ public class KernelFunctionFromPromptTests
builder.Services.AddSingleton(factory.Object);
Kernel kernel = builder.Build();
kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("jk", functions: new[] { kernel.CreateFunctionFromPrompt(promptTemplate: "Tell me a joke", functionName: "joker", description: "Nice fun") }));
kernel.ImportPluginFromFunctions("jk", functions: new[] { kernel.CreateFunctionFromPrompt(promptTemplate: "Tell me a joke", functionName: "joker", description: "Nice fun") });
// Act & Assert - 3 functions, var name is not case sensitive
Assert.True(kernel.Plugins.TryGetFunction("jk", "joker", out _));
@@ -573,7 +573,7 @@ public class KernelFunctionFromPromptTests
KernelFunction function1 = KernelFunctionFactory.CreateFromPrompt(new PromptTemplateConfig { Name = "Prompt1", Template = "Prompt1", ExecutionSettings = new() { ["service1"] = new OpenAIPromptExecutionSettings { MaxTokens = 1000 } } });
KernelFunction function2 = KernelFunctionFactory.CreateFromPrompt(new PromptTemplateConfig { Name = "Prompt2", Template = "Prompt2 {{MyPrompts.Prompt1}}", ExecutionSettings = new() { ["service2"] = new OpenAIPromptExecutionSettings { MaxTokens = 2000 } } });
kernel.Plugins.AddFromFunctions("MyPrompts", "Prompt1 and Prompt2", new[] { function1, function2 });
kernel.ImportPluginFromFunctions("MyPrompts", new[] { function1, function2 });
// Act
var result = await kernel.InvokeAsync(function2);
dotnet/src/SemanticKernel.UnitTests/KernelTests.cs
+1 -1
View File
@@ -503,7 +503,7 @@ public class KernelTests
var function = KernelFunctionFactory.CreateFromMethod(() => "fake result", "function");
var kernel = new Kernel();
kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { function }));
kernel.ImportPluginFromFunctions("plugin", new[] { function });
//Act
var result = await kernel.InvokeAsync("plugin", "function");
dotnet/src/SemanticKernel.UnitTests/PromptTemplate/KernelPromptTemplateTests.cs
+12 -12
View File
@@ -106,11 +106,11 @@ public sealed class KernelPromptTemplateTests
// Arrange
var template = "This {{$x11}} {{$a}}{{$missing}} test template {{p.bar $b}} and {{p.food c='literal \"c\"' d = $d}}";
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("p", "description", new[]
this._kernel.ImportPluginFromFunctions("p", new[]
{
KernelFunctionFactory.CreateFromMethod((string input) => "with function that accepts " + input, "bar"),
KernelFunctionFactory.CreateFromMethod((string c, string d) => "another one with " + c + d, "food"),
}));
});
var target = (KernelPromptTemplate)this._factory.Create(new PromptTemplateConfig(template));
@@ -186,7 +186,7 @@ public sealed class KernelPromptTemplateTests
canary = input;
}
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("p", "description", new[] { KernelFunctionFactory.CreateFromMethod(Foo, "bar") }));
this._kernel.ImportPluginFromFunctions("p", new[] { KernelFunctionFactory.CreateFromMethod(Foo, "bar") });
var template = "This is a test template that references variable that does not have argument. {{p.bar $foo}}.";
@@ -210,7 +210,7 @@ public sealed class KernelPromptTemplateTests
canary = input;
}
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("p", "description", new[] { KernelFunctionFactory.CreateFromMethod(Foo, "bar") }));
this._kernel.ImportPluginFromFunctions("p", new[] { KernelFunctionFactory.CreateFromMethod(Foo, "bar") });
var template = "This is a test template that references variable that have null argument{{p.bar $foo}}.";
@@ -236,7 +236,7 @@ public sealed class KernelPromptTemplateTests
canary = input;
}
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("p", "description", new[] { KernelFunctionFactory.CreateFromMethod(Foo, "bar") }));
this._kernel.ImportPluginFromFunctions("p", new[] { KernelFunctionFactory.CreateFromMethod(Foo, "bar") });
var template = "This is a test template that {{$zoo}}references variables that have null arguments{{p.bar $foo}}.";
@@ -265,7 +265,7 @@ public sealed class KernelPromptTemplateTests
canary = input;
}
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("p", "description", new[] { KernelFunctionFactory.CreateFromMethod(Foo, "bar") }));
this._kernel.ImportPluginFromFunctions("p", new[] { KernelFunctionFactory.CreateFromMethod(Foo, "bar") });
var template = "This is a test template that {{$zoo}}references variables that do not have arguments{{p.bar $foo}}.";
@@ -292,7 +292,7 @@ public sealed class KernelPromptTemplateTests
var func = KernelFunctionFactory.CreateFromMethod(MyFunctionAsync, "function");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { func }));
this._kernel.ImportPluginFromFunctions("plugin", new[] { func });
this._arguments[InputParameterName] = "INPUT-BAR";
@@ -318,7 +318,7 @@ public sealed class KernelPromptTemplateTests
var func = KernelFunctionFactory.CreateFromMethod(MyFunctionAsync, "function");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { func }));
this._kernel.ImportPluginFromFunctions("plugin", new[] { func });
this._arguments["myVar"] = "BAR";
var template = "foo-{{plugin.function $myVar}}-baz";
@@ -348,7 +348,7 @@ public sealed class KernelPromptTemplateTests
var func = KernelFunctionFactory.CreateFromMethod(MyFunctionAsync, "function");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { func }));
this._kernel.ImportPluginFromFunctions("plugin", new[] { func });
this._arguments[InputParameterName] = "Mario";
this._arguments["someDate"] = "2023-08-25T00:00:00";
@@ -395,7 +395,7 @@ public sealed class KernelPromptTemplateTests
KernelFunction func = KernelFunctionFactory.CreateFromMethod(MyFunctionAsync, "function");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { func }));
this._kernel.ImportPluginFromFunctions("plugin", new[] { func });
this._arguments[InputParameterName] = "Mario";
this._arguments["someDate"] = "2023-08-25T00:00:00";
@@ -461,7 +461,7 @@ public sealed class KernelPromptTemplateTests
KernelFunction func = KernelFunctionFactory.CreateFromMethod(MyFunctionAsync, "function");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { func }));
this._kernel.ImportPluginFromFunctions("plugin", new[] { func });
this._arguments["myVar"] = "BAR";
@@ -497,7 +497,7 @@ public sealed class KernelPromptTemplateTests
"f");
this._kernel.Culture = new CultureInfo("fr-FR"); //In French culture, a comma is used as a decimal separator, and a slash is used as a date separator. See the Assert below.
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("p", "description", new[] { func }));
this._kernel.ImportPluginFromFunctions("p", new[] { func });
var template = "int:{{$i}}, double:{{$d}}, {{p.f $s g=$g}}, DateTime:{{$dt}}, enum:{{$e}}";
dotnet/src/SemanticKernel.UnitTests/TemplateEngine/Blocks/CodeBlockTests.cs
+13 -17
View File
@@ -34,7 +34,7 @@ public class CodeBlockTests
static void method() => throw new FormatException("error");
var function = KernelFunctionFactory.CreateFromMethod(method, "function", "description");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { function }));
this._kernel.ImportPluginFromFunctions("plugin", new[] { function });
var target = new CodeBlock("plugin.function");
@@ -194,7 +194,7 @@ public class CodeBlockTests
},
"function");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { function }));
this._kernel.ImportPluginFromFunctions("plugin", new[] { function });
// Act
var codeBlock = new CodeBlock(new List<Block> { funcId, varBlock }, "");
@@ -222,7 +222,7 @@ public class CodeBlockTests
},
"function");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { function }));
this._kernel.ImportPluginFromFunctions("plugin", new[] { function });
// Act
var codeBlock = new CodeBlock(new List<Block> { funcBlock, valBlock }, "");
@@ -259,7 +259,7 @@ public class CodeBlockTests
},
"function");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { function }));
this._kernel.ImportPluginFromFunctions("plugin", new[] { function });
// Act
var codeBlock = new CodeBlock(new List<Block> { funcId, namedArgBlock1, namedArgBlock2 }, "");
@@ -282,10 +282,10 @@ public class CodeBlockTests
var varBlock = new VarBlock("$var");
var namedArgBlock = new NamedArgBlock("p1=$a1");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("p", "description", new[] { KernelFunctionFactory.CreateFromMethod((object p1) =>
this._kernel.ImportPluginFromFunctions("p", new[] { KernelFunctionFactory.CreateFromMethod((object p1) =>
{
canary = p1;
}, "f") }));
}, "f") });
// Act
var functionWithPositionedArgument = new CodeBlock(new List<Block> { funcId, varBlock }, "");
@@ -323,7 +323,7 @@ public class CodeBlockTests
var function = KernelFunctionFactory.CreateFromMethod((string foo, string baz) => { }, "function");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { function }));
this._kernel.ImportPluginFromFunctions("plugin", new[] { function });
// Act
var codeBlock = new CodeBlock(new List<Block> { funcId, namedArgBlock1, namedArgBlock2 }, "");
@@ -363,7 +363,7 @@ public class CodeBlockTests
var function = KernelFunctionFactory.CreateFromMethod(() => { }, "function");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { function }));
this._kernel.ImportPluginFromFunctions("plugin", new[] { function });
// Act
var codeBlock = new CodeBlock(blockList, "");
@@ -392,15 +392,13 @@ public class CodeBlockTests
new ValBlock($"'{FooValue}'")
};
kernel.Plugins.Add(
KernelPluginFactory.CreateFromFunctions("Plugin1", functions: new[]
kernel.ImportPluginFromFunctions("Plugin1", functions: new[]
{
kernel.CreateFunctionFromPrompt(
promptTemplate: $"\"This {{{{${parameterName}}}}}",
functionName: "Function1")
}
)
);
);
kernel.PromptRendering += (object? sender, PromptRenderingEventArgs e) =>
{
@@ -438,15 +436,13 @@ public class CodeBlockTests
new NamedArgBlock("x12='new'") // Extra parameters are ignored
};
kernel.Plugins.Add(
KernelPluginFactory.CreateFromFunctions("Plugin1", functions: new[]
kernel.ImportPluginFromFunctions("Plugin1", functions: new[]
{
kernel.CreateFunctionFromPrompt(
promptTemplate: "\"This {{$x11}}",
functionName: "Function1")
}
)
);
);
kernel.PromptRendering += (object? sender, PromptRenderingEventArgs e) =>
{
@@ -490,7 +486,7 @@ public class CodeBlockTests
},
"function");
this._kernel.Plugins.Add(KernelPluginFactory.CreateFromFunctions("plugin", "description", new[] { function }));
this._kernel.ImportPluginFromFunctions("plugin", new[] { function });
// Act
var codeBlock = new CodeBlock(new List<Block> { funcId, namedArgBlock1, namedArgBlock2 }, "");
python/.cspell.json
+46
View File
@@ -0,0 +1,46 @@
{
"version": "0.2",
"languageSettings": [
{
"languageId": "py",
"allowCompoundWords": true,
"locale": "en-US"
}
],
"language": "en-US",
"patterns": [
{
"name": "import",
"pattern": "import [a-zA-Z0-9_]+"
},
{
"name": "from import",
"pattern": "from [a-zA-Z0-9_]+ import [a-zA-Z0-9_]+"
}
],
"ignorePaths": [
"samples/**",
"notebooks/**"
],
"words": [
"aeiou",
"azuredocindex",
"azuredocs",
"contentvector",
"dotenv",
"logit",
"logprobs",
"mongocluster",
"ndarray",
"nopep",
"ollama",
"onyourdatatest",
"OPENAI",
"pydantic",
"retrywrites",
"skfunction",
"skprompt",
"templating",
"vectordb"
]
}
python/.vscode/settings.json
Vendored
-19
View File
@@ -1,7 +1,4 @@
{
"python.analysis.extraPaths": [
"./semantic_kernel"
],
"explorer.compactFolders": false,
"prettier.enable": true,
"editor.formatOnType": true,
@@ -16,22 +13,6 @@
"locale": "en-US"
}
],
"cSpell.words": [
"aeiou",
"nopep",
"OPENAI",
"skfunction"
],
"cSpell.patterns": [
{
"name": "import",
"pattern": "import [a-zA-Z0-9_]+"
},
{
"name": "from import",
"pattern": "from [a-zA-Z0-9_]+ import [a-zA-Z0-9_]+"
}
],
"python.testing.pytestArgs": [
"tests"
],
python/notebooks/00-getting-started.ipynb
+7 -3
View File
@@ -16,7 +16,7 @@
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.4.dev0"
"!python -m pip install semantic-kernel==0.4.5.dev0"
]
},
{
@@ -57,7 +57,9 @@
"\n",
"api_key, org_id = sk.openai_settings_from_dot_env()\n",
"\n",
"kernel.add_chat_service(\"chat-gpt\", OpenAIChatCompletion(ai_model_id=\"gpt-3.5-turbo-1106\", api_key=api_key, org_id=org_id))"
"kernel.add_chat_service(\n",
" \"chat-gpt\", OpenAIChatCompletion(ai_model_id=\"gpt-3.5-turbo-1106\", api_key=api_key, org_id=org_id)\n",
")"
]
},
{
@@ -88,7 +90,9 @@
"\n",
"deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()\n",
"\n",
"kernel.add_chat_service(\"chat_completion\", AzureChatCompletion(deployment_name=deployment, endpoint=endpoint, api_key=api_key))\n"
"kernel.add_chat_service(\n",
" \"chat_completion\", AzureChatCompletion(deployment_name=deployment, endpoint=endpoint, api_key=api_key)\n",
")"
]
},
{
python/notebooks/01-basic-loading-the-kernel.ipynb
+7 -7
View File
@@ -25,7 +25,7 @@
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.4.dev0"
"!python -m pip install semantic-kernel==0.4.5.dev0"
]
},
{
@@ -79,13 +79,13 @@
"source": [
"kernel = sk.Kernel()\n",
"\n",
"kernel.add_chat_service( # We are adding a text service\n",
" \"Azure_curie\", # The alias we can use in prompt templates' config.json\n",
"kernel.add_chat_service( # We are adding a text service\n",
" \"Azure_curie\", # The alias we can use in prompt templates' config.json\n",
" AzureChatCompletion(\n",
" deployment_name=\"my-finetuned-Curie\", # Azure OpenAI *Deployment name*\n",
" endpoint=\"https://contoso.openai.azure.com/\", # Azure OpenAI *Endpoint*\n",
" api_key=\"...your Azure OpenAI Key...\" # Azure OpenAI *Key*\n",
" )\n",
" deployment_name=\"my-finetuned-Curie\", # Azure OpenAI *Deployment name*\n",
" endpoint=\"https://contoso.openai.azure.com/\", # Azure OpenAI *Endpoint*\n",
" api_key=\"...your Azure OpenAI Key...\", # Azure OpenAI *Key*\n",
" ),\n",
")\n",
"\n",
"kernel.add_chat_service( # We are adding a text service\n",
python/notebooks/02-running-prompts-from-file.ipynb
+1 -1
View File
@@ -89,7 +89,7 @@
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.4.dev0"
"!python -m pip install semantic-kernel==0.4.5.dev0"
]
},
{
python/notebooks/03-semantic-function-inline.ipynb
+1 -1
View File
@@ -55,7 +55,7 @@
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.4.dev0"
"!python -m pip install semantic-kernel==0.4.5.dev0"
]
},
{
python/notebooks/04-context-variables-chat.ipynb
+1 -1
View File
@@ -26,7 +26,7 @@
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.4.dev0"
"!python -m pip install semantic-kernel==0.4.5.dev0"
]
},
{
python/notebooks/05-using-the-planner.ipynb
+1 -1
View File
@@ -23,7 +23,7 @@
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.4.dev0"
"!python -m pip install semantic-kernel==0.4.5.dev0"
]
},
{
python/notebooks/06-memory-and-embeddings.ipynb
+1 -1
View File
@@ -28,7 +28,7 @@
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.4.dev0"
"!python -m pip install semantic-kernel==0.4.5.dev0"
]
},
{
python/notebooks/07-hugging-face-for-skills.ipynb
+167 -189
View File
@@ -1,192 +1,170 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "68e1c158",
"metadata": {},
"source": [
"# Using Hugging Face With Skills\n",
"\n",
"In this notebook, we demonstrate using Hugging Face models for Skills using both SemanticMemory and text completions. \n",
"\n",
"SK supports downloading models from the Hugging Face that can perform the following tasks: text-generation, text2text-generation, summarization, and sentence-similarity. You can search for models by task at https://huggingface.co/models."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a77bdf89",
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.4.dev0\n",
"\n",
"# Note that additional dependencies are required for the Hugging Face connectors:\n",
"!python -m pip install torch==2.0.0\n",
"!python -m pip install transformers==^4.28.1\n",
"!python -m pip install sentence-transformers==^2.2.2"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "508ad44f",
"metadata": {},
"outputs": [],
"source": [
"import semantic_kernel as sk\n",
"import semantic_kernel.connectors.ai.hugging_face as sk_hf"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d8ddffc1",
"metadata": {},
"source": [
"First, we will create a kernel and add both text completion and embedding services. \n",
"\n",
"For text completion, we are choosing GPT2. This is a text-generation model. (Note: text-generation will repeat the input in the output, text2text-generation will not.)\n",
"For embeddings, we are using sentence-transformers/all-MiniLM-L6-v2. Vectors generated for this model are of length 384 (compared to a length of 1536 from OpenAI ADA).\n",
"\n",
"The following step may take a few minutes when run for the first time as the models will be downloaded to your local machine."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8f8dcbc6",
"metadata": {},
"outputs": [],
"source": [
"kernel = sk.Kernel()\n",
"\n",
"# Configure LLM service\n",
"kernel.add_text_completion_service(\n",
" service_id=\"gpt2\", \n",
" service=sk_hf.HuggingFaceTextCompletion(ai_model_id=\"gpt2\", task=\"text-generation\")\n",
")\n",
"kernel.add_text_embedding_generation_service(\n",
" service_id=\"sentence-transformers/all-MiniLM-L6-v2\",\n",
" service=sk_hf.HuggingFaceTextEmbedding(ai_model_id=\"sentence-transformers/all-MiniLM-L6-v2\"),\n",
")\n",
"kernel.register_memory_store(memory_store=sk.memory.VolatileMemoryStore())\n",
"kernel.import_skill(sk.core_skills.TextMemorySkill())"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "2a7e7ca4",
"metadata": {},
"source": [
"### Add Memories and Define a skill to use them\n",
"\n",
"Most models available on huggingface.co are not as powerful as OpenAI GPT-3+. Your skills will likely need to be simpler to accommodate this."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d096504c",
"metadata": {},
"outputs": [],
"source": [
"await kernel.memory.save_information_async(\n",
" collection=\"animal-facts\", id=\"info1\", text=\"Sharks are fish.\"\n",
")\n",
"await kernel.memory.save_information_async(\n",
" collection=\"animal-facts\", id=\"info2\", text=\"Whales are mammals.\"\n",
")\n",
"await kernel.memory.save_information_async(\n",
" collection=\"animal-facts\", id=\"info3\", text=\"Penguins are birds.\"\n",
")\n",
"await kernel.memory.save_information_async(\n",
" collection=\"animal-facts\", id=\"info4\", text=\"Dolphins are mammals.\"\n",
")\n",
"await kernel.memory.save_information_async(\n",
" collection=\"animal-facts\", id=\"info5\", text=\"Flies are insects.\"\n",
")\n",
"\n",
"# Define semantic function using SK prompt template language\n",
"my_prompt = \"\"\"I know these animal facts: {{recall $query1}} {{recall $query2}} {{recall $query3}} and \"\"\"\n",
"\n",
"# Create the semantic function\n",
"my_function = kernel.create_semantic_function(\n",
" prompt_template=my_prompt, max_tokens=45, temperature=0.5, top_p=0.5\n",
")\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "2calf857",
"metadata": {},
"source": [
"Let's now see what the completion looks like! Remember, \"gpt2\" is nowhere near as large as ChatGPT, so expect a much simpler answer."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "628c843e",
"metadata": {},
"outputs": [],
"source": [
"context = kernel.create_new_context()\n",
"context[sk.core_skills.TextMemorySkill.COLLECTION_PARAM] = \"animal-facts\"\n",
"context[sk.core_skills.TextMemorySkill.RELEVANCE_PARAM] = \"0.3\"\n",
"\n",
"context[\"query1\"] = \"animal that swims\"\n",
"context[\"query2\"] = \"animal that flies\"\n",
"context[\"query3\"] = \"penguins are?\"\n",
"output = await kernel.run_async(my_function, input_vars=context.variables)\n",
"\n",
"output = str(output).strip()\n",
"\n",
"query_result1 = await kernel.memory.search_async(\n",
" collection=\"animal-facts\",\n",
" query=context[\"query1\"],\n",
" limit=1,\n",
" min_relevance_score=0.3\n",
")\n",
"query_result2 = await kernel.memory.search_async(\n",
" collection=\"animal-facts\",\n",
" query=context[\"query2\"],\n",
" limit=1,\n",
" min_relevance_score=0.3\n",
")\n",
"query_result3 = await kernel.memory.search_async(\n",
" collection=\"animal-facts\",\n",
" query=context[\"query3\"],\n",
" limit=1,\n",
" min_relevance_score=0.3\n",
")\n",
"\n",
"print(f\"gpt2 completed prompt with: '{output}'\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
}
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "68e1c158",
"metadata": {},
"source": [
"# Using Hugging Face With Skills\n",
"\n",
"In this notebook, we demonstrate using Hugging Face models for Skills using both SemanticMemory and text completions. \n",
"\n",
"SK supports downloading models from the Hugging Face that can perform the following tasks: text-generation, text2text-generation, summarization, and sentence-similarity. You can search for models by task at https://huggingface.co/models."
]
},
"nbformat": 4,
"nbformat_minor": 5
{
"cell_type": "code",
"execution_count": null,
"id": "a77bdf89",
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.5.dev0\n",
"\n",
"# Note that additional dependencies are required for the Hugging Face connectors:\n",
"!python -m pip install torch==2.0.0\n",
"!python -m pip install transformers==^4.28.1\n",
"!python -m pip install sentence-transformers==^2.2.2"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "508ad44f",
"metadata": {},
"outputs": [],
"source": [
"import semantic_kernel as sk\n",
"import semantic_kernel.connectors.ai.hugging_face as sk_hf"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d8ddffc1",
"metadata": {},
"source": [
"First, we will create a kernel and add both text completion and embedding services. \n",
"\n",
"For text completion, we are choosing GPT2. This is a text-generation model. (Note: text-generation will repeat the input in the output, text2text-generation will not.)\n",
"For embeddings, we are using sentence-transformers/all-MiniLM-L6-v2. Vectors generated for this model are of length 384 (compared to a length of 1536 from OpenAI ADA).\n",
"\n",
"The following step may take a few minutes when run for the first time as the models will be downloaded to your local machine."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8f8dcbc6",
"metadata": {},
"outputs": [],
"source": [
"kernel = sk.Kernel()\n",
"\n",
"# Configure LLM service\n",
"kernel.add_text_completion_service(\n",
" service_id=\"gpt2\", service=sk_hf.HuggingFaceTextCompletion(ai_model_id=\"gpt2\", task=\"text-generation\")\n",
")\n",
"kernel.add_text_embedding_generation_service(\n",
" service_id=\"sentence-transformers/all-MiniLM-L6-v2\",\n",
" service=sk_hf.HuggingFaceTextEmbedding(ai_model_id=\"sentence-transformers/all-MiniLM-L6-v2\"),\n",
")\n",
"kernel.register_memory_store(memory_store=sk.memory.VolatileMemoryStore())\n",
"kernel.import_skill(sk.core_skills.TextMemorySkill())"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "2a7e7ca4",
"metadata": {},
"source": [
"### Add Memories and Define a skill to use them\n",
"\n",
"Most models available on huggingface.co are not as powerful as OpenAI GPT-3+. Your skills will likely need to be simpler to accommodate this."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d096504c",
"metadata": {},
"outputs": [],
"source": [
"await kernel.memory.save_information_async(collection=\"animal-facts\", id=\"info1\", text=\"Sharks are fish.\")\n",
"await kernel.memory.save_information_async(collection=\"animal-facts\", id=\"info2\", text=\"Whales are mammals.\")\n",
"await kernel.memory.save_information_async(collection=\"animal-facts\", id=\"info3\", text=\"Penguins are birds.\")\n",
"await kernel.memory.save_information_async(collection=\"animal-facts\", id=\"info4\", text=\"Dolphins are mammals.\")\n",
"await kernel.memory.save_information_async(collection=\"animal-facts\", id=\"info5\", text=\"Flies are insects.\")\n",
"\n",
"# Define semantic function using SK prompt template language\n",
"my_prompt = \"\"\"I know these animal facts: {{recall $query1}} {{recall $query2}} {{recall $query3}} and \"\"\"\n",
"\n",
"# Create the semantic function\n",
"my_function = kernel.create_semantic_function(prompt_template=my_prompt, max_tokens=45, temperature=0.5, top_p=0.5)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "2calf857",
"metadata": {},
"source": [
"Let's now see what the completion looks like! Remember, \"gpt2\" is nowhere near as large as ChatGPT, so expect a much simpler answer."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "628c843e",
"metadata": {},
"outputs": [],
"source": [
"context = kernel.create_new_context()\n",
"context[sk.core_skills.TextMemorySkill.COLLECTION_PARAM] = \"animal-facts\"\n",
"context[sk.core_skills.TextMemorySkill.RELEVANCE_PARAM] = \"0.3\"\n",
"\n",
"context[\"query1\"] = \"animal that swims\"\n",
"context[\"query2\"] = \"animal that flies\"\n",
"context[\"query3\"] = \"penguins are?\"\n",
"output = await kernel.run_async(my_function, input_vars=context.variables)\n",
"\n",
"output = str(output).strip()\n",
"\n",
"query_result1 = await kernel.memory.search_async(\n",
" collection=\"animal-facts\", query=context[\"query1\"], limit=1, min_relevance_score=0.3\n",
")\n",
"query_result2 = await kernel.memory.search_async(\n",
" collection=\"animal-facts\", query=context[\"query2\"], limit=1, min_relevance_score=0.3\n",
")\n",
"query_result3 = await kernel.memory.search_async(\n",
" collection=\"animal-facts\", query=context[\"query3\"], limit=1, min_relevance_score=0.3\n",
")\n",
"\n",
"print(f\"gpt2 completed prompt with: '{output}'\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
python/notebooks/08-native-function-inline.ipynb
+58 -63
View File
@@ -46,7 +46,7 @@
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.4.dev0"
"!python -m pip install semantic-kernel==0.4.5.dev0"
]
},
{
@@ -66,7 +66,9 @@
"# Configure AI service used by the kernel\n",
"if useAzureOpenAI:\n",
" deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()\n",
" azure_chat_service = AzureChatCompletion(deployment_name=\"turbo\", endpoint=endpoint, api_key=api_key) # set the deployment name to the value of your chat model\n",
" azure_chat_service = AzureChatCompletion(\n",
" deployment_name=\"turbo\", endpoint=endpoint, api_key=api_key\n",
" ) # set the deployment name to the value of your chat model\n",
" kernel.add_chat_service(\"chat_completion\", azure_chat_service)\n",
"else:\n",
" api_key, org_id = sk.openai_settings_from_dot_env()\n",
@@ -102,15 +104,13 @@
"import random\n",
"from semantic_kernel.skill_definition import sk_function\n",
"\n",
"\n",
"class GenerateNumberSkill:\n",
" \"\"\"\n",
" Description: Generate a number between 3-x.\n",
" \"\"\"\n",
"\n",
" @sk_function(\n",
" description=\"Generate a random number between 3-x\",\n",
" name=\"GenerateNumberThreeOrHigher\"\n",
" )\n",
" @sk_function(description=\"Generate a random number between 3-x\", name=\"GenerateNumberThreeOrHigher\")\n",
" def generate_number_three_or_higher(self, input: str) -> str:\n",
" \"\"\"\n",
" Generate a number between 3-<input>\n",
@@ -122,7 +122,7 @@
" int value\n",
" \"\"\"\n",
" try:\n",
" return str(random.randint(3, int(input))) \n",
" return str(random.randint(3, int(input)))\n",
" except ValueError as e:\n",
" print(f\"Invalid input {input}\")\n",
" raise e"
@@ -153,13 +153,15 @@
"- Be exactly {{$input}} paragraphs long\n",
"\"\"\"\n",
"\n",
"corgi_story = kernel.create_semantic_function(prompt_template=sk_prompt,\n",
" function_name=\"CorgiStory\",\n",
" skill_name=\"CorgiSkill\",\n",
" description=\"Write a short story about two Corgis on an adventure\",\n",
" max_tokens=500,\n",
" temperature=0.5,\n",
" top_p=0.5)\n",
"corgi_story = kernel.create_semantic_function(\n",
" prompt_template=sk_prompt,\n",
" function_name=\"CorgiStory\",\n",
" skill_name=\"CorgiSkill\",\n",
" description=\"Write a short story about two Corgis on an adventure\",\n",
" max_tokens=500,\n",
" temperature=0.5,\n",
" top_p=0.5,\n",
")\n",
"\n",
"generate_number_skill = kernel.import_skill(GenerateNumberSkill())"
]
@@ -217,6 +219,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "d54983d8",
"metadata": {},
"outputs": [],
"source": [
@@ -230,7 +233,9 @@
"# Configure AI service used by the kernel\n",
"if useAzureOpenAI:\n",
" deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()\n",
" azure_chat_service = AzureChatCompletion(deployment_name=\"turbo\", endpoint=endpoint, api_key=api_key) # set the deployment name to the value of your chat model\n",
" azure_chat_service = AzureChatCompletion(\n",
" deployment_name=\"turbo\", endpoint=endpoint, api_key=api_key\n",
" ) # set the deployment name to the value of your chat model\n",
" kernel.add_chat_service(\"chat_completion\", azure_chat_service)\n",
"else:\n",
" api_key, org_id = sk.openai_settings_from_dot_env()\n",
@@ -258,15 +263,13 @@
"from semantic_kernel.skill_definition import sk_function, sk_function_context_parameter\n",
"from semantic_kernel import SKContext\n",
"\n",
"\n",
"class GenerateNumberSkill:\n",
" \"\"\"\n",
" Description: Generate a number between a min and a max.\n",
" \"\"\"\n",
"\n",
" @sk_function(\n",
" description=\"Generate a random number between min and max\",\n",
" name=\"GenerateNumber\"\n",
" )\n",
" @sk_function(description=\"Generate a random number between min and max\", name=\"GenerateNumber\")\n",
" @sk_function_context_parameter(name=\"min\", description=\"Minimum number of paragraphs.\")\n",
" @sk_function_context_parameter(name=\"max\", description=\"Maximum number of paragraphs.\", default_value=\"10\")\n",
" def generate_number(self, context: SKContext) -> str:\n",
@@ -281,7 +284,7 @@
" int value\n",
" \"\"\"\n",
" try:\n",
" return str(random.randint(int(context[\"min\"]), int(context[\"max\"]))) \n",
" return str(random.randint(int(context[\"min\"]), int(context[\"max\"])))\n",
" except ValueError as e:\n",
" print(f\"Invalid input {context['min']} {context['max']}\")\n",
" raise e"
@@ -324,13 +327,15 @@
"- Be written in this language: {{$language}}\n",
"\"\"\"\n",
"\n",
"corgi_story = kernel.create_semantic_function(prompt_template=sk_prompt,\n",
" function_name=\"CorgiStory\",\n",
" skill_name=\"CorgiSkill\",\n",
" description=\"Write a short story about two Corgis on an adventure\",\n",
" max_tokens=500,\n",
" temperature=0.5,\n",
" top_p=0.5)"
"corgi_story = kernel.create_semantic_function(\n",
" prompt_template=sk_prompt,\n",
" function_name=\"CorgiStory\",\n",
" skill_name=\"CorgiSkill\",\n",
" description=\"Write a short story about two Corgis on an adventure\",\n",
" max_tokens=500,\n",
" temperature=0.5,\n",
" top_p=0.5,\n",
")"
]
},
{
@@ -349,11 +354,7 @@
"metadata": {},
"outputs": [],
"source": [
"context_variables = sk.ContextVariables(variables={\n",
" \"min\": \"1\",\n",
" \"max\": \"5\",\n",
" \"language\": \"Spanish\"\n",
"})"
"context_variables = sk.ContextVariables(variables={\"min\": \"1\", \"max\": \"5\", \"language\": \"Spanish\"})"
]
},
{
@@ -372,7 +373,7 @@
"metadata": {},
"outputs": [],
"source": [
"context_variables['paragraph_count'] = generate_number.invoke(variables=context_variables).result"
"context_variables[\"paragraph_count\"] = generate_number.invoke(variables=context_variables).result"
]
},
{
@@ -395,8 +396,11 @@
},
"outputs": [],
"source": [
"print(\"Generating a corgi story exactly {} paragraphs long in {} language: \".format(context_variables[\"paragraph_count\"],\n",
" context_variables[\"language\"]))\n",
"print(\n",
" \"Generating a corgi story exactly {} paragraphs long in {} language: \".format(\n",
" context_variables[\"paragraph_count\"], context_variables[\"language\"]\n",
" )\n",
")\n",
"print(\"=====================================================\")\n",
"print(story)"
]
@@ -426,6 +430,7 @@
"import random\n",
"from semantic_kernel.skill_definition import sk_function\n",
"\n",
"\n",
"class GenerateNamesSkill:\n",
" \"\"\"\n",
" Description: Generate character names.\n",
@@ -434,25 +439,14 @@
" # The default function name will be the name of the function itself, however you can override this\n",
" # by setting the name=<name override> in the @sk_function decorator. In this case, we're using\n",
" # the same name as the function name for simplicity.\n",
" @sk_function(\n",
" description=\"Generate character names\",\n",
" name=\"generate_names\"\n",
" )\n",
" @sk_function(description=\"Generate character names\", name=\"generate_names\")\n",
" def generate_names(self) -> str:\n",
" \"\"\"\n",
" Generate two names.\n",
" Returns:\n",
" str\n",
" \"\"\"\n",
" names = {\n",
" \"Hoagie\",\n",
" \"Hamilton\",\n",
" \"Bacon\",\n",
" \"Pizza\",\n",
" \"Boots\",\n",
" \"Shorts\",\n",
" \"Tuna\"\n",
" }\n",
" names = {\"Hoagie\", \"Hamilton\", \"Bacon\", \"Pizza\", \"Boots\", \"Shorts\", \"Tuna\"}\n",
" first_name = random.choice(list(names))\n",
" names.remove(first_name)\n",
" second_name = random.choice(list(names))\n",
@@ -496,13 +490,15 @@
"metadata": {},
"outputs": [],
"source": [
"corgi_story = kernel.create_semantic_function(prompt_template=sk_prompt,\n",
" function_name=\"CorgiStory\",\n",
" skill_name=\"CorgiSkill\",\n",
" description=\"Write a short story about two Corgis on an adventure\",\n",
" max_tokens=500,\n",
" temperature=0.5,\n",
" top_p=0.5)"
"corgi_story = kernel.create_semantic_function(\n",
" prompt_template=sk_prompt,\n",
" function_name=\"CorgiStory\",\n",
" skill_name=\"CorgiSkill\",\n",
" description=\"Write a short story about two Corgis on an adventure\",\n",
" max_tokens=500,\n",
" temperature=0.5,\n",
" top_p=0.5,\n",
")"
]
},
{
@@ -512,12 +508,8 @@
"metadata": {},
"outputs": [],
"source": [
"context_variables = sk.ContextVariables(variables={\n",
" \"min\": \"1\",\n",
" \"max\": \"5\",\n",
" \"language\": \"Spanish\"\n",
"})\n",
"context_variables['paragraph_count'] = generate_number.invoke(variables=context_variables).result"
"context_variables = sk.ContextVariables(variables={\"min\": \"1\", \"max\": \"5\", \"language\": \"Spanish\"})\n",
"context_variables[\"paragraph_count\"] = generate_number.invoke(variables=context_variables).result"
]
},
{
@@ -538,8 +530,11 @@
"metadata": {},
"outputs": [],
"source": [
"print(\"Generating a corgi story exactly {} paragraphs long in {} language: \".format(context_variables[\"paragraph_count\"],\n",
" context_variables[\"language\"]))\n",
"print(\n",
" \"Generating a corgi story exactly {} paragraphs long in {} language: \".format(\n",
" context_variables[\"paragraph_count\"], context_variables[\"language\"]\n",
" )\n",
")\n",
"print(\"=====================================================\")\n",
"print(story)"
]
python/notebooks/09-groundedness-checking.ipynb
+5 -2
View File
@@ -78,10 +78,11 @@
{
"cell_type": "code",
"execution_count": null,
"id": "7b22d324",
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.4.dev0"
"!python -m pip install semantic-kernel==0.4.5.dev0"
]
},
{
@@ -101,7 +102,9 @@
"# Configure AI service used by the kernel\n",
"if useAzureOpenAI:\n",
" deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()\n",
" azure_chat_service = AzureChatCompletion(deployment_name=\"turbo\", endpoint=endpoint, api_key=api_key) # set the deployment name to the value of your chat model\n",
" azure_chat_service = AzureChatCompletion(\n",
" deployment_name=\"turbo\", endpoint=endpoint, api_key=api_key\n",
" ) # set the deployment name to the value of your chat model\n",
" kernel.add_chat_service(\"chat_completion\", azure_chat_service)\n",
"else:\n",
" api_key, org_id = sk.openai_settings_from_dot_env()\n",
python/notebooks/10-multiple-results-per-prompt.ipynb
+385 -354
View File
@@ -1,356 +1,387 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "68e1c158",
"metadata": {},
"source": [
"# Multiple Results"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "fb81bacd",
"metadata": {},
"source": [
"In this notebook we show how you can in a single request, have the LLM model return multiple results per prompt. This is useful for running experiments where you want to evaluate the robustness of your prompt and the parameters of your config against a particular large language model."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a77bdf89",
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.4.dev0"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "508ad44f",
"metadata": {},
"outputs": [],
"source": [
"import semantic_kernel as sk\n",
"from semantic_kernel.connectors.ai.open_ai.request_settings.open_ai_request_settings import (\n",
" OpenAITextRequestSettings,\n",
" OpenAIChatRequestSettings,\n",
")\n",
"from semantic_kernel.connectors.ai.open_ai.request_settings.azure_chat_request_settings import (\n",
" AzureChatRequestSettings\n",
")\n",
"from semantic_kernel.connectors.ai.open_ai import AzureTextCompletion, AzureChatCompletion, OpenAITextCompletion, OpenAIChatCompletion\n",
"from semantic_kernel.connectors.ai.hugging_face import HuggingFaceTextCompletion"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d8ddffc1",
"metadata": {},
"source": [
"First, we will set up the text and chat services we will be submitting prompts to."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8f8dcbc6",
"metadata": {},
"outputs": [],
"source": [
"kernel = sk.Kernel()\n",
"\n",
"# Configure Azure LLM service\n",
"deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()\n",
"azure_text_service = AzureTextCompletion(deployment_name=\"text-davinci-003\", endpoint=endpoint, api_key=api_key) # set the deployment name to the value of your text model\n",
"azure_chat_service = AzureChatCompletion(deployment_name=\"gpt-35-turbo\", endpoint=endpoint, api_key=api_key) # set the deployment name to the value of your chat model\n",
"\n",
"# Configure OpenAI service\n",
"api_key, org_id = sk.openai_settings_from_dot_env()\n",
"oai_text_service = OpenAITextCompletion(ai_model_id=\"text-davinci-003\", api_key=api_key, org_id=org_id)\n",
"oai_chat_service = OpenAIChatCompletion(ai_model_id=\"gpt-3.5-turbo\", api_key=api_key, org_id=org_id)\n",
"\n",
"# Configure Hugging Face service\n",
"hf_text_service = HuggingFaceTextCompletion(ai_model_id=\"distilgpt2\", task=\"text-generation\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "50561d82",
"metadata": {},
"source": [
"Next, we'll set up the completion request settings for text completion services."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "628c843e",
"metadata": {},
"outputs": [],
"source": [
"oai_text_request_settings = OpenAITextRequestSettings(extension_data = { \"max_tokens\": 80, \"temperature\": 0.7, \"top_p\": 1, \"frequency_penalty\": 0.5, \"presence_penalty\": 0.5, \"number_of_responses\": 3})"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "857a9c89",
"metadata": {},
"source": [
"## Multiple Open AI Text Completions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e2979db8",
"metadata": {},
"outputs": [],
"source": [
"prompt = \"what is the purpose of a rubber duck?\"\n",
"results = await oai_text_service.complete_async(prompt=prompt, settings=oai_text_request_settings)\n",
"i = 1\n",
"for result in results:\n",
" print(f\"Result {i}: {result}\")\n",
" i += 1"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "4288d09f",
"metadata": {},
"source": [
"## Multiple Azure Open AI Text Completions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5319f14d",
"metadata": {},
"outputs": [],
"source": [
"prompt = \"provide me a list of possible meanings for the acronym 'ORLD'\"\n",
"results = await azure_text_service.complete_async(prompt=prompt, settings=oai_text_request_settings)\n",
"i = 1\n",
"for result in results:\n",
" print(f\"Result {i}: {result}\")\n",
" i += 1"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "eb548f9c",
"metadata": {},
"source": [
"## Multiple Hugging Face Text Completions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4a148709",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.connectors.ai.hugging_face.hf_request_settings import (\n",
" HuggingFaceRequestSettings,\n",
")\n",
"\n",
"hf_request_settings = HuggingFaceRequestSettings(extension_data = { \"max_new_tokens\": 80, \"temperature\": 0.7, \"top_p\": 1})"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9525e4f3",
"metadata": {},
"outputs": [],
"source": [
"prompt = \"The purpose of a rubber duck is\"\n",
"results = await hf_text_service.complete_async(prompt=prompt, request_settings=hf_request_settings)\n",
"print(\"\".join(results))"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "da632e12",
"metadata": {},
"source": [
"Here, we're setting up the settings for Chat completions."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e5f11e46",
"metadata": {},
"outputs": [],
"source": [
"oai_chat_request_settings = OpenAIChatRequestSettings(\n",
" max_tokens=80,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
" number_of_responses=3\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d6bf238e",
"metadata": {},
"source": [
"## Multiple OpenAI Chat Completions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dabc6a4c",
"metadata": {},
"outputs": [],
"source": [
"role = \"user\"\n",
"content = \"It's a beautiful day outside, birds are singing, flowers are blooming. On days like these, kids like you...\"\n",
"message = { \"role\":role, \"content\":content }\n",
"results = await oai_chat_service.complete_chat_async(messages=[message], settings=oai_chat_request_settings)\n",
"i = 0\n",
"for result in results:\n",
" print(f\"Result {i}: {result[0]}\")\n",
" i += 1"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "cdb8f740",
"metadata": {},
"source": [
"## Multiple Azure OpenAI Chat Completions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "66ba4767",
"metadata": {},
"outputs": [],
"source": [
"az_oai_request_settings = AzureChatRequestSettings(\n",
" max_tokens=80,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
" number_of_responses=3\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b74a64a9",
"metadata": {},
"outputs": [],
"source": [
"role = \"user\"\n",
"content = \"Tomorow is going to be a great day, I can feel it. I'm going to wake up early, go for a run, and then...\"\n",
"message = { \"role\":role, \"content\":content }\n",
"results = await azure_chat_service.complete_chat_async(messages=[message], settings=az_oai_request_settings)\n",
"i = 0\n",
"for result in results:\n",
" print(f\"Result {i}: {result[0]}\")\n",
" i += 1"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "98c8191d",
"metadata": {},
"source": [
"## Streaming Multiple Results\n",
"\n",
"Here is an example pattern if you want to stream your multiple results. Note that this is not supported for Hugging Face text completions at this time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "26a37702",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from IPython.display import clear_output\n",
"import time\n",
"\n",
"# Determine the clear command based on OS\n",
"clear_command = \"cls\" if os.name == \"nt\" else \"clear\"\n",
"\n",
"prompt = \"what is the purpose of a rubber duck?\"\n",
"stream = oai_text_service.complete_stream_async(prompt=prompt, settings=oai_text_request_settings)\n",
"number_of_responses = oai_text_request_settings.number_of_responses\n",
"texts = [\"\"] * number_of_responses\n",
"\n",
"last_clear_time = time.time()\n",
"clear_interval = 0.5 # seconds\n",
"\n",
"# Note: there are some quirks with displaying the output, which sometimes flashes and disappears.\n",
"# This could be influenced by a few factors specific to Jupyter notebooks and asynchronous processing.\n",
"# The following code attempts to buffer the results to avoid the output flashing on/off the screen.\n",
"\n",
"async for results in stream:\n",
" current_time = time.time()\n",
"\n",
" # Update texts with new results\n",
" for idx, result in enumerate(results):\n",
" if idx < number_of_responses:\n",
" texts[idx] += result\n",
"\n",
" # Clear and display output at intervals\n",
" if current_time - last_clear_time > clear_interval:\n",
" clear_output(wait=True)\n",
" for idx, text in enumerate(texts):\n",
" print(f\"Result {idx + 1}: {text}\")\n",
" last_clear_time = current_time\n",
"\n",
"print(\"----------------------------------------\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "68e1c158",
"metadata": {},
"source": [
"# Multiple Results"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "fb81bacd",
"metadata": {},
"source": [
"In this notebook we show how you can in a single request, have the LLM model return multiple results per prompt. This is useful for running experiments where you want to evaluate the robustness of your prompt and the parameters of your config against a particular large language model."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a77bdf89",
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.5.dev0"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3f4bfee4",
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = Service.AzureOpenAI"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "508ad44f",
"metadata": {},
"outputs": [],
"source": [
"import semantic_kernel as sk\n",
"if selectedService == Service.OpenAI or selectedService == Service.AzureOpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai.request_settings.open_ai_request_settings import (\n",
" OpenAITextRequestSettings,\n",
" OpenAIChatRequestSettings,\n",
" )\n",
" from semantic_kernel.connectors.ai.open_ai.request_settings.azure_chat_request_settings import AzureChatRequestSettings\n",
" from semantic_kernel.connectors.ai.open_ai import (\n",
" AzureTextCompletion,\n",
" AzureChatCompletion,\n",
" OpenAITextCompletion,\n",
" OpenAIChatCompletion,\n",
" )\n",
"if selectedService == Service.HuggingFace:\n",
" from semantic_kernel.connectors.ai.hugging_face import HuggingFaceTextCompletion"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d8ddffc1",
"metadata": {},
"source": [
"First, we will set up the text and chat services we will be submitting prompts to."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8f8dcbc6",
"metadata": {},
"outputs": [],
"source": [
"kernel = sk.Kernel()\n",
"\n",
"# Configure Azure LLM service\n",
"if selectedService == Service.AzureOpenAI:\n",
" deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()\n",
" azure_text_service = AzureTextCompletion(\n",
" deployment_name=\"gpt-35-turbo-instruct\", endpoint=endpoint, api_key=api_key\n",
" ) # set the deployment name to the value of your text model (e.g. gpt-35-turbo-instruct or text-davinci-003)\n",
" azure_chat_service = AzureChatCompletion(\n",
" deployment_name=\"gpt-35-turbo\", endpoint=endpoint, api_key=api_key\n",
" ) # set the deployment name to the value of your chat model\n",
"\n",
"# Configure OpenAI service\n",
"if selectedService == Service.OpenAI:\n",
" api_key, org_id = sk.openai_settings_from_dot_env()\n",
" oai_text_service = OpenAITextCompletion(ai_model_id=\"gpt-35-turbo-instruct\", api_key=api_key, org_id=org_id)\n",
" oai_chat_service = OpenAIChatCompletion(ai_model_id=\"gpt-3.5-turbo\", api_key=api_key, org_id=org_id)\n",
"\n",
"# Configure Hugging Face service\n",
"if selectedService == Service.HuggingFace:\n",
" hf_text_service = HuggingFaceTextCompletion(ai_model_id=\"distilgpt2\", task=\"text-generation\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "50561d82",
"metadata": {},
"source": [
"Next, we'll set up the completion request settings for text completion services."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "628c843e",
"metadata": {},
"outputs": [],
"source": [
"oai_text_request_settings = OpenAITextRequestSettings(\n",
" extension_data={\n",
" \"max_tokens\": 80,\n",
" \"temperature\": 0.7,\n",
" \"top_p\": 1,\n",
" \"frequency_penalty\": 0.5,\n",
" \"presence_penalty\": 0.5,\n",
" \"number_of_responses\": 3,\n",
" }\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "857a9c89",
"metadata": {},
"source": [
"## Multiple Open AI Text Completions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e2979db8",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.OpenAI:\n",
" prompt = \"what is the purpose of a rubber duck?\"\n",
" results = await oai_text_service.complete_async(prompt=prompt, settings=oai_text_request_settings)\n",
" i = 1\n",
" for result in results:\n",
" print(f\"Result {i}: {result}\")\n",
" i += 1"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "4288d09f",
"metadata": {},
"source": [
"## Multiple Azure Open AI Text Completions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5319f14d",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.AzureOpenAI:\n",
" prompt = \"provide me a list of possible meanings for the acronym 'ORLD'\"\n",
" results = await azure_text_service.complete_async(prompt=prompt, settings=oai_text_request_settings)\n",
" i = 1\n",
" for result in results:\n",
" print(f\"Result {i}: {result}\")\n",
" i += 1"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "eb548f9c",
"metadata": {},
"source": [
"## Multiple Hugging Face Text Completions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4a148709",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.HuggingFace:\n",
" from semantic_kernel.connectors.ai.hugging_face.hf_request_settings import (\n",
" HuggingFaceRequestSettings,\n",
" )\n",
"\n",
" hf_request_settings = HuggingFaceRequestSettings(extension_data={\"max_new_tokens\": 80, \"temperature\": 0.7, \"top_p\": 1})"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9525e4f3",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.HuggingFace:\n",
" prompt = \"The purpose of a rubber duck is\"\n",
" results = await hf_text_service.complete_async(prompt=prompt, request_settings=hf_request_settings)\n",
" print(\"\".join(results))"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "da632e12",
"metadata": {},
"source": [
"Here, we're setting up the settings for Chat completions."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e5f11e46",
"metadata": {},
"outputs": [],
"source": [
"oai_chat_request_settings = OpenAIChatRequestSettings(\n",
" max_tokens=80, temperature=0.7, top_p=1, frequency_penalty=0.5, presence_penalty=0.5, number_of_responses=3\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d6bf238e",
"metadata": {},
"source": [
"## Multiple OpenAI Chat Completions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dabc6a4c",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.OpenAI:\n",
" role = \"user\"\n",
" content = \"It's a beautiful day outside, birds are singing, flowers are blooming. On days like these, kids like you...\"\n",
" message = {\"role\": role, \"content\": content}\n",
" results = await oai_chat_service.complete_chat_async(messages=[message], settings=oai_chat_request_settings)\n",
" i = 0\n",
" for result in results:\n",
" print(f\"Result {i}: {result[0]}\")\n",
" i += 1"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "cdb8f740",
"metadata": {},
"source": [
"## Multiple Azure OpenAI Chat Completions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "66ba4767",
"metadata": {},
"outputs": [],
"source": [
"az_oai_request_settings = AzureChatRequestSettings(\n",
" max_tokens=80, temperature=0.7, top_p=1, frequency_penalty=0.5, presence_penalty=0.5, number_of_responses=3\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b74a64a9",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.AzureOpenAI:\n",
" role = \"user\"\n",
" content = \"Tomorow is going to be a great day, I can feel it. I'm going to wake up early, go for a run, and then...\"\n",
" message = {\"role\": role, \"content\": content}\n",
" results = await azure_chat_service.complete_chat_async(messages=[message], settings=az_oai_request_settings)\n",
" i = 0\n",
" for result in results:\n",
" print(f\"Result {i}: {result[0]}\")\n",
" i += 1"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "98c8191d",
"metadata": {},
"source": [
"## Streaming Multiple Results\n",
"\n",
"Here is an example pattern if you want to stream your multiple results. Note that this is not supported for Hugging Face text completions at this time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "26a37702",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.OpenAI:\n",
" import os\n",
" from IPython.display import clear_output\n",
" import time\n",
"\n",
" # Determine the clear command based on OS\n",
" clear_command = \"cls\" if os.name == \"nt\" else \"clear\"\n",
"\n",
" prompt = \"what is the purpose of a rubber duck?\"\n",
" stream = oai_text_service.complete_stream_async(prompt=prompt, settings=oai_text_request_settings)\n",
" number_of_responses = oai_text_request_settings.number_of_responses\n",
" texts = [\"\"] * number_of_responses\n",
"\n",
" last_clear_time = time.time()\n",
" clear_interval = 0.5 # seconds\n",
"\n",
" # Note: there are some quirks with displaying the output, which sometimes flashes and disappears.\n",
" # This could be influenced by a few factors specific to Jupyter notebooks and asynchronous processing.\n",
" # The following code attempts to buffer the results to avoid the output flashing on/off the screen.\n",
"\n",
" async for results in stream:\n",
" current_time = time.time()\n",
"\n",
" # Update texts with new results\n",
" for idx, result in enumerate(results):\n",
" if idx < number_of_responses:\n",
" texts[idx] += result\n",
"\n",
" # Clear and display output at intervals\n",
" if current_time - last_clear_time > clear_interval:\n",
" clear_output(wait=True)\n",
" for idx, text in enumerate(texts):\n",
" print(f\"Result {idx + 1}: {text}\")\n",
" last_clear_time = current_time\n",
"\n",
" print(\"----------------------------------------\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
python/notebooks/11-streaming-completions.ipynb
+327 -303
View File
@@ -1,305 +1,329 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "68e1c158",
"metadata": {},
"source": [
"# Streaming Results"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a77bdf89",
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.4.dev0"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "508ad44f",
"metadata": {},
"outputs": [],
"source": [
"import semantic_kernel as sk\n",
"from semantic_kernel.connectors.ai.open_ai.request_settings.open_ai_request_settings import (\n",
" OpenAITextRequestSettings,\n",
" OpenAIChatRequestSettings,\n",
")\n",
"from semantic_kernel.connectors.ai.open_ai.request_settings.azure_chat_request_settings import (\n",
" AzureChatRequestSettings,\n",
")\n",
"from semantic_kernel.connectors.ai.open_ai import (\n",
" AzureTextCompletion,\n",
" AzureChatCompletion,\n",
" OpenAITextCompletion,\n",
" OpenAIChatCompletion,\n",
")\n",
"from semantic_kernel.connectors.ai.hugging_face import HuggingFaceTextCompletion"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d8ddffc1",
"metadata": {},
"source": [
"First, we will set up the text and chat services we will be submitting prompts to."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8f8dcbc6",
"metadata": {},
"outputs": [],
"source": [
"kernel = sk.Kernel()\n",
"\n",
"# Configure Azure LLM service\n",
"deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()\n",
"azure_text_service = AzureTextCompletion(\n",
" deployment_name=\"text\", endpoint=endpoint, api_key=api_key\n",
") # set the deployment name to the value of your text model\n",
"azure_chat_service = AzureChatCompletion(\n",
" deployment_name=\"turbo\", endpoint=endpoint, api_key=api_key\n",
") # set the deployment name to the value of your chat model\n",
"\n",
"# Configure OpenAI service\n",
"api_key, org_id = sk.openai_settings_from_dot_env()\n",
"oai_text_service = OpenAITextCompletion(ai_model_id=\"text-davinci-003\", api_key=api_key, org_id=org_id)\n",
"oai_chat_service = OpenAIChatCompletion(ai_model_id=\"gpt-3.5-turbo\", api_key=api_key, org_id=org_id)\n",
"\n",
"# Configure Hugging Face service\n",
"hf_text_service = HuggingFaceTextCompletion(ai_model_id=\"distilgpt2\", task=\"text-generation\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "50561d82",
"metadata": {},
"source": [
"Next, we'll set up the completion request settings for text completion services."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "628c843e",
"metadata": {},
"outputs": [],
"source": [
"oai_request_settings = OpenAITextRequestSettings(\n",
" max_tokens=150,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "857a9c89",
"metadata": {},
"source": [
"## Streaming Open AI Text Completion"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e2979db8",
"metadata": {},
"outputs": [],
"source": [
"prompt = \"what is the purpose of a rubber duck?\"\n",
"stream = oai_text_service.complete_stream_async(prompt=prompt, settings=oai_request_settings)\n",
"async for text in stream:\n",
" print(text, end=\"\") # end = \"\" to avoid newlines"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "4288d09f",
"metadata": {},
"source": [
"## Streaming Azure Open AI Text Completion"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5319f14d",
"metadata": {},
"outputs": [],
"source": [
"prompt = \"provide me a list of possible meanings for the acronym 'ORLD'\"\n",
"stream = azure_text_service.complete_stream_async(prompt=prompt, settings=oai_request_settings)\n",
"async for text in stream:\n",
" print(text, end=\"\") # end = \"\" to avoid newlines"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "eb548f9c",
"metadata": {},
"source": [
"## Streaming Hugging Face Text Completion"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "be7b1c2e",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.connectors.ai.hugging_face.hf_request_settings import (\n",
" HuggingFaceRequestSettings,\n",
")\n",
"\n",
"hf_request_settings = HuggingFaceRequestSettings(\n",
" extension_data={\n",
" \"max_new_tokens\": 80,\n",
" \"top_p\": 1,\n",
" \"eos_token_id\": 11,\n",
" \"pad_token_id\": 0,\n",
" }\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9525e4f3",
"metadata": {},
"outputs": [],
"source": [
"prompt = \"The purpose of a rubber duck is\"\n",
"stream = hf_text_service.complete_stream_async(prompt=prompt, request_settings=hf_request_settings)\n",
"async for text in stream:\n",
" print(text, end=\"\") # end = \"\" to avoid newlines"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "da632e12",
"metadata": {},
"source": [
"Here, we're setting up the settings for Chat completions."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e5f11e46",
"metadata": {},
"outputs": [],
"source": [
"oai_chat_request_settings = OpenAIChatRequestSettings(\n",
" max_tokens=150,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d6bf238e",
"metadata": {},
"source": [
"## Streaming OpenAI Chat Completion"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dabc6a4c",
"metadata": {},
"outputs": [],
"source": [
"role = \"system\"\n",
"content = \"You are an AI assistant that helps people find information.\"\n",
"message = {\"role\": role, \"content\": content}\n",
"stream = oai_chat_service.complete_chat_stream_async(messages=[message], settings=oai_chat_request_settings)\n",
"async for text in stream:\n",
" print(text, end=\"\") # end = \"\" to avoid newlines"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "cdb8f740",
"metadata": {},
"source": [
"## Streaming Azure OpenAI Chat Completion"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "da1e9f59",
"metadata": {},
"outputs": [],
"source": [
"az_oai_chat_request_settings = AzureChatRequestSettings(\n",
" max_tokens=150,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b74a64a9",
"metadata": {},
"outputs": [],
"source": [
"role = \"system\"\n",
"content = \"You are an AI assistant that helps people find information.\"\n",
"message = {\"role\": role, \"content\": content}\n",
"stream = azure_chat_service.complete_chat_stream_async(messages=[message], settings=az_oai_chat_request_settings)\n",
"async for text in stream:\n",
" print(text, end=\"\") # end = \"\" to avoid newlines"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "68e1c158",
"metadata": {},
"source": [
"# Streaming Results\n",
"Here is an example pattern if you want to stream your multiple results. Note that this is not supported for Hugging Face text completions at this time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a77bdf89",
"metadata": {},
"outputs": [],
"source": [
"!python -m pip install semantic-kernel==0.4.5.dev0"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = Service.AzureOpenAI"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "508ad44f",
"metadata": {},
"outputs": [],
"source": [
"import semantic_kernel as sk\n",
"if selectedService == Service.OpenAI or selectedService == Service.AzureOpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai.request_settings.open_ai_request_settings import (\n",
" OpenAITextRequestSettings,\n",
" OpenAIChatRequestSettings,\n",
" )\n",
" from semantic_kernel.connectors.ai.open_ai.request_settings.azure_chat_request_settings import (\n",
" AzureChatRequestSettings,\n",
" )\n",
" from semantic_kernel.connectors.ai.open_ai import (\n",
" AzureTextCompletion,\n",
" AzureChatCompletion,\n",
" OpenAITextCompletion,\n",
" OpenAIChatCompletion,\n",
" )\n",
"if selectedService == Service.HuggingFace:\n",
" from semantic_kernel.connectors.ai.hugging_face import HuggingFaceTextCompletion"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d8ddffc1",
"metadata": {},
"source": [
"First, we will set up the text and chat services we will be submitting prompts to."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8f8dcbc6",
"metadata": {},
"outputs": [],
"source": [
"kernel = sk.Kernel()\n",
"\n",
"# Configure Azure LLM service\n",
"if selectedService == Service.AzureOpenAI:\n",
" deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()\n",
" azure_text_service = AzureTextCompletion(\n",
" deployment_name=\"gpt-35-turbo-instruct\", endpoint=endpoint, api_key=api_key\n",
" ) # set the deployment name to the value of your text model (e.g. gpt-35-turbo-instruct or text-davinci-003)\n",
" azure_chat_service = AzureChatCompletion(\n",
" deployment_name=\"gpt-35-turbo\", endpoint=endpoint, api_key=api_key\n",
" ) # set the deployment name to the value of your chat model\n",
"\n",
"# Configure OpenAI service\n",
"if selectedService == Service.OpenAI:\n",
" api_key, org_id = sk.openai_settings_from_dot_env()\n",
" oai_text_service = OpenAITextCompletion(ai_model_id=\"gpt-35-turbo-instruct\", api_key=api_key, org_id=org_id)\n",
" oai_chat_service = OpenAIChatCompletion(ai_model_id=\"gpt-3.5-turbo\", api_key=api_key, org_id=org_id)\n",
"\n",
"# Configure Hugging Face service\n",
"if selectedService == Service.HuggingFace:\n",
" hf_text_service = HuggingFaceTextCompletion(ai_model_id=\"distilgpt2\", task=\"text-generation\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "50561d82",
"metadata": {},
"source": [
"Next, we'll set up the completion request settings for text completion services."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "628c843e",
"metadata": {},
"outputs": [],
"source": [
"oai_request_settings = OpenAITextRequestSettings(\n",
" max_tokens=150,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "857a9c89",
"metadata": {},
"source": [
"## Streaming Open AI Text Completion"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e2979db8",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.OpenAI:\n",
" prompt = \"what is the purpose of a rubber duck?\"\n",
" stream = oai_text_service.complete_stream_async(prompt=prompt, settings=oai_request_settings)\n",
" async for text in stream:\n",
" print(text, end=\"\") # end = \"\" to avoid newlines"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "4288d09f",
"metadata": {},
"source": [
"## Streaming Azure Open AI Text Completion"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5319f14d",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.AzureOpenAI:\n",
" prompt = \"provide me a list of possible meanings for the acronym 'ORLD'\"\n",
" stream = azure_text_service.complete_stream_async(prompt=prompt, settings=oai_request_settings)\n",
" async for text in stream:\n",
" print(text, end=\"\") # end = \"\" to avoid newlines"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "eb548f9c",
"metadata": {},
"source": [
"## Streaming Hugging Face Text Completion"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "be7b1c2e",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.HuggingFace:\n",
" from semantic_kernel.connectors.ai.hugging_face.hf_request_settings import (\n",
" HuggingFaceRequestSettings,\n",
" )\n",
"\n",
" hf_request_settings = HuggingFaceRequestSettings(\n",
" extension_data={\n",
" \"max_new_tokens\": 80,\n",
" \"top_p\": 1,\n",
" \"eos_token_id\": 11,\n",
" \"pad_token_id\": 0,\n",
" }\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9525e4f3",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.HuggingFace:\n",
" prompt = \"The purpose of a rubber duck is\"\n",
" stream = hf_text_service.complete_stream_async(prompt=prompt, request_settings=hf_request_settings)\n",
" async for text in stream:\n",
" print(text, end=\"\") # end = \"\" to avoid newlines"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "da632e12",
"metadata": {},
"source": [
"Here, we're setting up the settings for Chat completions."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e5f11e46",
"metadata": {},
"outputs": [],
"source": [
"oai_chat_request_settings = OpenAIChatRequestSettings(\n",
" max_tokens=150,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d6bf238e",
"metadata": {},
"source": [
"## Streaming OpenAI Chat Completion"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dabc6a4c",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.OpenAI:\n",
" role = \"system\"\n",
" content = \"You are an AI assistant that helps people find information.\"\n",
" message = {\"role\": role, \"content\": content}\n",
" stream = oai_chat_service.complete_chat_stream_async(messages=[message], settings=oai_chat_request_settings)\n",
" async for text in stream:\n",
" print(text, end=\"\") # end = \"\" to avoid newlines"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "cdb8f740",
"metadata": {},
"source": [
"## Streaming Azure OpenAI Chat Completion"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "da1e9f59",
"metadata": {},
"outputs": [],
"source": [
"az_oai_chat_request_settings = AzureChatRequestSettings(\n",
" max_tokens=150,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b74a64a9",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.AzureOpenAI:\n",
" role = \"system\"\n",
" content = \"You are an AI assistant that helps people find information.\"\n",
" message = {\"role\": role, \"content\": content}\n",
" stream = azure_chat_service.complete_chat_stream_async(messages=[message], settings=az_oai_chat_request_settings)\n",
" async for text in stream:\n",
" print(text, end=\"\") # end = \"\" to avoid newlines"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
python/notebooks/services.py
+18
View File
@@ -0,0 +1,18 @@
"""
This module defines an enumeration representing different services.
"""
from enum import Enum
class Service(Enum):
"""
Attributes:
OpenAI (str): Represents the OpenAI service.
AzureOpenAI (str): Represents the Azure OpenAI service.
HuggingFace (str): Represents the HuggingFace service.
"""
OpenAI = "openai"
AzureOpenAI = "azureopenai"
HuggingFace = "huggingface"
python/poetry.lock
Generated
+3 -70
View File
@@ -946,73 +946,6 @@ tomli = {version = "*", optional = true, markers = "python_full_version <= \"3.1
[package.extras]
toml = ["tomli"]
[[package]]
name = "coverage"
version = "7.3.3"
description = "Code coverage measurement for Python"
optional = false
python-versions = ">=3.8"
files = [
{file = "coverage-7.3.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:d874434e0cb7b90f7af2b6e3309b0733cde8ec1476eb47db148ed7deeb2a9494"},
{file = "coverage-7.3.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:ee6621dccce8af666b8c4651f9f43467bfbf409607c604b840b78f4ff3619aeb"},
{file = "coverage-7.3.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1367aa411afb4431ab58fd7ee102adb2665894d047c490649e86219327183134"},
{file = "coverage-7.3.3-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:1f0f8f0c497eb9c9f18f21de0750c8d8b4b9c7000b43996a094290b59d0e7523"},
{file = "coverage-7.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:db0338c4b0951d93d547e0ff8d8ea340fecf5885f5b00b23be5aa99549e14cfd"},
{file = "coverage-7.3.3-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:d31650d313bd90d027f4be7663dfa2241079edd780b56ac416b56eebe0a21aab"},
{file = "coverage-7.3.3-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:9437a4074b43c177c92c96d051957592afd85ba00d3e92002c8ef45ee75df438"},
{file = "coverage-7.3.3-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:9e17d9cb06c13b4f2ef570355fa45797d10f19ca71395910b249e3f77942a837"},
{file = "coverage-7.3.3-cp310-cp310-win32.whl", hash = "sha256:eee5e741b43ea1b49d98ab6e40f7e299e97715af2488d1c77a90de4a663a86e2"},
{file = "coverage-7.3.3-cp310-cp310-win_amd64.whl", hash = "sha256:593efa42160c15c59ee9b66c5f27a453ed3968718e6e58431cdfb2d50d5ad284"},
{file = "coverage-7.3.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:8c944cf1775235c0857829c275c777a2c3e33032e544bcef614036f337ac37bb"},
{file = "coverage-7.3.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:eda7f6e92358ac9e1717ce1f0377ed2b9320cea070906ece4e5c11d172a45a39"},
{file = "coverage-7.3.3-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3c854c1d2c7d3e47f7120b560d1a30c1ca221e207439608d27bc4d08fd4aeae8"},
{file = "coverage-7.3.3-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:222b038f08a7ebed1e4e78ccf3c09a1ca4ac3da16de983e66520973443b546bc"},
{file = "coverage-7.3.3-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ff4800783d85bff132f2cc7d007426ec698cdce08c3062c8d501ad3f4ea3d16c"},
{file = "coverage-7.3.3-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:fc200cec654311ca2c3f5ab3ce2220521b3d4732f68e1b1e79bef8fcfc1f2b97"},
{file = "coverage-7.3.3-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:307aecb65bb77cbfebf2eb6e12009e9034d050c6c69d8a5f3f737b329f4f15fb"},
{file = "coverage-7.3.3-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:ffb0eacbadb705c0a6969b0adf468f126b064f3362411df95f6d4f31c40d31c1"},
{file = "coverage-7.3.3-cp311-cp311-win32.whl", hash = "sha256:79c32f875fd7c0ed8d642b221cf81feba98183d2ff14d1f37a1bbce6b0347d9f"},
{file = "coverage-7.3.3-cp311-cp311-win_amd64.whl", hash = "sha256:243576944f7c1a1205e5cd658533a50eba662c74f9be4c050d51c69bd4532936"},
{file = "coverage-7.3.3-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:a2ac4245f18057dfec3b0074c4eb366953bca6787f1ec397c004c78176a23d56"},
{file = "coverage-7.3.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f9191be7af41f0b54324ded600e8ddbcabea23e1e8ba419d9a53b241dece821d"},
{file = "coverage-7.3.3-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:31c0b1b8b5a4aebf8fcd227237fc4263aa7fa0ddcd4d288d42f50eff18b0bac4"},
{file = "coverage-7.3.3-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ee453085279df1bac0996bc97004771a4a052b1f1e23f6101213e3796ff3cb85"},
{file = "coverage-7.3.3-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1191270b06ecd68b1d00897b2daddb98e1719f63750969614ceb3438228c088e"},
{file = "coverage-7.3.3-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:007a7e49831cfe387473e92e9ff07377f6121120669ddc39674e7244350a6a29"},
{file = "coverage-7.3.3-cp312-cp312-musllinux_1_1_i686.whl", hash = "sha256:af75cf83c2d57717a8493ed2246d34b1f3398cb8a92b10fd7a1858cad8e78f59"},
{file = "coverage-7.3.3-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:811ca7373da32f1ccee2927dc27dc523462fd30674a80102f86c6753d6681bc6"},
{file = "coverage-7.3.3-cp312-cp312-win32.whl", hash = "sha256:733537a182b5d62184f2a72796eb6901299898231a8e4f84c858c68684b25a70"},
{file = "coverage-7.3.3-cp312-cp312-win_amd64.whl", hash = "sha256:e995efb191f04b01ced307dbd7407ebf6e6dc209b528d75583277b10fd1800ee"},
{file = "coverage-7.3.3-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:fbd8a5fe6c893de21a3c6835071ec116d79334fbdf641743332e442a3466f7ea"},
{file = "coverage-7.3.3-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:50c472c1916540f8b2deef10cdc736cd2b3d1464d3945e4da0333862270dcb15"},
{file = "coverage-7.3.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2e9223a18f51d00d3ce239c39fc41410489ec7a248a84fab443fbb39c943616c"},
{file = "coverage-7.3.3-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f501e36ac428c1b334c41e196ff6bd550c0353c7314716e80055b1f0a32ba394"},
{file = "coverage-7.3.3-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:475de8213ed95a6b6283056d180b2442eee38d5948d735cd3d3b52b86dd65b92"},
{file = "coverage-7.3.3-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:afdcc10c01d0db217fc0a64f58c7edd635b8f27787fea0a3054b856a6dff8717"},
{file = "coverage-7.3.3-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:fff0b2f249ac642fd735f009b8363c2b46cf406d3caec00e4deeb79b5ff39b40"},
{file = "coverage-7.3.3-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:a1f76cfc122c9e0f62dbe0460ec9cc7696fc9a0293931a33b8870f78cf83a327"},
{file = "coverage-7.3.3-cp38-cp38-win32.whl", hash = "sha256:757453848c18d7ab5d5b5f1827293d580f156f1c2c8cef45bfc21f37d8681069"},
{file = "coverage-7.3.3-cp38-cp38-win_amd64.whl", hash = "sha256:ad2453b852a1316c8a103c9c970db8fbc262f4f6b930aa6c606df9b2766eee06"},
{file = "coverage-7.3.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:3b15e03b8ee6a908db48eccf4e4e42397f146ab1e91c6324da44197a45cb9132"},
{file = "coverage-7.3.3-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:89400aa1752e09f666cc48708eaa171eef0ebe3d5f74044b614729231763ae69"},
{file = "coverage-7.3.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c59a3e59fb95e6d72e71dc915e6d7fa568863fad0a80b33bc7b82d6e9f844973"},
{file = "coverage-7.3.3-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:9ede881c7618f9cf93e2df0421ee127afdfd267d1b5d0c59bcea771cf160ea4a"},
{file = "coverage-7.3.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f3bfd2c2f0e5384276e12b14882bf2c7621f97c35320c3e7132c156ce18436a1"},
{file = "coverage-7.3.3-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:7f3bad1a9313401ff2964e411ab7d57fb700a2d5478b727e13f156c8f89774a0"},
{file = "coverage-7.3.3-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:65d716b736f16e250435473c5ca01285d73c29f20097decdbb12571d5dfb2c94"},
{file = "coverage-7.3.3-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:a702e66483b1fe602717020a0e90506e759c84a71dbc1616dd55d29d86a9b91f"},
{file = "coverage-7.3.3-cp39-cp39-win32.whl", hash = "sha256:7fbf3f5756e7955174a31fb579307d69ffca91ad163467ed123858ce0f3fd4aa"},
{file = "coverage-7.3.3-cp39-cp39-win_amd64.whl", hash = "sha256:cad9afc1644b979211989ec3ff7d82110b2ed52995c2f7263e7841c846a75348"},
{file = "coverage-7.3.3-pp38.pp39.pp310-none-any.whl", hash = "sha256:d299d379b676812e142fb57662a8d0d810b859421412b4d7af996154c00c31bb"},
{file = "coverage-7.3.3.tar.gz", hash = "sha256:df04c64e58df96b4427db8d0559e95e2df3138c9916c96f9f6a4dd220db2fdb7"},
]
[package.dependencies]
tomli = {version = "*", optional = true, markers = "python_full_version <= \"3.11.0a6\" and extra == \"toml\""}
[package.extras]
toml = ["tomli"]
[[package]]
name = "cryptography"
version = "41.0.7"
@@ -1975,13 +1908,13 @@ testing = ["Django", "attrs", "colorama", "docopt", "pytest (<7.0.0)"]
[[package]]
name = "jinja2"
version = "3.1.2"
version = "3.1.3"
description = "A very fast and expressive template engine."
optional = false
python-versions = ">=3.7"
files = [
{file = "Jinja2-3.1.2-py3-none-any.whl", hash = "sha256:6088930bfe239f0e6710546ab9c19c9ef35e29792895fed6e6e31a023a182a61"},
{file = "Jinja2-3.1.2.tar.gz", hash = "sha256:31351a702a408a9e7595a8fc6150fc3f43bb6bf7e319770cbc0db9df9437e852"},
{file = "Jinja2-3.1.3-py3-none-any.whl", hash = "sha256:7d6d50dd97d52cbc355597bd845fabfbac3f551e1f99619e39a35ce8c370b5fa"},
{file = "Jinja2-3.1.3.tar.gz", hash = "sha256:ac8bd6544d4bb2c9792bf3a159e80bba8fda7f07e81bc3aed565432d5925ba90"},
]
[package.dependencies]
python/pyproject.toml
+1 -7
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "semantic-kernel"
version = "0.4.4.dev"
version = "0.4.5.dev"
description = "Semantic Kernel Python SDK"
authors = ["Microsoft <[email protected]>"]
readme = "pip/README.md"
@@ -72,12 +72,6 @@ azure-identity = "^1.13.0"
usearch = "1.1.1"
pyarrow = ">=12.0.1,<15.0.0"
[tool.pytest.ini_options]
addopts = "--cov=semantic_kernel --cov-report term-missing"
testpaths = [
"tests"
]
[tool.isort]
profile = "ruff"
python/samples/kernel-syntax-examples/azure_chat_gpt_with_data_api.py
+16 -18
View File
@@ -4,6 +4,12 @@ import asyncio
import semantic_kernel as sk
import semantic_kernel.connectors.ai.open_ai as sk_oai
from semantic_kernel.connectors.ai.open_ai.request_settings.azure_chat_request_settings import (
AzureAISearchDataSources,
AzureChatRequestSettings,
AzureDataSources,
ExtraBody,
)
kernel = sk.Kernel()
@@ -27,16 +33,12 @@ azure_ai_search_settings["fieldsMapping"] = {
"filepathField": "source_file",
}
# Configure the Azure AI Search index as a data source.
azure_aisearch_datasource = sk_oai.OpenAIChatPromptTemplateWithDataConfig.AzureAISearchDataSource(
parameters=sk_oai.OpenAIChatPromptTemplateWithDataConfig.AzureAISearchDataSourceParameters(
**azure_ai_search_settings
)
)
azure_chat_with_data_settings = sk_oai.OpenAIChatPromptTemplateWithDataConfig.AzureChatWithDataSettings(
dataSources=[azure_aisearch_datasource]
)
# Create the data source settings
az_source = AzureAISearchDataSources(**azure_ai_search_settings)
az_data = AzureDataSources(type="AzureCognitiveSearch", parameters=az_source)
extra = ExtraBody(dataSources=[az_data])
req_settings = AzureChatRequestSettings(extra_body=extra)
prompt_config = sk.PromptTemplateConfig(completion=req_settings)
# When using data, set use_extensions=True and use the 2023-12-01-preview API version.
chat_service = sk_oai.AzureChatCompletion(
@@ -48,13 +50,6 @@ chat_service = sk_oai.AzureChatCompletion(
)
kernel.add_chat_service("chat-gpt", chat_service)
prompt_config = sk_oai.OpenAIChatPromptTemplateWithDataConfig.from_completion_parameters(
max_tokens=2000,
temperature=0.7,
top_p=0.8,
data_source_settings=azure_chat_with_data_settings,
)
prompt_template = sk.ChatPromptTemplate("{{$user_input}}", kernel.prompt_template_engine, prompt_config)
prompt_template.add_user_message("Hi there, who are you?")
@@ -62,6 +57,7 @@ prompt_template.add_assistant_message("I am an AI assistant here to answer your
function_config = sk.SemanticFunctionConfig(prompt_config, prompt_template)
chat_function = kernel.register_semantic_function("ChatBot", "Chat", function_config)
context = kernel.create_new_context()
async def chat() -> bool:
@@ -85,11 +81,13 @@ async def chat() -> bool:
# answer = await kernel.run_async(chat_function, input_vars=context_vars)
# print(f"Assistant:> {answer}")
answer = kernel.run_stream_async(chat_function, input_vars=context_vars)
answer = kernel.run_stream_async(chat_function, input_vars=context_vars, input_context=context)
print("Assistant:> ", end="")
async for message in answer:
print(message, end="")
print("\n")
# The tool message containing cited sources is available in the context
print(f"Tool:> {context.objects.get('tool_message')}")
return True
python/samples/kernel-syntax-examples/azure_chat_gpt_with_data_api_cross_lingual.py
+16 -20
View File
@@ -4,24 +4,29 @@ import asyncio
import semantic_kernel as sk
import semantic_kernel.connectors.ai.open_ai as sk_oai
from semantic_kernel.connectors.ai.open_ai.request_settings.azure_chat_request_settings import (
AzureAISearchDataSources,
AzureChatRequestSettings,
AzureDataSources,
ExtraBody,
)
kernel = sk.Kernel()
# Load Azure OpenAI Settings
deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()
# Load Azure OpenAI with data settings
azure_aisearch_datasource = sk_oai.OpenAIChatPromptTemplateWithDataConfig.AzureAISearchDataSource(
parameters=sk_oai.OpenAIChatPromptTemplateWithDataConfig.AzureAISearchDataSourceParameters(
**sk.azure_aisearch_settings_from_dot_env_as_dict()
)
)
# Load Azure AI Search settings
azure_ai_search_settings = sk.azure_aisearch_settings_from_dot_env_as_dict()
# Set index language
azure_aisearch_datasource.parameters.indexLanguage = "en"
azure_ai_search_settings["indexLanguage"] = "en"
azure_chat_with_data_settings = sk_oai.OpenAIChatPromptTemplateWithDataConfig.AzureChatWithDataSettings(
dataSources=[azure_aisearch_datasource]
)
# Create the data source settings
az_source = AzureAISearchDataSources(**azure_ai_search_settings)
az_data = AzureDataSources(type="AzureCognitiveSearch", parameters=az_source)
extra = ExtraBody(data_sources=[az_data], input_language="fr", output_language="de")
req_settings = AzureChatRequestSettings(extra_body=extra)
prompt_config = sk.PromptTemplateConfig(completion=req_settings)
# For example, AI Search index may contain the following document:
@@ -31,7 +36,7 @@ azure_chat_with_data_settings = sk_oai.OpenAIChatPromptTemplateWithDataConfig.Az
chat_service = sk_oai.AzureChatCompletion(
base_url=f"{str(endpoint).rstrip('/')}/openai/deployments/{deployment}/extensions",
use_extensions=True,
deployment_name=deployment,
api_key=api_key,
endpoint=endpoint,
@@ -42,15 +47,6 @@ kernel.add_chat_service(
chat_service,
)
prompt_config = sk_oai.OpenAIChatPromptTemplateWithDataConfig.from_completion_parameters(
max_tokens=2000,
temperature=0.7,
top_p=0.8,
inputLanguage="fr",
outputLanguage="de",
data_source_settings=azure_chat_with_data_settings,
)
prompt_template = sk.ChatPromptTemplate("{{$user_input}}", kernel.prompt_template_engine, prompt_config)
prompt_template.add_user_message("Bonjour!")
python/samples/kernel-syntax-examples/azure_chat_gpt_with_data_api_function_calling.py
+17 -20
View File
@@ -6,6 +6,12 @@ from typing import Tuple
import semantic_kernel as sk
import semantic_kernel.connectors.ai.open_ai as sk_oai
from semantic_kernel.connectors.ai.open_ai.request_settings.azure_chat_request_settings import (
AzureAISearchDataSources,
AzureChatRequestSettings,
AzureDataSources,
ExtraBody,
)
from semantic_kernel.connectors.ai.open_ai.semantic_functions.open_ai_chat_prompt_template import (
OpenAIChatPromptTemplate,
)
@@ -20,16 +26,12 @@ kernel = sk.Kernel()
# Load Azure OpenAI Settings
deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()
# Load Azure OpenAI with data settings
azure_aisearch_datasource = sk_oai.OpenAIChatPromptTemplateWithDataConfig.AzureAISearchDataSource(
parameters=sk_oai.OpenAIChatPromptTemplateWithDataConfig.AzureAISearchDataSourceParameters(
**sk.azure_aisearch_settings_from_dot_env_as_dict()
)
)
azure_chat_with_data_settings = sk_oai.OpenAIChatPromptTemplateWithDataConfig.AzureChatWithDataSettings(
dataSources=[azure_aisearch_datasource]
)
# Create the data source settings
azure_ai_search_settings = sk.azure_aisearch_settings_from_dot_env_as_dict()
az_source = AzureAISearchDataSources(**azure_ai_search_settings)
az_data = AzureDataSources(type="AzureCognitiveSearch", parameters=az_source)
extra = ExtraBody(dataSources=[az_data])
req_settings = AzureChatRequestSettings(extra_body=extra)
# For example, AI Search index may contain the following document:
@@ -56,18 +58,13 @@ kernel.import_semantic_skill_from_directory(skills_directory, "FunSkill")
# the math skill is a core skill and has the function calling enabled.
kernel.import_skill(TimeSkill(), skill_name="time")
# enabling or disabling function calling is done by setting the function_call parameter for the completion.
# when the function_call parameter is set to "auto" the model will decide which function to use, if any.
# if you only want to use a specific function, set the name of that function in this parameter,
# enabling or disabling function calling is done by setting the tool_choice parameter for the completion.
# when the tool_choice parameter is set to "auto" the model will decide which function to use, if any.
# if you only want to use a specific tool, set the name of that tool in this parameter,
# the format for that is 'SkillName-FunctionName', (i.e. 'math-Add').
# if the model or api version do not support this you will get an error.
prompt_config = sk_oai.OpenAIChatPromptTemplateWithDataConfig.from_completion_parameters(
max_tokens=2000,
temperature=0.7,
top_p=0.8,
function_call="auto",
data_source_settings=azure_chat_with_data_settings,
)
req_settings.tool_choice = "auto"
prompt_config = sk.PromptTemplateConfig(completion=req_settings)
prompt_template = OpenAIChatPromptTemplate("{{$user_input}}", kernel.prompt_template_engine, prompt_config)
prompt_template.add_user_message("Hi there, who are you?")
prompt_template.add_assistant_message("I am an AI assistant here to answer your questions.")
python/samples/kernel-syntax-examples/azure_chat_gpt_with_data_api_vector_search.py
+17 -18
View File
@@ -4,6 +4,12 @@ import asyncio
import semantic_kernel as sk
import semantic_kernel.connectors.ai.open_ai as sk_oai
from semantic_kernel.connectors.ai.open_ai.request_settings.azure_chat_request_settings import (
AzureAISearchDataSources,
AzureChatRequestSettings,
AzureDataSources,
ExtraBody,
)
kernel = sk.Kernel()
@@ -26,19 +32,15 @@ azure_ai_search_settings["fieldsMapping"] = {
"vectorFields": ["vector"],
}
# Add Ada embedding deployment name to the settings and use vector search.
azure_ai_search_settings["embeddingDeploymentName"] = "ada-002"
azure_ai_search_settings["embeddingDependency"] = {"type": "DeploymentName", "deploymentName": "ada-002"}
azure_ai_search_settings["queryType"] = "vector"
# Configure the Azure AI Search index as a data source.
azure_aisearch_datasource = sk_oai.OpenAIChatPromptTemplateWithDataConfig.AzureAISearchDataSource(
parameters=sk_oai.OpenAIChatPromptTemplateWithDataConfig.AzureAISearchDataSourceParameters(
**azure_ai_search_settings
)
)
azure_chat_with_data_settings = sk_oai.OpenAIChatPromptTemplateWithDataConfig.AzureChatWithDataSettings(
dataSources=[azure_aisearch_datasource]
)
# Create the data source settings
az_source = AzureAISearchDataSources(**azure_ai_search_settings)
az_data = AzureDataSources(type="AzureCognitiveSearch", parameters=az_source)
extra = ExtraBody(dataSources=[az_data])
req_settings = AzureChatRequestSettings(extra_body=extra)
prompt_config = sk.PromptTemplateConfig(completion=req_settings)
# When using data, set use_extensions=True and use the 2023-12-01-preview API version.
chat_service = sk_oai.AzureChatCompletion(
@@ -50,12 +52,6 @@ chat_service = sk_oai.AzureChatCompletion(
)
kernel.add_chat_service("chat-gpt", chat_service)
prompt_config = sk_oai.OpenAIChatPromptTemplateWithDataConfig.from_completion_parameters(
max_tokens=2000,
temperature=0.7,
top_p=0.8,
data_source_settings=azure_chat_with_data_settings,
)
prompt_template = sk.ChatPromptTemplate("{{$user_input}}", kernel.prompt_template_engine, prompt_config)
@@ -64,6 +60,7 @@ prompt_template.add_assistant_message("I am an AI assistant here to answer your
function_config = sk.SemanticFunctionConfig(prompt_config, prompt_template)
chat_function = kernel.register_semantic_function("ChatBot", "Chat", function_config)
context = kernel.create_new_context()
async def chat() -> bool:
@@ -87,11 +84,13 @@ async def chat() -> bool:
# answer = await kernel.run_async(chat_function, input_vars=context_vars)
# print(f"Assistant:> {answer}")
answer = kernel.run_stream_async(chat_function, input_vars=context_vars)
answer = kernel.run_stream_async(chat_function, input_vars=context_vars, input_context=context)
print("Assistant:> ", end="")
async for message in answer:
print(message, end="")
print("\n")
# The tool message containing cited sources is available in the context
print(f"Tool:> {context.objects.get('tool_message')}")
return True
python/samples/kernel-syntax-examples/self-critique_rag.py
+128
View File
@@ -0,0 +1,128 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from dotenv import dotenv_values
import semantic_kernel as sk
from semantic_kernel.connectors.ai.open_ai import (
AzureTextCompletion,
AzureTextEmbedding,
)
from semantic_kernel.connectors.memory.azure_cognitive_search import (
AzureCognitiveSearchMemoryStore,
)
from semantic_kernel.core_skills.text_memory_skill import TextMemorySkill
from semantic_kernel.orchestration.context_variables import ContextVariables
COLLECTION_NAME = "generic"
async def populate_memory(kernel: sk.Kernel) -> None:
# Add some documents to the ACS semantic memory
await kernel.memory.save_information_async(COLLECTION_NAME, id="info1", text="My name is Andrea")
await kernel.memory.save_information_async(COLLECTION_NAME, id="info2", text="I currently work as a tour guide")
await kernel.memory.save_information_async(
COLLECTION_NAME, id="info3", text="I've been living in Seattle since 2005"
)
await kernel.memory.save_information_async(
COLLECTION_NAME,
id="info4",
text="I visited France and Italy five times since 2015",
)
await kernel.memory.save_information_async(COLLECTION_NAME, id="info5", text="My family is from New York")
async def main() -> None:
kernel = sk.Kernel()
tms = TextMemorySkill()
kernel.import_skill(tms, "memory")
config = dotenv_values(".env")
AZURE_COGNITIVE_SEARCH_ENDPOINT = config["AZURE_COGNITIVE_SEARCH_ENDPOINT"]
AZURE_COGNITIVE_SEARCH_ADMIN_KEY = config["AZURE_COGNITIVE_SEARCH_ADMIN_KEY"]
AZURE_OPENAI_API_KEY = config["AZURE_OPENAI_API_KEY"]
AZURE_OPENAI_ENDPOINT = config["AZURE_OPENAI_ENDPOINT"]
vector_size = 1536
# Setting up OpenAI services for text completion and text embedding
kernel.add_text_completion_service(
"dv",
AzureTextCompletion(
deployment_name="gpt-35-turbo-instruct",
endpoint=AZURE_OPENAI_ENDPOINT,
api_key=AZURE_OPENAI_API_KEY,
),
)
kernel.add_text_embedding_generation_service(
"ada",
AzureTextEmbedding(
deployment_name="text-embedding-ada-002",
endpoint=AZURE_OPENAI_ENDPOINT,
api_key=AZURE_OPENAI_API_KEY,
),
)
connector = AzureCognitiveSearchMemoryStore(
vector_size, AZURE_COGNITIVE_SEARCH_ENDPOINT, AZURE_COGNITIVE_SEARCH_ADMIN_KEY
)
# Register the memory store with the kernel
kernel.register_memory_store(memory_store=connector)
print("Populating memory...")
await populate_memory(kernel)
sk_prompt_rag = """
Assistant can have a conversation with you about any topic.
It can give explicit instructions or say 'I don't know' if
it does not have an answer.
Here is some background information about the user that you should use to answer the question below:
{{ memory.recall $user_input }}
User: {{$user_input}}
Assistant: """.strip()
sk_prompt_rag_sc = """
You will get a question, background information to be used with that question and a answer that was given.
You have to answer Grounded or Ungrounded or Unclear.
Grounded if the answer is based on the background information and clearly answers the question.
Ungrounded if the answer could be true but is not based on the background information.
Unclear if the answer does not answer the question at all.
Question: {{$user_input}}
Background: {{ memory.recall $user_input }}
Answer: {{ $input }}
Remember, just answer Grounded or Ungrounded or Unclear: """.strip()
user_input = "Do I live in New York City?"
print(f"Question: {user_input}")
chat_func = kernel.create_semantic_function(sk_prompt_rag, max_tokens=1000, temperature=0.5)
self_critique_func = kernel.create_semantic_function(sk_prompt_rag_sc, max_tokens=4, temperature=0.0)
answer = await kernel.run_async(
chat_func,
input_vars=ContextVariables(variables={"user_input": user_input, "collection": COLLECTION_NAME, "limit": "2"}),
)
print(f"Answer: {str(answer).strip()}")
check = await kernel.run_async(self_critique_func, input_context=answer)
print(f"The answer was {str(check).strip()}")
print("-" * 50)
print(" Let's pretend the answer was wrong...")
answer.variables.variables["input"] = "Yes, you live in New York City."
print(f"Answer: {str(answer).strip()}")
check = await kernel.run_async(self_critique_func, input_context=answer)
print(f"The answer was {str(check).strip()}")
print("-" * 50)
print(" Let's pretend the answer is not related...")
answer.variables.variables["input"] = "Yes, the earth is not flat."
print(f"Answer: {str(answer).strip()}")
check = await kernel.run_async(self_critique_func, input_context=answer)
print(f"The answer was {str(check).strip()}")
await connector.close_async()
if __name__ == "__main__":
asyncio.run(main())
python/semantic_kernel/connectors/ai/ai_service_client_base.py
+16 -3
View File
@@ -1,8 +1,14 @@
# Copyright (c) Microsoft. All rights reserved.
import sys
from abc import ABC
from pydantic import constr
if sys.version_info >= (3, 9):
from typing import Annotated
else:
from typing_extensions import Annotated
from pydantic import StringConstraints
from semantic_kernel.connectors.ai.ai_request_settings import AIRequestSettings
from semantic_kernel.sk_pydantic import SKBaseModel
@@ -17,8 +23,15 @@ class AIServiceClientBase(SKBaseModel, ABC):
or can just be a string that is used to identify the service.
"""
ai_model_id: constr(strip_whitespace=True, min_length=1)
ai_model_id: Annotated[str, StringConstraints(strip_whitespace=True, min_length=1)]
def get_request_settings_class(self) -> "AIRequestSettings":
"""Create a request settings object."""
"""Get the request settings class."""
return AIRequestSettings # pragma: no cover
def instantiate_request_settings(self, **kwargs) -> "AIRequestSettings":
"""Create a request settings object.
All arguments are passed to the constructor of the request settings object.
"""
return self.get_request_settings_class()(**kwargs)
python/semantic_kernel/connectors/ai/google_palm/services/gp_chat_completion.py
+8 -2
View File
@@ -1,11 +1,17 @@
# Copyright (c) Microsoft. All rights reserved.
import logging
import sys
from typing import Any, List, Optional, Tuple, Union
if sys.version_info >= (3, 9):
from typing import Annotated
else:
from typing_extensions import Annotated
import google.generativeai as palm
from google.generativeai.types import ChatResponse
from pydantic import PrivateAttr, constr
from pydantic import PrivateAttr, StringConstraints
from semantic_kernel.connectors.ai.ai_exception import AIException
from semantic_kernel.connectors.ai.ai_request_settings import AIRequestSettings
@@ -25,7 +31,7 @@ logger: logging.Logger = logging.getLogger(__name__)
class GooglePalmChatCompletion(ChatCompletionClientBase, TextCompletionClientBase, AIServiceClientBase):
api_key: constr(strip_whitespace=True, min_length=1)
api_key: Annotated[str, StringConstraints(strip_whitespace=True, min_length=1)]
_message_history: Optional[ChatResponse] = PrivateAttr()
def __init__(
python/semantic_kernel/connectors/ai/google_palm/services/gp_text_completion.py
+7 -2
View File
@@ -1,10 +1,15 @@
# Copyright (c) Microsoft. All rights reserved.
import logging
import sys
from typing import Any, List, Optional, Union
if sys.version_info >= (3, 9):
from typing import Annotated
else:
from typing_extensions import Annotated
import google.generativeai as palm
from pydantic import constr
from pydantic import StringConstraints
from semantic_kernel.connectors.ai.ai_exception import AIException
from semantic_kernel.connectors.ai.ai_request_settings import AIRequestSettings
@@ -20,7 +25,7 @@ logger: logging.Logger = logging.getLogger(__name__)
class GooglePalmTextCompletion(TextCompletionClientBase, AIServiceClientBase):
api_key: constr(strip_whitespace=True, min_length=1)
api_key: Annotated[str, StringConstraints(strip_whitespace=True, min_length=1)]
def __init__(self, ai_model_id: str, api_key: str, log: Optional[Any] = None):
"""
python/semantic_kernel/connectors/ai/google_palm/services/gp_text_embedding.py
+7 -2
View File
@@ -1,11 +1,16 @@
# Copyright (c) Microsoft. All rights reserved.
import sys
from typing import List
if sys.version_info >= (3, 9):
from typing import Annotated
else:
from typing_extensions import Annotated
import google.generativeai as palm
from numpy import array, ndarray
from pydantic import constr
from pydantic import StringConstraints
from semantic_kernel.connectors.ai.ai_exception import AIException
from semantic_kernel.connectors.ai.ai_service_client_base import AIServiceClientBase
@@ -15,7 +20,7 @@ from semantic_kernel.connectors.ai.embeddings.embedding_generator_base import (
class GooglePalmTextEmbedding(EmbeddingGeneratorBase, AIServiceClientBase):
api_key: constr(strip_whitespace=True, min_length=1)
api_key: Annotated[str, StringConstraints(strip_whitespace=True, min_length=1)]
def __init__(self, ai_model_id: str, api_key: str) -> None:
"""
python/semantic_kernel/connectors/ai/ollama/__init__.py
View File
python/semantic_kernel/connectors/ai/ollama/ollama_request_settings.py
+24
View File
@@ -0,0 +1,24 @@
from typing import Any, Dict, List, Literal, Optional
from pydantic import Field
from semantic_kernel.connectors.ai.ai_request_settings import AIRequestSettings
class OllamaRequestSettings(AIRequestSettings):
ai_model_id: str = Field("", alias="model")
format: Optional[Literal["json"]] = None
options: Optional[Dict[str, Any]] = None
stream: bool = False
class OllamaTextRequestSettings(OllamaRequestSettings):
prompt: Optional[str] = None
context: Optional[str] = None
system: Optional[str] = None
template: Optional[str] = None
raw: bool = False
class OllamaChatRequestSettings(OllamaRequestSettings):
messages: Optional[List[Dict[str, str]]] = None
python/semantic_kernel/connectors/ai/ollama/services/ollama_chat_completion.py
+134
View File
@@ -0,0 +1,134 @@
# Copyright (c) Microsoft. All rights reserved.
import json
import logging
from typing import Dict, List, Optional, Union
import aiohttp
from pydantic import HttpUrl
from semantic_kernel.connectors.ai.ai_request_settings import AIRequestSettings
from semantic_kernel.connectors.ai.ai_service_client_base import AIServiceClientBase
from semantic_kernel.connectors.ai.chat_completion_client_base import ChatCompletionClientBase
from semantic_kernel.connectors.ai.ollama.ollama_request_settings import OllamaChatRequestSettings
from semantic_kernel.connectors.ai.ollama.utils import AsyncSession
from semantic_kernel.connectors.ai.text_completion_client_base import (
TextCompletionClientBase,
)
logger: logging.Logger = logging.getLogger(__name__)
class OllamaChatCompletion(TextCompletionClientBase, ChatCompletionClientBase, AIServiceClientBase):
"""
Initializes a new instance of the OllamaChatCompletion class.
Make sure to have the ollama service running either locally or remotely.
Arguments:
ai_model_id {str} -- Ollama model name, see https://ollama.ai/library
url {Optional[Union[str, HttpUrl]]} -- URL of the Ollama server, defaults to http://localhost:11434/api/chat
session {Optional[aiohttp.ClientSession]} -- Optional client session to use for requests.
"""
url: HttpUrl = "http://localhost:11434/api/chat"
session: Optional[aiohttp.ClientSession] = None
async def complete_chat_async(
self,
messages: List[Dict[str, str]],
request_settings: OllamaChatRequestSettings,
**kwargs,
) -> Union[str, List[str]]:
"""
This is the method that is called from the kernel to get a response from a chat-optimized LLM.
Arguments:
messages {List[ChatMessage]} -- A list of chat messages, that can be rendered into a
set of messages, from system, user, assistant and function.
settings {AIRequestSettings} -- Settings for the request.
logger {Logger} -- A logger to use for logging. (Deprecated)
Returns:
Union[str, List[str]] -- A string or list of strings representing the response(s) from the LLM.
"""
request_settings.messages = messages
request_settings.stream = False
async with AsyncSession(self.session) as session:
async with session.post(str(self.url), json=request_settings.prepare_settings_dict()) as response:
response.raise_for_status()
response_object = await response.json()
return response_object.get("message", {"content": None}).get("content", None)
async def complete_chat_stream_async(
self,
messages: List[Dict[str, str]],
settings: OllamaChatRequestSettings,
**kwargs,
):
"""
Streams a text completion using a Ollama model.
Note that this method does not support multiple responses.
Arguments:
prompt {str} -- Prompt to complete.
request_settings {OllamaChatRequestSettings} -- Request settings.
Yields:
str -- Completion result.
"""
settings.messages = messages
settings.stream = True
async with AsyncSession(self.session) as session:
async with session.post(str(self.url), json=settings.prepare_settings_dict()) as response:
response.raise_for_status()
async for line in response.content:
body = json.loads(line)
response_part = body.get("message", {"content": None}).get("content", None)
yield response_part
if body.get("done"):
break
async def complete_async(
self,
prompt: str,
request_settings: OllamaChatRequestSettings,
**kwargs,
) -> Union[str, List[str]]:
"""
This is the method that is called from the kernel to get a response from a text-optimized LLM.
Arguments:
prompt {str} -- The prompt to send to the LLM.
settings {AIRequestSettings} -- Settings for the request.
logger {Logger} -- A logger to use for logging (deprecated).
Returns:
Union[str, List[str]] -- A string or list of strings representing the response(s) from the LLM.
"""
return await self.complete_chat_async([{"role": "user", "content": prompt}], request_settings, **kwargs)
async def complete_stream_async(
self,
prompt: str,
request_settings: OllamaChatRequestSettings,
**kwargs,
):
"""
Streams a text completion using a Ollama model.
Note that this method does not support multiple responses.
Arguments:
prompt {str} -- Prompt to complete.
request_settings {OllamaChatRequestSettings} -- Request settings.
Yields:
str -- Completion result.
"""
response = self.complete_chat_stream_async([{"role": "user", "content": prompt}], request_settings, **kwargs)
async for line in response:
yield line
def get_request_settings_class(self) -> "AIRequestSettings":
"""Get the request settings class."""
return OllamaChatRequestSettings
python/semantic_kernel/connectors/ai/ollama/services/ollama_text_completion.py
+90
View File
@@ -0,0 +1,90 @@
# Copyright (c) Microsoft. All rights reserved.
import json
import logging
from typing import List, Optional, Union
import aiohttp
from pydantic import HttpUrl
from semantic_kernel.connectors.ai.ai_request_settings import AIRequestSettings
from semantic_kernel.connectors.ai.ai_service_client_base import AIServiceClientBase
from semantic_kernel.connectors.ai.ollama.ollama_request_settings import OllamaTextRequestSettings
from semantic_kernel.connectors.ai.ollama.utils import AsyncSession
from semantic_kernel.connectors.ai.text_completion_client_base import (
TextCompletionClientBase,
)
logger: logging.Logger = logging.getLogger(__name__)
class OllamaTextCompletion(TextCompletionClientBase, AIServiceClientBase):
"""
Initializes a new instance of the OllamaTextCompletion class.
Make sure to have the ollama service running either locally or remotely.
Arguments:
ai_model_id {str} -- Ollama model name, see https://ollama.ai/library
url {Optional[Union[str, HttpUrl]]} -- URL of the Ollama server, defaults to http://localhost:11434/api/generate
"""
url: HttpUrl = "http://localhost:11434/api/generate"
session: Optional[aiohttp.ClientSession] = None
async def complete_async(
self,
prompt: str,
request_settings: OllamaTextRequestSettings,
**kwargs,
) -> Union[str, List[str]]:
"""
This is the method that is called from the kernel to get a response from a text-optimized LLM.
Arguments:
prompt {str} -- The prompt to send to the LLM.
settings {AIRequestSettings} -- Settings for the request.
logger {Logger} -- A logger to use for logging (deprecated).
Returns:
Union[str, List[str]] -- A string or list of strings representing the response(s) from the LLM.
"""
request_settings.prompt = prompt
request_settings.stream = False
async with AsyncSession(self.session) as session:
async with session.post(self.url, json=request_settings.prepare_settings_dict()) as response:
response.raise_for_status()
return await response.text()
async def complete_stream_async(
self,
prompt: str,
request_settings: OllamaTextRequestSettings,
**kwargs,
):
"""
Streams a text completion using a Hugging Face model.
Note that this method does not support multiple responses.
Arguments:
prompt {str} -- Prompt to complete.
request_settings {HuggingFaceRequestSettings} -- Request settings.
Yields:
str -- Completion result.
"""
request_settings.prompt = prompt
request_settings.stream = True
async with AsyncSession(self.session) as session:
async with session.post(self.url, json=request_settings.prepare_settings_dict()) as response:
response.raise_for_status()
async for line in response.content:
body = json.loads(line)
response_part = body.get("response")
yield response_part
if body.get("done"):
break
def get_request_settings_class(self) -> "AIRequestSettings":
"""Get the request settings class."""
return OllamaTextRequestSettings
python/semantic_kernel/connectors/ai/ollama/services/ollama_text_embedding.py
+48
View File
@@ -0,0 +1,48 @@
# Copyright (c) Microsoft. All rights reserved.
import logging
from typing import List, Optional
import aiohttp
from numpy import array, ndarray
from pydantic import HttpUrl
from semantic_kernel.connectors.ai.ai_service_client_base import AIServiceClientBase
from semantic_kernel.connectors.ai.embeddings.embedding_generator_base import (
EmbeddingGeneratorBase,
)
from semantic_kernel.connectors.ai.ollama.utils import AsyncSession
logger: logging.Logger = logging.getLogger(__name__)
class OllamaTextEmbedding(EmbeddingGeneratorBase, AIServiceClientBase):
"""Ollama embeddings client.
Make sure to have the ollama service running either locally or remotely.
Arguments:
ai_model_id {str} -- Ollama model name, see https://ollama.ai/library
url {Optional[Union[str, HttpUrl]]} -- URL of the Ollama server, defaults to http://localhost:11434/api/embeddings
session {Optional[aiohttp.ClientSession]} -- Optional client session to use for requests.
"""
url: HttpUrl = "http://localhost:11434/api/embeddings"
session: Optional[aiohttp.ClientSession] = None
async def generate_embeddings_async(self, texts: List[str], **kwargs) -> ndarray:
"""
Generates embeddings for a list of texts.
Arguments:
texts {List[str]} -- Texts to generate embeddings for.
Returns:
ndarray -- Embeddings for the texts.
"""
async with AsyncSession(self.session) as session:
async with session.post(
self.url, json={"model": self.ai_model_id, "texts": texts, "options": kwargs}
) as response:
response.raise_for_status()
return array(await response.json())
python/semantic_kernel/connectors/ai/ollama/utils.py
+12
View File
@@ -0,0 +1,12 @@
import aiohttp
class AsyncSession:
def __init__(self, session: aiohttp.ClientSession = None):
self._session = session if session else aiohttp.ClientSession()
async def __aenter__(self):
return await self._session.__aenter__()
async def __aexit__(self, *args, **kwargs):
await self._session.close()
python/semantic_kernel/connectors/ai/open_ai/const.py
+4 -4
View File
@@ -1,7 +1,7 @@
# Copyright (c) Microsoft. All rights reserved.
from typing import final
from typing import Final
DEFAULT_AZURE_API_VERSION: final = "2023-05-15"
USER_AGENT: final = "User-Agent"
DEFAULT_CHAT_SYSTEM_PROMPT: final = "Assistant is a large language model."
DEFAULT_AZURE_API_VERSION: Final[str] = "2023-05-15"
USER_AGENT: Final[str] = "User-Agent"
DEFAULT_CHAT_SYSTEM_PROMPT: Final[str] = "Assistant is a large language model."
python/semantic_kernel/connectors/ai/open_ai/models/chat/azure_chat_with_data_response.py
+12 -4
View File
@@ -1,5 +1,6 @@
# Copyright (c) Microsoft. All rights reserved.
"""Azure OpenAI Chat With Data Streaming Response class."""
import json
from typing import Tuple
from openai import AsyncStream
@@ -34,10 +35,17 @@ class AzureChatWithDataStreamResponse:
if message.choices and len(message.choices) > 0:
delta = message.choices[0].delta
if delta and delta.model_extra and "context" in delta.model_extra:
for m in delta.model_extra["context"].get("messages", []):
if m["role"] == "tool":
self._tool_message = m.get("content", "")
break
if "messages" in delta.model_extra["context"]:
for m in delta.model_extra["context"]["messages"]:
if m.get("role") == "tool":
self._tool_message = m.get("content", "")
break
else:
self._tool_message = json.dumps(delta.model_extra["context"])
break
else:
# Ensure missing tool message doesn't cause empty assistant message
self._tool_message = "{}"
break
except StopIteration:
break
python/semantic_kernel/connectors/ai/open_ai/request_settings/azure_chat_request_settings.py
+19 -8
View File
@@ -1,7 +1,7 @@
import logging
from typing import Any, Dict, List, Literal, Optional, Union
from pydantic import Field
from pydantic import Field, SerializeAsAny
from pydantic.dataclasses import dataclass
from semantic_kernel.connectors.ai.open_ai.request_settings.open_ai_request_settings import (
@@ -13,13 +13,19 @@ logger = logging.getLogger(__name__)
@dataclass
class AzureCosmosDBAuthentication:
class ConnectionStringAuthentication:
type: Literal["ConnectionString"] = "ConnectionString"
connectionString: Optional[str] = None
@dataclass
class AzureCosmosDBEmbeddingDependencyType:
class ApiKeyAuthentication:
type: Literal["APIKey"] = "APIKey"
key: Optional[str] = None
@dataclass
class AzureEmbeddingDependency:
type: Literal["DeploymentName"] = "DeploymentName"
deploymentName: Optional[str] = None
@@ -38,14 +44,15 @@ class AzureDataSourceParameters:
embeddingEndpoint: Optional[str] = None
embeddingDeploymentName: Optional[str] = None
strictness: int = 3
embeddingDependency: Optional[AzureEmbeddingDependency] = None
@dataclass
class AzureCosmosDBDataSource(AzureDataSourceParameters):
authentication: Optional[AzureCosmosDBAuthentication] = None
authentication: Optional[ConnectionStringAuthentication] = None
databaseName: Optional[str] = None
containerName: Optional[str] = None
embeddingDependencyType: Optional[AzureCosmosDBEmbeddingDependencyType] = None
embeddingDependencyType: Optional[AzureEmbeddingDependency] = None
@dataclass
@@ -53,6 +60,7 @@ class AzureAISearchDataSources(AzureDataSourceParameters):
endpoint: Optional[str] = None
key: Optional[str] = None
queryType: Literal["simple", "semantic", "vector", "vectorSimpleHybrid", "vectorSemanticHybrid"] = "simple"
authentication: Optional[ApiKeyAuthentication] = None
@dataclass
@@ -60,14 +68,17 @@ class AzureDataSources:
"""Class to hold Azure AI data source parameters."""
type: Literal["AzureCognitiveSearch", "AzureCosmosDB"] = "AzureCognitiveSearch"
parameters: Optional[AzureDataSourceParameters] = None
parameters: Optional[SerializeAsAny[AzureDataSourceParameters]] = None
# @dataclass
class ExtraBody(SKBaseModel):
data_sources: Optional[List[AzureDataSources]] = Field(None, alias="dataSources")
# input_language: Optional[str] = Field(None, serialization_alias="inputLanguage")
# output_language: Optional[str] = Field(None, serialization_alias="outputLanguage")
input_language: Optional[str] = Field(None, serialization_alias="inputLanguage")
output_language: Optional[str] = Field(None, serialization_alias="outputLanguage")
def __getitem__(self, item):
return getattr(self, item)
class AzureChatRequestSettings(OpenAIChatRequestSettings):
python/semantic_kernel/connectors/ai/open_ai/request_settings/open_ai_request_settings.py
+3 -3
View File
@@ -42,9 +42,9 @@ class OpenAITextRequestSettings(OpenAIRequestSettings):
raise ValueError(
"When used with number_of_responses, best_of controls the number of candidate completions and n specifies how many to return, therefore best_of must be greater than number_of_responses." # noqa: E501
)
if self.extension_data.get("best_of") is not None and self.extension_data.get(
"best_of"
) < self.extension_data.get("number_of_responses"):
if self.extension_data.get("best_of") is not None and self.extension_data["best_of"] < self.extension_data.get(
"number_of_responses"
):
raise ValueError(
"When used with number_of_responses, best_of controls the number of candidate completions and n specifies how many to return, therefore best_of must be greater than number_of_responses." # noqa: E501
)
python/semantic_kernel/connectors/ai/open_ai/services/azure_chat_completion.py
-2
View File
@@ -21,8 +21,6 @@ from semantic_kernel.connectors.ai.ai_request_settings import AIRequestSettings
from semantic_kernel.connectors.ai.chat_completion_client_base import (
ChatCompletionClientBase,
)
# from semantic_kernel.connectors.ai.chat_request_settings import ChatRequestSettings
from semantic_kernel.connectors.ai.open_ai.const import DEFAULT_AZURE_API_VERSION
from semantic_kernel.connectors.ai.open_ai.models.chat.azure_chat_with_data_response import (
AzureChatWithDataStreamResponse,

Some files were not shown because too many files have changed in this diff Show More