Files
semantic-kernel/dotnet/samples/KernelSyntaxExamples/TestConfiguration.cs
T
Mustafa ZenginandSergeyMenshykh e4f9c750a0 .Net: API Manifest Plugins (#4961)
### Motivation and Context

This PR contains main implementation part of #4662 and adds it into
`Functions.OpenApi.Extensions` project. There were also couple of bug
fixes in the mentioned PR, those will be handled in subsequent PRs.

API Manifest can describe capabilities of an AI plugin for a chat-based
system among other things. For a full description of API Manifest, see
the [API Manifest
Specification](https://darrelmiller.github.io/api-manifest/draft-miller-api-manifest.html).

API Manifest refers to one or more OpenAPI documents as API dependencies
and it selects a subset of operations from those documents to describe
the capabilities of an AI plugin.

### Description

Semantic Kernel has a way of creating plugins from OpenAPI operations,
but it has a couple of limitations that can be addressed with API
Manifest:
1. It can only create functions from OpenAPI operations that are
described in a single OpenAPI document.
2. It cannot reference multiple OpenAPI documents in a single plugin.

Not being able to slice OpenAPI operations is a major limitation,
especially for large APIs such as Microsoft Graph, which has over 1000
operations.

Today, we can create a Semantic Kernel plugin from an OpenAPI file and
one OpenAPI file corresponds to one plugin. We don't have a way of
selecting a subset of operations.

```mermaid
graph TB

subgraph OpenAPI1
operation11
operation12
operation13
end

subgraph OpenAPI2
operation21
operation22
operation23
end

subgraph OpenAPIPlugin1
function11
function12
function13
end

subgraph OpenAPIPlugin2
function21
function22
function23
end

operation11 -.- function11
operation12 -.- function12
operation13 -.- function13

operation21 -.- function21
operation22 -.- function22
operation23 -.- function23

style OpenAPIPlugin1 fill:blue,color:white
style OpenAPIPlugin2 fill:blue,color:white

```

### Proposed Solution
After Semantic Kernel APIManifest support, we will have an APIManifest
file, which will be used as a filter on OpenAPI files and it can
reference more than one OpenAPI file. The filter can be used as a guide
to select only certain operations to be converted into functions.

Assume that we want to create a single plugin with `function11` from
`OpenAPI1` and `function21` and `function22` from `OpenAPI2`. We want to
preserve the one-to-one mapping between the APIManifest and the plugin,
e.g. between `ApiManifestX` and `PluginX`:

```mermaid
graph TB

subgraph OpenAPI1
operation11
operation12
operation13
end

subgraph OpenAPI2
operation21
operation22
operation23
end

subgraph APIManifestX
reference11
reference21
reference22
end

subgraph PluginX
function11
function21
function22
end

reference11 -.- operation11
reference21 -.- operation21
reference22 -.- operation22

function11 -.- reference11
function21 -.- reference21
function22 -.- reference22

style APIManifestX fill:green,color:white
style PluginX fill:blue,color:white

```

If we use existing OpenAPI to Plugin public methods after on-the-fly
slicing, we can only achieve the following because there is an
assumption that a single OpenAPI is converted into a single plugin,
which breaks APIManifest to Plugin one-to-one mapping. A single
`APIManifestX` file corresponds to two plugins, namely, `PluginX1` and
`PluginX2`:

```mermaid
graph TB

subgraph OpenAPI1
operation11
operation12
operation13
end

subgraph OpenAPI2
operation21
operation22
operation23
end

subgraph APIManifestX
reference11
reference21
reference22
end

subgraph OpenAPI1_Sliced
operation11_sliced[operation11]
end

subgraph OpenAPI2_Sliced
operation21_sliced[operation21]
operation22_sliced[operation22]
end

subgraph PluginX1
function11
end

subgraph PluginX2
function21
function22
end

reference11 -.- operation11
reference21 -.- operation21
reference22 -.- operation22

operation11 -.- operation11_sliced -.- function11
operation21 -.- operation21_sliced -.- function21
operation22 -.- operation22_sliced -.- function22

style APIManifestX fill:green,color:white
style PluginX1 fill:blue,color:white
style PluginX2 fill:blue,color:white

```

To avoid this, the implementation of regular OpenAPI plugin generation
and API Manifest plugin generation meets at the function generation
level, so reuse starts at `CreateRestApiFunction`.

The PR also adds a KernelSyntaxExample for `/me/messages` endpoint of
Microsoft Graph.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: SergeyMenshykh <[email protected]>
2024-02-15 11:56:30 +00:00

188 lines
6.0 KiB
C#

// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Runtime.CompilerServices;
using Microsoft.Extensions.Configuration;
using Microsoft.SemanticKernel.Plugins.MsGraph.Connectors.Client;
using Reliability;
public sealed class TestConfiguration
{
private readonly IConfigurationRoot _configRoot;
private static TestConfiguration? s_instance;
private TestConfiguration(IConfigurationRoot configRoot)
{
this._configRoot = configRoot;
}
public static void Initialize(IConfigurationRoot configRoot)
{
s_instance = new TestConfiguration(configRoot);
}
public static OpenAIConfig OpenAI => LoadSection<OpenAIConfig>();
public static AzureOpenAIConfig AzureOpenAI => LoadSection<AzureOpenAIConfig>();
public static AzureOpenAIConfig AzureOpenAIImages => LoadSection<AzureOpenAIConfig>();
public static AzureOpenAIEmbeddingsConfig AzureOpenAIEmbeddings => LoadSection<AzureOpenAIEmbeddingsConfig>();
public static AzureAISearchConfig AzureAISearch => LoadSection<AzureAISearchConfig>();
public static QdrantConfig Qdrant => LoadSection<QdrantConfig>();
public static WeaviateConfig Weaviate => LoadSection<WeaviateConfig>();
public static KeyVaultConfig KeyVault => LoadSection<KeyVaultConfig>();
public static HuggingFaceConfig HuggingFace => LoadSection<HuggingFaceConfig>();
public static PineconeConfig Pinecone => LoadSection<PineconeConfig>();
public static BingConfig Bing => LoadSection<BingConfig>();
public static GoogleConfig Google => LoadSection<GoogleConfig>();
public static GithubConfig Github => LoadSection<GithubConfig>();
public static PostgresConfig Postgres => LoadSection<PostgresConfig>();
public static RedisConfig Redis => LoadSection<RedisConfig>();
public static JiraConfig Jira => LoadSection<JiraConfig>();
public static ChromaConfig Chroma => LoadSection<ChromaConfig>();
public static KustoConfig Kusto => LoadSection<KustoConfig>();
public static MongoDBConfig MongoDB => LoadSection<MongoDBConfig>();
public static ChatGPTRetrievalPluginConfig ChatGPTRetrievalPlugin => LoadSection<ChatGPTRetrievalPluginConfig>();
public static MsGraphConfiguration MSGraph => LoadSection<MsGraphConfiguration>();
private static T LoadSection<T>([CallerMemberName] string? caller = null)
{
if (s_instance == null)
{
throw new InvalidOperationException(
"TestConfiguration must be initialized with a call to Initialize(IConfigurationRoot) before accessing configuration values.");
}
if (string.IsNullOrEmpty(caller))
{
throw new ArgumentNullException(nameof(caller));
}
return s_instance._configRoot.GetSection(caller).Get<T>() ??
throw new ConfigurationNotFoundException(section: caller);
}
#pragma warning disable CS8618 // Non-nullable field must contain a non-null value when exiting constructor.
public class OpenAIConfig
{
public string ModelId { get; set; }
public string ChatModelId { get; set; }
public string EmbeddingModelId { get; set; }
public string ApiKey { get; set; }
}
public class AzureOpenAIConfig
{
public string ServiceId { get; set; }
public string DeploymentName { get; set; }
public string ModelId { get; set; }
public string ChatDeploymentName { get; set; }
public string ChatModelId { get; set; }
public string ImageDeploymentName { get; set; }
public string ImageModelId { get; set; }
public string ImageEndpoint { get; set; }
public string Endpoint { get; set; }
public string ApiKey { get; set; }
public string ImageApiKey { get; set; }
}
public class AzureOpenAIEmbeddingsConfig
{
public string DeploymentName { get; set; }
public string Endpoint { get; set; }
public string ApiKey { get; set; }
}
public class AzureAISearchConfig
{
public string Endpoint { get; set; }
public string ApiKey { get; set; }
public string IndexName { get; set; }
}
public class QdrantConfig
{
public string Endpoint { get; set; }
public string Port { get; set; }
}
public class WeaviateConfig
{
public string Scheme { get; set; }
public string Endpoint { get; set; }
public string Port { get; set; }
public string ApiKey { get; set; }
}
public class KeyVaultConfig
{
public string Endpoint { get; set; }
public string ClientId { get; set; }
public string ClientSecret { get; set; }
}
public class HuggingFaceConfig
{
public string ApiKey { get; set; }
public string ModelId { get; set; }
}
public class PineconeConfig
{
public string ApiKey { get; set; }
public string Environment { get; set; }
}
public class BingConfig
{
public string ApiKey { get; set; }
}
public class GoogleConfig
{
public string ApiKey { get; set; }
public string SearchEngineId { get; set; }
}
public class GithubConfig
{
public string PAT { get; set; }
}
public class PostgresConfig
{
public string ConnectionString { get; set; }
}
public class RedisConfig
{
public string Configuration { get; set; }
}
public class JiraConfig
{
public string ApiKey { get; set; }
public string Email { get; set; }
public string Domain { get; set; }
}
public class ChromaConfig
{
public string Endpoint { get; set; }
}
public class KustoConfig
{
public string ConnectionString { get; set; }
}
public class MongoDBConfig
{
public string ConnectionString { get; set; }
}
public class ChatGPTRetrievalPluginConfig
{
public string Token { get; set; }
}
#pragma warning restore CS8618 // Non-nullable field must contain a non-null value when exiting constructor.
}