mirror of
https://github.com/microsoft/semantic-kernel.git
synced 2026-10-12 02:19:06 +00:00
Dmytro StrukandShawn Callegari
55dc7942e5
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This PR contains changes to improve observability for Semantic Kernel users, including: - Logging: using already existing `ILogger` interface with some improvements described below. - Metering: implemented using `Meter` and `MeterListener` classes from `System.Diagnostics.Metrics` namespace. - Tracing: implemented using `Activity`, `ActivitySource` and `ActivityListener` classes from `System.Diagnostics` namespace. Added telemetry uses native .NET methods, which means that it's not dependent on specific telemetry tool. Current PR contains console application using Application Insights as example to show telemetry capabilities in Kernel. Changes include instrumentation for `SequentialPlanner` as a starting point and later there will be new PRs for other types of planners and more improvements for kernel telemetry. Also, there are updates for `LogLevel` handling across codebase to align with described log level purpose: https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.logging.loglevel?view=dotnet-plat-ext-7.0 Particularly, there are changes for `LogLevel.Trace` to be used for sensitive data and most detailed messages, which should not be enabled in prodution environments. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Changes: - Marked `.WithLogger` method in KernelBuilder as obsolete and added new method `.WithLogging`. - Added new methods `.WithMetering` and `.WithTracing` in KernelBuilder. - Improved `LogLevel` handling across codebase to cover cases of logging sensitive data. - Implemented `InstrumentedSequentialPlanner` for instrumentation using decorator pattern. - Extended `InvokeAsync` method in `Plan` for instrumentation. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows SK Contribution Guidelines (https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) - [x] The code follows the .NET coding conventions (https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions) verified with `dotnet format` - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Shawn Callegari <[email protected]>
Semantic Kernel syntax examples
This project contains a collection of semi-random examples about various scenarios using SK components.
The examples are ordered by number, starting with very basic examples.
Most of the examples will require secrets and credentials, to access OpenAI, Azure OpenAI, Bing and other resources. We suggest using .NET Secret Manager to avoid the risk of leaking secrets into the repository, branches and pull requests. You can also use environment variables if you prefer.
To set your secrets with Secret Manager:
cd dotnet/samples/KernelSyntaxExamples
dotnet user-secrets init
dotnet user-secrets set "OpenAI:ModelId" "..."
dotnet user-secrets set "OpenAI:ChatModelId" "..."
dotnet user-secrets set "OpenAI:EmbeddingModelId" "..."
dotnet user-secrets set "OpenAI:ApiKey" "..."
dotnet user-secrets set "AzureOpenAI:ServiceId" "..."
dotnet user-secrets set "AzureOpenAI:DeploymentName" "..."
dotnet user-secrets set "AzureOpenAI:ChatDeploymentName" "..."
dotnet user-secrets set "AzureOpenAI:Endpoint" "https://... .openai.azure.com/"
dotnet user-secrets set "AzureOpenAI:ApiKey" "..."
dotnet user-secrets set "AzureOpenAIEmbeddings:DeploymentName" "..."
dotnet user-secrets set "AzureOpenAIEmbeddings:Endpoint" "https://... .openai.azure.com/"
dotnet user-secrets set "AzureOpenAIEmbeddings:ApiKey" "..."
dotnet user-secrets set "ACS:Endpoint" "https://... .search.windows.net"
dotnet user-secrets set "ACS:ApiKey" "..."
dotnet user-secrets set "Qdrant:Endpoint" "..."
dotnet user-secrets set "Qdrant:Port" "..."
dotnet user-secrets set "Weaviate:Scheme" "..."
dotnet user-secrets set "Weaviate:Endpoint" "..."
dotnet user-secrets set "Weaviate:Port" "..."
dotnet user-secrets set "Weaviate:ApiKey" "..."
dotnet user-secrets set "KeyVault:Endpoint" "..."
dotnet user-secrets set "KeyVault:ClientId" "..."
dotnet user-secrets set "KeyVault:TenantId" "..."
dotnet user-secrets set "HuggingFace:ApiKey" "..."
dotnet user-secrets set "HuggingFace:ModelId" "..."
dotnet user-secrets set "Pinecone:ApiKey" "..."
dotnet user-secrets set "Pinecone:Environment" "..."
dotnet user-secrets set "Jira:ApiKey" "..."
dotnet user-secrets set "Jira:Email" "..."
dotnet user-secrets set "Jira:Domain" "..."
dotnet user-secrets set "Bing:ApiKey" "..."
dotnet user-secrets set "Google:ApiKey" "..."
dotnet user-secrets set "Google:SearchEngineId" "..."
dotnet user-secrets set "Github:PAT" "github_pat_..."
dotnet user-secrets set "Apim:Endpoint" "https://apim...azure-api.net/"
dotnet user-secrets set "Apim:SubscriptionKey" "..."
dotnet user-secrets set "Postgres:ConnectionString" "..."
dotnet user-secrets set "Redis:Configuration" "..."
To set your secrets with environment variables, use these names:
# OpenAI
OpenAI__ModelId
OpenAI__ChatModelId
OpenAI__EmbeddingModelId
OpenAI__ApiKey
# Azure OpenAI
AzureOpenAI__ServiceId
AzureOpenAI__DeploymentName
AzureOpenAI__ChatDeploymentName
AzureOpenAI__Endpoint
AzureOpenAI__ApiKey
AzureOpenAIEmbeddings__DeploymentName
AzureOpenAIEmbeddings__Endpoint
AzureOpenAIEmbeddings__ApiKey
# Azure Cognitive Search
ACS__Endpoint
ACS__ApiKey
# Qdrant
Qdrant__Endpoint
Qdrant__Port
# Weaviate
Weaviate__Scheme
Weaviate__Endpoint
Weaviate__Port
Weaviate__ApiKey
# Azure Key Vault
KeyVault__Endpoint
KeyVault__ClientId
KeyVault__TenantId
# Hugging Face
HuggingFace__ApiKey
HuggingFace__ModelId
# Pinecone
Pinecone__ApiKey
Pinecone__Environment
# Jira
Jira__ApiKey
Jira__Email
Jira__Domain
# Bing
Bing__ApiKey
# Google
Google__ApiKey
Google__SearchEngineId
# Github
Github__PAT
# Azure API Management (APIM)
Apim__Endpoint
Apim__SubscriptionKey
# Other
Postgres__ConnectionString
Redis__Configuration