Python: Support Declarative Spec for OpenAIAssistantAgent & OpenAIResponsesAgent (#12247)

### Motivation and Context

This PR builds on the initial work/abstractions from #11982. The
`AzureAssistantAgent` / `OpenAIAssistantAgent` as well as the
`AzureResponsesAgent` / `OpenAIResponsesAgent` now support declarative
specs. Several examples on how to declare the spec are shown in the
respective dirs in `samples/concepts/agents`, as well as a simple
getting started example for each agent type.

To further simplify working with these two agents, the
`setup_resources()` methods for each agent type have been marked with a
deprecation tag. The new method to use is
`OpenAIAssistantAgent.create_client()` or
`OpenAIResponsesAgent.create_client()`. Further, when one needs to
access the underlying setting's, it's simple to do so with a settings
object and accessing the attribute.

In the previous PR for the AzureAIAgent, it lacked support for
templating - this is included in this current PR.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Bring in support for the declarative single agent spec for these agent
types.
- Add samples
- Add unit tests
- Add support for prompt templates for all current agent types
- Closes #11979

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
This commit is contained in:
Evan Mattson
2025-05-26 05:02:59 +00:00
committed by GitHub
parent 4027c5eb6a
commit ff1997047d
102 changed files with 4325 additions and 1116 deletions
+17 -1
View File
@@ -61,12 +61,22 @@
- [Mixed Chat Reset](./agents/mixed_chat/mixed_chat_reset.py)
- [Mixed Chat Streaming](./agents/mixed_chat/mixed_chat_streaming.py)
#### [OpenAI Assistant Agent](../../semantic_kernel/agents/open_ai/open_ai_assistant_agent.py)
#### [OpenAI Assistant Agent](../../semantic_kernel/agents/open_ai/openai_assistant_agent.py)
- [Azure OpenAI Assistant Declarative Code Interpreter](./agents/openai_assistant/azure_openai_assistant_declarative_code_interpreter.py)
- [Azure OpenAI Assistant Declarative File Search](./agents/openai_assistant/azure_openai_assistant_declarative_file_search.py)
- [Azure OpenAI Assistant Declarative Function Calling From File](./agents/openai_assistant/azure_openai_assistant_declarative_function_calling_from_file.py)
- [Azure OpenAI Assistant Declarative Templating](./agents/openai_assistant/azure_openai_assistant_declarative_templating.py)
- [Azure OpenAI Assistant Declarative With Existing Agent ID](./agents/openai_assistant/azure_openai_assistant_declarative_with_existing_agent_id.py)
- [OpenAI Assistant Auto Function Invocation Filter Streaming](./agents/openai_assistant/openai_assistant_auto_func_invocation_filter_streaming.py)
- [OpenAI Assistant Auto Function Invocation Filter](./agents/openai_assistant/openai_assistant_auto_func_invocation_filter.py)
- [OpenAI Assistant Chart Maker Streaming](./agents/openai_assistant/openai_assistant_chart_maker_streaming.py)
- [OpenAI Assistant Chart Maker](./agents/openai_assistant/openai_assistant_chart_maker.py)
- [OpenAI Assistant Declarative Code Interpreter](./agents/openai_assistant/openai_assistant_declarative_code_interpreter.py)
- [OpenAI Assistant Declarative File Search](./agents/openai_assistant/openai_assistant_declarative_file_search.py)
- [OpenAI Assistant Declarative Function Calling From File](./agents/openai_assistant/openai_assistant_declarative_function_calling_from_file.py)
- [OpenAI Assistant Declarative Templating](./agents/openai_assistant/openai_assistant_declarative_templating.py)
- [OpenAI Assistant Declarative With Existing Agent ID](./agents/openai_assistant/openai_assistant_declarative_with_existing_agent_id.py)
- [OpenAI Assistant File Manipulation Streaming](./agents/openai_assistant/openai_assistant_file_manipulation_streaming.py)
- [OpenAI Assistant File Manipulation](./agents/openai_assistant/openai_assistant_file_manipulation.py)
- [OpenAI Assistant Retrieval](./agents/openai_assistant/openai_assistant_retrieval.py)
@@ -79,6 +89,12 @@
#### [OpenAI Responses Agent](../../semantic_kernel/agents/open_ai/openai_responses_agent.py)
- [Azure OpenAI Responses Agent Declarative File Search](./agents/openai_responses/azure_openai_responses_agent_declarative_file_search.py)
- [Azure OpenAI Responses Agent Declarative Function Calling From File](./agents/openai_responses/azure_openai_responses_agent_declarative_function_calling_from_file.py)
- [Azure OpenAI Responses Agent Declarative Templating](./agents/openai_responses/azure_openai_responses_agent_declarative_templating.py)
- [OpenAI Responses Agent Declarative File Search](./agents/openai_responses/openai_responses_agent_declarative_file_search.py)
- [OpenAI Responses Agent Declarative Function Calling From File](./agents/openai_responses/openai_responses_agent_declarative_function_calling_from_file.py)
- [OpenAI Responses Agent Declarative Web Search](./agents/openai_responses/openai_responses_agent_declarative_web_search.py)
- [OpenAI Responses Message Callback Streaming](./agents/openai_responses/responses_agent_message_callback_streaming.py)
- [OpenAI Responses Message Callback](./agents/openai_responses/responses_agent_message_callback.py)
- [OpenAI Responses File Search Streaming](./agents/openai_responses/responses_agent_file_search_streaming.py)
@@ -4,8 +4,7 @@ import asyncio
from azure.identity.aio import DefaultAzureCredential
from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
from semantic_kernel.agents.agent import AgentRegistry
from semantic_kernel.agents import AgentRegistry, AzureAIAgent, AzureAIAgentSettings
from semantic_kernel.contents.chat_message_content import ChatMessageContent
from semantic_kernel.contents.function_call_content import FunctionCallContent
from semantic_kernel.contents.function_result_content import FunctionResultContent
@@ -4,8 +4,7 @@ import asyncio
from azure.identity.aio import DefaultAzureCredential
from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
from semantic_kernel.agents.agent import AgentRegistry
from semantic_kernel.agents import AgentRegistry, AzureAIAgent, AzureAIAgentSettings
"""
The following sample demonstrates how to create an Azure AI agent that answers
@@ -6,8 +6,7 @@ import os
from azure.ai.agents.models import FilePurpose
from azure.identity.aio import DefaultAzureCredential
from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
from semantic_kernel.agents.agent import AgentRegistry
from semantic_kernel.agents import AgentRegistry, AzureAIAgent, AzureAIAgentSettings
"""
The following sample demonstrates how to create an Azure AI agent that answers
@@ -6,25 +6,20 @@ import os
from azure.ai.agents.models import VectorStore
from azure.identity.aio import DefaultAzureCredential
from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
from semantic_kernel.agents.agent import AgentRegistry
from semantic_kernel.agents import AgentRegistry, AzureAIAgent, AzureAIAgentSettings
"""
The following sample demonstrates how to create an Azure AI agent that answers
user questions using the file search tool.
The agent is used to answer user questions that require file search to help ground
answers from the model.
user questions using the file search tool from a declarative spec.
"""
# Define the YAML string for the sample
spec = """
type: foundry_agent
name: FileSearchAgent
description: Agent with code interpreter tool.
description: Agent with file search tool.
instructions: >
Use the code interpreter tool to answer questions that require code to be generated
and executed.
Use the file search tool to answer questions from the user.
model:
id: ${AzureAI:ChatModelId}
connection:
@@ -4,8 +4,7 @@ import asyncio
from azure.identity.aio import DefaultAzureCredential
from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
from semantic_kernel.agents.agent import AgentRegistry
from semantic_kernel.agents import AgentRegistry, AzureAIAgent, AzureAIAgentSettings
"""
The following sample demonstrates how to create an Azure AI agent that answers
@@ -0,0 +1,72 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from azure.identity.aio import DefaultAzureCredential
from semantic_kernel.agents import AgentRegistry, AzureAIAgent
"""
The following sample demonstrates how to create an Azure AI Agent that invokes
a story generation task using a prompt template and a declarative spec.
"""
# Define the YAML string for the sample
spec = """
type: foundry_agent
name: StoryAgent
description: An agent that generates a story about a topic.
instructions: Tell a story about {{$topic}} that is {{$length}} sentences long.
model:
id: ${AzureAI:ChatModelId}
connection:
connection_string: ${AzureAI:Endpoint}
inputs:
topic:
description: The topic of the story.
required: true
default: Cats
length:
description: The number of sentences in the story.
required: true
default: 2
outputs:
output1:
description: The generated story.
template:
format: semantic-kernel
"""
async def main():
async with (
DefaultAzureCredential() as creds,
AzureAIAgent.create_client(credential=creds) as client,
):
try:
# Create the AzureAI Agent from the YAML spec
agent: AzureAIAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
)
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=None,
):
print(f"# {response.name}: {response}")
finally:
# Cleanup: Delete the agent, vector store, and file
await client.agents.delete_agent(agent.id)
"""
Sample output:
# StoryAgent: Under the silvery moon, three mischievous cats tiptoed across the rooftop, chasing
shadows and sharing secret whispers. By dawn, they curled up together, purring softly, dreaming
of adventures yet to come.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -4,8 +4,7 @@ import asyncio
from azure.identity.aio import DefaultAzureCredential
from semantic_kernel.agents import AzureAIAgent
from semantic_kernel.agents.agent import AgentRegistry
from semantic_kernel.agents import AgentRegistry, AzureAIAgent
"""
The following sample demonstrates how to create an Azure AI agent based
@@ -4,9 +4,9 @@ import asyncio
from typing import Annotated
from semantic_kernel.agents import AgentGroupChat, AzureAssistantAgent, ChatCompletionAgent
from semantic_kernel.agents.strategies.termination.termination_strategy import TerminationStrategy
from semantic_kernel.agents.strategies import TerminationStrategy
from semantic_kernel.connectors.ai import FunctionChoiceBehavior
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureOpenAISettings
from semantic_kernel.contents import AuthorRole
from semantic_kernel.functions import KernelArguments, kernel_function
from semantic_kernel.kernel import Kernel
@@ -86,11 +86,11 @@ async def main():
)
# Create the Assistant Agent using Azure OpenAI resources
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name=COPYWRITER_NAME,
instructions=COPYWRITER_INSTRUCTIONS,
)
@@ -4,7 +4,7 @@ import asyncio
import os
from semantic_kernel.agents import AgentGroupChat, AzureAssistantAgent, ChatCompletionAgent
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureOpenAISettings
from semantic_kernel.contents import AnnotationContent, AuthorRole
from semantic_kernel.kernel import Kernel
@@ -31,10 +31,10 @@ async def main():
)
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# If desired, create using OpenAI resources
# client, model = OpenAIAssistantAgent.setup_resources()
# client = OpenAIAssistantAgent.create_client()
# Load the text file as a FileObject
with open(file_path, "rb") as file:
@@ -45,7 +45,7 @@ async def main():
)
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
instructions="Create charts as requested without explanation.",
name="ChartMaker",
tools=code_interpreter_tool,
@@ -3,7 +3,7 @@
import asyncio
from semantic_kernel.agents import AgentGroupChat, AzureAssistantAgent, ChatCompletionAgent
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureOpenAISettings
from semantic_kernel.contents import AnnotationContent
from semantic_kernel.contents.utils.author_role import AuthorRole
from semantic_kernel.kernel import Kernel
@@ -24,14 +24,14 @@ def _create_kernel_with_chat_completion(service_id: str) -> Kernel:
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Get the code interpreter tool and resources
code_interpreter_tool, code_interpreter_resources = AzureAssistantAgent.configure_code_interpreter_tool()
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="Analyst",
instructions="Create charts as requested without explanation",
tools=code_interpreter_tool,
@@ -4,7 +4,7 @@ import asyncio
from typing import TYPE_CHECKING
from semantic_kernel.agents import AgentGroupChat, AzureAssistantAgent, ChatCompletionAgent
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureOpenAISettings
from semantic_kernel.contents import AuthorRole
from semantic_kernel.kernel import Kernel
@@ -41,11 +41,11 @@ async def main():
# Next, we will create the AzureAssistantAgent
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="copywriter",
instructions="""
The user may either provide information or query on information previously provided.
@@ -4,7 +4,7 @@ import asyncio
from semantic_kernel.agents import AgentGroupChat, AzureAssistantAgent, ChatCompletionAgent
from semantic_kernel.agents.strategies import TerminationStrategy
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureOpenAISettings
from semantic_kernel.contents import AuthorRole
from semantic_kernel.kernel import Kernel
@@ -46,11 +46,11 @@ async def main():
# Next, we will create the AzureAssistantAgent
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="CopyWriter",
instructions="""
You are a copywriter with ten years of experience and are known for brevity and a dry humor.
@@ -32,11 +32,11 @@ OpenAI Assistant Agents are created in the following way:
from semantic_kernel.agents import OpenAIAssistantAgent
# Create the client using OpenAI resources and configuration
client, model = OpenAIAssistantAgent.setup_resources()
client = OpenAIAssistantAgent.create_client()
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name
instructions="<instructions>",
name="<name>",
)
@@ -73,11 +73,11 @@ Alternatively, you can pass the `api_version` parameter when creating an `AzureA
from semantic_kernel.agents import AzureAssistantAgent
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name
instructions="<instructions>",
name="<name>",
)
@@ -0,0 +1,141 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import os
from semantic_kernel.agents import AgentRegistry, AzureAssistantAgent
"""
The following sample demonstrates how to create an Azure Assistant Agent that answers
user questions using the code interpreter tool.
The agent is then used to answer user questions that require code to be generated and
executed. The responses are handled in a streaming manner.
"""
# Define the YAML string for the sample
spec = """
type: azure_assistant
name: CodeInterpreterAgent
description: Agent with code interpreter tool.
instructions: >
Use the code interpreter tool to answer questions that require code to be generated
and executed.
model:
id: ${AzureOpenAI:ChatModelId}
connection:
api_key: ${AzureOpenAI:ApiKey}
tools:
- type: code_interpreter
options:
file_ids:
- ${AzureOpenAI:FileId1}
"""
async def main():
client = AzureAssistantAgent.create_client()
csv_file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
"resources",
"agent_assistant_file_manipulation",
"sales.csv",
)
# Load the employees PDF file as a FileObject
with open(csv_file_path, "rb") as file:
file = await client.files.create(file=file, purpose="assistants")
try:
# Create the Assistant Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `AzureOpenAI:` prefix).
# The short-format is used here for brevity
agent: AzureAssistantAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
extras={"AzureOpenAI:FileId1": file.id},
)
# Define the task for the agent
TASK = "Give me the code to calculate the total sales for all segments."
print(f"# User: '{TASK}'")
# Invoke the agent for the specified task
is_code = False
last_role = None
async for response in agent.invoke_stream(
messages=TASK,
):
current_is_code = response.metadata.get("code", False)
if current_is_code:
if not is_code:
print("\n\n```python")
is_code = True
print(response.content, end="", flush=True)
else:
if is_code:
print("\n```")
is_code = False
last_role = None
if hasattr(response, "role") and response.role is not None and last_role != response.role:
print(f"\n# {response.role}: ", end="", flush=True)
last_role = response.role
print(response.content, end="", flush=True)
if is_code:
print("```\n")
print()
finally:
# Cleanup: Delete the thread and agent
await client.beta.assistants.delete(agent.id)
await client.files.delete(file.id)
"""
Sample output:
# User: 'Give me the code to calculate the total sales for all segments.'
# AuthorRole.ASSISTANT: Let me first examine the contents of the uploaded file to determine its structure. This
will allow me to create the appropriate code for calculating the total sales for all segments. Hang tight!
```python
import pandas as pd
# Load the uploaded file to examine its contents
file_path = '/mnt/data/assistant-3nXizu2EX2EwXikUz71uNc'
data = pd.read_csv(file_path)
# Display the first few rows and column names to understand the structure of the dataset
data.head(), data.columns
```
# AuthorRole.ASSISTANT: The dataset contains several columns, including `Segment`, `Sales`, and others such as
`Country`, `Product`, and date-related information. To calculate the total sales for all segments, we will:
1. Group the data by the `Segment` column.
2. Sum the `Sales` column for each segment.
3. Calculate the grand total of all sales across all segments.
Here is the code snippet for this task:
```python
# Group by 'Segment' and sum up 'Sales'
segment_sales = data.groupby('Segment')['Sales'].sum()
# Calculate the total sales across all segments
total_sales = segment_sales.sum()
print("Total Sales per Segment:")
print(segment_sales)
print(f"\nGrand Total Sales: {total_sales}")
```
Would you like me to execute this directly for the uploaded data?
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,97 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import os
from semantic_kernel.agents import AgentRegistry, AzureAssistantAgent
"""
The following sample demonstrates how to create an Azure Assistant Agent that answers
user questions using the file search tool.
The agent is used to answer user questions that require file search to help ground
answers from the model.
"""
# Define the YAML string for the sample
spec = """
type: azure_assistant
name: FileSearchAgent
description: Agent with file search tool.
instructions: >
Use the file search tool to answer questions from the user.
model:
id: ${AzureOpenAI:ChatModelId}
connection:
api_key: ${AzureOpenAI:ApiKey}
tools:
- type: file_search
options:
vector_store_ids:
- ${AzureOpenAI:VectorStoreId}
"""
async def main():
# Setup the OpenAI Assistant client
client = AzureAssistantAgent.create_client()
# Read and upload the file to the OpenAI AI service
pdf_file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
"resources",
"file_search",
"employees.pdf",
)
# Upload the pdf file to the assistant service
with open(pdf_file_path, "rb") as file:
file = await client.files.create(file=file, purpose="assistants")
vector_store = await client.vector_stores.create(
name="assistant_file_search",
file_ids=[file.id],
)
try:
# Create the Assistant Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `AzureOpenAI:` prefix).
# The short-format is used here for brevity
agent: AzureAssistantAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
extras={"AzureOpenAI:VectorStoreId": vector_store.id},
)
# Define the task for the agent
TASK = "Who can help me if I have a sales question?"
print(f"# User: '{TASK}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=TASK,
):
print(f"# {response.name}: {response}")
finally:
# Cleanup: Delete the agent, vector store, and file
await client.beta.assistants.delete(agent.id)
await client.vector_stores.delete(vector_store.id)
await client.files.delete(file.id)
"""
Sample output:
# User: 'Who can help me if I have a sales question?'
# FileSearchAgent: If you have a sales question, you may contact the following individuals:
1. **Hicran Bea** - Sales Manager
2. **Mariam Jaslyn** - Sales Representative
3. **Angelino Embla** - Sales Representative
This information comes from the employee records【4:0†source】.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,100 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import os
from typing import Annotated
from semantic_kernel.agents import AgentRegistry, AzureAssistantAgent
from semantic_kernel.functions import kernel_function
"""
The following sample demonstrates how to create an Azure Assistant Agent that answers
user questions. The sample shows how to load a declarative spec from a file.
The plugins/functions must already exist in the kernel.
They are not created declaratively via the spec.
"""
class MenuPlugin:
"""A sample Menu Plugin used for the concept sample."""
@kernel_function(description="Provides a list of specials from the menu.")
def get_specials(self) -> Annotated[str, "Returns the specials from the menu."]:
return """
Special Soup: Clam Chowder
Special Salad: Cobb Salad
Special Drink: Chai Tea
"""
@kernel_function(description="Provides the price of the requested menu item.")
def get_item_price(
self, menu_item: Annotated[str, "The name of the menu item."]
) -> Annotated[str, "Returns the price of the menu item."]:
return "$9.99"
async def main():
try:
client = AzureAssistantAgent.create_client()
# Define the YAML file path for the sample
file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
"resources",
"declarative_spec",
"azure_assistant_spec.yaml",
)
# Create the Assistant Agent from the YAML spec
agent: AzureAssistantAgent = await AgentRegistry.create_from_file(
file_path,
plugins=[MenuPlugin()],
client=client,
)
# Create the agent
user_inputs = [
"Hello",
"What is the special soup?",
"How much does that cost?",
"Thank you",
]
# Create a thread for the agent
# If no thread is provided, a new thread will be
# created and returned with the initial response
thread = None
for user_input in user_inputs:
print(f"# User: '{user_input}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=user_input,
thread=thread,
):
print(f"# {response.name}: {response}")
# Store the thread for the next iteration
thread = response.thread
finally:
# Cleanup: Delete the thread and agent
await client.beta.assistants.delete(agent.id) if agent else None
await thread.delete() if thread else None
"""
Sample Output:
# User: 'Hello'
# Host: Hi there! How can I assist you today?
# User: 'What is the special soup?'
# Host: The special soup is Clam Chowder.
# User: 'What is the special drink?'
# Host: The special drink is Chai Tea.
# User: 'How much is it?'
# Host: The Chai Tea costs $9.99.
# User: 'Thank you'
# Host: You're welcome! If you have any more questions, feel free to ask.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,72 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from semantic_kernel.agents import AgentRegistry, AzureAssistantAgent
"""
The following sample demonstrates how to create an Azure Assistant Agent that invokes
a story generation task using a prompt template and a declarative spec.
"""
# Define the YAML string for the sample
spec = """
type: azure_assistant
name: StoryAgent
description: An agent that generates a story about a topic.
instructions: Tell a story about {{$topic}} that is {{$length}} sentences long.
model:
id: ${AzureOpenAI:ChatModelId}
connection:
endpoint: ${AzureOpenAI:Endpoint}
inputs:
topic:
description: The topic of the story.
required: true
default: Cats
length:
description: The number of sentences in the story.
required: true
default: 2
outputs:
output1:
description: The generated story.
template:
format: semantic-kernel
"""
async def main():
# Setup the OpenAI Assistant client
client = AzureAssistantAgent.create_client()
try:
# Create the Assistant Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `AzureOpenAI:` prefix).
# The short-format is used here for brevity
agent: AzureAssistantAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
)
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=None,
):
print(f"# {response.name}: {response}")
finally:
# Cleanup: Delete the agent, vector store, and file
await client.beta.assistants.delete(agent.id)
"""
Sample output:
# StoryAgent: Under the silvery moon, three mischievous cats tiptoed across the rooftop, chasing
shadows and sharing secret whispers. By dawn, they curled up together, purring softly, dreaming
of adventures yet to come.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,64 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from semantic_kernel.agents import AgentRegistry, AzureAssistantAgent
"""
The following sample demonstrates how to create an Azure Assistant Agent based
on an existing agent ID.
"""
# Define the YAML string for the sample
spec = """
id: ${AzureOpenAI:AgentId}
type: azure_assistant
instructions: You are helpful agent who always responds in French.
"""
async def main():
try:
client = AzureAssistantAgent.create_client()
# Create the Assistant Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `AzureOpenAI:` prefix).
# The short-format is used here for brevity
agent: AzureAssistantAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
extras={"AgentId": "<my-agent-id>"}, # Specify the existing agent ID
)
# Define the task for the agent
TASK = "Why is the sky blue?"
print(f"# User: '{TASK}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=TASK,
):
print(f"# {response.name}: {response}")
finally:
# Cleanup: Delete the thread and agent
await client.beta.assistants.delete(agent.id)
"""
Sample output:
# User: 'Why is the sky blue?'
# WeatherAgent: Le ciel est bleu à cause d'un phénomène appelé **diffusion de Rayleigh**. La lumière du
Soleil est composée de toutes les couleurs du spectre visible, mais lorsqu'elle traverse l'atmosphère
terrestre, elle entre en contact avec les molécules d'air et les particules présentes.
Les couleurs à courtes longueurs d'onde, comme le bleu et le violet, sont diffusées dans toutes les directions
beaucoup plus efficacement que les couleurs à longues longueurs d'onde, comme le rouge et l'orange. Bien que le
violet ait une longueur d'onde encore plus courte que le bleu, nos yeux sont moins sensibles à cette couleur,
et une partie du violet est également absorbée par la haute atmosphère. Ainsi, le bleu domine, donnant au ciel
sa couleur caractéristique.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -4,6 +4,7 @@ import asyncio
from typing import Annotated
from semantic_kernel.agents import AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import ChatMessageContent, FunctionCallContent, FunctionResultContent
from semantic_kernel.filters import (
AutoFunctionInvocationContext,
@@ -93,11 +94,11 @@ USER_INPUTS = ["What's the special food on the menu?", "What should I do then?"]
async def main() -> None:
# 1. Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# 2. Define the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="Host",
instructions="Answer questions about the menu.",
)
@@ -4,6 +4,7 @@ import asyncio
from typing import Annotated
from semantic_kernel.agents import AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import ChatMessageContent, FunctionCallContent, FunctionResultContent
from semantic_kernel.filters import (
AutoFunctionInvocationContext,
@@ -93,11 +94,11 @@ USER_INPUTS = ["What's the special food on the menu?", "What should I do then?"]
async def main() -> None:
# 1. Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# 2. Define the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="Host",
instructions="Answer questions about the menu.",
)
@@ -3,6 +3,7 @@ import asyncio
from samples.concepts.agents.openai_assistant.openai_assistant_sample_utils import download_response_images
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import FileReferenceContent
"""
@@ -15,14 +16,14 @@ in a streaming fashion.
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Get the code interpreter tool and resources
code_interpreter_tool, code_interpreter_resource = AzureAssistantAgent.configure_code_interpreter_tool()
# Define the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
instructions="Create charts as requested without explanation.",
name="ChartMaker",
tools=code_interpreter_tool,
@@ -3,6 +3,7 @@ import asyncio
from samples.concepts.agents.openai_assistant.openai_assistant_sample_utils import download_response_images
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import StreamingFileReferenceContent
"""
@@ -15,14 +16,14 @@ in a streaming fashion.
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Get the code interpreter tool and resources
code_interpreter_tool, code_interpreter_resource = AzureAssistantAgent.configure_code_interpreter_tool()
# Define the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
instructions="Create charts as requested without explanation.",
name="ChartMaker",
tools=code_interpreter_tool,
@@ -0,0 +1,141 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import os
from semantic_kernel.agents import AgentRegistry, OpenAIAssistantAgent
"""
The following sample demonstrates how to create an OpenAI Assistant Agent that answers
user questions using the code interpreter tool.
The agent is then used to answer user questions that require code to be generated and
executed. The responses are handled in a streaming manner.
"""
# Define the YAML string for the sample
spec = """
type: openai_assistant
name: CodeInterpreterAgent
description: Agent with code interpreter tool.
instructions: >
Use the code interpreter tool to answer questions that require code to be generated
and executed.
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
tools:
- type: code_interpreter
options:
file_ids:
- ${OpenAI:FileId1}
"""
async def main():
client = OpenAIAssistantAgent.create_client()
csv_file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
"resources",
"agent_assistant_file_manipulation",
"sales.csv",
)
# Load the employees PDF file as a FileObject
with open(csv_file_path, "rb") as file:
file = await client.files.create(file=file, purpose="assistants")
try:
# Create the Assistant Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `OpenAI:` prefix).
# The short-format is used here for brevity
agent: OpenAIAssistantAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
extras={"OpenAI:FileId1": file.id},
)
# Define the task for the agent
TASK = "Give me the code to calculate the total sales for all segments."
print(f"# User: '{TASK}'")
# Invoke the agent for the specified task
is_code = False
last_role = None
async for response in agent.invoke_stream(
messages=TASK,
):
current_is_code = response.metadata.get("code", False)
if current_is_code:
if not is_code:
print("\n\n```python")
is_code = True
print(response.content, end="", flush=True)
else:
if is_code:
print("\n```")
is_code = False
last_role = None
if hasattr(response, "role") and response.role is not None and last_role != response.role:
print(f"\n# {response.role}: ", end="", flush=True)
last_role = response.role
print(response.content, end="", flush=True)
if is_code:
print("```\n")
print()
finally:
# Cleanup: Delete the thread and agent
await client.beta.assistants.delete(agent.id)
await client.files.delete(file.id)
"""
Sample output:
# User: 'Give me the code to calculate the total sales for all segments.'
# AuthorRole.ASSISTANT: Let me first examine the contents of the uploaded file to determine its structure. This
will allow me to create the appropriate code for calculating the total sales for all segments. Hang tight!
```python
import pandas as pd
# Load the uploaded file to examine its contents
file_path = '/mnt/data/assistant-3nXizu2EX2EwXikUz71uNc'
data = pd.read_csv(file_path)
# Display the first few rows and column names to understand the structure of the dataset
data.head(), data.columns
```
# AuthorRole.ASSISTANT: The dataset contains several columns, including `Segment`, `Sales`, and others such as
`Country`, `Product`, and date-related information. To calculate the total sales for all segments, we will:
1. Group the data by the `Segment` column.
2. Sum the `Sales` column for each segment.
3. Calculate the grand total of all sales across all segments.
Here is the code snippet for this task:
```python
# Group by 'Segment' and sum up 'Sales'
segment_sales = data.groupby('Segment')['Sales'].sum()
# Calculate the total sales across all segments
total_sales = segment_sales.sum()
print("Total Sales per Segment:")
print(segment_sales)
print(f"\nGrand Total Sales: {total_sales}")
```
Would you like me to execute this directly for the uploaded data?
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,94 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import os
from semantic_kernel.agents import AgentRegistry, OpenAIAssistantAgent
"""
The following sample demonstrates how to create an OpenAI Assistant Agent that answers
user questions using the file search tool.
The agent is used to answer user questions that require file search to help ground
answers from the model.
"""
# Define the YAML string for the sample
spec = """
type: openai_assistant
name: FileSearchAgent
description: Agent with code interpreter tool.
instructions: >
Use the code interpreter tool to answer questions that require code to be generated
and executed.
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
tools:
- type: file_search
options:
vector_store_ids:
- ${OpenAI:VectorStoreId}
"""
async def main():
# Setup the OpenAI Assistant client
client = OpenAIAssistantAgent.create_client()
# Read and upload the file to the OpenAI AI service
pdf_file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
"resources",
"file_search",
"employees.pdf",
)
# Upload the pdf file to the assistant service
with open(pdf_file_path, "rb") as file:
file = await client.files.create(file=file, purpose="assistants")
vector_store = await client.vector_stores.create(
name="assistant_file_search",
file_ids=[file.id],
)
try:
# Create the Assistant Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `OpenAI:` prefix).
# The short-format is used here for brevity
agent: OpenAIAssistantAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
extras={"OpenAI:VectorStoreId": vector_store.id},
)
# Define the task for the agent
TASK = "Who can help me if I have a sales question?"
print(f"# User: '{TASK}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=TASK,
):
print(f"# {response.name}: {response}")
finally:
# Cleanup: Delete the agent, vector store, and file
await client.beta.assistants.delete(agent.id)
await client.vector_stores.delete(vector_store.id)
await client.files.delete(file.id)
"""
Sample output:
# User: 'Who can help me if I have a sales question?'
# FileSearchAgent: If you have a sales question, you can contact either Mariam Jaslyn or Angelino Embla, who
are both listed as Sales Representatives. Alternatively, you may also reach out to Hicran Bea,
the Sales Manager【4:0†employees.pdf】.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,100 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import os
from typing import Annotated
from semantic_kernel.agents import AgentRegistry, OpenAIAssistantAgent
from semantic_kernel.functions import kernel_function
"""
The following sample demonstrates how to create an OpenAI Assistant Agent that answers
user questions. The sample shows how to load a declarative spec from a file.
The plugins/functions must already exist in the kernel.
They are not created declaratively via the spec.
"""
class MenuPlugin:
"""A sample Menu Plugin used for the concept sample."""
@kernel_function(description="Provides a list of specials from the menu.")
def get_specials(self) -> Annotated[str, "Returns the specials from the menu."]:
return """
Special Soup: Clam Chowder
Special Salad: Cobb Salad
Special Drink: Chai Tea
"""
@kernel_function(description="Provides the price of the requested menu item.")
def get_item_price(
self, menu_item: Annotated[str, "The name of the menu item."]
) -> Annotated[str, "Returns the price of the menu item."]:
return "$9.99"
async def main():
try:
client = OpenAIAssistantAgent.create_client()
# Define the YAML file path for the sample
file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
"resources",
"declarative_spec",
"openai_assistant_spec.yaml",
)
# Create the Assistant Agent from the YAML spec
agent: OpenAIAssistantAgent = await AgentRegistry.create_from_file(
file_path,
plugins=[MenuPlugin()],
client=client,
)
# Create the agent
user_inputs = [
"Hello",
"What is the special soup?",
"How much does that cost?",
"Thank you",
]
# Create a thread for the agent
# If no thread is provided, a new thread will be
# created and returned with the initial response
thread = None
for user_input in user_inputs:
print(f"# User: '{user_input}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=user_input,
thread=thread,
):
print(f"# {response.name}: {response}")
# Store the thread for the next iteration
thread = response.thread
finally:
# Cleanup: Delete the thread and agent
await client.beta.assistants.delete(agent.id) if agent else None
await thread.delete() if thread else None
"""
Sample Output:
# User: 'Hello'
# Host: Hi there! How can I assist you today?
# User: 'What is the special soup?'
# Host: The special soup is Clam Chowder.
# User: 'What is the special drink?'
# Host: The special drink is Chai Tea.
# User: 'How much is it?'
# Host: The Chai Tea costs $9.99.
# User: 'Thank you'
# Host: You're welcome! If you have any more questions, feel free to ask.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,70 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from semantic_kernel.agents import AgentRegistry, OpenAIAssistantAgent
"""
The following sample demonstrates how to create an OpenAI Assistant Agent that invokes
a story generation task using a prompt template and a declarative spec.
"""
# Define the YAML string for the sample
spec = """
type: openai_assistant
name: StoryAgent
description: An agent that generates a story about a topic.
instructions: Tell a story about {{$topic}} that is {{$length}} sentences long.
model:
id: ${OpenAI:ChatModelId}
inputs:
topic:
description: The topic of the story.
required: true
default: Cats
length:
description: The number of sentences in the story.
required: true
default: 2
outputs:
output1:
description: The generated story.
template:
format: semantic-kernel
"""
async def main():
# Setup the OpenAI Assistant client
client = OpenAIAssistantAgent.create_client()
try:
# Create the Assistant Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `OpenAI:` prefix).
# The short-format is used here for brevity
agent: OpenAIAssistantAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
)
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=None,
):
print(f"# {response.name}: {response}")
finally:
# Cleanup: Delete the agent, vector store, and file
await client.beta.assistants.delete(agent.id)
"""
Sample output:
# StoryAgent: Under the silvery moon, three mischievous cats tiptoed across the rooftop, chasing
shadows and sharing secret whispers. By dawn, they curled up together, purring softly, dreaming
of adventures yet to come.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,65 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from semantic_kernel.agents import AgentRegistry, OpenAIAssistantAgent
"""
The following sample demonstrates how to create an OpenAI Assistant Agent based
on an existing agent ID.
"""
# Define the YAML string for the sample
spec = """
id: ${OpenAI:AgentId}
type: openai_assistant
instructions: You are helpful agent who always responds in French.
"""
async def main():
client = OpenAIAssistantAgent.create_client()
try:
# Create the Assistant Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `OpenAI:` prefix).
# The short-format is used here for brevity
agent: OpenAIAssistantAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
extras={"AgentId": "<my-agent-id>"}, # Specify the existing agent ID
)
# Define the task for the agent
TASK = "Why is the sky blue?"
print(f"# User: '{TASK}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=TASK,
):
print(f"# {response.name}: {response}")
finally:
# Cleanup: Delete the thread and agent
await client.agents.delete_agent(agent.id)
"""
Sample output:
# User: 'Why is the sky blue?'
# WeatherAgent: Le ciel est bleu à cause d'un phénomène appelé **diffusion de Rayleigh**. La lumière du
Soleil est composée de toutes les couleurs du spectre visible, mais lorsqu'elle traverse l'atmosphère
terrestre, elle entre en contact avec les molécules d'air et les particules présentes.
Les couleurs à courtes longueurs d'onde, comme le bleu et le violet, sont diffusées dans toutes les directions
beaucoup plus efficacement que les couleurs à longues longueurs d'onde, comme le rouge et l'orange. Bien que le
violet ait une longueur d'onde encore plus courte que le bleu, nos yeux sont moins sensibles à cette couleur,
et une partie du violet est également absorbée par la haute atmosphère. Ainsi, le bleu domine, donnant au ciel
sa couleur caractéristique.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -4,6 +4,7 @@ import os
from samples.concepts.agents.openai_assistant.openai_assistant_sample_utils import download_response_files
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import AnnotationContent
"""
@@ -16,7 +17,7 @@ uploaded files. This sample uses non-streaming responses.
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
csv_file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
@@ -34,7 +35,7 @@ async def main():
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="FileManipulation",
instructions="Find answers to the user's questions in the provided file.",
tools=code_interpreter_tool,
@@ -4,19 +4,19 @@ import os
from samples.concepts.agents.openai_assistant.openai_assistant_sample_utils import download_response_files
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import ChatMessageContent, StreamingAnnotationContent
"""
The following sample demonstrates how to create an OpenAI
assistant using either Azure OpenAI or OpenAI and leverage the
assistant's ability to have the code interpreter work with
The following sample demonstrates how to create an Azure Assistant Agent
to leverage the assistant's ability to have the code interpreter work with
uploaded files. This sample uses streaming responses.
"""
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
csv_file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
@@ -36,7 +36,7 @@ async def main():
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="FileManipulation",
instructions="Find answers to the user's questions in the provided file.",
tools=code_interpreter_tools,
@@ -3,6 +3,7 @@ import asyncio
from typing import Annotated
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import AuthorRole, FunctionCallContent, FunctionResultContent
from semantic_kernel.contents.chat_message_content import ChatMessageContent
from semantic_kernel.functions import kernel_function
@@ -54,11 +55,11 @@ async def handle_intermediate_steps(message: ChatMessageContent) -> None:
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Define the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="Host",
instructions="Answer questions about the menu.",
)
@@ -3,6 +3,7 @@ import asyncio
from typing import Annotated
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import AuthorRole, FunctionCallContent, FunctionResultContent
from semantic_kernel.contents.chat_message_content import ChatMessageContent
from semantic_kernel.functions import kernel_function
@@ -54,11 +55,11 @@ async def handle_streaming_intermediate_steps(message: ChatMessageContent) -> No
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Define the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="Host",
instructions="Answer questions about the menu.",
)
@@ -2,6 +2,7 @@
import asyncio
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
"""
The following sample demonstrates how to create an OpenAI
@@ -14,11 +15,11 @@ ID and creating a new instance of the assistant using the retrieved definition.
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="Assistant",
instructions="You are a helpful assistant answering questions about the world in one sentence.",
)
@@ -3,6 +3,7 @@ import asyncio
from typing import Annotated
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import AuthorRole
from semantic_kernel.functions import kernel_function
@@ -38,11 +39,11 @@ class MenuPlugin:
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Define the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="Host",
instructions="Answer questions about the menu.",
)
@@ -4,6 +4,7 @@ import asyncio
from pydantic import BaseModel
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
"""
The following sample demonstrates how to create an OpenAI
@@ -36,7 +37,7 @@ json_schema = {
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name
name="Assistant",
instructions="You are a helpful assistant answering questions about the world in one sentence.",
response_format=json_schema,
@@ -52,11 +53,11 @@ class ResponseModel(BaseModel):
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="Assistant",
instructions="You are a helpful assistant answering questions about the world in one sentence.",
response_format=AzureAssistantAgent.configure_response_format(ResponseModel),
@@ -3,6 +3,7 @@
import asyncio
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.functions import KernelArguments
from semantic_kernel.prompt_template import PromptTemplateConfig
from semantic_kernel.prompt_template.const import TEMPLATE_FORMAT_TYPES
@@ -30,14 +31,14 @@ async def invoke_agent_with_template(
template_str: str, template_format: TEMPLATE_FORMAT_TYPES, default_style: str = "haiku"
):
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Configure the prompt template
prompt_template_config = PromptTemplateConfig(template=template_str, template_format=template_format)
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="MyPoetAgent",
)
@@ -4,6 +4,7 @@ import asyncio
import os
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import AuthorRole, ChatMessageContent, FileReferenceContent, ImageContent, TextContent
"""
@@ -16,7 +17,7 @@ and answer questions about them and provide streaming responses.
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))), "resources", "cat.jpg"
@@ -27,7 +28,7 @@ async def main():
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
instructions="Answer questions about the menu.",
name="Host",
)
@@ -0,0 +1,93 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import os
from semantic_kernel.agents import AgentRegistry, AzureResponsesAgent
"""
The following sample demonstrates how to create an Azure Responses Agent that answers
user questions using the file search tool.
"""
# Define the YAML string for the sample
spec = """
type: azure_responses
name: FileSearchAgent
description: Agent with file search tool.
instructions: >
Use the file search tool to answer questions from the user.
model:
id: ${AzureOpenAI:ChatModelId}
connection:
endpoint: ${AzureOpenAI:Endpoint}
tools:
- type: file_search
options:
vector_store_ids:
- ${AzureOpenAI:VectorStoreId}
"""
async def main():
# Setup the Azure OpenAI client
client = AzureResponsesAgent.create_client()
# Read and upload the file to the OpenAI AI service
pdf_file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
"resources",
"file_search",
"employees.pdf",
)
# Upload the pdf file to the server
with open(pdf_file_path, "rb") as file:
file = await client.files.create(file=file, purpose="assistants")
vector_store = await client.vector_stores.create(
name="responses_file_search",
file_ids=[file.id],
)
try:
# Create the Responses Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `AzureOpenAI:` prefix).
# The short-format is used here for brevity
agent: AzureResponsesAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
extras={"AzureOpenAI:VectorStoreId": vector_store.id},
)
# Define the task for the agent
TASK = "Who can help me if I have a sales question?"
print(f"# User: '{TASK}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=TASK,
):
print(f"# {response.name}: {response}")
finally:
# Cleanup: Delete the agent, vector store, and file
await client.vector_stores.delete(vector_store.id)
await client.files.delete(file.id)
"""
Sample output:
# User: 'Who can help me if I have a sales question?'
# FileSearchAgent: If you have a sales question, you may contact the following individuals:
1. **Hicran Bea** - Sales Manager
2. **Mariam Jaslyn** - Sales Representative
3. **Angelino Embla** - Sales Representative
This information comes from the employee records【4:0†source】.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,99 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import os
from typing import Annotated
from semantic_kernel.agents import AgentRegistry, AzureResponsesAgent
from semantic_kernel.functions import kernel_function
"""
The following sample demonstrates how to create an Azure Responses Agent that answers
user questions. The sample shows how to load a declarative spec from a file.
The plugins/functions must already exist in the kernel.
They are not created declaratively via the spec.
"""
class MenuPlugin:
"""A sample Menu Plugin used for the concept sample."""
@kernel_function(description="Provides a list of specials from the menu.")
def get_specials(self) -> Annotated[str, "Returns the specials from the menu."]:
return """
Special Soup: Clam Chowder
Special Salad: Cobb Salad
Special Drink: Chai Tea
"""
@kernel_function(description="Provides the price of the requested menu item.")
def get_item_price(
self, menu_item: Annotated[str, "The name of the menu item."]
) -> Annotated[str, "Returns the price of the menu item."]:
return "$9.99"
async def main():
try:
client = AzureResponsesAgent.create_client()
# Define the YAML file path for the sample
file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
"resources",
"declarative_spec",
"azure_responses_spec.yaml",
)
# Create the Responses Agent from the YAML spec
agent: AzureResponsesAgent = await AgentRegistry.create_from_file(
file_path,
plugins=[MenuPlugin()],
client=client,
)
# Create the agent
user_inputs = [
"Hello",
"What is the special soup?",
"How much does that cost?",
"Thank you",
]
# Create a thread for the agent
# If no thread is provided, a new thread will be
# created and returned with the initial response
thread = None
for user_input in user_inputs:
print(f"# User: '{user_input}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=user_input,
thread=thread,
):
print(f"# {response.name}: {response}")
# Store the thread for the next iteration
thread = response.thread
finally:
# Cleanup: Delete the thread
await thread.delete() if thread else None
"""
Sample Output:
# User: 'Hello'
# Host: Hi there! How can I assist you today?
# User: 'What is the special soup?'
# Host: The special soup is Clam Chowder.
# User: 'What is the special drink?'
# Host: The special drink is Chai Tea.
# User: 'How much is it?'
# Host: The Chai Tea costs $9.99.
# User: 'Thank you'
# Host: You're welcome! If you have any more questions, feel free to ask.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,75 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from semantic_kernel.agents import AgentRegistry, AzureResponsesAgent
"""
The following sample demonstrates how to create an Azure Responses Agent that invokes
a story generation task using a prompt template and a declarative spec.
"""
# Define the YAML string for the sample
spec = """
type: azure_responses
name: StoryAgent
description: An agent that generates a story about a topic.
instructions: Tell a story about {{$topic}} that is {{$length}} sentences long.
model:
id: ${AzureOpenAI:ChatModelId}
connection:
endpoint: ${AzureOpenAI:Endpoint}
inputs:
topic:
description: The topic of the story.
required: true
default: Cats
length:
description: The number of sentences in the story.
required: true
default: 2
outputs:
output1:
description: The generated story.
template:
format: semantic-kernel
"""
async def main():
# Setup the Azure OpenAI client
client = AzureResponsesAgent.create_client()
# Create the Responses Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `AzureOpenAI:` prefix).
# The short-format is used here for brevity
agent: AzureResponsesAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
)
USER_INPUTS = ["Tell me a fun story."]
# Invoke the agent for the specified task
for user_input in USER_INPUTS:
# Print the user input
print(f"# User: '{user_input}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=user_input,
):
print(f"# {response.name}: {response}")
"""
Sample output:
# User: 'Tell me a fun story.'
# StoryAgent: Late at night, a mischievous cat named Whiskers tiptoed across the piano keys,
accidentally composing a tune so catchy that all the neighborhood felines gathered outside
to dance. By morning, the humans awoke to find a crowd of cats meowing for an encore performance.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,100 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import os
from semantic_kernel.agents import AgentRegistry, OpenAIResponsesAgent
"""
The following sample demonstrates how to create an OpenAI Responses Agent that answers
user questions using the file search tool based on a declarative spec.
"""
# Define the YAML string for the sample
spec = """
type: openai_responses
name: FileSearchAgent
description: Agent with file search tool.
instructions: >
Find answers to the user's questions in the provided file.
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
tools:
- type: file_search
description: File search for document retrieval.
options:
vector_store_ids:
- ${OpenAI:VectorStoreId}
"""
async def main():
# Setup the OpenAI Responses client
client = OpenAIResponsesAgent.create_client()
# Read and upload the file to the OpenAI AI service
pdf_file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
"resources",
"file_search",
"employees.pdf",
)
# Upload the pdf file to the assistant service
with open(pdf_file_path, "rb") as file:
file = await client.files.create(file=file, purpose="assistants")
vector_store = await client.vector_stores.create(
name="assistant_file_search",
file_ids=[file.id],
)
try:
# Create the Responses Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `OpenAI:` prefix).
# The short-format is used here for brevity
agent: OpenAIResponsesAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
extras={"OpenAI:VectorStoreId": vector_store.id},
)
# Define the task for the agent
USER_INPUTS = ["Who can help me if I have a sales question?", "Who works in sales?"]
thread = None
for user_input in USER_INPUTS:
# Print the user input
print(f"# User: '{user_input}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=user_input,
thread=thread,
):
print(f"# {response.name}: {response}")
thread = response.thread
finally:
# Cleanup: Delete the vector store, and file
await client.vector_stores.delete(vector_store.id)
await client.files.delete(file.id)
"""
Sample output:
# User: 'Who can help me if I have a sales question?'
# FileSearchAgent: If you have a sales question, you may contact the following individuals:
1. **Hicran Bea** - Sales Manager
2. **Mariam Jaslyn** - Sales Representative
3. **Angelino Embla** - Sales Representative
This information comes from the employee records【4:0†source】.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,99 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import os
from typing import Annotated
from semantic_kernel.agents import AgentRegistry, OpenAIResponsesAgent
from semantic_kernel.functions import kernel_function
"""
The following sample demonstrates how to create an OpenAI Responses Agent that answers
user questions. The sample shows how to load a declarative spec from a file.
The plugins/functions must already exist in the kernel.
They are not created declaratively via the spec.
"""
class MenuPlugin:
"""A sample Menu Plugin used for the concept sample."""
@kernel_function(description="Provides a list of specials from the menu.")
def get_specials(self) -> Annotated[str, "Returns the specials from the menu."]:
return """
Special Soup: Clam Chowder
Special Salad: Cobb Salad
Special Drink: Chai Tea
"""
@kernel_function(description="Provides the price of the requested menu item.")
def get_item_price(
self, menu_item: Annotated[str, "The name of the menu item."]
) -> Annotated[str, "Returns the price of the menu item."]:
return "$9.99"
async def main():
try:
client = OpenAIResponsesAgent.create_client()
# Define the YAML file path for the sample
file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))),
"resources",
"declarative_spec",
"openai_responses_spec.yaml",
)
# Create the Responses Agent from the YAML spec
agent: OpenAIResponsesAgent = await AgentRegistry.create_from_file(
file_path,
plugins=[MenuPlugin()],
client=client,
)
# Create the agent
user_inputs = [
"Hello",
"What is the special soup?",
"How much does that cost?",
"Thank you",
]
# Create a thread for the agent
# If no thread is provided, a new thread will be
# created and returned with the initial response
thread = None
for user_input in user_inputs:
print(f"# User: '{user_input}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=user_input,
thread=thread,
):
print(f"# {response.name}: {response}")
# Store the thread for the next iteration
thread = response.thread
finally:
# Cleanup: Delete the thread
await thread.delete() if thread else None
"""
Sample Output:
# User: 'Hello'
# Host: Hi there! How can I assist you today?
# User: 'What is the special soup?'
# Host: The special soup is Clam Chowder.
# User: 'What is the special drink?'
# Host: The special drink is Chai Tea.
# User: 'How much is it?'
# Host: The Chai Tea costs $9.99.
# User: 'Thank you'
# Host: You're welcome! If you have any more questions, feel free to ask.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,73 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from semantic_kernel.agents import AgentRegistry, OpenAIResponsesAgent
"""
The following sample demonstrates how to create an OpenAI Responses Agent that invokes
a story generation task using a prompt template and a declarative spec.
"""
# Define the YAML string for the sample
spec = """
type: openai_responses
name: StoryAgent
description: An agent that generates a story about a topic.
instructions: Tell a story about {{$topic}} that is {{$length}} sentences long.
model:
id: ${OpenAI:ChatModelId}
inputs:
topic:
description: The topic of the story.
required: true
default: Cats
length:
description: The number of sentences in the story.
required: true
default: 2
outputs:
output1:
description: The generated story.
template:
format: semantic-kernel
"""
async def main():
# Setup the OpenAI client
client = OpenAIResponsesAgent.create_client()
# Create the Responses Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `OpenAI:` prefix).
# The short-format is used here for brevity
agent: OpenAIResponsesAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
)
USER_INPUTS = ["Tell me a fun story."]
# Invoke the agent for the specified task
for user_input in USER_INPUTS:
# Print the user input
print(f"# User: '{user_input}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=user_input,
):
print(f"# {response.name}: {response}")
"""
Sample output:
# User: 'Tell me a fun story.'
# StoryAgent: Late at night, a mischievous cat named Whiskers tiptoed across the piano keys,
accidentally composing a tune so catchy that all the neighborhood felines gathered outside
to dance. By morning, the humans awoke to find a crowd of cats meowing for an encore performance.
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,86 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from semantic_kernel.agents import AgentRegistry, OpenAIResponsesAgent
"""
The following sample demonstrates how to create an OpenAI Responses Agent that answers
user questions using the web search tool based on a declarative spec.
"""
# Define the YAML string for the sample
spec = """
type: openai_responses
name: WebSearchAgent
description: Agent with web search tool.
instructions: >
Find answers to the user's questions using the provided tool.
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
tools:
- type: web_search
description: Search the internet for recent information.
options:
search_context_size: high
"""
async def main():
# Setup the OpenAI client
client = OpenAIResponsesAgent.create_client()
try:
# Create the Responses Agent from the YAML spec
# Note: the extras can be provided in the short-format (shown below) or
# in the long-format (as shown in the YAML spec, with the `OpenAI:` prefix).
# The short-format is used here for brevity
agent: OpenAIResponsesAgent = await AgentRegistry.create_from_yaml(
yaml_str=spec,
client=client,
)
# Define the task for the agent
USER_INPUTS = ["Who won the 2025 NCAA basketball championship?"]
thread = None
for user_input in USER_INPUTS:
# Print the user input
print(f"# User: '{user_input}'")
# Invoke the agent for the specified task
async for response in agent.invoke(
messages=user_input,
thread=thread,
):
print(f"# {response.name}: {response}")
thread = response.thread
finally:
await thread.delete() if thread else None
"""
Sample output:
# User: 'Who won the 2025 NCAA basketball championship?'
# WebSearchAgent: The Florida Gators won the 2025 NCAA men's basketball championship, defeating the Houston
Cougars 65-63 on April 7, 2025, at the Alamodome in San Antonio, Texas. This victory marked Florida's
third national title and their first since 2007. ([reuters.com](https://www.reuters.com/sports/basketball/florida-beat-houston-claim-third-ncaa-mens-basketball-title-2025-04-08/?utm_source=openai))
In the championship game, Florida overcame a 12-point deficit in the second half. Senior guard Walter Clayton
Jr. was instrumental in the comeback, scoring all 11 of his points in the second half and delivering a
crucial defensive stop in the final seconds to secure the win. Will Richard led the Gators with 18 points. ([apnews.com](https://apnews.com/article/74a9c790277595ce53ca130c5ec64429?utm_source=openai))
Head coach Todd Golden, in his third season, became the youngest coach to win the NCAA title since 1983. ([reuters.com](https://www.reuters.com/sports/basketball/florida-beat-houston-claim-third-ncaa-mens-basketball-title-2025-04-08/?utm_source=openai))
## Florida Gators' 2025 NCAA Championship Victory:
- [Florida overcome Houston in massive comeback to claim third NCAA title](https://www.reuters.com/sports/basketball/florida-beat-houston-claim-third-ncaa-mens-basketball-title-2025-04-08/?utm_source=openai)
- [Walter Clayton Jr.'s defensive stop gives Florida its 3rd national title with 65-63 win over Houston](https://apnews.com/article/74a9c790277595ce53ca130c5ec64429?utm_source=openai)
- [Reports: National champion Florida sets White House visit](https://www.reuters.com/sports/reports-national-champion-florida-sets-white-house-visit-2025-05-18/?utm_source=openai)
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -3,6 +3,7 @@ import asyncio
import os
from semantic_kernel.agents import OpenAIResponsesAgent
from semantic_kernel.connectors.ai.open_ai import OpenAISettings
from semantic_kernel.contents.streaming_chat_message_content import StreamingChatMessageContent
"""
@@ -28,7 +29,7 @@ USER_INPUTS = [
async def main():
# 1. Create the client using OpenAI resources and configuration
client, model = OpenAIResponsesAgent.setup_resources()
client = OpenAIResponsesAgent.create_client()
pdf_file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.realpath(__file__))), "resources", "employees.pdf"
@@ -46,7 +47,7 @@ async def main():
# 2. Create a Semantic Kernel agent for the OpenAI Responses API
agent = OpenAIResponsesAgent(
ai_model_id=model,
ai_model_id=OpenAISettings().chat_model_id,
client=client,
instructions="Find answers to the user's questions in the provided file.",
name="FileSearch",
@@ -3,6 +3,7 @@ import asyncio
from typing import Annotated
from semantic_kernel.agents import AzureResponsesAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import AuthorRole, FunctionCallContent, FunctionResultContent
from semantic_kernel.contents.chat_message_content import ChatMessageContent
from semantic_kernel.functions import kernel_function
@@ -53,11 +54,11 @@ async def handle_intermediate_steps(message: ChatMessageContent) -> None:
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client, model = AzureResponsesAgent.setup_resources()
client = AzureResponsesAgent.create_client()
# 2. Create a Semantic Kernel agent for the OpenAI Responses API
agent = AzureResponsesAgent(
ai_model_id=model,
ai_model_id=AzureOpenAISettings().chat_deployment_name,
client=client,
name="Host",
instructions="Answer questions about the menu.",
@@ -3,6 +3,7 @@ import asyncio
from typing import Annotated
from semantic_kernel.agents import OpenAIResponsesAgent
from semantic_kernel.connectors.ai.open_ai import OpenAISettings
from semantic_kernel.contents import AuthorRole, FunctionCallContent, FunctionResultContent
from semantic_kernel.contents.chat_message_content import ChatMessageContent
from semantic_kernel.functions import kernel_function
@@ -55,11 +56,11 @@ async def handle_streaming_intermediate_steps(message: ChatMessageContent) -> No
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client, model = OpenAIResponsesAgent.setup_resources()
client = OpenAIResponsesAgent.create_client()
# 2. Create a Semantic Kernel agent for the OpenAI Responses API
agent = OpenAIResponsesAgent(
ai_model_id=model,
ai_model_id=OpenAISettings().chat_model_id,
client=client,
name="Host",
instructions="Answer questions about the menu.",
@@ -3,6 +3,7 @@ import asyncio
from typing import Annotated
from semantic_kernel.agents import AzureResponsesAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.functions import kernel_function
"""
@@ -49,11 +50,11 @@ USER_INPUTS = [
async def main():
# 1. Create the client using OpenAI resources and configuration
client, model = AzureResponsesAgent.setup_resources()
client = AzureResponsesAgent.create_client()
# 2. Create a Semantic Kernel agent for the OpenAI Responses API
agent = AzureResponsesAgent(
ai_model_id=model,
ai_model_id=AzureOpenAISettings().chat_deployment_name,
client=client,
instructions="Answer questions about the menu.",
name="Host",
@@ -1,8 +1,8 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from semantic_kernel.agents import AzureResponsesAgent
from semantic_kernel.agents.open_ai.openai_responses_agent import ResponsesAgentThread
from semantic_kernel.agents import AzureResponsesAgent, ResponsesAgentThread
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
"""
The following sample demonstrates how to create an OpenAI Responses Agent.
@@ -24,13 +24,11 @@ USER_INPUTS = [
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client, model = AzureResponsesAgent.setup_resources()
print(f"Using model: {model}")
client = AzureResponsesAgent.create_client()
# 2. Create a Semantic Kernel agent for the OpenAI Responses API
agent = AzureResponsesAgent(
ai_model_id=model,
ai_model_id=AzureOpenAISettings().chat_deployment_name,
client=client,
instructions="Answer questions about from the user.",
name="Joker",
@@ -2,6 +2,7 @@
import asyncio
from semantic_kernel.agents import OpenAIResponsesAgent
from semantic_kernel.connectors.ai.open_ai import OpenAISettings
"""
The following sample demonstrates how to create an OpenAI Responses Agent.
@@ -24,13 +25,13 @@ USER_INPUTS = [
async def main():
# 1. Create the client using OpenAI resources and configuration
client, model = OpenAIResponsesAgent.setup_resources()
client = OpenAIResponsesAgent.create_client()
web_search_tool = OpenAIResponsesAgent.configure_web_search_tool()
# 2. Create a Semantic Kernel agent for the OpenAI Responses API
agent = OpenAIResponsesAgent(
ai_model_id=model,
ai_model_id=OpenAISettings().chat_model_id,
client=client,
instructions="Answer questions from the user.",
name="Host",
@@ -9,6 +9,7 @@ from pydantic import BaseModel, Field
from semantic_kernel import Kernel
from semantic_kernel.agents import OpenAIResponsesAgent
from semantic_kernel.connectors.ai.open_ai import OpenAISettings
from semantic_kernel.functions import kernel_function
from semantic_kernel.processes import ProcessBuilder
from semantic_kernel.processes.kernel_process import (
@@ -49,11 +50,11 @@ user queries and providing detailed outputs.
# 1) Helper to run OpenAI agent
#
async def run_openai_agent(instructions: str, prompt: str, agent_name: str = "GenericAgent") -> str:
client, model = OpenAIResponsesAgent.setup_resources()
client = OpenAIResponsesAgent.create_client()
agent_tools = [OpenAIResponsesAgent.configure_web_search_tool()]
agent = OpenAIResponsesAgent(
ai_model_id=model,
ai_model_id=OpenAISettings().responses_model_id,
client=client,
instructions=instructions,
name=agent_name,
@@ -0,0 +1,16 @@
type: azure_assistant
name: Host
description: This agent uses the provided functions to answer questions about the menu.
instructions: Use the provided functions to answer questions about the menu.
model:
id: ${AzureOpenAI:ChatModelId}
connection:
connection:
endpoint: ${AzureOpenAI:Endpoint}
options:
temperature: 0.4
tools:
- id: MenuPlugin.get_specials
type: function
- id: MenuPlugin.get_item_price
type: function
@@ -0,0 +1,16 @@
type: azure_responses
name: Host
description: This agent uses the provided functions to answer questions about the menu.
instructions: Use the provided functions to answer questions about the menu.
model:
id: ${AzureOpenAI:ChatModelId}
connection:
connection:
endpoint: ${AzureOpenAI:Endpoint}
options:
temperature: 0.4
tools:
- id: MenuPlugin.get_specials
type: function
- id: MenuPlugin.get_item_price
type: function
@@ -0,0 +1,15 @@
type: openai_assistant
name: Host
description: This agent uses the provided functions to answer questions about the menu.
instructions: Use the provided functions to answer questions about the menu.
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
options:
temperature: 0.4
tools:
- id: MenuPlugin.get_specials
type: function
- id: MenuPlugin.get_item_price
type: function
@@ -0,0 +1,15 @@
type: openai_responses
name: Host
description: This agent uses the provided functions to answer questions about the menu.
instructions: Use the provided functions to answer questions about the menu.
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
options:
temperature: 0.4
tools:
- id: MenuPlugin.get_specials
type: function
- id: MenuPlugin.get_item_price
type: function
@@ -59,6 +59,7 @@ Example|Description
[step3_assistant_vision](../getting_started_with_agents/openai_assistant/step3_assistant_vision.py)|How to provide an image as input to an OpenAI Assistant agent.
[step4_assistant_tool_code_interpreter](../getting_started_with_agents/openai_assistant/step4_assistant_tool_code_interpreter.py)|How to use the code-interpreter tool for an OpenAI Assistant agent.
[step5_assistant_tool_file_search](../getting_started_with_agents/openai_assistant/step5_assistant_tool_file_search.py)|How to use the file-search tool for an OpenAI Assistant agent.
[step6_assistant](../getting_started_with_agents/openai_assistant/step6_assistant_declarative.py)|How to create an Assistant Agent from a declarative spec.
## OpenAI Responses Agent
@@ -71,6 +72,7 @@ Example|Description
[step5_responses_agent_file_search](../getting_started_with_agents/openai_responses/step5_responses_agent_file_search.py)|How to use the file-search tool with an OpenAI Responses agent.
[step6_responses_agent_vision](../getting_started_with_agents/openai_responses/step6_responses_agent_vision.py)|How to provide an image as input to an OpenAI Responses agent.
[step7_responses_agent_structured_outputs](../getting_started_with_agents/openai_responses/step7_responses_agent_structured_outputs.py)|How to use have an OpenAI Responses agent use structured outputs.
[step8_assistant](../getting_started_with_agents/openai_responses/step8_responses_agent_declarative.py)|How to create a Responses Agent from a declarative spec.
## Multi-Agent Orchestration
@@ -10,7 +10,7 @@ from semantic_kernel.agents import (
StandardMagenticManager,
)
from semantic_kernel.agents.runtime import InProcessRuntime
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAISettings
from semantic_kernel.contents import ChatMessageContent
"""
@@ -47,10 +47,10 @@ async def agents() -> list[Agent]:
)
# Create an OpenAI Assistant agent with code interpreter capability
client, model = OpenAIAssistantAgent.setup_resources()
client = OpenAIAssistantAgent.create_client()
code_interpreter_tool, code_interpreter_tool_resources = OpenAIAssistantAgent.configure_code_interpreter_tool()
definition = await client.beta.assistants.create(
model=model,
model=OpenAISettings().chat_model_id,
name="CoderAgent",
description="A helpful assistant that writes and executes code to process and analyze data.",
instructions="You solve questions using code. Please provide detailed analysis and computation process.",
@@ -30,13 +30,14 @@ OpenAI Assistant Agents are created in the following way:
```python
from semantic_kernel.agents import OpenAIAssistantAgent
from semantic_kernel.connectors.ai.open_ai import OpenAISettings
# Create the client using OpenAI resources and configuration
client, model = OpenAIAssistantAgent.setup_resources()
client = OpenAIAssistantAgent.create_client()
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=OpenAISettings().chat_model_id,
instructions="<instructions>",
name="<name>",
)
@@ -78,7 +79,7 @@ Alternatively, you can pass the `api_version` parameter when creating an `AzureA
from semantic_kernel.agents import AzureAssistantAgent
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Create the assistant definition
definition = await client.beta.assistants.create(
@@ -2,6 +2,7 @@
import asyncio
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
"""
The following sample demonstrates how to create an OpenAI assistant using either
@@ -25,11 +26,11 @@ USER_INPUTS = [
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# 2. Create the assistant on the Azure OpenAI service
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
instructions="Answer questions about the world in one sentence.",
name="Assistant",
)
@@ -3,6 +3,7 @@ import asyncio
from typing import Annotated
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.functions import kernel_function
"""
@@ -44,11 +45,11 @@ USER_INPUTS = [
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# 2. Create the assistant on the Azure OpenAI service
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
instructions="Answer questions about the menu.",
name="Host",
)
@@ -4,6 +4,7 @@ import asyncio
import os
from semantic_kernel.agents import AssistantAgentThread, OpenAIAssistantAgent
from semantic_kernel.connectors.ai.open_ai import OpenAISettings
from semantic_kernel.contents import AuthorRole, ChatMessageContent, FileReferenceContent, ImageContent, TextContent
"""
@@ -17,7 +18,7 @@ and answer questions about them. This sample uses non-streaming responses.
async def main():
# 1. Create the OpenAI Assistant Agent client
# Note Azure OpenAI doesn't support vision files yet
client, model = OpenAIAssistantAgent.setup_resources()
client = OpenAIAssistantAgent.create_client()
# 2. Load a sample image of a cat used for the assistant to describe
file_path = os.path.join(os.path.dirname(os.path.dirname(os.path.realpath(__file__))), "resources", "cat.jpg")
@@ -27,7 +28,7 @@ async def main():
# 3. Create the assistant on the OpenAI service
definition = await client.beta.assistants.create(
model=model,
model=OpenAISettings().chat_model_id,
instructions="Answer questions about the provided images.",
name="Vision",
)
@@ -2,6 +2,7 @@
import asyncio
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
"""
The following sample demonstrates how to create an OpenAI
@@ -15,14 +16,14 @@ TASK = "Use code to determine the values in the Fibonacci sequence that that are
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# 2. Configure the code interpreter tool and resources for the Assistant
code_interpreter_tool, code_interpreter_tool_resources = AzureAssistantAgent.configure_code_interpreter_tool()
# 3. Create the assistant on the Azure OpenAI service
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
name="CodeRunner",
instructions="Run the provided request as code and return the result.",
tools=code_interpreter_tool,
@@ -4,6 +4,7 @@ import asyncio
import os
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
"""
The following sample demonstrates how to create an OpenAI
@@ -21,7 +22,7 @@ USER_INPUTS = {
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# 2. Read and upload the file to the Azure OpenAI assistant service
pdf_file_path = os.path.join(
@@ -41,7 +42,7 @@ async def main():
# 4. Create the assistant on the Azure OpenAI service with the file search tool
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
instructions="Find answers to the user's questions in the provided file.",
name="FileSearch",
tools=file_search_tool,
@@ -0,0 +1,98 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from typing import Annotated
from semantic_kernel.agents import AgentRegistry, OpenAIAssistantAgent
from semantic_kernel.functions import kernel_function
"""
The following sample demonstrates how to create an OpenAI Assistant agent that answers
questions about a sample menu using a Semantic Kernel Plugin. The agent is created
using a yaml declarative spec.
"""
# Simulate a conversation with the agent
USER_INPUTS = [
"Hello",
"What is the special soup?",
"How much does that cost?",
"Thank you",
]
# Define the YAML string for the sample
SPEC = """
type: openai_assistant
name: Host
instructions: Respond politely to the user's questions.
model:
id: ${OpenAI:ChatModelId}
tools:
- id: MenuPlugin.get_specials
type: function
- id: MenuPlugin.get_item_price
type: function
"""
# Define a sample plugin for the sample
class MenuPlugin:
"""A sample Menu Plugin used for the concept sample."""
@kernel_function(description="Provides a list of specials from the menu.")
def get_specials(self) -> Annotated[str, "Returns the specials from the menu."]:
return """
Special Soup: Clam Chowder
Special Salad: Cobb Salad
Special Drink: Chai Tea
"""
@kernel_function(description="Provides the price of the requested menu item.")
def get_item_price(
self, menu_item: Annotated[str, "The name of the menu item."]
) -> Annotated[str, "Returns the price of the menu item."]:
return "$9.99"
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client = OpenAIAssistantAgent.create_client()
# 2. Create the assistant on the Azure OpenAI service
agent: OpenAIAssistantAgent = await AgentRegistry.create_from_yaml(
SPEC,
plugins=[MenuPlugin()],
client=client,
)
# 3. Create a thread for the agent
# If no thread is provided, a new thread will be
# created and returned with the initial response
thread = None
try:
for user_input in USER_INPUTS:
print(f"# User: {user_input}")
# 4. Invoke the agent for the specified thread for response
async for response in agent.invoke(
messages=user_input,
thread=thread,
):
print(f"# {response.name}: {response}")
thread = response.thread
finally:
# 5. Clean up the resources
await thread.delete() if thread else None
await agent.client.beta.assistants.delete(assistant_id=agent.id)
"""
Sample Output:
# User: Hello
# Agent: Hello! How can I assist you today?
# User: What is the special soup?
# ...
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -2,6 +2,7 @@
import asyncio
from semantic_kernel.agents import AzureResponsesAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
"""
The following sample demonstrates how to create an OpenAI Responses Agent using either
@@ -28,11 +29,11 @@ USER_INPUTS = [
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client, model = AzureResponsesAgent.setup_resources()
client = AzureResponsesAgent.create_client()
# 2. Create a Semantic Kernel agent for the OpenAI Responses API
agent = AzureResponsesAgent(
ai_model_id=model,
ai_model_id=AzureOpenAISettings().responses_deployment_name,
client=client,
instructions="Answer questions about the world in one sentence.",
name="Expert",
@@ -2,6 +2,7 @@
import asyncio
from semantic_kernel.agents import AzureResponsesAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
"""
The following sample demonstrates how to create an OpenAI Responses Agent.
@@ -24,13 +25,11 @@ USER_INPUTS = [
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client, model = AzureResponsesAgent.setup_resources()
print(f"Using model: {model}")
client = AzureResponsesAgent.create_client()
# 2. Create a Semantic Kernel agent for the OpenAI Responses API
agent = AzureResponsesAgent(
ai_model_id=model,
ai_model_id=AzureOpenAISettings().responses_deployment_name,
client=client,
instructions="Answer questions about from the user.",
name="Joker",
@@ -3,6 +3,7 @@ import asyncio
from typing import Annotated
from semantic_kernel.agents import AzureResponsesAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.functions import kernel_function
"""
@@ -48,11 +49,11 @@ USER_INPUTS = [
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client, model = AzureResponsesAgent.setup_resources()
client = AzureResponsesAgent.create_client()
# 2. Create a Semantic Kernel agent for the OpenAI Responses API
agent = AzureResponsesAgent(
ai_model_id=model,
ai_model_id=AzureOpenAISettings().responses_deployment_name,
client=client,
instructions="Answer questions about the menu.",
name="Host",
@@ -2,6 +2,7 @@
import asyncio
from semantic_kernel.agents import OpenAIResponsesAgent
from semantic_kernel.connectors.ai.open_ai import OpenAISettings
"""
The following sample demonstrates how to create an OpenAI Responses Agent.
@@ -25,13 +26,13 @@ USER_INPUTS = [
async def main():
# 1. Create the client using OpenAI resources and configuration
# Note: the Azure OpenAI Responses API does not yet support the web search tool.
client, model = OpenAIResponsesAgent.setup_resources()
client = OpenAIResponsesAgent.create_client()
web_search_tool = OpenAIResponsesAgent.configure_web_search_tool()
# 2. Create a Semantic Kernel agent for the OpenAI Responses API
agent = OpenAIResponsesAgent(
ai_model_id=model,
ai_model_id=OpenAISettings().responses_model_id,
client=client,
instructions="Answer questions from the user about performing web searches for news.",
name="NewsSearcher",
@@ -3,6 +3,7 @@ import asyncio
import os
from semantic_kernel.agents import AzureResponsesAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
"""
The following sample demonstrates how to create an OpenAI Responses Agent.
@@ -27,7 +28,7 @@ USER_INPUTS = [
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client, model = AzureResponsesAgent.setup_resources()
client = AzureResponsesAgent.create_client()
pdf_file_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.realpath(__file__))), "resources", "employees.pdf"
@@ -45,7 +46,7 @@ async def main():
# 2. Create a Semantic Kernel agent for the OpenAI Responses API
agent = AzureResponsesAgent(
ai_model_id=model,
ai_model_id=AzureOpenAISettings().responses_deployment_name,
client=client,
instructions="Find answers to the user's questions in the provided file.",
name="FileSearch",
@@ -4,6 +4,7 @@ import asyncio
import os
from semantic_kernel.agents import OpenAIResponsesAgent
from semantic_kernel.connectors.ai.open_ai import OpenAISettings
from semantic_kernel.contents import ChatMessageContent
from semantic_kernel.contents.image_content import ImageContent
from semantic_kernel.contents.text_content import TextContent
@@ -22,14 +23,14 @@ maintain the conversation history if conversation context is desired.
async def main():
# 1. Create the client using OpenAI resources and configuration
client, model = OpenAIResponsesAgent.setup_resources()
client = OpenAIResponsesAgent.create_client()
# 2. Define a file path for an image that will be used in the conversation
file_path = os.path.join(os.path.dirname(os.path.dirname(os.path.realpath(__file__))), "resources", "cat.jpg")
# 3. Create a Semantic Kernel agent for the OpenAI Responses API
agent = OpenAIResponsesAgent(
ai_model_id=model,
ai_model_id=OpenAISettings().responses_model_id,
client=client,
instructions="Answer questions about the provided images.",
name="VisionAgent",
@@ -6,6 +6,7 @@ import json
from pydantic import BaseModel
from semantic_kernel.agents import OpenAIResponsesAgent
from semantic_kernel.connectors.ai.open_ai import OpenAISettings
"""
The following sample demonstrates how to create an OpenAI Responses Agent.
@@ -34,11 +35,11 @@ class Reasoning(BaseModel):
async def main():
# 1. Create the client using OpenAI resources and configuration
# Note: the Azure OpenAI Responses API does not yet support structured outputs.
client, model = OpenAIResponsesAgent.setup_resources()
client = OpenAIResponsesAgent.create_client()
# 2. Create a Semantic Kernel agent for the OpenAI Responses API
agent = OpenAIResponsesAgent(
ai_model_id=model,
ai_model_id=OpenAISettings().responses_model_id,
client=client,
instructions="Answer the user's questions.",
name="StructuredOutputsAgent",
@@ -0,0 +1,97 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from typing import Annotated
from semantic_kernel.agents import AgentRegistry, OpenAIResponsesAgent
from semantic_kernel.functions import kernel_function
"""
The following sample demonstrates how to create an OpenAI Assistant agent that answers
questions about a sample menu using a Semantic Kernel Plugin. The agent is created
using a yaml declarative spec.
"""
# Simulate a conversation with the agent
USER_INPUTS = [
"Hello",
"What is the special soup?",
"How much does that cost?",
"Thank you",
]
# Define the YAML string for the sample
SPEC = """
type: openai_responses
name: Host
instructions: Respond politely to the user's questions.
model:
id: ${OpenAI:ChatModelId}
tools:
- id: MenuPlugin.get_specials
type: function
- id: MenuPlugin.get_item_price
type: function
"""
# Define a sample plugin for the sample
class MenuPlugin:
"""A sample Menu Plugin used for the concept sample."""
@kernel_function(description="Provides a list of specials from the menu.")
def get_specials(self) -> Annotated[str, "Returns the specials from the menu."]:
return """
Special Soup: Clam Chowder
Special Salad: Cobb Salad
Special Drink: Chai Tea
"""
@kernel_function(description="Provides the price of the requested menu item.")
def get_item_price(
self, menu_item: Annotated[str, "The name of the menu item."]
) -> Annotated[str, "Returns the price of the menu item."]:
return "$9.99"
async def main():
# 1. Create the client using Azure OpenAI resources and configuration
client = OpenAIResponsesAgent.create_client()
# 2. Create the assistant on the Azure OpenAI service
agent: OpenAIResponsesAgent = await AgentRegistry.create_from_yaml(
SPEC,
plugins=[MenuPlugin()],
client=client,
)
# 3. Create a thread for the agent
# If no thread is provided, a new thread will be
# created and returned with the initial response
thread = None
try:
for user_input in USER_INPUTS:
print(f"# User: {user_input}")
# 4. Invoke the agent for the specified thread for response
async for response in agent.invoke(
messages=user_input,
thread=thread,
):
print(f"# {response.name}: {response}")
thread = response.thread
finally:
# 5. Clean up the resources
await thread.delete() if thread else None
"""
Sample Output:
# User: Hello
# Agent: Hello! How can I assist you today?
# User: What is the special soup?
# ...
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -5,6 +5,7 @@ import logging
import os
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import StreamingFileReferenceContent
logging.basicConfig(level=logging.ERROR)
@@ -65,7 +66,7 @@ async def download_response_image(agent: AzureAssistantAgent, file_ids: list[str
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Upload the files to the client
file_ids: list[str] = []
@@ -81,7 +82,7 @@ async def main():
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
instructions="""
Analyze the available data to provide an answer to the user's question.
Always format response using markdown.
@@ -4,6 +4,7 @@ import asyncio
import os
from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings
from semantic_kernel.contents import StreamingAnnotationContent
"""
@@ -34,7 +35,7 @@ filenames = [
async def main():
# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
# Upload the files to the client
file_ids: list[str] = []
@@ -55,7 +56,7 @@ async def main():
# Create the assistant definition
definition = await client.beta.assistants.create(
model=model,
model=AzureOpenAISettings().chat_deployment_name,
instructions="""
The document store contains the text of fictional stories.
Always analyze the document store to provide an answer to the user's question.
+2 -2
View File
@@ -16,7 +16,7 @@ _AGENTS = {
"AzureAIAgentSettings": ".azure_ai.azure_ai_agent_settings",
"AzureAIAgentThread": ".azure_ai.azure_ai_agent",
"AzureAssistantAgent": ".open_ai.azure_assistant_agent",
"AssistantAgentThread": ".open_ai.open_ai_assistant_agent",
"AssistantAgentThread": ".open_ai.openai_assistant_agent",
"AzureResponsesAgent": ".open_ai.azure_responses_agent",
"BedrockAgent": ".bedrock.bedrock_agent",
"BedrockAgentThread": ".bedrock.bedrock_agent",
@@ -27,7 +27,7 @@ _AGENTS = {
"CopilotStudioAgentSettings": ".copilot_studio.copilot_studio_agent_settings",
"CopilotStudioAgentThread": ".copilot_studio.copilot_studio_agent",
"DeclarativeSpecMixin": ".agent",
"OpenAIAssistantAgent": ".open_ai.open_ai_assistant_agent",
"OpenAIAssistantAgent": ".open_ai.openai_assistant_agent",
"OpenAIResponsesAgent": ".open_ai.openai_responses_agent",
"ModelConnection": ".agent",
"ModelSpec": ".agent",
+1 -1
View File
@@ -23,7 +23,7 @@ from .group_chat.agent_chat import AgentChat
from .group_chat.agent_group_chat import AgentGroupChat
from .open_ai.azure_assistant_agent import AzureAssistantAgent
from .open_ai.azure_responses_agent import AzureResponsesAgent
from .open_ai.open_ai_assistant_agent import AssistantAgentThread, OpenAIAssistantAgent
from .open_ai.openai_assistant_agent import AssistantAgentThread, OpenAIAssistantAgent
from .open_ai.openai_responses_agent import OpenAIResponsesAgent, ResponsesAgentThread
from .open_ai.run_polling_options import RunPollingOptions
from .orchestration.concurrent import ConcurrentOrchestration
+50 -10
View File
@@ -19,7 +19,6 @@ from semantic_kernel.contents.utils.author_role import AuthorRole
from semantic_kernel.exceptions.agent_exceptions import AgentExecutionException, AgentInitializationException
from semantic_kernel.functions import kernel_function
from semantic_kernel.functions.kernel_arguments import KernelArguments
from semantic_kernel.functions.kernel_function import TEMPLATE_FORMAT_MAP
from semantic_kernel.functions.kernel_plugin import KernelPlugin
from semantic_kernel.kernel import Kernel
from semantic_kernel.kernel_pydantic import KernelBaseModel
@@ -44,6 +43,21 @@ TThreadType = TypeVar("TThreadType", bound="AgentThread")
# region Declarative Spec Definitions
class InputSpec(KernelBaseModel):
"""Class representing an input specification."""
description: str | None = None
required: bool = False
default: Any = None
class OutputSpec(KernelBaseModel):
"""Class representing an output specification."""
description: str | None = None
type: str | None = None
class ModelConnection(KernelBaseModel):
"""Class representing a model connection."""
@@ -83,6 +97,8 @@ class AgentSpec(KernelBaseModel):
tools: list[ToolSpec] = Field(default_factory=list)
template: dict[str, Any] | None = None
extras: dict[str, Any] = Field(default_factory=dict)
inputs: dict[str, InputSpec] = Field(default_factory=dict)
outputs: dict[str, OutputSpec] = Field(default_factory=dict)
# endregion
@@ -642,6 +658,10 @@ _BUILTIN_AGENTS_LOCK = threading.Lock()
_BUILTIN_AGENT_MODULES = [
"semantic_kernel.agents.chat_completion.chat_completion_agent",
"semantic_kernel.agents.azure_ai.azure_ai_agent",
"semantic_kernel.agents.open_ai.openai_assistant_agent",
"semantic_kernel.agents.open_ai.azure_assistant_agent",
"semantic_kernel.agents.open_ai.openai_responses_agent",
"semantic_kernel.agents.open_ai.azure_responses_agent",
]
@@ -754,6 +774,7 @@ class AgentRegistry:
kernel: Kernel | None = None,
plugins: list[KernelPlugin | object] | dict[str, KernelPlugin | object] | None = None,
settings: "KernelBaseSettings | None" = None,
extras: dict[str, Any] | None = None,
**kwargs,
) -> _TAgent:
"""Create a single agent instance from a dictionary.
@@ -763,6 +784,7 @@ class AgentRegistry:
kernel: The Kernel instance to use for tool resolution and agent initialization.
plugins: The plugins to use for the agent.
settings: The settings to use for the agent.
extras: Additional parameters to resolve placeholders in the YAML.
**kwargs: Additional parameters passed to the agent constructor if required.
Returns:
@@ -794,7 +816,9 @@ class AgentRegistry:
return await agent_cls.from_dict(
data,
kernel=kernel,
plugins=plugins,
settings=settings,
extras=extras,
**kwargs,
)
@@ -895,12 +919,11 @@ class DeclarativeSpecMixin(ABC):
**kwargs,
) -> _D:
"""Default implementation: call the protected _from_dict."""
# Compose `data` and extracted common fields for the subclass
extracted, kernel = cls._normalize_spec_fields(data, kernel=kernel, plugins=plugins, **kwargs)
return await cls._from_dict(
{**data, **extracted},
kernel=kernel,
prompt_template_config=prompt_template_config,
prompt_template_config=extracted.get("prompt_template"),
settings=settings,
**kwargs,
)
@@ -959,11 +982,27 @@ class DeclarativeSpecMixin(ABC):
if "tools" in data:
cls._validate_tools(data["tools"], kernel)
model_options = data.get("model", {}).get("options", {}) if data.get("model") else {}
inputs = data.get("inputs", {})
input_defaults = {
k: v.get("default")
for k, v in (inputs.items() if isinstance(inputs, dict) else [])
if v.get("default") is not None
}
# Start with model options
arguments = KernelArguments(**model_options)
# Update with input defaults (only if not already provided by model options)
for k, v in input_defaults.items():
if k not in arguments:
arguments[k] = v
fields = {
"name": data.get("name"),
"description": data.get("description"),
"instructions": data.get("instructions"),
"arguments": KernelArguments(**(data.get("model", {}).get("options", {}))) if data.get("model") else None,
"arguments": arguments,
}
# Handle prompt_template if available
@@ -971,12 +1010,13 @@ class DeclarativeSpecMixin(ABC):
template_data = data.get("prompt_template") or data.get("template")
if isinstance(template_data, dict):
prompt_template_config = PromptTemplateConfig(**template_data)
fields["prompt_template"] = TEMPLATE_FORMAT_MAP[prompt_template_config.template_format](
prompt_template_config=prompt_template_config
)
# Overwrite instructions from prompt template if explicitly provided
if prompt_template_config.template is not None:
fields["instructions"] = prompt_template_config.template
# If 'instructions' is set in YAML, override the template field in config
instructions = data.get("instructions")
if instructions is not None:
prompt_template_config.template = instructions
fields["prompt_template"] = prompt_template_config
# Always set fields["instructions"] to the template being used
fields["instructions"] = prompt_template_config.template
return fields, kernel
@@ -487,6 +487,11 @@ class AzureAIAgent(DeclarativeSpecMixin, Agent):
if "settings" in kwargs:
kwargs.pop("settings")
args = data.pop("arguments", None)
arguments = None
if args:
arguments = KernelArguments(**args)
if spec.id:
existing_definition = await client.agents.get_agent(spec.id)
@@ -510,6 +515,7 @@ class AzureAIAgent(DeclarativeSpecMixin, Agent):
client=client,
kernel=kernel,
prompt_template_config=prompt_template_config,
arguments=arguments,
**kwargs,
)
@@ -539,6 +545,7 @@ class AzureAIAgent(DeclarativeSpecMixin, Agent):
definition=agent_definition,
client=client,
kernel=kernel,
arguments=arguments,
prompt_template_config=prompt_template_config,
**kwargs,
)
@@ -55,7 +55,7 @@ class OpenAIAssistantChannel(AgentChannel):
Yields:
tuple[bool, ChatMessageContent]: The conversation messages.
"""
from semantic_kernel.agents.open_ai.open_ai_assistant_agent import OpenAIAssistantAgent
from semantic_kernel.agents.open_ai.openai_assistant_agent import OpenAIAssistantAgent
if not isinstance(agent, OpenAIAssistantAgent):
raise AgentChatException(f"Agent is not of the expected type {type(OpenAIAssistantAgent)}.")
@@ -77,7 +77,7 @@ class OpenAIAssistantChannel(AgentChannel):
Yields:
tuple[bool, StreamingChatMessageContent]: The conversation messages.
"""
from semantic_kernel.agents.open_ai.open_ai_assistant_agent import OpenAIAssistantAgent
from semantic_kernel.agents.open_ai.openai_assistant_agent import OpenAIAssistantAgent
if not isinstance(agent, OpenAIAssistantAgent):
raise AgentChatException(f"Agent is not of the expected type {type(OpenAIAssistantAgent)}.")
@@ -49,7 +49,7 @@ if TYPE_CHECKING:
from openai.types.beta.threads.run import Run
from openai.types.beta.threads.run_create_params import AdditionalMessageAttachmentTool, TruncationStrategy
from semantic_kernel.agents.open_ai.open_ai_assistant_agent import OpenAIAssistantAgent
from semantic_kernel.agents.open_ai.openai_assistant_agent import OpenAIAssistantAgent
from semantic_kernel.contents.chat_history import ChatHistory
from semantic_kernel.contents.chat_message_content import ChatMessageContent
from semantic_kernel.contents.function_call_content import FunctionCallContent
@@ -1,25 +1,50 @@
# Copyright (c) Microsoft. All rights reserved.
import sys
from collections.abc import Awaitable, Callable
from copy import copy
from typing import Any
from typing import TYPE_CHECKING, Any
from openai import AsyncAzureOpenAI
from pydantic import ValidationError
from semantic_kernel.agents import OpenAIAssistantAgent
from semantic_kernel.agents.agent import register_agent_type
from semantic_kernel.connectors.ai.open_ai.settings.azure_open_ai_settings import AzureOpenAISettings
from semantic_kernel.exceptions.agent_exceptions import AgentInitializationException
from semantic_kernel.utils.authentication.entra_id_authentication import get_entra_auth_token
from semantic_kernel.utils.feature_stage_decorator import release_candidate
from semantic_kernel.utils.telemetry.user_agent import APP_INFO, prepend_semantic_kernel_to_user_agent
if TYPE_CHECKING:
from semantic_kernel.kernel_pydantic import KernelBaseSettings
if sys.version_info >= (3, 12):
from typing import override # pragma: no cover
else:
from typing_extensions import override # pragma: no cover
if sys.version < "3.11":
from typing_extensions import Self # pragma: no cover
else:
from typing import Self # type: ignore # pragma: no cover
if sys.version_info >= (3, 13):
from warnings import deprecated
else:
from typing_extensions import deprecated
@release_candidate
@register_agent_type("azure_assistant")
class AzureAssistantAgent(OpenAIAssistantAgent):
"""An Azure Assistant Agent class that extends the OpenAI Assistant Agent class."""
@staticmethod
@deprecated(
"setup_resources is deprecated. Use AzureAssistantAgent.create_client() instead. This method will be removed by 2025-06-15." # noqa: E501
)
def setup_resources(
*,
ad_token: str | None = None,
@@ -108,3 +133,147 @@ class AzureAssistantAgent(OpenAIAssistantAgent):
)
return client, azure_openai_settings.chat_deployment_name
@staticmethod
def create_client(
*,
ad_token: str | None = None,
ad_token_provider: Callable[[], str | Awaitable[str]] | None = None,
api_key: str | None = None,
api_version: str | None = None,
base_url: str | None = None,
default_headers: dict[str, str] | None = None,
deployment_name: str | None = None,
endpoint: str | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
token_scope: str | None = None,
**kwargs: Any,
) -> AsyncAzureOpenAI:
"""A method to create the Azure OpenAI client.
Any arguments provided will override the values in the environment variables/environment file.
Args:
ad_token: The Microsoft Entra (previously Azure AD) token represented as a string
ad_token_provider: The Microsoft Entra (previously Azure AD) token provider provided as a callback
api_key: The API key
api_version: The API version
base_url: The base URL in the form https://<resource>.azure.openai.com/openai/deployments/<deployment_name>
default_headers: The default headers to add to the client
deployment_name: The deployment name
endpoint: The endpoint in the form https://<resource>.azure.openai.com
env_file_path: The environment file path
env_file_encoding: The environment file encoding, defaults to utf-8
token_scope: The token scope
kwargs: Additional keyword arguments
Returns:
An Azure OpenAI client instance.
"""
try:
azure_openai_settings = AzureOpenAISettings(
api_key=api_key,
base_url=base_url,
endpoint=endpoint,
chat_deployment_name=deployment_name,
api_version=api_version,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
token_endpoint=token_scope,
)
except ValidationError as exc:
raise AgentInitializationException(f"Failed to create Azure OpenAI settings: {exc}") from exc
if (
azure_openai_settings.api_key is None
and ad_token_provider is None
and ad_token is None
and azure_openai_settings.token_endpoint
):
ad_token = get_entra_auth_token(azure_openai_settings.token_endpoint)
# If we still have no credentials, we can't proceed
if not azure_openai_settings.api_key and not ad_token and not ad_token_provider:
raise AgentInitializationException(
"Please provide either an api_key, ad_token or ad_token_provider for authentication."
)
merged_headers = dict(copy(default_headers)) if default_headers else {}
if default_headers:
merged_headers.update(default_headers)
if APP_INFO:
merged_headers.update(APP_INFO)
merged_headers = prepend_semantic_kernel_to_user_agent(merged_headers)
if not azure_openai_settings.endpoint:
raise AgentInitializationException("Please provide an Azure OpenAI endpoint")
if not azure_openai_settings.chat_deployment_name:
raise AgentInitializationException("Please provide an Azure OpenAI deployment name")
return AsyncAzureOpenAI(
azure_endpoint=str(azure_openai_settings.endpoint),
api_version=azure_openai_settings.api_version,
api_key=azure_openai_settings.api_key.get_secret_value() if azure_openai_settings.api_key else None,
azure_ad_token=ad_token,
azure_ad_token_provider=ad_token_provider,
default_headers=merged_headers,
**kwargs,
)
@override
@classmethod
def resolve_placeholders(
cls: type[Self],
yaml_str: str,
settings: "KernelBaseSettings | None" = None,
extras: dict[str, Any] | None = None,
) -> str:
"""Substitute ${AzureOpenAI:Key} placeholders with fields from AzureOpenAIAgentSettings and extras."""
import re
pattern = re.compile(r"\$\{([^}]+)\}")
# Build the mapping only if settings is provided and valid
field_mapping: dict[str, Any] = {}
if settings is None:
settings = AzureOpenAISettings()
if not isinstance(settings, AzureOpenAISettings):
raise AgentInitializationException(f"Expected AzureOpenAISettings, got {type(settings).__name__}")
field_mapping.update({
"ChatModelId": cls._get_setting(getattr(settings, "chat_deployment_name", None)),
"AgentId": cls._get_setting(getattr(settings, "agent_id", None)),
"ApiKey": cls._get_setting(getattr(settings, "api_key", None)),
"ApiVersion": cls._get_setting(getattr(settings, "api_version", None)),
"BaseUrl": cls._get_setting(getattr(settings, "base_url", None)),
"Endpoint": cls._get_setting(getattr(settings, "endpoint", None)),
"TokenEndpoint": cls._get_setting(getattr(settings, "token_endpoint", None)),
})
if extras:
field_mapping.update(extras)
def replacer(match: re.Match[str]) -> str:
"""Replace the matched placeholder with the corresponding value from field_mapping."""
full_key = match.group(1) # for example, OpenAI:ApiKey
section, _, key = full_key.partition(":")
if section != "AzureOpenAI":
return match.group(0)
# Try short key first (ApiKey), then full (OpenAI:ApiKey)
return str(field_mapping.get(key) or field_mapping.get(full_key) or match.group(0))
result = pattern.sub(replacer, yaml_str)
# Safety check for unresolved placeholders
unresolved = pattern.findall(result)
if unresolved:
raise AgentInitializationException(
f"Unresolved placeholders in spec: {', '.join(f'${{{key}}}' for key in unresolved)}"
)
return result
@@ -1,14 +1,16 @@
# Copyright (c) Microsoft. All rights reserved.
import logging
import sys
from collections.abc import Awaitable, Callable
from copy import copy
from typing import Any
from typing import TYPE_CHECKING, Any
from openai import AsyncAzureOpenAI
from pydantic import ValidationError
from semantic_kernel.agents import OpenAIResponsesAgent
from semantic_kernel.agents.agent import register_agent_type
from semantic_kernel.connectors.ai.open_ai.settings.azure_open_ai_settings import AzureOpenAISettings
from semantic_kernel.exceptions.agent_exceptions import (
AgentInitializationException,
@@ -17,10 +19,29 @@ from semantic_kernel.utils.authentication.entra_id_authentication import get_ent
from semantic_kernel.utils.feature_stage_decorator import experimental
from semantic_kernel.utils.telemetry.user_agent import APP_INFO, prepend_semantic_kernel_to_user_agent
if TYPE_CHECKING:
from semantic_kernel.kernel_pydantic import KernelBaseSettings
logger: logging.Logger = logging.getLogger(__name__)
if sys.version_info >= (3, 12):
from typing import override # pragma: no cover
else:
from typing_extensions import override # pragma: no cover
if sys.version < "3.11":
from typing_extensions import Self # pragma: no cover
else:
from typing import Self # type: ignore # pragma: no cover
if sys.version_info >= (3, 13):
from warnings import deprecated
else:
from typing_extensions import deprecated
@experimental
@register_agent_type("azure_responses")
class AzureResponsesAgent(OpenAIResponsesAgent):
"""Azure Responses Agent class.
@@ -28,6 +49,9 @@ class AzureResponsesAgent(OpenAIResponsesAgent):
"""
@staticmethod
@deprecated(
"setup_resources is deprecated. Use AzureResponsesAgent.create_client() instead. This method will be removed by 2025-06-15." # noqa: E501
)
def setup_resources(
*,
ad_token: str | None = None,
@@ -116,3 +140,147 @@ class AzureResponsesAgent(OpenAIResponsesAgent):
)
return client, azure_openai_settings.responses_deployment_name
@staticmethod
def create_client(
*,
ad_token: str | None = None,
ad_token_provider: Callable[[], str | Awaitable[str]] | None = None,
api_key: str | None = None,
api_version: str | None = None,
base_url: str | None = None,
default_headers: dict[str, str] | None = None,
deployment_name: str | None = None,
endpoint: str | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
token_scope: str | None = None,
**kwargs: Any,
) -> AsyncAzureOpenAI:
"""A method to create the Azure OpenAI client.
Any arguments provided will override the values in the environment variables/environment file.
Args:
ad_token: The Microsoft Entra (previously Azure AD) token represented as a string
ad_token_provider: The Microsoft Entra (previously Azure AD) token provider provided as a callback
api_key: The API key
api_version: The API version
base_url: The base URL in the form https://<resource>.azure.openai.com/openai/deployments/<deployment_name>
default_headers: The default headers to add to the client
deployment_name: The Responses deployment name
endpoint: The endpoint in the form https://<resource>.azure.openai.com
env_file_path: The environment file path
env_file_encoding: The environment file encoding, defaults to utf-8
token_scope: The token scope
kwargs: Additional keyword arguments
Returns:
An Azure OpenAI client instance.
"""
try:
azure_openai_settings = AzureOpenAISettings(
api_key=api_key,
base_url=base_url,
endpoint=endpoint,
responses_deployment_name=deployment_name,
api_version=api_version,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
token_endpoint=token_scope,
)
except ValidationError as exc:
raise AgentInitializationException(f"Failed to create Azure OpenAI settings: {exc}") from exc
if (
azure_openai_settings.api_key is None
and ad_token_provider is None
and ad_token is None
and azure_openai_settings.token_endpoint
):
ad_token = get_entra_auth_token(azure_openai_settings.token_endpoint)
# If we still have no credentials, we can't proceed
if not azure_openai_settings.api_key and not ad_token and not ad_token_provider:
raise AgentInitializationException(
"Please provide either an api_key, ad_token or ad_token_provider for authentication."
)
merged_headers = dict(copy(default_headers)) if default_headers else {}
if default_headers:
merged_headers.update(default_headers)
if APP_INFO:
merged_headers.update(APP_INFO)
merged_headers = prepend_semantic_kernel_to_user_agent(merged_headers)
if not azure_openai_settings.endpoint:
raise AgentInitializationException("Please provide an Azure OpenAI endpoint")
if not azure_openai_settings.responses_deployment_name:
raise AgentInitializationException("Please provide an Azure OpenAI Responses deployment name")
return AsyncAzureOpenAI(
azure_endpoint=str(azure_openai_settings.endpoint),
api_version=azure_openai_settings.api_version,
api_key=azure_openai_settings.api_key.get_secret_value() if azure_openai_settings.api_key else None,
azure_ad_token=ad_token,
azure_ad_token_provider=ad_token_provider,
default_headers=merged_headers,
**kwargs,
)
@override
@classmethod
def resolve_placeholders(
cls: type[Self],
yaml_str: str,
settings: "KernelBaseSettings | None" = None,
extras: dict[str, Any] | None = None,
) -> str:
"""Substitute ${AzureOpenAI:Key} placeholders with fields from AzureOpenAIAgentSettings and extras."""
import re
pattern = re.compile(r"\$\{([^}]+)\}")
# Build the mapping only if settings is provided and valid
field_mapping: dict[str, Any] = {}
if settings is None:
settings = AzureOpenAISettings()
if not isinstance(settings, AzureOpenAISettings):
raise AgentInitializationException(f"Expected AzureOpenAISettings, got {type(settings).__name__}")
field_mapping.update({
"ChatModelId": getattr(settings, "responses_deployment_name", None),
"AgentId": getattr(settings, "agent_id", None),
"ApiKey": getattr(settings, "api_key", None),
"ApiVersion": getattr(settings, "api_version", None),
"BaseUrl": getattr(settings, "base_url", None),
"Endpoint": getattr(settings, "endpoint", None),
"TokenEndpoint": getattr(settings, "token_endpoint", None),
})
if extras:
field_mapping.update(extras)
def replacer(match: re.Match[str]) -> str:
"""Replace the matched placeholder with the corresponding value from field_mapping."""
full_key = match.group(1) # for example, AzureOpenAI:ApiKey
section, _, key = full_key.partition(":")
if section != "AzureOpenAI":
return match.group(0)
# Try short key first (ApiKey), then full (AzureOpenAI:ApiKey)
return str(field_mapping.get(key) or field_mapping.get(full_key) or match.group(0))
result = pattern.sub(replacer, yaml_str)
# Safety check for unresolved placeholders
unresolved = pattern.findall(result)
if unresolved:
raise AgentInitializationException(
f"Unresolved placeholders in spec: {', '.join(f'${{{key}}}' for key in unresolved)}"
)
return result
@@ -1,15 +1,11 @@
# Copyright (c) Microsoft. All rights reserved.
import inspect
import logging
import sys
from collections.abc import AsyncIterable, Awaitable, Callable, Iterable
from copy import copy
from typing import TYPE_CHECKING, Any, ClassVar, Literal
if sys.version_info >= (3, 12):
from typing import override # pragma: no cover
else:
from typing_extensions import override # pragma: no cover
from copy import copy, deepcopy
from typing import TYPE_CHECKING, Any, ClassVar, Literal, TypeVar
from openai import NOT_GIVEN, AsyncOpenAI, NotGiven
from openai.lib._parsing._completions import type_to_response_format_param
@@ -20,11 +16,20 @@ from openai.types.beta.assistant_create_params import (
ToolResourcesFileSearch,
)
from openai.types.beta.assistant_response_format_option_param import AssistantResponseFormatOptionParam
from openai.types.beta.assistant_tool_param import AssistantToolParam
from openai.types.beta.code_interpreter_tool_param import CodeInterpreterToolParam
from openai.types.beta.file_search_tool_param import FileSearchToolParam
from pydantic import BaseModel, Field, ValidationError
from pydantic import BaseModel, Field, SecretStr, ValidationError
from semantic_kernel.agents import Agent
from semantic_kernel.agents.agent import AgentResponseItem, AgentThread
from semantic_kernel.agents.agent import (
AgentResponseItem,
AgentSpec,
AgentThread,
DeclarativeSpecMixin,
ToolSpec,
register_agent_type,
)
from semantic_kernel.agents.channels.agent_channel import AgentChannel
from semantic_kernel.agents.channels.open_ai_assistant_channel import OpenAIAssistantChannel
from semantic_kernel.agents.open_ai.assistant_thread_actions import AssistantThreadActions
@@ -42,6 +47,7 @@ from semantic_kernel.exceptions.agent_exceptions import (
from semantic_kernel.functions import KernelArguments
from semantic_kernel.functions.kernel_function import TEMPLATE_FORMAT_MAP
from semantic_kernel.functions.kernel_plugin import KernelPlugin
from semantic_kernel.kernel import Kernel
from semantic_kernel.schema.kernel_json_schema_builder import KernelJsonSchemaBuilder
from semantic_kernel.utils.feature_stage_decorator import release_candidate
from semantic_kernel.utils.naming import generate_random_ascii_name
@@ -53,16 +59,75 @@ from semantic_kernel.utils.telemetry.user_agent import APP_INFO, prepend_semanti
if TYPE_CHECKING:
from openai import AsyncOpenAI
from openai.types.beta.assistant_tool_param import AssistantToolParam
from openai.types.beta.code_interpreter_tool_param import CodeInterpreterToolParam
from openai.types.beta.thread_create_params import Message as ThreadCreateMessage
from openai.types.beta.threads.run_create_params import TruncationStrategy
from semantic_kernel.kernel import Kernel
from semantic_kernel.kernel_pydantic import KernelBaseSettings
from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig
if sys.version_info >= (3, 12):
from typing import override # pragma: no cover
else:
from typing_extensions import override # pragma: no cover
if sys.version_info >= (3, 13):
from warnings import deprecated
else:
from typing_extensions import deprecated
_T = TypeVar("_T", bound="OpenAIAssistantAgent")
logger: logging.Logger = logging.getLogger(__name__)
# region Declarative Spec
_TOOL_BUILDERS: dict[
str,
Callable[[ToolSpec, Kernel | None], tuple[list[AssistantToolParam], ToolResources]],
] = {}
def _register_tool(tool_type: str):
def decorator(
fn: Callable[[ToolSpec, Kernel | None], tuple[list[AssistantToolParam], ToolResources]],
):
_TOOL_BUILDERS[tool_type.lower()] = fn
return fn
return decorator
# Update _code_interpreter
@_register_tool("code_interpreter")
def _code_interpreter(spec: ToolSpec, kernel: Kernel | None = None) -> tuple[list[AssistantToolParam], ToolResources]:
file_ids = spec.options.get("file_ids")
return OpenAIAssistantAgent.configure_code_interpreter_tool(file_ids=file_ids)
# Update _file_search
@_register_tool("file_search")
def _file_search(spec: ToolSpec, kernel: Kernel | None = None) -> tuple[list[AssistantToolParam], ToolResources]:
vector_store_ids = spec.options.get("vector_store_ids")
if not vector_store_ids or not isinstance(vector_store_ids, list) or not vector_store_ids[0]:
raise AgentInitializationException(f"Missing or malformed 'vector_store_ids' in: {spec}")
return OpenAIAssistantAgent.configure_file_search_tool(vector_store_ids=vector_store_ids)
def _build_tool(spec: ToolSpec, kernel: "Kernel") -> tuple[list[AssistantToolParam], ToolResources]:
if not spec.type:
raise AgentInitializationException("Tool spec must include a 'type' field.")
try:
builder = _TOOL_BUILDERS[spec.type.lower()]
except KeyError as exc:
raise AgentInitializationException(f"Unsupported tool type: {spec.type}") from exc
sig = inspect.signature(builder)
return builder(spec) if len(sig.parameters) == 1 else builder(spec, kernel) # type: ignore[call-arg]
# endregion
@release_candidate
class AssistantAgentThread(AgentThread):
@@ -157,7 +222,8 @@ class AssistantAgentThread(AgentThread):
@release_candidate
class OpenAIAssistantAgent(Agent):
@register_agent_type("openai_assistant")
class OpenAIAssistantAgent(DeclarativeSpecMixin, Agent):
"""OpenAI Assistant Agent class.
Provides the ability to interact with OpenAI Assistants.
@@ -241,6 +307,9 @@ class OpenAIAssistantAgent(Agent):
super().__init__(**args)
@staticmethod
@deprecated(
"setup_resources is deprecated. Use OpenAIAssistantAgent.create_client() instead. This method will be removed by 2025-06-15." # noqa: E501
)
def setup_resources(
*,
ai_model_id: str | None = None,
@@ -300,6 +369,226 @@ class OpenAIAssistantAgent(Agent):
return client, openai_settings.chat_model_id
@staticmethod
def create_client(
*,
ai_model_id: str | None = None,
api_key: str | None = None,
org_id: str | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
default_headers: dict[str, str] | None = None,
**kwargs: Any,
) -> AsyncOpenAI:
"""A method to create the OpenAI client.
Any arguments provided will override the values in the environment variables/environment file.
Args:
ai_model_id: The AI model ID
api_key: The API key
org_id: The organization ID
env_file_path: The environment file path
env_file_encoding: The environment file encoding, defaults to utf-8
default_headers: The default headers to add to the client
kwargs: Additional keyword arguments
Returns:
An OpenAI client instance.
"""
try:
openai_settings = OpenAISettings(
chat_model_id=ai_model_id,
api_key=api_key,
org_id=org_id,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
except ValidationError as ex:
raise AgentInitializationException("Failed to create OpenAI settings.", ex) from ex
if not openai_settings.api_key:
raise AgentInitializationException("The OpenAI API key is required.")
if not openai_settings.chat_model_id:
raise AgentInitializationException("The OpenAI model ID is required.")
merged_headers = dict(copy(default_headers)) if default_headers else {}
if default_headers:
merged_headers.update(default_headers)
if APP_INFO:
merged_headers.update(APP_INFO)
merged_headers = prepend_semantic_kernel_to_user_agent(merged_headers)
return AsyncOpenAI(
api_key=openai_settings.api_key.get_secret_value() if openai_settings.api_key else None,
organization=openai_settings.org_id,
default_headers=merged_headers,
**kwargs,
)
# endregion
# region Declarative Spec
@override
@classmethod
async def _from_dict(
cls: type[_T],
data: dict,
*,
kernel: Kernel,
prompt_template_config: "PromptTemplateConfig | None" = None,
**kwargs,
) -> _T:
"""Create an Assistant Agent from the provided dictionary.
Args:
data: The dictionary containing the agent data.
kernel: The kernel to use for the agent.
prompt_template_config: The prompt template configuration.
kwargs: Additional keyword arguments. Note: unsupported keys may raise validation errors.
Returns:
AzureAIAgent: The OpenAI Assistant Agent instance.
"""
client: AsyncOpenAI = kwargs.pop("client", None)
if client is None:
raise AgentInitializationException("Missing required 'client' in OpenAIAssistantAgent._from_dict()")
spec = AgentSpec.model_validate(data)
if "settings" in kwargs:
kwargs.pop("settings")
args = data.pop("arguments", None)
arguments = None
if args:
arguments = KernelArguments(**args)
if spec.id:
existing_definition = await client.beta.assistants.retrieve(spec.id)
# Create a mutable clone
definition = deepcopy(existing_definition)
# Selectively override attributes from spec
if spec.name is not None:
setattr(definition, "name", spec.name)
if spec.description is not None:
setattr(definition, "description", spec.description)
if spec.instructions is not None:
setattr(definition, "instructions", spec.instructions)
if spec.extras:
merged_metadata = dict(getattr(definition, "metadata", {}) or {})
merged_metadata.update(spec.extras)
setattr(definition, "metadata", merged_metadata)
return cls(
definition=definition,
client=client,
kernel=kernel,
prompt_template_config=prompt_template_config,
arguments=arguments,
**kwargs,
)
if not (spec.model and spec.model.id):
raise ValueError("model.id required when creating a new Azure AI agent")
# Build tool definitions & resources
tool_objs = [
_build_tool(t, kernel) for t in spec.tools if t.type != "function"
] # List[tuple[list[ToolParam], ToolResources]]
all_tools: list[AssistantToolParam] = []
all_resources: ToolResources = {}
for tool_list, resource in tool_objs:
all_tools.extend(tool_list)
all_resources.update(resource)
try:
agent_definition = await client.beta.assistants.create(
model=spec.model.id,
name=spec.name,
description=spec.description,
instructions=spec.instructions,
tools=all_tools,
tool_resources=all_resources,
metadata=spec.extras,
**kwargs,
)
except Exception as ex:
print(f"Error creating agent: {ex}")
return cls(
definition=agent_definition,
client=client,
arguments=arguments,
kernel=kernel,
prompt_template_config=prompt_template_config,
**kwargs,
)
@classmethod
def _get_setting(cls: type[_T], value: Any) -> Any:
"""Return raw value if `SecretStr`, otherwise pass through."""
if isinstance(value, SecretStr):
return value.get_secret_value()
return value
@override
@classmethod
def resolve_placeholders(
cls: type[_T],
yaml_str: str,
settings: "KernelBaseSettings | None" = None,
extras: dict[str, Any] | None = None,
) -> str:
"""Substitute ${OpenAI:Key} placeholders with fields from OpenAIAgentSettings and extras."""
import re
pattern = re.compile(r"\$\{([^}]+)\}")
# Build the mapping only if settings is provided and valid
field_mapping: dict[str, Any] = {}
if settings is None:
settings = OpenAISettings()
if not isinstance(settings, OpenAISettings):
raise AgentInitializationException(f"Expected OpenAISettings, got {type(settings).__name__}")
field_mapping.update({
"ChatModelId": cls._get_setting(getattr(settings, "chat_model_id", None)),
"AgentId": cls._get_setting(getattr(settings, "agent_id", None)),
"ApiKey": cls._get_setting(getattr(settings, "api_key", None)),
})
if extras:
field_mapping.update(extras)
def replacer(match: re.Match[str]) -> str:
"""Replace the matched placeholder with the corresponding value from field_mapping."""
full_key = match.group(1) # for example, OpenAI:ApiKey
section, _, key = full_key.partition(":")
if section != "OpenAI":
return match.group(0)
# Try short key first (ApiKey), then full (OpenAI:ApiKey)
return str(field_mapping.get(key) or field_mapping.get(full_key) or match.group(0))
result = pattern.sub(replacer, yaml_str)
# Safety check for unresolved placeholders
unresolved = pattern.findall(result)
if unresolved:
raise AgentInitializationException(
f"Unresolved placeholders in spec: {', '.join(f'${{{key}}}' for key in unresolved)}"
)
return result
# endregion
# region Tool Handling
@@ -307,7 +596,7 @@ class OpenAIAssistantAgent(Agent):
@staticmethod
def configure_code_interpreter_tool(
file_ids: str | list[str] | None = None, **kwargs: Any
) -> tuple[list["CodeInterpreterToolParam"], ToolResources]:
) -> tuple[list["AssistantToolParam"], ToolResources]:
"""Generate tool + tool_resources for the code_interpreter."""
if isinstance(file_ids, str):
file_ids = [file_ids]
@@ -320,7 +609,7 @@ class OpenAIAssistantAgent(Agent):
@staticmethod
def configure_file_search_tool(
vector_store_ids: str | list[str], **kwargs: Any
) -> tuple[list[FileSearchToolParam], ToolResources]:
) -> tuple[list[AssistantToolParam], ToolResources]:
"""Generate tool + tool_resources for the file_search."""
if isinstance(vector_store_ids, str):
vector_store_ids = [vector_store_ids]
@@ -1,10 +1,11 @@
# Copyright (c) Microsoft. All rights reserved.
import inspect
import logging
import sys
from collections.abc import AsyncIterable, Awaitable, Callable
from copy import copy
from typing import TYPE_CHECKING, Any, Literal
from typing import TYPE_CHECKING, Any, Literal, TypeVar
from openai import AsyncOpenAI
from openai.lib._parsing._responses import type_to_text_format_param
@@ -18,9 +19,10 @@ from openai.types.responses.web_search_tool_param import UserLocation, WebSearch
from openai.types.shared_params.comparison_filter import ComparisonFilter
from openai.types.shared_params.compound_filter import CompoundFilter
from openai.types.shared_params.response_format_json_object import ResponseFormatJSONObject
from pydantic import BaseModel, Field, ValidationError
from pydantic import BaseModel, Field, SecretStr, ValidationError
from semantic_kernel.agents import Agent, AgentResponseItem, AgentThread, RunPollingOptions
from semantic_kernel.agents.agent import AgentSpec, DeclarativeSpecMixin, ToolSpec, register_agent_type
from semantic_kernel.agents.open_ai.responses_agent_thread_actions import ResponsesAgentThreadActions
from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior
from semantic_kernel.connectors.ai.open_ai.settings.open_ai_settings import OpenAISettings
@@ -37,6 +39,7 @@ from semantic_kernel.exceptions.agent_exceptions import (
from semantic_kernel.functions import KernelArguments
from semantic_kernel.functions.kernel_function import TEMPLATE_FORMAT_MAP
from semantic_kernel.functions.kernel_plugin import KernelPlugin
from semantic_kernel.kernel import Kernel
from semantic_kernel.schema.kernel_json_schema_builder import KernelJsonSchemaBuilder
from semantic_kernel.utils.feature_stage_decorator import experimental
from semantic_kernel.utils.naming import generate_random_ascii_name
@@ -46,21 +49,92 @@ from semantic_kernel.utils.telemetry.agent_diagnostics.decorators import (
)
from semantic_kernel.utils.telemetry.user_agent import APP_INFO, prepend_semantic_kernel_to_user_agent
if TYPE_CHECKING:
from openai import AsyncOpenAI
from semantic_kernel.kernel_pydantic import KernelBaseSettings
from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig
if sys.version_info >= (3, 12):
from typing import override # pragma: no cover
else:
from typing_extensions import override # pragma: no cover
if TYPE_CHECKING:
from openai import AsyncOpenAI
if sys.version_info >= (3, 13):
from warnings import deprecated
else:
from typing_extensions import deprecated
from semantic_kernel.kernel import Kernel
from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig
_T = TypeVar("_T", bound="OpenAIResponsesAgent")
ResponseFormatUnion = ResponseFormatText | ResponseFormatTextJSONSchemaConfigParam | ResponseFormatJSONObject
logger: logging.Logger = logging.getLogger(__name__)
# region Declarative Spec
_TOOL_BUILDERS: dict[str, Callable[[ToolSpec, Kernel | None], ToolParam]] = {}
def _register_tool(tool_type: str):
def decorator(fn: Callable[[ToolSpec, Kernel | None], ToolParam]):
_TOOL_BUILDERS[tool_type.lower()] = fn
return fn
return decorator
@_register_tool("file_search")
def _file_search(spec: ToolSpec, kernel: Kernel | None = None) -> FileSearchToolParam:
options = spec.options or {}
vector_store_ids = options.get("vector_store_ids")
if not vector_store_ids or not isinstance(vector_store_ids, list) or not vector_store_ids[0]:
raise AgentInitializationException(f"Missing or malformed 'vector_store_ids' in: {spec}")
filters = options.get("filters")
max_num_results = options.get("max_num_results")
ranking_options = options.get("ranking_options", {})
score_threshold = ranking_options.get("score_threshold")
ranker = ranking_options.get("ranker")
return OpenAIResponsesAgent.configure_file_search_tool(
vector_store_ids=vector_store_ids,
filters=filters,
max_num_results=max_num_results,
score_threshold=score_threshold,
ranker=ranker,
)
@_register_tool("web_search")
def _web_search(spec: ToolSpec, kernel: Kernel | None = None) -> WebSearchToolParam:
options = spec.options or {}
context_size = options.get("search_context_size")
user_location = options.get("user_location")
return OpenAIResponsesAgent.configure_web_search_tool(
context_size=context_size,
user_location=user_location,
)
def _build_tool(spec: ToolSpec, kernel: "Kernel") -> ToolParam:
if not spec.type:
raise AgentInitializationException("Tool spec must include a 'type' field.")
try:
builder = _TOOL_BUILDERS[spec.type.lower()]
except KeyError as exc:
raise AgentInitializationException(f"Unsupported tool type: {spec.type}") from exc
sig = inspect.signature(builder)
return builder(spec) if len(sig.parameters) == 1 else builder(spec, kernel) # type: ignore[call-arg]
# endregion
# region Agent Thread
@@ -177,7 +251,8 @@ class ResponsesAgentThread(AgentThread):
@experimental
class OpenAIResponsesAgent(Agent):
@register_agent_type("openai_responses")
class OpenAIResponsesAgent(DeclarativeSpecMixin, Agent):
"""OpenAI Responses Agent class.
Provides the ability to interact with OpenAI's Responses API.
@@ -303,6 +378,9 @@ class OpenAIResponsesAgent(Agent):
super().__init__(**args)
@staticmethod
@deprecated(
"setup_resources is deprecated. Use OpenAIResponsesAgent.create_client() instead. This method will be removed by 2025-06-15." # noqa: E501
)
def setup_resources(
*,
ai_model_id: str | None = None,
@@ -362,6 +440,181 @@ class OpenAIResponsesAgent(Agent):
return client, openai_settings.responses_model_id
@staticmethod
def create_client(
*,
ai_model_id: str | None = None,
api_key: str | None = None,
org_id: str | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
default_headers: dict[str, str] | None = None,
**kwargs: Any,
) -> AsyncOpenAI:
"""A method to create the OpenAI client.
Any arguments provided will override the values in the environment variables/environment file.
Args:
ai_model_id: The AI model ID
api_key: The API key
org_id: The organization ID
env_file_path: The environment file path
env_file_encoding: The environment file encoding, defaults to utf-8
default_headers: The default headers to add to the client
kwargs: Additional keyword arguments
Returns:
An OpenAI client instance.
"""
try:
openai_settings = OpenAISettings(
responses_model_id=ai_model_id,
api_key=api_key,
org_id=org_id,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
except ValidationError as ex:
raise AgentInitializationException("Failed to create OpenAI settings.", ex) from ex
if not openai_settings.api_key:
raise AgentInitializationException("The OpenAI API key is required.")
if not openai_settings.responses_model_id:
raise AgentInitializationException("The OpenAI Responses model ID is required.")
merged_headers = dict(copy(default_headers)) if default_headers else {}
if default_headers:
merged_headers.update(default_headers)
if APP_INFO:
merged_headers.update(APP_INFO)
merged_headers = prepend_semantic_kernel_to_user_agent(merged_headers)
return AsyncOpenAI(
api_key=openai_settings.api_key.get_secret_value() if openai_settings.api_key else None,
organization=openai_settings.org_id,
default_headers=merged_headers,
**kwargs,
)
# endregion
# region Declarative Spec
@override
@classmethod
async def _from_dict(
cls: type[_T],
data: dict,
*,
kernel: Kernel,
prompt_template_config: "PromptTemplateConfig | None" = None,
**kwargs,
) -> _T:
"""Create an Assistant Agent from the provided dictionary.
Args:
data: The dictionary containing the agent data.
kernel: The kernel to use for the agent.
prompt_template_config: The prompt template configuration.
kwargs: Additional keyword arguments. Note: unsupported keys may raise validation errors.
Returns:
AzureAIAgent: The OpenAI Assistant Agent instance.
"""
client: AsyncOpenAI = kwargs.pop("client", None)
if client is None:
raise AgentInitializationException("Missing required 'client' in OpenAIResponsesAgent._from_dict()")
spec = AgentSpec.model_validate(data)
if "settings" in kwargs:
kwargs.pop("settings")
args = data.pop("arguments", None)
arguments = None
if args:
arguments = KernelArguments(**args)
if not (spec.model and spec.model.id):
raise AgentInitializationException("model.id required when creating a new OpenAI Responses Agent.")
# Build tool definitions & resources
tool_objs = [_build_tool(t, kernel) for t in spec.tools if t.type != "function"]
return cls(
name=spec.name,
description=spec.description,
instruction_role=spec.instructions,
ai_model_id=spec.model.id,
client=client,
arguments=arguments,
kernel=kernel,
prompt_template_config=prompt_template_config,
tools=tool_objs,
**kwargs,
)
@classmethod
def _get_setting(cls: type[_T], value: Any) -> Any:
"""Return raw value if `SecretStr`, otherwise pass through."""
if isinstance(value, SecretStr):
return value.get_secret_value()
return value
@override
@classmethod
def resolve_placeholders(
cls: type[_T],
yaml_str: str,
settings: "KernelBaseSettings | None" = None,
extras: dict[str, Any] | None = None,
) -> str:
"""Substitute ${OpenAI:Key} placeholders with fields from OpenAIAgentSettings and extras."""
import re
pattern = re.compile(r"\$\{([^}]+)\}")
# Build the mapping only if settings is provided and valid
field_mapping: dict[str, Any] = {}
if settings is None:
settings = OpenAISettings()
if not isinstance(settings, OpenAISettings):
raise AgentInitializationException(f"Expected OpenAISettings, got {type(settings).__name__}")
field_mapping.update({
"ChatModelId": cls._get_setting(getattr(settings, "responses_model_id", None)),
"AgentId": cls._get_setting(getattr(settings, "agent_id", None)),
"ApiKey": cls._get_setting(getattr(settings, "api_key", None)),
})
if extras:
field_mapping.update(extras)
def replacer(match: re.Match[str]) -> str:
"""Replace the matched placeholder with the corresponding value from field_mapping."""
full_key = match.group(1) # for example, OpenAI:ApiKey
section, _, key = full_key.partition(":")
if section != "OpenAI":
return match.group(0)
# Try short key first (ApiKey), then full (OpenAI:ApiKey)
return str(field_mapping.get(key) or field_mapping.get(full_key) or match.group(0))
result = pattern.sub(replacer, yaml_str)
# Safety check for unresolved placeholders
unresolved = pattern.findall(result)
if unresolved:
raise AgentInitializationException(
f"Unresolved placeholders in spec: {', '.join(f'${{{key}}}' for key in unresolved)}"
)
return result
# endregion
# region Tool Handling
@@ -789,7 +789,7 @@ class ResponsesAgentThreadActions:
name=name,
role=AuthorRole(role_str),
items=items,
status=Status(response.status),
status=Status(response.status) if hasattr(response, "status") else None,
)
@classmethod
@@ -812,7 +812,7 @@ class ResponsesAgentThreadActions:
metadata=metadata,
role=AuthorRole(role_str),
items=items,
status=Status(response.status),
status=Status(response.status) if hasattr(response, "status") else None,
)
@classmethod
@@ -6,6 +6,7 @@ from typing import Annotated
import pytest
from semantic_kernel.agents import AzureAssistantAgent, OpenAIAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings, OpenAISettings
from semantic_kernel.contents import AuthorRole, ChatMessageContent, StreamingChatMessageContent
from semantic_kernel.contents.streaming_text_content import StreamingTextContent
from semantic_kernel.contents.text_content import TextContent
@@ -37,10 +38,12 @@ class TestOpenAIAssistantAgentIntegration:
tools, tool_resources, plugins = [], {}, []
if agent_type == "azure":
client, model = AzureAssistantAgent.setup_resources()
client = AzureAssistantAgent.create_client()
model = AzureOpenAISettings().chat_deployment_name
AgentClass = AzureAssistantAgent
else: # agent_type == "openai"
client, model = OpenAIAssistantAgent.setup_resources()
client = OpenAIAssistantAgent.create_client()
model = OpenAISettings().chat_model_id
AgentClass = OpenAIAssistantAgent
if params.get("enable_code_interpreter"):
@@ -7,6 +7,7 @@ import pytest
from pydantic import BaseModel
from semantic_kernel.agents import AzureResponsesAgent, OpenAIResponsesAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings, OpenAISettings
from semantic_kernel.contents import AuthorRole, ChatMessageContent, StreamingChatMessageContent
from semantic_kernel.contents.streaming_text_content import StreamingTextContent
from semantic_kernel.contents.text_content import TextContent
@@ -48,10 +49,12 @@ class TestOpenAIResponsesAgentIntegration:
tools, plugins, text = [], [], None
if agent_type == "azure":
client, model = AzureResponsesAgent.setup_resources()
client = AzureResponsesAgent.create_client()
model = AzureOpenAISettings().chat_deployment_name
AgentClass = AzureResponsesAgent
else: # agent_type == "openai"
client, model = OpenAIResponsesAgent.setup_resources()
client = OpenAIResponsesAgent.create_client()
model = OpenAISettings().chat_model_id
AgentClass = OpenAIResponsesAgent
if params.get("enable_web_search"):
@@ -6,5 +6,5 @@
}
],
"stream": false,
"model": "gpt-3.5-turbo"
"model": "gpt-4.1-nano"
}
@@ -14,5 +14,5 @@
}
],
"stream": false,
"model": "gpt-3.5-turbo"
"model": "gpt-4.1-nano"
}
@@ -6,5 +6,5 @@
}
],
"stream": false,
"model": "gpt-3.5-turbo"
"model": "gpt-4.1-nano"
}
@@ -10,5 +10,5 @@
}
],
"stream": false,
"model": "gpt-3.5-turbo"
"model": "gpt-4.1-nano"
}
@@ -6,5 +6,5 @@
}
],
"stream": false,
"model": "gpt-3.5-turbo"
"model": "gpt-4.1-nano"
}
@@ -12,8 +12,7 @@ import pytest
import pytest_asyncio
from openai import AsyncOpenAI
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion
from semantic_kernel.connectors.ai.open_ai.settings.open_ai_settings import OpenAISettings
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAISettings
from semantic_kernel.functions.kernel_arguments import KernelArguments
from semantic_kernel.functions.kernel_function import KernelFunction
from semantic_kernel.functions.kernel_function_decorator import kernel_function
@@ -23,7 +22,7 @@ from semantic_kernel.kernel import Kernel
logger = logging.getLogger(__name__)
OPENAI_MODEL_ID = "gpt-3.5-turbo"
OPENAI_MODEL_ID = "gpt-4.1-nano"
# region Test Prompts
@@ -53,7 +53,7 @@ from openai.types.shared.function_definition import FunctionDefinition
from semantic_kernel.agents.open_ai.assistant_thread_actions import AssistantThreadActions
from semantic_kernel.agents.open_ai.function_action_result import FunctionActionResult
from semantic_kernel.agents.open_ai.open_ai_assistant_agent import OpenAIAssistantAgent
from semantic_kernel.agents.open_ai.openai_assistant_agent import OpenAIAssistantAgent
from semantic_kernel.agents.open_ai.run_polling_options import RunPollingOptions
from semantic_kernel.contents.chat_message_content import ChatMessageContent
from semantic_kernel.contents.file_reference_content import FileReferenceContent
@@ -13,8 +13,9 @@ from openai.types.beta.threads.text import Text
from openai.types.beta.threads.text_content_block import TextContentBlock
from pydantic import BaseModel, ValidationError
from semantic_kernel.agents import AgentRegistry
from semantic_kernel.agents.open_ai.azure_assistant_agent import AzureAssistantAgent
from semantic_kernel.agents.open_ai.open_ai_assistant_agent import AssistantAgentThread
from semantic_kernel.agents.open_ai.openai_assistant_agent import AssistantAgentThread
from semantic_kernel.agents.open_ai.run_polling_options import RunPollingOptions
from semantic_kernel.contents.chat_message_content import ChatMessageContent
from semantic_kernel.contents.utils.author_role import AuthorRole
@@ -26,6 +27,28 @@ from semantic_kernel.kernel import Kernel
from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig
@pytest.fixture
def mock_azure_openai_client_and_definition():
client = AsyncMock(spec=AsyncOpenAI)
client.beta = MagicMock()
client.beta.assistants = MagicMock()
definition = AsyncMock(spec=Assistant)
definition.id = "agent123"
definition.name = "DeclarativeAgent"
definition.description = "desc"
definition.instructions = "test agent"
definition.tools = []
definition.model = "gpt-4o"
definition.temperature = 1.0
definition.top_p = 1.0
definition.metadata = {}
client.beta.assistants.create = AsyncMock(return_value=definition)
return client, definition
class SamplePlugin:
@kernel_function
def test_plugin(self, *args, **kwargs):
@@ -325,3 +348,121 @@ async def test_retrieve_agent_missing_chat_deployment_name_throws(kernel, azure_
endpoint="https://test_endpoint.com",
default_headers={"user_agent": "test"},
)
async def test_azure_assistant_agent_from_yaml_minimal(
azure_openai_unit_test_env, mock_azure_openai_client_and_definition
):
spec = """
type: azure_assistant
name: MinimalAgent
model:
id: ${AzureOpenAI:ChatModelId}
connection:
api_key: ${AzureOpenAI:ApiKey}
endpoint: ${AzureOpenAI:Endpoint}
"""
client, definition = mock_azure_openai_client_and_definition
definition.name = "MinimalAgent"
agent = await AgentRegistry.create_from_yaml(spec, client=client)
assert isinstance(agent, AzureAssistantAgent)
assert agent.name == "MinimalAgent"
assert agent.definition.model == "gpt-4o"
async def test_azure_assistant_agent_with_tools(azure_openai_unit_test_env, mock_azure_openai_client_and_definition):
spec = """
type: azure_assistant
name: CodeAgent
description: Uses code interpreter.
model:
id: ${AzureOpenAI:ChatModelId}
connection:
api_key: ${AzureOpenAI:ApiKey}
endpoint: ${AzureOpenAI:Endpoint}
tools:
- type: code_interpreter
options:
file_ids:
- ${AzureOpenAI:FileId1}
"""
client, definition = mock_azure_openai_client_and_definition
definition.name = "CodeAgent"
definition.tools = [{"type": "code_interpreter", "options": {"file_ids": ["file-123"]}}]
agent = await AgentRegistry.create_from_yaml(spec, client=client, extras={"AzureOpenAI:FileId1": "file-123"})
assert agent.name == "CodeAgent"
assert any(t["type"] == "code_interpreter" for t in agent.definition.tools)
async def test_azure_assistant_agent_with_inputs_outputs_template(
azure_openai_unit_test_env, mock_azure_openai_client_and_definition
):
spec = """
type: azure_assistant
name: StoryAgent
model:
id: ${AzureOpenAI:ChatModelId}
connection:
api_key: ${AzureOpenAI:ApiKey}
inputs:
topic:
description: The story topic.
required: true
default: AI
length:
description: The length of story.
required: true
default: 2
outputs:
output1:
description: The story.
template:
format: semantic-kernel
"""
client, definition = mock_azure_openai_client_and_definition
definition.name = "StoryAgent"
agent: AzureAssistantAgent = await AgentRegistry.create_from_yaml(spec, client=client)
assert agent.name == "StoryAgent"
assert agent.prompt_template.prompt_template_config.template_format == "semantic-kernel"
async def test_azure_assistant_agent_from_dict_missing_type():
data = {"name": "NoType"}
with pytest.raises(AgentInitializationException, match="Missing 'type'"):
await AgentRegistry.create_from_dict(data)
async def test_azure_assistant_agent_from_yaml_missing_required_fields():
spec = """
type: azure_assistant
"""
with pytest.raises(AgentInitializationException):
await AgentRegistry.create_from_yaml(spec)
async def test_agent_from_file_success(tmp_path, azure_openai_unit_test_env, mock_azure_openai_client_and_definition):
file_path = tmp_path / "spec.yaml"
file_path.write_text(
"""
type: azure_assistant
name: DeclarativeAgent
model:
id: ${AzureOpenAI:ChatModelId}
connection:
api_key: ${AzureOpenAI:ApiKey}
""",
encoding="utf-8",
)
client, _ = mock_azure_openai_client_and_definition
agent = await AgentRegistry.create_from_file(str(file_path), client=client)
assert agent.name == "DeclarativeAgent"
assert isinstance(agent, AzureAssistantAgent)
async def test_azure_assistant_agent_from_yaml_invalid_type():
spec = """
type: not_registered
name: ShouldFail
"""
with pytest.raises(AgentInitializationException, match="not registered"):
await AgentRegistry.create_from_yaml(spec)
@@ -14,7 +14,7 @@ from openai.types.beta.threads.text import Text
from openai.types.beta.threads.text_content_block import TextContentBlock
from semantic_kernel.agents.chat_completion.chat_completion_agent import ChatCompletionAgent
from semantic_kernel.agents.open_ai.open_ai_assistant_agent import OpenAIAssistantAgent
from semantic_kernel.agents.open_ai.openai_assistant_agent import OpenAIAssistantAgent
from semantic_kernel.contents.chat_message_content import ChatMessageContent
from semantic_kernel.contents.function_call_content import FunctionCallContent
from semantic_kernel.contents.text_content import TextContent
@@ -1,14 +1,14 @@
# Copyright (c) Microsoft. All rights reserved.
from unittest.mock import AsyncMock, patch
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from openai import AsyncOpenAI
from openai.types.beta.assistant import Assistant
from pydantic import BaseModel, ValidationError
from semantic_kernel.agents import OpenAIAssistantAgent
from semantic_kernel.agents.agent import AgentResponseItem
from semantic_kernel.agents.open_ai.open_ai_assistant_agent import AssistantAgentThread
from semantic_kernel.agents import AgentRegistry, AgentResponseItem, OpenAIAssistantAgent
from semantic_kernel.agents.open_ai.openai_assistant_agent import AssistantAgentThread
from semantic_kernel.agents.open_ai.run_polling_options import RunPollingOptions
from semantic_kernel.contents.chat_history import ChatHistory
from semantic_kernel.contents.chat_message_content import ChatMessageContent
@@ -24,6 +24,28 @@ from semantic_kernel.kernel import Kernel
from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig
@pytest.fixture
def mock_openai_client_and_definition():
client = AsyncMock(spec=AsyncOpenAI)
client.beta = MagicMock()
client.beta.assistants = MagicMock()
definition = AsyncMock(spec=Assistant)
definition.id = "agent123"
definition.name = "DeclarativeAgent"
definition.description = "desc"
definition.instructions = "test agent"
definition.tools = []
definition.model = "gpt-4o"
definition.temperature = 1.0
definition.top_p = 1.0
definition.metadata = {}
client.beta.assistants.create = AsyncMock(return_value=definition)
return client, definition
class SamplePlugin:
@kernel_function
def test_plugin(self, *args, **kwargs):
@@ -361,3 +383,117 @@ async def test_open_ai_agent_missing_chat_deployment_name_throws(kernel, openai_
api_key="test_api_key",
default_headers={"user_agent": "test"},
)
async def test_openai_assistant_agent_from_yaml_minimal(openai_unit_test_env, mock_openai_client_and_definition):
spec = """
type: openai_assistant
name: MinimalAgent
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
"""
client, definition = mock_openai_client_and_definition
definition.name = "MinimalAgent"
agent = await AgentRegistry.create_from_yaml(spec, client=client)
assert isinstance(agent, OpenAIAssistantAgent)
assert agent.name == "MinimalAgent"
assert agent.definition.model == "gpt-4o"
async def test_openai_assistant_agent_with_tools(openai_unit_test_env, mock_openai_client_and_definition):
spec = """
type: openai_assistant
name: CodeAgent
description: Uses code interpreter.
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
tools:
- type: code_interpreter
options:
file_ids:
- ${OpenAI:FileId1}
"""
client, definition = mock_openai_client_and_definition
definition.name = "CodeAgent"
definition.tools = [{"type": "code_interpreter", "options": {"file_ids": ["file-123"]}}]
agent = await AgentRegistry.create_from_yaml(spec, client=client, extras={"OpenAI:FileId1": "file-123"})
assert agent.name == "CodeAgent"
assert any(t["type"] == "code_interpreter" for t in agent.definition.tools)
async def test_openai_assistant_agent_with_inputs_outputs_template(
openai_unit_test_env, mock_openai_client_and_definition
):
spec = """
type: openai_assistant
name: StoryAgent
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
inputs:
topic:
description: The story topic.
required: true
default: AI
length:
description: The length of story.
required: true
default: 2
outputs:
output1:
description: The story.
template:
format: semantic-kernel
"""
client, definition = mock_openai_client_and_definition
definition.name = "StoryAgent"
agent: OpenAIAssistantAgent = await AgentRegistry.create_from_yaml(spec, client=client)
assert agent.name == "StoryAgent"
assert agent.prompt_template.prompt_template_config.template_format == "semantic-kernel"
async def test_openai_assistant_agent_from_dict_missing_type():
data = {"name": "NoType"}
with pytest.raises(AgentInitializationException, match="Missing 'type'"):
await AgentRegistry.create_from_dict(data)
async def test_openai_assistant_agent_from_yaml_missing_required_fields():
spec = """
type: openai_assistant
"""
with pytest.raises(AgentInitializationException):
await AgentRegistry.create_from_yaml(spec)
async def test_agent_from_file_success(tmp_path, openai_unit_test_env, mock_openai_client_and_definition):
file_path = tmp_path / "spec.yaml"
file_path.write_text(
"""
type: openai_assistant
name: DeclarativeAgent
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
""",
encoding="utf-8",
)
client, _ = mock_openai_client_and_definition
agent = await AgentRegistry.create_from_file(str(file_path), client=client)
assert agent.name == "DeclarativeAgent"
assert isinstance(agent, OpenAIAssistantAgent)
async def test_openai_assistant_agent_from_yaml_invalid_type():
spec = """
type: not_registered
name: ShouldFail
"""
with pytest.raises(AgentInitializationException, match="not registered"):
await AgentRegistry.create_from_yaml(spec)
@@ -6,6 +6,7 @@ import pytest
from openai import AsyncOpenAI
from pydantic import BaseModel, ValidationError
from semantic_kernel.agents import AgentRegistry
from semantic_kernel.agents.open_ai.openai_responses_agent import OpenAIResponsesAgent, ResponsesAgentThread
from semantic_kernel.agents.open_ai.run_polling_options import RunPollingOptions
from semantic_kernel.contents.chat_message_content import ChatMessageContent
@@ -18,6 +19,11 @@ from semantic_kernel.kernel import Kernel
from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig
@pytest.fixture
def mock_openai_client():
return AsyncMock(spec=AsyncOpenAI)
class SamplePlugin:
@kernel_function
def test_plugin(self, *args, **kwargs):
@@ -247,3 +253,114 @@ async def test_open_ai_agent_missing_chat_deployment_name_throws(kernel, openai_
api_key="test_api_key",
default_headers={"user_agent": "test"},
)
async def test_openai_assistant_agent_from_yaml_minimal(openai_unit_test_env, mock_openai_client):
spec = """
type: openai_responses
name: MinimalAgent
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
"""
client = mock_openai_client
agent: OpenAIResponsesAgent = await AgentRegistry.create_from_yaml(spec, client=client)
assert isinstance(agent, OpenAIResponsesAgent)
assert agent.name == "MinimalAgent"
assert agent.ai_model_id == openai_unit_test_env.get("OPENAI_RESPONSES_MODEL_ID")
async def test_openai_assistant_agent_with_tools(openai_unit_test_env, mock_openai_client):
spec = """
type: openai_responses
name: FileSearchAgent
description: Uses file search.
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
tools:
- type: file_search
description: File search for document retrieval.
options:
vector_store_ids:
- ${OpenAI:VectorStoreId}
"""
client = mock_openai_client
agent: OpenAIResponsesAgent = await AgentRegistry.create_from_yaml(
spec, client=client, extras={"OpenAI:VectorStoreId": "vector-store-123"}
)
assert agent.name == "FileSearchAgent"
assert any(t["type"] == "file_search" for t in agent.tools)
async def test_openai_assistant_agent_with_inputs_outputs_template(openai_unit_test_env, mock_openai_client):
spec = """
type: openai_responses
name: StoryAgent
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
inputs:
topic:
description: The story topic.
required: true
default: AI
length:
description: The length of story.
required: true
default: 2
outputs:
output1:
description: The story.
template:
format: semantic-kernel
"""
client = mock_openai_client
agent: OpenAIResponsesAgent = await AgentRegistry.create_from_yaml(spec, client=client)
assert agent.name == "StoryAgent"
assert agent.prompt_template.prompt_template_config.template_format == "semantic-kernel"
async def test_openai_assistant_agent_from_dict_missing_type():
data = {"name": "NoType"}
with pytest.raises(AgentInitializationException, match="Missing 'type'"):
await AgentRegistry.create_from_dict(data)
async def test_openai_assistant_agent_from_yaml_missing_required_fields():
spec = """
type: openai_responses
"""
with pytest.raises(AgentInitializationException):
await AgentRegistry.create_from_yaml(spec)
async def test_agent_from_file_success(tmp_path, openai_unit_test_env, mock_openai_client):
file_path = tmp_path / "spec.yaml"
file_path.write_text(
"""
type: openai_responses
name: DeclarativeAgent
model:
id: ${OpenAI:ChatModelId}
connection:
api_key: ${OpenAI:ApiKey}
""",
encoding="utf-8",
)
client = mock_openai_client
agent: OpenAIResponsesAgent = await AgentRegistry.create_from_file(str(file_path), client=client)
assert agent.name == "DeclarativeAgent"
assert isinstance(agent, OpenAIResponsesAgent)
async def test_openai_assistant_agent_from_yaml_invalid_type():
spec = """
type: not_registered
name: ShouldFail
"""
with pytest.raises(AgentInitializationException, match="not registered"):
await AgentRegistry.create_from_yaml(spec)
@@ -3,7 +3,7 @@
import pytest
from semantic_kernel.agents.chat_completion.chat_completion_agent import ChatCompletionAgent
from semantic_kernel.agents.open_ai.open_ai_assistant_agent import OpenAIAssistantAgent
from semantic_kernel.agents.open_ai.openai_assistant_agent import OpenAIAssistantAgent
pytestmark = pytest.mark.parametrize(
"decorated_method, expected_attribute",

Some files were not shown because too many files have changed in this diff Show More