langchain-ai/langsmith-docs
1
0
Fork 0
mirror of https://github.com/langchain-ai/langsmith-docs.git synced 2026-08-26 17:26:32 -04:00
Code Issues 16 Packages Projects Releases Wiki Activity

DOC: <Issue related to /observability/how_to_guides/trace_with_openai_agents_sdk> #88

New Issue
Closed
opened 2026-02-21 17:17:09 -05:00 by yindo · 5 comments
yindo commented 2026-02-21 17:17:09 -05:00
Owner
Copy Link
Copy Source

Originally created by @dast-draftwise on GitHub (Aug 18, 2025).

Originally assigned to: @angus-langchain on GitHub.

This approach doesnt track tool costs.
I have a simple tool responsible for making a string into lowercase using LLM as well as an triage Agent responsible for picking up the right tool for the task, in this case there is only one tool. I run the code with the OpenAIAgentsTracingProcessor defined.
This approach works well, the issue is that it created in the LangSmith UI a separate trace for the Agent vs Tool calls (LLM)
How can I have the agent and tool call under the same trace?
Image

if __name__ == "__main__":

    set_trace_processors([OpenAIAgentsTracingProcessor()])
    asyncio.run(main())

@function_tool(name_override="make_lowercase_tool", description_override="Convert the given text to lowercase.")
async def make_lowercase_tool(
    ctx: RunContextWrapper[Any],
    args: TextInput
) -> str:
    """Tool that uses LLM to convert text to lowercase."""
    client = wrap_openai(AsyncOpenAI())

    response = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Convert the given text to lowercase. Return only the result."},
            {"role": "user", "content": f"Convert to lowercase: {args.text}"}
        ],
        max_tokens=100,
        temperature=0
    )
    
    result = response.choices[0].message.content.strip()

    usage_info = {
        "model": response.model,
        "prompt_tokens": response.usage.prompt_tokens,
        "completion_tokens": response.usage.completion_tokens, # type: ignore
        "total_tokens": response.usage.total_tokens,
        "request_id": response.id,
        "created": response.created,
        "finish_reason": response.choices[0].finish_reason
    }

    print(f"make_lowercase_tool tokens | {usage_info.get("total_tokens")}")

    return json.dumps({
        "result": result,
        "operation": "lowercase",
        "original_text": args.text,
        "usage_metadata": {"input_tokens": usage_info.get("prompt_tokens"),
                        "output_tokens": usage_info.get("completion_tokens"),
                        "total_tokens": usage_info.get("total_tokens")
    }})

async def run_smart_text_agent(user_query: str, text: str) -> str:
    """Run the agent that uses the tool to process the input."""
        
    instructions = (
        "You are a text processing agent. You have 3 tools:\n"
        "- make_lowercase_tool: converts text to lowercase\n"
        "- make_capitalize_tool: capitalizes first letter of each word\n"
        "- count_words_tool: counts words in text\n\n"
        "IMPORTANT: When given a text and a request, immediately use the appropriate tool to process the text. "
        "Do NOT explain your capabilities - just do the work and return the results."
    )
    agent = Agent(
        name="Smart Text Processing Agent",
        instructions=instructions,
        tools=[make_lowercase_tool],
        model="gpt-4o-mini-2024-07-18"
    )
    prompt = f"Process this text: '{text}'. Request: {user_query}. Use the appropriate tool immediately."
    result = await Runner.run(agent, prompt)

    return result.final_output
Originally created by @dast-draftwise on GitHub (Aug 18, 2025). Originally assigned to: @angus-langchain on GitHub. This approach doesnt track tool costs. I have a simple tool responsible for making a string into lowercase using LLM as well as an triage Agent responsible for picking up the right tool for the task, in this case there is only one tool. I run the code with the `OpenAIAgentsTracingProcessor` defined. This approach works well, the issue is that it created in the LangSmith UI a separate trace for the Agent vs Tool calls (LLM) **How can I have the agent and tool call under the same trace?** <img width="990" height="140" alt="Image" src="https://github.com/user-attachments/assets/84943b12-6efd-49ff-bef3-5d33186a5261" /> ```py if __name__ == "__main__": set_trace_processors([OpenAIAgentsTracingProcessor()]) asyncio.run(main()) ``` ```py @function_tool(name_override="make_lowercase_tool", description_override="Convert the given text to lowercase.") async def make_lowercase_tool( ctx: RunContextWrapper[Any], args: TextInput ) -> str: """Tool that uses LLM to convert text to lowercase.""" client = wrap_openai(AsyncOpenAI()) response = await client.chat.completions.create( model="gpt-4o-mini", messages=[ {"role": "system", "content": "Convert the given text to lowercase. Return only the result."}, {"role": "user", "content": f"Convert to lowercase: {args.text}"} ], max_tokens=100, temperature=0 ) result = response.choices[0].message.content.strip() usage_info = { "model": response.model, "prompt_tokens": response.usage.prompt_tokens, "completion_tokens": response.usage.completion_tokens, # type: ignore "total_tokens": response.usage.total_tokens, "request_id": response.id, "created": response.created, "finish_reason": response.choices[0].finish_reason } print(f"make_lowercase_tool tokens | {usage_info.get("total_tokens")}") return json.dumps({ "result": result, "operation": "lowercase", "original_text": args.text, "usage_metadata": {"input_tokens": usage_info.get("prompt_tokens"), "output_tokens": usage_info.get("completion_tokens"), "total_tokens": usage_info.get("total_tokens") }}) async def run_smart_text_agent(user_query: str, text: str) -> str: """Run the agent that uses the tool to process the input.""" instructions = ( "You are a text processing agent. You have 3 tools:\n" "- make_lowercase_tool: converts text to lowercase\n" "- make_capitalize_tool: capitalizes first letter of each word\n" "- count_words_tool: counts words in text\n\n" "IMPORTANT: When given a text and a request, immediately use the appropriate tool to process the text. " "Do NOT explain your capabilities - just do the work and return the results." ) agent = Agent( name="Smart Text Processing Agent", instructions=instructions, tools=[make_lowercase_tool], model="gpt-4o-mini-2024-07-18" ) prompt = f"Process this text: '{text}'. Request: {user_query}. Use the appropriate tool immediately." result = await Runner.run(agent, prompt) return result.final_output ```
yindo closed this issue 2026-02-21 17:17:09 -05:00
yindo commented 2026-02-21 17:17:10 -05:00
Author
Owner
Copy Link
Copy Source

@angus-langchain commented on GitHub (Aug 18, 2025):

@dast-draftwise Currently the OpenAIAgentsTracingProcessor() doesn't work alongside other tracing methods, meaning you can't trace agents sdk and custom LangSmith/LangChain spans in the same trace.

@angus-langchain commented on GitHub (Aug 18, 2025): @dast-draftwise Currently the OpenAIAgentsTracingProcessor() doesn't work alongside other tracing methods, meaning you can't trace agents sdk and custom LangSmith/LangChain spans in the same trace.
yindo commented 2026-02-21 17:17:10 -05:00
Author
Owner
Copy Link
Copy Source

@dast-draftwise commented on GitHub (Aug 18, 2025):

Is there any plan to support that, or at least add more configuration options to it?
Its extremely not configurable, which makes it hard to integrate with other tools.

@dast-draftwise commented on GitHub (Aug 18, 2025): Is there any plan to support that, or at least add more configuration options to it? Its extremely not configurable, which makes it hard to integrate with other tools.
yindo commented 2026-02-21 17:17:10 -05:00
Author
Owner
Copy Link
Copy Source

@angus-langchain commented on GitHub (Aug 19, 2025):

I could support having the integration inherit langsmith context if available, but that would only work unidirectionally. Meaning openai agents could be parented by a langsmith span, but not vice versa. Would that help?

@angus-langchain commented on GitHub (Aug 19, 2025): I could support having the integration inherit langsmith context if available, but that would only work unidirectionally. Meaning openai agents could be parented by a langsmith span, but not vice versa. Would that help?
yindo commented 2026-02-21 17:17:10 -05:00
Author
Owner
Copy Link
Copy Source

@dast-draftwise commented on GitHub (Aug 19, 2025):

Would that preserve the order of the spans within the trace and place them under one trace? - if yes, that would help - if no, than probably nope.

I could support having the integration inherit langsmith context if available, but that would only work unidirectionally. Meaning openai agents could be parented by a langsmith span, but not vice versa. Would that help?

Given my example, which I think quite well demonstrates the integration - do you know how to add efficiently metadata?
I added @traceable(metadata={}) to the agent run_smart_text_agent but it doesnt set the metadata in the langsmith . Given that the spans cant be placed under same trace, I am trying to add a metadata to each of the spans (agent, tool) which they share, and can be filtered by.

Image
@dast-draftwise commented on GitHub (Aug 19, 2025): Would that preserve the order of the spans within the trace and place them under one trace? - if yes, that would help - if no, than probably nope. > I could support having the integration inherit langsmith context if available, but that would only work unidirectionally. Meaning openai agents could be parented by a langsmith span, but not vice versa. Would that help? Given my example, which I think quite well demonstrates the integration - do you know how to add efficiently metadata? I added @traceable(metadata={}) to the agent `run_smart_text_agent` but it doesnt set the metadata in the langsmith . Given that the spans cant be placed under same trace, I am trying to add a metadata to each of the spans (agent, tool) which they share, and can be filtered by. <img width="949" height="747" alt="Image" src="https://github.com/user-attachments/assets/8bd65018-3f54-43c8-bbd4-a70456bdf408" />
yindo commented 2026-02-21 17:17:10 -05:00
Author
Owner
Copy Link
Copy Source

@angus-langchain commented on GitHub (Aug 19, 2025):

I saw you opened a duplicate issue on our forum, I will continue the conversation there

https://forum.langchain.com/t/openai-agent-sdk-langsmith-integration/1254/2

@angus-langchain commented on GitHub (Aug 19, 2025): I saw you opened a duplicate issue on our forum, I will continue the conversation there https://forum.langchain.com/t/openai-agent-sdk-langsmith-integration/1254/2
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
pull-request
Mirrored from GitHub Pull Request
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langsmith-docs#88
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?