From 38039bf5bf5de2a2d912af73102543d236ace54a Mon Sep 17 00:00:00 2001 From: Kaustubh Date: Thu, 17 Sep 2026 14:12:48 +0530 Subject: [PATCH] cookbook: add Confident AI observability example (#10257) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Adds a runnable observability cookbook for [Confident AI](https://www.confident-ai.com/) using its OpenTelemetry-native `confident-trace` SDK, which detects Agno automatically. - Add `cookbook/observability/confident_ai.py`: calls `init()` once at startup, runs a Hacker News agent (OpenAIResponses, `gpt-5.6-luna`) twice, wraps the second run in `trace_context` to attach tags, metadata, and a user ID, and calls `shutdown()` in a `finally` block to flush spans. - List the example in `cookbook/observability/README.md`. - Record the test run in `cookbook/observability/TEST_LOG.md`. Companion docs: agno-agi/docs#767 (Mintlify) and agno-agi/agno-docs#141 (in-house). The in-house examples page links to this cookbook on `main`. ## Type of change - [ ] Bug fix - [ ] New feature - [ ] Breaking change - [x] Improvement - [ ] Model update - [ ] Other: --- ## Checklist - [x] Code complies with style guidelines - [x] Ran format/validation scripts (`./scripts/format.sh` and `./scripts/validate.sh`) - [x] Self-review completed - [x] Documentation updated (comments, docstrings) - [x] Examples and guides: Relevant cookbook examples have been included or updated (if applicable) - [x] Tested in clean environment - [ ] Tests added/updated (if applicable) ### Duplicate and AI-Generated PR Check - [x] I have searched existing [open pull requests](https://github.com/agno-agi/agno/pulls) and confirmed that no other PR already addresses this issue - [ ] If a similar PR exists, I have explained below why this PR is a better approach - [x] Check if this PR was entirely AI-generated (by Copilot, Claude Code, Cursor, etc.) --- ## Additional Notes - Verified against `confident-trace==0.1.3`. Its OpenAI integration patches the Responses API, so `OpenAIResponses` model spans are captured alongside the Agno agent and tool spans. - Ran end to end with `CONFIDENT_API_KEY` and `OPENAI_API_KEY` set. Both runs completed and span batches were accepted by the collector with HTTP 200. - `CONFIDENT_OTEL_ENDPOINT` must match the project's region. A US project key sent to the EU endpoint returns 401 on export, so only set it for EU projects. The module docstring calls this out. - `cookbook/scripts/check_cookbook_pattern.py` passes for the new file. The two existing violations it reports in `mlflow_via_autolog.py` are pre-existing and untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- cookbook/observability/README.md | 1 + cookbook/observability/TEST_LOG.md | 9 ++++ cookbook/observability/confident_ai.py | 65 ++++++++++++++++++++++++++ 3 files changed, 75 insertions(+) create mode 100644 cookbook/observability/confident_ai.py diff --git a/cookbook/observability/README.md b/cookbook/observability/README.md index 7d5227b0a0..d9397eb931 100644 --- a/cookbook/observability/README.md +++ b/cookbook/observability/README.md @@ -9,6 +9,7 @@ Observability examples for tracing and monitoring Agno agents, teams, and workfl - `arize_phoenix_via_openinference.py` - `arize_phoenix_via_openinference_local.py` - `atla_op.py` +- `confident_ai.py` - `langfuse_via_openinference.py` - `langfuse_via_openinference_response_model.py` - `langfuse_via_openlit.py` diff --git a/cookbook/observability/TEST_LOG.md b/cookbook/observability/TEST_LOG.md index d0ba982429..9e9762c54d 100644 --- a/cookbook/observability/TEST_LOG.md +++ b/cookbook/observability/TEST_LOG.md @@ -7,3 +7,12 @@ **Result:** Validation passed with zero violations. Runtime execution of individual cookbook scripts was not performed in this pass. --- +### confident_ai.py + +**Status:** PASS + +**Description:** Sends Agno agent traces to Confident AI through `confident-trace`. Calls `init()` once at startup, runs a HackerNews agent twice (one plain run, one inside `trace_context` with tags, metadata, and a user ID), and calls `shutdown()` in a `finally` block to flush spans. Verified with `ruff format`, `ruff check`, `cookbook/scripts/check_cookbook_pattern.py`, and a module import against `confident-trace==0.1.3` in `.venvs/demo`. + +**Result:** Static checks and import pass. Ran end to end with `CONFIDENT_API_KEY` and `OPENAI_API_KEY` set: both agent runs completed, tool and model calls executed, and span batches were accepted by the Confident AI collector with HTTP 200. Note that `CONFIDENT_OTEL_ENDPOINT` must match the project's region; a US project key against the EU endpoint returns 401 on export. + +--- diff --git a/cookbook/observability/confident_ai.py b/cookbook/observability/confident_ai.py new file mode 100644 index 0000000000..b4fc99d943 --- /dev/null +++ b/cookbook/observability/confident_ai.py @@ -0,0 +1,65 @@ +""" +Confident AI Observability Integration +====================================== + +Demonstrates sending Agno agent traces to Confident AI with confident-trace. + +confident-trace is an OpenTelemetry-native tracing SDK that detects Agno +automatically. Call init() once at startup and your agent, tool, and model +calls show up in the Confident AI Observatory with no other changes. + +Setup: + pip install agno openai confident-trace + +Set CONFIDENT_API_KEY and OPENAI_API_KEY before running this example. +For the EU region, also set CONFIDENT_OTEL_ENDPOINT to +https://eu.otel.confident-ai.com/v1/traces. +See https://www.confident-ai.com/docs/llm-tracing/introduction for details. +""" + +from agno.agent import Agent +from agno.models.openai import OpenAIResponses +from agno.tools.hackernews import HackerNewsTools +from confident_trace import init, shutdown, trace_context + +# --------------------------------------------------------------------------- +# Setup +# --------------------------------------------------------------------------- +# Initialize once at startup. This reads CONFIDENT_API_KEY from the +# environment and instruments Agno and the OpenAI SDK. +init() + + +# --------------------------------------------------------------------------- +# Create Agent +# --------------------------------------------------------------------------- +agent = Agent( + name="Hacker News Agent", + model=OpenAIResponses(id="gpt-5.6-luna"), + tools=[HackerNewsTools()], + instructions="You summarize Hacker News stories. Be concise and cite story titles.", + markdown=True, +) + + +# --------------------------------------------------------------------------- +# Run Example +# --------------------------------------------------------------------------- +if __name__ == "__main__": + try: + # A plain run is traced automatically. + agent.print_response("What are the top 3 stories on Hacker News right now?") + + # Attach tags, metadata, and a user ID to the trace before the run starts. + with trace_context( + tags=["cookbook"], + metadata={"release": "2026-09"}, + user_id="user-42", + ): + agent.print_response( + "Pick one of those stories and explain why it is trending.", + stream=True, + ) + finally: + # Flush pending spans before the process exits. + shutdown()