langchain-ai/langsmith-docs
1
0
Fork 0
mirror of https://github.com/langchain-ai/langsmith-docs.git synced 2026-08-27 01:41:19 -04:00
Code Issues 16 Packages Projects Releases Wiki Activity

DOC: <Issue related to /observability/how_to_guides/tracing/trace_with_opentelemetry> #48

New Issue
Closed
opened 2026-02-21 17:17:03 -05:00 by yindo · 5 comments
yindo commented 2026-02-21 17:17:03 -05:00
Owner
Copy Link
Copy Source

Originally created by @mindful-time on GitHub (Dec 23, 2024).

Originally assigned to: @angus-langchain on GitHub.

Issue: Unauthorized Access When Integrating Traceloop SDK with LangSmith with LLama-index

Description

When attempting to integrate Traceloop SDK with LangSmith for tracing LlamaIndex operations, receiving consistent 401 Unauthorized errors. The trace export attempts are failing with the following error:

ERROR:
opentelemetry.exporter.otlp.proto.http.trace_exporter: Failed to export batch 
code: 401, 
reason: {"error":"Unauthorized"}

Current Setup

  • Using Traceloop SDK to send traces to LangSmith's OpenTelemetry endpoint
  • Environment configured with Azure OpenAI for LlamaIndex operations
  • Traceloop initialization includes:

Steps to Reproduce

  1. Set up environment with LangSmith API key
  2. Initialize Traceloop SDK with LangSmith endpoint
  3. Run any LlamaIndex operation (in this case, a simple document query)
  4. Observe trace export failures in logs

Expected Behavior

  • Successful authentication with LangSmith
  • Proper export of OpenTelemetry traces to LangSmith

Actual Behavior

  • 401 Unauthorized errors when attempting to export traces
  • No traces being recorded in LangSmith

Potential Investigation Done

  1. Verifed LangSmith API key validity and permissions
  2. Confirmed correct environment variable name (LANGCHAIN_API_KEY)
  3. Validate LangSmith endpoint URL and authentication requirements
  4. Check Traceloop SDK configuration for LangSmith compatibility

Related Documentation

full implementation

# Initialize Traceloop SDK and LlamaIndex with Azure OpenAI
# This script sets up tracing and testing of the LlamaIndex integration with Azure OpenAI

import os
from traceloop.sdk import Traceloop

# Get LangSmith API key from environment variables for tracing
LANGSMITH_API_KEY = os.getenv("LANGCHAIN_API_KEY")

# Initialize Traceloop with LangSmith endpoint and authentication
# - api_endpoint: LangSmith OTEL endpoint for trace collection
# - headers: Authentication and content type headers
# - disable_batch: Send traces immediately without batching
# - app_name: Name of the application for trace identification
Traceloop.init(api_endpoint="https://api.smith.langchain.com/otel",
               headers=
               {
                "x-api-key": LANGSMITH_API_KEY, 
                "content-type": "application/protobuf"},
               disable_batch=True,
               app_name="test"
               )

# Import required LlamaIndex components
from llama_index.core import VectorStoreIndex, Document
from llama_index.llms.azure_openai import AzureOpenAI
from llama_index.embeddings.azure_openai import AzureOpenAIEmbedding
from llama_index.core import Settings
from app.core.config import get_settings

# Load application settings
settings = get_settings()

# Define required Azure OpenAI settings to validate configuration
required_settings = [
        ("Azure OpenAI API Key", settings.azure_openai_api_key),
        ("Azure OpenAI Endpoint", settings.azure_openai_endpoint),
        ("Azure OpenAI Deployment Name", settings.azure_openai_deployment_name),
        ("Azure OpenAI API Version", settings.azure_openai_api_version),
        ("Azure OpenAI Embeddings Name", settings.azure_openai_embeddings_name),
        ("Azure OpenAI Embeddings Endpoint", settings.azure_openai_embeddings_endpoint),
    ]

# Configure LlamaIndex to use Azure OpenAI for text generation
Settings.llm = AzureOpenAI(
            model=settings.text_model,
            engine=settings.azure_openai_deployment_name,
            deployment_name=settings.azure_openai_deployment_name,
            api_key=settings.azure_openai_api_key,
            azure_endpoint=settings.azure_openai_endpoint,
            api_version=settings.azure_openai_api_version,
        )

# Configure LlamaIndex to use Azure OpenAI for embeddings
Settings.embed_model = AzureOpenAIEmbedding(
            model=settings.azure_openai_embeddings_model,
            deployment_name=settings.azure_openai_embeddings_name,
            api_key=settings.azure_openai_api_key,
            azure_endpoint=settings.azure_openai_embeddings_endpoint,
            api_version=settings.azure_openai_embeddings_api_version,
        )

# Test the setup with a sample document and query
try:
    # Create test index with example document
    documents = [Document.example()]
    index = VectorStoreIndex.from_documents(documents)
    query_engine = index.as_query_engine()
    
    # Run test query
    response = query_engine.query("What is this document about?")
    print(f"Query Response: {response}")
except Exception as e:
    print(f"Error occurred: {str(e)}")

HTTP Request is working

image

Originally created by @mindful-time on GitHub (Dec 23, 2024). Originally assigned to: @angus-langchain on GitHub. # Issue: Unauthorized Access When Integrating Traceloop SDK with LangSmith with LLama-index ## Description When attempting to integrate Traceloop SDK with LangSmith for tracing LlamaIndex operations, receiving consistent 401 Unauthorized errors. The trace export attempts are failing with the following error: ``` ERROR: opentelemetry.exporter.otlp.proto.http.trace_exporter: Failed to export batch code: 401, reason: {"error":"Unauthorized"} ``` ## Current Setup - Using Traceloop SDK to send traces to LangSmith's OpenTelemetry endpoint - Environment configured with Azure OpenAI for LlamaIndex operations - Traceloop initialization includes: - API endpoint: https://api.smith.langchain.com/otel - Headers with LangSmith API key and content type - Batch disable flag set to true ## Steps to Reproduce 1. Set up environment with LangSmith API key 2. Initialize Traceloop SDK with LangSmith endpoint 3. Run any LlamaIndex operation (in this case, a simple document query) 4. Observe trace export failures in logs ## Expected Behavior - Successful authentication with LangSmith - Proper export of OpenTelemetry traces to LangSmith ## Actual Behavior - 401 Unauthorized errors when attempting to export traces - No traces being recorded in LangSmith ## Potential Investigation Done 1. Verifed LangSmith API key validity and permissions 2. Confirmed correct environment variable name (`LANGCHAIN_API_KEY`) 3. Validate LangSmith endpoint URL and authentication requirements 4. Check Traceloop SDK configuration for LangSmith compatibility ## Related Documentation - [OpenTelemetry Tracing Guide for Langsmith](https://docs.smith.langchain.com/observability/how_to_guides/tracing/trace_with_opentelemetry) - [OpenTelemetry Tracing Guide for Llama Index](https://docs.llamaindex.ai/en/stable/module_guides/observability/) - [OpenTelemetry Tracing Guide for Traceloop](https://www.traceloop.com/docs/openllmetry/integrations/introduction) ### full implementation ```python # Initialize Traceloop SDK and LlamaIndex with Azure OpenAI # This script sets up tracing and testing of the LlamaIndex integration with Azure OpenAI import os from traceloop.sdk import Traceloop # Get LangSmith API key from environment variables for tracing LANGSMITH_API_KEY = os.getenv("LANGCHAIN_API_KEY") # Initialize Traceloop with LangSmith endpoint and authentication # - api_endpoint: LangSmith OTEL endpoint for trace collection # - headers: Authentication and content type headers # - disable_batch: Send traces immediately without batching # - app_name: Name of the application for trace identification Traceloop.init(api_endpoint="https://api.smith.langchain.com/otel", headers= { "x-api-key": LANGSMITH_API_KEY, "content-type": "application/protobuf"}, disable_batch=True, app_name="test" ) # Import required LlamaIndex components from llama_index.core import VectorStoreIndex, Document from llama_index.llms.azure_openai import AzureOpenAI from llama_index.embeddings.azure_openai import AzureOpenAIEmbedding from llama_index.core import Settings from app.core.config import get_settings # Load application settings settings = get_settings() # Define required Azure OpenAI settings to validate configuration required_settings = [ ("Azure OpenAI API Key", settings.azure_openai_api_key), ("Azure OpenAI Endpoint", settings.azure_openai_endpoint), ("Azure OpenAI Deployment Name", settings.azure_openai_deployment_name), ("Azure OpenAI API Version", settings.azure_openai_api_version), ("Azure OpenAI Embeddings Name", settings.azure_openai_embeddings_name), ("Azure OpenAI Embeddings Endpoint", settings.azure_openai_embeddings_endpoint), ] # Configure LlamaIndex to use Azure OpenAI for text generation Settings.llm = AzureOpenAI( model=settings.text_model, engine=settings.azure_openai_deployment_name, deployment_name=settings.azure_openai_deployment_name, api_key=settings.azure_openai_api_key, azure_endpoint=settings.azure_openai_endpoint, api_version=settings.azure_openai_api_version, ) # Configure LlamaIndex to use Azure OpenAI for embeddings Settings.embed_model = AzureOpenAIEmbedding( model=settings.azure_openai_embeddings_model, deployment_name=settings.azure_openai_embeddings_name, api_key=settings.azure_openai_api_key, azure_endpoint=settings.azure_openai_embeddings_endpoint, api_version=settings.azure_openai_embeddings_api_version, ) # Test the setup with a sample document and query try: # Create test index with example document documents = [Document.example()] index = VectorStoreIndex.from_documents(documents) query_engine = index.as_query_engine() # Run test query response = query_engine.query("What is this document about?") print(f"Query Response: {response}") except Exception as e: print(f"Error occurred: {str(e)}") ``` # HTTP Request is working ![image](https://github.com/user-attachments/assets/e57665cf-7820-4252-9602-610d71c55484)
yindo closed this issue 2026-02-21 17:17:03 -05:00
yindo commented 2026-02-21 17:17:03 -05:00
Author
Owner
Copy Link
Copy Source

@angus-langchain commented on GitHub (Apr 5, 2025):

Hi @mindful-time, Can you confirm that content encoding is set to proto. Also, could you confirm that if you create a new API key in the app and re-run this code it still doesn't work?

@angus-langchain commented on GitHub (Apr 5, 2025): Hi @mindful-time, Can you confirm that content encoding is set to `proto`. Also, could you confirm that if you create a new API key in the app and re-run this code it still doesn't work?
yindo commented 2026-02-21 17:17:03 -05:00
Author
Owner
Copy Link
Copy Source

@mindful-time commented on GitHub (Apr 14, 2025):

Hi @angus-langchain sorry was busy, can we make this open. The issue still persists

@mindful-time commented on GitHub (Apr 14, 2025): Hi @angus-langchain sorry was busy, can we make this open. The issue still persists
yindo commented 2026-02-21 17:17:03 -05:00
Author
Owner
Copy Link
Copy Source

@angus-langchain commented on GitHub (Apr 14, 2025):

Hi @mindful-time, I'm unable to repro with the code you shared. Are you able to trace to LangSmith normally?

https://smith.langchain.com/public/277dcdd3-c0d1-412e-9cd9-ae7abf4b8262/r

@angus-langchain commented on GitHub (Apr 14, 2025): Hi @mindful-time, I'm unable to repro with the code you shared. Are you able to trace to LangSmith normally? https://smith.langchain.com/public/277dcdd3-c0d1-412e-9cd9-ae7abf4b8262/r
yindo commented 2026-02-21 17:17:03 -05:00
Author
Owner
Copy Link
Copy Source

@mindful-time commented on GitHub (Apr 16, 2025):

@angus-langchain do you mean with llama-index or in general with langchain/langgraph ? If it is general with langchain and langgrpah i am , but the llama-index isn't working based on the from traceloop.sdk

@mindful-time commented on GitHub (Apr 16, 2025): @angus-langchain do you mean with llama-index or in general with langchain/langgraph ? If it is general with langchain and langgrpah i am , but the llama-index isn't working based on the from traceloop.sdk
yindo commented 2026-02-21 17:17:03 -05:00
Author
Owner
Copy Link
Copy Source

@mindful-time commented on GitHub (Apr 16, 2025):

@angus-langchain changed the code to following seemed to work !

`# initialize Traceloop with LangSmith endpoint and authentication environment variables
os.environ["TRACELOOP_BASE_URL"] = "https://api.smith.langchain.com/otel"
os.environ["TRACELOOP_HEADERS"] = "x-api-key=" + langsmith_settings.api_key + ",Langsmith-Project=" + langsmith_settings.project

Traceloop.init()`

Changed code:
updated code repo langsmith-llamaindex
Btw any idea why all the trace aren't coming as one run and three

Image

I think we should use the create_global_handler in llama_index
https://docs.llamaindex.ai/en/stable/module_guides/observability/

Found here :
https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/callbacks/global_handlers.py

PHOENIX_API_KEY = "<PHOENIX_API_KEY>"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = f"api_key={PHOENIX_API_KEY}"
llama_index.core.set_global_handler(
"arize_phoenix", endpoint="https://llamatrace.com/v1/traces"
)

Image

@mindful-time commented on GitHub (Apr 16, 2025): @angus-langchain changed the code to following seemed to work ! `# initialize Traceloop with LangSmith endpoint and authentication environment variables os.environ["TRACELOOP_BASE_URL"] = "https://api.smith.langchain.com/otel" os.environ["TRACELOOP_HEADERS"] = "x-api-key=" + langsmith_settings.api_key + ",Langsmith-Project=" + langsmith_settings.project Traceloop.init()` Changed code: [updated code repo langsmith-llamaindex ](https://github.com/mindful-time/langsmith-llamaindex) Btw any idea why all the trace aren't coming as one run and three ![Image](https://github.com/user-attachments/assets/16f54541-e64b-489b-b5b9-95681732281d) I think we should use the create_global_handler in llama_index https://docs.llamaindex.ai/en/stable/module_guides/observability/ Found here : https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/callbacks/global_handlers.py PHOENIX_API_KEY = "<PHOENIX_API_KEY>" os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = f"api_key={PHOENIX_API_KEY}" llama_index.core.set_global_handler( "arize_phoenix", endpoint="https://llamatrace.com/v1/traces" ) ![Image](https://github.com/user-attachments/assets/a0a61f0a-552c-4f53-bf5d-01169076b256)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
pull-request
Mirrored from GitHub Pull Request
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langsmith-docs#48
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?