run-llama/workflows-py
1
0
Fork 0
mirror of https://github.com/run-llama/workflows-py.git synced 2026-07-21 04:05:25 -04:00
Code Issues 18 Packages Projects Releases 10 Wiki Activity

LlamaIndexInstrumentor doesn't automatically instrument the Workflow runs as a hierarchy #15

New Issue
Closed
opened 2026-02-16 02:16:05 -05:00 by yindo · 6 comments
yindo commented 2026-02-16 02:16:05 -05:00
Owner
Copy Link
Copy Source

Originally created by @ShresthaYadavilli on GitHub (Aug 8, 2025).

LlamaIndexInstrumentor to save traces in Arize Phoenix for my workflow but the traces in each workflow steps are saved as individual chat completion line items instead of workflow runs.

This issue is only when I use the new python libraries below:

from workflows import Workflow, step, Context
from workflows.events import StartEvent, StopEvent, Event

When I change the python libraries to the old workflow it instruments correctly:
from llama_index.core.workflow import StartEvent, StopEvent, Workflow, step, Context, Event

Originally created by @ShresthaYadavilli on GitHub (Aug 8, 2025). LlamaIndexInstrumentor to save traces in Arize Phoenix for my workflow but the traces in each workflow steps are saved as individual chat completion line items instead of workflow runs. This issue is only when I use the new python libraries below: from workflows import Workflow, step, Context from workflows.events import StartEvent, StopEvent, Event When I change the python libraries to the old workflow it instruments correctly: from llama_index.core.workflow import StartEvent, StopEvent, Workflow, step, Context, Event
yindo closed this issue 2026-02-16 02:16:05 -05:00
yindo commented 2026-02-16 02:16:05 -05:00
Author
Owner
Copy Link
Copy Source

@logan-markewich commented on GitHub (Aug 8, 2025):

There is no difference between those imports, they are quite literally the same thing lol

For example, from llama_index.core.workflow import Workflow is just mapped to this repo
https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/workflow/workflow.py

Can you provide a way to reproduce, maybe in google colab? Or maybe include some screenshots? Anything helps

@logan-markewich commented on GitHub (Aug 8, 2025): There is no difference between those imports, they are quite literally the same thing lol For example, `from llama_index.core.workflow import Workflow` is just mapped to this repo https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/workflow/workflow.py Can you provide a way to reproduce, maybe in google colab? Or maybe include some screenshots? Anything helps
yindo commented 2026-02-16 02:16:05 -05:00
Author
Owner
Copy Link
Copy Source

@ShresthaYadavilli commented on GitHub (Aug 8, 2025):

This is a test code that you can use to reproduce the issue:

import os
import asyncio
from workflows import Workflow, step, Context
from workflows.events import StartEvent, StopEvent, Event

from llama_index.core.workflow import Event

from llama_index.core.workflow import StartEvent, StopEvent, Workflow, step, Context

from openai import AsyncAzureOpenAI
from dotenv import load_dotenv
from openinference.instrumentation.llama_index import get_current_span
from opentelemetry import trace as trace_api
from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from phoenix.otel import register

tracer_provider = register(
project_name="test_workflow_run",
auto_instrument=True,
endpoint=os.environ.get("ARIZE_ENDPOINT", "")
)
LlamaIndexInstrumentor().instrument(tracer_provider=tracer_provider)
load_dotenv()

client = AsyncAzureOpenAI(
azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT", ""),
api_key=os.getenv("AZURE_OPENAI_API_KEY", ""),
api_version=os.getenv("AZURE_OPENAI_API_VERSION", ""),
max_retries=3,
)
model_name = "gpt-4.1-mini"

class InputEvent(Event):
file_path: str

class LLMResult(Event):
content: str

class SimpleTestWorkflow(Workflow):

@step
async def start(self, ctx: Context, ev: StartEvent) -> InputEvent:
    # Just pass the file_path as data
    return InputEvent(
        file_path=ev.file_path
    )

@step
async def get_file_name(self, ctx: Context, ev: InputEvent) -> LLMResult:
    prompt = f"Read the file path and respond with the name of the file: {ev.file_path}"
    messages = [
        {"role": "system", "content": "You are an AI assistant"},
        {"role": "user", "content": prompt}
    ]
    parent_span = get_current_span()
    with trace_api.use_span(parent_span, end_on_exit=False):
        response = await client.beta.chat.completions.parse(
            model = model_name,
            messages=messages,
            seed=42,
            temperature=0.1,
            max_completion_tokens=32000
        )
    content = response.choices[0].message.content
    return LLMResult(content=content)

@step
async def rename_file_name(self, ctx: Context, ev: LLMResult) -> StopEvent:
    prompt = f"Rename this given file name in a standard format: {ev.content}"
    messages = [
        {"role": "system", "content": "You are an AI assistant"},
        {"role": "user", "content": prompt}
    ]
    parent_span = get_current_span()
    with trace_api.use_span(parent_span, end_on_exit=False):
        response = await client.beta.chat.completions.parse(
            model = model_name,
            messages=messages,
            seed=42,
            temperature=0.1,
            max_completion_tokens=32000
        )
    content = response.choices[0].message.content
    return StopEvent(result=content)

async def main():
workflow = SimpleTestWorkflow(timeout=5000)
file_path = input("Enter any test string or file path: ")
result = await workflow.run(file_path=file_path)
print("Workflow finished with result:")
print(result)

if name == "main":
asyncio.run(main())

@ShresthaYadavilli commented on GitHub (Aug 8, 2025): This is a test code that you can use to reproduce the issue: > import os > import asyncio > from workflows import Workflow, step, Context > from workflows.events import StartEvent, StopEvent, Event > # from llama_index.core.workflow import Event > # from llama_index.core.workflow import StartEvent, StopEvent, Workflow, step, Context > from openai import AsyncAzureOpenAI > from dotenv import load_dotenv > from openinference.instrumentation.llama_index import get_current_span > from opentelemetry import trace as trace_api > from openinference.instrumentation.llama_index import LlamaIndexInstrumentor > from phoenix.otel import register > > tracer_provider = register( > project_name="test_workflow_run", > auto_instrument=True, > endpoint=os.environ.get("ARIZE_ENDPOINT", "") > ) > LlamaIndexInstrumentor().instrument(tracer_provider=tracer_provider) > load_dotenv() > > client = AsyncAzureOpenAI( > azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT", ""), > api_key=os.getenv("AZURE_OPENAI_API_KEY", ""), > api_version=os.getenv("AZURE_OPENAI_API_VERSION", ""), > max_retries=3, > ) > model_name = "gpt-4.1-mini" > > class InputEvent(Event): > file_path: str > > class LLMResult(Event): > content: str > > class SimpleTestWorkflow(Workflow): > > > @step > async def start(self, ctx: Context, ev: StartEvent) -> InputEvent: > # Just pass the file_path as data > return InputEvent( > file_path=ev.file_path > ) > > @step > async def get_file_name(self, ctx: Context, ev: InputEvent) -> LLMResult: > prompt = f"Read the file path and respond with the name of the file: {ev.file_path}" > messages = [ > {"role": "system", "content": "You are an AI assistant"}, > {"role": "user", "content": prompt} > ] > parent_span = get_current_span() > with trace_api.use_span(parent_span, end_on_exit=False): > response = await client.beta.chat.completions.parse( > model = model_name, > messages=messages, > seed=42, > temperature=0.1, > max_completion_tokens=32000 > ) > content = response.choices[0].message.content > return LLMResult(content=content) > > @step > async def rename_file_name(self, ctx: Context, ev: LLMResult) -> StopEvent: > prompt = f"Rename this given file name in a standard format: {ev.content}" > messages = [ > {"role": "system", "content": "You are an AI assistant"}, > {"role": "user", "content": prompt} > ] > parent_span = get_current_span() > with trace_api.use_span(parent_span, end_on_exit=False): > response = await client.beta.chat.completions.parse( > model = model_name, > messages=messages, > seed=42, > temperature=0.1, > max_completion_tokens=32000 > ) > content = response.choices[0].message.content > return StopEvent(result=content) > > async def main(): > workflow = SimpleTestWorkflow(timeout=5000) > file_path = input("Enter any test string or file path: ") > result = await workflow.run(file_path=file_path) > print("Workflow finished with result:") > print(result) > > if __name__ == "__main__": > asyncio.run(main()) >
yindo commented 2026-02-16 02:16:05 -05:00
Author
Owner
Copy Link
Copy Source

@logan-markewich commented on GitHub (Aug 10, 2025):

I cannot reproduce

Image

https://colab.research.google.com/drive/1yd8TSBkOqRFth0D_6LzcaBNBkXSXuvuZ?usp=sharing

Also, I might recommend avoiding touching open telemetry and parent spans directly? I would create my own spans and events using the instrumentation library directly
https://github.com/run-llama/workflows-py/blob/main/examples/observability/workflows_observability_pt1.ipynb

@logan-markewich commented on GitHub (Aug 10, 2025): I cannot reproduce <img width="1190" height="665" alt="Image" src="https://github.com/user-attachments/assets/7a83f60a-d2cc-4aca-baf6-71a9450d9922" /> https://colab.research.google.com/drive/1yd8TSBkOqRFth0D_6LzcaBNBkXSXuvuZ?usp=sharing Also, I might recommend avoiding touching open telemetry and parent spans directly? I would create my own spans and events using the instrumentation library directly https://github.com/run-llama/workflows-py/blob/main/examples/observability/workflows_observability_pt1.ipynb
yindo commented 2026-02-16 02:16:06 -05:00
Author
Owner
Copy Link
Copy Source

@ShresthaYadavilli commented on GitHub (Sep 9, 2025):

You can reproduce it if you comment out these imports:

from llama_index.core.workflow import Event
from llama_index.core.workflow import StartEvent, StopEvent, Workflow, step, Context

and only use the imports below:

from workflows import Workflow, step, Context
from workflows.events import StartEvent, StopEvent, Event

Image

Could you also let me know the versions of the libraries you are using currently?

@ShresthaYadavilli commented on GitHub (Sep 9, 2025): You can reproduce it if you comment out these imports: from llama_index.core.workflow import Event from llama_index.core.workflow import StartEvent, StopEvent, Workflow, step, Context and only use the imports below: from workflows import Workflow, step, Context from workflows.events import StartEvent, StopEvent, Event <img width="1538" height="414" alt="Image" src="https://github.com/user-attachments/assets/8351279a-abd5-403e-88b2-0ae9203ae6a6" /> Could you also let me know the versions of the libraries you are using currently?
yindo commented 2026-02-16 02:16:06 -05:00
Author
Owner
Copy Link
Copy Source

@CharlyJazz commented on GitHub (Oct 29, 2025):

I cannot reproduce

Image https://colab.research.google.com/drive/1yd8TSBkOqRFth0D_6LzcaBNBkXSXuvuZ?usp=sharing

Also, I might recommend avoiding touching open telemetry and parent spans directly? I would create my own spans and events using the instrumentation library directly https://github.com/run-llama/workflows-py/blob/main/examples/observability/workflows_observability_pt1.ipynb

which tool UI is that????

@CharlyJazz commented on GitHub (Oct 29, 2025): > I cannot reproduce > > <img alt="Image" width="1190" height="665" src="https://private-user-images.githubusercontent.com/22285038/476362950-7a83f60a-d2cc-4aca-baf6-71a9450d9922.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NjE3NDU5NjYsIm5iZiI6MTc2MTc0NTY2NiwicGF0aCI6Ii8yMjI4NTAzOC80NzYzNjI5NTAtN2E4M2Y2MGEtZDJjYy00YWNhLWJhZjYtNzFhOTQ1MGQ5OTIyLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTEwMjklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUxMDI5VDEzNDc0NlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPThiYWE1ODFhN2EwOTgyZjE2ODAxYzZjZjllYzhiMTQ0ODk0NmUyNzVkODg4MmU5Y2UwYWJmMjE3MDc5MzlhNTYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.lYyMwlbTlTyW-UI3K2seGx2qeoi6rZONYt_A4jlBz8s"> > https://colab.research.google.com/drive/1yd8TSBkOqRFth0D_6LzcaBNBkXSXuvuZ?usp=sharing > > Also, I might recommend avoiding touching open telemetry and parent spans directly? I would create my own spans and events using the instrumentation library directly https://github.com/run-llama/workflows-py/blob/main/examples/observability/workflows_observability_pt1.ipynb which tool UI is that????
yindo commented 2026-02-16 02:16:06 -05:00
Author
Owner
Copy Link
Copy Source

@logan-markewich commented on GitHub (Oct 29, 2025):

@CharlyJazz its arize https://arize.com/docs/phoenix

@logan-markewich commented on GitHub (Oct 29, 2025): @CharlyJazz its arize https://arize.com/docs/phoenix
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
enhancement
enhancement
good first issue
pull-request
Mirrored from GitHub Pull Request
 question
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: run-llama/workflows-py#15
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?