run-llama/workflows-py
1
0
Fork 0
mirror of https://github.com/run-llama/workflows-py.git synced 2026-07-21 04:05:25 -04:00
Code Issues 18 Packages Projects Releases 10 Wiki Activity

[Bug]: WorkflowValidationError: "Step signature must have at least one parameter annotated as type Event" Despite Proper Event Annotations #8

New Issue
Closed
opened 2026-02-16 02:16:03 -05:00 by yindo · 5 comments
yindo commented 2026-02-16 02:16:03 -05:00
Owner
Copy Link
Copy Source

Originally created by @AnandUgale on GitHub (Jul 29, 2025).

Bug Description

I'm encountering a persistent WorkflowValidationError when defining a custom Workflow class in LlamaIndex. The error occurs during class definition when using the @step decorator on methods that accept custom Event subclasses as parameters. The validation complains that the step signature must have at least one parameter annotated as type Event, even though the parameters are correctly annotated with custom Events that inherit from Event.

This seems to happen regardless of whether I use Pydantic *Field *for event attributes or not. I've tried multiple variations based on the documentation and examples (e.g., ReAct agent workflow and Text-to-SQL workflow), but the issue persists.

Version

llama-cloud==0.1.7
llama-index==0.12.14
llama-index-agent-openai==0.4.1
llama-index-cli==0.4.0
llama-index-core==0.12.14
llama-index-embeddings-fastembed==0.3.0
llama-index-embeddings-huggingface==0.3.1
llama-index-embeddings-openai==0.3.1
llama-index-embeddings-openai-like==0.1.0
llama-index-indices-managed-llama-cloud==0.6.3
llama-index-legacy==0.9.48
llama-index-llms-anyscale==0.1.4
llama-index-llms-fireworks==0.1.5
llama-index-llms-huggingface==0.2.0
llama-index-llms-langchain==0.1.4
llama-index-llms-ollama==0.5.0
llama-index-llms-openai==0.3.13
llama-index-llms-openai-like==0.3.4
llama-index-multi-modal-llms-openai==0.4.1
llama-index-program-openai==0.3.1
llama-index-question-gen-openai==0.3.0
llama-index-readers-file==0.4.1
llama-index-readers-llama-parse==0.4.0
llama-index-utils-workflow==0.3.0
llama-index-vector-stores-chroma==0.4.1
llama-index-vector-stores-milvus==0.5.0

Steps to Reproduce

  1. Use the following SQLAgentWorkflow definition.
  2. Ensure your environment has llama-index==0.12.14 and llama-index-core==0.12.14.
  3. Run the script below:
Code Snippet
# --- Import Statements ---
from __future__ import annotations
import os, ast, asyncio
from typing import List, Dict, Any, Union
import requests
from sqlalchemy import create_engine, text, inspect
from llama_index.core.workflow import (
    Workflow, step, StartEvent, StopEvent, Event, Context
)
from llama_index.core.tools import FunctionTool
from llama_index.core import Settings
from llama_index.core.prompts import PromptTemplate
from llama_index.llms.openai_like import OpenAILike

# --- Configure Gemini LLM ---
Settings.llm = OpenAILike(
    model="gemma3:latest",
    api_base="http://localhost:11434/v1",
    api_key="ollama",
    context_window=8000,
    is_chat_model=True,
    is_function_calling_model=True,
)

# --- Download DB ---
DB_PATH = "Chinook.db"
if not os.path.exists(DB_PATH):
    url = "https://storage.googleapis.com/benchmarks-artifacts/chinook/Chinook.db"
    with open(DB_PATH, "wb") as f:
        f.write(requests.get(url).content)

engine = create_engine(f"sqlite:///{DB_PATH}")

# --- Tool Functions ---
# (list_tables, schema, query, query_checker)
# ... (Omitted here for brevity — same as shared)

# --- Custom Event Classes ---
class ToolCallEvent(Event):
    tool_calls: List[Dict[str, Any]]

class ObservationEvent(Event):
    observations: List[str]

# --- ReAct Prompt ---
REACT_PROMPT = PromptTemplate("""
You are an agent designed to answer questions about a SQL database using the ReAct pattern.
...
""")

# --- SQLAgentWorkflow ---
class SQLAgentWorkflow(Workflow):

    async def _agent_common(self, scratchpad, question) -> Union[ToolCallEvent, StopEvent]:
        # (Tool parsing + ReAct logic)
        ...

    @step
    async def agent_start_step(self, ctx: Context, ev: StartEvent) -> Union[ToolCallEvent, StopEvent]:
        question = getattr(ev, "question", None) or getattr(ev, "kwargs", {}).get("question")
        if question is None:
            raise ValueError("No 'question' provided.")
        await ctx.set("question", question)
        scratchpad = f"Question: {question}\n"
        await ctx.set("scratchpad", scratchpad)
        return await self._agent_common(scratchpad, question)

    @step
    async def agent_continue_step(self, ctx: Context, ev: ObservationEvent) -> Union[ToolCallEvent, StopEvent]:
        scratchpad = await ctx.get("scratchpad", default="")
        for obs in ev.observations:
            scratchpad += f"Observation: {obs}\n"
        await ctx.set("scratchpad", scratchpad)
        question = await ctx.get("question")
        return await self._agent_common(scratchpad, question)

    @step
    async def tool_executor_step(self, ctx: Context, ev: ToolCallEvent) -> ObservationEvent:
        observations = []
        for call in ev.tool_calls:
            tool = next((t for t in tools if t.metadata.name == call["name"]), None)
            if tool:
                result = await asyncio.to_thread(tool.call, **call["args"])
                observations.append(str(result))
            else:
                observations.append(f"Tool '{call['name']}' not found.")
        return ObservationEvent(observations=observations)

# --- Runner ---
async def main():
    wf = SQLAgentWorkflow(timeout=180, verbose=True)
    result = await wf.run(question="Show the first 3 rows from the Artist table.")
    print(result)

if __name__ == "__main__":
    asyncio.run(main())

Relevant Logs/Tracbacks

>python SQL_Agent_4.py
Traceback (most recent call last):
  File "C:\Users\ananu\Documents\Jupyter_Notebook\Envs\llamaindex\SQL_Agent\SQL_Agent_4.py", line 185, in <module>
    class SQLAgentWorkflow(Workflow):
  File "C:\Users\ananu\Documents\Jupyter_Notebook\Envs\llamaindex\SQL_Agent\SQL_Agent_4.py", line 233, in SQLAgentWorkflow
    @step
     ^^^^
  File "C:\Users\ananu\AppData\Local\miniconda3\envs\llamaindex_test\Lib\site-packages\llama_index\core\workflow\decorators.py", line 86, in step
    decorator(func)
  File "C:\Users\ananu\AppData\Local\miniconda3\envs\llamaindex_test\Lib\site-packages\llama_index\core\workflow\decorators.py", line 60, in decorator
    validate_step_signature(spec)
  File "C:\Users\ananu\AppData\Local\miniconda3\envs\llamaindex_test\Lib\site-packages\llama_index\core\workflow\utils.py", line 97, in validate_step_signature
    raise WorkflowValidationError(msg)
llama_index.core.workflow.errors.WorkflowValidationError: Step signature must have at least one parameter annotated as type Event
Originally created by @AnandUgale on GitHub (Jul 29, 2025). ### Bug Description I'm encountering a persistent *WorkflowValidationError* when defining a custom Workflow class in LlamaIndex. The error occurs during class definition when using the *@step* decorator on methods that accept custom *Event* subclasses as parameters. The validation complains that the step signature must have at least one parameter annotated as type *Event*, even though the parameters are correctly annotated with custom Events that inherit from Event. This seems to happen regardless of whether I use Pydantic *Field *for event attributes or not. I've tried multiple variations based on the documentation and examples (e.g., ReAct agent workflow and Text-to-SQL workflow), but the issue persists. ### Version ``` llama-cloud==0.1.7 llama-index==0.12.14 llama-index-agent-openai==0.4.1 llama-index-cli==0.4.0 llama-index-core==0.12.14 llama-index-embeddings-fastembed==0.3.0 llama-index-embeddings-huggingface==0.3.1 llama-index-embeddings-openai==0.3.1 llama-index-embeddings-openai-like==0.1.0 llama-index-indices-managed-llama-cloud==0.6.3 llama-index-legacy==0.9.48 llama-index-llms-anyscale==0.1.4 llama-index-llms-fireworks==0.1.5 llama-index-llms-huggingface==0.2.0 llama-index-llms-langchain==0.1.4 llama-index-llms-ollama==0.5.0 llama-index-llms-openai==0.3.13 llama-index-llms-openai-like==0.3.4 llama-index-multi-modal-llms-openai==0.4.1 llama-index-program-openai==0.3.1 llama-index-question-gen-openai==0.3.0 llama-index-readers-file==0.4.1 llama-index-readers-llama-parse==0.4.0 llama-index-utils-workflow==0.3.0 llama-index-vector-stores-chroma==0.4.1 llama-index-vector-stores-milvus==0.5.0 ``` ### Steps to Reproduce 1. Use the following SQLAgentWorkflow definition. 2. Ensure your environment has llama-index==0.12.14 and llama-index-core==0.12.14. 3. Run the script below: ``` Code Snippet # --- Import Statements --- from __future__ import annotations import os, ast, asyncio from typing import List, Dict, Any, Union import requests from sqlalchemy import create_engine, text, inspect from llama_index.core.workflow import ( Workflow, step, StartEvent, StopEvent, Event, Context ) from llama_index.core.tools import FunctionTool from llama_index.core import Settings from llama_index.core.prompts import PromptTemplate from llama_index.llms.openai_like import OpenAILike # --- Configure Gemini LLM --- Settings.llm = OpenAILike( model="gemma3:latest", api_base="http://localhost:11434/v1", api_key="ollama", context_window=8000, is_chat_model=True, is_function_calling_model=True, ) # --- Download DB --- DB_PATH = "Chinook.db" if not os.path.exists(DB_PATH): url = "https://storage.googleapis.com/benchmarks-artifacts/chinook/Chinook.db" with open(DB_PATH, "wb") as f: f.write(requests.get(url).content) engine = create_engine(f"sqlite:///{DB_PATH}") # --- Tool Functions --- # (list_tables, schema, query, query_checker) # ... (Omitted here for brevity — same as shared) # --- Custom Event Classes --- class ToolCallEvent(Event): tool_calls: List[Dict[str, Any]] class ObservationEvent(Event): observations: List[str] # --- ReAct Prompt --- REACT_PROMPT = PromptTemplate(""" You are an agent designed to answer questions about a SQL database using the ReAct pattern. ... """) # --- SQLAgentWorkflow --- class SQLAgentWorkflow(Workflow): async def _agent_common(self, scratchpad, question) -> Union[ToolCallEvent, StopEvent]: # (Tool parsing + ReAct logic) ... @step async def agent_start_step(self, ctx: Context, ev: StartEvent) -> Union[ToolCallEvent, StopEvent]: question = getattr(ev, "question", None) or getattr(ev, "kwargs", {}).get("question") if question is None: raise ValueError("No 'question' provided.") await ctx.set("question", question) scratchpad = f"Question: {question}\n" await ctx.set("scratchpad", scratchpad) return await self._agent_common(scratchpad, question) @step async def agent_continue_step(self, ctx: Context, ev: ObservationEvent) -> Union[ToolCallEvent, StopEvent]: scratchpad = await ctx.get("scratchpad", default="") for obs in ev.observations: scratchpad += f"Observation: {obs}\n" await ctx.set("scratchpad", scratchpad) question = await ctx.get("question") return await self._agent_common(scratchpad, question) @step async def tool_executor_step(self, ctx: Context, ev: ToolCallEvent) -> ObservationEvent: observations = [] for call in ev.tool_calls: tool = next((t for t in tools if t.metadata.name == call["name"]), None) if tool: result = await asyncio.to_thread(tool.call, **call["args"]) observations.append(str(result)) else: observations.append(f"Tool '{call['name']}' not found.") return ObservationEvent(observations=observations) # --- Runner --- async def main(): wf = SQLAgentWorkflow(timeout=180, verbose=True) result = await wf.run(question="Show the first 3 rows from the Artist table.") print(result) if __name__ == "__main__": asyncio.run(main()) ``` ### Relevant Logs/Tracbacks ```shell >python SQL_Agent_4.py Traceback (most recent call last): File "C:\Users\ananu\Documents\Jupyter_Notebook\Envs\llamaindex\SQL_Agent\SQL_Agent_4.py", line 185, in <module> class SQLAgentWorkflow(Workflow): File "C:\Users\ananu\Documents\Jupyter_Notebook\Envs\llamaindex\SQL_Agent\SQL_Agent_4.py", line 233, in SQLAgentWorkflow @step ^^^^ File "C:\Users\ananu\AppData\Local\miniconda3\envs\llamaindex_test\Lib\site-packages\llama_index\core\workflow\decorators.py", line 86, in step decorator(func) File "C:\Users\ananu\AppData\Local\miniconda3\envs\llamaindex_test\Lib\site-packages\llama_index\core\workflow\decorators.py", line 60, in decorator validate_step_signature(spec) File "C:\Users\ananu\AppData\Local\miniconda3\envs\llamaindex_test\Lib\site-packages\llama_index\core\workflow\utils.py", line 97, in validate_step_signature raise WorkflowValidationError(msg) llama_index.core.workflow.errors.WorkflowValidationError: Step signature must have at least one parameter annotated as type Event ```
yindo added the bug label 2026-02-16 02:16:03 -05:00
yindo closed this issue 2026-02-16 02:16:03 -05:00
yindo commented 2026-02-16 02:16:04 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jul 29, 2025):

This error is almost always caused by one of these issues: inconsistent imports of the Event base class (e.g., importing Event from different modules in your code and the framework), missing or incorrect type annotations on your step parameters, or a version limitation in LlamaIndex 0.12.14.

For your version, here’s what you need to check:

  1. Make sure all your custom event classes (like ToolCallEvent, ObservationEvent) inherit from the exact Event class imported from llama_index.core.workflow.
  2. Every @step-decorated method must have at least one parameter explicitly annotated with your custom Event subclass (e.g., def my_step(self, ctx: Context, ev: ToolCallEvent)), and that subclass must inherit from Event.
  3. All Event imports in your workflow and event definitions must come from llama_index.core.workflow, not from different locations.
  4. Custom StopEvents (or any custom event that is “produced” as a terminal event) are not supported in 0.12.14—using them will trigger validation errors about events being consumed but never produced. You must use the built-in StopEvent only, or upgrade to >=0.12.21 for custom StopEvent support (see discussion).

Here’s a minimal working pattern for step signatures in 0.12.x:

from llama_index.core.workflow import Workflow, step, Event, StartEvent, StopEvent, Context

class MyCustomEvent(Event):
    value: int

class MyWorkflow(Workflow):
    @step
    async def first_step(self, ctx: Context, ev: StartEvent) -> MyCustomEvent:
        return MyCustomEvent(value=42)

    @step
    async def second_step(self, ctx: Context, ev: MyCustomEvent) -> StopEvent:
        return StopEvent(result=ev.value)

If you follow this pattern and still see the error, double-check that there are no duplicate Event definitions (e.g., from different llama_index modules or local copies) and that all type annotations are present and correct. If you need custom StopEvents, you’ll need to upgrade to llama-index-core 0.12.21 or later (see details).

Let me know if you want a more detailed example or want to troubleshoot a specific step signature!

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

@dosubot[bot] commented on GitHub (Jul 29, 2025): <!-- Answer --> This error is almost always caused by one of these issues: inconsistent imports of the Event base class (e.g., importing Event from different modules in your code and the framework), missing or incorrect type annotations on your step parameters, or a version limitation in LlamaIndex 0.12.14. For your version, here’s what you need to check: 1. Make sure all your custom event classes (like ToolCallEvent, ObservationEvent) inherit from the exact Event class imported from llama_index.core.workflow. 2. Every @step-decorated method must have at least one parameter explicitly annotated with your custom Event subclass (e.g., def my_step(self, ctx: Context, ev: ToolCallEvent)), and that subclass must inherit from Event. 3. All Event imports in your workflow and event definitions must come from llama_index.core.workflow, not from different locations. 4. Custom StopEvents (or any custom event that is “produced” as a terminal event) are not supported in 0.12.14—using them will trigger validation errors about events being consumed but never produced. You must use the built-in StopEvent only, or upgrade to >=0.12.21 for custom StopEvent support [(see discussion)](https://github.com/run-llama/llama_index/issues/18208). Here’s a minimal working pattern for step signatures in 0.12.x: ```python from llama_index.core.workflow import Workflow, step, Event, StartEvent, StopEvent, Context class MyCustomEvent(Event): value: int class MyWorkflow(Workflow): @step async def first_step(self, ctx: Context, ev: StartEvent) -> MyCustomEvent: return MyCustomEvent(value=42) @step async def second_step(self, ctx: Context, ev: MyCustomEvent) -> StopEvent: return StopEvent(result=ev.value) ``` If you follow this pattern and still see the error, double-check that there are no duplicate Event definitions (e.g., from different llama_index modules or local copies) and that all type annotations are present and correct. If you need custom StopEvents, you’ll need to upgrade to llama-index-core 0.12.21 or later [(see details)](https://github.com/run-llama/llama_index/issues/18208). Let me know if you want a more detailed example or want to troubleshoot a specific step signature! <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/659e4bf2-6097-40a7-a606-3adefa5cb9c6?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/659e4bf2-6097-40a7-a606-3adefa5cb9c6?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/659e4bf2-6097-40a7-a606-3adefa5cb9c6?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/659e4bf2-6097-40a7-a606-3adefa5cb9c6?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/659e4bf2-6097-40a7-a606-3adefa5cb9c6?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/659e4bf2-6097-40a7-a606-3adefa5cb9c6?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/659e4bf2-6097-40a7-a606-3adefa5cb9c6?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/a347f61c-1d6a-4435-903f-654845df5b7b/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/run-llama/llama_index/issues/19552)
yindo commented 2026-02-16 02:16:04 -05:00
Author
Owner
Copy Link
Copy Source

@AnandUgale commented on GitHub (Jul 29, 2025):

@dosu , I tried with minimal working pattern for step signatures in 0.12.x:, but still getting same error -

error -

>python SQL_Agent_4.py
Traceback (most recent call last):
  File "C:\Users\ananu\Documents\Jupyter_Notebook\Envs\llamaindex\SQL_Agent\SQL_Agent_4.py", line 160, in <module>
    class SQLAgentWorkflow(Workflow):
  File "C:\Users\ananu\Documents\Jupyter_Notebook\Envs\llamaindex\SQL_Agent\SQL_Agent_4.py", line 191, in SQLAgentWorkflow
    @step
     ^^^^
  File "C:\Users\ananu\AppData\Local\miniconda3\envs\llamaindex_test\Lib\site-packages\llama_index\core\workflow\decorators.py", line 86, in step
    decorator(func)
  File "C:\Users\ananu\AppData\Local\miniconda3\envs\llamaindex_test\Lib\site-packages\llama_index\core\workflow\decorators.py", line 60, in decorator
    validate_step_signature(spec)
  File "C:\Users\ananu\AppData\Local\miniconda3\envs\llamaindex_test\Lib\site-packages\llama_index\core\workflow\utils.py", line 97, in validate_step_signature
    raise WorkflowValidationError(msg)
llama_index.core.workflow.errors.WorkflowValidationError: Step signature must have at least one parameter annotated as type Event

code -

"""
SQL_Agent.py

A ReAct-style SQL agent built with LlamaIndex Workflow and Google Gemini (via
`llama-index-llms-google-genai`). It:
  1. Lists tables
  2. Fetches schemas/sample rows
  3. Lets the LLM plan tool calls (ReAct)
  4. Generates/checks SQL
  5. Executes queries and returns a final NL answer

Run:
    pip install -U "llama-index>=0.10" llama-index-llms-google-genai sqlalchemy requests
    # (Optional) silence gRPC warnings by forcing REST transport (already done below)

    set GOOGLE_API_KEY=YOUR_KEY   # Windows (cmd)
    export GOOGLE_API_KEY=YOUR_KEY  # Linux/macOS
"""

from __future__ import annotations
import os
import ast
import asyncio
from typing import List, Dict, Any, Union

import requests
from sqlalchemy import create_engine, text, inspect

from llama_index.core.workflow import (
    Workflow,
    step,
    Event,
    StartEvent,
    StopEvent,
    Context,
)
from llama_index.core.tools import FunctionTool
from llama_index.core import Settings
from llama_index.core.prompts import PromptTemplate
from llama_index.llms.openai_like import OpenAILike

# -----------------------------------------------------------------------------
# 0. Configure Gemini LLM
# -----------------------------------------------------------------------------
# Force REST transport to avoid gRPC shutdown warnings on Windows
Settings.llm = OpenAILike(
    model="gemma3:latest",
    api_base="http://localhost:11434/v1",
    api_key="ollama",
    context_window=8000,
)

# -----------------------------------------------------------------------------
# 1. Download and connect to the Chinook sample database
# -----------------------------------------------------------------------------
DB_PATH = "Chinook.db"
if not os.path.exists(DB_PATH):
    url = "https://storage.googleapis.com/benchmarks-artifacts/chinook/Chinook.db"
    resp = requests.get(url, timeout=30)
    resp.raise_for_status()
    with open(DB_PATH, "wb") as f:
        f.write(resp.content)

engine = create_engine(f"sqlite:///{DB_PATH}")

# -----------------------------------------------------------------------------
# 2. SQL helper tools exposed to the agent
# -----------------------------------------------------------------------------
def sql_db_list_tables() -> str:
    """List all tables in the database."""
    return ", ".join(inspect(engine).get_table_names())

def sql_db_schema(table_names: List[str]) -> str:
    """Return schema + 3 sample rows for each requested table."""
    insp = inspect(engine)
    chunks: List[str] = []
    for table in table_names:
        cols = insp.get_columns(table)
        col_str = ", ".join(f"{c['name']} ({c['type']})" for c in cols)
        sample_sql = f"SELECT * FROM {table} LIMIT 3;"
        rows = engine.connect().execute(text(sample_sql)).fetchall()
        chunks.append(f"Table: {table}\nColumns: {col_str}\nSample rows: {rows}\n")
    return "\n".join(chunks).strip()

def sql_db_query(query: str) -> str:
    """Execute arbitrary SQL (SELECT) and return rows."""
    rows = engine.connect().execute(text(query)).fetchall()
    return str(rows)

def sql_db_query_checker(query: str) -> str:
    """Sanity-check SQL via LLM; return corrected or original query."""
    chk = PromptTemplate(
        """Double-check the following SQL query for mistakes. If there's an issue,
return ONLY a corrected SQL statement. If it's fine, return the original query unchanged.
Do NOT add any explanation.
Query: {query}"""
    )
    resp = Settings.llm.complete(chk.format(query=query))
    return resp.text.strip()

# Wrap as FunctionTools
list_tables_tool = FunctionTool.from_defaults(
    fn=sql_db_list_tables,
    name="sql_db_list_tables",
    description="List all table names in the SQL database.",
)
schema_tool = FunctionTool.from_defaults(
    fn=sql_db_schema,
    name="sql_db_schema",
    description="Input a list of table names to get their schema and 3 sample rows.",
)
query_tool = FunctionTool.from_defaults(
    fn=sql_db_query,
    name="sql_db_query",
    description="Execute a SQL query and return the results.",
)
query_checker_tool = FunctionTool.from_defaults(
    fn=sql_db_query_checker,
    name="sql_db_query_checker",
    description="Check a SQL query for mistakes and return a corrected version only.",
)
tools = [list_tables_tool, schema_tool, query_tool, query_checker_tool]

# -----------------------------------------------------------------------------
# 3. Events for the workflow
# -----------------------------------------------------------------------------
class ToolCallEvent(Event):
    tool_calls: List[Dict[str, Any]]

class ObservationEvent(Event):
    observations: List[str]

# -----------------------------------------------------------------------------
# 4. ReAct prompt template
# -----------------------------------------------------------------------------
REACT_PROMPT = PromptTemplate(
    """
When interacting with the user, you follow ReAct:

Thought: think through what to do next
Action: the action to take, formatted as tool_name(args)
Observation: the result of the action

Repeat Thought/Action/Observation until you can answer:

<final_answer> your concise final answer here </final_answer>

Tools:
{tool_descs}

Question: {question}
Scratchpad:
{scratchpad}
"""
)

# -----------------------------------------------------------------------------
# 5. Workflow Implementation
# -----------------------------------------------------------------------------
class SQLAgentWorkflow(Workflow):
    """A two-step loop: agent_start_step -> tool_executor_step -> agent_continue_step ..."""

    async def _agent_common(self, scratchpad: str, question: str) -> Union[ToolCallEvent, StopEvent]:
        # Format tool list for prompt
        tool_descs = "\n".join(f"{t.metadata.name}: {t.metadata.description}" for t in tools)
        prompt = REACT_PROMPT.format(tool_descs=tool_descs, question=question, scratchpad=scratchpad)
        llm_resp = Settings.llm.complete(prompt)
        text_resp = llm_resp.text

        # Check for final answer
        if "<final_answer>" in text_resp:
            ans = text_resp.split("<final_answer>")[1].split("</final_answer>")[0].strip()
            return StopEvent(result=ans)

        # Parse Action for tool calls
        if "Action:" in text_resp:
            after = text_resp.split("Action:", 1)[1].strip()
            if "(" in after:
                name = after.split("(", 1)[0].strip()
                args_str = after.split("(", 1)[1].rsplit(")", 1)[0].strip()
                try:
                    parsed = ast.literal_eval(args_str)
                    args = parsed if isinstance(parsed, dict) else {"arg": parsed}
                except:
                    args = {"raw": args_str}
                return ToolCallEvent(tool_calls=[{"name": name, "args": args}])

        # Fallback
        return StopEvent(result="LLM did not return a valid Action or <final_answer>.\nResponse:\n" + text_resp)

    @step
    async def agent_start_step(self, ctx: Context, ev: StartEvent) -> Union[ToolCallEvent, StopEvent]:
        """Initialize scratchpad and question."""
        scratchpad = await ctx.get("scratchpad", default="")
        question = getattr(ev, "question", None) or ev.kwargs.get("question")
        if question is None:
            raise ValueError("No 'question' provided to StartEvent.")
        await ctx.set("question", question)
        scratchpad += f"Question: {question}\n"
        await ctx.set("scratchpad", scratchpad)
        return await self._agent_common(scratchpad, question)

    @step
    async def tool_executor_step(self, ctx: Context, ev: ToolCallEvent) -> ObservationEvent:
        """Execute the requested tool and collect observations."""
        obs: List[str] = []
        for call in ev.tool_calls:
            name, args = call["name"], call["args"] or {}
            tool = next((t for t in tools if t.metadata.name == name), None)
            if tool is None:
                obs.append(f"Unknown tool: {name}")
            else:
                res = tool.fn(**args)
                obs.append(str(res))
        return ObservationEvent(observations=obs)

    @step
    async def agent_continue_step(self, ctx: Context, ev: ObservationEvent) -> Union[ToolCallEvent, StopEvent]:
        """Append observations to scratchpad and continue planning."""
        scratchpad = await ctx.get("scratchpad", default="")
        for o in ev.observations:
            scratchpad += f"Observation: {o}\n"
        await ctx.set("scratchpad", scratchpad)
        question = await ctx.get("question")
        return await self._agent_common(scratchpad, question)

# -----------------------------------------------------------------------------
# 6. Runner
# -----------------------------------------------------------------------------
async def main() -> None:
    wf = SQLAgentWorkflow(timeout=180, verbose=True)
    result = await wf.run(question="Show the first 3 rows from the Artist table.")
    print(result)

if __name__ == "__main__":
    asyncio.run(main())
@AnandUgale commented on GitHub (Jul 29, 2025): @dosu , I tried with minimal working pattern for step signatures in 0.12.x:, but still getting same error - error - ``` >python SQL_Agent_4.py Traceback (most recent call last): File "C:\Users\ananu\Documents\Jupyter_Notebook\Envs\llamaindex\SQL_Agent\SQL_Agent_4.py", line 160, in <module> class SQLAgentWorkflow(Workflow): File "C:\Users\ananu\Documents\Jupyter_Notebook\Envs\llamaindex\SQL_Agent\SQL_Agent_4.py", line 191, in SQLAgentWorkflow @step ^^^^ File "C:\Users\ananu\AppData\Local\miniconda3\envs\llamaindex_test\Lib\site-packages\llama_index\core\workflow\decorators.py", line 86, in step decorator(func) File "C:\Users\ananu\AppData\Local\miniconda3\envs\llamaindex_test\Lib\site-packages\llama_index\core\workflow\decorators.py", line 60, in decorator validate_step_signature(spec) File "C:\Users\ananu\AppData\Local\miniconda3\envs\llamaindex_test\Lib\site-packages\llama_index\core\workflow\utils.py", line 97, in validate_step_signature raise WorkflowValidationError(msg) llama_index.core.workflow.errors.WorkflowValidationError: Step signature must have at least one parameter annotated as type Event ``` code - ``` """ SQL_Agent.py A ReAct-style SQL agent built with LlamaIndex Workflow and Google Gemini (via `llama-index-llms-google-genai`). It: 1. Lists tables 2. Fetches schemas/sample rows 3. Lets the LLM plan tool calls (ReAct) 4. Generates/checks SQL 5. Executes queries and returns a final NL answer Run: pip install -U "llama-index>=0.10" llama-index-llms-google-genai sqlalchemy requests # (Optional) silence gRPC warnings by forcing REST transport (already done below) set GOOGLE_API_KEY=YOUR_KEY # Windows (cmd) export GOOGLE_API_KEY=YOUR_KEY # Linux/macOS """ from __future__ import annotations import os import ast import asyncio from typing import List, Dict, Any, Union import requests from sqlalchemy import create_engine, text, inspect from llama_index.core.workflow import ( Workflow, step, Event, StartEvent, StopEvent, Context, ) from llama_index.core.tools import FunctionTool from llama_index.core import Settings from llama_index.core.prompts import PromptTemplate from llama_index.llms.openai_like import OpenAILike # ----------------------------------------------------------------------------- # 0. Configure Gemini LLM # ----------------------------------------------------------------------------- # Force REST transport to avoid gRPC shutdown warnings on Windows Settings.llm = OpenAILike( model="gemma3:latest", api_base="http://localhost:11434/v1", api_key="ollama", context_window=8000, ) # ----------------------------------------------------------------------------- # 1. Download and connect to the Chinook sample database # ----------------------------------------------------------------------------- DB_PATH = "Chinook.db" if not os.path.exists(DB_PATH): url = "https://storage.googleapis.com/benchmarks-artifacts/chinook/Chinook.db" resp = requests.get(url, timeout=30) resp.raise_for_status() with open(DB_PATH, "wb") as f: f.write(resp.content) engine = create_engine(f"sqlite:///{DB_PATH}") # ----------------------------------------------------------------------------- # 2. SQL helper tools exposed to the agent # ----------------------------------------------------------------------------- def sql_db_list_tables() -> str: """List all tables in the database.""" return ", ".join(inspect(engine).get_table_names()) def sql_db_schema(table_names: List[str]) -> str: """Return schema + 3 sample rows for each requested table.""" insp = inspect(engine) chunks: List[str] = [] for table in table_names: cols = insp.get_columns(table) col_str = ", ".join(f"{c['name']} ({c['type']})" for c in cols) sample_sql = f"SELECT * FROM {table} LIMIT 3;" rows = engine.connect().execute(text(sample_sql)).fetchall() chunks.append(f"Table: {table}\nColumns: {col_str}\nSample rows: {rows}\n") return "\n".join(chunks).strip() def sql_db_query(query: str) -> str: """Execute arbitrary SQL (SELECT) and return rows.""" rows = engine.connect().execute(text(query)).fetchall() return str(rows) def sql_db_query_checker(query: str) -> str: """Sanity-check SQL via LLM; return corrected or original query.""" chk = PromptTemplate( """Double-check the following SQL query for mistakes. If there's an issue, return ONLY a corrected SQL statement. If it's fine, return the original query unchanged. Do NOT add any explanation. Query: {query}""" ) resp = Settings.llm.complete(chk.format(query=query)) return resp.text.strip() # Wrap as FunctionTools list_tables_tool = FunctionTool.from_defaults( fn=sql_db_list_tables, name="sql_db_list_tables", description="List all table names in the SQL database.", ) schema_tool = FunctionTool.from_defaults( fn=sql_db_schema, name="sql_db_schema", description="Input a list of table names to get their schema and 3 sample rows.", ) query_tool = FunctionTool.from_defaults( fn=sql_db_query, name="sql_db_query", description="Execute a SQL query and return the results.", ) query_checker_tool = FunctionTool.from_defaults( fn=sql_db_query_checker, name="sql_db_query_checker", description="Check a SQL query for mistakes and return a corrected version only.", ) tools = [list_tables_tool, schema_tool, query_tool, query_checker_tool] # ----------------------------------------------------------------------------- # 3. Events for the workflow # ----------------------------------------------------------------------------- class ToolCallEvent(Event): tool_calls: List[Dict[str, Any]] class ObservationEvent(Event): observations: List[str] # ----------------------------------------------------------------------------- # 4. ReAct prompt template # ----------------------------------------------------------------------------- REACT_PROMPT = PromptTemplate( """ When interacting with the user, you follow ReAct: Thought: think through what to do next Action: the action to take, formatted as tool_name(args) Observation: the result of the action Repeat Thought/Action/Observation until you can answer: <final_answer> your concise final answer here </final_answer> Tools: {tool_descs} Question: {question} Scratchpad: {scratchpad} """ ) # ----------------------------------------------------------------------------- # 5. Workflow Implementation # ----------------------------------------------------------------------------- class SQLAgentWorkflow(Workflow): """A two-step loop: agent_start_step -> tool_executor_step -> agent_continue_step ...""" async def _agent_common(self, scratchpad: str, question: str) -> Union[ToolCallEvent, StopEvent]: # Format tool list for prompt tool_descs = "\n".join(f"{t.metadata.name}: {t.metadata.description}" for t in tools) prompt = REACT_PROMPT.format(tool_descs=tool_descs, question=question, scratchpad=scratchpad) llm_resp = Settings.llm.complete(prompt) text_resp = llm_resp.text # Check for final answer if "<final_answer>" in text_resp: ans = text_resp.split("<final_answer>")[1].split("</final_answer>")[0].strip() return StopEvent(result=ans) # Parse Action for tool calls if "Action:" in text_resp: after = text_resp.split("Action:", 1)[1].strip() if "(" in after: name = after.split("(", 1)[0].strip() args_str = after.split("(", 1)[1].rsplit(")", 1)[0].strip() try: parsed = ast.literal_eval(args_str) args = parsed if isinstance(parsed, dict) else {"arg": parsed} except: args = {"raw": args_str} return ToolCallEvent(tool_calls=[{"name": name, "args": args}]) # Fallback return StopEvent(result="LLM did not return a valid Action or <final_answer>.\nResponse:\n" + text_resp) @step async def agent_start_step(self, ctx: Context, ev: StartEvent) -> Union[ToolCallEvent, StopEvent]: """Initialize scratchpad and question.""" scratchpad = await ctx.get("scratchpad", default="") question = getattr(ev, "question", None) or ev.kwargs.get("question") if question is None: raise ValueError("No 'question' provided to StartEvent.") await ctx.set("question", question) scratchpad += f"Question: {question}\n" await ctx.set("scratchpad", scratchpad) return await self._agent_common(scratchpad, question) @step async def tool_executor_step(self, ctx: Context, ev: ToolCallEvent) -> ObservationEvent: """Execute the requested tool and collect observations.""" obs: List[str] = [] for call in ev.tool_calls: name, args = call["name"], call["args"] or {} tool = next((t for t in tools if t.metadata.name == name), None) if tool is None: obs.append(f"Unknown tool: {name}") else: res = tool.fn(**args) obs.append(str(res)) return ObservationEvent(observations=obs) @step async def agent_continue_step(self, ctx: Context, ev: ObservationEvent) -> Union[ToolCallEvent, StopEvent]: """Append observations to scratchpad and continue planning.""" scratchpad = await ctx.get("scratchpad", default="") for o in ev.observations: scratchpad += f"Observation: {o}\n" await ctx.set("scratchpad", scratchpad) question = await ctx.get("question") return await self._agent_common(scratchpad, question) # ----------------------------------------------------------------------------- # 6. Runner # ----------------------------------------------------------------------------- async def main() -> None: wf = SQLAgentWorkflow(timeout=180, verbose=True) result = await wf.run(question="Show the first 3 rows from the Artist table.") print(result) if __name__ == "__main__": asyncio.run(main()) ```
yindo commented 2026-02-16 02:16:04 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jul 29, 2025):

This error almost always means there’s an import mismatch or multiple versions of llama_index installed, causing the Event class in your code to differ from the one used by the framework’s validator—even if the code “looks” right. The validator checks the actual class object, not just the name or inheritance chain, so any difference in import path or environment will break it.

Here’s how to troubleshoot:

  1. Check for multiple installs: Run pip list | findstr llama (Windows) or pip list | grep llama (Linux/macOS) and make sure you only have one version of each llama_index package. Also check for stray copies or editable installs.

  2. Print Event class info: Add this to your script before defining your workflow:

    from llama_index.core.workflow import Event
print("Event repr:", repr(Event))
print("Event id:", id(Event))
print("Event file:", Event.__module__, Event.__qualname__)

    Do the same for any Event import in your custom event classes, and compare. If the id or __module__ differs, you have an import mismatch.

  3. PYTHONPATH and shadowing: Make sure there are no local files or folders named llama_index or workflow in your project directory that could shadow the installed package.

  4. Custom StopEvent: In 0.12.14, you cannot use custom StopEvent subclasses—only the built-in StopEvent is supported. If you need custom stop events, upgrade to llama-index-core >= 0.12.21 (see details).

If you find multiple versions or mismatched imports, uninstall all llama_index packages, delete any local copies, and reinstall a single version. This should resolve the validation error. The strictness of the validator is by design and is confirmed by the test suite and source code—matching import paths and class objects is required for step signature validation to pass.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

@dosubot[bot] commented on GitHub (Jul 29, 2025): <!-- Answer --> This error almost always means there’s an import mismatch or multiple versions of llama_index installed, causing the Event class in your code to differ from the one used by the framework’s validator—even if the code “looks” right. The validator checks the actual class object, not just the name or inheritance chain, so any difference in import path or environment will break it. Here’s how to troubleshoot: 1. **Check for multiple installs:** Run `pip list | findstr llama` (Windows) or `pip list | grep llama` (Linux/macOS) and make sure you only have one version of each llama_index package. Also check for stray copies or editable installs. 2. **Print Event class info:** Add this to your script before defining your workflow: ```python from llama_index.core.workflow import Event print("Event repr:", repr(Event)) print("Event id:", id(Event)) print("Event file:", Event.__module__, Event.__qualname__) ``` Do the same for any Event import in your custom event classes, and compare. If the `id` or `__module__` differs, you have an import mismatch. 3. **PYTHONPATH and shadowing:** Make sure there are no local files or folders named `llama_index` or `workflow` in your project directory that could shadow the installed package. 4. **Custom StopEvent:** In 0.12.14, you cannot use custom StopEvent subclasses—only the built-in StopEvent is supported. If you need custom stop events, upgrade to llama-index-core >= 0.12.21 ([see details](https://github.com/run-llama/llama_index/issues/18208)). If you find multiple versions or mismatched imports, uninstall all llama_index packages, delete any local copies, and reinstall a single version. This should resolve the validation error. The strictness of the validator is by design and is confirmed by the test suite and source code—matching import paths and class objects is required for step signature validation to pass. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/f202b5f3-e213-4422-b4d9-76f7f325a1a5?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/f202b5f3-e213-4422-b4d9-76f7f325a1a5?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/f202b5f3-e213-4422-b4d9-76f7f325a1a5?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/f202b5f3-e213-4422-b4d9-76f7f325a1a5?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/f202b5f3-e213-4422-b4d9-76f7f325a1a5?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/f202b5f3-e213-4422-b4d9-76f7f325a1a5?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/f202b5f3-e213-4422-b4d9-76f7f325a1a5?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/a347f61c-1d6a-4435-903f-654845df5b7b/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/run-llama/llama_index/issues/19552)
yindo commented 2026-02-16 02:16:04 -05:00
Author
Owner
Copy Link
Copy Source

@ZeyadAttia5 commented on GitHub (Jan 1, 2026):

This bug was already fixed in this repo

Root cause: When using from __future__ import annotations, Python stores annotations as strings (postponed evaluation). The original validation code used param.annotation directly, which returned a string like "StartEvent" instead of the actual class. The issubclass() check then failed because issubclass("StartEvent", Event) is invalid.

Fix: Commit e6bdef2 (March 27, 2025) — PR #18225 "support postponed type hints in step signatures"

The fix uses get_type_hints(fn) to resolve string annotations into actual types before validation.

Why you're still seeing this

Your traceback shows you're using llama_index.core.workflow — the workflow module bundled with llama-index-core==0.12.14. This bundled version lags behind the standalone llama-index-workflows package and likely doesn't include this fix.

Solution

pip install llama-index-workflows

Then change your imports:

from workflows import Workflow, step, Context
from workflows.events import Event, StartEvent, StopEvent

I've verified that the current workflows-py package handles your exact scenario correctly (custom Event subclasses, Union return types, and from future import annotations).

This issue can likely be closed. If you need the fix in the bundled llama_index.core.workflow, that would need to be tracked in run-llama/llama_index.

@ZeyadAttia5 commented on GitHub (Jan 1, 2026): ## This bug was already fixed in this repo **Root cause:** When using `from __future__ import annotations`, Python stores annotations as strings (postponed evaluation). The original validation code used `param.annotation` directly, which returned a string like `"StartEvent"` instead of the actual class. The `issubclass()` check then failed because `issubclass("StartEvent", Event)` is invalid. **Fix:** Commit [`e6bdef2`](https://github.com/run-llama/workflows-py/commit/e6bdef225ebf546b24ee91898540b2ebac040a78) (March 27, 2025) — PR #18225 "support postponed type hints in step signatures" The fix uses `get_type_hints(fn)` to resolve string annotations into actual types before validation. --- ### Why you're still seeing this Your traceback shows you're using `llama_index.core.workflow` — the workflow module bundled with `llama-index-core==0.12.14`. This bundled version lags behind the standalone `llama-index-workflows` package and likely doesn't include this fix. ### Solution ```bash pip install llama-index-workflows ``` Then change your imports: ```python from workflows import Workflow, step, Context from workflows.events import Event, StartEvent, StopEvent ``` I've verified that the current workflows-py package handles your exact scenario correctly (custom Event subclasses, Union return types, and from __future__ import annotations). This issue can likely be closed. If you need the fix in the bundled llama_index.core.workflow, that would need to be tracked in [run-llama/llama_index](https://github.com/run-llama/llama_index).
yindo commented 2026-02-16 02:16:04 -05:00
Author
Owner
Copy Link
Copy Source

@adrianlyjak commented on GitHub (Jan 7, 2026):

@AnandUgale if you're still seeing issues on the latest version of llama-index-workflows, feel free to re-open

@adrianlyjak commented on GitHub (Jan 7, 2026): @AnandUgale if you're still seeing issues on the latest version of llama-index-workflows, feel free to re-open
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
enhancement
enhancement
good first issue
pull-request
Mirrored from GitHub Pull Request
 question
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: run-llama/workflows-py#8
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?