QueryEngineTool returns "empty response" once workflow is deployed on llama-deploy #92

New Issue

2026-02-16T01:15:40-05:00

yindo commented

2026-02-16 01:15:40 -05:00

Originally created by @tituslhy on GitHub (Sep 10, 2024).

I've tested my code on the following levels:

query_engine.query()
I then wrapped this query engine into an agent. FunctionCallingAgent.chat() worked
I then wrapped the FunctionCallingAgent into a Workflow step. Workflow.run(query="...") worked.
I then deployed the workflow on llama_deploy. This was when I started getting empty responses from my query engine for RAG related tasks.

deploy_core.py

from llama_deploy import (
    deploy_core,
    ControlPlaneConfig,
)
from llama_deploy.message_queues.simple import (
    SimpleMessageQueueConfig
)

control_plane_config = ControlPlaneConfig(
    host = "0.0.0.0",
    port = 8288,
)
message_queue_config = SimpleMessageQueueConfig(
    host = "0.0.0.0",
    port = 8277,
)

if __name__ == "__main__":
    import asyncio
    
    asyncio.run(
        deploy_core(
            control_plane_config = control_plane_config,
            message_queue_config=message_queue_config,
            disable_control_plane=False,
            disable_message_queue=False,
        )
    )

deploy_workflow.py

from llama_deploy import (
    deploy_workflow,
    WorkflowServiceConfig,
    ControlPlaneConfig,
)

import os
from dotenv import load_dotenv, find_dotenv
import sys

_ = load_dotenv(find_dotenv())
__curdir__ = os.getcwd()

if "workflow" in __curdir__:
    sys.path.append(
        os.path.join(
            __curdir__,
            "../../src"
        )
    )
else: #in the main root folder
    sys.path.append("./src")

from multi_agent_system import MultiAgentSystem

control_plane_config = ControlPlaneConfig(
    host = "0.0.0.0",
    port = 8288,
)
workflow_config = WorkflowServiceConfig(
    host = "0.0.0.0",
    port = 8299,
    service_name="Assistant"
)

async def main():
    await deploy_workflow(
        workflow = MultiAgentSystem(timeout=600.0, verbose = True),
        workflow_config = workflow_config,
        control_plane_config = control_plane_config
    )

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())

Testing code on a Jupyter Notebook:

from llama_deploy import LlamaDeployClient, ControlPlaneConfig

control_plane_config = ControlPlaneConfig(
    host = "0.0.0.0",
    port = 8288,
)
client = LlamaDeployClient(control_plane_config)
session = client.create_session()
query = "What is the definition of a black swan event?"
result = session.run(
    "Assistant", 
    query=query,
    history = [
        {
            "role": "user",
            "content": query 
        }
    ]
)

The output I got in my terminal was:

=== Calling Function ===
Calling function: Definitions_Tool with args: {"input": "black swan event"}
=== Function Output ===
Empty Response

Not too sure why this is happening. I'm using ChromaDB as my vector database to do this.

Originally created by @tituslhy on GitHub (Sep 10, 2024). I've tested my code on the following levels: 1. query_engine.query() 2. I then wrapped this query engine into an agent. FunctionCallingAgent.chat() worked 3. I then wrapped the FunctionCallingAgent into a Workflow step. Workflow.run(query="...") worked. 4. I then deployed the workflow on llama_deploy. **This was when I started getting empty responses from my query engine for RAG related tasks.** deploy_core.py ``` from llama_deploy import ( deploy_core, ControlPlaneConfig, ) from llama_deploy.message_queues.simple import ( SimpleMessageQueueConfig ) control_plane_config = ControlPlaneConfig( host = "0.0.0.0", port = 8288, ) message_queue_config = SimpleMessageQueueConfig( host = "0.0.0.0", port = 8277, ) if __name__ == "__main__": import asyncio asyncio.run( deploy_core( control_plane_config = control_plane_config, message_queue_config=message_queue_config, disable_control_plane=False, disable_message_queue=False, ) ) ``` deploy_workflow.py ``` from llama_deploy import ( deploy_workflow, WorkflowServiceConfig, ControlPlaneConfig, ) import os from dotenv import load_dotenv, find_dotenv import sys _ = load_dotenv(find_dotenv()) __curdir__ = os.getcwd() if "workflow" in __curdir__: sys.path.append( os.path.join( __curdir__, "../../src" ) ) else: #in the main root folder sys.path.append("./src") from multi_agent_system import MultiAgentSystem control_plane_config = ControlPlaneConfig( host = "0.0.0.0", port = 8288, ) workflow_config = WorkflowServiceConfig( host = "0.0.0.0", port = 8299, service_name="Assistant" ) async def main(): await deploy_workflow( workflow = MultiAgentSystem(timeout=600.0, verbose = True), workflow_config = workflow_config, control_plane_config = control_plane_config ) if __name__ == "__main__": import asyncio asyncio.run(main()) ``` Testing code on a Jupyter Notebook: ``` from llama_deploy import LlamaDeployClient, ControlPlaneConfig control_plane_config = ControlPlaneConfig( host = "0.0.0.0", port = 8288, ) client = LlamaDeployClient(control_plane_config) session = client.create_session() query = "What is the definition of a black swan event?" result = session.run( "Assistant", query=query, history = [ { "role": "user", "content": query } ] ) ``` The output I got in my terminal was: ``` === Calling Function === Calling function: Definitions_Tool with args: {"input": "black swan event"} === Function Output === Empty Response ``` Not too sure why this is happening. I'm using ChromaDB as my vector database to do this.

yindo closed this issue

2026-02-16 01:15:40 -05:00

yindo commented

2026-02-16 01:15:41 -05:00

@logan-markewich commented on GitHub (Sep 11, 2024):

@tituslhy Might need more details to replicate this? What is MultiAgentSystem ? Is there some minimum workflow that replicates this?

@logan-markewich commented on GitHub (Sep 11, 2024): @tituslhy Might need more details to replicate this? What is `MultiAgentSystem` ? Is there some minimum workflow that replicates this?

yindo commented

2026-02-16 01:15:41 -05:00

@logan-markewich commented on GitHub (Sep 11, 2024):

Empty response from a query engine really only happens when zero results are retrieved. Perhaps it's not loading your existing chroma collection properly?

@logan-markewich commented on GitHub (Sep 11, 2024): Empty response from a query engine really only happens when zero results are retrieved. Perhaps it's not loading your existing chroma collection properly?

yindo commented

2026-02-16 01:15:44 -05:00

@tituslhy commented on GitHub (Sep 12, 2024):

Oh sorry. MultiAgentSystem is just the name of the workflow. Here's a minimally reproducible example:

llm = OpenAI()
embed_model = OpenAIEmbedding()

db_path = "./path/to/db"
chroma_client = chromadb.PersistentClient(path=db_path)
chroma_collection = chroma_client.get_or_create_collection(name="DefinitionsTool")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
index = VectorStoreIndex.from_vector_store(vector_store, embed_model=embed_model)
query_engine = index.as_query_engine(
        similarity_top_k = 4,
        llm = llm
    )
tools = [
        QueryEngineTool(
             query_engine = query_engine,
             metadata = ToolMetadata(
                 name = "Definitions_Tool",
                 description = """Use this tool to answer questions related to definitions such as
                 what is "yield"."""
                 )
             )
]
agent_worker = FunctionCallingAgentWorker.from_tools(tools=tools)
agent = agent_worker.as_agent()

class MultiAgentSystem(Workflow):

        def __init__(self):
              self.rag_agent = agent

        @step
         async def process_query(self, ev: StartEvent) -> RAGEvent:
              .... #some code here to route to RAGEvent
         
         @step
          async def get_definitions(self, ev: RAGEvent) -> StopEvent
               response = await self.rag_agent.achat(ev.query)
               return StopEvent(str(response))

This all works. When I use the query engine to query, or chat with the agent or run the task via workflow - I get the correct results. It's only once I deployed the workflow to llama-deploy that my RAG tool starts returning me empty responses for the exact same question - actually for any question.

@tituslhy commented on GitHub (Sep 12, 2024): Oh sorry. MultiAgentSystem is just the name of the workflow. Here's a minimally reproducible example: ``` llm = OpenAI() embed_model = OpenAIEmbedding() db_path = "./path/to/db" chroma_client = chromadb.PersistentClient(path=db_path) chroma_collection = chroma_client.get_or_create_collection(name="DefinitionsTool") vector_store = ChromaVectorStore(chroma_collection=chroma_collection) index = VectorStoreIndex.from_vector_store(vector_store, embed_model=embed_model) query_engine = index.as_query_engine( similarity_top_k = 4, llm = llm ) tools = [ QueryEngineTool( query_engine = query_engine, metadata = ToolMetadata( name = "Definitions_Tool", description = """Use this tool to answer questions related to definitions such as what is "yield".""" ) ) ] agent_worker = FunctionCallingAgentWorker.from_tools(tools=tools) agent = agent_worker.as_agent() class MultiAgentSystem(Workflow): def __init__(self): self.rag_agent = agent @step async def process_query(self, ev: StartEvent) -> RAGEvent: .... #some code here to route to RAGEvent @step async def get_definitions(self, ev: RAGEvent) -> StopEvent response = await self.rag_agent.achat(ev.query) return StopEvent(str(response)) ``` This all works. When I use the query engine to query, or chat with the agent or run the task via workflow - I get the correct results. It's only once I deployed the workflow to llama-deploy that my RAG tool starts returning me empty responses for the exact same question - actually for any question.

yindo commented

2026-02-16 01:15:44 -05:00

@tituslhy commented on GitHub (Sep 15, 2024):

I think this might be a ChromaDB problem. I changed the vector database to Qdrant and it immediately worked ok. Closing the issue.

@tituslhy commented on GitHub (Sep 15, 2024): I think this might be a ChromaDB problem. I changed the vector database to Qdrant and it immediately worked ok. Closing the issue.

yindo referenced this issue

2026-02-16 01:16:40 -05:00

[PR #106] [MERGED] [docs] Two fixes to provide better out-of-box experience to get started with llama agents #266

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: run-llama/llama_deploy#92