I encountered an error: cannot pickle 'module' object #23

Closed
opened 2026-02-16 02:16:09 -05:00 by yindo · 14 comments
Owner

Originally created by @lf2foce on GitHub (Oct 29, 2025).

Issue Description:
The multiagent workflow breaks when using llama-index-workflows version 2.9.0, while it works perfectly with version 2.8.3.

Problem Details:

  • Error: TypeError: cannot pickle 'module' object during workflow context serialization
  • The issue occurs specifically with deepcopy() operations in workflow context handling
  • This wasn't included in the main llamaindex v0.14.6 release, but when deployed to production, it breaks the entire multiagent system

Environment:

  • llama-index-workflows: 2.9.0 (broken) vs 2.8.3 (working)
  • Python deepcopy fails on workflow context objects containing module references

Temporary Workaround:
Downgraded to llama-index-workflows==2.8.3 and pinned the version to prevent auto-updates.

Request:
Could you please investigate the serialization changes introduced in version 2.9.0 that might be causing this pickling issue with workflow contexts?

Originally created by @lf2foce on GitHub (Oct 29, 2025). **Issue Description:** The multiagent workflow breaks when using llama-index-workflows version 2.9.0, while it works perfectly with version 2.8.3. **Problem Details:** - Error: `TypeError: cannot pickle 'module' object` during workflow context serialization - The issue occurs specifically with `deepcopy()` operations in workflow context handling - This wasn't included in the main llamaindex v0.14.6 release, but when deployed to production, it breaks the entire multiagent system **Environment:** - llama-index-workflows: 2.9.0 (broken) vs 2.8.3 (working) - Python deepcopy fails on workflow context objects containing module references **Temporary Workaround:** Downgraded to llama-index-workflows==2.8.3 and pinned the version to prevent auto-updates. **Request:** Could you please investigate the serialization changes introduced in version 2.9.0 that might be causing this pickling issue with workflow contexts?
yindo closed this issue 2026-02-16 02:16:09 -05:00
Author
Owner

@logan-markewich commented on GitHub (Oct 29, 2025):

@lf2foce any code to reproduce would help

@logan-markewich commented on GitHub (Oct 29, 2025): @lf2foce any code to reproduce would help
Author
Owner

@adrianlyjak commented on GitHub (Oct 29, 2025):

@lf2foce a stack trace would also be helpful

@adrianlyjak commented on GitHub (Oct 29, 2025): @lf2foce a stack trace would also be helpful
Author
Owner

@bambooblood commented on GitHub (Oct 29, 2025):

I've detected exactly the same with v2.9.0, and downgrading to v2.8.3 solved the issue. I'll try to provide a reproduction code tomorrow if still needed.

Disclaimer: In my case, it didn't raise any error directly, the agent.run() simply stops before really start working.

@bambooblood commented on GitHub (Oct 29, 2025): I've detected exactly the same with v2.9.0, and downgrading to v2.8.3 solved the issue. I'll try to provide a reproduction code tomorrow if still needed. Disclaimer: In my case, it didn't raise any error directly, the agent.run() simply stops before really start working.
Author
Owner

@adrianlyjak commented on GitHub (Oct 29, 2025):

Thanks @bambooblood. We've also pushed a fix for hashing issue, but it doesn't sound like the same. I'm curious if you see the same issues on v2.9.1

@adrianlyjak commented on GitHub (Oct 29, 2025): Thanks @bambooblood. We've also pushed a fix for hashing issue, but it doesn't sound like the same. I'm curious if you see the same issues on v2.9.1
Author
Owner

@bambooblood commented on GitHub (Oct 29, 2025):

Sure, I'll take a look tomorrow!

@bambooblood commented on GitHub (Oct 29, 2025): Sure, I'll take a look tomorrow!
Author
Owner

@bambooblood commented on GitHub (Oct 30, 2025):

Hey @adrianlyjak ! I've done some testing switching between different versions:

Both v2.8.3 and v2.9.1 are working as expected! Using the provided code (a very simplified version of my current implementation) you should be able to see the events in the console. So, seems like the issue is only happening in v2.9.0, where you'll see that the agent is not really running and it only calls the done_callback without any other kind of interaction. I've taken a look at your PR and I'm afraid I didn't see a direct relationship between it and any of this.

# requirements.txt
llama-index == 0.14.6
llama-index-core == 0.14.6
llama-index-llms-azure-openai == 0.4.2
# llama-index-workflows == 2.8.3 # Working as expected
llama-index-workflows == 2.9.0 # Does not work
# llama-index-workflows == 2.9.1 # Working as expected
# main.py
import asyncio
import logging

from llama_index.core.tools import FunctionTool
from llama_index.llms.azure_openai import AzureOpenAI
from llama_index.core.agent.workflow import ReActAgent

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()

MODEL = "gpt-4.1"
DEPLOYMENT_NAME = "gpt-4.1-20250414-global"
API_VERSION = "2024-12-01-preview"
API_BASE_URL = ""
API_KEY = ""

test_agent = ReActAgent(
    name="test",
    tools=[
        FunctionTool.from_defaults(
            fn=lambda: "2025-10-30 08:24:17", description="Get current date and time"
        )
    ],
    llm=AzureOpenAI(
        model=MODEL,
        azure_deployment=DEPLOYMENT_NAME,
        azure_endpoint=API_BASE_URL,
        api_key=API_KEY,
        api_version=API_VERSION,
    ),
    system_prompt="You are a helpful assistant.",
)




async def run(
    prompt: str,
):
    handler = test_agent.run(
        prompt,
    )
    handler.add_done_callback(lambda fut: logger.info("DONE"))

    async for event in handler.stream_events():
        logger.info(event)
        yield f"data: {event}\n\n"

    await handler


async def main():
    async_gen = run("Hello, world! What is the current date and time?")
    async for _ in async_gen:
        pass


asyncio.run(main())
@bambooblood commented on GitHub (Oct 30, 2025): Hey @adrianlyjak ! I've done some testing switching between different versions: Both v2.8.3 and v2.9.1 are working as expected! Using the provided code (a very simplified version of my current implementation) you should be able to see the events in the console. So, seems like the issue is only happening in v2.9.0, where you'll see that the agent is not really running and it only calls the done_callback without any other kind of interaction. I've taken a look at your PR and I'm afraid I didn't see a direct relationship between it and any of this. ``` # requirements.txt llama-index == 0.14.6 llama-index-core == 0.14.6 llama-index-llms-azure-openai == 0.4.2 # llama-index-workflows == 2.8.3 # Working as expected llama-index-workflows == 2.9.0 # Does not work # llama-index-workflows == 2.9.1 # Working as expected ``` ```python # main.py import asyncio import logging from llama_index.core.tools import FunctionTool from llama_index.llms.azure_openai import AzureOpenAI from llama_index.core.agent.workflow import ReActAgent logging.basicConfig(level=logging.INFO) logger = logging.getLogger() MODEL = "gpt-4.1" DEPLOYMENT_NAME = "gpt-4.1-20250414-global" API_VERSION = "2024-12-01-preview" API_BASE_URL = "" API_KEY = "" test_agent = ReActAgent( name="test", tools=[ FunctionTool.from_defaults( fn=lambda: "2025-10-30 08:24:17", description="Get current date and time" ) ], llm=AzureOpenAI( model=MODEL, azure_deployment=DEPLOYMENT_NAME, azure_endpoint=API_BASE_URL, api_key=API_KEY, api_version=API_VERSION, ), system_prompt="You are a helpful assistant.", ) async def run( prompt: str, ): handler = test_agent.run( prompt, ) handler.add_done_callback(lambda fut: logger.info("DONE")) async for event in handler.stream_events(): logger.info(event) yield f"data: {event}\n\n" await handler async def main(): async_gen = run("Hello, world! What is the current date and time?") async for _ in async_gen: pass asyncio.run(main()) ```
Author
Owner

@adrianlyjak commented on GitHub (Oct 30, 2025):

Thanks @bambooblood for the script. Yeah, if you remove the stream_events, you'll get the error logged from await handler that 2.9.1 fixes: TypeError: unhashable type: 'ReActAgent'. Unfortunately the event stream is stalled out in this failure scenario, so you never reach that if you stream events.

Were you getting the specific cannot pickle 'module' object error ever. If so under what circumstances?

@lf2foce have you had a chance to retest?

@adrianlyjak commented on GitHub (Oct 30, 2025): Thanks @bambooblood for the script. Yeah, if you remove the `stream_events`, you'll get the error logged from `await handler` that 2.9.1 fixes: `TypeError: unhashable type: 'ReActAgent'`. Unfortunately the event stream is stalled out in this failure scenario, so you never reach that if you stream events. Were you getting the specific `cannot pickle 'module' object` error ever. If so under what circumstances? @lf2foce have you had a chance to retest?
Author
Owner

@bambooblood commented on GitHub (Oct 31, 2025):

Following your instructions, yes, I'm seeing that TypeError: unhashable type: 'ReActAgent' using v2.9.0. Regarding cannot pickle 'module' object, no, I've never seen that.

So, at least from me side, all working as expected with the fixed 2.9.1 version. Thanks a lot!

@bambooblood commented on GitHub (Oct 31, 2025): Following your instructions, yes, I'm seeing that `TypeError: unhashable type: 'ReActAgent'` using v2.9.0. Regarding `cannot pickle 'module' object`, no, I've never seen that. So, at least from me side, all working as expected with the fixed 2.9.1 version. Thanks a lot!
Author
Owner

@na-proyectran commented on GitHub (Oct 31, 2025):

I'm having same cannot pickle 'module' object error using llama-index-workflows v. 2.10.0

It's still happening to me when I include memory object into the agents workflow.

example:

workflow = AgentWorkflow(
    agents=[director_agent, destinations_agent, flights_agent, hotels_agent, events_agent],
    root_agent=director_agent.name,

result = await workflow.run(user_msg=req.query, memory=condensed_memory)

update, seems to work only with v. 2.8.3

@na-proyectran commented on GitHub (Oct 31, 2025): I'm having same `cannot pickle 'module' object` error using llama-index-workflows v. 2.10.0 It's still happening to me when I include memory object into the agents workflow. example: ``` workflow = AgentWorkflow( agents=[director_agent, destinations_agent, flights_agent, hotels_agent, events_agent], root_agent=director_agent.name, result = await workflow.run(user_msg=req.query, memory=condensed_memory) ``` update, seems to work only with v. 2.8.3
Author
Owner

@lf2foce commented on GitHub (Oct 31, 2025):

Thanks @bambooblood for the script. Yeah, if you remove the stream_events, you'll get the error logged from await handler that 2.9.1 fixes: TypeError: unhashable type: 'ReActAgent'. Unfortunately the event stream is stalled out in this failure scenario, so you never reach that if you stream events.

Were you getting the specific cannot pickle 'module' object error ever. If so under what circumstances?

@lf2foce have you had a chance to retest?

2.10.0 still got that problem

@lf2foce commented on GitHub (Oct 31, 2025): > Thanks [@bambooblood](https://github.com/bambooblood) for the script. Yeah, if you remove the `stream_events`, you'll get the error logged from `await handler` that 2.9.1 fixes: `TypeError: unhashable type: 'ReActAgent'`. Unfortunately the event stream is stalled out in this failure scenario, so you never reach that if you stream events. > > Were you getting the specific `cannot pickle 'module' object` error ever. If so under what circumstances? > > [@lf2foce](https://github.com/lf2foce) have you had a chance to retest? 2.10.0 still got that problem
Author
Owner

@adrianlyjak commented on GitHub (Nov 1, 2025):

This seems somewhat nuanced. I can easily recreate a crash like this by adding a module object to A) the context store or B) the result value of a workflow event (e.g. return value of a function).

My speculation is that there's something new that is being serialized that wasn't previously. The only thing that I'm noticing is that the wait for event state has more granular serialization, so I would speculate that's related.

a precise code example recreating the bug, or a stack trace would be incredibly helpful to understand what particular part is different here. Is it an error pickling the dict returned from to_dict? Or an error being thrown from the to_dict call, and at what stage?

Basic examples of things that throw in either case

A) setting a module reference "somehow" on the store

    async for message in handler.stream_events():
        import time

        # Only set once to avoid spamming
        if await handler.ctx.store.get("module_obj", default=None) is None:
            await handler.ctx.store.set("module_obj", time)

        # This will raise: TypeError: cannot pickle 'module' object
        handler.ctx.to_dict(serializer=PickleSerializer())

B) or, from a return value of a function or event


director_agent = FunctionAgent(
    name="director",
    description="The director of the trip",
    tools=[],
)

def get_events():
    import time
    return {"time": time}


events_agent = FunctionAgent(
    name="events",
    description="The events of the trip",
    tools=[FunctionTool.from_defaults(get_events)],
)


workflow = AgentWorkflow(
    agents=[
        director_agent,
        events_agent,
    ],
    root_agent=director_agent.name,
)
@adrianlyjak commented on GitHub (Nov 1, 2025): This seems somewhat nuanced. I can easily recreate a crash like this by adding a module object to A) the context store or B) the result value of a workflow event (e.g. return value of a function). My speculation is that there's something new that is being serialized that wasn't previously. The only thing that I'm noticing is that the wait for event state has more granular serialization, so I would speculate that's related. a precise code example recreating the bug, or a stack trace would be incredibly helpful to understand what particular part is different here. Is it an error pickling the dict returned from `to_dict`? Or an error being thrown from the `to_dict` call, and at what stage? Basic examples of things that throw in either case A) setting a module reference "somehow" on the store ```py async for message in handler.stream_events(): import time # Only set once to avoid spamming if await handler.ctx.store.get("module_obj", default=None) is None: await handler.ctx.store.set("module_obj", time) # This will raise: TypeError: cannot pickle 'module' object handler.ctx.to_dict(serializer=PickleSerializer()) ``` B) or, from a return value of a function or event ```py director_agent = FunctionAgent( name="director", description="The director of the trip", tools=[], ) def get_events(): import time return {"time": time} events_agent = FunctionAgent( name="events", description="The events of the trip", tools=[FunctionTool.from_defaults(get_events)], ) workflow = AgentWorkflow( agents=[ director_agent, events_agent, ], root_agent=director_agent.name, ) ```
Author
Owner

@adrianlyjak commented on GitHub (Nov 1, 2025):

Ok, so this script, which sets a non-pickleable requirement fails on >=2.9.0, whereas it did not previously on 2.8.3.

@lf2foce @na-proyectran @bambooblood are you all using requirements in a call to wait_for_event?

import asyncio
from typing import Any
import pickle

from llama_index.core.agent.workflow import (
    AgentWorkflow,
    FunctionAgent,
    ToolCallResult,
)
from llama_index.core.memory import ChatMemoryBuffer
from llama_index.core.tools import FunctionTool
from workflows.context import PickleSerializer, Context
from workflows.events import HumanResponseEvent, InputRequiredEvent


class ResponseEvent(HumanResponseEvent):
    response: str
    extra: Any


class RequestEvent(InputRequiredEvent):
    request: str


async def get_events(ctx: Context):
    import time

    result = await ctx.wait_for_event(
        ResponseEvent,
        RequestEvent(request="what is your name?", extra="foo"),
        requirements={"extra": time},
    )
    return f"Hi {result.response}! The events are 1. driving, 2. flying, 3. sleeping. 4. eating."


events_agent = FunctionAgent(
    name="events",
    description="The events of the trip",
    tools=[FunctionTool.from_defaults(get_events)],
)


workflow = AgentWorkflow(
    agents=[
        events_agent,
    ],
    root_agent=events_agent.name,
)

condensed_memory = ChatMemoryBuffer.from_defaults(token_limit=60000)


class AlwaysEqual:
    def __eq__(self, other):
        return True


everything = AlwaysEqual()


async def main():
    handler = workflow.run(user_msg="I want to go to the moon", memory=condensed_memory)
    i = 0
    async for message in handler.stream_events():
        i += 1
        print(f"ev: {i} {type(message).__name__}")
        if isinstance(message, ToolCallResult):
            print(f"Tool call result: {message.tool_output}")
        if isinstance(message, InputRequiredEvent):
            ser = handler.ctx.to_dict(serializer=PickleSerializer())
            pickle.dumps(ser)
            print("post pickle dump")
            handler.ctx.send_event(ResponseEvent(response="Adrian", extra=everything))
    result = await handler
    print("FINAL RESULT: ", result)
    return result


if __name__ == "__main__":
    print("Starting...")
    import logging
    from importlib.metadata import version

    print("workflows version: ", version("llama-index-workflows"))

    logging.basicConfig(level=logging.INFO)
    asyncio.run(main())

stack trace

$ python test_issue_173.py                   
Starting...
workflows version:  2.10.0
ev: 1 AgentInput
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
ev: 2 AgentStream
ev: 3 AgentStream
ev: 4 AgentStream
ev: 5 AgentStream
ev: 6 AgentStream
ev: 7 AgentOutput
ev: 8 ToolCall
ev: 9 ToolCall
ev: 10 RequestEvent
Traceback (most recent call last):
  File "/Users/adrianlyjak/dev/llama_index/test_issue_173.py", line 87, in <module>
    asyncio.run(main())
    ~~~~~~~~~~~^^^^^^^^
  File "/Users/adrianlyjak/.local/share/uv/python/cpython-3.13.3-macos-aarch64-none/lib/python3.13/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/Users/adrianlyjak/.local/share/uv/python/cpython-3.13.3-macos-aarch64-none/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/adrianlyjak/.local/share/uv/python/cpython-3.13.3-macos-aarch64-none/lib/python3.13/asyncio/base_events.py", line 719, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/Users/adrianlyjak/dev/llama_index/test_issue_173.py", line 71, in main
    pickle.dumps(ser)
    ~~~~~~~~~~~~^^^^^
TypeError: cannot pickle 'module' object
@adrianlyjak commented on GitHub (Nov 1, 2025): Ok, so this script, which sets a non-pickleable requirement fails on >=2.9.0, whereas it did not previously on 2.8.3. @lf2foce @na-proyectran @bambooblood are you all using requirements in a call to `wait_for_event`? ```py import asyncio from typing import Any import pickle from llama_index.core.agent.workflow import ( AgentWorkflow, FunctionAgent, ToolCallResult, ) from llama_index.core.memory import ChatMemoryBuffer from llama_index.core.tools import FunctionTool from workflows.context import PickleSerializer, Context from workflows.events import HumanResponseEvent, InputRequiredEvent class ResponseEvent(HumanResponseEvent): response: str extra: Any class RequestEvent(InputRequiredEvent): request: str async def get_events(ctx: Context): import time result = await ctx.wait_for_event( ResponseEvent, RequestEvent(request="what is your name?", extra="foo"), requirements={"extra": time}, ) return f"Hi {result.response}! The events are 1. driving, 2. flying, 3. sleeping. 4. eating." events_agent = FunctionAgent( name="events", description="The events of the trip", tools=[FunctionTool.from_defaults(get_events)], ) workflow = AgentWorkflow( agents=[ events_agent, ], root_agent=events_agent.name, ) condensed_memory = ChatMemoryBuffer.from_defaults(token_limit=60000) class AlwaysEqual: def __eq__(self, other): return True everything = AlwaysEqual() async def main(): handler = workflow.run(user_msg="I want to go to the moon", memory=condensed_memory) i = 0 async for message in handler.stream_events(): i += 1 print(f"ev: {i} {type(message).__name__}") if isinstance(message, ToolCallResult): print(f"Tool call result: {message.tool_output}") if isinstance(message, InputRequiredEvent): ser = handler.ctx.to_dict(serializer=PickleSerializer()) pickle.dumps(ser) print("post pickle dump") handler.ctx.send_event(ResponseEvent(response="Adrian", extra=everything)) result = await handler print("FINAL RESULT: ", result) return result if __name__ == "__main__": print("Starting...") import logging from importlib.metadata import version print("workflows version: ", version("llama-index-workflows")) logging.basicConfig(level=logging.INFO) asyncio.run(main()) ``` stack trace ``` $ python test_issue_173.py Starting... workflows version: 2.10.0 ev: 1 AgentInput INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" ev: 2 AgentStream ev: 3 AgentStream ev: 4 AgentStream ev: 5 AgentStream ev: 6 AgentStream ev: 7 AgentOutput ev: 8 ToolCall ev: 9 ToolCall ev: 10 RequestEvent Traceback (most recent call last): File "/Users/adrianlyjak/dev/llama_index/test_issue_173.py", line 87, in <module> asyncio.run(main()) ~~~~~~~~~~~^^^^^^^^ File "/Users/adrianlyjak/.local/share/uv/python/cpython-3.13.3-macos-aarch64-none/lib/python3.13/asyncio/runners.py", line 195, in run return runner.run(main) ~~~~~~~~~~^^^^^^ File "/Users/adrianlyjak/.local/share/uv/python/cpython-3.13.3-macos-aarch64-none/lib/python3.13/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/Users/adrianlyjak/.local/share/uv/python/cpython-3.13.3-macos-aarch64-none/lib/python3.13/asyncio/base_events.py", line 719, in run_until_complete return future.result() ~~~~~~~~~~~~~^^ File "/Users/adrianlyjak/dev/llama_index/test_issue_173.py", line 71, in main pickle.dumps(ser) ~~~~~~~~~~~~^^^^^ TypeError: cannot pickle 'module' object ```
Author
Owner

@adrianlyjak commented on GitHub (Nov 3, 2025):

ah, there's another failure scenario. The deep copy in here was triggering pickling, which will crash if there are events that are pydantic serializable but not picklable. #187 will also fix this. Will release shortly

@adrianlyjak commented on GitHub (Nov 3, 2025): ah, there's another failure scenario. The deep copy in here was triggering pickling, which will crash if there are events that are pydantic serializable but not picklable. #187 will also fix this. Will release shortly
Author
Owner

@adrianlyjak commented on GitHub (Nov 3, 2025):

All cases should now be fixed in 2.10.3. If some flavor of this error is still occurring on that version, please open a new issue with a stack trace

@adrianlyjak commented on GitHub (Nov 3, 2025): All cases should now be fixed in 2.10.3. If some flavor of this error is still occurring on that version, please open a new issue with a stack trace
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: run-llama/workflows-py#23