run-llama/workflows-py
1
0
Fork 0
mirror of https://github.com/run-llama/workflows-py.git synced 2026-07-21 12:15:24 -04:00
Code Issues 18 Packages Projects Releases 10 Wiki Activity

Workflow Server eventing breaks with llamaparse in workflow #21

New Issue
Closed
opened 2026-02-16 02:16:08 -05:00 by yindo · 2 comments
yindo commented 2026-02-16 02:16:08 -05:00
Owner
Copy Link
Copy Source

Originally created by @hidhana on GitHub (Oct 28, 2025).

a) I am running LlamaIndex Workflow Server (FastAPI + Uvicorn).
b) The client connects using workflow server endpoint to stream events using /events/{handler_id} endpoint.
c) The server streams workflow output to the client properly from step 1 and step 2.
d) In 3rd step, workflow calls LlamaParse, which performs an async HTTP request.

When that happens, the eventstream breaks and I am not getting anymore events in client. Where as workflow continues to run completing parsing, classification and extract successfully.

Below are chatgpt suggestions.

SSE requires the HTTP connection to remain open and flush data continuously.
But when you perform an awaited blocking async I/O call (like calling LlamaParse’s remote API), a few things can go wrong:

a) Event Loop Starvation / Context Switch
The async HTTP call yields control back to the loop, but depending on how your SSE generator is structured, the framework (FastAPI/Uvicorn) may:

think your endpoint coroutine has completed, or

temporarily stop writing to the response stream, causing the client to time out.

b) Response Finalization During Await
If the async LlamaParse call happens inside the same coroutine that is producing SSE messages, Uvicorn/Starlette might flush headers or close the response iterator prematurely.

c) Thread/Worker Isolation (if using concurrency)
Some LlamaIndex tasks use asyncio.to_thread() internally. If that thread performs blocking work or logs to stdout, the SSE buffer may never get flushed properly, causing the browser to close the connection.

d) Content-Length Mismatch
SSE must be chunked (Transfer-Encoding: chunked), but if any middleware or downstream call (like LlamaParse’s internal client) tries to buffer or serialize the response, it can interfere with the streaming socket.

Originally created by @hidhana on GitHub (Oct 28, 2025). a) I am running LlamaIndex Workflow Server (FastAPI + Uvicorn). b) The client connects using workflow server endpoint to stream events using /events/{handler_id} endpoint. c) The server streams workflow output to the client properly from step 1 and step 2. d) In 3rd step, workflow calls LlamaParse, which performs an async HTTP request. When that happens, the eventstream breaks and I am not getting anymore events in client. Where as workflow continues to run completing parsing, classification and extract successfully. Below are chatgpt suggestions. SSE requires the HTTP connection to remain open and flush data continuously. But when you perform an awaited blocking async I/O call (like calling LlamaParse’s remote API), a few things can go wrong: a) Event Loop Starvation / Context Switch The async HTTP call yields control back to the loop, but depending on how your SSE generator is structured, the framework (FastAPI/Uvicorn) may: think your endpoint coroutine has completed, or temporarily stop writing to the response stream, causing the client to time out. b) Response Finalization During Await If the async LlamaParse call happens inside the same coroutine that is producing SSE messages, Uvicorn/Starlette might flush headers or close the response iterator prematurely. c) Thread/Worker Isolation (if using concurrency) Some LlamaIndex tasks use asyncio.to_thread() internally. If that thread performs blocking work or logs to stdout, the SSE buffer may never get flushed properly, causing the browser to close the connection. d) Content-Length Mismatch SSE must be chunked (Transfer-Encoding: chunked), but if any middleware or downstream call (like LlamaParse’s internal client) tries to buffer or serialize the response, it can interfere with the streaming socket.
yindo added the buggood first issue labels 2026-02-16 02:16:08 -05:00
yindo closed this issue 2026-02-16 02:16:08 -05:00
yindo commented 2026-02-16 02:16:08 -05:00
Author
Owner
Copy Link
Copy Source

@adrianlyjak commented on GitHub (Nov 3, 2025):

Adding some summaries to the thread from a debugging session.

There was a state store item that was non-serializable, this error was not logged and not apparent. I believe this can easily be recreated by writing anything to the store that is non pydantic serializable (in this case a dataclass). This failure scenario should have error logging around the call to the checkpoint API. I'll close this issue out when the error logging is corrected

@adrianlyjak commented on GitHub (Nov 3, 2025): Adding some summaries to the thread from a debugging session. There was a state store item that was non-serializable, this error was not logged and not apparent. I believe this can easily be recreated by writing anything to the store that is non pydantic serializable (in this case a dataclass). This failure scenario should have error logging around the call to the checkpoint API. I'll close this issue out when the error logging is corrected
yindo commented 2026-02-16 02:16:09 -05:00
Author
Owner
Copy Link
Copy Source

@adrianlyjak commented on GitHub (Nov 17, 2025):

Error logging improvements were released in v2.11.0 (https://github.com/run-llama/workflows-py/pull/194)

@adrianlyjak commented on GitHub (Nov 17, 2025): Error logging improvements were released in v2.11.0 (https://github.com/run-llama/workflows-py/pull/194)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
enhancement
enhancement
good first issue
pull-request
Mirrored from GitHub Pull Request
 question
No Label bug good first issue
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: run-llama/workflows-py#21
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?