langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-08-26 18:36:52 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

WorkflowExecution.outputs containing non-JSON-serializable objects causes Logstore persistence failure (ArrayFileSegment), leading to incomplete SLS and PG logs #21401

New Issue
Closed
opened 2026-02-21 20:12:24 -05:00 by yindo · 1 comment
yindo commented 2026-02-21 20:12:24 -05:00
Owner
Copy Link
Copy Source

Originally created by @scdeng on GitHub (Jan 1, 2026).

Originally assigned to: @fatelei on GitHub.

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.11.2

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

Environment:

  • Dify version: v1.11.2
  • Environment variables set:
CORE_WORKFLOW_EXECUTION_REPOSITORY=extensions.logstore.repositories.logstore_workflow_execution_repository.LogstoreWorkflowExecutionRepository
CORE_WORKFLOW_NODE_EXECUTION_REPOSITORY=extensions.logstore.repositories.logstore_workflow_node_execution_repository.LogstoreWorkflowNodeExecutionRepository
LOGSTORE_DUAL_WRITE_ENABLED=true

Reproduction:

  1. Run any workflow that produces outputs containing custom, non-JSON-serializable objects (e.g., ArrayFileSegment instances).
  2. During persistence, the Logstore repository attempts to serialize outputs via json.dumps() without a default serialization hook.
  3. Python raises a TypeError, causing the persistence step to fail.

Observed error log:

2025-12-31 11:11:36,825 ERROR [event_manager.py:186] b46082bd2e Error in layer on_event, layer_type=<class 'core.workflow.graph_engine.layers.persistence.WorkflowPersistenceLayer'>
Traceback (most recent call last):
  File "/home/admin/api/core/workflow/graph_engine/event_management/event_manager.py", line 184, in _notify_layers
    layer.on_event(event)
  File "/home/admin/dify/api/core/workflow/graph_engine/layers/persistence.py", line 113, in on_event
    self._handle_graph_run_succeeded(event)
  File "/home/admin/dify/api/core/workflow/graph_engine/layers/persistence.py", line 182, in _handle_graph_run_succeeded
    self._workflow_execution_repository.save(execution)
  File "/home/admin/dify/api/extensions/logstore/repositories/logstore_workflow_execution_repository.py", line 150, in save
    logstore_model = self._to_logstore_model(execution)
  File "/home/admin/dify/api/extensions/logstore/repositories/logstore_workflow_execution_repository.py", line 114, in _to_logstore_model
    ("outputs", json.dumps(domain_model.outputs, ensure_ascii=False) if domain_model.outputs else "{}"),
  File "/usr/local/lib/python3.12/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/local/lib/python3.12/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/lib/python3.12/json/encoder.py", line 258, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python3.12/json/encoder.py", line 180, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type ArrayFileSegment is not JSON serializable

Due to this error:

  1. In Aliyun SLS Logstore (workflow_runs), only a "running" status log is stored. The "succeeded" status log entry is missing.
    • Normally, there should be two entries (append-only write): "running" followed by "succeeded".
  2. In PostgreSQL, workflow status remains "running" because the final update to "succeeded" is never executed.
    • Normally, PostgreSQL should have one entry, initially "running", updated to "succeeded" when execution finishes.

✔️ Expected Behavior

  • SLS Logstore:
    Two log entries per workflow run:

    1. "running" status (at workflow start)
    2. "succeeded" status (at workflow completion)
      Since SLS is append-only, both entries are retained.

  • PostgreSQL:
    One workflow execution row, initially "running", updated to "succeeded" upon completion (update-write).

❌ Actual Behavior

  1. SLS Logstore:
    Only a single "running" status entry is stored. The "succeeded" entry is missing due to the persistence error.
  2. PostgreSQL:
    Only one row exists, permanently stuck in "running" status since the final status update fails when Logstore persistence raises an exception.

Root Cause Analysis

  • WorkflowExecution.outputs may contain custom Python objects (e.g., ArrayFileSegment), which cannot be serialized by Python’s default json.dumps() encoder.
  • The current implementation in LogstoreWorkflowExecutionRepository._to_logstore_model() does not define a default handler for non-serializable objects:
("outputs", json.dumps(domain_model.outputs, ensure_ascii=False) if domain_model.outputs else "{}"),
  • This causes a TypeError when attempting to serialize the outputs.

Proposed Solution

Introduce a reusable utility function to_serializable() in libs/helper.py:

def to_serializable(obj):
    """
    Convert non-JSON-serializable objects into JSON-compatible formats.

    - Uses `to_dict()` if available.
    - Uses `__dict__` if available.
    - Falls back to string representation.
    """
    if hasattr(obj, "to_dict"):
        return obj.to_dict()
    elif hasattr(obj, "__dict__"):
        return obj.__dict__
    else:
        return str(obj)

Update LogstoreWorkflowExecutionRepository to use it:

from libs.helper import to_serializable

("outputs", json.dumps(domain_model.outputs, ensure_ascii=False, default=to_serializable) if domain_model.outputs else "{}"),

Benefits:

  • Prevents TypeError for custom objects in outputs, graph, inputs.
  • Ensures both Logstore and PostgreSQL receive complete execution records.
  • Centralizes object-to-JSON conversion logic for reuse.

📌 Recommendation: Apply this change to all repository fields that are persisted via json.dumps() but may hold custom Python objects.

Originally created by @scdeng on GitHub (Jan 1, 2026). Originally assigned to: @fatelei on GitHub. ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.11.2 ### Cloud or Self Hosted Self Hosted (Source) ### Steps to reproduce **Environment:** - Dify version: `v1.11.2` - Environment variables set: ```bash CORE_WORKFLOW_EXECUTION_REPOSITORY=extensions.logstore.repositories.logstore_workflow_execution_repository.LogstoreWorkflowExecutionRepository CORE_WORKFLOW_NODE_EXECUTION_REPOSITORY=extensions.logstore.repositories.logstore_workflow_node_execution_repository.LogstoreWorkflowNodeExecutionRepository LOGSTORE_DUAL_WRITE_ENABLED=true ``` **Reproduction:** 1. Run any workflow that produces outputs containing custom, non-JSON-serializable objects (e.g., `ArrayFileSegment` instances). 2. During persistence, the Logstore repository attempts to serialize `outputs` via `json.dumps()` without a `default` serialization hook. 3. Python raises a `TypeError`, causing the persistence step to fail. **Observed error log:** ``` 2025-12-31 11:11:36,825 ERROR [event_manager.py:186] b46082bd2e Error in layer on_event, layer_type=<class 'core.workflow.graph_engine.layers.persistence.WorkflowPersistenceLayer'> Traceback (most recent call last): File "/home/admin/api/core/workflow/graph_engine/event_management/event_manager.py", line 184, in _notify_layers layer.on_event(event) File "/home/admin/dify/api/core/workflow/graph_engine/layers/persistence.py", line 113, in on_event self._handle_graph_run_succeeded(event) File "/home/admin/dify/api/core/workflow/graph_engine/layers/persistence.py", line 182, in _handle_graph_run_succeeded self._workflow_execution_repository.save(execution) File "/home/admin/dify/api/extensions/logstore/repositories/logstore_workflow_execution_repository.py", line 150, in save logstore_model = self._to_logstore_model(execution) File "/home/admin/dify/api/extensions/logstore/repositories/logstore_workflow_execution_repository.py", line 114, in _to_logstore_model ("outputs", json.dumps(domain_model.outputs, ensure_ascii=False) if domain_model.outputs else "{}"), File "/usr/local/lib/python3.12/json/__init__.py", line 238, in dumps **kw).encode(obj) File "/usr/local/lib/python3.12/json/encoder.py", line 200, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/local/lib/python3.12/json/encoder.py", line 258, in iterencode return _iterencode(o, 0) File "/usr/local/lib/python3.12/json/encoder.py", line 180, in default raise TypeError(f'Object of type {o.__class__.__name__} ' TypeError: Object of type ArrayFileSegment is not JSON serializable ``` **Due to this error:** 1. In **Aliyun SLS Logstore** (`workflow_runs`), only a `"running"` status log is stored. The `"succeeded"` status log entry is missing. - Normally, there should be **two entries** (append-only write): `"running"` followed by `"succeeded"`. 2. In **PostgreSQL**, workflow status remains `"running"` because the final update to `"succeeded"` is never executed. - Normally, PostgreSQL should have **one entry**, initially `"running"`, updated to `"succeeded"` when execution finishes. --- ### ✔️ Expected Behavior - **SLS Logstore:** Two log entries per workflow run: 1. `"running"` status (at workflow start) 2. `"succeeded"` status (at workflow completion) Since SLS is append-only, both entries are retained. - **PostgreSQL:** One workflow execution row, initially `"running"`, updated to `"succeeded"` upon completion (update-write). --- ### ❌ Actual Behavior 1. **SLS Logstore:** Only a single `"running"` status entry is stored. The `"succeeded"` entry is missing due to the persistence error. 2. **PostgreSQL:** Only one row exists, permanently stuck in `"running"` status since the final status update fails when Logstore persistence raises an exception. --- ### **Root Cause Analysis** - `WorkflowExecution.outputs` may contain custom Python objects (e.g., `ArrayFileSegment`), which cannot be serialized by Python’s default `json.dumps()` encoder. - The current implementation in `LogstoreWorkflowExecutionRepository._to_logstore_model()` does not define a `default` handler for non-serializable objects: ```python ("outputs", json.dumps(domain_model.outputs, ensure_ascii=False) if domain_model.outputs else "{}"), ``` - This causes a `TypeError` when attempting to serialize the outputs. --- ### **Proposed Solution** Introduce a reusable utility function `to_serializable()` in `libs/helper.py`: ```python def to_serializable(obj): """ Convert non-JSON-serializable objects into JSON-compatible formats. - Uses `to_dict()` if available. - Uses `__dict__` if available. - Falls back to string representation. """ if hasattr(obj, "to_dict"): return obj.to_dict() elif hasattr(obj, "__dict__"): return obj.__dict__ else: return str(obj) ``` Update `LogstoreWorkflowExecutionRepository` to use it: ```python from libs.helper import to_serializable ("outputs", json.dumps(domain_model.outputs, ensure_ascii=False, default=to_serializable) if domain_model.outputs else "{}"), ``` --- **Benefits:** - Prevents `TypeError` for custom objects in `outputs`, `graph`, `inputs`. - Ensures both Logstore and PostgreSQL receive complete execution records. - Centralizes object-to-JSON conversion logic for reuse. --- 📌 **Recommendation:** Apply this change to all repository fields that are persisted via `json.dumps()` but may hold custom Python objects.
yindo added the 🐞 bug label 2026-02-21 20:12:24 -05:00
yindo closed this issue 2026-02-21 20:12:24 -05:00
yindo commented 2026-02-21 20:12:25 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 1, 2026):

The root cause is that WorkflowExecution.outputs (and also inputs and graph) can contain custom objects that aren't JSON serializable, and the current code uses json.dumps() without a default handler, which leads to the TypeError and incomplete log persistence.

Instead of implementing a new to_serializable helper, Dify already provides a robust utility called jsonable_encoder in api/core/model_runtime/utils/encoders.py. This function recursively converts Pydantic models, dataclasses, enums, paths, decimals, and most custom objects (via dict() or vars()) into JSON-serializable formats, making it more comprehensive and maintainable than a simple fallback helper. You can use it like this:

from core.model_runtime.utils.encoders import jsonable_encoder

("outputs", json.dumps(jsonable_encoder(domain_model.outputs), ensure_ascii=False) if domain_model.outputs else "{}"),

Apply the same pattern to inputs and graph fields as well. This will prevent serialization errors for custom objects and ensure complete log and status updates in both Logstore and PostgreSQL. If you encounter edge cases with specific custom objects, you can extend jsonable_encoder with custom encoders as needed.
See implementation details

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Chat with Dosu Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Jan 1, 2026): <!-- Answer --> The root cause is that `WorkflowExecution.outputs` (and also `inputs` and `graph`) can contain custom objects that aren't JSON serializable, and the current code uses `json.dumps()` without a default handler, which leads to the TypeError and incomplete log persistence. Instead of implementing a new `to_serializable` helper, Dify already provides a robust utility called `jsonable_encoder` in `api/core/model_runtime/utils/encoders.py`. This function recursively converts Pydantic models, dataclasses, enums, paths, decimals, and most custom objects (via `dict()` or `vars()`) into JSON-serializable formats, making it more comprehensive and maintainable than a simple fallback helper. You can use it like this: ```python from core.model_runtime.utils.encoders import jsonable_encoder ("outputs", json.dumps(jsonable_encoder(domain_model.outputs), ensure_ascii=False) if domain_model.outputs else "{}"), ``` Apply the same pattern to `inputs` and `graph` fields as well. This will prevent serialization errors for custom objects and ensure complete log and status updates in both Logstore and PostgreSQL. If you encounter edge cases with specific custom objects, you can extend `jsonable_encoder` with custom encoders as needed. [See implementation details](https://github.com/langgenius/dify/blob/9b6b2f31950c50ec54a06985dadaf6577e295c83/api/core/model_runtime/utils/encoders.py#L53-L216) <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/78c75301-3d77-40cc-84ee-e66e24494485?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/78c75301-3d77-40cc-84ee-e66e24494485?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/78c75301-3d77-40cc-84ee-e66e24494485?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/78c75301-3d77-40cc-84ee-e66e24494485?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/78c75301-3d77-40cc-84ee-e66e24494485?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/78c75301-3d77-40cc-84ee-e66e24494485?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/78c75301-3d77-40cc-84ee-e66e24494485?feedback_type=other)</sup> [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=The%20root%20cause%20is%20that%20%60WorkflowExecution.outputs%60%20%28and%20also%20%60inputs%60%20and%20%60graph%60%29%20can%20contain%20custom%20objects%20that%20aren%27t%20JSON%20serializable%2C%20and%20the%20current%20code%20uses%20%60json.dumps%28%29%60%20without%20a%20default%20handler%2C%20which%20leads%20to%20the%20TypeError%20and%20incomplete%20log%20persistence.%0A%0AInstead%20of%20implementing%20a%20new%20%60to_serializable%60%20helper%2C%20Dify%20already%20provides%20a%20robust%20utility%20called%20%60jsonable_encoder%60%20in%20%60api/core/model_runtime/utils/encoders.py%60.%20This%20function%20recursively%20converts%20Pydantic%20models%2C%20dataclasses%2C%20enums%2C%20paths%2C%20decimals%2C%20and%20most%20custom%20objects%20%28via%20%60dict%28%29%60%20or%20%60vars%28%29%60%29%20into%20JSON-serializable%20formats%2C%20making%20it%20more%20comprehensive%20and%20maintainable%20than%20a%20simple%20fallback%20helper.%20You%20can%20use%20it%20like%20this%3A%0A%0A%60%60%60python%0Afrom%20core.model_runtime.utils.encoders%20import%20jsonable_encoder%0A%0A%28%22outputs%22%2C%20json.dumps%28jsonable_encoder%28domain_model.outputs%29%2C%20ensure_ascii%3DFalse%29%20if%20domain_model.outputs%20else%20%22%7B%7D%22%29%2C%0A%60%60%60%0A%0AApply%20the%20same%20pattern%20to%20%60inputs%60%20and%20%60graph%60%20fields%20as%20well.%20This%20will%20prevent%20serialization%20errors%20for%20custom%20objects%20and%20ensure%20complete%20log%20and%20status%20updates%20in%20both%20Logstore%20and%20PostgreSQL.%20If%20you%20encounter%20edge%20cases%20with%20specific%20custom%20objects%2C%20you%20can%20extend%20%60jsonable_encoder%60%20with%20custom%20encoders%20as%20needed.%20%20%0A%5BSee%20implementation%20details%5D%28https%3A//github.com/langgenius/dify/blob/9b6b2f31950c50ec54a06985dadaf6577e295c83/api/core/model_runtime/utils/encoders.py%23L53-L216%29)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/30454)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#21401
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?