mirror of
https://github.com/vxcontrol/pentagi.git
synced 2026-07-21 08:15:23 -04:00
[GH-ISSUE #31] [Bug]: json #15
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @JacksonLvX on GitHub (Apr 9, 2025).
Original GitHub issue: https://github.com/vxcontrol/pentagi/issues/31
Originally assigned to: @asdek on GitHub.
Affected Component
Core Services (Frontend UI/Backend API)
Describe the bug
When using the local deepseek model, the JSON format data is not parsed and all the content is put into the title. How should I solve this problem?
Steps to Reproduce
docker exec -it pentagi /opt/pentagi/bin/ctester -verbose
docker exec -it pentagi /opt/pentagi/bin/ctester -verbose
2025/04/09 03:52:16 Warning: Error loading .env file: open .env: no such file or directory
Testing Custom Provider with configuration:
Testing agent: simple
Model: deepseek-r1:8b
Test Results:
Basic Tests:
[✗] Completion: What is 2+2? Write only the... (102.359s)
[✓] Completion: Write 'Hello World' in uppe... (233.734s)
[✓] System-User: Count from 1 to 5, separate... (207.025s)
[✓] System-User: Calculate 5 * 10 and provid... (106.261s)
[✓] SimpleJSON: Return a JSON with a person... (176.476s)
[✗] Basic echo function (0.030s)
Error: API error: API returned unexpected status code: 400: registry.ollama.ai/library/deepseek-r1:8b does not support tools
Advanced Tests:
[✗] JSON response function (0.034s)
Error: API error: API returned unexpected status code: 400: registry.ollama.ai/library/deepseek-r1:8b does not support tools
[✗] Search query (0.020s)
Error: API error: API returned unexpected status code: 400: registry.ollama.ai/library/deepseek-r1:8b does not support tools
[✗] Ask advice (0.018s)
Error: API error: API returned unexpected status code: 400: registry.ollama.ai/library/deepseek-r1:8b does not support tools
Summary: 4/9 (44.44%) successful tests
Average latency: 91.773s
Testing agent: simple_json
Model: deepseek-r1:8b
Test Results:
Basic Tests:
[✗] SimpleJSON: Return a JSON with a person... (171.926s)
Error: JSON parsing error: invalid character 'a' after top-level value
[✗] SimpleJSON: Create a JSON object with f... (321.213s)
Error: JSON parsing error: invalid character 'I' after top-level value
[✓] SimpleJSON: Generate a JSON response fo... (161.093s)
[✗] SimpleJSON: Create a JSON array of 3 co... (166.866s)
Error: JSON array parsing error: invalid character 'T' after top-level value
System Configuration
.env
Custom LLM provider
LLM_SERVER_URL=http://192.168.173.223:11434/v1/
LLM_SERVER_KEY=null
LLM_SERVER_MODEL=deepseek-r1:8b
Logs and Artifacts
No response
Screenshots or Recordings
No response
Verification
@asdek commented on GitHub (Apr 9, 2025):
Hello @JacksonLvX
Thank you for bringing this issue to our attention.
I see a couple of potential problems:
1. Lack of Function Calling Support in the DeepSeek R1 Model:
From your test results and the error messages you're encountering-specifically:
it appears that the DeepSeek R1 model (
deepseek-r1:8b) does not support function calling. According to the official DeepSeek documentation about R1 model:PentAGI relies on the LLM's ability to perform function calls and output structured JSON data to interact correctly with agents and execute tasks. Since the R1 model doesn't support function calling or JSON output, it cannot process the tasks as expected by PentAGI.
2. Model Size Limitations:
Based on my experience, models of this size often struggle with the complex tasks that PentAGI agents are designed to perform.
In my tests, acceptable results were achievable only with highly detailed prompts on models with at least 34B parameters. A more consistent and successful completion of tasks generally starts with models having 70B parameters.
Smaller models may lack the necessary capacity to handle the reasoning, function execution, and adherence to structured outputs required by PentAGI.
JFYI currently, the R1 model is not included in the standard configuration for task execution in PentAGI because of its limitations. It's used only in the reflector agent, which operates in completion mode and doesn't call functions. You can view the configuration here: deepseek.provider.yml.
For now, I recommend configuring PentAGI to use an LLM that supports function calling and structured JSON outputs to ensure proper functionality.