[GH-ISSUE #313] [Bug]: Tool call calling issue in combination with llama.cpp backend #100

New Issue

2026-06-06T22:09:08-04:00

yindo commented

2026-06-06 22:09:08 -04:00

Originally created by @SebastianGode on GitHub (May 13, 2026).
Original GitHub issue: https://github.com/vxcontrol/pentagi/issues/313

Originally assigned to: @asdek on GitHub.

Affected Component

AI Agents (Researcher/Developer/...)

Describe the bug

When trying to run a simple test a few calls work and afterwards there's no response anymore in the UI to new questions:

Inside the log I get the following:
pentagi | time="2026-05-13T11:26:02Z" level=error msg="failed to perform assistant agent chain" error="failed to call agent chain: max retries reached, 3: API returned unexpected status code: 500: litellm.InternalServerError: InternalServerError: Custom_openaiException - Failed to parse tool call arguments as JSON: [json.exception.parse_error.101] parse error at line 1, column 131: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal. Received Model Group=qwen3.6-35b\nAvailable Model Group Fallbacks=None\nAPI returned unexpected status code: 500: litellm.InternalServerError: InternalServerError: Custom_openaiException - Failed to parse tool call arguments as JSON: [json.exception.parse_error.101] parse error at line 1, column 131: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal. Received Model Group=qwen3.6-35b\nAvailable Model Group Fallbacks=None\nAPI returned unexpected status code: 500: litellm.InternalServerError: InternalServerError: Custom_openaiException - Failed to parse tool call arguments as JSON: [json.exception.parse_error.101] parse error at line 1, column 131: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal. Received Model Group=qwen3.6-35b\nAvailable Model Group Fallbacks=None\nfailed to perform caller reflector: failed to call agent chain: max retries reached, 3: API returned unexpected status code: 429: No deployments available for selected model, Try again in 30 seconds.

The llama.cpp servers also show the following:

srv  log_server_r: done request: POST /v1/chat/completions 10.0.0.40 500
srv    operator(): got exception: {"error":{"code":500,"message":"Failed to parse tool call arguments as JSON: [json.exception.parse_error.101] parse error at line 1, column 131: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal","type":"server_error"}}

Steps to Reproduce

Access PentAGI UI
Open a new Flow
Do a simple task

System Configuration

Docker compose deployment with latest version, 3vCPU, 6GB RAM. LiteLLM Backend with llama.cpp as inference engines.

LLM_SERVER_URL=https://litellm.xyz.xyz/v1
LLM_SERVER_KEY=sk-mylitellmkey
LLM_SERVER_MODEL=qwen3.6-35b
LLM_SERVER_PROVIDER=
LLM_SERVER_CONFIG_PATH=
LLM_SERVER_LEGACY_REASONING=false
LLM_SERVER_PRESERVE_REASONING=false

Logs and Artifacts

No response

Screenshots or Recordings

No response

Verification

I have checked that this issue hasn't been already reported
I have provided all relevant configuration files (with sensitive data removed)
I have included relevant logs and error messages
I am running the latest version of PentAGI

Originally created by @SebastianGode on GitHub (May 13, 2026). Original GitHub issue: https://github.com/vxcontrol/pentagi/issues/313 Originally assigned to: @asdek on GitHub. ### Affected Component AI Agents (Researcher/Developer/...) ### Describe the bug When trying to run a simple test a few calls work and afterwards there's no response anymore in the UI to new questions: <img width="1670" height="1037" alt="Image" src="https://github.com/user-attachments/assets/6a650360-737d-4e25-b414-1caab332a3f8" /> Inside the log I get the following: `pentagi | time="2026-05-13T11:26:02Z" level=error msg="failed to perform assistant agent chain" error="failed to call agent chain: max retries reached, 3: API returned unexpected status code: 500: litellm.InternalServerError: InternalServerError: Custom_openaiException - Failed to parse tool call arguments as JSON: [json.exception.parse_error.101] parse error at line 1, column 131: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal. Received Model Group=qwen3.6-35b\nAvailable Model Group Fallbacks=None\nAPI returned unexpected status code: 500: litellm.InternalServerError: InternalServerError: Custom_openaiException - Failed to parse tool call arguments as JSON: [json.exception.parse_error.101] parse error at line 1, column 131: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal. Received Model Group=qwen3.6-35b\nAvailable Model Group Fallbacks=None\nAPI returned unexpected status code: 500: litellm.InternalServerError: InternalServerError: Custom_openaiException - Failed to parse tool call arguments as JSON: [json.exception.parse_error.101] parse error at line 1, column 131: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal. Received Model Group=qwen3.6-35b\nAvailable Model Group Fallbacks=None\nfailed to perform caller reflector: failed to call agent chain: max retries reached, 3: API returned unexpected status code: 429: No deployments available for selected model, Try again in 30 seconds.` The llama.cpp servers also show the following: ``` srv log_server_r: done request: POST /v1/chat/completions 10.0.0.40 500 srv operator(): got exception: {"error":{"code":500,"message":"Failed to parse tool call arguments as JSON: [json.exception.parse_error.101] parse error at line 1, column 131: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal","type":"server_error"}} ``` ### Steps to Reproduce 1. Access PentAGI UI 2. Open a new Flow 3. Do a simple task ### System Configuration Docker compose deployment with latest version, 3vCPU, 6GB RAM. LiteLLM Backend with llama.cpp as inference engines. ``` LLM_SERVER_URL=https://litellm.xyz.xyz/v1 LLM_SERVER_KEY=sk-mylitellmkey LLM_SERVER_MODEL=qwen3.6-35b LLM_SERVER_PROVIDER= LLM_SERVER_CONFIG_PATH= LLM_SERVER_LEGACY_REASONING=false LLM_SERVER_PRESERVE_REASONING=false ``` ### Logs and Artifacts _No response_ ### Screenshots or Recordings _No response_ ### Verification - [x] I have checked that this issue hasn't been already reported - [x] I have provided all relevant configuration files (with sensitive data removed) - [x] I have included relevant logs and error messages - [x] I am running the latest version of PentAGI

yindo added the bug label 2026-06-06 22:09:08 -04:00

yindo commented

2026-06-06 22:09:09 -04:00

@asdek commented on GitHub (May 29, 2026):

hey @SebastianGode

thank you for the feedback, it was fixed in the latest build, please update your installation.

it happens due to wrong previous response from llm side and while we are trying to fix it, was sending it with corrupted tool call args, now we are sanitizing wrong function arguments.

@asdek commented on GitHub (May 29, 2026): hey @SebastianGode thank you for the feedback, it was fixed in the latest build, please update your installation. it happens due to wrong previous response from llm side and while we are trying to fix it, was sending it with corrupted tool call args, now we are sanitizing wrong function arguments.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: vxcontrol/pentagi#100