[PR #330] docs: add tool-call parser troubleshooting for custom LLM backends #323

New Issue

2026-06-06T22:10:13-04:00

yindo commented

2026-06-06 22:10:13 -04:00

📋 Pull Request Information

Original PR: https://github.com/vxcontrol/pentagi/pull/330
Author: @mason5052
Created: 6/3/2026
Status: 🔄 Open

Base: main ← Head: codex/issue-313-tool-call-parser-troubleshooting

📝 Commits (1)

12f3d19 docs: add tool-call parser troubleshooting for custom LLM backends

📊 Changes

1 file changed (+20 additions, -0 deletions)

View changed files

📝 README.md (+20 -0)

📄 Description

Summary

Add a troubleshooting subsection under Custom LLM Provider Configuration explaining why tool-call (function-call) parser problems on self-hosted OpenAI-compatible backends (llama.cpp / SGLang / vLLM, often behind LiteLLM) cause stalled flows, and how to diagnose them. Docs only.

Problem

Issue #313 reported that flows stop responding after a few steps when running a custom backend configured through LLM_SERVER_* (LiteLLM in front of llama.cpp serving qwen3.6-35b). The logs showed:

Failed to parse tool call arguments as JSON: [json.exception.parse_error.101]
parse error at line 1, column 131: syntax error while parsing value -
unexpected end of input

surfaced through LiteLLM as an HTTP 500, followed by cascading retries and a 429. The maintainer confirmed the stall was fixed in the latest build by sanitizing malformed function-call arguments, and that the root cause was the model side returning corrupted tool-call arguments.

There is currently no documentation that explains this class of failure, even though it is a common pitfall with self-hosted backends and is closely related to the image-chooser failure (a flow's first action is an LLM tool call to pick the container image).

Solution

Add a #### Troubleshooting: tool-call (function-call) parser errors subsection right after the Custom LLM Provider Configuration content, covering:

Custom OpenAI-compatible backends must return valid tool-call JSON; llama.cpp, SGLang, and vLLM usually require a specific tool-call parser and a matching chat template, and not every setup produces valid tool calls out of the box (compatibility depends on the backend, not PentAGI alone).
Symptoms: Failed to parse tool call arguments as JSON, a flow that stalls after a few steps, looping tool calls, the start-of-flow failed to select primary docker image via llm call error, and unexpected backend 5xx/4xx responses.
How to investigate: check both PentAGI and backend/proxy logs, validate the provider with ctester before a full flow, confirm the parser/chat template match the model, and update PentAGI (recent builds sanitize malformed function-call arguments).

The new content links only to the existing Testing LLM Agents section and references the image-chooser error in prose (no new anchor), so it stands on its own against main.

User Impact

Users on self-hosted/llama.cpp/SGLang/vLLM backends get a clear explanation of the tool-call parser failure mode and a concrete diagnosis path, instead of an opaque Failed to parse tool call arguments as JSON stall.
Points users at ctester for pre-flight validation and at the update that sanitizes malformed arguments.
No behavior change.

Test Plan

git diff --check clean.
Docs-only diff: README.md (+20 lines). No tool-call parser code, provider runtime, schema, migration, or config-default changes.
Verified the referenced error string failed to select primary docker image via llm call exists in backend/pkg/providers/providers.go on main.
Verified LLM_SERVER_URL / LLM_SERVER_KEY / LLM_SERVER_MODEL / LLM_SERVER_PROVIDER exist in .env.example.
Verified the ctester utility exists and tests tool-calling agent types, and that the #testing-llm-agents anchor resolves.
Placed away from the README regions touched by open PRs #325 and #327 to avoid conflicts.
No unrelated files included.

Refs #313

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/vxcontrol/pentagi/pull/330 **Author:** [@mason5052](https://github.com/mason5052) **Created:** 6/3/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `codex/issue-313-tool-call-parser-troubleshooting` --- ### 📝 Commits (1) - [`12f3d19`](https://github.com/vxcontrol/pentagi/commit/12f3d1902102e9a340f204f38db199fdf384a8e2) docs: add tool-call parser troubleshooting for custom LLM backends ### 📊 Changes **1 file changed** (+20 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+20 -0) </details> ### 📄 Description ## Summary Add a troubleshooting subsection under **Custom LLM Provider Configuration** explaining why tool-call (function-call) parser problems on self-hosted OpenAI-compatible backends (llama.cpp / SGLang / vLLM, often behind LiteLLM) cause stalled flows, and how to diagnose them. Docs only. ## Problem Issue #313 reported that flows stop responding after a few steps when running a custom backend configured through `LLM_SERVER_*` (LiteLLM in front of llama.cpp serving `qwen3.6-35b`). The logs showed: ``` Failed to parse tool call arguments as JSON: [json.exception.parse_error.101] parse error at line 1, column 131: syntax error while parsing value - unexpected end of input ``` surfaced through LiteLLM as an HTTP 500, followed by cascading retries and a 429. The maintainer confirmed the stall was fixed in the latest build by sanitizing malformed function-call arguments, and that the root cause was the model side returning corrupted tool-call arguments. There is currently no documentation that explains this class of failure, even though it is a common pitfall with self-hosted backends and is closely related to the image-chooser failure (a flow's first action is an LLM tool call to pick the container image). ## Solution Add a `#### Troubleshooting: tool-call (function-call) parser errors` subsection right after the Custom LLM Provider Configuration content, covering: - Custom OpenAI-compatible backends must return valid tool-call JSON; llama.cpp, SGLang, and vLLM usually require a specific tool-call parser and a matching chat template, and not every setup produces valid tool calls out of the box (compatibility depends on the backend, not PentAGI alone). - Symptoms: `Failed to parse tool call arguments as JSON`, a flow that stalls after a few steps, looping tool calls, the start-of-flow `failed to select primary docker image via llm call` error, and unexpected backend 5xx/4xx responses. - How to investigate: check both PentAGI and backend/proxy logs, validate the provider with `ctester` before a full flow, confirm the parser/chat template match the model, and update PentAGI (recent builds sanitize malformed function-call arguments). The new content links only to the existing [Testing LLM Agents](https://github.com/vxcontrol/pentagi#testing-llm-agents) section and references the image-chooser error in prose (no new anchor), so it stands on its own against `main`. ## User Impact - Users on self-hosted/llama.cpp/SGLang/vLLM backends get a clear explanation of the tool-call parser failure mode and a concrete diagnosis path, instead of an opaque `Failed to parse tool call arguments as JSON` stall. - Points users at `ctester` for pre-flight validation and at the update that sanitizes malformed arguments. - No behavior change. ## Test Plan - [x] `git diff --check` clean. - [x] Docs-only diff: `README.md` (+20 lines). No tool-call parser code, provider runtime, schema, migration, or config-default changes. - [x] Verified the referenced error string `failed to select primary docker image via llm call` exists in `backend/pkg/providers/providers.go` on `main`. - [x] Verified `LLM_SERVER_URL` / `LLM_SERVER_KEY` / `LLM_SERVER_MODEL` / `LLM_SERVER_PROVIDER` exist in `.env.example`. - [x] Verified the `ctester` utility exists and tests tool-calling agent types, and that the `#testing-llm-agents` anchor resolves. - [x] Placed away from the README regions touched by open PRs #325 and #327 to avoid conflicts. - [x] No unrelated files included. Refs #313 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

yindo added the pull-request label 2026-06-06 22:10:13 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: vxcontrol/pentagi#323