vxcontrol/pentagi
1
0
Fork 0
mirror of https://github.com/vxcontrol/pentagi.git synced 2026-07-21 08:15:23 -04:00
Code Issues 45 Packages Projects Releases 9 Wiki Activity

[GH-ISSUE #285] [Bug]: Agent outputs Russian in message/result fields despite Flow.language = "English" #90

New Issue
Closed
opened 2026-06-06 22:09:04 -04:00 by yindo · 0 comments
yindo commented 2026-06-06 22:09:04 -04:00
Owner
Copy Link
Copy Source

Originally created by @HyggeHacker on GitHub (Apr 23, 2026).
Original GitHub issue: https://github.com/vxcontrol/pentagi/issues/285

Affected Component

Core Services (Frontend UI/Backend API)

Describe the bug

The agent emits Russian-language content in message (and some result) fields of tool calls, even when the Flow is configured with Language: "English" (the default set in backend/pkg/controller/flow.go). Other fields within the same tool call (e.g., question, query, subtasks[].description) consistently render in English.

Result: mixed-language output in the UI and in persisted flow artifacts. This contradicts the maintainer's stated English-only policy in #216.

Root cause

Tool-argument schemas in backend/pkg/tools/args.go use static jsonschema_description strings containing the phrase "in user's language only". Because jsonschema_description is a Go struct tag, the string is compiled into the binary — no template substitution occurs. The LLM receives a locale-ambiguous instruction at the schema level, while the system prompt (e.g., primary_agent.tmpl:240) separately says "communicate with the customer in their preferred language ({{.Lang}})" with {{.Lang}} rendered as "English".

Fields explicitly anchored to "in English" in the same file (see confirming evidence below) render reliably in English. The bug is specifically the "user's language" phrasing being ambiguous to the LLM.

Offending lines in backend/pkg/tools/args.go

All of the following jsonschema_description values end with "...in user's language only" or "(in user's language only)":

Line Struct field Observed in flow tools
20 File.Message file
34 HackResult.Message hack_result
44 SubtaskList.Message subtask_list (generator)
74 SubtaskPatch.Message subtask_patch (refiner)
79 Done.Result done
80 Done.Message done
84 AskUser.Message ask
89 ReportResult.Result report_result
90 ReportResult.Message report_result
98 Terminal.Message terminal
105 AskAdvice.Message advice
110 Search.Message search
116 SearchResult.Message (input) search_result
121 SearchResult.Message (output) search_result
129 SearchEngineSearch.Message google/duckduckgo/tavily/traversaal/perplexity/searxng/sploitus
145 EnricherResult.Message enricher_result
150 EnricherResult.Message (second struct) enricher_result
157 MemoristSearch.Message memorist
162 MemoristResult.Message memorist_result
182 StoreGuide.Message store_guide
195 StoreAnswer.Message store_answer
210 StoreCode.Message store_code
215 Maintenance.Message maintenance
220 MaintenanceResult.Message maintenance_result
225 Coder.Message coder
230 CodeResult.Message code_result
235 Pentester.Message pentester
240 HackResultInput.Message hack_result

Counter-evidence — fields anchored to "in English" in the same file that consistently render in English:

  • Line 102: AskAdvice.Question (...for clarifications in English)
  • Line 109: Search.Question (...in English)
  • Line 114: SearchEngineSearch.Query (...for better search result in English)
  • Line 120: SearchEngineSearch.Result (...in English)
  • Line 134: SearchInMemory.Query (...in English)
  • Line 149: EnricherResult.Result (...in English)
  • Line 154: MemoristSearch.Question (...in English)
  • Line 161: MemoristResult.Result (...in English)
  • Line 173: SearchGuide.Questions (...Formulate your queries in English)
  • Line 179: StoreGuide.Guide (...for future search in English)

The difference between the two groups is only the anchoring phrase. Every field phrased "in English" renders English; a consistent subset of fields phrased "in user's language only" renders Russian.

Steps to reproduce

  1. Fresh clone, cp .env.example .env, set ANTHROPIC_API_KEY, docker compose up -d.
  2. Log in as admin@pentagi.com / admin, change password on first login.
  3. Create a new Flow, provider anthropic, default Language (English).
  4. Enter a task in English, e.g.: "Perform web application reconnaissance on http://testphp.vulnweb.com/".
  5. Open the Flow's Automation tab and observe the first few tool calls produced by the generator and memorist agents.

System configuration

  • PentAGI 2.0.0 (2ec8ef3), fresh install from latest Docker images
  • Docker Desktop 4.60.0 on macOS
  • Provider: Anthropic (reproduced via both direct api.anthropic.com and via Azure Foundry at https://{resource}.services.ai.azure.com/anthropic/v1 using x-api-key)
  • Model: claude-sonnet-4-5 (generator falls to claude-opus-4-5 per backend/pkg/providers/anthropic/config.yml)

Evidence

From msglogs on a freshly-created Flow where the task description was 100% English:

id | type     | message
---+----------+----------------------------------------------------------
 3 | search   | Проверяю предыдущую информацию о целевом веб-приложении для тестирования безопасности.
 4 | search   | Searching for historical information about the target web application, IP address, URL...
 5 | thoughts | No previous target information found. Please provide the web application URL...
 7 | thoughts | Создаю план подзадач для разведки веб-приложения и выявления уязвимостей...

Same agent, same Flow, same English input — mixed output.

From msgchains.chain for the generator (flow_id=N, type='generator'), the final tool call's arguments:

{
  "message": "Создаю план подзадач для разведки веб-приложения и выявления уязвимостей. План начинается с получения информации о целевом приложении и завершается запросом одобрения перед любыми попытками эксплуатации.",
  "subtasks": [
    { "description": "Request the target web application URL or IP address from the user...", ... }
  ]
}

The message field (schema described as "in user's language only", args.go:44) is Russian. The subtasks[].description field (no locale instruction in its schema) is English.

Suggested fix

Two options:

Option A — static English (minimal). Global replace in backend/pkg/tools/args.go:

  • "...to send to the user in user's language only" → "...to send to the user in English"
  • "...(in user's language only)" → "...(in English)"

28 field descriptions, contained to one file. Half a day of work. Consistent with the English-only stance in #216.

Option B — runtime locale substitution. Move tool schema construction from the package-level var registryDefinitions to a builder function invoked per-Flow at NewFlowToolsExecutor time in backend/pkg/tools/tools.go, passing the Flow's Language and substituting it into the description strings. Preserves the original intent of the code if multi-language support is ever revisited, at the cost of ~1 day's refactor.

Option A is sufficient given #216's stated policy.

Verification

  • I have checked that this issue hasn't been already reported
  • I have provided all relevant configuration files (with sensitive data removed)
  • I have included relevant logs and error messages
  • I am running the latest version of PentAGI
Originally created by @HyggeHacker on GitHub (Apr 23, 2026). Original GitHub issue: https://github.com/vxcontrol/pentagi/issues/285 ### Affected Component Core Services (Frontend UI/Backend API) ### Describe the bug The agent emits Russian-language content in `message` (and some `result`) fields of tool calls, even when the Flow is configured with `Language: "English"` (the default set in `backend/pkg/controller/flow.go`). Other fields within the same tool call (e.g., `question`, `query`, `subtasks[].description`) consistently render in English. Result: mixed-language output in the UI and in persisted flow artifacts. This contradicts the maintainer's stated English-only policy in #216. ### Root cause Tool-argument schemas in `backend/pkg/tools/args.go` use static `jsonschema_description` strings containing the phrase `"in user's language only"`. Because `jsonschema_description` is a Go struct tag, the string is compiled into the binary — no template substitution occurs. The LLM receives a locale-ambiguous instruction at the schema level, while the system prompt (e.g., `primary_agent.tmpl:240`) separately says `"communicate with the customer in their preferred language ({{.Lang}})"` with `{{.Lang}}` rendered as `"English"`. Fields explicitly anchored to `"in English"` in the same file (see confirming evidence below) render reliably in English. The bug is specifically the `"user's language"` phrasing being ambiguous to the LLM. ### Offending lines in `backend/pkg/tools/args.go` All of the following `jsonschema_description` values end with `"...in user's language only"` or `"(in user's language only)"`: | Line | Struct field | Observed in flow tools | |------|--------------|------------------------| | 20 | `File.Message` | file | | 34 | `HackResult.Message` | hack_result | | 44 | `SubtaskList.Message` | subtask_list (generator) | | 74 | `SubtaskPatch.Message` | subtask_patch (refiner) | | 79 | `Done.Result` | done | | 80 | `Done.Message` | done | | 84 | `AskUser.Message` | ask | | 89 | `ReportResult.Result` | report_result | | 90 | `ReportResult.Message` | report_result | | 98 | `Terminal.Message` | terminal | | 105 | `AskAdvice.Message` | advice | | 110 | `Search.Message` | search | | 116 | `SearchResult.Message` (input) | search_result | | 121 | `SearchResult.Message` (output)| search_result | | 129 | `SearchEngineSearch.Message`| google/duckduckgo/tavily/traversaal/perplexity/searxng/sploitus | | 145 | `EnricherResult.Message` | enricher_result | | 150 | `EnricherResult.Message` (second struct) | enricher_result | | 157 | `MemoristSearch.Message` | memorist | | 162 | `MemoristResult.Message` | memorist_result | | 182 | `StoreGuide.Message` | store_guide | | 195 | `StoreAnswer.Message` | store_answer | | 210 | `StoreCode.Message` | store_code | | 215 | `Maintenance.Message` | maintenance | | 220 | `MaintenanceResult.Message` | maintenance_result | | 225 | `Coder.Message` | coder | | 230 | `CodeResult.Message` | code_result | | 235 | `Pentester.Message` | pentester | | 240 | `HackResultInput.Message` | hack_result | Counter-evidence — fields anchored to `"in English"` in the same file that consistently render in English: - Line 102: `AskAdvice.Question` (`...for clarifications in English`) - Line 109: `Search.Question` (`...in English`) - Line 114: `SearchEngineSearch.Query` (`...for better search result in English`) - Line 120: `SearchEngineSearch.Result` (`...in English`) - Line 134: `SearchInMemory.Query` (`...in English`) - Line 149: `EnricherResult.Result` (`...in English`) - Line 154: `MemoristSearch.Question` (`...in English`) - Line 161: `MemoristResult.Result` (`...in English`) - Line 173: `SearchGuide.Questions` (`...Formulate your queries in English`) - Line 179: `StoreGuide.Guide` (`...for future search in English`) The difference between the two groups is only the anchoring phrase. Every field phrased `"in English"` renders English; a consistent subset of fields phrased `"in user's language only"` renders Russian. ### Steps to reproduce 1. Fresh clone, `cp .env.example .env`, set `ANTHROPIC_API_KEY`, `docker compose up -d`. 2. Log in as `admin@pentagi.com / admin`, change password on first login. 3. Create a new Flow, provider `anthropic`, default Language (`English`). 4. Enter a task in English, e.g.: `"Perform web application reconnaissance on http://testphp.vulnweb.com/"`. 5. Open the Flow's Automation tab and observe the first few tool calls produced by the `generator` and `memorist` agents. ### System configuration - PentAGI 2.0.0 (`2ec8ef3`), fresh install from latest Docker images - Docker Desktop 4.60.0 on macOS - Provider: Anthropic (reproduced via both direct `api.anthropic.com` and via Azure Foundry at `https://{resource}.services.ai.azure.com/anthropic/v1` using `x-api-key`) - Model: `claude-sonnet-4-5` (generator falls to `claude-opus-4-5` per `backend/pkg/providers/anthropic/config.yml`) ### Evidence From `msglogs` on a freshly-created Flow where the task description was 100% English: ``` id | type | message ---+----------+---------------------------------------------------------- 3 | search | Проверяю предыдущую информацию о целевом веб-приложении для тестирования безопасности. 4 | search | Searching for historical information about the target web application, IP address, URL... 5 | thoughts | No previous target information found. Please provide the web application URL... 7 | thoughts | Создаю план подзадач для разведки веб-приложения и выявления уязвимостей... ``` Same agent, same Flow, same English input — mixed output. From `msgchains.chain` for the generator (`flow_id=N`, `type='generator'`), the final tool call's arguments: ```json { "message": "Создаю план подзадач для разведки веб-приложения и выявления уязвимостей. План начинается с получения информации о целевом приложении и завершается запросом одобрения перед любыми попытками эксплуатации.", "subtasks": [ { "description": "Request the target web application URL or IP address from the user...", ... } ] } ``` The `message` field (schema described as `"in user's language only"`, args.go:44) is Russian. The `subtasks[].description` field (no locale instruction in its schema) is English. ### Suggested fix Two options: **Option A — static English (minimal).** Global replace in `backend/pkg/tools/args.go`: - `"...to send to the user in user's language only"` → `"...to send to the user in English"` - `"...(in user's language only)"` → `"...(in English)"` 28 field descriptions, contained to one file. Half a day of work. Consistent with the English-only stance in #216. **Option B — runtime locale substitution.** Move tool schema construction from the package-level `var registryDefinitions` to a builder function invoked per-Flow at `NewFlowToolsExecutor` time in `backend/pkg/tools/tools.go`, passing the Flow's `Language` and substituting it into the description strings. Preserves the original intent of the code if multi-language support is ever revisited, at the cost of ~1 day's refactor. Option A is sufficient given #216's stated policy. ### Verification - [x] I have checked that this issue hasn't been already reported - [x] I have provided all relevant configuration files (with sensitive data removed) - [x] I have included relevant logs and error messages - [x] I am running the latest version of PentAGI
yindo closed this issue 2026-06-06 22:09:04 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
enhancement
help wanted
pull-request
Mirrored from GitHub Pull Request
 question
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: vxcontrol/pentagi#90
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?