Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-10 18:33:40 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #5762] Minify Generic OpenAI request bodies for better server compatibility #5572

New Issue
Open
opened 2026-06-07 16:45:28 -04:00 by yindo · 0 comments
yindo commented 2026-06-07 16:45:28 -04:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/5762
Author: @shatfield4
Created: 6/5/2026
Status: 🔄 Open

Base: master ← Head: 5752-bug-prompts-to-local-jan-endpoint-unresponsive

📝 Commits (1)

  • 3bba4d1 minify generic openai request bodies for better server compatibility

📊 Changes

2 files changed (+21 additions, -0 deletions)

View changed files

📝 server/utils/AiProviders/genericOpenAi/index.js (+20 -0)
📝 server/utils/agents/aibitat/providers/genericOpenAi.js (+1 -0)

📄 Description

Pull Request Type

  • ✨ feat (New feature)
  • 🐛 fix (Bug fix)
  • ♻️ refactor (Code refactoring without changing behavior)
  • 💄 style (UI style changes)
  • 🔨 chore (Build, CI, maintenance)
  • 📝 docs (Documentation updates)

Relevant Issues

resolves #5752

Description

  • Chats and agents sent to a Generic OpenAI endpoint would hang forever with no response, no error, and no timeout — even though the endpoint was healthy and curl/other clients got normal responses from the same URL.
  • Root cause: the openai Node SDK serializes every request body as pretty-printed JSON (JSON.stringify(body, null, 2)), so the body contains literal newlines, and some OpenAI-compatible servers mishandle that whitespace and never respond.
  • Confirmed by replaying the exact request with curl: a compact single-line body streamed instantly, while the same body pretty-printed with newlines hung indefinitely.
  • This is a server-side regression, not ours — the SDK has always pretty-printed bodies (verified 4.38.5 and 4.95.1 behave identically). It surfaced on Jan after the unified llama.cpp router process introduced in v0.8.0, which the reporter's v0.8.2 inherits.
  • Fix: a small fetch wrapper on the Generic OpenAI clients (chat + agent paths) re-serializes string JSON request bodies compactly before sending. Minified JSON is a lossless round-trip and the more universally compatible form on the wire, so it works across every backend the provider targets.
  • Safe for other providers: the transform only removes formatting whitespace — values, string contents (including \n/\t inside messages), escapes, and numbers stay byte-identical to what the SDK already produced. Only string JSON bodies are touched; multipart/binary and unparseable bodies pass through untouched.
  • Scoped to the Generic OpenAI provider's clients only — OpenAI, Anthropic, Groq, Together, etc. build their own clients that are not modified and keep sending requests exactly as before.
  • Jan regression reference: v0.8.0 changelog (router process introduced), v0.8.0 release, v0.8.2 release (version reported in #5752).

Visuals (if applicable)

Additional Information

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated (if applicable)
  • I have tested my code functionality
  • Docker build succeeds locally

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/5762 **Author:** [@shatfield4](https://github.com/shatfield4) **Created:** 6/5/2026 **Status:** 🔄 Open **Base:** `master` ← **Head:** `5752-bug-prompts-to-local-jan-endpoint-unresponsive` --- ### 📝 Commits (1) - [`3bba4d1`](https://github.com/Mintplex-Labs/anything-llm/commit/3bba4d1e67a28ef7bdbeaead8875fa773986e352) minify generic openai request bodies for better server compatibility ### 📊 Changes **2 files changed** (+21 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `server/utils/AiProviders/genericOpenAi/index.js` (+20 -0) 📝 `server/utils/agents/aibitat/providers/genericOpenAi.js` (+1 -0) </details> ### 📄 Description ### Pull Request Type <!-- For change type, change [ ] to [x]. --> - [x] ✨ feat (New feature) - [ ] 🐛 fix (Bug fix) - [ ] ♻️ refactor (Code refactoring without changing behavior) - [ ] 💄 style (UI style changes) - [ ] 🔨 chore (Build, CI, maintenance) - [ ] 📝 docs (Documentation updates) ### Relevant Issues <!-- Use "resolves #xxx" to auto resolve on merge. Otherwise, please use "connect #xxx" --> resolves #5752 ### Description <!-- Describe the changes in this PR that are impactful to the repo. What problem does it solve? --> - Chats and agents sent to a Generic OpenAI endpoint would hang forever with no response, no error, and no timeout — even though the endpoint was healthy and `curl`/other clients got normal responses from the same URL. - Root cause: the `openai` Node SDK serializes every request body as pretty-printed JSON (`JSON.stringify(body, null, 2)`), so the body contains literal newlines, and some OpenAI-compatible servers mishandle that whitespace and never respond. - Confirmed by replaying the exact request with `curl`: a compact single-line body streamed instantly, while the same body pretty-printed with newlines hung indefinitely. - This is a server-side regression, not ours — the SDK has always pretty-printed bodies (verified `4.38.5` and `4.95.1` behave identically). It surfaced on Jan after the unified llama.cpp router process introduced in v0.8.0, which the reporter's v0.8.2 inherits. - Fix: a small `fetch` wrapper on the Generic OpenAI clients (chat + agent paths) re-serializes string JSON request bodies compactly before sending. Minified JSON is a lossless round-trip and the more universally compatible form on the wire, so it works across every backend the provider targets. - Safe for other providers: the transform only removes formatting whitespace — values, string contents (including `\n`/`\t` inside messages), escapes, and numbers stay byte-identical to what the SDK already produced. Only string JSON bodies are touched; multipart/binary and unparseable bodies pass through untouched. - Scoped to the Generic OpenAI provider's clients only — OpenAI, Anthropic, Groq, Together, etc. build their own clients that are not modified and keep sending requests exactly as before. - Jan regression reference: [v0.8.0 changelog](https://www.jan.ai/changelog/2026-05-22-jan-v0.8.0) (router process introduced), [v0.8.0 release](https://github.com/janhq/jan/releases/tag/v0.8.0), [v0.8.2 release](https://github.com/janhq/jan/releases/tag/v0.8.2) (version reported in #5752). ### Visuals (if applicable) <!-- Add screenshots or screen recordings to demonstrate the changes, especially for UI updates. --> ### Additional Information <!-- Add any other context about the Pull Request here that was not captured above. --> ### Developer Validations <!-- All of the applicable items should be checked. --> - [x] I ran `yarn lint` from the root of the repo & committed changes - [x] Relevant documentation has been updated (if applicable) - [x] I have tested my code functionality - [x] Docker build succeeds locally --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-06-07 16:45:28 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5572
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?