Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4408] [BUG]: went wrong when using gemini2.5 pro in Openrouter.ai #2810

New Issue
Closed
opened 2026-02-22 18:31:22 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:31:22 -05:00
Owner
Copy Link
Copy Source

Originally created by @lcwxz1989 on GitHub (Sep 19, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4408

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

In anythingllm
Image
In openrouter webui

Image

Are there known steps to reproduce?

just configure openrouter in anythingllm, and use the geminixx pro will happened. every gemini pro version will encourter this problem

Originally created by @lcwxz1989 on GitHub (Sep 19, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4408 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? In anythingllm <img width="1336" height="958" alt="Image" src="https://github.com/user-attachments/assets/64af2a1e-eb2f-4f3f-b838-31b5dbff43cd" /> In openrouter webui <img width="1274" height="835" alt="Image" src="https://github.com/user-attachments/assets/891b09c5-285d-477f-a3b2-bb049437cb25" /> ### Are there known steps to reproduce? just configure openrouter in anythingllm, and use the geminixx pro will happened. every gemini pro version will encourter this problem
yindo added the possible bug label 2026-02-22 18:31:22 -05:00
yindo closed this issue 2026-02-22 18:31:22 -05:00
yindo commented 2026-02-22 18:31:22 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Sep 19, 2025):

This is actually because OR can be slower since it acts as a middleware for user<>provider interactions. This can increase the time between chunks compared to if you communicated directly with Gemini.

OpenRouter has some providers who have a habit of also not sending finish_reason in their chunks. To combat this, a long time ago we added a timeout to close the response if we go more than xms between chunks.

What is happening is that the timeout is closing the stream prematurely. To fix this now go to the LLM Provider screen for the system, select OR, and you will see under advanced options the timeout value - set this to something big like 3000.

Image

In the interim, I'm just going to bump up the default value.

With increased timeout the stream no longer hangs due to thinking chunk time-delays

Image
@timothycarambat commented on GitHub (Sep 19, 2025): This is actually because OR can be slower since it acts as a middleware for user<>provider interactions. This can increase the time between chunks compared to if you communicated directly with Gemini. OpenRouter has some providers who have a habit of also not sending `finish_reason` in their chunks. To combat this, a long time ago we [added a timeout](https://github.com/Mintplex-Labs/anything-llm/blob/95557ee16fd60094df0750fb9c913e2574cd2261/server/utils/AiProviders/openRouter/index.js#L90) to close the response if we go more than `x`ms between chunks. What is happening is that the timeout is closing the stream prematurely. To fix this _now_ go to the LLM Provider screen for the system, select OR, and you will see under advanced options the timeout value - set this to something big like `3000`. <img width="820" height="461" alt="Image" src="https://github.com/user-attachments/assets/feedde5c-e3fe-4f52-b4d1-267c4e08903a" /> In the interim, I'm just going to bump up the default value. --- With increased timeout the stream no longer hangs due to thinking chunk time-delays <img width="1170" height="631" alt="Image" src="https://github.com/user-attachments/assets/e5b525db-fd40-4246-bf2d-8e86d55c4fb2" />
yindo changed title from [BUG]: went wrong when using gemini2.5 pro in Openrouter.ai to [GH-ISSUE #4408] [BUG]: went wrong when using gemini2.5 pro in Openrouter.ai 2026-06-05 14:48:40 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2810
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?