Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1983] [BUG]: OpenRouter always cut off it's response #1288

New Issue
Closed
opened 2026-02-22 18:24:06 -05:00 by yindo · 9 comments
yindo commented 2026-02-22 18:24:06 -05:00
Owner
Copy Link
Copy Source

Originally created by @PierrunoYT on GitHub (Jul 27, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1983

Originally assigned to: @timothycarambat on GitHub.

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

[BUG]: OpenRouter always cut off it's response

grafik
grafik

Are there known steps to reproduce?

No response

Originally created by @PierrunoYT on GitHub (Jul 27, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1983 Originally assigned to: @timothycarambat on GitHub. ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? [BUG]: OpenRouter always cut off it's response ![grafik](https://github.com/user-attachments/assets/be99913e-f398-4423-a60e-d65f4e9b7d45) ![grafik](https://github.com/user-attachments/assets/c00eef2d-9cf5-42d4-a072-71be73aeb60b) ### Are there known steps to reproduce? _No response_
yindo added the possible buginvestigating labels 2026-02-22 18:24:06 -05:00
yindo closed this issue 2026-02-22 18:24:07 -05:00
yindo commented 2026-02-22 18:24:07 -05:00
Author
Owner
Copy Link
Copy Source

@PierrunoYT commented on GitHub (Jul 27, 2024):

https://discord.com/channels/1114740394715004990/1266674095286784051
https://discord.com/channels/1114740394715004990/1248000016786919546

@PierrunoYT commented on GitHub (Jul 27, 2024): https://discord.com/channels/1114740394715004990/1266674095286784051 https://discord.com/channels/1114740394715004990/1248000016786919546
yindo commented 2026-02-22 18:24:07 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 27, 2024):

What models are you using on OpenRouter so we can replicate - latency matters a lot because OR does normally return done finish responses :/

@timothycarambat commented on GitHub (Jul 27, 2024): What models are you using on OpenRouter so we can replicate - latency matters a lot because OR does normally return `done` finish responses :/
yindo commented 2026-02-22 18:24:07 -05:00
Author
Owner
Copy Link
Copy Source

@PierrunoYT commented on GitHub (Jul 28, 2024):

What models are you using on OpenRouter so we can replicate - latency matters a lot because OR does normally return done finish responses :/

I used Claude and ChatGPT but I think it will be on all models.

@PierrunoYT commented on GitHub (Jul 28, 2024): > What models are you using on OpenRouter so we can replicate - latency matters a lot because OR does normally return `done` finish responses :/ I used Claude and ChatGPT but I think it will be on all models.
yindo commented 2026-02-22 18:24:07 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 29, 2024):

Using meta-llama/llama-3-8b-instruct
Screenshot 2024-07-29 at 11 23 13 AM

Using meta-llama/llama-3.1-8b-instruct:free
Screenshot 2024-07-29 at 11 24 39 AM

Using GPT/Anthropic and other random models I get responses as well.

I think the issue here is the whole issue with OpenRouter we have to work around for here:
https://github.com/Mintplex-Labs/anything-llm/blob/296f04156455346a35cec4440239325523265d33/server/utils/AiProviders/openRouter/index.js#L164

Your internet connection to the model may be just slow enough per token response that you exceed the 500ms timeout which leads to the response being cut off. I have 5GB download, so my internet speeds are quite fast and that may not be the case on your and others ends.

We can make this timeout a config?

@timothycarambat commented on GitHub (Jul 29, 2024): Using `meta-llama/llama-3-8b-instruct` <img width="1374" alt="Screenshot 2024-07-29 at 11 23 13 AM" src="https://github.com/user-attachments/assets/98a27d0b-2c02-48b7-9644-50dfba1f4068"> Using `meta-llama/llama-3.1-8b-instruct:free` <img width="1412" alt="Screenshot 2024-07-29 at 11 24 39 AM" src="https://github.com/user-attachments/assets/9f61cc0f-2b72-447c-b6f0-4bc7f9b27b46"> Using GPT/Anthropic and other random models I get responses as well. I think the issue here is the whole issue with OpenRouter we have to work around for here: https://github.com/Mintplex-Labs/anything-llm/blob/296f04156455346a35cec4440239325523265d33/server/utils/AiProviders/openRouter/index.js#L164 Your internet connection to the model may be just slow enough per token response that you exceed the 500ms timeout which leads to the response being cut off. I have 5GB download, so my internet speeds are quite fast and that may not be the case on your and others ends. We can make this timeout a config?
yindo commented 2026-02-22 18:24:07 -05:00
Author
Owner
Copy Link
Copy Source

@PierrunoYT commented on GitHub (Jul 29, 2024):

Can you check this
grafik

Not sure how we can fix this issue.

@PierrunoYT commented on GitHub (Jul 29, 2024): Can you check this ![grafik](https://github.com/user-attachments/assets/00b5495e-f757-46e3-866f-eb45e078db0d) Not sure how we can fix this issue.
yindo commented 2026-02-22 18:24:07 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 29, 2024):

That is not the issue.

@timothycarambat commented on GitHub (Jul 29, 2024): That is not the issue.
yindo commented 2026-02-22 18:24:08 -05:00
Author
Owner
Copy Link
Copy Source

@PierrunoYT commented on GitHub (Jul 29, 2024):

That is not the issue.

Okay then he was wrong.

@PierrunoYT commented on GitHub (Jul 29, 2024): > That is not the issue. Okay then he was wrong.
yindo commented 2026-02-22 18:24:08 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 29, 2024):

Unless you are sending massive prompts the max_tokens response would not impact your output, which is not what the examples showed how you replicated the issue

@timothycarambat commented on GitHub (Jul 29, 2024): Unless you are sending massive prompts the max_tokens response would not impact your output, which is not what the examples showed how you replicated the issue
yindo commented 2026-02-22 18:24:08 -05:00
Author
Owner
Copy Link
Copy Source

@PierrunoYT commented on GitHub (Aug 4, 2024):

@timothycarambat I set it to 1000 and I still get this
grafik

@PierrunoYT commented on GitHub (Aug 4, 2024): @timothycarambat I set it to 1000 and I still get this ![grafik](https://github.com/user-attachments/assets/238ef5f7-e59c-4bce-857a-7d57b9ecfb2c)
yindo changed title from [BUG]: OpenRouter always cut off it's response to [GH-ISSUE #1983] [BUG]: OpenRouter always cut off it's response 2026-06-05 14:39:56 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1288
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?