Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1530] [BUG]: Getting mostly empty answers when using Generic OpenAI #986

New Issue
Closed
opened 2026-02-22 18:22:32 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:22:32 -05:00
Owner
Copy Link
Copy Source

Originally created by @HuidaQ on GitHub (May 24, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1530

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

When using Generic OpenAI option in LLM Preference, most answers I got was either empty or super short/incomplete. See screenshots below.

Incomplete answers: (the Lora and QLora papers are added to the workspace)
Screenshot 2024-05-24 at 12 05 35 PM

LLM setting:
Screenshot 2024-05-24 at 12 06 00 PM

Are there known steps to reproduce?

See screenshots above.

Originally created by @HuidaQ on GitHub (May 24, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1530 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? When using Generic OpenAI option in LLM Preference, most answers I got was either empty or super short/incomplete. See screenshots below. Incomplete answers: (the Lora and QLora papers are added to the workspace) <img width="943" alt="Screenshot 2024-05-24 at 12 05 35 PM" src="https://github.com/Mintplex-Labs/anything-llm/assets/25400853/3b001f39-fd08-4941-a26f-db130c1fc1b3"> LLM setting: <img width="1130" alt="Screenshot 2024-05-24 at 12 06 00 PM" src="https://github.com/Mintplex-Labs/anything-llm/assets/25400853/1cf2da6a-550a-453f-a6ba-d3041c9aacc6"> ### Are there known steps to reproduce? See screenshots above.
yindo added the possible buginvestigating labels 2026-02-22 18:22:32 -05:00
yindo closed this issue 2026-02-22 18:22:33 -05:00
yindo commented 2026-02-22 18:22:33 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (May 24, 2024):

What LLM API are you using here? This tends to happen when the LLM API you are connecting to does not form the streaming chunks the same way as OpenAI does. If we can get access to that API you are using, we can create a custom integration for that specific API so it can handle the streaming chunks differently and not end the stream after just a few tokens.

@shatfield4 commented on GitHub (May 24, 2024): What LLM API are you using here? This tends to happen when the LLM API you are connecting to does not form the streaming chunks the same way as OpenAI does. If we can get access to that API you are using, we can create a custom integration for that specific API so it can handle the streaming chunks differently and not end the stream after just a few tokens.
yindo commented 2026-02-22 18:22:33 -05:00
Author
Owner
Copy Link
Copy Source

@HuidaQ commented on GitHub (May 24, 2024):

I'm using the lingyiwanwu's API. See https://platform.lingyiwanwu.com/docs (sorry the page is in Chinese). I can't expose my own key but they have free trials (up to 36 RMB) once signed up.

API call example from their docs:

import openai
from openai import OpenAI
API_BASE = "https://api.lingyiwanwu.com/v1"
API_KEY = "your key"
client = OpenAI(
    api_key=API_KEY,
    base_url=API_BASE
)
completion = client.chat.completions.create(
    model="yi-large",
    messages=[{"role": "user", "content": "Hi, who are you?"}]
)
print(completion)
@HuidaQ commented on GitHub (May 24, 2024): I'm using the lingyiwanwu's API. See https://platform.lingyiwanwu.com/docs (sorry the page is in Chinese). I can't expose my own key but they have free trials (up to 36 RMB) once signed up. API call example from their docs: ``` import openai from openai import OpenAI API_BASE = "https://api.lingyiwanwu.com/v1" API_KEY = "your key" client = OpenAI( api_key=API_KEY, base_url=API_BASE ) completion = client.chat.completions.create( model="yi-large", messages=[{"role": "user", "content": "Hi, who are you?"}] ) print(completion) ```
yindo commented 2026-02-22 18:22:33 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 24, 2024):

This will be fixed in the next version of the desktop app. This was already patched with PR #1487 :+1

@timothycarambat commented on GitHub (May 24, 2024): This will be fixed in the next version of the desktop app. This was already patched with PR #1487 :+1
yindo changed title from [BUG]: Getting mostly empty answers when using Generic OpenAI to [GH-ISSUE #1530] [BUG]: Getting mostly empty answers when using Generic OpenAI 2026-06-05 14:38:18 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#986
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?