Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5481] [BUG]: Connection error with self-hosted gpt-oss-120b via API #5115

New Issue
Closed
opened 2026-06-05 14:52:05 -04:00 by yindo · 2 comments
yindo commented 2026-06-05 14:52:05 -04:00
Owner
Copy Link
Copy Source

Originally created by @pctapia on GitHub (Apr 22, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5481

How are you running AnythingLLM?

Docker (local)

What happened?

I’m running AnythingLLM in Docker with a local gpt-oss-120b model (accessed via API_KEY at https://ai-company-llm-enpoint.intra/v1/). The backend initializes successfully, but chat mode fails with:

Error: Connection error.
  at OpenAI.makeRequest (node_modules/openai/core.js:332:19)
  ...
  at async streamChatWithWorkspace (server/utils/chats/stream.js:267:20)

Observations:

LLM inference URL is correctly set (GenericOpenAiLLM logs confirm).
No network/firewall issues (other API calls succeed, like curl from the docker container).
Question: Is there a known issue with self-hosted OpenAI-compatible APIs in AnythingLLM? Any required config tweaks (e.g., headers, timeouts) for local endpoints?

Logs:

[backend] info: [GenericOpenAiEmbedder] Initialized bge-m3 {"baseURL":"https://ai-company-llm-enpoint.intra/v1/","maxConcurrentChunks":500,"embeddingMaxChunkLength":1000}
[backend] info: [GenericOpenAiLLM] Inference API: https://ai-company-llm-enpoint.intra/v1/⁠ Model: gpt-oss-120b
[backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-oss-120b
prisma:info Starting a sqlite pool with 21 connections.
[backend] warn: Child process exited with code 0 and signal null
[backend] info: Worker for job "cleanup-orphan-documents" exited with code 0
[backend] error: Error: Connection error.
    at OpenAI.makeRequest (/app/server/node_modules/openai/core.js:332:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async LLMPerformanceMonitor.measureStream (/app/server/utils/helpers/chat/LLMPerformanceMonitor.js:75:20)
    at async GenericOpenAiLLM.streamGetChatCompletion (/app/server/utils/AiProviders/genericOpenAi/index.js:252:35)
    at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:267:20)
    at async /app/server/endpoints/chat.js:63:9

Are there known steps to reproduce?

No response

LLM Provider & Model (if applicable)

LLM_PROVIDER='generic-openai'
GENERIC_OPEN_AI_BASE_PATH='https://ai-company-llm-enpoint.intra/v1/'
GENERIC_OPEN_AI_MODEL_PREF='gpt-oss-120b'
GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT='4096'
GENERIC_OPEN_AI_API_KEY='<MASKED_TOKEN>'

Embedder Provider & Model (if applicable)

EMBEDDING_MODEL_PREF='bge-m3'
EMBEDDING_ENGINE='generic-openai'
EMBEDDING_BASE_PATH='https://ai-company-llm-enpoint.intra/v1/'
GENERIC_OPEN_AI_EMBEDDING_API_KEY='<MASKED_TOKEN>'

Originally created by @pctapia on GitHub (Apr 22, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5481 ### How are you running AnythingLLM? Docker (local) ### What happened? I’m running AnythingLLM in Docker with a local gpt-oss-120b model (accessed via API_KEY at https://ai-company-llm-enpoint.intra/v1/). The backend initializes successfully, but chat mode fails with: ``` Error: Connection error. at OpenAI.makeRequest (node_modules/openai/core.js:332:19) ... at async streamChatWithWorkspace (server/utils/chats/stream.js:267:20) ``` Observations: LLM inference URL is correctly set (GenericOpenAiLLM logs confirm). No network/firewall issues (other API calls succeed, like curl from the docker container). Question: Is there a known issue with self-hosted OpenAI-compatible APIs in AnythingLLM? Any required config tweaks (e.g., headers, timeouts) for local endpoints? Logs: ``` [backend] info: [GenericOpenAiEmbedder] Initialized bge-m3 {"baseURL":"https://ai-company-llm-enpoint.intra/v1/","maxConcurrentChunks":500,"embeddingMaxChunkLength":1000} [backend] info: [GenericOpenAiLLM] Inference API: https://ai-company-llm-enpoint.intra/v1/⁠ Model: gpt-oss-120b [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-oss-120b prisma:info Starting a sqlite pool with 21 connections. [backend] warn: Child process exited with code 0 and signal null [backend] info: Worker for job "cleanup-orphan-documents" exited with code 0 [backend] error: Error: Connection error. at OpenAI.makeRequest (/app/server/node_modules/openai/core.js:332:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async LLMPerformanceMonitor.measureStream (/app/server/utils/helpers/chat/LLMPerformanceMonitor.js:75:20) at async GenericOpenAiLLM.streamGetChatCompletion (/app/server/utils/AiProviders/genericOpenAi/index.js:252:35) at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:267:20) at async /app/server/endpoints/chat.js:63:9 ``` ### Are there known steps to reproduce? _No response_ ### LLM Provider & Model (if applicable) LLM_PROVIDER='generic-openai' GENERIC_OPEN_AI_BASE_PATH='https://ai-company-llm-enpoint.intra/v1/' GENERIC_OPEN_AI_MODEL_PREF='gpt-oss-120b' GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT='4096' GENERIC_OPEN_AI_API_KEY='<MASKED_TOKEN>' ### Embedder Provider & Model (if applicable) EMBEDDING_MODEL_PREF='bge-m3' EMBEDDING_ENGINE='generic-openai' EMBEDDING_BASE_PATH='https://ai-company-llm-enpoint.intra/v1/' GENERIC_OPEN_AI_EMBEDDING_API_KEY='<MASKED_TOKEN>'
yindo added the possible bug label 2026-06-05 14:52:05 -04:00
yindo closed this issue 2026-06-05 14:52:05 -04:00
yindo commented 2026-06-05 14:52:06 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 22, 2026):

This is using a custom endpoint and its very likely whatever you have running does not acutally support the OpenAI compatible endpoint schema.

This works as-is with llama.cpp as well as hundreds of other cloud providers that go through this connector - so this is a bug with your internal service not being in spec, not our connector

<!-- gh-comment-id:4298083775 --> @timothycarambat commented on GitHub (Apr 22, 2026): This is using a custom endpoint and its very likely whatever you have running does not _acutally_ support the OpenAI compatible endpoint schema. This works as-is with llama.cpp as well as hundreds of other cloud providers that go through this connector - so this is a bug with your internal service not being in spec, not our connector
yindo commented 2026-06-05 14:52:06 -04:00
Author
Owner
Copy Link
Copy Source

@pctapia commented on GitHub (Apr 23, 2026):

My endpoints are OpenAI compatible. You might be right and the request is blocked due to missing the endpoint certificate.
Is there any way to add a root certificate to each request AnythingLLM executes? For instance to the LLM, to the repo, etc.

<!-- gh-comment-id:4302928218 --> @pctapia commented on GitHub (Apr 23, 2026): My endpoints are OpenAI compatible. You might be right and the request is blocked due to missing the endpoint certificate. Is there any way to add a root certificate to each request AnythingLLM executes? For instance to the LLM, to the repo, etc.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5115
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?