Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4313] Cannot configure Azure OpenAI with Anythingllm #2743

New Issue
Closed
opened 2026-02-22 18:31:03 -05:00 by yindo · 7 comments
yindo commented 2026-02-22 18:31:03 -05:00
Owner
Copy Link
Copy Source

Originally created by @yeldhomathew on GitHub (Aug 19, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4313

Hello All,

I have deployed anythingllm using docker (docker run -d -p 3001:3001 -it --rm -v "${STORAGE_LOCATION}:/app/server/storage" -v "${STORAGE_LOCATION}/.env:/app/server/.env" -e STORAGE_DIR="/app/server/storage" -e HTTP_PROXY=Proxyserver:port -e HTTPS_PROXY=Proxyserver:port mintplexlabs/anythingllm:latest) in my on prem server. I am able to access the same using serverIp:3001 via browser. I also hosted chroma Db for vector database on the same server using docker (docker run -d --rm --name chromadb -p 8000:8000 -v ./chromadb:/chroma/chroma -e IS_PERSISTENT=TRUE chromadb/chroma:0.6.3). Now i am trying to connect Azure Open AI LLM model. but while giving prompt, I am getting below error.

Could not respond to message.
Connection error.

I am able to access https://xxxxxx-openai.openai.azure.com/ from the container where AnythingLLM runs. The proxy i am using in Docker run command is enough? or do i want to set proxy anywhere else?

Originally created by @yeldhomathew on GitHub (Aug 19, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4313 Hello All, I have deployed anythingllm using docker (docker run -d -p 3001:3001 -it --rm -v "${STORAGE_LOCATION}:/app/server/storage" -v "${STORAGE_LOCATION}/.env:/app/server/.env" -e STORAGE_DIR="/app/server/storage" -e HTTP_PROXY=Proxyserver:port -e HTTPS_PROXY=Proxyserver:port mintplexlabs/anythingllm:latest) in my on prem server. I am able to access the same using serverIp:3001 via browser. I also hosted chroma Db for vector database on the same server using docker (docker run -d --rm --name chromadb -p 8000:8000 -v ./chromadb:/chroma/chroma -e IS_PERSISTENT=TRUE chromadb/chroma:0.6.3). Now i am trying to connect Azure Open AI LLM model. but while giving prompt, I am getting below error. **Could not respond to message. Connection error.** I am able to access https://xxxxxx-openai.openai.azure.com/ from the container where AnythingLLM runs. The proxy i am using in Docker run command is enough? or do i want to set proxy anywhere else?
yindo closed this issue 2026-02-22 18:31:03 -05:00
yindo commented 2026-02-22 18:31:03 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 19, 2025):

The endpoint + auth token + model deployment name all need to be present to interact with Azure.
Example:
Endpoint: https://xxxxxx.openai.azure.com
Key: c********123
Model: my-openai-deployment - This is the "deployment" on Azure you want to use. Not the base model.

I have a deployment of gpt-4 called my-openai-deployment - that is what the model name needs to be if that is not the case currently. Othewise, I would check the container logs as they will be more verbose.

@timothycarambat commented on GitHub (Aug 19, 2025): The endpoint + auth token + model deployment name all need to be present to interact with Azure. Example: Endpoint: https://xxxxxx.openai.azure.com Key: c********123 Model: `my-openai-deployment` - This is the "deployment" on Azure you want to use. Not the base model. I have a deployment of gpt-4 called `my-openai-deployment` - that is what the model name needs to be if that is not the case currently. Othewise, I would check the container logs as they will be more verbose.
yindo commented 2026-02-22 18:31:03 -05:00
Author
Owner
Copy Link
Copy Source

@yeldhomathew commented on GitHub (Aug 20, 2025):

Thanks for the response.
Below are the logs i fond in anythingLLM container and cromaDB container while simply prompt a question "what is LLM"

from AnythingLLM

[backend] info: [AzureOpenAi] Initialized. Model "gpt-4o" @ 128000 tokens.
API-Version: 2024-12-01-preview.
Model Type: reasoning
[backend] error: ChromaDB::namespaceExists Collection propel does not exist.
[backend] info: [fillSourceWindow] Need to backfill 4 chunks to fill in the source window for RAG!
[backend] info: [TokenManager] Returning existing instance for model: gpt-4o
[backend] info: [AzureOpenAi] Streaming will be disabled. AZURE_OPENAI_MODEL_TYPE is set to 'reasoning'.
[backend] info: [STREAMING DISABLED] Streaming is not available for AzureOpenAiLLM. Will use regular chat method.
[backend] error: Error: Connection error.
at AzureOpenAI.makeRequest (/app/server/node_modules/openai/core.js:332:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /app/server/utils/helpers/chat/LLMPerformanceMonitor.js:48:22
at async AzureOpenAiLLM.getChatCompletion (/app/server/utils/AiProviders/azureOpenAi/index.js:138:20)
at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:232:7)
at async /app/server/endpoints/chat.js:63:9

From ChromaDB

INFO: [20-08-2025 16:42:24] Backing off send_request(...) for 0.9s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='us.i.posthog.com', port=443): Max retries exceeded with url: /batch/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f746e38b090>: Failed to establish a new connection: [Errno 101] Network is unreachable')))
INFO: [20-08-2025 16:42:38] HostIp:40228 - "GET /api/v2/heartbeat HTTP/1.1" 200
INFO: [20-08-2025 16:42:38] HostIp:40236 - "GET /api/v2/tenants/default_tenant HTTP/1.1" 200
INFO: [20-08-2025 16:42:38] HostIp:40228 - "GET /api/v2/tenants/default_tenant/databases/default_database HTTP/1.1" 200
INFO: [20-08-2025 16:42:38] HostIp:40228 - "GET /api/v2/tenants/default_tenant/databases/default_database/collections/propel HTTP/1.1" 400
INFO: [20-08-2025 16:42:38] HostIp:40228 - "GET /api/v2/heartbeat HTTP/1.1" 200
INFO: [20-08-2025 16:42:38] HostIp:40236 - "GET /api/v2/tenants/default_tenant HTTP/1.1" 200
INFO: [20-08-2025 16:42:38] HostIp:40228 - "GET /api/v2/tenants/default_tenant/databases/default_database HTTP/1.1" 200
INFO: [20-08-2025 16:42:38] HostIp:40236 - "GET /api/v2/tenants/default_tenant/databases/default_database/collections/propel HTTP/1.1" 400
INFO: [20-08-2025 16:43:55] Backing off send_request(...) for 1.3s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='us.i.posthog.com', port=443): Max retries exceeded with url: /batch/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f746ecb5990>: Failed to establish a new connection: [Errno 101] Network is unreachable')))
INFO: [20-08-2025 16:45:26] Backing off send_request(...) for 1.8s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='us.i.posthog.com', port=443): Max retries exceeded with url: /batch/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f746e232950>: Failed to establish a new connection: [Errno 101] Network is unreachable')))
ERROR: [20-08-2025 16:46:58] Giving up send_request(...) after 4 tries (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='us.i.posthog.com', port=443): Max retries exceeded with url: /batch/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f746e1b8fd0>: Failed to establish a new connection: [Errno 101] Network is unreachable')))

While i am uploading a txt file( my data) for save and embedding i am getting below logs from anythingLLM container

[backend] info: Adding new vectorized document into namespace propel
[backend] info: [RecursiveSplitter] Will split with {"chunkSize":2048,"chunkOverlap":20,"chunkHeader":"<document_metadata>\nsourceDocument: propel.txt\npub..."}
[backend] info: Snippets created from document: 1
[backend] info: [AzureOpenAiEmbedder] Embedding 1 chunks...
[backend] error: addDocumentToNamespace Azure OpenAI Failed to embed: [failed_to_embed]: Connection error.
[backend] error: Failed to vectorize propel.txt
[backend] info: [TELEMETRY SENT] {"event":"documents_embedded_in_workspace","distinctId":"446c0c1a-fe31-4b22-8c99-abe9a122a01c","properties":{"LLMSelection":"azure","Embedder":"azure","VectorDbSelection":"chroma","TTSSelection":"native","LLMModel":"gpt-4o","runtime":"docker"}}
[backend] info: [Event Logged] - workspace_documents_added

No logs found from CromaDB container

@yeldhomathew commented on GitHub (Aug 20, 2025): Thanks for the response. Below are the logs i fond in anythingLLM container and cromaDB container while simply prompt a question "what is LLM" from AnythingLLM [backend] info: [AzureOpenAi] Initialized. Model "gpt-4o" @ 128000 tokens. API-Version: 2024-12-01-preview. Model Type: reasoning [backend] error: ChromaDB::namespaceExists Collection propel does not exist. [backend] info: [fillSourceWindow] Need to backfill 4 chunks to fill in the source window for RAG! [backend] info: [TokenManager] Returning existing instance for model: gpt-4o [backend] info: [AzureOpenAi] Streaming will be disabled. AZURE_OPENAI_MODEL_TYPE is set to 'reasoning'. [backend] info: [STREAMING DISABLED] Streaming is not available for AzureOpenAiLLM. Will use regular chat method. [backend] error: Error: Connection error. at AzureOpenAI.makeRequest (/app/server/node_modules/openai/core.js:332:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async /app/server/utils/helpers/chat/LLMPerformanceMonitor.js:48:22 at async AzureOpenAiLLM.getChatCompletion (/app/server/utils/AiProviders/azureOpenAi/index.js:138:20) at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:232:7) at async /app/server/endpoints/chat.js:63:9 From ChromaDB INFO: [20-08-2025 16:42:24] Backing off send_request(...) for 0.9s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='us.i.posthog.com', port=443): Max retries exceeded with url: /batch/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f746e38b090>: Failed to establish a new connection: [Errno 101] Network is unreachable'))) INFO: [20-08-2025 16:42:38] HostIp:40228 - "GET /api/v2/heartbeat HTTP/1.1" 200 INFO: [20-08-2025 16:42:38] HostIp:40236 - "GET /api/v2/tenants/default_tenant HTTP/1.1" 200 INFO: [20-08-2025 16:42:38] HostIp:40228 - "GET /api/v2/tenants/default_tenant/databases/default_database HTTP/1.1" 200 INFO: [20-08-2025 16:42:38] HostIp:40228 - "GET /api/v2/tenants/default_tenant/databases/default_database/collections/propel HTTP/1.1" 400 INFO: [20-08-2025 16:42:38] HostIp:40228 - "GET /api/v2/heartbeat HTTP/1.1" 200 INFO: [20-08-2025 16:42:38] HostIp:40236 - "GET /api/v2/tenants/default_tenant HTTP/1.1" 200 INFO: [20-08-2025 16:42:38] HostIp:40228 - "GET /api/v2/tenants/default_tenant/databases/default_database HTTP/1.1" 200 INFO: [20-08-2025 16:42:38] HostIp:40236 - "GET /api/v2/tenants/default_tenant/databases/default_database/collections/propel HTTP/1.1" 400 INFO: [20-08-2025 16:43:55] Backing off send_request(...) for 1.3s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='us.i.posthog.com', port=443): Max retries exceeded with url: /batch/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f746ecb5990>: Failed to establish a new connection: [Errno 101] Network is unreachable'))) INFO: [20-08-2025 16:45:26] Backing off send_request(...) for 1.8s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='us.i.posthog.com', port=443): Max retries exceeded with url: /batch/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f746e232950>: Failed to establish a new connection: [Errno 101] Network is unreachable'))) ERROR: [20-08-2025 16:46:58] Giving up send_request(...) after 4 tries (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='us.i.posthog.com', port=443): Max retries exceeded with url: /batch/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f746e1b8fd0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))) While i am uploading a txt file( my data) for save and embedding i am getting below logs from anythingLLM container [backend] info: Adding new vectorized document into namespace propel [backend] info: [RecursiveSplitter] Will split with {"chunkSize":2048,"chunkOverlap":20,"chunkHeader":"<document_metadata>\nsourceDocument: propel.txt\npub..."} [backend] info: Snippets created from document: 1 [backend] info: [AzureOpenAiEmbedder] Embedding 1 chunks... [backend] error: addDocumentToNamespace Azure OpenAI Failed to embed: [failed_to_embed]: Connection error. [backend] error: Failed to vectorize propel.txt [backend] info: [TELEMETRY SENT] {"event":"documents_embedded_in_workspace","distinctId":"446c0c1a-fe31-4b22-8c99-abe9a122a01c","properties":{"LLMSelection":"azure","Embedder":"azure","VectorDbSelection":"chroma","TTSSelection":"native","LLMModel":"gpt-4o","runtime":"docker"}} [backend] info: [Event Logged] - workspace_documents_added No logs found from CromaDB container
yindo commented 2026-02-22 18:31:03 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 20, 2025):

Okay this has to do with your network layer between or on top of the containers, you are getting failed connections left and right. Even chroma cannot reach the public internet

Max retries exceeded with url: /batch/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f746e38b090>: Failed to establish a new connection: [Errno 101] Network is unreachable')))

Its also worth noting that this command:

docker run -d 
-p 3001:3001 -it --rm 
-v "${STORAGE_LOCATION}:/app/server/storage" 
-v "${STORAGE_LOCATION}/.env:/app/server/.env" 
-e STORAGE_DIR="/app/server/storage" 
-e HTTP_PROXY=Proxyserver:port 
-e HTTPS_PROXY=Proxyserver:port
 mintplexlabs/anythingllm:latest

Is not the same as the official command:

# Assuming STORAGE_LOCATION resolves to a userspace dir
docker run -d -p 3001:3001 \
--cap-add SYS_ADMIN \ # used for web-scraping
-v ${STORAGE_LOCATION}:/app/server/storage \
-v ${STORAGE_LOCATION}/.env:/app/server/.env \
-e STORAGE_DIR="/app/server/storage" \
mintplexlabs/anythingllm:latest

The notable difference being HTTP_PROXY and HTTPS_PROXY not being ENVs we explicitly respect in our code. HOWEVER undici in NodeJS, which OpenAI and chromadb package both do respect those envs and likely explains the failure since those ENVs are being applied to the process that undici runs in.

Can you remove those ENVs and see if the errors resolve?

@timothycarambat commented on GitHub (Aug 20, 2025): Okay this has to do with your network layer between or on top of the containers, you are getting failed connections left and right. Even chroma cannot reach the public internet > Max retries exceeded with url: /batch/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f746e38b090>: Failed to establish a new connection: [Errno 101] Network is unreachable'))) Its also worth noting that this command: ```bash docker run -d -p 3001:3001 -it --rm -v "${STORAGE_LOCATION}:/app/server/storage" -v "${STORAGE_LOCATION}/.env:/app/server/.env" -e STORAGE_DIR="/app/server/storage" -e HTTP_PROXY=Proxyserver:port -e HTTPS_PROXY=Proxyserver:port mintplexlabs/anythingllm:latest ``` Is not the same as the [official](https://docs.anythingllm.com/installation-docker/quickstart) command: ``` # Assuming STORAGE_LOCATION resolves to a userspace dir docker run -d -p 3001:3001 \ --cap-add SYS_ADMIN \ # used for web-scraping -v ${STORAGE_LOCATION}:/app/server/storage \ -v ${STORAGE_LOCATION}/.env:/app/server/.env \ -e STORAGE_DIR="/app/server/storage" \ mintplexlabs/anythingllm:latest ``` The notable difference being `HTTP_PROXY` and `HTTPS_PROXY` not being ENVs we explicitly respect in our code. **HOWEVER** [`undici`](https://github.com/nodejs/undici/pull/2994) in NodeJS, which OpenAI and chromadb package both _do respect_ those envs and likely explains the failure since those ENVs are being applied to the process that undici runs in. Can you remove those ENVs and see if the errors resolve?
yindo commented 2026-02-22 18:31:03 -05:00
Author
Owner
Copy Link
Copy Source

@yeldhomathew commented on GitHub (Aug 20, 2025):

Thanks again.
Still i have issue. so i have started using inbuilt vector DB (LanceDB) and tried with and without Proxy. below are the logs from Docker

No pending migrations to apply.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo
[backend] info: [ContextWindowFinder] Pulling remote model map...
[backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo
[backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM.
[backend] info: prisma:info Starting a sqlite pool with 17 connections.
[backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"177dd959-c851-4f8b-abc9-be829af8c992","properties":{"commit":"--","runtime":"docker"}}
[backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[backend] info: Primary server in HTTP mode listening on port 3001
[backend] info: [BackgroundWorkerService] Feature is not enabled and will not be started.
[backend] info: [MetaGenerator] fetching custom meta tag settings...
[backend] info: [ContextWindowFinder] Error syncing remote model map TypeError: fetch failed
at node:internal/deps/undici/undici:12637:11
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async #pullRemoteModelMap (/app/server/utils/AiProviders/modelMap/index.js:99:37)
[backend] info: [Event Logged] - update_llm_provider
[backend] info: EmbeddingEngine changed from native to azure - resetting lancedb namespaces
[backend] info: [Event Logged] - workspace_vectors_reset
[backend] info: Resetting anythingllm managed vector namespaces for lancedb
[backend] error: Namespace by that name does not exist.
[backend] info: [Event Logged] - update_embedding_engine
[backend] info: [AzureOpenAi] Initialized. Model "gpt-4o" @ 128000 tokens.
API-Version: 2024-12-01-preview.
Model Type: default
[backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-4o
[backend] error: Error: Connection error.
at AzureOpenAI.makeRequest (/app/server/node_modules/openai/core.js:332:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async AzureOpenAiLLM.streamGetChatCompletion (/app/server/utils/AiProviders/azureOpenAi/index.js:171:7)
at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:248:20)
at async /app/server/endpoints/chat.js:63:9

@yeldhomathew commented on GitHub (Aug 20, 2025): Thanks again. Still i have issue. so i have started using inbuilt vector DB (LanceDB) and tried with and without Proxy. below are the logs from Docker No pending migrations to apply. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo [backend] info: [ContextWindowFinder] Pulling remote model map... [backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM. [backend] info: prisma:info Starting a sqlite pool with 17 connections. [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"177dd959-c851-4f8b-abc9-be829af8c992","properties":{"commit":"--","runtime":"docker"}} [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: Primary server in HTTP mode listening on port 3001 [backend] info: [BackgroundWorkerService] Feature is not enabled and will not be started. [backend] info: [MetaGenerator] fetching custom meta tag settings... [backend] info: [ContextWindowFinder] Error syncing remote model map TypeError: fetch failed at node:internal/deps/undici/undici:12637:11 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async #pullRemoteModelMap (/app/server/utils/AiProviders/modelMap/index.js:99:37) [backend] info: [Event Logged] - update_llm_provider [backend] info: EmbeddingEngine changed from native to azure - resetting lancedb namespaces [backend] info: [Event Logged] - workspace_vectors_reset [backend] info: Resetting anythingllm managed vector namespaces for lancedb [backend] error: Namespace by that name does not exist. [backend] info: [Event Logged] - update_embedding_engine [backend] info: [AzureOpenAi] Initialized. Model "gpt-4o" @ 128000 tokens. API-Version: 2024-12-01-preview. Model Type: default [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-4o [backend] error: Error: Connection error. at AzureOpenAI.makeRequest (/app/server/node_modules/openai/core.js:332:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async AzureOpenAiLLM.streamGetChatCompletion (/app/server/utils/AiProviders/azureOpenAi/index.js:171:7) at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:248:20) at async /app/server/endpoints/chat.js:63:9
yindo commented 2026-02-22 18:31:04 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 21, 2025):

Error syncing remote model map TypeError: fetch failed

This fetch is to this totally unrestricted file https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json. Are you sure your Docker container can even reach the public internet? That fetch is a very basic HTTP request that should not fail for any reason

@timothycarambat commented on GitHub (Aug 21, 2025): > Error syncing remote model map TypeError: fetch failed This fetch is to this totally unrestricted file https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json. Are you sure your Docker container can even reach the public internet? That fetch is a very basic HTTP request that should not fail for any reason
yindo commented 2026-02-22 18:31:04 -05:00
Author
Owner
Copy Link
Copy Source

@yeldhomathew commented on GitHub (Aug 22, 2025):

Thanks @timothycarambat,
We have docker behind proxy. I think this URL is not whitelisted yet. I will do and recheck. kindly add if you have any more public URl which is mandatorily accessible to anythingLLm.

@yeldhomathew commented on GitHub (Aug 22, 2025): Thanks @timothycarambat, We have docker behind proxy. I think this URL is not whitelisted yet. I will do and recheck. kindly add if you have any more public URl which is mandatorily accessible to anythingLLm.
yindo commented 2026-02-22 18:31:04 -05:00
Author
Owner
Copy Link
Copy Source

@yeldhomathew commented on GitHub (Aug 24, 2025):

After Whitelisting URL https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json I am able to reach out the same from the container. But still i am facing same error. no difference in logs also.

[collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance.
[collector] info: Collector hot directory and tmp storage wiped!
[collector] info: Document processor app listening on port 8888
Environment variables loaded from .env
Prisma schema loaded from prisma/schema.prisma

✔ Generated Prisma Client (v5.3.1) to ./node_modules/@prisma/client in 418ms

Start using Prisma Client in Node.js (See: https://pris.ly/d/client)

import { PrismaClient } from '@prisma/client'
const prisma = new PrismaClient()

or start using Prisma Client at the edge (See: https://pris.ly/d/accelerate)

import { PrismaClient } from '@prisma/client/edge'
const prisma = new PrismaClient()

See other ways of importing Prisma Client: http://pris.ly/d/importing-client

npm notice
npm notice New major version of npm available! 10.8.2 -> 11.5.2
npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.5.2
npm notice To update run: npm install -g npm@11.5.2
npm notice
Environment variables loaded from .env
Prisma schema loaded from prisma/schema.prisma
Datasource "db": SQLite database "anythingllm.db" at "file:../storage/anythingllm.db"

32 migrations found in prisma/migrations

No pending migrations to apply.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo
[backend] info: [ContextWindowFinder] Pulling remote model map...
[backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo
[backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM.
[backend] info: prisma:info Starting a sqlite pool with 17 connections.
[backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"177dd959-c851-4f8b-abc9-be829af8c992","properties":{"commit":"--","runtime":"docker"}}
[backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[backend] info: Primary server in HTTP mode listening on port 3001
[backend] info: [BackgroundWorkerService] Feature is not enabled and will not be started.
[backend] info: [MetaGenerator] fetching custom meta tag settings...
[backend] info: [ContextWindowFinder] Error syncing remote model map TypeError: fetch failed
at node:internal/deps/undici/undici:12637:11
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async #pullRemoteModelMap (/app/server/utils/AiProviders/modelMap/index.js:99:37)
[backend] info: [AzureOpenAi] Initialized. Model "gpt-4o" @ 128000 tokens.
API-Version: 2024-12-01-preview.
Model Type: default
[backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-4o
[backend] error: Error: Connection error.
at AzureOpenAI.makeRequest (/app/server/node_modules/openai/core.js:332:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async AzureOpenAiLLM.streamGetChatCompletion (/app/server/utils/AiProviders/azureOpenAi/index.js:171:7)
at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:248:20)
at async /app/server/endpoints/chat.js:63:9

@yeldhomathew commented on GitHub (Aug 24, 2025): After Whitelisting URL https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json I am able to reach out the same from the container. But still i am facing same error. no difference in logs also. [collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance. [collector] info: Collector hot directory and tmp storage wiped! [collector] info: Document processor app listening on port 8888 Environment variables loaded from .env Prisma schema loaded from prisma/schema.prisma ✔ Generated Prisma Client (v5.3.1) to ./node_modules/@prisma/client in 418ms Start using Prisma Client in Node.js (See: https://pris.ly/d/client) ``` import { PrismaClient } from '@prisma/client' const prisma = new PrismaClient() ``` or start using Prisma Client at the edge (See: https://pris.ly/d/accelerate) ``` import { PrismaClient } from '@prisma/client/edge' const prisma = new PrismaClient() ``` See other ways of importing Prisma Client: http://pris.ly/d/importing-client npm notice npm notice New major version of npm available! 10.8.2 -> 11.5.2 npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.5.2 npm notice To update run: npm install -g npm@11.5.2 npm notice Environment variables loaded from .env Prisma schema loaded from prisma/schema.prisma Datasource "db": SQLite database "anythingllm.db" at "file:../storage/anythingllm.db" 32 migrations found in prisma/migrations No pending migrations to apply. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo [backend] info: [ContextWindowFinder] Pulling remote model map... [backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM. [backend] info: prisma:info Starting a sqlite pool with 17 connections. [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"177dd959-c851-4f8b-abc9-be829af8c992","properties":{"commit":"--","runtime":"docker"}} [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: Primary server in HTTP mode listening on port 3001 [backend] info: [BackgroundWorkerService] Feature is not enabled and will not be started. [backend] info: [MetaGenerator] fetching custom meta tag settings... [backend] info: [ContextWindowFinder] Error syncing remote model map TypeError: fetch failed at node:internal/deps/undici/undici:12637:11 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async #pullRemoteModelMap (/app/server/utils/AiProviders/modelMap/index.js:99:37) [backend] info: [AzureOpenAi] Initialized. Model "gpt-4o" @ 128000 tokens. API-Version: 2024-12-01-preview. Model Type: default [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-4o [backend] error: Error: Connection error. at AzureOpenAI.makeRequest (/app/server/node_modules/openai/core.js:332:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async AzureOpenAiLLM.streamGetChatCompletion (/app/server/utils/AiProviders/azureOpenAi/index.js:171:7) at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:248:20) at async /app/server/endpoints/chat.js:63:9
yindo changed title from Cannot configure Azure OpenAI with Anythingllm to [GH-ISSUE #4313] Cannot configure Azure OpenAI with Anythingllm 2026-06-05 14:48:18 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2743
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?