Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4400] [BUG]: Generic OpenAPI connection error #2803

New Issue
Closed
opened 2026-02-22 18:31:20 -05:00 by yindo · 4 comments
yindo commented 2026-02-22 18:31:20 -05:00
Owner
Copy Link
Copy Source

Originally created by @XelaNimed on GitHub (Sep 18, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4400

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

Cannot connect to local OpenAPI compliant FastFlowLM server. When submitting a question, an error is displayed:

Could not respond to message.
Connection error.

Are there known steps to reproduce?

  1. Download and Install FastFlowLM from here.
  2. Download the LLM model: flm pull <model_name>, for ex.: flm pull llama3.1:8b
  3. Run a local OpenAPI-compatible server: flm serve llama3.1:8b
  4. In the AnythingLLM settings, configure the connection in "All Providers" -> "LLM":
    • LLM Provider -> Generic OpenAPI
    • Base URL: http://127.0.0.1:1234 or http://127.0.0.1:1234/v1 (same error)
    • API Key: "flm" (any string)
    • Chat model name: llama3.1:8b (the name of the model selected when starting flm)
    • Token context window: 128000
    • Max Tokens: 2048
  5. In Workspace configure settings for chat:
    • "Workspace LLM Provider": "Generic OpenAPI"
    • "Workspace Chat model": "llama3.1:8b" (the name of the model selected when starting flm)

Log:

[OllamaProcessManager] Ollama will bind on port 52559 when booted.
[ManualBrowser] Listening for IPC messages from main renderer process!
[Preferences] Will load window with last know bounds.
[collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo
[collector] info: Collector hot directory and tmp storage wiped!
[collector] info: [production] AnythingLLM Standalone Document processor listening on port 8888.
[backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo
[backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM.
[backend] info: prisma:info Starting a sqlite pool with 25 connections.
[backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"07436a59-d6e2-414d-be69-9d7457ad65e4","properties":{}}
[backend] info: Skipping preloading of AnythingLLMOllama - LLM_PROVIDER is generic-openai.
[backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[backend] info: [production] AnythingLLM Standalone Backend listening on port 3001. Network discovery is disabled.
[backend] info: [BackgroundWorkerService] Starting...
[backend] info: [BackgroundWorkerService] Service started with 1 jobs ["cleanup-orphan-documents"]
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [Ni] Inference API: http://127.0.0.1:1234/v1 Model: llama3.1:8b
[backend] info: [fillSourceWindow] Need to backfill 4 chunks to fill in the source window for RAG!
[backend] info: [TokenManager] Initialized new TokenManager instance for model: llama3.1:8b
[backend] error: Error: Connection error.
    at OpenAI.makeRequest (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\node_modules\openai\core.js:332:19)
    at async Sa.measureStream (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:21:8903)
    at async Ni.streamGetChatCompletion (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:198:1753)
    at async DM (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:3125)
    at async C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:4762
prisma:info Starting a sqlite pool with 25 connections.
[backend] warn: Child process exited with code 0 and signal null
[backend] info: Worker for job "cleanup-orphan-documents" exited with code 0
[backend] info: [Event Logged] - update_llm_provider
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [Ni] Inference API: http://127.0.0.1:1234 Model: llama3.1:8b
[backend] info: [fillSourceWindow] Need to backfill 4 chunks to fill in the source window for RAG!
[backend] info: [TokenManager] Returning existing instance for model: llama3.1:8b
[backend] error: Error: Connection error.
    at OpenAI.makeRequest (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\node_modules\openai\core.js:332:19)
    at async Sa.measureStream (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:21:8903)
    at async Ni.streamGetChatCompletion (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:198:1753)
    at async DM (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:3125)
    at async C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:4762
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [Ni] Inference API: http://127.0.0.1:1234 Model: llama3.1:8b
[backend] info: [fillSourceWindow] Need to backfill 4 chunks to fill in the source window for RAG!
[backend] info: [TokenManager] Returning existing instance for model: llama3.1:8b
[backend] error: Error: Connection error.
    at OpenAI.makeRequest (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\node_modules\openai\core.js:332:19)
    at async Sa.measureStream (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:21:8903)
    at async Ni.streamGetChatCompletion (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:198:1753)
    at async DM (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:3125)
    at async C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:4762
Originally created by @XelaNimed on GitHub (Sep 18, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4400 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? Cannot connect to local OpenAPI compliant [FastFlowLM](https://github.com/FastFlowLM/FastFlowLM) server. When submitting a question, an error is displayed: ``` Could not respond to message. Connection error. ``` ### Are there known steps to reproduce? 1. Download and Install FastFlowLM from [here](https://github.com/FastFlowLM/FastFlowLM/releases/latest/download/flm-setup.exe). 2. Download the LLM model: `flm pull <model_name>`, for ex.: `flm pull llama3.1:8b` 3. Run a local OpenAPI-compatible server: `flm serve llama3.1:8b` 4. In the AnythingLLM settings, configure the connection in "All Providers" -> "LLM": - LLM Provider -> Generic OpenAPI - Base URL: http://127.0.0.1:1234 or http://127.0.0.1:1234/v1 (same error) - API Key: "flm" (any string) - Chat model name: llama3.1:8b (the name of the model selected when starting flm) - Token context window: 128000 - Max Tokens: 2048 5. In Workspace configure settings for chat: - "Workspace LLM Provider": "Generic OpenAPI" - "Workspace Chat model": "llama3.1:8b" (the name of the model selected when starting flm) Log: ``` [OllamaProcessManager] Ollama will bind on port 52559 when booted. [ManualBrowser] Listening for IPC messages from main renderer process! [Preferences] Will load window with last know bounds. [collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo [collector] info: Collector hot directory and tmp storage wiped! [collector] info: [production] AnythingLLM Standalone Document processor listening on port 8888. [backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM. [backend] info: prisma:info Starting a sqlite pool with 25 connections. [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"07436a59-d6e2-414d-be69-9d7457ad65e4","properties":{}} [backend] info: Skipping preloading of AnythingLLMOllama - LLM_PROVIDER is generic-openai. [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: [production] AnythingLLM Standalone Backend listening on port 3001. Network discovery is disabled. [backend] info: [BackgroundWorkerService] Starting... [backend] info: [BackgroundWorkerService] Service started with 1 jobs ["cleanup-orphan-documents"] [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [Ni] Inference API: http://127.0.0.1:1234/v1 Model: llama3.1:8b [backend] info: [fillSourceWindow] Need to backfill 4 chunks to fill in the source window for RAG! [backend] info: [TokenManager] Initialized new TokenManager instance for model: llama3.1:8b [backend] error: Error: Connection error. at OpenAI.makeRequest (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\node_modules\openai\core.js:332:19) at async Sa.measureStream (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:21:8903) at async Ni.streamGetChatCompletion (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:198:1753) at async DM (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:3125) at async C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:4762 prisma:info Starting a sqlite pool with 25 connections. [backend] warn: Child process exited with code 0 and signal null [backend] info: Worker for job "cleanup-orphan-documents" exited with code 0 [backend] info: [Event Logged] - update_llm_provider [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [Ni] Inference API: http://127.0.0.1:1234 Model: llama3.1:8b [backend] info: [fillSourceWindow] Need to backfill 4 chunks to fill in the source window for RAG! [backend] info: [TokenManager] Returning existing instance for model: llama3.1:8b [backend] error: Error: Connection error. at OpenAI.makeRequest (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\node_modules\openai\core.js:332:19) at async Sa.measureStream (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:21:8903) at async Ni.streamGetChatCompletion (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:198:1753) at async DM (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:3125) at async C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:4762 [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [Ni] Inference API: http://127.0.0.1:1234 Model: llama3.1:8b [backend] info: [fillSourceWindow] Need to backfill 4 chunks to fill in the source window for RAG! [backend] info: [TokenManager] Returning existing instance for model: llama3.1:8b [backend] error: Error: Connection error. at OpenAI.makeRequest (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\node_modules\openai\core.js:332:19) at async Sa.measureStream (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:21:8903) at async Ni.streamGetChatCompletion (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:198:1753) at async DM (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:3125) at async C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:4762 ```
yindo added the possible bug label 2026-02-22 18:31:20 -05:00
yindo closed this issue 2026-02-22 18:31:20 -05:00
yindo commented 2026-02-22 18:31:20 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Sep 18, 2025):

According to their docs here, it looks like the default port is 11434. It looks like you're using port 1234 for the base URL so try changing this to be the correct port and it should work if they are OpenAI compatible.

@shatfield4 commented on GitHub (Sep 18, 2025): According to their docs [here](https://docs.fastflowlm.com/instructions/server/), it looks like the default port is 11434. It looks like you're using port 1234 for the base URL so try changing this to be the correct port and it should work if they are OpenAI compatible.
yindo commented 2026-02-22 18:31:20 -05:00
Author
Owner
Copy Link
Copy Source

@XelaNimed commented on GitHub (Sep 19, 2025):

@shatfield4 I apologize - the wrong section of the log was selected.
Below is a correct example of the log:

[OllamaProcessManager] Ollama will bind on port 55007 when booted.
[ManualBrowser] Listening for IPC messages from main renderer process!
[Preferences] Will load window with last know bounds.
[collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo
[collector] info: Collector hot directory and tmp storage wiped!
[collector] info: [production] AnythingLLM Standalone Document processor listening on port 8888.
[backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo
[backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM.
[backend] info: prisma:info Starting a sqlite pool with 25 connections.
[backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"07436a59-d6e2-414d-be69-9d7457ad65e4","properties":{}}
[backend] info: Skipping preloading of AnythingLLMOllama - LLM_PROVIDER is generic-openai.
[backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[backend] info: [production] AnythingLLM Standalone Backend listening on port 3001. Network discovery is enabled.
[backend] info: [BackgroundWorkerService] Starting...
[backend] info: [BackgroundWorkerService] Service started with 1 jobs ["cleanup-orphan-documents"]
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [Ni] Inference API: http://127.0.0.1:11434/v1 Model: llama3.1:8b
[backend] info: [TokenManager] Initialized new TokenManager instance for model: llama3.1:8b
[backend] error: Error: Connection error.
    at OpenAI.makeRequest (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\node_modules\openai\core.js:332:19)
    at async Sa.measureStream (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:21:8903)
    at async Ni.streamGetChatCompletion (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:198:1753)
    at async DM (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:3125)
    at async C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:6077
prisma:info Starting a sqlite pool with 25 connections.
[backend] warn: Child process exited with code 0 and signal null
[backend] info: Worker for job "cleanup-orphan-documents" exited with code 0
@XelaNimed commented on GitHub (Sep 19, 2025): @shatfield4 I apologize - the wrong section of the log was selected. Below is a correct example of the log: ``` [OllamaProcessManager] Ollama will bind on port 55007 when booted. [ManualBrowser] Listening for IPC messages from main renderer process! [Preferences] Will load window with last know bounds. [collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo [collector] info: Collector hot directory and tmp storage wiped! [collector] info: [production] AnythingLLM Standalone Document processor listening on port 8888. [backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM. [backend] info: prisma:info Starting a sqlite pool with 25 connections. [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"07436a59-d6e2-414d-be69-9d7457ad65e4","properties":{}} [backend] info: Skipping preloading of AnythingLLMOllama - LLM_PROVIDER is generic-openai. [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: [production] AnythingLLM Standalone Backend listening on port 3001. Network discovery is enabled. [backend] info: [BackgroundWorkerService] Starting... [backend] info: [BackgroundWorkerService] Service started with 1 jobs ["cleanup-orphan-documents"] [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [Ni] Inference API: http://127.0.0.1:11434/v1 Model: llama3.1:8b [backend] info: [TokenManager] Initialized new TokenManager instance for model: llama3.1:8b [backend] error: Error: Connection error. at OpenAI.makeRequest (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\node_modules\openai\core.js:332:19) at async Sa.measureStream (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:21:8903) at async Ni.streamGetChatCompletion (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:198:1753) at async DM (C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:3125) at async C:\Users\john.doe\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:283:6077 prisma:info Starting a sqlite pool with 25 connections. [backend] warn: Child process exited with code 0 and signal null [backend] info: Worker for job "cleanup-orphan-documents" exited with code 0 ```
yindo commented 2026-02-22 18:31:20 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Sep 19, 2025):

This has to be your flm service. You have flm serve llama3.1:8b running - so in theory you should be able to ping http://127.0.0.1:1234/v1/models in your browser to ensure the model server actually booted and successfully loaded a model into memory so it can run inference. It seems like either the server is starting and failing or something else?

Does the flm service have logs that can be shown - our logs just show that this service cannot be reached. Why this is the case is only discoverable within the service itself

@timothycarambat commented on GitHub (Sep 19, 2025): This has to be your `flm` service. You have `flm serve llama3.1:8b` running - so in theory you should be able to ping `http://127.0.0.1:1234/v1/models` in your browser to ensure the model server actually booted and successfully loaded a model into memory so it can run inference. It seems like either the server is starting and failing or something else? Does the flm service have logs that can be shown - our logs just show that this service cannot be reached. Why this is the case is only discoverable within the service itself
yindo commented 2026-02-22 18:31:20 -05:00
Author
Owner
Copy Link
Copy Source

@XelaNimed commented on GitHub (Sep 22, 2025):

@timothycarambat GET request http://127.0.0.1:11434/v1/models (pay attention to the port number) returns:

{
    "data": [
        {
            "created": 1758525535,
            "id": "deepseek-r1:8b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "deepseek-r1-0528:8b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "gemma3:1b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "gemma3:270m",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "gemma3:4b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "llama3.1:8b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "llama3.2:1b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "llama3.2:3b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "medgemma:4b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "qwen3:0.6b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "qwen3:1.7b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "qwen3:4b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "qwen3:8b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "qwen3-it:4b",
            "object": "model",
            "owned_by": "organization-owner"
        },
        {
            "created": 1758525535,
            "id": "qwen3-tk:4b",
            "object": "model",
            "owned_by": "organization-owner"
        }
    ],
    "object": "list"
}

But to view the loaded models I used the flm list command which returns the following:

PS C:\Users\john.doe> flm list
Models:
  - deepseek-r1:8b ✅
  - deepseek-r1-0528:8b ✅
  - gemma3:1b ⏬
  - gemma3:270m ⏬
  - gemma3:4b ⏬
  - llama3.1:8b ✅
  - llama3.2:1b ⏬
  - llama3.2:3b ⏬
  - medgemma:4b ⏬
  - qwen3:0.6b ⏬
  - qwen3:1.7b ⏬
  - qwen3:4b ⏬
  - qwen3:8b ✅
  - qwen3-it:4b ✅
  - qwen3-tk:4b ✅

The model is downloadable, accessible, and usable without any problems, for example in conjunction with LibreChat.
The flm logs only show incoming and outgoing requests and contain nothing else, since the connection is not established and there is nothing for flm to log. Below is a log of flm interaction with LibreChat:

PS C:\Users\john.doe> flm serve llama3.1:8b
[FLM]  Configuring NPU Power Mode to performance (flm default)
[FLM]  Loading model: C:\Users\john.doe\flm\models\Llama-3.1-8B-NPU2
[FLM]  Starting server on port 11434...
[LOG]  WebServer started on port 11434 with 5 I/O threads
[FLM]  Enter 'exit' to stop the server:
[🔗 ]  TCP connection established - Remote: 127.0.0.1:50442
[TCP]  Read 1392 bytes from socket
================================================
[⬇️ ]  Incoming Request: POST
[LOG]  Time stamp: 12:31:59 09:22:2025
[LOG]  Target: /v1/chat/completions
[LOG]  Version: 11
[LOG]  Keep-Alive: 1
[LOG]  Body:
{
    "messages": [
        {
            "content": "Hi, how are you?",
            "role": "user"
        }
    ],
    "model": "llama3.1:8b",
    "stream": true,
    "user": "68b009f4ad09e68746f6ba56"
}
[🟢 ]  NPU access granted for request: POST /v1/chat/completions
[FLM]  Start generating...
[🔗 ]  Keeping TCP connection alive for next request (streaming)
[🔵 ]  NPU access released for streaming request: req_0
[🔒 ]  TCP connection closed - Remote: 127.0.0.1:50442
@XelaNimed commented on GitHub (Sep 22, 2025): @timothycarambat GET request `http://127.0.0.1:11434/v1/models` (pay attention to the port number) returns: ```json { "data": [ { "created": 1758525535, "id": "deepseek-r1:8b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "deepseek-r1-0528:8b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "gemma3:1b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "gemma3:270m", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "gemma3:4b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "llama3.1:8b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "llama3.2:1b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "llama3.2:3b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "medgemma:4b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "qwen3:0.6b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "qwen3:1.7b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "qwen3:4b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "qwen3:8b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "qwen3-it:4b", "object": "model", "owned_by": "organization-owner" }, { "created": 1758525535, "id": "qwen3-tk:4b", "object": "model", "owned_by": "organization-owner" } ], "object": "list" } ``` But to view the loaded models I used the `flm list` command which returns the following: ``` PS C:\Users\john.doe> flm list Models: - deepseek-r1:8b ✅ - deepseek-r1-0528:8b ✅ - gemma3:1b ⏬ - gemma3:270m ⏬ - gemma3:4b ⏬ - llama3.1:8b ✅ - llama3.2:1b ⏬ - llama3.2:3b ⏬ - medgemma:4b ⏬ - qwen3:0.6b ⏬ - qwen3:1.7b ⏬ - qwen3:4b ⏬ - qwen3:8b ✅ - qwen3-it:4b ✅ - qwen3-tk:4b ✅ ``` The model is downloadable, accessible, and usable without any problems, for example in conjunction with LibreChat. The flm logs only show incoming and outgoing requests and contain nothing else, since the connection is not established and there is nothing for flm to log. Below is a log of flm interaction with LibreChat: ``` PS C:\Users\john.doe> flm serve llama3.1:8b [FLM] Configuring NPU Power Mode to performance (flm default) [FLM] Loading model: C:\Users\john.doe\flm\models\Llama-3.1-8B-NPU2 [FLM] Starting server on port 11434... [LOG] WebServer started on port 11434 with 5 I/O threads [FLM] Enter 'exit' to stop the server: [🔗 ] TCP connection established - Remote: 127.0.0.1:50442 [TCP] Read 1392 bytes from socket ================================================ [⬇️ ] Incoming Request: POST [LOG] Time stamp: 12:31:59 09:22:2025 [LOG] Target: /v1/chat/completions [LOG] Version: 11 [LOG] Keep-Alive: 1 [LOG] Body: { "messages": [ { "content": "Hi, how are you?", "role": "user" } ], "model": "llama3.1:8b", "stream": true, "user": "68b009f4ad09e68746f6ba56" } [🟢 ] NPU access granted for request: POST /v1/chat/completions [FLM] Start generating... [🔗 ] Keeping TCP connection alive for next request (streaming) [🔵 ] NPU access released for streaming request: req_0 [🔒 ] TCP connection closed - Remote: 127.0.0.1:50442 ```
yindo changed title from [BUG]: Generic OpenAPI connection error to [GH-ISSUE #4400] [BUG]: Generic OpenAPI connection error 2026-06-05 14:48:38 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2803
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?