Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5129] [BUG]: QNN Engine fails on Snapdragon X Plus (X1P42100) due to unrecognized SoC in cpuinfo (Windows ARM64) #4933

New Issue
Closed
opened 2026-06-05 14:50:57 -04:00 by yindo · 9 comments
yindo commented 2026-06-05 14:50:57 -04:00
Owner
Copy Link
Copy Source

Originally created by @IsaacTenorioLondono on GitHub (Mar 3, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5129

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

When running AnythingLLM on a Windows ARM64 device powered by a Snapdragon X Plus (X1P42100, Oryon CPU), the QNN Engine fails to initialize and remains offline.

Although:

  • The system is Windows ARM64
  • The NPU is correctly detected
  • QNN SDK loads successfully
  • QnnHtp.dll initializes
  • GenieProfile binds successfully

The engine ultimately fails with the following error:

Error in cpuinfo: Unknown chip model name 'Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU'.
Please add new Windows on Arm SoC/chip support to arm/windows/init.c!

After this, the backend repeatedly logs:

QNNEngine offline - retrying.
Error: QNN Engine is offline. Please reboot QNN Engine or AnythingLLM app.

The issue appears to stem from the bundled cpuinfo library not recognizing the Snapdragon X Plus (X1P42100) SoC, causing the QNN engine initialization to abort even though the SDK and NPU are functioning properly.

Environment

  • OS: Windows 11 ARM64
  • CPU: Snapdragon X Plus (X1P42100, Oryon)
  • Architecture: ARM64 (confirmed via PE header: 43620)
  • AnythingLLM: Desktop ARM64 build
  • QNN SDK: 2.42.0 (as reported in logs)

Observed Behavior

From logs:


[QnnProcessManager] Checking core Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU - MATCH
NPU Detected: true OS: windows (arm64)

SDK: 2.42.0.251225
DSP Targets: v81,v73
GenieProfile bound successfully - accurate metrics enabled.

Error in cpuinfo: Unknown chip model name 'Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU'.
Please add new Windows on Arm SoC/chip support to arm/windows/init.c!

The QNN API server boots but never transitions to ready state, causing the backend to report the engine as offline.

I watched the YouTube video "https://www.youtube.com/watch?v=iQvHtubnfcI&t=293s" and everything works perfectly in that video, but it’s because they are using the Snapdragon X Elite processor. I have the Snapdragon X Plus, which has two versions: one with 10 cores and another with 8. Both supposedly have the same NPU according to my research on the Qualcomm website, but still, AnythingLLM doesn't seem to recognize my processor.

Please note that the processor reference used in the video is the Snapdragon X Elite X1E80100, and the one I’m using is the Snapdragon X Plus X1P42100

Image Image Image Image Image Image Image

Are there known steps to reproduce?

Steps to Reproduce

  • Install AnythingLLM ARM64 on a Windows 11 ARM device with Snapdragon X Plus (X1P42100).
  • Enable QNN Engine as LLM provider.
  • Start the application.
  • Observe logs during QNN initialization.
  • The engine fails with “Unknown chip model name” error and remains offline.
Originally created by @IsaacTenorioLondono on GitHub (Mar 3, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5129 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? When running AnythingLLM on a Windows ARM64 device powered by a Snapdragon X Plus (X1P42100, Oryon CPU), the QNN Engine fails to initialize and remains offline. Although: - The system is Windows ARM64 - The NPU is correctly detected - QNN SDK loads successfully - QnnHtp.dll initializes - GenieProfile binds successfully The engine ultimately fails with the following error: ``` Error in cpuinfo: Unknown chip model name 'Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU'. Please add new Windows on Arm SoC/chip support to arm/windows/init.c! ``` After this, the backend repeatedly logs: ``` QNNEngine offline - retrying. Error: QNN Engine is offline. Please reboot QNN Engine or AnythingLLM app. ``` The issue appears to stem from the bundled cpuinfo library not recognizing the Snapdragon X Plus (X1P42100) SoC, causing the QNN engine initialization to abort even though the SDK and NPU are functioning properly. ### Environment - OS: Windows 11 ARM64 - CPU: Snapdragon X Plus (X1P42100, Oryon) - Architecture: ARM64 (confirmed via PE header: 43620) - AnythingLLM: Desktop ARM64 build - QNN SDK: 2.42.0 (as reported in logs) ### Observed Behavior From logs: ``` [QnnProcessManager] Checking core Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU - MATCH NPU Detected: true OS: windows (arm64) SDK: 2.42.0.251225 DSP Targets: v81,v73 GenieProfile bound successfully - accurate metrics enabled. Error in cpuinfo: Unknown chip model name 'Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU'. Please add new Windows on Arm SoC/chip support to arm/windows/init.c! ``` The QNN API server boots but never transitions to ready state, causing the backend to report the engine as offline. I watched the YouTube video "[https://www.youtube.com/watch?v=iQvHtubnfcI&t=293s](https://www.youtube.com/watch?v=iQvHtubnfcI&t=293s)" and everything works perfectly in that video, but it’s because they are using the Snapdragon X Elite processor. I have the Snapdragon X Plus, which has two versions: one with 10 cores and another with 8. Both supposedly have the same NPU according to my research on the Qualcomm website, but still, AnythingLLM doesn't seem to recognize my processor. Please note that the processor reference used in the video is the Snapdragon X Elite X1E80100, and the one I’m using is the Snapdragon X Plus X1P42100 <img width="960" height="564" alt="Image" src="https://github.com/user-attachments/assets/00536df5-5095-4e04-ad0a-78bfebb6d42e" /> <img width="956" height="561" alt="Image" src="https://github.com/user-attachments/assets/1edce899-50db-4c06-94f8-76f5f308c8c9" /> <img width="960" height="564" alt="Image" src="https://github.com/user-attachments/assets/4f3c324e-e9e7-4da3-b816-e003f78ce359" /> <img width="960" height="564" alt="Image" src="https://github.com/user-attachments/assets/06138f58-7a9f-4b8c-84d2-85fab537adc5" /> <img width="960" height="564" alt="Image" src="https://github.com/user-attachments/assets/e9382ff4-8bad-4a77-beed-88a900812847" /> <img width="960" height="564" alt="Image" src="https://github.com/user-attachments/assets/23c632a2-bda5-47fd-b5d2-2c496060ec45" /> <img width="960" height="562" alt="Image" src="https://github.com/user-attachments/assets/de017923-111c-40b3-b826-3ab1787149b1" /> ### Are there known steps to reproduce? ### Steps to Reproduce - Install AnythingLLM ARM64 on a Windows 11 ARM device with Snapdragon X Plus (X1P42100). - Enable QNN Engine as LLM provider. - Start the application. - Observe logs during QNN initialization. - The engine fails with “Unknown chip model name” error and remains offline.
yindo added the possible buginvestigating labels 2026-06-05 14:50:57 -04:00
yindo closed this issue 2026-06-05 14:50:57 -04:00
yindo commented 2026-06-05 14:50:58 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 4, 2026):

The Unknown chip model name is innocuous here and can be ignored - it always shows for any CPU. The earlier logs in your screenshot show MATCH so it is still trying to run the service anyway.

The fact the NPU based embedder in the 2nd terminal screenshot works but the underlying engine itself is not launching indicates that this could be a malloc issue with the model binaries themselves, since the embedder is very small for memory but the LLM is comparably massive.

Are you/have you tried the llama 3.2 3B 8K model? The phi model is slightly larger with a smaller context, but the binaries require more memory space on the NPU and with the 8-core model it could be the root issue. Discussion staying open and i'm subscribed to this issue so I will keep investigating with you - just closing the issue while under investigation.

<!-- gh-comment-id:3998595219 --> @timothycarambat commented on GitHub (Mar 4, 2026): The `Unknown chip model name` is innocuous here and can be ignored - it always shows for any CPU. The earlier logs in your screenshot show `MATCH` so it is still trying to run the service anyway. The fact the NPU based embedder in the 2nd terminal screenshot works but the underlying engine itself is not launching indicates that this could be a `malloc` issue with the model binaries themselves, since the embedder is _very_ small for memory but the LLM is comparably massive. Are you/have you tried the llama 3.2 3B 8K model? The phi model is slightly larger with a smaller context, but the binaries require more memory space on the NPU and with the 8-core model it could be the root issue. Discussion staying open and i'm subscribed to this issue so I will keep investigating with you - just closing the issue while under investigation.
yindo commented 2026-06-05 14:50:58 -04:00
Author
Owner
Copy Link
Copy Source

@IsaacTenorioLondono commented on GitHub (Mar 4, 2026):

Thank you for analyzing everything I sent: I ran the application again and captured the full output.

I am using the exact model you recommended:

llama_v3_2_3b_chat_8k

The logs show that the QNN engine initializes correctly with that model:

[QNN Engine] Initialized with model: llama_v3_2_3b_chat_8k. Context window: 8192

However, shortly after startup, the following error appears:

Error in cpuinfo: Unknown chip model name 'Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU'.
Please add new Windows on Arm SoC/chip support to arm/windows/init.c!

After that, the backend retries 5 times and eventually throws:

Error: QNN Engine is offline.

It seems the Snapdragon X Plus (X1P42100) is detected, but not recognized by the cpuinfo layer used by QNN.

Environment:

  • OS: Windows ARM64
  • CPU: Snapdragon X Plus (Oryon)
  • LLM_PROVIDER=qnnengine
  • Model: llama_v3_2_3b_chat_8k

Let me know if you need additional logs or testing.

<!-- gh-comment-id:3999056861 --> @IsaacTenorioLondono commented on GitHub (Mar 4, 2026): Thank you for analyzing everything I sent: I ran the application again and captured the full output. I am using the exact model you recommended: `llama_v3_2_3b_chat_8k` The logs show that the QNN engine initializes correctly with that model: `[QNN Engine] Initialized with model: llama_v3_2_3b_chat_8k. Context window: 8192` However, shortly after startup, the following error appears: ``` Error in cpuinfo: Unknown chip model name 'Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU'. Please add new Windows on Arm SoC/chip support to arm/windows/init.c! ``` After that, the backend retries 5 times and eventually throws: `Error: QNN Engine is offline.` It seems the Snapdragon X Plus (X1P42100) is detected, but not recognized by the cpuinfo layer used by QNN. ### Environment: - OS: Windows ARM64 - CPU: Snapdragon X Plus (Oryon) - LLM_PROVIDER=qnnengine - Model: llama_v3_2_3b_chat_8k Let me know if you need additional logs or testing.
yindo commented 2026-06-05 14:50:58 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 4, 2026):

Is it possible to post the full log so i can see everyline around the QNN startup? Are you comfortable with Powershell? I can give you some CLI commands to run to help trace the issue with a much more definitive crash

<!-- gh-comment-id:4000552640 --> @timothycarambat commented on GitHub (Mar 4, 2026): Is it possible to post the full log so i can see everyline around the QNN startup? Are you comfortable with Powershell? I can give you some CLI commands to run to help trace the issue with a much more definitive crash
yindo commented 2026-06-05 14:50:58 -04:00
Author
Owner
Copy Link
Copy Source

@IsaacTenorioLondono commented on GitHub (Mar 4, 2026):


Windows PowerShell
Copyright (C) Microsoft Corporation. Todos los derechos reservados.

Instale la versión más reciente de PowerShell para obtener nuevas características y mejoras. https://aka.ms/PSWindows

PS C:\Users\Isaac Tenorio> Start-Process "C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\AnythingLLM.exe"
PS C:\Users\Isaac Tenorio>
[Preferences] preference config stored at C:\Users\Isaac Tenorio\AppData\Roaming\anythingllm-desktop\config.json
[PrismaHelper: Generic] Setting Prisma Client output directory default. {
  clientOutputPath: 'C:\\Users\\Isaac Tenorio\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\node_modules\\.prisma\\client'
}
Prisma schema loaded from prisma\schema.prisma
Datasource "db": SQLite database "anythingllm.db" at "file:C:/Users/Isaac Tenorio/AppData/Roaming/anythingllm-desktop/storage/anythingllm.db"

Already in sync, no schema change or pending migration was found.

✔ Generated Prisma Client (v5.3.1, engine=binary) to .\node_modules\.prisma\clie
nt in 251ms


Prisma schema loaded from prisma\schema.prisma

✔ Generated Prisma Client (v5.3.1, engine=binary) to .\node_modules\.prisma\client in 3.15s

Start using Prisma Client in Node.js (See: https://pris.ly/d/client)

import { PrismaClient } from './node_modules/.prisma/client'
const prisma = new PrismaClient()

or start using Prisma Client at the edge (See: https://pris.ly/d/accelerate)

import { PrismaClient } from './node_modules/.prisma/client/edge'
const prisma = new PrismaClient()


See other ways of importing Prisma Client: http://pris.ly/d/importing-client

[QnnProcessManager] Checking for valid NPU cores.
[QnnProcessManager] Checking core Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU - MATCH
[QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager.
[QnnProcessManager] Checking for valid NPU cores.
[QnnProcessManager] Checking core Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU - MATCH
[OllamaProcessManager] Ollama will bind on port 11434 when booted.
[QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager.
[ManualBrowser] Listening for IPC messages from main renderer process!
[MeetingAssistantHelper.DownloadHelper] DownloadHelper initialized.
[MeetingAssistantHelper] Initialized & Registered
[meeting-assistant] info: Initialized & Registered
[FFmpegHelper.Bootstrap] Bootstrap is not supported on this platform.
[FFmpegHelper] Bootstrap is not supported on this platform.
[ffmpeg] info: Bootstrap is not supported on this platform.
[Preferences] Startup window preference: main
[Preferences] Will load window with last known bounds.
[TrayManager] System tray initialized
[MeetingAssistantHelper] FFMPEG bootstrapped successfully!
[meeting-assistant] info: FFMPEG bootstrapped successfully!
[18764:0304/170934.284:ERROR:ffmpeg_common.cc(959)] Unsupported pixel format: -1
[collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[collector] info: Collector hot directory and tmp storage wiped!
[collector] info: [production] AnythingLLM Standalone Document processor listening on port 8888.
[backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo
[backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo
[backend] info: prisma:info Starting a sqlite pool with 17 connections.
[backend] info: prisma:info Started query engine http server on http://127.0.0.1:64922
[backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM.
[backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"18284cb0-849f-4233-8d76-8ce93f28ee6b","properties":{}}
[backend] info: Skipping preloading of AnythingLLMOllama - LLM_PROVIDER is qnnengine.
[backend] info: Hot loading of QnnEngine - LLM_PROVIDER is qnnengine with model llama_v3_2_3b_chat_8k.
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [QNN Engine] Initialized with model: llama_v3_2_3b_chat_8k. Context window: 8192
[backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[backend] info: [production] AnythingLLM Standalone Backend listening on port 3001. Network discovery is disabled. NPU Detected: true OS: windows (arm64)
[QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager.
[QnnProcessManager] Checking for valid NPU cores.
[QnnProcessManager] Checking core Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU - MATCH
[QnnProcessManager] Booting QNN API server with model llama_v3_2_3b_chat_8k on port 3800.
[backend] info: [BackgroundWorkerService] Starting...
[backend] info: [BackgroundWorkerService] Service started with 3 jobs ["cleanup-orphan-documents","cleanup-orphan-recordings","cleanup-orphan-assistant-chats"]
[QnnProcessManager] [windows] QNN API server running. Port 3800. PID 17040.
[backend] info: QNNEngine offline - retrying. 1/5
[QnnProcessManager] [debug] stdout AnythingLLMQnnEngine API Server: Starting chat API on host 127.0.0.1:3800
SDK: 2.42.0.251225 DSP Targets: v81,v73
Build: 1.0.4 c022fb5 Fri Jan 16 11:00:32 2026

[QnnProcessManager] [debug] stdout GenieProfile bound successfully - accurate metrics enabled.

[backend] info: QNNEngine offline - retrying. 2/5
[backend] info: QNNEngine offline - retrying. 3/5
[backend] info: QNNEngine offline - retrying. 4/5
[backend] info: QNNEngine offline - retrying. 5/5
[backend] info: [2780:7940]: No meeting assistant storage path found - exiting.
[bg-worker][cleanup-orphan-recordings] info: [2780:7940]: No meeting assistant storage path found - exiting.
prisma:info Starting a sqlite pool with 17 connections.
prisma:info Started query engine http server on http://127.0.0.1:64961
[backend] warn: Child process exited with code 0 and signal null
[backend] info: Worker for job "cleanup-orphan-recordings" exited with code 0
[backend] warn: Child process exited with code 0 and signal null
[backend] info: Worker for job "cleanup-orphan-documents" exited with code 0
[backend] info: [TELEMETRY SENT] {"event":"workspace_thread_created","distinctId":"18284cb0-849f-4233-8d76-8ce93f28ee6b","properties":{"multiUserMode":false,"LLMSelection":"qnnengine","Embedder":"qnn-native","VectorDbSelection":"lancedb","TTSSelection":"native","LLMModel":"C:\\Users\\Isaac Tenorio\\AppData\\Roaming\\anythingllm-desktop\\storage\\models\\QNN\\llama_v3_2_3b_chat_8k"}}
[backend] info: [Event Logged] - workspace_thread_created
[backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll
[backend] info: [QnnNativeEmbedder] Checking for valid NPU cores.
[backend] info: [QnnNativeEmbedder] Checking core Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU - MATCH
[backend] info: [QnnNativeEmbedder] Initialized all-minilm-l6-v2 model
[backend] info: [QNN Engine] Initialized with model: llama_v3_2_3b_chat_8k. Context window: 8192
[backend] info: [QnnNativeEmbedder] Embedding 1 chunks
[backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll
Runtime options:
backend_path: C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll
soc_model: 60
htp_arch: 73
htp_performance_mode: high_performance
enable_htp_fp16_precision: 1
Error in cpuinfo: Unknown chip model name 'Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU'.
Please add new Windows on Arm SoC/chip support to arm/windows/init.c!
[backend] info: [QnnNativeEmbedder] Got 1 embeddings that are 384 long
[backend] info: [TokenManager] Initialized new TokenManager instance for model: C:\Users\Isaac Tenorio\AppData\Roaming\anythingllm-desktop\storage\models\QNN\llama_v3_2_3b_chat_8k
[QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager.
[QnnProcessManager] QNN API server already running. Killing and rebooting
[QnnProcessManager] QNN API server process 17040 killed.
[QnnProcessManager] Checking for valid NPU cores.
[QnnProcessManager] Checking core Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU - MATCH
[QnnProcessManager] Booting QNN API server with model llama_v3_2_3b_chat_8k on port 3800.
[QnnProcessManager] [windows] QNN API server running. Port 3800. PID 10916.
[backend] info: QNNEngine offline - retrying. 1/5
[QnnProcessManager] [debug] stdout AnythingLLMQnnEngine API Server: Starting chat API on host 127.0.0.1:3800
SDK: 2.42.0.251225 DSP Targets: v81,v73
Build: 1.0.4 c022fb5 Fri Jan 16 11:00:32 2026

[QnnProcessManager] [debug] stdout GenieProfile bound successfully - accurate metrics enabled.

[backend] info: QNNEngine offline - retrying. 2/5
[backend] info: QNNEngine offline - retrying. 3/5
[backend] error: Error: QNN Engine is offline. Please reboot QNN Engine or AnythingLLM app.
    at s.checkReady (C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:31:2420)
    at async s.streamGetChatCompletion (C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:31:3595)
    at async W$ (C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:316:3134)
    at async C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:316:6121
[backend] info: QNNEngine offline - retrying. 4/5
[backend] info: QNNEngine offline - retrying. 5/5

Yes, I’m comfortable using PowerShell.
I can run whatever CLI commands you provide.

I’ve also attached the full log above including the entire QNN startup sequence.

Let me know the commands you’d like me to execute and I’ll report back with the output.

<!-- gh-comment-id:4000643538 --> @IsaacTenorioLondono commented on GitHub (Mar 4, 2026): ``` Windows PowerShell Copyright (C) Microsoft Corporation. Todos los derechos reservados. Instale la versión más reciente de PowerShell para obtener nuevas características y mejoras. https://aka.ms/PSWindows PS C:\Users\Isaac Tenorio> Start-Process "C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\AnythingLLM.exe" PS C:\Users\Isaac Tenorio> [Preferences] preference config stored at C:\Users\Isaac Tenorio\AppData\Roaming\anythingllm-desktop\config.json [PrismaHelper: Generic] Setting Prisma Client output directory default. { clientOutputPath: 'C:\\Users\\Isaac Tenorio\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\node_modules\\.prisma\\client' } Prisma schema loaded from prisma\schema.prisma Datasource "db": SQLite database "anythingllm.db" at "file:C:/Users/Isaac Tenorio/AppData/Roaming/anythingllm-desktop/storage/anythingllm.db" Already in sync, no schema change or pending migration was found. ✔ Generated Prisma Client (v5.3.1, engine=binary) to .\node_modules\.prisma\clie nt in 251ms Prisma schema loaded from prisma\schema.prisma ✔ Generated Prisma Client (v5.3.1, engine=binary) to .\node_modules\.prisma\client in 3.15s Start using Prisma Client in Node.js (See: https://pris.ly/d/client) import { PrismaClient } from './node_modules/.prisma/client' const prisma = new PrismaClient() or start using Prisma Client at the edge (See: https://pris.ly/d/accelerate) import { PrismaClient } from './node_modules/.prisma/client/edge' const prisma = new PrismaClient() See other ways of importing Prisma Client: http://pris.ly/d/importing-client [QnnProcessManager] Checking for valid NPU cores. [QnnProcessManager] Checking core Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU - MATCH [QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager. [QnnProcessManager] Checking for valid NPU cores. [QnnProcessManager] Checking core Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU - MATCH [OllamaProcessManager] Ollama will bind on port 11434 when booted. [QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager. [ManualBrowser] Listening for IPC messages from main renderer process! [MeetingAssistantHelper.DownloadHelper] DownloadHelper initialized. [MeetingAssistantHelper] Initialized & Registered [meeting-assistant] info: Initialized & Registered [FFmpegHelper.Bootstrap] Bootstrap is not supported on this platform. [FFmpegHelper] Bootstrap is not supported on this platform. [ffmpeg] info: Bootstrap is not supported on this platform. [Preferences] Startup window preference: main [Preferences] Will load window with last known bounds. [TrayManager] System tray initialized [MeetingAssistantHelper] FFMPEG bootstrapped successfully! [meeting-assistant] info: FFMPEG bootstrapped successfully! [18764:0304/170934.284:ERROR:ffmpeg_common.cc(959)] Unsupported pixel format: -1 [collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [collector] info: Collector hot directory and tmp storage wiped! [collector] info: [production] AnythingLLM Standalone Document processor listening on port 8888. [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo [backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo [backend] info: prisma:info Starting a sqlite pool with 17 connections. [backend] info: prisma:info Started query engine http server on http://127.0.0.1:64922 [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM. [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"18284cb0-849f-4233-8d76-8ce93f28ee6b","properties":{}} [backend] info: Skipping preloading of AnythingLLMOllama - LLM_PROVIDER is qnnengine. [backend] info: Hot loading of QnnEngine - LLM_PROVIDER is qnnengine with model llama_v3_2_3b_chat_8k. [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [QNN Engine] Initialized with model: llama_v3_2_3b_chat_8k. Context window: 8192 [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: [production] AnythingLLM Standalone Backend listening on port 3001. Network discovery is disabled. NPU Detected: true OS: windows (arm64) [QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager. [QnnProcessManager] Checking for valid NPU cores. [QnnProcessManager] Checking core Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU - MATCH [QnnProcessManager] Booting QNN API server with model llama_v3_2_3b_chat_8k on port 3800. [backend] info: [BackgroundWorkerService] Starting... [backend] info: [BackgroundWorkerService] Service started with 3 jobs ["cleanup-orphan-documents","cleanup-orphan-recordings","cleanup-orphan-assistant-chats"] [QnnProcessManager] [windows] QNN API server running. Port 3800. PID 17040. [backend] info: QNNEngine offline - retrying. 1/5 [QnnProcessManager] [debug] stdout AnythingLLMQnnEngine API Server: Starting chat API on host 127.0.0.1:3800 SDK: 2.42.0.251225 DSP Targets: v81,v73 Build: 1.0.4 c022fb5 Fri Jan 16 11:00:32 2026 [QnnProcessManager] [debug] stdout GenieProfile bound successfully - accurate metrics enabled. [backend] info: QNNEngine offline - retrying. 2/5 [backend] info: QNNEngine offline - retrying. 3/5 [backend] info: QNNEngine offline - retrying. 4/5 [backend] info: QNNEngine offline - retrying. 5/5 [backend] info: [2780:7940]: No meeting assistant storage path found - exiting. [bg-worker][cleanup-orphan-recordings] info: [2780:7940]: No meeting assistant storage path found - exiting. prisma:info Starting a sqlite pool with 17 connections. prisma:info Started query engine http server on http://127.0.0.1:64961 [backend] warn: Child process exited with code 0 and signal null [backend] info: Worker for job "cleanup-orphan-recordings" exited with code 0 [backend] warn: Child process exited with code 0 and signal null [backend] info: Worker for job "cleanup-orphan-documents" exited with code 0 [backend] info: [TELEMETRY SENT] {"event":"workspace_thread_created","distinctId":"18284cb0-849f-4233-8d76-8ce93f28ee6b","properties":{"multiUserMode":false,"LLMSelection":"qnnengine","Embedder":"qnn-native","VectorDbSelection":"lancedb","TTSSelection":"native","LLMModel":"C:\\Users\\Isaac Tenorio\\AppData\\Roaming\\anythingllm-desktop\\storage\\models\\QNN\\llama_v3_2_3b_chat_8k"}} [backend] info: [Event Logged] - workspace_thread_created [backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll [backend] info: [QnnNativeEmbedder] Checking for valid NPU cores. [backend] info: [QnnNativeEmbedder] Checking core Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU - MATCH [backend] info: [QnnNativeEmbedder] Initialized all-minilm-l6-v2 model [backend] info: [QNN Engine] Initialized with model: llama_v3_2_3b_chat_8k. Context window: 8192 [backend] info: [QnnNativeEmbedder] Embedding 1 chunks [backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll Runtime options: backend_path: C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll soc_model: 60 htp_arch: 73 htp_performance_mode: high_performance enable_htp_fp16_precision: 1 Error in cpuinfo: Unknown chip model name 'Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU'. Please add new Windows on Arm SoC/chip support to arm/windows/init.c! [backend] info: [QnnNativeEmbedder] Got 1 embeddings that are 384 long [backend] info: [TokenManager] Initialized new TokenManager instance for model: C:\Users\Isaac Tenorio\AppData\Roaming\anythingllm-desktop\storage\models\QNN\llama_v3_2_3b_chat_8k [QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager. [QnnProcessManager] QNN API server already running. Killing and rebooting [QnnProcessManager] QNN API server process 17040 killed. [QnnProcessManager] Checking for valid NPU cores. [QnnProcessManager] Checking core Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU - MATCH [QnnProcessManager] Booting QNN API server with model llama_v3_2_3b_chat_8k on port 3800. [QnnProcessManager] [windows] QNN API server running. Port 3800. PID 10916. [backend] info: QNNEngine offline - retrying. 1/5 [QnnProcessManager] [debug] stdout AnythingLLMQnnEngine API Server: Starting chat API on host 127.0.0.1:3800 SDK: 2.42.0.251225 DSP Targets: v81,v73 Build: 1.0.4 c022fb5 Fri Jan 16 11:00:32 2026 [QnnProcessManager] [debug] stdout GenieProfile bound successfully - accurate metrics enabled. [backend] info: QNNEngine offline - retrying. 2/5 [backend] info: QNNEngine offline - retrying. 3/5 [backend] error: Error: QNN Engine is offline. Please reboot QNN Engine or AnythingLLM app. at s.checkReady (C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:31:2420) at async s.streamGetChatCompletion (C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:31:3595) at async W$ (C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:316:3134) at async C:\Users\Isaac Tenorio\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:316:6121 [backend] info: QNNEngine offline - retrying. 4/5 [backend] info: QNNEngine offline - retrying. 5/5 ``` Yes, I’m comfortable using PowerShell. I can run whatever CLI commands you provide. I’ve also attached the full log above including the entire QNN startup sequence. Let me know the commands you’d like me to execute and I’ll report back with the output.
yindo commented 2026-06-05 14:50:58 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 5, 2026):

Hm, okay then lets to debug this via calling the engine directly:
First open %APPDATA%/Local/Programs/AnythingLLM/resources/QnnSDK/LLMServer in terminal.
In here should be a file called AnythingLLMQnnEngine.exe

Next open explorer and navigate to %APPDATA%/Roaming/anythingllm-desktop/storage/models/QNN/

  • In here find the llama_v3_2_3b_chat_8k folder and right click -> Copy as path

In terminal you should run a command that looks like this

& .\AnythingLLMQnnEngine.exe --config "C:\Users\<usr>\AppData\Roaming\anythingllm-desktop\storage\models\QNN\llama_v3_2_3b_chat_8k"

Press enter, you should see the server begin to boot. Now simply wait! After GenieProfile bound successfully - accurate metrics enabled. you should hopefully see [INFO] "using create from Binary" and some other [INFO] logs.

Considering the machine, our timeout may not be long enough to create the binary on your machine and it times out before malloc completes.

If this works, you can try a simple request in another terminal window.

irm -Uri "http://127.0.0.1:8000/v1/chat/completions" -Method Post -ContentType "application/json" -Body '{"messages":[{"role":"user","content":"Tell me a joke."}]}' | Select-Object -ExpandProperty choices | % { $_.message.content }

Should output something like

Here's a joke for you:

What do you call a fake noodle?

An impasta!

I hope you found that one saucy! Do you want another?

Kill AnythingLLM before doing this just to be sure there is not port collision.

<!-- gh-comment-id:4006646329 --> @timothycarambat commented on GitHub (Mar 5, 2026): Hm, okay then lets to debug this via calling the engine directly: First open `%APPDATA%/Local/Programs/AnythingLLM/resources/QnnSDK/LLMServer` in terminal. In here should be a file called `AnythingLLMQnnEngine.exe` Next open explorer and navigate to `%APPDATA%/Roaming/anythingllm-desktop/storage/models/QNN/` - In here find the `llama_v3_2_3b_chat_8k` folder and right click -> Copy as path In terminal you should run a command that looks like this ```powershell & .\AnythingLLMQnnEngine.exe --config "C:\Users\<usr>\AppData\Roaming\anythingllm-desktop\storage\models\QNN\llama_v3_2_3b_chat_8k" ``` Press enter, you should see the server begin to boot. Now simply _wait_! After `GenieProfile bound successfully - accurate metrics enabled.` you should hopefully see `[INFO] "using create from Binary"` and some other `[INFO]` logs. Considering the machine, our timeout may not be long enough to create the binary on your machine and it times out before `malloc` completes. If this works, you can try a simple request in another terminal window. ```powershell irm -Uri "http://127.0.0.1:8000/v1/chat/completions" -Method Post -ContentType "application/json" -Body '{"messages":[{"role":"user","content":"Tell me a joke."}]}' | Select-Object -ExpandProperty choices | % { $_.message.content } ``` Should output something like ``` Here's a joke for you: What do you call a fake noodle? An impasta! I hope you found that one saucy! Do you want another? ``` Kill AnythingLLM before doing this just to be sure there is not port collision.
yindo commented 2026-06-05 14:50:58 -04:00
Author
Owner
Copy Link
Copy Source

@takeshy commented on GitHub (Mar 10, 2026):

I can confirm this issue also occurs on Snapdragon X Elite (Lenovo Yoga X Elite), not just X Plus.

Environment:

  • Device: Lenovo Yoga X Elite
  • CPU: Snapdragon X Elite (X1E)
  • OS: Windows 11 ARM64, Build 26200.7840 (24H2)
  • AnythingLLM QNN Engine: v1.0.4
  • QNN SDK: 2.42.0.251225 / DSP Targets: v81,v73

Error when running AnythingLLMQnnEngine.exe directly:

[INFO]  "Using create From Binary"
[INFO]  "Allocated total size = 816947712 across 8 buffers"
[ERROR] "Could not create context from binary for context index = 2 : err 1002"
[ERROR] "Create From Binary FAILED!"
Failed to load model: Failed to create the Genie Dialog.

Key finding — this is a regression:

A Japanese tech article from June 2025 documents Phi 3.5 Mini working correctly on the same Snapdragon X Elite hardware: https://forest.watch.impress.co.jp/docs/serial/usecopilotpc/2022243.html

I also verified that the model binaries currently bundled with AnythingLLM are identical to the official Snapdragon X Elite binaries distributed by Qualcomm AI Hub (phi_3_5_mini_instruct_4k, downloaded with "Snapdragon X Elite" selected). File sizes and dates match exactly (2024/12/13).

This means the model binaries are correct for X Elite — the issue lies in AnythingLLMQnnEngine.exe itself.

Additional finding: Llama 3.2 3B (llama_v3_2_3b_chat_8k) also fails with the same error on Snapdragon X Elite:

[INFO] "Using create From Binary"
[INFO] "Allocated total size = 506298880 across 5 buffers"
[ERROR] "Could not create context from binary for context index = 2 : err 1002"
[ERROR] "Create From Binary FAILED!"
Failed to load model: Failed to create the Genie Dialog.

This confirms the issue is not specific to the Phi 3.5 Mini model — err 1002 occurs with all models on this device. The problem is in AnythingLLMQnnEngine.exe itself.

Update: I also tested Microsoft Foundry Local (which uses ONNX Runtime QNN EP, completely independent from Genie SDK) and got the same failure:
Error code: 14001
Failed to load from EpContext model.
Failed to create context from binary.
This suggests the root cause may be in the Qualcomm NPU driver environment on this device rather than AnythingLLMQnnEngine.exe specifically. Will investigate driver updates.

<!-- gh-comment-id:4034787674 --> @takeshy commented on GitHub (Mar 10, 2026): I can confirm this issue also occurs on **Snapdragon X Elite (Lenovo Yoga X Elite)**, not just X Plus. **Environment:** - Device: Lenovo Yoga X Elite - CPU: Snapdragon X Elite (X1E) - OS: Windows 11 ARM64, Build 26200.7840 (24H2) - AnythingLLM QNN Engine: v1.0.4 - QNN SDK: 2.42.0.251225 / DSP Targets: v81,v73 **Error when running AnythingLLMQnnEngine.exe directly:** ``` [INFO] "Using create From Binary" [INFO] "Allocated total size = 816947712 across 8 buffers" [ERROR] "Could not create context from binary for context index = 2 : err 1002" [ERROR] "Create From Binary FAILED!" Failed to load model: Failed to create the Genie Dialog. ``` **Key finding — this is a regression:** A Japanese tech article from June 2025 documents Phi 3.5 Mini working correctly on the same Snapdragon X Elite hardware: https://forest.watch.impress.co.jp/docs/serial/usecopilotpc/2022243.html I also verified that the model binaries currently bundled with AnythingLLM are **identical** to the official Snapdragon X Elite binaries distributed by Qualcomm AI Hub (`phi_3_5_mini_instruct_4k`, downloaded with "Snapdragon X Elite" selected). File sizes and dates match exactly (2024/12/13). This means the model binaries are correct for X Elite — the issue lies in **`AnythingLLMQnnEngine.exe` itself**. Additional finding: Llama 3.2 3B (llama_v3_2_3b_chat_8k) also fails with the same error on Snapdragon X Elite: [INFO] "Using create From Binary" [INFO] "Allocated total size = 506298880 across 5 buffers" [ERROR] "Could not create context from binary for context index = 2 : err 1002" [ERROR] "Create From Binary FAILED!" Failed to load model: Failed to create the Genie Dialog. This confirms the issue is not specific to the Phi 3.5 Mini model — err 1002 occurs with all models on this device. The problem is in AnythingLLMQnnEngine.exe itself. Update: I also tested Microsoft Foundry Local (which uses ONNX Runtime QNN EP, completely independent from Genie SDK) and got the same failure: Error code: 14001 Failed to load from EpContext model. Failed to create context from binary. This suggests the root cause may be in the Qualcomm NPU driver environment on this device rather than AnythingLLMQnnEngine.exe specifically. Will investigate driver updates.
yindo commented 2026-06-05 14:50:58 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 10, 2026):

For our engine we use the QNN SDK - which if fails to load due to memory would be code 1005/1004 or something like that.

If MS ONNX format is also failing via the ExecutionProvider (which also uses what QNN SDK uses) then we know its not the code specifically, and it happens on every model size, then that only leaves hardware/drivers.

The above, in a different form has happened before and it was due to a driver issue on a new update of Windows, but IIRC it was on a specific Dell device - not a Lenovo. However, I believe the same situation could be applying here :/

<!-- gh-comment-id:4035093564 --> @timothycarambat commented on GitHub (Mar 10, 2026): For our engine we use the QNN SDK - which if fails to load due to memory would be code 1005/1004 or something like that. If MS ONNX format is **also failing** via the ExecutionProvider (which also uses what QNN SDK uses) then we know its not the code specifically, and it happens on every model size, then that only leaves hardware/drivers. The above, in a different form has happened before and it was due to a driver issue on a new update of Windows, but IIRC it was on a specific Dell device - not a Lenovo. However, I believe the same situation could be applying here :/
yindo commented 2026-06-05 14:50:58 -04:00
Author
Owner
Copy Link
Copy Source

@takeshy commented on GitHub (Mar 10, 2026):

Thanks for the detailed analysis!

Root cause found: Memory Integrity (HVCI) was blocking NPU access

After further investigation, I discovered the actual root cause on my Snapdragon X Elite device (Lenovo Yoga X Elite):

Windows Memory Integrity (HVCI) was enabled, which blocks the Hexagon DSP skeleton library from loading. This caused err 1002 (Genie SDK) and error 14001 (ONNX Runtime QNN EP) across all tools — not just AnythingLLM.

Verification:

Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\DeviceGuard\Scenarios\HypervisorEnforcedCodeIntegrity" -Name "Enabled"
# Result: Enabled = 1  ← this was the problem

Fix: Disabled Memory Integrity via Windows Security → Device Security → Core Isolation → Memory Integrity → OFF, then rebooted.

After disabling HVCI, both AnythingLLM QNN Engine and Microsoft Foundry Local NPU models load successfully.

I agree the error code doesn't point to malloc/memory — it was a DSP skel library load failure caused by HVCI blocking the unsigned kernel-level code.

Summary for anyone hitting this issue:

  • Check if Memory Integrity is enabled before debugging drivers or model binaries
  • This affects all QNN-based NPU inference on Windows (Genie SDK, ONNX Runtime QNN EP, Foundry Local)
  • Model binaries and drivers were correct all along — HVCI was silently blocking the DSP skel load
<!-- gh-comment-id:4035148829 --> @takeshy commented on GitHub (Mar 10, 2026): Thanks for the detailed analysis! **Root cause found: Memory Integrity (HVCI) was blocking NPU access** After further investigation, I discovered the actual root cause on my Snapdragon X Elite device (Lenovo Yoga X Elite): Windows Memory Integrity (HVCI) was enabled, which blocks the Hexagon DSP skeleton library from loading. This caused `err 1002` (Genie SDK) and `error 14001` (ONNX Runtime QNN EP) across all tools — not just AnythingLLM. **Verification:** ```powershell Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\DeviceGuard\Scenarios\HypervisorEnforcedCodeIntegrity" -Name "Enabled" # Result: Enabled = 1 ← this was the problem ``` **Fix:** Disabled Memory Integrity via Windows Security → Device Security → Core Isolation → Memory Integrity → OFF, then rebooted. After disabling HVCI, both AnythingLLM QNN Engine and Microsoft Foundry Local NPU models load successfully. I agree the error code doesn't point to malloc/memory — it was a DSP skel library load failure caused by HVCI blocking the unsigned kernel-level code. **Summary for anyone hitting this issue:** - Check if Memory Integrity is enabled before debugging drivers or model binaries - This affects all QNN-based NPU inference on Windows (Genie SDK, ONNX Runtime QNN EP, Foundry Local) - Model binaries and drivers were correct all along — HVCI was silently blocking the DSP skel load
yindo commented 2026-06-05 14:50:58 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 10, 2026):

@takeshy Excellent work! I will report this to Qualcomm directly since I have a line with them - this is a critical issue that should be fixed by them/vendor/MSFT - because its going to be an issue going forward it seems.

<!-- gh-comment-id:4035878773 --> @timothycarambat commented on GitHub (Mar 10, 2026): @takeshy Excellent work! I will report this to Qualcomm directly since I have a line with them - this is a critical issue that should be fixed by them/vendor/MSFT - because its going to be an issue going forward it seems.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#4933
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?