[GH-ISSUE #3194] NPU Compatibility Issue on Qualcomm X Elite (Surface Laptop 7 & Asus Vivobook S 15) #2055

New Issue

Originally created by @borkessb on GitHub (Feb 13, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3194

Version: v1.7.3-r2

I tested the latest release focusing on getting the NPU to finally work properly. Here are my findings:

Microsoft Surface Laptop 7 (X ELITE):
The LLM Provider Qualcomm QNN does not work in any configuration.
None of the available 3b and 8b chat models run on the NPU.

Asus Vivobook S 15 (X ELITE):
Only works with 3b models, and I can see NPU activity.
Performance is extremely slow, despite the small model size.

Also, the winget update for the same application leads to incorrect URLs.

At the address:
https://cdn.useanything.com/latest/AnythingLLMDesktop-Arm64.exe
there is nothing... The regular download is available at:
https://cdn.anythingllm.com/...

Originally created by @borkessb on GitHub (Feb 13, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3194 Version: v1.7.3-r2 I tested the latest release focusing on getting the NPU to finally work properly. Here are my findings: Microsoft Surface Laptop 7 (X ELITE): The LLM Provider Qualcomm QNN does not work in any configuration. None of the available 3b and 8b chat models run on the NPU. Asus Vivobook S 15 (X ELITE): Only works with 3b models, and I can see NPU activity. Performance is extremely slow, despite the small model size. Also, the winget update for the same application leads to incorrect URLs. At the address: https://cdn.useanything.com/latest/AnythingLLMDesktop-Arm64.exe there is nothing... The regular download is available at: https://cdn.anythingllm.com/...

yindo closed this issue

yindo commented

@timothycarambat commented on GitHub (Feb 13, 2025):

I dont know who built the winget installer because we do not maintain it. So if it needs to be fixed I dont even know where to correct it since we dont maintain that installer method.

Microsoft Surface Laptop 7 (X ELITE):
The LLM Provider Qualcomm QNN does not work in any configuration.
None of the available 3b and 8b chat models run on the NPU.

Asus Vivobook S 15 (X ELITE):
Only works with 3b models, and I can see NPU activity.
Performance is extremely slow, despite the small model size.

Any specs on these machines for RAM? The NPU uses a limited set of available memory on the device. If you are on 8GB it will never load a decent model and memory required will consume your entire available NPU RAM and probably move to Swap, which will be 10x slower.

@timothycarambat commented on GitHub (Feb 13, 2025): I dont know who built the `winget` installer because we do not maintain it. So if it needs to be fixed I dont even know where to correct it since we dont maintain that installer method. Microsoft Surface Laptop 7 (X ELITE): The LLM Provider Qualcomm QNN does not work in any configuration. None of the available 3b and 8b chat models run on the NPU. Asus Vivobook S 15 (X ELITE): Only works with 3b models, and I can see NPU activity. Performance is extremely slow, despite the small model size. Any specs on these machines for RAM? The NPU uses a limited set of available memory on the device. If you are on 8GB it will never load a decent model and memory required will consume your entire available NPU RAM and probably move to Swap, which will be 10x slower.

yindo commented

@borkessb commented on GitHub (Feb 13, 2025):

For winget, I assumed … Surface Laptop Snapdragon X 12-core X1E80100 @ 3.40 GHz with 16 GB of RAM,
ASUS Vivobook S15 … Snapdragon X Elite X1E with 32 GB of RAM

All in all, you’re doing an excellent job, and I will gladly try and test each new version. I have high hopes for the NPU and Qualcomm.

@borkessb commented on GitHub (Feb 13, 2025): For winget, I assumed … Surface Laptop Snapdragon X 12-core X1E80100 @ 3.40 GHz with 16 GB of RAM, ASUS Vivobook S15 … Snapdragon X Elite X1E with 32 GB of RAM All in all, you’re doing an excellent job, and I will gladly try and test each new version. I have high hopes for the NPU and Qualcomm.

yindo commented

@timothycarambat commented on GitHub (Feb 13, 2025):

Interesting 🤔! I have a 32GB Dell Latitude with 32GB RAM, same X-Elite chipset. When you say the models do not run do you just see "QNN Engine is offline" or something to that effect?

Once thing I have noticed is that until the next update, where we made some changes in the QNN runtime, people say that they have to run the app as admin for the QNN engine to boot up (#2962). Apparently Qualcomm made some changes that are in the next update that wont require that any longer (provided your device IT restrictions still allow running exe's).

As for the winget thing, we had this same thing happen with homebrew. I wasn't sure if winget has some OSS repo for managing packages we can bump on behalf of everyone. 🤷

@timothycarambat commented on GitHub (Feb 13, 2025): Interesting 🤔! I have a 32GB Dell Latitude with 32GB RAM, same X-Elite chipset. When you say the models do not run do you just see "QNN Engine is offline" or something to that effect? Once thing I have noticed is that until the next update, where we made some changes in the QNN runtime, people say that they have to run the app as admin for the QNN engine to boot up (#2962). Apparently Qualcomm made some changes that are in the next update that wont require that any longer (provided your device IT restrictions still allow running exe's). As for the winget thing, we had this same thing happen with homebrew. I wasn't sure if winget has some OSS repo for managing packages we can bump on behalf of everyone. 🤷

yindo commented

@borkessb commented on GitHub (Feb 13, 2025):

It turned out it was the “Run as administrator” issue after all. It’s actually working fine now. Thanks again. Let’s move on. As for winget, I think we should submit issues for manifest updates to the official repo.

@borkessb commented on GitHub (Feb 13, 2025): It turned out it was the “Run as administrator” issue after all. It’s actually working fine now. Thanks again. Let’s move on. As for winget, I think we should submit issues for manifest updates to the official repo.

yindo commented

@timothycarambat commented on GitHub (Feb 13, 2025):

Qualcomm eng told me that the next SDK version (which will be used in our engine next patch) will resolve the run as admin issue 🤞

Let’s move on. As for winget, I think we should submit issues for manifest updates to the official repo.

100%, i found out that all of this is done via winget/pkgs - so will make a PR to resolve that.

@timothycarambat commented on GitHub (Feb 13, 2025): Qualcomm eng told me that the next SDK version (which will be used in our engine next patch) will resolve the run as admin issue 🤞 > Let’s move on. As for winget, I think we should submit issues for manifest updates to the official repo. 100%, i found out that all of this is done via winget/pkgs - so will make a PR to resolve that.

yindo commented

@DanielDC88 commented on GitHub (Feb 24, 2025):

Looks like this is not working on a dell xps with an X1E80100. I tried running as administrator but didn't have any luck

My system has 32GB of RAM and supposedly 16GB of this is available to the NPU.

@DanielDC88 commented on GitHub (Feb 24, 2025): Looks like this is not working on a dell xps with an X1E80100. I tried running as administrator but didn't have any luck ![Image](https://github.com/user-attachments/assets/d6a70897-8476-4b34-9011-a595323ef1c5) My system has 32GB of RAM and supposedly 16GB of this is available to the NPU.

yindo commented

@timothycarambat commented on GitHub (Feb 25, 2025):

@DanielDC88 https://docs.anythingllm.com/installation-desktop/debug#general-debugging - please pull logs

@timothycarambat commented on GitHub (Feb 25, 2025): @DanielDC88 https://docs.anythingllm.com/installation-desktop/debug#general-debugging - please pull logs

yindo commented

@jinwater88 commented on GitHub (Apr 16, 2025):

@timothycarambat @DanielDC88 I also have this problem, what's going on? I uninstalled Qualcomm QNN's llama3.1 8B, and then the download was very slow and could not be downloaded again. What's going on?

@jinwater88 commented on GitHub (Apr 16, 2025): @timothycarambat @DanielDC88 I also have this problem, what's going on? I uninstalled Qualcomm QNN's llama3.1 8B, and then the download was very slow and could not be downloaded again. What's going on?

yindo commented

@timothycarambat commented on GitHub (Apr 16, 2025):

download was very slow

Well this comes from the CDN, so that is internet connection related

could not be downloaded again

How so? the model never finished downloading and you cannot redownload after app/close/reopen?

@timothycarambat commented on GitHub (Apr 16, 2025): > download was very slow Well this comes from the CDN, so that is internet connection related > could not be downloaded again How so? the model never finished downloading and you cannot redownload after app/close/reopen?

yindo commented

@LokimotiveUK commented on GitHub (Aug 21, 2025):

I am trying top get the best performance on this Surface laptop 7 with Snapdragon X Plus, besides trying to experiment with ONNX models and NPU performance, I have have tried AI Toolkit in VSCode and Anything LLM,. Anything LLM has a lot of promise with a lot of customisation, and in the positive side it intermittently allows full offload to the NPU at average of 85% to 90% usage. When this happens, inference speeds are great and I have a really good local LLM experience. This will stop working randomly, even with switching models, and NPU usage plummets to around 10% to 15%. When this happens the models performs sluggishly and is generally unusable. I have had limited success keeping this consistent as I cannot find any controls in the platform to manage NPU offload, similar to how LM Studio allows offload control to the GPU. I am trying this in AI Toolkit in VSCode and it is giving the same slow performance - which is sluggish and has low NPU usage. I need tips on how to solve this please.

@LokimotiveUK commented on GitHub (Aug 21, 2025): I am trying top get the best performance on this Surface laptop 7 with Snapdragon X Plus, besides trying to experiment with ONNX models and NPU performance, I have have tried AI Toolkit in VSCode and Anything LLM,. Anything LLM has a lot of promise with a lot of customisation, and in the positive side it intermittently allows full offload to the NPU at average of 85% to 90% usage. When this happens, inference speeds are great and I have a really good local LLM experience. This will stop working randomly, even with switching models, and NPU usage plummets to around 10% to 15%. When this happens the models performs sluggishly and is generally unusable. I have had limited success keeping this consistent as I cannot find any controls in the platform to manage NPU offload, similar to how LM Studio allows offload control to the GPU. I am trying this in AI Toolkit in VSCode and it is giving the same slow performance - which is sluggish and has low NPU usage. I need tips on how to solve this please.

yindo commented

@timothycarambat commented on GitHub (Aug 21, 2025):

This will stop working randomly, even with switching models, and NPU usage plummets to around 10% to 15%. When this happens the models performs sluggishly and is generally unusable.

I have seen this reported once before, I have never been able to consistently replicate it. It seems that if I leave a computer awake for too long for no reason at all, the NPU will be throttled to at max 15-30% ceiling. This is without anythingllm running by the way. I can start anythingllm later and it will do this.

Because of this, I do not think it is code we maintain in our specific engine but is instead some kind of memory manager or possibly power saving mode? I have tried every single power mode, every configuration I could try but to no avail. Simply, sometimes when the computer hibernates, I get this issue, and even then, it is rare and seemingly at random.

In fact, the last time I got the issue, I simply rebooted the computer, and it has not shown since. @LokimotiveUK, when is the last time you possibly rebooted the machine? It is almost like some OS-level process is doing the throttling since it can occur without the app running.

@timothycarambat commented on GitHub (Aug 21, 2025): > This will stop working randomly, even with switching models, and NPU usage plummets to around 10% to 15%. When this happens the models performs sluggishly and is generally unusable. I have seen this reported **once** before, I have never been able to consistently replicate it. It seems that if I leave a computer awake for too long for no reason at all, the NPU will be throttled to at max 15-30% ceiling. This is without anythingllm running by the way. I can start anythingllm later and it will do this. Because of this, I do not think it is code we maintain in our specific engine but is instead some kind of memory manager or possibly power saving mode? I have tried every single power mode, every configuration I could try but to no avail. Simply, sometimes when the computer hibernates, I get this issue, and even then, it is rare and seemingly at random. In fact, the last time I got the issue, I simply rebooted the computer, and it has not shown since. @LokimotiveUK, when is the last time you possibly rebooted the machine? It is almost like some OS-level process is doing the throttling since it can occur without the app running.

yindo commented

@LokimotiveUK commented on GitHub (Aug 25, 2025):

Heya @timothycarambat I reboot the machine regularly. My experience is probably the opposite to yours in that I very rarely get it to work as expected, almost all of the time it never offloads to the NPU and runs off the CPU. I've also downloaded all four AnythingLLM models in the category: "AnythingLLM NPU - Run LLMs locally on NPU exclusively on Snapdragon X CoPilot+ machines" and have only ever had success with the Llama 3.2 3B Chat 8K (2.49GB) model, albeit intermittently. If its helpful to you, let me know what data, logs or whatever I can get together if it helps with testing and troubleshooting? Also, feel free to dm me directly, I'm more than happy to help with testing this. I'll run a few scenarios and see if standby, hibernation or similar can recreate this, but as this morning I did a fresh bootup after the laptop had been shut down for a few days, opened Anything LLM as the first application after boot and had the resource monitor open next to it, kept the preffered model in and ran a simple text prompt, with no NPU offload. Hope this helps.

@LokimotiveUK commented on GitHub (Aug 25, 2025): Heya @timothycarambat I reboot the machine regularly. My experience is probably the opposite to yours in that I very rarely get it to work as expected, almost all of the time it never offloads to the NPU and runs off the CPU. I've also downloaded all four AnythingLLM models in the category: "AnythingLLM NPU - Run LLMs locally on NPU exclusively on Snapdragon X CoPilot+ machines" and have only ever had success with the Llama 3.2 3B Chat 8K (2.49GB) model, albeit intermittently. If its helpful to you, let me know what data, logs or whatever I can get together if it helps with testing and troubleshooting? Also, feel free to dm me directly, I'm more than happy to help with testing this. I'll run a few scenarios and see if standby, hibernation or similar can recreate this, but as this morning I did a fresh bootup after the laptop had been shut down for a few days, opened Anything LLM as the first application after boot and had the resource monitor open next to it, kept the preffered model in and ran a simple text prompt, with no NPU offload. Hope this helps.

yindo commented

@timothycarambat commented on GitHub (Aug 26, 2025):

with no NPU offload.

The AnythingLLM NPU models can only run on NPU - with the process monitor open you dont see any bumps in the performance graph during inferencing? Its not possible for the model to execute outside of the NPU so there must be some NPU inference occurring?

@timothycarambat commented on GitHub (Aug 26, 2025): > with no NPU offload. The AnythingLLM NPU models _can only_ run on NPU - with the process monitor open you dont see any bumps in the performance graph during inferencing? Its not possible for the model to execute outside of the NPU so there must be some NPU inference occurring?

yindo commented

@LokimotiveUK commented on GitHub (Aug 26, 2025):

There is some but minimal and intermittent. I just did a fresh restart and tried using the system, and have captured a series of screenshots with annotations showing the journey today. The first one show full NPU usage over the first two question to the model in a fresh chat, then the subsequent performance from the third question onwards. These subsequent score are still typically higher than I usually get, they're generally slower.

Screenshots

@LokimotiveUK commented on GitHub (Aug 26, 2025): There is some but minimal and intermittent. I just did a fresh restart and tried using the system, and have captured a series of screenshots with annotations showing the journey today. The first one show full NPU usage over the first two question to the model in a fresh chat, then the subsequent performance from the third question onwards. These subsequent score are still typically higher than I usually get, they're generally slower. [Screenshots](https://drive.google.com/drive/folders/1AZfcIKYnwlsdiW5M1kKhhQ0N4yImDu_s?usp=sharing)

yindo commented

@timothycarambat commented on GitHub (Aug 26, 2025):

@LokimotiveUK This is extremely helpful. I know NPU is currently memory-constrained. I wonder if some piece of inference is not being garbage collected, causing the NPU to under-utilize. Since the NPU can only use some sub-partition of RAM, if the available RAM is low, the sub-partition it can use becomes even smaller.

In those screenshots, what is the model being used? Also, do you have documents in this chat window? I would like to fully replicate this issue locally since I have an X Plus machine nearby.

@timothycarambat commented on GitHub (Aug 26, 2025): @LokimotiveUK This is extremely helpful. I know NPU is currently memory-constrained. I wonder if some piece of inference is not being garbage collected, causing the NPU to under-utilize. Since the NPU can only use some sub-partition of RAM, if the available RAM is low, the sub-partition it can use becomes even smaller. In those screenshots, what is the model being used? Also, do you have documents in this chat window? I would like to fully replicate this issue locally since I have an X Plus machine nearby.

yindo commented

@LokimotiveUK commented on GitHub (Aug 26, 2025):

Sure thing - no documents attached, only text based queries. Fresh laptop boot with the only two applications opened after boot, those being AnythingLLM and Task Manager (minus standard OS load, but I keep it light and little else starts up). I do want to maintain standard system usage as well though to replicate real world daily usage conditions.

Provider: AnythingLLM NPU
Model: Llama 3.2 3B Chat 8K - 2.49GB (Active)

Could well be RAM usage, I'll try a few other scenario and model combos and see if I can see a pattern. A useful feature may be something like the ability to allocate or control RAM or NPU thresholds? Not sure how this could stress other areas of the system, but it feels like it is currently reaching some kind of limit and then throttling based on that.

@LokimotiveUK commented on GitHub (Aug 26, 2025): Sure thing - no documents attached, only text based queries. Fresh laptop boot with the only two applications opened after boot, those being AnythingLLM and Task Manager (minus standard OS load, but I keep it light and little else starts up). I do want to maintain standard system usage as well though to replicate real world daily usage conditions. Provider: AnythingLLM NPU Model: Llama 3.2 3B Chat 8K - 2.49GB (Active) Could well be RAM usage, I'll try a few other scenario and model combos and see if I can see a pattern. A useful feature may be something like the ability to allocate or control RAM or NPU thresholds? Not sure how this could stress other areas of the system, but it feels like it is currently reaching some kind of limit and then throttling based on that.

yindo commented

@LokimotiveUK commented on GitHub (Aug 26, 2025):

Quick update: Changing Model

@LokimotiveUK commented on GitHub (Aug 26, 2025): Quick update: [Changing Model](https://drive.google.com/drive/folders/1pIHbAI3-IXrVjjz_rBGm83NEK08ZNTtO?usp=sharing)

yindo commented

@timothycarambat commented on GitHub (Aug 26, 2025):

@LokimotiveUK - I see citations in the response, so you for sure have documents embedded in that workspace. How large are those documents?

@timothycarambat commented on GitHub (Aug 26, 2025): @LokimotiveUK - I see citations in the response, so you for sure have documents embedded in that workspace. How large are those documents?

yindo commented

@timothycarambat commented on GitHub (Aug 26, 2025):

@LokimotiveUK Would it be possible for you to replicate this with AnythingLLM open and process window open (like the screnshots) but do a Snipping Tool video? This is all very helpful and I appreciate it a lot.

@timothycarambat commented on GitHub (Aug 26, 2025): @LokimotiveUK Would it be possible for you to replicate this with AnythingLLM open and process window open (like the screnshots) but do a Snipping Tool video? This is all very helpful and I appreciate it a lot.

yindo commented

@LokimotiveUK commented on GitHub (Aug 31, 2025):

@timothycarambat yes of course, apologies, I was on the road for a few days. Link here: Video capture

It started fine for a few seconds then dropped straight off. I tried switching between a few models and flipping around the resource monitor, let me know if there's a specific scenario and I'll try reproduce in sequence.

@LokimotiveUK commented on GitHub (Aug 31, 2025): @timothycarambat yes of course, apologies, I was on the road for a few days. Link here: [Video capture](https://drive.google.com/drive/folders/1yweCrHAPgTdWR-hX4_IFH6BCeTdrLwBo?usp=sharing) It started fine for a few seconds then dropped straight off. I tried switching between a few models and flipping around the resource monitor, let me know if there's a specific scenario and I'll try reproduce in sequence.

yindo commented

@renzocastillo commented on GitHub (Feb 4, 2026):

Hi guys, I'm dealing with the same situation with my Lenovo Yoga Slim 7 Laptop with Snapdragon X Elite

Windows PowerShell
Copyright (C) Microsoft Corporation. Todos los derechos reservados.

Instale la versión más reciente de PowerShell para obtener nuevas características y mejoras. https://aka.ms/PSWindows

PS C:\Users\renzo> C:\path\AnythingLLM\AnythingLLM.exe

PS C:\Users\renzo> [Preferences] preference config stored at C:\Users\renzo\AppData\Roaming\anythingllm-desktop\config.json
[PrismaHelper: Generic] Setting Prisma Client output directory default. {
  clientOutputPath: 'C:\\path\\AnythingLLM\\resources\\backend\\node_modules\\.prisma\\client'
}
Prisma schema loaded from prisma\schema.prisma
Datasource "db": SQLite database "anythingllm.db" at "file:C:/Users/renzo/AppData/Roaming/anythingllm-desktop/storage/anythingllm.db"

Already in sync, no schema change or pending migration was found.

✔ Generated Prisma Client (v5.3.1, engine=binary) to .\node_modules\.prisma\clie
nt in 169ms


Prisma schema loaded from prisma\schema.prisma

✔ Generated Prisma Client (v5.3.1, engine=binary) to .\node_modules\.prisma\client in 163ms

Start using Prisma Client in Node.js (See: https://pris.ly/d/client)

import { PrismaClient } from './node_modules/.prisma/client'
const prisma = new PrismaClient()

or start using Prisma Client at the edge (See: https://pris.ly/d/accelerate)

import { PrismaClient } from './node_modules/.prisma/client/edge'
const prisma = new PrismaClient()


See other ways of importing Prisma Client: http://pris.ly/d/importing-client

[QnnProcessManager] Checking for valid NPU cores.
[QnnProcessManager] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH
[QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager.
[QnnProcessManager] Checking for valid NPU cores.
[QnnProcessManager] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH
[OllamaProcessManager] Ollama will bind on port 11434 when booted.
[QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager.
[ManualBrowser] Listening for IPC messages from main renderer process!
[MeetingAssistantHelper.DownloadHelper] DownloadHelper initialized.
[MeetingAssistantHelper] Initialized & Registered
[meeting-assistant] info: Initialized & Registered
[MeetingAssistant.Bootstrap] Bootstrap is not supported on this platform.
[MeetingAssistantHelper] Bootstrap is not supported on this platform.
[meeting-assistant] info: Bootstrap is not supported on this platform.
[FFmpegHelper.Bootstrap] Bootstrap is not supported on this platform.
[FFmpegHelper] Bootstrap is not supported on this platform.
[ffmpeg] info: Bootstrap is not supported on this platform.
[Preferences] Will load window with last know bounds.
[MeetingAssistantHelper] Meeting assistant bootstrapped successfully!
[meeting-assistant] info: Meeting assistant bootstrapped successfully!
[MeetingAssistantHelper] FFMPEG bootstrapped successfully!
[meeting-assistant] info: FFMPEG bootstrapped successfully!
[49248:0204/192039.451:ERROR:ffmpeg_common.cc(959)] Unsupported pixel format: -1
[collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[collector] info: Collector hot directory and tmp storage wiped!
[collector] info: [production] AnythingLLM Standalone Document processor listening on port 8888.
[backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo
[backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo
[backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM.
[backend] info: prisma:info Starting a sqlite pool with 25 connections.
[backend] info: prisma:info Started query engine http server on http://127.0.0.1:55318
[backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"d566e655-2255-4788-bdef-d3a4099d4d6f","properties":{}}
[backend] info: Skipping preloading of AnythingLLMOllama - LLM_PROVIDER is qnnengine.
[backend] info: Hot loading of QnnEngine - LLM_PROVIDER is qnnengine with model llama_3_8b.
[backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2
[backend] info: [QNN Engine] Initialized with model: llama_3_8b. Context window: 4096
[backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[backend] info: [production] AnythingLLM Standalone Backend listening on port 3001. Network discovery is disabled. NPU Detected: true OS: windows (arm64)
[QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager.
[QnnProcessManager] Checking for valid NPU cores.
[QnnProcessManager] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH
[QnnProcessManager] Booting QNN API server with model llama_3_8b on port 3800.
[backend] info: [BackgroundWorkerService] Starting...
[backend] info: [BackgroundWorkerService] Service started with 2 jobs ["cleanup-orphan-documents","cleanup-orphan-recordings"]
[QnnProcessManager] [windows] QNN API server running. Port 3800. PID 48432.
[backend] info: QNNEngine offline - retrying. 1/5
[QnnProcessManager] [debug] stdout
Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b

Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b/genie_config.json

[QnnProcessManager] [debug] stdout
:::::::: AnythingLLMQnnEngine API Server 1.0.4 ::::::::

--config <Local directory path>: [Required] Base directory to set as the working directory, must have tokenizer.json, genie_config.json, and model.bin files.
--port <Port number>: [Optional] Port number to run the server on. Default is 8000.
--host <Host name>: [Optional] Host name to run the server on. Default is 127.0.0.1.
--version [Optional] Print the build version and exit.
--help [Optional] Print this help message and exit.

[backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\path\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll
[backend] info: [QnnNativeEmbedder] Checking for valid NPU cores.
[backend] info: [QnnNativeEmbedder] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH
[backend] info: [QnnNativeEmbedder] Initialized all-minilm-l6-v2 model
[backend] info: QNNEngine offline - retrying. 2/5
[backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\path\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll
[backend] info: [QnnNativeEmbedder] Checking for valid NPU cores.
[backend] info: [QnnNativeEmbedder] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH
[backend] info: [QnnNativeEmbedder] Initialized all-minilm-l6-v2 model
[backend] info: [QNN Engine] Initialized with model: llama_3_8b. Context window: 4096
[backend] info: [TokenManager] Initialized new TokenManager instance for model: llama_3_8b
[backend] error: Error: QNN Engine is booting. Please wait for it to finish and try again.
    at s.checkReady (C:\path\AnythingLLM\resources\backend\server.js:31:2064)
    at s.streamGetChatCompletion (C:\path\AnythingLLM\resources\backend\server.js:31:3384)
    at $D (C:\path\AnythingLLM\resources\backend\server.js:293:3142)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async C:\path\AnythingLLM\resources\backend\server.js:293:4792
[backend] info: QNNEngine offline - retrying. 3/5
[backend] info: QNNEngine offline - retrying. 4/5
[backend] info: QNNEngine offline - retrying. 5/5
[backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll
[backend] info: [QnnNativeEmbedder] Checking for valid NPU cores.
[backend] info: [QnnNativeEmbedder] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH
[backend] info: [QnnNativeEmbedder] Initialized all-minilm-l6-v2 model
[backend] info: [QNN Engine] Initialized with model: llama_3_8b. Context window: 4096
[backend] info: [TokenManager] Returning existing instance for model: llama_3_8b
[QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager.
[QnnProcessManager] QNN API server already running. Killing and rebooting
[QnnProcessManager] QNN API server process 48432 killed.
[QnnProcessManager] Checking for valid NPU cores.
[QnnProcessManager] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH
[QnnProcessManager] Booting QNN API server with model llama_3_8b on port 3800.
[QnnProcessManager] [windows] QNN API server running. Port 3800. PID 14704.
[backend] info: QNNEngine offline - retrying. 1/5
[QnnProcessManager] [debug] stdout
Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b

Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b/genie_config.json

[QnnProcessManager] [debug] stdout
:::::::: AnythingLLMQnnEngine API Server 1.0.4 ::::::::

--config <Local directory path>: [Required] Base directory to set as the working directory, must have tokenizer.json, genie_config.json, and model.bin files.
--port <Port number>: [Optional] Port number to run the server on. Default is 8000.
--host <Host name>: [Optional] Host name to run the server on. Default is 127.0.0.1.
--version [Optional] Print the build version and exit.
--help [Optional] Print this help message and exit.

[backend] info: QNNEngine offline - retrying. 2/5
[backend] info: QNNEngine offline - retrying. 3/5
[backend] error: Error: QNN Engine is offline. Please reboot QNN Engine or AnythingLLM app.
    at s.checkReady (C:\path\AnythingLLM\resources\backend\server.js:31:2198)
    at async s.streamGetChatCompletion (C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:31:3373)
    at async $D (C:\path\AnythingLLM\resources\backend\server.js:293:3134)
    at async C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:293:4792
[backend] info: QNNEngine offline - retrying. 4/5
[backend] info: QNNEngine offline - retrying. 5/5

Any clues about this issue?

@renzocastillo commented on GitHub (Feb 4, 2026): Hi guys, I'm dealing with the same situation with my Lenovo Yoga Slim 7 Laptop with Snapdragon X Elite ``` Windows PowerShell Copyright (C) Microsoft Corporation. Todos los derechos reservados. Instale la versión más reciente de PowerShell para obtener nuevas características y mejoras. https://aka.ms/PSWindows PS C:\Users\renzo> C:\path\AnythingLLM\AnythingLLM.exe PS C:\Users\renzo> [Preferences] preference config stored at C:\Users\renzo\AppData\Roaming\anythingllm-desktop\config.json [PrismaHelper: Generic] Setting Prisma Client output directory default. { clientOutputPath: 'C:\\path\\AnythingLLM\\resources\\backend\\node_modules\\.prisma\\client' } Prisma schema loaded from prisma\schema.prisma Datasource "db": SQLite database "anythingllm.db" at "file:C:/Users/renzo/AppData/Roaming/anythingllm-desktop/storage/anythingllm.db" Already in sync, no schema change or pending migration was found. ✔ Generated Prisma Client (v5.3.1, engine=binary) to .\node_modules\.prisma\clie nt in 169ms Prisma schema loaded from prisma\schema.prisma ✔ Generated Prisma Client (v5.3.1, engine=binary) to .\node_modules\.prisma\client in 163ms Start using Prisma Client in Node.js (See: https://pris.ly/d/client) ``` import { PrismaClient } from './node_modules/.prisma/client' const prisma = new PrismaClient() ``` or start using Prisma Client at the edge (See: https://pris.ly/d/accelerate) ``` import { PrismaClient } from './node_modules/.prisma/client/edge' const prisma = new PrismaClient() ``` See other ways of importing Prisma Client: http://pris.ly/d/importing-client [QnnProcessManager] Checking for valid NPU cores. [QnnProcessManager] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager. [QnnProcessManager] Checking for valid NPU cores. [QnnProcessManager] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [OllamaProcessManager] Ollama will bind on port 11434 when booted. [QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager. [ManualBrowser] Listening for IPC messages from main renderer process! [MeetingAssistantHelper.DownloadHelper] DownloadHelper initialized. [MeetingAssistantHelper] Initialized & Registered [meeting-assistant] info: Initialized & Registered [MeetingAssistant.Bootstrap] Bootstrap is not supported on this platform. [MeetingAssistantHelper] Bootstrap is not supported on this platform. [meeting-assistant] info: Bootstrap is not supported on this platform. [FFmpegHelper.Bootstrap] Bootstrap is not supported on this platform. [FFmpegHelper] Bootstrap is not supported on this platform. [ffmpeg] info: Bootstrap is not supported on this platform. [Preferences] Will load window with last know bounds. [MeetingAssistantHelper] Meeting assistant bootstrapped successfully! [meeting-assistant] info: Meeting assistant bootstrapped successfully! [MeetingAssistantHelper] FFMPEG bootstrapped successfully! [meeting-assistant] info: FFMPEG bootstrapped successfully! [49248:0204/192039.451:ERROR:ffmpeg_common.cc(959)] Unsupported pixel format: -1 [collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [collector] info: Collector hot directory and tmp storage wiped! [collector] info: [production] AnythingLLM Standalone Document processor listening on port 8888. [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo [backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM. [backend] info: prisma:info Starting a sqlite pool with 25 connections. [backend] info: prisma:info Started query engine http server on http://127.0.0.1:55318 [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"d566e655-2255-4788-bdef-d3a4099d4d6f","properties":{}} [backend] info: Skipping preloading of AnythingLLMOllama - LLM_PROVIDER is qnnengine. [backend] info: Hot loading of QnnEngine - LLM_PROVIDER is qnnengine with model llama_3_8b. [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [QNN Engine] Initialized with model: llama_3_8b. Context window: 4096 [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: [production] AnythingLLM Standalone Backend listening on port 3001. Network discovery is disabled. NPU Detected: true OS: windows (arm64) [QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager. [QnnProcessManager] Checking for valid NPU cores. [QnnProcessManager] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [QnnProcessManager] Booting QNN API server with model llama_3_8b on port 3800. [backend] info: [BackgroundWorkerService] Starting... [backend] info: [BackgroundWorkerService] Service started with 2 jobs ["cleanup-orphan-documents","cleanup-orphan-recordings"] [QnnProcessManager] [windows] QNN API server running. Port 3800. PID 48432. [backend] info: QNNEngine offline - retrying. 1/5 [QnnProcessManager] [debug] stdout Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b/genie_config.json [QnnProcessManager] [debug] stdout :::::::: AnythingLLMQnnEngine API Server 1.0.4 :::::::: --config <Local directory path>: [Required] Base directory to set as the working directory, must have tokenizer.json, genie_config.json, and model.bin files. --port <Port number>: [Optional] Port number to run the server on. Default is 8000. --host <Host name>: [Optional] Host name to run the server on. Default is 127.0.0.1. --version [Optional] Print the build version and exit. --help [Optional] Print this help message and exit. [backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\path\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll [backend] info: [QnnNativeEmbedder] Checking for valid NPU cores. [backend] info: [QnnNativeEmbedder] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [backend] info: [QnnNativeEmbedder] Initialized all-minilm-l6-v2 model [backend] info: QNNEngine offline - retrying. 2/5 [backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\path\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll [backend] info: [QnnNativeEmbedder] Checking for valid NPU cores. [backend] info: [QnnNativeEmbedder] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [backend] info: [QnnNativeEmbedder] Initialized all-minilm-l6-v2 model [backend] info: [QNN Engine] Initialized with model: llama_3_8b. Context window: 4096 [backend] info: [TokenManager] Initialized new TokenManager instance for model: llama_3_8b [backend] error: Error: QNN Engine is booting. Please wait for it to finish and try again. at s.checkReady (C:\path\AnythingLLM\resources\backend\server.js:31:2064) at s.streamGetChatCompletion (C:\path\AnythingLLM\resources\backend\server.js:31:3384) at $D (C:\path\AnythingLLM\resources\backend\server.js:293:3142) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async C:\path\AnythingLLM\resources\backend\server.js:293:4792 [backend] info: QNNEngine offline - retrying. 3/5 [backend] info: QNNEngine offline - retrying. 4/5 [backend] info: QNNEngine offline - retrying. 5/5 [backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll [backend] info: [QnnNativeEmbedder] Checking for valid NPU cores. [backend] info: [QnnNativeEmbedder] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [backend] info: [QnnNativeEmbedder] Initialized all-minilm-l6-v2 model [backend] info: [QNN Engine] Initialized with model: llama_3_8b. Context window: 4096 [backend] info: [TokenManager] Returning existing instance for model: llama_3_8b [QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager. [QnnProcessManager] QNN API server already running. Killing and rebooting [QnnProcessManager] QNN API server process 48432 killed. [QnnProcessManager] Checking for valid NPU cores. [QnnProcessManager] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [QnnProcessManager] Booting QNN API server with model llama_3_8b on port 3800. [QnnProcessManager] [windows] QNN API server running. Port 3800. PID 14704. [backend] info: QNNEngine offline - retrying. 1/5 [QnnProcessManager] [debug] stdout Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b/genie_config.json [QnnProcessManager] [debug] stdout :::::::: AnythingLLMQnnEngine API Server 1.0.4 :::::::: --config <Local directory path>: [Required] Base directory to set as the working directory, must have tokenizer.json, genie_config.json, and model.bin files. --port <Port number>: [Optional] Port number to run the server on. Default is 8000. --host <Host name>: [Optional] Host name to run the server on. Default is 127.0.0.1. --version [Optional] Print the build version and exit. --help [Optional] Print this help message and exit. [backend] info: QNNEngine offline - retrying. 2/5 [backend] info: QNNEngine offline - retrying. 3/5 [backend] error: Error: QNN Engine is offline. Please reboot QNN Engine or AnythingLLM app. at s.checkReady (C:\path\AnythingLLM\resources\backend\server.js:31:2198) at async s.streamGetChatCompletion (C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:31:3373) at async $D (C:\path\AnythingLLM\resources\backend\server.js:293:3134) at async C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:293:4792 [backend] info: QNNEngine offline - retrying. 4/5 [backend] info: QNNEngine offline - retrying. 5/5 ``` Any clues about this issue?

yindo commented

@timothycarambat commented on GitHub (Feb 5, 2026):

Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b
Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b/genie_config.json

Did you rename the folder or ENV for the model manually? The only available models for QNN are

llama_v3_2_3b_chat_8k
llama_v3_2_3b_chat_16k
llama_v3_1_8b_chat_8k
phi_3_5_mini_instruct_4k

It should be impossible to get another file/folder name? What does the storage directory models/QNN folder have in it?
@renzocastillo

@timothycarambat commented on GitHub (Feb 5, 2026): ``` Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b/genie_config.json ``` Did you rename the folder or ENV for the model manually? The only available models for QNN are ``` llama_v3_2_3b_chat_8k llama_v3_2_3b_chat_16k llama_v3_1_8b_chat_8k phi_3_5_mini_instruct_4k ``` It should be impossible to get another file/folder name? What does the storage directory `models/QNN` folder have in it? @renzocastillo

yindo commented

@renzocastillo commented on GitHub (Feb 5, 2026):

Thanks @timothycarambat , do you know Is there a way to clear the configuration ?

Windows PowerShell Copyright (C) Microsoft Corporation. Todos los derechos reservados. Instale la versión más reciente de PowerShell para obtener nuevas características y mejoras. https://aka.ms/PSWindows PS C:\Windows\system32> C:\Users\renzo\AppData\Local\Programs\AnythingLLM\AnythingLLM.exe PS C:\Windows\system32> C:\Users\renzo\AppData\Local\Programs\AnythingLLM[backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dllelper: Generic] Setting Prisma Client output directory default. { [backend] info: [QnnNativeEmbedder] Checking for valid NPU cores.AnythingLLM\resources\backend\node_modules\.prisma[backend] info: [QnnNativeEmbedder] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [backend] info: [QnnNativeEmbedder] Initialized all-minilm-l6-v2 model [backend] info: [QNN Engine] Initialized with model: llama_3_8b. Context window: 4096 [backend] info: [TokenManager] Initialized new TokenManager instance for model: llama_3_8banythingllm-desktop/storage/an[QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager. [QnnProcessManager] QNN API server already running. Killing and rebooting [QnnProcessManager] QNN API server process 28576 killed.as found. [QnnProcessManager] Checking for valid NPU cores. [QnnProcessManager] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [QnnProcessManager] Booting QNN API server with model llama_3_8b on port 3800. [QnnProcessManager] [windows] QNN API server running. Port 3800. PID 7576. [backend] info: QNNEngine offline - retrying. 1/5 [QnnProcessManager] [debug] stdout hema.prisma Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b ✔ Generated Prisma Client (v5.3.1, engine=binary) to .\node_modules.prisma\client in 198ms Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b/genie_config.json Start using Prisma Client in Node.js (See: https://pris.ly/d/client) [QnnProcessManager] [debug] stdout :::::::: AnythingLLMQnnEngine API Server 1.0.4 ::::::::ient' const prisma = new PrismaClient() --config : [Required] Base directory to set as the working directory, must have tokenizer.json, genie_config.json, and model.bin files.dge (See: https://pris.ly/d/accelerate) --port : [Optional] Port number to run the server on. Default is 8000. --host : [Optional] Host name to run the server on. Default is 127.0.0.1. --version [Optional] Print the build version and exit. --help [Optional] Print this help message and exit. [backend] info: QNNEngine offline - retrying. 2/5 [backend] info: QNNEngine offline - retrying. 3/5 [backend] error: Error: QNN Engine is offline. Please reboot QNN Engine or AnythingLLM app. at s.checkReady (C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:31:2198) at async s.streamGetChatCompletion (C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:31:3373) at async $D (C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:293:3134) at async C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:293:4792 [backend] info: QNNEngine offline - retrying. 4/5 [backend] info: QNNEngine offline - retrying. 5/5 [MeetingAssistantHelper] Initialized & Registered [meeting-assistant] info: Initialized & Registered [MeetingAssistant.Bootstrap] Bootstrap is not supported on this platform. [MeetingAssistantHelper] Bootstrap is not supported on this platform. [meeting-assistant] info: Bootstrap is not supported on this platform. [FFmpegHelper.Bootstrap] Bootstrap is not supported on this platform. [FFmpegHelper] Bootstrap is not supported on this platform. [ffmpeg] info: Bootstrap is not supported on this platform. [Preferences] Will load window with last know bounds. [MeetingAssistantHelper] Meeting assistant bootstrapped successfully! [meeting-assistant] info: Meeting assistant bootstrapped successfully! [MeetingAssistantHelper] FFMPEG bootstrapped successfully! [meeting-assistant] info: FFMPEG bootstrapped successfully! [11120:0205/195012.446:ERROR:ffmpeg_common.cc(959)] Unsupported pixel format: -1 [collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [collector] info: Collector hot directory and tmp storage wiped! [collector] info: [production] AnythingLLM Standalone Document processor listening on port 8888. [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo [backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM. [backend] info: prisma:info Starting a sqlite pool with 25 connections. [backend] info: prisma:info Started query engine http server on http://127.0.0.1:54898 [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"d566e655-2255-4788-bdef-d3a4099d4d6f","properties":{}} [backend] info: Skipping preloading of AnythingLLMOllama - LLM_PROVIDER is qnnengine. [backend] info: Hot loading of QnnEngine - LLM_PROVIDER is qnnengine with model llama_3_8b. [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [QNN Engine] Initialized with model: llama_3_8b. Context window: 4096 [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: [production] AnythingLLM Standalone Backend listening on port 3001. Network discovery is disabled. NPU Detected: true OS: windows (arm64) [QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager. [backend] info: [BackgroundWorkerService] Starting... [QnnProcessManager] Checking for valid NPU cores. [QnnProcessManager] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [QnnProcessManager] Booting QNN API server with model llama_3_8b on port 3800. [backend] info: [BackgroundWorkerService] Service started with 2 jobs ["cleanup-orphan-documents","cleanup-orphan-recordings"] [QnnProcessManager] [windows] QNN API server running. Port 3800. PID 28576. [backend] info: QNNEngine offline - retrying. 1/5 [QnnProcessManager] [debug] stdout Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b/genie_config.json [QnnProcessManager] [debug] stdout :::::::: AnythingLLMQnnEngine API Server 1.0.4 :::::::: --config : [Required] Base directory to set as the working directory, must have tokenizer.json, genie_config.json, and model.bin files. --port : [Optional] Port number to run the server on. Default is 8000. --host : [Optional] Host name to run the server on. Default is 127.0.0.1. --version [Optional] Print the build version and exit. --help [Optional] Print this help message and exit. [backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll [backend] info: [QnnNativeEmbedder] Checking for valid NPU cores. [backend] info: [QnnNativeEmbedder] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [backend] info: [QnnNativeEmbedder] Initialized all-minilm-l6-v2 model [backend] info: QNNEngine offline - retrying. 2/5 [backend] info: QNNEngine offline - retrying. 3/5 [backend] info: QNNEngine offline - retrying. 4/5 [backend] info: QNNEngine offline - retrying. 5/5 prisma:info Starting a sqlite pool with 25 connections. prisma:info Started query engine http server on http://127.0.0.1:61238 [backend] info: [7520:31056]: No direct uploads path found - exiting. [bg-worker][cleanup-orphan-documents] info: [7520:31056]: No direct uploads path found - exiting. prisma:info Starting a sqlite pool with 25 connections. [backend] warn: Child process exited with code 0 and signal null [backend] info: Worker for job "cleanup-orphan-documents" exited with code 0 prisma:info Started query engine http server on http://127.0.0.1:61240 [backend] warn: Child process exited with code 0 and signal null [backend] info: Worker for job "cleanup-orphan-recordings" exited with code 0

I keep installing and re-installing and actually I always have the same configuration

@renzocastillo commented on GitHub (Feb 5, 2026): Thanks @timothycarambat , do you know Is there a way to clear the configuration ? Windows PowerShell Copyright (C) Microsoft Corporation. Todos los derechos reservados. Instale la versión más reciente de PowerShell para obtener nuevas características y mejoras. https://aka.ms/PSWindows PS C:\Windows\system32> C:\Users\renzo\AppData\Local\Programs\AnythingLLM\AnythingLLM.exe PS C:\Windows\system32> C:\Users\renzo\AppData\Local\Programs\AnythingLLM\[backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dllelper: Generic] Setting Prisma Client output directory default. { [backend] info: [QnnNativeEmbedder] Checking for valid NPU cores.AnythingLLM\\resources\\backend\\node_modules\\.prisma\[backend] info: [QnnNativeEmbedder] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [backend] info: [QnnNativeEmbedder] Initialized all-minilm-l6-v2 model [backend] info: [QNN Engine] Initialized with model: llama_3_8b. Context window: 4096 [backend] info: [TokenManager] Initialized new TokenManager instance for model: llama_3_8banythingllm-desktop/storage/an[QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager. [QnnProcessManager] QNN API server already running. Killing and rebooting [QnnProcessManager] QNN API server process 28576 killed.as found. [QnnProcessManager] Checking for valid NPU cores. [QnnProcessManager] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [QnnProcessManager] Booting QNN API server with model llama_3_8b on port 3800. [QnnProcessManager] [windows] QNN API server running. Port 3800. PID 7576. [backend] info: QNNEngine offline - retrying. 1/5 [QnnProcessManager] [debug] stdout hema.prisma Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b ✔ Generated Prisma Client (v5.3.1, engine=binary) to .\node_modules\.prisma\client in 198ms Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b/genie_config.json Start using Prisma Client in Node.js (See: https://pris.ly/d/client) [QnnProcessManager] [debug] stdout :::::::: AnythingLLMQnnEngine API Server 1.0.4 ::::::::ient' const prisma = new PrismaClient() --config <Local directory path>: [Required] Base directory to set as the working directory, must have tokenizer.json, genie_config.json, and model.bin files.dge (See: https://pris.ly/d/accelerate) --port <Port number>: [Optional] Port number to run the server on. Default is 8000. --host <Host name>: [Optional] Host name to run the server on. Default is 127.0.0.1. --version [Optional] Print the build version and exit. --help [Optional] Print this help message and exit. [backend] info: QNNEngine offline - retrying. 2/5 [backend] info: QNNEngine offline - retrying. 3/5 [backend] error: Error: QNN Engine is offline. Please reboot QNN Engine or AnythingLLM app. at s.checkReady (C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:31:2198) at async s.streamGetChatCompletion (C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:31:3373) at async $D (C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:293:3134) at async C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\backend\server.js:293:4792 [backend] info: QNNEngine offline - retrying. 4/5 [backend] info: QNNEngine offline - retrying. 5/5 [MeetingAssistantHelper] Initialized & Registered [meeting-assistant] info: Initialized & Registered [MeetingAssistant.Bootstrap] Bootstrap is not supported on this platform. [MeetingAssistantHelper] Bootstrap is not supported on this platform. [meeting-assistant] info: Bootstrap is not supported on this platform. [FFmpegHelper.Bootstrap] Bootstrap is not supported on this platform. [FFmpegHelper] Bootstrap is not supported on this platform. [ffmpeg] info: Bootstrap is not supported on this platform. [Preferences] Will load window with last know bounds. [MeetingAssistantHelper] Meeting assistant bootstrapped successfully! [meeting-assistant] info: Meeting assistant bootstrapped successfully! [MeetingAssistantHelper] FFMPEG bootstrapped successfully! [meeting-assistant] info: FFMPEG bootstrapped successfully! [11120:0205/195012.446:ERROR:ffmpeg_common.cc(959)] Unsupported pixel format: -1 [collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [collector] info: Collector hot directory and tmp storage wiped! [collector] info: [production] AnythingLLM Standalone Document processor listening on port 8888. [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo [backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM. [backend] info: prisma:info Starting a sqlite pool with 25 connections. [backend] info: prisma:info Started query engine http server on http://127.0.0.1:54898 [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"d566e655-2255-4788-bdef-d3a4099d4d6f","properties":{}} [backend] info: Skipping preloading of AnythingLLMOllama - LLM_PROVIDER is qnnengine. [backend] info: Hot loading of QnnEngine - LLM_PROVIDER is qnnengine with model llama_3_8b. [backend] info: [NativeEmbedder] Initialized Xenova/all-MiniLM-L6-v2 [backend] info: [QNN Engine] Initialized with model: llama_3_8b. Context window: 4096 [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: [production] AnythingLLM Standalone Backend listening on port 3001. Network discovery is disabled. NPU Detected: true OS: windows (arm64) [QnnProcessManager] SINGLETON LOCK: Using existing QnnProcessManager. [backend] info: [BackgroundWorkerService] Starting... [QnnProcessManager] Checking for valid NPU cores. [QnnProcessManager] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [QnnProcessManager] Booting QNN API server with model llama_3_8b on port 3800. [backend] info: [BackgroundWorkerService] Service started with 2 jobs ["cleanup-orphan-documents","cleanup-orphan-recordings"] [QnnProcessManager] [windows] QNN API server running. Port 3800. PID 28576. [backend] info: QNNEngine offline - retrying. 1/5 [QnnProcessManager] [debug] stdout Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b Invalid argument for --config: It must be a valid and accessible local path. Provided: llama_3_8b/genie_config.json [QnnProcessManager] [debug] stdout :::::::: AnythingLLMQnnEngine API Server 1.0.4 :::::::: --config <Local directory path>: [Required] Base directory to set as the working directory, must have tokenizer.json, genie_config.json, and model.bin files. --port <Port number>: [Optional] Port number to run the server on. Default is 8000. --host <Host name>: [Optional] Host name to run the server on. Default is 127.0.0.1. --version [Optional] Print the build version and exit. --help [Optional] Print this help message and exit. [backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\Users\renzo\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll [backend] info: [QnnNativeEmbedder] Checking for valid NPU cores. [backend] info: [QnnNativeEmbedder] Checking core Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU - MATCH [backend] info: [QnnNativeEmbedder] Initialized all-minilm-l6-v2 model [backend] info: QNNEngine offline - retrying. 2/5 [backend] info: QNNEngine offline - retrying. 3/5 [backend] info: QNNEngine offline - retrying. 4/5 [backend] info: QNNEngine offline - retrying. 5/5 prisma:info Starting a sqlite pool with 25 connections. prisma:info Started query engine http server on http://127.0.0.1:61238 [backend] info: [7520:31056]: No direct uploads path found - exiting. [bg-worker][cleanup-orphan-documents] info: [7520:31056]: No direct uploads path found - exiting. prisma:info Starting a sqlite pool with 25 connections. [backend] warn: Child process exited with code 0 and signal null [backend] info: Worker for job "cleanup-orphan-documents" exited with code 0 prisma:info Started query engine http server on http://127.0.0.1:61240 [backend] warn: Child process exited with code 0 and signal null [backend] info: Worker for job "cleanup-orphan-recordings" exited with code 0 I keep installing and re-installing and actually I always have the same configuration

yindo commented

@timothycarambat commented on GitHub (Feb 6, 2026):

@renzocastillo you can do two things:

Open the App and go to Settings > AI Providers >LLM and select another provider/model or even just another model alltogether for the NPU provider.

You can also open C:\Users\<usr>\AppData\Roaming\anythingllm-desktop\storage\.env in notepad and modify the QNN_ENGINE_MODEL_PREF line to be the full path to where the real model is - somehow you have a totally invalid value in that field that is not even selectable in the UI of llama_3_8b

@timothycarambat commented on GitHub (Feb 6, 2026): @renzocastillo you can do two things: - Open the App and go to Settings > AI Providers >LLM and select another provider/model or even just another model alltogether for the NPU provider. You can also open `C:\Users\<usr>\AppData\Roaming\anythingllm-desktop\storage\.env` in notepad and modify the `QNN_ENGINE_MODEL_PREF` line to be the **full** path to where the real model is - somehow you have a totally invalid value in that field that is not even selectable in the UI of `llama_3_8b`

yindo commented

https://github.com/user-attachments/assets/09bf7cef-2743-4663-8896-68120dd6a798

@renzocastillo commented on GitHub (Feb 7, 2026):

Thanks @timothycarambat , I just downloaded the app and selected the "ANYTHING LLM NPU" as my LLM Provider.

I think there's a small bug whenever we switch from one LLM Provider to another, since the previous QNN_ENGINE_MODEL env value doesn't get updated when switching from ANYTHING_LLM to ANYTHING_LLM_NPU provider. Please check this video

That's the way I got this value in .env file:

QNN_ENGINE_MODEL_PREF='llama_3_8b'

@renzocastillo commented on GitHub (Feb 7, 2026): Thanks @timothycarambat , I just downloaded the app and selected the "ANYTHING LLM NPU" as my LLM Provider. I think there's a small bug whenever we switch from one LLM Provider to another, since the previous QNN_ENGINE_MODEL env value doesn't get updated when switching from ANYTHING_LLM to ANYTHING_LLM_NPU provider. Please check this video https://github.com/user-attachments/assets/09bf7cef-2743-4663-8896-68120dd6a798 That's the way I got this value in .env file: ``` QNN_ENGINE_MODEL_PREF='llama_3_8b' ```

yindo commented