Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4026] [BUG]: Fail to fetch using ARM NPU Embedder on Windows when user has non-ASCII characters in username. #2561

New Issue
Open
opened 2026-02-22 18:30:15 -05:00 by yindo · 4 comments
yindo commented 2026-02-22 18:30:15 -05:00
Owner
Copy Link
Copy Source

Originally created by @zacken on GitHub (Jun 20, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4026

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

When trying to embed documents on a Windows ARM (Snapdragon) PC using the NPU Embedder it fails with a "Failed to fetch" error.
From log (username is anonymized):
[backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\Users\HxxxxxFørxx\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll
Runtime options:
backend_path: C:\Users\HxxxxxFørxx\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll
soc_model: 60
htp_arch: 73
htp_performance_mode: high_performance
enable_htp_fp16_precision: 1
Load model from C:\Users\HenrikFￃﾸrli\AppData\Roaming\anythingllm-desktop\storage\models\qnn-embedder\all-minilm-l6-v2\model.onnx failed:Load model C:\Users\HxxxxxFￃﾸrxx\AppData\Roaming\anythingllm-desktop\storage\models\qnn-embedder\all-minilm-l6-v2\model.onnx failed. File doesn't exist

In this case the character ø is replaced with ￃﾸ in the path when loading model.onnx

When loading LLM models the correct path is created:
[backend] info: [TokenManager] Initialized new TokenManager instance for model: C:\Users\HxxxxxFørxx\AppData\Roaming\anythingllm-desktop\storage\models\QNN\phi_3_5_mini_instruct_4k

Are there known steps to reproduce?

Can be reproduced by using Windows account with international characters on a Windows ARM Snapdragon PC, install AnythingLLM, select NPU Embedder, embed document.

Work if no international characters in Windows username.

Workaround

Start a command prompt in administrator mode and create a symlink between the wrong folder and the correct one:

mklink /d C:\Users\HxxxxxFￃﾸrxx C:\Users\HxxxxxFørxx

Verified to work.

Originally created by @zacken on GitHub (Jun 20, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4026 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? When trying to embed documents on a Windows ARM (Snapdragon) PC using the NPU Embedder it fails with a "Failed to fetch" error. From log (username is anonymized): [backend] info: [QnnNativeEmbedder] NPU is enabled, using QnnSDK backend at C:\Users\HxxxxxFørxx\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll Runtime options: backend_path: C:\Users\HxxxxxFørxx\AppData\Local\Programs\AnythingLLM\resources\QnnSDK\aarch64-windows-msvc\QnnHtp.dll soc_model: 60 htp_arch: 73 htp_performance_mode: high_performance enable_htp_fp16_precision: 1 Load model from C:\Users\HenrikFￃﾸrli\AppData\Roaming\anythingllm-desktop\storage\models\qnn-embedder\all-minilm-l6-v2\model.onnx failed:Load model C:\Users\HxxxxxFￃﾸrxx\AppData\Roaming\anythingllm-desktop\storage\models\qnn-embedder\all-minilm-l6-v2\model.onnx failed. File doesn't exist In this case the character ø is replaced with ￃﾸ in the path when loading model.onnx When loading LLM models the correct path is created: [backend] info: [TokenManager] Initialized new TokenManager instance for model: C:\Users\HxxxxxFørxx\AppData\Roaming\anythingllm-desktop\storage\models\QNN\phi_3_5_mini_instruct_4k ### Are there known steps to reproduce? Can be reproduced by using Windows account with international characters on a Windows ARM Snapdragon PC, install AnythingLLM, select NPU Embedder, embed document. Work if no international characters in Windows username. ### Workaround Start a command prompt in administrator mode and create a symlink between the wrong folder and the correct one: `mklink /d C:\Users\HxxxxxFￃﾸrxx C:\Users\HxxxxxFørxx` Verified to work.
yindo added the bugDesktopOS: Windows labels 2026-02-22 18:30:15 -05:00
yindo commented 2026-02-22 18:30:15 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 24, 2025):

Interesting, thanks for writing this up but also providing a temporay workaround while we address that

@timothycarambat commented on GitHub (Jun 24, 2025): Interesting, thanks for writing this up but also providing a temporay workaround while we address that
yindo changed title from [BUG]: Fail to fetch using ARM NPU Embedder on Windows when user has non-ASCII characters in username. to [GH-ISSUE #4026] [BUG]: Fail to fetch using ARM NPU Embedder on Windows when user has non-ASCII characters in username. 2026-06-05 14:47:15 -04:00
yindo commented 2026-06-05 14:47:15 -04:00
Author
Owner
Copy Link
Copy Source

@majiayu000 commented on GitHub (Mar 21, 2026):

Hi, I've been looking into this and traced the root cause.

Proposed approach: Find the NPU/QNN embedder path construction code (likely in server/utils/EmbeddingEngines/ or similar). Compare how the LLM model path is constructed (which works correctly with non-ASCII characters) vs. how the embedder model path is constructed. The fix likely involves ensuring proper UTF-8 encoding or using Buffer/path APIs that handle Unicode correctly when resolving the model.onnx path. The broken encoding 'ￃﾸ' for 'ø' suggests a double-encoding or incorrect charset conversion — fix by using native path APIs (e.g., Node.js path.join) instead of string concatenation, or ensuring proper encoding when passing paths to the ONNX runtime.

I'll open a draft PR shortly. Happy to adjust based on your preference.

<!-- gh-comment-id:4102720654 --> @majiayu000 commented on GitHub (Mar 21, 2026): Hi, I've been looking into this and traced the root cause. **Proposed approach:** Find the NPU/QNN embedder path construction code (likely in server/utils/EmbeddingEngines/ or similar). Compare how the LLM model path is constructed (which works correctly with non-ASCII characters) vs. how the embedder model path is constructed. The fix likely involves ensuring proper UTF-8 encoding or using Buffer/path APIs that handle Unicode correctly when resolving the model.onnx path. The broken encoding 'ￃﾸ' for 'ø' suggests a double-encoding or incorrect charset conversion — fix by using native path APIs (e.g., Node.js path.join) instead of string concatenation, or ensuring proper encoding when passing paths to the ONNX runtime. I'll open a draft PR shortly. Happy to adjust based on your preference.
yindo commented 2026-06-05 14:47:15 -04:00
Author
Owner
Copy Link
Copy Source

@majiayu000 commented on GitHub (Mar 21, 2026):

Hi, I've been looking into this and traced the root cause.

Proposed approach: 1. Clone/fork the repo, locate NPU/QNN embedder code (likely server/utils/EmbeddingEngines/ or similar QnnNativeEmbedder class). 2. Compare how LLM model path is constructed (works correctly with ø) vs embedder path (corrupts ø to ￃﾸ — classic double UTF-8 encoding / mojibake). 3. Fix by ensuring proper UTF-8 handling — likely replace string concatenation with path.join() or fix charset conversion when passing paths to ONNX runtime. 4. Add unit test for non-ASCII path handling. 5. PR title: 'fix: handle non-ASCII characters in NPU embedder model path'.

I'll open a draft PR shortly. Happy to adjust based on your preference.

<!-- gh-comment-id:4102876262 --> @majiayu000 commented on GitHub (Mar 21, 2026): Hi, I've been looking into this and traced the root cause. **Proposed approach:** 1. Clone/fork the repo, locate NPU/QNN embedder code (likely server/utils/EmbeddingEngines/ or similar QnnNativeEmbedder class). 2. Compare how LLM model path is constructed (works correctly with ø) vs embedder path (corrupts ø to ￃﾸ — classic double UTF-8 encoding / mojibake). 3. Fix by ensuring proper UTF-8 handling — likely replace string concatenation with path.join() or fix charset conversion when passing paths to ONNX runtime. 4. Add unit test for non-ASCII path handling. 5. PR title: 'fix: handle non-ASCII characters in NPU embedder model path'. I'll open a draft PR shortly. Happy to adjust based on your preference.
yindo commented 2026-06-05 14:47:16 -04:00
Author
Owner
Copy Link
Copy Source

@leavedrop commented on GitHub (May 25, 2026):

Root cause looks like a classic Windows + native binding UTF-8 path bug. The character ø (U+00F8, two UTF-8 bytes: 0xC3 0xB8) is being interpreted by the QnnSDK / ONNX Runtime ARM64 native binding as a Latin-1 / OEM codepage string rather than UTF-8 — the bytes 0xC3 0xB8 are then re-encoded into the path, producing the garbage ￃﾸ you see in the failure log. The LLM model load path works because that path goes through a different code path (TokenManager uses a JS-only layer that doesn't cross the native ABI for path strings).

The reason Load model from <path> succeeds in the log message but actual file open fails is that the log string is still a JS UTF-8 string, but by the time the path is handed off to the QnnSDK native function it's been silently transcoded.

Three fix layers, ordered by how invasive they are:

  1. Userland workaround (zero-code, immediate): document that the QNN embedder storage directory must use a Windows short path. Operators can set the storage env var to e.g. C:\AnythingLLM (no non-ASCII) and the bug disappears. This is what most native-binding-with-broken-utf8 projects ship as the interim fix.

  2. Resolve path to 8.3 short name before handing to native (small, robust): before calling into the QnnSDK / ONNX Runtime, resolve the storage-relative model path through Windows' GetShortPathName (Node has fs.realpath + win32 short-path APIs via node-shortpath or a direct ffi call). The resulting path is ASCII-only by construction. Doesn't require touching the native binding.

  3. Fix the native binding to take UTF-8 (right but invasive): patch the QnnSDK Node addon to accept std::wstring / Path16 and use fopen_s with the wide-char filename path instead of fopen. This is the correct fix but requires a native build and coordination with the QnnSDK vendor.

I'd suggest shipping (1) as an immediate doc note and (2) as the proper code fix — happy to PR (2) plus a regression test that asserts the resolved storage path is ASCII-only before the native call. Don't have an ARM NPU machine to verify end-to-end though, so someone with the hardware would need to confirm the embed actually completes after the path resolution change.

Same class of bug recurs across native bindings on Windows whenever a user has any non-ASCII char in their Windows username (Cyrillic, CJK, accented Latin like this one). Worth fixing once at the path-handoff layer.

<!-- gh-comment-id:4531882385 --> @leavedrop commented on GitHub (May 25, 2026): Root cause looks like a classic Windows + native binding UTF-8 path bug. The character `ø` (U+00F8, two UTF-8 bytes: `0xC3 0xB8`) is being interpreted by the QnnSDK / ONNX Runtime ARM64 native binding as a Latin-1 / OEM codepage string rather than UTF-8 — the bytes `0xC3 0xB8` are then re-encoded into the path, producing the garbage `ￃﾸ` you see in the failure log. The LLM model load path works because that path goes through a different code path (TokenManager uses a JS-only layer that doesn't cross the native ABI for path strings). The reason `Load model from <path>` succeeds in the log message but actual file open fails is that the *log* string is still a JS UTF-8 string, but by the time the path is handed off to the QnnSDK native function it's been silently transcoded. Three fix layers, ordered by how invasive they are: 1. **Userland workaround** (zero-code, immediate): document that the QNN embedder storage directory must use a Windows short path. Operators can set the storage env var to e.g. `C:\AnythingLLM` (no non-ASCII) and the bug disappears. This is what most native-binding-with-broken-utf8 projects ship as the interim fix. 2. **Resolve path to 8.3 short name before handing to native** (small, robust): before calling into the QnnSDK / ONNX Runtime, resolve the storage-relative model path through Windows' `GetShortPathName` (Node has `fs.realpath` + `win32` short-path APIs via `node-shortpath` or a direct `ffi` call). The resulting path is ASCII-only by construction. Doesn't require touching the native binding. 3. **Fix the native binding to take UTF-8** (right but invasive): patch the QnnSDK Node addon to accept `std::wstring` / `Path16` and use `fopen_s` with the wide-char filename path instead of `fopen`. This is the correct fix but requires a native build and coordination with the QnnSDK vendor. I'd suggest shipping (1) as an immediate doc note and (2) as the proper code fix — happy to PR (2) plus a regression test that asserts the resolved storage path is ASCII-only before the native call. Don't have an ARM NPU machine to verify end-to-end though, so someone with the hardware would need to confirm the embed actually completes after the path resolution change. Same class of bug recurs across native bindings on Windows whenever a user has any non-ASCII char in their Windows username (Cyrillic, CJK, accented Latin like this one). Worth fixing once at the path-handoff layer.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label Desktop OS: Windows bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2561
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?