Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #539] Error: Unsupported model type: whisper #316

New Issue
Closed
opened 2026-02-22 18:18:55 -05:00 by yindo · 4 comments
yindo commented 2026-02-22 18:18:55 -05:00
Owner
Copy Link
Copy Source

Originally created by @lsiem on GitHub (Jan 5, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/539

When uploading an mp4 file I get the error: failed fetch.

The docker log output shows the following:

2024-01-05 09:53:44 -- Working The PisaSales user interface.mp4 --
2024-01-05 09:53:44 [Conversion Required] .mp4 file detected - converting to .wav
2024-01-05 09:53:45 [Conversion Processing]: 4KB converted
2024-01-05 09:53:46 [Conversion Processing]: 37376KB converted
2024-01-05 09:53:46 [Conversion Processing]: 54596KB converted
2024-01-05 09:53:46 [Conversion Complete]: File converted to .wav!
2024-01-05 09:58:45 fetch failed
2024-01-05 09:49:08 ┌─────────────────────────────────────────────────────────┐
2024-01-05 09:49:08 │  Update available 5.3.1 -> 5.7.1                        │
2024-01-05 09:49:08 │  Run the following to update                            │
2024-01-05 09:49:08 │    npm i --save-dev prisma@latest                       │
2024-01-05 09:49:08 │    npm i @prisma/client@latest                          │
2024-01-05 09:49:08 └─────────────────────────────────────────────────────────┘
2024-01-05 09:49:09 fatal: not a git repository (or any of the parent directories): .git
2024-01-05 09:49:09 getGitVersion Command failed: git rev-parse HEAD
2024-01-05 09:49:09 fatal: not a git repository (or any of the parent directories): .git
2024-01-05 09:49:09 
2024-01-05 10:01:27 Failed to load the native whisper model: Error: Unsupported model type: whisper
2024-01-05 10:01:27     at AutoModelForCTC.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/models.js:4667:19)
2024-01-05 10:01:27     at async file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2677:33
2024-01-05 10:01:27 node:internal/process/promises:288
2024-01-05 10:01:27             triggerUncaughtException(err, true /* fromPromise */);
2024-01-05 10:01:27             ^
2024-01-05 10:01:27 
2024-01-05 10:01:27 Error: Unsupported model type: whisper
2024-01-05 10:01:27     at AutoModelForCTC.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/models.js:4667:19)
2024-01-05 10:01:27     at async file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2677:33
2024-01-05 10:01:27 
2024-01-05 10:01:27 Node.js v18.19.0

I am running anythingllm with my Apple M1 Max running macOS 14.2.1 (23C71)

Originally created by @lsiem on GitHub (Jan 5, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/539 When uploading an mp4 file I get the error: `failed fetch`. The docker log output shows the following: ``` 2024-01-05 09:53:44 -- Working The PisaSales user interface.mp4 -- 2024-01-05 09:53:44 [Conversion Required] .mp4 file detected - converting to .wav 2024-01-05 09:53:45 [Conversion Processing]: 4KB converted 2024-01-05 09:53:46 [Conversion Processing]: 37376KB converted 2024-01-05 09:53:46 [Conversion Processing]: 54596KB converted 2024-01-05 09:53:46 [Conversion Complete]: File converted to .wav! 2024-01-05 09:58:45 fetch failed 2024-01-05 09:49:08 ┌─────────────────────────────────────────────────────────┐ 2024-01-05 09:49:08 │ Update available 5.3.1 -> 5.7.1 │ 2024-01-05 09:49:08 │ Run the following to update │ 2024-01-05 09:49:08 │ npm i --save-dev prisma@latest │ 2024-01-05 09:49:08 │ npm i @prisma/client@latest │ 2024-01-05 09:49:08 └─────────────────────────────────────────────────────────┘ 2024-01-05 09:49:09 fatal: not a git repository (or any of the parent directories): .git 2024-01-05 09:49:09 getGitVersion Command failed: git rev-parse HEAD 2024-01-05 09:49:09 fatal: not a git repository (or any of the parent directories): .git 2024-01-05 09:49:09 2024-01-05 10:01:27 Failed to load the native whisper model: Error: Unsupported model type: whisper 2024-01-05 10:01:27 at AutoModelForCTC.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/models.js:4667:19) 2024-01-05 10:01:27 at async file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2677:33 2024-01-05 10:01:27 node:internal/process/promises:288 2024-01-05 10:01:27 triggerUncaughtException(err, true /* fromPromise */); 2024-01-05 10:01:27 ^ 2024-01-05 10:01:27 2024-01-05 10:01:27 Error: Unsupported model type: whisper 2024-01-05 10:01:27 at AutoModelForCTC.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/models.js:4667:19) 2024-01-05 10:01:27 at async file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2677:33 2024-01-05 10:01:27 2024-01-05 10:01:27 Node.js v18.19.0 ``` I am running anythingllm with my Apple M1 Max running macOS 14.2.1 (23C71)
yindo added the bugDocker labels 2026-02-22 18:18:55 -05:00
yindo closed this issue 2026-02-22 18:18:55 -05:00
yindo commented 2026-02-22 18:18:55 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 5, 2024):

Related: https://github.com/xenova/transformers.js/issues/314

@timothycarambat commented on GitHub (Jan 5, 2024): Related: https://github.com/xenova/transformers.js/issues/314
yindo commented 2026-02-22 18:18:55 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 5, 2024):

Can you shell into your container and see if there are any files in /app/server/storage/models/Xenova/whisper-small. The model should only download when needed for the first time and be loaded from the disk on subsequent runs. However, if for whatever reason HuggingFace blocks the download, then the model would fail to download and thus, inference would fail

I do not have an M-series chip so I am unable to replicate this issue currently. We have the "patched" version of transformers running already so that should be resolved.

@timothycarambat commented on GitHub (Jan 5, 2024): Can you shell into your container and see if there are any files in `/app/server/storage/models/Xenova/whisper-small`. The model should only download when needed for the first time and be loaded from the disk on subsequent runs. However, if for whatever reason HuggingFace blocks the download, then the model would fail to download and thus, inference would fail I do not have an M-series chip so I am unable to replicate this issue currently. We have the "patched" version of transformers running already so that should be resolved.
yindo commented 2026-02-22 18:18:55 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Jan 5, 2024):

I get a similar fetch failed error in my docker logs as well on an M1 Pro running macOS 12.4

2024-01-05 09:45:41 -- Working jfk.wav --
2024-01-05 09:45:41 [INFO] The native whisper model has never been run and will be downloaded right now. Subsequent runs will be faster. (~250MB)
2024-01-05 09:45:41 
2024-01-05 09:45:41 
2024-01-05 09:46:17 Failed to load the native whisper model: TypeError: fetch failed
2024-01-05 09:46:17     at Object.fetch (node:internal/deps/undici/undici:11730:11)
2024-01-05 09:46:17     at async getModelFile (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:471:24)
2024-01-05 09:46:17     at async getModelJSON (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:575:18)
2024-01-05 09:46:17     at async AutoProcessor.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/processors.js:1752:44)
2024-01-05 09:46:17     at async Promise.all (index 2)
2024-01-05 09:46:17     at async loadItems (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2694:5)
2024-01-05 09:46:17     at async pipeline (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2640:19)
2024-01-05 09:46:17     at async LocalWhisper.client (/app/collector/utils/WhisperProviders/localWhisper.js:34:14) {
2024-01-05 09:46:17   cause: SocketError: other side closed
2024-01-05 09:46:17       at TLSSocket.onSocketEnd (node:internal/deps/undici/undici:8280:26)
2024-01-05 09:46:17       at TLSSocket.emit (node:events:529:35)
2024-01-05 09:46:17       at endReadableNT (node:internal/streams/readable:1400:12)
2024-01-05 09:46:17       at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
2024-01-05 09:46:17     code: 'UND_ERR_SOCKET',
2024-01-05 09:46:17     socket: {
2024-01-05 09:46:17       localAddress: '172.17.0.2',
2024-01-05 09:46:17       localPort: 40518,
2024-01-05 09:46:17       remoteAddress: undefined,
2024-01-05 09:46:17       remotePort: undefined,
2024-01-05 09:46:17       remoteFamily: undefined,
2024-01-05 09:46:17       timeout: undefined,
2024-01-05 09:46:17       bytesWritten: 263,
2024-01-05 09:46:17       bytesRead: 0
2024-01-05 09:46:17     }
2024-01-05 09:46:17   }
2024-01-05 09:46:17 }
2024-01-05 09:46:17 node:internal/process/promises:288
2024-01-05 09:46:17             triggerUncaughtException(err, true /* fromPromise */);
2024-01-05 09:46:17             ^
2024-01-05 09:46:17 
2024-01-05 09:46:17 TypeError: fetch failed
2024-01-05 09:46:17     at Object.fetch (node:internal/deps/undici/undici:11730:11)
2024-01-05 09:46:17     at async getModelFile (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:471:24)
2024-01-05 09:46:17     at async getModelJSON (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:575:18)
2024-01-05 09:46:17     at async AutoProcessor.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/processors.js:1752:44)
2024-01-05 09:46:17     at async Promise.all (index 2)
2024-01-05 09:46:17     at async loadItems (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2694:5)
2024-01-05 09:46:17     at async pipeline (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2640:19)
2024-01-05 09:46:17     at async LocalWhisper.client (/app/collector/utils/WhisperProviders/localWhisper.js:34:14) {
2024-01-05 09:46:17   cause: SocketError: other side closed
2024-01-05 09:46:17       at TLSSocket.onSocketEnd (node:internal/deps/undici/undici:8280:26)
2024-01-05 09:46:17       at TLSSocket.emit (node:events:529:35)
2024-01-05 09:46:17       at endReadableNT (node:internal/streams/readable:1400:12)
2024-01-05 09:46:17       at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
2024-01-05 09:46:17     code: 'UND_ERR_SOCKET',
2024-01-05 09:46:17     socket: {
2024-01-05 09:46:17       localAddress: '172.17.0.2',
2024-01-05 09:46:17       localPort: 40518,
2024-01-05 09:46:17       remoteAddress: undefined,
2024-01-05 09:46:17       remotePort: undefined,
2024-01-05 09:46:17       remoteFamily: undefined,
2024-01-05 09:46:17       timeout: undefined,
2024-01-05 09:46:17       bytesWritten: 263,
2024-01-05 09:46:17       bytesRead: 0
2024-01-05 09:46:17     }
2024-01-05 09:46:17   }
2024-01-05 09:46:17 }
2024-01-05 09:46:17 
2024-01-05 09:46:17 Node.js v18.19.0
@shatfield4 commented on GitHub (Jan 5, 2024): I get a similar `fetch failed` error in my docker logs as well on an M1 Pro running macOS 12.4 ``` 2024-01-05 09:45:41 -- Working jfk.wav -- 2024-01-05 09:45:41 [INFO] The native whisper model has never been run and will be downloaded right now. Subsequent runs will be faster. (~250MB) 2024-01-05 09:45:41 2024-01-05 09:45:41 2024-01-05 09:46:17 Failed to load the native whisper model: TypeError: fetch failed 2024-01-05 09:46:17 at Object.fetch (node:internal/deps/undici/undici:11730:11) 2024-01-05 09:46:17 at async getModelFile (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:471:24) 2024-01-05 09:46:17 at async getModelJSON (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:575:18) 2024-01-05 09:46:17 at async AutoProcessor.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/processors.js:1752:44) 2024-01-05 09:46:17 at async Promise.all (index 2) 2024-01-05 09:46:17 at async loadItems (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2694:5) 2024-01-05 09:46:17 at async pipeline (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2640:19) 2024-01-05 09:46:17 at async LocalWhisper.client (/app/collector/utils/WhisperProviders/localWhisper.js:34:14) { 2024-01-05 09:46:17 cause: SocketError: other side closed 2024-01-05 09:46:17 at TLSSocket.onSocketEnd (node:internal/deps/undici/undici:8280:26) 2024-01-05 09:46:17 at TLSSocket.emit (node:events:529:35) 2024-01-05 09:46:17 at endReadableNT (node:internal/streams/readable:1400:12) 2024-01-05 09:46:17 at process.processTicksAndRejections (node:internal/process/task_queues:82:21) { 2024-01-05 09:46:17 code: 'UND_ERR_SOCKET', 2024-01-05 09:46:17 socket: { 2024-01-05 09:46:17 localAddress: '172.17.0.2', 2024-01-05 09:46:17 localPort: 40518, 2024-01-05 09:46:17 remoteAddress: undefined, 2024-01-05 09:46:17 remotePort: undefined, 2024-01-05 09:46:17 remoteFamily: undefined, 2024-01-05 09:46:17 timeout: undefined, 2024-01-05 09:46:17 bytesWritten: 263, 2024-01-05 09:46:17 bytesRead: 0 2024-01-05 09:46:17 } 2024-01-05 09:46:17 } 2024-01-05 09:46:17 } 2024-01-05 09:46:17 node:internal/process/promises:288 2024-01-05 09:46:17 triggerUncaughtException(err, true /* fromPromise */); 2024-01-05 09:46:17 ^ 2024-01-05 09:46:17 2024-01-05 09:46:17 TypeError: fetch failed 2024-01-05 09:46:17 at Object.fetch (node:internal/deps/undici/undici:11730:11) 2024-01-05 09:46:17 at async getModelFile (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:471:24) 2024-01-05 09:46:17 at async getModelJSON (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:575:18) 2024-01-05 09:46:17 at async AutoProcessor.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/processors.js:1752:44) 2024-01-05 09:46:17 at async Promise.all (index 2) 2024-01-05 09:46:17 at async loadItems (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2694:5) 2024-01-05 09:46:17 at async pipeline (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2640:19) 2024-01-05 09:46:17 at async LocalWhisper.client (/app/collector/utils/WhisperProviders/localWhisper.js:34:14) { 2024-01-05 09:46:17 cause: SocketError: other side closed 2024-01-05 09:46:17 at TLSSocket.onSocketEnd (node:internal/deps/undici/undici:8280:26) 2024-01-05 09:46:17 at TLSSocket.emit (node:events:529:35) 2024-01-05 09:46:17 at endReadableNT (node:internal/streams/readable:1400:12) 2024-01-05 09:46:17 at process.processTicksAndRejections (node:internal/process/task_queues:82:21) { 2024-01-05 09:46:17 code: 'UND_ERR_SOCKET', 2024-01-05 09:46:17 socket: { 2024-01-05 09:46:17 localAddress: '172.17.0.2', 2024-01-05 09:46:17 localPort: 40518, 2024-01-05 09:46:17 remoteAddress: undefined, 2024-01-05 09:46:17 remotePort: undefined, 2024-01-05 09:46:17 remoteFamily: undefined, 2024-01-05 09:46:17 timeout: undefined, 2024-01-05 09:46:17 bytesWritten: 263, 2024-01-05 09:46:17 bytesRead: 0 2024-01-05 09:46:17 } 2024-01-05 09:46:17 } 2024-01-05 09:46:17 } 2024-01-05 09:46:17 2024-01-05 09:46:17 Node.js v18.19.0 ```
yindo commented 2026-02-22 18:18:55 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 5, 2024):

Okay, here is the root cause.

On AMD machines, Docker runs AnythingLLM in an x86 emulator on the container. This emulator overhead causes ONNX models to run very slowly if at all. This slowness also leads to timeouts on the HTTPS call to HF when downloading the models needed to embed,whisper, etc.

The fix for this is to publish an AMD64 image for AnythingLLM so that AMD machines can download an appropriate image and not run in a compatibility mode.

  • Implement arm64 build and publish for anythingllm docker image

This support causes issues with lanceDB, who require Rust binaries for it to run.

  • We need to bump LanceDB to a version that publishes support for linux_arm64 binaries so that the non-emulated docker container can use LanceDB.
  • puppeteer does not publish an arm86 compatible chromium with it - so we need to patch this as well.
@timothycarambat commented on GitHub (Jan 5, 2024): Okay, here is the root cause. On AMD machines, Docker runs AnythingLLM in an x86 emulator on the container. This emulator overhead causes ONNX models to run very slowly if at all. This slowness also leads to timeouts on the HTTPS call to HF when downloading the models needed to embed,whisper, etc. The fix for this is to publish an AMD64 image for AnythingLLM so that AMD machines can download an appropriate image and not run in a compatibility mode. - [ ] Implement `arm64` build and publish for anythingllm docker image This support causes issues with lanceDB, who require Rust binaries for it to run. - [ ] We need to bump LanceDB to a version that publishes support for `linux_arm64` binaries so that the non-emulated docker container can use LanceDB. - [ ] `puppeteer` does not publish an arm86 compatible chromium with it - so we need to patch this as well.
yindo changed title from Error: Unsupported model type: whisper to [GH-ISSUE #539] Error: Unsupported model type: whisper 2026-06-05 14:34:40 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label Docker bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#316
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?