Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2983] [BUG]: Cannot enable local whisper STT #1907

New Issue
Closed
opened 2026-02-22 18:27:10 -05:00 by yindo · 8 comments
yindo commented 2026-02-22 18:27:10 -05:00
Owner
Copy Link
Copy Source

Originally created by @ethanfrey on GitHub (Jan 17, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2983

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

This is similar to #2651 but I have ruled out HF connectivity issues and even attempted to manually download the whisper model.

Which I click on the STT icon, I get the error message:
Cannot find file "http://localhost:3001/static/stt/models/Xenova/whisper-tiny.en/tokenizer.json".

I discovered the internal storage is available (under Linux) at: $HOME/.config/anythingllm-desktop/storage. In particular, it seems the whisper stuff is available at: $HOME/.config/anythingllm-desktop/storage/models/Xenova

I manually copied the whisper model with the following commands

git lfs install

cd $HOME/.config/anythingllm-desktop/storage/models/Xenova
git clone https://huggingface.co/Xenova/whisper-tiny whisper-tiny.en

And the files are there

~/.config/anythingllm-desktop/storage$ du -h models/Xenova/whisper-tiny.en/tokenizer.json 
2,4M	models/Xenova/whisper-tiny.en/tokenizer.json

But I still get the same error message (even after restarting AnythingLLM).

I cannot find any better error messages. I am technical and happy to download models or configure any text files, just would love to see the voice interaction work

Are there known steps to reproduce?

No response

Originally created by @ethanfrey on GitHub (Jan 17, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2983 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? This is similar to #2651 but I have ruled out HF connectivity issues and even attempted to manually download the whisper model. Which I click on the STT icon, I get the error message: `Cannot find file "http://localhost:3001/static/stt/models/Xenova/whisper-tiny.en/tokenizer.json".` I discovered the internal storage is available (under Linux) at: `$HOME/.config/anythingllm-desktop/storage`. In particular, it seems the whisper stuff is available at: `$HOME/.config/anythingllm-desktop/storage/models/Xenova` I manually copied the whisper model with the following commands ```bash git lfs install cd $HOME/.config/anythingllm-desktop/storage/models/Xenova git clone https://huggingface.co/Xenova/whisper-tiny whisper-tiny.en ``` And the files are there ``` ~/.config/anythingllm-desktop/storage$ du -h models/Xenova/whisper-tiny.en/tokenizer.json 2,4M models/Xenova/whisper-tiny.en/tokenizer.json ``` But I still get the same error message (even after restarting AnythingLLM). I cannot find any better error messages. I am technical and happy to download models or configure any text files, just would love to see the voice interaction work ### Are there known steps to reproduce? _No response_
yindo added the Desktopinvestigating labels 2026-02-22 18:27:10 -05:00
yindo closed this issue 2026-02-22 18:27:10 -05:00
yindo commented 2026-02-22 18:27:10 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 17, 2025):

Any possibility the user running AnythingLLM cannot read those files due to permissions?

Additionally, while the app is running - does going to http://localhost:3001/static/stt/models/Xenova/whisper-tiny.en/tokenizer.json load a JSON response?

If not - the file is not readable/parseable by the server and may generate a log that can be read
https://docs.anythingllm.com/installation-desktop/debug

@timothycarambat commented on GitHub (Jan 17, 2025): Any possibility the user running AnythingLLM cannot read those files due to permissions? Additionally, while the app is running - does going to http://localhost:3001/static/stt/models/Xenova/whisper-tiny.en/tokenizer.json load a JSON response? If not - the file is not readable/parseable by the server and may generate a log that can be read https://docs.anythingllm.com/installation-desktop/debug
yindo commented 2026-02-22 18:27:10 -05:00
Author
Owner
Copy Link
Copy Source

@ethanfrey commented on GitHub (Jan 20, 2025):

Thanks for the quick response, I was off the weekend and just testing now.

Going to http://localhost:3001/static/stt/models/Xenova/whisper-tiny.en/tokenizer.json gives me:
Cannot GET /static/stt/models/Xenova/whisper-tiny.en/tokenizer.json

I looked inside ~/.config/anythingllm-desktop/ and the storage subdirs and such, and everything is owned by ethan (my user) and I run anythingllm as my user from the terminal, so there should not be a user permission mismatch.

I am having trouble getting debug info. I checked ~/.config/anythingllm-desktop/storage/logs as per the debug guide and it just has the startup info, nothing new when I keep hitting the tokenizer.json url.

{"level":"info","message":"Skipping preloading of AnythingLLMOllama - LLM_PROVIDER is ollama.","service":"backend"}
{"level":"info","message":"\u001b[36m[CommunicationKey]\u001b[0m RSA key pair generated for signed payloads within AnythingLLM services.","service":"backend"}
{"level":"info","message":"\u001b[36m[EncryptionManager]\u001b[0m Loaded existing key & salt for encrypting arbitrary data.","service":"backend"}
{"level":"info","message":"[production] AnythingLLM Standalone Backend listening on port 33969. Network discovery is disabled.","service":"backend"}
{"level":"info","message":"\u001b[36m[BackgroundWorkerService]\u001b[0m Feature is not enabled and will not be started.","service":"backend"}

However, I was unable to follow these debug instructions as the file was not there:

To run the AnythingLLM Application in debug mode (if you are getting errors) you can open a Terminal and navigate to ~/.config/anythingllm-desktop/AnythingLLMDesktop.AppImage where you can then run the executable and see all AppImage logs while running AnythingLLM.

~/.config/anythingllm-desktop has no executable. I find it however, in ~/AnythingLLMDesktop/anythingllm-desktop/AppRun which then calls into ~/AnythingLLMDesktop/anythingllm-desktop/anythingllm-desktop:

$ du -h anythingllm-desktop
161M	anythingllm-desktop
$ file anythingllm-desktop
anythingllm-desktop: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=da0fd670ca5cf8aa6fedcdcee4861fe22ce72e68, not stripped

I installed according to https://docs.anythingllm.com/installation-desktop/linux

I couldn't figure out any flags to add to the anythingllm-desktop to increase the log level

@ethanfrey commented on GitHub (Jan 20, 2025): Thanks for the quick response, I was off the weekend and just testing now. Going to http://localhost:3001/static/stt/models/Xenova/whisper-tiny.en/tokenizer.json gives me: `Cannot GET /static/stt/models/Xenova/whisper-tiny.en/tokenizer.json` I looked inside `~/.config/anythingllm-desktop/` and the storage subdirs and such, and everything is owned by `ethan` (my user) and I run anythingllm as my user from the terminal, so there should not be a user permission mismatch. I am having trouble getting debug info. I checked `~/.config/anythingllm-desktop/storage/logs` as per [the debug guide](https://docs.anythingllm.com/installation-desktop/debug) and it just has the startup info, nothing new when I keep hitting the tokenizer.json url. ``` {"level":"info","message":"Skipping preloading of AnythingLLMOllama - LLM_PROVIDER is ollama.","service":"backend"} {"level":"info","message":"\u001b[36m[CommunicationKey]\u001b[0m RSA key pair generated for signed payloads within AnythingLLM services.","service":"backend"} {"level":"info","message":"\u001b[36m[EncryptionManager]\u001b[0m Loaded existing key & salt for encrypting arbitrary data.","service":"backend"} {"level":"info","message":"[production] AnythingLLM Standalone Backend listening on port 33969. Network discovery is disabled.","service":"backend"} {"level":"info","message":"\u001b[36m[BackgroundWorkerService]\u001b[0m Feature is not enabled and will not be started.","service":"backend"} ``` However, I was unable to follow these [debug instructions](https://docs.anythingllm.com/installation-desktop/debug) as the file was not there: ``` To run the AnythingLLM Application in debug mode (if you are getting errors) you can open a Terminal and navigate to ~/.config/anythingllm-desktop/AnythingLLMDesktop.AppImage where you can then run the executable and see all AppImage logs while running AnythingLLM. ``` ` ~/.config/anythingllm-desktop` has no executable. I find it however, in `~/AnythingLLMDesktop/anythingllm-desktop/AppRun` which then calls into `~/AnythingLLMDesktop/anythingllm-desktop/anythingllm-desktop`: ```bash $ du -h anythingllm-desktop 161M anythingllm-desktop $ file anythingllm-desktop anythingllm-desktop: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=da0fd670ca5cf8aa6fedcdcee4861fe22ce72e68, not stripped ``` I installed according to https://docs.anythingllm.com/installation-desktop/linux I couldn't figure out any flags to add to the anythingllm-desktop to increase the log level
yindo commented 2026-02-22 18:27:10 -05:00
Author
Owner
Copy Link
Copy Source

@ethanfrey commented on GitHub (Jan 20, 2025):

Digging in more, I realized the server declared itself as listening to 33969, but I am hitting port 3001 looking for the model.

Using netstat and docker, I find this is:

6df478a1f11c itzcrazykns1337/perplexica-backend:main "docker-entrypoint.s…" 5 days ago Up 24 minutes 0.0.0.0:3001->3001/tcp, :::3001->3001/tcp perplexica-perplexica-backend-1

Which I had installed and tested earlier and doesn't seem to have anything to do with AnythingLLM.
I shut down that service and suddenly get a very nice long JSON file at http://localhost:3001/static/stt/models/Xenova/whisper-tiny.en/tokenizer.json

And netstat show this process listening on 3001: /home/ethan/AnythingLLMDesktop/anythingllm-desktop/anythingllm-desktop

So I guess it was user error on my side. I guess it would be nice to have a bigger warning message if something else was blocking 3001 (which is not an uncommon port). But I should also double-check services when running.

The logs say:

{"level":"info","message":"[production] AnythingLLM Standalone Backend listening on port 3001. Network discovery is disabled.","service":"backend"}
{"level":"info","message":"\u001b[36m[BackgroundWorkerService]\u001b[0m Feature is not enabled and will not be started.","service":"backend"}
{"level":"info","message":"\u001b[36m[NativeEmbedder]\u001b[0m Initialized","service":"backend"}

Which seems to suggest that it picks a random port (eg 33969) if it cannot bind to 3001.

@ethanfrey commented on GitHub (Jan 20, 2025): Digging in more, I realized the server declared itself as listening to 33969, but I am hitting port 3001 looking for the model. Using netstat and docker, I find this is: `6df478a1f11c itzcrazykns1337/perplexica-backend:main "docker-entrypoint.s…" 5 days ago Up 24 minutes 0.0.0.0:3001->3001/tcp, :::3001->3001/tcp perplexica-perplexica-backend-1` Which I had installed and tested earlier and doesn't seem to have anything to do with AnythingLLM. I shut down that service and suddenly get a very nice long JSON file at http://localhost:3001/static/stt/models/Xenova/whisper-tiny.en/tokenizer.json And `netstat` show this process listening on 3001: `/home/ethan/AnythingLLMDesktop/anythingllm-desktop/anythingllm-desktop` So I guess it was user error on my side. I guess it would be nice to have a bigger warning message if something else was blocking 3001 (which is not an uncommon port). But I should also double-check services when running. The logs say: ``` {"level":"info","message":"[production] AnythingLLM Standalone Backend listening on port 3001. Network discovery is disabled.","service":"backend"} {"level":"info","message":"\u001b[36m[BackgroundWorkerService]\u001b[0m Feature is not enabled and will not be started.","service":"backend"} {"level":"info","message":"\u001b[36m[NativeEmbedder]\u001b[0m Initialized","service":"backend"} ``` Which seems to suggest that it picks a random port (eg 33969) if it cannot bind to 3001.
yindo commented 2026-02-22 18:27:10 -05:00
Author
Owner
Copy Link
Copy Source

@ethanfrey commented on GitHub (Jan 20, 2025):

Interesting, however, if that even though I have voice and speech pointing to the local whisper model, and I click the voice button without any errors this time. But nothing happens. No events are logged to backend-<date>.log like with sent_chat or rag. And no more memory is claimed in nvidia compute (cuda) for the whisper model.

I removed my manually installed whisper, restarted anythingllm, and did it again. Nothing happened either. When I checked http://localhost:3001/static/stt/models/Xenova/whisper-tiny.en/tokenizer.json it showed me the tokenizer (and made the file in ~/.config/anythingllm-desktop/storage/models/Xenova/whisper-tiny.en/tokenizer.json but no more files in that directory)

Unless you have some great inspiration, I am going to leave off debugging this for now, as voice was a nice to have an not essential. But I hope this can be useful (like giving an error if unable to bind port 3001). And if you have any further insights or things I need to adjust, I am happy to play with this. For now, I'm going to experiment with RAG and the agent tools.

@ethanfrey commented on GitHub (Jan 20, 2025): Interesting, however, if that even though I have voice and speech pointing to the local whisper model, and I click the voice button without any errors this time. But nothing happens. No events are logged to `backend-<date>.log` like with sent_chat or rag. And no more memory is claimed in nvidia compute (cuda) for the whisper model. I removed my manually installed whisper, restarted anythingllm, and did it again. Nothing happened either. When I checked http://localhost:3001/static/stt/models/Xenova/whisper-tiny.en/tokenizer.json it showed me the tokenizer (and made the file in `~/.config/anythingllm-desktop/storage/models/Xenova/whisper-tiny.en/tokenizer.json` but no more files in that directory) Unless you have some great inspiration, I am going to leave off debugging this for now, as voice was a nice to have an not essential. But I hope this can be useful (like giving an error if unable to bind port 3001). And if you have any further insights or things I need to adjust, I am happy to play with this. For now, I'm going to experiment with RAG and the agent tools.
yindo commented 2026-02-22 18:27:11 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 20, 2025):

Ah, so the reason the server moved ports is that on boot we dont force our bind on 3001. We prefer that port, but if it is occupied in this case then we hunt for a free port in a nearby open range. That explains that.

That being said, the localwhisper stuff all runs in a web-worker on the frontend, not the backend - we simple source the files from the backend since the web-worker needs them. I will have to investigate on my side to see if I can replicate as there are also microphone permissions and other concerns that are not obvious to debug but could be the root cause as well.

Going to close for now, but keep investigating tag on so we can come back to it

@timothycarambat commented on GitHub (Jan 20, 2025): Ah, so the reason the server moved ports is that on boot we dont _force_ our bind on 3001. We _prefer_ that port, but if it is occupied in this case then we hunt for a free port in a nearby open range. That explains that. That being said, the localwhisper stuff all runs in a web-worker on the frontend, not the backend - we simple source the files from the backend since the web-worker needs them. I will have to investigate on my side to see if I can replicate as there are _also_ microphone permissions and other concerns that are not obvious to debug but could be the root cause as well. Going to close for now, but keep _investigating_ tag on so we can come back to it
yindo commented 2026-02-22 18:27:11 -05:00
Author
Owner
Copy Link
Copy Source

@ethanfrey commented on GitHub (Jan 21, 2025):

Thank you for the explanation and it doesn't seem like any concrete action needed on your side.

One thing I did notice is the log line of BackgroundWorkerService: feature is not enabled. Maybe that is required?

Also, the UI does look like a web view and I keep trying to open up the inspect pane on it to see console log or such. It would be great to be able to do so. Maybe there is some config setting (or build time config) to enable that?

@ethanfrey commented on GitHub (Jan 21, 2025): Thank you for the explanation and it doesn't seem like any concrete action needed on your side. One thing I did notice is the log line of `BackgroundWorkerService: feature is not enabled`. Maybe that is required? Also, the UI does look like a web view and I keep trying to open up the inspect pane on it to see console log or such. It would be great to be able to do so. Maybe there is some config setting (or build time config) to enable that?
yindo commented 2026-02-22 18:27:11 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 21, 2025):

BackgroundWorkerService: feature is not enabled

This is for the local document automatic sync - it is unrelated.

@timothycarambat commented on GitHub (Jan 21, 2025): > BackgroundWorkerService: feature is not enabled This is for the local document automatic sync - it is unrelated.
yindo commented 2026-02-22 18:27:11 -05:00
Author
Owner
Copy Link
Copy Source

@fabapp2 commented on GitHub (Feb 24, 2025):

Ah, so the reason the server moved ports is that on boot we dont force our bind on 3001. We prefer that port, but if it is occupied in this case then we hunt for a free port in a nearby open range. That explains that.

That being said, the localwhisper stuff all runs in a web-worker on the frontend, not the backend - we simple source the files from the backend since the web-worker needs them. I will have to investigate on my side to see if I can replicate as there are also microphone permissions and other concerns that are not obvious to debug but could be the root cause as well.

Going to close for now, but keep investigating tag on so we can come back to it

For me , being on AnythingLLM 1.7.4, the problem was resolved after stopping the process that occupied 3001.
It seems there's something not working when 3001 is occupied (at least for me).

Thanks for AnythingLLM!

@fabapp2 commented on GitHub (Feb 24, 2025): > Ah, so the reason the server moved ports is that on boot we dont _force_ our bind on 3001. We _prefer_ that port, but if it is occupied in this case then we hunt for a free port in a nearby open range. That explains that. > > That being said, the localwhisper stuff all runs in a web-worker on the frontend, not the backend - we simple source the files from the backend since the web-worker needs them. I will have to investigate on my side to see if I can replicate as there are _also_ microphone permissions and other concerns that are not obvious to debug but could be the root cause as well. > > Going to close for now, but keep _investigating_ tag on so we can come back to it For me , being on AnythingLLM 1.7.4, the problem was resolved after stopping the process that occupied 3001. It seems there's something not working when 3001 is occupied (at least for me). Thanks for AnythingLLM!
yindo changed title from [BUG]: Cannot enable local whisper STT to [GH-ISSUE #2983] [BUG]: Cannot enable local whisper STT 2026-06-05 14:43:23 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label Desktop investigating
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1907
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?