mirror of
https://github.com/Mintplex-Labs/anything-llm.git
synced 2026-07-19 14:13:35 -04:00
[GH-ISSUE #1745] [BUG]: Ollama models not loading with http://172.17.0.1:11434 on Debian 12 #1137
Closed
opened 2026-02-22 18:23:18 -05:00 by yindo
·
11 comments
No Branch/Tag Specified
master
5846-bug-when-scrolling-up-scrolling-jumps
refactor-remove-workspace-pfp
5969-bug-stopgenerationbutton-disappears-on-the-first-prompt-that-initiates-an-agent-session
2235-bug-how-to-upload-a-folder-with-subfolders-with-files-to-anythingllm
5990-workspace-update-fails-with-unknown-argument-router_id-v1130v1150-intel-mac
opencomputer-examples
pg
feat/image-generation-translations
feat/image-generation
5924-bug-meeting-summary-fails-with-sincludes-is-not-a-function-when-default-llm-is-anthropic-claude
render
feat/uniform-modal-component
5883-bug-unescaped-content-in-json-strings-being-passed-to-document-generator-tools
5901-bug-api-update-embeddings-fails-prisma-argument-filename-is-missing-on-workspace_documentscreate-desktop-windows-v1141
hybrid-search
1981-translations
5752-bug-prompts-to-local-jan-endpoint-unresponsive
feat-disable-native-tool-calling-env-var
5676-bug-non-ollama-agent-providers-do-not-parse-and-present-reasoning-content
feat/markdown-web-scraping
5717-bug-apiv1documentupload-silently-drops-metadata-field-in-desktop-1130-arg-count-mismatch-nested-payload-key
fix/aibitat-context-overflow
5711-bug-erratic-deepseek-v4-flash-the-agent-model-failed-to-respond-400-the-reasoning_content
5631-feat-custom-api-request-timeouts-for-ai-providers
feat-reasoning-control
feat-agent-clarifying-questions-translations
5583-bug-lm-studio-provider-does-not-present-reasoning-output
5313-normalize-translations
feat/memory-translations
5305-lemonade-embedding-engine-swallows-errors-falsely-reports-documents-as-embedded
5060-bug-agent-interactions-agent-are-not-persisted-to-thread-history-via-api
pptx-subagent
feat-render-images-from-mcp-tool-results
stt-provider-expansion-openai-api-compatible
feat-file-search-agent-tool
feat-native-embedder-job-queue
5189-normalize-translations
feat-file-search-agent-tool-translations
i18n-eslint
5140-auto-migration
5112-bug-openrouter-failed-message-bug
3506-feat-parameters-for-openrouter-models
4992-feat-preserve-scroll-position
4973-bug-markdown-numbered-list-display-in-reasoning-pane
desktop
4938-bug-pending-chat-rerendering-ui-bug
quickstart-env
node-llama-cpp-in-container-cuda
node-llama-cpp-in-container
ollama-in-container
4817-feat-set-cooldown-per-mcp-server
4845-keyboard-shortcuts-to-navigate-in-chat
4844-feat-reorder-threads-by-latest-interaction
standardize-username-constraints-normalize-translations
4792-feat-refactor-workspacepfp-image
1382-embed-ip-improvements
1382-bug-embed-api-improvements
refactor-eslint-frontend
4687-feat-refactor-vector-db-providers
4615-feat-disable-apidocs-with-environment-variable
4559-feat-agent-web-search-enable-ordering-of-results
4599-bug-ollama-race-condition-bug
4572-bug-lmstudio-provided-llm-stopped-working-with-anything-llm-after-upgrading-to-190
4508-agent-youtube-transcript-analysis
4497-feat-workspace-names
frontend-eslint
ollama-lmstudio-auto-context-window
4431-validate-vector-database-connectioN
2019-slash-command-keyboard-selection
microsoft-foundry-provider
4431-validate-vector-database-connection
4325-sys-prompt-var-improvements
3209-feat-apiv1workspacestream-chat-sources-citations
4210-bug-voice-to-text-overwrite
4136-feat-jan-as-a-backend-server-option
4172-feat-openai-o3-support
1.8.3-rerelease
web-push-notifications-service
tasks
3955-feat-jinaai-embedder-provider-support
3921-feat-agent-skills-uiux-improvements
3901-bug-validfunccall-checks-optional-arguments
keyboard-dev
1787-custom-roles-and-permissions
add-jira-slack-data-connector
office-extension-wip
lightmode-dropdown-color-update
3586-bug-agent-flow-function-description-provided-by-user-is-not-seen-in-the-llm-query
3463-bug-agent-continues-to-run-if-request-failed-even-after-exit
3439-feat-call-variables-within-the-flow-api-block-url-field
3282-manager-view-models-workspace
3280-token-counting-server-side-truncation-improvements
3147-bug-embedded-chat-widget---not-considering-query-mode-option-always-working-in-chat-mode
2995-feat-disable-temperature-setting-for-deepseek-r1-deepseek-reasoner-model
2827-feat-perplexity-citations
2866-feat-finally-a-gemini-models-endpoint
2647-feat-hpp-header-for-a-c++-code-file-mime-addition
lancedb-revert
1656-feat-implement-tooltip-ui-designs
2011-feat-bump-perplexity-models
1873-feat-auto-add-and-watch-folder-for-document-uploads
1297-feat-gemini-agent-support
1759-bug-ui-bug-fixes
1686-feat-implement-winston-for-logging
1536-bug-toggling-on-users-can-delete-workspaces-does-not-take-effect
agent-ui-mobile-styles
1522-feat-chromadb-support
1595-bug-unable-to-get-live-web-search-and-browsing-agent-working-using-google-custom-search-engine-error-getaddrinfo-enotfound-http-errno-3008
1582-bug-lm-studio-does-not-allow-for-different-model-selection
1312-bug-usernames-should-not-be-case-sensitive-when-logging-in
1029-feat-hf-serverless-inference-api
1086-feat-implement-normalized-input-fields
knowledge-graph-support
644-bug-uploaded-file-name-does-not-match-the-displayed-file-name-after-the-upload
v1.15.0
v1.14.2
v1.14.1
v1.14.0
v1.13.0
v1.12.1
v1.12.0
v1.11.2
v1.11.1
v1.11.0
v1.10.0
v1.9.1
v1.9.0
v1.8.5
v1.8.4
v1.8.3
v1.8.2
v1.8.1
v1.8.0
v1.7.8
v1.7.6
v1.7.5
v1.7.4
v1.4.0
v1.3.0
v1.2.4
v1.2.3
v1.2.2
v1.2.1
v1.2.0
v1.1.1
v1.1.0
v1.0.0
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
question
stage: specifications
wontfix
Mirrored from GitHub Pull Request
No Label
possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: Mintplex-Labs/anything-llm#1137
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Mattias-Code on GitHub (Jun 22, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1745
How are you running AnythingLLM?
Docker (local)
What happened?
Stuck at loading Ollama models, verified that Ollama is running on 127.0.0.1:11434
and used 172.17.0.1:11434 as url according to documentation.
AnythingLLM logs:
[NativeEmbedder] Initialized
TypeError: fetch failed
at Object.fetch (node:internal/deps/undici/undici:11731:11)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async ollamaAIModels (/app/server/utils/helpers/customModels.js:245:18)
at async getCustomModels (/app/server/utils/helpers/customModels.js:33:14)
at async /app/server/endpoints/system.js:925:35 {
cause: Error: connect ECONNREFUSED 172.17.0.1:11434
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) {
errno: -111,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '172.17.0.1',
port: 11434
}
}
Really appreciate the job that you guys put down to make this software a reality. I've had a flawless time using AnythingLLM with Anthropic API!
Are there known steps to reproduce?
No response
@alex-dna-tech commented on GitHub (Jun 23, 2024):
The same problem on MacOS Desktop version v1.5.7

@timothycarambat commented on GitHub (Jun 23, 2024):
The connector does work, you just are using the wrong URL or Ollama is actually not running on the host.
assuming both are on the same machine and you _confirmed that
ollama serveis running.If on Docker:
However, on Ubuntu/Linux docker the above URL does not work and you need to use 172.17.0.1:11434. If this is not your situation you dont need to use this URL. This also presumes you have not already tried to boot Ollama with the proper ENVs for ollama to be able to listen on the correct loopback. 172.17.0.1 = 0.0.0.0
On desktop:
HTTP://127.0.01:11434
Things to read about this:
https://docs.useanything.com/ollama-models-not-loading
https://github.com/Mintplex-Labs/anything-llm/blob/master/server/utils/AiProviders/ollama/README.md
@Mattias-Code commented on GitHub (Jun 24, 2024):
After following the documentation at https://github.com/Mintplex-Labs/anything-llm/blob/master/server/utils/AiProviders/ollama/README.md I managed to resolve the issue by binding Ollama to 0.0.0.0 and using 172.17.0.1:11434 as base URL.
Thank you. For future problems I'll take a closer look at the github before creating a new issue!
@timothycarambat commented on GitHub (Jun 24, 2024):
@Mattias-Code Thanks for the reply + resolution. Also no big deal, we get Ollama connection issues about 5-10 times a week because of that caveat on address binding Ollama has - you are not alone!
@QRMarketing commented on GitHub (Aug 17, 2024):
Not sure whether this is an issue or not - since this is closed. But ... I read all the links the env variables are all ok. Every time I restart the server the only way I have been able to wake up ollama is to go and get it to auto detect the end point and save it. Quite annoying ... but it is what it is.
I notice a JavaScript error on the save - so maybe it doesn't really save it - so the server restart doesn't remember. Though the toast say it was saved.
index-a13552ba.js:203 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'preventDefault')
at handleSubmit (index-a13552ba.js:203:7)
function GeneralLLMPreference() {
var _a, _b;
const [saving, setSaving] = reactExports.useState(false);
const [hasChanges, setHasChanges] = reactExports.useState(false);
const [settings, setSettings] = reactExports.useState(null);
const [loading, setLoading] = reactExports.useState(true);
const [searchQuery, setSearchQuery] = reactExports.useState("");
const [filteredLLMs, setFilteredLLMs] = reactExports.useState([]);
const [selectedLLM, setSelectedLLM] = reactExports.useState(null);
const [searchMenuOpen, setSearchMenuOpen] = reactExports.useState(false);
const searchInputRef = reactExports.useRef(null);
const isHosted = window.location.hostname.includes("useanything.com");
const { t } = useTranslation();
const handleSubmit = async (e) => { // e is undefined
e.preventDefault();
const form = e.target;
const data = { LLMProvider: selectedLLM };
const formData = new FormData(form);
for (var [key, value] of formData.entries())
data[key] = value;
const { error } = await System.updateSystem(data);
setSaving(true);
if (error) {
showToast(
Failed to save LLM settings: ${error}, "error");} else {
showToast("LLM preferences saved successfully.", "success");
}
setSaving(false);
setHasChanges(!!error);
};
@QRMarketing commented on GitHub (Aug 17, 2024):
For those that care. I did fix this. Yes, I set up the environment variables and the .env to no avail. I suppose there is a real fix either in the code or my environment. But I did a hack in the server at startup to get it to work:
In server.js near start of the script:
const { ResetOllama } = require('./ollamareset.js'); /* kluge wake up ollama */ setTimeout(async () => { await ResetOllama(); }, 1000);And ollamareset.js':
`
const { updateENV } = require("./utils/helpers/updateENV");
async function ResetOllama() {
}
module.exports = {
ResetOllama
};
`
@timothycarambat commented on GitHub (Aug 18, 2024):
If you have to do that everytime you start the app then the app is not saving information to the .env file? Are you in Docker?
The error with the no preventDefault is especially off, since that is just a handler for the form, where
ewould for sure exists so we need to check that out as well. What is your environment?@timothycarambat commented on GitHub (Aug 18, 2024):
https://github.com/Mintplex-Labs/anything-llm/blob/a8d25c7dd31e90b06c9ea8575ada2321928dcfa4/frontend/src/pages/GeneralSettings/LLMPreference/index.jsx#L341
@QRMarketing commented on GitHub (Aug 18, 2024):
STOP wasting your time researching my Agent Configuration problem - it was my stupidity in not setting the Agent Configuration correctly. The issues of the Javascript errors and my kluge fix is another matter. With all this it is now functioning.
Below is my .env same as well as my Windows environment variables. Yes stand alone on Windows. Everything works, Embed widget and workspace chats, etc ... (with the kluge fix - but i removed it and it works as well, you just need to resave the LLM preference each server restart to wake it up.). The only thing that does not work, which is all related to this point; I discovered is Agents (yes refreshed source - I see you have made recent changes) see below. Why the other functions see ollama but not Agents?
There seems to be a discussion over this OLLAMA_BASE_PATH='http://127.17.0.1:11434' or OLLAMA_BASE_PATH='http://127.0.0.1:11434'. http://127.17.0.1:11434 nothing works....
As far as the Javascript error you mention. I put a break point on it. It seems to get there twice. First time with e undefined and the second time ok. So the function actually works (i.e. saves the LLM selection.)
Any help would be great.
The error log:
PS C:\Users\User\Desktop\Any\anything-llm\server> [TELEMETRY SENT] sent_chat
sent_chat { workspaceName: 'ecologene', chatModel: 'llama3.1:latest' } User: 1
[AgentHandler] Start 88a7d05f-baab-49de-aacb-d7d554ff9fba::ollama:llama3:latest
[TELEMETRY SENT] agent_chat_started
[AgentHandler] Attached websocket plugin to Agent cluster
[AgentHandler] Attached chat-history plugin to Agent cluster
[AgentHandler] Attaching user and default agent to Agent cluster.
[AgentHandler] Attached rag-memory plugin to Agent cluster
[AgentHandler] Attached document-summarizer plugin to Agent cluster
[AgentHandler] Attached web-scraping plugin to Agent cluster
[AgentHandler] Attached create-chart plugin to Agent cluster
model "llama3:latest" not found, try pulling it first ResponseError: model "llama3:latest" not found, try pulling it first
at checkOk (C:\Users\User\Desktop\Any\anything-llm\server\node_modules\ollama\dist\shared\ollama.a247cdd6.cjs:38:11)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async post (C:\Users\User\Desktop\Any\anything-llm\server\node_modules\ollama\dist\shared\ollama.a247cdd6.cjs:86:3)
at async Ollama.processStreamableRequest (C:\Users\User\Desktop\Any\anything-llm\server\node_modules\ollama\dist\shared\ollama.a247cdd6.cjs:183:22)
at async #handleFunctionCallChat (C:\Users\User\Desktop\Any\anything-llm\server\utils\agents\aibitat\providers\ollama.js:29:22)
at async OllamaProvider.functionCall (C:\Users\User\Desktop\Any\anything-llm\server\utils\agents\aibitat\providers\helpers\untooled.js:110:22)
at async OllamaProvider.complete (C:\Users\User\Desktop\Any\anything-llm\server\utils\agents\aibitat\providers\ollama.js:50:36)
at async AIbitat.handleExecution (C:\Users\User\Desktop\Any\anything-llm\server\utils\agents\aibitat\index.js:592:24)
at async AIbitat.reply (C:\Users\User\Desktop\Any\anything-llm\server\utils\agents\aibitat\index.js:574:21)
at async AIbitat.chat (C:\Users\User\Desktop\Any\anything-llm\server\utils\agents\aibitat\index.js:374:15) {
error: 'model "llama3:latest" not found, try pulling it first',
status_code: 404
}
[AgentHandler] End 88a7d05f-baab-49de-aacb-d7d554ff9fba::ollama:llama3:latest
My .env
######## ANYTHINGLLM SETTINGS ############
LLM_PROVIDER='ollama'
OLLAMA_HOST='0.0.0.0'
OLLAMA_BASE_PATH='http://127.0.0.1:11434'
OLLAMA_MODEL_PREF='llama3:latest'
OLLAMA_MODEL_TOKEN_LIMIT='4096'
OLLAMA_KEEP_ALIVE_TIMEOUT='300'
@timothycarambat commented on GitHub (Aug 19, 2024):
Seems like the connection works fine.
The model you have set for agents (if set at all) is not installed in that Ollama service so it can't invoke llama 3.
ollama pull llama3:latestif that is what you are tying to run.This sets the model you want to use, it does not download it for you in Ollama. You still need to run
ollama pull llama3:latest@Kentauros10710 commented on GitHub (Dec 9, 2024):
Based on my own testing, I ultimately managed it by downloading Ollama using Docker as well. Since my Docker environment is running both Anything LLM and Ollama simultaneously, they are able to interact and integrate with each other.
[BUG]: Ollama models not loading with http://172.17.0.1:11434 on Debian 12to [GH-ISSUE #1745] [BUG]: Ollama models not loading with http://172.17.0.1:11434 on Debian 12