mirror of
https://github.com/ollama/ollama-python.git
synced 2026-07-21 09:05:23 -04:00
[Errno 111] Connection refused while embedding in docker #163
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @LiZijunApril on GitHub (Nov 10, 2024).
Originally assigned to: @ParthSareen on GitHub.
For some reason I have to use ollama (version is 0.3.3) in my docker network and when I add two containers together and embed via ollama.embeddings it shows:
File "/home/test.py", line 22, in <module> ollama.embeddings(model='nomic-embed-text', prompt='The sky is blue because of rayleigh scattering') File "/usr/local/lib/python3.12/site-packages/ollama/_client.py", line 281, in embeddings return self._request( ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/ollama/_client.py", line 70, in _request response = self._client.request(method, url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 837, in request return self.send(request, auth=auth, follow_redirects=follow_redirects) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 926, in send response = self._send_handling_auth( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 954, in _send_handling_auth response = self._send_handling_redirects( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 991, in _send_handling_redirects response = self._send_single_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1027, in _send_single_request response = transport.handle_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 235, in handle_request with map_httpcore_exceptions(): File "/usr/local/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 89, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectError: [Errno 111] Connection refusedI run my code in docker, the LLM is correct but the embedding module is not. And I guess the error only happened when using ollama in docker network. My solution method is:
# host = split.hostname or '127.0.0.1' host = split.hostname or 'ollama'Path is '/usr/local/lib/python3.12/site-packages/ollama/_client.py', line 1033.
'ollama' is the container in docker network where the ollama serve is running.
I have to say the method is really ugly. I just want to know if there is better way.
@gabripo commented on GitHub (Nov 13, 2024):
it happens to me when just querying the available models with the /api/tag endpoint (docker-easyrag is my Docker container):
docker-easyrag | File "/home/easyrag/ollama_manager.py", line 17, in is_llm_available docker-easyrag | return llm_name in list_available_models() docker-easyrag | File "/home/easyrag/ollama_manager.py", line 7, in list_available_models docker-easyrag | response = requests.get(url) docker-easyrag | File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 73, in get docker-easyrag | return request("get", url, params=params, **kwargs) docker-easyrag | File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 59, in request docker-easyrag | return session.request(method=method, url=url, **kwargs) docker-easyrag | File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 589, in request docker-easyrag | resp = self.send(prep, **send_kwargs) docker-easyrag | File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 703, in send docker-easyrag | r = adapter.send(request, **kwargs) docker-easyrag | File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 700, in send docker-easyrag | raise ConnectionError(e, request=request) docker-easyrag | requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/tags (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffff80d68820>: Failed to establish a new connection: [Errno 111] Connection refused')) docker-easyrag | 172.19.0.1 - - [13/Nov/2024 22:25:00] "POST /streamlit-kill/17 HTTP/1.1" 200 - docker-easyrag | 172.19.0.1 - - [13/Nov/2024 22:25:00] "GET / HTTP/1.1" 200 -In the source file, I just call the API endpoint as per documentation:
def list_available_models() -> set[str]: url = "http://localhost:11434/api/tags" response = requests.get(url) if response.status_code == 200: models = response.json() model_names = {model["name"].split(":")[0] for model in models["models"]} return model_names else: return {}@ParthSareen commented on GitHub (Nov 14, 2024):
Hey! @LiZijunApril @gabripo I'll check this out and see what a good pattern should be. Thanks for bringing it up!
@gabripo commented on GitHub (Nov 15, 2024):
@ParthSareen I fixed the issue by manually substituting the http://localhost:11434 API call with http://host.docker.internal:11434
This will allow API calls from the Docker container to the shared network with host - where Ollama runs.
You can see the patch within the Dockerfile in my project:
https://github.com/gabripo/easyrag/
... in the commit:
https://github.com/gabripo/easyrag/commit/583803f1e1b3d2801f98dcd7ae391ac3d88c3226
P.S. you can try the package by yourself by building a Docker container and running it with the scripts:
On top of that I implemented a solution for the case Ollama runs in a container and a dockerized app calls the API, by exploiting the environmental variables to be set with Docker Compsoe, see the commit:
https://github.com/gabripo/easyrag/commit/ff430d4356033b07d673a9bc260bbc07485bea18
... and the source code.
@ParthSareen commented on GitHub (Nov 15, 2024):
Looked a bit into it - are you currently passing in the URL to
base_url=_parse_host(host or os.getenv('OLLAMA_HOST')),? https://github.com/ollama/ollama-python/blob/dc38fe467585037e02a4bc27a100cf6cb10202ff/ollama/_client.py#L90@ParthSareen commented on GitHub (Nov 25, 2024):
Closed as stale
@LiZijunApril commented on GitHub (Nov 28, 2024):
Thank you very much