mirror of
https://github.com/ollama/ollama-python.git
synced 2026-07-21 09:05:23 -04:00
After changing OLLAMA_HOST=0.0.0.0, ollama.chat() and ollama.generate() not work #236
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @hkbb2014 on GitHub (Mar 16, 2025).
Before setting OLLAMA_HOST, ollama.generate() was working.
response = ollama.generate(model=model, prompt=prompt, stream=True)
After setting OLLAMA_HOST=0.0.0.0, ollama.generate() did not work at all.
But directly connecting to http://localhost:11434/api/generate without python work well.
How can I fix this?
Traceback (most recent call last):
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_transports\default.py", line 101, in map_httpcore_exceptions
yield
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_transports\default.py", line 250, in handle_request
resp = self._pool.handle_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_sync\connection_pool.py", line 256, in handle_request
raise exc from None
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_sync\connection_pool.py", line 236, in handle_request
response = connection.handle_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_sync\connection.py", line 101, in handle_request
raise exc
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_sync\connection.py", line 78, in handle_request
stream = self._connect(request)
^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_sync\connection.py", line 124, in _connect
stream = self._network_backend.connect_tcp(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_backends\sync.py", line 207, in connect_tcp
with map_exceptions(exc_map):
^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\contextlib.py", line 158, in exit
self.gen.throw(value)
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectError: [WinError 10049] 內容中所要求的位址不正確。
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "x:\pythonProjects\llmDemo\1_ollama\1_generate\generate_python.py", line 35, in
completion_with_ollama(prompt)
File "x:\pythonProjects\llmDemo\1_ollama\1_generate\generate_python.py", line 18, in completion_with_ollama
for chunk in response:
^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\ollama_client.py", line 163, in inner
with self._client.stream(*args, **kwargs) as r:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\contextlib.py", line 137, in enter
return next(self.gen)
^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_client.py", line 868, in stream
response = self.send(
^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_client.py", line 914, in send
response = self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_client.py", line 942, in _send_handling_auth
response = self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_client.py", line 979, in _send_handling_redirects
response = self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_client.py", line 1014, in _send_single_request
response = transport.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_transports\default.py", line 249, in handle_request
with map_httpcore_exceptions():
^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\contextlib.py", line 158, in exit
self.gen.throw(value)
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_transports\default.py", line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectError: [WinError 10049] 內容中所要求的位址不正確。
@hkbb2014 commented on GitHub (Mar 16, 2025):
I print the request in httpx _client.py, it seems that it is making request to http://0.0.0.0:11434/api/generate
<Request('POST', 'http://0.0.0.0:11434/api/generate')>
I guess this is the reason, how can I specify the default host for for ollama.generate()
It seems that langChain's ChatOllama and OllamaLLM also have the same problem. How to fix this other than specifying other OLLAMA_HOST?
@nickdickinson commented on GitHub (Mar 18, 2025):
I can confirm I have the same problem (windows).
I believe this is a bug but not sure why it does this. It is common to have OLLAMA_HOST set to 0.0.0.0 and that value is commonly used by other software such as Continue.dev in VSCode for code completion etc.
@hkbb2014 My workaround at the moment is to do the following in a script and to import that BEFORE importing ollama.
os.environ['OLLAMA_HOST'] = 'http://127.0.0.1:11434'
import ollama
A few ways to do this in a project:
@hkbb2014 commented on GitHub (Mar 18, 2025):
@nickdickinson set OLLAMA_HOST before importing ollama works. It works well for langchain too, thanks!
@ParthSareen commented on GitHub (Mar 18, 2025):
Are you also serving ollama with
OLLAMA_HOSTset?export OLLAMA_HOST=...ollama serve@nickdickinson commented on GitHub (Mar 18, 2025):
@ParthSareen Yes, but the global system env value is 0.0.0.0 which does not work in the Python package (at least on my Windows system). It does work when I explicitly set it to http://127.0.0.1:11434/
@ParthSareen commented on GitHub (Mar 18, 2025):
Got it, thanks @nickdickinson. Will dig in
@ParthSareen commented on GitHub (Apr 9, 2025):
@nickdickinson
This is expected behavior - probably surfaced with the httpx bump from a bit ago.
OLLAMA_HOSTshould not be0.0.0.0from the client side.See:
https://github.com/ollama/ollama-python/pull/491#issuecomment-2790636938