Breaking Change for langchain-ollama==0.2.1 on ollama==0.4.3 #181

New Issue

Closed

opened 2026-02-15 16:28:31 -05:00 by yindo · 6 comments

yindo commented 2026-02-15 16:28:31 -05:00

Owner

Copy Link

Copy Source

Originally created by @ross-harrison-fox on GitHub (Dec 7, 2024).

Receive Validation code when running the bellow snippet with the following package combination:

• langchain==0.3.10
• ollama==0.4.3
• langchain-ollama==0.2.1

# get embedding and LLM models
from langchain_ollama import OllamaEmbeddings, OllamaLLM

embeddings = OllamaEmbeddings(model="llama3", base_url="http://localhost:11434")
llm = OllamaLLM(model="llama3", format='', base_url="http://localhost:11434", verbose=True)

prompt = "Hello There"
response = llm.invoke(prompt)

Expected Behavior:

Some response

Actual:

See validation error

19 prompt = "Hello There"
---> 21 response = llm.invoke(prompt)

File ~\miniconda3\Lib\site-packages\langchain_core\language_models\chat_models.py:286, in BaseChatModel.invoke(self, input, config, stop, **kwargs)
    275 def invoke(
    276     self,
    277     input: LanguageModelInput,
   (...)
    281     **kwargs: Any,
    282 ) -> BaseMessage:
    283     config = ensure_config(config)
    284     return cast(
    285         ChatGeneration,
--> 286         self.generate_prompt(
    287             [self._convert_input(input)],
    288             stop=stop,
    289             callbacks=config.get("callbacks"),
    290             tags=config.get("tags"),
    291             metadata=config.get("metadata"),
    292             run_name=config.get("run_name"),
    293             run_id=config.pop("run_id", None),
    294             **kwargs,
    295         ).generations[0][0],
    296     ).message

File ~\miniconda3\Lib\site-packages\langchain_core\language_models\chat_models.py:786, in BaseChatModel.generate_prompt(self, prompts, stop, callbacks, **kwargs)
    778 def generate_prompt(
    779     self,
    780     prompts: list[PromptValue],
   (...)
    783     **kwargs: Any,
    784 ) -> LLMResult:
    785     prompt_messages = [p.to_messages() for p in prompts]
--> 786     return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)

File ~\miniconda3\Lib\site-packages\langchain_core\language_models\chat_models.py:643, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs)
    641         if run_managers:
    642             run_managers[i].on_llm_error(e, response=LLMResult(generations=[]))
--> 643         raise e
    644 flattened_outputs = [
    645     LLMResult(generations=[res.generations], llm_output=res.llm_output)  # type: ignore[list-item]
    646     for res in results
    647 ]
    648 llm_output = self._combine_llm_outputs([res.llm_output for res in results])

File ~\miniconda3\Lib\site-packages\langchain_core\language_models\chat_models.py:633, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs)
    630 for i, m in enumerate(messages):
    631     try:
    632         results.append(
--> 633             self._generate_with_cache(
    634                 m,
    635                 stop=stop,
    636                 run_manager=run_managers[i] if run_managers else None,
    637                 **kwargs,
    638             )
    639         )
    640     except BaseException as e:
    641         if run_managers:

File ~\miniconda3\Lib\site-packages\langchain_core\language_models\chat_models.py:851, in BaseChatModel._generate_with_cache(self, messages, stop, run_manager, **kwargs)
    849 else:
    850     if inspect.signature(self._generate).parameters.get("run_manager"):
--> 851         result = self._generate(
    852             messages, stop=stop, run_manager=run_manager, **kwargs
    853         )
    854     else:
    855         result = self._generate(messages, stop=stop, **kwargs)

File ~\miniconda3\Lib\site-packages\langchain_ollama\chat_models.py:690, in ChatOllama._generate(self, messages, stop, run_manager, **kwargs)
    683 def _generate(
    684     self,
    685     messages: List[BaseMessage],
   (...)
    688     **kwargs: Any,
    689 ) -> ChatResult:
--> 690     final_chunk = self._chat_stream_with_aggregation(
    691         messages, stop, run_manager, verbose=self.verbose, **kwargs
    692     )
    693     generation_info = final_chunk.generation_info
    694     chat_generation = ChatGeneration(
    695         message=AIMessage(
    696             content=final_chunk.text,
   (...)
    700         generation_info=generation_info,
    701     )

File ~\miniconda3\Lib\site-packages\langchain_ollama\chat_models.py:591, in ChatOllama._chat_stream_with_aggregation(self, messages, stop, run_manager, verbose, **kwargs)
    582 def _chat_stream_with_aggregation(
    583     self,
    584     messages: List[BaseMessage],
   (...)
    588     **kwargs: Any,
    589 ) -> ChatGenerationChunk:
    590     final_chunk = None
--> 591     for stream_resp in self._create_chat_stream(messages, stop, **kwargs):
    592         if not isinstance(stream_resp, str):
    593             chunk = ChatGenerationChunk(
    594                 message=AIMessageChunk(
    595                     content=(
   (...)
    608                 ),
    609             )

File ~\miniconda3\Lib\site-packages\langchain_ollama\chat_models.py:578, in ChatOllama._create_chat_stream(self, messages, stop, **kwargs)
    575 chat_params = self._chat_params(messages, stop, **kwargs)
    577 if chat_params["stream"]:
--> 578     yield from self._client.chat(**chat_params)
    579 else:
    580     yield self._client.chat(**chat_params)

File ~\miniconda3\Lib\site-packages\ollama\_client.py:336, in Client.chat(self, model, messages, tools, stream, format, options, keep_alive)
    288 def chat(
    289   self,
    290   model: str = '',
   (...)
    297   keep_alive: Optional[Union[float, str]] = None,
    298 ) -> Union[ChatResponse, Iterator[ChatResponse]]:
    299   """
    300   Create a chat response using the requested model.
    301 
   (...)
    330   Returns `ChatResponse` if `stream` is `False`, otherwise returns a `ChatResponse` generator.
    331   """
    332   return self._request(
    333     ChatResponse,
    334     'POST',
    335     '/api/chat',
--> 336     json=ChatRequest(
    337       model=model,
    338       messages=[message for message in _copy_messages(messages)],
    339       tools=[tool for tool in _copy_tools(tools)],
    340       stream=stream,
    341       format=format,
    342       options=options,
    343       keep_alive=keep_alive,
    344     ).model_dump(exclude_none=True),
    345     stream=stream,
    346   )

File ~\miniconda3\Lib\site-packages\pydantic\main.py:214, in BaseModel.__init__(self, **data)
    212 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    213 __tracebackhide__ = True
--> 214 validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    215 if self is not validated_self:
    216     warnings.warn(
    217         'A custom validator is returning a value other than `self`.\n'
    218         "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
    219         'See the `model_validator` docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.',
    220         stacklevel=2,
    221     )

ValidationError: 2 validation errors for ChatRequest
format.literal['json']
  Input should be 'json' [type=literal_error, input_value='', input_type=str]
    For further information visit https://errors.pydantic.dev/2.10/v/literal_error
format.dict[str,any]
  Input should be a valid dictionary [type=dict_type, input_value='', input_type=str]
    For further information visit https://errors.pydantic.dev/2.10/v/dict_type

Additional Details

• Receive correct response when invoking via CLI and ollama package directly
• Downgrading to ollama==0.4.2 resolves issue

Originally created by @ross-harrison-fox on GitHub (Dec 7, 2024). Receive Validation code when running the bellow snippet with the following package combination: * `langchain==0.3.10` * `ollama==0.4.3` * `langchain-ollama==0.2.1` ```python # get embedding and LLM models from langchain_ollama import OllamaEmbeddings, OllamaLLM embeddings = OllamaEmbeddings(model="llama3", base_url="http://localhost:11434") llm = OllamaLLM(model="llama3", format='', base_url="http://localhost:11434", verbose=True) prompt = "Hello There" response = llm.invoke(prompt) ``` Expected Behavior: Some response Actual: See validation error ``` 19 prompt = "Hello There" ---> 21 response = llm.invoke(prompt) File ~\miniconda3\Lib\site-packages\langchain_core\language_models\chat_models.py:286, in BaseChatModel.invoke(self, input, config, stop, **kwargs) 275 def invoke( 276 self, 277 input: LanguageModelInput, (...) 281 **kwargs: Any, 282 ) -> BaseMessage: 283 config = ensure_config(config) 284 return cast( 285 ChatGeneration, --> 286 self.generate_prompt( 287 [self._convert_input(input)], 288 stop=stop, 289 callbacks=config.get("callbacks"), 290 tags=config.get("tags"), 291 metadata=config.get("metadata"), 292 run_name=config.get("run_name"), 293 run_id=config.pop("run_id", None), 294 **kwargs, 295 ).generations[0][0], 296 ).message File ~\miniconda3\Lib\site-packages\langchain_core\language_models\chat_models.py:786, in BaseChatModel.generate_prompt(self, prompts, stop, callbacks, **kwargs) 778 def generate_prompt( 779 self, 780 prompts: list[PromptValue], (...) 783 **kwargs: Any, 784 ) -> LLMResult: 785 prompt_messages = [p.to_messages() for p in prompts] --> 786 return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs) File ~\miniconda3\Lib\site-packages\langchain_core\language_models\chat_models.py:643, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs) 641 if run_managers: 642 run_managers[i].on_llm_error(e, response=LLMResult(generations=[])) --> 643 raise e 644 flattened_outputs = [ 645 LLMResult(generations=[res.generations], llm_output=res.llm_output) # type: ignore[list-item] 646 for res in results 647 ] 648 llm_output = self._combine_llm_outputs([res.llm_output for res in results]) File ~\miniconda3\Lib\site-packages\langchain_core\language_models\chat_models.py:633, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs) 630 for i, m in enumerate(messages): 631 try: 632 results.append( --> 633 self._generate_with_cache( 634 m, 635 stop=stop, 636 run_manager=run_managers[i] if run_managers else None, 637 **kwargs, 638 ) 639 ) 640 except BaseException as e: 641 if run_managers: File ~\miniconda3\Lib\site-packages\langchain_core\language_models\chat_models.py:851, in BaseChatModel._generate_with_cache(self, messages, stop, run_manager, **kwargs) 849 else: 850 if inspect.signature(self._generate).parameters.get("run_manager"): --> 851 result = self._generate( 852 messages, stop=stop, run_manager=run_manager, **kwargs 853 ) 854 else: 855 result = self._generate(messages, stop=stop, **kwargs) File ~\miniconda3\Lib\site-packages\langchain_ollama\chat_models.py:690, in ChatOllama._generate(self, messages, stop, run_manager, **kwargs) 683 def _generate( 684 self, 685 messages: List[BaseMessage], (...) 688 **kwargs: Any, 689 ) -> ChatResult: --> 690 final_chunk = self._chat_stream_with_aggregation( 691 messages, stop, run_manager, verbose=self.verbose, **kwargs 692 ) 693 generation_info = final_chunk.generation_info 694 chat_generation = ChatGeneration( 695 message=AIMessage( 696 content=final_chunk.text, (...) 700 generation_info=generation_info, 701 ) File ~\miniconda3\Lib\site-packages\langchain_ollama\chat_models.py:591, in ChatOllama._chat_stream_with_aggregation(self, messages, stop, run_manager, verbose, **kwargs) 582 def _chat_stream_with_aggregation( 583 self, 584 messages: List[BaseMessage], (...) 588 **kwargs: Any, 589 ) -> ChatGenerationChunk: 590 final_chunk = None --> 591 for stream_resp in self._create_chat_stream(messages, stop, **kwargs): 592 if not isinstance(stream_resp, str): 593 chunk = ChatGenerationChunk( 594 message=AIMessageChunk( 595 content=( (...) 608 ), 609 ) File ~\miniconda3\Lib\site-packages\langchain_ollama\chat_models.py:578, in ChatOllama._create_chat_stream(self, messages, stop, **kwargs) 575 chat_params = self._chat_params(messages, stop, **kwargs) 577 if chat_params["stream"]: --> 578 yield from self._client.chat(**chat_params) 579 else: 580 yield self._client.chat(**chat_params) File ~\miniconda3\Lib\site-packages\ollama\_client.py:336, in Client.chat(self, model, messages, tools, stream, format, options, keep_alive) 288 def chat( 289 self, 290 model: str = '', (...) 297 keep_alive: Optional[Union[float, str]] = None, 298 ) -> Union[ChatResponse, Iterator[ChatResponse]]: 299 """ 300 Create a chat response using the requested model. 301 (...) 330 Returns `ChatResponse` if `stream` is `False`, otherwise returns a `ChatResponse` generator. 331 """ 332 return self._request( 333 ChatResponse, 334 'POST', 335 '/api/chat', --> 336 json=ChatRequest( 337 model=model, 338 messages=[message for message in _copy_messages(messages)], 339 tools=[tool for tool in _copy_tools(tools)], 340 stream=stream, 341 format=format, 342 options=options, 343 keep_alive=keep_alive, 344 ).model_dump(exclude_none=True), 345 stream=stream, 346 ) File ~\miniconda3\Lib\site-packages\pydantic\main.py:214, in BaseModel.__init__(self, **data) 212 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks 213 __tracebackhide__ = True --> 214 validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self) 215 if self is not validated_self: 216 warnings.warn( 217 'A custom validator is returning a value other than `self`.\n' 218 "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n" 219 'See the `model_validator` docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.', 220 stacklevel=2, 221 ) ValidationError: 2 validation errors for ChatRequest format.literal['json'] Input should be 'json' [type=literal_error, input_value='', input_type=str] For further information visit https://errors.pydantic.dev/2.10/v/literal_error format.dict[str,any] Input should be a valid dictionary [type=dict_type, input_value='', input_type=str] For further information visit https://errors.pydantic.dev/2.10/v/dict_type ``` ## Additional Details * Receive correct response when invoking via CLI and `ollama` package directly * Downgrading to `ollama==0.4.2` resolves issue

yindo closed this issue 2026-02-15 16:28:31 -05:00

yindo commented 2026-02-15 16:28:32 -05:00

Author

Owner

Copy Link

Copy Source

@anton-b commented on GitHub (Dec 7, 2024):

Observing the same issue.

ollama/_types.py

0.4.3:

class BaseGenerateRequest(BaseStreamableRequest):
  options: Optional[Union[Mapping[str, Any], Options]] = None
  'Options to use for the request.'

  format: Optional[Union[Literal['json'], JsonSchemaValue]] = None
  'Format of the response.'

  keep_alive: Optional[Union[float, str]] = None
  'Keep model alive for the specified duration.'

0.4.2:

class BaseGenerateRequest(BaseStreamableRequest):
 options: Optional[Union[Mapping[str, Any], Options]] = None
 'Options to use for the request.'

 format: Optional[Literal['', 'json']] = None
 'Format of the response.'

 keep_alive: Optional[Union[float, str]] = None
 'Keep model alive for the specified duration.'

@anton-b commented on GitHub (Dec 7, 2024): Observing the same issue. `ollama/_types.py` 0.4.3: ``` class BaseGenerateRequest(BaseStreamableRequest): options: Optional[Union[Mapping[str, Any], Options]] = None 'Options to use for the request.' format: Optional[Union[Literal['json'], JsonSchemaValue]] = None 'Format of the response.' keep_alive: Optional[Union[float, str]] = None 'Keep model alive for the specified duration.' ``` 0.4.2: ``` class BaseGenerateRequest(BaseStreamableRequest): options: Optional[Union[Mapping[str, Any], Options]] = None 'Options to use for the request.' format: Optional[Literal['', 'json']] = None 'Format of the response.' keep_alive: Optional[Union[float, str]] = None 'Keep model alive for the specified duration.' ```

yindo commented 2026-02-15 16:28:32 -05:00

Author

Owner

Copy Link

Copy Source

@edmcman commented on GitHub (Dec 7, 2024):

Can we maybe add a test to make sure that future ollama-python releases don't break langchain before we release them? This is the second issue in a week now.

@edmcman commented on GitHub (Dec 7, 2024): Can we maybe add a test to make sure that future `ollama-python` releases don't break langchain before we release them? This is the second issue in a week now.

yindo commented 2026-02-15 16:28:32 -05:00

Author

Owner

Copy Link

Copy Source

@nadeemcite commented on GitHub (Dec 7, 2024):

The issue stems from a recent update, where the BaseGenerateRequest class only allows 'json' as a valid value for the format field. Previously, an empty string was also permitted, but this support was removed in the latest update. To resolve this, we can modify the _types.py file in the ollama-python package to allow an empty string alongside 'json'. I've also added a PR to address this issue https://github.com/ollama/ollama-python/pull/368.

@nadeemcite commented on GitHub (Dec 7, 2024): The issue stems from a recent update, where the [BaseGenerateRequest class only allows 'json' as a valid value](https://github.com/ollama/ollama-python/blob/4b10dee2b2a73242bb3ac89332f634cda2b6f633/ollama/_types.py#L154) for the format field. Previously, an empty string was also permitted, but this support was removed in the latest update. To resolve this, we can modify the _types.py file in the ollama-python package to allow an empty string alongside 'json'. I've also added a PR to address this issue https://github.com/ollama/ollama-python/pull/368.

yindo commented 2026-02-15 16:28:32 -05:00

Author

Owner

Copy Link

Copy Source

@ParthSareen commented on GitHub (Dec 7, 2024):

Apologies for the breaking change! Just rolled out the change - thank you for pointing this!

@ParthSareen commented on GitHub (Dec 7, 2024): Apologies for the breaking change! Just rolled out the change - thank you for pointing this!

yindo commented 2026-02-15 16:28:32 -05:00

Author

Owner

Copy Link

Copy Source

@jmorganca commented on GitHub (Dec 7, 2024):

Sorry this broke all. You can update the ollama Python library to fix it:

pip install -U ollama

@jmorganca commented on GitHub (Dec 7, 2024): Sorry this broke all. You can update the `ollama` Python library to fix it: ``` pip install -U ollama ```

yindo commented 2026-02-15 16:28:33 -05:00

Author

Owner

Copy Link

Copy Source

@ross-harrison-fox commented on GitHub (Dec 10, 2024):

Thank you for the quick fix!

@ross-harrison-fox commented on GitHub (Dec 10, 2024): Thank you for the quick fix!

yindo referenced this issue 2026-02-15 16:30:16 -05:00

[PR #181] [MERGED] Add type overloads to methods #401

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

dependabot/github_actions/actions/setup-python-7

dependabot/github_actions/actions/checkout-7

dependabot/pip/certifi-2026.4.22

dependabot/pip/pydantic-core-2.46.3

dependabot/pip/httpcore-1.0.9

dependabot/pip/anyio-4.12.1

dependabot/pip/typing-extensions-4.15.0

add-hatch-test-matrix

dependabot/github_actions/astral-sh/setup-uv-7

parth/point-to-local-websearch

nicole/example-refactor

nicole/browser-tool

parth/turbo-auth

parth/proxy-auth

parth/test

parth/tool-prototype

brucemacd/list-capabilities

tool-parsing-json-package

parth/revert-462

parth/tokenize-detokenize

revert-330-parth/qol-disable-tests-for-readmes-and-examples

parth/examples-pydantic-tools-and-more

jyan/tools

mxyng/layers-from-files

v0.6.2

v0.6.1

v0.6.0

v0.5.4

v0.5.3

v0.5.2

v0.5.1

v0.5.0

v0.4.9

v0.4.8

v0.4.7

v0.4.6

v0.4.5

v0.4.4

v0.4.3

v0.4.2

v0.4.1

v0.4.0

v0.3.3

v0.3.2

v0.3.1

v0.3.0

v0.2.1

v0.2.0

v0.1.9

v0.1.8

v0.1.7

v0.1.6

v0.1.5

v0.1.4

v0.1.3

v0.1.2

v0.1.0

v0.0.1

Labels

Clear labels

bug

enhancement

pull-request

Mirrored from GitHub Pull Request

thinking

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

yindo

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ollama/ollama-python#181