Tonkenization of the context #213

New Issue

2026-02-15T16:28:47-05:00

yindo commented

2026-02-15 16:28:47 -05:00

Originally created by @Gavin-WangSC on GitHub (Feb 5, 2025).

We see the definition of generate() in client file, there's a context that should be Sequence[int]:

  def generate(
    self,
    model: str = '',
    prompt: Optional[str] = None,
    suffix: Optional[str] = None,
    *,
    system: Optional[str] = None,
    template: Optional[str] = None,
    context: Optional[Sequence[int]] = None,
    stream: bool = False,
    raw: Optional[bool] = None,
    format: Optional[Union[Literal['', 'json'], JsonSchemaValue]] = None,
    images: Optional[Sequence[Union[str, bytes]]] = None,
    options: Optional[Union[Mapping[str, Any], Options]] = None,
    keep_alive: Optional[Union[float, str]] = None,
  ) -> Union[GenerateResponse, Iterator[GenerateResponse]]:
    """
    Create a response using the requested model.

    Raises `RequestError` if a model is not provided.

    Raises `ResponseError` if the request could not be fulfilled.

    Returns `GenerateResponse` if `stream` is `False`, otherwise returns a `GenerateResponse` generator.
    """

    return self._request(
      GenerateResponse,
      'POST',
      '/api/generate',
      json=GenerateRequest(
        model=model,
        prompt=prompt,
        suffix=suffix,
        system=system,
        template=template,
        context=context,
        stream=stream,
        raw=raw,
        format=format,
        images=[image for image in _copy_images(images)] if images else None,
        options=options,
        keep_alive=keep_alive,
      ).model_dump(exclude_none=True),
      stream=stream,
    )

class GenerateResponse(BaseGenerateResponse):
  """
  Response returned by generate requests.
  """

  response: str
  'Response content. When streaming, this contains a fragment of the response.'

  context: Optional[Sequence[int]] = None
  'Tokenized history up to the point of the response.'

But how can our text be tokenized into the int? Is there such a function inside this ollama libriary?

Originally created by @Gavin-WangSC on GitHub (Feb 5, 2025). We see the definition of generate() in client file, there's a context that should be Sequence[int]: ``` def generate( self, model: str = '', prompt: Optional[str] = None, suffix: Optional[str] = None, *, system: Optional[str] = None, template: Optional[str] = None, context: Optional[Sequence[int]] = None, stream: bool = False, raw: Optional[bool] = None, format: Optional[Union[Literal['', 'json'], JsonSchemaValue]] = None, images: Optional[Sequence[Union[str, bytes]]] = None, options: Optional[Union[Mapping[str, Any], Options]] = None, keep_alive: Optional[Union[float, str]] = None, ) -> Union[GenerateResponse, Iterator[GenerateResponse]]: """ Create a response using the requested model. Raises `RequestError` if a model is not provided. Raises `ResponseError` if the request could not be fulfilled. Returns `GenerateResponse` if `stream` is `False`, otherwise returns a `GenerateResponse` generator. """ return self._request( GenerateResponse, 'POST', '/api/generate', json=GenerateRequest( model=model, prompt=prompt, suffix=suffix, system=system, template=template, context=context, stream=stream, raw=raw, format=format, images=[image for image in _copy_images(images)] if images else None, options=options, keep_alive=keep_alive, ).model_dump(exclude_none=True), stream=stream, ) ``` ``` class GenerateResponse(BaseGenerateResponse): """ Response returned by generate requests. """ response: str 'Response content. When streaming, this contains a fragment of the response.' context: Optional[Sequence[int]] = None 'Tokenized history up to the point of the response.' ``` But how can our text be tokenized into the int? Is there such a function inside this ollama libriary?

yindo closed this issue

2026-02-15 16:28:47 -05:00

yindo commented

2026-02-15 16:28:47 -05:00

@ParthSareen commented on GitHub (Feb 11, 2025):

context should be a deprecated param: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-completion

Sorry for the confusion, should remove soon.

If you are trying to pass chat history, would recommend using the chat endpoint instead

@ParthSareen commented on GitHub (Feb 11, 2025): `context` should be a deprecated param: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-completion Sorry for the confusion, should remove soon. If you are trying to pass chat history, would recommend using the chat endpoint instead

yindo referenced this issue

2026-02-15 16:30:19 -05:00

[PR #213] [MERGED] integrate tool calls #415

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ollama/ollama-python#213