How to set temprature and output token size in the chat mode? #39

New Issue

2026-02-15T16:27:38-05:00

yindo commented

2026-02-15 16:27:38 -05:00

Originally created by @jojogh on GitHub (Feb 28, 2024).

Is there any examples to guide how to set temprature and output token size in the chat mode?

Originally created by @jojogh on GitHub (Feb 28, 2024). Is there any examples to guide how to set temprature and output token size in the chat mode?

yindo closed this issue

2026-02-15 16:27:38 -05:00

yindo commented

2026-02-15 16:27:38 -05:00

@eliranwong commented on GitHub (Feb 28, 2024):

You may read this example:

examples/fill-in-middle/main.py

@eliranwong commented on GitHub (Feb 28, 2024): You may read this example: examples/fill-in-middle/main.py

yindo commented

2026-02-15 16:27:39 -05:00

@jojogh commented on GitHub (Mar 1, 2024):

thanks first, i have seen this example, it is generate mode, not in chat mode.

@jojogh commented on GitHub (Mar 1, 2024): thanks first, i have seen this example, it is generate mode, not in chat mode.

yindo commented

2026-02-15 16:27:39 -05:00

@eliranwong commented on GitHub (Mar 1, 2024):

For example, I use the following for one of my projects:

from ollama import Options

        completion = ollama.chat(
            #keep_alive=0,
            model=config.ollamaDefaultModel,
            messages=[
                *ongoingMessages,
                {
                    "role": "user",
                    "content": prompt,
                },
            ],
            format="json",
            stream=True,
            options=Options(
                temperature=0.0,
                num_ctx=8192,
                num_predict=-1,
            ),
        )

@eliranwong commented on GitHub (Mar 1, 2024): For example, I use the following for one of my projects: ``` from ollama import Options ``` ``` completion = ollama.chat( #keep_alive=0, model=config.ollamaDefaultModel, messages=[ *ongoingMessages, { "role": "user", "content": prompt, }, ], format="json", stream=True, options=Options( temperature=0.0, num_ctx=8192, num_predict=-1, ), ) ```

yindo commented

2026-02-15 16:27:39 -05:00

@deepcoder commented on GitHub (Mar 1, 2024):

and "keep_alive" as well, thanks!

@deepcoder commented on GitHub (Mar 1, 2024): and "keep_alive" as well, thanks!

yindo commented

2026-02-15 16:27:39 -05:00

@jojogh commented on GitHub (Mar 2, 2024):

For example, I use the following for one of my projects:

from ollama import Options

        completion = ollama.chat(
            #keep_alive=0,
            model=config.ollamaDefaultModel,
            messages=[
                *ongoingMessages,
                {
                    "role": "user",
                    "content": prompt,
                },
            ],
            format="json",
            stream=True,
            options=Options(
                temperature=0.0,
                num_ctx=8192,
                num_predict=-1,
            ),
        )

Thanks Wong, I will try. Many thanks.

@jojogh commented on GitHub (Mar 2, 2024): > For example, I use the following for one of my projects: > > ``` > from ollama import Options > ``` > > ``` > completion = ollama.chat( > #keep_alive=0, > model=config.ollamaDefaultModel, > messages=[ > *ongoingMessages, > { > "role": "user", > "content": prompt, > }, > ], > format="json", > stream=True, > options=Options( > temperature=0.0, > num_ctx=8192, > num_predict=-1, > ), > ) > ``` Thanks Wong, I will try. Many thanks.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ollama/ollama-python#39