Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2886] [FEAT]: Gemini - Better handling for thinking models #1837

New Issue
Closed
opened 2026-02-22 18:26:45 -05:00 by yindo · 5 comments
yindo commented 2026-02-22 18:26:45 -05:00
Owner
Copy Link
Copy Source

Originally created by @xoconoch on GitHub (Dec 21, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2886

What would you like to see?

Been experimenting with Gemni 2.0 Flash Thinking (awful name) lately and it seems like the anythingllm frontend handles the api answer as such:

[Thought process][Answer]

It puts all the text in the same "space" as if it was one unique answer, but it's not.

This is an example of an answer showcased in anythingllm:

image

the highlighted part is where I think both thought and answer are concatenated, not only because it clearly looks like, but also because that's the only part of the whole answer where a period is missing its respective subsequent space.

This is an example of the same prompt in google ai studio

image

If I expand that Thought box, it shows all the previous process.

I'm not sure if the api response body provides the possibility for implementing this, but I sure hope it does! Also, I wonder if this a mistake also happening with OpenAI's o1.

Edit: One workaround I found is to specify on the workspace's prompt to structure its responses like so:

[Thought process]
[Enter]
[Response]
Originally created by @xoconoch on GitHub (Dec 21, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2886 ### What would you like to see? Been experimenting with Gemni 2.0 Flash Thinking (awful name) lately and it seems like the anythingllm frontend handles the api answer as such: [Thought process][Answer] It puts all the text in the same "space" as if it was one unique answer, but it's not. This is an example of an answer showcased in anythingllm: ![image](https://github.com/user-attachments/assets/32ab3402-810c-461d-aae4-cac78ad1f355) the highlighted part is where I think both thought and answer are concatenated, not only because it clearly looks like, but also because that's the only part of the whole answer where a period is missing its respective subsequent space. This is an example of the same prompt in google ai studio ![image](https://github.com/user-attachments/assets/e55597ab-664e-4eed-8eb4-e1b6d6b058dc) If I expand that Thought box, it shows all the previous process. I'm not sure if the api response body provides the possibility for implementing this, but I sure hope it does! Also, I wonder if this a mistake also happening with OpenAI's o1. Edit: One workaround I found is to specify on the workspace's prompt to structure its responses like so: ``` [Thought process] [Enter] [Response] ```
yindo added the enhancementfeature request labels 2026-02-22 18:26:45 -05:00
yindo closed this issue 2026-02-22 18:26:45 -05:00
yindo commented 2026-02-22 18:26:46 -05:00
Author
Owner
Copy Link
Copy Source

@spencerthayer commented on GitHub (Dec 22, 2024):

This a good idea. I like how OpenAI had the thinking process as an unordered list that one would open and close.

@spencerthayer commented on GitHub (Dec 22, 2024): This a good idea. I like how OpenAI had the thinking process as an unordered list that one would open and close.
yindo commented 2026-02-22 18:26:46 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 3, 2025):

Apparently they just removed it from the API - following this fourm post

Why they would do this, I have no idea. The docs dont show anything on this in their SDK, so for now this isn't possible and is simply only visible in their AI Studio UI?

Crazy work by Google to do that after enabling it for so long

@timothycarambat commented on GitHub (Feb 3, 2025): Apparently they _just_ removed it from the API - [following this fourm post](https://discuss.ai.google.dev/t/thoughts-are-missing-cot-not-included-anymore/63653/9) Why they would do this, I have no idea. The docs dont show anything on this in their SDK, so for now this isn't possible and is[ simply only visible in their AI Studio UI?](https://discuss.ai.google.dev/t/thoughts-are-missing-cot-not-included-anymore/63653/10) Crazy work by Google to do that after enabling it for so long
yindo commented 2026-02-22 18:26:46 -05:00
Author
Owner
Copy Link
Copy Source

@spencerthayer commented on GitHub (Feb 6, 2025):

@timothycarambat

It looks like the lead product dev for Google AI Studio posted on Twitter that you can turn it back on.

FYI, based on feedback we also disabled thoughts by default in the API, you can turn them back on with config = {'thinking_config': {'include_thoughts': True}}.

@spencerthayer commented on GitHub (Feb 6, 2025): @timothycarambat It looks like the lead product dev for Google AI Studio [posted on Twitter](https://x.com/OfficialLoganK/status/1881848134718075109) that you can turn it back on. > FYI, based on feedback we also disabled thoughts by default in the API, you can turn them back on with `config = {'thinking_config': {'include_thoughts': True}}`.
yindo commented 2026-02-22 18:26:46 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 6, 2025):

@spencerthayer That was posted Jan 21, 2025 - the form I linked is from Feb 3, 2025. So the current state is what is on the form - not from that post on Twitter.

They used to have it, then undid it :(

@timothycarambat commented on GitHub (Feb 6, 2025): @spencerthayer That was posted Jan 21, 2025 - the form I linked is from Feb 3, 2025. So the current state is what is on the form - not from that post on Twitter. They _used_ to have it, then undid it :(
yindo commented 2026-02-22 18:26:46 -05:00
Author
Owner
Copy Link
Copy Source

@spencerthayer commented on GitHub (Feb 6, 2025):

Ah. Sad. It's real shame.

@spencerthayer commented on GitHub (Feb 6, 2025): Ah. Sad. It's real shame.
yindo changed title from [FEAT]: Gemini - Better handling for thinking models to [GH-ISSUE #2886] [FEAT]: Gemini - Better handling for thinking models 2026-06-05 14:42:57 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1837
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?