Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1297] [FEAT]: Gemini Agent Support #815

New Issue
Closed
opened 2026-02-22 18:21:31 -05:00 by yindo · 13 comments
yindo commented 2026-02-22 18:21:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @giovanniscalar on GitHub (May 6, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1297

Originally assigned to: @shatfield4 on GitHub.

What would you like to see?

Add support to use Gemini as the agent

Originally created by @giovanniscalar on GitHub (May 6, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1297 Originally assigned to: @shatfield4 on GitHub. ### What would you like to see? Add support to use Gemini as the agent
yindo added the enhancementfeature requestblocked labels 2026-02-22 18:21:31 -05:00
yindo closed this issue 2026-02-22 18:21:31 -05:00
yindo commented 2026-02-22 18:21:32 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (May 7, 2024):

We tried to get this implemented but unfortunately gemini function calling does not work reliably enough to be used as an agent. After some research and testing we came to the conclusion that we will have to wait until Google improves gemini models for function calling before we can implement this.

Gemini failing to call tools correctly

@shatfield4 commented on GitHub (May 7, 2024): We tried to get this implemented but unfortunately gemini function calling does not work reliably enough to be used as an agent. After some research and testing we came to the conclusion that we will have to wait until Google improves gemini models for function calling before we can implement this. [Gemini failing to call tools correctly](https://www.googlecloudcommunity.com/gc/AI-ML/Gemini-Pro-function-call-tooling-errors-quot-I-am-sorry-I-cannot/m-p/711763)
yindo commented 2026-02-22 18:21:32 -05:00
Author
Owner
Copy Link
Copy Source

@DangerousBerries commented on GitHub (Nov 1, 2024):

We tried to get this implemented but unfortunately gemini function calling does not work reliably enough to be used as an agent. After some research and testing we came to the conclusion that we will have to wait until Google improves gemini models for function calling before we can implement this.

Gemini failing to call tools correctly

Is this still the case?

@DangerousBerries commented on GitHub (Nov 1, 2024): > We tried to get this implemented but unfortunately gemini function calling does not work reliably enough to be used as an agent. After some research and testing we came to the conclusion that we will have to wait until Google improves gemini models for function calling before we can implement this. > > [Gemini failing to call tools correctly](https://www.googlecloudcommunity.com/gc/AI-ML/Gemini-Pro-function-call-tooling-errors-quot-I-am-sorry-I-cannot/m-p/711763) Is this still the case?
yindo commented 2026-02-22 18:21:32 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Nov 1, 2024):

The last time we checked on this issue, the issue was not with tool calling, but rate-limiting that google applies to models, making continuous agent calling impossible

@timothycarambat commented on GitHub (Nov 1, 2024): The last time we checked on this issue, the issue was not with tool calling, but rate-limiting that google applies to models, making continuous agent calling impossible
yindo commented 2026-02-22 18:21:32 -05:00
Author
Owner
Copy Link
Copy Source

@DangerousBerries commented on GitHub (Nov 12, 2024):

The last time we checked on this issue, the issue was not with tool calling, but rate-limiting that google applies to models, making continuous agent calling impossible

I thought Flash was fine with limits, can't the agent just turn off when rate limits are reached?

@DangerousBerries commented on GitHub (Nov 12, 2024): > The last time we checked on this issue, the issue was not with tool calling, but rate-limiting that google applies to models, making continuous agent calling impossible I thought Flash was fine with limits, can't the agent just turn off when rate limits are reached?
yindo commented 2026-02-22 18:21:32 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Nov 12, 2024):

It would produce an error, which then leads to someone opening a GitHub issue about it because they think it's broken!

@timothycarambat commented on GitHub (Nov 12, 2024): It would produce an error, which then leads to someone opening a GitHub issue about it because they think it's broken!
yindo commented 2026-02-22 18:21:32 -05:00
Author
Owner
Copy Link
Copy Source

@bermudi commented on GitHub (Feb 16, 2025):

until Google improves gemini models for function calling

I think it is time

@bermudi commented on GitHub (Feb 16, 2025): > until Google improves gemini models for function calling I think it is time
yindo commented 2026-02-22 18:21:32 -05:00
Author
Owner
Copy Link
Copy Source

@ringge commented on GitHub (Mar 5, 2025):

a workaround that I'm using is setup a LiteLLM instance then add Google gemini as a model, then use LiteLLM in anything LLM
but hopefully gemini is officially supported soon

@ringge commented on GitHub (Mar 5, 2025): a workaround that I'm using is setup a LiteLLM instance then add Google gemini as a model, then use LiteLLM in anything LLM but hopefully gemini is officially supported soon
yindo commented 2026-02-22 18:21:33 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 5, 2025):

@ringge You arent hitting rate limits in Gemini with agent tool calling even through LiteLLM? What model are you using btw?

@timothycarambat commented on GitHub (Mar 5, 2025): @ringge You arent hitting rate limits in Gemini with agent tool calling even through LiteLLM? What model are you using btw?
yindo commented 2026-02-22 18:21:33 -05:00
Author
Owner
Copy Link
Copy Source

@ringge commented on GitHub (Mar 5, 2025):

@ringge You arent hitting rate limits in Gemini with agent tool calling even through LiteLLM? What model are you using btw?

No I didn't see any issue with rate limits. I'm using gemini-2.0-pro-exp-02-05

@ringge commented on GitHub (Mar 5, 2025): > [@ringge](https://github.com/ringge) You arent hitting rate limits in Gemini with agent tool calling even through LiteLLM? What model are you using btw? No I didn't see any issue with rate limits. I'm using gemini-2.0-pro-exp-02-05
yindo commented 2026-02-22 18:21:33 -05:00
Author
Owner
Copy Link
Copy Source

@Benniepie commented on GitHub (Apr 5, 2025):

Any news on this?

Really surprised that there's no Gemini agent support given the models Google has been releasing over the last few months and their free API.

Please could you allow use of Gemini, specifically gemini-2.0-flash-thinking-exp-01-21 for agents as it is quick, has decent rate limits, massive context window and 65k token output. Offered for free via Google Generative AI API.

@Benniepie commented on GitHub (Apr 5, 2025): Any news on this? Really surprised that there's no Gemini agent support given the models Google has been releasing over the last few months and their free API. Please could you allow use of Gemini, specifically gemini-2.0-flash-thinking-exp-01-21 for agents as it is quick, has decent rate limits, massive context window and 65k token output. Offered for free via Google Generative AI API.
yindo commented 2026-02-22 18:21:33 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 5, 2025):

and their free API.

This is often what people try to do, the rate-limiting on the free Gemini API that 99% of tools calls with agents will fail since often you are running queries back-to-back to get faster responses. It's one of the main reasons we don't have Gemini agents, besides the fact that Google has a specific way of doing tools calls.

@timothycarambat commented on GitHub (Apr 5, 2025): > and their free API. This is often what people try to do, the rate-limiting on the free Gemini API that 99% of tools calls with agents will fail since often you are running queries back-to-back to get faster responses. It's one of the main reasons we don't have Gemini agents, besides the fact that Google has a specific way of doing tools calls.
yindo commented 2026-02-22 18:21:33 -05:00
Author
Owner
Copy Link
Copy Source

@bermudi commented on GitHub (Apr 5, 2025):

gemini-2.0-flash is the best LLM by a long shot when considering capabilities and cost. Not supporting it is just dumb.

There aren't many places to compare model use but if we go by openrouter metrics, gemini-2.0-flash is the most used model out there.

Image

btw that's the paid version... free version is no. 13 on the list

@bermudi commented on GitHub (Apr 5, 2025): gemini-2.0-flash is the best LLM by a long shot when considering capabilities and cost. Not supporting it is just dumb. There aren't many places to compare model use but if we go by openrouter metrics, gemini-2.0-flash is the most used model out there. ![Image](https://github.com/user-attachments/assets/4dddcb55-a2f4-4ea5-894d-4daff0230baa) btw that's the **paid** version... free version is no. 13 on the list
yindo commented 2026-02-22 18:21:33 -05:00
Author
Owner
Copy Link
Copy Source

@Benniepie commented on GitHub (Apr 5, 2025):

and their free API.

This is often what people try to do, the rate-limiting on the free Gemini API that 99% of tools calls with agents will fail since often you are running queries back-to-back to get faster responses. It's one of the main reasons we don't have Gemini agents, besides the fact that Google has a specific way of doing tools calls.

I don't buy that. Not in April 2025.

Have you tried the free API for the newer Gemini Flash models?

Or looked at the Gemini 2.0 live api options for function calls? Eg OpenAI compatibility, Multi-tool use, automatic calls, composition calls, mcpserver function call support...etc

Google has increased the free limits considerably and dropped the prices for the paid tiers..."built for the era of agents" says Google...

The same Generative AI API can also have much higher limits by adding billing to the cloud project, not all users are on the "free" API tier.

If you're thinking of the Gemini models and rate limits from a few months ago, it's all changed for the 2.0 models, below is 1.5 pro for comparison.

gemini-1.5-pro:
2 RPM 32,000 TPM 50 RPD (still heavily rate limited)

gemini-2.0-flash-001:
(Rank. 9 model on LM Arena)
15 RPM 1,000,000 TPM 1,500 RPD

gemini-2.0-flash-exp:
(Ranked 1 for image generation on LM Arena - imagen 3.0)
10 RPM 1,000,000 TPM 1,500 RPD

gemini-2.0-flash-lite
(Ranked 16 on LM Arena)
30 RPM 1,000,000 TPM 1,500 RPD

gemini-2.0-flash-thinking-01-21
(Ranked 6 on LM Arena / 65k token output)
10 RPM 4,000,000 TPM 1,500 RPD

gemma-3-27b-it
(Ranked 14 on LM Arena)
30 RPM 15,000 TPM 14,400 RPD

gemini-2.5-pro-exp-03-25
(Ranked 1 on LM Arena / 65k token output)
5 RPM 1,000,000 TPM 25 RPD

Google is releasing model after model with incredible performance/innovation at the moment and outperforming OpenAI/Anthropic, not supporting their models, even if the code is a bit different, is not good for customers.

Sources:
https://lmarena.ai/?leaderboard
https://ai.google.dev/gemini-api/docs/rate-limits#current-rate-limits
https://ai.google.dev/gemini-api/docs/function-calling?example=meeting#compositional_function_calling
https://ai.google.dev/gemini-api/docs/openai#function-calling

@Benniepie commented on GitHub (Apr 5, 2025): > > and their free API. > > This is often what people try to do, the rate-limiting on the free Gemini API that 99% of tools calls with agents will fail since often you are running queries back-to-back to get faster responses. It's one of the main reasons we don't have Gemini agents, besides the fact that Google has a specific way of doing tools calls. I don't buy that. Not in April 2025. Have you tried the free API for the newer Gemini Flash models? Or looked at the Gemini 2.0 live api options for function calls? Eg OpenAI compatibility, Multi-tool use, automatic calls, composition calls, mcpserver function call support...etc Google has increased the free limits considerably and dropped the prices for the paid tiers..."built for the era of agents" says Google... The same Generative AI API can also have much higher limits by adding billing to the cloud project, not all users are on the "free" API tier. If you're thinking of the Gemini models and rate limits from a few months ago, it's all changed for the 2.0 models, below is 1.5 pro for comparison. gemini-1.5-pro: 2 RPM 32,000 TPM 50 RPD (still heavily rate limited) gemini-2.0-flash-001: (Rank. 9 model on LM Arena) 15 RPM 1,000,000 TPM 1,500 RPD gemini-2.0-flash-exp: (Ranked 1 for image generation on LM Arena - imagen 3.0) 10 RPM 1,000,000 TPM 1,500 RPD gemini-2.0-flash-lite (Ranked 16 on LM Arena) 30 RPM 1,000,000 TPM 1,500 RPD gemini-2.0-flash-thinking-01-21 (Ranked 6 on LM Arena / 65k token output) 10 RPM 4,000,000 TPM 1,500 RPD gemma-3-27b-it (Ranked 14 on LM Arena) 30 RPM 15,000 TPM 14,400 RPD gemini-2.5-pro-exp-03-25 (Ranked 1 on LM Arena / 65k token output) 5 RPM 1,000,000 TPM 25 RPD Google is releasing model after model with incredible performance/innovation at the moment and outperforming OpenAI/Anthropic, not supporting their models, even if the code is a bit different, is not good for customers. Sources: https://lmarena.ai/?leaderboard https://ai.google.dev/gemini-api/docs/rate-limits#current-rate-limits https://ai.google.dev/gemini-api/docs/function-calling?example=meeting#compositional_function_calling https://ai.google.dev/gemini-api/docs/openai#function-calling
yindo changed title from [FEAT]: Gemini Agent Support to [GH-ISSUE #1297] [FEAT]: Gemini Agent Support 2026-06-05 14:37:20 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label blocked enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#815
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?