Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3779] [FEAT]: Enable Google Search Tool for Gemini Models #2427

New Issue
Closed
opened 2026-02-22 18:29:38 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:29:38 -05:00
Owner
Copy Link
Copy Source

Originally created by @unforced on GitHub (May 7, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3779

What would you like to see?

Is your feature request related to a problem? Please describe.
Currently, AnythingLLM's Gemini integration does not leverage the "Grounding with Google Search" feature available through the native Gemini API. This feature allows Gemini models to ground their responses in real-time Google Search results, improving factual accuracy, recency, and providing users with source attribution.

Describe the solution you'd like
We should update the Gemini provider in AnythingLLM to support enabling the Google Search tool. This would likely involve:

  1. Transitioning to the native Google Generative AI SDK for Node.js:

    • The current implementation uses the openai Node.js library, configured with a custom baseURL to point to Google's OpenAI-compatible endpoint (https://generativelanguage.googleapis.com/v1beta/openai/).
    • To properly support Gemini-specific features like Google Search tooling, we should migrate to the official @google/generative-ai SDK.

  2. Modifying API Calls:

    • The method for generating content will need to change from openai.chat.completions.create to the equivalent in the @google/generative-ai SDK (e.g., genAI.getGenerativeModel(...).generateContentStream(...) or generateContent(...)).
    • Crucially, we need to add the tools parameter to the model configuration when making API calls. This parameter will include an instance of GoogleSearch to enable the feature. 
      // Example snippet for enabling Google Search
const { GoogleGenerativeAI } = require("@google/generative-ai");
const { GoogleSearch } = require("@google/generative-ai/server"); // Or appropriate import

const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);
const model = genAI.getGenerativeModel({
  model: "gemini-2.0-flash", // Or other compatible model
  tools: [{ googleSearch: new GoogleSearch() }], // Or [{ googleSearch: {} }]
});
// ... then call model.generateContentStream(request) or model.generateContent(request)

  3. Updating Request and Response Handling:

    • The structure of the request payload (specifically messages or contents) will need to align with the native Google SDK's format.
    • The response handling logic will need to be updated to extract and potentially display groundingMetadata from the API response. This metadata includes search entry points (rendered content for suggested queries) and grounding chunks with URIs to the source documents.
    • The documentation specifies requirements for displaying Google Search Suggestions.

  4. User Interface (UI) Changes:

    • A new setting should be added in the AnythingLLM UI for Gemini configurations to allow users to enable or disable Google Search grounding.
    • Consideration should be given to how grounding sources and search suggestions are displayed to the end-user in the chat interface.

  5. Model Compatibility:

    • Ensure that the feature is enabled only for Gemini models that support this tooling (e.g., "gemini-2.0-flash" and potentially 1.5 models, though the documentation has some nuances here – "Search-as-a-tool" is mentioned for Gemini 2.0).

Describe alternatives you've considered
Attempting to use this feature via the current OpenAI-compatible endpoint is unlikely to work, as this is a Gemini-specific tooling feature best accessed via its native SDK.

Additional context
The official Google AI documentation for grounding with Google Search can be found here: https://ai.google.dev/gemini-api/docs/grounding?lang=python (Note: link is for Python, but concepts for tool configuration are relevant).

The existing Gemini integration is located in server/utils/AiProviders/gemini/index.js. Key methods to be refactored would include the constructor, getChatCompletion, and streamGetChatCompletion.

This enhancement would significantly improve the reliability and verifiability of responses from Gemini models within AnythingLLM, especially for queries requiring up-to-date information.

Originally created by @unforced on GitHub (May 7, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3779 ### What would you like to see? **Is your feature request related to a problem? Please describe.** Currently, AnythingLLM's Gemini integration does not leverage the "Grounding with Google Search" feature available through the native Gemini API. This feature allows Gemini models to ground their responses in real-time Google Search results, improving factual accuracy, recency, and providing users with source attribution. **Describe the solution you'd like** We should update the Gemini provider in AnythingLLM to support enabling the Google Search tool. This would likely involve: 1. **Transitioning to the native Google Generative AI SDK for Node.js:** * The current implementation uses the `openai` Node.js library, configured with a custom `baseURL` to point to Google's OpenAI-compatible endpoint (`https://generativelanguage.googleapis.com/v1beta/openai/`). * To properly support Gemini-specific features like Google Search tooling, we should migrate to the official `@google/generative-ai` SDK. 2. **Modifying API Calls:** * The method for generating content will need to change from `openai.chat.completions.create` to the equivalent in the `@google/generative-ai` SDK (e.g., `genAI.getGenerativeModel(...).generateContentStream(...)` or `generateContent(...)`). * Crucially, we need to add the `tools` parameter to the model configuration when making API calls. This parameter will include an instance of `GoogleSearch` to enable the feature. ```javascript // Example snippet for enabling Google Search const { GoogleGenerativeAI } = require("@google/generative-ai"); const { GoogleSearch } = require("@google/generative-ai/server"); // Or appropriate import const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY); const model = genAI.getGenerativeModel({ model: "gemini-2.0-flash", // Or other compatible model tools: [{ googleSearch: new GoogleSearch() }], // Or [{ googleSearch: {} }] }); // ... then call model.generateContentStream(request) or model.generateContent(request) ``` 3. **Updating Request and Response Handling:** * The structure of the request payload (specifically `messages` or `contents`) will need to align with the native Google SDK's format. * The response handling logic will need to be updated to extract and potentially display `groundingMetadata` from the API response. This metadata includes search entry points (rendered content for suggested queries) and grounding chunks with URIs to the source documents. * The documentation specifies requirements for displaying Google Search Suggestions. 4. **User Interface (UI) Changes:** * A new setting should be added in the AnythingLLM UI for Gemini configurations to allow users to enable or disable Google Search grounding. * Consideration should be given to how grounding sources and search suggestions are displayed to the end-user in the chat interface. 5. **Model Compatibility:** * Ensure that the feature is enabled only for Gemini models that support this tooling (e.g., "gemini-2.0-flash" and potentially 1.5 models, though the documentation has some nuances here – "Search-as-a-tool" is mentioned for Gemini 2.0). **Describe alternatives you've considered** Attempting to use this feature via the current OpenAI-compatible endpoint is unlikely to work, as this is a Gemini-specific tooling feature best accessed via its native SDK. **Additional context** The official Google AI documentation for grounding with Google Search can be found here: [https://ai.google.dev/gemini-api/docs/grounding?lang=python](https://ai.google.dev/gemini-api/docs/grounding?lang=python) (Note: link is for Python, but concepts for tool configuration are relevant). The existing Gemini integration is located in `server/utils/AiProviders/gemini/index.js`. Key methods to be refactored would include the constructor, `getChatCompletion`, and `streamGetChatCompletion`. This enhancement would significantly improve the reliability and verifiability of responses from Gemini models within AnythingLLM, especially for queries requiring up-to-date information.
yindo added the enhancementfeature request labels 2026-02-22 18:29:38 -05:00
yindo closed this issue 2026-02-22 18:29:38 -05:00
yindo commented 2026-02-22 18:29:38 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 7, 2025):

Transitioning to the native Google Generative AI SDK for Node.js:

We previously supported the generative AI SDK from google and it blocks gemini from being able to use the tool calling built into AnythingLLM, since their API is very difficult to work with and is vastly different from the more standardized OpenAI SDK. We will not be changing it back to reduce complexity in maintainability.

Next, since Gemini (all models) now can work with native AnythingLLM's agent tooling our built-in web-search tool that can use Google, Bing, DuckDuckGo, or whatever SERP you want easily fills in this gap with no changes required since using the "built-in" search from the Gemini SDK requires us to make a ton of Google-specific UI changes as well as not unlimited usage - so its the same as just using the agent web-search tool we already have which is model agnostic

@timothycarambat commented on GitHub (May 7, 2025): > Transitioning to the native Google Generative AI SDK for Node.js: We previously supported the generative AI SDK from google and it blocks gemini from being able to use the tool calling built into AnythingLLM, since their API is very difficult to work with and is vastly different from the more standardized OpenAI SDK. We will not be changing it back to reduce complexity in maintainability. Next, since Gemini (all models) now can work with native AnythingLLM's agent tooling our built-in `web-search` tool that can use Google, Bing, DuckDuckGo, or whatever SERP you want easily fills in this gap with no changes required since using the "built-in" search from the Gemini SDK requires us to make a [ton of Google-specific UI changes](https://ai.google.dev/gemini-api/docs/grounding/search-suggestions#requirements) as well as not unlimited usage - so its the same as just using the agent web-search tool we already have which is model agnostic
yindo changed title from [FEAT]: Enable Google Search Tool for Gemini Models to [GH-ISSUE #3779] [FEAT]: Enable Google Search Tool for Gemini Models 2026-06-05 14:46:28 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2427
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?