Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 09:35:22 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #771] [FEAT]: VertexAI Support as LLM #446

New Issue
Open
opened 2026-02-22 18:19:31 -05:00 by yindo · 7 comments
yindo commented 2026-02-22 18:19:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @timothycarambat on GitHub (Feb 21, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/771

What would you like to see?

Add support for GCP's VertexAI provider.
Docs: https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview#gen-ai-studio

This may be quite complex to support given the nature of Vertex and how it can run multiple models. Functionally, this is probably quite similar to AWS BedRock models

Originally created by @timothycarambat on GitHub (Feb 21, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/771 ### What would you like to see? Add support for GCP's VertexAI provider. Docs: https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview#gen-ai-studio This may be quite complex to support given the nature of Vertex and how it can run multiple models. Functionally, this is probably quite similar to AWS BedRock models
yindo added the enhancementIntegration Requestfeature request labels 2026-02-22 18:19:31 -05:00
yindo commented 2026-02-22 18:19:31 -05:00
Author
Owner
Copy Link
Copy Source

@SeaDude commented on GitHub (Oct 30, 2024):

This would be lovely.

Example: I'm still waiting for Claude access in my AWS account (because AWS makes you contact support ((boooo!)) ), but I was immediately granted access in my Google Vertex account.

I mean I could just pipe in my Anthropic API key but... the data privacy and security are not guaranteed by the cloud provider (important to me).

@SeaDude commented on GitHub (Oct 30, 2024): This would be lovely. Example: I'm still waiting for Claude access in my AWS account (because AWS makes you contact support ((boooo!)) ), but I was immediately granted access in my Google Vertex account. I mean I could just pipe in my Anthropic API key but... the data privacy and security are not guaranteed by the cloud provider (important to me).
yindo commented 2026-02-22 18:19:31 -05:00
Author
Owner
Copy Link
Copy Source

@chronometer commented on GitHub (Nov 3, 2024):

This would be amazing.
Especially as you can launch models from huggingface models via VertexAI.

@chronometer commented on GitHub (Nov 3, 2024): This would be amazing. Especially as you can launch models from huggingface models via VertexAI.
yindo commented 2026-02-22 18:19:31 -05:00
Author
Owner
Copy Link
Copy Source

@Divyanshu9822 commented on GitHub (Dec 23, 2024):

I came to AnythingLLM looking for this because I also want to run Claude Sonnet via APIs but I prefer to use this via Vertex AI

@Divyanshu9822 commented on GitHub (Dec 23, 2024): I came to AnythingLLM looking for this because I also want to run Claude Sonnet via APIs but I prefer to use this via Vertex AI
yindo commented 2026-02-22 18:19:31 -05:00
Author
Owner
Copy Link
Copy Source

@savager commented on GitHub (Jun 29, 2025):

+1 for this even as a not fully integrated feature. Setting the Vertex model in the admin and not in the chat would even be beneficial. Thanks

@savager commented on GitHub (Jun 29, 2025): +1 for this even as a not fully integrated feature. Setting the Vertex model in the admin and not in the chat would even be beneficial. Thanks
yindo commented 2026-02-22 18:19:31 -05:00
Author
Owner
Copy Link
Copy Source

@grantcarthew commented on GitHub (Aug 1, 2025):

I'm setting up a new interface for a group of people to use AI. AnythingLLM was my first goto. This feature, or lack of, is a blocker. We can't use AnythingLLM.

For reference, Google Gemini 2.5 Pro is the top of the leaderboards as of this post (Aug 2025):
https://lmarena.ai/leaderboard

Image

Anyone wanting paid access to the full "model garden" is going to need Vertex AI support.

@grantcarthew commented on GitHub (Aug 1, 2025): I'm setting up a new interface for a group of people to use AI. AnythingLLM was my first goto. This feature, or lack of, is a blocker. We can't use AnythingLLM. For reference, Google Gemini 2.5 Pro is the top of the leaderboards as of this post (Aug 2025): https://lmarena.ai/leaderboard <img width="1273" height="1196" alt="Image" src="https://github.com/user-attachments/assets/add6d644-d301-46b7-ac96-915cf51d4d11" /> Anyone wanting paid access to the full "model garden" is going to need Vertex AI support.
yindo commented 2026-02-22 18:19:31 -05:00
Author
Owner
Copy Link
Copy Source

@juwy commented on GitHub (Jan 10, 2026):

Just added a feature request (#4852 ) to support Google Cloud Auth for the Generic OpenAI Provider (as there is a OpenAI compatible API for Vertex, that just lacks support for permanent API keys).

That would be a lightweight workaround to use Vertex AI without the need to implement Vertex AI support from scratch.

However, I'd still appreciate a native Vertex AI, but are aware that it's a much bigger effort.

@juwy commented on GitHub (Jan 10, 2026): Just added a feature request (#4852 ) to support Google Cloud Auth for the Generic OpenAI Provider (as there is a OpenAI compatible API for Vertex, that just lacks support for permanent API keys). That would be a lightweight workaround to use Vertex AI without the need to implement Vertex AI support from scratch. However, I'd still appreciate a native Vertex AI, but are aware that it's a much bigger effort.
yindo changed title from [FEAT]: VertexAI Support as LLM to [GH-ISSUE #771] [FEAT]: VertexAI Support as LLM 2026-06-05 14:35:21 -04:00
yindo commented 2026-06-05 14:35:21 -04:00
Author
Owner
Copy Link
Copy Source

@spartacruz commented on GitHub (Mar 24, 2026):

Really look forward for this feat!
is there any current development on this? @timothycarambat

<!-- gh-comment-id:4122783295 --> @spartacruz commented on GitHub (Mar 24, 2026): Really look forward for this feat! is there any current development on this? @timothycarambat
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label Integration Request enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#446
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?