Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #157] Anthropic Claude 2 API Integration #101

New Issue
Closed
opened 2026-02-22 18:17:54 -05:00 by yindo · 6 comments
yindo commented 2026-02-22 18:17:54 -05:00
Owner
Copy Link
Copy Source

Originally created by @jignnsd on GitHub (Jul 22, 2023).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/157

Hey Tim, thank you for this work, it works very good so far on my tests.
I have one question, is it possible to have support (using the same DB and Embeddings API) of the just introduced AnthropIc Claude 2 to use it instead of Openai chat API on an easy way, or it will implicate a lot of changes to be done?
I’m curios because I just compare the answers between openai (on anything-llm) and claude 2 with the same pdf file, and the answer was much better on claude 2, maybe because of the bigger token limit, so maybe will be a good thing to try if this does not implicate a huge change.
Also, the pricing drop using Claude 2 instead of Openai is important, like half of the price.
Again, many thanks for the good work and hopefully you can add a link to make a donation to compensate this great effort.
Javier

Originally created by @jignnsd on GitHub (Jul 22, 2023). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/157 Hey Tim, thank you for this work, it works very good so far on my tests. I have one question, is it possible to have support (using the same DB and Embeddings API) of the just introduced AnthropIc Claude 2 to use it instead of Openai chat API on an easy way, or it will implicate a lot of changes to be done? I’m curios because I just compare the answers between openai (on anything-llm) and claude 2 with the same pdf file, and the answer was much better on claude 2, maybe because of the bigger token limit, so maybe will be a good thing to try if this does not implicate a huge change. Also, the pricing drop using Claude 2 instead of Openai is important, like half of the price. Again, many thanks for the good work and hopefully you can add a link to make a donation to compensate this great effort. Javier
yindo added the enhancementIntegration Request labels 2026-02-22 18:17:54 -05:00
yindo closed this issue 2026-02-22 18:17:54 -05:00
yindo commented 2026-02-22 18:17:54 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 25, 2023):

Oh, that sounds very cool. The LLM side of things needs to diversity (even beyond Azure which is just OpenAI anyway).

If they have a support JS client it should keep things even more simple. Otherwise rolling our own internal API might be a pain. I do think that embedding will still have to be done with OpenAI though, as I don't think Anthropic has an embedded yet?

Chat should be straighforward

@timothycarambat commented on GitHub (Jul 25, 2023): Oh, that sounds very cool. The LLM side of things needs to diversity (even beyond Azure which is just OpenAI anyway). If they have a support JS client it should keep things even more simple. Otherwise rolling our own internal API might be a pain. I do think that embedding will still have to be done with OpenAI though, as I don't think Anthropic has an embedded yet? Chat should be straighforward
yindo commented 2026-02-22 18:17:54 -05:00
Author
Owner
Copy Link
Copy Source

@jignnsd commented on GitHub (Jul 25, 2023):

You are right, should keept openai for embeddings and also same vectordb, just the chat part will be changed to have the option of Anthropic.

@jignnsd commented on GitHub (Jul 25, 2023): You are right, should keept openai for embeddings and also same vectordb, just the chat part will be changed to have the option of Anthropic.
yindo commented 2026-02-22 18:17:54 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 25, 2023):

For Implementation:
Node client: https://www.npmjs.com/package/@anthropic-ai/sdk
API: https://docs.anthropic.com/claude/reference/getting-started-with-the-api

Will require some new ENV keys in settings to support this config.

@timothycarambat commented on GitHub (Jul 25, 2023): For Implementation: Node client: https://www.npmjs.com/package/@anthropic-ai/sdk API: https://docs.anthropic.com/claude/reference/getting-started-with-the-api Will require some new ENV keys in settings to support this config.
yindo commented 2026-02-22 18:17:54 -05:00
Author
Owner
Copy Link
Copy Source

@madruga8 commented on GitHub (Jul 30, 2023):

the big context of claude2-100k really makes a huge difference...

@madruga8 commented on GitHub (Jul 30, 2023): the big context of claude2-100k really makes a huge difference...
yindo commented 2026-02-22 18:17:54 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 3, 2023):

Mintplex Labs has applied to the Anthropic Claude 2 API access so we can integrate it fully. This issue is blocked until we or someone else can provide an API key for development (do not share key in issue please)

@timothycarambat commented on GitHub (Aug 3, 2023): Mintplex Labs has applied to the Anthropic Claude 2 API access so we can integrate it fully. This issue is blocked until we or someone else can provide an API key for development (do not share key in issue please)
yindo commented 2026-02-22 18:17:54 -05:00
Author
Owner
Copy Link
Copy Source

@ishaan-jaff commented on GitHub (Sep 21, 2023):

Hi @jignnsd @timothycarambat I believe I can help with this issue. I’m the maintainer of LiteLLM https://github.com/BerriAI/litellm - we allow you to use any LLM as a drop in replacement for gpt-3.5-turbo.

You can use LiteLLM in the following ways:

With your own API KEY:

This calls the provider API directly

from litellm import completion
import os
## set ENV variables 
os.environ["OPENAI_API_KEY"] = "your-key" # 
os.environ["COHERE_API_KEY"] = "your-key" # 

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion(model="command-nightly", messages=messages)

Using the LiteLLM Proxy with a LiteLLM Key

this is great if you don’t have access to claude but want to use the open source LiteLLM proxy to access claude

from litellm import completion
import os

## set ENV variables 
os.environ["OPENAI_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your openai key
os.environ["COHERE_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your cohere key

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion(model="command-nightly", messages=messages)
@ishaan-jaff commented on GitHub (Sep 21, 2023): Hi @jignnsd @timothycarambat I believe I can help with this issue. I’m the maintainer of LiteLLM https://github.com/BerriAI/litellm - we allow you to use any LLM as a drop in replacement for `gpt-3.5-turbo`. You can use LiteLLM in the following ways: ## With your own API KEY: This calls the provider API directly ```python from litellm import completion import os ## set ENV variables os.environ["OPENAI_API_KEY"] = "your-key" # os.environ["COHERE_API_KEY"] = "your-key" # messages = [{ "content": "Hello, how are you?","role": "user"}] # openai call response = completion(model="gpt-3.5-turbo", messages=messages) # cohere call response = completion(model="command-nightly", messages=messages) ``` ## Using the LiteLLM Proxy with a LiteLLM Key this is great if you don’t have access to claude but want to use the open source LiteLLM proxy to access claude ```python from litellm import completion import os ## set ENV variables os.environ["OPENAI_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your openai key os.environ["COHERE_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your cohere key messages = [{ "content": "Hello, how are you?","role": "user"}] # openai call response = completion(model="gpt-3.5-turbo", messages=messages) # cohere call response = completion(model="command-nightly", messages=messages) ```
yindo changed title from Anthropic Claude 2 API Integration to [GH-ISSUE #157] Anthropic Claude 2 API Integration 2026-06-05 14:33:29 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label Integration Request enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#101
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?