Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3085] [CHORE]: OpenAI o# models require different max_tokens parameter #1977

New Issue
Closed
opened 2026-02-22 18:27:31 -05:00 by yindo · 5 comments
yindo commented 2026-02-22 18:27:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @gianni-rosato on GitHub (Feb 1, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3085

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

Image

This is encountered when attempting to use the OpenAI-compatible option in AnythingLLM using GitHub Copilot Chat's model endpoint. I believe this is related to this OpenAI Community post. Reproduction steps are included below.

Are there known steps to reproduce?

Originally created by @gianni-rosato on GitHub (Feb 1, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3085 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? ![Image](https://github.com/user-attachments/assets/63630030-b985-4f70-8d2d-8712038a8ad1) This is encountered when attempting to use the OpenAI-compatible option in AnythingLLM using GitHub Copilot Chat's model endpoint. I believe this is related to [this OpenAI Community post](https://community.openai.com/t/why-was-max-tokens-changed-to-max-completion-tokens/938077). Reproduction steps are included below. ### Are there known steps to reproduce? - Go to https://github.com/marketplace/models/azure-openai/o3-mini/playground - Click "Use This Model" in the top right corner - Select "OpenAI SDK" from the SDK dropdown menu - Enter "https://models.inference.ai.azure.com" as your endpoint in AnythingLLM's OpenAI Compatible setup screen - Generate a Personal Access Token via GitHub and enter it as your API key - Set your token context window & max tokens - Set chat model name to `o3-mini` - Send a message in a Workspace chat window - Observe the error
yindo added the enhancement label 2026-02-22 18:27:31 -05:00
yindo closed this issue 2026-02-22 18:27:31 -05:00
yindo commented 2026-02-22 18:27:31 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 1, 2025):

Interestingly, the generic provide is probably bad for this and we should make it its own provider. That being said, the o# series has nuances to their calls where yes, these things can occur while using non o models would work fine.

Truthfully, this is the first I am even hearing about being able to use Github to run LLMs. Do you have some higher level or paid access to this service? I dont even have o1 or o3 in the marketplace available to me.

Reference: https://github.com/Mintplex-Labs/anything-llm/blob/121fbea2848707e870613e2edbbdb3c94df64ecf/server/utils/AiProviders/genericOpenAi/index.js#L152

@timothycarambat commented on GitHub (Feb 1, 2025): Interestingly, the generic provide is probably bad for this and we should make it its own provider. That being said, the `o#` series has nuances to their calls where yes, these things can occur while using non `o` models would work fine. Truthfully, this is the first I am even hearing about being able to use Github to run LLMs. Do you have some higher level or paid access to this service? I dont even have `o1` or `o3` in the marketplace available to me. Reference: https://github.com/Mintplex-Labs/anything-llm/blob/121fbea2848707e870613e2edbbdb3c94df64ecf/server/utils/AiProviders/genericOpenAi/index.js#L152
yindo commented 2026-02-22 18:27:31 -05:00
Author
Owner
Copy Link
Copy Source

@gianni-rosato commented on GitHub (Feb 1, 2025):

Are you able to go to https://github.com/marketplace/models/azure-openai/o3-mini/playground? That is where I'm seeing that I have access to o3 mini.

@gianni-rosato commented on GitHub (Feb 1, 2025): Are you able to go to `https://github.com/marketplace/models/azure-openai/o3-mini/playground`? That is where I'm seeing that I have access to o3 mini.
yindo commented 2026-02-22 18:27:31 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 1, 2025):

No, i get a 404

@timothycarambat commented on GitHub (Feb 1, 2025): No, i get a 404
yindo commented 2026-02-22 18:27:31 -05:00
Author
Owner
Copy Link
Copy Source

@gianni-rosato commented on GitHub (Feb 1, 2025):

Huh, interesting. I do have Copilot Pro, but I thought there was a free plan for GitHub Models.

@gianni-rosato commented on GitHub (Feb 1, 2025): Huh, interesting. I do have Copilot Pro, but I thought there was a free plan for GitHub Models.
yindo commented 2026-02-22 18:27:31 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Oct 1, 2025):

Resolved via #4467

@timothycarambat commented on GitHub (Oct 1, 2025): Resolved via #4467
yindo changed title from [CHORE]: OpenAI `o#` models require different `max_tokens` parameter to [GH-ISSUE #3085] [CHORE]: OpenAI `o#` models require different `max_tokens` parameter 2026-06-05 14:43:47 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1977
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?