[GH-ISSUE #51] [Bug]: PentAGI doesn't actually work well with custom models hosted locally such as Azure OpenAI models. #26

Closed
opened 2026-06-06 22:08:05 -04:00 by yindo · 2 comments
Owner

Originally created by @Aishani20278 on GitHub (Aug 16, 2025).
Original GitHub issue: https://github.com/vxcontrol/pentagi/issues/51

Originally assigned to: @asdek on GitHub.

Affected Component

AI Agents (Researcher/Developer/...)

Describe the bug

Configuring the custom LLMs section to include Azure OpenAI model deployments along with a custom config file doesn't actually configure pentagi to run and use Azure OpenAI. Not even a basic flow is being created.

Steps to Reproduce

  1. Go to the .env file and in the custom LLMs section fill out the following details -
    LLM_SERVER_URL=
    LLM_SERVER_KEY=
    LLM_SERVER_MODEL=
    LLM_SERVER_CONFIG_PATH=
    LLM_SERVER_LEGACY_REASONING=false

  2. Run PentAGI with the following changes.

System Configuration

Deployment type: Custom Deployment.
LLM Provider: Azure OpenAI

Logs and Artifacts

No response

Screenshots or Recordings

No response

Verification

  • I have checked that this issue hasn't been already reported
  • I have provided all relevant configuration files (with sensitive data removed)
  • I have included relevant logs and error messages
  • I am running the latest version of PentAGI
Originally created by @Aishani20278 on GitHub (Aug 16, 2025). Original GitHub issue: https://github.com/vxcontrol/pentagi/issues/51 Originally assigned to: @asdek on GitHub. ### Affected Component AI Agents (Researcher/Developer/...) ### Describe the bug Configuring the custom LLMs section to include Azure OpenAI model deployments along with a custom config file doesn't actually configure pentagi to run and use Azure OpenAI. Not even a basic flow is being created. ### Steps to Reproduce 1. Go to the .env file and in the custom LLMs section fill out the following details - LLM_SERVER_URL=<your llm server url> LLM_SERVER_KEY=<your azure model key> LLM_SERVER_MODEL=<azure deployment name> LLM_SERVER_CONFIG_PATH=<custom config file created based on example config files> LLM_SERVER_LEGACY_REASONING=false 2. Run PentAGI with the following changes. ### System Configuration Deployment type: Custom Deployment. LLM Provider: Azure OpenAI ### Logs and Artifacts _No response_ ### Screenshots or Recordings _No response_ ### Verification - [x] I have checked that this issue hasn't been already reported - [x] I have provided all relevant configuration files (with sensitive data removed) - [x] I have included relevant logs and error messages - [x] I am running the latest version of PentAGI
yindo added the enhancement label 2026-06-06 22:08:05 -04:00
yindo closed this issue 2026-06-06 22:08:05 -04:00
Author
Owner

@asdek commented on GitHub (Aug 17, 2025):

Hello @Aishani20278

Thanks for reporting this issue. Unfortunately, Azure OpenAI is not currently supported because the client implementation in langchaingo is quite outdated and doesn't fully support reasoning models.

I'll work on adding proper Azure OpenAI support in the upcoming release and will update this issue once Azure OpenAI support is implemented.

<!-- gh-comment-id:3194392349 --> @asdek commented on GitHub (Aug 17, 2025): Hello @Aishani20278 Thanks for reporting this issue. Unfortunately, Azure OpenAI is not currently supported because the client implementation in langchaingo is quite outdated and doesn't fully support reasoning models. I'll work on adding proper Azure OpenAI support in the upcoming release and will update this issue once Azure OpenAI support is implemented.
Author
Owner

@asdek commented on GitHub (Feb 21, 2026):

hey @Aishani20278

now PentAGI fully supported LiteLLM and you can try to join Azure OpenAI through LiteLLM

Closes #102

<!-- gh-comment-id:3939803455 --> @asdek commented on GitHub (Feb 21, 2026): hey @Aishani20278 now PentAGI fully supported LiteLLM and you can try to join Azure OpenAI through LiteLLM Closes #102
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: vxcontrol/pentagi#26