Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4088] [BUG]: Multi-modal Content Array Not Supported in Database Storage #2605

New Issue
Closed
opened 2026-02-22 18:30:25 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:30:25 -05:00
Owner
Copy Link
Copy Source

Originally created by @bernarduswillson on GitHub (Jul 2, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4088

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

Docker (remote machine)

What happened?

AnythingLLM successfully processes multi-modal prompts (text + images) and generates responses, but fails when attempting to save the conversation to the database due to Prisma schema expecting a string for the prompt field instead of an array of content objects.

Are there known steps to reproduce?

  1. Send a request to AnythingLLM with multi-modal content format:
{
  messages: [
    {
      role: "user",
      content: [
        {
          type: "text", 
          text: "Please evaluate these image entries\n\n ID: 1\nCaption: \"This is a test caption\""
        },
        {
          type: "image_url",
          image_url: {
            url: "data:image/jpeg;base64,{base64_image}",
            detail: "low"
          }
        }
      ]
    }
  ]
}
  1. Observe that the LLM processes the request successfully and logs a valid response
  2. Check logs for database errors during conversation saving

Error logs:

Invalid `prisma.workspace_chats.create()` invocation:
{
  data: {
    workspaceId: 13,
    prompt: [
      {
        type: "text",
        text: "Please evaluate these image entries\n\n ID: 1\nCaption: \"This is a test caption\""
      },
      {
        type: "image_url",
        image_url: {
          url: "data:image/jpeg;base64,/9j/2wBDA..."
        }
      }
    ],
    ...
  }
}

Argument `prompt`: Invalid value provided. Expected String, provided (Object, Object).

LLM response logs:

{
  "text": "```json\n{\n  \"results\": [\n    {\n      \"id\": \"1\",\n      \"is_image_valid\": false,\n      \"is_caption_valid\": false,\n      \"explanation\": \"The image appears to be a screenshot of an app interface, not suitable for a daycare setting. The caption 'This is a test caption' is not meaningful or descriptive of the image content.\"\n    }\n  ]\n}\n```",
  "sources": [],
  "type": "chat",
  "metrics": {
    "prompt_tokens": 696,
    "completion_tokens": 84,
    "total_tokens": 780,
    "outputTps": 30.08595988538682,
    "duration": 2.792
  }
}
  • LLM successfully processes the multi-modal prompt
  • Valid response is generated (as seen in logs)
  • Database save operation fails with Prisma validation error
  • Error: Argument 'prompt': Invalid value provided. Expected String, provided (Object, Object).
Originally created by @bernarduswillson on GitHub (Jul 2, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4088 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? Docker (remote machine) ### What happened? AnythingLLM successfully processes multi-modal prompts (text + images) and generates responses, but fails when attempting to save the conversation to the database due to Prisma schema expecting a string for the prompt field instead of an array of content objects. ### Are there known steps to reproduce? 1. Send a request to AnythingLLM with multi-modal content format: ``` { messages: [ { role: "user", content: [ { type: "text", text: "Please evaluate these image entries\n\n ID: 1\nCaption: \"This is a test caption\"" }, { type: "image_url", image_url: { url: "data:image/jpeg;base64,{base64_image}", detail: "low" } } ] } ] } ``` 2. Observe that the LLM processes the request successfully and logs a valid response 3. Check logs for database errors during conversation saving Error logs: ``` Invalid `prisma.workspace_chats.create()` invocation: { data: { workspaceId: 13, prompt: [ { type: "text", text: "Please evaluate these image entries\n\n ID: 1\nCaption: \"This is a test caption\"" }, { type: "image_url", image_url: { url: "data:image/jpeg;base64,/9j/2wBDA..." } } ], ... } } Argument `prompt`: Invalid value provided. Expected String, provided (Object, Object). ``` LLM response logs: ``` { "text": "```json\n{\n \"results\": [\n {\n \"id\": \"1\",\n \"is_image_valid\": false,\n \"is_caption_valid\": false,\n \"explanation\": \"The image appears to be a screenshot of an app interface, not suitable for a daycare setting. The caption 'This is a test caption' is not meaningful or descriptive of the image content.\"\n }\n ]\n}\n```", "sources": [], "type": "chat", "metrics": { "prompt_tokens": 696, "completion_tokens": 84, "total_tokens": 780, "outputTps": 30.08595988538682, "duration": 2.792 } } ``` - LLM **successfully** processes the multi-modal prompt - Valid response is generated (as seen in logs) - Database save operation fails with Prisma validation error - Error: Argument 'prompt': Invalid value provided. Expected String, provided (Object, Object).
yindo added the possible buginvestigating labels 2026-02-22 18:30:25 -05:00
yindo closed this issue 2026-02-22 18:30:25 -05:00
yindo commented 2026-02-22 18:30:26 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Jul 2, 2025):

@bernarduswillson Could you tell me what model you're using for this? Also, are you doing this via the developer API or just via the UI?

I'm assuming that this is done via the API because of the message structure you provided but please correct me if I'm wrong.

@shatfield4 commented on GitHub (Jul 2, 2025): @bernarduswillson Could you tell me what model you're using for this? Also, are you doing this via the developer API or just via the UI? I'm assuming that this is done via the API because of the message structure you provided but please correct me if I'm wrong.
yindo commented 2026-02-22 18:30:26 -05:00
Author
Owner
Copy Link
Copy Source

@bernarduswillson commented on GitHub (Jul 2, 2025):

@shatfield4 I'm using GPT-4o for this, and yes, I'm doing it via the developer API.

@bernarduswillson commented on GitHub (Jul 2, 2025): @shatfield4 I'm using GPT-4o for this, and yes, I'm doing it via the developer API.
yindo changed title from [BUG]: Multi-modal Content Array Not Supported in Database Storage to [GH-ISSUE #4088] [BUG]: Multi-modal Content Array Not Supported in Database Storage 2026-06-05 14:47:31 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2605
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?