Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5628] [BUG]: Statistical / Analytical Queries Returning Incorrect or Off-Target Results #5205

New Issue
Closed
opened 2026-06-05 14:52:38 -04:00 by yindo · 2 comments
yindo commented 2026-06-05 14:52:38 -04:00
Owner
Copy Link
Copy Source

Originally created by @punk1234567 on GitHub (May 14, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5628

Hi Team,

I am seeing inconsistent and inaccurate responses from AnythingLLM when running statistical or analytical queries against tabular datasets. I need accurate and precise responses to statistical questions for decision-making purposes. If the outputs are inaccurate, the system becomes unreliable for analysis workflows. Please advise on the recommended approach to improve analytical accuracy in AnythingLLM.

Issue:
The model frequently returns:

  • Incorrect aggregations
  • Inaccurate counts or sums
  • Responses that do not align with the source dataset

In some cases, it appears the LLM is interpreting chunks of text semantically rather than performing structured data analysis on the full dataset.

Environment:

  • AnythingLLM Version: Desktop v1.12.0
  • LLM Provider: Google Gemma 4 31B 1T
  • Embedding Model: Cohere Embed English v3
  • Vector Database: LanceDB
  • Text Chunk Size: 1945
  • Text Chunk Overlap: 400
  • OS: Windows 11

Sample Dataset:
Below are 2 dummy records out of ~120 records provided to the workspace as JSON input.

[
  {
    "#": 1,
    "Product Name": "Smart Retail Display",
    "Product Value": "$30,875",
    "Client": "Nova Retail",
    "Status": "20 smart display packages deployed across multiple retail locations.",
    "Account Manager": "Daniel"
  },
  {
    "#": 2,
    "Product Name": "Coffee Machine",
    "Product Value": "$29,750",
    "Client": "BrewTech",
    "Status": "Machine Setup: Draft shared on 19th Jan 2023.",
    "Account Manager": "Daniel"
  }
]

Steps to Reproduce:

  1. Upload JSON dataset into workspace
  2. Query the dataset through chat
  3. Ask statistical or analytical questions such as totals, counts, or aggregations

Observed Result:
The returned values are often inaccurate or inconsistent with the actual dataset.

Expected Result:
AnythingLLM should either:

  • Perform deterministic structured analysis against the dataset, or
  • Clearly indicate limitations when the model is estimating instead of calculating

Additional Notes:

  • I tested with multiple models including Gemma 4 and Cohere models, with similar results.
  • My expectation is accurate tabular analysis rather than semantic summarization.
  • Please let me know whether a different ingestion method, agent/tool configuration workflow is recommended for analytical use cases.

Thanks in advance for your guidance.
Sam

Originally created by @punk1234567 on GitHub (May 14, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5628 Hi Team, I am seeing inconsistent and inaccurate responses from AnythingLLM when running statistical or analytical queries against tabular datasets. I need accurate and precise responses to statistical questions for decision-making purposes. If the outputs are inaccurate, the system becomes unreliable for analysis workflows. Please advise on the recommended approach to improve analytical accuracy in AnythingLLM. Issue: The model frequently returns: * Incorrect aggregations * Inaccurate counts or sums * Responses that do not align with the source dataset In some cases, it appears the LLM is interpreting chunks of text semantically rather than performing structured data analysis on the full dataset. Environment: * AnythingLLM Version: Desktop v1.12.0 * LLM Provider: Google Gemma 4 31B 1T * Embedding Model: Cohere Embed English v3 * Vector Database: LanceDB * Text Chunk Size: 1945 * Text Chunk Overlap: 400 * OS: Windows 11 Sample Dataset: Below are 2 dummy records out of ~120 records provided to the workspace as JSON input. ```json [ { "#": 1, "Product Name": "Smart Retail Display", "Product Value": "$30,875", "Client": "Nova Retail", "Status": "20 smart display packages deployed across multiple retail locations.", "Account Manager": "Daniel" }, { "#": 2, "Product Name": "Coffee Machine", "Product Value": "$29,750", "Client": "BrewTech", "Status": "Machine Setup: Draft shared on 19th Jan 2023.", "Account Manager": "Daniel" } ] ``` Steps to Reproduce: 1. Upload JSON dataset into workspace 2. Query the dataset through chat 3. Ask statistical or analytical questions such as totals, counts, or aggregations Observed Result: The returned values are often inaccurate or inconsistent with the actual dataset. Expected Result: AnythingLLM should either: * Perform deterministic structured analysis against the dataset, or * Clearly indicate limitations when the model is estimating instead of calculating Additional Notes: * I tested with multiple models including Gemma 4 and Cohere models, with similar results. * My expectation is accurate tabular analysis rather than semantic summarization. * Please let me know whether a different ingestion method, agent/tool configuration workflow is recommended for analytical use cases. Thanks in advance for your guidance. Sam
yindo added the possible bug label 2026-06-05 14:52:38 -04:00
yindo closed this issue 2026-06-05 14:52:38 -04:00
yindo commented 2026-06-05 14:52:38 -04:00
Author
Owner
Copy Link
Copy Source

@punk1234567 commented on GitHub (May 15, 2026):

Dear team, please let me know if there is any update. Thank you.

Sam

<!-- gh-comment-id:4459517568 --> @punk1234567 commented on GitHub (May 15, 2026): Dear team, please let me know if there is any update. Thank you. Sam
yindo commented 2026-06-05 14:52:38 -04:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (May 15, 2026):

This is just caused by the Gemma 4 model hallucinating. I would suggest testing out other models (cloud models like Claude/Codex) and/or cutting out as much context as you can from the chat when asking questions. In general, when using up more of the context window, LLM models tend to hallucinate much more. If you can give the LLM only the required context to answer the question it should perform better. Unfortunately this will just take more testing on your end to figure out which model is the best for your use case.

Closing since this is a model issue, not a bug with AnythingLLM.

<!-- gh-comment-id:4462880753 --> @shatfield4 commented on GitHub (May 15, 2026): This is just caused by the Gemma 4 model hallucinating. I would suggest testing out other models (cloud models like Claude/Codex) and/or cutting out as much context as you can from the chat when asking questions. In general, when using up more of the context window, LLM models tend to hallucinate much more. If you can give the LLM only the required context to answer the question it should perform better. Unfortunately this will just take more testing on your end to figure out which model is the best for your use case. Closing since this is a model issue, not a bug with AnythingLLM.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5205
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?