Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1501] [BUG]: Confluence connector only save partial document body in JSON #962

New Issue
Closed
opened 2026-02-22 18:22:26 -05:00 by yindo · 5 comments
yindo commented 2026-02-22 18:22:26 -05:00
Owner
Copy Link
Copy Source

Originally created by @jazelly on GitHub (May 22, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1501

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

Local development

What happened?

After confluence connector scraping confluence documents, the document bodies are not fully saved in JSON under storage.

After embedding them, it will not provide useful info as expected. For example, we have a confluence doc containing some code snippets and would like to ask questions to retrieve that. The code snippets is lost after scraping, however, which caused the LLM to response basic info.

I am not sure if this is a limitation of Atlassian API but surely users would expect more than just some basic info of the confluence documents.

Are there known steps to reproduce?

No response

Originally created by @jazelly on GitHub (May 22, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1501 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? Local development ### What happened? After confluence connector scraping confluence documents, the document bodies are not fully saved in JSON under `storage`. After embedding them, it will not provide useful info as expected. For example, we have a confluence doc containing some code snippets and would like to ask questions to retrieve that. The code snippets is lost after scraping, however, which caused the LLM to response basic info. I am not sure if this is a limitation of Atlassian API but surely users would expect more than just some basic info of the confluence documents. ### Are there known steps to reproduce? _No response_
yindo added the possible bugcore-team-onlyinvestigating labels 2026-02-22 18:22:26 -05:00
yindo closed this issue 2026-02-22 18:22:26 -05:00
yindo commented 2026-02-22 18:22:26 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 23, 2024):

@jazelly the pageContent of the associated docment is empty?

@timothycarambat commented on GitHub (May 23, 2024): @jazelly the `pageContent` of the associated docment is empty?
yindo commented 2026-02-22 18:22:26 -05:00
Author
Owner
Copy Link
Copy Source

@jazelly commented on GitHub (May 23, 2024):

@timothycarambat the pageContent is not empty. It has content, but just not include script content, e.g.

VIEW ALL\nsql\nASSIGN TO AN ACCOUNT\nThe account must already exist.\nsql\n

Notice the sql in the pageContent, which is supposed to be a SQL command. LLM makes up the answers when we ask a question related to that, since the prompt contains no reference to the real command

@jazelly commented on GitHub (May 23, 2024): @timothycarambat the `pageContent` is not empty. It has content, but just not include script content, e.g. ``` VIEW ALL\nsql\nASSIGN TO AN ACCOUNT\nThe account must already exist.\nsql\n ``` Notice the `sql` in the `pageContent`, which is supposed to be a SQL command. LLM makes up the answers when we ask a question related to that, since the prompt contains no reference to the real command
yindo commented 2026-02-22 18:22:26 -05:00
Author
Owner
Copy Link
Copy Source

@jainpradeep commented on GitHub (May 30, 2024):

Issue faced with local deployment as well. LLM responses are poor.

@jainpradeep commented on GitHub (May 30, 2024): Issue faced with local deployment as well. LLM responses are poor.
yindo commented 2026-02-22 18:22:26 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 30, 2024):

Issue faced with local deployment as well. LLM responses are poor.

Has nothing to do with the deployment method or RAG structure, the RAG results are bad because the scraper is returning poor information from the documents. As @jazelly mentions, it seems like some non-text blocks are not returned or parsed using the Langchain parser - which is where this lies

@timothycarambat commented on GitHub (May 30, 2024): > Issue faced with local deployment as well. LLM responses are poor. Has nothing to do with the deployment method or RAG structure, the RAG results are bad because the scraper is returning poor information from the documents. As @jazelly mentions, it seems like some non-text blocks are not returned or parsed using the Langchain parser - which is where this lies
yindo commented 2026-02-22 18:22:26 -05:00
Author
Owner
Copy Link
Copy Source

@jazelly commented on GitHub (May 30, 2024):

This might be an issue better for LangChain community.

To us, the current solution is nothing more than writing our own scraper to download these documents, and upload them to anything-llm via APIs

@jazelly commented on GitHub (May 30, 2024): This might be an issue better for LangChain community. To us, the current solution is nothing more than writing our own scraper to download these documents, and upload them to anything-llm via APIs
yindo changed title from [BUG]: Confluence connector only save partial document body in JSON to [GH-ISSUE #1501] [BUG]: Confluence connector only save partial document body in JSON 2026-06-05 14:38:10 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label core-team-only investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#962
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?