Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1854] [BUG]: Scraping my Confluence and I get a weird error about invalid data #1205

New Issue
Closed
opened 2026-02-22 18:23:41 -05:00 by yindo · 6 comments
yindo commented 2026-02-22 18:23:41 -05:00
Owner
Copy Link
Copy Source

Originally created by @JayCroghan on GitHub (Jul 11, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1854

Originally assigned to: @shatfield4, @timothycarambat on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

When I try to import my Confluence space I get the following error popup:

image

But then it continues in the background, I can see it is still pulling pages because it shows

[collector] info: [Confluence Loader]: Saving: ... to ...

in the docker logs for a long time after the initial error while it fetches the rest of the pages before it finally says

[backend] info: [CollectorApi] fetch failed

Which then leaves my Vector db empty. Anything I can do to make it ignore that one specific error coming from one specific page? My space is massive, 1.3gb.

Are there known steps to reproduce?

Just run the Confluence collector on my local Confluence or the one hosted on Attlassian with the same space imported.

Originally created by @JayCroghan on GitHub (Jul 11, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1854 Originally assigned to: @shatfield4, @timothycarambat on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? When I try to import my Confluence space I get the following error popup: <img width="316" alt="image" src="https://github.com/user-attachments/assets/68b4c350-e22f-4e6b-9c25-a87f63d2bf2c"> But then it continues in the background, I can see it is still pulling pages because it shows `[collector] info: [Confluence Loader]: Saving: ... to ...` in the docker logs for a long time after the initial error while it fetches the rest of the pages before it finally says `[backend] info: [CollectorApi] fetch failed` Which then leaves my Vector db empty. Anything I can do to make it ignore that one specific error coming from one specific page? My space is massive, 1.3gb. ### Are there known steps to reproduce? Just run the Confluence collector on my local Confluence or the one hosted on Attlassian with the same space imported.
yindo added the needs info / can't replicateinvestigating labels 2026-02-22 18:23:41 -05:00
yindo closed this issue 2026-02-22 18:23:41 -05:00
yindo commented 2026-02-22 18:23:41 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Jul 11, 2024):

What content do you have in the Confluence space? Are you able to make another space with just some text in it and test to see if that works?

@shatfield4 commented on GitHub (Jul 11, 2024): What content do you have in the Confluence space? Are you able to make another space with just some text in it and test to see if that works?
yindo commented 2026-02-22 18:23:41 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 11, 2024):

Collecting the space does not automatically embed it. First we fetch and then you pick and choose what files you wish to embed. As @shatfield4 suggested, is there a small space you can test with just to first ensure that its not some kind of install/config issue.

Also are there API key limits in place here? I imagine not for local installs, but likely yes for cloud

@timothycarambat commented on GitHub (Jul 11, 2024): Collecting the space _does not_ automatically embed it. First we fetch and then you pick and choose what files you wish to embed. As @shatfield4 suggested, is there a small space you can test with just to first ensure that its not some kind of install/config issue. Also are there API key limits in place here? I imagine not for local installs, but likely yes for cloud
yindo commented 2026-02-22 18:23:41 -05:00
Author
Owner
Copy Link
Copy Source

@JayCroghan commented on GitHub (Jul 12, 2024):

Ah ok my mistake. So I tried this from two confluence pages with the same space imported. If I attempt it from the hosted Atlassian page it does actually load the list all of the pages for me to import. I have just clicked on that right now and it seems to have worked.

The locally hosted one did not have the same success, once I put the URL for which I had to use a reverse proxy on https/443 because it would not allow me to use a non-standard port due to the regex you use to determine if the URL is valid or not, it throws the following error. I have checked 100s of times, this space is active at this address, if I add a page after the address it does not help, I have regenerated the access key, the email address is correct, Confluence is setup to use nginx reverse proxy (the conf needs changing when access this way) and it still always gives the same error.

[collector] error: undefined
[backend] info: [CollectorApi] Response could not be completed
[backend] info: [TELEMETRY SENT]
@JayCroghan commented on GitHub (Jul 12, 2024): Ah ok my mistake. So I tried this from two confluence pages with the same space imported. If I attempt it from the hosted Atlassian page it does actually load the list all of the pages for me to import. I have just clicked on that right now and it seems to have worked. The locally hosted one did not have the same success, once I put the URL for which I had to use a reverse proxy on https/443 because it would not allow me to use a non-standard port due to the regex you use to determine if the URL is valid or not, it throws the following error. I have checked 100s of times, this space is active at this address, if I add a page after the address it does not help, I have regenerated the access key, the email address is correct, Confluence is setup to use nginx reverse proxy (the conf needs changing when access this way) and it still always gives the same error. ``` [collector] error: undefined [backend] info: [CollectorApi] Response could not be completed [backend] info: [TELEMETRY SENT] ```
yindo commented 2026-02-22 18:23:41 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 12, 2024):

What does the url look like schema-wise? Does it have a port number or something to that effect? The regex for the URL pattern is pretty strict and I can imagine a locally hosted one will not pass

@timothycarambat commented on GitHub (Jul 12, 2024): What does the url look like schema-wise? Does it have a port number or something to that effect? The regex for the URL pattern is pretty strict and I can imagine a locally hosted one will not pass
yindo commented 2026-02-22 18:23:42 -05:00
Author
Owner
Copy Link
Copy Source

@JayCroghan commented on GitHub (Jul 12, 2024):

What does the url look like schema-wise? Does it have a port number or something to that effect? The regex for the URL pattern is pretty strict and I can imagine a locally hosted one will not pass

https://domain.net/display/XXX

The nginx reverse proxy on 443 connects to the non-standard port so the domain on https can be a normal port. It has a valid cert for domain.net too.

If I browse the URL I get exactly what I would expect.

image

@JayCroghan commented on GitHub (Jul 12, 2024): > What does the url look like schema-wise? Does it have a port number or something to that effect? The regex for the URL pattern is pretty strict and I can imagine a locally hosted one will not pass https://domain.net/display/XXX The nginx reverse proxy on 443 connects to the non-standard port so the domain on https can be a normal port. It has a valid cert for domain.net too. If I browse the URL I get exactly what I would expect. ![image](https://github.com/user-attachments/assets/25d31e20-8059-4e1b-b0bd-52a4ad6a13fe)
yindo commented 2026-02-22 18:23:42 -05:00
Author
Owner
Copy Link
Copy Source

@JayCroghan commented on GitHub (Jul 13, 2024):

What’s the best way for me to debug this and see what’s throwing the error? I didn’t see any options to add additional logging to the docker logs? I’m a software engineer so I’m handy enough with stuff. I was thinking of changing the regex on the URL and compiling from source before remembering I could use an nginx reverse proxy 😂 Any way to add verbose debug logging? Am I just stupid and missed that in the docs?

@JayCroghan commented on GitHub (Jul 13, 2024): What’s the best way for me to debug this and see what’s throwing the error? I didn’t see any options to add additional logging to the docker logs? I’m a software engineer so I’m handy enough with stuff. I was thinking of changing the regex on the URL and compiling from source before remembering I could use an nginx reverse proxy 😂 Any way to add verbose debug logging? Am I just stupid and missed that in the docs?
yindo changed title from [BUG]: Scraping my Confluence and I get a weird error about invalid data to [GH-ISSUE #1854] [BUG]: Scraping my Confluence and I get a weird error about invalid data 2026-06-05 14:39:31 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating needs info / can't replicate
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1205
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?