Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2459] [BUG]: Bulk Link Scraper does NOT work for localhost website #1597

New Issue
Closed
opened 2026-02-22 18:25:36 -05:00 by yindo · 0 comments
yindo commented 2026-02-22 18:25:36 -05:00
Owner
Copy Link
Copy Source

Originally created by @lawrenceching on GitHub (Oct 11, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2459

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

I can't use Bulk Link Scraper to scrap localhost website

I'm using windows desktop app in version v1.6.7

image

"Scraping website" forever.

And the localhost website looks like:

$ curl localhost:8000 -v
* Host localhost:8000 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:8000...
*   Trying 127.0.0.1:8000...
* Connected to localhost (127.0.0.1) port 8000
> GET / HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/8.8.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 11 Oct 2024 17:09:23 GMT
< Last-modified: Fri, 11 Oct 2024 16:58:23 GMT
< Content-type: text/html; charset=UTF-8
< Content-length: 175
<
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
</head>
<body>
<h1>Directory listing for &#x2F;</h1>
<ul>
<li><a href="test.txt">test.txt</a></li>
</ul>
</body>
</html>

Logs

collector-2024-10-12.log
backend-2024-10-12.log

Are there known steps to reproduce?

Scrap website served in localhost

Originally created by @lawrenceching on GitHub (Oct 11, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2459 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? I can't use Bulk Link Scraper to scrap localhost website I'm using windows desktop app in version v1.6.7 ![image](https://github.com/user-attachments/assets/e79669f0-d504-4459-b827-66afcfc779e4) "Scraping website" forever. And the localhost website looks like: ```bash $ curl localhost:8000 -v * Host localhost:8000 was resolved. * IPv6: ::1 * IPv4: 127.0.0.1 * Trying [::1]:8000... * Trying 127.0.0.1:8000... * Connected to localhost (127.0.0.1) port 8000 > GET / HTTP/1.1 > Host: localhost:8000 > User-Agent: curl/8.8.0 > Accept: */* > < HTTP/1.1 200 OK < Date: Fri, 11 Oct 2024 17:09:23 GMT < Last-modified: Fri, 11 Oct 2024 16:58:23 GMT < Content-type: text/html; charset=UTF-8 < Content-length: 175 < <!DOCTYPE html> <html> <head> <meta charset="utf-8"/> </head> <body> <h1>Directory listing for &#x2F;</h1> <ul> <li><a href="test.txt">test.txt</a></li> </ul> </body> </html> ``` ## Logs [collector-2024-10-12.log](https://github.com/user-attachments/files/17345670/collector-2024-10-12.log) [backend-2024-10-12.log](https://github.com/user-attachments/files/17345671/backend-2024-10-12.log) ### Are there known steps to reproduce? Scrap website served in localhost
yindo added the possible buginvestigating labels 2026-02-22 18:25:36 -05:00
yindo closed this issue 2026-02-22 18:25:36 -05:00
yindo changed title from [BUG]: Bulk Link Scraper does NOT work for localhost website to [GH-ISSUE #2459] [BUG]: Bulk Link Scraper does NOT work for localhost website 2026-06-05 14:41:39 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1597
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?