Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #334] Collector failing with timeout error, cannot restart. #193

New Issue
Closed
opened 2026-02-22 18:18:18 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:18:18 -05:00
Owner
Copy Link
Copy Source

Originally created by @Britman72 on GitHub (Nov 4, 2023).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/334

I was trying to collect an entire website when it bailed near the end. Here's the log below. When I try to restart the menu, it also throws an error.

Working on https://www.nbtclothing.com... [INFO] Starting Chromium download. 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 86.8M/86.8M [00:02<00:00, 35.7Mb/s] [INFO] Beginning extraction [INFO] Chromium extracted to: /Users/britman/Library/Application Support/pyppeteer/local-chromium/588429 [nltk_data] Downloading package punkt to /Users/britman/nltk_data... [nltk_data] Unzipping tokenizers/punkt.zip. [nltk_data] Downloading package averaged_perceptron_tagger to [nltk_data] /Users/britman/nltk_data... [nltk_data] Unzipping taggers/averaged_perceptron_tagger.zip. Working on https://www.nbtclothing.com/collections/all... Working on https://www.nbtclothing.com/account/login... Working on https://www.nbtclothing.com/collections/best-sellers... Working on https://www.nbtclothing.com/collections/best-sellers... Working on https://www.nbtclothing.com/policies/shipping-policy... Working on https://www.nbtclothing.com/collections/all... Working on https://www.nbtclothing.com/pages/returns-exchanges... Working on https://www.nbtclothing.com/products/nomad... Traceback (most recent call last): File "/Users/britman/textrify/AnythingLLM/collector/main.py", line 84, in <module> main() File "/Users/britman/textrify/AnythingLLM/collector/main.py", line 58, in main crawler() File "/Users/britman/textrify/AnythingLLM/collector/scripts/link.py", line 97, in crawler parse_links(links) File "/Users/britman/textrify/AnythingLLM/collector/scripts/link.py", line 134, in parse_links req.html.render(timeout=10) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/britman/textrify/AnythingLLM/collector/v-env/lib/python3.11/site-packages/requests_html.py", line 605, in render raise MaxRetries("Unable to render the page. Try increasing timeout") requests_html.MaxRetries: Unable to render the page. Try increasing timeout

Restarting it:

(v-env) britman@Als-Office collector % python main.py Traceback (most recent call last): File "/Users/britman/textrify/AnythingLLM/collector/main.py", line 4, in <module> from scripts.link import link, links, crawler File "/Users/britman/textrify/AnythingLLM/collector/scripts/link.py", line 3, in <module> from requests_html import HTMLSession File "/Users/britman/textrify/AnythingLLM/collector/v-env/lib/python3.11/site-packages/requests_html.py", line 489 return self IndentationError: unexpected indent (v-env) britman@Als-Office collector % python main.py Traceback (most recent call last): File "/Users/britman/textrify/AnythingLLM/collector/main.py", line 4, in <module> from scripts.link import link, links, crawler File "/Users/britman/textrify/AnythingLLM/collector/scripts/link.py", line 3, in <module> from requests_html import HTMLSession File "/Users/britman/textrify/AnythingLLM/collector/v-env/lib/python3.11/site-packages/requests_html.py", line 489 return self IndentationError: unexpected indent

Originally created by @Britman72 on GitHub (Nov 4, 2023). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/334 I was trying to collect an entire website when it bailed near the end. Here's the log below. When I try to restart the menu, it also throws an error. `Working on https://www.nbtclothing.com... [INFO] Starting Chromium download. 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 86.8M/86.8M [00:02<00:00, 35.7Mb/s] [INFO] Beginning extraction [INFO] Chromium extracted to: /Users/britman/Library/Application Support/pyppeteer/local-chromium/588429 [nltk_data] Downloading package punkt to /Users/britman/nltk_data... [nltk_data] Unzipping tokenizers/punkt.zip. [nltk_data] Downloading package averaged_perceptron_tagger to [nltk_data] /Users/britman/nltk_data... [nltk_data] Unzipping taggers/averaged_perceptron_tagger.zip. Working on https://www.nbtclothing.com/collections/all... Working on https://www.nbtclothing.com/account/login... Working on https://www.nbtclothing.com/collections/best-sellers... Working on https://www.nbtclothing.com/collections/best-sellers... Working on https://www.nbtclothing.com/policies/shipping-policy... Working on https://www.nbtclothing.com/collections/all... Working on https://www.nbtclothing.com/pages/returns-exchanges... Working on https://www.nbtclothing.com/products/nomad... Traceback (most recent call last): File "/Users/britman/textrify/AnythingLLM/collector/main.py", line 84, in <module> main() File "/Users/britman/textrify/AnythingLLM/collector/main.py", line 58, in main crawler() File "/Users/britman/textrify/AnythingLLM/collector/scripts/link.py", line 97, in crawler parse_links(links) File "/Users/britman/textrify/AnythingLLM/collector/scripts/link.py", line 134, in parse_links req.html.render(timeout=10) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/britman/textrify/AnythingLLM/collector/v-env/lib/python3.11/site-packages/requests_html.py", line 605, in render raise MaxRetries("Unable to render the page. Try increasing timeout") requests_html.MaxRetries: Unable to render the page. Try increasing timeout` Restarting it: `(v-env) britman@Als-Office collector % python main.py Traceback (most recent call last): File "/Users/britman/textrify/AnythingLLM/collector/main.py", line 4, in <module> from scripts.link import link, links, crawler File "/Users/britman/textrify/AnythingLLM/collector/scripts/link.py", line 3, in <module> from requests_html import HTMLSession File "/Users/britman/textrify/AnythingLLM/collector/v-env/lib/python3.11/site-packages/requests_html.py", line 489 return self IndentationError: unexpected indent (v-env) britman@Als-Office collector % python main.py Traceback (most recent call last): File "/Users/britman/textrify/AnythingLLM/collector/main.py", line 4, in <module> from scripts.link import link, links, crawler File "/Users/britman/textrify/AnythingLLM/collector/scripts/link.py", line 3, in <module> from requests_html import HTMLSession File "/Users/britman/textrify/AnythingLLM/collector/v-env/lib/python3.11/site-packages/requests_html.py", line 489 return self IndentationError: unexpected indent `
yindo closed this issue 2026-02-22 18:18:18 -05:00
yindo commented 2026-02-22 18:18:19 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Nov 6, 2023):

The restart error is bizzare since the error is coming from a provider and not code we wrote (requests_html.py). Additionally the original error came from the HTTP timeout - again not related to the code but related to the website being scrape in question and on a timeout it should abort.

Closing since this is not part of the code we wrote or maintain. I do notice you are using python3.11, can you do a new v-env with python3.9? This should not impact the requests library

@timothycarambat commented on GitHub (Nov 6, 2023): The restart error is bizzare since the error is coming from a provider and not code we wrote (requests_html.py). Additionally the original error came from the HTTP timeout - again not related to the code but related to the website being scrape in question and on a timeout it should abort. Closing since this is not part of the code we wrote or maintain. I do notice you are using python3.11, can you do a new v-env with python3.9? This should not impact the requests library
yindo changed title from Collector failing with timeout error, cannot restart. to [GH-ISSUE #334] Collector failing with timeout error, cannot restart. 2026-06-05 14:33:59 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#193
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?