Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2098] [BUG]: Bulk Link Scraper in docker version throws error #1366

New Issue
Closed
opened 2026-02-22 18:24:28 -05:00 by yindo · 5 comments
yindo commented 2026-02-22 18:24:28 -05:00
Owner
Copy Link
Copy Source

Originally created by @troymroberts on GitHub (Aug 12, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2098

How are you running AnythingLLM?

Docker (remote machine)

What happened?

embedding a single page works fine. Embedding via the bulk link scraper doesn't work. Receive the following errors in the log

[collector] info: Discovering links...
[collector] error: Failed to get page links from https://learn.microsoft.com/en-us/azure/well-architected/reliability. Error: Failed to launch the browser process!
[1006:1006:0810/184605.831237:FATAL:zygote_host_impl_linux.cc(127)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/main/docs/linux/suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox.

Are there known steps to reproduce?

No response

Originally created by @troymroberts on GitHub (Aug 12, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2098 ### How are you running AnythingLLM? Docker (remote machine) ### What happened? embedding a single page works fine. Embedding via the bulk link scraper doesn't work. Receive the following errors in the log [collector] info: Discovering links... [collector] error: Failed to get page links from https://learn.microsoft.com/en-us/azure/well-architected/reliability. Error: Failed to launch the browser process! [1006:1006:0810/184605.831237:FATAL:zygote_host_impl_linux.cc(127)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/main/docs/linux/suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox. ### Are there known steps to reproduce? _No response_
yindo added the possible buginvestigating labels 2026-02-22 18:24:28 -05:00
yindo closed this issue 2026-02-22 18:24:28 -05:00
yindo commented 2026-02-22 18:24:28 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Aug 12, 2024):

When did you pull the docker image for this? We just pushed an update a few hours ago to the bulk link scraper and I want to verify if that is what's causing the issue or not.

@shatfield4 commented on GitHub (Aug 12, 2024): When did you pull the docker image for this? We just pushed an update a few hours ago to the bulk link scraper and I want to verify if that is what's causing the issue or not.
yindo commented 2026-02-22 18:24:28 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 12, 2024):

@shatfield4 This is not the case. The issue @troymroberts is having is that they have run the docker container without the --cap-add SYS_ADMIN as defined in our documentation in all places. Without this param the puppeteer process cannot safely run a sandbox chrome process
https://docs.anythingllm.com/installation/self-hosted/local-docker#recommend-way-to-run-dockerized-anythingllm

@timothycarambat commented on GitHub (Aug 12, 2024): @shatfield4 This is not the case. The issue @troymroberts is having is that they have run the docker container without the `--cap-add SYS_ADMIN` as defined in our documentation in all places. Without this param the puppeteer process cannot safely run a sandbox chrome process https://docs.anythingllm.com/installation/self-hosted/local-docker#recommend-way-to-run-dockerized-anythingllm
yindo commented 2026-02-22 18:24:28 -05:00
Author
Owner
Copy Link
Copy Source

@troymroberts commented on GitHub (Aug 13, 2024):

@shatfield4 This is not the case. The issue @troymroberts is having is that they have run the docker container without the --cap-add SYS_ADMIN as defined in our documentation in all places. Without this param the puppeteer process cannot safely run a sandbox chrome process https://docs.anythingllm.com/installation/self-hosted/local-docker#recommend-way-to-run-dockerized-anythingllm

This is correct, this was the issue. My apologies.

@troymroberts commented on GitHub (Aug 13, 2024): > @shatfield4 This is not the case. The issue @troymroberts is having is that they have run the docker container without the `--cap-add SYS_ADMIN` as defined in our documentation in all places. Without this param the puppeteer process cannot safely run a sandbox chrome process https://docs.anythingllm.com/installation/self-hosted/local-docker#recommend-way-to-run-dockerized-anythingllm This is correct, this was the issue. My apologies.
yindo commented 2026-02-22 18:24:28 -05:00
Author
Owner
Copy Link
Copy Source

@coffeverton commented on GitHub (Oct 29, 2024):

Hi, I'm also having this issue. The error message is the same. Can you help me?

anythingllm  | [collector] info: Found 1 links to scrape.
anythingllm  | [collector] info: Starting bulk scraping...
anythingllm  | [collector] info: Scraping 1/1: https://suporte.agencialink.com
anythingllm  | [collector] error: Failed to scrape https://suporte.agencialink.com. Error: Failed to launch the browser process!
anythingllm  | [179:179:1029/185118.583950:FATAL:zygote_host_impl_linux.cc(127)] No usable sandbox! Update your kernel or see https://chrom...

My docker-compose.yml is based on this one: https://github.com/Mintplex-Labs/anything-llm/blob/master/docker/HOW_TO_USE_DOCKER.md:

version: '3.8'
services:
  anythingllm:
    image: mintplexlabs/anythingllm
    container_name: anythingllm
    ports:
      - "3007:3001"
    cap_add:
      - SYS_ADMIN
    environment:
    # Adjust for your environment
      - STORAGE_DIR=/app/server/storage
      - JWT_SECRET="f..."
      - EMBEDDING_ENGINE='openai'
      - OPEN_AI_KEY=sk-....
      - EMBEDDING_MODEL_PREF='text-embedding-ada-002'
      - VECTOR_DB=lancedb
      - WHISPER_PROVIDER="openai"
      - TTS_PROVIDER="openai"
      - PASSWORDMINCHAR=8
      # Add any other keys here for services or settings
      # you can find in the docker/.env.example file
      - DISABLE_TELEMETRY="true"
    volumes:
      - anythingllm_storage:/app/server/storage
    restart: always

volumes:
  anythingllm_storage:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /path/to/storage
@coffeverton commented on GitHub (Oct 29, 2024): Hi, I'm also having this issue. The error message is the same. Can you help me? ``` anythingllm | [collector] info: Found 1 links to scrape. anythingllm | [collector] info: Starting bulk scraping... anythingllm | [collector] info: Scraping 1/1: https://suporte.agencialink.com anythingllm | [collector] error: Failed to scrape https://suporte.agencialink.com. Error: Failed to launch the browser process! anythingllm | [179:179:1029/185118.583950:FATAL:zygote_host_impl_linux.cc(127)] No usable sandbox! Update your kernel or see https://chrom... ``` My docker-compose.yml is based on this one: https://github.com/Mintplex-Labs/anything-llm/blob/master/docker/HOW_TO_USE_DOCKER.md: ``` version: '3.8' services: anythingllm: image: mintplexlabs/anythingllm container_name: anythingllm ports: - "3007:3001" cap_add: - SYS_ADMIN environment: # Adjust for your environment - STORAGE_DIR=/app/server/storage - JWT_SECRET="f..." - EMBEDDING_ENGINE='openai' - OPEN_AI_KEY=sk-.... - EMBEDDING_MODEL_PREF='text-embedding-ada-002' - VECTOR_DB=lancedb - WHISPER_PROVIDER="openai" - TTS_PROVIDER="openai" - PASSWORDMINCHAR=8 # Add any other keys here for services or settings # you can find in the docker/.env.example file - DISABLE_TELEMETRY="true" volumes: - anythingllm_storage:/app/server/storage restart: always volumes: anythingllm_storage: driver: local driver_opts: type: none o: bind device: /path/to/storage ```
yindo commented 2026-02-22 18:24:29 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Oct 29, 2024):

again

Error: Failed to launch the browser process!
You cannot run puppeteer because https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#setting-up-chrome-linux-sandbox

This is presuming you are also not running the container as root. These are user config issues

@timothycarambat commented on GitHub (Oct 29, 2024): again > Error: Failed to launch the browser process! You cannot run puppeteer because https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#setting-up-chrome-linux-sandbox This is presuming you are also not running the container as root. These are user config issues
yindo changed title from [BUG]: Bulk Link Scraper in docker version throws error to [GH-ISSUE #2098] [BUG]: Bulk Link Scraper in docker version throws error 2026-06-05 14:40:22 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1366
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?