langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 04:25:29 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

dify 1.11.3 webscraper tool error #21666

New Issue
Closed
opened 2026-02-21 20:13:45 -05:00 by yindo · 12 comments
yindo commented 2026-02-21 20:13:45 -05:00
Owner
Copy Link
Copy Source

Originally created by @Capricorn0010 on GitHub (Jan 15, 2026).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.11.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

dify webscraper tools error {"webscraper": "tool invoke error: Command '['npm', 'install']' returned non-zero exit status 1."}

Image

✔️ Expected Behavior

true

❌ Actual Behavior

dify webscraper tools error {"webscraper": "tool invoke error: Command '['npm', 'install']' returned non-zero exit status 1."}

Image
Originally created by @Capricorn0010 on GitHub (Jan 15, 2026). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.11.3 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce dify webscraper tools error {"webscraper": "tool invoke error: Command '['npm', 'install']' returned non-zero exit status 1."} <img width="1660" height="654" alt="Image" src="https://github.com/user-attachments/assets/c9b22bc6-0ba3-45a2-8f06-ee237ff82d43" /> ### ✔️ Expected Behavior true ### ❌ Actual Behavior dify webscraper tools error {"webscraper": "tool invoke error: Command '['npm', 'install']' returned non-zero exit status 1."} <img width="1660" height="654" alt="Image" src="https://github.com/user-attachments/assets/c9b22bc6-0ba3-45a2-8f06-ee237ff82d43" />
yindo added the 🐞 bug label 2026-02-21 20:13:45 -05:00
yindo closed this issue 2026-02-21 20:13:45 -05:00
yindo commented 2026-02-21 20:13:46 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 15, 2026):

The npm install error you’re seeing with the webscraper tool in Dify 1.11.3 (Docker) is a known issue and is likely caused by a missing or misconfigured dependency in the container environment. The built-in webscraper tool is implemented in Python and doesn’t directly invoke npm, so this error probably comes from a supplementary script or plugin that tries to run npm install at runtime, relying on the Node.js/npm environment provided by the Docker image. The Dockerfile does install Node.js and system libraries, but does not run npm install for the webscraper tool at build time, so any runtime invocation depends on the environment being correct and all dependencies being present and writable by the non-root user "dify".

This class of error is common in Dify’s Docker setups, especially with plugins or tools that require extra dependencies not present in the base image. The best workaround is to manually install any missing dependencies inside the running container, or to ensure all required dependencies are listed in the tool’s or plugin’s requirements file so they’re installed automatically at startup. For persistent fixes, update the Dockerfile or plugin requirements to include everything needed for npm install to succeed at runtime. You can also check for more detailed npm error logs inside the container to pinpoint the missing package or permission issue. See similar troubleshooting steps and best practices for plugin dependency errors in Dify’s Docker environment here.

If you need a temporary workaround, you can try using an alternative plugin for web scraping, as was done for similar issues in earlier Dify versions here.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Chat with Dosu Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Jan 15, 2026): <!-- Answer --> The npm install error you’re seeing with the webscraper tool in Dify 1.11.3 (Docker) is a known issue and is likely caused by a missing or misconfigured dependency in the container environment. The built-in webscraper tool is implemented in Python and doesn’t directly invoke npm, so this error probably comes from a supplementary script or plugin that tries to run npm install at runtime, relying on the Node.js/npm environment provided by the Docker image. The Dockerfile does install Node.js and system libraries, but does not run npm install for the webscraper tool at build time, so any runtime invocation depends on the environment being correct and all dependencies being present and writable by the non-root user ["dify"](https://github.com/langgenius/dify/blob/52af829f1fadf2afeb3c62e57137c160e8179050/api/Dockerfile#L1-L122). This class of error is common in Dify’s Docker setups, especially with plugins or tools that require extra dependencies not present in the base image. The best workaround is to manually install any missing dependencies inside the running container, or to ensure all required dependencies are listed in the tool’s or plugin’s requirements file so they’re installed automatically at startup. For persistent fixes, update the Dockerfile or plugin requirements to include everything needed for npm install to succeed at runtime. You can also check for more detailed npm error logs inside the container to pinpoint the missing package or permission issue. See similar troubleshooting steps and best practices for plugin dependency errors in Dify’s Docker environment [here](https://github.com/langgenius/dify-official-plugins/issues/2169). If you need a temporary workaround, you can try using an alternative plugin for web scraping, as was done for similar issues in earlier Dify versions [here](https://github.com/langgenius/dify/issues/19778#issuecomment-2883640081). <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/8e264836-164c-488f-99e6-372bbf5ac181?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/8e264836-164c-488f-99e6-372bbf5ac181?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/8e264836-164c-488f-99e6-372bbf5ac181?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/8e264836-164c-488f-99e6-372bbf5ac181?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/8e264836-164c-488f-99e6-372bbf5ac181?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/8e264836-164c-488f-99e6-372bbf5ac181?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/8e264836-164c-488f-99e6-372bbf5ac181?feedback_type=other)</sup> [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=The%20npm%20install%20error%20you%E2%80%99re%20seeing%20with%20the%20webscraper%20tool%20in%20Dify%201.11.3%20%28Docker%29%20is%20a%20known%20issue%20and%20is%20likely%20caused%20by%20a%20missing%20or%20misconfigured%20dependency%20in%20the%20container%20environment.%20The%20built-in%20webscraper%20tool%20is%20implemented%20in%20Python%20and%20doesn%E2%80%99t%20directly%20invoke%20npm%2C%20so%20this%20error%20probably%20comes%20from%20a%20supplementary%20script%20or%20plugin%20that%20tries%20to%20run%20npm%20install%20at%20runtime%2C%20relying%20on%20the%20Node.js/npm%20environment%20provided%20by%20the%20Docker%20image.%20The%20Dockerfile%20does%20install%20Node.js%20and%20system%20libraries%2C%20but%20does%20not%20run%20npm%20install%20for%20the%20webscraper%20tool%20at%20build%20time%2C%20so%20any%20runtime%20invocation%20depends%20on%20the%20environment%20being%20correct%20and%20all%20dependencies%20being%20present%20and%20writable%20by%20the%20non-root%20user%20%5B%22dify%22%5D%28https%3A//github.com/langgenius/dify/blob/52af829f1fadf2afeb3c62e57137c160e8179050/api/Dockerfile%23L1-L122%29.%0A%0AThis%20class%20of%20error%20is%20common%20in%20Dify%E2%80%99s%20Docker%20setups%2C%20especially%20with%20plugins%20or%20tools%20that%20require%20extra%20dependencies%20not%20present%20in%20the%20base%20image.%20The%20best%20workaround%20is%20to%20manually%20install%20any%20missing%20dependencies%20inside%20the%20running%20container%2C%20or%20to%20ensure%20all%20required%20dependencies%20are%20listed%20in%20the%20tool%E2%80%99s%20or%20plugin%E2%80%99s%20requirements%20file%20so%20they%E2%80%99re%20installed%20automatically%20at%20startup.%20For%20persistent%20fixes%2C%20update%20the%20Dockerfile%20or%20plugin%20requirements%20to%20include%20everything%20needed%20for%20npm%20install%20to%20succeed%20at%20runtime.%20You%20can%20also%20check%20for%20more%20detailed%20npm%20error%20logs%20inside%20the%20container%20to%20pinpoint%20the%20missing%20package%20or%20permission%20issue.%20See%20similar%20troubleshooting%20steps%20and%20best%20practices%20for%20plugin%20dependency%20errors%20in%20Dify%E2%80%99s%20Docker%20environment%20%5Bhere%5D%28https%3A//github.com/langgenius/dify-official-plugins/issues/2169%29.%0A%0AIf%20you%20need%20a%20temporary%20workaround%2C%20you%20can%20try%20using%20an%20alternative%20plugin%20for%20web%20scraping%2C%20as%20was%20done%20for%20similar%20issues%20in%20earlier%20Dify%20versions%20%5Bhere%5D%28https%3A//github.com/langgenius/dify/issues/19778%23issuecomment-2883640081%29.)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/31036)
yindo commented 2026-02-21 20:13:46 -05:00
Author
Owner
Copy Link
Copy Source

@fatelei commented on GitHub (Jan 15, 2026):

Self Checks

  • I have read the Contributing Guide and Language Policy.[x] This is only for bug report, if you would like to ask a question, please head to Discussions.[x] I have searched for existing issues search for existing issues, including closed ones.[x] I confirm that I am using English to submit this report, otherwise it will be closed.[x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）[x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.11.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

dify webscraper tools error {"webscraper": "tool invoke error: Command '['npm', 'install']' returned non-zero exit status 1."}

Image ### ✔️ Expected Behavior true

❌ Actual Behavior

dify webscraper tools error {"webscraper": "tool invoke error: Command '['npm', 'install']' returned non-zero exit status 1."}

Image

Enter the container

docker exec -it docker-api-1 bash

Install readabilipy npm dependencies

cd $(python -c "import os; from readabilipy import simple_json;print(os.path.join(os.path.dirname(simple_json.file), 'javascript'))")
npm install

@fatelei commented on GitHub (Jan 15, 2026): > ### Self Checks > * [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).[x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).[x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.[x] I confirm that I am using English to submit this report, otherwise it will be closed.[x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）[x] Please do not modify this template :) and fill in all the required fields. > > ### Dify version > 1.11.3 > > ### Cloud or Self Hosted > Self Hosted (Docker) > > ### Steps to reproduce > dify webscraper tools error {"webscraper": "tool invoke error: Command '['npm', 'install']' returned non-zero exit status 1."} > > <img alt="Image" width="1660" height="654" src="https://private-user-images.githubusercontent.com/54889571/536080886-c9b22bc6-0ba3-45a2-8f06-ee237ff82d43.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Njg0Njg5OTEsIm5iZiI6MTc2ODQ2ODY5MSwicGF0aCI6Ii81NDg4OTU3MS81MzYwODA4ODYtYzliMjJiYzYtMGJhMy00NWEyLThmMDYtZWUyMzdmZjgyZDQzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjAxMTUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwMTE1VDA5MTgxMVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWIwNTEzMjI4MGJiYjllNjFhN2RiNGFkYzA1YjA5ZTZmNWJhZTE3MmMyODg3ZWVkZWVjYjRlOWU4ZjgxYTY2ODcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.tVcjPanBTAXAOAuizYhXF8iHHYvx9tPaS0pWz98_K7M"> > ### ✔️ Expected Behavior > true > > ### ❌ Actual Behavior > dify webscraper tools error {"webscraper": "tool invoke error: Command '['npm', 'install']' returned non-zero exit status 1."} > > <img alt="Image" width="1660" height="654" src="https://private-user-images.githubusercontent.com/54889571/536080886-c9b22bc6-0ba3-45a2-8f06-ee237ff82d43.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Njg0Njg5OTEsIm5iZiI6MTc2ODQ2ODY5MSwicGF0aCI6Ii81NDg4OTU3MS81MzYwODA4ODYtYzliMjJiYzYtMGJhMy00NWEyLThmMDYtZWUyMzdmZjgyZDQzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjAxMTUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwMTE1VDA5MTgxMVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWIwNTEzMjI4MGJiYjllNjFhN2RiNGFkYzA1YjA5ZTZmNWJhZTE3MmMyODg3ZWVkZWVjYjRlOWU4ZjgxYTY2ODcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.tVcjPanBTAXAOAuizYhXF8iHHYvx9tPaS0pWz98_K7M"> # Enter the container docker exec -it docker-api-1 bash # Install readabilipy npm dependencies cd $(python -c "import os; from readabilipy import simple_json;print(os.path.join(os.path.dirname(simple_json.__file__), 'javascript'))") npm install
yindo commented 2026-02-21 20:13:46 -05:00
Author
Owner
Copy Link
Copy Source

@Capricorn0010 commented on GitHub (Jan 15, 2026):

cd $(python -c "import os; from readabilipy import simple_json;print(os.path.join(os.path.dirname(simple_json.file), 'javascript'))")
Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'readabilipy.simple_json' has no attribute 'file' @fatelei

@Capricorn0010 commented on GitHub (Jan 15, 2026): cd $(python -c "import os; from readabilipy import simple_json;print(os.path.join(os.path.dirname(simple_json.file), 'javascript'))") Traceback (most recent call last): File "<string>", line 1, in <module> AttributeError: module 'readabilipy.simple_json' has no attribute 'file' @fatelei
yindo commented 2026-02-21 20:13:46 -05:00
Author
Owner
Copy Link
Copy Source

@fatelei commented on GitHub (Jan 15, 2026):

cd $(python -c "import os; from readabilipy import simple_json;print(os.path.join(os.path.dirname(simple_json.file), 'javascript'))") Traceback (most recent call last): File "", line 1, in AttributeError: module 'readabilipy.simple_json' has no attribute 'file' @fatelei

Image no problem

you can cd to /app/api/.venv/lib/python3.12/site-packages/readabilipy/javascript

@fatelei commented on GitHub (Jan 15, 2026): > cd $(python -c "import os; from readabilipy import simple_json;print(os.path.join(os.path.dirname(simple_json.file), 'javascript'))") Traceback (most recent call last): File "", line 1, in AttributeError: module 'readabilipy.simple_json' has no attribute 'file' [@fatelei](https://github.com/fatelei) <img width="1340" height="308" alt="Image" src="https://github.com/user-attachments/assets/3a2c757d-28c2-44c3-ad36-74b04b1b9b14" /> no problem you can cd to /app/api/.venv/lib/python3.12/site-packages/readabilipy/javascript
yindo commented 2026-02-21 20:13:46 -05:00
Author
Owner
Copy Link
Copy Source

@fatelei commented on GitHub (Jan 15, 2026):

and execute npm install

@fatelei commented on GitHub (Jan 15, 2026): and execute npm install
yindo commented 2026-02-21 20:13:46 -05:00
Author
Owner
Copy Link
Copy Source

@linsz2603 commented on GitHub (Jan 19, 2026):

The same issue is occurring on Cloud Dify. Is this a bug in 1.11.4?

@linsz2603 commented on GitHub (Jan 19, 2026): The same issue is occurring on Cloud Dify. Is this a bug in 1.11.4?
yindo commented 2026-02-21 20:13:46 -05:00
Author
Owner
Copy Link
Copy Source

@fatelei commented on GitHub (Jan 19, 2026):

The same issue is occurring on Cloud Dify. Is this a bug in 1.11.4?

this is not a bug for 1.11.4, because dify use readabilipy library, but it do not install nodejs dependencies auto, so it fail in cloud

@fatelei commented on GitHub (Jan 19, 2026): > The same issue is occurring on Cloud Dify. Is this a bug in 1.11.4? this is not a bug for 1.11.4, because dify use readabilipy library, but it do not install nodejs dependencies auto, so it fail in cloud
yindo commented 2026-02-21 20:13:46 -05:00
Author
Owner
Copy Link
Copy Source

@linsz2603 commented on GitHub (Jan 19, 2026):

@fatelei Thank you for your response. Regarding the SaaS version, is it correct to understand that the tool's error will be fixed in the next version update?

@linsz2603 commented on GitHub (Jan 19, 2026): @fatelei Thank you for your response. Regarding the SaaS version, is it correct to understand that the tool's error will be fixed in the next version update?
yindo commented 2026-02-21 20:13:46 -05:00
Author
Owner
Copy Link
Copy Source

@gsm1258 commented on GitHub (Jan 22, 2026):

I have deployed Dify version 1.11.4 in an offline environment and cannot access the internet. How should I proceed? @dosu
Currently, webscraper tools running Dify 1.11.4 on the internal network shows no prompts at all and does not display the error message: "Command '['npm', 'install']' returned non-zero exit status 1."

@gsm1258 commented on GitHub (Jan 22, 2026): I have deployed Dify version 1.11.4 in an offline environment and cannot access the internet. How should I proceed? @dosu Currently, webscraper tools running Dify 1.11.4 on the internal network shows no prompts at all and does not display the error message: "Command '['npm', 'install']' returned non-zero exit status 1."
yindo commented 2026-02-21 20:13:46 -05:00
Author
Owner
Copy Link
Copy Source

@fatelei commented on GitHub (Jan 22, 2026):

I have deployed Dify version 1.11.4 in an offline environment and cannot access the internet. How should I proceed? @dosu Currently, webscraper tools running Dify 1.11.4 on the internal network shows no prompts at all and does not display the error message: "Command '['npm', 'install']' returned non-zero exit status 1."

you can modify the source code

from

def extract_using_readabilipy(html: str):
    json_article: dict[str, Any] = simple_json_from_html_string(html, use_readability=True)
    article = Article(
        title=json_article.get("title") or "",
        author=json_article.get("byline") or "",
        text=json_article.get("plain_text") or [],
    )

    return article

to

def extract_using_readabilipy(html: str):
    json_article: dict[str, Any] = simple_json_from_html_string(html, use_readability=False)
    article = Article(
        title=json_article.get("title") or "",
        author=json_article.get("byline") or "",
        text=json_article.get("plain_text") or [],
    )

    return article
@fatelei commented on GitHub (Jan 22, 2026): > I have deployed Dify version 1.11.4 in an offline environment and cannot access the internet. How should I proceed? [@dosu](https://github.com/dosu) Currently, webscraper tools running Dify 1.11.4 on the internal network shows no prompts at all and does not display the error message: "Command '['npm', 'install']' returned non-zero exit status 1." you can modify the source code from ``` def extract_using_readabilipy(html: str): json_article: dict[str, Any] = simple_json_from_html_string(html, use_readability=True) article = Article( title=json_article.get("title") or "", author=json_article.get("byline") or "", text=json_article.get("plain_text") or [], ) return article ``` to ``` def extract_using_readabilipy(html: str): json_article: dict[str, Any] = simple_json_from_html_string(html, use_readability=False) article = Article( title=json_article.get("title") or "", author=json_article.get("byline") or "", text=json_article.get("plain_text") or [], ) return article ```
yindo commented 2026-02-21 20:13:46 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 22, 2026):

For Dify 1.11.4 in an offline environment, you’ll need to manually pre-package the npm dependencies required by the webscraper tool (specifically, for the readabilipy library). On a machine with internet access, install the same versions of Dify and readabilipy, then go to the readabilipy/javascript directory in your Python site-packages, run npm install, and archive the entire javascript directory (including node_modules, package.json, and package-lock.json). Transfer this archive to your offline environment, stop the Dify container, and replace the corresponding directory inside the container (e.g., /app/api/.venv/lib/python3.12/site-packages/readabilipy/javascript). Restart the container and the webscraper tool should work without needing to run npm install in the offline environment. Repeat this process if you upgrade Dify or readabilipy, as dependencies may change. This manual approach is currently the only reliable workaround for air-gapped deployments source.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Chat with Dosu Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Jan 22, 2026): <!-- Answer --> For Dify 1.11.4 in an offline environment, you’ll need to manually pre-package the npm dependencies required by the webscraper tool (specifically, for the readabilipy library). On a machine with internet access, install the same versions of Dify and readabilipy, then go to the readabilipy/javascript directory in your Python site-packages, run npm install, and archive the entire javascript directory (including node_modules, package.json, and package-lock.json). Transfer this archive to your offline environment, stop the Dify container, and replace the corresponding directory inside the container (e.g., /app/api/.venv/lib/python3.12/site-packages/readabilipy/javascript). Restart the container and the webscraper tool should work without needing to run npm install in the offline environment. Repeat this process if you upgrade Dify or readabilipy, as dependencies may change. This manual approach is currently the only reliable workaround for air-gapped deployments [source](https://github.com/langgenius/dify/issues/31291#issuecomment-3772168784). <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/63d7963e-6bf0-42a9-93b1-362c70e3dee1?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/63d7963e-6bf0-42a9-93b1-362c70e3dee1?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/63d7963e-6bf0-42a9-93b1-362c70e3dee1?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/63d7963e-6bf0-42a9-93b1-362c70e3dee1?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/63d7963e-6bf0-42a9-93b1-362c70e3dee1?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/63d7963e-6bf0-42a9-93b1-362c70e3dee1?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/63d7963e-6bf0-42a9-93b1-362c70e3dee1?feedback_type=other)</sup> [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=For%20Dify%201.11.4%20in%20an%20offline%20environment%2C%20you%E2%80%99ll%20need%20to%20manually%20pre-package%20the%20npm%20dependencies%20required%20by%20the%20webscraper%20tool%20%28specifically%2C%20for%20the%20readabilipy%20library%29.%20On%20a%20machine%20with%20internet%20access%2C%20install%20the%20same%20versions%20of%20Dify%20and%20readabilipy%2C%20then%20go%20to%20the%20readabilipy/javascript%20directory%20in%20your%20Python%20site-packages%2C%20run%20npm%20install%2C%20and%20archive%20the%20entire%20javascript%20directory%20%28including%20node_modules%2C%20package.json%2C%20and%20package-lock.json%29.%20Transfer%20this%20archive%20to%20your%20offline%20environment%2C%20stop%20the%20Dify%20container%2C%20and%20replace%20the%20corresponding%20directory%20inside%20the%20container%20%28e.g.%2C%20/app/api/.venv/lib/python3.12/site-packages/readabilipy/javascript%29.%20Restart%20the%20container%20and%20the%20webscraper%20tool%20should%20work%20without%20needing%20to%20run%20npm%20install%20in%20the%20offline%20environment.%20Repeat%20this%20process%20if%20you%20upgrade%20Dify%20or%20readabilipy%2C%20as%20dependencies%20may%20change.%20This%20manual%20approach%20is%20currently%20the%20only%20reliable%20workaround%20for%20air-gapped%20deployments%20%5Bsource%5D%28https%3A//github.com/langgenius/dify/issues/31291%23issuecomment-3772168784%29.)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/31036)
yindo commented 2026-02-21 20:13:46 -05:00
Author
Owner
Copy Link
Copy Source

@zqr10159 commented on GitHub (Jan 27, 2026):

It seems like the dify user lacks a writable home directory (/home/dify) in the current Docker environment, which causes npm to fail when attempting to create a cache directory.

To resolve this and install the missing dependencies for readabilipy, you can explicitly point the npm cache to a writable temporary directory.

Run the following commands:

Bash

1. Navigate to the Readabilipy javascript directory

cd /app/api/.venv/lib/python3.12/site-packages/readabilipy/javascript

2. Run npm install with a custom cache path to avoid permission issues

npm install --cache /tmp/npm-cache --prefer-offline
Why this works: By using --cache /tmp/npm-cache, we bypass the default attempt to write to /home/dify/.npm, ensuring the installation completes successfully even if the user home directory is restricted or non-existent.

@zqr10159 commented on GitHub (Jan 27, 2026): It seems like the dify user lacks a writable home directory (/home/dify) in the current Docker environment, which causes npm to fail when attempting to create a cache directory. To resolve this and install the missing dependencies for readabilipy, you can explicitly point the npm cache to a writable temporary directory. Run the following commands: Bash # 1. Navigate to the Readabilipy javascript directory cd /app/api/.venv/lib/python3.12/site-packages/readabilipy/javascript # 2. Run npm install with a custom cache path to avoid permission issues npm install --cache /tmp/npm-cache --prefer-offline Why this works: By using --cache /tmp/npm-cache, we bypass the default attempt to write to /home/dify/.npm, ensuring the installation completes successfully even if the user home directory is restricted or non-existent.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#21666
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?