langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

When setting ETL_TYPE to Unstructured, the local docker runs and reports an error when creating the knowledge base. #4851

New Issue
Closed
opened 2026-02-21 18:08:17 -05:00 by yindo · 7 comments
yindo commented 2026-02-21 18:08:17 -05:00
Owner
Copy Link
Copy Source

Originally created by @biofer on GitHub (Aug 1, 2024).

Originally assigned to: @JohnJyong on GitHub.

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

dify-api:0.6.15

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

image

Local docker run, set environment variables .env

ETL_TYPE=Unstructured
UNSTRUCTURED_API_URL=https://api.unstructuredapp.io/general/v0/general
UNSTRUCTURED_API_KEY=*****

The above error occurs when creating the knowledge base and starting the setup.

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

Originally created by @biofer on GitHub (Aug 1, 2024). Originally assigned to: @JohnJyong on GitHub. ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version dify-api:0.6.15 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce <img width="1376" alt="image" src="https://github.com/user-attachments/assets/eee6574f-da97-480a-b7b9-ccabece90071"> Local docker run, set environment variables `.env` ``` ETL_TYPE=Unstructured UNSTRUCTURED_API_URL=https://api.unstructuredapp.io/general/v0/general UNSTRUCTURED_API_KEY=***** ``` The above error occurs when creating the knowledge base and starting the setup. ### ✔️ Expected Behavior _No response_ ### ❌ Actual Behavior _No response_
yindo added the 🐞 bug label 2026-02-21 18:08:17 -05:00
yindo closed this issue 2026-02-21 18:08:17 -05:00
yindo commented 2026-02-21 18:08:17 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 1, 2024):

Hello @biofer! 👋 I'm here to help you with any bugs, questions, or contributions you may have regarding the repository. Let me know how I can assist you!

To resolve the error indicating that the NLTK 'punkt' resource is not found when setting ETL_TYPE to Unstructured in your local Docker instance, you need to download the 'punkt' resource. You can do this by adding the following code to your Dockerfile or by running it in your Docker container:

import nltk
nltk.download('punkt')

If you are using a Dockerfile, you can add the following lines to ensure the 'punkt' resource is downloaded during the build process:

RUN python -m nltk.downloader punkt

This will ensure that the 'punkt' tokenizer is available when you run your application in the Docker container [1].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Aug 1, 2024): <!-- Greeting --> Hello @biofer! 👋 I'm here to help you with any bugs, questions, or contributions you may have regarding the repository. Let me know how I can assist you! <!-- Answer --> To resolve the error indicating that the NLTK 'punkt' resource is not found when setting `ETL_TYPE` to `Unstructured` in your local Docker instance, you need to download the 'punkt' resource. You can do this by adding the following code to your Dockerfile or by running it in your Docker container: ```python import nltk nltk.download('punkt') ``` If you are using a Dockerfile, you can add the following lines to ensure the 'punkt' resource is downloaded during the build process: ```dockerfile RUN python -m nltk.downloader punkt ``` This will ensure that the 'punkt' tokenizer is available when you run your application in the Docker container <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/core/rag/extractor/extract_processor.py)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:08:17 -05:00
Author
Owner
Copy Link
Copy Source

@biofer commented on GitHub (Aug 1, 2024):

I followed the official tutorial to start docker, if there is something wrong with the docker image, why don't you provide the latest docker image?

@biofer commented on GitHub (Aug 1, 2024): I followed the official tutorial to start docker, if there is something wrong with the docker image, why don't you provide the latest docker image?
yindo commented 2026-02-21 18:08:17 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 1, 2024):

This is a tough nut to crack, could really use your help on this, @takatost.

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Aug 1, 2024): <!-- Answer --> This is a tough nut to crack, could really use your help on this, @takatost. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:08:18 -05:00
Author
Owner
Copy Link
Copy Source

@biofer commented on GitHub (Aug 1, 2024):

Any plans or details to divulge, my friend? @JohnJyong
After the local image goes in, following this fails, so hopefully I can get the official integrated image.

import nltk
nltk.download('punkt')
@biofer commented on GitHub (Aug 1, 2024): Any plans or details to divulge, my friend? @JohnJyong After the local image goes in, following this fails, so hopefully I can get the official integrated image. ``` import nltk nltk.download('punkt') ```
yindo commented 2026-02-21 18:08:18 -05:00
Author
Owner
Copy Link
Copy Source

@biofer commented on GitHub (Aug 1, 2024):

I would think that since the official docs give instructions, shouldn't they be followed and shouldn't it just work silky smooth?
Or is the experience of the domestic user unimportant and untested?

@biofer commented on GitHub (Aug 1, 2024): I would think that since the official docs give instructions, shouldn't they be followed and shouldn't it just work silky smooth? Or is the experience of the domestic user unimportant and untested?
yindo commented 2026-02-21 18:08:18 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Aug 2, 2024):

He is currently on vacation. Here is a similar issue raised long time ago. https://github.com/langgenius/dify/issues/4659

@crazywoola commented on GitHub (Aug 2, 2024): He is currently on vacation. Here is a similar issue raised long time ago. https://github.com/langgenius/dify/issues/4659
yindo commented 2026-02-21 18:08:18 -05:00
Author
Owner
Copy Link
Copy Source

@biofer commented on GitHub (Aug 2, 2024):

Any plans or details to divulge, my friend? @JohnJyong After the local image goes in, following this fails, so hopefully I can get the official integrated image.

import nltk
nltk.download('punkt')

So hopefully I can get a mirror image with no problems.

@biofer commented on GitHub (Aug 2, 2024): > Any plans or details to divulge, my friend? @JohnJyong After the local image goes in, following this fails, so hopefully I can get the official integrated image. > > ``` > import nltk > nltk.download('punkt') > ``` So hopefully I can get a mirror image with no problems.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#4851
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?