langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 20:45:26 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Meta endpoint! Unexpected status code: 404 using weaviate vdb #6611

New Issue
Closed
opened 2026-02-21 18:16:32 -05:00 by yindo · 6 comments
yindo commented 2026-02-21 18:16:32 -05:00
Owner
Copy Link
Copy Source

Originally created by @Derican on GitHub (Nov 7, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

v0.11.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

  1. use docker-compose to create dify
  2. add docs to knowledge, select high quality and openai embedding

✔️ Expected Behavior

docs are indexed

❌ Actual Behavior

Screenshot 2024-11-06 224858

Originally created by @Derican on GitHub (Nov 7, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version v0.11.0 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce 1. use docker-compose to create dify 2. add docs to knowledge, select high quality and openai embedding ### ✔️ Expected Behavior docs are indexed ### ❌ Actual Behavior ![Screenshot 2024-11-06 224858](https://github.com/user-attachments/assets/c9010d3f-f52e-48ff-84e5-3b0bcbddc138)
yindo added the 👻 feat:ragstale labels 2026-02-21 18:16:33 -05:00
yindo closed this issue 2026-02-21 18:16:33 -05:00
yindo commented 2026-02-21 18:16:33 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Nov 7, 2024):

I found a similar unsolved discussion regarding a 404 error with Weaviate: Error: Response output is missing or does not contain embeddings [1].

Additionally, here are some potential solutions and considerations for resolving the 404 error when using Dify with Weaviate:

  1. Configuration Settings: Ensure that your Weaviate configuration settings are correct. You should have the following environment variables set:

    • VECTOR_STORE set to weaviate.
    • WEAVIATE_ENDPOINT pointing to the correct Weaviate endpoint (e.g., http://weaviate:8080).
    • Ensure that API key authentication is correctly configured if enabled [2].

  2. Network Configuration: Verify that your Docker network settings allow communication between Dify and Weaviate. Dify uses a custom Docker network for internal communication, so ensure that Weaviate is accessible on the default network [3].

  3. Error Handling: Dify tolerates 404 errors during certain operations, such as deleting non-existent objects. However, if you're encountering a 404 during indexing, it might indicate a misconfiguration or connectivity issue [4].

  4. Logs and Debugging: Check the logs for any error messages that might provide more context on the issue. Dify logs errors at the INFO level and above, which can help diagnose connectivity issues [5].

If these steps do not resolve the issue, consider reviewing the detailed configuration and network settings to ensure everything is set up correctly.

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Nov 7, 2024): <!-- Answer --> I found a similar unsolved discussion regarding a 404 error with Weaviate: [Error: Response output is missing or does not contain embeddings](https://github.com/langgenius/dify/discussions/10032) <sup>[[1]](https://github.com/langgenius/dify/discussions/10032)</sup>. Additionally, here are some potential solutions and considerations for resolving the 404 error when using Dify with Weaviate: 1. **Configuration Settings**: Ensure that your Weaviate configuration settings are correct. You should have the following environment variables set: - `VECTOR_STORE` set to `weaviate`. - `WEAVIATE_ENDPOINT` pointing to the correct Weaviate endpoint (e.g., `http://weaviate:8080`). - Ensure that API key authentication is correctly configured if enabled <sup>[[2]](https://github.com/langgenius/dify-docs/blob/main/jp/getting-started/install-self-hosted/environments.md)</sup>. 2. **Network Configuration**: Verify that your Docker network settings allow communication between Dify and Weaviate. Dify uses a custom Docker network for internal communication, so ensure that Weaviate is accessible on the default network <sup>[[3]](https://github.com/langgenius/dify/blob/main/docker/docker-compose.middleware.yaml)</sup>. 3. **Error Handling**: Dify tolerates 404 errors during certain operations, such as deleting non-existent objects. However, if you're encountering a 404 during indexing, it might indicate a misconfiguration or connectivity issue <sup>[[4]](https://github.com/langgenius/dify/blob/main/api/core/rag/datasource/vdb/weaviate/weaviate_vector.py)</sup>. 4. **Logs and Debugging**: Check the logs for any error messages that might provide more context on the issue. Dify logs errors at the INFO level and above, which can help diagnose connectivity issues <sup>[[5]](https://github.com/langgenius/dify/blob/main/api/core/rag/datasource/vdb/lindorm/lindorm_vector.py)</sup>. If these steps do not resolve the issue, consider reviewing the detailed configuration and network settings to ensure everything is set up correctly. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:16:33 -05:00
Author
Owner
Copy Link
Copy Source

@Derican commented on GitHub (Nov 7, 2024):

I have mentioned info in Issue#3216 and #734 , but neither of the solutions worked. The network to openai is tested and it worked, proxy setting is in #9960 . Logs of worker and weaviate are pasted below:
image
image

@Derican commented on GitHub (Nov 7, 2024): I have mentioned info in Issue#3216 and #734 , but neither of the solutions worked. The network to openai is tested and it worked, proxy setting is in #9960 . Logs of `worker` and `weaviate` are pasted below: ![image](https://github.com/user-attachments/assets/d9235433-76f0-4780-a4a8-c6f09898c4e7) ![image](https://github.com/user-attachments/assets/b8f31bd4-31d8-44bd-ac80-63e1476f494c)
yindo commented 2026-02-21 18:16:33 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Nov 7, 2024):

Can you attach the results of docker ps.

@crazywoola commented on GitHub (Nov 7, 2024): Can you attach the results of `docker ps`.
yindo commented 2026-02-21 18:16:33 -05:00
Author
Owner
Copy Link
Copy Source

@Derican commented on GitHub (Nov 7, 2024):

Can you attach the results of docker ps.

image

@Derican commented on GitHub (Nov 7, 2024): > Can you attach the results of `docker ps`. ![image](https://github.com/user-attachments/assets/c79f9bf3-8da6-47f5-aed8-dd80fb2b5f23)
yindo commented 2026-02-21 18:16:33 -05:00
Author
Owner
Copy Link
Copy Source

@Derican commented on GitHub (Nov 7, 2024):

I figure out by adding networks to weaviate:

weaviate:
    image: semitechnologies/weaviate:1.19.0
    profiles:
      - ''
      - weaviate
    restart: always
    volumes:
      # Mount the Weaviate data directory to the con tainer.
      - ./volumes/weaviate:/var/lib/weaviate
    environment:
      # The Weaviate configurations
      # You can refer to the [Weaviate](https://weaviate.io/developers/weaviate/config-refs/env-vars) documentation for more information.
      PERSISTENCE_DATA_PATH: ${WEAVIATE_PERSISTENCE_DATA_PATH:-/var/lib/weaviate}
      QUERY_DEFAULTS_LIMIT: ${WEAVIATE_QUERY_DEFAULTS_LIMIT:-25}
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: ${WEAVIATE_AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED:-false}
      DEFAULT_VECTORIZER_MODULE: ${WEAVIATE_DEFAULT_VECTORIZER_MODULE:-none}
      CLUSTER_HOSTNAME: ${WEAVIATE_CLUSTER_HOSTNAME:-node1}
      AUTHENTICATION_APIKEY_ENABLED: ${WEAVIATE_AUTHENTICATION_APIKEY_ENABLED:-true}
      AUTHENTICATION_APIKEY_ALLOWED_KEYS: ${WEAVIATE_AUTHENTICATION_APIKEY_ALLOWED_KEYS:-WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih}
      AUTHENTICATION_APIKEY_USERS: ${WEAVIATE_AUTHENTICATION_APIKEY_USERS:-hello@dify.ai}
      AUTHORIZATION_ADMINLIST_ENABLED: ${WEAVIATE_AUTHORIZATION_ADMINLIST_ENABLED:-true}
      AUTHORIZATION_ADMINLIST_USERS: ${WEAVIATE_AUTHORIZATION_ADMINLIST_USERS:-hello@dify.ai}
    networks:
      - ssrf_proxy_network
      - default

and adding no_proxy to api and worker:

worker:
    image: langgenius/dify-api:0.11.0
    restart: always
    environment:
      # Use the shared environment variables.
      <<: *shared-api-worker-env
      # Startup mode, 'worker' starts the Celery worker for processing the queue.
      MODE: worker
      HTTP_PROXY: ${SANDBOX_HTTP_PROXY:-http://ssrf_proxy:3128}
      HTTPS_PROXY: ${SANDBOX_HTTPS_PROXY:-http://ssrf_proxy:3128}
      no_proxy: 'localhost,127.0.0.1,172.*.*.*,.local,weaviate'

And be aware that knowledge base must be re-created as the index of previously created base will not work on retrying indexing.

@Derican commented on GitHub (Nov 7, 2024): I figure out by adding networks to weaviate: ```yaml weaviate: image: semitechnologies/weaviate:1.19.0 profiles: - '' - weaviate restart: always volumes: # Mount the Weaviate data directory to the con tainer. - ./volumes/weaviate:/var/lib/weaviate environment: # The Weaviate configurations # You can refer to the [Weaviate](https://weaviate.io/developers/weaviate/config-refs/env-vars) documentation for more information. PERSISTENCE_DATA_PATH: ${WEAVIATE_PERSISTENCE_DATA_PATH:-/var/lib/weaviate} QUERY_DEFAULTS_LIMIT: ${WEAVIATE_QUERY_DEFAULTS_LIMIT:-25} AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: ${WEAVIATE_AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED:-false} DEFAULT_VECTORIZER_MODULE: ${WEAVIATE_DEFAULT_VECTORIZER_MODULE:-none} CLUSTER_HOSTNAME: ${WEAVIATE_CLUSTER_HOSTNAME:-node1} AUTHENTICATION_APIKEY_ENABLED: ${WEAVIATE_AUTHENTICATION_APIKEY_ENABLED:-true} AUTHENTICATION_APIKEY_ALLOWED_KEYS: ${WEAVIATE_AUTHENTICATION_APIKEY_ALLOWED_KEYS:-WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih} AUTHENTICATION_APIKEY_USERS: ${WEAVIATE_AUTHENTICATION_APIKEY_USERS:-hello@dify.ai} AUTHORIZATION_ADMINLIST_ENABLED: ${WEAVIATE_AUTHORIZATION_ADMINLIST_ENABLED:-true} AUTHORIZATION_ADMINLIST_USERS: ${WEAVIATE_AUTHORIZATION_ADMINLIST_USERS:-hello@dify.ai} networks: - ssrf_proxy_network - default ``` and adding `no_proxy` to `api` and `worker`: ```yaml worker: image: langgenius/dify-api:0.11.0 restart: always environment: # Use the shared environment variables. <<: *shared-api-worker-env # Startup mode, 'worker' starts the Celery worker for processing the queue. MODE: worker HTTP_PROXY: ${SANDBOX_HTTP_PROXY:-http://ssrf_proxy:3128} HTTPS_PROXY: ${SANDBOX_HTTPS_PROXY:-http://ssrf_proxy:3128} no_proxy: 'localhost,127.0.0.1,172.*.*.*,.local,weaviate' ``` And be aware that knowledge base must be re-created as the index of previously created base will not work on retrying indexing.
yindo commented 2026-02-21 18:16:33 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Dec 8, 2024):

Hi, @Derican. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary

  • You reported a 404 error with Weaviate in Dify version 0.11.0 using a self-hosted Docker setup.
  • Suggested solutions included checking configuration settings, network configurations, and logs.
  • You resolved the issue by modifying the Docker setup with specific network configurations and environment variables.
  • Recreating the knowledge base was necessary for proper indexing.

Next Steps

  • Is this issue still relevant to the latest version of the Dify repository? If so, please comment to keep the discussion open.
  • Otherwise, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Dec 8, 2024): Hi, @Derican. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog. I'm marking this issue as stale. **Issue Summary** - You reported a 404 error with Weaviate in Dify version 0.11.0 using a self-hosted Docker setup. - Suggested solutions included checking configuration settings, network configurations, and logs. - You resolved the issue by modifying the Docker setup with specific network configurations and environment variables. - Recreating the knowledge base was necessary for proper indexing. **Next Steps** - Is this issue still relevant to the latest version of the Dify repository? If so, please comment to keep the discussion open. - Otherwise, this issue will be automatically closed in 15 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label stale 👻 feat:rag
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#6611
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?