langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Ollama's Embedding endpoint is wrong #5500

New Issue
Closed
opened 2026-02-21 18:11:17 -05:00 by yindo · 4 comments
yindo commented 2026-02-21 18:11:17 -05:00
Owner
Copy Link
Copy Source

Originally created by @killkli on GitHub (Sep 10, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

latest

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Ollama's embedding's endpoint should be
/api/embeddings

However, Dify's implementation uses
/api/embed
This cause embedding model unusable with ollama
Please fix~

Code in current git:
https://github.com/langgenius/dify/blob/85ff82a69461ce45388996907af7705bfa1b5e51/api/core/model_runtime/model_providers/ollama/text_embedding/text_embedding.py#L62

✔️ Expected Behavior

Use correct endpoint

❌ Actual Behavior

Use wrong endpoint, always result in 404

Originally created by @killkli on GitHub (Sep 10, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version latest ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce Ollama's embedding's endpoint should be /api/embeddings However, Dify's implementation uses /api/embed This cause embedding model unusable with ollama Please fix~ Code in current git: https://github.com/langgenius/dify/blob/85ff82a69461ce45388996907af7705bfa1b5e51/api/core/model_runtime/model_providers/ollama/text_embedding/text_embedding.py#L62 ### ✔️ Expected Behavior Use correct endpoint ### ❌ Actual Behavior Use wrong endpoint, always result in 404
yindo added the 🐞 bug label 2026-02-21 18:11:17 -05:00
yindo closed this issue 2026-02-21 18:11:17 -05:00
yindo commented 2026-02-21 18:11:17 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Sep 10, 2024):

Hello @killkli! I'm here to help you out with bug fixes, questions, and contributions. Let me know how I can assist you while you wait for a human maintainer.

I found a similar issue that was recently closed. The issue was about the Ollama embedding endpoint being incorrect, and it was resolved by updating the endpoint from /api/embed to /api/embeddings [1]. The fix was implemented in this pull request: fix: ollama text embedding 500 error.

To resolve the issue, you need to update the endpoint in the code from http://localhost:11434/api/embed to http://localhost:11434/api/embeddings. Here is the updated code snippet:

endpoint_url = urljoin(endpoint_url, 'api/embeddings')

The modified _invoke method should look like this:

def _invoke(self, model: str, credentials: dict,
            texts: list[str], user: Optional[str] = None) \
        -> TextEmbeddingResult:
    """
    Invoke text embedding model

    :param model: model name
    :param credentials: model credentials
    :param texts: texts to embed
    :param user: unique user id
    :return: embeddings result
    """

    # Prepare headers and payload for the request
    headers = {
        'Content-Type': 'application/json'
    }

    endpoint_url = credentials.get('base_url')
    if not endpoint_url.endswith('/'):
        endpoint_url += '/'

    endpoint_url = urljoin(endpoint_url, 'api/embeddings')

    # get model properties
    context_size = self._get_context_size(model, credentials)

    inputs = []
    used_tokens = 0

    for i, text in enumerate(texts):
        # Here token count is only an approximation based on the GPT2 tokenizer
        num_tokens = self._get_num_tokens_by_gpt2(text)

        if (num_tokens >= context_size):
            cutoff = int(np.floor(len(text) * (context_size / num_tokens)))
            # if num tokens is larger than context length, only use the start
            inputs.append(text[0: cutoff])
        else:
            inputs.append(text)

    # Prepare the payload for the request
    payload = {
        'input': inputs,
        'model': model,
    }

    # Make the request to the OpenAI API
    response = requests.post(
        endpoint_url,
        headers=headers,
        data=json.dumps(payload),
        timeout=(10, 300),
        options={"use_mmap": "true"}
    )

    response.raise_for_status()  # Raise an exception for HTTP errors
    response_data = response.json()
    # Extract embeddings and used tokens from the response
    embeddings = response_data['embeddings']
    embedding_used_tokens = self.get_num_tokens(model, credentials, inputs)

    used_tokens += embedding_used_tokens

    # calc usage
    usage = self._calc_response_usage(
        model=model,
        credentials=credentials,
        tokens=used_tokens
    )

    return TextEmbeddingResult(
        embeddings=embeddings,
        usage=usage,
        model=model
    )

Make sure to replace all instances of the old endpoint in your code with the new one. Additionally, ensure that your Ollama service is correctly configured and accessible at http://localhost:11434. If you are using Docker, you might need to expose the Ollama service to the network properly. For Docker deployments, consider using the host's IP address or host.docker.internal instead of localhost to avoid connection issues [2].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Sep 10, 2024): <!-- Greeting --> Hello @killkli! I'm here to help you out with bug fixes, questions, and contributions. Let me know how I can assist you while you wait for a human maintainer. <!-- Answer --> I found a similar issue that was recently closed. The issue was about the Ollama embedding endpoint being incorrect, and it was resolved by updating the endpoint from `/api/embed` to `/api/embeddings` <sup>[[1]](https://github.com/langgenius/dify/issues/8128)</sup>. The fix was implemented in this pull request: [fix: ollama text embedding 500 error](https://github.com/langgenius/dify/pull/8131). To resolve the issue, you need to update the endpoint in the code from `http://localhost:11434/api/embed` to `http://localhost:11434/api/embeddings`. Here is the updated code snippet: ```python endpoint_url = urljoin(endpoint_url, 'api/embeddings') ``` The modified `_invoke` method should look like this: ```python def _invoke(self, model: str, credentials: dict, texts: list[str], user: Optional[str] = None) \ -> TextEmbeddingResult: """ Invoke text embedding model :param model: model name :param credentials: model credentials :param texts: texts to embed :param user: unique user id :return: embeddings result """ # Prepare headers and payload for the request headers = { 'Content-Type': 'application/json' } endpoint_url = credentials.get('base_url') if not endpoint_url.endswith('/'): endpoint_url += '/' endpoint_url = urljoin(endpoint_url, 'api/embeddings') # get model properties context_size = self._get_context_size(model, credentials) inputs = [] used_tokens = 0 for i, text in enumerate(texts): # Here token count is only an approximation based on the GPT2 tokenizer num_tokens = self._get_num_tokens_by_gpt2(text) if (num_tokens >= context_size): cutoff = int(np.floor(len(text) * (context_size / num_tokens))) # if num tokens is larger than context length, only use the start inputs.append(text[0: cutoff]) else: inputs.append(text) # Prepare the payload for the request payload = { 'input': inputs, 'model': model, } # Make the request to the OpenAI API response = requests.post( endpoint_url, headers=headers, data=json.dumps(payload), timeout=(10, 300), options={"use_mmap": "true"} ) response.raise_for_status() # Raise an exception for HTTP errors response_data = response.json() # Extract embeddings and used tokens from the response embeddings = response_data['embeddings'] embedding_used_tokens = self.get_num_tokens(model, credentials, inputs) used_tokens += embedding_used_tokens # calc usage usage = self._calc_response_usage( model=model, credentials=credentials, tokens=used_tokens ) return TextEmbeddingResult( embeddings=embeddings, usage=usage, model=model ) ``` Make sure to replace all instances of the old endpoint in your code with the new one. Additionally, ensure that your Ollama service is correctly configured and accessible at `http://localhost:11434`. If you are using Docker, you might need to expose the Ollama service to the network properly. For Docker deployments, consider using the host's IP address or `host.docker.internal` instead of `localhost` to avoid connection issues <sup>[[2]](https://github.com/langgenius/dify/issues/7322)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:11:17 -05:00
Author
Owner
Copy Link
Copy Source

@killkli commented on GitHub (Sep 10, 2024):

No, https://github.com/langgenius/dify/pull/8131 Doesn't fix the code~ Bot your text is right but thr reference pull req is wrong XD

@killkli commented on GitHub (Sep 10, 2024): No, https://github.com/langgenius/dify/pull/8131 Doesn't fix the code~ Bot your text is right but thr reference pull req is wrong XD
yindo commented 2026-02-21 18:11:18 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Sep 10, 2024):

To fix the incorrect endpoint for Ollama's embedding in Dify, you need to update the endpoint in the code from /api/embed to /api/embeddings. Here is the corrected code snippet for the _invoke method:

def _invoke(self, model: str, credentials: dict,
            texts: list[str], user: Optional[str] = None) \
        -> TextEmbeddingResult:
    """
    Invoke text embedding model

    :param model: model name
    :param credentials: model credentials
    :param texts: texts to embed
    :param user: unique user id
    :return: embeddings result
    """

    # Prepare headers and payload for the request
    headers = {
        'Content-Type': 'application/json'
    }

    endpoint_url = credentials.get('base_url')
    if not endpoint_url.endswith('/'):
        endpoint_url += '/'

    endpoint_url = urljoin(endpoint_url, 'api/embeddings')

    # get model properties
    context_size = self._get_context_size(model, credentials)

    inputs = []
    used_tokens = 0

    for i, text in enumerate(texts):
        # Here token count is only an approximation based on the GPT2 tokenizer
        num_tokens = self._get_num_tokens_by_gpt2(text)

        if (num_tokens >= context_size):
            cutoff = int(np.floor(len(text) * (context_size / num_tokens)))
            # if num tokens is larger than context length, only use the start
            inputs.append(text[0: cutoff])
        else:
            inputs.append(text)

    # Prepare the payload for the request
    payload = {
        'input': inputs,
        'model': model,
    }

    # Make the request to the OpenAI API
    response = requests.post(
        endpoint_url,
        headers=headers,
        data=json.dumps(payload),
        timeout=(10, 300),
        options={"use_mmap": "true"}
    )

    response.raise_for_status()  # Raise an exception for HTTP errors
    response_data = response.json()
    # Extract embeddings and used tokens from the response
    embeddings = response_data['embeddings']
    embedding_used_tokens = self.get_num_tokens(model, credentials, inputs)

    used_tokens += embedding_used_tokens

    # calc usage
    usage = self._calc_response_usage(
        model=model,
        credentials=credentials,
        tokens=used_tokens
    )

    return TextEmbeddingResult(
        embeddings=embeddings,
        usage=usage,
        model=model
    )

Make sure to replace all instances of the old endpoint in your code with the new one. Additionally, ensure that your Ollama service is correctly configured and accessible at http://localhost:11434. If you are using Docker, you might need to expose the Ollama service to the network properly. For Docker deployments, consider using the host's IP address or host.docker.internal instead of localhost to avoid connection issues [1].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Sep 10, 2024): <!-- Answer --> To fix the incorrect endpoint for Ollama's embedding in Dify, you need to update the endpoint in the code from `/api/embed` to `/api/embeddings`. Here is the corrected code snippet for the `_invoke` method: ```python def _invoke(self, model: str, credentials: dict, texts: list[str], user: Optional[str] = None) \ -> TextEmbeddingResult: """ Invoke text embedding model :param model: model name :param credentials: model credentials :param texts: texts to embed :param user: unique user id :return: embeddings result """ # Prepare headers and payload for the request headers = { 'Content-Type': 'application/json' } endpoint_url = credentials.get('base_url') if not endpoint_url.endswith('/'): endpoint_url += '/' endpoint_url = urljoin(endpoint_url, 'api/embeddings') # get model properties context_size = self._get_context_size(model, credentials) inputs = [] used_tokens = 0 for i, text in enumerate(texts): # Here token count is only an approximation based on the GPT2 tokenizer num_tokens = self._get_num_tokens_by_gpt2(text) if (num_tokens >= context_size): cutoff = int(np.floor(len(text) * (context_size / num_tokens))) # if num tokens is larger than context length, only use the start inputs.append(text[0: cutoff]) else: inputs.append(text) # Prepare the payload for the request payload = { 'input': inputs, 'model': model, } # Make the request to the OpenAI API response = requests.post( endpoint_url, headers=headers, data=json.dumps(payload), timeout=(10, 300), options={"use_mmap": "true"} ) response.raise_for_status() # Raise an exception for HTTP errors response_data = response.json() # Extract embeddings and used tokens from the response embeddings = response_data['embeddings'] embedding_used_tokens = self.get_num_tokens(model, credentials, inputs) used_tokens += embedding_used_tokens # calc usage usage = self._calc_response_usage( model=model, credentials=credentials, tokens=used_tokens ) return TextEmbeddingResult( embeddings=embeddings, usage=usage, model=model ) ``` Make sure to replace all instances of the old endpoint in your code with the new one. Additionally, ensure that your Ollama service is correctly configured and accessible at `http://localhost:11434`. If you are using Docker, you might need to expose the Ollama service to the network properly. For Docker deployments, consider using the host's IP address or `host.docker.internal` instead of `localhost` to avoid connection issues <sup>[[1]](https://github.com/langgenius/dify-docs/blob/main/en/guides/model-configuration/ollama.md)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:11:18 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Sep 10, 2024):

Ollama's embedding's endpoint should be /api/embeddings

This endpoint is deprecated. See https://github.com/langgenius/dify/issues/7322

You need to upgrade your ollama client version.

https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings
https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embedding

See their docs:

Note: this endpoint has been superseded by /api/embed

POST /api/embeddings

https://github.com/langgenius/dify/pull/6876

@crazywoola commented on GitHub (Sep 10, 2024): > Ollama's embedding's endpoint should be /api/embeddings This endpoint is deprecated. See https://github.com/langgenius/dify/issues/7322 You need to upgrade your ollama client version. https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embedding See their docs: > Note: this endpoint has been superseded by /api/embed > POST /api/embeddings https://github.com/langgenius/dify/pull/6876
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#5500
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?