langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Could you kindly provide a full sample code for the custom model? #13148

New Issue
Closed
opened 2026-02-21 19:10:47 -05:00 by yindo · 3 comments
yindo commented 2026-02-21 19:10:47 -05:00
Owner
Copy Link
Copy Source

Originally created by @TFannie on GitHub (Apr 20, 2025).

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Provide a description of requested docs changes

Could you kindly provide a full sample code for the custom model? After reviewing the documentation, I found it a bit challenging to fully understand without an example. Your assistance would be greatly appreciated.

Originally created by @TFannie on GitHub (Apr 20, 2025). ### Self Checks - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Provide a description of requested docs changes Could you kindly provide a full sample code for the custom model? After reviewing the documentation, I found it a bit challenging to fully understand without an example. Your assistance would be greatly appreciated.
yindo added the 📚 documentation label 2026-02-21 19:10:47 -05:00
yindo closed this issue 2026-02-21 19:10:47 -05:00
yindo commented 2026-02-21 19:10:47 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Apr 20, 2025):

Hello @TFannie! I'm Dosu, your friendly bot here to help you with bugs, answer questions, and guide you on your journey to becoming a contributor. I'm here to assist you while you wait for a human maintainer. Let's tackle this together! 🤖

Here's a sample code for implementing a custom model in Dify, focusing on the _invoke method:

import json
from typing import Mapping
from werkzeug import Request, Response
from dify_plugin import Endpoint

class CustomModel(Endpoint):
    def _invoke(self, r: Request, values: Mapping, settings: Mapping) -> Response:
        """
        Invokes the endpoint with the given request.
        """
        app_id = values["app_id"]
        def generator():
            yield f"{app_id} <br>"
        return Response(generator(), status=200, content_type="text/html")

This example demonstrates how to create a simple endpoint that responds with the app_id provided in the request. You can expand this logic to include more complex operations based on your model's requirements.

For a tool that requests an LLM model, here's another example:

from collections.abc import Generator
from typing import Any

from dify_plugin import Tool
from dify_plugin.entities.model.llm import LLMModelConfig
from dify_plugin.entities.tool import ToolInvokeMessage
from dify_plugin.entities.model.message import SystemPromptMessage, UserPromptMessage

class LLMTool(Tool):
    def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage]:
        response = self.session.model.llm.invoke(
            model_config=LLMModelConfig(
                provider='openai',
                model='gpt-4o-mini',
                mode='chat',
                completion_params={}
            ),
            prompt_messages=[
                SystemPromptMessage(
                    content='you are a helpful assistant'
                ),
                UserPromptMessage(
                    content=tool_parameters.get('query')
                )
            ],
            stream=True
        )

        for chunk in response:
            if chunk.delta.message:
                assert isinstance(chunk.delta.message.content, str)
                yield self.create_text_message(text=chunk.delta.message.content)

This example shows how to invoke an OpenAI model using the invoke method within a tool, streaming the response back to the user [1][2].

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Apr 20, 2025): <!-- Greeting --> Hello @TFannie! I'm [Dosu](https://dosu.dev), your friendly bot here to help you with bugs, answer questions, and guide you on your journey to becoming a contributor. I'm here to assist you while you wait for a human maintainer. Let's tackle this together! 🤖 <!-- Answer --> Here's a sample code for implementing a custom model in Dify, focusing on the `_invoke` method: ```python import json from typing import Mapping from werkzeug import Request, Response from dify_plugin import Endpoint class CustomModel(Endpoint): def _invoke(self, r: Request, values: Mapping, settings: Mapping) -> Response: """ Invokes the endpoint with the given request. """ app_id = values["app_id"] def generator(): yield f"{app_id} <br>" return Response(generator(), status=200, content_type="text/html") ``` This example demonstrates how to create a simple endpoint that responds with the `app_id` provided in the request. You can expand this logic to include more complex operations based on your model's requirements. For a tool that requests an LLM model, here's another example: ```python from collections.abc import Generator from typing import Any from dify_plugin import Tool from dify_plugin.entities.model.llm import LLMModelConfig from dify_plugin.entities.tool import ToolInvokeMessage from dify_plugin.entities.model.message import SystemPromptMessage, UserPromptMessage class LLMTool(Tool): def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage]: response = self.session.model.llm.invoke( model_config=LLMModelConfig( provider='openai', model='gpt-4o-mini', mode='chat', completion_params={} ), prompt_messages=[ SystemPromptMessage( content='you are a helpful assistant' ), UserPromptMessage( content=tool_parameters.get('query') ) ], stream=True ) for chunk in response: if chunk.delta.message: assert isinstance(chunk.delta.message.content, str) yield self.create_text_message(text=chunk.delta.message.content) ``` This example shows how to invoke an OpenAI model using the `invoke` method within a tool, streaming the response back to the user <sup>[[1]](https://github.com/langgenius/dify-docs/blob/main/en/plugins/schema-definition/endpoint.md)</sup><sup>[[2]](https://github.com/langgenius/dify-docs/blob/main/en/plugins/schema-definition/reverse-invocation-of-the-dify-service/model.md)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/1c3ddd7d-6b55-48ec-be44-d993efcbe27d?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/1c3ddd7d-6b55-48ec-be44-d993efcbe27d?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/1c3ddd7d-6b55-48ec-be44-d993efcbe27d?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/1c3ddd7d-6b55-48ec-be44-d993efcbe27d?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/1c3ddd7d-6b55-48ec-be44-d993efcbe27d?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/1c3ddd7d-6b55-48ec-be44-d993efcbe27d?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/1c3ddd7d-6b55-48ec-be44-d993efcbe27d?feedback_type=other)</sup>
yindo commented 2026-02-21 19:10:47 -05:00
Author
Owner
Copy Link
Copy Source

@TFannie commented on GitHub (Apr 20, 2025):

The robot can't solve my problem. I follow customizable-model.md. The sample files below https://github.com/langgenius/dify-runtime/blob/main/lib/model_providers/anthropic/llm/llm.py does not exist

@TFannie commented on GitHub (Apr 20, 2025): The robot can't solve my problem. I follow customizable-model.md. The sample files below https://github.com/langgenius/dify-runtime/blob/main/lib/model_providers/anthropic/llm/llm.py does not exist
yindo commented 2026-02-21 19:10:47 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Apr 20, 2025):

https://github.com/langgenius/dify/issues/18425

Hello, here are the source code of the model providers.

https://github.com/langgenius/dify-official-plugins/tree/main/models

@crazywoola commented on GitHub (Apr 20, 2025): https://github.com/langgenius/dify/issues/18425 Hello, here are the source code of the model providers. https://github.com/langgenius/dify-official-plugins/tree/main/models
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 📚 documentation
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#13148
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?