langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

System Prompt Not Recognized When Using LocalAI with Starling 7b Model via Docker Deployment #763

New Issue
Closed
opened 2026-02-21 17:28:19 -05:00 by yindo · 3 comments
yindo commented 2026-02-21 17:28:19 -05:00
Owner
Copy Link
Copy Source

Originally created by @jlvihv on GitHub (Dec 1, 2023).

Originally assigned to: @takatost on GitHub.

Dify version

0.3.32

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Hello Dify Team,

I am encountering an issue with the system prompt functionality when integrating a model with the Dify application. I have deployed Dify using Docker and have also set up a local instance running a model API compatible with the OpenAI API specification. Specifically, the model I am using is Starling 7b, and I have registered it as a LocalAI model provider.

While I can communicate with the model normally, when I attempt to build the application, the system prompt does not seem to take effect. The model does not recognize the role I have set for it within the system prompt. Attached is a screenshot that illustrates my setup.

I am using the latest Docker version available for Dify, and I have followed all the standard procedures for deployment. The communication with the model works for standard prompts, but the system prompts are not producing the expected behavior.

Could you please assist me in identifying what might be the issue here? Is there a specific configuration or additional step I need to ensure the system prompt is processed correctly by the model?

I am happy to provide further details, logs, configuration settings, or any additional information you may need to diagnose the issue.

Thank you for your assistance.

✔️ Expected Behavior

When setting a system prompt, I anticipate that the Starling 7b model will recognize and respond to the prompt as it is designed to, adopting the specified role or following the instructions laid out in the system prompt. This should guide the model's responses and influence its behavior according to the defined role.

❌ Actual Behavior

Despite setting the system prompt as per the documentation, the model does not seem to acknowledge the additional context or instructions. The responses received are generic and do not reflect the role or scenario described in the system prompt. It is as if the system prompt is completely ignored, and only the user input is processed for generating responses.

Originally created by @jlvihv on GitHub (Dec 1, 2023). Originally assigned to: @takatost on GitHub. ### Dify version 0.3.32 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce Hello Dify Team, I am encountering an issue with the system prompt functionality when integrating a model with the Dify application. I have deployed Dify using Docker and have also set up a local instance running a model API compatible with the OpenAI API specification. Specifically, the model I am using is Starling 7b, and I have registered it as a LocalAI model provider. While I can communicate with the model normally, when I attempt to build the application, the system prompt does not seem to take effect. The model does not recognize the role I have set for it within the system prompt. Attached is a screenshot that illustrates my setup. I am using the latest Docker version available for Dify, and I have followed all the standard procedures for deployment. The communication with the model works for standard prompts, but the system prompts are not producing the expected behavior. Could you please assist me in identifying what might be the issue here? Is there a specific configuration or additional step I need to ensure the system prompt is processed correctly by the model? I am happy to provide further details, logs, configuration settings, or any additional information you may need to diagnose the issue. Thank you for your assistance. ### ✔️ Expected Behavior When setting a system prompt, I anticipate that the Starling 7b model will recognize and respond to the prompt as it is designed to, adopting the specified role or following the instructions laid out in the system prompt. This should guide the model's responses and influence its behavior according to the defined role. ### ❌ Actual Behavior Despite setting the system prompt as per the documentation, the model does not seem to acknowledge the additional context or instructions. The responses received are generic and do not reflect the role or scenario described in the system prompt. It is as if the system prompt is completely ignored, and only the user input is processed for generating responses.
yindo added the 🐞 bug label 2026-02-21 17:28:19 -05:00
yindo closed this issue 2026-02-21 17:28:19 -05:00
yindo commented 2026-02-21 17:28:19 -05:00
Author
Owner
Copy Link
Copy Source

@jlvihv commented on GitHub (Dec 1, 2023):

For reference, here is a screenshot illustrating the issue:

https://imgur.com/4bPXzFK

@jlvihv commented on GitHub (Dec 1, 2023): For reference, here is a screenshot illustrating the issue: ![](https://imgur.com/4bPXzFK) https://imgur.com/4bPXzFK
yindo commented 2026-02-21 17:28:19 -05:00
Author
Owner
Copy Link
Copy Source

@defiways commented on GitHub (Dec 14, 2023):

"set up a local instance running a model API compatible with the OpenAI API specification. Specifically, the model I am using is Starling 7b, and I have registered it as a LocalAI model provider." What version of Open AI specification is it compatible with? And I would like to know what version of Open AI is required by LocalAI model provider?

@defiways commented on GitHub (Dec 14, 2023): "set up a local instance running a model API compatible with the OpenAI API specification. Specifically, the model I am using is Starling 7b, and I have registered it as a LocalAI model provider." What version of Open AI specification is it compatible with? And I would like to know what version of Open AI is required by LocalAI model provider?
yindo commented 2026-02-21 17:28:20 -05:00
Author
Owner
Copy Link
Copy Source

@jlvihv commented on GitHub (Dec 18, 2023):

"set up a local instance running a model API compatible with the OpenAI API specification. Specifically, the model I am using is Starling 7b, and I have registered it as a LocalAI model provider." What version of Open AI specification is it compatible with? And I would like to know what version of Open AI is required by LocalAI model provider?

Dear Support Team,

Thank you for your response. I would like to clarify that I am using the API functionality provided by the library at https://github.com/oobabooga/text-generation-webui, not the one at https://github.com/mudler/LocalAI.

It doesn't appear to be an issue of API compatibility, as I was later able to successfully use this feature without encountering the problems mentioned in my report. It seems that the bug may be intermittent or require specific conditions to trigger. Regrettably, I do not have more time to conduct detailed testing at the moment.

You may consider closing this issue, as it does not consistently reproduce and may not be easily resolvable without further specific details.

Thank you for your assistance.

@jlvihv commented on GitHub (Dec 18, 2023): > "set up a local instance running a model API compatible with the OpenAI API specification. Specifically, the model I am using is Starling 7b, and I have registered it as a LocalAI model provider." What version of Open AI specification is it compatible with? And I would like to know what version of Open AI is required by LocalAI model provider? Dear Support Team, Thank you for your response. I would like to clarify that I am using the API functionality provided by the library at https://github.com/oobabooga/text-generation-webui, not the one at https://github.com/mudler/LocalAI. It doesn't appear to be an issue of API compatibility, as I was later able to successfully use this feature without encountering the problems mentioned in my report. It seems that the bug may be intermittent or require specific conditions to trigger. Regrettably, I do not have more time to conduct detailed testing at the moment. You may consider closing this issue, as it does not consistently reproduce and may not be easily resolvable without further specific details. Thank you for your assistance.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#763
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?