langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

input llama2 model uid in Xinference but error #731

New Issue
Closed
opened 2026-02-21 17:28:12 -05:00 by yindo · 6 comments
yindo commented 2026-02-21 17:28:12 -05:00
Owner
Copy Link
Copy Source

Originally created by @hey0223 on GitHub (Nov 22, 2023).

Dify version

0.3.21

Cloud or Self Hosted

Self Hosted

Steps to reproduce

AttributeError: [address=0.0.0.0:33767, pid=5848] Model model_format='pytorch' model_size_in_billions=13 quantizations=['4-bit', '8-bit', 'none'] model_id='FlagAlpha/Llama2-Chinese-13b-Chat' model_hub='huggingface' model_uri='file:///root/chinese-llama2' model_revision=None is not for chat.

✔️ Expected Behavior

I downloaded the model of llama2 locally and deployed it on Xinreference. I hope it can be deployed to Dify through the model supplier

❌ Actual Behavior

After entering the corresponding uid and URL, an error message has been continuously reported as shown above. I am certain that I downloaded the chat version, but I don't know why not

Originally created by @hey0223 on GitHub (Nov 22, 2023). ### Dify version 0.3.21 ### Cloud or Self Hosted Self Hosted ### Steps to reproduce AttributeError: [address=0.0.0.0:33767, pid=5848] Model model_format='pytorch' model_size_in_billions=13 quantizations=['4-bit', '8-bit', 'none'] model_id='FlagAlpha/Llama2-Chinese-13b-Chat' model_hub='huggingface' model_uri='file:///root/chinese-llama2' model_revision=None is not for chat. ### ✔️ Expected Behavior I downloaded the model of llama2 locally and deployed it on Xinreference. I hope it can be deployed to Dify through the model supplier ### ❌ Actual Behavior After entering the corresponding uid and URL, an error message has been continuously reported as shown above. I am certain that I downloaded the chat version, but I don't know why not
yindo added the 🐞 bug label 2026-02-21 17:28:12 -05:00
yindo closed this issue 2026-02-21 17:28:12 -05:00
yindo commented 2026-02-21 17:28:13 -05:00
Author
Owner
Copy Link
Copy Source

@hey0223 commented on GitHub (Nov 22, 2023):

image

@hey0223 commented on GitHub (Nov 22, 2023): ![image](https://github.com/langgenius/dify/assets/105712475/d198393e-cebf-470d-9844-76da38118444)
yindo commented 2026-02-21 17:28:13 -05:00
Author
Owner
Copy Link
Copy Source

@hey0223 commented on GitHub (Nov 22, 2023):

if don’t have generate in ability ，it can‘t use xinference deploy to Dify？

@hey0223 commented on GitHub (Nov 22, 2023): if don’t have generate in ability ，it can‘t use xinference deploy to Dify？
yindo commented 2026-02-21 17:28:13 -05:00
Author
Owner
Copy Link
Copy Source

@takatost commented on GitHub (Nov 22, 2023):

Have you confirmed that the model can run and be called successfully in xinference?

@takatost commented on GitHub (Nov 22, 2023): Have you confirmed that the model can run and be called successfully in `xinference`?
yindo commented 2026-02-21 17:28:13 -05:00
Author
Owner
Copy Link
Copy Source

@hey0223 commented on GitHub (Nov 22, 2023):

您是否确认该模型可以运行并成功调用？xinference

微信图片_20231123103913

@hey0223 commented on GitHub (Nov 22, 2023): > 您是否确认该模型可以运行并成功调用？`xinference` ![微信图片_20231123103913](https://github.com/langgenius/dify/assets/105712475/8f95f36b-56df-4966-8cb6-5f36886bce95)
yindo commented 2026-02-21 17:28:13 -05:00
Author
Owner
Copy Link
Copy Source

@hey0223 commented on GitHub (Nov 22, 2023):

您是否确认该模型可以运行并成功调用？xinference

This can be considered a model that can be called

@hey0223 commented on GitHub (Nov 22, 2023): > 您是否确认该模型可以运行并成功调用？`xinference` This can be considered a model that can be called
yindo commented 2026-02-21 17:28:13 -05:00
Author
Owner
Copy Link
Copy Source

@takatost commented on GitHub (Nov 25, 2023):

model_id='FlagAlpha/Llama2-Chinese-13b-Chat' model_hub='huggingface' model_uri='file:///root/chinese-llama2' model_revision=None is not for chat.

This might be the reason why the model's pattern seems mismatched.
You can ask xinference first to see if the configuration is correct.

@takatost commented on GitHub (Nov 25, 2023): ``` model_id='FlagAlpha/Llama2-Chinese-13b-Chat' model_hub='huggingface' model_uri='file:///root/chinese-llama2' model_revision=None is not for chat. ``` This might be the reason why the model's pattern seems mismatched. You can ask `xinference` first to see if the configuration is correct.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#731
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?