vllm plugin only support llm, but vllm can do text embedding too #125

New Issue

2026-02-22T17:19:27-05:00

yindo commented

2026-02-22 17:19:27 -05:00

Originally created by @qiulang on GitHub (Jun 25, 2025).

vllm can do llm as well as text embedding

e.g. https://docs.vllm.ai/en/v0.7.1/serving/engine_args.html
--task
Possible choices: auto, generate, embedding, embed, classify, score, reward

I start bge model like this vllm serve ./BAAI/bge-m3 --task embed but I can not use it with vllm plugin

Originally created by @qiulang on GitHub (Jun 25, 2025). vllm can do llm as well as text embedding e.g. https://docs.vllm.ai/en/v0.7.1/serving/engine_args.html --task Possible choices: auto, generate, embedding, embed, classify, score, reward I start bge model like this `vllm serve ./BAAI/bge-m3 --task embed` but I can not use it with vllm plugin

yindo added the enhancement label 2026-02-22 17:19:27 -05:00

yindo commented

2026-02-22 17:19:27 -05:00