anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

qwen3-coder:latest does not support tools #1111

New Issue
Closed
opened 2026-02-16 17:29:29 -05:00 by yindo · 16 comments
yindo commented 2026-02-16 17:29:29 -05:00
Owner
Copy Link
Copy Source

Originally created by @mihaim on GitHub (Aug 5, 2025).

As per title, can't make qwen3-coder work under Ollama due to the tool error.

What to do about it?

Originally created by @mihaim on GitHub (Aug 5, 2025). As per title, can't make qwen3-coder work under Ollama due to the tool error. What to do about it?
yindo added the model-problem label 2026-02-16 17:29:29 -05:00
yindo closed this issue 2026-02-16 17:29:29 -05:00
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@ReidBraswell commented on GitHub (Aug 5, 2025):

https://x.com/thdxr/status/1952724502799843431

@ReidBraswell commented on GitHub (Aug 5, 2025): https://x.com/thdxr/status/1952724502799843431
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@GriffinMeyer commented on GitHub (Aug 5, 2025):

I also had the same issue with Ollama, the LM studio beta fixed it!

@GriffinMeyer commented on GitHub (Aug 5, 2025): I also had the same issue with Ollama, the LM studio beta fixed it!
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@SiddharthPant commented on GitHub (Aug 5, 2025):

Use lm-studio latest beta version to run the coder locally instead of Ollama. It has fixes for Qwen3-coder

Image
@SiddharthPant commented on GitHub (Aug 5, 2025): Use lm-studio latest beta version to run the coder locally instead of Ollama. It has fixes for Qwen3-coder <img width="1152" height="568" alt="Image" src="https://github.com/user-attachments/assets/bf5c2be0-74ef-4850-8df7-1ee74dfa6c8d" />
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@thdxr commented on GitHub (Aug 5, 2025):

^ yes

@thdxr commented on GitHub (Aug 5, 2025): ^ yes
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@desugar-64 commented on GitHub (Aug 5, 2025):

It works well with the latest beta of LM Studio, which I’m already using with OpenCode. For me, it works very well, I’m using it right now.

@desugar-64 commented on GitHub (Aug 5, 2025): It works well with the latest beta of LM Studio, which I’m already using with OpenCode. For me, it works very well, I’m using it right now.
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@johan-lejdung commented on GitHub (Sep 3, 2025):

It works well with the latest beta of LM Studio, which I’m already using with OpenCode. For me, it works very well, I’m using it right now.
@desugar-64
What kind of performance are you getting? Mine is abysmal. I'm sitting on a good CPU, Radeon RX 6750 XT and 64GB ram. I'm wondering if I have to do anything special to tune performance

@johan-lejdung commented on GitHub (Sep 3, 2025): > It works well with the latest beta of LM Studio, which I’m already using with OpenCode. For me, it works very well, I’m using it right now. @desugar-64 What kind of performance are you getting? Mine is abysmal. I'm sitting on a good CPU, Radeon RX 6750 XT and 64GB ram. I'm wondering if I have to do anything special to tune performance
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@desugar-64 commented on GitHub (Sep 4, 2025):

@johan-lejdung Hi.
I’m using a tuned model from Unsloth. It’s faster and more memory-efficient than the original, running with Q8XL UD quantization, a 127000-token context window, and generating around 17–20 tokens per second. With a smaller context window (e.g., 65536 tokens), performance ramps up to about 30 tokens per second.

My machine:

  • CPU: AMD Ryzen R9 9950X3D
  • RAM: 64GB DDR5 6400
  • GPU: Nvidia RTX 5070, 12GB VRAM

The model’s experts are forced to run on the CPU, but the KV cache is on the GPU using Flash Attention and Q8 quantization.

Image
@desugar-64 commented on GitHub (Sep 4, 2025): @johan-lejdung Hi. I’m using a tuned model from [Unsloth](https://docs.unsloth.ai/basics/qwen3-coder-how-to-run-locally#recommended-settings). It’s faster and more memory-efficient than the original, running with Q8XL UD quantization, a 127000-token context window, and generating around 17–20 tokens per second. With a smaller context window (e.g., 65536 tokens), performance ramps up to about 30 tokens per second. My machine: - CPU: AMD Ryzen R9 9950X3D - RAM: 64GB DDR5 6400 - GPU: Nvidia RTX 5070, 12GB VRAM The model’s experts are forced to run on the CPU, but the KV cache is on the GPU using Flash Attention and Q8 quantization. <img width="3834" height="1678" alt="Image" src="https://github.com/user-attachments/assets/0e57ba4e-9361-468c-a3ee-deef7b9dd8ad" />
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@desugar-64 commented on GitHub (Sep 4, 2025):

To tune performance, it definitely helps to move the KV cache (basically your chat history) into GPU VRAM. If it doesn’t fit, you can either quantize it or reduce the context window size. As for the model’s experts, they’re forced to run on the CPU — but since they’re relatively small, the CPU can handle them quite well.

But, performance is a relative measure, it depends on what you consider abysmal. For instance, 20 tokens per second is perfectly fine for my development. It outputs text faster than I can read it, which is good enough to me. The most time-consuming part is prompt processing: when you add huge chunks of code into the chat history, it takes some time. But moving the KV cache to the GPU speeds this up.

@desugar-64 commented on GitHub (Sep 4, 2025): To tune performance, it definitely helps to move the KV cache (basically your chat history) into GPU VRAM. If it doesn’t fit, you can either quantize it or reduce the context window size. As for the model’s experts, they’re forced to run on the CPU — but since they’re relatively small, the CPU can handle them quite well. But, performance is a relative measure, it depends on what you consider abysmal. For instance, 20 tokens per second is perfectly fine for my development. It outputs text faster than I can read it, which is good enough to me. The most time-consuming part is prompt processing: when you add huge chunks of code into the chat history, it takes some time. But moving the KV cache to the GPU speeds this up.
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@johan-lejdung commented on GitHub (Sep 4, 2025):

@desugar-64

Thank you very much! Well for me using qwen3-coder with default settings and opencode it took around 30-60sec to even start generating tokens, once it did it was fast enough. I'll try the one you suggested and see if it's good enough to run as a local refactoring agent :)

@johan-lejdung commented on GitHub (Sep 4, 2025): @desugar-64 Thank you very much! Well for me using qwen3-coder with default settings and opencode it took around 30-60sec to even start generating tokens, once it did it was fast enough. I'll try the one you suggested and see if it's good enough to run as a local refactoring agent :)
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@desugar-64 commented on GitHub (Sep 4, 2025):

around 30-60sec

The delay you’re observing is due to prompt processing. OpenCode uses a fairly lengthy system prompt. You can see this in the LM Studios logs, which show how it processes the prompt sequentially. One way to speed this up is by moving the KV cache to GPU memory and quantizing it.

Alternatively, you can try a different coding agents. From my experience, OpenCode doesn’t perform very well with this particular model, it often gets stuck editing files for several dozen seconds. Crush, on the other hand, is much faster at file editing with this model. I’ve also found RooCode to be a good alternative; their system prompts aren’t as long, and the system is smart enough to continue iterative development efficiently.

@desugar-64 commented on GitHub (Sep 4, 2025): > around 30-60sec The delay you’re observing is due to prompt processing. OpenCode uses a fairly lengthy system prompt. You can see this in the LM Studios logs, which show how it processes the prompt sequentially. One way to speed this up is by moving the KV cache to GPU memory and quantizing it. Alternatively, you can try a different coding agents. From my experience, OpenCode doesn’t perform very well with this particular model, it often gets stuck editing files for several dozen seconds. Crush, on the other hand, is much faster at file editing with this model. I’ve also found RooCode to be a good alternative; their system prompts aren’t as long, and the system is smart enough to continue iterative development efficiently.
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@johan-lejdung commented on GitHub (Sep 4, 2025):

@desugar-64 sorry to keep bothering you, but are you using a different prompt template?

I'm getting this error after spinning it up on my desktop and then trying to access it on my laptop.
Image

Did you use the template from here? https://docs.unsloth.ai/basics/qwen3-coder-how-to-run-locally#tool-calling-fixes

@johan-lejdung commented on GitHub (Sep 4, 2025): @desugar-64 sorry to keep bothering you, but are you using a different prompt template? I'm getting this error after spinning it up on my desktop and then trying to access it on my laptop. <img width="1120" height="135" alt="Image" src="https://github.com/user-attachments/assets/c97f4d4d-810e-4fd4-bc12-cf22438bdd80" /> Did you use the template from here? https://docs.unsloth.ai/basics/qwen3-coder-how-to-run-locally#tool-calling-fixes
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@desugar-64 commented on GitHub (Sep 4, 2025):

Ah, yes, that error. For some reason, the default prompt template is broken, LMStudio doesn’t understand it. I learned about this from GosuCoder's YouTube video. He provided a fixed version of the template, and it worked really well for me.
Here’s the fixed version I use:

{#- Copyright 2025-present the Unsloth team. All rights reserved. #}
{#- Licensed under the Apache License, Version 2.0 (the "License") #}
{#- Edits made by Unsloth to fix the chat template #}
{% macro render_item_list(item_list, tag_name='required') %}
    {%- if item_list is defined and item_list is iterable and item_list | length > 0 %}
        {%- if tag_name %}{{- '\n<' ~ tag_name ~ '>' -}}{% endif %}
            {{- '[' }}
                {%- for item in item_list -%}
                    {%- if loop.index > 1 %}{{- ", "}}{% endif -%}
                    {%- if item is string -%}
                        {{ "`" ~ item ~ "`" }}
                    {%- else -%}
                        {{ item }}
                    {%- endif -%}
                {%- endfor -%}
            {{- ']' }}
        {%- if tag_name %}{{- '</' ~ tag_name ~ '>' -}}{% endif %}
    {%- endif %}
{% endmacro %}

{%- if messages[0]["role"] == "system" %}
    {%- set system_message = messages[0]["content"] %}
    {%- set loop_messages = messages[1:] %}
{%- else %}
    {%- set loop_messages = messages %}
{%- endif %}

{%- if not tools is defined %}
    {%- set tools = [] %}
{%- endif %}

{%- if system_message is defined %}
    {{- "<|im_start|>system\n" + system_message }}
{%- else %}
    {%- if tools is iterable and tools | length > 0 %}
        {{- "<|im_start|>system\nYou are Qwen, a helpful AI assistant that can interact with a computer to solve tasks." }}
    {%- endif %}
{%- endif %}
{%- if tools is iterable and tools | length > 0 %}
    {{- "\n\nYou have access to the following functions:\n\n" }}
    {{- "<tools>" }}
    {%- for tool in tools %}
        {%- if tool.function is defined %}
            {%- set tool = tool.function %}
        {%- endif %}
        {{- "\n<function>\n<name>" ~ tool.name ~ "</name>" }}
        {{- '\n<description>' ~ (tool.description | trim) ~ '</description>' }}
        {{- '\n<parameters>' }}
        {%- for param_name, param_fields in tool.parameters.properties|items %}
            {{- '\n<parameter>' }}
            {{- '\n<name>' ~ param_name ~ '</name>' }}
            {%- if param_fields.type is defined %}
                {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
            {%- endif %}
            {%- if param_fields.description is defined %}
                {{- '\n<description>' ~ (param_fields.description | trim) ~ '</description>' }}
            {%- endif %}
            {{- render_item_list(param_fields.enum, 'enum') }}
            {%- set handled_keys = ['type', 'description', 'enum', 'required'] %}
            {%- for json_key in param_fields %}
                {%- if json_key not in handled_keys %}
                    {%- set normed_json_key = json_key | replace("-", "_") | replace(" ", "_") | replace("$", "") %}
                    {%- if param_fields[json_key] is mapping %}
                        {{- '\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | tojson) ~ '</' ~ normed_json_key ~ '>' }}
                    {%- else %}
                        {{- '\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | string) ~ '</' ~ normed_json_key ~ '>' }}
                    {%- endif %}
                {%- endif %}
            {%- endfor %}
            {{- render_item_list(param_fields.required, 'required') }}
            {{- '\n</parameter>' }}
        {%- endfor %}
        {{- render_item_list(tool.parameters.required, 'required') }}
        {{- '\n</parameters>' }}
        {%- if tool.return is defined %}
            {%- if tool.return is mapping %}
                {{- '\n<return>' ~ (tool.return | tojson) ~ '</return>' }}
            {%- else %}
                {{- '\n<return>' ~ (tool.return | string) ~ '</return>' }}
            {%- endif %}
        {%- endif %}
        {{- '\n</function>' }}
    {%- endfor %}
    {{- "\n</tools>" }}
    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
{%- endif %}
{%- if system_message is defined %}
    {{- '<|im_end|>\n' }}
{%- else %}
    {%- if tools is iterable and tools | length > 0 %}
        {{- '<|im_end|>\n' }}
    {%- endif %}
{%- endif %}
{%- for message in loop_messages %}
    {%- if message.role == "assistant" and message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
        {{- '<|im_start|>' + message.role }}
        {%- if message.content is defined and message.content is string and message.content | trim | length > 0 %}
            {{- '\n' + message.content | trim + '\n' }}
        {%- endif %}
        {%- for tool_call in message.tool_calls %}
            {%- if tool_call.function is defined %}
                {%- set tool_call = tool_call.function %}
            {%- endif %}
            {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
            {%- if tool_call.arguments is defined %}
                {%- for args_name, args_value in tool_call.arguments|items %}
                    {{- '<parameter=' + args_name + '>\n' }}
                    {%- set args_value = args_value if args_value is string else args_value | string %}
                    {{- args_value }}
                    {{- '\n</parameter>\n' }}
                {%- endfor %}
            {%- endif %}
            {{- '</function>\n</tool_call>' }}
        {%- endfor %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "user" or message.role == "system" or message.role == "assistant" %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
    {%- elif message.role == "tool" %}
        {%- if loop.previtem and loop.previtem.role != "tool" %}
            {{- '<|im_start|>user\n' }}
        {%- endif %}
        {{- '<tool_response>\n' }}
        {{- message.content }}
        {{- '\n</tool_response>\n' }}
        {%- if not loop.last and loop.nextitem.role != "tool" %}
            {{- '<|im_end|>\n' }}
        {%- elif loop.last %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- else %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
{%- endif %}
{#- Copyright 2025-present the Unsloth team. All rights reserved. #}
{#- Licensed under the Apache License, Version 2.0 (the "License") #}
@desugar-64 commented on GitHub (Sep 4, 2025): Ah, yes, that error. For some reason, the default prompt template is broken, LMStudio doesn’t understand it. I learned about this from [GosuCoder's YouTube video](https://youtu.be/HQ7dNWqjv7E?t=1000). He provided a fixed version of the template, and it worked really well for me. Here’s the fixed version I use: ```python {#- Copyright 2025-present the Unsloth team. All rights reserved. #} {#- Licensed under the Apache License, Version 2.0 (the "License") #} {#- Edits made by Unsloth to fix the chat template #} {% macro render_item_list(item_list, tag_name='required') %} {%- if item_list is defined and item_list is iterable and item_list | length > 0 %} {%- if tag_name %}{{- '\n<' ~ tag_name ~ '>' -}}{% endif %} {{- '[' }} {%- for item in item_list -%} {%- if loop.index > 1 %}{{- ", "}}{% endif -%} {%- if item is string -%} {{ "`" ~ item ~ "`" }} {%- else -%} {{ item }} {%- endif -%} {%- endfor -%} {{- ']' }} {%- if tag_name %}{{- '</' ~ tag_name ~ '>' -}}{% endif %} {%- endif %} {% endmacro %} {%- if messages[0]["role"] == "system" %} {%- set system_message = messages[0]["content"] %} {%- set loop_messages = messages[1:] %} {%- else %} {%- set loop_messages = messages %} {%- endif %} {%- if not tools is defined %} {%- set tools = [] %} {%- endif %} {%- if system_message is defined %} {{- "<|im_start|>system\n" + system_message }} {%- else %} {%- if tools is iterable and tools | length > 0 %} {{- "<|im_start|>system\nYou are Qwen, a helpful AI assistant that can interact with a computer to solve tasks." }} {%- endif %} {%- endif %} {%- if tools is iterable and tools | length > 0 %} {{- "\n\nYou have access to the following functions:\n\n" }} {{- "<tools>" }} {%- for tool in tools %} {%- if tool.function is defined %} {%- set tool = tool.function %} {%- endif %} {{- "\n<function>\n<name>" ~ tool.name ~ "</name>" }} {{- '\n<description>' ~ (tool.description | trim) ~ '</description>' }} {{- '\n<parameters>' }} {%- for param_name, param_fields in tool.parameters.properties|items %} {{- '\n<parameter>' }} {{- '\n<name>' ~ param_name ~ '</name>' }} {%- if param_fields.type is defined %} {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }} {%- endif %} {%- if param_fields.description is defined %} {{- '\n<description>' ~ (param_fields.description | trim) ~ '</description>' }} {%- endif %} {{- render_item_list(param_fields.enum, 'enum') }} {%- set handled_keys = ['type', 'description', 'enum', 'required'] %} {%- for json_key in param_fields %} {%- if json_key not in handled_keys %} {%- set normed_json_key = json_key | replace("-", "_") | replace(" ", "_") | replace("$", "") %} {%- if param_fields[json_key] is mapping %} {{- '\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | tojson) ~ '</' ~ normed_json_key ~ '>' }} {%- else %} {{- '\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | string) ~ '</' ~ normed_json_key ~ '>' }} {%- endif %} {%- endif %} {%- endfor %} {{- render_item_list(param_fields.required, 'required') }} {{- '\n</parameter>' }} {%- endfor %} {{- render_item_list(tool.parameters.required, 'required') }} {{- '\n</parameters>' }} {%- if tool.return is defined %} {%- if tool.return is mapping %} {{- '\n<return>' ~ (tool.return | tojson) ~ '</return>' }} {%- else %} {{- '\n<return>' ~ (tool.return | string) ~ '</return>' }} {%- endif %} {%- endif %} {{- '\n</function>' }} {%- endfor %} {{- "\n</tools>" }} {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }} {%- endif %} {%- if system_message is defined %} {{- '<|im_end|>\n' }} {%- else %} {%- if tools is iterable and tools | length > 0 %} {{- '<|im_end|>\n' }} {%- endif %} {%- endif %} {%- for message in loop_messages %} {%- if message.role == "assistant" and message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %} {{- '<|im_start|>' + message.role }} {%- if message.content is defined and message.content is string and message.content | trim | length > 0 %} {{- '\n' + message.content | trim + '\n' }} {%- endif %} {%- for tool_call in message.tool_calls %} {%- if tool_call.function is defined %} {%- set tool_call = tool_call.function %} {%- endif %} {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }} {%- if tool_call.arguments is defined %} {%- for args_name, args_value in tool_call.arguments|items %} {{- '<parameter=' + args_name + '>\n' }} {%- set args_value = args_value if args_value is string else args_value | string %} {{- args_value }} {{- '\n</parameter>\n' }} {%- endfor %} {%- endif %} {{- '</function>\n</tool_call>' }} {%- endfor %} {{- '<|im_end|>\n' }} {%- elif message.role == "user" or message.role == "system" or message.role == "assistant" %} {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }} {%- elif message.role == "tool" %} {%- if loop.previtem and loop.previtem.role != "tool" %} {{- '<|im_start|>user\n' }} {%- endif %} {{- '<tool_response>\n' }} {{- message.content }} {{- '\n</tool_response>\n' }} {%- if not loop.last and loop.nextitem.role != "tool" %} {{- '<|im_end|>\n' }} {%- elif loop.last %} {{- '<|im_end|>\n' }} {%- endif %} {%- else %} {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }} {%- endif %} {%- endfor %} {%- if add_generation_prompt %} {{- '<|im_start|>assistant\n' }} {%- endif %} {#- Copyright 2025-present the Unsloth team. All rights reserved. #} {#- Licensed under the Apache License, Version 2.0 (the "License") #} ```
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@johan-lejdung commented on GitHub (Sep 4, 2025):

Thank you!

I got it running but it is easily 2-3min to first token. I think my GPU isn't used at all because it's not CUDA supported. And CPU could have more threads. I'll give this another go once I've upgraded my hardware down the line.

Much appreciation with the help!

@johan-lejdung commented on GitHub (Sep 4, 2025): Thank you! I got it running but it is easily 2-3min to first token. I think my GPU isn't used at all because it's not CUDA supported. And CPU could have more threads. I'll give this another go once I've upgraded my hardware down the line. Much appreciation with the help!
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@desugar-64 commented on GitHub (Sep 4, 2025):

I think my GPU isn't used at all because it's not CUDA supported.

Since you have an AMD Radeon GPU, you should use the Vulkan-based runtime for LM Studio. Verify this in the settings to ensure correctness.

@desugar-64 commented on GitHub (Sep 4, 2025): > I think my GPU isn't used at all because it's not CUDA supported. Since you have an AMD Radeon GPU, you should use the Vulkan-based runtime for LM Studio. Verify this in the settings to ensure correctness.
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@johan-lejdung commented on GitHub (Sep 4, 2025):

Good call, that was turned on! I'm kind of stumped though, on my desktop I get great performance using that model in local chat. 25tk/s

I fed it 3 pages of a book and asked it to summarise, took maybe 5-10s to process the prompt.

I ended up installing opencode on my windows machine, and it seems the issue is somewhat related to opencode. Took it at least a minute to respond to "Hello"

I also gave RooCode a try, it was not really better. I think context length is limiting me in the end, cant seem to go above 22k without failing to load the model.

I had some fun debugging this though!

@johan-lejdung commented on GitHub (Sep 4, 2025): Good call, that was turned on! I'm kind of stumped though, on my desktop I get great performance using that model in local chat. 25tk/s I fed it 3 pages of a book and asked it to summarise, took maybe 5-10s to process the prompt. I ended up installing opencode on my windows machine, and it seems the issue is somewhat related to opencode. Took it at least a minute to respond to "Hello" I also gave RooCode a try, it was not really better. I think context length is limiting me in the end, cant seem to go above 22k without failing to load the model. I had some fun debugging this though!
yindo commented 2026-02-16 17:29:30 -05:00
Author
Owner
Copy Link
Copy Source

@desugar-64 commented on GitHub (Sep 5, 2025):

Make sure to keep at least 1 GB of VRAM free. Once VRAM is oversubscribed, performance drops severely

@desugar-64 commented on GitHub (Sep 5, 2025): Make sure to keep at least 1 GB of VRAM free. Once VRAM is oversubscribed, performance drops severely
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label model-problem
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#1111
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?