OpenCode as a txtai LLM

OpenCode is an open source AI coding agent. It's rapidly growing in popularity as an open alternative to Claude Code.

OpenCode shows the power of permissive open source. It's rapidly undergoing mass adoption and as of January 2026 sits at 81K+ GitHub ⭐'s. OpenCode supports running as a local server via opencode serve.

This notebook will explore how TxtAI integrates this as a LLM pipeline.

Install dependencies

Install txtai and all dependencies.

In [ ]:

%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[api,pipeline-llm]

# Install OpenCode if not already installed and run serve
!curl -fsSL https://opencode.ai/install | bash
!opencode serve &

Create a OpenCode LLM

Now that everything is installed, let's test it out!

In [6]:

from IPython.display import Markdown, display

from txtai import LLM

def generate(request):
    display(Markdown(llm(request)))

# Connect to OpenCode server and use default LLM provider
llm = LLM("opencode")

generate("Show a Python Hello World example")

<IPython.core.display.Markdown object>

OK, now let's try this for a more advanced use case.

In [8]:

generate("Show a simple and basic TxtAI RAG example")

<IPython.core.display.Markdown object>

Interesting! It's not limited to Python though.

In [9]:

generate("Show a x86 assembly hello world")

<IPython.core.display.Markdown object>

In [11]:

generate("""
Fix the following error in the code below and explain what's wrong.

Error:
AttributeError: 'NoneType' object has no attribute 'startswith'

Code:
a = None
a.startswith("hello")
""")

<IPython.core.display.Markdown object>

OpenAI-compatible Endpoint

Now that OpenCode is integrated into the TxtAI ecosystem, it opens a lot of different opportunities. Let's say we want to host a local OpenAI compatible endpoint, no problem!

Write the following file.

config.yml

# Enable OpenAI compat endpoint
openai: True

llm:
  path: opencode

and start the following process.

CONFIG=config.yml uvicorn "txtai.api:app"

In [16]:

from IPython.display import Markdown, display

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="api-key",
)

response = client.chat.completions.create(
    messages=[{
        "role": "user",
        "content": "Show a Python factorial function",
    }],
    model="opencode",
)

display(Markdown((response.choices[0].message.content)))

<IPython.core.display.Markdown object>

Just like before we get an answer! This time via the OpenAI client. Fun times.

Wrapping up

This notebook showed how to connect TxtAI and OpenCode. This will lead to some interesting integrations. One such example is ncoder, an AI coding agent that integrates with Jupyter Notebooks. Stay tuned for more!

9.9 KiB Raw Blame History