neuml/txtai
1
0
Fork 0
mirror of https://github.com/neuml/txtai.git synced 2026-10-11 21:39:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add example notebook covering OpenCode LLM, closes #1011

Browse Source
This commit is contained in:
davidmezzetti
2026-01-21 08:28:41 -05:00
parent c48ecb3062
commit 001b97b0bf
2 changed files with 378 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/examples.md
+1
View File
@@ -49,6 +49,7 @@ Autonomous agents, retrieval augmented generation (RAG), chat with your data, pi
| [Medical RAG Research with txtai](https://github.com/neuml/txtai/blob/master/examples/75_Medical_RAG_Research_with_txtai.ipynb) | Analyze PubMed article metadata with RAG | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/75_Medical_RAG_Research_with_txtai.ipynb) |
| [GraphRAG with Wikipedia and GPT OSS](https://github.com/neuml/txtai/blob/master/examples/77_GraphRAG_with_Wikipedia_and_GPT_OSS.ipynb) | Deep graph search powered RAG | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/77_GraphRAG_with_Wikipedia_and_GPT_OSS.ipynb) |
| [RAG is more than Vector Search](https://github.com/neuml/txtai/blob/master/examples/79_RAG_is_more_than_Vector_Search.ipynb) | Context retrieval via Web, SQL and other sources | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/79_RAG_is_more_than_Vector_Search.ipynb) |
| [OpenCode as a txtai LLM](https://github.com/neuml/txtai/blob/master/examples/81_OpenCode_as_a_txtai_LLM.ipynb) | Integrate OpenCode with the txtai ecosystem | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/81_OpenCode_as_a_txtai_LLM.ipynb) |
## Pipelines
examples/81_OpenCode_as_a_txtai_LLM.ipynb
+377
View File
@@ -0,0 +1,377 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "f95d2373",
"metadata": {},
"source": [
"# OpenCode as a txtai LLM\n",
"\n",
"[OpenCode](https://github.com/anomalyco/opencode) is an open source AI coding agent. It's rapidly growing in popularity as an open alternative to [Claude Code](https://code.claude.com/docs/en/overview). \n",
"\n",
"OpenCode shows the power of permissive open source. It's rapidly undergoing mass adoption and as of January 2026 sits at `81K+` GitHub ⭐'s. OpenCode supports running as a [local server](https://opencode.ai/docs/server/) via `opencode serve`.\n",
"\n",
"This notebook will explore how TxtAI integrates this as a LLM pipeline."
]
},
{
"cell_type": "markdown",
"id": "a8719b92",
"metadata": {},
"source": [
"# Install dependencies\n",
"\n",
"Install `txtai` and all dependencies."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e00c85bc",
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"%%capture\n",
"!pip install git+https://github.com/neuml/txtai#egg=txtai[api,pipeline-llm]\n",
"\n",
"# Install OpenCode if not already installed and run serve\n",
"!curl -fsSL https://opencode.ai/install | bash\n",
"!opencode serve &"
]
},
{
"cell_type": "markdown",
"id": "73ffe60e",
"metadata": {},
"source": [
"# Create a OpenCode LLM\n",
"\n",
"Now that everything is installed, let's test it out!"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "0e35a304",
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"```python\n",
"print(\"Hello, World!\")\n",
"```"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from IPython.display import Markdown, display\n",
"\n",
"from txtai import LLM\n",
"\n",
"def generate(request):\n",
" display(Markdown(llm(request)))\n",
"\n",
"# Connect to OpenCode server and use default LLM provider\n",
"llm = LLM(\"opencode\")\n",
"\n",
"generate(\"Show a Python Hello World example\")"
]
},
{
"cell_type": "markdown",
"id": "55dd0fad",
"metadata": {},
"source": [
"OK, now let's try this for a more advanced use case."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "b3080841",
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"```python\n",
"from txtai import Embeddings, LLM\n",
"\n",
"# Create embeddings model\n",
"embeddings = Embeddings(path=\"sentence-transformers/all-MiniLM-L6-v2\")\n",
"\n",
"# Create LLM\n",
"llm = LLM(\"huggingface/Qwen/Qwen2.5-1.5B-Instruct\")\n",
"\n",
"# Sample documents\n",
"documents = [\n",
" \"Python is a programming language created by Guido van Rossum\",\n",
" \"Machine learning is a subset of artificial intelligence\",\n",
" \"Deep learning uses neural networks with multiple layers\"\n",
"]\n",
"\n",
"# Build the index\n",
"embeddings.index([(i, doc) for i, doc in enumerate(documents)])\n",
"\n",
"# RAG function\n",
"def rag_query(question):\n",
" # Search for relevant documents\n",
" results = embeddings.search(question, 2)\n",
" \n",
" # Build context from retrieved documents\n",
" context = \" \".join([documents[result[0]] for result in results])\n",
" \n",
" # Generate response using LLM with context\n",
" prompt = f\"Context: {context}\\n\\nQuestion: {question}\\n\\nAnswer:\"\n",
" return llm(prompt)\n",
"\n",
"# Example usage\n",
"question = \"What is Python?\"\n",
"answer = rag_query(question)\n",
"print(f\"Question: {question}\")\n",
"print(f\"Answer: {answer}\")\n",
"```"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"generate(\"Show a simple and basic TxtAI RAG example\")"
]
},
{
"cell_type": "markdown",
"id": "b44e0309",
"metadata": {},
"source": [
"Interesting! It's not limited to Python though."
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "0a2ec1b4",
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"```assembly\n",
"section .data\n",
" hello db 'Hello, World!', 10 ; String with newline\n",
" hello_len equ $ - hello ; Length of string\n",
"\n",
"section .text\n",
" global _start\n",
"\n",
"_start:\n",
" ; sys_write system call\n",
" mov eax, 4 ; sys_write\n",
" mov ebx, 1 ; stdout\n",
" mov ecx, hello ; message\n",
" mov edx, hello_len ; message length\n",
" int 0x80 ; kernel interrupt\n",
"\n",
" ; sys_exit system call\n",
" mov eax, 1 ; sys_exit\n",
" mov ebx, 0 ; exit code 0\n",
" int 0x80 ; kernel interrupt\n",
"```\n",
"\n",
"Compile and run:\n",
"```bash\n",
"nasm -f elf32 hello.asm -o hello.o\n",
"ld -m elf_i386 hello.o -o hello\n",
"./hello\n",
"```"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"generate(\"Show a x86 assembly hello world\")"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "4e28d5c4",
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"```python\n",
"a = None\n",
"if a is not None:\n",
" a.startswith(\"hello\")\n",
"else:\n",
" print(\"a is None, cannot call startswith\")\n",
"```\n",
"\n",
"**What's wrong:** The variable `a` is `None`, which is a `NoneType` object. The `startswith()` method only exists for strings, not for `NoneType`. You need to check if `a` is not `None` before calling string methods on it."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"generate(\"\"\"\n",
"Fix the following error in the code below and explain what's wrong.\n",
"\n",
"Error:\n",
"AttributeError: 'NoneType' object has no attribute 'startswith'\n",
"\n",
"Code:\n",
"a = None\n",
"a.startswith(\"hello\")\n",
"\"\"\")"
]
},
{
"cell_type": "markdown",
"id": "156122f0",
"metadata": {},
"source": [
"# OpenAI-compatible Endpoint\n",
"\n",
"Now that OpenCode is integrated into the TxtAI ecosystem, it opens a lot of different opportunities. Let's say we want to host a local OpenAI compatible endpoint, no problem!\n",
"\n",
"Write the following file.\n",
"\n",
"## config.yml\n",
"\n",
"```yaml\n",
"# Enable OpenAI compat endpoint\n",
"openai: True\n",
"\n",
"llm:\n",
" path: opencode\n",
"```\n",
"\n",
"and start the following process.\n",
"\n",
"```\n",
"CONFIG=config.yml uvicorn \"txtai.api:app\"\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "33057f59",
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"```python\n",
"def factorial(n):\n",
" if n < 0:\n",
" raise ValueError(\"Factorial is not defined for negative numbers\")\n",
" if n == 0 or n == 1:\n",
" return 1\n",
" result = 1\n",
" for i in range(2, n + 1):\n",
" result *= i\n",
" return result\n",
"\n",
"# Example usage:\n",
"# print(factorial(5)) # Output: 120\n",
"```"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from IPython.display import Markdown, display\n",
"\n",
"from openai import OpenAI\n",
"\n",
"client = OpenAI(\n",
" base_url=\"http://localhost:8000/v1\",\n",
" api_key=\"api-key\",\n",
")\n",
"\n",
"response = client.chat.completions.create(\n",
" messages=[{\n",
" \"role\": \"user\",\n",
" \"content\": \"Show a Python factorial function\",\n",
" }],\n",
" model=\"opencode\",\n",
")\n",
"\n",
"display(Markdown((response.choices[0].message.content)))"
]
},
{
"cell_type": "markdown",
"id": "58a9169b",
"metadata": {},
"source": [
"Just like before we get an answer! This time via the OpenAI client. Fun times."
]
},
{
"cell_type": "markdown",
"id": "47f9dbbd",
"metadata": {},
"source": [
"# Wrapping up\n",
"\n",
"This notebook showed how to connect TxtAI and OpenCode. This will lead to some interesting integrations. One such example is [ncoder](https://github.com/neuml/ncoder), an AI coding agent that integrates with Jupyter Notebooks. Stay tuned for more!"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "local",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.19"
}
},
"nbformat": 4,
"nbformat_minor": 5
}