txtai/examples/44_Prompt_templates_and_task_chains.ipynb

Prompt templates and task chains

txtai has long had support for workflows. Workflows connect the input and outputs of machine learning models together to create powerful transformation and processing functions.

There has been a recent surge in interest in "model prompting", which is the process of building a natural language description of a task and passing it to a large language model (LLM). txtai has recently improved support for task templating, which builds string outputs from a set of parameters.

This notebook demonstrates how txtai workflows can be used to apply prompt templates and chain those tasks together.

Install dependencies

Install txtai and all dependencies.

In [ ]:

%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[api]

Prompt workflow

First, we'll look at building a workflow with a series of model prompts. This workflow creates a conditional translation using a statement and target language. Another task reads that output text and detects the language.

This workflow uses a LLM pipeline. The LLM pipeline loads a local model for inference, in this case Qwen3-4B-Instruct-2507. The LLM pipeline supports local transformers models, llama.cpp models and LLM APIs such as Ollama, vLLM, OpenAI, Claude etc.

It's important to note that a pipeline is simply a callable function. It can easily be replaced with a call to an external API.

In [ ]:

from txtai import LLM, Workflow
from txtai.workflow import TemplateTask

# Create LLM
llm = LLM("Qwen/Qwen3-4B-Instruct-2507")

# Define workflow or chaining of tasks together.
workflow = Workflow([
    TemplateTask(
        template="Translate text '{statement}' to {language} if the text is English, otherwise keep the original text",
        action=llm
    ),
    TemplateTask(
        template="What language is the following text. Only print the answer? {text}",
        action=llm
    )
])

inputs = [
    {"statement": "Hello, how are you", "language": "French"},
    {"statement": "Hallo, wie geht's dir", "language": "French"}
]

print(list(workflow(inputs)))

['French', 'German']

Let's recap what happened here. The first workflow task conditionally translates text to a language if it's English.

The first statement is Hello, how are you with a target language of French. So the statement is translated to French.

The second statement is German, so it's not converted to French.

The next step asks the model what the language is and it correctly prints French and German.

Prompt Workflow as YAML

The same workflow above can be created with YAML configuration.

In [2]:

%%writefile workflow.yml

llm:
  path: Qwen/Qwen3-4B-Instruct-2507

workflow:
  chain:
    tasks:
      - task: template
        template: Translate text '{statement}' to {language} if the text is English, otherwise keep the original text
        action: llm
      - task: template
        template: What language is the following text. Only print the answer? {text}
        action: llm

Writing workflow.yml

In [2]:

from txtai import Application

app = Application("workflow.yml")
print(list(app.workflow("chain", inputs)))

['French', 'German']

As expected, the same result! This is a matter of preference on how you want to create a workflow. One advantage of YAML workflows is that an API can easily be created from the workflow file.

Prompt Workflow via an API call

Let's say you want the workflow to be available via an API call. Well good news, txtai has a built in API mechanism using FastAPI.

In [ ]:

# Start an API service
!CONFIG=workflow.yml nohup uvicorn "txtai.api:app" &> api.log &
!sleep 60

In [3]:

import requests

# Run API request
requests.post("http://localhost:8000/workflow", json={"name": "chain", "elements": inputs}).json()

Out [3]:

['French', 'German']

Just like the previous steps, except through an API call. Let's run via cURL for good measure.

In [4]:

%%bash

curl -s -X POST "http://localhost:8000/workflow" \
     -H "Content-Type: application/json" \
     --data @- << EOF
{
  "name": "chain",
  "elements": [
    {"statement": "Hello, how are you", "language": "French"},
    {"statement": "Hallo, wie geht's dir", "language": "French"}
  ]
}
EOF

["French","German"]

One last time, the same output is shown.

If your primary development environment isn't Python, txtai does have API bindings for JavaScript, Rust, Go and Java.

More information on the API is available here.

Chat with your data

"Chat with your data" is a popular entry point into the AI space. Let's run an example.

In [10]:

%%writefile search.yml

writable: false
cloud:
  provider: huggingface-hub
  container: neuml/txtai-intro

rag:
  path: Qwen/Qwen3-4B-Instruct-2507
  output: reference
  template: |
    Answer the following question using only the context below.

    Question: {question}
    Context: {context}

workflow:
  search:
    tasks:
      - action: rag
      - task: template
        template: "{answer}\n\nReference: {reference}"
        rules:
          answer: I don't have data on that

Writing search.yml

In [ ]:

app = Application("search.yml")
print(list(app.workflow("search", ["Find something about North America"])))

["Canada's last fully intact ice shelf has suddenly collapsed, forming a Manhattan-sized iceberg.\n\nReference: 1"]

The first thing the code above does is run an embeddings search to build a conversational context. That context is then used to build a prompt and inference is run against the LLM.

The next task formats the outputs with a reference to the best matching record. In this case, it's only an id of 1. But this can be much more useful if the id is a URL or there is logic to format the id back to a unique reference string.

Wrapping up

This notebook covered how to build prompt templates and task chains through a series of results. txtai has long had a robust and efficient workflow framework for connecting models together. This can be small and simple models and/or prompting with large models. Go ahead and give it a try!