Add Workflow Quickstart Example, closes #996

This commit is contained in:
davidmezzetti
2025-12-01 18:44:14 -05:00
parent 4d6c0ecc47
commit 441d0eabd7
4 changed files with 64 additions and 1 deletions
+2
View File
@@ -165,6 +165,8 @@ Language model workflows, also known as semantic workflows, connect language mod
While LLMs are powerful, there are plenty of smaller, more specialized models that work better and faster for specific tasks. This includes models for extractive question-answering, automatic summarization, text-to-speech, transcription and translation.
Check out this [Workflow Quickstart Example](https://github.com/neuml/txtai/blob/master/examples/workflow_quickstart.py). Additional examples are listed below.
| Notebook | Description | |
|:----------|:-------------|------:|
| [Run pipeline workflows](https://github.com/neuml/txtai/blob/master/examples/14_Run_pipeline_workflows.ipynb) [▶️](https://www.youtube.com/watch?v=UBMPDCn1gEU) | Simple yet powerful constructs to efficiently process data | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/14_Run_pipeline_workflows.ipynb) |
+2
View File
@@ -77,6 +77,8 @@ Language model workflows, also known as semantic workflows, connect language mod
While LLMs are powerful, there are plenty of smaller, more specialized models that work better and faster for specific tasks. This includes models for extractive question-answering, automatic summarization, text-to-speech, transcription and translation.
Check out this [Workflow Quickstart Example](https://github.com/neuml/txtai/blob/master/examples/workflow_quickstart.py). Additional examples are listed below.
| Notebook | Description | |
|:----------|:-------------|------:|
| [Run pipeline workflows](https://github.com/neuml/txtai/blob/master/examples/14_Run_pipeline_workflows.ipynb) [▶️](https://www.youtube.com/watch?v=UBMPDCn1gEU) | Simple yet powerful constructs to efficiently process data | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/14_Run_pipeline_workflows.ipynb) |
+1 -1
View File
@@ -202,4 +202,4 @@ Workflows are callable objects. Workflows take an input of iterable data element
## More examples
See [this link](../examples/#workflows) for a full list of workflow examples.
Check out this [Workflow Quickstart Example](https://github.com/neuml/txtai/blob/master/examples/workflow_quickstart.py). See [this link](../examples/#workflows) for a full list of workflow examples.
+59
View File
@@ -0,0 +1,59 @@
"""
Workflow Quick Start
Easy to use way to get started with deterministic workflows.
TxtAI has many example notebooks covering everything the framework provides
Examples: https://neuml.github.io/txtai/examples
Install TxtAI
pip install txtai[pipeline-data]
"""
from txtai import LLM, Workflow
from txtai.pipeline import Summary, Textractor, Translation
from txtai.workflow import Task
# Step 1: Define available pipelines
textractor = Textractor(backend="docling", headers={"user-agent": "Mozilla/5.0"})
summary = Summary()
translate = Translation()
# Step 2: Define workflow tasks
workflow = Workflow([Task(textractor), Task(summary), Task(lambda inputs: [translate(x, "fr") for x in inputs])])
# Step 3: Run the workflow
print(list(workflow(["https://neuml.com"])))
# Each component above is a single model that specializes in a task
# LLMs can also be used to accomplish the same tasks
# pylint: disable=E0102,C0116
def summary(text):
return f"""
Summarize the following text in 40 words or less.
{text}
"""
def translate(text, language):
return f"""
Translate the following text to {language}.
{text}
"""
textractor = Textractor(backend="docling", headers={"user-agent": "Mozilla/5.0"})
llm = LLM("Qwen/Qwen3-4B-Instruct-2507")
workflow = Workflow(
[
Task(textractor),
Task(lambda inputs: llm([summary(x) for x in inputs], maxlength=25000, defaultrole="user")),
Task(lambda inputs: llm([translate(x, "fr") for x in inputs], maxlength=25000, defaultrole="user")),
]
)
print(list(workflow(["https://neuml.com"])))