neuml/txtai
1
0
Fork 0
mirror of https://github.com/neuml/txtai.git synced 2026-10-12 02:19:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update RAG notebooks to add clarifying notes on LLM inference, closes #847

Browse Source
This commit is contained in:
davidmezzetti
2025-01-09 07:28:53 -05:00
parent 2d8cf5d80d
commit 1352040add
3 changed files with 79 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/52_Build_RAG_pipelines_with_txtai.ipynb
+30 -2
View File
@@ -71,9 +71,9 @@
"source": [
"# Start with the basics\n",
"\n",
"Let's jump right in and start with a simple LLM pipeline. The [LLM pipeline](https://neuml.github.io/txtai/pipeline/text/llm/) supports loading local LLM models from the [Hugging Face Hub](https://huggingface.co/models).\n",
"Let's jump right in and start with a simple LLM pipeline. The [LLM pipeline](https://neuml.github.io/txtai/pipeline/text/llm/) supports local LLM models via [Hugging Face Transformers](https://github.com/huggingface/transformers) and [llama.cpp](https://github.com/abetlen/llama-cpp-python).\n",
"\n",
"For those using LLM API services (OpenAI, Cohere, etc), this call can easily be replaced with an API call."
"The LLM pipeline also supports [API services (i.e. OpenAI, Claude, Bedrock etc) via LiteLLM](https://github.com/BerriAI/litellm). The LLM pipeline automatically detects the underlying LLM framework from the `path` parameter.\n"
],
"metadata": {
"id": "xmPN8RDF1pXd"
@@ -296,6 +296,27 @@
"id": "WaVeEHrIMpFr"
}
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_Before continuing, it's important to note that txtai has multiple ways to run LLM inference. In the past, prior to \"Chat Templates\", it was expected that the submitted text had all the required chat tokens embedded. The same prompt above can also be written with chat messages. This is especially important when working with LLM APIs (i.e. OpenAI, Claude, Bedrock etc)._\n",
"\n",
"```python\n",
"llm([\n",
" {\"role\": \"system\": \"You are a friendly assistant. You answer questions from users.\"}\n",
" {\"role\": \"user\", \"content\": f\"\"\"\n",
" Answer the following question using only the context below. Only include information specifically discussed.\n",
"\n",
" question: {question}\n",
" context: {text} \n",
" \"\"\"}\n",
"])\n",
"```\n",
"\n",
"_See the [LLM pipeline documentation](https://neuml.github.io/txtai/pipeline/text/llm/) for more information._"
]
},
{
"cell_type": "markdown",
"source": [
@@ -574,6 +595,13 @@
"id": "vHdE2Q59jNnF"
}
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_As with the LLM pipeline, the RAG pipeline also supports chat messages. See the [RAG pipeline documentation](https://neuml.github.io/txtai/pipeline/text/rag/) for more._"
]
},
{
"cell_type": "markdown",
"source": [
examples/58_Advanced_RAG_with_graph_path_traversal.ipynb
+21
View File
@@ -280,6 +280,27 @@
"print(rag(\"Write a book covering the end of Roman Britain, Saxons, Vikings and Norman conquest of England\", context))\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_It's important to note that txtai has multiple ways to run LLM inference. In the past, prior to \"Chat Templates\", it was expected that the submitted text had all the required chat tokens embedded. The same prompt above can also be written with chat messages. This is especially important when working with LLM APIs (i.e. OpenAI, Claude, Bedrock etc)._\n",
"\n",
"```python\n",
"llm([\n",
" {\"role\": \"system\": \"You are a friendly assistant. You answer questions from users.\"}\n",
" {\"role\": \"user\", \"content\": f\"\"\"\n",
" Answer the following question using only the context below. Only include information specifically discussed.\n",
"\n",
" question: {question}\n",
" context: {text} \n",
" \"\"\"}\n",
"])\n",
"```\n",
"\n",
"_See the [LLM pipeline documentation](https://neuml.github.io/txtai/pipeline/text/llm/) for more information._"
]
},
{
"cell_type": "markdown",
"metadata": {
examples/63_How_RAG_with_txtai_works.ipynb
+28
View File
@@ -186,6 +186,27 @@
"id": "CSNbxxIMN_26"
}
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_Before continuing, it's important to note that txtai has multiple ways to run LLM inference. In the past, prior to \"Chat Templates\", it was expected that the submitted text had all the required chat tokens embedded. The same prompt above can also be written with chat messages. This is especially important when working with LLM APIs (i.e. OpenAI, Claude, Bedrock etc)._\n",
"\n",
"```python\n",
"llm([\n",
" {\"role\": \"system\": \"You are a friendly assistant. You answer questions from users.\"}\n",
" {\"role\": \"user\", \"content\": f\"\"\"\n",
" Answer the following question using only the context below. Only include information specifically discussed.\n",
"\n",
" question: {question}\n",
" context: {text} \n",
" \"\"\"}\n",
"])\n",
"```\n",
"\n",
"_See the [LLM pipeline documentation](https://neuml.github.io/txtai/pipeline/text/llm/) for more information._"
]
},
{
"cell_type": "markdown",
"source": [
@@ -259,6 +280,13 @@
}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_As with the LLM pipeline, the RAG pipeline also supports chat messages. See the [RAG pipeline documentation](https://neuml.github.io/txtai/pipeline/text/rag/) for more._"
]
},
{
"cell_type": "markdown",
"source": [