Release 0.5.22 (#1164)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

apps/docs/CHANGELOG.md

@@ -1,5 +1,55 @@
 # docs
 
+## 0.0.63
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
+## 0.0.62
+
+### Patch Changes
+
+- Updated dependencies [ae1149f]
+- Updated dependencies [2411c9f]
+- Updated dependencies [e8f229c]
+- Updated dependencies [11b3856]
+- Updated dependencies [83d7f41]
+- Updated dependencies [0148354]
+- Updated dependencies [1711f6d]
+  - llamaindex@0.5.21
+
+## 0.0.61
+
+### Patch Changes
+
+- Updated dependencies [d9d6c56]
+- Updated dependencies [22ff486]
+- Updated dependencies [eed0b04]
+  - llamaindex@0.5.20
+
+## 0.0.60
+
+### Patch Changes
+
+- Updated dependencies [fcbf183]
+  - llamaindex@0.5.19
+
+## 0.0.59
+
+### Patch Changes
+
+- Updated dependencies [8b66cf4]
+  - llamaindex@0.5.18
+
+## 0.0.58
+
+### Patch Changes
+
+- Updated dependencies [c654398]
+  - llamaindex@0.5.17
+
 ## 0.0.57
 
 ### Patch Changes

apps/docs/docs/getting_started/environments.md

@@ -6,10 +6,17 @@ sidebar_position: 2
 
 We support Node.JS versions 18, 20 and 22, with experimental support for Deno, Bun and Vercel Edge functions.
 
-## NextJS App Router
+## NextJS
 
-If you're using NextJS App Router route handlers/serverless functions, you'll need to use the NodeJS mode:
+If you're using NextJS you'll need to add `withLlamaIndex` to your `next.config.js` file. This will add the necessary configuration for included 3rd-party libraries to your build:
 
 ```js
-export const runtime = "nodejs"; // default
+// next.config.js
+const withLlamaIndex = require("llamaindex/next");
+
+module.exports = withLlamaIndex({
+  // your next.js config
+});
 ```
+
+For details, check the latest [withLlamaIndex](https://github.com/run-llama/LlamaIndexTS/blob/main/packages/llamaindex/src/next.ts) implementation.

apps/docs/docs/introduction.md

@@ -21,7 +21,7 @@ LlamaIndex.TS handles several major use cases:
 
 - **Structured Data Extraction**: turning complex, unstructured and semi-structured data into uniform, programmatically accessible formats.
 - **Retrieval-Augmented Generation (RAG)**: answering queries across your internal data by providing LLMs with up-to-date, semantically relevant context including Question and Answer systems and chat bots.
-- **Autonomous Agents**: building software that is capable of intelligently selecting and using tools to accomplish tasks in an interative, unsupervised manner.
+- **Autonomous Agents**: building software that is capable of intelligently selecting and using tools to accomplish tasks in an interactive, unsupervised manner.
 
 ## 👨‍👩‍👧‍👦 Who is LlamaIndex for?
 

apps/docs/docs/modules/chat_engine.md

@@ -27,3 +27,4 @@ for await (const chunk of stream) {
 
 - [ContextChatEngine](../api/classes/ContextChatEngine.md)
 - [CondenseQuestionChatEngine](../api/classes/ContextChatEngine.md)
+- [SimpleChatEngine](../api/classes/SimpleChatEngine.md)

apps/docs/docs/modules/data_index.md

@@ -21,3 +21,4 @@ const index = await VectorStoreIndex.fromDocuments([document]);
 
 - [SummaryIndex](../api/classes/SummaryIndex.md)
 - [VectorStoreIndex](../api/classes/VectorStoreIndex.md)
+- [KeywordTableIndex](../api/classes/KeywordTableIndex.md)

apps/docs/docs/modules/data_loaders/json.md

@@ -2,6 +2,7 @@
 
 A simple JSON data loader with various options.
 Either parses the entire string, cleaning it and treat each line as an embedding or performs a recursive depth-first traversal yielding JSON paths.
+Supports streaming of large JSON data using [@discoveryjs/json-ext](https://github.com/discoveryjs/json-ext)
 
 ## Usage
 
@@ -20,12 +21,16 @@ const docsFromContent = reader.loadDataAsContent(content);
 
 Basic:
 
+- `streamingThreshold?`: The threshold for using streaming mode in MB of the JSON Data. CEstimates characters by calculating bytes: `(streamingThreshold * 1024 * 1024) / 2` and comparing against `.length` of the JSON string. Set `undefined` to disable streaming or `0` to always use streaming. Default is `50` MB.
+
 - `ensureAscii?`: Wether to ensure only ASCII characters be present in the output by converting non-ASCII characters to their unicode escape sequence. Default is `false`.
 
-- `isJsonLines?`: Wether the JSON is in JSON Lines format. If true, will split into lines, remove empty one and parse each line as JSON. Default is `false`
+- `isJsonLines?`: Wether the JSON is in JSON Lines format. If true, will split into lines, remove empty one and parse each line as JSON. Note: Uses a custom streaming parser, most likely less robust than json-ext. Default is `false`
 
 - `cleanJson?`: Whether to clean the JSON by filtering out structural characters (`{}, [], and ,`). If set to false, it will just parse the JSON, not removing structural characters. Default is `true`.
 
+- `logger?`: A placeholder for a custom logger function.
+
 Depth-First-Traversal:
 
 - `levelsBack?`: Specifies how many levels up the JSON structure to include in the output. `cleanJson` will be ignored. If set to 0, all levels are included. If undefined, parses the entire JSON, treat each line as an embedding and create a document per top-level array. Default is `undefined`

apps/docs/docs/modules/embeddings/available_embeddings/mixedbreadai.md

@@ -98,3 +98,7 @@ Use the `embedDocuments` method to generate embeddings for the texts.
 const result = await embeddings.embedDocuments(texts);
 console.log(result); // Perfectly customized embeddings, ready to serve.
 ```
+
+## API Reference
+
+- [MixedbreadAIEmbeddings](../../../api/classes/MixedbreadAIEmbeddings.md)

apps/docs/docs/modules/evaluation/index.md

@@ -2,7 +2,7 @@
 
 ## Concept
 
-Evaluation and benchmarking are crucial concepts in LLM development. To improve the perfomance of an LLM app (RAG, agents) you must have a way to measure it.
+Evaluation and benchmarking are crucial concepts in LLM development. To improve the performance of an LLM app (RAG, agents) you must have a way to measure it.
 
 LlamaIndex offers key modules to measure the quality of generated results. We also offer key modules to measure retrieval quality.
 

apps/docs/docs/modules/ingestion_pipeline/index.md

@@ -16,7 +16,7 @@ import {
   MetadataMode,
   OpenAIEmbedding,
   TitleExtractor,
-  SimpleNodeParser,
+  SentenceSplitter,
 } from "llamaindex";
 
 async function main() {
@@ -29,7 +29,7 @@ async function main() {
   const document = new Document({ text: essay, id_: path });
   const pipeline = new IngestionPipeline({
     transformations: [
-      new SimpleNodeParser({ chunkSize: 1024, chunkOverlap: 20 }),
+      new SentenceSplitter({ chunkSize: 1024, chunkOverlap: 20 }),
       new TitleExtractor(),
       new OpenAIEmbedding(),
     ],
@@ -62,7 +62,7 @@ import {
   MetadataMode,
   OpenAIEmbedding,
   TitleExtractor,
-  SimpleNodeParser,
+  SentenceSplitter,
   QdrantVectorStore,
   VectorStoreIndex,
 } from "llamaindex";
@@ -81,7 +81,7 @@ async function main() {
   const document = new Document({ text: essay, id_: path });
   const pipeline = new IngestionPipeline({
     transformations: [
-      new SimpleNodeParser({ chunkSize: 1024, chunkOverlap: 20 }),
+      new SentenceSplitter({ chunkSize: 1024, chunkOverlap: 20 }),
       new TitleExtractor(),
       new OpenAIEmbedding(),
     ],

apps/docs/docs/modules/ingestion_pipeline/transformations.md

@@ -4,7 +4,7 @@ A transformation is something that takes a list of nodes as an input, and return
 
 Currently, the following components are Transformation objects:
 
-- [SimpleNodeParser](../../api/classes/SimpleNodeParser.md)
+- [SentenceSplitter](../../api/classes/SentenceSplitter.md)
 - [MetadataExtractor](../documents_and_nodes/metadata_extraction.md)
 - [Embeddings](../embeddings/index.md)
 
@@ -13,10 +13,10 @@ Currently, the following components are Transformation objects:
 While transformations are best used with with an IngestionPipeline, they can also be used directly.
 
 ```ts
-import { SimpleNodeParser, TitleExtractor, Document } from "llamaindex";
+import { SentenceSplitter, TitleExtractor, Document } from "llamaindex";
 
 async function main() {
-  let nodes = new SimpleNodeParser().getNodesFromDocuments([
+  let nodes = new SentenceSplitter().getNodesFromDocuments([
     new Document({ text: "I am 10 years old. John is 20 years old." }),
   ]);
 
@@ -34,15 +34,15 @@ main().catch(console.error);
 
 ## Custom Transformations
 
-You can implement any transformation yourself by implementing the `TransformerComponent`.
+You can implement any transformation yourself by implementing the `TransformComponent`.
 
-The following custom transformation will remove any special characters or punctutaion in text.
+The following custom transformation will remove any special characters or punctuation in text.
 
 ```ts
-import { TransformerComponent, Node } from "llamaindex";
+import { TransformComponent, TextNode } from "llamaindex";
 
-class RemoveSpecialCharacters extends TransformerComponent {
-  async transform(nodes: Node[]): Promise<Node[]> {
+export class RemoveSpecialCharacters extends TransformComponent {
+  async transform(nodes: TextNode[]): Promise<TextNode[]> {
     for (const node of nodes) {
       node.text = node.text.replace(/[^\w\s]/gi, "");
     }
@@ -75,3 +75,7 @@ async function main() {
 
 main().catch(console.error);
 ```
+
+## API Reference
+
+- [TransformComponent](../../api/classes/TransformComponent.md)

apps/docs/docs/modules/llms/available_llms/deepseek.md

@@ -1,5 +1,7 @@
 # DeepSeek LLM
 
+[DeepSeek Platform](https://platform.deepseek.com/)
+
 ## Usage
 
 ```ts
@@ -45,6 +47,6 @@ Currently does not support function calling.
 
 [Currently does not support json-output param while still is very good at json generating.](https://platform.deepseek.com/api-docs/faq#does-your-api-support-json-output)
 
-## API platform
+## API Reference
 
-- [DeepSeek platform](https://platform.deepseek.com/)
+- [DeepSeekLLM](../../../api/classes/DeepSeekLLM.md)

apps/docs/docs/modules/node_parser.md

@@ -7,9 +7,9 @@ sidebar_position: 4
 The `NodeParser` in LlamaIndex is responsible for splitting `Document` objects into more manageable `Node` objects. When you call `.fromDocuments()`, the `NodeParser` from the `Settings` is used to do this automatically for you. Alternatively, you can use it to split documents ahead of time.
 
 ```typescript
-import { Document, SimpleNodeParser } from "llamaindex";
+import { Document, SentenceSplitter } from "llamaindex";
 
-const nodeParser = new SimpleNodeParser();
+const nodeParser = new SentenceSplitter();
 
 Settings.nodeParser = nodeParser;
 ```
@@ -93,6 +93,5 @@ The output metadata will be something like:
 
 ## API Reference
 
-- [SimpleNodeParser](../api/classes/SimpleNodeParser.md)
 - [SentenceSplitter](../api/classes/SentenceSplitter.md)
 - [MarkdownNodeParser](../api/classes/MarkdownNodeParser.md)

apps/docs/docs/modules/node_postprocessors/mixedbreadiai_reranker.md

@@ -163,3 +163,7 @@ Use the `rerank` method to reorder the documents based on the query.
 const result = await reranker.rerank(documents, query);
 console.log(result); // Perfectly customized results, ready to serve.
 ```
+
+## API Reference
+
+- [MixedbreadAIReranker](../../api/classes/MixedbreadAIReranker.md)

apps/docs/docs/modules/query_engines/index.md

@@ -1,6 +1,6 @@
 # QueryEngine
 
-A query engine wraps a `Retriever` and a `ResponseSynthesizer` into a pipeline, that will use the query string to fetech nodes and then send them to the LLM to generate a response.
+A query engine wraps a `Retriever` and a `ResponseSynthesizer` into a pipeline, that will use the query string to fetch nodes and then send them to the LLM to generate a response.
 
 ```typescript
 const queryEngine = index.asQueryEngine();

apps/docs/docs/modules/query_engines/router_query_engine.md

@@ -15,7 +15,7 @@ import {
   OpenAI,
   RouterQueryEngine,
   SimpleDirectoryReader,
-  SimpleNodeParser,
+  SentenceSplitter,
   SummaryIndex,
   VectorStoreIndex,
   Settings,
@@ -34,11 +34,11 @@ const documents = await new SimpleDirectoryReader().loadData({
 
 ## Service Context
 
-Next, we need to define some basic rules and parse the documents into nodes. We will use the `SimpleNodeParser` to parse the documents into nodes and `Settings` to define the rules (eg. LLM API key, chunk size, etc.):
+Next, we need to define some basic rules and parse the documents into nodes. We will use the `SentenceSplitter` to parse the documents into nodes and `Settings` to define the rules (eg. LLM API key, chunk size, etc.):
 
 ```ts
 Settings.llm = new OpenAI();
-Settings.nodeParser = new SimpleNodeParser({
+Settings.nodeParser = new SentenceSplitter({
   chunkSize: 1024,
 });
 ```
@@ -104,14 +104,14 @@ import {
   OpenAI,
   RouterQueryEngine,
   SimpleDirectoryReader,
-  SimpleNodeParser,
+  SentenceSplitter,
   SummaryIndex,
   VectorStoreIndex,
   Settings,
 } from "llamaindex";
 
 Settings.llm = new OpenAI();
-Settings.nodeParser = new SimpleNodeParser({
+Settings.nodeParser = new SentenceSplitter({
   chunkSize: 1024,
 });
 

apps/docs/docs/modules/retriever.md

@@ -4,7 +4,14 @@ sidebar_position: 5
 
 # Retriever
 
-A retriever in LlamaIndex is what is used to fetch `Node`s from an index using a query string. Aa `VectorIndexRetriever` will fetch the top-k most similar nodes. Meanwhile, a `SummaryIndexRetriever` will fetch all nodes no matter the query.
+A retriever in LlamaIndex is what is used to fetch `Node`s from an index using a query string.
+
+- [VectorIndexRetriever](../api/classes/VectorIndexRetriever.md) will fetch the top-k most similar nodes. Ideal for dense retrieval to find most relevant nodes.
+- [SummaryIndexRetriever](../api/classes/SummaryIndexRetriever.md) will fetch all nodes no matter the query. Ideal when complete context is necessary, e.g. analyzing large datasets.
+- [SummaryIndexLLMRetriever](../api/classes/SummaryIndexLLMRetriever.md) utilizes an LLM to score and filter nodes based on relevancy to the query.
+- [KeywordTableLLMRetriever](../api/classes/KeywordTableLLMRetriever.md) uses an LLM to extract keywords from the query and retrieve relevant nodes based on keyword matches.
+- [KeywordTableSimpleRetriever](../api/classes/KeywordTableSimpleRetriever.md) uses a basic frequency-based approach to extract keywords and retrieve nodes.
+- [KeywordTableRAKERetriever](../api/classes/KeywordTableRAKERetriever.md) uses the RAKE (Rapid Automatic Keyword Extraction) algorithm to extract keywords from the query, focusing on co-occurrence and context for keyword-based retrieval.
 
 ```typescript
 const retriever = vectorIndex.asRetriever({
@@ -14,9 +21,3 @@ const retriever = vectorIndex.asRetriever({
 // Fetch nodes!
 const nodesWithScore = await retriever.retrieve({ query: "query string" });
 ```
-
-## API Reference
-
-- [SummaryIndexRetriever](../api/classes/SummaryIndexRetriever.md)
-- [SummaryIndexLLMRetriever](../api/classes/SummaryIndexLLMRetriever.md)
-- [VectorIndexRetriever](../api/classes/VectorIndexRetriever.md)

apps/docs/package.json

@@ -1,6 +1,6 @@
 {
   "name": "docs",
-  "version": "0.0.57",
+  "version": "0.0.63",
   "private": true,
   "scripts": {
     "docusaurus": "docusaurus",
@@ -15,29 +15,29 @@
     "typecheck": "tsc"
   },
   "dependencies": {
-    "@docusaurus/core": "3.4.0",
-    "@docusaurus/remark-plugin-npm2yarn": "3.4.0",
+    "@docusaurus/core": "3.5.2",
+    "@docusaurus/remark-plugin-npm2yarn": "3.5.2",
     "@llamaindex/examples": "workspace:*",
     "@mdx-js/react": "3.0.1",
     "clsx": "2.1.1",
     "llamaindex": "workspace:*",
-    "postcss": "8.4.39",
-    "prism-react-renderer": "2.3.1",
+    "postcss": "8.4.41",
+    "prism-react-renderer": "2.4.0",
     "raw-loader": "4.0.2",
     "react": "18.3.1",
     "react-dom": "18.3.1"
   },
   "devDependencies": {
-    "@docusaurus/module-type-aliases": "3.4.0",
-    "@docusaurus/preset-classic": "3.4.0",
-    "@docusaurus/theme-classic": "3.4.0",
-    "@docusaurus/types": "3.4.0",
+    "@docusaurus/module-type-aliases": "3.5.2",
+    "@docusaurus/preset-classic": "3.5.2",
+    "@docusaurus/theme-classic": "3.5.2",
+    "@docusaurus/types": "3.5.2",
     "@tsconfig/docusaurus": "2.0.3",
-    "@types/node": "^20.12.11",
-    "docusaurus-plugin-typedoc": "1.0.3",
-    "typedoc": "0.26.4",
-    "typedoc-plugin-markdown": "4.1.2",
-    "typescript": "^5.5.3"
+    "@types/node": "^22.5.1",
+    "docusaurus-plugin-typedoc": "1.0.5",
+    "typedoc": "0.26.6",
+    "typedoc-plugin-markdown": "4.2.6",
+    "typescript": "^5.5.4"
   },
   "browserslist": {
     "production": [

examples/agent/multi_document_agent.ts

@@ -6,8 +6,8 @@ import {
   OpenAI,
   OpenAIAgent,
   QueryEngineTool,
+  SentenceSplitter,
   Settings,
-  SimpleNodeParser,
   SimpleToolNodeMapping,
   SummaryIndex,
   VectorStoreIndex,
@@ -43,7 +43,7 @@ async function main() {
   for (const title of wikiTitles) {
     console.log(`Processing ${title}`);
 
-    const nodes = new SimpleNodeParser({
+    const nodes = new SentenceSplitter({
       chunkSize: 200,
       chunkOverlap: 20,
     }).getNodesFromDocuments([countryDocs[title]]);

examples/agent/openai-task.ts

@@ -1,5 +1,4 @@
 import { ChatResponseChunk, OpenAIAgent } from "llamaindex";
-import { ReadableStream } from "node:stream/web";
 import {
   getCurrentIDTool,
   getUserInfoTool,

examples/agent/react-task.ts

@@ -1,5 +1,4 @@
 import { ChatResponseChunk, ReActAgent } from "llamaindex";
-import { ReadableStream } from "node:stream/web";
 import {
   getCurrentIDTool,
   getUserInfoTool,

examples/extractors/keywordExtractor.ts

@@ -2,13 +2,13 @@ import {
   Document,
   KeywordExtractor,
   OpenAI,
-  SimpleNodeParser,
+  SentenceSplitter,
 } from "llamaindex";
 
 (async () => {
   const openaiLLM = new OpenAI({ model: "gpt-3.5-turbo", temperature: 0 });
 
-  const nodeParser = new SimpleNodeParser();
+  const nodeParser = new SentenceSplitter();
 
   const nodes = nodeParser.getNodesFromDocuments([
     new Document({ text: "banana apple orange pear peach watermelon" }),

examples/extractors/questionsAnsweredExtractor.ts

@@ -2,13 +2,13 @@ import {
   Document,
   OpenAI,
   QuestionsAnsweredExtractor,
-  SimpleNodeParser,
+  SentenceSplitter,
 } from "llamaindex";
 
 (async () => {
   const openaiLLM = new OpenAI({ model: "gpt-3.5-turbo", temperature: 0 });
 
-  const nodeParser = new SimpleNodeParser();
+  const nodeParser = new SentenceSplitter();
 
   const nodes = nodeParser.getNodesFromDocuments([
     new Document({

examples/extractors/summaryExtractor.ts

@@ -1,14 +1,14 @@
 import {
   Document,
   OpenAI,
-  SimpleNodeParser,
+  SentenceSplitter,
   SummaryExtractor,
 } from "llamaindex";
 
 (async () => {
   const openaiLLM = new OpenAI({ model: "gpt-3.5-turbo", temperature: 0 });
 
-  const nodeParser = new SimpleNodeParser();
+  const nodeParser = new SentenceSplitter();
 
   const nodes = nodeParser.getNodesFromDocuments([
     new Document({

examples/extractors/titleExtractor.ts

@@ -1,11 +1,11 @@
-import { Document, OpenAI, SimpleNodeParser, TitleExtractor } from "llamaindex";
+import { Document, OpenAI, SentenceSplitter, TitleExtractor } from "llamaindex";
 
 import essay from "../essay";
 
 (async () => {
   const openaiLLM = new OpenAI({ model: "gpt-3.5-turbo-0125", temperature: 0 });
 
-  const nodeParser = new SimpleNodeParser({});
+  const nodeParser = new SentenceSplitter({});
 
   const nodes = nodeParser.getNodesFromDocuments([
     new Document({

examples/jupyter/nodeparser.ipynb

@@ -7,10 +7,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import {\n",
-    "  Document,\n",
-    "  SimpleNodeParser\n",
-    "} from \"npm:llamaindex\";"
+    "import { Document, SentenceSplitter } from \"npm:llamaindex\";"
    ]
   },
   {
@@ -45,7 +42,7 @@
     }
    ],
    "source": [
-    "const nodeParser = new SimpleNodeParser();\n",
+    "const nodeParser = new SentenceSplitter();\n",
     "const nodes = nodeParser.getNodesFromDocuments([\n",
     "    new Document({ text: \"I am 10 years old. John is 20 years old.\" }),\n",
     "]);\n",

examples/lowlevel.ts

@@ -2,12 +2,12 @@ import {
   Document,
   NodeWithScore,
   ResponseSynthesizer,
-  SimpleNodeParser,
+  SentenceSplitter,
   TextNode,
 } from "llamaindex";
 
 (async () => {
-  const nodeParser = new SimpleNodeParser();
+  const nodeParser = new SentenceSplitter();
   const nodes = nodeParser.getNodesFromDocuments([
     new Document({ text: "I am 10 years old. John is 20 years old." }),
   ]);

examples/mongodb/2_load_and_index.ts

@@ -28,12 +28,23 @@ async function loadAndIndex() {
     "full_text",
   ]);
 
+  const FILTER_METADATA_FIELD = "content_type";
+
+  documents.forEach((document, index) => {
+    const contentType = ["tweet", "post", "story"][index % 3]; // assign a random content type to each document
+    document.metadata = {
+      ...document.metadata,
+      [FILTER_METADATA_FIELD]: contentType,
+    };
+  });
+
   // create Atlas as a vector store
   const vectorStore = new MongoDBAtlasVectorSearch({
     mongodbClient: client,
     dbName: databaseName,
     collectionName: vectorCollectionName, // this is where your embeddings will be stored
     indexName: indexName, // this is the name of the index you will need to create
+    indexedMetadataFields: [FILTER_METADATA_FIELD], // this is the field that will be used for the query
   });
 
   // now create an index from all the Documents and store them in Atlas
@@ -45,39 +56,4 @@ async function loadAndIndex() {
   await client.close();
 }
 
-/**
- * This method is document in https://www.mongodb.com/docs/atlas/atlas-search/create-index/#create-an-fts-index-programmatically
- * But, while testing a 'CommandNotFound' error occurred, so we're not using this here.
- */
-async function createSearchIndex() {
-  const client = new MongoClient(mongoUri);
-  const database = client.db(databaseName);
-  const collection = database.collection(vectorCollectionName);
-
-  // define your Atlas Search index
-  const index = {
-    name: indexName,
-    definition: {
-      /* search index definition fields */
-      mappings: {
-        dynamic: true,
-        fields: [
-          {
-            type: "vector",
-            path: "embedding",
-            numDimensions: 1536,
-            similarity: "cosine",
-          },
-        ],
-      },
-    },
-  };
-  // run the helper method
-  const result = await collection.createSearchIndex(index);
-  console.log("Successfully created search index:", result);
-  await client.close();
-}
-
 loadAndIndex().catch(console.error);
-
-// you can't query your index yet because you need to create a vector search index in mongodb's UI now

examples/mongodb/3_query.ts

@@ -14,14 +14,26 @@ async function query() {
     dbName: process.env.MONGODB_DATABASE!,
     collectionName: process.env.MONGODB_VECTORS!,
     indexName: process.env.MONGODB_VECTOR_INDEX!,
+    indexedMetadataFields: ["content_type"],
   });
 
   const index = await VectorStoreIndex.fromVectorStore(store);
 
   const retriever = index.asRetriever({ similarityTopK: 20 });
-  const queryEngine = index.asQueryEngine({ retriever });
+  const queryEngine = index.asQueryEngine({
+    retriever,
+    preFilters: {
+      filters: [
+        {
+          key: "content_type",
+          value: "story", // try "tweet" or "post" to see the difference
+          operator: "==",
+        },
+      ],
+    },
+  });
   const result = await queryEngine.query({
-    query: "What does the author think of web frameworks?",
+    query: "What does author receive when he was 11 years old?", // Isaac Asimov's "Foundation" for Christmas
   });
   console.log(result.response);
   await client.close();

examples/mongodb/README.md

@@ -68,45 +68,6 @@ What you're doing here is creating a Reader which loads the data out of Mongo in
 
 Now you're creating a vector search client for Mongo. In addition to a MongoDB client object, you again tell it what database everything is in. This time you give it the name of the collection where you'll store the vector embeddings, and the name of the vector search index you'll create in the next step.
 
-### Create a vector search index
-
-Now if all has gone well you should be able to log in to the Mongo Atlas UI and see two collections in your database: the original data in `tiny_tweets_collection`, and the vector embeddings in `tiny_tweets_vectors`.
-
-![MongoDB Atlas collections](./docs/3_vectors_in_db.png)
-
-Now it's time to create the vector search index so that you can query the data.
-It's not yet possible to programmatically create a vector search index using the [`createIndex`](https://www.mongodb.com/docs/manual/reference/method/db.collection.createIndex/) function, therefore we have to create one manually in the UI.
-To do so, first, click the 'Atlas Search' tab, and then click "Create Search Index":
-
-![MongoDB Atlas create search index](./docs/4_search_tab.png)
-
-We have to use the JSON editor, as the Visual Editor does not yet support to create a vector search index:
-
-![MongoDB Atlas JSON editor](./docs/5_json_editor.png)
-
-Now under "database and collection" select `tiny_tweets_db` and within that select `tiny_tweets_vectors`. Then under "Index name" enter `tiny_tweets_vector_index` (or whatever value you put for MONGODB_VECTOR_INDEX in `.env`). Under that, you'll want to enter this JSON object:
-
-```json
-{
-  "fields": [
-    {
-      "type": "vector",
-      "path": "embedding",
-      "numDimensions": 1536,
-      "similarity": "cosine"
-    }
-  ]
-}
-```
-
-This tells Mongo that the `embedding` field in each document (in the `tiny_tweets_vectors` collection) is a vector of 1536 dimensions (this is the size of embeddings used by OpenAI), and that we want to use cosine similarity to compare vectors. You don't need to worry too much about these values unless you want to use a different LLM to OpenAI entirely.
-
-The UI will ask you to review and confirm your choices, then you need to wait a minute or two while it generates the index. If all goes well, you should see something like this screen:
-
-![MongoDB Atlas index created](./docs/7_index_created.png)
-
-Now you're ready to query your data!
-
 ### Run a test query
 
 You can do this by running

examples/multimodal/chat.ts

@@ -0,0 +1,40 @@
+// call pnpm tsx multimodal/load.ts first to init the storage
+import { OpenAI, Settings, SimpleChatEngine, imageToDataUrl } from "llamaindex";
+import fs from "node:fs/promises";
+
+import path from "path";
+// Update llm
+Settings.llm = new OpenAI({ model: "gpt-4o-mini", maxTokens: 512 });
+
+async function main() {
+  const chatEngine = new SimpleChatEngine();
+
+  // Load the image and convert it to a data URL
+  const imagePath = path.join(__dirname, ".", "data", "60.jpg");
+
+  // 1. you can read the buffer from the file
+  const imageBuffer = await fs.readFile(imagePath);
+  const dataUrl = await imageToDataUrl(imageBuffer);
+  // or 2. you can just pass the file path to the imageToDataUrl function
+  // const dataUrl = await imageToDataUrl(imagePath);
+
+  // Update the image_url in the chat message
+  const response = await chatEngine.chat({
+    message: [
+      {
+        type: "text",
+        text: "What is in this image?",
+      },
+      {
+        type: "image_url",
+        image_url: {
+          url: dataUrl,
+        },
+      },
+    ],
+  });
+
+  console.log(response.message.content);
+}
+
+main().catch(console.error);

examples/package.json

@@ -4,24 +4,24 @@
   "version": "0.0.7",
   "dependencies": {
     "@aws-crypto/sha256-js": "^5.2.0",
-    "@azure/identity": "^4.2.1",
-    "@datastax/astra-db-ts": "^1.2.1",
+    "@azure/identity": "^4.4.1",
+    "@datastax/astra-db-ts": "^1.4.1",
     "@llamaindex/core": "^0.1.0",
     "@notionhq/client": "^2.2.15",
-    "@pinecone-database/pinecone": "^2.2.2",
-    "@zilliz/milvus2-sdk-node": "^2.4.4",
+    "@pinecone-database/pinecone": "^3.0.2",
+    "@zilliz/milvus2-sdk-node": "^2.4.6",
     "chromadb": "^1.8.1",
     "commander": "^12.1.0",
     "dotenv": "^16.4.5",
-    "js-tiktoken": "^1.0.12",
+    "js-tiktoken": "^1.0.14",
     "llamaindex": "^0.5.0",
     "mongodb": "^6.7.0",
     "pathe": "^1.1.2"
   },
   "devDependencies": {
-    "@types/node": "^20.14.1",
-    "tsx": "^4.15.6",
-    "typescript": "^5.5.3"
+    "@types/node": "^22.5.1",
+    "tsx": "^4.19.0",
+    "typescript": "^5.5.4"
   },
   "scripts": {
     "lint": "eslint ."

examples/pipeline/ingestion.ts

@@ -5,7 +5,7 @@ import {
   IngestionPipeline,
   MetadataMode,
   OpenAIEmbedding,
-  SimpleNodeParser,
+  SentenceSplitter,
 } from "llamaindex";
 
 async function main() {
@@ -18,7 +18,7 @@ async function main() {
   const document = new Document({ text: essay, id_: path });
   const pipeline = new IngestionPipeline({
     transformations: [
-      new SimpleNodeParser({ chunkSize: 1024, chunkOverlap: 20 }),
+      new SentenceSplitter({ chunkSize: 1024, chunkOverlap: 20 }),
       new OpenAIEmbedding(),
     ],
   });

examples/prompts/promptMixin.ts

@@ -1,21 +1,22 @@
 import {
   Document,
+  PromptTemplate,
   ResponseSynthesizer,
   TreeSummarize,
   TreeSummarizePrompt,
   VectorStoreIndex,
 } from "llamaindex";
 
-const treeSummarizePrompt: TreeSummarizePrompt = ({ context, query }) => {
-  return `Context information from multiple sources is below.
+const treeSummarizePrompt: TreeSummarizePrompt = new PromptTemplate({
+  template: `Context information from multiple sources is below.
 ---------------------
-${context}
+{context}
 ---------------------
 Given the information from multiple sources and not prior knowledge.
 Answer the query in the style of a Shakespeare play"
-Query: ${query}
-Answer:`;
-};
+Query: {query}
+Answer:`,
+});
 
 async function main() {
   const documents = new Document({

examples/readers/package.json

@@ -14,14 +14,15 @@
     "start:assemblyai": "node --import tsx ./src/assemblyai.ts",
     "start:llamaparse-dir": "node --import tsx ./src/simple-directory-reader-with-llamaparse.ts",
     "start:llamaparse-json": "node --import tsx ./src/llamaparse-json.ts",
-    "start:discord": "node --import tsx ./src/discord.ts"
+    "start:discord": "node --import tsx ./src/discord.ts",
+    "start:json": "node --import tsx ./src/json.ts"
   },
   "dependencies": {
     "llamaindex": "*"
   },
   "devDependencies": {
-    "@types/node": "^20.12.11",
-    "tsx": "^4.15.6",
-    "typescript": "^5.5.3"
+    "@types/node": "^22.5.1",
+    "tsx": "^4.19.0",
+    "typescript": "^5.5.4"
   }
 }

examples/readers/src/csv.ts

@@ -1,6 +1,7 @@
 import {
   CompactAndRefine,
   OpenAI,
+  PromptTemplate,
   ResponseSynthesizer,
   Settings,
   VectorStoreIndex,
@@ -18,14 +19,15 @@ async function main() {
   // Split text and create embeddings. Store them in a VectorStoreIndex
   const index = await VectorStoreIndex.fromDocuments(documents);
 
-  const csvPrompt = ({ context = "", query = "" }) => {
-    return `The following CSV file is loaded from ${path}
+  const csvPrompt = new PromptTemplate({
+    templateVars: ["query", "context"],
+    template: `The following CSV file is loaded from ${path}
 \`\`\`csv
-${context}
+{context}
 \`\`\`
-Given the CSV file, generate me Typescript code to answer the question: ${query}. You can use built in NodeJS functions but avoid using third party libraries.
-`;
-  };
+Given the CSV file, generate me Typescript code to answer the question: {query}. You can use built in NodeJS functions but avoid using third party libraries.
+`,
+  });
 
   const responseSynthesizer = new ResponseSynthesizer({
     responseBuilder: new CompactAndRefine(undefined, csvPrompt),

examples/readers/src/llamaparse-json.ts

@@ -3,6 +3,7 @@ import {
   ImageNode,
   LlamaParseReader,
   OpenAI,
+  PromptTemplate,
   VectorStoreIndex,
 } from "llamaindex";
 import { createMessageContent } from "llamaindex/synthesizers/utils";
@@ -50,7 +51,9 @@ async function getImageTextDocs(
 
   for (const imageDict of imageDicts) {
     const imageDoc = new ImageNode({ image: imageDict.path });
-    const prompt = () => `Describe the image as alt text`;
+    const prompt = new PromptTemplate({
+      template: `Describe the image as alt text`,
+    });
     const message = await createMessageContent(prompt, [imageDoc]);
 
     const response = await llm.complete({

examples/routerQueryEngine.ts

@@ -1,9 +1,9 @@
 import {
   OpenAI,
   RouterQueryEngine,
+  SentenceSplitter,
   Settings,
   SimpleDirectoryReader,
-  SimpleNodeParser,
   SummaryIndex,
   VectorStoreIndex,
 } from "llamaindex";
@@ -12,7 +12,7 @@ import {
 Settings.llm = new OpenAI();
 
 // Update node parser
-Settings.nodeParser = new SimpleNodeParser({
+Settings.nodeParser = new SentenceSplitter({
   chunkSize: 1024,
 });
 

examples/summaryIndex.ts

@@ -1,7 +1,7 @@
 import {
   Document,
+  SentenceSplitter,
   Settings,
-  SimpleNodeParser,
   SummaryIndex,
   SummaryRetrieverMode,
 } from "llamaindex";
@@ -9,7 +9,7 @@ import {
 import essay from "./essay";
 
 // Update node parser
-Settings.nodeParser = new SimpleNodeParser({
+Settings.nodeParser = new SentenceSplitter({
   chunkSize: 40,
 });
 

examples/tsconfig.json

@@ -7,8 +7,6 @@
     "forceConsistentCasingInFileNames": true,
     "strict": true,
     "skipLibCheck": true,
-    "lib": ["ES2022"],
-    "types": ["node"],
     "outDir": "./lib",
     "tsBuildInfoFile": "./lib/.tsbuildinfo",
     "incremental": true,

examples/weaviate/README.md

@@ -0,0 +1,31 @@
+# Weaviate Vector Store
+
+Here are two sample scripts which work with loading and querying data from a Weaviate Vector Store.
+
+## Prerequisites
+
+- An Weaviate Vector Database
+  - Hosted https://weaviate.io/
+  - Self Hosted https://weaviate.io/developers/weaviate/installation/docker-compose#starter-docker-compose-file
+- An OpenAI API Key
+
+## Setup
+
+1. Set your env variables:
+
+- `WEAVIATE_CLUSTER_URL`: Address of your Weaviate Vector Store (like localhost:8080)
+- `WEAVIATE_API_KEY`: Your Weaviate API key
+- `OPENAI_API_KEY`: Your OpenAI key
+
+2. `cd` Into the `examples` directory
+3. run `npm i`
+
+## Load the data
+
+This sample loads the same dataset of movie reviews as sample dataset
+
+run `npx tsx weaviate/load`
+
+## Use RAG to Query the data
+
+run `npx tsx weaviate/query`

examples/weaviate/load.ts

@@ -0,0 +1,23 @@
+import {
+  PapaCSVReader,
+  storageContextFromDefaults,
+  VectorStoreIndex,
+  WeaviateVectorStore,
+} from "llamaindex";
+
+const indexName = "MovieReviews";
+
+async function main() {
+  try {
+    const reader = new PapaCSVReader(false);
+    const docs = await reader.loadData("./data/movie_reviews.csv");
+    const vectorStore = new WeaviateVectorStore({ indexName });
+    const storageContext = await storageContextFromDefaults({ vectorStore });
+    await VectorStoreIndex.fromDocuments(docs, { storageContext });
+    console.log("Successfully loaded data into Weaviate");
+  } catch (e) {
+    console.error(e);
+  }
+}
+
+void main();

examples/weaviate/query.ts

@@ -0,0 +1,46 @@
+import { VectorStoreIndex, WeaviateVectorStore } from "llamaindex";
+
+const indexName = "MovieReviews";
+
+async function main() {
+  try {
+    const query = "Get all movie titles.";
+    const vectorStore = new WeaviateVectorStore({ indexName });
+    const index = await VectorStoreIndex.fromVectorStore(vectorStore);
+    const retriever = index.asRetriever({ similarityTopK: 20 });
+
+    const queryEngine = index.asQueryEngine({ retriever });
+    const results = await queryEngine.query({ query });
+    console.log(`Query from ${results.sourceNodes?.length} nodes`);
+    console.log(results.response);
+
+    console.log("\n=====\nQuerying the index with filters");
+    const queryEngineWithFilters = index.asQueryEngine({
+      retriever,
+      preFilters: {
+        filters: [
+          {
+            key: "document_id",
+            value: "./data/movie_reviews.csv_37",
+            operator: "==",
+          },
+          {
+            key: "document_id",
+            value: "./data/movie_reviews.csv_21",
+            operator: "==",
+          },
+        ],
+        condition: "or",
+      },
+    });
+    const resultAfterFilter = await queryEngineWithFilters.query({
+      query: "Get all movie titles.",
+    });
+    console.log(`Query from ${resultAfterFilter.sourceNodes?.length} nodes`);
+    console.log(resultAfterFilter.response);
+  } catch (e) {
+    console.error(e);
+  }
+}
+
+void main();

package.json

@@ -19,25 +19,24 @@
   },
   "devDependencies": {
     "@changesets/cli": "^2.27.5",
-    "@typescript-eslint/eslint-plugin": "^7.13.1",
-    "eslint": "^8.57.0",
-    "eslint-config-next": "^14.2.5",
+    "@typescript-eslint/eslint-plugin": "^8.3.0",
+    "eslint": "8.57.0",
+    "eslint-config-next": "^14.2.7",
     "eslint-config-prettier": "^9.1.0",
-    "eslint-config-turbo": "^2.0.5",
-    "eslint-plugin-react": "7.34.3",
-    "husky": "^9.0.11",
-    "lint-staged": "^15.2.7",
-    "madge": "^7.0.0",
-    "prettier": "^3.3.2",
+    "eslint-config-turbo": "^2.1.0",
+    "eslint-plugin-react": "7.35.0",
+    "husky": "^9.1.5",
+    "lint-staged": "^15.2.9",
+    "madge": "^8.0.0",
+    "prettier": "^3.3.3",
     "prettier-plugin-organize-imports": "^4.0.0",
-    "turbo": "^2.0.5",
-    "typescript": "^5.5.3"
+    "turbo": "^2.1.0",
+    "typescript": "^5.5.4"
   },
   "packageManager": "pnpm@9.5.0",
   "pnpm": {
     "overrides": {
       "trim": "1.0.1",
-      "@babel/traverse": "7.23.2",
       "protobufjs": "7.2.6"
     }
   },

packages/autotool/examples/01_node/CHANGELOG.md

@@ -1,5 +1,27 @@
 # @llamaindex/autotool-01-node-example
 
+## 0.0.3
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+  - @llamaindex/autotool@2.0.1
+
+## 0.0.2
+
+### Patch Changes
+
+- Updated dependencies [ae1149f]
+- Updated dependencies [2411c9f]
+- Updated dependencies [e8f229c]
+- Updated dependencies [11b3856]
+- Updated dependencies [83d7f41]
+- Updated dependencies [0148354]
+- Updated dependencies [1711f6d]
+  - llamaindex@0.5.21
+  - @llamaindex/autotool@2.0.1
+
 ## null
 
 ### Patch Changes

packages/autotool/examples/01_node/package.json

@@ -5,13 +5,13 @@
   "dependencies": {
     "@llamaindex/autotool": "workspace:*",
     "llamaindex": "workspace:*",
-    "openai": "^4.52.5"
+    "openai": "^4.57.0"
   },
   "devDependencies": {
-    "tsx": "^4.15.6"
+    "tsx": "^4.19.0"
   },
   "scripts": {
     "start": "node --import tsx --import @llamaindex/autotool/node ./src/index.ts"
   },
-  "version": null
+  "version": "0.0.3"
 }

packages/autotool/examples/02_nextjs/CHANGELOG.md

@@ -1,5 +1,61 @@
 # @llamaindex/autotool-02-next-example
 
+## 0.1.47
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+  - @llamaindex/autotool@2.0.1
+
+## 0.1.46
+
+### Patch Changes
+
+- Updated dependencies [ae1149f]
+- Updated dependencies [2411c9f]
+- Updated dependencies [e8f229c]
+- Updated dependencies [11b3856]
+- Updated dependencies [83d7f41]
+- Updated dependencies [0148354]
+- Updated dependencies [1711f6d]
+  - llamaindex@0.5.21
+  - @llamaindex/autotool@2.0.1
+
+## 0.1.45
+
+### Patch Changes
+
+- Updated dependencies [d9d6c56]
+- Updated dependencies [22ff486]
+- Updated dependencies [eed0b04]
+  - llamaindex@0.5.20
+  - @llamaindex/autotool@2.0.1
+
+## 0.1.44
+
+### Patch Changes
+
+- Updated dependencies [fcbf183]
+  - llamaindex@0.5.19
+  - @llamaindex/autotool@2.0.1
+
+## 0.1.43
+
+### Patch Changes
+
+- Updated dependencies [8b66cf4]
+  - llamaindex@0.5.18
+  - @llamaindex/autotool@2.0.1
+
+## 0.1.42
+
+### Patch Changes
+
+- Updated dependencies [c654398]
+  - llamaindex@0.5.17
+  - @llamaindex/autotool@2.0.1
+
 ## 0.1.41
 
 ### Patch Changes

packages/autotool/examples/02_nextjs/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/autotool-02-next-example",
   "private": true,
-  "version": "0.1.41",
+  "version": "0.1.47",
   "scripts": {
     "dev": "next dev",
     "build": "next build",
@@ -10,28 +10,28 @@
   "dependencies": {
     "@llamaindex/autotool": "workspace:*",
     "@radix-ui/react-slot": "^1.1.0",
-    "ai": "^3.2.1",
+    "ai": "^3.3.21",
     "class-variance-authority": "^0.7.0",
     "dotenv": "^16.3.1",
     "llamaindex": "workspace:*",
-    "lucide-react": "^0.407.0",
+    "lucide-react": "^0.436.0",
     "next": "14.3.0-canary.51",
     "react": "^18.3.1",
     "react-dom": "^18.3.1",
     "react-markdown": "^9.0.1",
     "react-syntax-highlighter": "^15.5.0",
     "sonner": "^1.5.0",
-    "tailwind-merge": "^2.1.0"
+    "tailwind-merge": "^2.5.2"
   },
   "devDependencies": {
-    "@types/node": "^20.12.11",
-    "@types/react": "^18.3.3",
+    "@types/node": "^22.5.1",
+    "@types/react": "^18.3.5",
     "@types/react-dom": "^18.3.0",
     "@types/react-syntax-highlighter": "^15.5.11",
-    "autoprefixer": "^10.4.16",
+    "autoprefixer": "^10.4.20",
     "cross-env": "^7.0.3",
-    "postcss": "^8.4.32",
-    "tailwindcss": "^3.4.4",
-    "typescript": "^5.5.3"
+    "postcss": "^8.4.41",
+    "tailwindcss": "^3.4.10",
+    "typescript": "^5.5.4"
   }
 }

packages/autotool/package.json

@@ -45,13 +45,13 @@
     "dev": "bunchee --watch"
   },
   "dependencies": {
-    "@swc/core": "^1.6.3",
-    "jotai": "^2.8.3",
-    "typedoc": "^0.26.4",
-    "unplugin": "^1.10.1"
+    "@swc/core": "^1.7.22",
+    "jotai": "2.8.4",
+    "typedoc": "^0.26.6",
+    "unplugin": "^1.12.2"
   },
   "peerDependencies": {
-    "llamaindex": "^0.5.16",
+    "llamaindex": "^0.5.22",
     "openai": "^4",
     "typescript": "^4"
   },
@@ -67,16 +67,16 @@
     }
   },
   "devDependencies": {
-    "@swc/types": "^0.1.8",
+    "@swc/types": "^0.1.12",
     "@types/json-schema": "^7.0.15",
-    "@types/node": "^20.12.11",
-    "bunchee": "5.3.1",
+    "@types/node": "^22.5.1",
+    "bunchee": "5.3.2",
     "llamaindex": "workspace:*",
-    "next": "14.2.5",
-    "rollup": "^4.18.0",
-    "tsx": "^4.15.6",
-    "typescript": "^5.5.3",
-    "vitest": "^2.0.2",
-    "webpack": "^5.92.1"
+    "next": "14.2.7",
+    "rollup": "^4.21.2",
+    "tsx": "^4.19.0",
+    "typescript": "^5.5.4",
+    "vitest": "^2.0.5",
+    "webpack": "^5.94.0"
   }
 }

packages/cloud/package.json

@@ -34,7 +34,7 @@
     "directory": "packages/cloud"
   },
   "devDependencies": {
-    "@hey-api/openapi-ts": "^0.48.0",
-    "bunchee": "5.3.1"
+    "@hey-api/openapi-ts": "^0.52.11",
+    "bunchee": "5.3.2"
   }
 }

packages/community/CHANGELOG.md

@@ -1,5 +1,27 @@
 # @llamaindex/community
 
+## 0.0.32
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - @llamaindex/env@0.1.10
+  - @llamaindex/core@0.1.11
+
+## 0.0.31
+
+### Patch Changes
+
+- Updated dependencies [0148354]
+  - @llamaindex/core@0.1.10
+
+## 0.0.30
+
+### Patch Changes
+
+- Updated dependencies [e27e7dd]
+  - @llamaindex/core@0.1.9
+
 ## 0.0.29
 
 ### Patch Changes

packages/community/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/community",
   "description": "Community package for LlamaIndexTS",
-  "version": "0.0.29",
+  "version": "0.0.32",
   "type": "module",
   "types": "dist/type/index.d.ts",
   "main": "dist/cjs/index.js",
@@ -42,11 +42,11 @@
     "dev": "bunchee --watch"
   },
   "devDependencies": {
-    "@types/node": "^20.14.2",
-    "bunchee": "5.3.1"
+    "@types/node": "^22.5.1",
+    "bunchee": "5.3.2"
   },
   "dependencies": {
-    "@aws-sdk/client-bedrock-runtime": "^3.613.0",
+    "@aws-sdk/client-bedrock-runtime": "^3.642.0",
     "@llamaindex/core": "workspace:*",
     "@llamaindex/env": "workspace:*"
   }

packages/core/CHANGELOG.md

@@ -1,5 +1,26 @@
 # @llamaindex/core
 
+## 0.1.11
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - @llamaindex/env@0.1.10
+
+## 0.1.10
+
+### Patch Changes
+
+- 0148354: refactor: prompt system
+
+  Add `PromptTemplate` module with strong type check.
+
+## 0.1.9
+
+### Patch Changes
+
+- e27e7dd: chore: bump `natural` to 8.0.1
+
 ## 0.1.8
 
 ### Patch Changes

packages/core/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/core",
   "type": "module",
-  "version": "0.1.8",
+  "version": "0.1.11",
   "description": "LlamaIndex Core Module",
   "exports": {
     "./node-parser": {
@@ -115,6 +115,20 @@
         "types": "./dist/utils/index.d.ts",
         "default": "./dist/utils/index.js"
       }
+    },
+    "./prompts": {
+      "require": {
+        "types": "./dist/prompts/index.d.cts",
+        "default": "./dist/prompts/index.cjs"
+      },
+      "import": {
+        "types": "./dist/prompts/index.d.ts",
+        "default": "./dist/prompts/index.js"
+      },
+      "default": {
+        "types": "./dist/prompts/index.d.ts",
+        "default": "./dist/prompts/index.js"
+      }
     }
   },
   "files": [
@@ -130,13 +144,14 @@
     "url": "https://github.com/himself65/LlamaIndexTS.git"
   },
   "devDependencies": {
-    "ajv": "^8.16.0",
-    "bunchee": "5.3.1",
-    "natural": "^7.1.0"
+    "ajv": "^8.17.1",
+    "bunchee": "5.3.2",
+    "natural": "^8.0.1",
+    "python-format-js": "^1.4.3"
   },
   "dependencies": {
     "@llamaindex/env": "workspace:*",
-    "@types/node": "^20.14.9",
+    "@types/node": "^22.5.1",
     "zod": "^3.23.8"
   }
 }

packages/core/src/global/constants.ts

@@ -0,0 +1,24 @@
+import { path } from "@llamaindex/env";
+
+//#region llm
+export const DEFAULT_CONTEXT_WINDOW = 3900;
+export const DEFAULT_NUM_OUTPUTS = 256;
+export const DEFAULT_CHUNK_SIZE = 1024;
+export const DEFAULT_CHUNK_OVERLAP = 20;
+export const DEFAULT_CHUNK_OVERLAP_RATIO = 0.1;
+export const DEFAULT_PADDING = 5;
+//#endregion
+//#region storage
+export const DEFAULT_COLLECTION = "data";
+export const DEFAULT_PERSIST_DIR = path.join("./storage");
+export const DEFAULT_INDEX_STORE_PERSIST_FILENAME = "index_store.json";
+export const DEFAULT_DOC_STORE_PERSIST_FILENAME = "doc_store.json";
+export const DEFAULT_VECTOR_STORE_PERSIST_FILENAME = "vector_store.json";
+export const DEFAULT_GRAPH_STORE_PERSIST_FILENAME = "graph_store.json";
+export const DEFAULT_NAMESPACE = "docstore";
+export const DEFAULT_IMAGE_VECTOR_NAMESPACE = "images";
+//#endregion
+//#region llama cloud
+export const DEFAULT_PROJECT_NAME = "Default";
+export const DEFAULT_BASE_URL = "https://api.cloud.llamaindex.ai";
+//#endregion

packages/core/src/global/index.ts

@@ -1,3 +1,4 @@
+export * from "./constants";
 export { Settings } from "./settings";
 export { CallbackManager } from "./settings/callback-manager";
 export type {

packages/core/src/node-parser/sentence_tokenizer.d.ts

@@ -1,4 +1,5 @@
 declare class SentenceTokenizer {
+  constructor(abbreviations?: string[]);
   tokenize(text: string): string[];
 }
 

packages/core/src/node-parser/sentence_tokenizer.js

@@ -0,0 +1,222 @@
+var __getOwnPropNames = Object.getOwnPropertyNames;
+var __commonJS = (cb, mod) =>
+  function __require() {
+    return (
+      mod ||
+        (0, cb[__getOwnPropNames(cb)[0]])((mod = { exports: {} }).exports, mod),
+      mod.exports
+    );
+  };
+
+// lib/natural/tokenizers/tokenizer.js
+var require_tokenizer = __commonJS({
+  "lib/natural/tokenizers/tokenizer.js"(exports, module) {
+    "use strict";
+    var Tokenizer = class {
+      trim(array) {
+        while (array[array.length - 1] === "") {
+          array.pop();
+        }
+        while (array[0] === "") {
+          array.shift();
+        }
+        return array;
+      }
+    };
+    module.exports = Tokenizer;
+  },
+});
+
+// lib/natural/tokenizers/sentence_tokenizer.js
+var require_sentence_tokenizer = __commonJS({
+  "lib/natural/tokenizers/sentence_tokenizer.js"(exports, module) {
+    var Tokenizer = require_tokenizer();
+    var NUM = "NUMBER";
+    var DELIM = "DELIM";
+    var URI = "URI";
+    var ABBREV = "ABBREV";
+    var DEBUG = false;
+    function generateUniqueCode(base, index) {
+      return `{{${base}_${index}}}`;
+    }
+    function escapeRegExp(string) {
+      return string.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+    }
+    var SentenceTokenizer = class extends Tokenizer {
+      constructor(abbreviations) {
+        super();
+        if (abbreviations) {
+          this.abbreviations = abbreviations;
+        } else {
+          this.abbreviations = [];
+        }
+        this.replacementMap = null;
+        this.replacementCounter = 0;
+      }
+      replaceUrisWithPlaceholders(text) {
+        const urlPattern =
+          /(https?:\/\/\S+|www\.\S+|ftp:\/\/\S+|(mailto:)?[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}|file:\/\/\S+)/gi;
+        const modifiedText = text.replace(urlPattern, (match) => {
+          const placeholder = generateUniqueCode(
+            URI,
+            this.replacementCounter++,
+          );
+          this.replacementMap.set(placeholder, match);
+          return placeholder;
+        });
+        return modifiedText;
+      }
+      replaceAbbreviations(text) {
+        if (this.abbreviations.length === 0) {
+          return text;
+        }
+        const pattern = new RegExp(
+          `(${this.abbreviations.map((abbrev) => escapeRegExp(abbrev)).join("|")})`,
+          "gi",
+        );
+        const replacedText = text.replace(pattern, (match) => {
+          const code = generateUniqueCode(ABBREV, this.replacementCounter++);
+          this.replacementMap.set(code, match);
+          return code;
+        });
+        return replacedText;
+      }
+      replaceDelimitersWithPlaceholders(text) {
+        const delimiterPattern = /([.?!… ]*)([.?!…])(["'”’)}\]]?)/g;
+        const modifiedText = text.replace(
+          delimiterPattern,
+          (match, p1, p2, p3) => {
+            const placeholder = generateUniqueCode(
+              DELIM,
+              this.replacementCounter++,
+            );
+            this.delimiterMap.set(placeholder, p1 + p2 + p3);
+            return placeholder;
+          },
+        );
+        return modifiedText;
+      }
+      splitOnPlaceholders(text, placeholders) {
+        if (this.delimiterMap.size === 0) {
+          return [text];
+        }
+        const keys = Array.from(this.delimiterMap.keys());
+        const pattern = new RegExp(`(${keys.map(escapeRegExp).join("|")})`);
+        const parts = text.split(pattern);
+        const sentences = [];
+        for (let i = 0; i < parts.length; i += 2) {
+          const sentence = parts[i];
+          const placeholder = parts[i + 1] || "";
+          sentences.push(sentence + placeholder);
+        }
+        return sentences;
+      }
+      replaceNumbersWithCode(text) {
+        const numberPattern = /\b\d{1,3}(?:,\d{3})*(?:\.\d+)?\b/g;
+        const replacedText = text.replace(numberPattern, (match) => {
+          const code = generateUniqueCode(NUM, this.replacementCounter++);
+          this.replacementMap.set(code, match);
+          return code;
+        });
+        return replacedText;
+      }
+      revertReplacements(text) {
+        let originalText = text;
+        for (const [
+          placeholder,
+          replacement,
+        ] of this.replacementMap.entries()) {
+          const pattern = new RegExp(escapeRegExp(placeholder), "g");
+          originalText = originalText.replace(pattern, replacement);
+        }
+        return originalText;
+      }
+      revertDelimiters(text) {
+        let originalText = text;
+        for (const [placeholder, replacement] of this.delimiterMap.entries()) {
+          const pattern = new RegExp(escapeRegExp(placeholder), "g");
+          originalText = originalText.replace(pattern, replacement);
+        }
+        return originalText;
+      }
+      tokenize(text) {
+        this.replacementCounter = 0;
+        this.replacementMap = /* @__PURE__ */ new Map();
+        this.delimiterMap = /* @__PURE__ */ new Map();
+        DEBUG &&
+          console.log(
+            "---Start of sentence tokenization-----------------------",
+          );
+        DEBUG && console.log("Original input: >>>" + text + "<<<");
+        const result1 = this.replaceAbbreviations(text);
+        DEBUG &&
+          console.log(
+            "Phase 1: replacing abbreviations: " + JSON.stringify(result1),
+          );
+        const result2 = this.replaceUrisWithPlaceholders(result1);
+        DEBUG &&
+          console.log("Phase 2: replacing URIs: " + JSON.stringify(result2));
+        const result3 = this.replaceNumbersWithCode(result2);
+        DEBUG &&
+          console.log(
+            "Phase 3: replacing numbers with placeholders: " +
+              JSON.stringify(result3),
+          );
+        const result4 = this.replaceDelimitersWithPlaceholders(result3);
+        DEBUG &&
+          console.log(
+            "Phase 4: replacing delimiters with placeholders: " +
+              JSON.stringify(result4),
+          );
+        const sentences = this.splitOnPlaceholders(result4);
+        DEBUG &&
+          console.log(
+            "Phase 5: splitting into sentences on placeholders: " +
+              JSON.stringify(sentences),
+          );
+        const newSentences = sentences.map((s) => {
+          const s1 = this.revertReplacements(s);
+          return this.revertDelimiters(s1);
+        });
+        DEBUG &&
+          console.log(
+            "Phase 6: replacing back abbreviations, URIs, numbers and delimiters: " +
+              JSON.stringify(newSentences),
+          );
+        const trimmedSentences = this.trim(newSentences);
+        DEBUG &&
+          console.log(
+            "Phase 7: trimming array of empty sentences: " +
+              JSON.stringify(trimmedSentences),
+          );
+        const trimmedSentences2 = trimmedSentences.map((sent) => sent.trim());
+        DEBUG &&
+          console.log(
+            "Phase 8: trimming sentences from surrounding whitespace: " +
+              JSON.stringify(trimmedSentences2),
+          );
+        DEBUG &&
+          console.log(
+            "---End of sentence tokenization--------------------------",
+          );
+        DEBUG &&
+          console.log(
+            "---Replacement map---------------------------------------",
+          );
+        DEBUG && console.log([...this.replacementMap.entries()]);
+        DEBUG &&
+          console.log(
+            "---Delimiter map-----------------------------------------",
+          );
+        DEBUG && console.log([...this.delimiterMap.entries()]);
+        DEBUG &&
+          console.log(
+            "---------------------------------------------------------",
+          );
+        return trimmedSentences2;
+      }
+    };
+    module.exports = SentenceTokenizer;
+  },
+});
+export default require_sentence_tokenizer();

packages/core/src/node-parser/utils.ts

@@ -1,5 +1,5 @@
 import type { TextSplitter } from "./base";
-import SentenceTokenizerNew from "./sentence-tokenizer-parser.js";
+import SentenceTokenizer from "./sentence_tokenizer";
 
 export type TextSplitterFn = (text: string) => string[];
 
@@ -31,11 +31,17 @@ export const splitByChar = (): TextSplitterFn => {
   return (text: string) => text.split("");
 };
 
-let sentenceTokenizer: SentenceTokenizerNew | null = null;
+let sentenceTokenizer: SentenceTokenizer | null = null;
 
 export const splitBySentenceTokenizer = (): TextSplitterFn => {
   if (!sentenceTokenizer) {
-    sentenceTokenizer = new SentenceTokenizerNew();
+    sentenceTokenizer = new SentenceTokenizer([
+      "i.e.",
+      "etc.",
+      "vs.",
+      "Inc.",
+      "A.S.A.P.",
+    ]);
   }
   const tokenizer = sentenceTokenizer;
   return (text: string) => {

packages/core/src/prompts/base.ts

@@ -0,0 +1,225 @@
+import format from "python-format-js";
+import type { ChatMessage } from "../llms";
+import type { BaseOutputParser, Metadata } from "../schema";
+import { objectEntries } from "../utils";
+import { PromptType } from "./prompt-type";
+
+type MappingFn<TemplatesVar extends string[] = string[]> = (
+  options: Record<TemplatesVar[number], string>,
+) => string;
+
+export type BasePromptTemplateOptions<
+  TemplatesVar extends readonly string[],
+  Vars extends readonly string[],
+> = {
+  metadata?: Metadata;
+  templateVars?:
+    | TemplatesVar
+    // loose type for better type inference
+    | readonly string[];
+  options?: Partial<Record<TemplatesVar[number] | (string & {}), string>>;
+  outputParser?: BaseOutputParser;
+  templateVarMappings?: Partial<
+    Record<Vars[number] | (string & {}), TemplatesVar[number] | (string & {})>
+  >;
+  functionMappings?: Partial<
+    Record<TemplatesVar[number] | (string & {}), MappingFn>
+  >;
+};
+
+export abstract class BasePromptTemplate<
+  const TemplatesVar extends readonly string[] = string[],
+  const Vars extends readonly string[] = string[],
+> {
+  metadata: Metadata = {};
+  templateVars: Set<string> = new Set();
+  options: Partial<Record<TemplatesVar[number] | (string & {}), string>> = {};
+  outputParser?: BaseOutputParser;
+  templateVarMappings: Partial<
+    Record<Vars[number] | (string & {}), TemplatesVar[number] | (string & {})>
+  > = {};
+  functionMappings: Partial<
+    Record<TemplatesVar[number] | (string & {}), MappingFn>
+  > = {};
+
+  protected constructor(
+    options: BasePromptTemplateOptions<TemplatesVar, Vars>,
+  ) {
+    const {
+      metadata,
+      templateVars,
+      outputParser,
+      templateVarMappings,
+      functionMappings,
+    } = options;
+    if (metadata) {
+      this.metadata = metadata;
+    }
+    if (templateVars) {
+      this.templateVars = new Set(templateVars);
+    }
+    if (options.options) {
+      this.options = options.options;
+    }
+    this.outputParser = outputParser;
+    if (templateVarMappings) {
+      this.templateVarMappings = templateVarMappings;
+    }
+    if (functionMappings) {
+      this.functionMappings = functionMappings;
+    }
+  }
+
+  protected mapTemplateVars(
+    options: Record<TemplatesVar[number] | (string & {}), string>,
+  ) {
+    const templateVarMappings = this.templateVarMappings;
+    return Object.fromEntries(
+      objectEntries(options).map(([k, v]) => [templateVarMappings[k] || k, v]),
+    );
+  }
+
+  protected mapFunctionVars(
+    options: Record<TemplatesVar[number] | (string & {}), string>,
+  ) {
+    const functionMappings = this.functionMappings;
+    const newOptions = {} as Record<TemplatesVar[number], string>;
+    for (const [k, v] of objectEntries(functionMappings)) {
+      newOptions[k] = v!(options);
+    }
+
+    for (const [k, v] of objectEntries(options)) {
+      if (!(k in newOptions)) {
+        newOptions[k] = v;
+      }
+    }
+
+    return newOptions;
+  }
+
+  protected mapAllVars(
+    options: Record<TemplatesVar[number] | (string & {}), string>,
+  ): Record<string, string> {
+    const newOptions = this.mapFunctionVars(options);
+    return this.mapTemplateVars(newOptions);
+  }
+
+  abstract partialFormat(
+    options: Partial<Record<TemplatesVar[number] | (string & {}), string>>,
+  ): BasePromptTemplate<TemplatesVar, Vars>;
+
+  abstract format(
+    options?: Partial<Record<TemplatesVar[number] | (string & {}), string>>,
+  ): string;
+
+  abstract formatMessages(
+    options?: Partial<Record<TemplatesVar[number] | (string & {}), string>>,
+  ): ChatMessage[];
+
+  abstract get template(): string;
+}
+
+type Permutation<T, K = T> = [T] extends [never]
+  ? []
+  : K extends K
+    ? [K, ...Permutation<Exclude<T, K>>]
+    : never;
+
+type Join<T extends any[], U extends string> = T extends [infer F, ...infer R]
+  ? R["length"] extends 0
+    ? `${F & string}`
+    : `${F & string}${U}${Join<R, U>}`
+  : never;
+
+type WrapStringWithBracket<T extends string> = `{${T}}`;
+
+export type StringTemplate<Var extends readonly string[]> =
+  Var["length"] extends 0
+    ? string
+    : Var["length"] extends number
+      ? number extends Var["length"]
+        ? string
+        : `${string}${Join<Permutation<WrapStringWithBracket<Var[number]>>, `${string}`>}${string}`
+      : never;
+
+export type PromptTemplateOptions<
+  TemplatesVar extends readonly string[],
+  Vars extends readonly string[],
+  Template extends StringTemplate<TemplatesVar>,
+> = BasePromptTemplateOptions<TemplatesVar, Vars> & {
+  template: Template;
+  promptType?: PromptType;
+};
+
+export class PromptTemplate<
+  const TemplatesVar extends readonly string[] = string[],
+  const Vars extends readonly string[] = string[],
+  const Template extends
+    StringTemplate<TemplatesVar> = StringTemplate<TemplatesVar>,
+> extends BasePromptTemplate<TemplatesVar, Vars> {
+  #template: Template;
+  promptType: PromptType;
+
+  constructor(options: PromptTemplateOptions<TemplatesVar, Vars, Template>) {
+    const { template, promptType, ...rest } = options;
+    super(rest);
+    this.#template = template;
+    this.promptType = promptType ?? PromptType.custom;
+  }
+
+  partialFormat(
+    options: Partial<Record<TemplatesVar[number] | (string & {}), string>>,
+  ): PromptTemplate<TemplatesVar, Vars, Template> {
+    const prompt = new PromptTemplate({
+      template: this.template,
+      templateVars: [...this.templateVars],
+      options: this.options,
+      outputParser: this.outputParser,
+      templateVarMappings: this.templateVarMappings,
+      functionMappings: this.functionMappings,
+      metadata: this.metadata,
+      promptType: this.promptType,
+    });
+
+    prompt.options = {
+      ...prompt.options,
+      ...options,
+    };
+
+    return prompt;
+  }
+
+  format(
+    options?: Partial<Record<TemplatesVar[number] | (string & {}), string>>,
+  ): string {
+    const allOptions = {
+      ...this.options,
+      ...options,
+    } as Record<TemplatesVar[number], string>;
+
+    const mappedAllOptions = this.mapAllVars(allOptions);
+
+    const prompt = format(this.template, mappedAllOptions);
+
+    if (this.outputParser) {
+      return this.outputParser.format(prompt);
+    }
+    return prompt;
+  }
+
+  formatMessages(
+    options?: Partial<Record<TemplatesVar[number] | (string & {}), string>>,
+  ): ChatMessage[] {
+    const prompt = this.format(options);
+    return [
+      {
+        role: "user",
+        content: prompt,
+      },
+    ];
+  }
+
+  get template(): Template {
+    return this.#template;
+  }
+}

packages/core/src/prompts/index.ts

@@ -0,0 +1,33 @@
+export { BasePromptTemplate, PromptTemplate } from "./base";
+export type {
+  BasePromptTemplateOptions,
+  PromptTemplateOptions,
+  StringTemplate,
+} from "./base";
+export { PromptMixin, type ModuleRecord, type PromptsRecord } from "./mixin";
+export {
+  anthropicSummaryPrompt,
+  anthropicTextQaPrompt,
+  defaultChoiceSelectPrompt,
+  defaultCondenseQuestionPrompt,
+  defaultContextSystemPrompt,
+  defaultKeywordExtractPrompt,
+  defaultQueryKeywordExtractPrompt,
+  defaultRefinePrompt,
+  defaultSubQuestionPrompt,
+  defaultSummaryPrompt,
+  defaultTextQAPrompt,
+  defaultTreeSummarizePrompt,
+} from "./prompt";
+export type {
+  ChoiceSelectPrompt,
+  CondenseQuestionPrompt,
+  ContextSystemPrompt,
+  KeywordExtractPrompt,
+  QueryKeywordExtractPrompt,
+  RefinePrompt,
+  SubQuestionPrompt,
+  SummaryPrompt,
+  TextQAPrompt,
+  TreeSummarizePrompt,
+} from "./prompt";

packages/core/src/prompts/mixin.ts

@@ -0,0 +1,79 @@
+import { objectEntries } from "../utils";
+import type { BasePromptTemplate } from "./base";
+
+export type PromptsRecord = Record<string, BasePromptTemplate>;
+export type ModuleRecord = Record<string, PromptMixin>;
+
+export abstract class PromptMixin {
+  validatePrompts(promptsDict: PromptsRecord, moduleDict: ModuleRecord): void {
+    for (const key of Object.keys(promptsDict)) {
+      if (key.includes(":")) {
+        throw new Error(`Prompt key ${key} cannot contain ':'.`);
+      }
+    }
+
+    for (const key of Object.keys(moduleDict)) {
+      if (key.includes(":")) {
+        throw new Error(`Module key ${key} cannot contain ':'.`);
+      }
+    }
+  }
+
+  getPrompts(): PromptsRecord {
+    const promptsDict: PromptsRecord = this._getPrompts();
+
+    const moduleDict = this._getPromptModules();
+
+    this.validatePrompts(promptsDict, moduleDict);
+
+    const allPrompts: PromptsRecord = { ...promptsDict };
+
+    for (const [module_name, prompt_module] of objectEntries(moduleDict)) {
+      for (const [key, prompt] of objectEntries(prompt_module.getPrompts())) {
+        allPrompts[`${module_name}:${key}`] = prompt;
+      }
+    }
+    return allPrompts;
+  }
+
+  updatePrompts(prompts: PromptsRecord): void {
+    const promptModules = this._getPromptModules();
+
+    this._updatePrompts(prompts);
+
+    const subPrompt: Record<string, PromptsRecord> = {};
+
+    for (const key in prompts) {
+      if (key.includes(":")) {
+        const [module_name, sub_key] = key.split(":");
+
+        if (!subPrompt[module_name]) {
+          subPrompt[module_name] = {};
+        }
+        subPrompt[module_name][sub_key] = prompts[key];
+      }
+    }
+
+    for (const [module_name, subPromptDict] of Object.entries(subPrompt)) {
+      if (!promptModules[module_name]) {
+        throw new Error(`Module ${module_name} not found.`);
+      }
+
+      const moduleToUpdate = promptModules[module_name];
+
+      moduleToUpdate.updatePrompts(subPromptDict);
+    }
+  }
+
+  protected abstract _getPrompts(): PromptsRecord;
+  protected abstract _updatePrompts(prompts: PromptsRecord): void;
+
+  /**
+   *
+   * Return a dictionary of sub-modules within the current module
+   * that also implement PromptMixin (so that their prompts can also be get/set).
+   *
+   * Can be blank if no sub-modules.
+   */
+  protected abstract _getPromptModules(): ModuleRecord;
+}

packages/core/src/prompts/prompt-type.ts

@@ -0,0 +1,64 @@
+import { z } from "zod";
+
+const promptType = {
+  SUMMARY: "summary",
+  TREE_INSERT: "insert",
+  TREE_SELECT: "tree_select",
+  TREE_SELECT_MULTIPLE: "tree_select_multiple",
+  QUESTION_ANSWER: "text_qa",
+  REFINE: "refine",
+  KEYWORD_EXTRACT: "keyword_extract",
+  QUERY_KEYWORD_EXTRACT: "query_keyword_extract",
+  SCHEMA_EXTRACT: "schema_extract",
+  TEXT_TO_SQL: "text_to_sql",
+  TEXT_TO_GRAPH_QUERY: "text_to_graph_query",
+  TABLE_CONTEXT: "table_context",
+  KNOWLEDGE_TRIPLET_EXTRACT: "knowledge_triplet_extract",
+  SIMPLE_INPUT: "simple_input",
+  PANDAS: "pandas",
+  JSON_PATH: "json_path",
+  SINGLE_SELECT: "single_select",
+  MULTI_SELECT: "multi_select",
+  VECTOR_STORE_QUERY: "vector_store_query",
+  SUB_QUESTION: "sub_question",
+  SQL_RESPONSE_SYNTHESIS: "sql_response_synthesis",
+  SQL_RESPONSE_SYNTHESIS_V2: "sql_response_synthesis_v2",
+  CONVERSATION: "conversation",
+  DECOMPOSE: "decompose",
+  CHOICE_SELECT: "choice_select",
+  CUSTOM: "custom",
+  RANKGPT_RERANK: "rankgpt_rerank",
+} as const;
+
+const promptTypeSchema = z.enum([
+  promptType.SUMMARY,
+  promptType.TREE_INSERT,
+  promptType.TREE_SELECT,
+  promptType.TREE_SELECT_MULTIPLE,
+  promptType.QUESTION_ANSWER,
+  promptType.REFINE,
+  promptType.KEYWORD_EXTRACT,
+  promptType.QUERY_KEYWORD_EXTRACT,
+  promptType.SCHEMA_EXTRACT,
+  promptType.TEXT_TO_SQL,
+  promptType.TEXT_TO_GRAPH_QUERY,
+  promptType.TABLE_CONTEXT,
+  promptType.KNOWLEDGE_TRIPLET_EXTRACT,
+  promptType.SIMPLE_INPUT,
+  promptType.PANDAS,
+  promptType.JSON_PATH,
+  promptType.SINGLE_SELECT,
+  promptType.MULTI_SELECT,
+  promptType.VECTOR_STORE_QUERY,
+  promptType.SUB_QUESTION,
+  promptType.SQL_RESPONSE_SYNTHESIS,
+  promptType.SQL_RESPONSE_SYNTHESIS_V2,
+  promptType.CONVERSATION,
+  promptType.DECOMPOSE,
+  promptType.CHOICE_SELECT,
+  promptType.CUSTOM,
+  promptType.RANKGPT_RERANK,
+]);
+
+export const PromptType = promptTypeSchema.enum;
+export type PromptType = z.infer<typeof promptTypeSchema>;

packages/core/src/prompts/prompt.ts

@@ -0,0 +1,253 @@
+import type { ChatMessage, ToolMetadata } from "../llms";
+import { PromptTemplate } from "./base";
+
+export type TextQAPrompt = PromptTemplate<["context", "query"]>;
+export type SummaryPrompt = PromptTemplate<["context"]>;
+export type RefinePrompt = PromptTemplate<
+  ["query", "existingAnswer", "context"]
+>;
+export type TreeSummarizePrompt = PromptTemplate<["context", "query"]>;
+export type ChoiceSelectPrompt = PromptTemplate<["context", "query"]>;
+export type SubQuestionPrompt = PromptTemplate<["toolsStr", "queryStr"]>;
+export type CondenseQuestionPrompt = PromptTemplate<
+  ["chatHistory", "question"]
+>;
+export type ContextSystemPrompt = PromptTemplate<["context"]>;
+export type KeywordExtractPrompt = PromptTemplate<["context"]>;
+export type QueryKeywordExtractPrompt = PromptTemplate<["question"]>;
+
+export const defaultTextQAPrompt: TextQAPrompt = new PromptTemplate({
+  templateVars: ["context", "query"],
+  template: `Context information is below.
+---------------------
+{context}
+---------------------
+Given the context information and not prior knowledge, answer the query.
+Query: {query}
+Answer:`,
+});
+
+export const anthropicTextQaPrompt: TextQAPrompt = new PromptTemplate({
+  templateVars: ["context", "query"],
+  template: `Context information:
+<context>
+{context}
+</context>
+Given the context information and not prior knowledge, answer the query.
+Query: {query}`,
+});
+
+export const defaultSummaryPrompt: SummaryPrompt = new PromptTemplate({
+  templateVars: ["context"],
+  template: `Write a summary of the following. Try to use only the information provided. Try to include as many key details as possible.
+
+
+{context}
+
+
+SUMMARY:"""
+`,
+});
+
+export const anthropicSummaryPrompt: SummaryPrompt = new PromptTemplate({
+  templateVars: ["context"],
+  template: `Summarize the following text. Try to use only the information provided. Try to include as many key details as possible.
+<original-text>
+{context}
+</original-text>
+
+SUMMARY:
+`,
+});
+
+export const defaultRefinePrompt: RefinePrompt = new PromptTemplate({
+  templateVars: ["query", "existingAnswer", "context"],
+  template: `The original query is as follows: {query}
+We have provided an existing answer: {existingAnswer}
+We have the opportunity to refine the existing answer (only if needed) with some more context below.
+------------
+{context}
+------------
+Given the new context, refine the original answer to better answer the query. If the context isn't useful, return the original answer.
+Refined Answer:`,
+});
+
+export const defaultTreeSummarizePrompt = new PromptTemplate({
+  templateVars: ["context", "query"],
+  template: `Context information from multiple sources is below.
+---------------------
+{context}
+---------------------
+Given the information from multiple sources and not prior knowledge, answer the query.
+Query: {query}
+Answer:`,
+});
+
+export const defaultChoiceSelectPrompt = new PromptTemplate({
+  templateVars: ["context", "query"],
+  template: `A list of documents is shown below. Each document has a number next to it along 
+with a summary of the document. A question is also provided.
+Respond with the numbers of the documents
+you should consult to answer the question, in order of relevance, as well
+as the relevance score. The relevance score is a number from 1-10 based on
+how relevant you think the document is to the question.
+Do not include any documents that are not relevant to the question.
+Example format:
+Document 1:
+<summary of document 1>
+
+Document 2:
+<summary of document 2>
+
+...
+
+Document 10:\n<summary of document 10>
+
+Question: <question>
+Answer:
+Doc: 9, Relevance: 7
+Doc: 3, Relevance: 4
+Doc: 7, Relevance: 3
+
+Let's try this now:
+
+{context}
+Question: {query}
+Answer:`,
+});
+
+export function buildToolsText(tools: ToolMetadata[]) {
+  const toolsObj = tools.reduce<Record<string, string>>((acc, tool) => {
+    acc[tool.name] = tool.description;
+    return acc;
+  }, {});
+
+  return JSON.stringify(toolsObj, null, 4);
+}
+
+const exampleTools: ToolMetadata[] = [
+  {
+    name: "uber_10k",
+    description: "Provides information about Uber financials for year 2021",
+  },
+  {
+    name: "lyft_10k",
+    description: "Provides information about Lyft financials for year 2021",
+  },
+];
+
+const exampleQueryStr = `Compare and contrast the revenue growth and EBITDA of Uber and Lyft for year 2021`;
+
+const exampleOutput = [
+  {
+    subQuestion: "What is the revenue growth of Uber",
+    toolName: "uber_10k",
+  },
+  {
+    subQuestion: "What is the EBITDA of Uber",
+    toolName: "uber_10k",
+  },
+  {
+    subQuestion: "What is the revenue growth of Lyft",
+    toolName: "lyft_10k",
+  },
+  {
+    subQuestion: "What is the EBITDA of Lyft",
+    toolName: "lyft_10k",
+  },
+] as const;
+
+export const defaultSubQuestionPrompt: SubQuestionPrompt = new PromptTemplate({
+  templateVars: ["toolsStr", "queryStr"],
+  template: `Given a user question, and a list of tools, output a list of relevant sub-questions that when composed can help answer the full user question:
+
+# Example 1
+<Tools>
+\`\`\`json
+${buildToolsText(exampleTools)}
+\`\`\`
+
+<User Question>
+${exampleQueryStr}
+
+<Output>
+\`\`\`json
+${JSON.stringify(exampleOutput, null, 4)}
+\`\`\`
+
+# Example 2
+<Tools>
+\`\`\`json
+{toolsStr}
+\`\`\`
+
+<User Question>
+{queryStr}
+
+<Output>
+`,
+});
+
+export const defaultCondenseQuestionPrompt = new PromptTemplate({
+  templateVars: ["chatHistory", "question"],
+  template: `Given a conversation (between Human and Assistant) and a follow up message from Human, rewrite the message to be a standalone question that captures all relevant context from the conversation.
+
+<Chat History>
+{chatHistory}
+
+<Follow Up Message>
+{question}
+
+<Standalone question>
+`,
+});
+
+export function messagesToHistoryStr(messages: ChatMessage[]) {
+  return messages.reduce((acc, message) => {
+    acc += acc ? "\n" : "";
+    if (message.role === "user") {
+      acc += `Human: ${message.content}`;
+    } else {
+      acc += `Assistant: ${message.content}`;
+    }
+    return acc;
+  }, "");
+}
+
+export const defaultContextSystemPrompt: ContextSystemPrompt =
+  new PromptTemplate({
+    templateVars: ["context"],
+    template: `Context information is below.
+---------------------
+{context}
+---------------------`,
+  });
+
+export const defaultKeywordExtractPrompt: KeywordExtractPrompt =
+  new PromptTemplate({
+    templateVars: ["maxKeywords", "context"],
+    template: `
+Some text is provided below. Given the text, extract up to {maxKeywords} keywords from the text. Avoid stopwords.
+---------------------
+{context}
+---------------------
+Provide keywords in the following comma-separated format: 'KEYWORDS: <keywords>'
+`,
+  }).partialFormat({
+    maxKeywords: "10",
+  });
+
+export const defaultQueryKeywordExtractPrompt = new PromptTemplate({
+  templateVars: ["maxKeywords", "question"],
+  template: `(
+  "A question is provided below. Given the question, extract up to {maxKeywords} "
+  "keywords from the text. Focus on extracting the keywords that we can use "
+  "to best lookup answers to the question. Avoid stopwords."
+  "---------------------"
+  "{question}"
+  "---------------------"
+  "Provide keywords in the following comma-separated format: 'KEYWORDS: <keywords>'"
+)`,
+}).partialFormat({
+  maxKeywords: "10",
+});

packages/core/src/schema/index.ts

@@ -1,4 +1,5 @@
 export * from "./node";
 export { FileReader, TransformComponent, type BaseReader } from "./type";
+export type { BaseOutputParser } from "./type/base-output-parser";
 export { EngineResponse } from "./type/engine–response";
 export * from "./zod";

packages/core/src/schema/type/base-output-parser.ts

@@ -0,0 +1,8 @@
+/**
+ * An OutputParser is used to extract structured data from the raw output of the LLM.
+ */
+export interface BaseOutputParser<T = any> {
+  parse(output: string): T;
+
+  format(output: string): string;
+}

packages/core/src/utils/index.ts

@@ -51,4 +51,8 @@ export {
   extractImage,
   extractSingleText,
   extractText,
+  messagesToHistory,
+  toToolDescriptions,
 } from "./llms";
+
+export { objectEntries } from "./object-entries";

packages/core/src/utils/llms.ts

@@ -1,7 +1,9 @@
 import type {
+  ChatMessage,
   MessageContent,
   MessageContentDetail,
   MessageContentTextDetail,
+  ToolMetadata,
 } from "../llms";
 import type { QueryType } from "../query-engine";
 import type { ImageType } from "../schema";
@@ -84,3 +86,24 @@ export const extractDataUrlComponents = (
     base64,
   };
 };
+
+export function messagesToHistory(messages: ChatMessage[]): string {
+  return messages.reduce((acc, message) => {
+    acc += acc ? "\n" : "";
+    if (message.role === "user") {
+      acc += `Human: ${message.content}`;
+    } else {
+      acc += `Assistant: ${message.content}`;
+    }
+    return acc;
+  }, "");
+}
+
+export function toToolDescriptions(tools: ToolMetadata[]): string {
+  const toolsObj = tools.reduce<Record<string, string>>((acc, tool) => {
+    acc[tool.name] = tool.description;
+    return acc;
+  }, {});
+
+  return JSON.stringify(toolsObj, null, 4);
+}

packages/core/src/utils/object-entries.ts

@@ -0,0 +1,14 @@
+type ObjectEntries<T extends Record<string, any>> = {
+  [K in keyof T]: [K, T[K]];
+}[keyof T][];
+
+/**
+ * Type safe version of `Object.entries`
+ */
+export function objectEntries<T extends Record<string, any>>(
+  obj: T,
+): ObjectEntries<{
+  [K in keyof T]-?: NonNullable<T[K]>;
+}> {
+  return Object.entries(obj);
+}

packages/core/tests/package.json

@@ -7,6 +7,6 @@
   },
   "devDependencies": {
     "@llamaindex/core": "workspace:*",
-    "vitest": "^2.0.2"
+    "vitest": "^2.0.5"
   }
 }

packages/core/tests/prompts.test.ts

@@ -0,0 +1,161 @@
+import { PromptTemplate, type StringTemplate } from "@llamaindex/core/prompts";
+import type { BaseOutputParser } from "@llamaindex/core/schema";
+import { describe, expect, expectTypeOf, test } from "vitest";
+
+describe("type system", () => {
+  test("StringTemplate", () => {
+    {
+      type Test = StringTemplate<["var1", "var2"]>;
+      expectTypeOf<"{var1}{var2}">().toMatchTypeOf<Test>();
+      expectTypeOf<"{var1}">().not.toMatchTypeOf<Test>();
+      expectTypeOf<"{var1} var2">().not.toMatchTypeOf<Test>();
+      expectTypeOf<"{var2}{var1}">().toMatchTypeOf<Test>();
+    }
+    {
+      const arr = ["var1", "var2"] as const;
+      type Test = StringTemplate<typeof arr>;
+      expectTypeOf<"{var1}{var2}">().toMatchTypeOf<Test>();
+      expectTypeOf<"{var1}">().not.toMatchTypeOf<Test>();
+      expectTypeOf<"{var1} var2">().not.toMatchTypeOf<Test>();
+      expectTypeOf<"{var2}{var1}">().toMatchTypeOf<Test>();
+    }
+    {
+      const template =
+        `Act as a natural language processing software. Analyze the given text and return me only a parsable and minified JSON object.
+
+
+Here's the JSON Object structure:
+{
+  "key1": /* Some instructions */,
+  "key2": /* Some instructions */,
+}
+
+Here are the rules you must follow:
+- You MUST return a valid, parsable JSON object.
+- More rules…
+
+Here are some examples to help you out:
+- Example 1…
+- Example 2…
+
+Text: {selection}
+
+JSON Data:` as const;
+      type Test = StringTemplate<["selection"]>;
+      expectTypeOf(template).toMatchTypeOf<Test>();
+    }
+    {
+      // matrix
+      type Test = StringTemplate<["a", "b", "c"]>;
+      expectTypeOf<"{a}{b}{c}">().toMatchTypeOf<Test>();
+      expectTypeOf<"{a}{c}{b}">().toMatchTypeOf<Test>();
+      expectTypeOf<"{b}{a}{c}">().toMatchTypeOf<Test>();
+      expectTypeOf<"{b}{c}{a}">().toMatchTypeOf<Test>();
+      expectTypeOf<"{c}{a}{b}">().toMatchTypeOf<Test>();
+      expectTypeOf<"{c}{b}{a}">().toMatchTypeOf<Test>();
+    }
+  });
+
+  test("PromptTemplate", () => {
+    {
+      new PromptTemplate({
+        // @ts-expect-error
+        template: "",
+        templateVars: ["var1"],
+      });
+    }
+    {
+      new PromptTemplate({
+        template: "something{var1}",
+        templateVars: ["var1"],
+      });
+    }
+    {
+      new PromptTemplate({
+        // @ts-expect-error
+        template: "{var1 }",
+        templateVars: ["var1"],
+      });
+    }
+    {
+      // in this case, type won't be inferred
+      const template = "{var2}";
+      const templateVars = ["var1"];
+      new PromptTemplate({
+        template,
+        templateVars,
+      });
+    }
+    {
+      const template = "{var2}" as const;
+      const templateVars = ["var1"] as const;
+      new PromptTemplate({
+        // @ts-expect-error
+        template,
+        templateVars,
+      });
+    }
+    {
+      const prompt = new PromptTemplate({
+        template: "hello {text} {foo}",
+        templateVars: ["text", "foo"],
+      });
+
+      prompt.partialFormat({
+        foo: "bar",
+        goo: "baz",
+      });
+    }
+  });
+});
+
+describe("PromptTemplate", () => {
+  test("basic usage", () => {
+    {
+      const template = "hello {text} {foo}";
+      const prompt = new PromptTemplate({
+        template,
+      });
+      const partialPrompt = prompt.partialFormat({
+        foo: "bar",
+      });
+      expect(partialPrompt).instanceof(PromptTemplate);
+      expect(
+        partialPrompt.format({
+          text: "world",
+        }),
+      ).toBe("hello world bar");
+    }
+  });
+  test("should partially format and fully format a prompt", () => {
+    const prompt = new PromptTemplate({
+      template: "hello {text} {foo}",
+      templateVars: ["text", "foo"],
+    });
+
+    const partialPrompt = prompt.partialFormat({ foo: "bar" });
+    expect(partialPrompt).toBeInstanceOf(PromptTemplate);
+    expect(partialPrompt.format({ text: "world" })).toBe("hello world bar");
+  });
+
+  test("should use output parser in formatting", () => {
+    const outputParser: BaseOutputParser = {
+      parse(output: string) {
+        return { output: output };
+      },
+
+      format(query: string) {
+        return `${query}\noutput_instruction`;
+      },
+    };
+
+    const prompt = new PromptTemplate({
+      template: "hello {text} {foo}",
+      templateVars: ["text", "foo"],
+      outputParser: outputParser,
+    });
+
+    const formatted = prompt.format({ text: "world", foo: "bar" });
+    expect(formatted).toBe("hello world bar\noutput_instruction");
+  });
+});

packages/core/tests/prompts/mixin.test.ts

@@ -1,45 +1,52 @@
-import { PromptMixin } from "llamaindex/prompts/index";
+import {
+  type ModuleRecord,
+  PromptMixin,
+  PromptTemplate,
+} from "@llamaindex/core/prompts";
 import { describe, expect, it } from "vitest";
 
-type MockPrompt = {
-  context: string;
-  query: string;
-};
+const mockPrompt = new PromptTemplate({
+  templateVars: ["context", "query"],
+  template: `context: {context} query: {query}`,
+});
 
-const mockPrompt = ({ context, query }: MockPrompt) =>
-  `context: ${context} query: ${query}`;
+const mockPrompt2 = new PromptTemplate({
+  templateVars: ["context", "query"],
+  template: `query: {query} context: {context}`,
+});
 
-const mockPrompt2 = ({ context, query }: MockPrompt) =>
-  `query: ${query} context: ${context}`;
-
-type MockPromptFunction = typeof mockPrompt;
+type MockPrompt = typeof mockPrompt;
 
 class MockObject2 extends PromptMixin {
-  _prompt_dict_2: MockPromptFunction;
+  _prompt_dict_2: MockPrompt;
 
   constructor() {
     super();
     this._prompt_dict_2 = mockPrompt;
   }
 
-  protected _getPrompts(): { [x: string]: MockPromptFunction } {
+  protected _getPrompts() {
     return {
       abc: this._prompt_dict_2,
     };
   }
 
-  _updatePrompts(promptsDict: { [x: string]: MockPromptFunction }): void {
-    if ("abc" in promptsDict) {
-      this._prompt_dict_2 = promptsDict["abc"];
+  protected _updatePrompts(prompts: { abc: MockPrompt }): void {
+    if ("abc" in prompts) {
+      this._prompt_dict_2 = prompts["abc"];
     }
   }
+
+  protected _getPromptModules(): ModuleRecord {
+    return {};
+  }
 }
 
 class MockObject1 extends PromptMixin {
   mockObject2: MockObject2;
 
-  fooPrompt: MockPromptFunction;
-  barPrompt: MockPromptFunction;
+  fooPrompt: MockPrompt;
+  barPrompt: MockPrompt;
 
   constructor() {
     super();
@@ -76,14 +83,14 @@ describe("PromptMixin", () => {
     const prompts = mockObj1.getPrompts();
 
     expect(
-      mockObj1.fooPrompt({
+      mockObj1.fooPrompt.format({
         context: "{foo}",
         query: "{foo}",
       }),
     ).toEqual("context: {foo} query: {foo}");
 
     expect(
-      mockObj1.barPrompt({
+      mockObj1.barPrompt.format({
         context: "{foo} {bar}",
         query: "{foo} {bar}",
       }),
@@ -103,18 +110,22 @@ describe("PromptMixin", () => {
     const mockObj1 = new MockObject1();
 
     expect(
-      mockObj1.barPrompt({
+      mockObj1.barPrompt.format({
         context: "{foo} {bar}",
         query: "{foo} {bar}",
       }),
-    ).toEqual(mockPrompt({ context: "{foo} {bar}", query: "{foo} {bar}" }));
+    ).toEqual(
+      mockPrompt.format({ context: "{foo} {bar}", query: "{foo} {bar}" }),
+    );
 
     expect(
-      mockObj1.mockObject2._prompt_dict_2({
+      mockObj1.mockObject2._prompt_dict_2.format({
         context: "{bar} testing",
         query: "{bar} testing",
       }),
-    ).toEqual(mockPrompt({ context: "{bar} testing", query: "{bar} testing" }));
+    ).toEqual(
+      mockPrompt.format({ context: "{bar} testing", query: "{bar} testing" }),
+    );
 
     mockObj1.updatePrompts({
       bar: mockPrompt2,
@@ -122,18 +133,20 @@ describe("PromptMixin", () => {
     });
 
     expect(
-      mockObj1.barPrompt({
+      mockObj1.barPrompt.format({
         context: "{foo} {bar}",
         query: "{bar} {foo}",
       }),
-    ).toEqual(mockPrompt2({ context: "{foo} {bar}", query: "{bar} {foo}" }));
+    ).toEqual(
+      mockPrompt2.format({ context: "{foo} {bar}", query: "{bar} {foo}" }),
+    );
     expect(
-      mockObj1.mockObject2._prompt_dict_2({
+      mockObj1.mockObject2._prompt_dict_2.format({
         context: "{bar} testing",
         query: "{bar} testing",
       }),
     ).toEqual(
-      mockPrompt2({ context: "{bar} testing", query: "{bar} testing" }),
+      mockPrompt2.format({ context: "{bar} testing", query: "{bar} testing" }),
     );
   });
 });

packages/env/CHANGELOG.md

@@ -1,5 +1,11 @@
 # @llamaindex/env
 
+## 0.1.10
+
+### Patch Changes
+
+- 4648da6: fix: wrong tiktoken version caused NextJs CL template run fail
+
 ## 0.1.9
 
 ### Patch Changes

packages/env/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/env",
   "description": "environment wrapper, supports all JS environment including node, deno, bun, edge runtime, and cloudflare worker",
-  "version": "0.1.9",
+  "version": "0.1.10",
   "type": "module",
   "types": "dist/type/index.d.ts",
   "main": "dist/cjs/index.js",
@@ -69,14 +69,14 @@
   "devDependencies": {
     "@aws-crypto/sha256-js": "^5.2.0",
     "@swc/cli": "^0.4.0",
-    "@swc/core": "^1.6.3",
+    "@swc/core": "^1.7.22",
     "concurrently": "^8.2.2",
     "pathe": "^1.1.2",
-    "vitest": "^2.0.2"
+    "vitest": "^2.0.5"
   },
   "dependencies": {
-    "@types/lodash": "^4.17.5",
-    "@types/node": "^20.12.11"
+    "@types/lodash": "^4.17.7",
+    "@types/node": "^22.5.1"
   },
   "peerDependencies": {
     "@aws-crypto/sha256-js": "^5.2.0",

packages/env/src/index.ts

@@ -16,11 +16,6 @@ import { createHash, randomUUID } from "node:crypto";
 import { EOL } from "node:os";
 import path from "node:path";
 import { Readable } from "node:stream";
-import {
-  ReadableStream,
-  TransformStream,
-  WritableStream,
-} from "node:stream/web";
 import { fileURLToPath } from "node:url";
 import { createWriteStream, fs } from "./fs/node.js";
 import type { SHA256 } from "./polyfill.js";
@@ -48,7 +43,4 @@ export {
   path,
   randomUUID,
   Readable,
-  ReadableStream,
-  TransformStream,
-  WritableStream,
 };

packages/env/src/polyfill.ts

@@ -46,12 +46,4 @@ export function randomUUID(): string {
   return crypto.randomUUID();
 }
 
-// @ts-expect-error
-const ReadableStream = globalThis.ReadableStream;
-// @ts-expect-error
-const TransformStream = globalThis.TransformStream;
-// @ts-expect-error
-const WritableStream = globalThis.WritableStream;
-
 export { AsyncLocalStorage, CustomEvent, getEnv, setEnvs } from "./utils.js";
-export { ReadableStream, TransformStream, WritableStream };

packages/experimental/CHANGELOG.md

@@ -1,5 +1,55 @@
 # @llamaindex/experimental
 
+## 0.0.72
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
+## 0.0.71
+
+### Patch Changes
+
+- Updated dependencies [ae1149f]
+- Updated dependencies [2411c9f]
+- Updated dependencies [e8f229c]
+- Updated dependencies [11b3856]
+- Updated dependencies [83d7f41]
+- Updated dependencies [0148354]
+- Updated dependencies [1711f6d]
+  - llamaindex@0.5.21
+
+## 0.0.70
+
+### Patch Changes
+
+- Updated dependencies [d9d6c56]
+- Updated dependencies [22ff486]
+- Updated dependencies [eed0b04]
+  - llamaindex@0.5.20
+
+## 0.0.69
+
+### Patch Changes
+
+- Updated dependencies [fcbf183]
+  - llamaindex@0.5.19
+
+## 0.0.68
+
+### Patch Changes
+
+- Updated dependencies [8b66cf4]
+  - llamaindex@0.5.18
+
+## 0.0.67
+
+### Patch Changes
+
+- Updated dependencies [c654398]
+  - llamaindex@0.5.17
+
 ## 0.0.66
 
 ### Patch Changes

packages/experimental/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/experimental",
   "description": "Experimental package for LlamaIndexTS",
-  "version": "0.0.66",
+  "version": "0.0.72",
   "type": "module",
   "types": "dist/type/index.d.ts",
   "main": "dist/cjs/index.js",
@@ -57,14 +57,14 @@
   "devDependencies": {
     "@aws-crypto/sha256-js": "^5.2.0",
     "@swc/cli": "^0.4.0",
-    "@swc/core": "^1.6.3",
+    "@swc/core": "^1.7.22",
     "@types/jsonpath": "^0.2.4",
     "concurrently": "^8.2.2",
     "pathe": "^1.1.2"
   },
   "dependencies": {
-    "@types/lodash": "^4.17.5",
-    "@types/node": "^20.12.11",
+    "@types/lodash": "^4.17.7",
+    "@types/node": "^22.5.1",
     "jsonpath": "^1.1.1",
     "llamaindex": "workspace:*",
     "lodash": "^4.17.21"

packages/llamaindex/CHANGELOG.md

@@ -1,5 +1,66 @@
 # llamaindex
 
+## 0.5.22
+
+### Patch Changes
+
+- 4648da6: fix: wrong tiktoken version caused NextJs CL template run fail
+- Updated dependencies [4648da6]
+  - @llamaindex/env@0.1.10
+  - @llamaindex/core@0.1.11
+
+## 0.5.21
+
+### Patch Changes
+
+- ae1149f: feat: add JSON streaming to JSONReader
+- 2411c9f: Auto-create index for MongoDB vector store (if not exists)
+- e8f229c: Remove logging from MongoDB Atlas Vector Store
+- 11b3856: implement filters for MongoDBAtlasVectorSearch
+- 83d7f41: Fix database insertion for `PGVectorStore`
+
+  It will now:
+
+  - throw an error if there is an insertion error.
+  - Upsert documents with the same id.
+  - add all documents to the database as a single `INSERT` call (inside a transaction).
+
+- 0148354: refactor: prompt system
+
+  Add `PromptTemplate` module with strong type check.
+
+- 1711f6d: Export imageToDataUrl for using images in chat
+- Updated dependencies [0148354]
+  - @llamaindex/core@0.1.10
+
+## 0.5.20
+
+### Patch Changes
+
+- d9d6c56: Add support for MetadataFilters for PostgreSQL
+- 22ff486: Add tiktoken WASM to withLlamaIndex
+- eed0b04: fix: use LLM metadata mode for generating context of ContextChatEngine
+
+## 0.5.19
+
+### Patch Changes
+
+- fcbf183: implement llamacloud file service
+
+## 0.5.18
+
+### Patch Changes
+
+- 8b66cf4: feat: support organization id in llamacloud index
+- Updated dependencies [e27e7dd]
+  - @llamaindex/core@0.1.9
+
+## 0.5.17
+
+### Patch Changes
+
+- c654398: Implement Weaviate Vector Store in TS
+
 ## 0.5.16
 
 ### Patch Changes

packages/llamaindex/e2e/examples/cloudflare-worker-agent/CHANGELOG.md

@@ -1,5 +1,55 @@
 # @llamaindex/cloudflare-worker-agent-test
 
+## 0.0.56
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
+## 0.0.55
+
+### Patch Changes
+
+- Updated dependencies [ae1149f]
+- Updated dependencies [2411c9f]
+- Updated dependencies [e8f229c]
+- Updated dependencies [11b3856]
+- Updated dependencies [83d7f41]
+- Updated dependencies [0148354]
+- Updated dependencies [1711f6d]
+  - llamaindex@0.5.21
+
+## 0.0.54
+
+### Patch Changes
+
+- Updated dependencies [d9d6c56]
+- Updated dependencies [22ff486]
+- Updated dependencies [eed0b04]
+  - llamaindex@0.5.20
+
+## 0.0.53
+
+### Patch Changes
+
+- Updated dependencies [fcbf183]
+  - llamaindex@0.5.19
+
+## 0.0.52
+
+### Patch Changes
+
+- Updated dependencies [8b66cf4]
+  - llamaindex@0.5.18
+
+## 0.0.51
+
+### Patch Changes
+
+- Updated dependencies [c654398]
+  - llamaindex@0.5.17
+
 ## 0.0.50
 
 ### Patch Changes

packages/llamaindex/e2e/examples/cloudflare-worker-agent/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/cloudflare-worker-agent-test",
-  "version": "0.0.50",
+  "version": "0.0.56",
   "type": "module",
   "private": true,
   "scripts": {
@@ -12,13 +12,13 @@
     "cf-typegen": "wrangler types"
   },
   "devDependencies": {
-    "@cloudflare/vitest-pool-workers": "^0.4.10",
-    "@cloudflare/workers-types": "^4.20240620.0",
+    "@cloudflare/vitest-pool-workers": "^0.4.27",
+    "@cloudflare/workers-types": "^4.20240821.1",
     "@vitest/runner": "1.5.3",
     "@vitest/snapshot": "1.5.3",
-    "typescript": "^5.5.3",
+    "typescript": "^5.5.4",
     "vitest": "1.5.3",
-    "wrangler": "^3.63.2"
+    "wrangler": "^3.73.0"
   },
   "dependencies": {
     "llamaindex": "workspace:*"

packages/llamaindex/e2e/examples/cloudflare-worker-agent/src/index.ts

@@ -18,14 +18,12 @@ export default {
     console.log(2);
     const textEncoder = new TextEncoder();
     const response = responseStream.pipeThrough<Uint8Array>(
-      // @ts-expect-error: see https://github.com/cloudflare/workerd/issues/2067
       new TransformStream({
         transform: (chunk, controller) => {
           controller.enqueue(textEncoder.encode(chunk.delta));
         },
       }),
     );
-    // @ts-expect-error: see https://github.com/cloudflare/workerd/issues/2067
     return new Response(response);
   },
 };

packages/llamaindex/e2e/examples/nextjs-agent/CHANGELOG.md

@@ -1,5 +1,55 @@
 # @llamaindex/next-agent-test
 
+## 0.1.56
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
+## 0.1.55
+
+### Patch Changes
+
+- Updated dependencies [ae1149f]
+- Updated dependencies [2411c9f]
+- Updated dependencies [e8f229c]
+- Updated dependencies [11b3856]
+- Updated dependencies [83d7f41]
+- Updated dependencies [0148354]
+- Updated dependencies [1711f6d]
+  - llamaindex@0.5.21
+
+## 0.1.54
+
+### Patch Changes
+
+- Updated dependencies [d9d6c56]
+- Updated dependencies [22ff486]
+- Updated dependencies [eed0b04]
+  - llamaindex@0.5.20
+
+## 0.1.53
+
+### Patch Changes
+
+- Updated dependencies [fcbf183]
+  - llamaindex@0.5.19
+
+## 0.1.52
+
+### Patch Changes
+
+- Updated dependencies [8b66cf4]
+  - llamaindex@0.5.18
+
+## 0.1.51
+
+### Patch Changes
+
+- Updated dependencies [c654398]
+  - llamaindex@0.5.17
+
 ## 0.1.50
 
 ### Patch Changes

packages/llamaindex/e2e/examples/nextjs-agent/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/next-agent-test",
-  "version": "0.1.50",
+  "version": "0.1.56",
   "private": true,
   "scripts": {
     "dev": "next dev",
@@ -9,20 +9,20 @@
     "lint": "next lint"
   },
   "dependencies": {
-    "ai": "^3.2.1",
+    "ai": "^3.3.21",
     "llamaindex": "workspace:*",
-    "next": "14.2.5",
+    "next": "14.2.7",
     "react": "18.3.1",
     "react-dom": "18.3.1"
   },
   "devDependencies": {
-    "@types/node": "^20.12.11",
-    "@types/react": "^18.3.3",
+    "@types/node": "^22.5.1",
+    "@types/react": "^18.3.5",
     "@types/react-dom": "^18.3.0",
-    "eslint": "^8.57.0",
-    "eslint-config-next": "14.2.5",
-    "postcss": "^8",
-    "tailwindcss": "^3.4.4",
-    "typescript": "^5.5.3"
+    "eslint": "8.57.0",
+    "eslint-config-next": "14.2.7",
+    "postcss": "^8.4.41",
+    "tailwindcss": "^3.4.10",
+    "typescript": "^5.5.4"
   }
 }

packages/llamaindex/e2e/examples/nextjs-edge-runtime/CHANGELOG.md

@@ -1,5 +1,55 @@
 # test-edge-runtime
 
+## 0.1.55
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
+## 0.1.54
+
+### Patch Changes
+
+- Updated dependencies [ae1149f]
+- Updated dependencies [2411c9f]
+- Updated dependencies [e8f229c]
+- Updated dependencies [11b3856]
+- Updated dependencies [83d7f41]
+- Updated dependencies [0148354]
+- Updated dependencies [1711f6d]
+  - llamaindex@0.5.21
+
+## 0.1.53
+
+### Patch Changes
+
+- Updated dependencies [d9d6c56]
+- Updated dependencies [22ff486]
+- Updated dependencies [eed0b04]
+  - llamaindex@0.5.20
+
+## 0.1.52
+
+### Patch Changes
+
+- Updated dependencies [fcbf183]
+  - llamaindex@0.5.19
+
+## 0.1.51
+
+### Patch Changes
+
+- Updated dependencies [8b66cf4]
+  - llamaindex@0.5.18
+
+## 0.1.50
+
+### Patch Changes
+
+- Updated dependencies [c654398]
+  - llamaindex@0.5.17
+
 ## 0.1.49
 
 ### Patch Changes

packages/llamaindex/e2e/examples/nextjs-edge-runtime/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/nextjs-edge-runtime-test",
-  "version": "0.1.49",
+  "version": "0.1.55",
   "private": true,
   "scripts": {
     "dev": "next dev",
@@ -9,14 +9,14 @@
   },
   "dependencies": {
     "llamaindex": "workspace:*",
-    "next": "14.2.5",
+    "next": "14.2.7",
     "react": "^18.3.1",
     "react-dom": "^18.3.1"
   },
   "devDependencies": {
-    "@types/node": "^20.12.11",
-    "@types/react": "^18.3.3",
+    "@types/node": "^22.5.1",
+    "@types/react": "^18.3.5",
     "@types/react-dom": "^18.3.0",
-    "typescript": "^5.5.3"
+    "typescript": "^5.5.4"
   }
 }

packages/llamaindex/e2e/examples/nextjs-node-runtime/CHANGELOG.md

@@ -1,5 +1,55 @@
 # @llamaindex/next-node-runtime
 
+## 0.0.37
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
+## 0.0.36
+
+### Patch Changes
+
+- Updated dependencies [ae1149f]
+- Updated dependencies [2411c9f]
+- Updated dependencies [e8f229c]
+- Updated dependencies [11b3856]
+- Updated dependencies [83d7f41]
+- Updated dependencies [0148354]
+- Updated dependencies [1711f6d]
+  - llamaindex@0.5.21
+
+## 0.0.35
+
+### Patch Changes
+
+- Updated dependencies [d9d6c56]
+- Updated dependencies [22ff486]
+- Updated dependencies [eed0b04]
+  - llamaindex@0.5.20
+
+## 0.0.34
+
+### Patch Changes
+
+- Updated dependencies [fcbf183]
+  - llamaindex@0.5.19
+
+## 0.0.33
+
+### Patch Changes
+
+- Updated dependencies [8b66cf4]
+  - llamaindex@0.5.18
+
+## 0.0.32
+
+### Patch Changes
+
+- Updated dependencies [c654398]
+  - llamaindex@0.5.17
+
 ## 0.0.31
 
 ### Patch Changes

packages/llamaindex/e2e/examples/nextjs-node-runtime/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/next-node-runtime-test",
-  "version": "0.0.31",
+  "version": "0.0.37",
   "private": true,
   "scripts": {
     "dev": "next dev",
@@ -10,18 +10,18 @@
   },
   "dependencies": {
     "llamaindex": "workspace:*",
-    "next": "14.2.5",
+    "next": "14.2.7",
     "react": "18.3.1",
     "react-dom": "18.3.1"
   },
   "devDependencies": {
-    "@types/node": "^20.12.11",
-    "@types/react": "^18.3.3",
+    "@types/node": "^22.5.1",
+    "@types/react": "^18.3.5",
     "@types/react-dom": "^18.3.0",
-    "eslint": "^8.57.0",
-    "eslint-config-next": "14.2.5",
-    "postcss": "^8",
-    "tailwindcss": "^3.4.4",
-    "typescript": "^5.5.3"
+    "eslint": "8.57.0",
+    "eslint-config-next": "14.2.7",
+    "postcss": "^8.4.41",
+    "tailwindcss": "^3.4.10",
+    "typescript": "^5.5.4"
   }
 }

packages/llamaindex/e2e/examples/waku-query-engine/CHANGELOG.md

@@ -1,5 +1,55 @@
 # @llamaindex/waku-query-engine-test
 
+## 0.0.56
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
+## 0.0.55
+
+### Patch Changes
+
+- Updated dependencies [ae1149f]
+- Updated dependencies [2411c9f]
+- Updated dependencies [e8f229c]
+- Updated dependencies [11b3856]
+- Updated dependencies [83d7f41]
+- Updated dependencies [0148354]
+- Updated dependencies [1711f6d]
+  - llamaindex@0.5.21
+
+## 0.0.54
+
+### Patch Changes
+
+- Updated dependencies [d9d6c56]
+- Updated dependencies [22ff486]
+- Updated dependencies [eed0b04]
+  - llamaindex@0.5.20
+
+## 0.0.53
+
+### Patch Changes
+
+- Updated dependencies [fcbf183]
+  - llamaindex@0.5.19
+
+## 0.0.52
+
+### Patch Changes
+
+- Updated dependencies [8b66cf4]
+  - llamaindex@0.5.18
+
+## 0.0.51
+
+### Patch Changes
+
+- Updated dependencies [c654398]
+  - llamaindex@0.5.17
+
 ## 0.0.50
 
 ### Patch Changes

packages/llamaindex/e2e/examples/waku-query-engine/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/waku-query-engine-test",
-  "version": "0.0.50",
+  "version": "0.0.56",
   "type": "module",
   "private": true,
   "scripts": {
@@ -10,16 +10,16 @@
   },
   "dependencies": {
     "llamaindex": "workspace:*",
-    "react": "19.0.0-beta-e7d213dfb0-20240507",
-    "react-dom": "19.0.0-beta-e7d213dfb0-20240507",
-    "react-server-dom-webpack": "19.0.0-beta-e7d213dfb0-20240507",
-    "waku": "0.20.2"
+    "react": "19.0.0-rc-7771d3a7-20240827",
+    "react-dom": "19.0.0-rc-7771d3a7-20240827",
+    "react-server-dom-webpack": "19.0.0-rc-7771d3a7-20240827",
+    "waku": "0.21.1"
   },
   "devDependencies": {
-    "@types/react": "18.3.3",
+    "@types/react": "18.3.5",
     "@types/react-dom": "18.3.0",
-    "autoprefixer": "10.4.19",
-    "tailwindcss": "3.4.4",
-    "typescript": "5.5.3"
+    "autoprefixer": "10.4.20",
+    "tailwindcss": "3.4.10",
+    "typescript": "5.5.4"
   }
 }

packages/llamaindex/e2e/examples/waku-query-engine/tsconfig.json

@@ -10,6 +10,7 @@
     "noUncheckedIndexedAccess": true,
     "exactOptionalPropertyTypes": true,
     "types": ["react/experimental"],
-    "jsx": "react-jsx"
+    "jsx": "react-jsx",
+    "outDir": "./lib"
   }
 }

packages/llamaindex/e2e/package.json

@@ -11,9 +11,9 @@
   "devDependencies": {
     "@faker-js/faker": "^8.4.1",
     "@llamaindex/core": "workspace:*",
-    "@types/node": "^20.12.11",
+    "@types/node": "^22.5.1",
     "consola": "^3.2.3",
     "llamaindex": "workspace:*",
-    "tsx": "^4.15.6"
+    "tsx": "^4.19.0"
   }
 }

packages/llamaindex/e2e/tsconfig.json

@@ -5,7 +5,7 @@
     "module": "node16",
     "moduleResolution": "node16",
     "target": "ESNext",
-    "lib": ["ES2022"],
+    "lib": ["ES2022", "DOM.AsyncIterable"],
     "types": ["node"]
   },
   "include": ["./node", "./mock-module.js", "./mock-register.js", "./fixtures"],

packages/llamaindex/package.json

@@ -1,6 +1,6 @@
 {
   "name": "llamaindex",
-  "version": "0.5.16",
+  "version": "0.5.22",
   "license": "MIT",
   "type": "module",
   "keywords": [
@@ -20,42 +20,43 @@
     "llamaindex"
   ],
   "dependencies": {
-    "@anthropic-ai/sdk": "0.21.1",
+    "@anthropic-ai/sdk": "0.27.1",
     "@aws-crypto/sha256-js": "^5.2.0",
-    "@azure/identity": "^4.2.1",
-    "@datastax/astra-db-ts": "^1.2.1",
+    "@azure/identity": "^4.4.1",
+    "@datastax/astra-db-ts": "^1.4.1",
     "@discordjs/rest": "^2.3.0",
-    "@google-cloud/vertexai": "^1.2.0",
+    "@discoveryjs/json-ext": "^0.6.1",
+    "@google-cloud/vertexai": "1.2.0",
     "@google/generative-ai": "0.12.0",
-    "@grpc/grpc-js": "^1.10.11",
-    "@huggingface/inference": "^2.7.0",
+    "@grpc/grpc-js": "^1.11.1",
+    "@huggingface/inference": "^2.8.0",
     "@llamaindex/cloud": "workspace:*",
     "@llamaindex/core": "workspace:*",
     "@llamaindex/env": "workspace:*",
-    "@mistralai/mistralai": "^0.5.0",
+    "@mistralai/mistralai": "^1.0.4",
     "@mixedbread-ai/sdk": "^2.2.11",
-    "@pinecone-database/pinecone": "^2.2.2",
-    "@qdrant/js-client-rest": "^1.9.0",
-    "@types/lodash": "^4.17.4",
-    "@types/node": "^20.14.5",
+    "@pinecone-database/pinecone": "^3.0.2",
+    "@qdrant/js-client-rest": "^1.11.0",
+    "@types/lodash": "^4.17.7",
+    "@types/node": "^22.5.1",
     "@types/papaparse": "^5.3.14",
-    "@types/pg": "^8.11.6",
+    "@types/pg": "^8.11.8",
     "@xenova/transformers": "^2.17.2",
-    "@zilliz/milvus2-sdk-node": "^2.4.4",
-    "ajv": "^8.16.0",
-    "assemblyai": "^4.6.0",
+    "@zilliz/milvus2-sdk-node": "^2.4.6",
+    "ajv": "^8.17.1",
+    "assemblyai": "^4.7.0",
     "chromadb": "1.8.1",
-    "cohere-ai": "7.10.6",
-    "discord-api-types": "^0.37.92",
-    "groq-sdk": "^0.5.0",
-    "js-tiktoken": "^1.0.12",
+    "cohere-ai": "7.13.0",
+    "discord-api-types": "^0.37.98",
+    "groq-sdk": "^0.6.1",
+    "js-tiktoken": "^1.0.14",
     "lodash": "^4.17.21",
     "magic-bytes.js": "^1.10.0",
     "mammoth": "^1.7.2",
     "md-utils-ts": "^2.0.0",
     "mongodb": "^6.7.0",
     "notion-md-crawler": "^1.0.0",
-    "openai": "^4.55.3",
+    "openai": "^4.57.0",
     "papaparse": "^5.4.1",
     "pathe": "^1.1.2",
     "pg": "^8.12.0",
@@ -65,6 +66,7 @@
     "string-strip-html": "^13.4.8",
     "tiktoken": "^1.0.15",
     "unpdf": "^0.11.0",
+    "weaviate-client": "^3.1.4",
     "wikipedia": "^2.1.2",
     "wink-nlp": "^2.3.0",
     "zod": "^3.23.8"
@@ -80,10 +82,10 @@
   "devDependencies": {
     "@notionhq/client": "^2.2.15",
     "@swc/cli": "^0.4.0",
-    "@swc/core": "^1.6.3",
+    "@swc/core": "^1.7.22",
     "concurrently": "^8.2.2",
     "glob": "^11.0.0",
-    "typescript": "^5.5.3"
+    "typescript": "^5.5.4"
   },
   "engines": {
     "node": ">=18.0.0"

packages/llamaindex/src/ChatHistory.ts

@@ -1,8 +1,10 @@
 import type { ChatMessage, LLM, MessageType } from "@llamaindex/core/llms";
-import { extractText } from "@llamaindex/core/utils";
+import {
+  defaultSummaryPrompt,
+  type SummaryPrompt,
+} from "@llamaindex/core/prompts";
+import { extractText, messagesToHistory } from "@llamaindex/core/utils";
 import { tokenizers, type Tokenizer } from "@llamaindex/env";
-import type { SummaryPrompt } from "./Prompt.js";
-import { defaultSummaryPrompt, messagesToHistoryStr } from "./Prompt.js";
 import { OpenAI } from "./llm/openai.js";
 
 /**
@@ -106,8 +108,8 @@ export class SummaryChatHistory extends ChatHistory {
     do {
       promptMessages = [
         {
-          content: this.summaryPrompt({
-            context: messagesToHistoryStr(messagesToSummarize),
+          content: this.summaryPrompt.format({
+            context: messagesToHistory(messagesToSummarize),
           }),
           role: "user" as MessageType,
           options: {},

packages/llamaindex/src/OutputParser.ts

@@ -1,5 +1,6 @@
+import type { BaseOutputParser } from "@llamaindex/core/schema";
 import type { SubQuestion } from "./engines/query/types.js";
-import type { BaseOutputParser, StructuredOutput } from "./types.js";
+import type { StructuredOutput } from "./types.js";
 
 /**
  * Error class for output parsing. Due to the nature of LLMs, anytime we use LLM

packages/llamaindex/src/Prompt.ts

@@ -1,411 +0,0 @@
-import type { ChatMessage, ToolMetadata } from "@llamaindex/core/llms";
-import type { SubQuestion } from "./engines/query/types.js";
-
-/**
- * A SimplePrompt is a function that takes a dictionary of inputs and returns a string.
- * NOTE this is a different interface compared to LlamaIndex Python
- * NOTE 2: we default to empty string to make it easy to calculate prompt sizes
- */
-export type SimplePrompt = (
-  input: Record<string, string | undefined>,
-) => string;
-
-/*
-DEFAULT_TEXT_QA_PROMPT_TMPL = (
-    "Context information is below.\n"
-    "---------------------\n"
-    "{context_str}\n"
-    "---------------------\n"
-    "Given the context information and not prior knowledge, "
-    "answer the query.\n"
-    "Query: {query_str}\n"
-    "Answer: "
-)
-*/
-
-export const defaultTextQaPrompt = ({ context = "", query = "" }) => {
-  return `Context information is below.
----------------------
-${context}
----------------------
-Given the context information and not prior knowledge, answer the query.
-Query: ${query}
-Answer:`;
-};
-
-export type TextQaPrompt = typeof defaultTextQaPrompt;
-
-export const anthropicTextQaPrompt: TextQaPrompt = ({
-  context = "",
-  query = "",
-}) => {
-  return `Context information:
-<context>
-${context}
-</context>
-Given the context information and not prior knowledge, answer the query.
-Query: ${query}`;
-};
-
-/*
-DEFAULT_SUMMARY_PROMPT_TMPL = (
-    "Write a summary of the following. Try to use only the "
-    "information provided. "
-    "Try to include as many key details as possible.\n"
-    "\n"
-    "\n"
-    "{context_str}\n"
-    "\n"
-    "\n"
-    'SUMMARY:"""\n'
-)
-*/
-
-export const defaultSummaryPrompt = ({ context = "" }) => {
-  return `Write a summary of the following. Try to use only the information provided. Try to include as many key details as possible.
-
-
-${context}
-
-
-SUMMARY:"""
-`;
-};
-
-export type SummaryPrompt = typeof defaultSummaryPrompt;
-
-export const anthropicSummaryPrompt: SummaryPrompt = ({ context = "" }) => {
-  return `Summarize the following text. Try to use only the information provided. Try to include as many key details as possible.
-<original-text>
-${context}
-</original-text>
-
-SUMMARY:
-`;
-};
-
-/*
-DEFAULT_REFINE_PROMPT_TMPL = (
-    "The original query is as follows: {query_str}\n"
-    "We have provided an existing answer: {existing_answer}\n"
-    "We have the opportunity to refine the existing answer "
-    "(only if needed) with some more context below.\n"
-    "------------\n"
-    "{context_msg}\n"
-    "------------\n"
-    "Given the new context, refine the original answer to better "
-    "answer the query. "
-    "If the context isn't useful, return the original answer.\n"
-    "Refined Answer: "
-)
-*/
-
-export const defaultRefinePrompt = ({
-  query = "",
-  existingAnswer = "",
-  context = "",
-}) => {
-  return `The original query is as follows: ${query}
-We have provided an existing answer: ${existingAnswer}
-We have the opportunity to refine the existing answer (only if needed) with some more context below.
-------------
-${context}
-------------
-Given the new context, refine the original answer to better answer the query. If the context isn't useful, return the original answer.
-Refined Answer:`;
-};
-
-export type RefinePrompt = typeof defaultRefinePrompt;
-
-/*
-DEFAULT_TREE_SUMMARIZE_TMPL = (
-  "Context information from multiple sources is below.\n"
-  "---------------------\n"
-  "{context_str}\n"
-  "---------------------\n"
-  "Given the information from multiple sources and not prior knowledge, "
-  "answer the query.\n"
-  "Query: {query_str}\n"
-  "Answer: "
-)
-*/
-
-export const defaultTreeSummarizePrompt = ({ context = "", query = "" }) => {
-  return `Context information from multiple sources is below.
----------------------
-${context}
----------------------
-Given the information from multiple sources and not prior knowledge, answer the query.
-Query: ${query}
-Answer:`;
-};
-
-export type TreeSummarizePrompt = typeof defaultTreeSummarizePrompt;
-
-export const defaultChoiceSelectPrompt = ({ context = "", query = "" }) => {
-  return `A list of documents is shown below. Each document has a number next to it along 
-with a summary of the document. A question is also provided.
-Respond with the numbers of the documents
-you should consult to answer the question, in order of relevance, as well
-as the relevance score. The relevance score is a number from 1-10 based on
-how relevant you think the document is to the question.
-Do not include any documents that are not relevant to the question.
-Example format:
-Document 1:
-<summary of document 1>
-
-Document 2:
-<summary of document 2>
-
-...
-
-Document 10:\n<summary of document 10>
-
-Question: <question>
-Answer:
-Doc: 9, Relevance: 7
-Doc: 3, Relevance: 4
-Doc: 7, Relevance: 3
-
-Let's try this now:
-
-${context}
-Question: ${query}
-Answer:`;
-};
-
-export type ChoiceSelectPrompt = typeof defaultChoiceSelectPrompt;
-
-/*
-PREFIX = """\
-Given a user question, and a list of tools, output a list of relevant sub-questions \
-that when composed can help answer the full user question:
-
-"""
-
-
-example_query_str = (
-    "Compare and contrast the revenue growth and EBITDA of Uber and Lyft for year 2021"
-)
-example_tools = [
-    ToolMetadata(
-        name="uber_10k",
-        description="Provides information about Uber financials for year 2021",
-    ),
-    ToolMetadata(
-        name="lyft_10k",
-        description="Provides information about Lyft financials for year 2021",
-    ),
-]
-example_tools_str = build_tools_text(example_tools)
-example_output = [
-    SubQuestion(
-        sub_question="What is the revenue growth of Uber", tool_name="uber_10k"
-    ),
-    SubQuestion(sub_question="What is the EBITDA of Uber", tool_name="uber_10k"),
-    SubQuestion(
-        sub_question="What is the revenue growth of Lyft", tool_name="lyft_10k"
-    ),
-    SubQuestion(sub_question="What is the EBITDA of Lyft", tool_name="lyft_10k"),
-]
-example_output_str = json.dumps([x.dict() for x in example_output], indent=4)
-
-EXAMPLES = (
-    """\
-# Example 1
-<Tools>
-```json
-{tools_str}
-```
-
-<User Question>
-{query_str}
-
-
-<Output>
-```json
-{output_str}
-```
-
-""".format(
-        query_str=example_query_str,
-        tools_str=example_tools_str,
-        output_str=example_output_str,
-    )
-    .replace("{", "{{")
-    .replace("}", "}}")
-)
-
-SUFFIX = """\
-# Example 2
-<Tools>
-```json
-{tools_str}
-```
-
-<User Question>
-{query_str}
-
-<Output>
-"""
-
-DEFAULT_SUB_QUESTION_PROMPT_TMPL = PREFIX + EXAMPLES + SUFFIX
-*/
-
-export function buildToolsText(tools: ToolMetadata[]) {
-  const toolsObj = tools.reduce<Record<string, string>>((acc, tool) => {
-    acc[tool.name] = tool.description;
-    return acc;
-  }, {});
-
-  return JSON.stringify(toolsObj, null, 4);
-}
-
-const exampleTools: ToolMetadata[] = [
-  {
-    name: "uber_10k",
-    description: "Provides information about Uber financials for year 2021",
-  },
-  {
-    name: "lyft_10k",
-    description: "Provides information about Lyft financials for year 2021",
-  },
-];
-
-const exampleQueryStr = `Compare and contrast the revenue growth and EBITDA of Uber and Lyft for year 2021`;
-
-const exampleOutput: SubQuestion[] = [
-  {
-    subQuestion: "What is the revenue growth of Uber",
-    toolName: "uber_10k",
-  },
-  {
-    subQuestion: "What is the EBITDA of Uber",
-    toolName: "uber_10k",
-  },
-  {
-    subQuestion: "What is the revenue growth of Lyft",
-    toolName: "lyft_10k",
-  },
-  {
-    subQuestion: "What is the EBITDA of Lyft",
-    toolName: "lyft_10k",
-  },
-];
-
-export const defaultSubQuestionPrompt = ({ toolsStr = "", queryStr = "" }) => {
-  return `Given a user question, and a list of tools, output a list of relevant sub-questions that when composed can help answer the full user question:
-
-# Example 1
-<Tools>
-\`\`\`json
-${buildToolsText(exampleTools)}
-\`\`\`
-
-<User Question>
-${exampleQueryStr}
-
-<Output>
-\`\`\`json
-${JSON.stringify(exampleOutput, null, 4)}
-\`\`\`
-
-# Example 2
-<Tools>
-\`\`\`json
-${toolsStr}
-\`\`\`
-
-<User Question>
-${queryStr}
-
-<Output>
-`;
-};
-
-export type SubQuestionPrompt = typeof defaultSubQuestionPrompt;
-
-// DEFAULT_TEMPLATE = """\
-// Given a conversation (between Human and Assistant) and a follow up message from Human, \
-// rewrite the message to be a standalone question that captures all relevant context \
-// from the conversation.
-
-// <Chat History>
-// {chat_history}
-
-// <Follow Up Message>
-// {question}
-
-// <Standalone question>
-// """
-
-export const defaultCondenseQuestionPrompt = ({
-  chatHistory = "",
-  question = "",
-}) => {
-  return `Given a conversation (between Human and Assistant) and a follow up message from Human, rewrite the message to be a standalone question that captures all relevant context from the conversation.
-
-<Chat History>
-${chatHistory}
-
-<Follow Up Message>
-${question}
-
-<Standalone question>
-`;
-};
-
-export type CondenseQuestionPrompt = typeof defaultCondenseQuestionPrompt;
-
-export function messagesToHistoryStr(messages: ChatMessage[]) {
-  return messages.reduce((acc, message) => {
-    acc += acc ? "\n" : "";
-    if (message.role === "user") {
-      acc += `Human: ${message.content}`;
-    } else {
-      acc += `Assistant: ${message.content}`;
-    }
-    return acc;
-  }, "");
-}
-
-export const defaultContextSystemPrompt = ({ context = "" }) => {
-  if (!context) return "";
-  return `Context information is below.
----------------------
-${context}
----------------------`;
-};
-
-export type ContextSystemPrompt = typeof defaultContextSystemPrompt;
-
-export const defaultKeywordExtractPrompt = ({
-  context = "",
-  maxKeywords = 10,
-}) => {
-  return `
-Some text is provided below. Given the text, extract up to ${maxKeywords} keywords from the text. Avoid stopwords.
----------------------
-${context}
----------------------
-Provide keywords in the following comma-separated format: 'KEYWORDS: <keywords>'
-`;
-};
-
-export type KeywordExtractPrompt = typeof defaultKeywordExtractPrompt;
-
-export const defaultQueryKeywordExtractPrompt = ({
-  question = "",
-  maxKeywords = 10,
-}) => {
-  return `(
-  "A question is provided below. Given the question, extract up to ${maxKeywords} "
-  "keywords from the text. Focus on extracting the keywords that we can use "
-  "to best lookup answers to the question. Avoid stopwords."
-  "---------------------"
-  "${question}"
-  "---------------------"
-  "Provide keywords in the following comma-separated format: 'KEYWORDS: <keywords>'"
-)`;
-};
-export type QueryKeywordExtractPrompt = typeof defaultQueryKeywordExtractPrompt;

packages/llamaindex/src/PromptHelper.ts

@@ -1,24 +1,29 @@
-import { SentenceSplitter } from "@llamaindex/core/node-parser";
-import { type Tokenizer, tokenizers } from "@llamaindex/env";
-import type { SimplePrompt } from "./Prompt.js";
 import {
   DEFAULT_CHUNK_OVERLAP_RATIO,
   DEFAULT_CONTEXT_WINDOW,
   DEFAULT_NUM_OUTPUTS,
   DEFAULT_PADDING,
-} from "./constants.js";
+} from "@llamaindex/core/global";
+import { SentenceSplitter } from "@llamaindex/core/node-parser";
+import type { PromptTemplate } from "@llamaindex/core/prompts";
+import { type Tokenizer, tokenizers } from "@llamaindex/env";
 
-export function getEmptyPromptTxt(prompt: SimplePrompt) {
-  return prompt({});
+/**
+ * Get the empty prompt text given a prompt.
+ */
+export function getEmptyPromptTxt(prompt: PromptTemplate) {
+  return prompt.format({
+    ...Object.fromEntries(
+      [...prompt.templateVars.keys()].map((key) => [key, ""]),
+    ),
+  });
 }
 
 /**
  * Get biggest empty prompt size from a list of prompts.
  * Used to calculate the maximum size of inputs to the LLM.
- * @param prompts
- * @returns
  */
-export function getBiggestPrompt(prompts: SimplePrompt[]) {
+export function getBiggestPrompt(prompts: PromptTemplate[]) {
   const emptyPromptTexts = prompts.map(getEmptyPromptTxt);
   const emptyPromptLengths = emptyPromptTexts.map((text) => text.length);
   const maxEmptyPromptLength = Math.max(...emptyPromptLengths);
@@ -59,7 +64,7 @@ export class PromptHelper {
    * @param prompt
    * @returns
    */
-  private getAvailableContextSize(prompt: SimplePrompt) {
+  private getAvailableContextSize(prompt: PromptTemplate) {
     const emptyPromptText = getEmptyPromptTxt(prompt);
     const promptTokens = this.tokenizer.encode(emptyPromptText);
     const numPromptTokens = promptTokens.length;
@@ -69,13 +74,9 @@ export class PromptHelper {
 
   /**
    * Find the maximum size of each chunk given a prompt.
-   * @param prompt
-   * @param numChunks
-   * @param padding
-   * @returns
    */
   private getAvailableChunkSize(
-    prompt: SimplePrompt,
+    prompt: PromptTemplate,
     numChunks = 1,
     padding = 5,
   ) {
@@ -92,13 +93,9 @@ export class PromptHelper {
 
   /**
    * Creates a text splitter with the correct chunk sizes and overlaps given a prompt.
-   * @param prompt
-   * @param numChunks
-   * @param padding
-   * @returns
    */
   getTextSplitterGivenPrompt(
-    prompt: SimplePrompt,
+    prompt: PromptTemplate,
     numChunks = 1,
     padding = DEFAULT_PADDING,
   ) {
@@ -112,13 +109,9 @@ export class PromptHelper {
 
   /**
    * Repack resplits the strings based on the optimal text splitter.
-   * @param prompt
-   * @param textChunks
-   * @param padding
-   * @returns
    */
   repack(
-    prompt: SimplePrompt,
+    prompt: PromptTemplate,
     textChunks: string[],
     padding = DEFAULT_PADDING,
   ) {

packages/llamaindex/src/QuestionGenerator.ts

@@ -1,16 +1,20 @@
 import type { LLM, ToolMetadata } from "@llamaindex/core/llms";
+import {
+  defaultSubQuestionPrompt,
+  type ModuleRecord,
+  PromptMixin,
+  type SubQuestionPrompt,
+} from "@llamaindex/core/prompts";
 import type { QueryType } from "@llamaindex/core/query-engine";
-import { extractText } from "@llamaindex/core/utils";
+import type { BaseOutputParser } from "@llamaindex/core/schema";
+import { extractText, toToolDescriptions } from "@llamaindex/core/utils";
 import { SubQuestionOutputParser } from "./OutputParser.js";
-import type { SubQuestionPrompt } from "./Prompt.js";
-import { buildToolsText, defaultSubQuestionPrompt } from "./Prompt.js";
 import type {
   BaseQuestionGenerator,
   SubQuestion,
 } from "./engines/query/types.js";
 import { OpenAI } from "./llm/openai.js";
-import { PromptMixin } from "./prompts/index.js";
-import type { BaseOutputParser, StructuredOutput } from "./types.js";
+import type { StructuredOutput } from "./types.js";
 
 /**
  * LLMQuestionGenerator uses the LLM to generate new questions for the LLM using tools and a user query.
@@ -49,11 +53,11 @@ export class LLMQuestionGenerator
     tools: ToolMetadata[],
     query: QueryType,
   ): Promise<SubQuestion[]> {
-    const toolsStr = buildToolsText(tools);
+    const toolsStr = toToolDescriptions(tools);
     const queryStr = extractText(query);
     const prediction = (
       await this.llm.complete({
-        prompt: this.prompt({
+        prompt: this.prompt.format({
           toolsStr,
           queryStr,
         }),
@@ -64,4 +68,8 @@ export class LLMQuestionGenerator
 
     return structuredOutput.parsedOutput;
   }
+
+  protected _getPromptModules(): ModuleRecord {
+    return {};
+  }
 }

packages/llamaindex/src/agent/base.ts

@@ -7,7 +7,7 @@ import type {
 } from "@llamaindex/core/llms";
 import { EngineResponse } from "@llamaindex/core/schema";
 import { wrapEventCaller } from "@llamaindex/core/utils";
-import { ReadableStream, TransformStream, randomUUID } from "@llamaindex/env";
+import { randomUUID } from "@llamaindex/env";
 import { ChatHistory } from "../ChatHistory.js";
 import { Settings } from "../Settings.js";
 import {
@@ -16,7 +16,7 @@ import {
   type ChatEngineParamsStreaming,
 } from "../engines/chat/index.js";
 import { consoleLogger, emptyLogger } from "../internal/logger.js";
-import { isAsyncIterable } from "../internal/utils.js";
+import { isReadableStream } from "../internal/utils.js";
 import { ObjectRetriever } from "../objects/index.js";
 import type {
   AgentTaskContext,
@@ -374,7 +374,7 @@ export abstract class AgentRunner<
       this.#chatHistory = [...stepOutput.taskStep.context.store.messages];
       if (stepOutput.isLast) {
         const { output } = stepOutput;
-        if (isAsyncIterable(output)) {
+        if (isReadableStream(output)) {
           return output.pipeThrough<EngineResponse>(
             new TransformStream({
               transform(chunk, controller) {