Release 0.5.25 (#1182 )

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
docs: update README.md (#1183 )
2026-07-03 19:19:08 -04:00 · 2024-09-11 12:08:39 -07:00 · 2024-09-11 11:07:00 -07:00 · 2024-09-11 10:20:55 -07:00 · 2024-09-11 10:19:59 -07:00 · 2024-09-10 23:48:51 -07:00
185 changed files with 2015 additions and 744 deletions
@@ -19,6 +19,11 @@ jobs:
      matrix:
        node-version: [18.x, 20.x, 22.x]
    name: E2E on Node.js ${{ matrix.node-version }}
+
+    env: POSTGRES_DB=vectordb
+      POSTGRES_USER=testuser
+      POSTGRES_PASSWORD=testpwd
+      POSTGRES_HOST_AUTH_METHOD=trust
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
@@ -45,6 +50,16 @@ jobs:

    steps:
      - uses: actions/checkout@v4
+      - uses: ankane/setup-postgres@v1
+        with:
+          database: llamaindex_node_test
+          dev-files: true
+      - run: |
+          cd /tmp
+          git clone --branch v0.7.0 https://github.com/pgvector/pgvector.git
+          cd pgvector
+          make
+          sudo make install
      - uses: pnpm/action-setup@v4
      - name: Setup Node.js
        uses: actions/setup-node@v4
@@ -36,9 +36,44 @@ For now, browser support is limited due to the lack of support for [AsyncLocalSt
 npm install llamaindex
 pnpm install llamaindex
 yarn add llamaindex
-jsr install @llamaindex/core
 ```

+### Setup TypeScript
+
+```json5
+{
+  compilerOptions: {
+    // ⬇️ add this line to your tsconfig.json
+    moduleResolution: "bundler", // or "node16"
+  },
+}
+```
+
+<details>
+   <summary>Why?</summary>
+  We are shipping both ESM and CJS module, and compatible with Vercel Edge, Cloudflare Workers, and other serverless platforms.
+
+So we are using [conditional exports](https://nodejs.org/api/packages.html#conditional-exports) to support all environments.
+
+This is a kind of modern way of shipping packages, but might cause TypeScript type check to fail because of legacy module resolution.
+
+Imaging you put output file into `/dist/openai.js` but you are importing `llamaindex/openai` in your code, and set `package.json` like this:
+
+```json
+{
+  "exports": {
+    "./openai": "./dist/openai.js"
+  }
+}
+```
+
+In old module resolution, TypeScript will not be able to find the module because it is not follow the file structure, even you run `node index.js` successfully. (on Node.js >=16)
+
+See more about [moduleResolution](https://www.typescriptlang.org/docs/handbook/modules/theory.html#module-resolution) or
+[TypeScript 5.0 blog](https://devblogs.microsoft.com/typescript/announcing-typescript-5-0/#--moduleresolution-bundler7).
+
+</details>
+
 ### Node.js

 ```ts
@@ -1,5 +1,32 @@
 # docs

+## 0.0.66
+
+### Patch Changes
+
+- Updated dependencies [4810364]
+- Updated dependencies [d3bc663]
+  - llamaindex@0.5.25
+
+## 0.0.65
+
+### Patch Changes
+
+- llamaindex@0.5.24
+
+## 0.0.64
+
+### Patch Changes
+
+- llamaindex@0.5.23
+
+## 0.0.63
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.0.62

 ### Patch Changes
@@ -50,10 +50,10 @@ We want to see what our agent is up to, so we're going to hook into some events

 ```javascript
 Settings.callbackManager.on("llm-tool-call", (event) => {
-  console.log(event.detail.payload);
+  console.log(event.detail);
 });
 Settings.callbackManager.on("llm-tool-result", (event) => {
-  console.log(event.detail.payload);
+  console.log(event.detail);
 });
 ```

@@ -21,7 +21,7 @@ LlamaIndex.TS handles several major use cases:

 - **Structured Data Extraction**: turning complex, unstructured and semi-structured data into uniform, programmatically accessible formats.
 - **Retrieval-Augmented Generation (RAG)**: answering queries across your internal data by providing LLMs with up-to-date, semantically relevant context including Question and Answer systems and chat bots.
- **Autonomous Agents**: building software that is capable of intelligently selecting and using tools to accomplish tasks in an interative, unsupervised manner.
+- **Autonomous Agents**: building software that is capable of intelligently selecting and using tools to accomplish tasks in an interactive, unsupervised manner.

 ## 👨‍👩‍👧‍👦 Who is LlamaIndex for?

@@ -27,3 +27,4 @@ for await (const chunk of stream) {

 - [ContextChatEngine](../api/classes/ContextChatEngine.md)
 - [CondenseQuestionChatEngine](../api/classes/ContextChatEngine.md)
+- [SimpleChatEngine](../api/classes/SimpleChatEngine.md)
@@ -21,3 +21,4 @@ const index = await VectorStoreIndex.fromDocuments([document]);

 - [SummaryIndex](../api/classes/SummaryIndex.md)
 - [VectorStoreIndex](../api/classes/VectorStoreIndex.md)
+- [KeywordTableIndex](../api/classes/KeywordTableIndex.md)
@@ -98,3 +98,7 @@ Use the `embedDocuments` method to generate embeddings for the texts.
 const result = await embeddings.embedDocuments(texts);
 console.log(result); // Perfectly customized embeddings, ready to serve.
 ```
+
+## API Reference
+
+- [MixedbreadAIEmbeddings](../../../api/classes/MixedbreadAIEmbeddings.md)
@@ -2,7 +2,7 @@

 ## Concept

-Evaluation and benchmarking are crucial concepts in LLM development. To improve the perfomance of an LLM app (RAG, agents) you must have a way to measure it.
+Evaluation and benchmarking are crucial concepts in LLM development. To improve the performance of an LLM app (RAG, agents) you must have a way to measure it.

 LlamaIndex offers key modules to measure the quality of generated results. We also offer key modules to measure retrieval quality.

@@ -36,7 +36,7 @@ main().catch(console.error);

 You can implement any transformation yourself by implementing the `TransformComponent`.

-The following custom transformation will remove any special characters or punctutation in text.
+The following custom transformation will remove any special characters or punctuation in text.

 ```ts
 import { TransformComponent, TextNode } from "llamaindex";
@@ -75,3 +75,7 @@ async function main() {

 main().catch(console.error);
 ```
+
+## API Reference
+
+- [TransformComponent](../../api/classes/TransformComponent.md)
@@ -1,5 +1,7 @@
 # DeepSeek LLM

+[DeepSeek Platform](https://platform.deepseek.com/)
+
 ## Usage

 ```ts
@@ -45,6 +47,6 @@ Currently does not support function calling.

 [Currently does not support json-output param while still is very good at json generating.](https://platform.deepseek.com/api-docs/faq#does-your-api-support-json-output)

-## API platform
+## API Reference

- [DeepSeek platform](https://platform.deepseek.com/)
+- [DeepSeekLLM](../../../api/classes/DeepSeekLLM.md)
@@ -163,3 +163,7 @@ Use the `rerank` method to reorder the documents based on the query.
 const result = await reranker.rerank(documents, query);
 console.log(result); // Perfectly customized results, ready to serve.
 ```
+
+## API Reference
+
+- [MixedbreadAIReranker](../../api/classes/MixedbreadAIReranker.md)
@@ -1,6 +1,6 @@
 # QueryEngine

-A query engine wraps a `Retriever` and a `ResponseSynthesizer` into a pipeline, that will use the query string to fetech nodes and then send them to the LLM to generate a response.
+A query engine wraps a `Retriever` and a `ResponseSynthesizer` into a pipeline, that will use the query string to fetch nodes and then send them to the LLM to generate a response.

 ```typescript
 const queryEngine = index.asQueryEngine();
@@ -4,7 +4,14 @@ sidebar_position: 5

 # Retriever

-A retriever in LlamaIndex is what is used to fetch `Node`s from an index using a query string. Aa `VectorIndexRetriever` will fetch the top-k most similar nodes. Meanwhile, a `SummaryIndexRetriever` will fetch all nodes no matter the query.
+A retriever in LlamaIndex is what is used to fetch `Node`s from an index using a query string.
+
+- [VectorIndexRetriever](../api/classes/VectorIndexRetriever.md) will fetch the top-k most similar nodes. Ideal for dense retrieval to find most relevant nodes.
+- [SummaryIndexRetriever](../api/classes/SummaryIndexRetriever.md) will fetch all nodes no matter the query. Ideal when complete context is necessary, e.g. analyzing large datasets.
+- [SummaryIndexLLMRetriever](../api/classes/SummaryIndexLLMRetriever.md) utilizes an LLM to score and filter nodes based on relevancy to the query.
+- [KeywordTableLLMRetriever](../api/classes/KeywordTableLLMRetriever.md) uses an LLM to extract keywords from the query and retrieve relevant nodes based on keyword matches.
+- [KeywordTableSimpleRetriever](../api/classes/KeywordTableSimpleRetriever.md) uses a basic frequency-based approach to extract keywords and retrieve nodes.
+- [KeywordTableRAKERetriever](../api/classes/KeywordTableRAKERetriever.md) uses the RAKE (Rapid Automatic Keyword Extraction) algorithm to extract keywords from the query, focusing on co-occurrence and context for keyword-based retrieval.

 ```typescript
 const retriever = vectorIndex.asRetriever({
@@ -14,9 +21,3 @@ const retriever = vectorIndex.asRetriever({
 // Fetch nodes!
 const nodesWithScore = await retriever.retrieve({ query: "query string" });
 ```
-
-## API Reference
-
- [SummaryIndexRetriever](../api/classes/SummaryIndexRetriever.md)
- [SummaryIndexLLMRetriever](../api/classes/SummaryIndexLLMRetriever.md)
- [VectorIndexRetriever](../api/classes/VectorIndexRetriever.md)
@@ -1,6 +1,6 @@
 {
  "name": "docs",
-  "version": "0.0.62",
+  "version": "0.0.66",
  "private": true,
  "scripts": {
    "docusaurus": "docusaurus",
@@ -3,7 +3,7 @@ import { DeepInfraEmbedding } from "llamaindex";
 async function main() {
  // API token can be provided as an environment variable too
  // using DEEPINFRA_API_TOKEN variable
-  const apiToken = "YOUR_API_TOKEN" ?? process.env.DEEPINFRA_API_TOKEN;
+  const apiToken = process.env.DEEPINFRA_API_TOKEN ?? "YOUR_API_TOKEN";
  const model = "BAAI/bge-large-en-v1.5";
  const embedModel = new DeepInfraEmbedding({
    model,
@@ -38,6 +38,9 @@
    "overrides": {
      "trim": "1.0.1",
      "protobufjs": "7.2.6"
+    },
+    "patchedDependencies": {
+      "python-format-js@1.4.3": "patches/python-format-js@1.4.3.patch"
    }
  },
  "lint-staged": {
@@ -1,5 +1,36 @@
 # @llamaindex/autotool-01-node-example

+## 0.0.6
+
+### Patch Changes
+
+- Updated dependencies [4810364]
+- Updated dependencies [d3bc663]
+  - llamaindex@0.5.25
+  - @llamaindex/autotool@2.0.1
+
+## 0.0.5
+
+### Patch Changes
+
+- llamaindex@0.5.24
+- @llamaindex/autotool@2.0.1
+
+## 0.0.4
+
+### Patch Changes
+
+- llamaindex@0.5.23
+- @llamaindex/autotool@2.0.1
+
+## 0.0.3
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+  - @llamaindex/autotool@2.0.1
+
 ## 0.0.2

 ### Patch Changes
@@ -13,5 +13,5 @@
  "scripts": {
    "start": "node --import tsx --import @llamaindex/autotool/node ./src/index.ts"
  },
-  "version": "0.0.2"
+  "version": "0.0.6"
 }
@@ -16,7 +16,7 @@ const openai = new OpenAI();
    stream: false,
  });

-  const toolCalls = response.choices[0].message.tool_calls ?? [];
+  const toolCalls = response.choices[0]!.message.tool_calls ?? [];
  for (const toolCall of toolCalls) {
    toolCall.function.name;
  }
@@ -1,5 +1,36 @@
 # @llamaindex/autotool-02-next-example

+## 0.1.50
+
+### Patch Changes
+
+- Updated dependencies [4810364]
+- Updated dependencies [d3bc663]
+  - llamaindex@0.5.25
+  - @llamaindex/autotool@2.0.1
+
+## 0.1.49
+
+### Patch Changes
+
+- llamaindex@0.5.24
+- @llamaindex/autotool@2.0.1
+
+## 0.1.48
+
+### Patch Changes
+
+- llamaindex@0.5.23
+- @llamaindex/autotool@2.0.1
+
+## 0.1.47
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+  - @llamaindex/autotool@2.0.1
+
 ## 0.1.46

 ### Patch Changes
@@ -1,7 +1,7 @@
 {
  "name": "@llamaindex/autotool-02-next-example",
  "private": true,
-  "version": "0.1.46",
+  "version": "0.1.50",
  "scripts": {
    "dev": "next dev",
    "build": "next build",
@@ -51,7 +51,7 @@
    "unplugin": "^1.12.2"
  },
  "peerDependencies": {
-    "llamaindex": "^0.5.21",
+    "llamaindex": "^0.5.25",
    "openai": "^4",
    "typescript": "^4"
  },
@@ -16,11 +16,16 @@ const openaiToolsAtom = atom<ChatCompletionTool[]>((get) => {
  const metadata = get(toolMetadataAtom);
  return metadata.map(([metadata]) => ({
    type: "function",
-    function: {
-      parameters: metadata.parameters,
-      name: metadata.name,
-      description: metadata.description,
-    },
+    function: metadata.parameters
+      ? {
+          parameters: metadata.parameters,
+          name: metadata.name,
+          description: metadata.description,
+        }
+      : {
+          name: metadata.name,
+          description: metadata.description,
+        },
  }));
 });

@@ -17,7 +17,7 @@ export type Info = {
 * @internal
 */
 export type InfoString = {
-  originalFunction?: string;
+  originalFunction: string | undefined;
  parameterMapping: Record<string, number>;
 };

@@ -1,5 +1,17 @@
 # @llamaindex/cloud

+## 0.2.4
+
+### Patch Changes
+
+- 4810364: fix: bump version
+
+## 0.2.3
+
+### Patch Changes
+
+- 0bf8d80: fix: bump version
+
 ## 0.2.2

 ### Patch Changes
@@ -4,6 +4,7 @@ export default defineConfig({
  // you can download this file to get the latest version of the OpenAPI document
  // @link https://api.cloud.llamaindex.ai/api/openapi.json
  input: "./openapi.json",
+  client: "@hey-api/client-fetch",
  output: {
    path: "./src/client",
    format: "prettier",
@@ -1,10 +1,10 @@
 {
  "name": "@llamaindex/cloud",
-  "version": "0.2.2",
+  "version": "0.2.4",
  "type": "module",
  "license": "MIT",
  "scripts": {
-    "generate": "pnpm dlx @hey-api/openapi-ts@0.49.0",
+    "generate": "pnpx @hey-api/openapi-ts@0.53.0",
    "build": "pnpm run generate && bunchee"
  },
  "files": [
@@ -34,7 +34,8 @@
    "directory": "packages/cloud"
  },
  "devDependencies": {
-    "@hey-api/openapi-ts": "^0.52.11",
+    "@hey-api/client-fetch": "^0.2.4",
+    "@hey-api/openapi-ts": "^0.53.0",
    "bunchee": "5.3.2"
  }
 }
@@ -1,5 +1,20 @@
 # @llamaindex/community

+## 0.0.33
+
+### Patch Changes
+
+- Updated dependencies [711c814]
+  - @llamaindex/core@0.1.12
+
+## 0.0.32
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - @llamaindex/env@0.1.10
+  - @llamaindex/core@0.1.11
+
 ## 0.0.31

 ### Patch Changes
@@ -1,7 +1,7 @@
 {
  "name": "@llamaindex/community",
  "description": "Community package for LlamaIndexTS",
-  "version": "0.0.31",
+  "version": "0.0.33",
  "type": "module",
  "types": "dist/type/index.d.ts",
  "main": "dist/cjs/index.js",
@@ -28,9 +28,9 @@ export const mergeNeighboringSameRoleMessages = (
 ): AnthropicMessage[] => {
  return messages.reduce(
    (result: AnthropicMessage[], current: AnthropicMessage, index: number) => {
-      if (index > 0 && messages[index - 1].role === current.role) {
-        result[result.length - 1].content = [
-          ...result[result.length - 1].content,
+      if (index > 0 && messages[index - 1]!.role === current.role) {
+        result[result.length - 1]!.content = [
+          ...result[result.length - 1]!.content,
          ...current.content,
        ];
      } else {
@@ -128,7 +128,7 @@ export const mapChatMessagesToAnthropicMessages = <
      );
    })
    .filter((message: AnthropicMessage) => {
-      const content = message.content[0];
+      const content = message.content[0]!;
      if (content.type === "text" && !content.text) return false;
      if (content.type === "image" && !content.source.data) return false;
      if (content.type === "image" && message.role === "assistant")
@@ -151,12 +151,12 @@ export const extractDataUrlComponents = (
 } => {
  const parts = dataUrl.split(";base64,");

-  if (parts.length !== 2 || !parts[0].startsWith("data:")) {
+  if (parts.length !== 2 || !parts[0]!.startsWith("data:")) {
    throw new Error("Invalid data URL");
  }

-  const mimeType = parts[0].slice(5);
-  const base64 = parts[1];
+  const mimeType = parts[0]!.slice(5);
+  const base64 = parts[1]!;

  return {
    mimeType,
@@ -153,12 +153,15 @@ export const TOOL_CALL_MODELS = [

 const getProvider = (model: string): Provider => {
  const providerName = model.split(".")[0];
+  if (!providerName) {
+    throw new Error(`Model ${model} is not supported`);
+  }
  if (!(providerName in PROVIDERS)) {
    throw new Error(
      `Provider ${providerName} for model ${model} is not supported`,
    );
  }
-  return PROVIDERS[providerName];
+  return PROVIDERS[providerName]!;
 };

 export type BedrockModelParams = {
@@ -34,7 +34,7 @@ export class MetaProvider extends Provider<MetaStreamEvent> {
    const result = this.getResultFromResponse(response);
    if (!result.generation.trim().startsWith(TOKENS.TOOL_CALL)) return [];
    const tool = JSON.parse(
-      result.generation.trim().split(TOKENS.TOOL_CALL)[1],
+      result.generation.trim().split(TOKENS.TOOL_CALL)[1]!,
    );
    return [
      {
@@ -1,5 +1,18 @@
 # @llamaindex/core

+## 0.1.12
+
+### Patch Changes
+
+- 711c814: fix: patch `python-format-js`
+
+## 0.1.11
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - @llamaindex/env@0.1.10
+
 ## 0.1.10

 ### Patch Changes
@@ -1,7 +1,7 @@
 {
  "name": "@llamaindex/core",
  "type": "module",
-  "version": "0.1.10",
+  "version": "0.1.12",
  "description": "LlamaIndex Core Module",
  "exports": {
    "./node-parser": {
@@ -129,6 +129,20 @@
        "types": "./dist/prompts/index.d.ts",
        "default": "./dist/prompts/index.js"
      }
+    },
+    "./indices": {
+      "require": {
+        "types": "./dist/indices/index.d.cts",
+        "default": "./dist/indices/index.cjs"
+      },
+      "import": {
+        "types": "./dist/indices/index.d.ts",
+        "default": "./dist/indices/index.js"
+      },
+      "default": {
+        "types": "./dist/indices/index.d.ts",
+        "default": "./dist/indices/index.js"
+      }
    }
  },
  "files": [
@@ -34,7 +34,7 @@ export abstract class BaseEmbedding extends TransformComponent {
        const embeddings = await this.getTextEmbeddingsBatch(texts, options);

        for (let i = 0; i < nodes.length; i++) {
-          nodes[i].embedding = embeddings[i];
+          nodes[i]!.embedding = embeddings[i];
        }

        return nodes;
@@ -120,7 +120,7 @@ export async function batchEmbeddings<T>(
  const curBatch: T[] = [];

  for (let i = 0; i < queue.length; i++) {
-    curBatch.push(queue[i]);
+    curBatch.push(queue[i]!);
    if (i == queue.length - 1 || curBatch.length == chunkSize) {
      const embeddings = await embedFunc(curBatch);

@@ -35,20 +35,20 @@ export function similarity(
  function norm(x: number[]): number {
    let result = 0;
    for (let i = 0; i < x.length; i++) {
-      result += x[i] * x[i];
+      result += x[i]! * x[i]!;
    }
    return Math.sqrt(result);
  }

  switch (mode) {
    case SimilarityType.EUCLIDEAN: {
-      const difference = embedding1.map((x, i) => x - embedding2[i]);
+      const difference = embedding1.map((x, i) => x - embedding2[i]!);
      return -norm(difference);
    }
    case SimilarityType.DOT_PRODUCT: {
      let result = 0;
      for (let i = 0; i < embedding1.length; i++) {
-        result += embedding1[i] * embedding2[i];
+        result += embedding1[i]! * embedding2[i]!;
      }
      return result;
    }
@@ -0,0 +1,24 @@
+import { path } from "@llamaindex/env";
+
+//#region llm
+export const DEFAULT_CONTEXT_WINDOW = 3900;
+export const DEFAULT_NUM_OUTPUTS = 256;
+export const DEFAULT_CHUNK_SIZE = 1024;
+export const DEFAULT_CHUNK_OVERLAP = 20;
+export const DEFAULT_CHUNK_OVERLAP_RATIO = 0.1;
+export const DEFAULT_PADDING = 5;
+//#endregion
+//#region storage
+export const DEFAULT_COLLECTION = "data";
+export const DEFAULT_PERSIST_DIR = path.join("./storage");
+export const DEFAULT_INDEX_STORE_PERSIST_FILENAME = "index_store.json";
+export const DEFAULT_DOC_STORE_PERSIST_FILENAME = "doc_store.json";
+export const DEFAULT_VECTOR_STORE_PERSIST_FILENAME = "vector_store.json";
+export const DEFAULT_GRAPH_STORE_PERSIST_FILENAME = "graph_store.json";
+export const DEFAULT_NAMESPACE = "docstore";
+export const DEFAULT_IMAGE_VECTOR_NAMESPACE = "images";
+//#endregion
+//#region llama cloud
+export const DEFAULT_PROJECT_NAME = "Default";
+export const DEFAULT_BASE_URL = "https://api.cloud.llamaindex.ai";
+//#endregion
@@ -1,3 +1,4 @@
+export * from "./constants";
 export { Settings } from "./settings";
 export { CallbackManager } from "./settings/callback-manager";
 export type {
@@ -1,4 +1,5 @@
 import type { Tokenizer } from "@llamaindex/env";
+import type { LLM } from "../llms";
 import {
  type CallbackManager,
  getCallbackManager,
@@ -10,6 +11,7 @@ import {
  setChunkSize,
  withChunkSize,
 } from "./settings/chunk-size";
+import { getLLM, setLLM, withLLM } from "./settings/llm";
 import {
  getTokenizer,
  setTokenizer,
@@ -17,6 +19,15 @@ import {
 } from "./settings/tokenizer";

 export const Settings = {
+  get llm() {
+    return getLLM();
+  },
+  set llm(llm) {
+    setLLM(llm);
+  },
+  withLLM<Result>(llm: LLM, fn: () => Result): Result {
+    return withLLM(llm, fn);
+  },
  get tokenizer() {
    return getTokenizer();
  },
@@ -0,0 +1,23 @@
+import { AsyncLocalStorage } from "@llamaindex/env";
+import type { LLM } from "../../llms";
+
+const llmAsyncLocalStorage = new AsyncLocalStorage<LLM>();
+let globalLLM: LLM | undefined;
+
+export function getLLM(): LLM {
+  const currentLLM = globalLLM ?? llmAsyncLocalStorage.getStore();
+  if (!currentLLM) {
+    throw new Error(
+      "Cannot find LLM, please set `Settings.llm = ...` on the top of your code",
+    );
+  }
+  return currentLLM;
+}
+
+export function setLLM(llm: LLM): void {
+  globalLLM = llm;
+}
+
+export function withLLM<Result>(llm: LLM, fn: () => Result): Result {
+  return llmAsyncLocalStorage.run(llm, fn);
+}
@@ -0,0 +1,5 @@
+export {
+  PromptHelper,
+  getBiggestPrompt,
+  type PromptHelperOptions,
+} from "./prompt-helper";
@@ -1,17 +1,17 @@
-import { SentenceSplitter } from "@llamaindex/core/node-parser";
-import type { PromptTemplate } from "@llamaindex/core/prompts";
 import { type Tokenizer, tokenizers } from "@llamaindex/env";
 import {
  DEFAULT_CHUNK_OVERLAP_RATIO,
  DEFAULT_CONTEXT_WINDOW,
  DEFAULT_NUM_OUTPUTS,
  DEFAULT_PADDING,
-} from "./constants.js";
+} from "../global";
+import { SentenceSplitter } from "../node-parser";
+import type { PromptTemplate } from "../prompts";

 /**
 * Get the empty prompt text given a prompt.
 */
-export function getEmptyPromptTxt(prompt: PromptTemplate) {
+function getEmptyPromptTxt(prompt: PromptTemplate) {
  return prompt.format({
    ...Object.fromEntries(
      [...prompt.templateVars.keys()].map((key) => [key, ""]),
@@ -23,14 +23,23 @@ export function getEmptyPromptTxt(prompt: PromptTemplate) {
 * Get biggest empty prompt size from a list of prompts.
 * Used to calculate the maximum size of inputs to the LLM.
 */
-export function getBiggestPrompt(prompts: PromptTemplate[]) {
+export function getBiggestPrompt(prompts: PromptTemplate[]): PromptTemplate {
  const emptyPromptTexts = prompts.map(getEmptyPromptTxt);
  const emptyPromptLengths = emptyPromptTexts.map((text) => text.length);
  const maxEmptyPromptLength = Math.max(...emptyPromptLengths);
  const maxEmptyPromptIndex = emptyPromptLengths.indexOf(maxEmptyPromptLength);
-  return prompts[maxEmptyPromptIndex];
+  return prompts[maxEmptyPromptIndex]!;
 }

+export type PromptHelperOptions = {
+  contextWindow?: number;
+  numOutput?: number;
+  chunkOverlapRatio?: number;
+  chunkSizeLimit?: number;
+  tokenizer?: Tokenizer;
+  separator?: string;
+};
+
 /**
 * A collection of helper functions for working with prompts.
 */
@@ -38,19 +47,19 @@ export class PromptHelper {
  contextWindow = DEFAULT_CONTEXT_WINDOW;
  numOutput = DEFAULT_NUM_OUTPUTS;
  chunkOverlapRatio = DEFAULT_CHUNK_OVERLAP_RATIO;
-  chunkSizeLimit?: number;
+  chunkSizeLimit: number | undefined;
  tokenizer: Tokenizer;
  separator = " ";

-  // eslint-disable-next-line max-params
-  constructor(
-    contextWindow = DEFAULT_CONTEXT_WINDOW,
-    numOutput = DEFAULT_NUM_OUTPUTS,
-    chunkOverlapRatio = DEFAULT_CHUNK_OVERLAP_RATIO,
-    chunkSizeLimit?: number,
-    tokenizer?: Tokenizer,
-    separator = " ",
-  ) {
+  constructor(options: PromptHelperOptions = {}) {
+    const {
+      contextWindow = DEFAULT_CONTEXT_WINDOW,
+      numOutput = DEFAULT_NUM_OUTPUTS,
+      chunkOverlapRatio = DEFAULT_CHUNK_OVERLAP_RATIO,
+      chunkSizeLimit,
+      tokenizer,
+      separator = " ",
+    } = options;
    this.contextWindow = contextWindow;
    this.numOutput = numOutput;
    this.chunkOverlapRatio = chunkOverlapRatio;
@@ -79,7 +88,7 @@ export class PromptHelper {
    prompt: PromptTemplate,
    numChunks = 1,
    padding = 5,
-  ) {
+  ): number {
    const availableContextSize = this.getAvailableContextSize(prompt);

    const result = Math.floor(availableContextSize / numChunks) - padding;
@@ -104,7 +113,12 @@ export class PromptHelper {
      throw new Error("Got 0 as available chunk size");
    }
    const chunkOverlap = this.chunkOverlapRatio * chunkSize;
-    return new SentenceSplitter({ chunkSize, chunkOverlap });
+    return new SentenceSplitter({
+      chunkSize,
+      chunkOverlap,
+      separator: this.separator,
+      tokenizer: this.tokenizer,
+    });
  }

  /**
@@ -103,7 +103,7 @@ export type LLMMetadata = {
  model: string;
  temperature: number;
  topP: number;
-  maxTokens?: number;
+  maxTokens?: number | undefined;
  contextWindow: number;
  tokenizer: Tokenizers | undefined;
 };
@@ -141,7 +141,7 @@ export interface LLMCompletionParamsStreaming extends LLMCompletionParamsBase {

 export interface LLMCompletionParamsNonStreaming
  extends LLMCompletionParamsBase {
-  stream?: false | null;
+  stream?: false | null | undefined;
 }

 export type MessageContentTextDetail = {
@@ -122,7 +122,7 @@ export abstract class MetadataAwareTextSplitter extends TextSplitter {
      throw new TypeError("`texts` and `metadata` must have the same length");
    }
    return texts.flatMap((text, i) =>
-      this.splitTextMetadataAware(text, metadata[i]),
+      this.splitTextMetadataAware(text, metadata[i]!),
    );
  }

@@ -35,8 +35,8 @@ export class MarkdownNodeParser extends NodeParser {
        }
        metadata = this.updateMetadata(
          metadata,
-          headerMatch[2],
-          headerMatch[1].trim().length,
+          headerMatch[2]!,
+          headerMatch[1]!.trim().length,
        );
        currentSection = `${headerMatch[2]}\n`;
      } else {
@@ -63,7 +63,7 @@ export class MarkdownNodeParser extends NodeParser {
    for (let i = 1; i < newHeaderLevel; i++) {
      const key = `Header_${i}`;
      if (key in headersMetadata) {
-        updatedHeaders[key] = headersMetadata[key];
+        updatedHeaders[key] = headersMetadata[key]!;
      }
    }

@@ -76,10 +76,10 @@ export class MarkdownNodeParser extends NodeParser {
    node: TextNode,
    metadata: Metadata,
  ): TextNode {
-    const newNode = buildNodeFromSplits([textSplit], node, undefined)[0];
+    const newNode = buildNodeFromSplits([textSplit], node, undefined)[0]!;

    if (this.includeMetadata) {
-      newNode.metadata = { ...newNode.metadata, ...metadata };
+      newNode.metadata = { ...newNode!.metadata, ...metadata };
    }

    return newNode;
@@ -168,9 +168,9 @@ export class SentenceSplitter extends MetadataAwareTextSplitter {
      let lastIndex = lastChunk.length - 1;
      while (
        lastIndex >= 0 &&
-        currentChunkLength + lastChunk[lastIndex][1] <= this.chunkOverlap
+        currentChunkLength + lastChunk[lastIndex]![1] <= this.chunkOverlap
      ) {
-        const [text, length] = lastChunk[lastIndex];
+        const [text, length] = lastChunk[lastIndex]!;
        currentChunkLength += length;
        currentChunk.unshift([text, length]);
        lastIndex -= 1;
@@ -178,7 +178,7 @@ export class SentenceSplitter extends MetadataAwareTextSplitter {
    };

    while (splits.length > 0) {
-      const curSplit = splits[0];
+      const curSplit = splits[0]!;
      if (curSplit.tokenSize > chunkSize) {
        throw new Error("Single token exceeded chunk size");
      }
@@ -5,7 +5,7 @@ export type TextSplitterFn = (text: string) => string[];

 const truncateText = (text: string, textSplitter: TextSplitter): string => {
  const chunks = textSplitter.splitText(text);
-  return chunks[0];
+  return chunks[0] ?? text;
 };

 const splitTextKeepSeparator = (text: string, separator: string): string[] => {
@@ -18,7 +18,7 @@ export type BasePromptTemplateOptions<
    // loose type for better type inference
    | readonly string[];
  options?: Partial<Record<TemplatesVar[number] | (string & {}), string>>;
-  outputParser?: BaseOutputParser;
+  outputParser?: BaseOutputParser | undefined;
  templateVarMappings?: Partial<
    Record<Vars[number] | (string & {}), TemplatesVar[number] | (string & {})>
  >;
@@ -34,7 +34,7 @@ export abstract class BasePromptTemplate<
  metadata: Metadata = {};
  templateVars: Set<string> = new Set();
  options: Partial<Record<TemplatesVar[number] | (string & {}), string>> = {};
-  outputParser?: BaseOutputParser;
+  outputParser: BaseOutputParser | undefined;
  templateVarMappings: Partial<
    Record<Vars[number] | (string & {}), TemplatesVar[number] | (string & {})>
  > = {};
@@ -45,21 +45,21 @@ export abstract class PromptMixin {

    for (const key in prompts) {
      if (key.includes(":")) {
-        const [module_name, sub_key] = key.split(":");
+        const [moduleName, subKey] = key.split(":") as [string, string];

-        if (!subPrompt[module_name]) {
-          subPrompt[module_name] = {};
+        if (!subPrompt[moduleName]) {
+          subPrompt[moduleName] = {};
        }
-        subPrompt[module_name][sub_key] = prompts[key];
+        subPrompt[moduleName][subKey] = prompts[key]!;
      }
    }

-    for (const [module_name, subPromptDict] of Object.entries(subPrompt)) {
-      if (!promptModules[module_name]) {
-        throw new Error(`Module ${module_name} not found.`);
+    for (const [moduleName, subPromptDict] of Object.entries(subPrompt)) {
+      if (!promptModules[moduleName]) {
+        throw new Error(`Module ${moduleName} not found.`);
      }

-      const moduleToUpdate = promptModules[module_name];
+      const moduleToUpdate = promptModules[moduleName];

      moduleToUpdate.updatePrompts(subPromptDict);
    }
@@ -38,13 +38,15 @@ export type RelatedNodeType<T extends Metadata = Metadata> =
  | RelatedNodeInfo<T>[];

 export type BaseNodeParams<T extends Metadata = Metadata> = {
-  id_?: string;
-  metadata?: T;
-  excludedEmbedMetadataKeys?: string[];
-  excludedLlmMetadataKeys?: string[];
-  relationships?: Partial<Record<NodeRelationship, RelatedNodeType<T>>>;
-  hash?: string;
-  embedding?: number[];
+  id_?: string | undefined;
+  metadata?: T | undefined;
+  excludedEmbedMetadataKeys?: string[] | undefined;
+  excludedLlmMetadataKeys?: string[] | undefined;
+  relationships?:
+    | Partial<Record<NodeRelationship, RelatedNodeType<T>>>
+    | undefined;
+  hash?: string | undefined;
+  embedding?: number[] | undefined;
 };

 /**
@@ -58,7 +60,7 @@ export abstract class BaseNode<T extends Metadata = Metadata> {
   * Set to a UUID by default.
   */
  id_: string;
-  embedding?: number[];
+  embedding: number[] | undefined;

  // Metadata fields
  metadata: T;
@@ -198,11 +200,11 @@ export abstract class BaseNode<T extends Metadata = Metadata> {

 export type TextNodeParams<T extends Metadata = Metadata> =
  BaseNodeParams<T> & {
-    text?: string;
-    textTemplate?: string;
-    startCharIdx?: number;
-    endCharIdx?: number;
-    metadataSeparator?: string;
+    text?: string | undefined;
+    textTemplate?: string | undefined;
+    startCharIdx?: number | undefined;
+    endCharIdx?: number | undefined;
+    metadataSeparator?: string | undefined;
  };

 /**
@@ -418,7 +420,7 @@ export class ImageDocument<T extends Metadata = Metadata> extends ImageNode<T> {
 */
 export interface NodeWithScore<T extends Metadata = Metadata> {
  node: BaseNode<T>;
-  score?: number;
+  score?: number | undefined;
 }

 export enum ModalityType {
@@ -3,7 +3,7 @@ import { extractText } from "../../utils";
 import type { Metadata, NodeWithScore } from "../node";

 export class EngineResponse implements ChatResponse, ChatResponseChunk {
-  sourceNodes?: NodeWithScore[];
+  sourceNodes: NodeWithScore[] | undefined;

  metadata: Metadata = {};

@@ -74,12 +74,12 @@ export const extractDataUrlComponents = (
 } => {
  const parts = dataUrl.split(";base64,");

-  if (parts.length !== 2 || !parts[0].startsWith("data:")) {
+  if (parts.length !== 2 || !parts[0]!.startsWith("data:")) {
    throw new Error("Invalid data URL");
  }

-  const mimeType = parts[0].slice(5);
-  const base64 = parts[1];
+  const mimeType = parts[0]!.slice(5);
+  const base64 = parts[1]!;

  return {
    mimeType,
@@ -19,12 +19,12 @@ Header 2 content
    ]);

    expect(splits.length).toBe(2);
-    expect(splits[0].metadata).toEqual({ Header_1: "Main Header" });
-    expect(splits[1].metadata).toEqual({ Header_1: "Header 2" });
-    expect(splits[0].getContent(MetadataMode.NONE)).toStrictEqual(
+    expect(splits[0]!.metadata).toEqual({ Header_1: "Main Header" });
+    expect(splits[1]!.metadata).toEqual({ Header_1: "Header 2" });
+    expect(splits[0]!.getContent(MetadataMode.NONE)).toStrictEqual(
      "Main Header\n\nHeader 1 content",
    );
-    expect(splits[1].getContent(MetadataMode.NONE)).toStrictEqual(
+    expect(splits[1]!.getContent(MetadataMode.NONE)).toStrictEqual(
      "Header 2\nHeader 2 content",
    );
  });
@@ -89,16 +89,16 @@ Content
      }),
    ]);
    expect(splits.length).toBe(4);
-    expect(splits[0].metadata).toEqual({ Header_1: "Main Header" });
-    expect(splits[1].metadata).toEqual({
+    expect(splits[0]!.metadata).toEqual({ Header_1: "Main Header" });
+    expect(splits[1]!.metadata).toEqual({
      Header_1: "Main Header",
      Header_2: "Sub-header",
    });
-    expect(splits[2].metadata).toEqual({
+    expect(splits[2]!.metadata).toEqual({
      Header_1: "Main Header",
      Header_2: "Sub-header",
      Header_3: "Sub-sub header",
    });
-    expect(splits[3].metadata).toEqual({ Header_1: "New title" });
+    expect(splits[3]!.metadata).toEqual({ Header_1: "New title" });
  });
 });
@@ -22,7 +22,7 @@ describe("SentenceSplitter", () => {
    });
    const result = sentenceSplitter.getNodesFromDocuments([doc]);
    expect(result.length).toEqual(1);
-    const node = result[0];
+    const node = result[0]!;
    // check not the same object
    expect(node.metadata).not.toBe(doc.metadata);
    expect(node.excludedLlmMetadataKeys).not.toBe(doc.excludedLlmMetadataKeys);
@@ -1,5 +1,11 @@
 # @llamaindex/env

+## 0.1.10
+
+### Patch Changes
+
+- 4648da6: fix: wrong tiktoken version caused NextJs CL template run fail
+
 ## 0.1.9

 ### Patch Changes
@@ -1,7 +1,7 @@
 {
  "name": "@llamaindex/env",
  "description": "environment wrapper, supports all JS environment including node, deno, bun, edge runtime, and cloudflare worker",
-  "version": "0.1.9",
+  "version": "0.1.10",
  "type": "module",
  "types": "dist/type/index.d.ts",
  "main": "dist/cjs/index.js",
@@ -82,7 +82,7 @@
    "@aws-crypto/sha256-js": "^5.2.0",
    "js-tiktoken": "^1.0.12",
    "pathe": "^1.1.2",
-    "tiktoken": "1.0.14"
+    "tiktoken": "^1.0.15"
  },
  "peerDependenciesMeta": {
    "@aws-crypto/sha256-js": {
@@ -1,5 +1,32 @@
 # @llamaindex/experimental

+## 0.0.75
+
+### Patch Changes
+
+- Updated dependencies [4810364]
+- Updated dependencies [d3bc663]
+  - llamaindex@0.5.25
+
+## 0.0.74
+
+### Patch Changes
+
+- llamaindex@0.5.24
+
+## 0.0.73
+
+### Patch Changes
+
+- llamaindex@0.5.23
+
+## 0.0.72
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.0.71

 ### Patch Changes
@@ -1,7 +1,7 @@
 {
  "name": "@llamaindex/experimental",
  "description": "Experimental package for LlamaIndexTS",
-  "version": "0.0.71",
+  "version": "0.0.75",
  "type": "module",
  "types": "dist/type/index.d.ts",
  "main": "dist/cjs/index.js",
@@ -1,5 +1,40 @@
 # llamaindex

+## 0.5.25
+
+### Patch Changes
+
+- 4810364: fix: handle `RouterQueryEngine` with string query
+- d3bc663: refactor: export vector store only in nodejs environment on top level
+
+  If you see some missing modules error, please change vector store related imports to `llamaindex/vector-store`
+
+- Updated dependencies [4810364]
+  - @llamaindex/cloud@0.2.4
+
+## 0.5.24
+
+### Patch Changes
+
+- Updated dependencies [0bf8d80]
+  - @llamaindex/cloud@0.2.3
+
+## 0.5.23
+
+### Patch Changes
+
+- Updated dependencies [711c814]
+  - @llamaindex/core@0.1.12
+
+## 0.5.22
+
+### Patch Changes
+
+- 4648da6: fix: wrong tiktoken version caused NextJs CL template run fail
+- Updated dependencies [4648da6]
+  - @llamaindex/env@0.1.10
+  - @llamaindex/core@0.1.11
+
 ## 0.5.21

 ### Patch Changes
@@ -1,5 +1,32 @@
 # @llamaindex/cloudflare-worker-agent-test

+## 0.0.59
+
+### Patch Changes
+
+- Updated dependencies [4810364]
+- Updated dependencies [d3bc663]
+  - llamaindex@0.5.25
+
+## 0.0.58
+
+### Patch Changes
+
+- llamaindex@0.5.24
+
+## 0.0.57
+
+### Patch Changes
+
+- llamaindex@0.5.23
+
+## 0.0.56
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.0.55

 ### Patch Changes
@@ -1,6 +1,6 @@
 {
  "name": "@llamaindex/cloudflare-worker-agent-test",
-  "version": "0.0.55",
+  "version": "0.0.59",
  "type": "module",
  "private": true,
  "scripts": {
@@ -1,5 +1,32 @@
 # @llamaindex/next-agent-test

+## 0.1.59
+
+### Patch Changes
+
+- Updated dependencies [4810364]
+- Updated dependencies [d3bc663]
+  - llamaindex@0.5.25
+
+## 0.1.58
+
+### Patch Changes
+
+- llamaindex@0.5.24
+
+## 0.1.57
+
+### Patch Changes
+
+- llamaindex@0.5.23
+
+## 0.1.56
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.1.55

 ### Patch Changes
@@ -1,6 +1,6 @@
 {
  "name": "@llamaindex/next-agent-test",
-  "version": "0.1.55",
+  "version": "0.1.59",
  "private": true,
  "scripts": {
    "dev": "next dev",
@@ -1,5 +1,32 @@
 # test-edge-runtime

+## 0.1.58
+
+### Patch Changes
+
+- Updated dependencies [4810364]
+- Updated dependencies [d3bc663]
+  - llamaindex@0.5.25
+
+## 0.1.57
+
+### Patch Changes
+
+- llamaindex@0.5.24
+
+## 0.1.56
+
+### Patch Changes
+
+- llamaindex@0.5.23
+
+## 0.1.55
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.1.54

 ### Patch Changes
@@ -1,6 +1,6 @@
 {
  "name": "@llamaindex/nextjs-edge-runtime-test",
-  "version": "0.1.54",
+  "version": "0.1.58",
  "private": true,
  "scripts": {
    "dev": "next dev",
@@ -1,5 +1,32 @@
 # @llamaindex/next-node-runtime

+## 0.0.40
+
+### Patch Changes
+
+- Updated dependencies [4810364]
+- Updated dependencies [d3bc663]
+  - llamaindex@0.5.25
+
+## 0.0.39
+
+### Patch Changes
+
+- llamaindex@0.5.24
+
+## 0.0.38
+
+### Patch Changes
+
+- llamaindex@0.5.23
+
+## 0.0.37
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.0.36

 ### Patch Changes
@@ -1,6 +1,6 @@
 {
  "name": "@llamaindex/next-node-runtime-test",
-  "version": "0.0.36",
+  "version": "0.0.40",
  "private": true,
  "scripts": {
    "dev": "next dev",
@@ -1,5 +1,32 @@
 # @llamaindex/waku-query-engine-test

+## 0.0.59
+
+### Patch Changes
+
+- Updated dependencies [4810364]
+- Updated dependencies [d3bc663]
+  - llamaindex@0.5.25
+
+## 0.0.58
+
+### Patch Changes
+
+- llamaindex@0.5.24
+
+## 0.0.57
+
+### Patch Changes
+
+- llamaindex@0.5.23
+
+## 0.0.56
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.0.55

 ### Patch Changes
@@ -1,6 +1,6 @@
 {
  "name": "@llamaindex/waku-query-engine-test",
-  "version": "0.0.55",
+  "version": "0.0.59",
  "type": "module",
  "private": true,
  "scripts": {
@@ -11,7 +11,7 @@ export class OpenAIEmbedding
  extends TransformComponent
  implements BaseEmbedding
 {
-  embedInfo?: EmbeddingInfo | undefined;
+  embedInfo?: EmbeddingInfo;
  embedBatchSize = 512;

  constructor() {
@@ -48,8 +48,8 @@ export class OpenAI implements LLM {
        llmCompleteMockStorage.llmEventStart.shift()!["messages"];
      strictEqual(params.messages.length, chatMessage.length);
      for (let i = 0; i < chatMessage.length; i++) {
-        strictEqual(params.messages[i].role, chatMessage[i].role);
-        deepStrictEqual(params.messages[i].content, chatMessage[i].content);
+        strictEqual(params.messages[i]!.role, chatMessage[i]!.role);
+        deepStrictEqual(params.messages[i]!.content, chatMessage[i]!.content);
      }

      if (llmCompleteMockStorage.llmEventEnd.length > 0) {
@@ -64,7 +64,7 @@ export class OpenAI implements LLM {
                if (idx === -1) {
                  break;
                }
-                const chunk = llmCompleteMockStorage.llmEventStream[idx].chunk;
+                const chunk = llmCompleteMockStorage.llmEventStream[idx]!.chunk;
                llmCompleteMockStorage.llmEventStream.splice(idx, 1);
                yield chunk;
              }
@@ -90,8 +90,8 @@ export class OpenAI implements LLM {
      const chatMessage =
        llmCompleteMockStorage.llmEventStart.shift()!["messages"];
      strictEqual(1, chatMessage.length);
-      strictEqual("user", chatMessage[0].role);
-      strictEqual(params.prompt, chatMessage[0].content);
+      strictEqual("user", chatMessage[0]!.role);
+      strictEqual(params.prompt, chatMessage[0]!.content);
    }
    if (llmCompleteMockStorage.llmEventEnd.length > 0) {
      const response = llmCompleteMockStorage.llmEventEnd.shift()!["response"];
@@ -0,0 +1,67 @@
+import { LLMSingleSelector, Settings } from "llamaindex";
+import assert from "node:assert";
+import { test } from "node:test";
+import { mockLLMEvent } from "./utils.js";
+
+await test("#1177", async (t) => {
+  await mockLLMEvent(t, "#1177");
+  await t.test(async () => {
+    const selector = new LLMSingleSelector({
+      llm: Settings.llm,
+    });
+    {
+      const result = await selector.select(
+        [
+          {
+            description: "Math calculation",
+          },
+          {
+            description: "Search from google",
+          },
+        ],
+        "calculate 2 + 2",
+      );
+      assert.equal(result.selections.length, 1);
+      assert.equal(result.selections.at(0)!.index, 0);
+    }
+    {
+      const result = await selector.select(
+        [
+          {
+            description: "Math calculation",
+          },
+          {
+            description: "Search from google",
+          },
+        ],
+        {
+          query: "calculate 2 + 2",
+        },
+      );
+      assert.equal(result.selections.length, 1);
+      assert.equal(result.selections.at(0)!.index, 0);
+    }
+    {
+      const result = await selector.select(
+        [
+          {
+            description: "Math calculation",
+          },
+          {
+            description: "Search from google",
+          },
+        ],
+        {
+          query: [
+            {
+              type: "text",
+              text: "calculate 2 + 2",
+            },
+          ],
+        },
+      );
+      assert.equal(result.selections.length, 1);
+      assert.equal(result.selections.at(0)!.index, 0);
+    }
+  });
+});
@@ -163,11 +163,11 @@ For questions about more specific sections, please use the vector_tool.`,
    }),
  ];

-  const originalCall = queryEngineTools[1].call.bind(queryEngineTools[1]);
+  const originalCall = queryEngineTools[1]!.call.bind(queryEngineTools[1]);
  const mockCall = t.mock.fn(({ query }: { query: string }) => {
    return originalCall({ query });
  });
-  queryEngineTools[1].call = mockCall;
+  queryEngineTools[1]!.call = mockCall;

  const toolMapping = SimpleToolNodeMapping.fromObjects(queryEngineTools);

@@ -0,0 +1,67 @@
+{
+  "llmEventStart": [
+    {
+      "id": "PRESERVE_0",
+      "messages": [
+        {
+          "content": "Some choices are given below. It is provided in a numbered list (1 to 42), where each item in the list corresponds to a summary.\n---------------------\n(1) Math calculation(2) Search from google\n---------------------\nUsing only the choices above and not prior knowledge, return the choice that is most relevant to the question: 'calculate 2 + 2'\n\n\nThe output should be ONLY JSON formatted as a JSON instance.\n\nHere is an example:\n[\n    {\n        \"choice\": 1,\n        \"reason\": \"<insert reason for choice>\"\n    },\n    ...\n]\n",
+          "role": "user"
+        }
+      ]
+    },
+    {
+      "id": "PRESERVE_1",
+      "messages": [
+        {
+          "content": "Some choices are given below. It is provided in a numbered list (1 to 42), where each item in the list corresponds to a summary.\n---------------------\n(1) Math calculation(2) Search from google\n---------------------\nUsing only the choices above and not prior knowledge, return the choice that is most relevant to the question: 'calculate 2 + 2'\n\n\nThe output should be ONLY JSON formatted as a JSON instance.\n\nHere is an example:\n[\n    {\n        \"choice\": 1,\n        \"reason\": \"<insert reason for choice>\"\n    },\n    ...\n]\n",
+          "role": "user"
+        }
+      ]
+    },
+    {
+      "id": "PRESERVE_2",
+      "messages": [
+        {
+          "content": "Some choices are given below. It is provided in a numbered list (1 to 42), where each item in the list corresponds to a summary.\n---------------------\n(1) Math calculation(2) Search from google\n---------------------\nUsing only the choices above and not prior knowledge, return the choice that is most relevant to the question: 'calculate 2 + 2'\n\n\nThe output should be ONLY JSON formatted as a JSON instance.\n\nHere is an example:\n[\n    {\n        \"choice\": 1,\n        \"reason\": \"<insert reason for choice>\"\n    },\n    ...\n]\n",
+          "role": "user"
+        }
+      ]
+    }
+  ],
+  "llmEventEnd": [
+    {
+      "id": "PRESERVE_0",
+      "response": {
+        "raw": null,
+        "message": {
+          "content": "[\n    {\n        \"choice\": 1,\n        \"reason\": \"The question 'calculate 2 + 2' is directly asking for a math calculation, which corresponds to choice 1.\"\n    }\n]",
+          "role": "assistant",
+          "options": {}
+        }
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "response": {
+        "raw": null,
+        "message": {
+          "content": "[\n    {\n        \"choice\": 1,\n        \"reason\": \"The question 'calculate 2 + 2' is asking for a mathematical calculation, which directly corresponds to choice 1: Math calculation.\"\n    }\n]",
+          "role": "assistant",
+          "options": {}
+        }
+      }
+    },
+    {
+      "id": "PRESERVE_2",
+      "response": {
+        "raw": null,
+        "message": {
+          "content": "[\n    {\n        \"choice\": 1,\n        \"reason\": \"The question 'calculate 2 + 2' is asking for a mathematical calculation, which directly corresponds to choice 1: Math calculation.\"\n    }\n]",
+          "role": "assistant",
+          "options": {}
+        }
+      }
+    }
+  ],
+  "llmEventStream": []
+}
@@ -1,6 +1,6 @@
 {
  "name": "llamaindex",
-  "version": "0.5.21",
+  "version": "0.5.25",
  "license": "MIT",
  "type": "module",
  "keywords": [
@@ -59,12 +59,10 @@
    "openai": "^4.57.0",
    "papaparse": "^5.4.1",
    "pathe": "^1.1.2",
-    "pg": "^8.12.0",
-    "pgvector": "^0.2.0",
    "portkey-ai": "0.1.16",
    "rake-modified": "^1.0.8",
    "string-strip-html": "^13.4.8",
-    "tiktoken": "1.0.14",
+    "tiktoken": "^1.0.15",
    "unpdf": "^0.11.0",
    "weaviate-client": "^3.1.4",
    "wikipedia": "^2.1.2",
@@ -72,11 +70,19 @@
    "zod": "^3.23.8"
  },
  "peerDependencies": {
-    "@notionhq/client": "^2.2.15"
+    "@notionhq/client": "^2.2.15",
+    "pg": "^8.12.0",
+    "pgvector": "0.2.0"
  },
  "peerDependenciesMeta": {
    "@notionhq/client": {
      "optional": true
+    },
+    "pg": {
+      "optional": true
+    },
+    "pgvector": {
+      "optional": true
    }
  },
  "devDependencies": {
@@ -85,6 +91,8 @@
    "@swc/core": "^1.7.22",
    "concurrently": "^8.2.2",
    "glob": "^11.0.0",
+    "pg": "^8.12.0",
+    "pgvector": "0.2.0",
    "typescript": "^5.5.4"
  },
  "engines": {
@@ -42,7 +42,7 @@ export class SimpleChatHistory extends ChatHistory {
  messages: ChatMessage[];
  private messagesBefore: number;

-  constructor(init?: Partial<SimpleChatHistory>) {
+  constructor(init?: { messages?: ChatMessage[] | undefined }) {
    super();
    this.messages = init?.messages ?? [];
    this.messagesBefore = this.messages.length;
@@ -118,7 +118,7 @@ export class SummaryChatHistory extends ChatHistory {
      // remove oldest message until the chat history is short enough for the context window
      messagesToSummarize.shift();
    } while (
-      this.tokenizer.encode(promptMessages[0].content).length >
+      this.tokenizer.encode(promptMessages[0]!.content).length >
      this.tokensToSummarize
    );

@@ -146,7 +146,7 @@ export class SummaryChatHistory extends ChatHistory {

  public getLastSummary(): ChatMessage | null {
    const lastSummaryIndex = this.getLastSummaryIndex();
-    return lastSummaryIndex ? this.messages[lastSummaryIndex] : null;
+    return lastSummaryIndex ? this.messages[lastSummaryIndex]! : null;
  }

  private get systemMessages() {
@@ -174,10 +174,10 @@ export class SummaryChatHistory extends ChatHistory {
      // and convert summary message so it can be send to the LLM
      const summaryMessage: ChatMessage = transformSummary
        ? {
-            content: `Summary of the conversation so far: ${this.messages[lastSummaryIndex].content}`,
+            content: `Summary of the conversation so far: ${this.messages[lastSummaryIndex]!.content}`,
            role: "system",
          }
-        : this.messages[lastSummaryIndex];
+        : this.messages[lastSummaryIndex]!;
      return [summaryMessage, ...this.messages.slice(lastSummaryIndex + 1)];
    }
  }
@@ -13,6 +13,8 @@ export type RetrieveParams = {
 export interface BaseRetriever {
  retrieve(params: RetrieveParams): Promise<NodeWithScore[]>;

-  // to be deprecated soon
-  serviceContext?: ServiceContext;
+  /**
+   * @deprecated to be deprecated soon
+   */
+  serviceContext?: ServiceContext | undefined;
 }
@@ -1,10 +1,10 @@
 import type { BaseEmbedding } from "@llamaindex/core/embeddings";
+import { PromptHelper } from "@llamaindex/core/indices";
 import type { LLM } from "@llamaindex/core/llms";
 import {
  type NodeParser,
  SentenceSplitter,
 } from "@llamaindex/core/node-parser";
-import { PromptHelper } from "./PromptHelper.js";
 import { OpenAIEmbedding } from "./embeddings/OpenAIEmbedding.js";
 import { OpenAI } from "./llm/openai.js";

@@ -4,7 +4,7 @@ import {
 } from "@llamaindex/core/global";
 import { OpenAI } from "./llm/openai.js";

-import { PromptHelper } from "./PromptHelper.js";
+import { PromptHelper } from "@llamaindex/core/indices";

 import type { BaseEmbedding } from "@llamaindex/core/embeddings";
 import type { LLM } from "@llamaindex/core/llms";
@@ -27,13 +27,12 @@ export type PromptConfig = {

 export interface Config {
  prompt: PromptConfig;
-  llm: LLM | null;
  promptHelper: PromptHelper | null;
  embedModel: BaseEmbedding | null;
  nodeParser: NodeParser | null;
  callbackManager: CallbackManager | null;
-  chunkSize?: number;
-  chunkOverlap?: number;
+  chunkSize: number | undefined;
+  chunkOverlap: number | undefined;
 }

 /**
@@ -41,12 +40,10 @@ export interface Config {
 */
 class GlobalSettings implements Config {
  #prompt: PromptConfig = {};
-  #llm: LLM | null = null;
  #promptHelper: PromptHelper | null = null;
  #nodeParser: NodeParser | null = null;
  #chunkOverlap?: number;

-  #llmAsyncLocalStorage = new AsyncLocalStorage<LLM>();
  #promptHelperAsyncLocalStorage = new AsyncLocalStorage<PromptHelper>();
  #nodeParserAsyncLocalStorage = new AsyncLocalStorage<NodeParser>();
  #chunkOverlapAsyncLocalStorage = new AsyncLocalStorage<number>();
@@ -62,19 +59,21 @@ class GlobalSettings implements Config {
  }

  get llm(): LLM {
-    if (this.#llm === null) {
-      this.#llm = new OpenAI();
+    // fixme: we might need check internal error instead of try-catch here
+    try {
+      CoreSettings.llm;
+    } catch (error) {
+      CoreSettings.llm = new OpenAI();
    }
-
-    return this.#llmAsyncLocalStorage.getStore() ?? this.#llm;
+    return CoreSettings.llm;
  }

  set llm(llm: LLM) {
-    this.#llm = llm;
+    CoreSettings.llm = llm;
  }

  withLLM<Result>(llm: LLM, fn: () => Result): Result {
-    return this.#llmAsyncLocalStorage.run(llm, fn);
+    return CoreSettings.withLLM(llm, fn);
  }

  get promptHelper(): PromptHelper {
@@ -159,7 +158,9 @@ class GlobalSettings implements Config {
  }

  set chunkOverlap(chunkOverlap: number | undefined) {
-    this.#chunkOverlap = chunkOverlap;
+    if (typeof chunkOverlap === "number") {
+      this.#chunkOverlap = chunkOverlap;
+    }
  }

  withChunkOverlap<Result>(chunkOverlap: number, fn: () => Result): Result {
@@ -53,7 +53,7 @@ export function createTaskOutputStream<
        nextSteps: new Set(),
      };
      if (steps.length > 0) {
-        step.prevStep = steps[steps.length - 1];
+        step.prevStep = steps[steps.length - 1]!;
      }
      const taskOutputs: TaskStepOutput<
        Model,
@@ -77,7 +77,7 @@ export function createTaskOutputStream<
      context.logger.log("Finished step(id, %s).", step.id);
      // fixme: support multi-thread when there are multiple outputs
      // todo: for now we pretend there is only one task output
-      const { isLast, taskStep } = taskOutputs[0];
+      const { isLast, taskStep } = taskOutputs[0]!;
      context = {
        ...taskStep.context,
        store: {
@@ -89,8 +89,8 @@ function extractFinalResponse(
    );
  }

-  const thought = match[1].trim();
-  const answer = match[2].trim();
+  const thought = match[1]!.trim();
+  const answer = match[2]!.trim();
  return [thought, answer];
 }

@@ -108,9 +108,9 @@ function extractToolUse(
    );
  }

-  const thought = match[1].trim();
-  const action = match[2].trim();
-  const actionInput = match[3].trim();
+  const thought = match[1]!.trim();
+  const action = match[2]!.trim();
+  const actionInput = match[3]!.trim();
  return [thought, action, actionInput];
 }

@@ -12,9 +12,14 @@ export class LLamaCloudFileService {
  public static async getAllProjectsWithPipelines() {
    initService();
    try {
-      const projects = await ProjectsService.listProjectsApiV1ProjectsGet();
-      const pipelines =
-        await PipelinesService.searchPipelinesApiV1PipelinesGet();
+      const { data: projects } =
+        await ProjectsService.listProjectsApiV1ProjectsGet({
+          throwOnError: true,
+        });
+      const { data: pipelines } =
+        await PipelinesService.searchPipelinesApiV1PipelinesGet({
+          throwOnError: true,
+        });
      return projects.map((project) => ({
        ...project,
        pipelines: pipelines.filter((p) => p.project_id === project.id),
@@ -35,11 +40,12 @@ export class LLamaCloudFileService {
    customMetadata: Record<string, any> = {},
  ) {
    initService();
-    const file = await FilesService.uploadFileApiV1FilesPost({
-      projectId,
-      formData: {
+    const { data: file } = await FilesService.uploadFileApiV1FilesPost({
+      path: { project_id: projectId },
+      body: {
        upload_file: uploadFile,
      },
+      throwOnError: true,
    });
    const files = [
      {
@@ -48,19 +54,24 @@ export class LLamaCloudFileService {
      },
    ];
    await PipelinesService.addFilesToPipelineApiV1PipelinesPipelineIdFilesPut({
-      pipelineId,
-      requestBody: files,
+      path: {
+        pipeline_id: pipelineId,
+      },
+      body: files,
    });

    // Wait 2s for the file to be processed
    const maxAttempts = 20;
    let attempt = 0;
    while (attempt < maxAttempts) {
-      const result =
+      const { data: result } =
        await PipelinesService.getPipelineFileStatusApiV1PipelinesPipelineIdFilesFileIdStatusGet(
          {
-            pipelineId,
-            fileId: file.id,
+            path: {
+              pipeline_id: pipelineId,
+              file_id: file.id,
+            },
+            throwOnError: true,
          },
        );
      if (result.status === "ERROR") {
@@ -83,16 +94,24 @@ export class LLamaCloudFileService {
   */
  public static async getFileUrl(pipelineId: string, filename: string) {
    initService();
-    const allPipelineFiles =
+    const { data: allPipelineFiles } =
      await PipelinesService.listPipelineFilesApiV1PipelinesPipelineIdFilesGet({
-        pipelineId,
+        path: {
+          pipeline_id: pipelineId,
+        },
+        throwOnError: true,
      });
    const file = allPipelineFiles.find((file) => file.name === filename);
    if (!file?.file_id) return null;
-    const fileContent =
+    const { data: fileContent } =
      await FilesService.readFileContentApiV1FilesIdContentGet({
-        id: file.file_id,
-        projectId: file.project_id,
+        path: {
+          id: file.file_id,
+        },
+        query: {
+          project_id: file.project_id,
+        },
+        throwOnError: true,
      });
    return fileContent.url;
  }
@@ -7,7 +7,7 @@ import type { QueryEngine } from "../types.js";
 import type { CloudRetrieveParams } from "./LlamaCloudRetriever.js";
 import { LlamaCloudRetriever } from "./LlamaCloudRetriever.js";
 import { getPipelineCreate } from "./config.js";
-import type { CloudConstructorParams } from "./constants.js";
+import type { CloudConstructorParams } from "./type.js";
 import { getAppBaseUrl, getProjectId, initService } from "./utils.js";

 import { PipelinesService, ProjectsService } from "@llamaindex/cloud/api";
@@ -38,10 +38,13 @@ export class LlamaCloudIndex {
    }

    while (true) {
-      const pipelineStatus =
+      const { data: pipelineStatus } =
        await PipelinesService.getPipelineStatusApiV1PipelinesPipelineIdStatusGet(
          {
-            pipelineId,
+            path: {
+              pipeline_id: pipelineId,
+            },
+            throwOnError: true,
          },
        );

@@ -90,9 +93,14 @@ export class LlamaCloudIndex {
      const docsToRemove = new Set<string>();

      for (const doc of pendingDocs) {
-        const { status } =
+        const {
+          data: { status },
+        } =
          await PipelinesService.getPipelineDocumentStatusApiV1PipelinesPipelineIdDocumentsDocumentIdStatusGet(
-            { pipelineId, documentId: doc },
+            {
+              path: { pipeline_id: pipelineId, document_id: doc },
+              throwOnError: true,
+            },
          );

        if (status === "NOT_STARTED" || status === "IN_PROGRESS") {
@@ -136,12 +144,16 @@ export class LlamaCloudIndex {
    name?: string,
    projectName?: string,
  ): Promise<string> {
-    const pipelines = await PipelinesService.searchPipelinesApiV1PipelinesGet({
-      projectId: await this.getProjectId(projectName),
-      pipelineName: name ?? this.params.name,
-    });
+    const { data: pipelines } =
+      await PipelinesService.searchPipelinesApiV1PipelinesGet({
+        path: {
+          project_id: await this.getProjectId(projectName),
+          project_name: name ?? this.params.name,
+        },
+        throwOnError: true,
+      });

-    return pipelines[0].id;
+    return pipelines[0]!.id;
  }

  public async getProjectId(
@@ -177,26 +189,37 @@ export class LlamaCloudIndex {
      transformations: params.transformations ?? defaultTransformations,
    });

-    const project = await ProjectsService.upsertProjectApiV1ProjectsPut({
-      organizationId: params.organizationId,
-      requestBody: {
-        name: params.projectName ?? "default",
-      },
-    });
+    const { data: project } =
+      await ProjectsService.upsertProjectApiV1ProjectsPut({
+        path: {
+          organization_id: params.organizationId,
+        },
+        body: {
+          name: params.projectName ?? "default",
+        },
+        throwOnError: true,
+      });

    if (!project.id) {
      throw new Error("Project ID should be defined");
    }

-    const pipeline = await PipelinesService.upsertPipelineApiV1PipelinesPut({
-      projectId: project.id,
-      requestBody: {
-        name: params.name,
-        configured_transformations:
-          pipelineCreateParams.configured_transformations,
-        pipeline_type: pipelineCreateParams.pipeline_type,
-      },
-    });
+    const { data: pipeline } =
+      await PipelinesService.upsertPipelineApiV1PipelinesPut({
+        path: {
+          project_id: project.id,
+        },
+        body: pipelineCreateParams.configured_transformations
+          ? {
+              name: params.name,
+              configured_transformations:
+                pipelineCreateParams.configured_transformations,
+            }
+          : {
+              name: params.name,
+            },
+        throwOnError: true,
+      });

    if (!pipeline.id) {
      throw new Error("Pipeline ID must be defined");
@@ -208,8 +231,10 @@ export class LlamaCloudIndex {

    await PipelinesService.upsertBatchPipelineDocumentsApiV1PipelinesPipelineIdDocumentsPut(
      {
-        pipelineId: pipeline.id,
-        requestBody: params.documents.map((doc) => ({
+        path: {
+          pipeline_id: pipeline.id,
+        },
+        body: params.documents.map((doc) => ({
          metadata: doc.metadata,
          text: doc.text,
          excluded_embed_metadata_keys: doc.excludedEmbedMetadataKeys,
@@ -220,10 +245,11 @@ export class LlamaCloudIndex {
    );

    while (true) {
-      const pipelineStatus =
+      const { data: pipelineStatus } =
        await PipelinesService.getPipelineStatusApiV1PipelinesPipelineIdStatusGet(
          {
-            pipelineId: pipeline.id,
+            path: { pipeline_id: pipeline.id },
+            throwOnError: true,
          },
        );

@@ -299,8 +325,10 @@ export class LlamaCloudIndex {

    await PipelinesService.createBatchPipelineDocumentsApiV1PipelinesPipelineIdDocumentsPost(
      {
-        pipelineId: pipelineId,
-        requestBody: [
+        path: {
+          pipeline_id: pipelineId,
+        },
+        body: [
          {
            metadata: document.metadata,
            text: document.text,
@@ -327,8 +355,10 @@ export class LlamaCloudIndex {

    await PipelinesService.deletePipelineDocumentApiV1PipelinesPipelineIdDocumentsDocumentIdDelete(
      {
-        pipelineId,
-        documentId: document.id_,
+        path: {
+          pipeline_id: pipelineId,
+          document_id: document.id_,
+        },
      },
    );

@@ -347,8 +377,10 @@ export class LlamaCloudIndex {

    await PipelinesService.upsertBatchPipelineDocumentsApiV1PipelinesPipelineIdDocumentsPut(
      {
-        pipelineId,
-        requestBody: [
+        path: {
+          pipeline_id: pipelineId,
+        },
+        body: [
          {
            metadata: document.metadata,
            text: document.text,
@@ -4,13 +4,12 @@ import {
  type RetrievalParams,
  type TextNodeWithScore,
 } from "@llamaindex/cloud/api";
-import { Settings } from "@llamaindex/core/global";
+import { DEFAULT_PROJECT_NAME, Settings } from "@llamaindex/core/global";
 import type { NodeWithScore } from "@llamaindex/core/schema";
 import { jsonToNode, ObjectType } from "@llamaindex/core/schema";
 import { extractText, wrapEventCaller } from "@llamaindex/core/utils";
 import type { BaseRetriever, RetrieveParams } from "../Retriever.js";
-import type { ClientParams, CloudConstructorParams } from "./constants.js";
-import { DEFAULT_PROJECT_NAME } from "./constants.js";
+import type { ClientParams, CloudConstructorParams } from "./type.js";
 import { getProjectId, initService } from "./utils.js";

 export type CloudRetrieveParams = Omit<
@@ -60,20 +59,27 @@ export class LlamaCloudRetriever implements BaseRetriever {
    query,
    preFilters,
  }: RetrieveParams): Promise<NodeWithScore[]> {
-    const pipelines = await PipelinesService.searchPipelinesApiV1PipelinesGet({
-      projectId: await getProjectId(this.projectName, this.organizationId),
-      pipelineName: this.pipelineName,
-    });
+    const { data: pipelines } =
+      await PipelinesService.searchPipelinesApiV1PipelinesGet({
+        query: {
+          project_id: await getProjectId(this.projectName, this.organizationId),
+          project_name: this.pipelineName,
+        },
+        throwOnError: true,
+      });

-    if (pipelines.length === 0 || !pipelines[0].id) {
+    if (pipelines.length === 0 || !pipelines[0]!.id) {
      throw new Error(
        `No pipeline found with name ${this.pipelineName} in project ${this.projectName}`,
      );
    }

-    const pipeline =
+    const { data: pipeline } =
      await PipelinesService.getPipelineApiV1PipelinesPipelineIdGet({
-        pipelineId: pipelines[0].id,
+        path: {
+          pipeline_id: pipelines[0]!.id,
+        },
+        throwOnError: true,
      });

    if (!pipeline) {
@@ -82,15 +88,18 @@ export class LlamaCloudRetriever implements BaseRetriever {
      );
    }

-    const results =
+    const { data: results } =
      await PipelinesService.runSearchApiV1PipelinesPipelineIdRetrievePost({
-        pipelineId: pipeline.id,
-        requestBody: {
+        throwOnError: true,
+        path: {
+          pipeline_id: pipeline.id,
+        },
+        body: {
          ...this.retrieveParams,
          query: extractText(query),
          search_filters:
            this.retrieveParams.filters ?? (preFilters as MetadataFilters),
-          dense_similarity_top_k: this.retrieveParams.similarityTopK,
+          dense_similarity_top_k: this.retrieveParams.similarityTopK!,
        },
      });

@@ -1,13 +0,0 @@
-import type { ServiceContext } from "../ServiceContext.js";
-
-export const DEFAULT_PROJECT_NAME = "Default";
-export const DEFAULT_BASE_URL = "https://api.cloud.llamaindex.ai";
-
-export type ClientParams = { apiKey?: string; baseUrl?: string };
-
-export type CloudConstructorParams = {
-  name: string;
-  projectName: string;
-  organizationId?: string;
-  serviceContext?: ServiceContext;
-} & ClientParams;
@@ -1,7 +1,7 @@
-export type { CloudConstructorParams } from "./constants.js";
 export { LLamaCloudFileService } from "./LLamaCloudFileService.js";
 export { LlamaCloudIndex } from "./LlamaCloudIndex.js";
 export {
  LlamaCloudRetriever,
  type CloudRetrieveParams,
 } from "./LlamaCloudRetriever.js";
+export type { CloudConstructorParams } from "./type.js";
@@ -0,0 +1,13 @@
+import type { ServiceContext } from "../ServiceContext.js";
+
+export type ClientParams = {
+  apiKey?: string | undefined;
+  baseUrl?: string | undefined;
+};
+
+export type CloudConstructorParams = {
+  name: string;
+  projectName: string;
+  organizationId?: string | undefined;
+  serviceContext?: ServiceContext | undefined;
+} & ClientParams;
@@ -1,20 +1,33 @@
-import { OpenAPI, ProjectsService } from "@llamaindex/cloud/api";
+import { client, ProjectsService } from "@llamaindex/cloud/api";
+import { DEFAULT_BASE_URL } from "@llamaindex/core/global";
 import { getEnv } from "@llamaindex/env";
-import type { ClientParams } from "./constants.js";
-import { DEFAULT_BASE_URL } from "./constants.js";
+import type { ClientParams } from "./type.js";

 function getBaseUrl(baseUrl?: string): string {
  return baseUrl ?? getEnv("LLAMA_CLOUD_BASE_URL") ?? DEFAULT_BASE_URL;
 }

 export function getAppBaseUrl(): string {
-  return OpenAPI.BASE.replace(/api\./, "");
+  return client.getConfig().baseUrl?.replace(/api\./, "") ?? "";
 }

+// fixme: refactor this to init at the top level or module level
+let initOnce = false;
 export function initService({ apiKey, baseUrl }: ClientParams = {}) {
-  OpenAPI.TOKEN = apiKey ?? getEnv("LLAMA_CLOUD_API_KEY");
-  OpenAPI.BASE = getBaseUrl(baseUrl);
-  if (!OpenAPI.TOKEN) {
+  if (initOnce) {
+    return;
+  }
+  initOnce = true;
+  client.setConfig({
+    baseUrl: getBaseUrl(baseUrl),
+    throwOnError: true,
+  });
+  const token = apiKey ?? getEnv("LLAMA_CLOUD_API_KEY");
+  client.interceptors.request.use((request) => {
+    request.headers.set("Authorization", `Bearer ${token}`);
+    return request;
+  });
+  if (!token) {
    throw new Error(
      "API Key is required for LlamaCloudIndex. Please pass the apiKey parameter",
    );
@@ -25,10 +38,15 @@ export async function getProjectId(
  projectName: string,
  organizationId?: string,
 ): Promise<string> {
-  const projects = await ProjectsService.listProjectsApiV1ProjectsGet({
-    projectName: projectName,
-    organizationId: organizationId,
-  });
+  const { data: projects } = await ProjectsService.listProjectsApiV1ProjectsGet(
+    {
+      path: {
+        project_name: projectName,
+        organization_id: organizationId,
+      },
+      throwOnError: true,
+    },
+  );

  if (projects.length === 0) {
    throw new Error(
@@ -40,7 +58,7 @@ export async function getProjectId(
    );
  }

-  const project = projects[0];
+  const project = projects[0]!;

  if (!project.id) {
    throw new Error(`No project found with name ${projectName}`);
@@ -1,8 +0,0 @@
-export const DEFAULT_CONTEXT_WINDOW = 3900;
-export const DEFAULT_NUM_OUTPUTS = 256;
-
-export const DEFAULT_CHUNK_SIZE = 1024;
-export const DEFAULT_CHUNK_OVERLAP = 20;
-export const DEFAULT_CHUNK_OVERLAP_RATIO = 0.1;
-
-export const DEFAULT_PADDING = 5;
@@ -87,7 +87,7 @@ export class DeepInfraEmbedding extends BaseEmbedding {
  async getTextEmbedding(text: string): Promise<number[]> {
    const texts = mapPrefixWithInputs(this.textPrefix, [text]);
    const embeddings = await this.getDeepInfraEmbedding(texts);
-    return embeddings[0];
+    return embeddings[0]!;
  }

  async getQueryEmbedding(
@@ -97,7 +97,7 @@ export class DeepInfraEmbedding extends BaseEmbedding {
    if (text) {
      const queries = mapPrefixWithInputs(this.queryPrefix, [text]);
      const embeddings = await this.getDeepInfraEmbedding(queries);
-      return embeddings[0];
+      return embeddings[0]!;
    } else {
      return null;
    }
@@ -37,13 +37,13 @@ export class JinaAIEmbedding extends MultiModalEmbedding {

  async getTextEmbedding(text: string): Promise<number[]> {
    const result = await this.getJinaEmbedding({ input: [{ text }] });
-    return result.data[0].embedding;
+    return result.data[0]!.embedding;
  }

  async getImageEmbedding(image: ImageType): Promise<number[]> {
    const img = await this.getImageInput(image);
    const result = await this.getJinaEmbedding({ input: [img] });
-    return result.data[0].embedding;
+    return result.data[0]!.embedding;
  }

  // Retrieve multiple text embeddings in a single request
@@ -81,7 +81,7 @@ export class JinaAIEmbedding extends MultiModalEmbedding {
  ): Promise<{ bytes: string } | { url: string }> {
    if (isLocal(image) || image instanceof Blob) {
      const base64 = await imageToDataUrl(image);
-      const bytes = base64.split(",")[1];
+      const bytes = base64.split(",")[1]!;
      return { bytes };
    } else {
      return { url: image.toString() };
@@ -105,8 +105,10 @@ export class MixedbreadAIEmbeddings extends BaseEmbedding {
    }

    this.embedBatchSize = params?.embedBatchSize ?? 128;
-    this.embedInfo = params?.embedInfo;
-    this.requestParams = {
+    if (params?.embedInfo) {
+      this.embedInfo = params?.embedInfo;
+    }
+    this.requestParams = <EmbeddingsRequestWithoutInput>{
      model: params?.model ?? "mixedbread-ai/mxbai-embed-large-v1",
      normalized: params?.normalized,
      dimensions: params?.dimensions,
@@ -123,10 +125,16 @@ export class MixedbreadAIEmbeddings extends BaseEmbedding {
        "user-agent": "@mixedbread-ai/llamaindex-ts-sdk",
      },
    };
-    this.client = new MixedbreadAIClient({
-      apiKey,
-      environment: params?.baseUrl,
-    });
+    this.client = new MixedbreadAIClient(
+      params?.baseUrl
+        ? {
+            apiKey,
+            environment: params?.baseUrl,
+          }
+        : {
+            apiKey,
+          },
+    );
  }

  /**
@@ -140,7 +148,7 @@ export class MixedbreadAIEmbeddings extends BaseEmbedding {
   * console.log(result);
   */
  async getTextEmbedding(text: string): Promise<number[]> {
-    return (await this.getTextEmbeddings([text]))[0];
+    return (await this.getTextEmbeddings([text]))[0]!;
  }

  /**
@@ -39,7 +39,7 @@ export abstract class MultiModalEmbedding extends BaseEmbedding {
      _options,
    );
    for (let i = 0; i < textNodes.length; i++) {
-      textNodes[i].embedding = embeddings[i];
+      textNodes[i]!.embedding = embeddings[i];
    }

    const imageEmbeddings = await batchEmbeddings(
@@ -49,7 +49,7 @@ export abstract class MultiModalEmbedding extends BaseEmbedding {
      _options,
    );
    for (let i = 0; i < imageNodes.length; i++) {
-      imageNodes[i].embedding = imageEmbeddings[i];
+      imageNodes[i]!.embedding = imageEmbeddings[i];
    }

    return nodes;
@@ -36,21 +36,20 @@ export class OpenAIEmbedding extends BaseEmbedding {
  /** embeddding model. defaults to "text-embedding-ada-002" */
  model: string;
  /** number of dimensions of the resulting vector, for models that support choosing fewer dimensions. undefined will default to model default */
-  dimensions: number | undefined;
+  dimensions?: number | undefined;

  // OpenAI session params

  /** api key */
-  apiKey?: string = undefined;
+  apiKey?: string | undefined = undefined;
  /** maximum number of retries, default 10 */
  maxRetries: number;
  /** timeout in ms, default 60 seconds  */
-  timeout?: number;
+  timeout?: number | undefined;
  /** other session options for OpenAI */
-  additionalSessionOptions?: Omit<
-    Partial<OpenAIClientOptions>,
-    "apiKey" | "maxRetries" | "timeout"
-  >;
+  additionalSessionOptions?:
+    | Omit<Partial<OpenAIClientOptions>, "apiKey" | "maxRetries" | "timeout">
+    | undefined;

  /** session object */
  session: OpenAISession;
@@ -119,11 +118,18 @@ export class OpenAIEmbedding extends BaseEmbedding {
    // TODO: ensure this for every sub class by calling it in the base class
    input = this.truncateMaxTokens(input);

-    const { data } = await this.session.openai.embeddings.create({
-      model: this.model,
-      dimensions: this.dimensions, // only sent to OpenAI if set by user
-      input,
-    });
+    const { data } = await this.session.openai.embeddings.create(
+      this.dimensions
+        ? {
+            model: this.model,
+            dimensions: this.dimensions, // only sent to OpenAI if set by user
+            input,
+          }
+        : {
+            model: this.model,
+            input,
+          },
+    );

    return data.map((d) => d.embedding);
  }
@@ -141,6 +147,6 @@ export class OpenAIEmbedding extends BaseEmbedding {
   * @param texts
   */
  async getTextEmbedding(text: string): Promise<number[]> {
-    return (await this.getOpenAIEmbedding([text]))[0];
+    return (await this.getOpenAIEmbedding([text]))[0]!;
  }
 }
@@ -38,16 +38,16 @@ export class ContextChatEngine extends PromptMixin implements ChatEngine {
  chatModel: LLM;
  chatHistory: ChatHistory;
  contextGenerator: ContextGenerator & PromptMixin;
-  systemPrompt?: string;
+  systemPrompt?: string | undefined;

  constructor(init: {
    retriever: BaseRetriever;
-    chatModel?: LLM;
-    chatHistory?: ChatMessage[];
-    contextSystemPrompt?: ContextSystemPrompt;
-    nodePostprocessors?: BaseNodePostprocessor[];
-    systemPrompt?: string;
-    contextRole?: MessageType;
+    chatModel?: LLM | undefined;
+    chatHistory?: ChatMessage[] | undefined;
+    contextSystemPrompt?: ContextSystemPrompt | undefined;
+    nodePostprocessors?: BaseNodePostprocessor[] | undefined;
+    systemPrompt?: string | undefined;
+    contextRole?: MessageType | undefined;
  }) {
    super();
    this.chatModel = init.chatModel ?? Settings.llm;
@@ -23,10 +23,10 @@ export class DefaultContextGenerator

  constructor(init: {
    retriever: BaseRetriever;
-    contextSystemPrompt?: ContextSystemPrompt;
-    nodePostprocessors?: BaseNodePostprocessor[];
-    contextRole?: MessageType;
-    metadataMode?: MetadataMode;
+    contextSystemPrompt?: ContextSystemPrompt | undefined;
+    nodePostprocessors?: BaseNodePostprocessor[] | undefined;
+    contextRole?: MessageType | undefined;
+    metadataMode?: MetadataMode | undefined;
  }) {
    super();

@@ -64,9 +64,9 @@ export class RouterQueryEngine extends PromptMixin implements QueryEngine {
  constructor(init: {
    selector: BaseSelector;
    queryEngineTools: RouterQueryEngineTool[];
-    serviceContext?: ServiceContext;
-    summarizer?: TreeSummarize;
-    verbose?: boolean;
+    serviceContext?: ServiceContext | undefined;
+    summarizer?: TreeSummarize | undefined;
+    verbose?: boolean | undefined;
  }) {
    super();

@@ -138,14 +138,14 @@ export class RouterQueryEngine extends PromptMixin implements QueryEngine {
    if (result.selections.length > 1) {
      const responses: EngineResponse[] = [];
      for (let i = 0; i < result.selections.length; i++) {
-        const engineInd = result.selections[i];
-        const logStr = `Selecting query engine ${engineInd.index}: ${result.selections[i].index}.`;
+        const engineInd = result.selections[i]!;
+        const logStr = `Selecting query engine ${engineInd.index}: ${result.selections[i]!.index}.`;

        if (this.verbose) {
          console.log(logStr + "\n");
        }

-        const selectedQueryEngine = this.queryEngines[engineInd.index];
+        const selectedQueryEngine = this.queryEngines[engineInd.index]!;
        responses.push(
          await selectedQueryEngine.query({
            query: extractText(query),
@@ -163,15 +163,15 @@ export class RouterQueryEngine extends PromptMixin implements QueryEngine {

        return finalResponse;
      } else {
-        return responses[0];
+        return responses[0]!;
      }
    } else {
      let selectedQueryEngine;

      try {
-        selectedQueryEngine = this.queryEngines[result.selections[0].index];
+        selectedQueryEngine = this.queryEngines[result.selections[0]!.index];

-        const logStr = `Selecting query engine ${result.selections[0].index}: ${result.selections[0].reason}`;
+        const logStr = `Selecting query engine ${result.selections[0]!.index}: ${result.selections[0]!.reason}`;

        if (this.verbose) {
          console.log(logStr + "\n");
@@ -22,16 +22,16 @@ export class FaithfulnessEvaluator
  extends PromptMixin
  implements BaseEvaluator
 {
-  private serviceContext?: ServiceContext;
+  private serviceContext?: ServiceContext | undefined;
  private raiseError: boolean;
  private evalTemplate: FaithfulnessTextQAPrompt;
  private refineTemplate: FaithfulnessRefinePrompt;

  constructor(params?: {
-    serviceContext?: ServiceContext;
-    raiseError?: boolean;
-    faithfulnessSystemPrompt?: FaithfulnessTextQAPrompt;
-    faithFulnessRefinePrompt?: FaithfulnessRefinePrompt;
+    serviceContext?: ServiceContext | undefined;
+    raiseError?: boolean | undefined;
+    faithfulnessSystemPrompt?: FaithfulnessTextQAPrompt | undefined;
+    faithFulnessRefinePrompt?: FaithfulnessRefinePrompt | undefined;
  }) {
    super();
    this.serviceContext = params?.serviceContext;
@@ -16,14 +16,14 @@ import type {
 } from "./types.js";

 type RelevancyParams = {
-  serviceContext?: ServiceContext;
-  raiseError?: boolean;
-  evalTemplate?: RelevancyEvalPrompt;
-  refineTemplate?: RelevancyRefinePrompt;
+  serviceContext?: ServiceContext | undefined;
+  raiseError?: boolean | undefined;
+  evalTemplate?: RelevancyEvalPrompt | undefined;
+  refineTemplate?: RelevancyRefinePrompt | undefined;
 };

 export class RelevancyEvaluator extends PromptMixin implements BaseEvaluator {
-  private serviceContext?: ServiceContext;
+  private serviceContext?: ServiceContext | undefined;
  private raiseError: boolean;

  private evalTemplate: RelevancyEvalPrompt;
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
github-actions[bot]	28b877e31f	Release 0.5.25 (#1182 ) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2024-09-11 12:08:39 -07:00
Alex Yang	4389b80a52	docs: update README.md (#1183 )	2024-09-11 11:07:00 -07:00
Alex Yang	d3bc663951	fix: vector store cleanup (#1175 )	2024-09-11 10:20:55 -07:00
Kieran Simkin	4810364788	fix: handle `RouterQueryEngine` with string query (#1181 )	2024-09-11 10:19:59 -07:00
github-actions[bot]	2dcad52dd9	Release 0.5.24 (#1178 ) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2024-09-10 23:48:51 -07:00
Alex Yang	0bf8d80b12	fix: llama cloud api build	2024-09-10 23:39:58 -07:00
github-actions[bot]	e4bba02aec	Release 0.5.23 (#1174 ) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2024-09-10 11:41:49 -07:00
Alex Yang	1caa0da657	chore: fix changeset	2024-09-10 11:34:07 -07:00
Alex Yang	711c814bb2	fix: patch `python-format-js` (#1173 )	2024-09-10 09:49:36 -07:00
Alex Yang	5b832eb927	fix: strict type check (#1170 )	2024-09-10 09:28:44 -07:00
Alex Yang	49988431f6	refactor: move settings.llm into core package (#1165 )	2024-09-09 10:45:57 -07:00
Alex Yang	72d65dd51a	docs: fix example (#1168 )	2024-09-09 10:45:30 -07:00
Alex Yang	553bc55b19	refactor: move `PromptHelper` into core package (#1166 )	2024-09-09 10:15:21 -07:00
Alex Yang	fc6f69833c	fix: example code (#1167 )	2024-09-09 10:11:58 -07:00
github-actions[bot]	c7fd06841f	Release 0.5.22 (#1164 ) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2024-09-09 09:27:26 -07:00
Thuc Pham	4648da6849	fix: wrong tiktoken version caused NextJs CL template run fail (#1162 )	2024-09-09 09:23:13 -07:00
Fabian Wimmer	0188cf3bb6	docs: fix typos, add API references (#1161 )	2024-09-09 11:40:05 +07:00
Alex Yang	e0b4f9c047	refactor: move constant into core module (#1158 )	2024-09-06 20:29:08 -07:00