Release 0.5.22 (#1164)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

apps/docs/CHANGELOG.md

@@ -1,5 +1,12 @@
 # docs
 
+## 0.0.63
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.0.62
 
 ### Patch Changes

apps/docs/docs/introduction.md

@@ -21,7 +21,7 @@ LlamaIndex.TS handles several major use cases:
 
 - **Structured Data Extraction**: turning complex, unstructured and semi-structured data into uniform, programmatically accessible formats.
 - **Retrieval-Augmented Generation (RAG)**: answering queries across your internal data by providing LLMs with up-to-date, semantically relevant context including Question and Answer systems and chat bots.
-- **Autonomous Agents**: building software that is capable of intelligently selecting and using tools to accomplish tasks in an interative, unsupervised manner.
+- **Autonomous Agents**: building software that is capable of intelligently selecting and using tools to accomplish tasks in an interactive, unsupervised manner.
 
 ## 👨‍👩‍👧‍👦 Who is LlamaIndex for?
 

apps/docs/docs/modules/chat_engine.md

@@ -27,3 +27,4 @@ for await (const chunk of stream) {
 
 - [ContextChatEngine](../api/classes/ContextChatEngine.md)
 - [CondenseQuestionChatEngine](../api/classes/ContextChatEngine.md)
+- [SimpleChatEngine](../api/classes/SimpleChatEngine.md)

apps/docs/docs/modules/data_index.md

@@ -21,3 +21,4 @@ const index = await VectorStoreIndex.fromDocuments([document]);
 
 - [SummaryIndex](../api/classes/SummaryIndex.md)
 - [VectorStoreIndex](../api/classes/VectorStoreIndex.md)
+- [KeywordTableIndex](../api/classes/KeywordTableIndex.md)

apps/docs/docs/modules/embeddings/available_embeddings/mixedbreadai.md

@@ -98,3 +98,7 @@ Use the `embedDocuments` method to generate embeddings for the texts.
 const result = await embeddings.embedDocuments(texts);
 console.log(result); // Perfectly customized embeddings, ready to serve.
 ```
+
+## API Reference
+
+- [MixedbreadAIEmbeddings](../../../api/classes/MixedbreadAIEmbeddings.md)

apps/docs/docs/modules/evaluation/index.md

@@ -2,7 +2,7 @@
 
 ## Concept
 
-Evaluation and benchmarking are crucial concepts in LLM development. To improve the perfomance of an LLM app (RAG, agents) you must have a way to measure it.
+Evaluation and benchmarking are crucial concepts in LLM development. To improve the performance of an LLM app (RAG, agents) you must have a way to measure it.
 
 LlamaIndex offers key modules to measure the quality of generated results. We also offer key modules to measure retrieval quality.
 

apps/docs/docs/modules/ingestion_pipeline/transformations.md

@@ -36,7 +36,7 @@ main().catch(console.error);
 
 You can implement any transformation yourself by implementing the `TransformComponent`.
 
-The following custom transformation will remove any special characters or punctutation in text.
+The following custom transformation will remove any special characters or punctuation in text.
 
 ```ts
 import { TransformComponent, TextNode } from "llamaindex";
@@ -75,3 +75,7 @@ async function main() {
 
 main().catch(console.error);
 ```
+
+## API Reference
+
+- [TransformComponent](../../api/classes/TransformComponent.md)

apps/docs/docs/modules/llms/available_llms/deepseek.md

@@ -1,5 +1,7 @@
 # DeepSeek LLM
 
+[DeepSeek Platform](https://platform.deepseek.com/)
+
 ## Usage
 
 ```ts
@@ -45,6 +47,6 @@ Currently does not support function calling.
 
 [Currently does not support json-output param while still is very good at json generating.](https://platform.deepseek.com/api-docs/faq#does-your-api-support-json-output)
 
-## API platform
+## API Reference
 
-- [DeepSeek platform](https://platform.deepseek.com/)
+- [DeepSeekLLM](../../../api/classes/DeepSeekLLM.md)

apps/docs/docs/modules/node_postprocessors/mixedbreadiai_reranker.md

@@ -163,3 +163,7 @@ Use the `rerank` method to reorder the documents based on the query.
 const result = await reranker.rerank(documents, query);
 console.log(result); // Perfectly customized results, ready to serve.
 ```
+
+## API Reference
+
+- [MixedbreadAIReranker](../../api/classes/MixedbreadAIReranker.md)

apps/docs/docs/modules/query_engines/index.md

@@ -1,6 +1,6 @@
 # QueryEngine
 
-A query engine wraps a `Retriever` and a `ResponseSynthesizer` into a pipeline, that will use the query string to fetech nodes and then send them to the LLM to generate a response.
+A query engine wraps a `Retriever` and a `ResponseSynthesizer` into a pipeline, that will use the query string to fetch nodes and then send them to the LLM to generate a response.
 
 ```typescript
 const queryEngine = index.asQueryEngine();

apps/docs/docs/modules/retriever.md

@@ -4,7 +4,14 @@ sidebar_position: 5
 
 # Retriever
 
-A retriever in LlamaIndex is what is used to fetch `Node`s from an index using a query string. Aa `VectorIndexRetriever` will fetch the top-k most similar nodes. Meanwhile, a `SummaryIndexRetriever` will fetch all nodes no matter the query.
+A retriever in LlamaIndex is what is used to fetch `Node`s from an index using a query string.
+
+- [VectorIndexRetriever](../api/classes/VectorIndexRetriever.md) will fetch the top-k most similar nodes. Ideal for dense retrieval to find most relevant nodes.
+- [SummaryIndexRetriever](../api/classes/SummaryIndexRetriever.md) will fetch all nodes no matter the query. Ideal when complete context is necessary, e.g. analyzing large datasets.
+- [SummaryIndexLLMRetriever](../api/classes/SummaryIndexLLMRetriever.md) utilizes an LLM to score and filter nodes based on relevancy to the query.
+- [KeywordTableLLMRetriever](../api/classes/KeywordTableLLMRetriever.md) uses an LLM to extract keywords from the query and retrieve relevant nodes based on keyword matches.
+- [KeywordTableSimpleRetriever](../api/classes/KeywordTableSimpleRetriever.md) uses a basic frequency-based approach to extract keywords and retrieve nodes.
+- [KeywordTableRAKERetriever](../api/classes/KeywordTableRAKERetriever.md) uses the RAKE (Rapid Automatic Keyword Extraction) algorithm to extract keywords from the query, focusing on co-occurrence and context for keyword-based retrieval.
 
 ```typescript
 const retriever = vectorIndex.asRetriever({
@@ -14,9 +21,3 @@ const retriever = vectorIndex.asRetriever({
 // Fetch nodes!
 const nodesWithScore = await retriever.retrieve({ query: "query string" });
 ```
-
-## API Reference
-
-- [SummaryIndexRetriever](../api/classes/SummaryIndexRetriever.md)
-- [SummaryIndexLLMRetriever](../api/classes/SummaryIndexLLMRetriever.md)
-- [VectorIndexRetriever](../api/classes/VectorIndexRetriever.md)

apps/docs/package.json

@@ -1,6 +1,6 @@
 {
   "name": "docs",
-  "version": "0.0.62",
+  "version": "0.0.63",
   "private": true,
   "scripts": {
     "docusaurus": "docusaurus",

packages/autotool/examples/01_node/CHANGELOG.md

@@ -1,5 +1,13 @@
 # @llamaindex/autotool-01-node-example
 
+## 0.0.3
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+  - @llamaindex/autotool@2.0.1
+
 ## 0.0.2
 
 ### Patch Changes

packages/autotool/examples/01_node/package.json

@@ -13,5 +13,5 @@
   "scripts": {
     "start": "node --import tsx --import @llamaindex/autotool/node ./src/index.ts"
   },
-  "version": "0.0.2"
+  "version": "0.0.3"
 }

packages/autotool/examples/02_nextjs/CHANGELOG.md

@@ -1,5 +1,13 @@
 # @llamaindex/autotool-02-next-example
 
+## 0.1.47
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+  - @llamaindex/autotool@2.0.1
+
 ## 0.1.46
 
 ### Patch Changes

packages/autotool/examples/02_nextjs/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/autotool-02-next-example",
   "private": true,
-  "version": "0.1.46",
+  "version": "0.1.47",
   "scripts": {
     "dev": "next dev",
     "build": "next build",

packages/autotool/package.json

@@ -51,7 +51,7 @@
     "unplugin": "^1.12.2"
   },
   "peerDependencies": {
-    "llamaindex": "^0.5.21",
+    "llamaindex": "^0.5.22",
     "openai": "^4",
     "typescript": "^4"
   },

packages/community/CHANGELOG.md

@@ -1,5 +1,13 @@
 # @llamaindex/community
 
+## 0.0.32
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - @llamaindex/env@0.1.10
+  - @llamaindex/core@0.1.11
+
 ## 0.0.31
 
 ### Patch Changes

packages/community/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/community",
   "description": "Community package for LlamaIndexTS",
-  "version": "0.0.31",
+  "version": "0.0.32",
   "type": "module",
   "types": "dist/type/index.d.ts",
   "main": "dist/cjs/index.js",

packages/core/CHANGELOG.md

@@ -1,5 +1,12 @@
 # @llamaindex/core
 
+## 0.1.11
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - @llamaindex/env@0.1.10
+
 ## 0.1.10
 
 ### Patch Changes

packages/core/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/core",
   "type": "module",
-  "version": "0.1.10",
+  "version": "0.1.11",
   "description": "LlamaIndex Core Module",
   "exports": {
     "./node-parser": {

packages/core/src/global/constants.ts

@@ -0,0 +1,24 @@
+import { path } from "@llamaindex/env";
+
+//#region llm
+export const DEFAULT_CONTEXT_WINDOW = 3900;
+export const DEFAULT_NUM_OUTPUTS = 256;
+export const DEFAULT_CHUNK_SIZE = 1024;
+export const DEFAULT_CHUNK_OVERLAP = 20;
+export const DEFAULT_CHUNK_OVERLAP_RATIO = 0.1;
+export const DEFAULT_PADDING = 5;
+//#endregion
+//#region storage
+export const DEFAULT_COLLECTION = "data";
+export const DEFAULT_PERSIST_DIR = path.join("./storage");
+export const DEFAULT_INDEX_STORE_PERSIST_FILENAME = "index_store.json";
+export const DEFAULT_DOC_STORE_PERSIST_FILENAME = "doc_store.json";
+export const DEFAULT_VECTOR_STORE_PERSIST_FILENAME = "vector_store.json";
+export const DEFAULT_GRAPH_STORE_PERSIST_FILENAME = "graph_store.json";
+export const DEFAULT_NAMESPACE = "docstore";
+export const DEFAULT_IMAGE_VECTOR_NAMESPACE = "images";
+//#endregion
+//#region llama cloud
+export const DEFAULT_PROJECT_NAME = "Default";
+export const DEFAULT_BASE_URL = "https://api.cloud.llamaindex.ai";
+//#endregion

packages/core/src/global/index.ts

@@ -1,3 +1,4 @@
+export * from "./constants";
 export { Settings } from "./settings";
 export { CallbackManager } from "./settings/callback-manager";
 export type {

packages/env/CHANGELOG.md

@@ -1,5 +1,11 @@
 # @llamaindex/env
 
+## 0.1.10
+
+### Patch Changes
+
+- 4648da6: fix: wrong tiktoken version caused NextJs CL template run fail
+
 ## 0.1.9
 
 ### Patch Changes

packages/env/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/env",
   "description": "environment wrapper, supports all JS environment including node, deno, bun, edge runtime, and cloudflare worker",
-  "version": "0.1.9",
+  "version": "0.1.10",
   "type": "module",
   "types": "dist/type/index.d.ts",
   "main": "dist/cjs/index.js",
@@ -82,7 +82,7 @@
     "@aws-crypto/sha256-js": "^5.2.0",
     "js-tiktoken": "^1.0.12",
     "pathe": "^1.1.2",
-    "tiktoken": "1.0.14"
+    "tiktoken": "^1.0.15"
   },
   "peerDependenciesMeta": {
     "@aws-crypto/sha256-js": {

packages/experimental/CHANGELOG.md

@@ -1,5 +1,12 @@
 # @llamaindex/experimental
 
+## 0.0.72
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.0.71
 
 ### Patch Changes

packages/experimental/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/experimental",
   "description": "Experimental package for LlamaIndexTS",
-  "version": "0.0.71",
+  "version": "0.0.72",
   "type": "module",
   "types": "dist/type/index.d.ts",
   "main": "dist/cjs/index.js",

packages/llamaindex/CHANGELOG.md

@@ -1,5 +1,14 @@
 # llamaindex
 
+## 0.5.22
+
+### Patch Changes
+
+- 4648da6: fix: wrong tiktoken version caused NextJs CL template run fail
+- Updated dependencies [4648da6]
+  - @llamaindex/env@0.1.10
+  - @llamaindex/core@0.1.11
+
 ## 0.5.21
 
 ### Patch Changes

packages/llamaindex/e2e/examples/cloudflare-worker-agent/CHANGELOG.md

@@ -1,5 +1,12 @@
 # @llamaindex/cloudflare-worker-agent-test
 
+## 0.0.56
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.0.55
 
 ### Patch Changes

packages/llamaindex/e2e/examples/cloudflare-worker-agent/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/cloudflare-worker-agent-test",
-  "version": "0.0.55",
+  "version": "0.0.56",
   "type": "module",
   "private": true,
   "scripts": {

packages/llamaindex/e2e/examples/nextjs-agent/CHANGELOG.md

@@ -1,5 +1,12 @@
 # @llamaindex/next-agent-test
 
+## 0.1.56
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.1.55
 
 ### Patch Changes

packages/llamaindex/e2e/examples/nextjs-agent/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/next-agent-test",
-  "version": "0.1.55",
+  "version": "0.1.56",
   "private": true,
   "scripts": {
     "dev": "next dev",

packages/llamaindex/e2e/examples/nextjs-edge-runtime/CHANGELOG.md

@@ -1,5 +1,12 @@
 # test-edge-runtime
 
+## 0.1.55
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.1.54
 
 ### Patch Changes

packages/llamaindex/e2e/examples/nextjs-edge-runtime/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/nextjs-edge-runtime-test",
-  "version": "0.1.54",
+  "version": "0.1.55",
   "private": true,
   "scripts": {
     "dev": "next dev",

packages/llamaindex/e2e/examples/nextjs-node-runtime/CHANGELOG.md

@@ -1,5 +1,12 @@
 # @llamaindex/next-node-runtime
 
+## 0.0.37
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.0.36
 
 ### Patch Changes

packages/llamaindex/e2e/examples/nextjs-node-runtime/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/next-node-runtime-test",
-  "version": "0.0.36",
+  "version": "0.0.37",
   "private": true,
   "scripts": {
     "dev": "next dev",

packages/llamaindex/e2e/examples/waku-query-engine/CHANGELOG.md

@@ -1,5 +1,12 @@
 # @llamaindex/waku-query-engine-test
 
+## 0.0.56
+
+### Patch Changes
+
+- Updated dependencies [4648da6]
+  - llamaindex@0.5.22
+
 ## 0.0.55
 
 ### Patch Changes

packages/llamaindex/e2e/examples/waku-query-engine/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/waku-query-engine-test",
-  "version": "0.0.55",
+  "version": "0.0.56",
   "type": "module",
   "private": true,
   "scripts": {

packages/llamaindex/package.json

@@ -1,6 +1,6 @@
 {
   "name": "llamaindex",
-  "version": "0.5.21",
+  "version": "0.5.22",
   "license": "MIT",
   "type": "module",
   "keywords": [
@@ -64,7 +64,7 @@
     "portkey-ai": "0.1.16",
     "rake-modified": "^1.0.8",
     "string-strip-html": "^13.4.8",
-    "tiktoken": "1.0.14",
+    "tiktoken": "^1.0.15",
     "unpdf": "^0.11.0",
     "weaviate-client": "^3.1.4",
     "wikipedia": "^2.1.2",

packages/llamaindex/src/PromptHelper.ts

@@ -1,12 +1,12 @@
-import { SentenceSplitter } from "@llamaindex/core/node-parser";
-import type { PromptTemplate } from "@llamaindex/core/prompts";
-import { type Tokenizer, tokenizers } from "@llamaindex/env";
 import {
   DEFAULT_CHUNK_OVERLAP_RATIO,
   DEFAULT_CONTEXT_WINDOW,
   DEFAULT_NUM_OUTPUTS,
   DEFAULT_PADDING,
-} from "./constants.js";
+} from "@llamaindex/core/global";
+import { SentenceSplitter } from "@llamaindex/core/node-parser";
+import type { PromptTemplate } from "@llamaindex/core/prompts";
+import { type Tokenizer, tokenizers } from "@llamaindex/env";
 
 /**
  * Get the empty prompt text given a prompt.

packages/llamaindex/src/cloud/LlamaCloudIndex.ts

@@ -7,7 +7,7 @@ import type { QueryEngine } from "../types.js";
 import type { CloudRetrieveParams } from "./LlamaCloudRetriever.js";
 import { LlamaCloudRetriever } from "./LlamaCloudRetriever.js";
 import { getPipelineCreate } from "./config.js";
-import type { CloudConstructorParams } from "./constants.js";
+import type { CloudConstructorParams } from "./type.js";
 import { getAppBaseUrl, getProjectId, initService } from "./utils.js";
 
 import { PipelinesService, ProjectsService } from "@llamaindex/cloud/api";

packages/llamaindex/src/cloud/LlamaCloudRetriever.ts

@@ -4,13 +4,12 @@ import {
   type RetrievalParams,
   type TextNodeWithScore,
 } from "@llamaindex/cloud/api";
-import { Settings } from "@llamaindex/core/global";
+import { DEFAULT_PROJECT_NAME, Settings } from "@llamaindex/core/global";
 import type { NodeWithScore } from "@llamaindex/core/schema";
 import { jsonToNode, ObjectType } from "@llamaindex/core/schema";
 import { extractText, wrapEventCaller } from "@llamaindex/core/utils";
 import type { BaseRetriever, RetrieveParams } from "../Retriever.js";
-import type { ClientParams, CloudConstructorParams } from "./constants.js";
-import { DEFAULT_PROJECT_NAME } from "./constants.js";
+import type { ClientParams, CloudConstructorParams } from "./type.js";
 import { getProjectId, initService } from "./utils.js";
 
 export type CloudRetrieveParams = Omit<

packages/llamaindex/src/cloud/index.ts

@@ -1,7 +1,7 @@
-export type { CloudConstructorParams } from "./constants.js";
 export { LLamaCloudFileService } from "./LLamaCloudFileService.js";
 export { LlamaCloudIndex } from "./LlamaCloudIndex.js";
 export {
   LlamaCloudRetriever,
   type CloudRetrieveParams,
 } from "./LlamaCloudRetriever.js";
+export type { CloudConstructorParams } from "./type.js";

packages/llamaindex/src/cloud/type.ts

@@ -1,8 +1,5 @@
 import type { ServiceContext } from "../ServiceContext.js";
 
-export const DEFAULT_PROJECT_NAME = "Default";
-export const DEFAULT_BASE_URL = "https://api.cloud.llamaindex.ai";
-
 export type ClientParams = { apiKey?: string; baseUrl?: string };
 
 export type CloudConstructorParams = {

packages/llamaindex/src/cloud/utils.ts

@@ -1,7 +1,7 @@
 import { OpenAPI, ProjectsService } from "@llamaindex/cloud/api";
+import { DEFAULT_BASE_URL } from "@llamaindex/core/global";
 import { getEnv } from "@llamaindex/env";
-import type { ClientParams } from "./constants.js";
-import { DEFAULT_BASE_URL } from "./constants.js";
+import type { ClientParams } from "./type.js";
 
 function getBaseUrl(baseUrl?: string): string {
   return baseUrl ?? getEnv("LLAMA_CLOUD_BASE_URL") ?? DEFAULT_BASE_URL;

packages/llamaindex/src/constants.ts

@@ -1,8 +0,0 @@
-export const DEFAULT_CONTEXT_WINDOW = 3900;
-export const DEFAULT_NUM_OUTPUTS = 256;
-
-export const DEFAULT_CHUNK_SIZE = 1024;
-export const DEFAULT_CHUNK_OVERLAP = 20;
-export const DEFAULT_CHUNK_OVERLAP_RATIO = 0.1;
-
-export const DEFAULT_PADDING = 5;

packages/llamaindex/src/index.edge.ts

@@ -1,6 +1,36 @@
 import type { AgentEndEvent, AgentStartEvent } from "./agent/types.js";
 import type { RetrievalEndEvent, RetrievalStartEvent } from "./llm/types.js";
 
+export {
+  CallbackManager,
+  DEFAULT_BASE_URL,
+  DEFAULT_CHUNK_OVERLAP,
+  DEFAULT_CHUNK_OVERLAP_RATIO,
+  DEFAULT_CHUNK_SIZE,
+  DEFAULT_COLLECTION,
+  DEFAULT_CONTEXT_WINDOW,
+  DEFAULT_DOC_STORE_PERSIST_FILENAME,
+  DEFAULT_GRAPH_STORE_PERSIST_FILENAME,
+  DEFAULT_IMAGE_VECTOR_NAMESPACE,
+  DEFAULT_INDEX_STORE_PERSIST_FILENAME,
+  DEFAULT_NAMESPACE,
+  DEFAULT_NUM_OUTPUTS,
+  DEFAULT_PADDING,
+  DEFAULT_PERSIST_DIR,
+  DEFAULT_PROJECT_NAME,
+  DEFAULT_VECTOR_STORE_PERSIST_FILENAME,
+} from "@llamaindex/core/global";
+export type {
+  JSONArray,
+  JSONObject,
+  JSONValue,
+  LLMEndEvent,
+  LLMStartEvent,
+  LLMStreamEvent,
+  LLMToolCallEvent,
+  LLMToolResultEvent,
+} from "@llamaindex/core/global";
+export * from "@llamaindex/core/llms";
 export * from "@llamaindex/core/prompts";
 export * from "@llamaindex/core/schema";
 
@@ -14,23 +44,11 @@ declare module "@llamaindex/core/global" {
   }
 }
 
-export { CallbackManager } from "@llamaindex/core/global";
-export type {
-  JSONArray,
-  JSONObject,
-  JSONValue,
-  LLMEndEvent,
-  LLMStartEvent,
-  LLMStreamEvent,
-  LLMToolCallEvent,
-  LLMToolResultEvent,
-} from "@llamaindex/core/global";
 export * from "@llamaindex/core/llms";
 export * from "@llamaindex/core/schema";
 export * from "./agent/index.js";
 export * from "./ChatHistory.js";
 export * from "./cloud/index.js";
-export * from "./constants.js";
 export * from "./embeddings/index.js";
 export * from "./engines/chat/index.js";
 export * from "./engines/query/index.js";

packages/llamaindex/src/storage/StorageContext.ts

@@ -1,10 +1,10 @@
-import { ModalityType, ObjectType } from "@llamaindex/core/schema";
-import { path } from "@llamaindex/env";
-import { getImageEmbedModel } from "../internal/settings/image-embed-model.js";
 import {
   DEFAULT_IMAGE_VECTOR_NAMESPACE,
   DEFAULT_NAMESPACE,
-} from "./constants.js";
+} from "@llamaindex/core/global";
+import { ModalityType, ObjectType } from "@llamaindex/core/schema";
+import { path } from "@llamaindex/env";
+import { getImageEmbedModel } from "../internal/settings/image-embed-model.js";
 import { SimpleDocumentStore } from "./docStore/SimpleDocumentStore.js";
 import type { BaseDocumentStore } from "./docStore/types.js";
 import { SimpleIndexStore } from "./indexStore/SimpleIndexStore.js";

packages/llamaindex/src/storage/constants.ts

@@ -1,8 +0,0 @@
-export const DEFAULT_COLLECTION = "data";
-export const DEFAULT_PERSIST_DIR = "./storage";
-export const DEFAULT_INDEX_STORE_PERSIST_FILENAME = "index_store.json";
-export const DEFAULT_DOC_STORE_PERSIST_FILENAME = "doc_store.json";
-export const DEFAULT_VECTOR_STORE_PERSIST_FILENAME = "vector_store.json";
-export const DEFAULT_GRAPH_STORE_PERSIST_FILENAME = "graph_store.json";
-export const DEFAULT_NAMESPACE = "docstore";
-export const DEFAULT_IMAGE_VECTOR_NAMESPACE = "images";

packages/llamaindex/src/storage/docStore/KVDocumentStore.ts

@@ -1,7 +1,7 @@
+import { DEFAULT_NAMESPACE } from "@llamaindex/core/global";
 import type { BaseNode } from "@llamaindex/core/schema";
 import { ObjectType } from "@llamaindex/core/schema";
 import _ from "lodash";
-import { DEFAULT_NAMESPACE } from "../constants.js";
 import type { BaseKVStore } from "../kvStore/types.js";
 import type { RefDocInfo } from "./types.js";
 import { BaseDocumentStore } from "./types.js";

packages/llamaindex/src/storage/docStore/SimpleDocumentStore.ts

@@ -1,10 +1,10 @@
-import { path } from "@llamaindex/env";
-import _ from "lodash";
 import {
   DEFAULT_DOC_STORE_PERSIST_FILENAME,
   DEFAULT_NAMESPACE,
   DEFAULT_PERSIST_DIR,
-} from "../constants.js";
+} from "@llamaindex/core/global";
+import { path } from "@llamaindex/env";
+import _ from "lodash";
 import { SimpleKVStore } from "../kvStore/SimpleKVStore.js";
 import { BaseInMemoryKVStore } from "../kvStore/types.js";
 import { KVDocumentStore } from "./KVDocumentStore.js";

packages/llamaindex/src/storage/docStore/types.ts

@@ -1,8 +1,8 @@
-import { BaseNode } from "@llamaindex/core/schema";
 import {
   DEFAULT_DOC_STORE_PERSIST_FILENAME,
   DEFAULT_PERSIST_DIR,
-} from "../constants.js";
+} from "@llamaindex/core/global";
+import { BaseNode } from "@llamaindex/core/schema";
 
 const defaultPersistPath = `${DEFAULT_PERSIST_DIR}/${DEFAULT_DOC_STORE_PERSIST_FILENAME}`;
 

packages/llamaindex/src/storage/index.ts

@@ -1,6 +1,5 @@
 export { SimpleChatStore } from "./chatStore/SimpleChatStore.js";
 export * from "./chatStore/types.js";
-export * from "./constants.js";
 export { SimpleDocumentStore } from "./docStore/SimpleDocumentStore.js";
 export * from "./docStore/types.js";
 export * from "./FileSystem.js";

packages/llamaindex/src/storage/indexStore/KVIndexStore.ts

@@ -1,7 +1,7 @@
+import { DEFAULT_NAMESPACE } from "@llamaindex/core/global";
 import _ from "lodash";
 import type { IndexStruct } from "../../indices/IndexStruct.js";
 import { jsonToIndexStruct } from "../../indices/json-to-index-struct.js";
-import { DEFAULT_NAMESPACE } from "../constants.js";
 import type { BaseKVStore } from "../kvStore/types.js";
 import { BaseIndexStore } from "./types.js";
 

packages/llamaindex/src/storage/indexStore/SimpleIndexStore.ts

@@ -1,8 +1,8 @@
-import { path } from "@llamaindex/env";
 import {
   DEFAULT_INDEX_STORE_PERSIST_FILENAME,
   DEFAULT_PERSIST_DIR,
-} from "../constants.js";
+} from "@llamaindex/core/global";
+import { path } from "@llamaindex/env";
 import type { DataType } from "../kvStore/SimpleKVStore.js";
 import { SimpleKVStore } from "../kvStore/SimpleKVStore.js";
 import type { BaseInMemoryKVStore } from "../kvStore/types.js";

packages/llamaindex/src/storage/indexStore/types.ts

@@ -1,10 +1,14 @@
-import type { IndexStruct } from "../../indices/IndexStruct.js";
 import {
   DEFAULT_INDEX_STORE_PERSIST_FILENAME,
   DEFAULT_PERSIST_DIR,
-} from "../constants.js";
+} from "@llamaindex/core/global";
+import { path } from "@llamaindex/env";
+import type { IndexStruct } from "../../indices/IndexStruct.js";
 
-const defaultPersistPath = `${DEFAULT_PERSIST_DIR}/${DEFAULT_INDEX_STORE_PERSIST_FILENAME}`;
+const defaultPersistPath = path.join(
+  DEFAULT_PERSIST_DIR,
+  DEFAULT_INDEX_STORE_PERSIST_FILENAME,
+);
 
 export abstract class BaseIndexStore {
   abstract getIndexStructs(): Promise<IndexStruct[]>;

packages/llamaindex/src/storage/kvStore/SimpleKVStore.ts

@@ -1,6 +1,6 @@
+import { DEFAULT_COLLECTION } from "@llamaindex/core/global";
 import { fs, path } from "@llamaindex/env";
 import { exists } from "../FileSystem.js";
-import { DEFAULT_COLLECTION } from "../constants.js";
 import { BaseKVStore } from "./types.js";
 
 export type DataType = Record<string, Record<string, any>>;

packages/llamaindex/src/storage/vectorStore/MongoDBAtlasVectorStore.ts

@@ -129,9 +129,9 @@ export class MongoDBAtlasVectorSearch
    * Function to determine the number of candidates to retrieve for a given query.
    * In case your results are not good, you might tune this value.
    *
-   * {@link https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-stage/|Run Vector Search Queries}
+   * {@link https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-stage/ | Run Vector Search Queries}
    *
-   * {@link https://arxiv.org/abs/1603.09320|Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs}
+   * {@link https://arxiv.org/abs/1603.09320 | Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs}
    *
    *
    * Default: query.similarityTopK * 10

packages/llamaindex/src/storage/vectorStore/SimpleVectorStore.ts

@@ -1,4 +1,5 @@
 import type { BaseEmbedding } from "@llamaindex/core/embeddings";
+import { DEFAULT_PERSIST_DIR } from "@llamaindex/core/global";
 import type { BaseNode } from "@llamaindex/core/schema";
 import { fs, path } from "@llamaindex/env";
 import {
@@ -6,7 +7,6 @@ import {
   getTopKMMREmbeddings,
 } from "../../internal/utils.js";
 import { exists } from "../FileSystem.js";
-import { DEFAULT_PERSIST_DIR } from "../constants.js";
 import {
   FilterOperator,
   VectorStoreBase,

pnpm-lock.yaml

@@ -162,7 +162,7 @@ importers:
         version: link:../packages/llamaindex
       mongodb:
         specifier: ^6.7.0
-        version: 6.8.0(@aws-sdk/credential-providers@3.637.0)
+        version: 6.8.0(@aws-sdk/credential-providers@3.637.0(@aws-sdk/client-sso-oidc@3.637.0(@aws-sdk/client-sts@3.637.0)))
       pathe:
         specifier: ^1.1.2
         version: 1.1.2
@@ -408,8 +408,8 @@ importers:
         specifier: ^1.0.12
         version: 1.0.12
       tiktoken:
-        specifier: 1.0.14
-        version: 1.0.14
+        specifier: ^1.0.15
+        version: 1.0.16
     devDependencies:
       '@aws-crypto/sha256-js':
         specifier: ^5.2.0
@@ -573,7 +573,7 @@ importers:
         version: 2.0.0
       mongodb:
         specifier: ^6.7.0
-        version: 6.8.0(@aws-sdk/credential-providers@3.637.0)
+        version: 6.8.0(@aws-sdk/credential-providers@3.637.0(@aws-sdk/client-sso-oidc@3.637.0(@aws-sdk/client-sts@3.637.0)))
       notion-md-crawler:
         specifier: ^1.0.0
         version: 1.0.0(encoding@0.1.13)
@@ -602,8 +602,8 @@ importers:
         specifier: ^13.4.8
         version: 13.4.8
       tiktoken:
-        specifier: 1.0.14
-        version: 1.0.14
+        specifier: ^1.0.15
+        version: 1.0.16
       unpdf:
         specifier: ^0.11.0
         version: 0.11.0(encoding@0.1.13)
@@ -10247,8 +10247,8 @@ packages:
   thunky@1.1.0:
     resolution: {integrity: sha512-eHY7nBftgThBqOyHGVN+l8gF0BucP09fMo0oO/Lb0w1OF80dJv+lDVpXG60WMQvkcxAkNybKsrEIE3ZtKGmPrA==}
 
-  tiktoken@1.0.14:
-    resolution: {integrity: sha512-g5zd5r/DoH8Kw0fiYbYpVhb6WO8BHO1unXqmBBWKwoT17HwSounnDtMDFUKm2Pko8U47sjQarOe+9aUrnqmmTg==}
+  tiktoken@1.0.16:
+    resolution: {integrity: sha512-hRcORIGF2YlAgWx3nzrGJOrKSJwLoc81HpXmMQk89632XAgURc7IeV2FgQ2iXo9z/J96fCvpsHg2kWoHcbj9fg==}
 
   tiny-invariant@1.3.3:
     resolution: {integrity: sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg==}
@@ -17734,6 +17734,16 @@ snapshots:
     transitivePeerDependencies:
       - supports-color
 
+  eslint-module-utils@2.8.2(@typescript-eslint/parser@8.3.0(eslint@8.57.0)(typescript@5.5.4))(eslint-import-resolver-node@0.3.9)(eslint@8.57.0):
+    dependencies:
+      debug: 3.2.7
+    optionalDependencies:
+      '@typescript-eslint/parser': 8.3.0(eslint@8.57.0)(typescript@5.5.4)
+      eslint: 8.57.0
+      eslint-import-resolver-node: 0.3.9
+    transitivePeerDependencies:
+      - supports-color
+
   eslint-plugin-import@2.29.1(@typescript-eslint/parser@8.3.0(eslint@8.57.0)(typescript@5.5.4))(eslint@8.57.0):
     dependencies:
       array-includes: 3.1.8
@@ -17744,7 +17754,7 @@ snapshots:
       doctrine: 2.1.0
       eslint: 8.57.0
       eslint-import-resolver-node: 0.3.9
-      eslint-module-utils: 2.8.2(@typescript-eslint/parser@7.2.0(eslint@8.57.0)(typescript@5.5.4))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@3.6.3(@typescript-eslint/parser@7.2.0(eslint@8.57.0)(typescript@5.5.4))(eslint-import-resolver-node@0.3.9)(eslint-plugin-import@2.29.1)(eslint@8.57.0))(eslint@8.57.0)
+      eslint-module-utils: 2.8.2(@typescript-eslint/parser@8.3.0(eslint@8.57.0)(typescript@5.5.4))(eslint-import-resolver-node@0.3.9)(eslint@8.57.0)
       hasown: 2.0.2
       is-core-module: 2.15.1
       is-glob: 4.0.3
@@ -20472,7 +20482,7 @@ snapshots:
     optionalDependencies:
       '@aws-sdk/credential-providers': 3.637.0(@aws-sdk/client-sso-oidc@3.637.0(@aws-sdk/client-sts@3.637.0))
 
-  mongodb@6.8.0(@aws-sdk/credential-providers@3.637.0):
+  mongodb@6.8.0(@aws-sdk/credential-providers@3.637.0(@aws-sdk/client-sso-oidc@3.637.0(@aws-sdk/client-sts@3.637.0))):
     dependencies:
       '@mongodb-js/saslprep': 1.1.7
       bson: 6.8.0
@@ -23055,7 +23065,7 @@ snapshots:
 
   thunky@1.1.0: {}
 
-  tiktoken@1.0.14: {}
+  tiktoken@1.0.16: {}
 
   tiny-invariant@1.3.3: {}