Compare commits

..

4 Commits

Author SHA1 Message Date
github-actions[bot] 1e296ebe72 Release 0.11.24 (#2141)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: marcusschiesser <17126+marcusschiesser@users.noreply.github.com>
2025-07-30 12:56:45 -04:00
Marcus Schiesser f9f1de9516 chore: use Logger for core (#2139) 2025-07-30 11:43:45 +08:00
Twisha Bansal f576812e7a docs: Using MCP Toolbox for Databases with LlamaIndex (#2138) 2025-07-30 11:19:34 +08:00
Adrian Lyjak c3bf3c7178 Adding support for page citations, and refactor the confidence into the field metadata (#2140) 2025-07-30 10:25:19 +08:00
143 changed files with 885 additions and 199 deletions
+14
View File
@@ -1,5 +1,19 @@
# @llamaindex/doc # @llamaindex/doc
## 0.2.47
### Patch Changes
- Updated dependencies [c3bf3c7]
- Updated dependencies [f9f1de9]
- @llamaindex/cloud@4.0.28
- @llamaindex/core@0.6.19
- llamaindex@0.11.24
- @llamaindex/node-parser@2.0.19
- @llamaindex/openai@0.4.14
- @llamaindex/readers@3.1.18
- @llamaindex/workflow@1.1.20
## 0.2.46 ## 0.2.46
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@llamaindex/doc", "name": "@llamaindex/doc",
"version": "0.2.46", "version": "0.2.47",
"private": true, "private": true,
"scripts": { "scripts": {
"postinstall": "fumadocs-mdx", "postinstall": "fumadocs-mdx",
Binary file not shown.

After

Width:  |  Height:  |  Size: 154 KiB

@@ -0,0 +1,85 @@
---
title: MCP Toolbox For Databases
description: MCP Toolbox for Databases is an open source MCP server for databases.
---
# MCP Toolbox for Databases
[MCP Toolbox for Databases](https://github.com/googleapis/genai-toolbox) is an open source MCP server for databases. It was designed with enterprise-grade and production-quality in mind. It enables you to develop tools easier, faster, and more securely by handling the complexities such as connection pooling, authentication, and more.
Toolbox Tools can be seemlessly integrated with LlamaIndex applications. For more
information on [getting
started](https://googleapis.github.io/genai-toolbox/getting-started/local_quickstart_js/) or
[configuring](https://googleapis.github.io/genai-toolbox/getting-started/configure/)
Toolbox, see the
[documentation](https://googleapis.github.io/genai-toolbox/getting-started/introduction/).
![architecture](/images/mcp_db_toolbox.png)
### Configure and deploy
Toolbox is an open source server that you deploy and manage yourself. For more
instructions on deploying and configuring, see the official Toolbox
documentation:
* [Installing the Server](https://googleapis.github.io/genai-toolbox/getting-started/introduction/#installing-the-server)
* [Configuring Toolbox](https://googleapis.github.io/genai-toolbox/getting-started/configure/)
### Install client SDK
LlamaIndex relies on the `@toolbox-sdk/core` node package to use Toolbox. Install the
package before getting started:
```shell
npm install @toolbox-sdk/core
```
### Loading Toolbox Tools
Once your Toolbox server is configured and up and running, you can load tools
from your server using the SDK:
```javascript
import { gemini, GEMINI_MODEL } from "@llamaindex/google";
import { agent } from "@llamaindex/workflow";
import { tool } from "llamaindex";
import { ToolboxClient } from "@toolbox-sdk/core";
// Initialize LLM
const llm = gemini({
model: GEMINI_MODEL.GEMINI_2_0_FLASH,
apiKey: process.env.GOOGLE_API_KEY,
});
// Replace with your Toolbox Server URL
const URL = 'https://127.0.0.1:5000';
const client = new ToolboxClient("http://127.0.0.1:5000");
const toolboxTools = await client.loadToolset("my-toolset");
const getTool = (toolboxTool) => tool({
name: toolboxTool.getName(),
description: toolboxTool.getDescription(),
parameters: toolboxTool.getParamSchema(),
execute: toolboxTool
});
const tools = toolboxTools.map(getTool);
const myAgent = agent({
tools: tools,
llm,
memory,
systemPrompt: prompt,
});
const result = await myAgent.run(query);
console.log(result);
```
### Advanced Toolbox Features
Toolbox has a variety of features to make developing Gen AI tools for databases seamless.
For more information, read more about the following:
- [Authenticated Parameters](https://googleapis.github.io/genai-toolbox/resources/tools/#authenticated-parameters): bind tool inputs to values from OIDC tokens automatically, making it easy to run sensitive queries without potentially leaking data
- [Authorized Invocations](https://googleapis.github.io/genai-toolbox/resources/tools/#authorized-invocations): restrict access to use a tool based on the users Auth token
- [OpenTelemetry](https://googleapis.github.io/genai-toolbox/how-to/export_telemetry/): get metrics and tracing from Toolbox with [OpenTelemetry](https://opentelemetry.io/docs/)
@@ -1,5 +1,5 @@
{ {
"title": "Integration", "title": "Integration",
"description": "See our integrations", "description": "See our integrations",
"pages": ["open-llm-metry", "lang-trace", "vercel"] "pages": ["open-llm-metry", "lang-trace", "mcp-toolbox", "vercel"]
} }
@@ -101,6 +101,9 @@ const agent = agent({
}); });
``` ```
You can also use [MCP Toolbox for
Databases](/docs/llamaindex/integration/mcp-toolbox) to interact with MCP tools.
## Function tool ## Function tool
@@ -1,5 +1,11 @@
# @llamaindex/cloudflare-worker-agent-test # @llamaindex/cloudflare-worker-agent-test
## 0.0.185
### Patch Changes
- llamaindex@0.11.24
## 0.0.184 ## 0.0.184
### Patch Changes ### Patch Changes
@@ -1,6 +1,6 @@
{ {
"name": "@llamaindex/cloudflare-worker-agent-test", "name": "@llamaindex/cloudflare-worker-agent-test",
"version": "0.0.184", "version": "0.0.185",
"type": "module", "type": "module",
"private": true, "private": true,
"scripts": { "scripts": {
@@ -1,5 +1,12 @@
# @llamaindex/llama-parse-browser-test # @llamaindex/llama-parse-browser-test
## 0.0.83
### Patch Changes
- Updated dependencies [c3bf3c7]
- @llamaindex/cloud@4.0.28
## 0.0.82 ## 0.0.82
### Patch Changes ### Patch Changes
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/llama-parse-browser-test", "name": "@llamaindex/llama-parse-browser-test",
"private": true, "private": true,
"version": "0.0.82", "version": "0.0.83",
"type": "module", "type": "module",
"scripts": { "scripts": {
"dev": "vite", "dev": "vite",
+6
View File
@@ -1,5 +1,11 @@
# @llamaindex/next-agent-test # @llamaindex/next-agent-test
## 0.1.185
### Patch Changes
- llamaindex@0.11.24
## 0.1.184 ## 0.1.184
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@llamaindex/next-agent-test", "name": "@llamaindex/next-agent-test",
"version": "0.1.184", "version": "0.1.185",
"private": true, "private": true,
"scripts": { "scripts": {
"dev": "next dev", "dev": "next dev",
@@ -1,5 +1,11 @@
# test-edge-runtime # test-edge-runtime
## 0.1.184
### Patch Changes
- llamaindex@0.11.24
## 0.1.183 ## 0.1.183
### Patch Changes ### Patch Changes
@@ -1,6 +1,6 @@
{ {
"name": "@llamaindex/nextjs-edge-runtime-test", "name": "@llamaindex/nextjs-edge-runtime-test",
"version": "0.1.183", "version": "0.1.184",
"private": true, "private": true,
"scripts": { "scripts": {
"dev": "next dev", "dev": "next dev",
@@ -1,5 +1,13 @@
# @llamaindex/next-node-runtime # @llamaindex/next-node-runtime
## 0.1.54
### Patch Changes
- llamaindex@0.11.24
- @llamaindex/huggingface@0.1.24
- @llamaindex/readers@3.1.18
## 0.1.53 ## 0.1.53
### Patch Changes ### Patch Changes
@@ -1,6 +1,6 @@
{ {
"name": "@llamaindex/next-node-runtime-test", "name": "@llamaindex/next-node-runtime-test",
"version": "0.1.53", "version": "0.1.54",
"private": true, "private": true,
"scripts": { "scripts": {
"dev": "next dev", "dev": "next dev",
@@ -1,5 +1,11 @@
# vite-import-llamaindex # vite-import-llamaindex
## 0.0.51
### Patch Changes
- llamaindex@0.11.24
## 0.0.50 ## 0.0.50
### Patch Changes ### Patch Changes
@@ -1,7 +1,7 @@
{ {
"name": "vite-import-llamaindex", "name": "vite-import-llamaindex",
"private": true, "private": true,
"version": "0.0.50", "version": "0.0.51",
"type": "module", "type": "module",
"scripts": { "scripts": {
"build": "vite build", "build": "vite build",
@@ -1,5 +1,11 @@
# @llamaindex/waku-query-engine-test # @llamaindex/waku-query-engine-test
## 0.0.185
### Patch Changes
- llamaindex@0.11.24
## 0.0.184 ## 0.0.184
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@llamaindex/waku-query-engine-test", "name": "@llamaindex/waku-query-engine-test",
"version": "0.0.184", "version": "0.0.185",
"type": "module", "type": "module",
"private": true, "private": true,
"scripts": { "scripts": {
+53
View File
@@ -1,5 +1,58 @@
# examples # examples
## 0.3.35
### Patch Changes
- Updated dependencies [c3bf3c7]
- Updated dependencies [f9f1de9]
- @llamaindex/cloud@4.0.28
- @llamaindex/core@0.6.19
- llamaindex@0.11.24
- @llamaindex/node-parser@2.0.19
- @llamaindex/anthropic@0.3.21
- @llamaindex/assemblyai@0.1.18
- @llamaindex/clip@0.0.70
- @llamaindex/cohere@0.0.33
- @llamaindex/deepinfra@0.0.70
- @llamaindex/discord@0.1.18
- @llamaindex/google@0.3.18
- @llamaindex/huggingface@0.1.24
- @llamaindex/jinaai@0.0.30
- @llamaindex/mistral@0.1.19
- @llamaindex/mixedbread@0.0.33
- @llamaindex/notion@0.1.18
- @llamaindex/ollama@0.1.19
- @llamaindex/openai@0.4.14
- @llamaindex/perplexity@0.0.27
- @llamaindex/portkey-ai@0.0.61
- @llamaindex/replicate@0.0.61
- @llamaindex/bm25-retriever@0.0.8
- @llamaindex/astra@0.0.33
- @llamaindex/azure@0.1.31
- @llamaindex/chroma@0.0.33
- @llamaindex/elastic-search@0.1.19
- @llamaindex/firestore@1.0.26
- @llamaindex/milvus@0.1.28
- @llamaindex/mongodb@0.0.34
- @llamaindex/pinecone@0.1.19
- @llamaindex/postgres@0.0.62
- @llamaindex/qdrant@0.1.29
- @llamaindex/supabase@0.1.20
- @llamaindex/upstash@0.0.33
- @llamaindex/weaviate@0.0.34
- @llamaindex/vercel@0.1.19
- @llamaindex/voyage-ai@1.0.25
- @llamaindex/readers@3.1.18
- @llamaindex/tools@0.1.9
- @llamaindex/workflow@1.1.20
- @llamaindex/deepseek@0.0.31
- @llamaindex/fireworks@0.0.30
- @llamaindex/groq@0.0.86
- @llamaindex/together@0.0.30
- @llamaindex/vllm@0.0.56
- @llamaindex/xai@0.0.17
## 0.3.34 ## 0.3.34
### Patch Changes ### Patch Changes
+47 -47
View File
@@ -1,6 +1,6 @@
{ {
"name": "@llamaindex/examples", "name": "@llamaindex/examples",
"version": "0.3.34", "version": "0.3.35",
"private": true, "private": true,
"scripts": { "scripts": {
"lint": "eslint .", "lint": "eslint .",
@@ -11,52 +11,52 @@
"@azure/cosmos": "^4.1.1", "@azure/cosmos": "^4.1.1",
"@azure/identity": "^4.4.1", "@azure/identity": "^4.4.1",
"@azure/search-documents": "^12.1.0", "@azure/search-documents": "^12.1.0",
"@llamaindex/anthropic": "^0.3.20", "@llamaindex/anthropic": "^0.3.21",
"@llamaindex/assemblyai": "^0.1.17", "@llamaindex/assemblyai": "^0.1.18",
"@llamaindex/astra": "^0.0.32", "@llamaindex/astra": "^0.0.33",
"@llamaindex/azure": "^0.1.30", "@llamaindex/azure": "^0.1.31",
"@llamaindex/bm25-retriever": "^0.0.7", "@llamaindex/bm25-retriever": "^0.0.8",
"@llamaindex/chroma": "^0.0.32", "@llamaindex/chroma": "^0.0.33",
"@llamaindex/clip": "^0.0.69", "@llamaindex/clip": "^0.0.70",
"@llamaindex/cloud": "^4.0.27", "@llamaindex/cloud": "^4.0.28",
"@llamaindex/cohere": "^0.0.32", "@llamaindex/cohere": "^0.0.33",
"@llamaindex/core": "^0.6.18", "@llamaindex/core": "^0.6.19",
"@llamaindex/deepinfra": "^0.0.69", "@llamaindex/deepinfra": "^0.0.70",
"@llamaindex/deepseek": "^0.0.30", "@llamaindex/deepseek": "^0.0.31",
"@llamaindex/discord": "^0.1.17", "@llamaindex/discord": "^0.1.18",
"@llamaindex/elastic-search": "^0.1.18", "@llamaindex/elastic-search": "^0.1.19",
"@llamaindex/env": "^0.1.30", "@llamaindex/env": "^0.1.30",
"@llamaindex/firestore": "^1.0.25", "@llamaindex/firestore": "^1.0.26",
"@llamaindex/fireworks": "^0.0.29", "@llamaindex/fireworks": "^0.0.30",
"@llamaindex/google": "^0.3.17", "@llamaindex/google": "^0.3.18",
"@llamaindex/groq": "^0.0.85", "@llamaindex/groq": "^0.0.86",
"@llamaindex/huggingface": "^0.1.23", "@llamaindex/huggingface": "^0.1.24",
"@llamaindex/jinaai": "^0.0.29", "@llamaindex/jinaai": "^0.0.30",
"@llamaindex/milvus": "^0.1.27", "@llamaindex/milvus": "^0.1.28",
"@llamaindex/mistral": "^0.1.18", "@llamaindex/mistral": "^0.1.19",
"@llamaindex/mixedbread": "^0.0.32", "@llamaindex/mixedbread": "^0.0.33",
"@llamaindex/mongodb": "^0.0.33", "@llamaindex/mongodb": "^0.0.34",
"@llamaindex/node-parser": "^2.0.18", "@llamaindex/node-parser": "^2.0.19",
"@llamaindex/notion": "^0.1.17", "@llamaindex/notion": "^0.1.18",
"@llamaindex/ollama": "^0.1.18", "@llamaindex/ollama": "^0.1.19",
"@llamaindex/openai": "^0.4.13", "@llamaindex/openai": "^0.4.14",
"@llamaindex/perplexity": "^0.0.26", "@llamaindex/perplexity": "^0.0.27",
"@llamaindex/pinecone": "^0.1.18", "@llamaindex/pinecone": "^0.1.19",
"@llamaindex/portkey-ai": "^0.0.60", "@llamaindex/portkey-ai": "^0.0.61",
"@llamaindex/postgres": "^0.0.61", "@llamaindex/postgres": "^0.0.62",
"@llamaindex/qdrant": "^0.1.28", "@llamaindex/qdrant": "^0.1.29",
"@llamaindex/readers": "^3.1.17", "@llamaindex/readers": "^3.1.18",
"@llamaindex/replicate": "^0.0.60", "@llamaindex/replicate": "^0.0.61",
"@llamaindex/supabase": "^0.1.19", "@llamaindex/supabase": "^0.1.20",
"@llamaindex/together": "^0.0.29", "@llamaindex/together": "^0.0.30",
"@llamaindex/tools": "^0.1.8", "@llamaindex/tools": "^0.1.9",
"@llamaindex/upstash": "^0.0.32", "@llamaindex/upstash": "^0.0.33",
"@llamaindex/vercel": "^0.1.18", "@llamaindex/vercel": "^0.1.19",
"@llamaindex/vllm": "^0.0.55", "@llamaindex/vllm": "^0.0.56",
"@llamaindex/voyage-ai": "^1.0.24", "@llamaindex/voyage-ai": "^1.0.25",
"@llamaindex/weaviate": "^0.0.33", "@llamaindex/weaviate": "^0.0.34",
"@llamaindex/workflow": "^1.1.19", "@llamaindex/workflow": "^1.1.20",
"@llamaindex/xai": "^0.0.16", "@llamaindex/xai": "^0.0.17",
"@notionhq/client": "^4.0.0", "@notionhq/client": "^4.0.0",
"@pinecone-database/pinecone": "^4.0.0", "@pinecone-database/pinecone": "^4.0.0",
"@vercel/postgres": "^0.10.0", "@vercel/postgres": "^0.10.0",
@@ -65,7 +65,7 @@
"commander": "^12.1.0", "commander": "^12.1.0",
"dotenv": "^17.2.0", "dotenv": "^17.2.0",
"js-tiktoken": "^1.0.14", "js-tiktoken": "^1.0.14",
"llamaindex": "^0.11.23", "llamaindex": "^0.11.24",
"mongodb": "6.7.0", "mongodb": "6.7.0",
"postgres": "^3.4.4", "postgres": "^3.4.4",
"wikipedia": "^2.1.2", "wikipedia": "^2.1.2",
+6
View File
@@ -1,5 +1,11 @@
# @llamaindex/autotool # @llamaindex/autotool
## 8.0.24
### Patch Changes
- llamaindex@0.11.24
## 8.0.23 ## 8.0.23
### Patch Changes ### Patch Changes
@@ -1,5 +1,12 @@
# @llamaindex/autotool-01-node-example # @llamaindex/autotool-01-node-example
## 0.0.132
### Patch Changes
- llamaindex@0.11.24
- @llamaindex/autotool@8.0.24
## 0.0.131 ## 0.0.131
### Patch Changes ### Patch Changes
@@ -13,5 +13,5 @@
"scripts": { "scripts": {
"start": "node --import tsx --import @llamaindex/autotool/node ./src/index.ts" "start": "node --import tsx --import @llamaindex/autotool/node ./src/index.ts"
}, },
"version": "0.0.131" "version": "0.0.132"
} }
+1 -1
View File
@@ -6,7 +6,7 @@
"url": "git+https://github.com/run-llama/LlamaIndexTS.git", "url": "git+https://github.com/run-llama/LlamaIndexTS.git",
"directory": "packages/autotool" "directory": "packages/autotool"
}, },
"version": "8.0.23", "version": "8.0.24",
"description": "auto transpile your JS function to LLM Agent compatible", "description": "auto transpile your JS function to LLM Agent compatible",
"files": [ "files": [
"dist", "dist",
+8
View File
@@ -1,5 +1,13 @@
# @llamaindex/cloud # @llamaindex/cloud
## 4.0.28
### Patch Changes
- c3bf3c7: Adding support for citations to beta agent data schema
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 4.0.27 ## 4.0.27
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@llamaindex/cloud", "name": "@llamaindex/cloud",
"version": "4.0.27", "version": "4.0.28",
"type": "module", "type": "module",
"license": "MIT", "license": "MIT",
"scripts": { "scripts": {
+28 -3
View File
@@ -28,6 +28,29 @@ export type ComparisonOperator =
*/ */
export type FilterOperation = RawFilterOperation; export type FilterOperation = RawFilterOperation;
/**
* Metadata for an extracted field, including confidence and citation information
*/
export interface ExtractedFieldMetadata {
/** The confidence score for the field, combined with parsing confidence if applicable */
confidence?: number;
/** The confidence score for the field based on the extracted text only */
extracted_confidence?: number;
/** The page number that the field occurred on */
page_number?: number;
/** The original text this field's value was derived from */
matching_text?: string;
}
/**
* Dictionary mapping field names to their metadata
* Values can be ExtractedFieldMetadata objects, nested dictionaries, or arrays
*/
export type ExtractedFieldMetadataDict = Record<
string,
ExtractedFieldMetadata | Record<string, unknown> | unknown[]
>;
/** /**
* Base extracted data interface * Base extracted data interface
*/ */
@@ -35,11 +58,13 @@ export interface ExtractedData<T = unknown> {
/** The original data that was extracted from the document. For tracking changes. Should not be updated. */ /** The original data that was extracted from the document. For tracking changes. Should not be updated. */
original_data: T; original_data: T;
/** The latest state of the data. Will differ if data has been updated. */ /** The latest state of the data. Will differ if data has been updated. */
data?: T; data: T;
/** The status of the extracted data. Prefer to use the StatusType values, but any string is allowed. */ /** The status of the extracted data. Prefer to use the StatusType values, but any string is allowed. */
status: StatusType | string; status: StatusType | string;
/** Confidence scores, if any, for each primitive field in the original_data data. */ /** The overall confidence score for the extracted data. */
confidence?: Record<string, unknown>; overall_confidence?: number;
/** Page links, and perhaps eventually bounding boxes, for individual fields in the extracted data. */
field_metadata?: ExtractedFieldMetadataDict;
/** The ID of the file that was used to extract the data. */ /** The ID of the file that was used to extract the data. */
file_id?: string; file_id?: string;
/** The name of the file that was used to extract the data. */ /** The name of the file that was used to extract the data. */
+7
View File
@@ -1,5 +1,12 @@
# @llamaindex/community # @llamaindex/community
## 0.0.101
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.100 ## 0.0.100
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/community", "name": "@llamaindex/community",
"description": "Community package for LlamaIndexTS", "description": "Community package for LlamaIndexTS",
"version": "0.0.100", "version": "0.0.101",
"type": "module", "type": "module",
"types": "dist/type/index.d.ts", "types": "dist/type/index.d.ts",
"main": "dist/cjs/index.js", "main": "dist/cjs/index.js",
+6
View File
@@ -1,5 +1,11 @@
# @llamaindex/core # @llamaindex/core
## 0.6.19
### Patch Changes
- f9f1de9: Use logger interface instead of directly hardcoding console.log
## 0.6.18 ## 0.6.18
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/core", "name": "@llamaindex/core",
"type": "module", "type": "module",
"version": "0.6.18", "version": "0.6.19",
"description": "LlamaIndex Core Module", "description": "LlamaIndex Core Module",
"exports": { "exports": {
"./agent": { "./agent": {
+6 -1
View File
@@ -1,3 +1,4 @@
import { consoleLogger, emptyLogger, type Logger } from "@llamaindex/env";
import type { Tokenizers } from "@llamaindex/env/tokenizers"; import type { Tokenizers } from "@llamaindex/env/tokenizers";
import type { MessageContentDetail } from "../llms"; import type { MessageContentDetail } from "../llms";
import { BaseNode, MetadataMode, TransformComponent } from "../schema"; import { BaseNode, MetadataMode, TransformComponent } from "../schema";
@@ -18,6 +19,7 @@ export type EmbeddingInfo = {
export type BaseEmbeddingOptions = { export type BaseEmbeddingOptions = {
logProgress?: boolean; logProgress?: boolean;
progressCallback?: (current: number, total: number) => void; progressCallback?: (current: number, total: number) => void;
logger?: Logger;
}; };
export abstract class BaseEmbedding extends TransformComponent< export abstract class BaseEmbedding extends TransformComponent<
@@ -133,6 +135,9 @@ export async function batchEmbeddings<T>(
const curBatch: T[] = []; const curBatch: T[] = [];
const logger =
options?.logger ?? (options?.logProgress ? consoleLogger : emptyLogger);
for (let i = 0; i < queue.length; i++) { for (let i = 0; i < queue.length; i++) {
curBatch.push(queue[i]!); curBatch.push(queue[i]!);
if (i == queue.length - 1 || curBatch.length == chunkSize) { if (i == queue.length - 1 || curBatch.length == chunkSize) {
@@ -143,7 +148,7 @@ export async function batchEmbeddings<T>(
options?.progressCallback?.(i + 1, queue.length); options?.progressCallback?.(i + 1, queue.length);
} }
if (options?.logProgress) { if (options?.logProgress) {
console.log(`getting embedding progress: ${i + 1} / ${queue.length}`); logger.log(`getting embedding progress: ${i + 1} / ${queue.length}`);
} }
curBatch.length = 0; curBatch.length = 0;
+13 -2
View File
@@ -1,3 +1,4 @@
import { consoleLogger, type Logger } from "@llamaindex/env";
import { Settings } from "../global"; import { Settings } from "../global";
import type { ChatMessage, LLM } from "../llms"; import type { ChatMessage, LLM } from "../llms";
import { extractText } from "../utils"; import { extractText } from "../utils";
@@ -38,6 +39,11 @@ export type MemoryOptions<TMessageOptions extends object = object> = {
* This default LLM can be overridden by the LLM passed in the `getLLM` method. * This default LLM can be overridden by the LLM passed in the `getLLM` method.
*/ */
llm?: LLM | undefined; llm?: LLM | undefined;
/**
* Logger for memory operations
*/
logger?: Logger;
}; };
export class Memory< export class Memory<
@@ -76,6 +82,10 @@ export class Memory<
* The default LLM to use for memory retrieval. * The default LLM to use for memory retrieval.
*/ */
private llm: LLM | undefined; private llm: LLM | undefined;
/**
* Logger for memory operations
*/
private logger: Logger;
constructor( constructor(
messages: MemoryMessage<TMessageOptions>[] = [], messages: MemoryMessage<TMessageOptions>[] = [],
@@ -87,6 +97,7 @@ export class Memory<
options.shortTermTokenLimitRatio ?? DEFAULT_SHORT_TERM_TOKEN_LIMIT_RATIO; options.shortTermTokenLimitRatio ?? DEFAULT_SHORT_TERM_TOKEN_LIMIT_RATIO;
this.memoryBlocks = options.memoryBlocks ?? []; this.memoryBlocks = options.memoryBlocks ?? [];
this.memoryCursor = options.memoryCursor ?? 0; this.memoryCursor = options.memoryCursor ?? 0;
this.logger = options.logger ?? consoleLogger;
this.initLLM(options.llm); this.initLLM(options.llm);
this.adapters = { this.adapters = {
@@ -309,7 +320,7 @@ export class Memory<
addedTokenCount += messageTokenCount; addedTokenCount += messageTokenCount;
} }
} catch (error) { } catch (error) {
console.warn( this.logger.warn(
`Failed to get content from memory block ${block.id}:`, `Failed to get content from memory block ${block.id}:`,
error, error,
); );
@@ -371,7 +382,7 @@ export class Memory<
try { try {
await block.put(newMessages); await block.put(newMessages);
} catch (error) { } catch (error) {
console.warn( this.logger.warn(
`Failed to process messages into memory block ${block.id}:`, `Failed to process messages into memory block ${block.id}:`,
error, error,
); );
@@ -1,3 +1,4 @@
import { consoleLogger, type Logger } from "@llamaindex/env";
import type { Tokenizer } from "@llamaindex/env/tokenizers"; import type { Tokenizer } from "@llamaindex/env/tokenizers";
import { z } from "zod"; import { z } from "zod";
import { Settings } from "../global"; import { Settings } from "../global";
@@ -48,9 +49,11 @@ export class SentenceSplitter extends MetadataAwareTextSplitter {
#splitFns: Set<TextSplitterFn> = new Set(); #splitFns: Set<TextSplitterFn> = new Set();
#subSentenceSplitFns: Set<TextSplitterFn> = new Set(); #subSentenceSplitFns: Set<TextSplitterFn> = new Set();
#tokenizer: Tokenizer; #tokenizer: Tokenizer;
#logger: Logger;
constructor( constructor(
params?: z.input<typeof sentenceSplitterSchema> & SplitterParams, params?: z.input<typeof sentenceSplitterSchema> &
SplitterParams & { logger?: Logger },
) { ) {
super(); super();
if (params) { if (params) {
@@ -66,6 +69,7 @@ export class SentenceSplitter extends MetadataAwareTextSplitter {
this.extraAbbreviations, this.extraAbbreviations,
); );
this.#tokenizer = params?.tokenizer ?? Settings.tokenizer; this.#tokenizer = params?.tokenizer ?? Settings.tokenizer;
this.#logger = params?.logger ?? consoleLogger;
this.#splitFns.add(splitBySep(this.paragraphSeparator)); this.#splitFns.add(splitBySep(this.paragraphSeparator));
this.#splitFns.add(this.#chunkingTokenizerFn); this.#splitFns.add(this.#chunkingTokenizerFn);
@@ -82,7 +86,7 @@ export class SentenceSplitter extends MetadataAwareTextSplitter {
`Metadata length (${metadataLength}) is longer than chunk size (${this.chunkSize}). Consider increasing the chunk size or decreasing the size of your metadata to avoid this.`, `Metadata length (${metadataLength}) is longer than chunk size (${this.chunkSize}). Consider increasing the chunk size or decreasing the size of your metadata to avoid this.`,
); );
} else if (effectiveChunkSize < 50) { } else if (effectiveChunkSize < 50) {
console.log( this.#logger.log(
`Metadata length (${metadataLength}) is close to chunk size (${this.chunkSize}). Resulting chunks are less than 50 tokens. Consider increasing the chunk size or decreasing the size of your metadata to avoid this.`, `Metadata length (${metadataLength}) is close to chunk size (${this.chunkSize}). Resulting chunks are less than 50 tokens. Consider increasing the chunk size or decreasing the size of your metadata to avoid this.`,
); );
} }
@@ -1,3 +1,4 @@
import { consoleLogger, type Logger } from "@llamaindex/env";
import type { Tokenizer } from "@llamaindex/env/tokenizers"; import type { Tokenizer } from "@llamaindex/env/tokenizers";
import { z } from "zod"; import { z } from "zod";
import { DEFAULT_CHUNK_OVERLAP, DEFAULT_CHUNK_SIZE, Settings } from "../global"; import { DEFAULT_CHUNK_OVERLAP, DEFAULT_CHUNK_SIZE, Settings } from "../global";
@@ -21,9 +22,11 @@ export class TokenTextSplitter extends MetadataAwareTextSplitter {
backupSeparators: string[] = ["\n"]; backupSeparators: string[] = ["\n"];
#tokenizer: Tokenizer; #tokenizer: Tokenizer;
#splitFns: Array<(text: string) => string[]> = []; #splitFns: Array<(text: string) => string[]> = [];
#logger: Logger;
constructor( constructor(
params?: SplitterParams & Partial<z.infer<typeof tokenTextSplitterSchema>>, params?: SplitterParams &
Partial<z.infer<typeof tokenTextSplitterSchema>> & { logger?: Logger },
) { ) {
super(); super();
@@ -42,6 +45,7 @@ export class TokenTextSplitter extends MetadataAwareTextSplitter {
} }
this.#tokenizer = params?.tokenizer ?? Settings.tokenizer; this.#tokenizer = params?.tokenizer ?? Settings.tokenizer;
this.#logger = params?.logger ?? consoleLogger;
const allSeparators = [this.separator, ...this.backupSeparators]; const allSeparators = [this.separator, ...this.backupSeparators];
this.#splitFns = allSeparators.map((sep) => splitBySep(sep)); this.#splitFns = allSeparators.map((sep) => splitBySep(sep));
@@ -65,7 +69,7 @@ export class TokenTextSplitter extends MetadataAwareTextSplitter {
`Consider increasing the chunk size or decreasing the size of your metadata to avoid this.`, `Consider increasing the chunk size or decreasing the size of your metadata to avoid this.`,
); );
} else if (effectiveChunkSize < 50) { } else if (effectiveChunkSize < 50) {
console.warn( this.#logger.warn(
`Metadata length (${metadataLength}) is close to chunk size (${this.chunkSize}). ` + `Metadata length (${metadataLength}) is close to chunk size (${this.chunkSize}). ` +
`Resulting chunks are less than 50 tokens. Consider increasing the chunk size or decreasing the size of your metadata to avoid this.`, `Resulting chunks are less than 50 tokens. Consider increasing the chunk size or decreasing the size of your metadata to avoid this.`,
); );
@@ -148,7 +152,7 @@ export class TokenTextSplitter extends MetadataAwareTextSplitter {
const splitLength = this.tokenSize(split); const splitLength = this.tokenSize(split);
if (splitLength > chunkSize) { if (splitLength > chunkSize) {
console.warn( this.#logger.warn(
`Got a split of size ${splitLength}, larger than chunk size ${chunkSize}.`, `Got a split of size ${splitLength}, larger than chunk size ${chunkSize}.`,
); );
} }
@@ -1,3 +1,4 @@
import { consoleLogger, type Logger } from "@llamaindex/env";
import { DEFAULT_NAMESPACE } from "../../global"; import { DEFAULT_NAMESPACE } from "../../global";
import { BaseNode, ObjectType, type StoredValue } from "../../schema"; import { BaseNode, ObjectType, type StoredValue } from "../../schema";
import type { BaseKVStore } from "../kv-store"; import type { BaseKVStore } from "../kv-store";
@@ -16,13 +17,19 @@ export class KVDocumentStore extends BaseDocumentStore {
private nodeCollection: string; private nodeCollection: string;
private refDocCollection: string; private refDocCollection: string;
private metadataCollection: string; private metadataCollection: string;
private logger: Logger;
constructor(kvstore: BaseKVStore, namespace: string = DEFAULT_NAMESPACE) { constructor(
kvstore: BaseKVStore,
namespace: string = DEFAULT_NAMESPACE,
options?: { logger?: Logger },
) {
super(); super();
this.kvstore = kvstore; this.kvstore = kvstore;
this.nodeCollection = `${namespace}/data`; this.nodeCollection = `${namespace}/data`;
this.refDocCollection = `${namespace}/ref_doc_info`; this.refDocCollection = `${namespace}/ref_doc_info`;
this.metadataCollection = `${namespace}/metadata`; this.metadataCollection = `${namespace}/metadata`;
this.logger = options?.logger ?? consoleLogger;
} }
async docs(): Promise<Record<string, BaseNode>> { async docs(): Promise<Record<string, BaseNode>> {
@@ -33,7 +40,7 @@ export class KVDocumentStore extends BaseDocumentStore {
if (isValidDocJson(value)) { if (isValidDocJson(value)) {
docs[key] = jsonToDoc(value, this.serializer); docs[key] = jsonToDoc(value, this.serializer);
} else { } else {
console.warn(`Invalid JSON for docId ${key}`); this.logger.warn(`Invalid JSON for docId ${key}`);
} }
} }
return docs; return docs;
+12 -5
View File
@@ -1,4 +1,4 @@
import { path } from "@llamaindex/env"; import { path, type Logger } from "@llamaindex/env";
import { IndexStruct, jsonToIndexStruct } from "../../data-structs"; import { IndexStruct, jsonToIndexStruct } from "../../data-structs";
import { import {
DEFAULT_INDEX_STORE_PERSIST_FILENAME, DEFAULT_INDEX_STORE_PERSIST_FILENAME,
@@ -8,8 +8,8 @@ import {
import { import {
BaseInMemoryKVStore, BaseInMemoryKVStore,
BaseKVStore, BaseKVStore,
type DataType,
SimpleKVStore, SimpleKVStore,
type DataType,
} from "../kv-store"; } from "../kv-store";
export const DEFAULT_PERSIST_PATH = path.join( export const DEFAULT_PERSIST_PATH = path.join(
@@ -84,16 +84,23 @@ export class SimpleIndexStore extends KVIndexStore {
static async fromPersistDir( static async fromPersistDir(
persistDir: string = DEFAULT_PERSIST_DIR, persistDir: string = DEFAULT_PERSIST_DIR,
options?: { logger?: Logger },
): Promise<SimpleIndexStore> { ): Promise<SimpleIndexStore> {
const persistPath = path.join( const persistPath = path.join(
persistDir, persistDir,
DEFAULT_INDEX_STORE_PERSIST_FILENAME, DEFAULT_INDEX_STORE_PERSIST_FILENAME,
); );
return this.fromPersistPath(persistPath); return this.fromPersistPath(persistPath, options);
} }
static async fromPersistPath(persistPath: string): Promise<SimpleIndexStore> { static async fromPersistPath(
const simpleKVStore = await SimpleKVStore.fromPersistPath(persistPath); persistPath: string,
options?: { logger?: Logger },
): Promise<SimpleIndexStore> {
const simpleKVStore = await SimpleKVStore.fromPersistPath(
persistPath,
options,
);
return new SimpleIndexStore(simpleKVStore); return new SimpleIndexStore(simpleKVStore);
} }
+7 -3
View File
@@ -1,4 +1,4 @@
import { fs, path } from "@llamaindex/env"; import { consoleLogger, fs, path, type Logger } from "@llamaindex/env";
import { DEFAULT_COLLECTION } from "../../global"; import { DEFAULT_COLLECTION } from "../../global";
import type { StoredValue } from "../../schema"; import type { StoredValue } from "../../schema";
@@ -98,7 +98,11 @@ export class SimpleKVStore extends BaseKVStore {
await fs.writeFile(persistPath, JSON.stringify(this.data)); await fs.writeFile(persistPath, JSON.stringify(this.data));
} }
static async fromPersistPath(persistPath: string): Promise<SimpleKVStore> { static async fromPersistPath(
persistPath: string,
options?: { logger?: Logger },
): Promise<SimpleKVStore> {
const logger = options?.logger ?? consoleLogger;
const dirPath = path.dirname(persistPath); const dirPath = path.dirname(persistPath);
if (!(await exists(dirPath))) { if (!(await exists(dirPath))) {
await fs.mkdir(dirPath, { recursive: true }); await fs.mkdir(dirPath, { recursive: true });
@@ -106,7 +110,7 @@ export class SimpleKVStore extends BaseKVStore {
let data: DataType = {}; let data: DataType = {};
if (!(await exists(persistPath))) { if (!(await exists(persistPath))) {
console.info(`Starting new store from path: ${persistPath}`); logger.log(`Starting new store from path: ${persistPath}`);
} else { } else {
try { try {
const fileData = await fs.readFile(persistPath); const fileData = await fs.readFile(persistPath);
+5 -1
View File
@@ -1,3 +1,4 @@
import { consoleLogger, type Logger } from "@llamaindex/env";
import type { JSONSchemaType } from "ajv"; import type { JSONSchemaType } from "ajv";
import { z } from "zod"; import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema"; import { zodToJsonSchema } from "zod-to-json-schema";
@@ -14,11 +15,13 @@ export class FunctionTool<
#additionalArg: AdditionalToolArgument | undefined; #additionalArg: AdditionalToolArgument | undefined;
readonly #metadata: ToolMetadata<JSONSchemaType<T>>; readonly #metadata: ToolMetadata<JSONSchemaType<T>>;
readonly #zodType: z.ZodType<T> | null = null; readonly #zodType: z.ZodType<T> | null = null;
readonly #logger: Logger;
constructor( constructor(
fn: (input: T, additionalArg?: AdditionalToolArgument) => R, fn: (input: T, additionalArg?: AdditionalToolArgument) => R,
metadata: ToolMetadata<JSONSchemaType<T>>, metadata: ToolMetadata<JSONSchemaType<T>>,
zodType?: z.ZodType<T>, zodType?: z.ZodType<T>,
additionalArg?: AdditionalToolArgument, additionalArg?: AdditionalToolArgument,
logger?: Logger,
) { ) {
this.#fn = fn; this.#fn = fn;
this.#metadata = metadata; this.#metadata = metadata;
@@ -26,6 +29,7 @@ export class FunctionTool<
this.#zodType = zodType; this.#zodType = zodType;
} }
this.#additionalArg = additionalArg; this.#additionalArg = additionalArg;
this.#logger = logger ?? consoleLogger;
} }
static from<T, AdditionalToolArgument extends object = object>( static from<T, AdditionalToolArgument extends object = object>(
@@ -140,7 +144,7 @@ export class FunctionTool<
if (result.success) { if (result.success) {
params = result.data; params = result.data;
} else { } else {
console.warn(result.error.errors); this.#logger.warn(result.error.errors);
} }
} }
return this.#fn.call(null, params, this.#additionalArg); return this.#fn.call(null, params, this.#additionalArg);
+6
View File
@@ -1,5 +1,11 @@
# @llamaindex/experimental # @llamaindex/experimental
## 0.0.201
### Patch Changes
- llamaindex@0.11.24
## 0.0.200 ## 0.0.200
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/experimental", "name": "@llamaindex/experimental",
"description": "Experimental package for LlamaIndexTS", "description": "Experimental package for LlamaIndexTS",
"version": "0.0.200", "version": "0.0.201",
"type": "module", "type": "module",
"types": "dist/type/index.d.ts", "types": "dist/type/index.d.ts",
"main": "dist/cjs/index.js", "main": "dist/cjs/index.js",
+11
View File
@@ -1,5 +1,16 @@
# llamaindex # llamaindex
## 0.11.24
### Patch Changes
- Updated dependencies [c3bf3c7]
- Updated dependencies [f9f1de9]
- @llamaindex/cloud@4.0.28
- @llamaindex/core@0.6.19
- @llamaindex/node-parser@2.0.19
- @llamaindex/workflow@1.1.20
## 0.11.23 ## 0.11.23
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "llamaindex", "name": "llamaindex",
"version": "0.11.23", "version": "0.11.24",
"license": "MIT", "license": "MIT",
"type": "module", "type": "module",
"keywords": [ "keywords": [
@@ -8,7 +8,7 @@ import {
BaseInMemoryKVStore, BaseInMemoryKVStore,
SimpleKVStore, SimpleKVStore,
} from "@llamaindex/core/storage/kv-store"; } from "@llamaindex/core/storage/kv-store";
import { path } from "@llamaindex/env"; import { path, type Logger } from "@llamaindex/env";
import _ from "lodash"; import _ from "lodash";
// eslint-disable-next-line @typescript-eslint/no-explicit-any // eslint-disable-next-line @typescript-eslint/no-explicit-any
@@ -27,19 +27,28 @@ export class SimpleDocumentStore extends KVDocumentStore {
static async fromPersistDir( static async fromPersistDir(
persistDir: string = DEFAULT_PERSIST_DIR, persistDir: string = DEFAULT_PERSIST_DIR,
namespace?: string, namespace?: string,
options?: { logger?: Logger },
): Promise<SimpleDocumentStore> { ): Promise<SimpleDocumentStore> {
const persistPath = path.join( const persistPath = path.join(
persistDir, persistDir,
DEFAULT_DOC_STORE_PERSIST_FILENAME, DEFAULT_DOC_STORE_PERSIST_FILENAME,
); );
return await SimpleDocumentStore.fromPersistPath(persistPath, namespace); return await SimpleDocumentStore.fromPersistPath(
persistPath,
namespace,
options,
);
} }
static async fromPersistPath( static async fromPersistPath(
persistPath: string, persistPath: string,
namespace?: string, namespace?: string,
options?: { logger?: Logger },
): Promise<SimpleDocumentStore> { ): Promise<SimpleDocumentStore> {
const simpleKVStore = await SimpleKVStore.fromPersistPath(persistPath); const simpleKVStore = await SimpleKVStore.fromPersistPath(
persistPath,
options,
);
return new SimpleDocumentStore(simpleKVStore, namespace); return new SimpleDocumentStore(simpleKVStore, namespace);
} }
@@ -18,7 +18,7 @@ import {
type VectorStoreQuery, type VectorStoreQuery,
type VectorStoreQueryResult, type VectorStoreQueryResult,
} from "@llamaindex/core/vector-store"; } from "@llamaindex/core/vector-store";
import { fs, path } from "@llamaindex/env"; import { consoleLogger, fs, path, type Logger } from "@llamaindex/env";
import { exists } from "../storage/FileSystem.js"; import { exists } from "../storage/FileSystem.js";
const LEARNER_MODES = new Set<VectorStoreQueryMode>([ const LEARNER_MODES = new Set<VectorStoreQueryMode>([
@@ -139,9 +139,14 @@ export class SimpleVectorStore extends BaseVectorStore {
static async fromPersistDir( static async fromPersistDir(
persistDir: string = DEFAULT_PERSIST_DIR, persistDir: string = DEFAULT_PERSIST_DIR,
embedModel?: BaseEmbedding, embedModel?: BaseEmbedding,
options?: { logger?: Logger },
): Promise<SimpleVectorStore> { ): Promise<SimpleVectorStore> {
const persistPath = path.join(persistDir, "vector_store.json"); const persistPath = path.join(persistDir, "vector_store.json");
return await SimpleVectorStore.fromPersistPath(persistPath, embedModel); return await SimpleVectorStore.fromPersistPath(
persistPath,
embedModel,
options,
);
} }
client() { client() {
@@ -273,7 +278,9 @@ export class SimpleVectorStore extends BaseVectorStore {
static async fromPersistPath( static async fromPersistPath(
persistPath: string, persistPath: string,
embedModel?: BaseEmbedding, embedModel?: BaseEmbedding,
options?: { logger?: Logger },
): Promise<SimpleVectorStore> { ): Promise<SimpleVectorStore> {
const logger = options?.logger ?? consoleLogger;
const dirPath = path.dirname(persistPath); const dirPath = path.dirname(persistPath);
if (!(await exists(dirPath))) { if (!(await exists(dirPath))) {
await fs.mkdir(dirPath, { recursive: true }); await fs.mkdir(dirPath, { recursive: true });
@@ -281,7 +288,7 @@ export class SimpleVectorStore extends BaseVectorStore {
let dataDict: Record<string, unknown> = {}; let dataDict: Record<string, unknown> = {};
if (!(await exists(persistPath))) { if (!(await exists(persistPath))) {
console.info(`Starting new store from path: ${persistPath}`); logger.log(`Starting new store from path: ${persistPath}`);
} else { } else {
try { try {
const fileData = await fs.readFile(persistPath); const fileData = await fs.readFile(persistPath);
+6
View File
@@ -1,5 +1,11 @@
# @llamaindex/core-test # @llamaindex/core-test
## 0.1.15
### Patch Changes
- @llamaindex/openai@0.4.14
## 0.1.14 ## 0.1.14
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/llamaindex-test", "name": "@llamaindex/llamaindex-test",
"private": true, "private": true,
"version": "0.1.14", "version": "0.1.15",
"type": "module", "type": "module",
"scripts": { "scripts": {
"test": "vitest run" "test": "vitest run"
@@ -47,22 +47,31 @@ describe("StorageContext", () => {
test("persists and loads", async () => { test("persists and loads", async () => {
const doc = new Document({ text: "test document" }); const doc = new Document({ text: "test document" });
const consoleInfoSpy = vi // Create a Logger that spies on log (info) calls
.spyOn(console, "info") const spyLogger = {
.mockImplementation(() => {}); log: vi.fn(),
error: vi.fn(),
warn: vi.fn(),
};
// storage context from individual stores // storage context from individual stores
const storageContext = await storageContextFromDefaults({ const storageContext = await storageContextFromDefaults({
docStore: await SimpleDocumentStore.fromPersistDir(testDir), docStore: await SimpleDocumentStore.fromPersistDir(testDir, undefined, {
vectorStore: await SimpleVectorStore.fromPersistDir(testDir), logger: spyLogger,
indexStore: await SimpleIndexStore.fromPersistDir(testDir), }),
vectorStore: await SimpleVectorStore.fromPersistDir(testDir, undefined, {
logger: spyLogger,
}),
indexStore: await SimpleIndexStore.fromPersistDir(testDir, {
logger: spyLogger,
}),
}); });
const index = await VectorStoreIndex.fromDocuments([doc], { const index = await VectorStoreIndex.fromDocuments([doc], {
storageContext, storageContext,
}); });
expect(consoleInfoSpy).toHaveBeenCalledTimes(3); expect(spyLogger.log).toHaveBeenCalledTimes(3);
expect(consoleInfoSpy).toHaveBeenCalledWith( expect(spyLogger.log).toHaveBeenCalledWith(
expect.stringContaining("Starting new store"), expect.stringContaining("Starting new store"),
); );
expect(index).toBeDefined(); expect(index).toBeDefined();
@@ -75,13 +84,19 @@ describe("StorageContext", () => {
// Check that the test data files exist // Check that the test data files exist
await expectTestDataFilesExist(testDir); await expectTestDataFilesExist(testDir);
consoleInfoSpy.mockClear(); spyLogger.log.mockClear();
// Now, load it again. Since data was persisted, we should not see the error. // Now, load it again. Since data was persisted, we should not see the error.
const newStorageContext = await storageContextFromDefaults({ const newStorageContext = await storageContextFromDefaults({
docStore: await SimpleDocumentStore.fromPersistDir(testDir), docStore: await SimpleDocumentStore.fromPersistDir(testDir, undefined, {
vectorStore: await SimpleVectorStore.fromPersistDir(testDir), logger: spyLogger,
indexStore: await SimpleIndexStore.fromPersistDir(testDir), }),
vectorStore: await SimpleVectorStore.fromPersistDir(testDir, undefined, {
logger: spyLogger,
}),
indexStore: await SimpleIndexStore.fromPersistDir(testDir, {
logger: spyLogger,
}),
}); });
const loadedIndex = await VectorStoreIndex.init({ const loadedIndex = await VectorStoreIndex.init({
@@ -94,9 +109,7 @@ describe("StorageContext", () => {
await expectTestDataFilesExist(testDir); await expectTestDataFilesExist(testDir);
expect(consoleInfoSpy).not.toHaveBeenCalled(); expect(spyLogger.log).not.toHaveBeenCalled();
consoleInfoSpy.mockRestore();
}); });
test("throws error on corrupted data", async () => { test("throws error on corrupted data", async () => {
+7
View File
@@ -1,5 +1,12 @@
# @llamaindex/node-parser # @llamaindex/node-parser
## 2.0.19
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 2.0.18 ## 2.0.18
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@llamaindex/node-parser", "name": "@llamaindex/node-parser",
"version": "2.0.18", "version": "2.0.19",
"description": "Node parser for LlamaIndex", "description": "Node parser for LlamaIndex",
"type": "module", "type": "module",
"exports": { "exports": {
@@ -1,5 +1,12 @@
# @llamaindex/anthropic # @llamaindex/anthropic
## 0.3.21
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.3.20 ## 0.3.20
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/anthropic", "name": "@llamaindex/anthropic",
"description": "Anthropic Adapter for LlamaIndex", "description": "Anthropic Adapter for LlamaIndex",
"version": "0.3.20", "version": "0.3.21",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
@@ -1,5 +1,12 @@
# @llamaindex/assemblyai # @llamaindex/assemblyai
## 0.1.18
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.1.17 ## 0.1.17
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/assemblyai", "name": "@llamaindex/assemblyai",
"description": "AssemblyAI Reader for LlamaIndex", "description": "AssemblyAI Reader for LlamaIndex",
"version": "0.1.17", "version": "0.1.18",
"type": "module", "type": "module",
"types": "dist/index.d.ts", "types": "dist/index.d.ts",
"main": "dist/index.cjs", "main": "dist/index.cjs",
+7
View File
@@ -1,5 +1,12 @@
# @llamaindex/community # @llamaindex/community
## 0.0.115
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.114 ## 0.0.114
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/aws", "name": "@llamaindex/aws",
"description": "AWS package for LlamaIndexTS", "description": "AWS package for LlamaIndexTS",
"version": "0.0.114", "version": "0.0.115",
"type": "module", "type": "module",
"types": "dist/type/index.d.ts", "types": "dist/type/index.d.ts",
"main": "dist/cjs/index.js", "main": "dist/cjs/index.js",
+8
View File
@@ -1,5 +1,13 @@
# @llamaindex/clip # @llamaindex/clip
## 0.0.70
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
- @llamaindex/openai@0.4.14
## 0.0.69 ## 0.0.69
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/clip", "name": "@llamaindex/clip",
"description": "Clip Embedding Adapter for LlamaIndex", "description": "Clip Embedding Adapter for LlamaIndex",
"version": "0.0.69", "version": "0.0.70",
"type": "module", "type": "module",
"types": "dist/index.d.ts", "types": "dist/index.d.ts",
"main": "dist/index.cjs", "main": "dist/index.cjs",
+7
View File
@@ -1,5 +1,12 @@
# @llamaindex/cohere # @llamaindex/cohere
## 0.0.33
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.32 ## 0.0.32
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/cohere", "name": "@llamaindex/cohere",
"description": "Cohere Adapter for LlamaIndex", "description": "Cohere Adapter for LlamaIndex",
"version": "0.0.32", "version": "0.0.33",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
@@ -1,5 +1,13 @@
# @llamaindex/deepinfra # @llamaindex/deepinfra
## 0.0.70
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
- @llamaindex/openai@0.4.14
## 0.0.69 ## 0.0.69
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/deepinfra", "name": "@llamaindex/deepinfra",
"description": "Deepinfra Adapter for LlamaIndex", "description": "Deepinfra Adapter for LlamaIndex",
"version": "0.0.69", "version": "0.0.70",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
+6
View File
@@ -1,5 +1,11 @@
# @llamaindex/deepseek # @llamaindex/deepseek
## 0.0.31
### Patch Changes
- @llamaindex/openai@0.4.14
## 0.0.30 ## 0.0.30
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/deepseek", "name": "@llamaindex/deepseek",
"description": "DeepSeek Adapter for LlamaIndex", "description": "DeepSeek Adapter for LlamaIndex",
"version": "0.0.30", "version": "0.0.31",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
+7
View File
@@ -1,5 +1,12 @@
# @llamaindex/discord # @llamaindex/discord
## 0.1.18
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.1.17 ## 0.1.17
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/discord", "name": "@llamaindex/discord",
"description": "Discord Reader for LlamaIndex", "description": "Discord Reader for LlamaIndex",
"version": "0.1.17", "version": "0.1.18",
"type": "module", "type": "module",
"types": "dist/index.d.ts", "types": "dist/index.d.ts",
"main": "dist/index.cjs", "main": "dist/index.cjs",
+7
View File
@@ -1,5 +1,12 @@
# @llamaindex/excel # @llamaindex/excel
## 0.1.19
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.1.18 ## 0.1.18
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/excel", "name": "@llamaindex/excel",
"description": "Excel Reader for LlamaIndex", "description": "Excel Reader for LlamaIndex",
"version": "0.1.18", "version": "0.1.19",
"type": "module", "type": "module",
"types": "dist/index.d.ts", "types": "dist/index.d.ts",
"main": "dist/index.cjs", "main": "dist/index.cjs",
@@ -1,5 +1,11 @@
# @llamaindex/fireworks # @llamaindex/fireworks
## 0.0.30
### Patch Changes
- @llamaindex/openai@0.4.14
## 0.0.29 ## 0.0.29
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/fireworks", "name": "@llamaindex/fireworks",
"description": "Fireworks Adapter for LlamaIndex", "description": "Fireworks Adapter for LlamaIndex",
"version": "0.0.29", "version": "0.0.30",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
+7
View File
@@ -1,5 +1,12 @@
# @llamaindex/google # @llamaindex/google
## 0.3.18
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.3.17 ## 0.3.17
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/google", "name": "@llamaindex/google",
"description": "Google Adapter for LlamaIndex", "description": "Google Adapter for LlamaIndex",
"version": "0.3.17", "version": "0.3.18",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
+6
View File
@@ -1,5 +1,11 @@
# @llamaindex/groq # @llamaindex/groq
## 0.0.86
### Patch Changes
- @llamaindex/openai@0.4.14
## 0.0.85 ## 0.0.85
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/groq", "name": "@llamaindex/groq",
"description": "Groq Adapter for LlamaIndex", "description": "Groq Adapter for LlamaIndex",
"version": "0.0.85", "version": "0.0.86",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
@@ -1,5 +1,13 @@
# @llamaindex/huggingface # @llamaindex/huggingface
## 0.1.24
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
- @llamaindex/openai@0.4.14
## 0.1.23 ## 0.1.23
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/huggingface", "name": "@llamaindex/huggingface",
"description": "Huggingface Adapter for LlamaIndex", "description": "Huggingface Adapter for LlamaIndex",
"version": "0.1.23", "version": "0.1.24",
"type": "module", "type": "module",
"types": "dist/index.d.ts", "types": "dist/index.d.ts",
"main": "dist/index.cjs", "main": "dist/index.cjs",
+8
View File
@@ -1,5 +1,13 @@
# @llamaindex/jinaai # @llamaindex/jinaai
## 0.0.30
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
- @llamaindex/openai@0.4.14
## 0.0.29 ## 0.0.29
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/jinaai", "name": "@llamaindex/jinaai",
"description": "JinaAI Adapter for LlamaIndex", "description": "JinaAI Adapter for LlamaIndex",
"version": "0.0.29", "version": "0.0.30",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
+7
View File
@@ -1,5 +1,12 @@
# @llamaindex/mistral # @llamaindex/mistral
## 0.1.19
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.1.18 ## 0.1.18
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/mistral", "name": "@llamaindex/mistral",
"description": "Mistral Adapter for LlamaIndex", "description": "Mistral Adapter for LlamaIndex",
"version": "0.1.18", "version": "0.1.19",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
@@ -1,5 +1,12 @@
# @llamaindex/mixedbread # @llamaindex/mixedbread
## 0.0.33
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.32 ## 0.0.32
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/mixedbread", "name": "@llamaindex/mixedbread",
"description": "Mixedbread Adapter for LlamaIndex", "description": "Mixedbread Adapter for LlamaIndex",
"version": "0.0.32", "version": "0.0.33",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
+7
View File
@@ -1,5 +1,12 @@
# @llamaindex/notion # @llamaindex/notion
## 0.1.18
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.1.17 ## 0.1.17
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/notion", "name": "@llamaindex/notion",
"description": "Notion Reader for LlamaIndex", "description": "Notion Reader for LlamaIndex",
"version": "0.1.17", "version": "0.1.18",
"type": "module", "type": "module",
"types": "dist/index.d.ts", "types": "dist/index.d.ts",
"main": "dist/index.cjs", "main": "dist/index.cjs",
+7
View File
@@ -1,5 +1,12 @@
# @llamaindex/ollama # @llamaindex/ollama
## 0.1.19
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.1.18 ## 0.1.18
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/ollama", "name": "@llamaindex/ollama",
"description": "Ollama Adapter for LlamaIndex", "description": "Ollama Adapter for LlamaIndex",
"version": "0.1.18", "version": "0.1.19",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
+7
View File
@@ -1,5 +1,12 @@
# @llamaindex/openai # @llamaindex/openai
## 0.4.14
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.4.13 ## 0.4.13
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/openai", "name": "@llamaindex/openai",
"description": "OpenAI Adapter for LlamaIndex", "description": "OpenAI Adapter for LlamaIndex",
"version": "0.4.13", "version": "0.4.14",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
@@ -1,5 +1,13 @@
# @llamaindex/perplexity # @llamaindex/perplexity
## 0.0.27
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
- @llamaindex/openai@0.4.14
## 0.0.26 ## 0.0.26
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/perplexity", "name": "@llamaindex/perplexity",
"description": "Perplexity Adapter for LlamaIndex", "description": "Perplexity Adapter for LlamaIndex",
"version": "0.0.26", "version": "0.0.27",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
@@ -1,5 +1,12 @@
# @llamaindex/portkey-ai # @llamaindex/portkey-ai
## 0.0.61
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.60 ## 0.0.60
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/portkey-ai", "name": "@llamaindex/portkey-ai",
"description": "Portkey Adapter for LlamaIndex", "description": "Portkey Adapter for LlamaIndex",
"version": "0.0.60", "version": "0.0.61",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
@@ -1,5 +1,12 @@
# @llamaindex/replicate # @llamaindex/replicate
## 0.0.61
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.60 ## 0.0.60
### Patch Changes ### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/replicate", "name": "@llamaindex/replicate",
"description": "Replicate Adapter for LlamaIndex", "description": "Replicate Adapter for LlamaIndex",
"version": "0.0.60", "version": "0.0.61",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
@@ -1,5 +1,12 @@
# @llamaindex/bm25-retriever # @llamaindex/bm25-retriever
## 0.0.8
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.7 ## 0.0.7
### Patch Changes ### Patch Changes
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/bm25-retriever", "name": "@llamaindex/bm25-retriever",
"description": "BM25 Retriever for LlamaIndex", "description": "BM25 Retriever for LlamaIndex",
"version": "0.0.7", "version": "0.0.8",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",
@@ -1,5 +1,12 @@
# @llamaindex/astra # @llamaindex/astra
## 0.0.33
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.32 ## 0.0.32
### Patch Changes ### Patch Changes
@@ -1,7 +1,7 @@
{ {
"name": "@llamaindex/astra", "name": "@llamaindex/astra",
"description": "Astra Storage for LlamaIndex", "description": "Astra Storage for LlamaIndex",
"version": "0.0.32", "version": "0.0.33",
"type": "module", "type": "module",
"main": "./dist/index.cjs", "main": "./dist/index.cjs",
"module": "./dist/index.js", "module": "./dist/index.js",

Some files were not shown because too many files have changed in this diff Show More