Compare commits

...

8 Commits

Author SHA1 Message Date
github-actions[bot] 1e296ebe72 Release 0.11.24 (#2141)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: marcusschiesser <17126+marcusschiesser@users.noreply.github.com>
2025-07-30 12:56:45 -04:00
Marcus Schiesser f9f1de9516 chore: use Logger for core (#2139) 2025-07-30 11:43:45 +08:00
Twisha Bansal f576812e7a docs: Using MCP Toolbox for Databases with LlamaIndex (#2138) 2025-07-30 11:19:34 +08:00
Adrian Lyjak c3bf3c7178 Adding support for page citations, and refactor the confidence into the field metadata (#2140) 2025-07-30 10:25:19 +08:00
github-actions[bot] 38487da65d Release 0.11.23 (#2136)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: marcusschiesser <17126+marcusschiesser@users.noreply.github.com>
2025-07-28 14:07:23 +08:00
Marcus Schiesser f29799e385 feat: Add toolcall callbacks to agent workflows (#2137) 2025-07-24 15:37:14 +08:00
Marcus Schiesser 9bca30620b fix: docs build 2025-07-23 12:55:35 +08:00
Marcus Schiesser 7224c06409 feat: Add logger and callbacks to llm.exec (#2135) 2025-07-23 12:37:02 +08:00
150 changed files with 1652 additions and 235 deletions
+28
View File
@@ -1,5 +1,33 @@
# @llamaindex/doc
## 0.2.47
### Patch Changes
- Updated dependencies [c3bf3c7]
- Updated dependencies [f9f1de9]
- @llamaindex/cloud@4.0.28
- @llamaindex/core@0.6.19
- llamaindex@0.11.24
- @llamaindex/node-parser@2.0.19
- @llamaindex/openai@0.4.14
- @llamaindex/readers@3.1.18
- @llamaindex/workflow@1.1.20
## 0.2.46
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/workflow@1.1.19
- @llamaindex/core@0.6.18
- llamaindex@0.11.23
- @llamaindex/cloud@4.0.27
- @llamaindex/node-parser@2.0.18
- @llamaindex/openai@0.4.13
- @llamaindex/readers@3.1.17
## 0.2.45
### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@llamaindex/doc",
"version": "0.2.45",
"version": "0.2.47",
"private": true,
"scripts": {
"postinstall": "fumadocs-mdx",
Binary file not shown.

After

Width:  |  Height:  |  Size: 154 KiB

@@ -0,0 +1,85 @@
---
title: MCP Toolbox For Databases
description: MCP Toolbox for Databases is an open source MCP server for databases.
---
# MCP Toolbox for Databases
[MCP Toolbox for Databases](https://github.com/googleapis/genai-toolbox) is an open source MCP server for databases. It was designed with enterprise-grade and production-quality in mind. It enables you to develop tools easier, faster, and more securely by handling the complexities such as connection pooling, authentication, and more.
Toolbox Tools can be seemlessly integrated with LlamaIndex applications. For more
information on [getting
started](https://googleapis.github.io/genai-toolbox/getting-started/local_quickstart_js/) or
[configuring](https://googleapis.github.io/genai-toolbox/getting-started/configure/)
Toolbox, see the
[documentation](https://googleapis.github.io/genai-toolbox/getting-started/introduction/).
![architecture](/images/mcp_db_toolbox.png)
### Configure and deploy
Toolbox is an open source server that you deploy and manage yourself. For more
instructions on deploying and configuring, see the official Toolbox
documentation:
* [Installing the Server](https://googleapis.github.io/genai-toolbox/getting-started/introduction/#installing-the-server)
* [Configuring Toolbox](https://googleapis.github.io/genai-toolbox/getting-started/configure/)
### Install client SDK
LlamaIndex relies on the `@toolbox-sdk/core` node package to use Toolbox. Install the
package before getting started:
```shell
npm install @toolbox-sdk/core
```
### Loading Toolbox Tools
Once your Toolbox server is configured and up and running, you can load tools
from your server using the SDK:
```javascript
import { gemini, GEMINI_MODEL } from "@llamaindex/google";
import { agent } from "@llamaindex/workflow";
import { tool } from "llamaindex";
import { ToolboxClient } from "@toolbox-sdk/core";
// Initialize LLM
const llm = gemini({
model: GEMINI_MODEL.GEMINI_2_0_FLASH,
apiKey: process.env.GOOGLE_API_KEY,
});
// Replace with your Toolbox Server URL
const URL = 'https://127.0.0.1:5000';
const client = new ToolboxClient("http://127.0.0.1:5000");
const toolboxTools = await client.loadToolset("my-toolset");
const getTool = (toolboxTool) => tool({
name: toolboxTool.getName(),
description: toolboxTool.getDescription(),
parameters: toolboxTool.getParamSchema(),
execute: toolboxTool
});
const tools = toolboxTools.map(getTool);
const myAgent = agent({
tools: tools,
llm,
memory,
systemPrompt: prompt,
});
const result = await myAgent.run(query);
console.log(result);
```
### Advanced Toolbox Features
Toolbox has a variety of features to make developing Gen AI tools for databases seamless.
For more information, read more about the following:
- [Authenticated Parameters](https://googleapis.github.io/genai-toolbox/resources/tools/#authenticated-parameters): bind tool inputs to values from OIDC tokens automatically, making it easy to run sensitive queries without potentially leaking data
- [Authorized Invocations](https://googleapis.github.io/genai-toolbox/resources/tools/#authorized-invocations): restrict access to use a tool based on the users Auth token
- [OpenTelemetry](https://googleapis.github.io/genai-toolbox/how-to/export_telemetry/): get metrics and tracing from Toolbox with [OpenTelemetry](https://opentelemetry.io/docs/)
@@ -1,5 +1,5 @@
{
"title": "Integration",
"description": "See our integrations",
"pages": ["open-llm-metry", "lang-trace", "vercel"]
"pages": ["open-llm-metry", "lang-trace", "mcp-toolbox", "vercel"]
}
@@ -0,0 +1,164 @@
---
title: Low-Level LLM Execution
---
Sometimes your need more control over LLM interactions than what high-level agents provide. The `llm.exec` method makes it simple for you to make a single LLM call with tools but hides the complexity of executing the tools and generating the tool messages.
## When to Use `llm.exec`
Use `llm.exec` when you need to:
- Build custom agent logic in [workflow](/docs/llamaindex/modules/agents/workflows) steps
- Have precise control over message handling and tool execution
## Basic Usage
The `llm.exec` method takes messages and tools as parameter and executes one LLM call.
The LLM might either request to call one or more of the tools or generate an assistant message as result.
For each tool call that is requested, `llm.exec` executes it and generates the two tool call messages (call and result). If no tool call is requested, just the assistant message is returned.
```ts
import { openai } from "@llamaindex/openai";
import { ChatMessage, tool } from "llamaindex";
import z from "zod";
const llm = openai({ model: "gpt-4.1-mini" });
const messages = [
{
content: "What's the weather like in San Francisco?",
role: "user",
} as ChatMessage,
];
const { newMessages, toolCalls } = await llm.exec({
messages,
tools: [
tool({
name: "get_weather",
description: "Get the current weather for a location",
parameters: z.object({
address: z.string().describe("The address"),
}),
execute: ({ address }) => {
return `It's sunny in ${address}!`;
},
}),
],
});
// Add the new messages (including tool calls and responses) to your conversation
messages.push(...newMessages);
```
> `newMessages` is an array as each tool call generates two messages: a tool call message and the tool call result message.
## Agent Loop Pattern
A common pattern is to use `llm.exec` in a loop until the LLM stops making tool calls:
```ts
import { openai } from "@llamaindex/openai";
import { ChatMessage, tool } from "llamaindex";
import z from "zod";
async function runAgentLoop() {
const llm = openai({ model: "gpt-4.1-mini" });
const messages = [
{
content: "What's the weather like in San Francisco?",
role: "user",
} as ChatMessage,
];
let exit = false;
do {
const { newMessages, toolCalls } = await llm.exec({
messages,
tools: [
tool({
name: "get_weather",
description: "Get the current weather for a location",
parameters: z.object({
address: z.string().describe("The address"),
}),
execute: ({ address }) => {
return `It's sunny in ${address}!`;
},
}),
],
});
console.log(newMessages);
messages.push(...newMessages);
// Exit when no more tool calls are made
exit = toolCalls.length === 0;
} while (!exit);
}
```
## Streaming Support
For real-time responses, use the `stream` option to get the assistant's response as streamed tokens:
```ts
import { openai } from "@llamaindex/openai";
import { tool } from "llamaindex";
import z from "zod";
async function streamingAgentLoop() {
const llm = openai({ model: "gpt-4o-mini" });
const messages = [
{
content: "What's the weather like in San Francisco?",
role: "user",
} as ChatMessage,
];
let exit = false;
do {
const { stream, newMessages, toolCalls } = await llm.exec({
messages,
tools: [
tool({
name: "get_weather",
description: "Get the current weather for a location",
parameters: z.object({
address: z.string().describe("The address"),
}),
execute: ({ address }) => {
return `It's sunny in ${address}!`;
},
}),
],
stream: true,
});
// Stream the response token by token
for await (const chunk of stream) {
process.stdout.write(chunk.delta);
}
messages.push(...newMessages());
exit = toolCalls.length === 0;
} while (!exit);
}
```
> `newMessages` is a function when streaming. The reason is that the result only is available after streaming. Calling it before, will throw an error.
## Return Values
`llm.exec` returns an object with:
- **`newMessages`**: Array of new chat messages including the LLM response and any tool call messages (call or result). This is a function return the array when streaming.
- **`toolCalls`**: Array of tool calls made by the LLM
- **`stream`**: Async iterable for streaming responses (only when `stream: true`)
## Best Practices
For using `llm.exec` in an agent loop, take care to:
1. **Maintain message history**: Always add `newMessages` to your conversation history
2. **Set exit conditions**: Implement proper logic to avoid infinite loops
@@ -1,4 +1,10 @@
{
"title": "Agents",
"pages": ["tool", "agent_workflow", "workflows", "natural_language_workflow"]
"pages": [
"tool",
"agent_workflow",
"workflows",
"low-level",
"natural_language_workflow"
]
}
@@ -101,6 +101,9 @@ const agent = agent({
});
```
You can also use [MCP Toolbox for
Databases](/docs/llamaindex/integration/mcp-toolbox) to interact with MCP tools.
## Function tool
@@ -1,5 +1,17 @@
# @llamaindex/cloudflare-worker-agent-test
## 0.0.185
### Patch Changes
- llamaindex@0.11.24
## 0.0.184
### Patch Changes
- llamaindex@0.11.23
## 0.0.183
### Patch Changes
@@ -1,6 +1,6 @@
{
"name": "@llamaindex/cloudflare-worker-agent-test",
"version": "0.0.183",
"version": "0.0.185",
"type": "module",
"private": true,
"scripts": {
@@ -1,5 +1,18 @@
# @llamaindex/llama-parse-browser-test
## 0.0.83
### Patch Changes
- Updated dependencies [c3bf3c7]
- @llamaindex/cloud@4.0.28
## 0.0.82
### Patch Changes
- @llamaindex/cloud@4.0.27
## 0.0.81
### Patch Changes
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/llama-parse-browser-test",
"private": true,
"version": "0.0.81",
"version": "0.0.83",
"type": "module",
"scripts": {
"dev": "vite",
+12
View File
@@ -1,5 +1,17 @@
# @llamaindex/next-agent-test
## 0.1.185
### Patch Changes
- llamaindex@0.11.24
## 0.1.184
### Patch Changes
- llamaindex@0.11.23
## 0.1.183
### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@llamaindex/next-agent-test",
"version": "0.1.183",
"version": "0.1.185",
"private": true,
"scripts": {
"dev": "next dev",
@@ -1,5 +1,17 @@
# test-edge-runtime
## 0.1.184
### Patch Changes
- llamaindex@0.11.24
## 0.1.183
### Patch Changes
- llamaindex@0.11.23
## 0.1.182
### Patch Changes
@@ -1,6 +1,6 @@
{
"name": "@llamaindex/nextjs-edge-runtime-test",
"version": "0.1.182",
"version": "0.1.184",
"private": true,
"scripts": {
"dev": "next dev",
@@ -1,5 +1,21 @@
# @llamaindex/next-node-runtime
## 0.1.54
### Patch Changes
- llamaindex@0.11.24
- @llamaindex/huggingface@0.1.24
- @llamaindex/readers@3.1.18
## 0.1.53
### Patch Changes
- llamaindex@0.11.23
- @llamaindex/huggingface@0.1.23
- @llamaindex/readers@3.1.17
## 0.1.52
### Patch Changes
@@ -1,6 +1,6 @@
{
"name": "@llamaindex/next-node-runtime-test",
"version": "0.1.52",
"version": "0.1.54",
"private": true,
"scripts": {
"dev": "next dev",
@@ -1,5 +1,17 @@
# vite-import-llamaindex
## 0.0.51
### Patch Changes
- llamaindex@0.11.24
## 0.0.50
### Patch Changes
- llamaindex@0.11.23
## 0.0.49
### Patch Changes
@@ -1,7 +1,7 @@
{
"name": "vite-import-llamaindex",
"private": true,
"version": "0.0.49",
"version": "0.0.51",
"type": "module",
"scripts": {
"build": "vite build",
@@ -1,5 +1,17 @@
# @llamaindex/waku-query-engine-test
## 0.0.185
### Patch Changes
- llamaindex@0.11.24
## 0.0.184
### Patch Changes
- llamaindex@0.11.23
## 0.0.183
### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@llamaindex/waku-query-engine-test",
"version": "0.0.183",
"version": "0.0.185",
"type": "module",
"private": true,
"scripts": {
+106
View File
@@ -1,5 +1,111 @@
# examples
## 0.3.35
### Patch Changes
- Updated dependencies [c3bf3c7]
- Updated dependencies [f9f1de9]
- @llamaindex/cloud@4.0.28
- @llamaindex/core@0.6.19
- llamaindex@0.11.24
- @llamaindex/node-parser@2.0.19
- @llamaindex/anthropic@0.3.21
- @llamaindex/assemblyai@0.1.18
- @llamaindex/clip@0.0.70
- @llamaindex/cohere@0.0.33
- @llamaindex/deepinfra@0.0.70
- @llamaindex/discord@0.1.18
- @llamaindex/google@0.3.18
- @llamaindex/huggingface@0.1.24
- @llamaindex/jinaai@0.0.30
- @llamaindex/mistral@0.1.19
- @llamaindex/mixedbread@0.0.33
- @llamaindex/notion@0.1.18
- @llamaindex/ollama@0.1.19
- @llamaindex/openai@0.4.14
- @llamaindex/perplexity@0.0.27
- @llamaindex/portkey-ai@0.0.61
- @llamaindex/replicate@0.0.61
- @llamaindex/bm25-retriever@0.0.8
- @llamaindex/astra@0.0.33
- @llamaindex/azure@0.1.31
- @llamaindex/chroma@0.0.33
- @llamaindex/elastic-search@0.1.19
- @llamaindex/firestore@1.0.26
- @llamaindex/milvus@0.1.28
- @llamaindex/mongodb@0.0.34
- @llamaindex/pinecone@0.1.19
- @llamaindex/postgres@0.0.62
- @llamaindex/qdrant@0.1.29
- @llamaindex/supabase@0.1.20
- @llamaindex/upstash@0.0.33
- @llamaindex/weaviate@0.0.34
- @llamaindex/vercel@0.1.19
- @llamaindex/voyage-ai@1.0.25
- @llamaindex/readers@3.1.18
- @llamaindex/tools@0.1.9
- @llamaindex/workflow@1.1.20
- @llamaindex/deepseek@0.0.31
- @llamaindex/fireworks@0.0.30
- @llamaindex/groq@0.0.86
- @llamaindex/together@0.0.30
- @llamaindex/vllm@0.0.56
- @llamaindex/xai@0.0.17
## 0.3.34
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/workflow@1.1.19
- @llamaindex/core@0.6.18
- llamaindex@0.11.23
- @llamaindex/cloud@4.0.27
- @llamaindex/node-parser@2.0.18
- @llamaindex/anthropic@0.3.20
- @llamaindex/assemblyai@0.1.17
- @llamaindex/clip@0.0.69
- @llamaindex/cohere@0.0.32
- @llamaindex/deepinfra@0.0.69
- @llamaindex/discord@0.1.17
- @llamaindex/google@0.3.17
- @llamaindex/huggingface@0.1.23
- @llamaindex/jinaai@0.0.29
- @llamaindex/mistral@0.1.18
- @llamaindex/mixedbread@0.0.32
- @llamaindex/notion@0.1.17
- @llamaindex/ollama@0.1.18
- @llamaindex/openai@0.4.13
- @llamaindex/perplexity@0.0.26
- @llamaindex/portkey-ai@0.0.60
- @llamaindex/replicate@0.0.60
- @llamaindex/bm25-retriever@0.0.7
- @llamaindex/astra@0.0.32
- @llamaindex/azure@0.1.30
- @llamaindex/chroma@0.0.32
- @llamaindex/elastic-search@0.1.18
- @llamaindex/firestore@1.0.25
- @llamaindex/milvus@0.1.27
- @llamaindex/mongodb@0.0.33
- @llamaindex/pinecone@0.1.18
- @llamaindex/postgres@0.0.61
- @llamaindex/qdrant@0.1.28
- @llamaindex/supabase@0.1.19
- @llamaindex/upstash@0.0.32
- @llamaindex/weaviate@0.0.33
- @llamaindex/vercel@0.1.18
- @llamaindex/voyage-ai@1.0.24
- @llamaindex/readers@3.1.17
- @llamaindex/tools@0.1.8
- @llamaindex/deepseek@0.0.30
- @llamaindex/fireworks@0.0.29
- @llamaindex/groq@0.0.85
- @llamaindex/together@0.0.29
- @llamaindex/vllm@0.0.55
- @llamaindex/xai@0.0.16
## 0.3.33
### Patch Changes
+47 -47
View File
@@ -1,6 +1,6 @@
{
"name": "@llamaindex/examples",
"version": "0.3.33",
"version": "0.3.35",
"private": true,
"scripts": {
"lint": "eslint .",
@@ -11,52 +11,52 @@
"@azure/cosmos": "^4.1.1",
"@azure/identity": "^4.4.1",
"@azure/search-documents": "^12.1.0",
"@llamaindex/anthropic": "^0.3.19",
"@llamaindex/assemblyai": "^0.1.16",
"@llamaindex/astra": "^0.0.31",
"@llamaindex/azure": "^0.1.29",
"@llamaindex/bm25-retriever": "^0.0.6",
"@llamaindex/chroma": "^0.0.31",
"@llamaindex/clip": "^0.0.68",
"@llamaindex/cloud": "^4.0.26",
"@llamaindex/cohere": "^0.0.31",
"@llamaindex/core": "^0.6.17",
"@llamaindex/deepinfra": "^0.0.68",
"@llamaindex/deepseek": "^0.0.29",
"@llamaindex/discord": "^0.1.16",
"@llamaindex/elastic-search": "^0.1.17",
"@llamaindex/anthropic": "^0.3.21",
"@llamaindex/assemblyai": "^0.1.18",
"@llamaindex/astra": "^0.0.33",
"@llamaindex/azure": "^0.1.31",
"@llamaindex/bm25-retriever": "^0.0.8",
"@llamaindex/chroma": "^0.0.33",
"@llamaindex/clip": "^0.0.70",
"@llamaindex/cloud": "^4.0.28",
"@llamaindex/cohere": "^0.0.33",
"@llamaindex/core": "^0.6.19",
"@llamaindex/deepinfra": "^0.0.70",
"@llamaindex/deepseek": "^0.0.31",
"@llamaindex/discord": "^0.1.18",
"@llamaindex/elastic-search": "^0.1.19",
"@llamaindex/env": "^0.1.30",
"@llamaindex/firestore": "^1.0.24",
"@llamaindex/fireworks": "^0.0.28",
"@llamaindex/google": "^0.3.16",
"@llamaindex/groq": "^0.0.84",
"@llamaindex/huggingface": "^0.1.22",
"@llamaindex/jinaai": "^0.0.28",
"@llamaindex/milvus": "^0.1.26",
"@llamaindex/mistral": "^0.1.17",
"@llamaindex/mixedbread": "^0.0.31",
"@llamaindex/mongodb": "^0.0.32",
"@llamaindex/node-parser": "^2.0.17",
"@llamaindex/notion": "^0.1.16",
"@llamaindex/ollama": "^0.1.17",
"@llamaindex/openai": "^0.4.12",
"@llamaindex/perplexity": "^0.0.25",
"@llamaindex/pinecone": "^0.1.17",
"@llamaindex/portkey-ai": "^0.0.59",
"@llamaindex/postgres": "^0.0.60",
"@llamaindex/qdrant": "^0.1.27",
"@llamaindex/readers": "^3.1.16",
"@llamaindex/replicate": "^0.0.59",
"@llamaindex/supabase": "^0.1.18",
"@llamaindex/together": "^0.0.28",
"@llamaindex/tools": "^0.1.7",
"@llamaindex/upstash": "^0.0.31",
"@llamaindex/vercel": "^0.1.17",
"@llamaindex/vllm": "^0.0.54",
"@llamaindex/voyage-ai": "^1.0.23",
"@llamaindex/weaviate": "^0.0.32",
"@llamaindex/workflow": "^1.1.17",
"@llamaindex/xai": "^0.0.15",
"@llamaindex/firestore": "^1.0.26",
"@llamaindex/fireworks": "^0.0.30",
"@llamaindex/google": "^0.3.18",
"@llamaindex/groq": "^0.0.86",
"@llamaindex/huggingface": "^0.1.24",
"@llamaindex/jinaai": "^0.0.30",
"@llamaindex/milvus": "^0.1.28",
"@llamaindex/mistral": "^0.1.19",
"@llamaindex/mixedbread": "^0.0.33",
"@llamaindex/mongodb": "^0.0.34",
"@llamaindex/node-parser": "^2.0.19",
"@llamaindex/notion": "^0.1.18",
"@llamaindex/ollama": "^0.1.19",
"@llamaindex/openai": "^0.4.14",
"@llamaindex/perplexity": "^0.0.27",
"@llamaindex/pinecone": "^0.1.19",
"@llamaindex/portkey-ai": "^0.0.61",
"@llamaindex/postgres": "^0.0.62",
"@llamaindex/qdrant": "^0.1.29",
"@llamaindex/readers": "^3.1.18",
"@llamaindex/replicate": "^0.0.61",
"@llamaindex/supabase": "^0.1.20",
"@llamaindex/together": "^0.0.30",
"@llamaindex/tools": "^0.1.9",
"@llamaindex/upstash": "^0.0.33",
"@llamaindex/vercel": "^0.1.19",
"@llamaindex/vllm": "^0.0.56",
"@llamaindex/voyage-ai": "^1.0.25",
"@llamaindex/weaviate": "^0.0.34",
"@llamaindex/workflow": "^1.1.20",
"@llamaindex/xai": "^0.0.17",
"@notionhq/client": "^4.0.0",
"@pinecone-database/pinecone": "^4.0.0",
"@vercel/postgres": "^0.10.0",
@@ -65,7 +65,7 @@
"commander": "^12.1.0",
"dotenv": "^17.2.0",
"js-tiktoken": "^1.0.14",
"llamaindex": "^0.11.21",
"llamaindex": "^0.11.24",
"mongodb": "6.7.0",
"postgres": "^3.4.4",
"wikipedia": "^2.1.2",
+12
View File
@@ -1,5 +1,17 @@
# @llamaindex/autotool
## 8.0.24
### Patch Changes
- llamaindex@0.11.24
## 8.0.23
### Patch Changes
- llamaindex@0.11.23
## 8.0.22
### Patch Changes
@@ -1,5 +1,19 @@
# @llamaindex/autotool-01-node-example
## 0.0.132
### Patch Changes
- llamaindex@0.11.24
- @llamaindex/autotool@8.0.24
## 0.0.131
### Patch Changes
- llamaindex@0.11.23
- @llamaindex/autotool@8.0.23
## 0.0.130
### Patch Changes
@@ -13,5 +13,5 @@
"scripts": {
"start": "node --import tsx --import @llamaindex/autotool/node ./src/index.ts"
},
"version": "0.0.130"
"version": "0.0.132"
}
+1 -1
View File
@@ -6,7 +6,7 @@
"url": "git+https://github.com/run-llama/LlamaIndexTS.git",
"directory": "packages/autotool"
},
"version": "8.0.22",
"version": "8.0.24",
"description": "auto transpile your JS function to LLM Agent compatible",
"files": [
"dist",
+16
View File
@@ -1,5 +1,21 @@
# @llamaindex/cloud
## 4.0.28
### Patch Changes
- c3bf3c7: Adding support for citations to beta agent data schema
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 4.0.27
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 4.0.26
### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@llamaindex/cloud",
"version": "4.0.26",
"version": "4.0.28",
"type": "module",
"license": "MIT",
"scripts": {
+28 -3
View File
@@ -28,6 +28,29 @@ export type ComparisonOperator =
*/
export type FilterOperation = RawFilterOperation;
/**
* Metadata for an extracted field, including confidence and citation information
*/
export interface ExtractedFieldMetadata {
/** The confidence score for the field, combined with parsing confidence if applicable */
confidence?: number;
/** The confidence score for the field based on the extracted text only */
extracted_confidence?: number;
/** The page number that the field occurred on */
page_number?: number;
/** The original text this field's value was derived from */
matching_text?: string;
}
/**
* Dictionary mapping field names to their metadata
* Values can be ExtractedFieldMetadata objects, nested dictionaries, or arrays
*/
export type ExtractedFieldMetadataDict = Record<
string,
ExtractedFieldMetadata | Record<string, unknown> | unknown[]
>;
/**
* Base extracted data interface
*/
@@ -35,11 +58,13 @@ export interface ExtractedData<T = unknown> {
/** The original data that was extracted from the document. For tracking changes. Should not be updated. */
original_data: T;
/** The latest state of the data. Will differ if data has been updated. */
data?: T;
data: T;
/** The status of the extracted data. Prefer to use the StatusType values, but any string is allowed. */
status: StatusType | string;
/** Confidence scores, if any, for each primitive field in the original_data data. */
confidence?: Record<string, unknown>;
/** The overall confidence score for the extracted data. */
overall_confidence?: number;
/** Page links, and perhaps eventually bounding boxes, for individual fields in the extracted data. */
field_metadata?: ExtractedFieldMetadataDict;
/** The ID of the file that was used to extract the data. */
file_id?: string;
/** The name of the file that was used to extract the data. */
+15
View File
@@ -1,5 +1,20 @@
# @llamaindex/community
## 0.0.101
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.100
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.0.99
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/community",
"description": "Community package for LlamaIndexTS",
"version": "0.0.99",
"version": "0.0.101",
"type": "module",
"types": "dist/type/index.d.ts",
"main": "dist/cjs/index.js",
+13
View File
@@ -1,5 +1,18 @@
# @llamaindex/core
## 0.6.19
### Patch Changes
- f9f1de9: Use logger interface instead of directly hardcoding console.log
## 0.6.18
### Patch Changes
- f29799e: Add toolcall callbacks to agent workflows
- 7224c06: Add logger and callbacks to llm.exec
## 0.6.17
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/core",
"type": "module",
"version": "0.6.17",
"version": "0.6.19",
"description": "LlamaIndex Core Module",
"exports": {
"./agent": {
+2
View File
@@ -15,6 +15,7 @@ import type {
} from "../llms";
import { baseToolWithCallSchema } from "../schema";
import {
assertIsJSONValue,
isAsyncIterable,
prettifyError,
stringifyJSONToMessageContent,
@@ -227,6 +228,7 @@ export async function callTool(
`Tool ${tool.metadata.name} (remote:${toolCall.name}) succeeded.`,
);
logger.log(`Output: ${JSON.stringify(output)}`);
assertIsJSONValue(output);
const toolOutput: ToolOutput = {
tool,
input,
+6 -1
View File
@@ -1,3 +1,4 @@
import { consoleLogger, emptyLogger, type Logger } from "@llamaindex/env";
import type { Tokenizers } from "@llamaindex/env/tokenizers";
import type { MessageContentDetail } from "../llms";
import { BaseNode, MetadataMode, TransformComponent } from "../schema";
@@ -18,6 +19,7 @@ export type EmbeddingInfo = {
export type BaseEmbeddingOptions = {
logProgress?: boolean;
progressCallback?: (current: number, total: number) => void;
logger?: Logger;
};
export abstract class BaseEmbedding extends TransformComponent<
@@ -133,6 +135,9 @@ export async function batchEmbeddings<T>(
const curBatch: T[] = [];
const logger =
options?.logger ?? (options?.logProgress ? consoleLogger : emptyLogger);
for (let i = 0; i < queue.length; i++) {
curBatch.push(queue[i]!);
if (i == queue.length - 1 || curBatch.length == chunkSize) {
@@ -143,7 +148,7 @@ export async function batchEmbeddings<T>(
options?.progressCallback?.(i + 1, queue.length);
}
if (options?.logProgress) {
console.log(`getting embedding progress: ${i + 1} / ${queue.length}`);
logger.log(`getting embedding progress: ${i + 1} / ${queue.length}`);
}
curBatch.length = 0;
+16 -9
View File
@@ -1,6 +1,7 @@
import { emptyLogger } from "@llamaindex/env";
import { extractText } from "../utils/llms";
import { streamConverter } from "../utils/stream";
import { callTool, getToolCallsFromResponse } from "./tool-call";
import { callToolToMessage, getToolCallsFromResponse } from "./tool-call";
import type {
ChatMessage,
ChatResponse,
@@ -99,16 +100,19 @@ export abstract class BaseLLM<
if (params.stream) {
return this.streamExec(params);
}
const logger = params.logger ?? emptyLogger;
const newMessages: ChatMessage<AdditionalMessageOptions>[] = [];
const response = await this.chat(params);
newMessages.push(response.message);
const toolCalls = getToolCallsFromResponse(response);
if (params.tools && toolCalls.length > 0) {
for (const toolCall of toolCalls) {
const toolResultMessage = await callTool<AdditionalMessageOptions>(
params.tools,
toolCall,
);
const toolResultMessage =
await callToolToMessage<AdditionalMessageOptions>(
params.tools,
toolCall,
logger,
);
if (toolResultMessage) {
newMessages.push(toolResultMessage);
}
@@ -126,6 +130,7 @@ export abstract class BaseLLM<
AdditionalMessageOptions
>,
): Promise<ExecStreamResponse<AdditionalMessageOptions>> {
const logger = params.logger ?? emptyLogger;
const responseStream = await this.chat(params);
const iterator = responseStream[Symbol.asyncIterator]();
const first = await iterator.next();
@@ -220,10 +225,12 @@ export abstract class BaseLLM<
} as AdditionalMessageOptions,
});
for (const toolCall of toolCalls) {
const toolResultMessage = await callTool<AdditionalMessageOptions>(
params.tools,
toolCall,
);
const toolResultMessage =
await callToolToMessage<AdditionalMessageOptions>(
params.tools,
toolCall,
logger,
);
if (toolResultMessage) {
messages.push(toolResultMessage);
}
+20 -17
View File
@@ -1,3 +1,5 @@
import { type Logger } from "@llamaindex/env";
import { callTool } from "../agent/utils.js";
import { stringifyJSONToMessageContent } from "../utils";
import type {
BaseTool,
@@ -35,27 +37,28 @@ export const getToolCallsFromResponse = (
return [];
};
export const callTool = async <
export const callToolToMessage = async <
AdditionalMessageOptions extends object = object,
>(
tools: BaseTool[],
toolCall: ToolCall,
logger: Logger,
): Promise<ChatMessage<AdditionalMessageOptions> | null> => {
const tool = tools?.find((t) => t.metadata.name === toolCall.name);
// TODO: consider using BaseToolWithCall instead of BaseTool to avoid checking for tool.call
if (tool && tool.call) {
const result = await tool.call(toolCall.input);
const toolResultMessage: ChatMessage<AdditionalMessageOptions> = {
role: "user",
content: stringifyJSONToMessageContent(result),
options: {
toolResult: {
id: toolCall.id,
result,
},
} as AdditionalMessageOptions,
};
return toolResultMessage;
}
return null;
const toolOutput = await callTool(tool, toolCall, logger);
const toolResultMessage: ChatMessage<AdditionalMessageOptions> = {
role: "user",
content: stringifyJSONToMessageContent(toolOutput.output),
options: {
toolResult: {
id: toolCall.id,
result: toolOutput.output,
isError: toolOutput.isError,
},
} as AdditionalMessageOptions,
};
return toolResultMessage;
};
+2
View File
@@ -1,3 +1,4 @@
import type { Logger } from "@llamaindex/env";
import type { Tokenizers } from "@llamaindex/env/tokenizers";
import type { JSONSchemaType } from "ajv";
import { z } from "zod";
@@ -139,6 +140,7 @@ export interface LLMChatParamsBase<
additionalChatOptions?: AdditionalChatOptions | undefined;
tools?: BaseTool[] | undefined;
responseFormat?: z.ZodType | object | undefined;
logger?: Logger | undefined;
}
export interface LLMChatParamsStreaming<
+13 -2
View File
@@ -1,3 +1,4 @@
import { consoleLogger, type Logger } from "@llamaindex/env";
import { Settings } from "../global";
import type { ChatMessage, LLM } from "../llms";
import { extractText } from "../utils";
@@ -38,6 +39,11 @@ export type MemoryOptions<TMessageOptions extends object = object> = {
* This default LLM can be overridden by the LLM passed in the `getLLM` method.
*/
llm?: LLM | undefined;
/**
* Logger for memory operations
*/
logger?: Logger;
};
export class Memory<
@@ -76,6 +82,10 @@ export class Memory<
* The default LLM to use for memory retrieval.
*/
private llm: LLM | undefined;
/**
* Logger for memory operations
*/
private logger: Logger;
constructor(
messages: MemoryMessage<TMessageOptions>[] = [],
@@ -87,6 +97,7 @@ export class Memory<
options.shortTermTokenLimitRatio ?? DEFAULT_SHORT_TERM_TOKEN_LIMIT_RATIO;
this.memoryBlocks = options.memoryBlocks ?? [];
this.memoryCursor = options.memoryCursor ?? 0;
this.logger = options.logger ?? consoleLogger;
this.initLLM(options.llm);
this.adapters = {
@@ -309,7 +320,7 @@ export class Memory<
addedTokenCount += messageTokenCount;
}
} catch (error) {
console.warn(
this.logger.warn(
`Failed to get content from memory block ${block.id}:`,
error,
);
@@ -371,7 +382,7 @@ export class Memory<
try {
await block.put(newMessages);
} catch (error) {
console.warn(
this.logger.warn(
`Failed to process messages into memory block ${block.id}:`,
error,
);
@@ -1,3 +1,4 @@
import { consoleLogger, type Logger } from "@llamaindex/env";
import type { Tokenizer } from "@llamaindex/env/tokenizers";
import { z } from "zod";
import { Settings } from "../global";
@@ -48,9 +49,11 @@ export class SentenceSplitter extends MetadataAwareTextSplitter {
#splitFns: Set<TextSplitterFn> = new Set();
#subSentenceSplitFns: Set<TextSplitterFn> = new Set();
#tokenizer: Tokenizer;
#logger: Logger;
constructor(
params?: z.input<typeof sentenceSplitterSchema> & SplitterParams,
params?: z.input<typeof sentenceSplitterSchema> &
SplitterParams & { logger?: Logger },
) {
super();
if (params) {
@@ -66,6 +69,7 @@ export class SentenceSplitter extends MetadataAwareTextSplitter {
this.extraAbbreviations,
);
this.#tokenizer = params?.tokenizer ?? Settings.tokenizer;
this.#logger = params?.logger ?? consoleLogger;
this.#splitFns.add(splitBySep(this.paragraphSeparator));
this.#splitFns.add(this.#chunkingTokenizerFn);
@@ -82,7 +86,7 @@ export class SentenceSplitter extends MetadataAwareTextSplitter {
`Metadata length (${metadataLength}) is longer than chunk size (${this.chunkSize}). Consider increasing the chunk size or decreasing the size of your metadata to avoid this.`,
);
} else if (effectiveChunkSize < 50) {
console.log(
this.#logger.log(
`Metadata length (${metadataLength}) is close to chunk size (${this.chunkSize}). Resulting chunks are less than 50 tokens. Consider increasing the chunk size or decreasing the size of your metadata to avoid this.`,
);
}
@@ -1,3 +1,4 @@
import { consoleLogger, type Logger } from "@llamaindex/env";
import type { Tokenizer } from "@llamaindex/env/tokenizers";
import { z } from "zod";
import { DEFAULT_CHUNK_OVERLAP, DEFAULT_CHUNK_SIZE, Settings } from "../global";
@@ -21,9 +22,11 @@ export class TokenTextSplitter extends MetadataAwareTextSplitter {
backupSeparators: string[] = ["\n"];
#tokenizer: Tokenizer;
#splitFns: Array<(text: string) => string[]> = [];
#logger: Logger;
constructor(
params?: SplitterParams & Partial<z.infer<typeof tokenTextSplitterSchema>>,
params?: SplitterParams &
Partial<z.infer<typeof tokenTextSplitterSchema>> & { logger?: Logger },
) {
super();
@@ -42,6 +45,7 @@ export class TokenTextSplitter extends MetadataAwareTextSplitter {
}
this.#tokenizer = params?.tokenizer ?? Settings.tokenizer;
this.#logger = params?.logger ?? consoleLogger;
const allSeparators = [this.separator, ...this.backupSeparators];
this.#splitFns = allSeparators.map((sep) => splitBySep(sep));
@@ -65,7 +69,7 @@ export class TokenTextSplitter extends MetadataAwareTextSplitter {
`Consider increasing the chunk size or decreasing the size of your metadata to avoid this.`,
);
} else if (effectiveChunkSize < 50) {
console.warn(
this.#logger.warn(
`Metadata length (${metadataLength}) is close to chunk size (${this.chunkSize}). ` +
`Resulting chunks are less than 50 tokens. Consider increasing the chunk size or decreasing the size of your metadata to avoid this.`,
);
@@ -148,7 +152,7 @@ export class TokenTextSplitter extends MetadataAwareTextSplitter {
const splitLength = this.tokenSize(split);
if (splitLength > chunkSize) {
console.warn(
this.#logger.warn(
`Got a split of size ${splitLength}, larger than chunk size ${chunkSize}.`,
);
}
@@ -1,3 +1,4 @@
import { consoleLogger, type Logger } from "@llamaindex/env";
import { DEFAULT_NAMESPACE } from "../../global";
import { BaseNode, ObjectType, type StoredValue } from "../../schema";
import type { BaseKVStore } from "../kv-store";
@@ -16,13 +17,19 @@ export class KVDocumentStore extends BaseDocumentStore {
private nodeCollection: string;
private refDocCollection: string;
private metadataCollection: string;
private logger: Logger;
constructor(kvstore: BaseKVStore, namespace: string = DEFAULT_NAMESPACE) {
constructor(
kvstore: BaseKVStore,
namespace: string = DEFAULT_NAMESPACE,
options?: { logger?: Logger },
) {
super();
this.kvstore = kvstore;
this.nodeCollection = `${namespace}/data`;
this.refDocCollection = `${namespace}/ref_doc_info`;
this.metadataCollection = `${namespace}/metadata`;
this.logger = options?.logger ?? consoleLogger;
}
async docs(): Promise<Record<string, BaseNode>> {
@@ -33,7 +40,7 @@ export class KVDocumentStore extends BaseDocumentStore {
if (isValidDocJson(value)) {
docs[key] = jsonToDoc(value, this.serializer);
} else {
console.warn(`Invalid JSON for docId ${key}`);
this.logger.warn(`Invalid JSON for docId ${key}`);
}
}
return docs;
+12 -5
View File
@@ -1,4 +1,4 @@
import { path } from "@llamaindex/env";
import { path, type Logger } from "@llamaindex/env";
import { IndexStruct, jsonToIndexStruct } from "../../data-structs";
import {
DEFAULT_INDEX_STORE_PERSIST_FILENAME,
@@ -8,8 +8,8 @@ import {
import {
BaseInMemoryKVStore,
BaseKVStore,
type DataType,
SimpleKVStore,
type DataType,
} from "../kv-store";
export const DEFAULT_PERSIST_PATH = path.join(
@@ -84,16 +84,23 @@ export class SimpleIndexStore extends KVIndexStore {
static async fromPersistDir(
persistDir: string = DEFAULT_PERSIST_DIR,
options?: { logger?: Logger },
): Promise<SimpleIndexStore> {
const persistPath = path.join(
persistDir,
DEFAULT_INDEX_STORE_PERSIST_FILENAME,
);
return this.fromPersistPath(persistPath);
return this.fromPersistPath(persistPath, options);
}
static async fromPersistPath(persistPath: string): Promise<SimpleIndexStore> {
const simpleKVStore = await SimpleKVStore.fromPersistPath(persistPath);
static async fromPersistPath(
persistPath: string,
options?: { logger?: Logger },
): Promise<SimpleIndexStore> {
const simpleKVStore = await SimpleKVStore.fromPersistPath(
persistPath,
options,
);
return new SimpleIndexStore(simpleKVStore);
}
+7 -3
View File
@@ -1,4 +1,4 @@
import { fs, path } from "@llamaindex/env";
import { consoleLogger, fs, path, type Logger } from "@llamaindex/env";
import { DEFAULT_COLLECTION } from "../../global";
import type { StoredValue } from "../../schema";
@@ -98,7 +98,11 @@ export class SimpleKVStore extends BaseKVStore {
await fs.writeFile(persistPath, JSON.stringify(this.data));
}
static async fromPersistPath(persistPath: string): Promise<SimpleKVStore> {
static async fromPersistPath(
persistPath: string,
options?: { logger?: Logger },
): Promise<SimpleKVStore> {
const logger = options?.logger ?? consoleLogger;
const dirPath = path.dirname(persistPath);
if (!(await exists(dirPath))) {
await fs.mkdir(dirPath, { recursive: true });
@@ -106,7 +110,7 @@ export class SimpleKVStore extends BaseKVStore {
let data: DataType = {};
if (!(await exists(persistPath))) {
console.info(`Starting new store from path: ${persistPath}`);
logger.log(`Starting new store from path: ${persistPath}`);
} else {
try {
const fileData = await fs.readFile(persistPath);
+5 -1
View File
@@ -1,3 +1,4 @@
import { consoleLogger, type Logger } from "@llamaindex/env";
import type { JSONSchemaType } from "ajv";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
@@ -14,11 +15,13 @@ export class FunctionTool<
#additionalArg: AdditionalToolArgument | undefined;
readonly #metadata: ToolMetadata<JSONSchemaType<T>>;
readonly #zodType: z.ZodType<T> | null = null;
readonly #logger: Logger;
constructor(
fn: (input: T, additionalArg?: AdditionalToolArgument) => R,
metadata: ToolMetadata<JSONSchemaType<T>>,
zodType?: z.ZodType<T>,
additionalArg?: AdditionalToolArgument,
logger?: Logger,
) {
this.#fn = fn;
this.#metadata = metadata;
@@ -26,6 +29,7 @@ export class FunctionTool<
this.#zodType = zodType;
}
this.#additionalArg = additionalArg;
this.#logger = logger ?? consoleLogger;
}
static from<T, AdditionalToolArgument extends object = object>(
@@ -140,7 +144,7 @@ export class FunctionTool<
if (result.success) {
params = result.data;
} else {
console.warn(result.error.errors);
this.#logger.warn(result.error.errors);
}
}
return this.#fn.call(null, params, this.#additionalArg);
+12
View File
@@ -1,5 +1,17 @@
# @llamaindex/experimental
## 0.0.201
### Patch Changes
- llamaindex@0.11.24
## 0.0.200
### Patch Changes
- llamaindex@0.11.23
## 0.0.199
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/experimental",
"description": "Experimental package for LlamaIndexTS",
"version": "0.0.199",
"version": "0.0.201",
"type": "module",
"types": "dist/type/index.d.ts",
"main": "dist/cjs/index.js",
+22
View File
@@ -1,5 +1,27 @@
# llamaindex
## 0.11.24
### Patch Changes
- Updated dependencies [c3bf3c7]
- Updated dependencies [f9f1de9]
- @llamaindex/cloud@4.0.28
- @llamaindex/core@0.6.19
- @llamaindex/node-parser@2.0.19
- @llamaindex/workflow@1.1.20
## 0.11.23
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/workflow@1.1.19
- @llamaindex/core@0.6.18
- @llamaindex/cloud@4.0.27
- @llamaindex/node-parser@2.0.18
## 0.11.22
### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "llamaindex",
"version": "0.11.22",
"version": "0.11.24",
"license": "MIT",
"type": "module",
"keywords": [
@@ -8,7 +8,7 @@ import {
BaseInMemoryKVStore,
SimpleKVStore,
} from "@llamaindex/core/storage/kv-store";
import { path } from "@llamaindex/env";
import { path, type Logger } from "@llamaindex/env";
import _ from "lodash";
// eslint-disable-next-line @typescript-eslint/no-explicit-any
@@ -27,19 +27,28 @@ export class SimpleDocumentStore extends KVDocumentStore {
static async fromPersistDir(
persistDir: string = DEFAULT_PERSIST_DIR,
namespace?: string,
options?: { logger?: Logger },
): Promise<SimpleDocumentStore> {
const persistPath = path.join(
persistDir,
DEFAULT_DOC_STORE_PERSIST_FILENAME,
);
return await SimpleDocumentStore.fromPersistPath(persistPath, namespace);
return await SimpleDocumentStore.fromPersistPath(
persistPath,
namespace,
options,
);
}
static async fromPersistPath(
persistPath: string,
namespace?: string,
options?: { logger?: Logger },
): Promise<SimpleDocumentStore> {
const simpleKVStore = await SimpleKVStore.fromPersistPath(persistPath);
const simpleKVStore = await SimpleKVStore.fromPersistPath(
persistPath,
options,
);
return new SimpleDocumentStore(simpleKVStore, namespace);
}
@@ -18,7 +18,7 @@ import {
type VectorStoreQuery,
type VectorStoreQueryResult,
} from "@llamaindex/core/vector-store";
import { fs, path } from "@llamaindex/env";
import { consoleLogger, fs, path, type Logger } from "@llamaindex/env";
import { exists } from "../storage/FileSystem.js";
const LEARNER_MODES = new Set<VectorStoreQueryMode>([
@@ -139,9 +139,14 @@ export class SimpleVectorStore extends BaseVectorStore {
static async fromPersistDir(
persistDir: string = DEFAULT_PERSIST_DIR,
embedModel?: BaseEmbedding,
options?: { logger?: Logger },
): Promise<SimpleVectorStore> {
const persistPath = path.join(persistDir, "vector_store.json");
return await SimpleVectorStore.fromPersistPath(persistPath, embedModel);
return await SimpleVectorStore.fromPersistPath(
persistPath,
embedModel,
options,
);
}
client() {
@@ -273,7 +278,9 @@ export class SimpleVectorStore extends BaseVectorStore {
static async fromPersistPath(
persistPath: string,
embedModel?: BaseEmbedding,
options?: { logger?: Logger },
): Promise<SimpleVectorStore> {
const logger = options?.logger ?? consoleLogger;
const dirPath = path.dirname(persistPath);
if (!(await exists(dirPath))) {
await fs.mkdir(dirPath, { recursive: true });
@@ -281,7 +288,7 @@ export class SimpleVectorStore extends BaseVectorStore {
let dataDict: Record<string, unknown> = {};
if (!(await exists(persistPath))) {
console.info(`Starting new store from path: ${persistPath}`);
logger.log(`Starting new store from path: ${persistPath}`);
} else {
try {
const fileData = await fs.readFile(persistPath);
+12
View File
@@ -1,5 +1,17 @@
# @llamaindex/core-test
## 0.1.15
### Patch Changes
- @llamaindex/openai@0.4.14
## 0.1.14
### Patch Changes
- @llamaindex/openai@0.4.13
## 0.1.13
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/llamaindex-test",
"private": true,
"version": "0.1.13",
"version": "0.1.15",
"type": "module",
"scripts": {
"test": "vitest run"
@@ -47,22 +47,31 @@ describe("StorageContext", () => {
test("persists and loads", async () => {
const doc = new Document({ text: "test document" });
const consoleInfoSpy = vi
.spyOn(console, "info")
.mockImplementation(() => {});
// Create a Logger that spies on log (info) calls
const spyLogger = {
log: vi.fn(),
error: vi.fn(),
warn: vi.fn(),
};
// storage context from individual stores
const storageContext = await storageContextFromDefaults({
docStore: await SimpleDocumentStore.fromPersistDir(testDir),
vectorStore: await SimpleVectorStore.fromPersistDir(testDir),
indexStore: await SimpleIndexStore.fromPersistDir(testDir),
docStore: await SimpleDocumentStore.fromPersistDir(testDir, undefined, {
logger: spyLogger,
}),
vectorStore: await SimpleVectorStore.fromPersistDir(testDir, undefined, {
logger: spyLogger,
}),
indexStore: await SimpleIndexStore.fromPersistDir(testDir, {
logger: spyLogger,
}),
});
const index = await VectorStoreIndex.fromDocuments([doc], {
storageContext,
});
expect(consoleInfoSpy).toHaveBeenCalledTimes(3);
expect(consoleInfoSpy).toHaveBeenCalledWith(
expect(spyLogger.log).toHaveBeenCalledTimes(3);
expect(spyLogger.log).toHaveBeenCalledWith(
expect.stringContaining("Starting new store"),
);
expect(index).toBeDefined();
@@ -75,13 +84,19 @@ describe("StorageContext", () => {
// Check that the test data files exist
await expectTestDataFilesExist(testDir);
consoleInfoSpy.mockClear();
spyLogger.log.mockClear();
// Now, load it again. Since data was persisted, we should not see the error.
const newStorageContext = await storageContextFromDefaults({
docStore: await SimpleDocumentStore.fromPersistDir(testDir),
vectorStore: await SimpleVectorStore.fromPersistDir(testDir),
indexStore: await SimpleIndexStore.fromPersistDir(testDir),
docStore: await SimpleDocumentStore.fromPersistDir(testDir, undefined, {
logger: spyLogger,
}),
vectorStore: await SimpleVectorStore.fromPersistDir(testDir, undefined, {
logger: spyLogger,
}),
indexStore: await SimpleIndexStore.fromPersistDir(testDir, {
logger: spyLogger,
}),
});
const loadedIndex = await VectorStoreIndex.init({
@@ -94,9 +109,7 @@ describe("StorageContext", () => {
await expectTestDataFilesExist(testDir);
expect(consoleInfoSpy).not.toHaveBeenCalled();
consoleInfoSpy.mockRestore();
expect(spyLogger.log).not.toHaveBeenCalled();
});
test("throws error on corrupted data", async () => {
+15
View File
@@ -1,5 +1,20 @@
# @llamaindex/node-parser
## 2.0.19
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 2.0.18
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 2.0.17
### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@llamaindex/node-parser",
"version": "2.0.17",
"version": "2.0.19",
"description": "Node parser for LlamaIndex",
"type": "module",
"exports": {
+15
View File
@@ -1,5 +1,20 @@
# @llamaindex/anthropic
## 0.3.21
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.3.20
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.3.19
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/anthropic",
"description": "Anthropic Adapter for LlamaIndex",
"version": "0.3.19",
"version": "0.3.21",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
@@ -1,5 +1,20 @@
# @llamaindex/assemblyai
## 0.1.18
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.1.17
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.1.16
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/assemblyai",
"description": "AssemblyAI Reader for LlamaIndex",
"version": "0.1.16",
"version": "0.1.18",
"type": "module",
"types": "dist/index.d.ts",
"main": "dist/index.cjs",
+15
View File
@@ -1,5 +1,20 @@
# @llamaindex/community
## 0.0.115
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.114
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.0.113
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/aws",
"description": "AWS package for LlamaIndexTS",
"version": "0.0.113",
"version": "0.0.115",
"type": "module",
"types": "dist/type/index.d.ts",
"main": "dist/cjs/index.js",
+17
View File
@@ -1,5 +1,22 @@
# @llamaindex/clip
## 0.0.70
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
- @llamaindex/openai@0.4.14
## 0.0.69
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
- @llamaindex/openai@0.4.13
## 0.0.68
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/clip",
"description": "Clip Embedding Adapter for LlamaIndex",
"version": "0.0.68",
"version": "0.0.70",
"type": "module",
"types": "dist/index.d.ts",
"main": "dist/index.cjs",
+15
View File
@@ -1,5 +1,20 @@
# @llamaindex/cohere
## 0.0.33
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.32
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.0.31
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/cohere",
"description": "Cohere Adapter for LlamaIndex",
"version": "0.0.31",
"version": "0.0.33",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
+17
View File
@@ -1,5 +1,22 @@
# @llamaindex/deepinfra
## 0.0.70
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
- @llamaindex/openai@0.4.14
## 0.0.69
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
- @llamaindex/openai@0.4.13
## 0.0.68
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/deepinfra",
"description": "Deepinfra Adapter for LlamaIndex",
"version": "0.0.68",
"version": "0.0.70",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
+12
View File
@@ -1,5 +1,17 @@
# @llamaindex/deepseek
## 0.0.31
### Patch Changes
- @llamaindex/openai@0.4.14
## 0.0.30
### Patch Changes
- @llamaindex/openai@0.4.13
## 0.0.29
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/deepseek",
"description": "DeepSeek Adapter for LlamaIndex",
"version": "0.0.29",
"version": "0.0.31",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
+15
View File
@@ -1,5 +1,20 @@
# @llamaindex/discord
## 0.1.18
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.1.17
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.1.16
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/discord",
"description": "Discord Reader for LlamaIndex",
"version": "0.1.16",
"version": "0.1.18",
"type": "module",
"types": "dist/index.d.ts",
"main": "dist/index.cjs",
+15
View File
@@ -1,5 +1,20 @@
# @llamaindex/excel
## 0.1.19
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.1.18
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.1.17
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/excel",
"description": "Excel Reader for LlamaIndex",
"version": "0.1.17",
"version": "0.1.19",
"type": "module",
"types": "dist/index.d.ts",
"main": "dist/index.cjs",
+12
View File
@@ -1,5 +1,17 @@
# @llamaindex/fireworks
## 0.0.30
### Patch Changes
- @llamaindex/openai@0.4.14
## 0.0.29
### Patch Changes
- @llamaindex/openai@0.4.13
## 0.0.28
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/fireworks",
"description": "Fireworks Adapter for LlamaIndex",
"version": "0.0.28",
"version": "0.0.30",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
+15
View File
@@ -1,5 +1,20 @@
# @llamaindex/google
## 0.3.18
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.3.17
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.3.16
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/google",
"description": "Google Adapter for LlamaIndex",
"version": "0.3.16",
"version": "0.3.18",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
+12
View File
@@ -1,5 +1,17 @@
# @llamaindex/groq
## 0.0.86
### Patch Changes
- @llamaindex/openai@0.4.14
## 0.0.85
### Patch Changes
- @llamaindex/openai@0.4.13
## 0.0.84
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/groq",
"description": "Groq Adapter for LlamaIndex",
"version": "0.0.84",
"version": "0.0.86",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
@@ -1,5 +1,22 @@
# @llamaindex/huggingface
## 0.1.24
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
- @llamaindex/openai@0.4.14
## 0.1.23
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
- @llamaindex/openai@0.4.13
## 0.1.22
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/huggingface",
"description": "Huggingface Adapter for LlamaIndex",
"version": "0.1.22",
"version": "0.1.24",
"type": "module",
"types": "dist/index.d.ts",
"main": "dist/index.cjs",
+17
View File
@@ -1,5 +1,22 @@
# @llamaindex/jinaai
## 0.0.30
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
- @llamaindex/openai@0.4.14
## 0.0.29
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
- @llamaindex/openai@0.4.13
## 0.0.28
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/jinaai",
"description": "JinaAI Adapter for LlamaIndex",
"version": "0.0.28",
"version": "0.0.30",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
+15
View File
@@ -1,5 +1,20 @@
# @llamaindex/mistral
## 0.1.19
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.1.18
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.1.17
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/mistral",
"description": "Mistral Adapter for LlamaIndex",
"version": "0.1.17",
"version": "0.1.19",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
@@ -1,5 +1,20 @@
# @llamaindex/mixedbread
## 0.0.33
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.32
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.0.31
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/mixedbread",
"description": "Mixedbread Adapter for LlamaIndex",
"version": "0.0.31",
"version": "0.0.33",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
+15
View File
@@ -1,5 +1,20 @@
# @llamaindex/notion
## 0.1.18
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.1.17
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.1.16
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/notion",
"description": "Notion Reader for LlamaIndex",
"version": "0.1.16",
"version": "0.1.18",
"type": "module",
"types": "dist/index.d.ts",
"main": "dist/index.cjs",
+15
View File
@@ -1,5 +1,20 @@
# @llamaindex/ollama
## 0.1.19
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.1.18
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.1.17
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/ollama",
"description": "Ollama Adapter for LlamaIndex",
"version": "0.1.17",
"version": "0.1.19",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
+15
View File
@@ -1,5 +1,20 @@
# @llamaindex/openai
## 0.4.14
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.4.13
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.4.12
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/openai",
"description": "OpenAI Adapter for LlamaIndex",
"version": "0.4.12",
"version": "0.4.14",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
@@ -1,5 +1,22 @@
# @llamaindex/perplexity
## 0.0.27
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
- @llamaindex/openai@0.4.14
## 0.0.26
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
- @llamaindex/openai@0.4.13
## 0.0.25
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/perplexity",
"description": "Perplexity Adapter for LlamaIndex",
"version": "0.0.25",
"version": "0.0.27",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
@@ -1,5 +1,20 @@
# @llamaindex/portkey-ai
## 0.0.61
### Patch Changes
- Updated dependencies [f9f1de9]
- @llamaindex/core@0.6.19
## 0.0.60
### Patch Changes
- Updated dependencies [f29799e]
- Updated dependencies [7224c06]
- @llamaindex/core@0.6.18
## 0.0.59
### Patch Changes
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@llamaindex/portkey-ai",
"description": "Portkey Adapter for LlamaIndex",
"version": "0.0.59",
"version": "0.0.61",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",

Some files were not shown because too many files have changed in this diff Show More