mirror of
https://github.com/run-llama/LlamaIndexTS.git
synced 2026-07-15 06:52:45 -04:00
Compare commits
7 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 2234d8ff6d | |||
| ca2405811b | |||
| 9b2e25a184 | |||
| b29521bf6c | |||
| 73e25787e7 | |||
| 3ce80540fe | |||
| dbc1ee3089 |
@@ -0,0 +1,10 @@
|
||||
---
|
||||
"@llamaindex/anthropic": patch
|
||||
"@llamaindex/google": patch
|
||||
"@llamaindex/openai": patch
|
||||
"@llamaindex/core": patch
|
||||
"@llamaindex/env": patch
|
||||
"@llamaindex/examples": patch
|
||||
---
|
||||
|
||||
Use Uint8Array instead of Buffer for file type messages (works with non-NodeJS)
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
"llamaindex": patch
|
||||
---
|
||||
|
||||
Fix using new workflow with npm by forcing to use the current version of core in llamaindex (and not the one from the old workflow)
|
||||
@@ -9,10 +9,10 @@ To install llamaindex, run the following command:
|
||||
npm i llamaindex
|
||||
```
|
||||
|
||||
In most cases, you'll also need an LLM package to use LlamaIndex. For example, to use the OpenAI LLM, you would install the following:
|
||||
In most cases, you'll also need an LLM package and the Workflow package to use LlamaIndex. For example, to use the OpenAI LLM with agents, you would install the following:
|
||||
|
||||
```package-install
|
||||
npm i @llamaindex/openai
|
||||
npm i @llamaindex/openai @llamaindex/workflow
|
||||
```
|
||||
|
||||
Go to [LLM APIs](/docs/llamaindex/modules/models/llms) to find out how to use other LLMs.
|
||||
|
||||
@@ -6,171 +6,13 @@ A `Workflow` in LlamaIndex is a lightweight, event-driven abstraction used to ch
|
||||
|
||||
Workflows are designed to be flexible and can be used to build agents, RAG flows, extraction flows, or anything else you want to implement.
|
||||
|
||||
To use workflows install this package:
|
||||
|
||||
```package-install
|
||||
npm i @llamaindex/workflow @llamaindex/openai
|
||||
npm i @llamaindex/workflow
|
||||
```
|
||||
|
||||
## Getting Started
|
||||
This package is a stable, production-ready version of our [llama-flow](../../../llamaflow) project.
|
||||
|
||||
Let's explore a simple workflow example where a joke is generated and then critiqued and iterated on:
|
||||
While you can still reference the llama-flow documentation for detailed information about the underlying concepts, we recommend using the `@llamaindex/workflow` package for all new projects to ensure stability and long-term availability.
|
||||
|
||||
<include cwd>../../examples/agents/workflow/joke.ts</include>
|
||||
|
||||
There are a few moving pieces here, so let's go through this step by step.
|
||||
|
||||
### Defining Workflow Events
|
||||
|
||||
```typescript
|
||||
const startEvent = workflowEvent<string>(); // Input topic for joke
|
||||
const jokeEvent = workflowEvent<{ joke: string }>(); // Intermediate joke
|
||||
const critiqueEvent = workflowEvent<{ joke: string; critique: string }>(); // Intermediate critique
|
||||
const resultEvent = workflowEvent<{ joke: string; critique: string }>(); // Final joke + critique
|
||||
```
|
||||
|
||||
Events are defined using the `workflowEvent` function and contain arbitrary data provided as a generic type. In this example, we have four events:
|
||||
- `startEvent`: Takes a string input (the joke topic)
|
||||
- `jokeEvent`: Contains an object with a joke property
|
||||
- `critiqueEvent`: Contains both the joke and its critique, used for the feedback loop
|
||||
- `resultEvent`: Contains the final joke and critique after any iterations
|
||||
|
||||
### Setting up the Workflow with Stateful Middleware
|
||||
|
||||
```typescript
|
||||
const { withState, getContext } = createStatefulMiddleware(() => ({
|
||||
numIterations: 0,
|
||||
maxIterations: 3,
|
||||
}));
|
||||
const jokeFlow = withState(createWorkflow());
|
||||
```
|
||||
|
||||
Our workflow is implemented using the `createWorkflow()` function, enhanced with the `withState` middleware. This middleware provides shared state across all handlers, which in this case tracks:
|
||||
- `numIterations`: Counts how many iterations of joke improvement we've done
|
||||
- `maxIterations`: Sets a limit to prevent infinite loops
|
||||
|
||||
This state will be accessible within workflows by using the `getContext().state` function.
|
||||
|
||||
### Adding Handlers with Loops
|
||||
|
||||
We have three key handlers in our workflow:
|
||||
|
||||
1. The first handler processes the `startEvent`, generates an initial joke, and emits a `jokeEvent`:
|
||||
|
||||
```typescript
|
||||
jokeFlow.handle([startEvent], async (event) => {
|
||||
// Prompt the LLM to write a joke
|
||||
const prompt = `Write your best joke about ${event.data}. Write the joke between <joke> and </joke> tags.`;
|
||||
const response = await llm.complete({ prompt });
|
||||
|
||||
// Parse the joke from the response
|
||||
const joke =
|
||||
response.text.match(/<joke>([\s\S]*?)<\/joke>/)?.[1]?.trim() ??
|
||||
response.text;
|
||||
return jokeEvent.with({ joke: joke });
|
||||
});
|
||||
```
|
||||
|
||||
2. The second handler handles the `jokeEvent`, critiques the joke, and either:
|
||||
- Emits a `critiqueEvent` if the joke needs improvement
|
||||
- Emits a `resultEvent` if the joke is good enough
|
||||
|
||||
```typescript
|
||||
jokeFlow.handle([jokeEvent], async (event) => {
|
||||
// Prompt the LLM to critique the joke
|
||||
const prompt = `Give a thorough critique of the following joke. If the joke needs improvement, put "IMPROVE" somewhere in the critique: ${event.data.joke}`;
|
||||
const response = await llm.complete({ prompt });
|
||||
|
||||
// If the critique includes "IMPROVE", keep iterating, else, return the result
|
||||
if (response.text.includes("IMPROVE")) {
|
||||
return critiqueEvent.with({
|
||||
joke: event.data.joke,
|
||||
critique: response.text,
|
||||
});
|
||||
}
|
||||
|
||||
return resultEvent.with({ joke: event.data.joke, critique: response.text });
|
||||
});
|
||||
```
|
||||
|
||||
3. The third handler processes the `critiqueEvent`, generates an improved joke based on the critique, and either:
|
||||
- Loops back to the joke evaluation (if under the iteration limit)
|
||||
- Emits the final `resultEvent` (if iteration limit reached)
|
||||
|
||||
```typescript
|
||||
jokeFlow.handle([critiqueEvent], async (event) => {
|
||||
// Keep track of the number of iterations
|
||||
const state = getContext().state;
|
||||
state.numIterations++;
|
||||
|
||||
// Write a new joke based on the previous joke and critique
|
||||
const prompt = `Write a new joke based on the following critique and the original joke. Write the joke between <joke> and </joke> tags.\n\nJoke: ${event.data.joke}\n\nCritique: ${event.data.critique}`;
|
||||
const response = await llm.complete({ prompt });
|
||||
|
||||
// Parse the joke from the response
|
||||
const joke =
|
||||
response.text.match(/<joke>([\s\S]*?)<\/joke>/)?.[1]?.trim() ??
|
||||
response.text;
|
||||
|
||||
// If we've done less than the max number of iterations, keep iterating
|
||||
// else, return the result
|
||||
if (state.numIterations < state.maxIterations) {
|
||||
return jokeEvent.with({ joke: joke });
|
||||
}
|
||||
|
||||
return resultEvent.with({ joke: joke, critique: event.data.critique });
|
||||
});
|
||||
```
|
||||
|
||||
### Running the Workflow
|
||||
|
||||
```typescript
|
||||
async function main() {
|
||||
const { stream, sendEvent } = jokeFlow.createContext();
|
||||
sendEvent(startEvent.with("pirates"));
|
||||
|
||||
let result: { joke: string, critique: string } | undefined;
|
||||
|
||||
for await (const event of stream) {
|
||||
// console.log(event.data); optionally log the event data
|
||||
if (resultEvent.include(event)) {
|
||||
result = event.data;
|
||||
break; // Stop when we get the final result
|
||||
}
|
||||
}
|
||||
|
||||
console.log(result);
|
||||
}
|
||||
```
|
||||
|
||||
To run the workflow, we:
|
||||
1. Create a workflow context with `createContext()`
|
||||
2. Trigger the initial event with `sendEvent()`
|
||||
3. Listen to the event stream and process events as they arrive
|
||||
4. Use `include()` to check if an event is of a specific type
|
||||
5. Break the loop when we receive our final result
|
||||
|
||||
### Using Stream Utilities
|
||||
|
||||
The `stream` returned by `createContext` contains utility functions to make working with event streams easier:
|
||||
|
||||
```typescript
|
||||
// Create a workflow context and send the initial event
|
||||
const { stream, sendEvent } = jokeFlow.createContext();
|
||||
sendEvent(startEvent.with("pirates"));
|
||||
|
||||
// Collect all events until we get a resultEvent
|
||||
const allEvents = await stream.until(resultEvent).toArray();
|
||||
|
||||
// The last event will be the resultEvent
|
||||
const finalEvent = allEvents.at(-1);
|
||||
console.log(finalEvent.data); // Output the joke and critique
|
||||
```
|
||||
|
||||
The stream utilities make it easier to work with the asynchronous event flow. In this example, we use:
|
||||
- `toArray`: Aggregates all events into an array
|
||||
- `until`: Creates a stream that emits events until a condition is met (in this case, until a resultEvent is received)
|
||||
|
||||
You can combine these utilities with other stream operators like `filter` and `map` to create powerful processing pipelines.
|
||||
|
||||
## Next Steps
|
||||
|
||||
To learn more about workflows, check out [the documentation in the tutorial section](../../../llamaflow).
|
||||
|
||||
@@ -8,9 +8,10 @@ We have a comprehensive, step-by-step [guide to building agents in LlamaIndex.TS
|
||||
|
||||
In a new folder:
|
||||
|
||||
```bash npm2yarn
|
||||
```package-install
|
||||
npm init
|
||||
npm i -D typescript @types/node
|
||||
npm i @llamaindex/openai @llamaindex/workflow llamaindex zod
|
||||
```
|
||||
|
||||
## Run agent
|
||||
@@ -20,15 +21,14 @@ Create the file `example.ts`. This code will:
|
||||
- Create two tools for use by the agent:
|
||||
- A `sumNumbers` tool that adds two numbers
|
||||
- A `divideNumbers` tool that divides numbers
|
||||
-
|
||||
- Give an example of the data structure we wish to generate
|
||||
- Prompt the LLM with instructions and the example, plus a sample transcript
|
||||
|
||||
<include cwd>../../examples/agent/openai.ts</include>
|
||||
<include cwd>../../examples/agents/agent/openai.ts</include>
|
||||
|
||||
To run the code:
|
||||
|
||||
```bash
|
||||
```package-install
|
||||
npx tsx example.ts
|
||||
```
|
||||
|
||||
@@ -36,9 +36,18 @@ You should expect output something like:
|
||||
|
||||
```
|
||||
{
|
||||
content: 'The sum of 5 + 5 is 10. When you divide 10 by 2, you get 5.',
|
||||
role: 'assistant',
|
||||
options: {}
|
||||
result: '5 + 5 is 10. Then, 10 divided by 2 is 5.',
|
||||
state: {
|
||||
memory: ChatMemoryBuffer {
|
||||
chatStore: SimpleChatStore {},
|
||||
chatStoreKey: 'chat_history',
|
||||
tokenLimit: 750000
|
||||
},
|
||||
scratchpad: [],
|
||||
currentAgentName: 'Agent',
|
||||
agents: [ 'Agent' ],
|
||||
nextAgentName: null
|
||||
}
|
||||
}
|
||||
Done
|
||||
```
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
"basic_agent",
|
||||
"rag",
|
||||
"agents",
|
||||
"../../llamaflow",
|
||||
"workflows",
|
||||
"local_llm",
|
||||
"chatbot",
|
||||
"structured_data_extraction"
|
||||
|
||||
@@ -16,7 +16,7 @@ LlamaIndex uses a two stage method when using an LLM with your data:
|
||||
1. **indexing stage**: preparing a knowledge base, and
|
||||
2. **querying stage**: retrieving relevant context from the knowledge to assist the LLM in responding to a question
|
||||
|
||||

|
||||

|
||||
|
||||
This process is also known as Retrieval Augmented Generation (RAG).
|
||||
|
||||
@@ -28,7 +28,7 @@ Let's explore each stage in detail.
|
||||
|
||||
LlamaIndex.TS help you prepare the knowledge base with a suite of data connectors and indexes.
|
||||
|
||||

|
||||

|
||||
|
||||
[**Data Loaders**](/docs/llamaindex/modules/data/readers):
|
||||
A data connector (i.e. `Reader`) ingest data from different data sources and data formats into a simple `Document` representation (text and simple metadata).
|
||||
@@ -54,7 +54,7 @@ LlamaIndex provides composable modules that help you build and integrate RAG pip
|
||||
|
||||
These building blocks can be customized to reflect ranking preferences, as well as composed to reason over multiple knowledge bases in a structured way.
|
||||
|
||||

|
||||

|
||||
|
||||
#### Building Blocks
|
||||
|
||||
|
||||
@@ -8,9 +8,10 @@ One of the most common use-cases for LlamaIndex is Retrieval-Augmented Generatio
|
||||
|
||||
In a new folder, run:
|
||||
|
||||
```bash npm2yarn
|
||||
```package-install
|
||||
npm init
|
||||
npm i -D typescript @types/node
|
||||
npm i llamaindex
|
||||
```
|
||||
|
||||
Then, check out the [installation](/docs/llamaindex/getting_started/installation) steps to install LlamaIndex.TS and prepare an OpenAI key.
|
||||
@@ -34,7 +35,7 @@ Create a `tsconfig.json` file in the same folder:
|
||||
|
||||
Now you can run the code with
|
||||
|
||||
```bash
|
||||
```package-install
|
||||
npx tsx example.ts
|
||||
```
|
||||
|
||||
|
||||
@@ -10,9 +10,10 @@ You can use [other LLMs](/docs/llamaindex/modules/models/llms) via their APIs; i
|
||||
|
||||
In a new folder:
|
||||
|
||||
```bash npm2yarn
|
||||
```package-install
|
||||
npm init
|
||||
npm i -D typescript @types/node
|
||||
npm i @llamaindex/openai zod
|
||||
```
|
||||
|
||||
## Extract data
|
||||
@@ -27,7 +28,7 @@ Create the file `example.ts`. This code will:
|
||||
|
||||
To run the code:
|
||||
|
||||
```bash
|
||||
```package-install
|
||||
npx tsx example.ts
|
||||
```
|
||||
|
||||
|
||||
@@ -0,0 +1,176 @@
|
||||
---
|
||||
title: Workflows
|
||||
---
|
||||
|
||||
A `Workflow` in LlamaIndex is a lightweight, event-driven abstraction used to chain together several events. Workflows are made up of `handlers`, with each one responsible for processing specific event types and emitting new events.
|
||||
|
||||
Workflows are designed to be flexible and can be used to build agents, RAG flows, extraction flows, or anything else you want to implement.
|
||||
|
||||
```package-install
|
||||
npm i @llamaindex/workflow @llamaindex/openai
|
||||
```
|
||||
|
||||
## Getting Started
|
||||
|
||||
Let's explore a simple workflow example where a joke is generated and then critiqued and iterated on:
|
||||
|
||||
<include cwd>../../examples/agents/workflow/joke.ts</include>
|
||||
|
||||
There are a few moving pieces here, so let's go through this step by step.
|
||||
|
||||
### Defining Workflow Events
|
||||
|
||||
```typescript
|
||||
const startEvent = workflowEvent<string>(); // Input topic for joke
|
||||
const jokeEvent = workflowEvent<{ joke: string }>(); // Intermediate joke
|
||||
const critiqueEvent = workflowEvent<{ joke: string; critique: string }>(); // Intermediate critique
|
||||
const resultEvent = workflowEvent<{ joke: string; critique: string }>(); // Final joke + critique
|
||||
```
|
||||
|
||||
Events are defined using the `workflowEvent` function and contain arbitrary data provided as a generic type. In this example, we have four events:
|
||||
- `startEvent`: Takes a string input (the joke topic)
|
||||
- `jokeEvent`: Contains an object with a joke property
|
||||
- `critiqueEvent`: Contains both the joke and its critique, used for the feedback loop
|
||||
- `resultEvent`: Contains the final joke and critique after any iterations
|
||||
|
||||
### Setting up the Workflow with Stateful Middleware
|
||||
|
||||
```typescript
|
||||
const { withState, getContext } = createStatefulMiddleware(() => ({
|
||||
numIterations: 0,
|
||||
maxIterations: 3,
|
||||
}));
|
||||
const jokeFlow = withState(createWorkflow());
|
||||
```
|
||||
|
||||
Our workflow is implemented using the `createWorkflow()` function, enhanced with the `withState` middleware. This middleware provides shared state across all handlers, which in this case tracks:
|
||||
- `numIterations`: Counts how many iterations of joke improvement we've done
|
||||
- `maxIterations`: Sets a limit to prevent infinite loops
|
||||
|
||||
This state will be accessible within workflows by using the `getContext().state` function.
|
||||
|
||||
### Adding Handlers with Loops
|
||||
|
||||
We have three key handlers in our workflow:
|
||||
|
||||
1. The first handler processes the `startEvent`, generates an initial joke, and emits a `jokeEvent`:
|
||||
|
||||
```typescript
|
||||
jokeFlow.handle([startEvent], async (event) => {
|
||||
// Prompt the LLM to write a joke
|
||||
const prompt = `Write your best joke about ${event.data}. Write the joke between <joke> and </joke> tags.`;
|
||||
const response = await llm.complete({ prompt });
|
||||
|
||||
// Parse the joke from the response
|
||||
const joke =
|
||||
response.text.match(/<joke>([\s\S]*?)<\/joke>/)?.[1]?.trim() ??
|
||||
response.text;
|
||||
return jokeEvent.with({ joke: joke });
|
||||
});
|
||||
```
|
||||
|
||||
2. The second handler handles the `jokeEvent`, critiques the joke, and either:
|
||||
- Emits a `critiqueEvent` if the joke needs improvement
|
||||
- Emits a `resultEvent` if the joke is good enough
|
||||
|
||||
```typescript
|
||||
jokeFlow.handle([jokeEvent], async (event) => {
|
||||
// Prompt the LLM to critique the joke
|
||||
const prompt = `Give a thorough critique of the following joke. If the joke needs improvement, put "IMPROVE" somewhere in the critique: ${event.data.joke}`;
|
||||
const response = await llm.complete({ prompt });
|
||||
|
||||
// If the critique includes "IMPROVE", keep iterating, else, return the result
|
||||
if (response.text.includes("IMPROVE")) {
|
||||
return critiqueEvent.with({
|
||||
joke: event.data.joke,
|
||||
critique: response.text,
|
||||
});
|
||||
}
|
||||
|
||||
return resultEvent.with({ joke: event.data.joke, critique: response.text });
|
||||
});
|
||||
```
|
||||
|
||||
3. The third handler processes the `critiqueEvent`, generates an improved joke based on the critique, and either:
|
||||
- Loops back to the joke evaluation (if under the iteration limit)
|
||||
- Emits the final `resultEvent` (if iteration limit reached)
|
||||
|
||||
```typescript
|
||||
jokeFlow.handle([critiqueEvent], async (event) => {
|
||||
// Keep track of the number of iterations
|
||||
const state = getContext().state;
|
||||
state.numIterations++;
|
||||
|
||||
// Write a new joke based on the previous joke and critique
|
||||
const prompt = `Write a new joke based on the following critique and the original joke. Write the joke between <joke> and </joke> tags.\n\nJoke: ${event.data.joke}\n\nCritique: ${event.data.critique}`;
|
||||
const response = await llm.complete({ prompt });
|
||||
|
||||
// Parse the joke from the response
|
||||
const joke =
|
||||
response.text.match(/<joke>([\s\S]*?)<\/joke>/)?.[1]?.trim() ??
|
||||
response.text;
|
||||
|
||||
// If we've done less than the max number of iterations, keep iterating
|
||||
// else, return the result
|
||||
if (state.numIterations < state.maxIterations) {
|
||||
return jokeEvent.with({ joke: joke });
|
||||
}
|
||||
|
||||
return resultEvent.with({ joke: joke, critique: event.data.critique });
|
||||
});
|
||||
```
|
||||
|
||||
### Running the Workflow
|
||||
|
||||
```typescript
|
||||
async function main() {
|
||||
const { stream, sendEvent } = jokeFlow.createContext();
|
||||
sendEvent(startEvent.with("pirates"));
|
||||
|
||||
let result: { joke: string, critique: string } | undefined;
|
||||
|
||||
for await (const event of stream) {
|
||||
// console.log(event.data); optionally log the event data
|
||||
if (resultEvent.include(event)) {
|
||||
result = event.data;
|
||||
break; // Stop when we get the final result
|
||||
}
|
||||
}
|
||||
|
||||
console.log(result);
|
||||
}
|
||||
```
|
||||
|
||||
To run the workflow, we:
|
||||
1. Create a workflow context with `createContext()`
|
||||
2. Trigger the initial event with `sendEvent()`
|
||||
3. Listen to the event stream and process events as they arrive
|
||||
4. Use `include()` to check if an event is of a specific type
|
||||
5. Break the loop when we receive our final result
|
||||
|
||||
### Using Stream Utilities
|
||||
|
||||
The `stream` returned by `createContext` contains utility functions to make working with event streams easier:
|
||||
|
||||
```typescript
|
||||
// Create a workflow context and send the initial event
|
||||
const { stream, sendEvent } = jokeFlow.createContext();
|
||||
sendEvent(startEvent.with("pirates"));
|
||||
|
||||
// Collect all events until we get a resultEvent
|
||||
const allEvents = await stream.until(resultEvent).toArray();
|
||||
|
||||
// The last event will be the resultEvent
|
||||
const finalEvent = allEvents.at(-1);
|
||||
console.log(finalEvent.data); // Output the joke and critique
|
||||
```
|
||||
|
||||
The stream utilities make it easier to work with the asynchronous event flow. In this example, we use:
|
||||
- `toArray`: Aggregates all events into an array
|
||||
- `until`: Creates a stream that emits events until a condition is met (in this case, until a resultEvent is received)
|
||||
|
||||
You can combine these utilities with other stream operators like `filter` and `map` to create powerful processing pipelines.
|
||||
|
||||
## Next Steps
|
||||
|
||||
To learn more about workflows, check out [the Workflows documentation](/docs/llamaindex/modules/agents/workflows).
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
"pages": ["llamaindex", "api"]
|
||||
"pages": ["llamaindex", "api", "llamaflow"]
|
||||
}
|
||||
|
||||
@@ -26,7 +26,7 @@ const divideNumbers = tool({
|
||||
async function main() {
|
||||
const mathAgent = agent({
|
||||
tools: [sumNumbers, divideNumbers],
|
||||
llm: openai({ model: "gpt-4o-mini" }),
|
||||
llm: openai({ model: "gpt-4.1-mini" }),
|
||||
verbose: false,
|
||||
});
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@ async function main() {
|
||||
},
|
||||
{
|
||||
type: "file",
|
||||
data: fs.readFileSync("./data/manga.pdf"),
|
||||
data: Uint8Array.from(fs.readFileSync("./data/manga.pdf")),
|
||||
mimeType: "application/pdf",
|
||||
},
|
||||
],
|
||||
|
||||
@@ -32,7 +32,7 @@ import fs from "fs";
|
||||
},
|
||||
{
|
||||
type: "file",
|
||||
data: fs.readFileSync("./data/manga.pdf"),
|
||||
data: Uint8Array.from(fs.readFileSync("./data/manga.pdf")),
|
||||
mimeType: "application/pdf",
|
||||
},
|
||||
],
|
||||
|
||||
@@ -26,7 +26,7 @@ import fs from "fs";
|
||||
},
|
||||
{
|
||||
type: "file",
|
||||
data: fs.readFileSync("./data/manga.pdf"),
|
||||
data: new Uint8Array(fs.readFileSync("./data/manga.pdf")),
|
||||
mimeType: "application/pdf",
|
||||
},
|
||||
],
|
||||
|
||||
@@ -21,7 +21,7 @@ async function main() {
|
||||
},
|
||||
{
|
||||
type: "file",
|
||||
data: fs.readFileSync("./data/manga.pdf"),
|
||||
data: Uint8Array.from(fs.readFileSync("./data/manga.pdf")),
|
||||
mimeType: "application/pdf",
|
||||
},
|
||||
],
|
||||
|
||||
@@ -165,7 +165,7 @@ export type MessageContentImageDetail = {
|
||||
|
||||
export type MessageContentFileDetail = {
|
||||
type: "file";
|
||||
data: Buffer;
|
||||
data: Uint8Array;
|
||||
mimeType: string;
|
||||
};
|
||||
|
||||
|
||||
Vendored
+2
-1
@@ -6,7 +6,8 @@
|
||||
import "./global-check.js";
|
||||
|
||||
export * from "./als/index.web.js";
|
||||
export { consoleLogger, emptyLogger, type Logger } from "./logger/index.js";
|
||||
export * from "./logger/index.js";
|
||||
export * from "./utils/base64.js";
|
||||
export { NotSupportCurrentRuntimeClass } from "./utils/shared.js";
|
||||
export * from "./web-polyfill.js";
|
||||
if (typeof window === "undefined") {
|
||||
|
||||
Vendored
+2
-1
@@ -5,6 +5,7 @@
|
||||
*/
|
||||
|
||||
export * from "./als/index.non-node.js";
|
||||
export { consoleLogger, emptyLogger, type Logger } from "./logger/index.js";
|
||||
export * from "./logger/index.js";
|
||||
export * from "./node-polyfill.js";
|
||||
export * from "./utils/base64.js";
|
||||
export { NotSupportCurrentRuntimeClass } from "./utils/shared.js";
|
||||
|
||||
Vendored
+2
-1
@@ -37,7 +37,8 @@ export function createSHA256(): SHA256 {
|
||||
export const process = globalThis.process;
|
||||
|
||||
export * from "./als/index.node.js";
|
||||
export { consoleLogger, emptyLogger, type Logger } from "./logger/index.js";
|
||||
export * from "./logger/index.js";
|
||||
export * from "./utils/base64.js";
|
||||
export { CustomEvent, getEnv, setEnvs } from "./utils/index.js";
|
||||
export { NotSupportCurrentRuntimeClass } from "./utils/shared.js";
|
||||
export {
|
||||
|
||||
Vendored
+2
-1
@@ -11,9 +11,10 @@ export * from "./als/index.workerd.js";
|
||||
export { NotSupportCurrentRuntimeClass } from "./utils/shared.js";
|
||||
|
||||
export * from "./node-polyfill.js";
|
||||
export * from "./utils/base64.js";
|
||||
|
||||
export function getEnv(name: string): string | undefined {
|
||||
return INTERNAL_ENV[name];
|
||||
}
|
||||
|
||||
export { consoleLogger, emptyLogger, type Logger } from "./logger/index.js";
|
||||
export * from "./logger/index.js";
|
||||
|
||||
Vendored
+38
@@ -0,0 +1,38 @@
|
||||
/**
|
||||
* Converts a Uint8Array to a base64 string.
|
||||
* For large arrays, it processes the data in chunks to avoid memory issues.
|
||||
* Falls back to Buffer if available for better performance.
|
||||
*
|
||||
* @param bytes - The Uint8Array to convert
|
||||
* @returns The base64 string representation
|
||||
*/
|
||||
export function uint8ArrayToBase64(bytes: Uint8Array): string {
|
||||
// Use Buffer if available (Node.js environment)
|
||||
if (typeof Buffer !== "undefined") {
|
||||
return Buffer.from(bytes).toString("base64");
|
||||
}
|
||||
|
||||
// For browsers and other environments without Buffer
|
||||
// Process in chunks for large arrays to avoid memory issues
|
||||
const CHUNK_SIZE = 32768; // 32KB chunks
|
||||
let result = "";
|
||||
|
||||
// For small arrays, use the built-in btoa function directly
|
||||
if (bytes.length < CHUNK_SIZE) {
|
||||
const binary = Array.from(bytes)
|
||||
.map((byte) => String.fromCharCode(byte))
|
||||
.join("");
|
||||
return globalThis.btoa(binary);
|
||||
}
|
||||
|
||||
// For large arrays, process in chunks
|
||||
for (let i = 0; i < bytes.length; i += CHUNK_SIZE) {
|
||||
const chunk = bytes.subarray(i, i + CHUNK_SIZE);
|
||||
const binary = Array.from(chunk)
|
||||
.map((byte) => String.fromCharCode(byte))
|
||||
.join("");
|
||||
result += globalThis.btoa(binary);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
@@ -21,7 +21,7 @@
|
||||
],
|
||||
"dependencies": {
|
||||
"@llamaindex/cloud": "workspace:*",
|
||||
"@llamaindex/core": "workspace:*",
|
||||
"@llamaindex/core": "0.6.3",
|
||||
"@llamaindex/env": "workspace:*",
|
||||
"@llamaindex/node-parser": "workspace:*",
|
||||
"@llamaindex/openai": "workspace:*",
|
||||
|
||||
@@ -28,7 +28,7 @@ import type {
|
||||
} from "@llamaindex/core/llms";
|
||||
import { ToolCallLLM } from "@llamaindex/core/llms";
|
||||
import { extractText } from "@llamaindex/core/utils";
|
||||
import { getEnv } from "@llamaindex/env";
|
||||
import { getEnv, uint8ArrayToBase64 } from "@llamaindex/env";
|
||||
import { isDeepEqual } from "remeda";
|
||||
|
||||
export class AnthropicSession {
|
||||
@@ -332,7 +332,7 @@ export class Anthropic extends ToolCallLLM<
|
||||
source: {
|
||||
type: "base64" as const,
|
||||
media_type: content.mimeType,
|
||||
data: content.data.toString("base64"),
|
||||
data: uint8ArrayToBase64(content.data),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# @llamaindex/google
|
||||
|
||||
## 0.2.6
|
||||
|
||||
### Patch Changes
|
||||
|
||||
- 73e2578: Add support for gemini-2.5-pro-preview-05-06
|
||||
|
||||
## 0.2.5
|
||||
|
||||
### Patch Changes
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "@llamaindex/google",
|
||||
"description": "Google Adapter for LlamaIndex",
|
||||
"version": "0.2.5",
|
||||
"version": "0.2.6",
|
||||
"type": "module",
|
||||
"main": "./dist/index.cjs",
|
||||
"module": "./dist/index.js",
|
||||
|
||||
@@ -64,6 +64,7 @@ export const GEMINI_MODEL_INFO_MAP: Record<GEMINI_MODEL, GeminiModelInfo> = {
|
||||
[GEMINI_MODEL.GEMINI_2_0_FLASH_THINKING_EXP]: { contextWindow: 32768 },
|
||||
[GEMINI_MODEL.GEMINI_2_0_PRO_EXPERIMENTAL]: { contextWindow: 2 * 10 ** 6 },
|
||||
[GEMINI_MODEL.GEMINI_2_5_PRO_PREVIEW]: { contextWindow: 10 ** 6 },
|
||||
[GEMINI_MODEL.GEMINI_2_5_PRO_PREVIEW_LATEST]: { contextWindow: 10 ** 6 },
|
||||
[GEMINI_MODEL.GEMINI_2_5_FLASH_PREVIEW]: { contextWindow: 10 ** 6 },
|
||||
};
|
||||
|
||||
@@ -82,6 +83,7 @@ export const SUPPORT_TOOL_CALL_MODELS: GEMINI_MODEL[] = [
|
||||
GEMINI_MODEL.GEMINI_2_0_FLASH,
|
||||
GEMINI_MODEL.GEMINI_2_0_PRO_EXPERIMENTAL,
|
||||
GEMINI_MODEL.GEMINI_2_5_PRO_PREVIEW,
|
||||
GEMINI_MODEL.GEMINI_2_5_PRO_PREVIEW_LATEST,
|
||||
GEMINI_MODEL.GEMINI_2_5_FLASH_PREVIEW,
|
||||
];
|
||||
|
||||
|
||||
@@ -74,6 +74,7 @@ export enum GEMINI_MODEL {
|
||||
GEMINI_2_0_FLASH_THINKING_EXP = "gemini-2.0-flash-thinking-exp-01-21",
|
||||
GEMINI_2_0_PRO_EXPERIMENTAL = "gemini-2.0-pro-exp-02-05",
|
||||
GEMINI_2_5_PRO_PREVIEW = "gemini-2.5-pro-preview-03-25",
|
||||
GEMINI_2_5_PRO_PREVIEW_LATEST = "gemini-2.5-pro-preview-05-06",
|
||||
GEMINI_2_5_FLASH_PREVIEW = "gemini-2.5-flash-preview-04-17",
|
||||
}
|
||||
|
||||
|
||||
@@ -289,7 +289,7 @@ export class GeminiHelper {
|
||||
if (fileContents.length > 0) {
|
||||
for (const file of fileContents) {
|
||||
const uploadResponse = await GeminiHelper.uploadFile(
|
||||
file.data,
|
||||
Buffer.from(file.data),
|
||||
file.mimeType,
|
||||
);
|
||||
parts.push({
|
||||
|
||||
@@ -13,7 +13,7 @@ import {
|
||||
type ToolCallLLMMessageOptions,
|
||||
} from "@llamaindex/core/llms";
|
||||
import { extractText } from "@llamaindex/core/utils";
|
||||
import { getEnv } from "@llamaindex/env";
|
||||
import { getEnv, uint8ArrayToBase64 } from "@llamaindex/env";
|
||||
import { Tokenizers } from "@llamaindex/env/tokenizers";
|
||||
import type {
|
||||
AzureClientOptions,
|
||||
@@ -217,10 +217,11 @@ export class OpenAI extends ToolCallLLM<OpenAIAdditionalChatOptions> {
|
||||
if (item.mimeType !== "application/pdf") {
|
||||
throw new Error("Only PDF files are supported");
|
||||
}
|
||||
const base64Data = uint8ArrayToBase64(item.data);
|
||||
return {
|
||||
type: "file",
|
||||
file: {
|
||||
file_data: `data:${item.mimeType};base64,${item.data.toString("base64")}`,
|
||||
file_data: `data:${item.mimeType};base64,${base64Data}`,
|
||||
filename: `part-${index}.pdf`,
|
||||
},
|
||||
} satisfies ChatCompletionContentPart.File;
|
||||
|
||||
@@ -16,7 +16,7 @@ import {
|
||||
} from "@llamaindex/core/llms";
|
||||
import type { StoredValue } from "@llamaindex/core/schema";
|
||||
import { extractText } from "@llamaindex/core/utils";
|
||||
import { getEnv } from "@llamaindex/env";
|
||||
import { getEnv, uint8ArrayToBase64 } from "@llamaindex/env";
|
||||
import {
|
||||
OpenAI as OpenAILLM,
|
||||
type AzureClientOptions,
|
||||
@@ -703,10 +703,11 @@ export class OpenAIResponses extends ToolCallLLM<OpenAIResponsesChatOptions> {
|
||||
);
|
||||
}
|
||||
|
||||
const base64Data = uint8ArrayToBase64(item.data);
|
||||
return {
|
||||
type: "input_file",
|
||||
filename: `part-${index}.pdf`,
|
||||
file_data: `data:${item.mimeType};base64,${item.data.toString("base64")}`,
|
||||
file_data: `data:${item.mimeType};base64,${base64Data}`,
|
||||
};
|
||||
}
|
||||
throw new Error("Unsupported content type");
|
||||
|
||||
@@ -205,7 +205,7 @@ describe("OpenAIResponses Unit Tests", () => {
|
||||
{
|
||||
type: "file",
|
||||
mimeType: "image/jpeg",
|
||||
data: Buffer.from("test image content"),
|
||||
data: Uint8Array.from(Buffer.from("test image content")),
|
||||
},
|
||||
];
|
||||
// @ts-expect-error accessing private method
|
||||
|
||||
@@ -2,7 +2,9 @@ import { describe, expect, test } from "vitest";
|
||||
import { duckduckgo, type DuckDuckGoToolOutput } from "../src/tools/duckduckgo";
|
||||
|
||||
describe("DuckDuckGo Tool", () => {
|
||||
test("performs search and returns results", async () => {
|
||||
// Needs to be skipped: duck-duck-scrape@2.2.7 throws an error:
|
||||
// DDG detected an anomaly in the request, you are likely making requests too quickly.
|
||||
test.skip("performs search and returns results", async () => {
|
||||
const searchTool = duckduckgo();
|
||||
const results = (await searchTool.call({
|
||||
query: "OpenAI ChatGPT",
|
||||
|
||||
Generated
+1
-1
@@ -1028,7 +1028,7 @@ importers:
|
||||
specifier: workspace:*
|
||||
version: link:../cloud
|
||||
'@llamaindex/core':
|
||||
specifier: workspace:*
|
||||
specifier: 0.6.3
|
||||
version: link:../core
|
||||
'@llamaindex/env':
|
||||
specifier: workspace:*
|
||||
|
||||
Reference in New Issue
Block a user