Release 0.8.11 (#1471)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

CONTRIBUTING.md

@@ -2,86 +2,58 @@
 
 ## Structure
 
-This is a monorepo built with Turborepo
+LlamaIndex.TS uses pnpm monorepo.
 
-Right now, for first-time contributors, these three packages are of the highest importance:
+We recommend you to understand the basics of Node.js, TypeScript, pnpm, and of course, LLM before contributing.
 
-- `packages/llamaindex` which is the main NPM library `llamaindex`
+There are some important folders in the repository:
-- `examples` is where the demo code lives
-- `apps/docs` is where the code for the documentation of https://ts.llamaindex.ai/ is located
 
-### Turborepo docs
+- `packages/*`: Contains the source code of the packages. Each package is a separate npm package.
-
+  - `llamaindex`: The starter package for LlamaIndex.TS, which contains the all sub-packages.
-You can checkout how Turborepo works using the default [README-turborepo.md](/README-turborepo.md)
+  - `core`: The core package of LlamaIndex.TS, which contains the abstract classes and interfaces. It is designed for
+    all JS runtime environments.
+  - `env`: The environment package of LlamaIndex.TS, which contains the environment-specific classes and interfaces. It
+    includes compatibility layers for Node.js, Deno, Vercel Edge Runtime, Cloudflare Workers...
+- `apps/*`: The applications based on LlamaIndex.TS.
+  - `next`: Our documentation website based on Next.js.
+- `examples`: The code examples of LlamaIndex.TS using Node.js.
 
 ## Getting Started
 
-Install NodeJS. Preferably v18 using nvm or n.
+Make sure you have Node.js LIS (Long-term Support) installed. You can check your Node.js version by running:
-
-Inside the LlamaIndexTS directory:
 
+```shell
+node -v
+# v20.x.x
 ```
-npm i -g pnpm ts-node
+
+### Use pnpm
+
+```shell
+corepack enable
+```
+
+### Install dependencies
+
+```shell
 pnpm install
 ```
 
-Note: we use pnpm in this repo, which has a lot of the same functionality and CLI options as npm but it does do some things better in a monorepo, like centralizing dependencies and caching.
+### Build the packages
 
-PNPM's has documentation on its [workspace feature](https://pnpm.io/workspaces) and Turborepo had some [useful documentation also](https://turbo.build/repo/docs/core-concepts/monorepos/running-tasks).
+```shell
-
+# Build all packages
-### Running Typescript
+turbo build --filter "./packages/*"
-
-When we publish to NPM we will have a tsc compiled version of the library in JS. For now, the easiest thing to do is use ts-node.
-
-### Test cases
-
-To run them, run
-
-```
-pnpm run test
-```
-
-To write new test cases write them in [packages/llamaindex/tests](/packages/llamaindex/tests)
-
-We use Vitest https://vitest.dev to write our test cases. Vitest comes with a bunch of built-in assertions using the expect function: https://vitest.dev/api/expect.html#expect
-
-### Demo applications
-
-There is an existing ["example"](/examples/README.md) demos folder with mainly NodeJS scripts. Feel free to add additional demos to that folder. If you would like to try out your changes in the `llamaindex` package with a new demo, you need to run the build command in the README.
-
-You can create new demo applications in the apps folder. Just run pnpm init in the folder after you create it to create its own package.json
-
-### Installing packages
-
-To install packages for a specific package or demo application, run
-
-```
-pnpm add [NPM Package] --filter [package or application i.e. llamaindex or docs]
-```
-
-To install packages for every package or application run
-
-```
-pnpm add -w [NPM Package]
 ```
 
 ### Docs
 
-To contribute to the docs, go to the docs website folder and run the Docusaurus instance.
+See the [docs](./apps/next/README.md) for more information.
-
-```bash
-cd apps/docs
-pnpm install
-pnpm start
-```
-
-That should start a webserver which will serve the docs on https://localhost:3000
-
-Any changes you make should be reflected in the browser. If you need to regenerate the API docs and find that your TSDoc isn't getting the updates, feel free to remove apps/docs/api. It will automatically regenerate itself when you run pnpm start again.
 
 ## Changeset
 
-We use [changesets](https://github.com/changesets/changesets) for managing versions and changelogs. To create a new changeset, run in the root folder:
+We use [changesets](https://github.com/changesets/changesets) for managing versions and changelogs. To create a new
+changeset, run in the root folder:
 
 ```
 pnpm changeset
@@ -95,6 +67,6 @@ The [Release Github Action](.github/workflows/release.yml) is automatically gene
 PR called "Release {version}".
 
 This PR will update the `package.json` and `CHANGELOG.md` files of each package according to
-the current changesets in the [.changeset](.changeset/) folder.
+the current changesets in the [.changeset](.changeset) folder.
 
 If this PR is merged it will automatically add version tags to the repository and publish the updated packages to NPM.

apps/docs/CHANGELOG.md

@@ -1,5 +1,41 @@
 # docs
 
+## 0.0.115
+
+### Patch Changes
+
+- llamaindex@0.8.11
+
+## 0.0.114
+
+### Patch Changes
+
+- Updated dependencies [f066e50]
+  - llamaindex@0.8.10
+  - @llamaindex/examples@0.0.14
+
+## 0.0.113
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+- Updated dependencies [4d4cd8a]
+  - llamaindex@0.8.9
+
+## 0.0.112
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - llamaindex@0.8.8
+  - @llamaindex/examples@0.0.13
+
+## 0.0.111
+
+### Patch Changes
+
+- llamaindex@0.8.7
+
 ## 0.0.110
 
 ### Patch Changes

apps/docs/package.json

@@ -1,6 +1,6 @@
 {
   "name": "docs",
-  "version": "0.0.110",
+  "version": "0.0.115",
   "private": true,
   "scripts": {
     "docusaurus": "docusaurus",

apps/next/CHANGELOG.md

@@ -1,5 +1,70 @@
 # @llamaindex/doc
 
+## 0.0.13
+
+### Patch Changes
+
+- Updated dependencies [f865c98]
+  - @llamaindex/core@0.4.8
+  - @llamaindex/cloud@2.0.8
+  - llamaindex@0.8.11
+  - @llamaindex/node-parser@0.0.9
+  - @llamaindex/openai@0.1.33
+  - @llamaindex/readers@1.0.9
+
+## 0.0.12
+
+### Patch Changes
+
+- Updated dependencies [f066e50]
+- Updated dependencies [d89ebe0]
+- Updated dependencies [fd8c882]
+- Updated dependencies [fd8c882]
+  - llamaindex@0.8.10
+  - @llamaindex/core@0.4.7
+  - @llamaindex/workflow@0.0.4
+  - @llamaindex/cloud@2.0.7
+  - @llamaindex/node-parser@0.0.8
+  - @llamaindex/openai@0.1.32
+  - @llamaindex/readers@1.0.8
+
+## 0.0.11
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+- Updated dependencies [4d4cd8a]
+  - llamaindex@0.8.9
+  - @llamaindex/cloud@2.0.6
+  - @llamaindex/core@0.4.6
+  - @llamaindex/node-parser@0.0.7
+  - @llamaindex/openai@0.1.31
+  - @llamaindex/readers@1.0.7
+
+## 0.0.10
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - @llamaindex/core@0.4.5
+  - llamaindex@0.8.8
+  - @llamaindex/node-parser@0.0.6
+  - @llamaindex/workflow@0.0.3
+  - @llamaindex/cloud@2.0.5
+  - @llamaindex/openai@0.1.30
+  - @llamaindex/readers@1.0.6
+
+## 0.0.9
+
+### Patch Changes
+
+- @llamaindex/cloud@2.0.4
+- @llamaindex/core@0.4.4
+- llamaindex@0.8.7
+- @llamaindex/node-parser@0.0.5
+- @llamaindex/openai@0.1.29
+- @llamaindex/readers@1.0.5
+
 ## 0.0.8
 
 ### Patch Changes

apps/next/README.md

@@ -1,4 +1,4 @@
-# next
+# Docs
 
 This is a Next.js application generated with
 [Create Fumadocs](https://github.com/fuma-nama/fumadocs).
@@ -6,15 +6,10 @@ This is a Next.js application generated with
 Run development server:
 
 ```bash
-npm run dev
+turbo run dev
-# or
+# turbo will build all required packages before running the dev server
-pnpm dev
-# or
-yarn dev
 ```
 
-Open http://localhost:3000 with your browser to see the result.
-
 ## Learn More
 
 To learn more about Next.js and Fumadocs, take a look at the following

apps/next/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/doc",
-  "version": "0.0.8",
+  "version": "0.0.13",
   "private": true,
   "scripts": {
     "build": "pnpm run build:docs && next build",

apps/next/src/content/docs/llamaindex/setup/typescript.mdx

@@ -93,6 +93,35 @@ See more about [moduleResolution](https://www.typescriptlang.org/docs/handbook/m
 	</Accordion>
 </Accordions>
 
+## Enable AsyncIterable for `Web Stream` API
+
+Some modules uses `Web Stream` API like `ReadableStream` and `WritableStream`, you need to enable `DOM.AsyncIterable` in your `tsconfig.json`.
+
+```json5
+{
+  compilerOptions: {
+    // ⬇️ add this lib to your tsconfig.json
+    lib: ["DOM.AsyncIterable"],
+  },
+}
+```
+
+```ts twoslash
+import { OpenAIAgent } from '@llamaindex/openai'
+
+const agent = new OpenAIAgent({
+  tools: []
+})
+
+const response = await agent.chat({
+  message: 'Hello, how are you?',
+  stream: true
+})
+for await (const _ of response) {
+                      //^?
+  // ...
+}
+```
 
 ## Run TypeScript Script in Node.js
 

apps/next/turbo.json

@@ -0,0 +1,16 @@
+{
+  "extends": ["//"],
+  "tasks": {
+    "build": {
+      "outputs": [
+        ".next",
+        ".source",
+        "next-env.d.ts",
+        "src/content/docs/cloud/api/**"
+      ]
+    },
+    "dev": {
+      "dependsOn": ["^build"]
+    }
+  }
+}

e2e/examples/cloudflare-worker-agent/CHANGELOG.md

@@ -1,5 +1,39 @@
 # @llamaindex/cloudflare-worker-agent-test
 
+## 0.0.107
+
+### Patch Changes
+
+- llamaindex@0.8.11
+
+## 0.0.106
+
+### Patch Changes
+
+- Updated dependencies [f066e50]
+  - llamaindex@0.8.10
+
+## 0.0.105
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+- Updated dependencies [4d4cd8a]
+  - llamaindex@0.8.9
+
+## 0.0.104
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - llamaindex@0.8.8
+
+## 0.0.103
+
+### Patch Changes
+
+- llamaindex@0.8.7
+
 ## 0.0.102
 
 ### Patch Changes

e2e/examples/cloudflare-worker-agent/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/cloudflare-worker-agent-test",
-  "version": "0.0.102",
+  "version": "0.0.107",
   "type": "module",
   "private": true,
   "scripts": {

e2e/examples/llama-parse-browser/CHANGELOG.md

@@ -1,5 +1,35 @@
 # @llamaindex/llama-parse-browser-test
 
+## 0.0.28
+
+### Patch Changes
+
+- @llamaindex/cloud@2.0.8
+
+## 0.0.27
+
+### Patch Changes
+
+- @llamaindex/cloud@2.0.7
+
+## 0.0.26
+
+### Patch Changes
+
+- @llamaindex/cloud@2.0.6
+
+## 0.0.25
+
+### Patch Changes
+
+- @llamaindex/cloud@2.0.5
+
+## 0.0.24
+
+### Patch Changes
+
+- @llamaindex/cloud@2.0.4
+
 ## 0.0.23
 
 ### Patch Changes

e2e/examples/llama-parse-browser/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/llama-parse-browser-test",
   "private": true,
-  "version": "0.0.23",
+  "version": "0.0.28",
   "type": "module",
   "scripts": {
     "dev": "vite",

e2e/examples/nextjs-agent/CHANGELOG.md

@@ -1,5 +1,39 @@
 # @llamaindex/next-agent-test
 
+## 0.1.107
+
+### Patch Changes
+
+- llamaindex@0.8.11
+
+## 0.1.106
+
+### Patch Changes
+
+- Updated dependencies [f066e50]
+  - llamaindex@0.8.10
+
+## 0.1.105
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+- Updated dependencies [4d4cd8a]
+  - llamaindex@0.8.9
+
+## 0.1.104
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - llamaindex@0.8.8
+
+## 0.1.103
+
+### Patch Changes
+
+- llamaindex@0.8.7
+
 ## 0.1.102
 
 ### Patch Changes

e2e/examples/nextjs-agent/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/next-agent-test",
-  "version": "0.1.102",
+  "version": "0.1.107",
   "private": true,
   "scripts": {
     "dev": "next dev",

e2e/examples/nextjs-edge-runtime/CHANGELOG.md

@@ -1,5 +1,39 @@
 # test-edge-runtime
 
+## 0.1.106
+
+### Patch Changes
+
+- llamaindex@0.8.11
+
+## 0.1.105
+
+### Patch Changes
+
+- Updated dependencies [f066e50]
+  - llamaindex@0.8.10
+
+## 0.1.104
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+- Updated dependencies [4d4cd8a]
+  - llamaindex@0.8.9
+
+## 0.1.103
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - llamaindex@0.8.8
+
+## 0.1.102
+
+### Patch Changes
+
+- llamaindex@0.8.7
+
 ## 0.1.101
 
 ### Patch Changes

e2e/examples/nextjs-edge-runtime/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/nextjs-edge-runtime-test",
-  "version": "0.1.101",
+  "version": "0.1.106",
   "private": true,
   "scripts": {
     "dev": "next dev",

e2e/examples/nextjs-node-runtime/CHANGELOG.md

@@ -1,5 +1,39 @@
 # @llamaindex/next-node-runtime
 
+## 0.0.88
+
+### Patch Changes
+
+- llamaindex@0.8.11
+
+## 0.0.87
+
+### Patch Changes
+
+- Updated dependencies [f066e50]
+  - llamaindex@0.8.10
+
+## 0.0.86
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+- Updated dependencies [4d4cd8a]
+  - llamaindex@0.8.9
+
+## 0.0.85
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - llamaindex@0.8.8
+
+## 0.0.84
+
+### Patch Changes
+
+- llamaindex@0.8.7
+
 ## 0.0.83
 
 ### Patch Changes

e2e/examples/nextjs-node-runtime/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/next-node-runtime-test",
-  "version": "0.0.83",
+  "version": "0.0.88",
   "private": true,
   "scripts": {
     "dev": "next dev",

e2e/examples/nextjs-node-runtime/src/actions/openai.ts

@@ -15,7 +15,6 @@ Settings.llm = new OpenAI({
 });
 Settings.embedModel = new HuggingFaceEmbedding({
   modelType: "BAAI/bge-small-en-v1.5",
-  quantized: false,
 });
 Settings.callbackManager.on("llm-tool-call", (event) => {
   console.log(event.detail);

e2e/examples/nextjs-node-runtime/src/utils/tokenizer.ts

@@ -1,5 +1,5 @@
 // test runtime
-import { Tokenizers, tokenizers } from "@llamaindex/env";
+import { Tokenizers, tokenizers } from "@llamaindex/env/tokenizers";
 import "llamaindex";
 
 // @ts-expect-error EdgeRuntime is not defined in type

e2e/examples/waku-query-engine/CHANGELOG.md

@@ -1,5 +1,39 @@
 # @llamaindex/waku-query-engine-test
 
+## 0.0.107
+
+### Patch Changes
+
+- llamaindex@0.8.11
+
+## 0.0.106
+
+### Patch Changes
+
+- Updated dependencies [f066e50]
+  - llamaindex@0.8.10
+
+## 0.0.105
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+- Updated dependencies [4d4cd8a]
+  - llamaindex@0.8.9
+
+## 0.0.104
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - llamaindex@0.8.8
+
+## 0.0.103
+
+### Patch Changes
+
+- llamaindex@0.8.7
+
 ## 0.0.102
 
 ### Patch Changes

e2e/examples/waku-query-engine/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/waku-query-engine-test",
-  "version": "0.0.102",
+  "version": "0.0.107",
   "type": "module",
   "private": true,
   "scripts": {

e2e/fixtures/llm/ollama.ts

@@ -0,0 +1,3 @@
+import { OpenAI } from "./openai.js";
+
+export class Ollama extends OpenAI {}

e2e/mock-module.js

@@ -15,7 +15,17 @@ export async function resolve(specifier, context, nextResolve) {
   const targetUrl = fileURLToPath(result.url).replace(/\.js$/, ".ts");
   let relativePath = relative(packageDistDir, targetUrl);
   // todo: make it more generic if we have more sub modules fixtures in the future
-  if (relativePath.startsWith("../../llm/openai")) {
+  if (relativePath.startsWith("../../llm/anthropic")) {
+    relativePath = relativePath.replace(
+      "../../llm/ollama/dist/index.ts",
+      "llm/anthropic.ts",
+    );
+  } else if (relativePath.startsWith("../../llm/ollama")) {
+    relativePath = relativePath.replace(
+      "../../llm/ollama/dist/index.ts",
+      "llm/ollama.ts",
+    );
+  } else if (relativePath.startsWith("../../llm/openai")) {
     relativePath = relativePath.replace(
       "../../llm/openai/dist/index.ts",
       "llm/openai.ts",

e2e/node/embedding/clip.e2e.ts

@@ -64,7 +64,7 @@ await test("clip embedding", async (t) => {
   });
 
   await t.test("custom transformer", async () => {
-    const transformers = await import("@xenova/transformers");
+    const transformers = await import("@huggingface/transformers");
     const getter = test.mock.fn((t, k, r) => {
       return Reflect.get(t, k, r);
     });

e2e/node/issue.e2e.ts

@@ -1,6 +1,13 @@
-import { LLMSingleSelector, Settings } from "llamaindex";
+import type { TaskStep } from "@llamaindex/core/agent";
+import {
+  LLMSingleSelector,
+  OpenAIAgent,
+  Settings,
+  type ChatMessage,
+} from "llamaindex";
 import assert from "node:assert";
 import { test } from "node:test";
+import { divideNumbersTool, sumNumbersTool } from "./fixtures/tools.js";
 import { mockLLMEvent } from "./utils.js";
 
 await test("#1177", async (t) => {
@@ -65,3 +72,28 @@ await test("#1177", async (t) => {
     }
   });
 });
+
+await test("#1281", async (t) => {
+  await mockLLMEvent(t, "#1281");
+  await t.test(async () => {
+    const chatHistory: ChatMessage[] = [];
+    const agent = new OpenAIAgent({
+      chatHistory,
+      tools: [sumNumbersTool, divideNumbersTool],
+    });
+    {
+      const stream = agent.createTask(
+        "calculate 2 + 2",
+        true,
+        true,
+        chatHistory,
+      );
+      const steps: TaskStep[] = [];
+      for await (const task of stream) {
+        steps.push(task.taskStep);
+      }
+      const lastStep = steps.at(-1)!;
+      assert.equal(lastStep.context.store.messages.length, 4);
+    }
+  });
+});

e2e/node/ollama.e2e.ts

@@ -0,0 +1,35 @@
+import { Ollama } from "@llamaindex/ollama";
+import assert from "node:assert";
+import { test } from "node:test";
+import { getWeatherTool } from "./fixtures/tools.js";
+import { mockLLMEvent } from "./utils.js";
+
+await test("ollama", async (t) => {
+  await mockLLMEvent(t, "ollama");
+  await t.test("ollama function call", async (t) => {
+    const llm = new Ollama({
+      model: "llama3.2",
+    });
+    const chatResponse = await llm.chat({
+      messages: [
+        {
+          role: "user",
+          content: "What is the weather in Paris?",
+        },
+      ],
+      tools: [getWeatherTool],
+    });
+    if (
+      chatResponse.message.options &&
+      "toolCall" in chatResponse.message.options
+    ) {
+      assert.equal(chatResponse.message.options.toolCall.length, 1);
+      assert.equal(
+        chatResponse.message.options.toolCall[0]!.name,
+        getWeatherTool.metadata.name,
+      );
+    } else {
+      throw new Error("Expected tool calls in response");
+    }
+  });
+});

e2e/node/openai.e2e.ts

@@ -167,6 +167,7 @@ For questions about more specific sections, please use the vector_tool.`,
   const mockCall = t.mock.fn(({ query }: { query: string }) => {
     return originalCall({ query });
   });
+  // @ts-expect-error what?
   queryEngineTools[1]!.call = mockCall;
 
   const toolMapping = SimpleToolNodeMapping.fromObjects(queryEngineTools);

e2e/node/snapshot/#1281.snap

@@ -0,0 +1,393 @@
+{
+  "llmEventStart": [
+    {
+      "id": "PRESERVE_0",
+      "messages": [
+        {
+          "role": "user",
+          "content": "calculate 2 + 2"
+        }
+      ]
+    },
+    {
+      "id": "PRESERVE_1",
+      "messages": [
+        {
+          "role": "user",
+          "content": "calculate 2 + 2"
+        },
+        {
+          "role": "assistant",
+          "content": "",
+          "options": {
+            "toolCall": [
+              {
+                "name": "sumNumbers",
+                "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+                "input": {
+                  "a": 2,
+                  "b": 2
+                }
+              }
+            ]
+          }
+        },
+        {
+          "role": "user",
+          "content": "4",
+          "options": {
+            "toolResult": {
+              "result": "4",
+              "isError": false,
+              "id": "call_S2x0FUa475GVpNQJ796Rc9fd"
+            }
+          }
+        }
+      ]
+    }
+  ],
+  "llmEventEnd": [
+    {
+      "id": "PRESERVE_0",
+      "response": {
+        "raw": null,
+        "message": {
+          "content": "",
+          "role": "assistant",
+          "options": {
+            "toolCall": [
+              {
+                "name": "sumNumbers",
+                "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+                "input": {
+                  "a": 2,
+                  "b": 2
+                }
+              }
+            ]
+          }
+        }
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "response": {
+        "raw": null,
+        "message": {
+          "content": "The result of \\(2 + 2\\) is \\(4\\).",
+          "role": "assistant",
+          "options": {}
+        }
+      }
+    }
+  ],
+  "llmEventStream": [
+    {
+      "id": "PRESERVE_0",
+      "chunk": {
+        "raw": null,
+        "options": {
+          "toolCall": [
+            {
+              "name": "sumNumbers",
+              "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+              "input": "{\"a\":2,\"b\":2}"
+            }
+          ]
+        },
+        "delta": ""
+      }
+    },
+    {
+      "id": "PRESERVE_0",
+      "chunk": {
+        "raw": null,
+        "options": {
+          "toolCall": [
+            {
+              "name": "sumNumbers",
+              "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+              "input": "{\"a\":2,\"b\":2}"
+            }
+          ]
+        },
+        "delta": ""
+      }
+    },
+    {
+      "id": "PRESERVE_0",
+      "chunk": {
+        "raw": null,
+        "options": {
+          "toolCall": [
+            {
+              "name": "sumNumbers",
+              "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+              "input": "{\"a\":2,\"b\":2}"
+            }
+          ]
+        },
+        "delta": ""
+      }
+    },
+    {
+      "id": "PRESERVE_0",
+      "chunk": {
+        "raw": null,
+        "options": {
+          "toolCall": [
+            {
+              "name": "sumNumbers",
+              "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+              "input": "{\"a\":2,\"b\":2}"
+            }
+          ]
+        },
+        "delta": ""
+      }
+    },
+    {
+      "id": "PRESERVE_0",
+      "chunk": {
+        "raw": null,
+        "options": {
+          "toolCall": [
+            {
+              "name": "sumNumbers",
+              "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+              "input": "{\"a\":2,\"b\":2}"
+            }
+          ]
+        },
+        "delta": ""
+      }
+    },
+    {
+      "id": "PRESERVE_0",
+      "chunk": {
+        "raw": null,
+        "options": {
+          "toolCall": [
+            {
+              "name": "sumNumbers",
+              "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+              "input": "{\"a\":2,\"b\":2}"
+            }
+          ]
+        },
+        "delta": ""
+      }
+    },
+    {
+      "id": "PRESERVE_0",
+      "chunk": {
+        "raw": null,
+        "options": {
+          "toolCall": [
+            {
+              "name": "sumNumbers",
+              "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+              "input": "{\"a\":2,\"b\":2}"
+            }
+          ]
+        },
+        "delta": ""
+      }
+    },
+    {
+      "id": "PRESERVE_0",
+      "chunk": {
+        "raw": null,
+        "options": {
+          "toolCall": [
+            {
+              "name": "sumNumbers",
+              "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+              "input": "{\"a\":2,\"b\":2}"
+            }
+          ]
+        },
+        "delta": ""
+      }
+    },
+    {
+      "id": "PRESERVE_0",
+      "chunk": {
+        "raw": null,
+        "options": {
+          "toolCall": [
+            {
+              "name": "sumNumbers",
+              "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+              "input": "{\"a\":2,\"b\":2}"
+            }
+          ]
+        },
+        "delta": ""
+      }
+    },
+    {
+      "id": "PRESERVE_0",
+      "chunk": {
+        "raw": null,
+        "options": {
+          "toolCall": [
+            {
+              "name": "sumNumbers",
+              "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+              "input": "{\"a\":2,\"b\":2}"
+            }
+          ]
+        },
+        "delta": ""
+      }
+    },
+    {
+      "id": "PRESERVE_0",
+      "chunk": {
+        "raw": null,
+        "options": {
+          "toolCall": [
+            {
+              "name": "sumNumbers",
+              "id": "call_S2x0FUa475GVpNQJ796Rc9fd",
+              "input": {
+                "a": 2,
+                "b": 2
+              }
+            }
+          ]
+        },
+        "delta": ""
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": "The"
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": " result"
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": " of"
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": " \\("
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": "2"
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": " +"
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": " "
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": "2"
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": "\\"
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": ")"
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": " is"
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": " \\("
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": "4"
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": "\\"
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": ")."
+      }
+    },
+    {
+      "id": "PRESERVE_1",
+      "chunk": {
+        "raw": null,
+        "options": {},
+        "delta": ""
+      }
+    }
+  ]
+}

e2e/node/snapshot/ollama.snap

@@ -0,0 +1,37 @@
+{
+  "llmEventStart": [
+    {
+      "id": "PRESERVE_0",
+      "messages": [
+        {
+          "role": "user",
+          "content": "What is the weather in Paris?"
+        }
+      ]
+    }
+  ],
+  "llmEventEnd": [
+    {
+      "id": "PRESERVE_0",
+      "response": {
+        "message": {
+          "role": "assistant",
+          "content": "",
+          "options": {
+            "toolCall": [
+              {
+                "name": "getWeather",
+                "input": {
+                  "city": "Paris"
+                },
+                "id": "5d198775-5268-4552-993b-9ecb4425385b"
+              }
+            ]
+          }
+        },
+        "raw": null
+      }
+    }
+  ],
+  "llmEventStream": []
+}

e2e/package.json

@@ -12,10 +12,11 @@
     "@faker-js/faker": "^9.2.0",
     "@llamaindex/core": "workspace:*",
     "@llamaindex/env": "workspace:*",
+    "@llamaindex/ollama": "workspace:*",
     "@llamaindex/openai": "workspace:*",
     "@types/node": "^22.9.0",
     "@types/pg": "^8.11.8",
-    "@xenova/transformers": "^2.17.2",
+    "@huggingface/transformers": "^3.0.2",
     "consola": "^3.2.3",
     "dotenv": "^16.4.5",
     "llamaindex": "workspace:*",

e2e/tsconfig.json

@@ -5,7 +5,6 @@
     "module": "node16",
     "moduleResolution": "node16",
     "target": "ESNext",
-    "lib": ["ES2022", "DOM.AsyncIterable"],
     "types": ["node"]
   },
   "include": ["./node", "./mock-module.js", "./mock-register.js", "./fixtures"],

examples/CHANGELOG.md

@@ -1,5 +1,28 @@
 # examples
 
+## 0.0.14
+
+### Patch Changes
+
+- Updated dependencies [f066e50]
+- Updated dependencies [d89ebe0]
+- Updated dependencies [fd8c882]
+- Updated dependencies [fd8c882]
+  - llamaindex@0.8.10
+  - @llamaindex/core@0.4.7
+  - @llamaindex/workflow@0.0.4
+  - @llamaindex/readers@1.0.8
+
+## 0.0.13
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - @llamaindex/core@0.4.5
+  - llamaindex@0.8.8
+  - @llamaindex/workflow@0.0.3
+  - @llamaindex/readers@1.0.6
+
 ## 0.0.12
 
 ### Patch Changes

examples/package.json

@@ -1,15 +1,15 @@
 {
   "name": "@llamaindex/examples",
   "private": true,
-  "version": "0.0.12",
+  "version": "0.0.14",
   "dependencies": {
     "@aws-crypto/sha256-js": "^5.2.0",
     "@azure/cosmos": "^4.1.1",
     "@azure/identity": "^4.4.1",
     "@datastax/astra-db-ts": "^1.4.1",
-    "@llamaindex/core": "^0.4.0",
+    "@llamaindex/core": "^0.4.7",
-    "@llamaindex/readers": "^1.0.0",
+    "@llamaindex/readers": "^1.0.8",
-    "@llamaindex/workflow": "^0.0.2",
+    "@llamaindex/workflow": "^0.0.4",
     "@notionhq/client": "^2.2.15",
     "@pinecone-database/pinecone": "^3.0.2",
     "@vercel/postgres": "^0.10.0",
@@ -18,7 +18,7 @@
     "commander": "^12.1.0",
     "dotenv": "^16.4.5",
     "js-tiktoken": "^1.0.14",
-    "llamaindex": "^0.8.0",
+    "llamaindex": "^0.8.10",
     "mongodb": "^6.7.0",
     "pathe": "^1.1.2",
     "postgres": "^3.4.4"

examples/vectorIndexLocal.ts

@@ -14,7 +14,6 @@ Settings.llm = new Ollama({
 
 Settings.embedModel = new HuggingFaceEmbedding({
   modelType: "BAAI/bge-small-en-v1.5",
-  quantized: false,
 });
 
 async function main() {

examples/vllm.ts

@@ -0,0 +1,16 @@
+import { VLLM } from "llamaindex";
+
+const llm = new VLLM({
+  model: "NousResearch/Meta-Llama-3-8B-Instruct",
+});
+
+const response = await llm.chat({
+  messages: [
+    {
+      role: "user",
+      content: "Hello?",
+    },
+  ],
+});
+
+console.log(response.message.content);

examples/workflow/app-creator.ts

@@ -1,14 +1,19 @@
 import {
-  Context,
+  HandlerContext,
   StartEvent,
   StopEvent,
   Workflow,
   WorkflowEvent,
-} from "@llamaindex/core/workflow";
+} from "@llamaindex/workflow";
 import { OpenAI } from "llamaindex";
 
 const MAX_REVIEWS = 3;
 
+type Context = {
+  specification: string;
+  numberReviews: number;
+};
+
 // Using the o1-preview model (see https://platform.openai.com/docs/guides/reasoning?reasoning-prompt-examples=coding-planning)
 const llm = new OpenAI({ model: "o1-preview", temperature: 1 });
 
@@ -20,7 +25,9 @@ stores the question/answer pair in the database.`;
 
 // Create custom event types
 export class MessageEvent extends WorkflowEvent<{ msg: string }> {}
+
 export class CodeEvent extends WorkflowEvent<{ code: string }> {}
+
 export class ReviewEvent extends WorkflowEvent<{
   review: string;
   code: string;
@@ -34,12 +41,13 @@ const truncate = (str: string) => {
 };
 
 // the architect is responsible for writing the structure and the initial code based on the specification
-const architect = async (context: Context, ev: StartEvent) => {
+const architect = async (
-  // get the specification from the start event and save it to context
+  context: HandlerContext<Context>,
-  context.set("specification", ev.data.input);
+  _: StartEvent<string>,
-  const spec = context.get("specification");
+) => {
+  const spec = context.data.specification;
   // write a message to send an update to the user
-  context.writeEventToStream(
+  context.sendEvent(
     new MessageEvent({
       msg: `Writing app using this specification: ${truncate(spec)}`,
     }),
@@ -50,13 +58,13 @@ const architect = async (context: Context, ev: StartEvent) => {
 };
 
 // the coder is responsible for updating the code based on the review
-const coder = async (context: Context, ev: ReviewEvent) => {
+const coder = async (context: HandlerContext<Context>, ev: ReviewEvent) => {
   // get the specification from the context
-  const spec = context.get("specification");
+  const spec = context.data.specification;
   // get the latest review and code
   const { review, code } = ev.data;
   // write a message to send an update to the user
-  context.writeEventToStream(
+  context.sendEvent(
     new MessageEvent({
       msg: `Update code based on review: ${truncate(review)}`,
     }),
@@ -67,32 +75,35 @@ const coder = async (context: Context, ev: ReviewEvent) => {
 };
 
 // the reviewer is responsible for reviewing the code and providing feedback
-const reviewer = async (context: Context, ev: CodeEvent) => {
+const reviewer = async (context: HandlerContext<Context>, ev: CodeEvent) => {
   // get the specification from the context
-  const spec = context.get("specification");
+  const spec = context.data.specification;
   // get latest code from the event
   const { code } = ev.data;
   // update and check the number of reviews
-  const numberReviews = context.get("numberReviews", 0) + 1;
+  context.data.numberReviews++;
-  context.set("numberReviews", numberReviews);
+  if (context.data.numberReviews > MAX_REVIEWS) {
-  if (numberReviews > MAX_REVIEWS) {
     // the we've done this too many times - return the code
-    context.writeEventToStream(
+    context.sendEvent(
       new MessageEvent({
-        msg: `Already reviewed ${numberReviews - 1} times, stopping!`,
+        msg: `Already reviewed ${
+          context.data.numberReviews - 1
+        } times, stopping!`,
       }),
     );
     return new StopEvent({ result: code });
   }
   // write a message to send an update to the user
-  context.writeEventToStream(
+  context.sendEvent(
-    new MessageEvent({ msg: `Review #${numberReviews}: ${truncate(code)}` }),
+    new MessageEvent({
+      msg: `Review #${context.data.numberReviews}: ${truncate(code)}`,
+    }),
   );
   const prompt = `Review this code: <code>${code}</code>. Check if the code quality and whether it correctly implements this specification: <spec>${spec}</spec>. If you're satisfied, just return 'Looks great', nothing else. If not, return a review with a list of changes you'd like to see.`;
   const review = (await llm.complete({ prompt })).text;
   if (review.includes("Looks great")) {
     // the reviewer is satisfied with the code, let's return the review
-    context.writeEventToStream(
+    context.sendEvent(
       new MessageEvent({
         msg: `Reviewer says: ${review}`,
       }),
@@ -103,20 +114,44 @@ const reviewer = async (context: Context, ev: CodeEvent) => {
   return new ReviewEvent({ review, code });
 };
 
-const codeAgent = new Workflow({ validate: true });
+const codeAgent = new Workflow<Context, string, string>();
-codeAgent.addStep(StartEvent, architect, { outputs: CodeEvent });
+codeAgent.addStep(
-codeAgent.addStep(ReviewEvent, coder, { outputs: CodeEvent });
+  {
-codeAgent.addStep(CodeEvent, reviewer, { outputs: ReviewEvent });
+    inputs: [StartEvent<string>],
+    outputs: [CodeEvent],
+  },
+  architect,
+);
+codeAgent.addStep(
+  {
+    inputs: [ReviewEvent],
+    outputs: [CodeEvent],
+  },
+  coder,
+);
+codeAgent.addStep(
+  {
+    inputs: [CodeEvent],
+    outputs: [ReviewEvent, StopEvent],
+  },
+  reviewer,
+);
 
 // Usage
 async function main() {
-  const run = codeAgent.run(specification);
+  const run = codeAgent.run(specification).with({
-  for await (const event of codeAgent.streamEvents()) {
+    specification,
-    const msg = (event as MessageEvent).data.msg;
+    numberReviews: 0,
-    console.log(`${msg}\n`);
+  });
+  for await (const event of run) {
+    if (event instanceof MessageEvent) {
+      const msg = (event as MessageEvent).data.msg;
+      console.log(`${msg}\n`);
+    } else if (event instanceof StopEvent) {
+      const result = (event as StopEvent<string>).data;
+      console.log("Final code:\n", result);
+    }
   }
-  const result = await run;
-  console.log("Final code:\n", result.data.result);
 }
 
 main().catch(console.error);

examples/workflow/conditional.ts

@@ -1,10 +1,10 @@
 import {
-  Context,
+  HandlerContext,
   StartEvent,
   StopEvent,
   Workflow,
   WorkflowEvent,
-} from "@llamaindex/core/workflow";
+} from "@llamaindex/workflow";
 import { OpenAI } from "llamaindex";
 
 // Create LLM instance
@@ -12,59 +12,77 @@ const llm = new OpenAI();
 
 // Create custom event types
 export class JokeEvent extends WorkflowEvent<{ joke: string }> {}
+
 export class CritiqueEvent extends WorkflowEvent<{ critique: string }> {}
+
 export class AnalysisEvent extends WorkflowEvent<{ analysis: string }> {}
 
-const generateJoke = async (_context: Context, ev: StartEvent) => {
+const generateJoke = async (_: unknown, ev: StartEvent<string>) => {
-  const prompt = `Write your best joke about ${ev.data.input}.`;
+  const prompt = `Write your best joke about ${ev.data}.`;
   const response = await llm.complete({ prompt });
   return new JokeEvent({ joke: response.text });
 };
 
-const critiqueJoke = async (_context: Context, ev: JokeEvent) => {
+const critiqueJoke = async (_: unknown, ev: JokeEvent) => {
   const prompt = `Give a thorough critique of the following joke: ${ev.data.joke}`;
   const response = await llm.complete({ prompt });
   return new CritiqueEvent({ critique: response.text });
 };
 
-const analyzeJoke = async (_context: Context, ev: JokeEvent) => {
+const analyzeJoke = async (_: unknown, ev: JokeEvent) => {
   const prompt = `Give a thorough analysis of the following joke: ${ev.data.joke}`;
   const response = await llm.complete({ prompt });
   return new AnalysisEvent({ analysis: response.text });
 };
 
 const reportJoke = async (
-  context: Context,
+  context: HandlerContext,
-  ev: AnalysisEvent | CritiqueEvent,
+  ev1: AnalysisEvent,
+  ev2: CritiqueEvent,
 ) => {
-  const events = context.collectEvents(ev, [AnalysisEvent, CritiqueEvent]);
+  const subPrompts = [ev1.data.analysis, ev2.data.critique];
-  if (!events) {
-    return;
-  }
-  const subPrompts = events.map((event) => {
-    if (event instanceof AnalysisEvent) {
-      return `Analysis: ${event.data.analysis}`;
-    } else if (event instanceof CritiqueEvent) {
-      return `Critique: ${event.data.critique}`;
-    }
-    return "";
-  });
 
-  const prompt = `Based on the following information about a joke:\n${subPrompts.join("\n")}\nProvide a comprehensive report on the joke's quality and impact.`;
+  const prompt = `Based on the following information about a joke:\n${subPrompts.join(
+    "\n",
+  )}\nProvide a comprehensive report on the joke's quality and impact.`;
   const response = await llm.complete({ prompt });
-  return new StopEvent({ result: response.text });
+  return new StopEvent(response.text);
 };
 
-const jokeFlow = new Workflow();
+const jokeFlow = new Workflow<unknown, string, string>();
-jokeFlow.addStep(StartEvent, generateJoke);
+jokeFlow.addStep(
-jokeFlow.addStep(JokeEvent, critiqueJoke);
+  {
-jokeFlow.addStep(JokeEvent, analyzeJoke);
+    inputs: [StartEvent<string>],
-jokeFlow.addStep([AnalysisEvent, CritiqueEvent], reportJoke);
+    outputs: [JokeEvent],
+  },
+  generateJoke,
+);
+jokeFlow.addStep(
+  {
+    inputs: [JokeEvent],
+    outputs: [CritiqueEvent],
+  },
+  critiqueJoke,
+);
+jokeFlow.addStep(
+  {
+    inputs: [JokeEvent],
+    outputs: [AnalysisEvent],
+  },
+  analyzeJoke,
+);
+jokeFlow.addStep(
+  {
+    inputs: [AnalysisEvent, CritiqueEvent],
+    outputs: [StopEvent<string>],
+  },
+  reportJoke,
+);
 
 // Usage
 async function main() {
   const result = await jokeFlow.run("pirates");
-  console.log(result.data.result);
+  console.log(result.data);
 }
 
 main().catch(console.error);

examples/workflow/joke.ts

@@ -1,10 +1,9 @@
 import {
-  Context,
   StartEvent,
   StopEvent,
   Workflow,
   WorkflowEvent,
-} from "@llamaindex/core/workflow";
+} from "@llamaindex/workflow";
 import { OpenAI } from "llamaindex";
 
 // Create LLM instance
@@ -13,26 +12,38 @@ const llm = new OpenAI();
 // Create a custom event type
 export class JokeEvent extends WorkflowEvent<{ joke: string }> {}
 
-const generateJoke = async (_context: Context, ev: StartEvent) => {
+const generateJoke = async (_: unknown, ev: StartEvent<string>) => {
-  const prompt = `Write your best joke about ${ev.data.input}.`;
+  const prompt = `Write your best joke about ${ev.data}.`;
   const response = await llm.complete({ prompt });
   return new JokeEvent({ joke: response.text });
 };
 
-const critiqueJoke = async (_context: Context, ev: JokeEvent) => {
+const critiqueJoke = async (_: unknown, ev: JokeEvent) => {
   const prompt = `Give a thorough critique of the following joke: ${ev.data.joke}`;
   const response = await llm.complete({ prompt });
-  return new StopEvent({ result: response.text });
+  return new StopEvent(response.text);
 };
 
-const jokeFlow = new Workflow({ verbose: true });
+const jokeFlow = new Workflow<unknown, string, string>();
-jokeFlow.addStep(StartEvent, generateJoke);
+jokeFlow.addStep(
-jokeFlow.addStep(JokeEvent, critiqueJoke);
+  {
+    inputs: [StartEvent<string>],
+    outputs: [JokeEvent],
+  },
+  generateJoke,
+);
+jokeFlow.addStep(
+  {
+    inputs: [JokeEvent],
+    outputs: [StopEvent<string>],
+  },
+  critiqueJoke,
+);
 
 // Usage
 async function main() {
   const result = await jokeFlow.run("pirates");
-  console.log(result.data.result);
+  console.log(result.data);
 }
 
 main().catch(console.error);

examples/workflow/stream-events.ts

@@ -1,10 +1,10 @@
 import {
-  Context,
+  HandlerContext,
   StartEvent,
   StopEvent,
   Workflow,
   WorkflowEvent,
-} from "@llamaindex/core/workflow";
+} from "@llamaindex/workflow";
 import { OpenAI } from "llamaindex";
 
 // Create LLM instance
@@ -12,38 +12,55 @@ const llm = new OpenAI();
 
 // Create custom event types
 export class JokeEvent extends WorkflowEvent<{ joke: string }> {}
+
 export class MessageEvent extends WorkflowEvent<{ msg: string }> {}
 
-const generateJoke = async (context: Context, ev: StartEvent) => {
+const generateJoke = async (context: HandlerContext, ev: StartEvent) => {
-  context.writeEventToStream(
+  context.sendEvent(
-    new MessageEvent({ msg: `Generating a joke about: ${ev.data.input}` }),
+    new MessageEvent({ msg: `Generating a joke about: ${ev.data}` }),
   );
-  const prompt = `Write your best joke about ${ev.data.input}.`;
+  const prompt = `Write your best joke about ${ev.data}.`;
   const response = await llm.complete({ prompt });
   return new JokeEvent({ joke: response.text });
 };
 
-const critiqueJoke = async (context: Context, ev: JokeEvent) => {
+const critiqueJoke = async (context: HandlerContext, ev: JokeEvent) => {
-  context.writeEventToStream(
+  context.sendEvent(
     new MessageEvent({ msg: `Write a critique of this joke: ${ev.data.joke}` }),
   );
   const prompt = `Give a thorough critique of the following joke: ${ev.data.joke}`;
   const response = await llm.complete({ prompt });
-  return new StopEvent({ result: response.text });
+  return new StopEvent(response.text);
 };
 
 const jokeFlow = new Workflow();
-jokeFlow.addStep(StartEvent, generateJoke);
+jokeFlow.addStep(
-jokeFlow.addStep(JokeEvent, critiqueJoke);
+  {
+    inputs: [StartEvent<string>],
+    outputs: [JokeEvent],
+  },
+  generateJoke,
+);
+jokeFlow.addStep(
+  {
+    inputs: [JokeEvent],
+    outputs: [StopEvent<string>],
+  },
+  critiqueJoke,
+);
 
 // Usage
 async function main() {
   const run = jokeFlow.run("pirates");
-  for await (const event of jokeFlow.streamEvents()) {
+  for await (const event of run) {
-    console.log((event as MessageEvent).data.msg);
+    if (event instanceof MessageEvent) {
+      console.log("Message:");
+      console.log((event as MessageEvent).data.msg);
+    } else if (event instanceof StopEvent) {
+      console.log("Result:");
+      console.log((event as StopEvent<string>).data);
+    }
   }
-  const result = await run;
-  console.log(result.data.result);
 }
 
 main().catch(console.error);

examples/workflow/timeout.ts

@@ -1,19 +1,21 @@
-import {
+import { StartEvent, StopEvent, Workflow } from "@llamaindex/workflow";
-  Context,
-  StartEvent,
-  StopEvent,
-  Workflow,
-} from "@llamaindex/core/workflow";
 
-const longRunning = async (_context: Context, ev: StartEvent) => {
+const longRunning = async (_: unknown, ev: StartEvent<string>) => {
   await new Promise((resolve) => setTimeout(resolve, 2000)); // Wait for 2 seconds
-  return new StopEvent({ result: "We waited 2 seconds" });
+  return new StopEvent("We waited 2 seconds");
 };
 
 async function timeout() {
-  const workflow = new Workflow({ verbose: true, timeout: 1 });
+  const workflow = new Workflow<unknown, string, string>({
-  workflow.addStep(StartEvent, longRunning);
+    timeout: 1,
-  // This will timeout
+  });
+  workflow.addStep(
+    {
+      inputs: [StartEvent<string>],
+      outputs: [StopEvent<string>],
+    },
+    longRunning,
+  );
   try {
     await workflow.run("Let's start");
   } catch (error) {
@@ -23,14 +25,23 @@ async function timeout() {
 
 async function notimeout() {
   // Increase timeout to 3 seconds - no timeout
-  const workflow = new Workflow({ verbose: true, timeout: 3 });
+  const workflow = new Workflow<unknown, string, string>({
-  workflow.addStep(StartEvent, longRunning);
+    timeout: 3,
+  });
+  workflow.addStep(
+    {
+      inputs: [StartEvent<string>],
+      outputs: [StopEvent<string>],
+    },
+    longRunning,
+  );
   const result = await workflow.run("Let's start");
-  console.log(result.data.result);
+  console.log(result.data);
 }
 
 async function main() {
   await timeout();
+  console.log("---");
   await notimeout();
 }
 

examples/workflow/validation.ts

@@ -1,10 +1,9 @@
 import {
-  Context,
   StartEvent,
   StopEvent,
   Workflow,
   WorkflowEvent,
-} from "@llamaindex/core/workflow";
+} from "@llamaindex/workflow";
 import { OpenAI } from "llamaindex";
 
 // Create LLM instance
@@ -13,40 +12,66 @@ const llm = new OpenAI();
 // Create a custom event type
 export class JokeEvent extends WorkflowEvent<{ joke: string }> {}
 
-const generateJoke = async (_context: Context, ev: StartEvent) => {
+const generateJoke = async (_: unknown, ev: StartEvent<string>) => {
-  const prompt = `Write your best joke about ${ev.data.input}.`;
+  const prompt = `Write your best joke about ${ev.data}.`;
   const response = await llm.complete({ prompt });
   return new JokeEvent({ joke: response.text });
 };
 
-const critiqueJoke = async (_context: Context, ev: JokeEvent) => {
+const critiqueJoke = async (_: unknown, ev: JokeEvent) => {
   const prompt = `Give a thorough critique of the following joke: ${ev.data.joke}`;
   const response = await llm.complete({ prompt });
-  return new StopEvent({ result: response.text });
+  return new StopEvent(response.text);
 };
 
 async function validateFails() {
   try {
-    const jokeFlow = new Workflow({ verbose: true, validate: true });
+    const jokeFlow = new Workflow();
-    jokeFlow.addStep(StartEvent, generateJoke, { outputs: StopEvent });
+    jokeFlow.addStep(
-    jokeFlow.addStep(JokeEvent, critiqueJoke, { outputs: StopEvent });
+      {
-    await jokeFlow.run("pirates");
+        inputs: [StartEvent<string>],
+        outputs: [StopEvent<string>],
+      },
+      // @ts-expect-error outputs should be JokeEvent
+      generateJoke,
+    );
+    jokeFlow.addStep(
+      {
+        inputs: [JokeEvent],
+        outputs: [StopEvent],
+      },
+      critiqueJoke,
+    );
+    await jokeFlow.run("pirates").strict();
   } catch (e) {
     console.error("Validation failed:", e);
   }
 }
 
 async function validate() {
-  const jokeFlow = new Workflow({ verbose: true, validate: true });
+  const jokeFlow = new Workflow();
-  jokeFlow.addStep(StartEvent, generateJoke, { outputs: JokeEvent });
+  jokeFlow.addStep(
-  jokeFlow.addStep(JokeEvent, critiqueJoke, { outputs: StopEvent });
+    {
-  const result = await jokeFlow.run("pirates");
+      inputs: [StartEvent<string>],
-  console.log(result.data.result);
+      outputs: [JokeEvent],
+    },
+    generateJoke,
+  );
+  jokeFlow.addStep(
+    {
+      inputs: [JokeEvent],
+      outputs: [StopEvent<string>],
+    },
+    critiqueJoke,
+  );
+  const result = await jokeFlow.run("pirates").strict();
+  console.log(result.data);
 }
 
 // Usage
 async function main() {
   await validateFails();
+  console.log("---");
   await validate();
 }
 

package.json

@@ -35,12 +35,6 @@
     "typescript-eslint": "^8.13.0"
   },
   "packageManager": "pnpm@9.12.3",
-  "pnpm": {
-    "overrides": {
-      "trim": "1.0.1",
-      "protobufjs": "7.2.6"
-    }
-  },
   "lint-staged": {
     "(!apps/docs/i18n/**/docusaurus-plugin-content-docs/current/api/*).{js,jsx,ts,tsx,md}": "prettier --write"
   }

packages/autotool/CHANGELOG.md

@@ -1,5 +1,39 @@
 # @llamaindex/autotool
 
+## 5.0.11
+
+### Patch Changes
+
+- llamaindex@0.8.11
+
+## 5.0.10
+
+### Patch Changes
+
+- Updated dependencies [f066e50]
+  - llamaindex@0.8.10
+
+## 5.0.9
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+- Updated dependencies [4d4cd8a]
+  - llamaindex@0.8.9
+
+## 5.0.8
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - llamaindex@0.8.8
+
+## 5.0.7
+
+### Patch Changes
+
+- llamaindex@0.8.7
+
 ## 5.0.6
 
 ### Patch Changes

packages/autotool/examples/01_node/CHANGELOG.md

@@ -1,5 +1,44 @@
 # @llamaindex/autotool-01-node-example
 
+## 0.0.54
+
+### Patch Changes
+
+- llamaindex@0.8.11
+- @llamaindex/autotool@5.0.11
+
+## 0.0.53
+
+### Patch Changes
+
+- Updated dependencies [f066e50]
+  - llamaindex@0.8.10
+  - @llamaindex/autotool@5.0.10
+
+## 0.0.52
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+- Updated dependencies [4d4cd8a]
+  - llamaindex@0.8.9
+  - @llamaindex/autotool@5.0.9
+
+## 0.0.51
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - llamaindex@0.8.8
+  - @llamaindex/autotool@5.0.8
+
+## 0.0.50
+
+### Patch Changes
+
+- llamaindex@0.8.7
+- @llamaindex/autotool@5.0.7
+
 ## 0.0.49
 
 ### Patch Changes

packages/autotool/examples/01_node/package.json

@@ -13,5 +13,5 @@
   "scripts": {
     "start": "node --import tsx --import @llamaindex/autotool/node ./src/index.ts"
   },
-  "version": "0.0.49"
+  "version": "0.0.54"
 }

packages/autotool/examples/02_nextjs/CHANGELOG.md

@@ -1,5 +1,44 @@
 # @llamaindex/autotool-02-next-example
 
+## 0.1.98
+
+### Patch Changes
+
+- llamaindex@0.8.11
+- @llamaindex/autotool@5.0.11
+
+## 0.1.97
+
+### Patch Changes
+
+- Updated dependencies [f066e50]
+  - llamaindex@0.8.10
+  - @llamaindex/autotool@5.0.10
+
+## 0.1.96
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+- Updated dependencies [4d4cd8a]
+  - llamaindex@0.8.9
+  - @llamaindex/autotool@5.0.9
+
+## 0.1.95
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - llamaindex@0.8.8
+  - @llamaindex/autotool@5.0.8
+
+## 0.1.94
+
+### Patch Changes
+
+- llamaindex@0.8.7
+- @llamaindex/autotool@5.0.7
+
 ## 0.1.93
 
 ### Patch Changes

packages/autotool/examples/02_nextjs/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/autotool-02-next-example",
   "private": true,
-  "version": "0.1.93",
+  "version": "0.1.98",
   "scripts": {
     "dev": "next dev",
     "build": "next build",

packages/autotool/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/autotool",
   "type": "module",
-  "version": "5.0.6",
+  "version": "5.0.11",
   "description": "auto transpile your JS function to LLM Agent compatible",
   "files": [
     "dist",

packages/cloud/CHANGELOG.md

@@ -1,5 +1,44 @@
 # @llamaindex/cloud
 
+## 2.0.8
+
+### Patch Changes
+
+- Updated dependencies [f865c98]
+  - @llamaindex/core@0.4.8
+
+## 2.0.7
+
+### Patch Changes
+
+- Updated dependencies [d89ebe0]
+- Updated dependencies [fd8c882]
+  - @llamaindex/core@0.4.7
+
+## 2.0.6
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+  - @llamaindex/env@0.1.20
+  - @llamaindex/core@0.4.6
+
+## 2.0.5
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - @llamaindex/core@0.4.5
+  - @llamaindex/env@0.1.19
+
+## 2.0.4
+
+### Patch Changes
+
+- Updated dependencies [a8d3fa6]
+  - @llamaindex/env@0.1.18
+  - @llamaindex/core@0.4.4
+
 ## 2.0.3
 
 ### Patch Changes

packages/cloud/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@llamaindex/cloud",
-  "version": "2.0.3",
+  "version": "2.0.8",
   "type": "module",
   "license": "MIT",
   "scripts": {

packages/cloud/tsconfig.json

@@ -8,7 +8,6 @@
     "moduleResolution": "Bundler",
     "skipLibCheck": true,
     "strict": true,
-    "lib": ["DOM", "ESNext"],
     "types": []
   },
   "include": ["./src"],

packages/community/CHANGELOG.md

@@ -1,5 +1,44 @@
 # @llamaindex/community
 
+## 0.0.66
+
+### Patch Changes
+
+- Updated dependencies [f865c98]
+  - @llamaindex/core@0.4.8
+
+## 0.0.65
+
+### Patch Changes
+
+- Updated dependencies [d89ebe0]
+- Updated dependencies [fd8c882]
+  - @llamaindex/core@0.4.7
+
+## 0.0.64
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+  - @llamaindex/env@0.1.20
+  - @llamaindex/core@0.4.6
+
+## 0.0.63
+
+### Patch Changes
+
+- Updated dependencies [ad85bd0]
+  - @llamaindex/core@0.4.5
+  - @llamaindex/env@0.1.19
+
+## 0.0.62
+
+### Patch Changes
+
+- Updated dependencies [a8d3fa6]
+  - @llamaindex/env@0.1.18
+  - @llamaindex/core@0.4.4
+
 ## 0.0.61
 
 ### Patch Changes

packages/community/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/community",
   "description": "Community package for LlamaIndexTS",
-  "version": "0.0.61",
+  "version": "0.0.66",
   "type": "module",
   "types": "dist/type/index.d.ts",
   "main": "dist/cjs/index.js",

packages/core/CHANGELOG.md

@@ -1,5 +1,41 @@
 # @llamaindex/core
 
+## 0.4.8
+
+### Patch Changes
+
+- f865c98: feat: async get message on chat store
+
+## 0.4.7
+
+### Patch Changes
+
+- d89ebe0: feat: better support for zod schema
+- fd8c882: chore: add warning on legacy workflow API
+
+## 0.4.6
+
+### Patch Changes
+
+- Updated dependencies [4fc001c]
+  - @llamaindex/env@0.1.20
+
+## 0.4.5
+
+### Patch Changes
+
+- ad85bd0: - fix agent chat message not saved into the task context when streaming
+  - fix async local storage might use `node:async_hook` in edge-light/workerd condition
+- Updated dependencies [ad85bd0]
+  - @llamaindex/env@0.1.19
+
+## 0.4.4
+
+### Patch Changes
+
+- Updated dependencies [a8d3fa6]
+  - @llamaindex/env@0.1.18
+
 ## 0.4.3
 
 ### Patch Changes

packages/core/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/core",
   "type": "module",
-  "version": "0.4.3",
+  "version": "0.4.8",
   "description": "LlamaIndex Core Module",
   "exports": {
     "./agent": {
@@ -392,7 +392,7 @@
     "@edge-runtime/vm": "^4.0.3",
     "ajv": "^8.17.1",
     "bunchee": "5.6.1",
-    "happy-dom": "^15.10.0",
+    "happy-dom": "^15.11.0",
     "natural": "^8.0.1"
   },
   "dependencies": {

packages/core/src/agent/base.ts

@@ -3,7 +3,7 @@ import {
   BaseChatEngine,
   type NonStreamingChatEngineParams,
   type StreamingChatEngineParams,
-} from "../chat-engine/base";
+} from "../chat-engine";
 import { wrapEventCaller } from "../decorator";
 import { Settings } from "../global";
 import type {
@@ -172,7 +172,6 @@ export abstract class AgentWorker<
       start: async (controller) => {
         for await (const stepOutput of taskOutputStream) {
           this.#taskSet.add(stepOutput.taskStep);
-          controller.enqueue(stepOutput);
           if (stepOutput.isLast) {
             let currentStep: TaskStep<
               AI,
@@ -183,7 +182,30 @@ export abstract class AgentWorker<
               this.#taskSet.delete(currentStep);
               currentStep = currentStep.prevStep;
             }
+            const { output, taskStep } = stepOutput;
+            if (output instanceof ReadableStream) {
+              const [pipStream, finalStream] = output.tee();
+              stepOutput.output = finalStream;
+              const reader = pipStream.getReader();
+              const { value } = await reader.read();
+              reader.releaseLock();
+              let content: string = value!.delta;
+              for await (const chunk of pipStream) {
+                content += chunk.delta;
+              }
+              taskStep.context.store.messages = [
+                ...taskStep.context.store.messages,
+                {
+                  role: "assistant",
+                  content,
+                  options: value!.options,
+                },
+              ];
+            }
+            controller.enqueue(stepOutput);
             controller.close();
+          } else {
+            controller.enqueue(stepOutput);
           }
         }
       },
@@ -373,10 +395,15 @@ export abstract class AgentRunner<
       if (stepOutput.isLast) {
         const { output } = stepOutput;
         if (output instanceof ReadableStream) {
-          return output.pipeThrough<EngineResponse>(
+          return output.pipeThrough(
-            new TransformStream({
+            new TransformStream<EngineResponse>({
               transform(chunk, controller) {
-                controller.enqueue(EngineResponse.fromChatResponseChunk(chunk));
+                controller.enqueue(
+                  EngineResponse.fromChatResponseChunk(
+                    chunk,
+                    chunk.sourceNodes,
+                  ),
+                );
               },
             }),
           );

packages/core/src/agent/utils.ts

@@ -79,7 +79,7 @@ export async function stepToolsStreaming<Model extends LLM>({
     for await (const chunk of pipStream) {
       if (chunk.options && "toolCall" in chunk.options) {
         const toolCall = chunk.options.toolCall;
-        toolCall.forEach((toolCall) => {
+        toolCall.forEach((toolCall: ToolCall | PartialToolCall) => {
           toolCalls.set(toolCall.id, toolCall);
         });
       }

packages/core/src/embeddings/base.ts

@@ -1,4 +1,4 @@
-import { type Tokenizers } from "@llamaindex/env";
+import type { Tokenizers } from "@llamaindex/env/tokenizers";
 import type { MessageContentDetail } from "../llms";
 import { BaseNode, MetadataMode, TransformComponent } from "../schema";
 import { extractSingleText } from "../utils";

packages/core/src/embeddings/tokenizer.ts

@@ -1,4 +1,4 @@
-import { Tokenizers, tokenizers } from "@llamaindex/env";
+import { Tokenizers, tokenizers } from "@llamaindex/env/tokenizers";
 
 export function truncateMaxTokens(
   tokenizer: Tokenizers,

packages/core/src/global/settings.ts

@@ -1,4 +1,5 @@
-import { getEnv, type Tokenizer } from "@llamaindex/env";
+import { getEnv } from "@llamaindex/env";
+import type { Tokenizer } from "@llamaindex/env/tokenizers";
 import type { LLM } from "../llms";
 import {
   type CallbackManager,

packages/core/src/global/settings/tokenizer.ts

@@ -1,4 +1,5 @@
-import { AsyncLocalStorage, type Tokenizer, tokenizers } from "@llamaindex/env";
+import { AsyncLocalStorage } from "@llamaindex/env";
+import { type Tokenizer, tokenizers } from "@llamaindex/env/tokenizers";
 
 const chunkSizeAsyncLocalStorage = new AsyncLocalStorage<Tokenizer>();
 let globalTokenizer: Tokenizer = tokenizers.tokenizer();

packages/core/src/indices/prompt-helper.ts

@@ -1,4 +1,4 @@
-import { type Tokenizer, tokenizers } from "@llamaindex/env";
+import type { Tokenizer } from "@llamaindex/env/tokenizers";
 import {
   DEFAULT_CHUNK_OVERLAP_RATIO,
   DEFAULT_CONTEXT_WINDOW,
@@ -64,7 +64,7 @@ export class PromptHelper {
     this.numOutput = numOutput;
     this.chunkOverlapRatio = chunkOverlapRatio;
     this.chunkSizeLimit = chunkSizeLimit;
-    this.tokenizer = tokenizer ?? tokenizers.tokenizer();
+    this.tokenizer = tokenizer ?? Settings.tokenizer;
     this.separator = separator;
   }
 

packages/core/src/llms/base.ts

@@ -1,5 +1,4 @@
-import { streamConverter } from "../utils";
+import { extractText, streamConverter } from "../utils";
-import { extractText } from "../utils/llms";
 import type {
   ChatResponse,
   ChatResponseChunk,

packages/core/src/llms/type.ts

@@ -1,6 +1,6 @@
-import type { Tokenizers } from "@llamaindex/env";
+import type { Tokenizers } from "@llamaindex/env/tokenizers";
 import type { JSONSchemaType } from "ajv";
-import type { JSONObject, JSONValue } from "../global/type";
+import type { JSONObject, JSONValue } from "../global";
 
 /**
  * @internal

packages/core/src/memory/base.ts

@@ -65,7 +65,9 @@ export abstract class BaseChatStoreMemory<
     super();
   }
 
-  getAllMessages(): ChatMessage<AdditionalMessageOptions>[] {
+  getAllMessages():
+    | ChatMessage<AdditionalMessageOptions>[]
+    | Promise<ChatMessage<AdditionalMessageOptions>[]> {
     return this.chatStore.getMessages(this.chatStoreKey);
   }
 

packages/core/src/memory/chat-memory-buffer.ts

@@ -33,11 +33,11 @@ export class ChatMemoryBuffer<
     }
   }
 
-  getMessages(
+  async getMessages(
     transientMessages?: ChatMessage<AdditionalMessageOptions>[] | undefined,
     initialTokenCount: number = 0,
   ) {
-    const messages = this.getAllMessages();
+    const messages = await this.getAllMessages();
 
     if (initialTokenCount > this.tokenLimit) {
       throw new Error("Initial token count exceeds token limit");

packages/core/src/memory/summary-memory.ts

@@ -1,4 +1,4 @@
-import { type Tokenizer, tokenizers } from "@llamaindex/env";
+import { type Tokenizer, tokenizers } from "@llamaindex/env/tokenizers";
 import { Settings } from "../global";
 import type { ChatMessage, LLM, MessageType } from "../llms";
 import { defaultSummaryPrompt, type SummaryPrompt } from "../prompts";

packages/core/src/node-parser/sentence-splitter.ts

@@ -1,4 +1,4 @@
-import type { Tokenizer } from "@llamaindex/env";
+import type { Tokenizer } from "@llamaindex/env/tokenizers";
 import { z } from "zod";
 import { Settings } from "../global";
 import { sentenceSplitterSchema } from "../schema";

packages/core/src/node-parser/token-text-splitter.ts

@@ -1,4 +1,4 @@
-import type { Tokenizer } from "@llamaindex/env";
+import type { Tokenizer } from "@llamaindex/env/tokenizers";
 import { z } from "zod";
 import { DEFAULT_CHUNK_OVERLAP, DEFAULT_CHUNK_SIZE, Settings } from "../global";
 import { MetadataAwareTextSplitter } from "./base";

packages/core/src/node-parser/type.ts

@@ -1,4 +1,4 @@
-import type { Tokenizer } from "@llamaindex/env";
+import type { Tokenizer } from "@llamaindex/env/tokenizers";
 
 export type SplitterParams = {
   tokenizer?: Tokenizer;

packages/core/src/storage/chat-store/base-chat-store.ts

@@ -7,7 +7,11 @@ export abstract class BaseChatStore<
     key: string,
     messages: ChatMessage<AdditionalMessageOptions>[],
   ): void;
-  abstract getMessages(key: string): ChatMessage<AdditionalMessageOptions>[];
+  abstract getMessages(
+    key: string,
+  ):
+    | ChatMessage<AdditionalMessageOptions>[]
+    | Promise<ChatMessage<AdditionalMessageOptions>[]>;
   abstract addMessage(
     key: string,
     message: ChatMessage<AdditionalMessageOptions>,

packages/core/src/tools/function-tool.ts

@@ -4,18 +4,12 @@ import { zodToJsonSchema } from "zod-to-json-schema";
 import type { JSONValue } from "../global";
 import type { BaseTool, ToolMetadata } from "../llms";
 
-const kOriginalFn = Symbol("originalFn");
-
 export class FunctionTool<T, R extends JSONValue | Promise<JSONValue>>
   implements BaseTool<T>
 {
-  [kOriginalFn]?: (input: T) => R;
-
   #fn: (input: T) => R;
-  #metadata: ToolMetadata<JSONSchemaType<T>>;
+  readonly #metadata: ToolMetadata<JSONSchemaType<T>>;
-  // todo: for the future, we can use zod to validate the input parameters
+  readonly #zodType: z.ZodType<T> | null = null;
-  // eslint-disable-next-line no-unused-private-class-members
-  #zodType: z.ZodType<T> | null = null;
   constructor(
     fn: (input: T) => R,
     metadata: ToolMetadata<JSONSchemaType<T>>,
@@ -32,6 +26,12 @@ export class FunctionTool<T, R extends JSONValue | Promise<JSONValue>>
     fn: (input: T) => JSONValue | Promise<JSONValue>,
     schema: ToolMetadata<JSONSchemaType<T>>,
   ): FunctionTool<T, JSONValue | Promise<JSONValue>>;
+  static from<R extends z.ZodType>(
+    fn: (input: z.infer<R>) => JSONValue | Promise<JSONValue>,
+    schema: Omit<ToolMetadata, "parameters"> & {
+      parameters: R;
+    },
+  ): FunctionTool<z.infer<R>, JSONValue | Promise<JSONValue>>;
   static from<T, R extends z.ZodType<T>>(
     fn: (input: T) => JSONValue | Promise<JSONValue>,
     schema: Omit<ToolMetadata, "parameters"> & {
@@ -40,15 +40,15 @@ export class FunctionTool<T, R extends JSONValue | Promise<JSONValue>>
   ): FunctionTool<T, JSONValue>;
   // eslint-disable-next-line @typescript-eslint/no-explicit-any
   static from(fn: any, schema: any): any {
-    if (schema.parameter instanceof z.ZodSchema) {
+    if (schema.parameters instanceof z.ZodSchema) {
-      const jsonSchema = zodToJsonSchema(schema.parameter);
+      const jsonSchema = zodToJsonSchema(schema.parameters);
       return new FunctionTool(
         fn,
         {
           ...schema,
           parameters: jsonSchema,
         },
-        schema.parameter,
+        schema.parameters,
       );
     }
     return new FunctionTool(fn, schema);
@@ -58,7 +58,15 @@ export class FunctionTool<T, R extends JSONValue | Promise<JSONValue>>
     return this.#metadata as BaseTool<T>["metadata"];
   }
 
-  call(input: T) {
+  call = (input: T) => {
+    if (this.#zodType) {
+      const result = this.#zodType.safeParse(input);
+      if (result.success) {
+        return this.#fn.call(null, result.data);
+      } else {
+        console.warn(result.error.errors);
+      }
+    }
     return this.#fn.call(null, input);
-  }
+  };
 }

packages/core/src/workflow/workflow.ts

@@ -13,6 +13,8 @@ export type StepFunction<T extends WorkflowEvent = WorkflowEvent> = (
 
 type EventTypeParam = EventTypes | EventTypes[];
 
+let once = false;
+
 export class Workflow {
   #steps: Map<
     // eslint-disable-next-line @typescript-eslint/no-explicit-any
@@ -29,8 +31,20 @@ export class Workflow {
       verbose?: boolean;
       timeout?: number;
       validate?: boolean;
+      ignoreDeprecatedWarning?: boolean;
     } = {},
   ) {
+    if (!once && !params.ignoreDeprecatedWarning) {
+      console.warn(
+        "@llamaindex/core/workflow is going to use the new workflow API in the next major version.",
+        "Please update your imports to @llamaindex/workflow",
+      );
+      console.warn(
+        "See https://ts.llamaindex.ai/docs/llamaindex/guide/workflow for more information",
+      );
+      once = true;
+    }
+
     this.#verbose = params.verbose ?? false;
     this.#timeout = params.timeout ?? null;
     this.#validate = params.validate ?? false;

packages/core/tests/embeddings.test.ts

@@ -1,5 +1,5 @@
 import { truncateMaxTokens } from "@llamaindex/core/embeddings";
-import { Tokenizers, tokenizers } from "@llamaindex/env";
+import { Tokenizers, tokenizers } from "@llamaindex/env/tokenizers";
 import { describe, expect, test } from "vitest";
 
 describe("truncateMaxTokens", () => {

packages/core/tests/memory/chat-memory-buffer.test.ts

@@ -19,7 +19,7 @@ describe("ChatMemoryBuffer", () => {
     expect(buffer.tokenLimit).toBe(500);
   });
 
-  test("getMessages returns all messages when under token limit", () => {
+  test("getMessages returns all messages when under token limit", async () => {
     const messages: ChatMessage[] = [
       { role: "user", content: "Hello" },
       { role: "assistant", content: "Hi there!" },
@@ -30,11 +30,11 @@ describe("ChatMemoryBuffer", () => {
       chatHistory: messages,
     });
 
-    const result = buffer.getMessages();
+    const result = await buffer.getMessages();
     expect(result).toEqual(messages);
   });
 
-  test("getMessages truncates messages when over token limit", () => {
+  test("getMessages truncates messages when over token limit", async () => {
     const messages: ChatMessage[] = [
       { role: "user", content: "This is a long message" },
       { role: "assistant", content: "This is also a long reply" },
@@ -45,11 +45,11 @@ describe("ChatMemoryBuffer", () => {
       chatHistory: messages,
     });
 
-    const result = buffer.getMessages();
+    const result = await buffer.getMessages();
     expect(result).toEqual([{ role: "user", content: "Short" }]);
   });
 
-  test("getMessages handles input messages", () => {
+  test("getMessages handles input messages", async () => {
     const storedMessages: ChatMessage[] = [
       { role: "user", content: "Hello" },
       { role: "assistant", content: "Hi there!" },
@@ -62,13 +62,13 @@ describe("ChatMemoryBuffer", () => {
     const inputMessages: ChatMessage[] = [
       { role: "user", content: "New message" },
     ];
-    const result = buffer.getMessages(inputMessages);
+    const result = await buffer.getMessages(inputMessages);
     expect(result).toEqual([...inputMessages, ...storedMessages]);
   });
 
   test("getMessages throws error when initial token count exceeds limit", () => {
     const buffer = new ChatMemoryBuffer({ tokenLimit: 10 });
-    expect(() => buffer.getMessages(undefined, 20)).toThrow(
+    expect(async () => buffer.getMessages(undefined, 20)).rejects.toThrow(
       "Initial token count exceeds token limit",
     );
   });

packages/core/tests/node-parser/sentence-spiller.test.ts

@@ -1,6 +1,6 @@
 import { SentenceSplitter } from "@llamaindex/core/node-parser";
 import { Document } from "@llamaindex/core/schema";
-import { tokenizers } from "@llamaindex/env";
+import { tokenizers } from "@llamaindex/env/tokenizers";
 import { beforeEach, describe, expect, test } from "vitest";
 
 describe("SentenceSplitter", () => {

packages/core/tsconfig.json

@@ -8,7 +8,6 @@
     "moduleResolution": "Bundler",
     "skipLibCheck": true,
     "strict": true,
-    "lib": ["ESNext", "DOM", "DOM.AsyncIterable"],
     "types": ["node"]
   },
   "include": ["./src"],

packages/env/CHANGELOG.md

@@ -1,5 +1,26 @@
 # @llamaindex/env
 
+## 0.1.20
+
+### Patch Changes
+
+- 4fc001c: chore: bump `@huggingface/transformers`
+
+  Upgrade to v3, please read https://github.com/huggingface/transformers.js/releases/tag/3.0.0 for more information.
+
+## 0.1.19
+
+### Patch Changes
+
+- ad85bd0: - fix agent chat message not saved into the task context when streaming
+  - fix async local storage might use `node:async_hook` in edge-light/workerd condition
+
+## 0.1.18
+
+### Patch Changes
+
+- a8d3fa6: fix: exports in package.json
+
 ## 0.1.17
 
 ### Patch Changes

packages/env/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@llamaindex/env",
   "description": "environment wrapper, supports all JS environment including node, deno, bun, edge runtime, and cloudflare worker",
-  "version": "0.1.17",
+  "version": "0.1.20",
   "type": "module",
   "types": "dist/index.d.ts",
   "module": "dist/index.js",
@@ -51,34 +51,61 @@
         "default": "./dist/index.cjs"
       }
     },
-    "./multi-model": {
+    "./tokenizers": {
       "workerd": {
-        "types": "./multi-model/dist/multi-model.workerd.d.ts",
+        "types": "./tokenizers/dist/index.workerd.d.ts",
-        "default": "./multi-model/dist/multi-model.workerd.js"
+        "default": "./tokenizers/dist/index.workerd.js"
       },
       "edge-light": {
-        "types": "./multi-model/dist/multi-model.edge-light.d.ts",
+        "types": "./tokenizers/dist/index.edge-light.d.ts",
-        "default": "./multi-model/dist/multi-model.edge-light.js"
+        "default": "./tokenizers/dist/index.edge-light.js"
       },
       "browser": {
-        "types": "./multi-model/dist/multi-model.browser.d.ts",
+        "types": "./tokenizers/dist/index.browser.d.ts",
-        "default": "./multi-model/dist/multi-model.browser.js"
+        "default": "./tokenizers/dist/index.browser.js"
       },
       "import": {
-        "types": "./multi-model/dist/multi-model.d.ts",
+        "types": "./tokenizers/dist/index.d.ts",
-        "default": "./multi-model/dist/multi-model.js"
+        "default": "./tokenizers/dist/index.js"
       },
       "require": {
-        "types": "./multi-model/dist/multi-model.d.cts",
+        "types": "./tokenizers/dist/index.d.cts",
-        "default": "./multi-model/dist/multi-model.cjs"
+        "default": "./tokenizers/dist/index.cjs"
       },
       "default": {
-        "types": "./multi-model/dist/multi-model.d.ts",
+        "types": "./tokenizers/dist/index.d.ts",
-        "default": "./multi-model/dist/multi-model.js"
+        "default": "./tokenizers/dist/index.js"
+      }
+    },
+    "./multi-model": {
+      "workerd": {
+        "types": "./multi-model/dist/index.workerd.d.ts",
+        "default": "./multi-model/dist/index.workerd.js"
+      },
+      "edge-light": {
+        "types": "./multi-model/dist/index.edge-light.d.ts",
+        "default": "./multi-model/dist/index.edge-light.js"
+      },
+      "browser": {
+        "types": "./multi-model/dist/index.browser.d.ts",
+        "default": "./multi-model/dist/index.browser.js"
+      },
+      "import": {
+        "types": "./multi-model/dist/index.d.ts",
+        "default": "./multi-model/dist/index.js"
+      },
+      "require": {
+        "types": "./multi-model/dist/index.d.cts",
+        "default": "./multi-model/dist/index.cjs"
+      },
+      "default": {
+        "types": "./multi-model/dist/index.d.ts",
+        "default": "./multi-model/dist/index.js"
       }
     }
   },
   "files": [
+    "tokenizers",
     "multi-model",
     "dist",
     "CHANGELOG.md",
@@ -97,7 +124,7 @@
   "devDependencies": {
     "@types/node": "^22.9.0",
     "@types/readable-stream": "^4.0.15",
-    "@xenova/transformers": "^2.17.2",
+    "@huggingface/transformers": "^3.0.2",
     "bunchee": "5.6.1",
     "gpt-tokenizer": "^2.6.0",
     "pathe": "^1.1.2",
@@ -105,7 +132,7 @@
   },
   "peerDependencies": {
     "@aws-crypto/sha256-js": "^5.2.0",
-    "@xenova/transformers": "^2.17.2",
+    "@huggingface/transformers": "^3.0.2",
     "gpt-tokenizer": "^2.5.0",
     "js-tiktoken": "^1.0.12",
     "pathe": "^1.1.2"
@@ -114,7 +141,7 @@
     "@aws-crypto/sha256-js": {
       "optional": true
     },
-    "@xenova/transformers": {
+    "@huggingface/transformers": {
       "optional": true
     },
     "pathe": {

packages/env/src/als/index.node.ts

@@ -0,0 +1 @@
+export { AsyncLocalStorage } from "node:async_hooks";

packages/env/src/als/index.non-node.ts

@@ -0,0 +1,3 @@
+// Async Local Storage is available cross different JS runtimes
+// @ts-expect-error AsyncLocalStorage is not defined in Non Node.js environment
+export const AsyncLocalStorage = globalThis.AsyncLocalStorage;

packages/env/src/als/index.web.ts

@@ -0,0 +1,32 @@
+// Web doesn't have AsyncLocalStorage and there's no alternative way to implement it
+// Wait for https://github.com/tc39/proposal-async-context
+export class AsyncLocalStorage<T> {
+  #store: T = null!;
+
+  // eslint-disable-next-line @typescript-eslint/no-explicit-any
+  static bind<Func extends (...args: any[]) => any>(fn: Func): Func {
+    return fn;
+  }
+
+  // eslint-disable-next-line @typescript-eslint/no-explicit-any
+  static snapshot(): <R, TArgs extends any[]>(
+    fn: (...args: TArgs) => R,
+    ...args: TArgs
+  ) => R {
+    // eslint-disable-next-line @typescript-eslint/no-explicit-any
+    return (cb: any, ...args: any[]) => cb(...args);
+  }
+
+  getStore() {
+    return this.#store;
+  }
+
+  run<R>(store: T, cb: () => R): R {
+    this.#store = store;
+    if (cb.constructor.name === "AsyncFunction") {
+      console.warn("AsyncLocalStorage is not supported in the web environment");
+      console.warn("Please note that some features may not work as expected");
+    }
+    return cb();
+  }
+}

packages/env/src/index.browser.ts

@@ -5,11 +5,10 @@
  */
 import "./global-check.js";
 
+export * from "./als/index.web.js";
 export { consoleLogger, emptyLogger, type Logger } from "./logger/index.js";
-export { Tokenizers, tokenizers, type Tokenizer } from "./tokenizers/js.js";
 export { NotSupportCurrentRuntimeClass } from "./utils/shared.js";
 export * from "./web-polyfill.js";
-// @ts-expect-error no type
 if (typeof window === "undefined") {
   console.warn(
     "You are not in a browser environment. This module is not supposed to be used in a non-browser environment.",

packages/env/src/index.edge-light.ts

@@ -3,8 +3,8 @@
  *
  * @module
  */
-import "./global-check.js";
+
+export * from "./als/index.non-node.js";
 export { consoleLogger, emptyLogger, type Logger } from "./logger/index.js";
 export * from "./node-polyfill.js";
-export { Tokenizers, tokenizers, type Tokenizer } from "./tokenizers/js.js";
 export { NotSupportCurrentRuntimeClass } from "./utils/shared.js";

packages/env/src/index.ts

@@ -34,14 +34,9 @@ export function createSHA256(): SHA256 {
   };
 }
 
+export * from "./als/index.node.js";
 export { consoleLogger, emptyLogger, type Logger } from "./logger/index.js";
-export { Tokenizers, tokenizers, type Tokenizer } from "./tokenizers/node.js";
+export { CustomEvent, getEnv, setEnvs } from "./utils/index.js";
-export {
-  AsyncLocalStorage,
-  CustomEvent,
-  getEnv,
-  setEnvs,
-} from "./utils/index.js";
 export { NotSupportCurrentRuntimeClass } from "./utils/shared.js";
 export {
   EOL,

packages/env/src/index.workerd.ts

@@ -7,6 +7,7 @@
  */
 import { INTERNAL_ENV } from "./utils/index.js";
 
+export * from "./als/index.non-node.js";
 export { NotSupportCurrentRuntimeClass } from "./utils/shared.js";
 
 export * from "./node-polyfill.js";
@@ -16,4 +17,3 @@ export function getEnv(name: string): string | undefined {
 }
 
 export { consoleLogger, emptyLogger, type Logger } from "./logger/index.js";
-export { Tokenizers, tokenizers, type Tokenizer } from "./tokenizers/js.js";

packages/env/src/internal/multi-model/browser.ts

@@ -8,8 +8,10 @@ export {
 export async function loadTransformers(onLoad: OnLoad) {
   if (getTransformers() === null) {
     setTransformers(
-      // @ts-expect-error no type
+      await import(
-      await import("https://cdn.jsdelivr.net/npm/@xenova/transformers@2.17.2"),
+        // @ts-expect-error no type
+        "https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.0.2"
+      ),
     );
   } else {
     return getTransformers()!;

packages/env/src/internal/multi-model/node.ts

@@ -8,7 +8,7 @@ export {
 
 export async function loadTransformers(onLoad: OnLoad) {
   if (getTransformers() === null) {
-    setTransformers(await import("@xenova/transformers"));
+    setTransformers(await import("@huggingface/transformers"));
   } else {
     return getTransformers()!;
   }

packages/env/src/internal/multi-model/non-node.ts

@@ -9,7 +9,7 @@ export async function loadTransformers(onLoad: OnLoad) {
   if (getTransformers() === null) {
     /**
      * If you see this warning, it means that the current environment does not support the transformer.
-     *  because "@xeonva/transformers" highly depends on Node.js APIs.
+     *  because "@huggingface/transformers" highly depends on Node.js APIs.
      *
      * One possible solution is to fix their implementation to make it work in the non-Node.js environment,
      *  but it's not worth the effort because Edge Runtime and Cloudflare Workers are not the for heavy Machine Learning task.
@@ -17,14 +17,14 @@ export async function loadTransformers(onLoad: OnLoad) {
      * Or you can provide an RPC server that runs the transformer in a Node.js environment.
      * Or you just run the code in a Node.js environment.
      *
-     * Refs: https://github.com/xenova/transformers.js/issues/309
+     * Refs: https://github.com/huggingface/transformers.js/issues/309
      */
     console.warn(
-      '"@xenova/transformers" is not officially supported in this environment, some features may not work as expected.',
+      '"@huggingface/transformers" is not officially supported in this environment, some features may not work as expected.',
     );
     setTransformers(
       // @ts-expect-error no type
-      await import("@xenova/transformers/dist/transformers"),
+      await import("@huggingface/transformers/dist/transformers.js"),
     );
   } else {
     return getTransformers()!;

packages/env/src/internal/multi-model/shared.ts

@@ -1,17 +1,17 @@
-let transformer: typeof import("@xenova/transformers") | null = null;
+let transformer: typeof import("@huggingface/transformers") | null = null;
 
 export function getTransformers() {
   return transformer;
 }
 
-export function setTransformers(t: typeof import("@xenova/transformers")) {
+export function setTransformers(t: typeof import("@huggingface/transformers")) {
   transformer = t;
 }
 
 export type OnLoad = (
-  transformer: typeof import("@xenova/transformers"),
+  transformer: typeof import("@huggingface/transformers"),
 ) => void;
 
 export type LoadTransformerEvent = {
-  transformer: typeof import("@xenova/transformers");
+  transformer: typeof import("@huggingface/transformers");
 };

packages/env/src/internal/tokenizers/js.ts

@@ -1,4 +1,4 @@
-// Note: js-tiktoken it's 60x slower than the WASM implementation - use it only for unsupported environments
+// Note: js-tiktoken it's 60x slower than gpt-tokenizer
 import { getEncoding } from "js-tiktoken";
 import type { Tokenizer } from "./types.js";
 import { Tokenizers } from "./types.js";

packages/env/src/internal/tokenizers/node.ts

@@ -1,4 +1,3 @@
-// Note: This is using th WASM implementation of tiktoken which is 60x faster
 import type { Tokenizer } from "./types.js";
 import { Tokenizers } from "./types.js";
 

packages/env/src/node-polyfill.ts

@@ -56,9 +56,4 @@ export const process: NodeJS.Process = globalThis.process ?? {
   versions: {},
 };
 
-export {
+export { CustomEvent, getEnv, setEnvs } from "./utils/index.js";
-  AsyncLocalStorage,
-  CustomEvent,
-  getEnv,
-  setEnvs,
-} from "./utils/index.js";

packages/env/src/tokenizers.browser.ts

@@ -0,0 +1,5 @@
+export {
+  Tokenizers,
+  tokenizers,
+  type Tokenizer,
+} from "./internal/tokenizers/js.js";

packages/env/src/tokenizers.edge-light.ts

@@ -0,0 +1,5 @@
+export {
+  Tokenizers,
+  tokenizers,
+  type Tokenizer,
+} from "./internal/tokenizers/js.js";

packages/env/src/tokenizers.ts

@@ -0,0 +1,5 @@
+export {
+  Tokenizers,
+  tokenizers,
+  type Tokenizer,
+} from "./internal/tokenizers/node.js";

packages/env/src/tokenizers.workerd.ts

@@ -0,0 +1,5 @@
+export {
+  Tokenizers,
+  tokenizers,
+  type Tokenizer,
+} from "./internal/tokenizers/js.js";

packages/env/src/utils/index.ts

@@ -56,9 +56,6 @@ interface CustomEventInit<T = any> extends EventInit {
   detail?: T;
 }
 
-// Async Local Storage is available cross different JS runtimes
-export { AsyncLocalStorage } from "node:async_hooks";
-
 // Node.js 18 doesn't have CustomEvent by default
 // Refs: https://github.com/nodejs/node/issues/40678
 // eslint-disable-next-line @typescript-eslint/no-explicit-any