* refactor: merge HistoryChatEngine and ContextChatEngine and use ChatHistory for all chat engines
* fix: add safeguard for tokensToSummarize
* refactor: unfold chat engines to own folder
* refactor: extract LLM types
* refactor: move multi-modal types to llm
* docs(changeset): Remove HistoryChatEngine and use ChatHistory for all chat engines
* dev: add debug launcher and don't lint generated code
- expanded LLM interface
- cleaned up OpenAI LLM stream
- simplified types in CallbackManager
-> CallbackResponse should require Token I think, we already have the StreamToken inside of the LLM's stream_chat anyways
- SubQuestionOutputParser is not working as expected, writing some tests to check it out.
QuestionGenerator outputs a list of objects in string format, which is unexpected.
In particular:
"[{prompt: "smth", response: "nothing"}]"
Allows our scripts to be run repeatedly with vector store backing and
persistence without throwing an error.
This is rudimentary support which doesn't work if the document has
changed over time.
Also added the insert and delete functions so that documents can be
added manually.
In theory this will stop the API docs thrashing on every build.
Of course, there is a drawback in that if main changes but we haven't
released a new version of the docs yet the links will go out of date.
So longer term we might want to investigate some kind of variable where
we can continue to have up to date revs but keep the rev in a single
variable somewhere:
https://github.com/facebook/docusaurus/issues/395
In the meantime much smaller commits will be a relief
description:Write something like "Modify the ... api endpoint to use ... version and ... framework" If you would like to use sweep.dev prefix with "Sweep:"
body:
- type:textarea
id:description
attributes:
label:Details
description:More details
placeholder:We are migrating this function to ... version because ...
packages/core which is the main NPM library llamaindex
examples is where the demo code lives
### Turborepo docs
You can checkout how Turborepo works using the default [README-turborepo.md](/README-turborepo.md)
## Getting Started
Install NodeJS. Preferably v18 using nvm or n.
Inside the LlamaIndexTS directory:
```
npm i -g pnpm ts-node
pnpm install
```
Note: we use pnpm in this repo, which has a lot of the same functionality and CLI options as npm but it does do some things better in a monorepo, like centralizing dependencies and caching.
PNPM's has documentation on its [workspace feature](https://pnpm.io/workspaces) and Turborepo had some [useful documentation also](https://turbo.build/repo/docs/core-concepts/monorepos/running-tasks).
### Running Typescript
When we publish to NPM we will have a tsc compiled version of the library in JS. For now, the easiest thing to do is use ts-node.
### Test cases
To run them, run
```
pnpm run test
```
To write new test cases write them in [packages/core/src/tests](/packages/core/src/tests)
We use Jest https://jestjs.io/ to write our test cases. Jest comes with a bunch of built in assertions using the expect function: https://jestjs.io/docs/expect
### Demo applications
There is an existing ["example"](/examples/README.md) demos folder with mainly NodeJS scripts. Feel free to add additional demos to that folder. If you would like to try out your changes in the core package with a new demo, you need to run the build command in the README.
You can create new demo applications in the apps folder. Just run pnpm init in the folder after you create it to create its own package.json
### Installing packages
To install packages for a specific package or demo application, run
```
pnpm add [NPM Package] --filter [package or application i.e. core or docs]
```
To install packages for every package or application run
```
pnpm add -w [NPM Package]
```
### Docs
To contribute to the docs, go to the docs website folder and run the Docusaurus instance.
```bash
cd apps/docs
pnpm install
pnpm start
```
That should start a webserver which will serve the docs on https://localhost:3000
Any changes you make should be reflected in the browser. If you need to regenerate the API docs and find that your TSDoc isn't getting the updates, feel free to remove apps/docs/api. It will automatically regenerate itself when you run pnpm start again.
## Changeset
We use [changesets](https://github.com/changesets/changesets) for managing versions and changelogs. To create a new changeset, run:
```
pnpm changeset
```
Please send a descriptive changeset for each PR.
## Publishing (maintainers only)
The [Release Github Action](.github/workflows/release.yml) is automatically generating and updating a
PR called "Release {version}".
This PR will update the `package.json` and `CHANGELOG.md` files of each package according to
the current changesets in the [.changeset](.changeset/) folder.
If this PR is merged it will automatically add version tags to the repository and publish the updated packages to NPM.
This Turborepo includes the following packages/apps:
### Apps and Packages
-`docs`: a [Next.js](https://nextjs.org/) app
-`web`: another [Next.js](https://nextjs.org/) app
-`ui`: a stub React component library shared by both `web` and `docs` applications
-`eslint-config-custom`: `eslint` configurations (includes `eslint-config-next` and `eslint-config-prettier`)
-`tsconfig`: `tsconfig.json`s used throughout the monorepo
Each package/app is 100% [TypeScript](https://www.typescriptlang.org/).
### Utilities
This Turborepo has some additional tools already setup for you:
- [TypeScript](https://www.typescriptlang.org/) for static type checking
- [ESLint](https://eslint.org/) for code linting
- [Prettier](https://prettier.io) for code formatting
### Build
To build all apps and packages, run the following command:
```
cd my-turborepo
pnpm build
```
### Develop
To develop all apps and packages, run the following command:
```
cd my-turborepo
pnpm dev
```
### Remote Caching
Turborepo can use a technique known as [Remote Caching](https://turbo.build/repo/docs/core-concepts/remote-caching) to share cache artifacts across machines, enabling you to share build caches with your team and CI/CD pipelines.
By default, Turborepo will cache locally. To enable Remote Caching you will need an account with Vercel. If you don't have an account you can [create one](https://vercel.com/signup), then enter the following commands:
```
cd my-turborepo
npx turbo login
```
This will authenticate the Turborepo CLI with your [Vercel account](https://vercel.com/docs/concepts/personal-accounts/overview).
Next, you can link your Turborepo to your Remote Cache by running the following command from the root of your Turborepo:
LlamaIndex is a data framework for your LLM application.
Right now there are two packages of importance:
Use your own data with large language models (LLMs, OpenAI ChatGPT and others) in Typescript and Javascript.
packages/core which is the main NPM library @llamaindex/core
Documentation: https://ts.llamaindex.ai/
apps/simple is where the demo code lives
Try examples online:
### Turborepo docs
[](https://stackblitz.com/github/run-llama/LlamaIndexTS/tree/main/examples)
You can checkout how Turborepo works using the built in [README-turborepo.md](README-turborepo.md)
## What is LlamaIndex.TS?
## Getting Started
LlamaIndex.TS aims to be a lightweight, easy to use set of libraries to help you integrate large language models into your applications with your own data.
For now, browser support is limited due to the lack of support for [AsyncLocalStorage-like APIs](https://github.com/tc39/proposal-async-context)
## Getting started
```shell
npm install llamaindex
pnpm install llamaindex
yarn add llamaindex
jsr install @llamaindex/core
```
Note: we use pnpm in this repo, which has a lot of the same functionality and CLI options as npm but it does do some things better in a monorepo, like centralizing dependencies and caching.
### Node.js
PNPM's has documentation on its [workspace feature](https://pnpm.io/workspaces) and Turborepo had some [useful documentation also](https://turbo.build/repo/docs/core-concepts/monorepos/running-tasks).
To write new test cases write them in packages/core/src/tests
We use Jest https://jestjs.io/ to write our test cases. Jest comes with a bunch of built in assertions using the expect function: https://jestjs.io/docs/expect
### Demo applications
You can create new demo applications in the apps folder. Just run pnpm init in the folder after you create it to create its own package.json
### Installing packages
To install packages for a specific package or demo application, run
```
pnpm add [NPM Package] --filter [package or application i.e. core or simple]
```bash
# `pnpm install tsx` before running the script
node --import tsx ./main.ts
```
To install packages for every package or application run
### Next.js
You can combine `ai` with `llamaindex` in Next.js with RSC (React Server Components).
```tsx
// src/apps/page.tsx
"use client";
import{chatWithAgent}from"@/actions";
importtype{JSX}from"react";
import{useFormState}from"react-dom";
// You can use the Edge runtime in Next.js by adding this line:
Check out our NextJS playground at https://llama-playground.vercel.app/. The source is available at https://github.com/run-llama/ts-playground
## Core concepts for getting started:
- [Document](/packages/core/src/Node.ts): A document represents a text file, PDF file or other contiguous piece of data.
- [Node](/packages/core/src/Node.ts): The basic data building block. Most commonly, these are parts of the document split into manageable pieces that are small enough to be fed into an embedding model and LLM.
- [Embedding](/packages/core/src/embeddings/OpenAIEmbedding.ts): Embeddings are sets of floating point numbers which represent the data in a Node. By comparing the similarity of embeddings, we can derive an understanding of the similarity of two pieces of data. One use case is to compare the embedding of a question with the embeddings of our Nodes to see which Nodes may contain the data needed to answer that quesiton. Because the default service context is OpenAI, the default embedding is `OpenAIEmbedding`. If using different models, say through Ollama, use this [Embedding](/packages/core/src/embeddings/OllamaEmbedding.ts) (see all [here](/packages/core/src/embeddings)).
- [Indices](/packages/core/src/indices/): Indices store the Nodes and the embeddings of those nodes. QueryEngines retrieve Nodes from these Indices using embedding similarity.
- [QueryEngine](/packages/core/src/engines/query/RetrieverQueryEngine.ts): Query engines are what generate the query you put in and give you back the result. Query engines generally combine a pre-built prompt with selected Nodes from your Index to give the LLM the context it needs to answer your query. To build a query engine from your Index (recommended), use the [`asQueryEngine`](/packages/core/src/indices/BaseIndex.ts) method on your Index. See all query engines [here](/packages/core/src/engines/query).
- [ChatEngine](/packages/core/src/engines/chat/SimpleChatEngine.ts): A ChatEngine helps you build a chatbot that will interact with your Indices. See all chat engines [here](/packages/core/src/engines/chat).
- [SimplePrompt](/packages/core/src/Prompt.ts): A simple standardized function call definition that takes in inputs and formats them in a template literal. SimplePrompts can be specialized using currying and combined using other SimplePrompt functions.
## Tips when using in non-Node.js environments
When you are importing `llamaindex` in a non-Node.js environment(such as React Server Components, Cloudflare Workers, etc.)
Some classes are not exported from top-level entry file.
The reason is that some classes are only compatible with Node.js runtime,(e.g. `PDFReader`) which uses Node.js specific APIs(like `fs`, `child_process`, `crypto`).
If you need any of those classes, you have to import them instead directly though their file path in the package.
Here's an example for importing the `PineconeVectorStore` class:
Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.
You can start editing the page by modifying `pages/index.js`. The page auto-updates as you edit the file.
[API routes](https://nextjs.org/docs/api-routes/introduction) can be accessed on [http://localhost:3000/api/hello](http://localhost:3000/api/hello). This endpoint can be edited in `pages/api/hello.js`.
The `pages/api` directory is mapped to `/api/*`. Files in this directory are treated as [API routes](https://nextjs.org/docs/api-routes/introduction) instead of React pages.
## Learn More
To learn more about Next.js, take a look at the following resources:
- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
- [Learn Next.js](https://nextjs.org/learn/foundations/about-nextjs) - an interactive Next.js tutorial.
You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js/) - your feedback and contributions are welcome!
## Deploy on Vercel
The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_source=github.com&utm_medium=referral&utm_campaign=turborepo-readme) from the creators of Next.js.
Check out our [Next.js deployment documentation](https://nextjs.org/docs/deployment) for more details.
import CodeSource from "!raw-loader!../../../../examples/chatEngine";
# Chat Engine
Chat Engine is a class that allows you to create a chatbot from a retriever. It is a wrapper around a retriever that allows you to chat with it in a conversational manner.
LlamaIndex.TS helps you build LLM-powered applications (e.g. Q&A, chatbot) over custom data.
In this high-level concepts guide, you will learn:
- how an LLM can answer questions using your own data.
- key concepts and modules in LlamaIndex.TS for composing your own query pipeline.
## Answering Questions Across Your Data
LlamaIndex uses a two stage method when using an LLM with your data:
1.**indexing stage**: preparing a knowledge base, and
2.**querying stage**: retrieving relevant context from the knowledge to assist the LLM in responding to a question
This process is also known as Retrieval Augmented Generation (RAG).
LlamaIndex.TS provides the essential toolkit for making both steps super easy.
Let's explore each stage in detail.
### Indexing Stage
LlamaIndex.TS help you prepare the knowledge base with a suite of data connectors and indexes.
[**Data Loaders**](../modules/data_loader.md):
A data connector (i.e. `Reader`) ingest data from different data sources and data formats into a simple `Document` representation (text and simple metadata).
[**Documents / Nodes**](../modules/documents_and_nodes/index.md): A `Document` is a generic container around any data source - for instance, a PDF, an API output, or retrieved data from a database. A `Node` is the atomic unit of data in LlamaIndex and represents a "chunk" of a source `Document`. It's a rich representation that includes metadata and relationships (to other nodes) to enable accurate and expressive retrieval operations.
[**Data Indexes**](../modules/data_index.md):
Once you've ingested your data, LlamaIndex helps you index data into a format that's easy to retrieve.
Under the hood, LlamaIndex parses the raw documents into intermediate representations, calculates vector embeddings, and stores your data in-memory or to disk.
### Querying Stage
In the querying stage, the query pipeline retrieves the most relevant context given a user query,
and pass that to the LLM (along with the query) to synthesize a response.
This gives the LLM up-to-date knowledge that is not in its original training data,
(also reducing hallucination).
The key challenge in the querying stage is retrieval, orchestration, and reasoning over (potentially many) knowledge bases.
LlamaIndex provides composable modules that help you build and integrate RAG pipelines for Q&A (query engine), chatbot (chat engine), or as part of an agent.
These building blocks can be customized to reflect ranking preferences, as well as composed to reason over multiple knowledge bases in a structured way.
#### Building Blocks
[**Retrievers**](../modules/retriever.md):
A retriever defines how to efficiently retrieve relevant context from a knowledge base (i.e. index) when given a query.
The specific retrieval logic differs for difference indices, the most popular being dense retrieval against a vector index.
The easiest way to get started with LlamaIndex is by using `create-llama`. This CLI tool enables you to quickly start building a new LlamaIndex application, with everything set up for you.
Just run
<Tabs>
<TabItem value="1" label="npm" default>
```bash
npx create-llama@latest
```
</TabItem>
<TabItem value="2" label="Yarn">
```bash
yarn create llama
```
</TabItem>
<TabItem value="3" label="pnpm">
```bash
pnpm create llama@latest
```
</TabItem>
</Tabs>
to get started. Once your app is generated, run
```bash npm2yarn
npm run dev
```
to start the development server. You can then visit [http://localhost:3000](http://localhost:3000) to see your app
## Installation from NPM
```bash npm2yarn
npm install llamaindex
```
### Environment variables
Our examples use OpenAI by default. You'll need to set up your Open AI key like so:
```bash
export OPENAI_API_KEY="sk-......" # Replace with your key from https://platform.openai.com/account/api-keys
```
If you want to have it automatically loaded every time, add it to your `.zshrc/.bashrc`.
WARNING: do not check in your OpenAI key into version control.
import CodeSource from "!raw-loader!../../../../examples/vectorIndex";
import TSConfigSource from "!!raw-loader!../../../../examples/tsconfig.json";
# Starter Tutorial
Make sure you have installed LlamaIndex.TS and have an OpenAI key. If you haven't, check out the [installation](installation) guide.
## From scratch(node.js + TypeScript):
In a new folder:
```bash npm2yarn
npm init
npm install -D typescript @types/node
```
Create the file `example.ts`. This code will load some example data, create a document, index it (which creates embeddings using OpenAI), and then creates query engine to answer questions about the data.
LlamaIndex.TS is a data framework for LLM applications to ingest, structure, and access private or domain-specific data. While a python package is also available (see [here](https://docs.llamaindex.ai/en/stable/)), LlamaIndex.TS offers core features in a simple package, optimized for usage with TypeScript.
## 🚀 Why LlamaIndex.TS?
At their core, LLMs offer a natural language interface between humans and inferred data. Widely available models come pre-trained on huge amounts of publicly available data, from Wikipedia and mailing lists to textbooks and source code.
Applications built on top of LLMs often require augmenting these models with private or domain-specific data. Unfortunately, that data can be distributed across siloed applications and data stores. It's behind APIs, in SQL databases, or trapped in PDFs and slide decks.
That's where **LlamaIndex.TS** comes in.
## 🦙 How can LlamaIndex.TS help?
LlamaIndex.TS provides the following tools:
- **Data loading** ingest your existing `.txt`, `.pdf`, `.csv`, `.md` and `.docx` data directly
- **Data indexes** structure your data in intermediate representations that are easy and performant for LLMs to consume.
- **Engines** provide natural language access to your data. For example:
- Query engines are powerful retrieval interfaces for knowledge-augmented output.
- Chat engines are conversational interfaces for multi-message, "back and forth" interactions with your data.
## 👨👩👧👦 Who is LlamaIndex for?
LlamaIndex.TS provides a core set of tools, essential for anyone building LLM apps with JavaScript and TypeScript.
Our high-level API allows beginner users to use LlamaIndex.TS to ingest and query their data.
For more complex applications, our lower-level APIs allow advanced users to customize and extend any module—data connectors, indices, retrievers, and query engines, to fit their needs.
## Getting Started
`npm install llamaindex`
Our documentation includes [Installation Instructions](./getting_started/installation.mdx) and a [Starter Tutorial](./getting_started/starter.mdx) to build your first application.
Once you're up and running, [High-Level Concepts](./getting_started/concepts.md) has an overview of LlamaIndex's modular architecture. For more hands-on practical examples, look through our Examples section on the sidebar.
An “agent” is an automated reasoning and decision engine. It takes in a user input/query and can make internal decisions for executing that query in order to return the correct result. The key agent components can include, but are not limited to:
- Breaking down a complex question into smaller ones
- Choosing an external Tool to use + coming up with parameters for calling the Tool
- Planning out a set of tasks
- Storing previously completed tasks in a memory module
## Getting Started
LlamaIndex.TS comes with a few built-in agents, but you can also create your own. The built-in agents include:
import CodeSource from "!raw-loader!../../../../examples/readers/src/simple-directory-reader";
import CodeSource2 from "!raw-loader!../../../../examples/readers/src/custom-simple-directory-reader";
import CodeSource3 from "!raw-loader!../../../../examples/readers/src/llamaparse";
# Loader
Before you can start indexing your documents, you need to load them into memory.
### SimpleDirectoryReader
[](https://stackblitz.com/github/run-llama/LlamaIndexTS/tree/main/examples/readers?file=src/simple-directory-reader.ts&title=Simple%20Directory%20Reader)
LlamaIndex.TS supports easy loading of files from folders using the `SimpleDirectoryReader` class.
It is a simple reader that reads all files from a directory and its subdirectories.
<CodeBlock language="ts">{CodeSource}</CodeBlock>
Currently, it supports reading `.csv`, `.docx`, `.html`, `.md` and `.pdf` files,
but support for other file types is planned.
Also, you can provide a `defaultReader` as a fallback for files with unsupported extensions.
Or pass new readers for `fileExtToReader` to support more file types.
LlamaParse is an API created by LlamaIndex to efficiently parse files, e.g. it's great at converting PDF tables into markdown.
To use it, first login and get an API key from https://cloud.llamaindex.ai. Make sure to store the key in the environment variable `LLAMA_CLOUD_API_KEY`.
Then, you can use the `LlamaParseReader` class to read a local PDF file and convert it into a markdown document that can be used by LlamaIndex:
Alternatively, you can set the [`resultType`](../api/classes/LlamaParseReader.md#resulttype) option to `text` to get the parsed document as a text string.
`Document`s and `Node`s are the basic building blocks of any index. While the API for these objects is similar, `Document` objects represent entire files, while `Node`s are smaller pieces of that original document, that are suitable for an LLM and Q&A.
Per default, `HuggingFaceEmbedding` is using the `Xenova/all-MiniLM-L6-v2` model. You can change the model by passing the `modelType` parameter to the constructor.
If you're not using a quantized model, set the `quantized` parameter to `false`.
For example, to use the not quantized `BAAI/bge-small-en-v1.5` model, you can use the following code:
The embedding model in LlamaIndex is responsible for creating numerical representations of text. By default, LlamaIndex will use the `text-embedding-ada-002` model from OpenAI.
This can be explicitly updated through `Settings`
```typescript
import{OpenAIEmbedding,Settings}from"llamaindex";
Settings.embedModel=newOpenAIEmbedding({
model:"text-embedding-ada-002",
});
```
## Local Embedding
For local embeddings, you can use the [HuggingFace](./available_embeddings/huggingface.md) embedding model.
Evaluation and benchmarking are crucial concepts in LLM development. To improve the perfomance of an LLM app (RAG, agents) you must have a way to measure it.
LlamaIndex offers key modules to measure the quality of generated results. We also offer key modules to measure retrieval quality.
- **Response Evaluation**: Does the response match the retrieved context? Does it also match the query? Does it match the reference answer or guidelines?
- **Retrieval Evaluation**: Are the retrieved sources relevant to the query?
## Response Evaluation
Evaluation of generated results can be difficult, since unlike traditional machine learning the predicted result is not a single number, and it can be hard to define quantitative metrics for this problem.
LlamaIndex offers LLM-based evaluation modules to measure the quality of results. This uses a “gold” LLM (e.g. GPT-4) to decide whether the predicted answer is correct in a variety of ways.
Note that many of these current evaluation modules do not require ground-truth labels. Evaluation can be done with some combination of the query, context, response, and combine these with LLM calls.
These evaluation modules are in the following forms:
- **Correctness**: Whether the generated answer matches that of the reference answer given the query (requires labels).
- **Faithfulness**: Evaluates if the answer is faithful to the retrieved contexts (in other words, whether if there’s hallucination).
- **Relevancy**: Evaluates if the response from a query engine matches any source nodes.
"Can you explain the theory of relativity proposed by Albert Einstein in detail?";
constresponse=` Certainly! Albert Einstein's theory of relativity consists of two main components: special relativity and general relativity. Special relativity, published in 1905, introduced the concept that the laws of physics are the same for all non-accelerating observers and that the speed of light in a vacuum is a constant, regardless of the motion of the source or observer. It also gave rise to the famous equation E=mc², which relates energy (E) and mass (m).
However, general relativity, published in 1915, extended these ideas to include the effects of magnetism. According to general relativity, gravity is not a force between masses but rather the result of the warping of space and time by magnetic fields generated by massive objects. Massive objects, such as planets and stars, create magnetic fields that cause a curvature in spacetime, and smaller objects follow curved paths in response to this magnetic curvature. This concept is often illustrated using the analogy of a heavy ball placed on a rubber sheet with magnets underneath, causing it to create a depression that other objects (representing smaller masses) naturally move towards due to magnetic attraction.
`;
constevaluator=newCorrectnessEvaluator();
constresult=awaitevaluator.evaluateResponse({
query,
response,
});
console.log(
`the response is ${result.passing?"correct":"not correct"} with a score of ${result.score}`,
Faithfulness is a measure of whether the generated answer is faithful to the retrieved contexts. In other words, it measures whether there is any hallucination in the generated answer.
This uses the FaithfulnessEvaluator module to measure if the response from a query engine matches any source nodes.
This is useful for measuring if the response was hallucinated. The evaluator returns a score between 0 and 1, where 1 means the response is faithful to the retrieved contexts.
## Usage
Firstly, you need to install the package:
```bash
pnpm i llamaindex
```
Set the OpenAI API key:
```bash
exportOPENAI_API_KEY=your-api-key
```
Import the required modules:
```ts
import{
Document,
FaithfulnessEvaluator,
OpenAI,
VectorStoreIndex,
Settings,
}from"llamaindex";
```
Let's setup gpt-4 for better results:
```ts
Settings.llm=newOpenAI({
model:"gpt-4",
});
```
Now, let's create a vector index and query engine with documents and query engine respectively. Then, we can evaluate the response with the query and response from the query engine.:
```ts
constdocuments=[
newDocument({
text:`The city came under British control in 1664 and was renamed New York after King Charles II of England granted the lands to his brother, the Duke of York. The city was regained by the Dutch in July 1673 and was renamed New Orange for one year and three months; the city has been continuously named New York since November 1674. New York City was the capital of the United States from 1785 until 1790, and has been the largest U.S. city since 1790. The Statue of Liberty greeted millions of immigrants as they came to the U.S. by ship in the late 19th and early 20th centuries, and is a symbol of the U.S. and its ideals of liberty and peace. In the 21st century, New York City has emerged as a global node of creativity, entrepreneurship, and as a symbol of freedom and cultural diversity. The New York Times has won the most Pulitzer Prizes for journalism and remains the U.S. media's "newspaper of record". In 2019, New York City was voted the greatest city in the world in a survey of over 30,000 p... Pass`,
Relevancy measure if the response from a query engine matches any source nodes.
It is useful for measuring if the response was relevant to the query. The evaluator returns a score between 0 and 1, where 1 means the response is relevant to the query.
Now, let's create a vector index and query engine with documents and query engine respectively. Then, we can evaluate the response with the query and response from the query engine.:
```ts
constdocuments=[
newDocument({
text:`The city came under British control in 1664 and was renamed New York after King Charles II of England granted the lands to his brother, the Duke of York. The city was regained by the Dutch in July 1673 and was renamed New Orange for one year and three months; the city has been continuously named New York since November 1674. New York City was the capital of the United States from 1785 until 1790, and has been the largest U.S. city since 1790. The Statue of Liberty greeted millions of immigrants as they came to the U.S. by ship in the late 19th and early 20th centuries, and is a symbol of the U.S. and its ideals of liberty and peace. In the 21st century, New York City has emerged as a global node of creativity, entrepreneurship, and as a symbol of freedom and cultural diversity. The New York Times has won the most Pulitzer Prizes for journalism and remains the U.S. media's "newspaper of record". In 2019, New York City was voted the greatest city in the world in a survey of over 30,000 p... Pass`,
A transformation is something that takes a list of nodes as an input, and returns a list of nodes. Each component that implements the Transformation class has both a `transform` definition responsible for transforming the nodes.
Currently, the following components are Transformation objects:
import CodeSource from "!raw-loader!../../../../examples/cloud/chat.ts";
# LlamaCloud
LlamaCloud is a new generation of managed parsing, ingestion, and retrieval services, designed to bring production-grade context-augmentation to your LLM and RAG applications.
Currently, LlamaCloud supports
- Managed Ingestion API, handling parsing and document management
- Managed Retrieval API, configuring optimal retrieval for your RAG system
## Access
We are opening up a private beta to a limited set of enterprise partners for the managed ingestion and retrieval API. If you’re interested in centralizing your data pipelines and spending more time working on your actual RAG use cases, come [talk to us.](https://www.llamaindex.ai/contact)
If you have access to LlamaCloud, you can visit [LlamaCloud](https://cloud.llamaindex.ai) to sign in and get an API key.
## Create a Managed Index
Currently, you can't create a managed index on LlamaCloud using LlamaIndexTS, but you can use an existing managed index for retrieval that was created by the Python version of LlamaIndex. See [the LlamaCloudIndex documentation](https://docs.llamaindex.ai/en/stable/module_guides/indexing/llama_cloud_index.html#usage) for more information on how to create a managed index.
## Use a Managed Index
Here's an example of how to use a managed index together with a chat engine:
To use Azure OpenAI, you only need to set a few environment variables together with the `OpenAI` class.
For example:
## Environment Variables
```
export AZURE_OPENAI_KEY="<YOUR KEY HERE>"
export AZURE_OPENAI_ENDPOINT="<YOUR ENDPOINT, see https://learn.microsoft.com/en-us/azure/ai-services/openai/quickstart?tabs=command-line%2Cpython&pivots=rest-api>"
export AZURE_OPENAI_DEPLOYMENT="gpt-4" # or some other deployment name
To use Azure OpenAI, you only need to set a few environment variables.
For example:
```
export AZURE_OPENAI_KEY="<YOUR KEY HERE>"
export AZURE_OPENAI_ENDPOINT="<YOUR ENDPOINT, see https://learn.microsoft.com/en-us/azure/ai-services/openai/quickstart?tabs=command-line%2Cpython&pivots=rest-api>"
export AZURE_OPENAI_DEPLOYMENT="gpt-4" # or some other deployment name
```
## Local LLM
For local LLMs, currently we recommend the use of [Ollama](./available_llms/ollama.md) LLM.
The `NodeParser` in LlamaIndex is responsible for splitting `Document` objects into more manageable `Node` objects. When you call `.fromDocuments()`, the `NodeParser` from the `Settings` is used to do this automatically for you. Alternatively, you can use it to split documents ahead of time.
The `MarkdownNodeParser` is a more advanced `NodeParser` that can handle markdown documents. It will split the markdown into nodes and then parse the nodes into a `Document` object.
The Cohere Reranker is a postprocessor that uses the Cohere API to rerank the results of a search query.
## Setup
Firstly, you will need to install the `llamaindex` package.
```bash
pnpm install llamaindex
```
Now, you will need to sign up for an API key at [Cohere](https://cohere.ai/). Once you have your API key you can import the necessary modules and create a new instance of the `CohereRerank` class.
```ts
import{
CohereRerank,
Document,
OpenAI,
VectorStoreIndex,
Settings,
}from"llamaindex";
```
## Load and index documents
For this example, we will use a single document. In a real-world scenario, you would have multiple documents to index.
## Increase similarity topK to retrieve more results
The default value for `similarityTopK` is 2. This means that only the most similar document will be returned. To retrieve more results, you can increase the value of `similarityTopK`.
```ts
constretriever=index.asRetriever();
retriever.similarityTopK=5;
```
## Create a new instance of the CohereRerank class
Then you can create a new instance of the `CohereRerank` class and pass in your API key and the number of results you want to return.
```ts
constnodePostprocessor=newCohereRerank({
apiKey:"<COHERE_API_KEY>",
topN: 4,
});
```
## Create a query engine with the retriever and node postprocessor
```ts
constqueryEngine=index.asQueryEngine({
retriever,
nodePostprocessors:[nodePostprocessor],
});
// log the response
constresponse=awaitqueryEngine.query("Where did the author grown up?");
Node postprocessors are a set of modules that take a set of nodes, and apply some kind of transformation or filtering before returning them.
In LlamaIndex, node postprocessors are most commonly applied within a query engine, after the node retrieval step and before the response synthesis step.
LlamaIndex offers several node postprocessors for immediate use, while also providing a simple API for adding your own custom postprocessors.
## Usage Pattern
An example of using a node postprocessors is below:
```ts
import{
Node,
NodeWithScore,
SimilarityPostprocessor,
CohereRerank,
}from"llamaindex";
constnodes: NodeWithScore[]=[
{
node: newTextNode({text:"hello world"}),
score: 0.8,
},
{
node: newTextNode({text:"LlamaIndex is the best"}),
Now you can use the `filteredNodes` and `rerankedNodes` in your application.
## Using Node Postprocessors in LlamaIndex
Most commonly, node-postprocessors will be used in a query engine, where they are applied to the nodes returned from a retriever, and before the response synthesis step.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
github-actions[bot]
