Add Chroma for vector storage #3

@yisding commented on GitHub (Jul 27, 2023):

Hi Jeff, sorry about the VS Code search. I think there's something in the exclude settings that's stopping VS Code from searching through the code properly. I'll take a look at that.

@yisding commented on GitHub (Jul 27, 2023): Hi Jeff, sorry about the VS Code search. I think there's something in the exclude settings that's stopping VS Code from searching through the code properly. I'll take a look at that.

yindo commented

@yisding commented on GitHub (Jul 27, 2023):

So this is the interface we're using to implement our Vector Stores:

export interface VectorStore {
  storesText: boolean;
  isEmbeddingQuery?: boolean;
  client(): any;
  add(embeddingResults: NodeWithEmbedding[]): string[];
  delete(refDocId: string, deleteKwargs?: any): void;
  query(query: VectorStoreQuery, kwargs?: any): VectorStoreQueryResult;
  persist(persistPath: string, fs?: GenericFileSystem): void;
}

It's generally the same as the python API.

Persist here wouldn't apply. Right don't worry about it.

get is an implementation detail of SimpleVectorStore so that wouldn't apply here also.

So just add, delete, and query.

For the add case, the document you can get that using node.getContent(MetadataMode.NONE)

@yisding commented on GitHub (Jul 27, 2023): So this is the interface we're using to implement our Vector Stores: ``` export interface VectorStore { storesText: boolean; isEmbeddingQuery?: boolean; client(): any; add(embeddingResults: NodeWithEmbedding[]): string[]; delete(refDocId: string, deleteKwargs?: any): void; query(query: VectorStoreQuery, kwargs?: any): VectorStoreQueryResult; persist(persistPath: string, fs?: GenericFileSystem): void; } ``` It's generally the same as the python API. Persist here wouldn't apply. Right don't worry about it. get is an implementation detail of SimpleVectorStore so that wouldn't apply here also. So just add, delete, and query. For the add case, the document you can get that using `node.getContent(MetadataMode.NONE)`

yindo commented

@yisding commented on GitHub (Jul 27, 2023):

OK the VS Code issue should be fixed in main, sorry about that.

@yisding commented on GitHub (Jul 27, 2023): OK the VS Code issue should be fixed in main, sorry about that.

yindo commented

@Jaikant commented on GitHub (Aug 23, 2023):

I am new here, I wanted to use the "Keyword Table Index", I assume in the current implementation this index will be in the memory and is lost once the application shuts down.
My question is would this index get automatically supported with chroma or would need additional work to get this working with chroma?

@Jaikant commented on GitHub (Aug 23, 2023): I am new here, I wanted to use the "Keyword Table Index", I assume in the current implementation this index will be in the memory and is lost once the application shuts down. My question is would this index get automatically supported with chroma or would need additional work to get this working with chroma?

yindo commented

@orpic commented on GitHub (Nov 1, 2023):

Hi, is this issue still being worked on, if not, would like to give it a shot.

@orpic commented on GitHub (Nov 1, 2023): Hi, is this issue still being worked on, if not, would like to give it a shot.

yindo commented

@jeffchuber commented on GitHub (Nov 3, 2023):

@orpic that would be great- please target this version of chromadb.js for now https://www.npmjs.com/package/chromadb/v/1.5.12-beta.0

@jeffchuber commented on GitHub (Nov 3, 2023): @orpic that would be great- please target this version of chromadb.js for now https://www.npmjs.com/package/chromadb/v/1.5.12-beta.0

yindo commented

@orpic commented on GitHub (Nov 3, 2023):

Cool , Thanks for the heads up.

@orpic commented on GitHub (Nov 3, 2023): Cool , Thanks for the heads up.

yindo commented

2026-02-15 19:15:34 -05:00

@boehlerlukas commented on GitHub (Dec 28, 2023):

What about something like this?

import { ChromaClient } from 'chromadb';
import { VectorStore, VectorStoreQuery, VectorStoreQueryResult, MetadataFilters, BaseNode, Document, Metadata, MetadataMode } from "llamaindex";

export class ChromaVectorStore implements VectorStore {
    storesText: boolean = true;
    chromaClient: ChromaClient;
    collectionName: string;

    constructor(collectionName: string = "defaultCollection") {
        const chromaURL = process.env.CHROMA_API_URL;
        if (!chromaURL) {
            throw new Error(
                "Must specify CHROMA_API_URL via env variable.",
            );
        }

        const chromaAPIKey = process.env.CHROMA_API_KEY;
        if (!chromaAPIKey) {
            throw new Error(
                "Must specify CHROMA_API_KEY via env variable.",
            );
        }

        this.chromaClient = new ChromaClient({
            path: chromaURL,
            auth: {
                provider: "token",
                credentials: chromaAPIKey,
            },
        });
        this.collectionName = collectionName;
    }

    client(): ChromaClient {
        return this.chromaClient;
    }

    async initializeCollection() {
        try {
            await this.chromaClient.createCollection({ name: this.collectionName });
        } catch (error) {
            console.error("Error initializing collection:", error);
        }
    }

    async add(embeddingResults: BaseNode<Metadata>[]): Promise<string[]> {
        if (embeddingResults.length === 0) {
            return;
        }

        const collection = await this.chromaClient.getCollection({ name: this.collectionName });
        const items = embeddingResults.map(this.nodeToChromaRecord);

        for (let item of items) {
            await collection.upsert(item);
        }

        return Promise.resolve([]);
    }

    async query(query: VectorStoreQuery): Promise<VectorStoreQueryResult> {
        const collection = await this.chromaClient.getCollection({ name: this.collectionName });

        const results = await collection.query({
            queryEmbeddings: [query.queryEmbedding],
            nResults: query.similarityTopK,
            where: this.toChromaFilter(query.filters)
        });

        return this.convertToQueryResult(results);
    }

    private nodeToChromaRecord(node: BaseNode<Metadata>): any {
        return {
            ids: node.id_,
            embeddings: node.getEmbedding(),
            metadatas: node.metadata,
            documents: node.getContent(MetadataMode.EMBED)
        };
    }

    private toChromaFilter(stdFilters?: MetadataFilters): any {
        return stdFilters?.filters?.reduce((carry, item) => {
            carry[item.key] = item.value;
            return carry;
        }, {});
    }

    private convertToQueryResult(results: any): VectorStoreQueryResult {
        const nodes = results.map(result => new Document({
            id_: result.id,
            text: result.documents,
            metadata: result.metadatas,
            embedding: result.embeddings
        }));

        return {
            nodes: nodes,
            similarities: results.map(result => result.score),
            ids: results.map(result => result.id)
        };
    }

    async delete(refDocId: string): Promise<void> {
        const collection = await this.chromaClient.getCollection({ name: this.collectionName });
        await collection.delete({ ids: [refDocId] });
    }

    async clearCollection(): Promise<void> {
        await this.chromaClient.deleteCollection({ name: this.collectionName });
        await this.initializeCollection();
    }

    async countItems(): Promise<number> {
        const collection = await this.chromaClient.getCollection({ name: this.collectionName });
        return await collection.count();
    }

    async resetDatabase(): Promise<void> {
        await this.chromaClient.reset();
    }
}

@boehlerlukas commented on GitHub (Dec 28, 2023): What about something like this? ``` import { ChromaClient } from 'chromadb'; import { VectorStore, VectorStoreQuery, VectorStoreQueryResult, MetadataFilters, BaseNode, Document, Metadata, MetadataMode } from "llamaindex"; export class ChromaVectorStore implements VectorStore { storesText: boolean = true; chromaClient: ChromaClient; collectionName: string; constructor(collectionName: string = "defaultCollection") { const chromaURL = process.env.CHROMA_API_URL; if (!chromaURL) { throw new Error( "Must specify CHROMA_API_URL via env variable.", ); } const chromaAPIKey = process.env.CHROMA_API_KEY; if (!chromaAPIKey) { throw new Error( "Must specify CHROMA_API_KEY via env variable.", ); } this.chromaClient = new ChromaClient({ path: chromaURL, auth: { provider: "token", credentials: chromaAPIKey, }, }); this.collectionName = collectionName; } client(): ChromaClient { return this.chromaClient; } async initializeCollection() { try { await this.chromaClient.createCollection({ name: this.collectionName }); } catch (error) { console.error("Error initializing collection:", error); } } async add(embeddingResults: BaseNode<Metadata>[]): Promise<string[]> { if (embeddingResults.length === 0) { return; } const collection = await this.chromaClient.getCollection({ name: this.collectionName }); const items = embeddingResults.map(this.nodeToChromaRecord); for (let item of items) { await collection.upsert(item); } return Promise.resolve([]); } async query(query: VectorStoreQuery): Promise<VectorStoreQueryResult> { const collection = await this.chromaClient.getCollection({ name: this.collectionName }); const results = await collection.query({ queryEmbeddings: [query.queryEmbedding], nResults: query.similarityTopK, where: this.toChromaFilter(query.filters) }); return this.convertToQueryResult(results); } private nodeToChromaRecord(node: BaseNode<Metadata>): any { return { ids: node.id_, embeddings: node.getEmbedding(), metadatas: node.metadata, documents: node.getContent(MetadataMode.EMBED) }; } private toChromaFilter(stdFilters?: MetadataFilters): any { return stdFilters?.filters?.reduce((carry, item) => { carry[item.key] = item.value; return carry; }, {}); } private convertToQueryResult(results: any): VectorStoreQueryResult { const nodes = results.map(result => new Document({ id_: result.id, text: result.documents, metadata: result.metadatas, embedding: result.embeddings })); return { nodes: nodes, similarities: results.map(result => result.score), ids: results.map(result => result.id) }; } async delete(refDocId: string): Promise<void> { const collection = await this.chromaClient.getCollection({ name: this.collectionName }); await collection.delete({ ids: [refDocId] }); } async clearCollection(): Promise<void> { await this.chromaClient.deleteCollection({ name: this.collectionName }); await this.initializeCollection(); } async countItems(): Promise<number> { const collection = await this.chromaClient.getCollection({ name: this.collectionName }); return await collection.count(); } async resetDatabase(): Promise<void> { await this.chromaClient.reset(); } } ```

yindo commented

@jeffchuber commented on GitHub (Dec 28, 2023):

thanks @boehlerlukas :) did you test this?

i have a branch on-going here and my branch is pretty similar.

I've mainly had trouble figuring out how llamaindex serializes and deserializes all their custom metadata.

@jeffchuber commented on GitHub (Dec 28, 2023): thanks @boehlerlukas :) did you test this? i have a branch on-going here and my branch is pretty similar. I've mainly had trouble figuring out how llamaindex serializes and deserializes all their custom metadata.

yindo commented

2026-02-15 19:15:34 -05:00

@boehlerlukas commented on GitHub (Dec 28, 2023):

@jeffchuber still some early early WiP - just wanted to share it so someone else could make use of it as I'm not sure how much time i will have to spend on it.

We will probably only use LlamaIndex on a pretty low level / pick some functionality that we need and do everything else directly with the Chroma client for now.

@boehlerlukas commented on GitHub (Dec 28, 2023): @jeffchuber still some early early WiP - just wanted to share it so someone else could make use of it as I'm not sure how much time i will have to spend on it. We will probably only use LlamaIndex on a pretty low level / pick some functionality that we need and do everything else directly with the Chroma client for now.

yindo commented

@jeffchuber commented on GitHub (Dec 28, 2023):

@boehlerlukas sgtm! I'll update this thread with my progress on a full integration

@jeffchuber commented on GitHub (Dec 28, 2023): @boehlerlukas sgtm! I'll update this thread with my progress on a full integration

yindo commented

@boehlerlukas commented on GitHub (Dec 28, 2023):

If you need a code-review or some second thoughts on it, let me know!

@boehlerlukas commented on GitHub (Dec 28, 2023): If you need a code-review or some second thoughts on it, let me know!

yindo commented

@marcusschiesser commented on GitHub (Jan 1, 2024):

@jeffchuber as you're working on the integration, have you seen this PR: https://github.com/run-llama/LlamaIndexTS/pull/300 ?

@marcusschiesser commented on GitHub (Jan 1, 2024): @jeffchuber as you're working on the integration, have you seen this PR: https://github.com/run-llama/LlamaIndexTS/pull/300 ?

yindo commented

@ianschmitz commented on GitHub (Jan 30, 2024):

Was this closed by https://github.com/run-llama/LlamaIndexTS/pull/310?

@ianschmitz commented on GitHub (Jan 30, 2024): Was this closed by https://github.com/run-llama/LlamaIndexTS/pull/310?

yindo commented