How to query CollectionStore and EmbeddingStore models directly in a clean way? #29

New Issue

2026-02-16T05:16:09-05:00

yindo commented

2026-02-16 05:16:09 -05:00

Originally created by @darahayes on GitHub (Jul 11, 2024).

Hello, thanks for this great project I've found it very useful. I have a use case right now where within one application I want to create and manage multiple collections as well as being able to fetch and return details about some collections, e.g. the name and the collection metadata - Essentially my use case is CRUD for collections.

Currently I don't really see any way to do that cleanly other than dropping down to raw SQL queries in my application. Would this be the recommended approach?

I see in the source code in vectorstores.py that there are "private"/unexposed SQLAlchemy models defined for CollectionStore and EmbeddingsStore. Having them exposed would make querying against the tables a lot easier, at least for my particular use case.

I can understand why you might want to keep them private - they might be subject to change and any user code that touches those models potentially breaks. But I think even when the models are not exposed, if there were changes that resulted in the database tables being different, this would still be a breaking change for a lot of apps anyways.

Is exposing those models something you might consider? Or would you recommend going with raw SQL? Would be more than happy to submit a PR. Thanks!

Originally created by @darahayes on GitHub (Jul 11, 2024). Hello, thanks for this great project I've found it very useful. I have a use case right now where within one application I want to create and manage multiple collections as well as being able to fetch and return details about some collections, e.g. the name and the collection metadata - Essentially my use case is CRUD for collections. Currently I don't really see any way to do that cleanly other than dropping down to raw SQL queries in my application. Would this be the recommended approach? I see in the [source code in `vectorstores.py`](https://github.com/langchain-ai/langchain-postgres/blob/533e9655d1cd766c65acffc02008ec0e6a966ada/langchain_postgres/vectorstores.py#L108-L228) that there are "private"/unexposed SQLAlchemy models defined for `CollectionStore` and `EmbeddingsStore.` Having them exposed would make querying against the tables a lot easier, at least for my particular use case. I can understand why you might want to keep them private - they might be subject to change and any user code that touches those models potentially breaks. But I think even when the models are not exposed, if there were changes that resulted in the database tables being different, this would still be a breaking change for a lot of apps anyways. Is exposing those models something you might consider? Or would you recommend going with raw SQL? Would be more than happy to submit a PR. Thanks!

yindo added the help wanted label 2026-02-16 05:16:09 -05:00

yindo commented

2026-02-16 05:16:10 -05:00

@eyurtsev commented on GitHub (Jul 12, 2024):

Hi @darahayes, there's no current way to do this.

This code needs to be refactored to support two things:

Add a control plane (IndexAdmin) that will do exactly what you need it to do.
Create different tables for the actual embeddings (e.g., to support different embedding dimensions)

Here's a stub at the abstraction that's needed: https://github.com/langchain-ai/langchain/pull/23990/files

This would also open up the pathway for being able to apply specific types of indices on the collections and do schema migration down the roads if necessary.

If you're interested in helping out, I can help provide some guidance if needed!

@eyurtsev commented on GitHub (Jul 12, 2024): Hi @darahayes, there's no current way to do this. This code needs to be refactored to support two things: 1. Add a control plane (IndexAdmin) that will do exactly what you need it to do. 2. Create different tables for the actual embeddings (e.g., to support different embedding dimensions) Here's a stub at the abstraction that's needed: https://github.com/langchain-ai/langchain/pull/23990/files This would also open up the pathway for being able to apply specific types of indices on the collections and do schema migration down the roads if necessary. If you're interested in helping out, I can help provide some guidance if needed!

yindo commented

2026-02-16 05:16:10 -05:00

@Sachin-Bhat commented on GitHub (Jul 18, 2024):

Hey @eyurtsev,

If more information is given I can take this up.

Cheers,
Sachin

@Sachin-Bhat commented on GitHub (Jul 18, 2024): Hey @eyurtsev, If more information is given I can take this up. Cheers, Sachin

yindo commented

2026-02-16 05:16:10 -05:00

@ghost commented on GitHub (May 23, 2025):

Langchain is exposing its UpsertionRecord what can be referenced as a type or used to fetch rows from DB without raw SQL
https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/indexes/_sql_record_manager.py#L53-L83

What's the exact blocker to expose CollectionStore and EmbeddingStore similar way in langchain-postgres package?

@ghost commented on GitHub (May 23, 2025): Langchain is exposing its `UpsertionRecord` what can be referenced as a type or used to fetch rows from DB without raw SQL https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/indexes/_sql_record_manager.py#L53-L83 What's the exact blocker to expose `CollectionStore` and `EmbeddingStore` similar way in `langchain-postgres` package?

yindo referenced this issue

2026-02-16 05:16:41 -05:00

[PR #29] [MERGED] Add security policy #129

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: langchain-ai/langchain-postgres#29