Add Question-Level Caching to LangChain for RAG Pipelines #90

New Issue

2026-02-16T05:16:31-05:00

yindo commented

2026-02-16 05:16:31 -05:00

Originally created by @bharatht19 on GitHub (Sep 18, 2025).

Currently, RedisSemanticCache and other cache integrations in LangChain operate at the LLM layer — caching on the rendered prompt (question + retrieved context).

For many RAG use cases, it would be very useful to cache before retrieval, i.e., at the question level, so that repeated or semantically similar questions can bypass vector DB retrieval entirely.

Why it matters:
• Saves cost and latency (skip retrieval + LLM if cached).
• Supports both exact-match and semantic caching of questions.
• Fits common enterprise use cases where the same queries recur frequently.

What I’m asking:

Could LangChain provide a QuestionCache or extend RedisSemanticCache (and similar backends) with an option to store & lookup only on the raw user question (optionally semantic), before retrieval?

This would complement the existing LLM-level cache and make caching more flexible in RAG pipelines.

Originally created by @bharatht19 on GitHub (Sep 18, 2025). Currently, RedisSemanticCache and other cache integrations in LangChain operate at the LLM layer — caching on the rendered prompt (question + retrieved context). For many RAG use cases, it would be very useful to cache before retrieval, i.e., at the question level, so that repeated or semantically similar questions can bypass vector DB retrieval entirely. Why it matters: • Saves cost and latency (skip retrieval + LLM if cached). • Supports both exact-match and semantic caching of questions. • Fits common enterprise use cases where the same queries recur frequently. What I’m asking: Could LangChain provide a QuestionCache or extend RedisSemanticCache (and similar backends) with an option to store & lookup only on the raw user question (optionally semantic), before retrieval? This would complement the existing LLM-level cache and make caching more flexible in RAG pipelines.

yindo referenced this issue

2026-02-16 05:16:47 -05:00

[PR #90] [MERGED] Release 0.0.10 #159

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: langchain-ai/langchain-postgres#90