langchain-ai/docs
1
0
Fork 0
mirror of https://github.com/langchain-ai/docs.git synced 2026-07-22 04:15:25 -04:00
Code Issues 490 Packages Projects Releases Wiki Activity

[GH-ISSUE #3033] [langchain]: Add HighSNR compressor integration docs (document compression for retrievers) #2701

New Issue
Open
opened 2026-06-05 17:26:21 -04:00 by yindo · 0 comments
yindo commented 2026-06-05 17:26:21 -04:00
Owner
Copy Link
Copy Source

Originally created by @georgeSkoumas on GitHub (Mar 10, 2026).
Original GitHub issue: https://github.com/langchain-ai/docs/issues/3033

Type of issue

request for content

Language

Python

Description

Context

langchain-highsnr is a published third-party
integration for deterministic document compression.

LLM pipelines that process long documents — whether for RAG, summarization, classification,
or extraction — often hit context window limits or waste tokens on low-signal content.
HighSNR compresses documents to a token budget, keeping the highest-signal chunks and
discarding the rest. It's deterministic, privacy-first (zero text retention), and sub-second
for most documents.

It provides two LangChain components:

  • HighSNRDocumentCompressor — post-retrieval compression via ContextualCompressionRetriever
  • HighSNRDocumentTransformer — pre-indexing compression before embedding

Why this is useful

  • Retrieved chunks often exceed the context window or budget — HighSNR fits them within a
    token limit while preserving the most informative content
  • Pre-indexing compression produces leaner, higher-quality vector stores with less noise
  • Benchmarked on LongBench v1 with GPT-4o: at 80% budget, exceeds full-context F1 on
    HotpotQA (70.96 vs 69.71)

Proposed docs addition

A single page at src/oss/python/integrations/retrievers/highsnr-compressor.mdx covering:

  • Setup (install + API key)
  • Post-retrieval compression with ContextualCompressionRetriever
  • Pre-indexing compression with HighSNRDocumentTransformer
  • Parameter tables for both components
  • Benchmark summary with link to reproduction scripts

I have a draft PR ready and will link it once this issue is created.

References

Originally created by @georgeSkoumas on GitHub (Mar 10, 2026). Original GitHub issue: https://github.com/langchain-ai/docs/issues/3033 ### Type of issue request for content ### Language Python ### Description ## Context [langchain-highsnr](https://pypi.org/project/langchain-highsnr/) is a published third-party integration for deterministic document compression. LLM pipelines that process long documents — whether for RAG, summarization, classification, or extraction — often hit context window limits or waste tokens on low-signal content. HighSNR compresses documents to a token budget, keeping the highest-signal chunks and discarding the rest. It's deterministic, privacy-first (zero text retention), and sub-second for most documents. It provides two LangChain components: - `HighSNRDocumentCompressor` — post-retrieval compression via `ContextualCompressionRetriever` - `HighSNRDocumentTransformer` — pre-indexing compression before embedding ## Why this is useful - Retrieved chunks often exceed the context window or budget — HighSNR fits them within a token limit while preserving the most informative content - Pre-indexing compression produces leaner, higher-quality vector stores with less noise - Benchmarked on LongBench v1 with GPT-4o: at 80% budget, exceeds full-context F1 on HotpotQA (70.96 vs 69.71) ## Proposed docs addition A single page at `src/oss/python/integrations/retrievers/highsnr-compressor.mdx` covering: - Setup (install + API key) - Post-retrieval compression with `ContextualCompressionRetriever` - Pre-indexing compression with `HighSNRDocumentTransformer` - Parameter tables for both components - Benchmark summary with link to reproduction scripts I have a draft PR ready and will link it once this issue is created. ## References - PyPI: https://pypi.org/project/langchain-highsnr/ - Source: https://github.com/HighSNRInc/langchain-highsnr - Benchmarks (LongBench v1, GPT-4o): https://github.com/HighSNRInc/highsnr-benchmarks - API docs: https://www.high-snr.com/docs.html
yindo added the langchainexternal labels 2026-06-05 17:26:21 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
deepagents
docs-infra
external
external
help wanted
integration
internal
internal
javascript
js/ts
langchain
langgraph
langsmith
needs-title
pull-request
Mirrored from GitHub Pull Request
 python
reference
No Label external langchain
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/docs#2701
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?