langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-20 19:50:06 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #32263] feat(api): optimize OceanBase vector store performance and configurability #33638

New Issue
Closed
opened 2026-02-21 20:53:38 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:53:38 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/32263

State: closed
Merged: Yes

Important

  1. Make sure you have read our contribution guidelines
  2. Ensure there is an associated issue and you have been assigned to it
  3. Use the correct syntax to link this PR: Fixes #<issue number>.

Fixes #32262

Summary

Optimize OceanBase vector store with several performance and functionality improvements. All new configuration options have sensible defaults and are fully backward-compatible with existing deployments.

Changes Overview

1. Batch Insertion

Replace single-row inserts with configurable batch insertion, reducing database round-trips. When >= 1000 documents are inserted, the HNSW index is automatically refreshed to ensure search quality.

2. Multiple Distance Metrics

Support three distance metrics with correct distance-to-score conversion for each:

Metric Score Formula Best For
l2 1.0 / (1.0 + distance) General-purpose, unnormalized embeddings
cosine 1.0 - distance Normalized embeddings, semantic similarity
inner_product -distance Models optimized for dot-product similarity

3. Configurable HNSW Index Parameters

Expose HNSW parameters for fine-tuning the balance between index build time, memory usage, and search quality.

4. Connection Pool Configuration

Add SQLAlchemy connection pool settings with pool_recycle=3600 and pool_pre_ping=True hardcoded for connection reliability.

5. Metadata Functional Index

Automatically create a functional index on metadata->>'$.document_id' during collection initialization, reducing query latency variance on filtered queries.

New Environment Variables

Variable Type Default Description
OCEANBASE_VECTOR_BATCH_SIZE PositiveInt 100 Number of documents to insert per batch. Larger values reduce round-trips but increase memory usage per batch.
OCEANBASE_VECTOR_METRIC_TYPE str l2 Distance metric for vector index. Supported values: l2, cosine, inner_product. Must match the embedding model's expected similarity measure.
OCEANBASE_HNSW_M PositiveInt 16 HNSW M parameter — max number of bi-directional links per node. Higher values improve recall but increase memory and build time. Typical range: 8–64.
OCEANBASE_HNSW_EF_CONSTRUCTION PositiveInt 256 HNSW efConstruction — search width during index building. Higher values produce a better-quality graph at the cost of slower indexing. Typical range: 100–500.
OCEANBASE_HNSW_EF_SEARCH int -1 HNSW efSearch — search width at query time. Higher values improve recall at the cost of latency. Set to -1 to use the OceanBase server default. Typical range: 64–512.
OCEANBASE_VECTOR_POOL_SIZE PositiveInt 5 SQLAlchemy connection pool size — number of persistent connections kept open. Increase for high-concurrency workloads.
OCEANBASE_VECTOR_MAX_OVERFLOW int 10 SQLAlchemy max overflow — additional temporary connections allowed beyond pool size. Total max connections = pool_size + max_overflow.

Performance Benchmark

Tested on local single-node OceanBase CE v4.3.5.5, 1536-dimension random vectors, macOS:

Insertion throughput — single-row vs batch (batch_size=100):

Documents Single-row Batch(100) Speedup
100 0.28s 0.11s 2.6x
500 1.36s 0.55s 2.5x
1,000 2.64s 1.29s 2.1x
5,000 25.59s 11.41s 2.2x

Metadata filter query — WHERE metadata->>'$.document_id' = ? on 5,000 rows (~1,000 matches), 10 iterations:

Condition Latency (mean ± stdev)
Without functional index 3.3 ± 1.4 ms
With functional index 2.7 ± 0.6 ms

At 5k rows the latency difference is modest; the functional index primarily stabilizes variance and is expected to show larger gains on bigger tables where full-scan cost grows linearly.

Vector search — top-10 ANN, 20 queries, on 1,000 rows:

Metric Latency (mean ± stdev)
l2 5.2 ± 0.9 ms
cosine 5.0 ± 1.2 ms
inner_product 7.7 ± 0.4 ms

Benchmark script: api/tests/integration_tests/vdb/oceanbase/bench_oceanbase.py

Screenshots

N/A — Backend-only changes, no UI modifications.

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran make lint and make type-check (backend) and cd web && npx lint-staged (frontend) to appease the lint gods
**Original Pull Request:** https://github.com/langgenius/dify/pull/32263 **State:** closed **Merged:** Yes --- > [!IMPORTANT] > > 1. Make sure you have read our [contribution guidelines](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) > 1. Ensure there is an associated issue and you have been assigned to it > 1. Use the correct syntax to link this PR: `Fixes #<issue number>`. Fixes #32262 ## Summary Optimize OceanBase vector store with several performance and functionality improvements. All new configuration options have sensible defaults and are fully backward-compatible with existing deployments. ### Changes Overview #### 1. Batch Insertion Replace single-row inserts with configurable batch insertion, reducing database round-trips. When >= 1000 documents are inserted, the HNSW index is automatically refreshed to ensure search quality. #### 2. Multiple Distance Metrics Support three distance metrics with correct distance-to-score conversion for each: | Metric | Score Formula | Best For | |--------|--------------|----------| | `l2` | `1.0 / (1.0 + distance)` | General-purpose, unnormalized embeddings | | `cosine` | `1.0 - distance` | Normalized embeddings, semantic similarity | | `inner_product` | `-distance` | Models optimized for dot-product similarity | #### 3. Configurable HNSW Index Parameters Expose HNSW parameters for fine-tuning the balance between index build time, memory usage, and search quality. #### 4. Connection Pool Configuration Add SQLAlchemy connection pool settings with `pool_recycle=3600` and `pool_pre_ping=True` hardcoded for connection reliability. #### 5. Metadata Functional Index Automatically create a functional index on `metadata->>'$.document_id'` during collection initialization, reducing query latency variance on filtered queries. ### New Environment Variables | Variable | Type | Default | Description | |----------|------|---------|-------------| | `OCEANBASE_VECTOR_BATCH_SIZE` | `PositiveInt` | `100` | Number of documents to insert per batch. Larger values reduce round-trips but increase memory usage per batch. | | `OCEANBASE_VECTOR_METRIC_TYPE` | `str` | `l2` | Distance metric for vector index. Supported values: `l2`, `cosine`, `inner_product`. Must match the embedding model's expected similarity measure. | | `OCEANBASE_HNSW_M` | `PositiveInt` | `16` | HNSW M parameter — max number of bi-directional links per node. Higher values improve recall but increase memory and build time. Typical range: 8–64. | | `OCEANBASE_HNSW_EF_CONSTRUCTION` | `PositiveInt` | `256` | HNSW efConstruction — search width during index building. Higher values produce a better-quality graph at the cost of slower indexing. Typical range: 100–500. | | `OCEANBASE_HNSW_EF_SEARCH` | `int` | `-1` | HNSW efSearch — search width at query time. Higher values improve recall at the cost of latency. Set to `-1` to use the OceanBase server default. Typical range: 64–512. | | `OCEANBASE_VECTOR_POOL_SIZE` | `PositiveInt` | `5` | SQLAlchemy connection pool size — number of persistent connections kept open. Increase for high-concurrency workloads. | | `OCEANBASE_VECTOR_MAX_OVERFLOW` | `int` | `10` | SQLAlchemy max overflow — additional temporary connections allowed beyond pool size. Total max connections = pool_size + max_overflow. | ### Performance Benchmark Tested on local single-node OceanBase CE v4.3.5.5, 1536-dimension random vectors, macOS: **Insertion throughput — single-row vs batch (batch_size=100):** | Documents | Single-row | Batch(100) | Speedup | |-----------|-----------|-----------|---------| | 100 | 0.28s | 0.11s | **2.6x** | | 500 | 1.36s | 0.55s | **2.5x** | | 1,000 | 2.64s | 1.29s | **2.1x** | | 5,000 | 25.59s | 11.41s | **2.2x** | **Metadata filter query — `WHERE metadata->>'$.document_id' = ?` on 5,000 rows (~1,000 matches), 10 iterations:** | Condition | Latency (mean ± stdev) | |-----------|----------------------| | Without functional index | 3.3 ± 1.4 ms | | With functional index | 2.7 ± 0.6 ms | > At 5k rows the latency difference is modest; the functional index primarily **stabilizes variance** and is expected to show larger gains on bigger tables where full-scan cost grows linearly. **Vector search — top-10 ANN, 20 queries, on 1,000 rows:** | Metric | Latency (mean ± stdev) | |--------|----------------------| | `l2` | 5.2 ± 0.9 ms | | `cosine` | 5.0 ± 1.2 ms | | `inner_product` | 7.7 ± 0.4 ms | > Benchmark script: `api/tests/integration_tests/vdb/oceanbase/bench_oceanbase.py` ## Screenshots N/A — Backend-only changes, no UI modifications. ## Checklist - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs) - [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!) - [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change. - [x] I've updated the documentation accordingly. - [x] I ran `make lint` and `make type-check` (backend) and `cd web && npx lint-staged` (frontend) to appease the lint gods
yindo added the pull-request label 2026-02-21 20:53:38 -05:00
yindo closed this issue 2026-02-21 20:53:38 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#33638
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?