## Overview
Adds a unified membership-filter surface syntax and merges the
previously separate `bloom_match` / `roaring_match` chains into one
parameterized execution path.
issue: #52777
## What's in this PR
### 1. Unified surface syntax
```
membership_match(<field>, {<blob-bytes-template>})
membership_match(<field>, {<blob-bytes-template>}, type=bloom|roaring)
not membership_match(<field>, {<blob-bytes-template>})
```
The filter kind is derived from the blob's self-describing magic header
— `MBF1` lowers to the existing `BloomFilterExpr` plan node, `MRB1` to
`RoaringFilterExpr`, and unknown headers fail closed. The optional
`type=` argument is a readability aid and a consistency pin (it must
agree with the magic). The unreleased `bloom_match` / `roaring_match`
surface names are intentionally not retained — neither has shipped in a
release. The wire plan bytes are unchanged from the two predecessor
implementations, so the plan proto stays frozen and rolling-upgrade
behavior for clients using `membership_match` matches what the
predecessor docs specified.
### 2. One control flow, per-kind data planes
- **Go parser/proxy**: one visitor/fill skeleton (`visitMembershipCall`
/ `fillMembershipMatchExpressionValue`), one spec table for behavior
switches (delete safety), and unified tree walkers serving delete
guards, element-level guards, plan-size accounting, and log redaction.
- **segcore**: a single `PhyMembershipFilterExpr<LogicalExpr,
ProbePolicy>` template carries the shared control flow;
`BloomMembershipProbe` (zero-copy SBBF view) and
`RoaringMembershipProbe` (decode-once Roaring64) remain separate data
planes. Historical class names remain aliases, so factory construction
and logs are unchanged.
- Kind-specific admission gates (MBF1 envelope + domain check, MRB1 body
walk + decoded-memory estimate) remain in their own files.
- Client builders move under `client/membership/` (`sbbf`,
`roaringfilter`); `bloom_match`/`roaring_match` builder packages are
removed with the surface names.
### 3. Preserve existing mask/null semantics
The unified executor preserves current `master` behavior on both
raw-data and index-only paths: candidate-mask-excluded rows are left at
their initialized `(false, valid)` value, including excluded NULL rows;
probed NULL rows produce `(false, invalid)`. This PR does not change the
framework reverse-lookup helper.
### 4. Two independent proxy budgets
- `proxy.maxMembershipFilterSize` (default 64 MiB) limits one MBF1 or
MRB1 body. It falls back to the released `proxy.maxBloomFilterSize` key
and the development-only `proxy.maxRoaringFilterSize` key.
- `proxy.maxMembershipFilterPlanSize` (default 128 MiB) remains the
separate request-wide aggregate serialized-plan limit. It falls back to
`proxy.maxBloomFilterPlanSize`.
- Main predicates, scorer filters, and HybridSearch sub-requests share
the same preflight budget. The preflight charges body bytes, matching
the per-blob gate that allows the fixed 32-byte envelope header on top;
final `proto.Size` accounting remains authoritative. Keeping the plan
budget (128 MiB) well above the per-body limit (64 MiB) means a
maximum-size body plus its 32-byte envelope and plan overhead always
fits — no silent halving of the usable tier.
## Deliberate behavior changes
1. Adds `membership_match` with magic-based MBF1/MRB1 dispatch, an
optional `type=` consistency pin, and fail-closed handling for unknown
headers.
2. Removes the unreleased `bloom_match`/`roaring_match` surface names
(and their client builder packages); the wire plan bytes for equivalent
inputs are unchanged.
3. Updates error wording in bytes-template, element-level, MATCH_* and
delete guards.
4. MATCH_* element-predicate guards now reject every membership-filter
surface name.
5. Preflight aggregate accounting charges blob bodies rather than whole
envelopes, matching the per-blob admission rule.
## Non-goals
- No change to MBF1/MRB1 formats, golden vectors, or SDK builders.
- No change to the plan proto or rolling-upgrade contract: fields 22/23
remain frozen.
- No repo-wide NULL/candidate-mask semantic change.
## Design document
[20260822-membership-match-expression.md](https://github.com/xiaofan-luan/milvus/blob/unify-membership-match/docs/design-docs/design_docs/20260822-membership-match-expression.md)
## Test plan
- [x] `internal/parser/planparserv2`: full suite green, including magic
dispatch, alias byte identity, fail-closed cases, delete safety,
redaction, and shared preflight budgeting.
- [x] `pkg/util/paramtable`: fallback-key precedence tests.
- [x] `internal/proxy`: membership size/error/search/query focused tests
green; the full local suite is green except `TestProxy`, which requires
unavailable integration infrastructure and fails identically on clean
HEAD.
- [x] segcore: all 73 bloom/roaring/membership expression tests green;
full `all_tests` has only failures reproduced on clean HEAD (6
ConjunctExprTest + 1 ExprJsonIndexTest abort).
- [x] Local segcore build and unit-test binary link with llvm@19.
---------
Signed-off-by: xiaofanluan <[email protected]>
Signed-off-by: xiaofanluan <[email protected]>
Co-authored-by: xiaofanluan <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
issue: #52094
Review follow-up to #51968. No behaviour change: the MRB1 wire format,
the validation rules and the limits are all unchanged, and every
surviving production function -- `Build`, `Validate`,
`scanPortableRoaring64Body`, `validateEnvelope` -- is byte-identical to
master. The segcore edits are comments only. The Go production changes
are deletions, plus one behaviour-preserving refactor:
`PlanContainsBloomFilter` and `PlanContainsRoaringFilter` now share
`planContainsFilter` (see "Two smaller items").
## The duplicate halves of the codec
`client/v3/roaringfilter` and `pkg/v3/util/roaringfilter` each exported
the whole codec — `Build`, `Validate`, `Parse`, `Filter` — while
production reached half of each:
| | production entry point | reachable only from tests |
| --- | --- | --- |
| `client/v3/roaringfilter` | `NewRoaringBitmapBlob` → `Build` |
`Validate`, `Parse`, `Filter`, and all 284 lines of
`portable_roaring64.go` |
| `pkg/v3/util/roaringfilter` | `validateRoaringBitmapBlob` → `Validate`
| `Build`, `Parse`, `Filter` |
The two `portable_roaring64.go` files were 2 lines apart, because one
was copied from the other. The two test files were 19 lines apart out of
~985. `pkg`'s copy even carried a wrapper,
`validatePortableRoaring64Body`, that nothing called at all.
Neither package appears in any release tag — `client/v3.0.0-beta`
(2026-07-18), `pkg/v3.0-beta`, `pkg/v3.0.0` and `v3.0.0` (2026-07-29)
all predate #51968, which merged 2026-08-17 — so trimming the two
package APIs breaks no published surface. `git ls-tree` on each of those
tags finds neither directory. The only residual exposure is a consumer
pinned to a master pseudo-version inside a two-day window on an
unreleased package.
Each package now owns one job: **the client builds, pkg validates,
segcore decodes.** The builder and decoder pieces pkg's own tests still
need moved to `fixtures_test.go`, so the validator suite is unchanged
and the package's API is `Validate` alone. That fixture builder is
trimmed to an encoder: the SDK additionally pre-rejects member sets the
proxy would reject, and none of the `buildFixture` call sites assert a
rejection, so reproducing that admission logic there would be a second
copy of it that nothing checks. The bytes are the same either way — a
Roaring bitmap is a set, so insertion order cannot change the encoding.
One side effect worth naming. `pkg` is linked into every server
component, and it was the only non-test importer of
`RoaringBitmap/roaring` outside the SDK, so after this change no
production Go code outside `client/` imports the library. `go.mod` moves
it from the indirect block to a direct require in the **root** module,
because a root-module test file now imports `roaring64` directly;
`pkg/go.mod` keeps it as a direct require for the same reason. So the
module graph is unchanged for anyone depending on `pkg/v3` — what
changed is the production import graph, not the dependency list.
`TestClientAndServerRoaringBuildAgree` existed to compare two builders.
With one builder there is nothing to compare, so it becomes
`TestClientBuiltBlobsPassProxyValidation`: the contract that still spans
two implementations is "a blob the SDK builds is one the proxy accepts".
It keeps the same distribution shapes and sizes, adds a shape that
reaches the bitmap-container and offset-table branches with bytes a real
builder produced, and grades the validator's structural scan against
counts recomputed a third time from the key set, so neither side is
graded against itself.
## The C++ copy of the limits
The nine MRB1 envelope and admission constants exist in three places.
The Go pair was already pinned to each other. segcore consumes its own
copy — `RoaringMembership.cpp` reads all nine — but **nothing pinned
them to the Go values**, so editing `kMaxHighContainerCount` there alone
compiled and passed CI, and the symptom only showed up in production as
the proxy admitting a blob every querynode refuses, or the SDK refusing
to build one the cluster would take.
segcore's copy is now **generated** from the Go one rather than checked
against it. `cmd/tools/genmrb1limits` writes
`internal/core/src/common/MRB1Limits.generated.h` from
`pkg/util/roaringfilter`'s constants, and `RoaringMembership.h` aliases
the names in it:
```cpp
static constexpr auto kHeaderSize = mrb1::kHeaderSize;
```
There is one definition, in Go, and segcore resolves it.
`RoaringMembership.cpp` is untouched: `RoaringMembership::kFoo` still
names the same value at the same width.
An earlier revision did this the other way — a Go test opened the two
segcore sources and parsed the declarations out of them. It worked, and
fault injection said so, but it was 623 lines, and the reason it needed
them is that every way of being shown the wrong declaration is a way to
pass over a diverged constant: comments carrying an old value,
non-literal initializers, a declared type that silently truncates
(`uint16_t kX = uint64_t{1} << 18` is 0), a name mentioned rather than
declared, an enum member or `#ifdef` pair hiding the live declaration
behind a readable decoy, and a macro defeating the check that catches
those. The cost landed on C++: those two files had to stay macro-free,
one-declarator-per-statement, and evaluable, enforced by a Go test in
another directory that takes the whole `internal` UT job down with it —
in a tree that has 172 `#define` and five headers using include guards.
Generating removes the need for all of it; segcore may spell its
constants however it likes now, because nothing reads them.
### What each half guarantees
- **Values and types reach segcore intact.** A standalone TU reproducing
the class, the aliases and the call-site shapes from
`RoaringMembership.cpp` compiles under `clang++ -std=c++20 -Wall
-Wextra` and `static_assert`s that every constant still equals the
literal it replaced and still has its old declared type (`const
std::string_view` / `uint16_t` / `size_t` / `uint64_t`), so no
comparison in the `.cpp` changes width.
- **The header cannot go stale.** `TestGeneratedHeaderIsCurrent` fails
when the file on disk is not what the generator would write. Injected
both directions — a Go constant changed without regenerating, and a
hand-edited generated header — and both fail with the regenerate
instruction.
- **A wrong type in the generator's table is a compile error, not a
truncation.** The values are brace-initialised for exactly this,
measured rather than assumed: clang accepts `constexpr uint16_t kX =
262144` without a word and gives it 0, while `constexpr uint16_t
kX{262144}` is `-Wc++11-narrowing`, an error by default. Injected:
narrowing one row fails at the generated header and again at every
`static_assert` downstream.
It also lands somewhere better to run. `internal/parser` is covered by
the codecov job only; `cmd/tools` is named by both it and
`scripts/run_go_unittest.sh`, so `make test-go` now covers this where it
did not cover the pin.
### What this gives up
- **The completeness direction.** The scan required every `k`-prefixed
constant in those two files to be classified as pinned or deliberately
unpinned, so a limit added to segcore alone was caught. Nothing replaces
that: a new MRB1 limit has to be added to the generator's table, and
adding one only to C++ is now caught by review rather than by CI.
- **Use sites, which neither approach ever covered.** Changing `/
kPortableRoaring64MinEntryBytes` to `/ 16`, or a `>=` to a `>`, diverges
segcore from the Go validator exactly as a changed constant would, and
passes.
The seven portable-Roaring wire constants in `RoaringMembership.cpp`
remain deliberately **not** pinned by value. They are RoaringFormatSpec
values rather than a Milvus contract, and two of them are derived rather
than chosen. Each side is held to them behaviourally, at the boundary,
by fixtures written independently on each side. That catches something a
value check cannot — a value check reads declarations, so flipping a `>`
to a `>=` passes it while the fixtures fail — but it is not strictly
stronger, and injection says so: `kPortableRoaring64MinEntryBytes` 12 →
16 fails on both sides, 12 → 8 fails on neither. Loosening is benign
here, since the bound is a pre-filter and the full scan behind it
rejects the same input, but the two sides really can drift in that
direction unnoticed. That reasoning now lives next to the constants in
`RoaringMembership.cpp` rather than in a test file.
## Two smaller items
- `PlanContainsBloomFilter` and `PlanContainsRoaringFilter` were
line-for-line copies of each other, and both re-implemented the switch
`planPredicates` already does — three copies of one switch in one
package. They share `planContainsFilter` now. Its nil check is gone with
them: the generated getters are nil-safe and `walkExpr` reports false
for a nil expression, and both nil cases are covered by existing tests.
- `PhyRoaringFilterExpr` passes a `nullptr` `skip_func`, so no chunk is
ever skipped. Bloom has no choice — an SBBF exposes no range — but a
Roaring bitmap has `Minimum()`/`Maximum()` in O(1). Left as a TODO
against this issue, carrying the two's-complement caveat, so it doesn't
read as "roaring can't".
## Verification
Coverage of the deleted client-side tests was checked rather than
assumed: all 18 `Test*` functions removed from
`client/roaringfilter/roaring_filter_test.go` have a counterpart in
`pkg/util/roaringfilter/roaring_filter_test.go`, which is where the
validator they exercise now lives. 4 keep their name outright; the other
14 are `TestParse*`/`TestBuildParse*` names renamed to `TestValidate*`
because the package no longer exports `Build` or `Parse`. The
differential oracle the old cross-module test provided is not lost:
`TestBuildMatchesLegacyByteForByte` pins the SDK's output byte-for-byte
against a naive builder, which is a better oracle than comparing two
copies of the same wrapper.
`FuzzParse` becomes `FuzzValidate`. One direction of its equivalence
became trivially true, because `parseFixture` calls `Validate` first;
the comment says so rather than implying otherwise. The load-bearing
comparison survives: the cardinality the hand-rolled scanner computes is
cross-checked against what `roaring64.ReadFrom` decodes, so the grader
is the upstream library, not the implementation under test.
Passed in this checkout:
- `client/v3/roaringfilter`, `client/v3/milvusclient`
- `pkg/v3/util/roaringfilter` — the full validator suite, plus
`FuzzValidate`
- `internal/parser/planparserv2` — the whole package, including the
rewritten `TestClientBuiltBlobsPassProxyValidation`
- `cmd/tools/genmrb1limits` — `TestGeneratedHeaderIsCurrent` and its two
table guards; the generator is idempotent
- `go vet` on all three modules; `gofmt -l` clean; `clang-format-15`,
the version `internal/core` pins, reports all four touched C++ files
clean
Running `internal/parser/planparserv2` locally needs `LOCALSTORAGEPATH`
pointed somewhere writable: paramtable's `LOCAL_STORAGE_SIZE` formatter
mkdirs `/var/lib/milvus/data/` and `mlog.Fatal`s if it cannot, which on
a stock macOS checkout takes the package down before any assertion runs.
That reproduces identically on `upstream/master`.
Two things I could not check locally and am not claiming:
`internal/proxy` and `internal/distributed/proxy/httpserver` need a
built `milvus_core`/`rocksdb`, so the one external caller of the trimmed
API — `internal/proxy/task_delete_test.go`, which uses the client's
`Build`, still exported — is grep-verified only. segcore was not
rebuilt: the standalone TU above is the strongest check I could run
without it, and CI is the first real segcore build. Its non-comment edit
is nine one-line aliases and one `#include`. `golangci-lint` crashes
locally on a Go toolchain mismatch, so CI is the first real lint run.
Where the new guard runs: `cmd/tools`, which both `make
codecov-go-without-build` → `scripts/run_go_codecov.sh` and
`scripts/run_go_unittest.sh` → `make test-go` cover. The test's own doc
comment says this.
## Follow-ups
- `client/sbbf` is in the same shape this PR fixes for roaring: it
exports `Parse`, `Filter` and six accessors with no non-test caller, it
is likewise absent from every release tag, and MBF1 has the same
three-way limit contract with no guard. Left alone deliberately to keep
this diff to one format; adding it now means one more table in
`cmd/tools/genmrb1limits`, not a second mechanism.
- `redactExprParams`' doc comment is attached to `hasExprParams` in
`internal/distributed/proxy/httpserver/handler_v2.go`, leaving
`redactExprParams` undocumented in godoc. That fix was in an earlier
revision of this PR and was taken out: it is another subsystem, and it
belongs in its own `doc:` PR.
---------
Signed-off-by: Li Liu <[email protected]>
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
Related to #49398
Bump Go module references from pkg/v2 and milvus-proto/go-api/v2 to
pkg/v3 and milvus-proto/go-api/v3 so the client tracks the Milvus 3.x
release line.
This prepares the repository for the upcoming 3.x.y release by aligning
imports, module dependencies, and proto API references with the new
major-version module paths.
---------
Signed-off-by: Congqi Xia <[email protected]>
## Summary
Removes the `IsTriggerKill` / SIGINT block from
`datacoord.stopServiceWatch()` and `querycoordv2.watchNodes()`.
### Root Cause
In MixCoord mode all three coordinators share the same etcd `Session`
object. During shutdown, when any coordinator calls `session.Stop()` it
cancels the shared context, closing the other coordinators' etcd watches
— triggering `stopServiceWatch()` / `watchNodes()` which sent SIGINT to
the process during a normal, coordinated teardown.
### Why remove SIGINT?
- `go s.Stop()` already handles unexpected session loss — SIGINT was
just a backstop in case `Stop()` hangs
- No other component (rootcoord, proxy, datanode, querynode) has this
logic
- The false-positive risk (killing the process during normal shutdown)
outweighs the marginal benefit
issue: #48242
---------
Signed-off-by: xiaofanluan <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
The config gen-yaml tool initialized BaseTable with default options,
which connected to etcd and loaded environment variables. This caused
runtime config values (e.g. mq.type=pulsar from etcd) to override
code-defined defaults, producing incorrect generated yaml files.
Signed-off-by: Congqi Xia <[email protected]>
## Summary
- Replace spf13/viper with yaml.v3 in the only 2 files that used it
(printer.go, server_test.go)
- Remove viper + 7 transitive deps from go.mod (8 packages total)
- C++ ghost dep removal (libsodium, simde) deferred to separate PR to
avoid CI conan issues
### Removed packages
spf13/viper, fsnotify/fsnotify, hashicorp/hcl, mitchellh/mapstructure,
pelletier/go-toml, spf13/afero, spf13/jwalterweatherman, subosito/gotenv
## Test plan
- [ ] CI passes (Go build + code-check)
- [ ] `go mod tidy` produces no diff
- [ ] querynodev2 TestInit_QueryHook passes with new yamlConfigWriter
issue: https://github.com/milvus-io/milvus/issues/47919🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Li Liu <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
Related to #46199
## Summary
Remove 5 unused or misused Go dependencies to reduce module bloat and
consolidate overlapping libraries:
- **`mgutz/ansi`** → replaced with inline ANSI escape codes (only used
for 3 color constants in migration console)
- **`valyala/fastjson`** → replaced with `tidwall/gjson` (only 1 file
used fastjson; gjson is already used in 22+ files)
- **`google.golang.org/grpc/examples`** → replaced with existing
`rootcoordpb` (test file pulled in entire grpc examples repo for a mock
server)
- **`remeh/sizedwaitgroup`** → replaced with `chan` semaphore +
`sync.WaitGroup` (only 2 files, trivial pattern)
- **`pkg/errors`** → replaced with `cockroachdb/errors` (the project
standard; `pkg/errors` was used in 1 file)
## Behavior change: DeleteLog.Parse() fail-fast on missing fields
The `fastjson` → `gjson` migration adds explicit `Exists()` validation
for `ts`, `pk`, and `pkType` fields in the JSON parsing branch.
Previously, both fastjson and gjson would silently return zero values
for missing fields, causing `dl.Pk` to remain nil and panicking
downstream. The new code fails fast with a descriptive error at parse
time. This is a defensive improvement (the original code had identical
silent-failure behavior).
## Performance impact
| Change | Path type | Perf delta | Matters? |
|--------|-----------|------------|----------|
| `pkg/errors` → `cockroachdb/errors` | Cold (offline CLI tool
`config-docs-generator`) | Negligible | No |
| `mgutz/ansi` → inline ANSI codes | Cold (offline CLI tool
`migration/console`) | Marginally faster (eliminates map lookup) | No |
| `fastjson` → `gjson` (`DeleteLog.Parse`) | Warm — old-format deltalog
deserialization only | **~2.5x slower** per JSON parse (143ns→361ns) |
**No** — see below |
| `grpc/examples` → `rootcoordpb` | Test only (`client_test.go`) | None
| No |
| `sizedwaitgroup` → chan+WaitGroup | Test only (`wal_test.go`,
`test_framework.go`) | None | No |
### fastjson → gjson regression detail
`DeleteLog.Parse()` is called per-row during deltalog deserialization,
but **only for the legacy single-field format**. The new multi-field
parquet format (`newDeltalogMultiFieldReader`) reads pk/ts as separate
Arrow columns and bypasses `Parse()` entirely. Legacy deltalogs are
rewritten to parquet format during compaction, so this is a dying code
path. Additionally, deltalog loading is I/O-bound — the JSON parse cost
(~361ns/row) is negligible compared to disk read and Arrow
deserialization overhead.
Benchmark (Go 1.24, arm64):
```
BenchmarkFastjsonSmall-4 8,315,624 143.1 ns/op 0 B/op 0 allocs/op
BenchmarkGjsonOptimized-4 3,321,613 361.4 ns/op 96 B/op 1 allocs/op
```
## Test plan
- [x] CI build passes
- [x] CI code-check passes
- [ ] CI ut-go passes
- [ ] CI e2e passes
- [x] Boundary test cases added (bare number, missing pkType/ts/pk)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Li Liu <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
generated a library that wraps the go expr parser, and embedded that
into libmilvus-core.so
issue: https://github.com/milvus-io/milvus/issues/45702
see `internal/core/src/plan/milvus_plan_parser.h` for the exposed
interface
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced C++ API for plan parsing with schema registration and
expression parsing capabilities.
* Plan parser now available as shared libraries instead of a standalone
binary tool.
* **Refactor**
* Reorganized build system to produce shared library artifacts instead
of executable binaries.
* Build outputs relocated to standardized library and include
directories.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Buqian Zheng <[email protected]>
Related to #44620
Related to unstable ut "internal/querycoordv2 TestServer/TestNodeUp"
Introduce SessionWatcher interface to fix race condition and goroutine
leak that caused unstable unit test TestServer/TestNodeUp.
Changes:
- Add SessionWatcher interface with EventChannel() and Stop() methods
- Refactor WatchServices() to return SessionWatcher instead of raw
channel
- Fix cleanup order in QueryCoordV2: stop watcher before session
- Update DataCoord, ConnectionManager to use SessionWatcher
- Add MockSessionWatcher for testing
Fixes race condition between session context cancellation and internal
loop exit. Eliminates goroutine leak by providing explicit lifecycle
management.
---------
Signed-off-by: Congqi Xia <[email protected]>
issue: #43897
- Alter collection/database is implemented by WAL-based DDL framework
now.
- Support AlterCollection/AlterDatabase in wal now.
- Alter operation can be synced by new CDC now.
- Refactor some UT for alter DDL.
---------
Signed-off-by: chyezh <[email protected]>
related: #39173
Core Features
* Parquet File Analysis: Analyze Milvus binlog Parquet files with
metadata extraction
* MinIO Integration: Direct connection to MinIO storage for remote file
analysis
* Vector Data Deserialization: Specialized handling of Milvus vector
data in binlog files
* Interactive CLI: Command-line interface with interactive exploration
Analysis Capabilities
* Metadata & Vector Analysis: Extract schema info, row counts, and
vector statistics
* Data Export: Export data to JSON format with configurable limits
* Query Functionality: Search for specific records by ID
* Batch Processing: Analyze multiple Parquet files simultaneously
User Experience
* Verbose Output: Detailed logging for debugging
* Error Handling: Robust error handling for file access and parsing
* Flexible Output: Support for single file and batch analysis formats
---------
Signed-off-by: shaoting-huang <[email protected]>
Co-authored-by: nico <[email protected]>
#35856
1. Add function-related configuration in milvus.yaml
2. Add null and empty value check to TextEmbeddingFunction
Signed-off-by: junjie.jiang <[email protected]>
issue: #38715
- Current milvus use a serialized index size(compressed) for estimate
resource for loading.
- Add a new field `MemSize` (before compressing) for index to estimate
resource.
---------
Signed-off-by: chyezh <[email protected]>
issue: ##36621
- For simple types in a struct, add "string" to the JSON tag for
automatic string conversion during JSON encoding.
- For complex types in a struct, replace "int64" with "string."
Signed-off-by: jaime <[email protected]>
issue: #33285
- using streaming service in insert/upsert/flush/delete/querynode
- fixup flusher bugs and refactor the flush operation
- enable streaming service for dml and ddl
- pass the e2e when enabling streaming service
- pass the integration tst when enabling streaming service
---------
Signed-off-by: chyezh <[email protected]>
Signed-off-by: bigsheeper <[email protected]>
