mirror of
https://github.com/milvus-io/milvus.git
synced 2026-10-12 02:19:10 +00:00
tinswzyandClaude Opus 5
d731ae60b6
issue: #43638 ### What Bumps `github.com/zilliztech/woodpecker` from v0.1.42 to v0.1.45 in the root and `pkg` modules, moves the woodpecker image tag in the three service-mode e2e helm values to match, and **exposes the six configuration keys this range adds** — the previous bumps in this series were dependency-only because woodpecker's config schema had not moved; this one has. `v0.1.42..v0.1.45` is twenty-one commits. The ten through v0.1.44: ``` dc9666e fix(hardware): drop the unused GPU memory helpers (#333) 2c072a4 enhance(logstore): bound what a node takes on, and watch its memory while doing it (#329) 335f151 enhance(log): bound the compaction pass and each attempt against a node (#324) d5f5d09 fix(ci): take MinIO from quay.io, which is where it still exists (#331) 9b65f40 fix(segment): index the fence LAC by the replicas that may be missing it (#326) 31ca09d enhance(logstore): size the shared flush pool from config, not the host's cores (#325) f36986b enhance(compaction): drop one payload copy and the pool around a contiguous read (#322) 068fb0b enhance(log): jitter the auditor's first tick and bound the compaction pass (#323) 7efb0a5 enhance(config): expose the two settings missing from woodpecker.yaml (#321) 0d8ac26 enhance(log): stop the auditor re-logging a no-op compaction pass on local storage (#313) ``` and eleven more in v0.1.45, all observability and operator work this embedding does not reach — new metrics (#342 #343 #344 #345 #355), a monitoring dashboard row and its fixtures (#348 #350), an admin endpoint for locating orphaned local data (#353), and the k8s operator's init image (#334 #336). The exception is #347, which is the only one that touches the module graph; see below. v0.1.44..v0.1.45 adds no configuration keys, so the six exposed here are still the whole set. ### The GPU link fix `#333` deletes woodpecker's unused GPU memory helpers. Their cgo preamble defined `getAllGPUMemoryInfo` without `static`, so the symbol had external linkage and collided with Milvus's own copy in `pkg/util/hardware`. A build tag is a property of a build rather than of a module, so a GPU build compiled woodpecker's file even though woodpecker never passes `-tags cuda` itself. **This branch is not affected** — [#49383](https://github.com/milvus-io/milvus/pull/49383) made its copy `static` in April, leaving one definition and a clean link. 2.6 never received that fix, so its GPU build is broken today; #53495 resolves it with this same bump. What this branch gains is the removal of a CUDA cgo compilation unit from every Milvus GPU build, for code woodpecker never called. ### What matters for Milvus **#326 — a recovery boundary below data a client was told was durable.** `resolveFenceLAC` indexed the sorted fence replies at `len(results)-ackQuorum`, which is the acknowledged boundary only when every replica answered. One failed reply shifted the index down a position: with tails `[0,0,-1]` and `Aq=2`, a lost reply left `[0,-1]` and selected `-1`, completion succeeded at `-1`, compaction sealed an empty object, and a fresh reader could not retrieve an entry the client had already been told was committed. A missing reply is not evidence of an empty replica, so the index is now `writeQuorum-ackQuorum` — the same tolerance the append path spends before giving up. That change forced a second one in the same PR. Completion had required `ackQuorum` replicas to *cover* the boundary locally, which cannot be satisfied once the boundary may be held by fewer, and failing it is not free: `CompleteSegment` has already reached every node by the time the count happens, and in service mode `Finalize` burns the boundary into a durable footer that refuses any later `Finalize` at a different one. So the segment could never complete — and because `OpenLogWriter` fences and completes every active segment before returning a writer, **one wedged segment stops writes on the whole log**. Completion now requires an ack quorum to have *recorded* the boundary rather than to cover it. **#324 / #323 / #329 bound work that was previously unbounded**, all on the auditor's compaction path, which runs its passes in sequence so an unbounded one stalls that log's truncate state, snapshot publication and orphan sweep behind it: - **#324** gives each compaction attempt against a node its own deadline — without one, a node that hangs rather than failing holds the call forever, since the auditor's context carries no deadline and the node's own ceiling is only enforced while the node is still running it. It also bounds how long one cycle spends *starting* compactions. - **#323** caps how many segments one cycle compacts (64, oldest first) and jitters each auditor's first tick, so writers created together — a node restart, a WAL failover, a channel rebalance — stop ticking in lockstep. - **#329** adds node-wide admission control for compaction memory. A node holds segments from many logs, each driven by its own auditor on its own schedule, and nothing bounded how many it took on at once. **#322** removes a payload copy from every merged block and replaces a pooled per-block read with one contiguous `ReadAt`, on the staged-storage compaction path. The remainder are woodpecker's own CI and a local-storage logging fix. ### Config This is the part that differs from the previous bumps. Milvus builds its woodpecker configuration from `config.NewConfiguration()` and overrides selected fields, so a key it does not map silently takes woodpecker's in-code default and cannot be tuned at all. All six new keys are exposed: | Key | Default | Why it is worth reaching | |---|---|---| | `woodpecker.logstore.syncScheduler.maxWorkers` | 32 | The node's flush concurrency. #325 made it configurable precisely because deriving it from the host's core count could not saturate a device's IOPS — leaving it unexposed would waste that change. | | `...segmentCompactionPolicy.maxInflightMemory` | 1000000000 | Memory a node reserves for concurrent compaction. Milvus already exposes `maxSize` and `maxParallelUploads`, which raise what one compaction costs; without this the node has no say in the total. | | `...segmentCompactionPolicy.memoryHighWatermark` | 0.7 | Stops new compactions when the node is near its memory limit for any reason. Pairs with the above. | | `...segmentCompactionPolicy.timeout` | 300s | The node's ceiling on one compaction. | | `woodpecker.client.auditor.compactionAttemptTimeout` | 330s | The client's deadline for one node to answer. **Exposed together with the ceiling above on purpose:** the two combine as a minimum, so exposing either alone offers a knob that cannot take effect. | | `woodpecker.client.auditor.compactionPassBudget` | 60s | How long one auditor cycle keeps starting compactions. | **Every default matches what woodpecker ships, so behaviour is unchanged for a deployment that sets none of them.** `maxInflightMemory` is written as a decimal `1000000000` rather than `"1G"` for that reason — `GetAsSize()` reads `"1G"` as 2^30, which would raise the ceiling 7% above the library's own default. (The pre-existing `segmentCompactionPolicy.maxSize: 2M` has the same 2^20-vs-decimal divergence against woodpecker's 2000000; it is left as it is.) `syncScheduler.maxWorkers` is marked `refreshable:"false"` — it sizes the worker pool built at logstore construction, so a runtime change would not rebuild it. `configs/milvus.yaml` is regenerated by the config tool, not hand-edited. ### Compatibility - **API**: unchanged for every symbol Milvus uses. `pkg/streaming/walimpls/impls/wp` consumes `NewClient` / `NewEmbedClient` / `StopEmbedLogStore` / `LogExists` / `CreateLog` / `OpenLog` / `LogReader` and the message-id helpers; nothing in that surface moved. - **Transitive deps**: woodpecker's `go.mod` does move this time. #347 takes gopsutil from v3.23.7 to v4.25.10, adding `ebitengine/purego` and bumping `perfstat`, `tklauser` and `wmi`. **Milvus's graph is unaffected**: it already requires every one of those at the same or a higher version — `gopsutil/v4 v4.25.10` and `purego v0.9.0` match exactly — so MVS resolves nothing new and the `go.sum` delta is still the two woodpecker lines per module. `gopsutil/v3` stays in `pkg/go.mod`, now reached through pulsar-client-go's testcontainers dependency rather than through woodpecker. - **On-disk format**: unchanged. #326 changes which boundary recovery selects, not what is written. - **Metrics**: #329 adds five `woodpecker_server_compaction_admission_*` series. No dashboard under `deployments/` references any `woodpecker_*` series, so nothing in this repo needs updating. - **New behaviour worth knowing about**, none of which is a regression: - A logstore node under memory pressure or at its compaction ceiling now refuses a compaction with a retryable error. The client tries the next replica and the auditor retries next cycle, so this classifies as `transient` in `WpSegmentCompactionFailuresTotal` rather than as a failure — but a series that was previously always zero can become non-zero under load. - Each auditor's first cycle after a writer opens is delayed by a random fraction of its interval (#323), up to 10s on the default. ### Test `pkg/util/paramtable` gains default-value assertions for the six new keys. Beyond that this is a dependency bump plus configuration plumbing, so the coverage is woodpecker's own tests for the commits above and the existing Milvus e2e suites. Verified locally: - both modules resolve to v0.1.45, and the `go.sum` delta is exactly the two woodpecker lines per module — populated additively with `go list` under `-mod=mod` (which only adds) rather than by `go mod tidy`, and checked with `go mod verify` - `pkg/util/paramtable` builds, vets and tests clean - `pkg/streaming/walimpls/impls/wp` and `pkg/mq/msgstream/mqwrapper/wp` vet clean against v0.1.45 - the six values reach the library end to end: a throwaway test drove `paramtable.Init()` → `builderImpl.getWpConfig()` and asserted each landed in the woodpecker `Configuration` with `Validate()` passing. Removed before commit. - `configs/milvus.yaml` regenerated with the config tool; the diff is 31 added lines and no reordering of existing content No Milvus-side behavioural claim is made beyond the API / config / dependency-graph audit above. The effects of #326, #324, #323 and #329 are woodpecker-internal and have not been reproduced or measured on a Milvus cluster in this PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: tinswzy <[email protected]> Co-authored-by: Claude Opus 5 (1M context) <[email protected]>