## What this fixes
Linux builds currently enable split DWARF by default, but Milvus
installation and published debug images do not include the generated
`.dwo` files. The installed ELF files therefore contain dangling DWO
references and GDB cannot load the expected debug information.
## Changes
- make split DWARF an explicit CMake opt-in, defaulting to OFF
- expose `USE_SPLIT_DWARF=ON` through `scripts/core_build.sh` for
build-only workflows that preserve `.dwo` files
- keep the existing release image, tag, multi-arch manifest, and strip
pipeline unchanged
With the default build, debug information remains self-contained in the
installed ELF files. The existing release pipeline can continue
publishing the unstripped build as `-debug` and stripping the same build
for the regular image without copying sidecar debug files into
production images.
## Release impact
- regular stripped images should retain their current layout and strip
behavior
- debug images can grow because debug sections are embedded instead of
being left behind in the build tree
- Release builds still use the existing `-g1`; changing the debug level
or publishing `.dwp` artifacts is intentionally out of scope
- the first C++ build after this flag change may have lower cache reuse
## Validation
- `bash -n scripts/core_build.sh`
- parsed `DefineOptions.cmake` with CMake
- verified `MILVUS_USE_SPLIT_DWARF` defaults to OFF and accepts explicit
ON
- `git diff --check`
Supersedes #53188, which was recreated from the contributor fork to
follow the repository contribution workflow.
Fixes#53182
Signed-off-by: Zhikun Yao <[email protected]>
## Summary
Add `scripts/milvus-debug.sh` to automate GDB coredump analysis using
the `-debug` image that accompanies each Milvus release starting from
v2.6.14.
## Background
Starting with v2.6.14, each Milvus release publishes two image variants:
| Tag | Content | Size | Use |
|-----|---------|------|-----|
| `milvusdb/milvus:<tag>` | **stripped** (production) | ~1/3 |
Production deployment |
| `milvusdb/milvus:<tag>-debug` | unstripped (full symbols) | full | GDB
coredump analysis |
The stripped production image is **byte-identical** to the debug image
in its code sections (`strip --strip-debug` removes only `.debug_*`
sections and symbol tables). So GDB can use the debug image's symbols to
resolve addresses in a coredump produced by the stripped image.
## What the script does
1. Pulls the debug image
2. Mounts the coredump file read-only into a container
3. Installs `gdb` on demand (not present in the runtime image by
default)
4. Launches GDB with `solib-search-path=/milvus/lib` so symbols from all
shared libraries (`libknowhere.so`, `libtantivy-binding.so`, etc.)
resolve correctly
5. Auto-prints the initial backtrace
## Typical usage after a production crash
```bash
# 1. Copy the coredump out of the crashed pod
kubectl cp <ns>/<pod>:/tmp/cores/core.<pid> ./core.<pid>
# 2. Run this script (GDB starts automatically with backtrace)
./scripts/milvus-debug.sh ./core.<pid> \
milvusdb/milvus:v2.6.14 milvusdb/milvus:v2.6.14-debug
```
## Test plan
- [x] `bash -n scripts/milvus-debug.sh` (syntax check passes)
- [x] Mirrors the documented procedure in the
strip-image-coredump-debug-guide Feishu doc
- [ ] Manual validation on a real coredump (waiting for v2.6.14
production crash; can be exercised with any coredump for now)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Zhikun Yao <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
zhikunyao