## What changed
- Replace the unavailable Quay MinIO image in Docker and Ansible
deployment definitions with
`milvusdb/minio:RELEASE.2024-12-18T13-15-44Z` from Docker Hub.
- Use a multi-architecture image supporting Linux amd64, arm64, and
ppc64le.
This is a follow-up to #53549 after the Quay image became unavailable.
## Validation
- Synced the image from the approved AWS US VDC Harbor source to Docker
Hub through CICD Portal record `11462` / Jenkins
`vdc-harbor-to-dockerhub #20`.
- Verified the Docker Hub OCI index digest is
`sha256:1dce27c494a16bae114774f1cec295493f3613142713130c2d22dd5696be6ad3`,
matching the source.
- Verified the index contains Linux amd64, arm64, and ppc64le manifests.
- Started the Docker Hub image on a Linux arm64 Kubernetes node and
verified `mc ready local` succeeds.
- Ran `docker compose config --quiet` for the root, dev, Apple Silicon,
standalone CPU, and standalone GPU Compose files.
- Parsed the Ansible MinIO task as YAML and ran `git diff --check`.
Signed-off-by: Zhikun Yao <[email protected]>
issue: #51221
related: #49775
## What this changes
- Add a one-shot volume initializer to the CPU and GPU standalone
Compose files.
- Resolve the numeric UID/GID from the same Milvus image and migrate
`/var/lib/milvus` before the non-root server starts.
- Write a UID/GID-specific marker only after a successful recursive
migration, so large existing volumes are not scanned on every start and
interrupted migrations remain retryable.
- Prepare the bind mount in `standalone_embed.sh`, including when a
previously created container is stopped after a permission failure.
- Teach Docker deployment health checks to exclude one-shot services.
- Document the ownership and restore contract.
The initializer has no network, uses a read-only root filesystem, and
mounts only the Milvus data directory. The Milvus server continues to
run as `milvus:milvus`.
This fixes both fresh rootful-Docker installations, where Docker creates
the bind source as `root:root`, and upgrades from root-running images
with root-owned data or cache files. It avoids the ineffective
image-layer `chmod` approach from #51226 because a bind mount hides
image-layer ownership and mode.
## Compatibility
- The recursive ownership migration runs once for each image UID/GID.
- A failed or interrupted migration does not create the marker and is
retried.
- Root-running older images can still read and write the migrated files
on rollback.
- After restoring files owned by another user, remove
`.milvus-volume-owner-*` before starting Milvus to rerun migration.
## Validation
- `git diff --check`
- `bash -n scripts/standalone_embed.sh
tests/python_client/deploy/utils.sh
tests/python_client/deploy/check_healthy.sh`
- `docker compose -f deployments/docker/standalone/docker-compose.yml
config --quiet`
- `docker compose -f
deployments/docker/gpu/standalone/docker-compose.yml config --quiet`
- Verified the deployment health-count expressions return the three
long-running services.
- Simulated the existing release bump regexes and confirmed both
init/server image references and `MILVUS_IMAGE` are updated.
Container-level startup was not run locally because the Docker daemon is
unavailable; the Linux deployment CI must exercise the fresh-install and
root-owned upgrade paths.
---------
Signed-off-by: Zhikun Yao <[email protected]>
Native support for Google cloud storage using the Google Cloud Storage
libraries. Authentication is performed using GCS service account
credentials JSON.
Currently, Milvus supports Google Cloud Storage using S3-compatible APIs
via the AWS SDK. This approach has the following limitations:
1. Overhead: Translating requests between S3-compatible APIs and GCS can
introduce additional overhead.
2. Compatibility Limitations: Some features of the original S3 API may
not fully translate or work as expected with GCS.
To address these limitations, This enhancement is needed.
Related Issue: #36212
issue: #33285
- using streaming service in insert/upsert/flush/delete/querynode
- fixup flusher bugs and refactor the flush operation
- enable streaming service for dml and ddl
- pass the e2e when enabling streaming service
- pass the integration tst when enabling streaming service
---------
Signed-off-by: chyezh <[email protected]>
Signed-off-by: bigsheeper <[email protected]>
- fix graceful shutdown by using "exec bin/pulsar" when starting in a shell script
so that OS signals can get delivered to Pulsar process to fix graceful shutdown
- simplify and fix configuration issues
- disable Pulsar Functions worker and related key-value storage service so that
resource usage is reduced
- upgrade to use Pulsar 2.7.3 since 2.6.x isn't actively maintained
Signed-off-by: Lari Hotari <[email protected]>
zhikunyao