John SmithandClaude Code 8239009233 Review and harden CachyLlama build system ...

Retire the broken model downloader (BYO model system), then tighten the
stack around it.

Build / image hardening
- Dockerfile: pin CachyLLama commit via build arg, run as non-root
  llama user (UID/GID 1000), chown data dirs instead of chmod 777,
  fix LD_LIBRARY_PATH mismatch, raise healthcheck start-period for
  Vulkan init.
- Dockerfile.litellm: pin litellm base to v1.40.13-stable, run as
  non-root USER litellm.
- docker-compose.yml: drop local-breaking image: override, drop
  ipc:host, restore plan-mandated flags (--cache-ssd-system-prompts,
  --cache-ssd-system-max-days, -ub 2048 on embedder), externalize
  cache flags into .env, switch command: to JSON-array form.

Security / reliability
- Redis: requirepass via REDIS_PASSWORD, drop host-port publishing,
  pin redis-stack to 7.4.0-v3.
- litellm-config.yaml: switch os.environ/VAR to portable ${VAR}
  Jinja syntax, pass REDIS_PASSWORD to the redis-semantic cache.
- docker-compose.yml: add healthcheck on every service, upgrade
  depends_on to condition: service_healthy, segregate internal vs
  edge networks.

CI
- Bump build-push-action to v6 with provenance: true.
- Add hadolint, smoke test (llama-server --help), Trivy CRITICAL scan.
- Build and publish a separate cachyllama-litellm image.

Repo hygiene
- Add .dockerignore (keeps .env, data/, docs, mitm db out of context).
- Add README documenting the BYO workflow, API examples, verification
  commands, and known limitations.

Co-Authored-By: Claude Code <[email protected]>

2026-09-08 18:07:44 +00:00

..

docker-build.yml

Review and harden CachyLlama build system

2026-09-08 18:07:44 +00:00