The bridge commands never go through the dry-run backend, so the
inherited --dry-run flag was silently ignored: `bridge convert` still
wiped and rewrote the output directory, pulled the transformer image
and ran it, and `bridge transformations create` still wrote its
destination folder.
Fail fast with an explicit error before any side effect instead of
pretending to honor the flag.
Signed-off-by: Guillaume Lours <[email protected]>
compose-go's loader now silently renames both deprecated
x-initialSync and x-develop extensions to their official
initial_sync and develop attributes before compose ever sees them
as extensions, so the legacy lookups and deprecation warnings for
both can no longer be reached. That's why the e2e test asserting on
the x-initialSync warning broke under the bump.
Simplify both lookups to rely on the promoted fields directly, and
rewrite the e2e test to check the legacy attribute still triggers a
real initial sync instead of asserting on an unreachable message.
Signed-off-by: Guillaume Lours <[email protected]>
down's outer ops dispatch bounded only how many ops ran concurrently,
separate from the semaphore already shared by image removal -- a
single network plus --rmi could spend up to 2x the configured
budget. Network and volume removal now share that same semaphore.
up's post-attach log streaming called ContainerInspect directly and
passed no limiter to doLogContainer, unlike `compose logs --follow`'s
equivalent path, so it never shared --parallel's budget either.
The plan executor also held a node's slot for its whole dependency
wait instead of just its own work, and internal/sync/tar.go's file
sync used a hardcoded cap of 16 instead of --parallel.
Signed-off-by: Guillaume Lours <[email protected]>
Co-Authored-By: Claude Sonnet 5 <[email protected]>
`docker compose cp` fanned out to every matched container through a
bare unbounded errgroup, ignoring --parallel entirely -- the same
class of gap already closed everywhere else in this package. Routed
it through the existing forEachContainerConcurrent helper instead of
a bespoke fan-out.
inspectWithSlot and doLogContainer both release a semaphore slot
early via an explicit call rather than a defer, so a panic between
acquire and that release leaked the slot forever. Guarded both with
a panic-safe release, scoped tightly in doLogContainer so a later
panic in the same function can't trigger a double release.
Signed-off-by: Guillaume Lours <[email protected]>
Co-Authored-By: Claude Sonnet 5 <[email protected]>
down and stop rebuilt their per-container limiter on every service
visit, so independent services could each spend a full budget
concurrently -- the same leak already fixed on restart. start's
InDependencyOrder call had no concurrency option, leaving every
`up`'s container-start phase fully unbounded. wait.go goes back to
unbounded on purpose: ContainerWait blocks per container
indefinitely, so capping it would serialize waits meant to run
concurrently.
Also extracts the shared per-container fan-out into one helper reused
by restart/down/stop, and dedupes the concurrency-guard test fixtures.
Signed-off-by: Guillaume Lours <[email protected]>
Logs --follow reused the bounded errgroup for both the indefinite log
streams and the monitor goroutine, so once maxConcurrency streams
opened the monitor never started and further services never showed
logs. restart gave each service its own budget, so ready services
could multiply maxConcurrency instead of sharing it.
Bound both with a semaphore that gates only the connect/restart call,
never the indefinite work that follows, sharing one instance across
restart's services. Adds regression tests, plus one guarding the
topological node order a bounded errgroup now relies on in executor.
Signed-off-by: Guillaume Lours <[email protected]>
errgroup.SetLimit(0) means "allow zero goroutines", not "unlimited",
and maxConcurrency's Go zero-value is 0 — only NewComposeService sets
it to -1 explicitly. Any composeService{} literal built without it
(common in tests) silently deadlocked at every call site that called
SetLimit unconditionally.
Separately, --parallel/COMPOSE_PARALLEL_LIMIT was only ever wired
into pull, push, and the dependency-graph traversal, despite the docs
promising a generic bound on "concurrent engine calls". Every other
bulk operation (kill, pause, down, the up/create plan executor, logs,
ps, top, wait, restart, remove, images, model pulls, watch) launched
one goroutine per container/image/DAG-node with no cap at all.
Fix both via a shared newLimitedErrgroup helper applied at every
fan-out site, threading maxConcurrency through forEachContainerConcurrent
and ImagePruner, which had no access to composeService. Add a
regression test for the zero-value case.
service_containers.go's waitDependencies is intentionally left
unguarded: it's a per-dependency ticker poll, not a burst of engine
calls.
Signed-off-by: Guillaume Lours <[email protected]>
Wait has no real container to observe in dry-run: getContainers
falls back to the live daemon and finds nothing, so the command
failed with a misleading "no containers for project" error instead
of signaling upfront that the operation doesn't apply under
simulation.
Signed-off-by: Guillaume Lours <[email protected]>
DryRunClient fakes container state as permanently running/healthy,
which never satisfies the exited-state check that
service_completed_successfully waits for. With up -d passing no
default timeout, the wait loop polled forever and dry-run never
returned, even though no container was ever really started.
Signed-off-by: Guillaume Lours <[email protected]>
publish was promoted to a regular Compose command in 13618756d
(2025-03-12) but was left registered under 'alpha' as well.
Remove the leftover duplicate registration and the now-orphaned
generated docs (compose_alpha_publish.md,
docker_compose_alpha_publish.yaml), and regenerate
docker_compose_alpha.yaml accordingly.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
removeTaggedImagesOp switched to a plain errgroup.Group so one
image's failure can't cancel the others still in flight, but
Wait() only ever surfaces the first error returned to it,
silently dropping any other concurrent failures. Aggregate every
failure instead, the same way removeImages already separates
tolerated outcomes (still-in-use, already-gone) from real errors
before joining them.
Also give a tagged-image listing failure its own "Tagged images"
label instead of the generic Compose-wide one, matching the
dedicated label the dangling-image path already has.
Signed-off-by: Guillaume Lours <[email protected]>
removeTaggedImagesOp shared an errgroup.WithContext-derived ctx across
every image removal goroutine. A genuine failure on one image canceled
that ctx, so every image still queued behind it failed with "context
canceled" instead of ever being attempted -- a regression from the
independent per-image ops it replaced.
Also extracts the Conflict/NotFound tolerance rule and the
orphaned-image check into single shared helpers instead of leaving two
independently maintained copies to drift apart, and drops a return
bucket nothing outside its own test consumed.
Signed-off-by: Guillaume Lours <[email protected]>
When a dangling-image batch had both a genuine removal error and an
image tolerated as still in use, with nothing actually removed, the
error path reported only the genuine failure and silently dropped the
still-in-use count — the same information loss the "Removed" label
was already fixed to avoid, just on its error-reporting sibling
branch. Both facts now show up together regardless of which path the
batch's outcome takes.
Signed-off-by: Guillaume Lours <[email protected]>
A dangling image still in use failed the whole op, while a tagged
image or volume in the same state only gets a swallowed warning via
removeResource — same daemon condition, inconsistent outcome. Dangling
images now get the same tolerance. Also stop claiming "Removed" when
a removal failure means nothing actually was, bound the tagged-image
fan-out the same way the dangling one already is, and share the
listing+filter logic between the down and watch paths instead of
duplicating it.
Signed-off-by: Guillaume Lours <[email protected]>
Found during review of #14219: `ensureImagesDown` still listed tagged
images to prune synchronously and returned the error immediately,
before `down` had scheduled the already-built network/volume cleanup
ops onto its errgroup — the exact same hazard just fixed for dangling
images, on a different code path. The listing is now deferred to when
its own op runs, so a failure surfaces as a visible error without
blocking sibling ops.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
`down --rmi` always printed "Dangling images ... Removed" even when
there was nothing to remove, on every repeat `down` (#14219). The
dangling-images removal now: stays silent when there's nothing
eligible, same as removeNetwork's silent skip; shows a visible error
instead of the misleading label when the listing itself fails, without
aborting the rest of `down`; and keeps the "Removed" status visible
alongside the real daemon error when some images fail to remove for a
genuine reason (not a benign already-gone race).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
Match the sentence-case convention used elsewhere in our doc
guidelines instead of title-casing each heading.
Signed-off-by: Guillaume Lours <[email protected]>
AI_POLICY.md duplicated rules moby/.github now defines once for every
Docker/Moby project; the two copies would drift the moment either side
changed a rule. Sync it in daily via a scheduled workflow, modeled on
compose-spec's existing schema sync, and move every Compose-specific
rule (approved-issue gate, the AI_AGENT_DISCLOSURE.md checkpoint,
platform-testing restriction, make fmt) into CONTRIBUTING.md and
AGENTS.md so it survives the next sync instead of being overwritten.
Signed-off-by: Guillaume Lours <[email protected]>
thaJeztah, vvoland, and crazy-max raised several gaps during review:
AI co-author trailers don't belong in commits (disclosure belongs in
the PR description), unreviewed AI replies and wall-of-text issues
waste maintainer time, and unrelated-code scope creep and contribution
farming weren't called out anywhere.
Addresses review comments on docker/compose#13550.
Signed-off-by: Guillaume Lours <[email protected]>
Unit tests alone didn't cover the exact failure: an attached up
hanging when the project is stopped/removed externally while a
service sits in restart backoff. The new test reproduces that
sequence end to end, so a future regression on this path breaks
loudly instead of slipping through as a passing unit suite. A
RestartCount guard narrows the race with the backoff expiring, so a
missed window fails loudly instead of passing silently through the
pre-existing die-based path.
Signed-off-by: Guillaume Lours <[email protected]>
Since 2.39.3 the monitor relies solely on die events to stop tracking
containers, but a container stopped while in restart backoff emits only
stop and destroy events, so an attached `up` hangs forever after an
external `stop`/`down`. Handle destroy as terminal, and on stop inspect
the container to distinguish a definitive stop from the transient one
emitted during a ContainerRestart, which keeps watch sync+restart
(#13161) working.
A plain stop and its die aren't ordered, so evicting as soon as a stop
looks definitive can end the loop before an already-queued die is
drained, losing the exit code --exit-code-from and
--abort-on-container-exit depend on. Eviction on stop now requires a
die already observed for that container; otherwise it defers to the
pending die.
Fixes#13985
Signed-off-by: Guillaume Lours <[email protected]>
`docker compose port SERVICE PORT` picks the first matching publisher
as its answer, but containerPublishers()'s sort.Slice does not
guarantee tie order, so a dual-stack publish (same target port on
both an IPv4 and an IPv6 address) could non-deterministically return
either one. Switch to sort.SliceStable to preserve the daemon's
original order for ties.
Also drops a dead length check and a duplicated host:port formatter.
Signed-off-by: Guillaume Lours <[email protected]>
compose watch triggered one rebuild per debounced batch, so a burst of
edits arriving faster than the debounce window (e.g. from a coding
agent editing several files) queued up several rebuilds in a row
instead of a single consolidated one.
Rebuilds now run asynchronously through a small scheduler that
coalesces requests received while a rebuild is in flight into one
trailing rebuild, without adding latency to an isolated edit. This
also closes a race where a plain restart could run concurrently with
an asynchronous rebuild of the same service.
Signed-off-by: Guillaume Lours <[email protected]>
Compose silently ignored several spec-valid attributes it doesn't
honor outside Swarm mode: deploy.update_config, credential_spec,
ports[].mode: host, and others had no visible effect on a plain
`docker compose up`.
Detection is wired into compose-go's loader as a fail-closed
allowlist: every attribute path the schema declares, minus the
handful this runtime deliberately doesn't implement, plus a few
value-conditional checks for attributes only unsupported for one
specific value. A newly-specified attribute gets flagged
automatically, with no corresponding code change needed here.
Depends on an unreleased compose-go change; go.mod carries a
temporary replace to a fork branch until it's released.
Signed-off-by: Guillaume Lours <[email protected]>
`bridge convert -o <dir>` unconditionally ran `os.RemoveAll` on the
output directory and discarded any error, so a typo (`-o .`, `-o
$PWD`, `-o ~`) silently destroyed unrelated user data.
An empty or missing output directory is still created silently, but a
non-empty one now requires explicit confirmation (prompted
interactively, or via the new `--yes` flag for scripts/CI) before
being wiped.
Signed-off-by: Guillaume Lours <[email protected]>
Split SetRecursive into the existing per-GOOS watcher files instead of
a runtime check + nolint: the non-Windows stub never calls the
always-failing SetRecursive, and Windows failures now propagate as a
real error instead of silently downgrading to non-recursive.
Addresses review from thaJeztah on #14164.
Signed-off-by: Guillaume Lours <[email protected]>
bridgeCommand and transformersCommand had no RunE, so an unknown
subcommand (e.g. "bridge zzz") fell through cobra's default
non-Runnable path: help printed, exit 0. Scripts couldn't distinguish
a typo from success. Mirrors the same fix already shipped on the
compose root command.
Signed-off-by: Guillaume Lours <[email protected]>
When a service set `build.provenance: false` (or `sbom: false`), the
bake attest entry was silently omitted (the field is `omitempty`),
so BuildKit applied its defaults and attached a provenance attestation
anyway. The attestation changes the manifest-list digest on every build,
which caused containers to be needlessly recreated on `up --build`.
Emit `"type=<t>,disabled=true"` instead, which BuildKit's
`Attest.UnmarshalText` parses as `Disabled:true` and forwards to
BuildKit as an explicit disable signal.
Extract `appendAttest` to avoid duplicating the three-branch switch
for both provenance and sbom.
Fixes#14111
Signed-off-by: Guillaume Lours <[email protected]>
Fixes#14114: Done() blocked forever once the render goroutine had
already exited via ctx.Done() (e.g. after SIGTERM), since it sent on an
unbuffered channel with nobody left to receive.
Closing the channel instead of sending never blocks, but a bare close()
panics the moment ttyWriter runs a second Start/Done cycle on the same
shared bus - which already happens today (run's nested create/start,
publish's nested push, rm --stop's sequential stop then remove). Start
now hands out a fresh, once-only-closable signal each cycle instead of
a single shared channel, and run/publish stop opening a redundant
nested bracket around calls already inside one.
Signed-off-by: Guillaume Lours <[email protected]>
- compose_down.md: note that retained pre_start hook containers (those
kept after a hook failure for post-mortem) are automatically removed
by compose down, with the label filter to list them beforehand.
- compose_up.md: note the retention behavior on pre_start hook failure,
how to inspect the retained container, and that it is cleaned up on
the next compose up or compose down.
Signed-off-by: Guillaume Lours <[email protected]>
A pre_start hook container retained after failure (introduced in the
previous commit) is created without ConfigHashLabel so getContainers()
and every downstream compose operation (down, ps, stop, rm) is blind
to it. Without explicit cleanup it would survive compose down.
This commit adds removePreStartHookContainers(), called from down()
after regular container teardown and before network removal (so hook
containers are gone before their networks are removed). The cleanup is
project-wide on a full compose down; scoped to the requested services
when compose down <svc...> is used. Individual ContainerRemove errors
are logged at warn level and do not abort the teardown.
Changes:
- pkg/compose/down.go: removePreStartHookContainers helper + call in down()
- pkg/compose/filters.go: add //nolint:unparam to hookFilter (param is
architecturally correct; today only preStartHookType is in use)
- pkg/compose/down_test.go: update all existing down tests to expect the
new ContainerList call; add TestDownRemovesRetainedPreStartHookContainers
and TestDownHookContainerRemovalFailureIsNonFatal
Signed-off-by: Guillaume Lours <[email protected]>
A failing pre_start hook container is now retained (AutoRemove: false)
so operators can run 'docker logs <id>' and 'docker ps -a' to diagnose
the failure. On success the container is removed explicitly, mirroring
the old AutoRemove behaviour including anonymous volumes.
Before each pre_start run, stale hook containers from a previous failed
run are detected via project+service+HookLabel filters and force-removed
so they do not accumulate.
Changes:
- pkg/api/labels.go: add HookLabel (com.docker.compose.hook)
- pkg/compose/filters.go: add hookFilter helper
- pkg/compose/pre_start.go:
- AutoRemove: false in createPreStartContainer
- HookLabel added to container labels
- runPreStartHook: explicit ContainerRemove on success; retain on failure
- removeOrphanPreStartContainers: new helper called in runPreStart
- Tests: update all existing pre_start tests for the new flow; add
feature tests (success removes, failure retains, orphan cleanup) and
coverage-gap tests (lowestNumberedContainer, waitPreStart cancel,
preStartResultErr, streamPreStartLogs error paths, old-API network
paths, ExecCreate/Attach/Inspect errors, hookExitError branches)
Coverage after: hook.go 100%, pre_start.go most functions 100%
(was 89% and 64% respectively per Codecov delta).
Signed-off-by: Guillaume Lours <[email protected]>
On top of the output-tail infrastructure from the previous commit:
Stderr bias: split the combined output tail into separate stdout and stderr
buffers. stdcopy.StdCopy(wOut, wErr, reader) now routes each stream to its
own outputTail. The error message prefers stderr (where hooks write their
actual error) and falls back to stdout when stderr is empty:
db hook exited with status 1: Table 'service.sites' doesn't exist
A new hookExitError helper encapsulates this preference.
pre_start parity: streamPreStartLogs previously short-circuited and returned
immediately when listener was nil (detached mode / compose up -d), leaving
the tail buffers empty and failure errors without context. The log stream is
now always opened. The demultiplexed streams fill separate tailOut / tailErr
buffers. On non-zero exit the stderr-biased tail is appended to the error
(same behaviour as exec hooks).
Tests:
- TestRunHook_StderrBias: stdout noise stays out of the error when stderr
has content.
- TestPreStart_DetachedModeAttachesLogs: ContainerLogs is called even with
a nil listener.
- TestPreStart_FailureIncludesTail: a failing pre_start hook carries its
stderr output in the returned error.
Signed-off-by: Guillaume Lours <[email protected]>
The --format flag defaulted to "table" instead of "", so the
opts.Format == "" guard in runPs never fired for a bare compose ps.
Setting the default to "" lets the existing config-fallback path work,
matching the behaviour of docker ps.
Fixes#13643
Supersedes #14032
Signed-off-by: Guillaume Lours <[email protected]>
createNetwork built the IPAM object in two separate blocks; the second
silently overwrote createOpts.IPAM with a fresh empty struct and never
forwarded n.Ipam.Options to the daemon. Consolidate into a single block
covering driver, config pool, and options.
Fixes#13785
Supersedes #13936
Signed-off-by: Guillaume Lours <[email protected]>
initialSyncFiles and initialSyncDirectory filtered out any host file
whose mtime predated the image creation time — which is exactly the
files that initial_sync is meant to copy. The imageCreatedTime lookup
(ContainerList + ImageInspect) was also a silent failure point when the
container list returned nothing. Remove the filter and the helper
entirely: sync all files that are not ignored or bind-mounted.
Fixes#13725
Supersedes #14031
Signed-off-by: Guillaume Lours <[email protected]>
Several e2e tests used fixed host ports (8070, 8080, 8090, etc.), global
buildx builders registered with --use, and a hardcoded IPC container name.
Under parallel execution these shared daemon-level resources caused port
conflicts, buildkitd container-name collisions, and intermittent failures.
Separately, the standalone make target hardcoded -parallel=1, forcing the
entire standalone suite to run serially even though the same tests run in
parallel in plugin mode.
Changes:
- Convert fixed host ports in four fixtures (network-test, sentences,
build-test, volume-test) to ephemeral bindings; add ServicePublishedPort
helper to framework.go to resolve the actual mapped port at runtime.
- Give each buildx-builder test a unique daemon-scoped name via BuilderName
helper, preventing container-name collisions between parallel goroutines.
Applies to TestBuildPlatformsWithCorrectBuildxConfig, TestBuildPrivileged,
TestBuildBuilder, TestBuildEntitlements, TestBuildTLS.
- Rename the fixed ipc_mode_container to a project-scoped name
(ipc_e2e-src) and update the ipc-test fixture accordingly.
- Replace the hardcoded -parallel=1 in make e2e-compose-standalone with
E2E_PARALLEL_PLUGIN?=4 and E2E_STANDALONE_PARALLEL?=4, aligning both
modes with the effective GOMAXPROCS on 4-core GitHub Actions runners.
Both variables are overridable (E2E_STANDALONE_PARALLEL=1 restores
serial local debugging).
- Raise the poll delay in watch_test.go long-running polls (90s/120s
timeout) from the default 100ms to 1s; bump cascade_test.go and
publish_test.go from 100ms to 500ms. Reduces CPU pressure without
affecting effective test duration.
The logging-driver fixture retains its fixed port 24224: the Docker fluentd
log driver encodes fluentd-address at service-start time, making ephemeral
resolution non-trivial. TestLoggingDriver is serial so there is no
collision risk in practice.
Signed-off-by: Guillaume Lours <[email protected]>
docker compose events <name> was passing service arguments straight to
the event filter without checking they exist in the project, so a typo
would produce an event stream matching nothing and block forever with
no output and no error.
Switch to projectOrName (which validates args via ToProject) before
calling backend.Events, consistent with logs, kill, and restart.
Signed-off-by: Guillaume Lours <[email protected]>
When a locally-installed binary with the same name (e.g.
example-provider) sits in an earlier PATH directory than
bin/build/, exec.LookPath resolves the wrong binary. The
old tests appended the build directory to the end of PATH,
so any pre-existing system binary shadowed the test one.
Prepending instead ensures the freshly built binary always
wins, regardless of what is installed on the developer's
machine. CI is unaffected since no such system binary
exists there.
Signed-off-by: Guillaume Lours <[email protected]>
docker-agent-action v2.0.3 raised the review job's required permission
from actions:read to actions:write (cache delete for review-lock
release, feedback artifact management). GitHub refuses to start a
reusable workflow requesting more permissions than the caller grants,
so every PR Review run since the v2.0.3 bump ended in startup_failure
and docker-agent stopped launching automatically.
Signed-off-by: Guillaume Lours <[email protected]>
compose pull switched on the raw pull_policy string: daily/weekly/every_N
never matched a case and fell through to an unconditional re-pull, and
the hook-image loop was a second interpreter that ignored the refresh
window entirely. Delegate the decision to the exact interpreter the up
path uses (mustPull), with hook images routed through the same decision
(build mapped to missing — a hook image can't be built as a fallback).
Two deliberate differences with up are kept and documented in
shouldPullImage: a service without an explicit pull_policy is always
refreshed (skipping it would turn an explicit compose pull into a no-op
once images exist), and a present latest tag is still refreshed under
missing/if_not_present — the tag is expected to move, and triggering the
pull lets the daemon negotiate with the registry, a manifest check with
no download when the local image is already current.
User-visible change (changelog): compose pull now honors
daily/weekly/every_N refresh windows instead of always re-pulling.
Signed-off-by: Guillaume Lours <[email protected]>
scale and run were the only container-creating commands that never
called applyPlatforms, yet both go through the regular create path and
its config-hash comparison (run for the dependencies it starts). With
DOCKER_DEFAULT_PLATFORM set, they hashed an empty service Platform where
up had hashed the resolved one, so every invocation recreated the
affected containers. run's project preparation is extracted to a helper
to keep runCommand under the complexity threshold.
No unit test: neither command has a test harness and the fix is the one
missing call, aligned on create/watch; the config-hash equality is
covered by the reconciler tests.
Signed-off-by: Guillaume Lours <[email protected]>
Image identities recorded for staleness detection were produced by
several independent paths yielding different digest kinds for the same
image: the platform check compared flat inspect fields while the digest
picked a manifest with the host matcher (never the service's pinned
platform), a wrong-platform summary just discarded still leaked its
digest into the label, bake substituted digests host-side in batch, and
the classic builder recorded the raw build-stream ID as-is. Any of those
mismatches makes the next up see a phantom image change and recreate
containers.
Converge every producer on one selection (matchLocalManifest /
localContentDigest): the shared parallel inspect feeds both the digest
and the platform check, platform-pinned services resolve THEIR platform's
manifest in-process (no extra API call), and both builders route through
canonicalBuiltDigest. Registry-only builds (push-only, multi-platform
without load) keep the builder-reported digest — volatile but honest, an
actual rebuild is still detected, where a stable placeholder would hide
real image changes. ensureImagesExists' final loop becomes the label's
single writer so the pinned resolution can't be overwritten, superseded
only by pull/build results already platform-resolved by their producers —
and when a pull or build refreshed the shared entry mid-run (a digest
resolved for whichever service triggered it), a service pinned on another
platform re-resolves its own with one extra inspect, in that case only.
With every producer converged, TestUpIdempotentContainerdStore is
un-skipped here.
Signed-off-by: Guillaume Lours <[email protected]>
The e2e suite only ran on graphdriver daemons, where the different kinds
of image digests coincide — the blind spot that let #13636, #13998 and
#14005 through. Add one matrix entry enabling the containerd image store,
plus TestUpIdempotentContainerdStore: two consecutive `up` runs with no
change must not recreate any container. The test is red on this
configuration (the com.docker.compose.image label is written from the
index digest on the pulling run, then compared against the per-platform
manifest digest on the next run) and skipped until the next commit
resolves the pull-path digest.
Signed-off-by: Guillaume Lours <[email protected]>
`docker compose bridge transformations create` panicked with an index
out of range when invoked without its PATH argument, as RunE indexed
args[0] without any validator. Add cli.ExactArgs(1) so the CLI reports
a proper usage error instead.
Also add cobra.NoArgs to `bridge convert` and `bridge transformations
list`, which silently ignored stray arguments.
Signed-off-by: Guillaume Lours <[email protected]>
Containers are created from a project whose service environment is
resolved (env_file merged into environment), but since v2.22.0 `config
--hash` skipped that resolution, so hashes diverged from the
com.docker.compose.config-hash label for services using env_file.
Resolve the environment of the hashed services only, so a broken
env_file or platforms on an unrelated service still doesn't prevent
hashing, and honor --no-env-resolution as an escape hatch.
Fixes#14001
Signed-off-by: Guillaume Lours <[email protected]>
pruneDanglingImagesOnRebuild probed the name-keyed built-images map with
an image ID, so the spare-check never matched and every dangling image
carrying the project label was removed on each rebuild with --prune —
not just the superseded ones. Match dangling IDs against the map's
values (the freshly built image IDs) instead.
Signed-off-by: Guillaume Lours <[email protected]>
Pulling an oci:// resource resolved each layer digest through the
registry manifests endpoint, which answers 500 when the digest points
to a non-manifest blob. containerd v2.3.0+ (pulled in by buildx v0.36
and buildkit v0.32) no longer falls back to the blobs endpoint unless
manifests returned 404, so publish/pull of compose artifacts broke.
Fetch layers directly with the descriptors already listed in the
manifest instead of resolving them again.
Signed-off-by: Guillaume Lours <[email protected]>
Test the upcoming buildx and BuildKit releases ahead of GA.
buildx v0.36.0-rc2 requires Go >= 1.26.3, so the go directive
moves from 1.25.9 to 1.26.3.
Signed-off-by: Guillaume Lours <[email protected]>
Replace the long-lived DOCKERPUBLICBOT PAT with short-lived tokens
minted through the Docker Hub OIDC connection, using the
registry-identities input of the github-builder bake workflow.
The connection rulesets cover both triggers of this workflow
(refs/heads/main and refs/tags/v*) for compose-bin and
compose-desktop-module.
Signed-off-by: Guillaume Lours <[email protected]>
Follow-up to #13603: scale, watch and shell completion loaded the
project without any tolerance option, so a missing env_file on a
service not involved in the operation aborted the command, while
up/exec/ps already tolerate this since #13156 and #13603.
Mirror the WithServices pattern: load with WithoutEnvironmentResolution
and resolve the environment once the project has been reduced to the
selected services, so targeted services still get their env_file
validated. Completion only needs names and never resolves. This also
aligns the config hash of scale-created containers with up-created
ones.
Signed-off-by: Guillaume Lours <[email protected]>
The runServices, runVolumes, runNetworks, runModels and runHash paths
called ProjectOptions.ToProject directly, bypassing the configOptions
wrapper that applies --no-consistency, --no-interpolate, --no-normalize,
--no-path-resolution, --profile filtering and env_file discarding.
Restore the variadic wrapper (mirroring configOptions.ToModel) and route
the five call sites through it.
Regression introduced by b80bb0586 (LoadProject API migration).
Fixes#13974
Signed-off-by: Guillaume Lours <[email protected]>
The caller in pkg/compose/cp.go unconditionally defers
res.Content.Close() once the call succeeds. The dry-run client
returned a zero-value result with a nil Content reader, so
`docker compose cp --dry-run <ctr>:<path> <dst>` panicked with a
nil pointer dereference. Return an empty NopCloser instead, matching
the pattern already used for the other stream results in this file.
Signed-off-by: Guillaume Lours <[email protected]>
AI_AGENT_DISCLOSURE.md was merged into the repository by accident as
part of a reconcile change (44ac2c94e). With the file permanently
present, the AGENTS.md instruction to "add" it became a no-op and
AI-prepared pull requests no longer carried any disclosure signal.
Remove the committed copy and change the instruction to add the file
with the current date, or refresh the date when the file is already
present, so every AI-prepared change includes the disclosure in its
diff.
Signed-off-by: Guillaume Lours <[email protected]>
Rely on compose-go WithImagesResolved, which now resolves dependent
images — `type: image` volume sources and pre_start hook images —
with its already-digested guard, per-call memoization and
sibling-service detection (compose-spec/compose-go#894, #899), rather
than duplicating resolution logic CLI-side. The interpolated path gets
this for free; --no-interpolate maps the raw model onto a
pseudo-project keyed by service names to reuse the same resolution,
and --lock-image-digests keeps type:image volumes in its output.
pre_start hooks can't be carried into the lock override (hook lists
are appended on merge), so generating a lock warns that hook images
stay unpinned there.
As a side effect, `compose publish` now fails fast on unresolvable
dependent images.
Fixes#13827
Signed-off-by: Guillaume Lours <[email protected]>
Move the project guide to AGENTS.md (the agents.md convention used
by Claude Code, Codex, Cursor, etc.) so all coding agents pick it
up — not just Claude. Keep CLAUDE.md as a symlink so Claude-specific
tooling keeps working unchanged.
Add an "Issue and PR Guidelines" section asking AI agents to disclose
their involvement when filing PRs or issues on a human's behalf,
mirroring the pattern used in ghostty-org/ghostty's AGENTS.md.
Signed-off-by: Guillaume Lours <[email protected]>
With the containerd image store and BuildKit provenance attestations
(the default), a built image is stored as an attested index whose
top-level digest also covers the attestation manifest. That digest
churns on every build even when the runnable content is unchanged,
so compose recreated containers on every `up --build`.
Compare the digest of the "image" kind manifest instead, selected for
the target platform and restricted to locally available manifests, so
it is deterministic and reflects only config + layers. Both the build
and up sides of the staleness check go through the same selection, and
registry-only images keep the Bake-reported digest.
Fixes#13636
Signed-off-by: Guillaume Lours <[email protected]>
`docker compose config --no-interpolate <service>` and
`docker compose config --variables <service>` load the raw model
without applying service filtering, so the full model is rendered
regardless of the services passed as arguments. Filtering will not
be supported on these paths, so emit a warning to make sure users
are no longer misled by silently ignored arguments.
Fixes#13614
Signed-off-by: Guillaume Lours <[email protected]>
- Replace ncipollo/release-action with softprops/action-gh-release
(v3.0.1, SHA-pinned), the action used across Docker repositories;
input mapping is one-to-one.
- Duplicate the artifact listing/type checks into binary-finalize:
one copy inspects the artifact as assembled, on every push/PR (lost
when the tag guard moved to job level); the release-job copy still
inspects what was downloaded and gets attached to the release.
- Set overwrite_files: false so a job re-run never replaces assets
already uploaded to an existing release: unlike ncipollo, softprops
updates an existing release instead of failing. A re-run may still
refresh the release name/notes.
Signed-off-by: Guillaume Lours <[email protected]>
The pinned codeql-action/upload-sarif v2 (v2.28.1) falls in the
vulnerable range of CVE-2025-24362 and the v2 line has no patched
release, so bump to v3.36.3. Scope the release job to tag refs so its
contents:write token is only minted when a release is actually
created. In merge.yml, drop a dead conditional (workflow only triggers
on push) and pass DOCKERDESKTOP_REPO to github-script via env rather
than inline interpolation, as recommended against script injection.
Signed-off-by: Guillaume Lours <[email protected]>
runPreStart executes a service's pre_start hooks sequentially as
ephemeral containers that share the first non-running replica's
volumes via VolumesFrom and attach to the same networks. A non-zero
hook exit gates service start.
per_replica: false is the only currently supported mode; per_replica:
true is rejected up front. The donor replica is the lowest-numbered
one so the choice is deterministic. ContainerWait uses
WaitConditionNextExit, and the wait loop deterministically handles
the daemon's clean-close (nil on Error + exit code on Result) and
transport-error races to avoid spurious hook failures.
The log stream is opened before ContainerStart to avoid racing
AutoRemove on fast-exiting hooks, and runs under a derived context
so a daemon that keeps the connection open cannot deadlock the call.
Hook containers carry project/service/version labels; the two
cleanup paths force-remove the never-started container explicitly
and warn when removal fails.
pre_start runs once per service when no replica is already running
(initial up, force-recreate or spec change), and is skipped on
scale-up so additional replicas don't re-trigger the hooks.
Coverage: 11 unit tests (including scheduler-race stress) with
goroutine-leak verification via goleak, plus 10 E2E tests (success
path, hook failure gating start, build-image inheritance, idempotent
re-up, spec change, force-recreate, mid-sequence failure, ordering,
scale-up, scaled service).
Signed-off-by: Guillaume Lours <[email protected]>
The reconciler hashed the raw service config while the executor
hashed the form with network_mode/ipc/pid/volumes_from references
resolved to container IDs. Persisted hash and recomputed hash
never matched, so dependents were recreated on every `up`.
Resolve references against observed containers before hashing,
and cascade recreation to namespace-sharing dependents when a
parent is replaced — otherwise the dependent would keep a stale
"container:<old_id>" reference. Also dedup stops in
planStopDependents via stoppedByPlan: with the cascade restored,
the dependent would otherwise receive two Stop nodes.
Fixes#13878.
Signed-off-by: Guillaume Lours <[email protected]>
emitRunningEvents iterated the full ObservedState.Containers map,
which is intentionally broader than the operation scope (it covers
DisabledServices for orphan classification). compose run --no-deps
SERVICE leaves project.Services empty and moves every other service
to DisabledServices, so their running containers were reported as
Running even though this command must not manage them.
Filter the iteration by project.Services, matching the reconciler
scope, and document the contract on the function.
Fixes#13882
Signed-off-by: Guillaume Lours <[email protected]>
The transport passed to NewResolver was applied only via
docker.WithClient. Pass it via docker.WithAuthClient too so the
authorizer's OAuth token fetches use it instead of http.DefaultClient.
Signed-off-by: Guillaume Lours <[email protected]>
Output was aligned with `docker ps --format json` in v2.21.0 (#10918)
but the docs were never updated.
Fixes#13850
Signed-off-by: Guillaume Lours <[email protected]>
The run parameter was always passed as false at the single call site
and the run==true branch was dead code. Remove it so unparam stops
flagging callers added by PR #13742.
Signed-off-by: Guillaume Lours <[email protected]>
maxBeforeStatusWidth used len(l.taskID) (bytes) while applyPadding
used utf8.RuneCountInString (runes). For ASCII task IDs the two
agree and no symptom surfaces, but a taskID containing multi-byte
UTF-8 chars (CJK, emoji, accented Latin) reported a width larger
than its visual columns. computeOverflow then triggered truncation
where none was needed, and truncateLongestTaskID's byte-indexed
slice could land mid-multibyte sequence, corrupting the displayed
string.
Align the two measurements on rune count.
Signed-off-by: Guillaume Lours <[email protected]>
In narrow terminals (e.g. tmux panes), the TTY progress UI emitted
lines wider than terminalWidth because adjustLineWidth could shrink
details and taskID but never the progress field. When progress
carried the "X.XMB / Y.YMB" size suffix, the truncation loop exited
with overflow > 0 and applyPadding's max(timerPad, 1) floor pushed
the line one char over. tmux then wrapped the line visually while
print() kept counting logical lines, desyncing aec.Up() on the next
render and producing the mangled "[+] pull X/Y" header overwriting
prior task lines.
Track the size suffix byte length on lineData and let
adjustLineWidth drop it as an intermediate truncation step before
abbreviating the taskID.
Fixesdocker/compose#13595
Signed-off-by: Guillaume Lours <[email protected]>
The progress UI writes to dockerCli.Err() but the auto-mode selector
was probing dockerCli.Out().IsTerminal(), introduced when the
EventProcessor was moved to the CLI layer. Any context that pipes
stdout while keeping stderr attached — `docker compose up | tee log`,
some CI wrappers, PowerShell native-command capture — silently
dropped to plain mode.
Align the detection with the stream the renderer actually targets,
restoring v4 behavior. Extract the switch into selectEventProcessor
so the auto-mode logic can be unit-tested with a real pty pair.
Fixesdocker/compose#13570
Signed-off-by: Guillaume Lours <[email protected]>
External reviewer noted that the alreadyStopped branch adds createNode
to removeDeps while the !alreadyStopped branch does not — semantically
correct but fragile, since it relies on the implicit invariant that
stopNode.DependsOn contains createNode in the !alreadyStopped path.
Spell out the invariant in a comment so a future maintainer who edits
the stop → create edge in the normal path knows they must also add
createNode unconditionally in the remove deps.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
Two coverage gaps surfaced by an external review:
- TestReconcileContainers_DependsOnChain: asserts that a service B with
depends_on: [A] produces a CreateContainer for B that depends on A's
last plan node (the serviceNodes mechanism in infrastructureDeps).
This was the only depends_on-via-plan-DAG behavior untested before.
- TestReconcileContainers_DependsOnScaleDown: companion test that
exercises the scale-down → dependent path specifically, verifying
that the previous commit's lastNode-on-scale-down fix actually wires
the dependency through.
- TestOperationTypeString: adds OpRunProvider to the table; all other
OperationType values were already covered.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
Three fixes surfaced by an external code review of the new reconciler:
1. Scale-down now propagates through serviceNodes. When a service is
scaled down (all containers in excess), reconcileService used to
continue without assigning lastNode, leaving r.serviceNodes[svc]
unset. Dependent services then declared no edge on the scale-down
ops and could start before the cleanup finished. Track the
RemoveContainer node as lastNode so depends_on chains pick it up.
2. mustRecreate errors are no longer silently ignored by sortContainers.
The comparator used `obsi, _ := r.mustRecreate(...)`, falling back
to false on any hashing error. Pre-compute obsolescence into a map
keyed by container ID before sorting and propagate the error to
reconcileService.
3. A container is no longer Stopped twice when its network and its
config both diverge. planRecreateNetwork already stops the affected
container as part of the disconnect/remove/recreate dance; the
subsequent planRecreateContainer (triggered via hasNetworkMismatch)
used to add another OpStopContainer against the now-stopped target.
Track stops in r.stoppedByPlan; planRecreateContainer reuses an
existing Stop node when present, and chains its Remove on both that
Stop and the replacement Create.
Two golden tests (TestReconcileNetworks_Diverged*) are updated to
reflect the new, dedupe'd plan shape (one Stop instead of two per
recreated container).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
Three improvements identified in the Principal Engineer pass but
deliberately deferred:
1. Test fidelity. Split executePlan into newPlanExecutor (constructs
the executor seeded from observed state) and (*planExecutor).run
(walks the DAG). Production callers go through executePlan
unchanged. TestExecutePlanRemoveContainerDropsFromCache now uses
newPlanExecutor + run, exercising the same errgroup, done-channel
and group-tracker wiring as production instead of a hand-rolled
loop over executeNode.
2. //nolint:unused chain. The three preserved helpers
(reconciler.prompt, planRecreateVolume, servicesUsingVolume) each
carried a separate "kept for future" comment. Consolidate the
rationale on the reconciler.prompt field doc and point the helper
nolint directives there, so a future cleanup is a single grep.
3. Concurrency test. Add TestExecutePlanConcurrentRemovesCacheCoherence
which builds N independent Stop→Remove chains in one plan; the
errgroup fans them out across goroutines that all hit
containersByService under the mutex. Passes under -race. Failure
would expose a missing or incorrect lock.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
Two related changes to the executor, plus the small cleanups they
attracted in review:
* Rename node consults a planner-set CreateNodeID instead of walking
ancestors. The old execRenameContainer searched node.DependsOn for a
CreateContainer result and fell back to a recursive walk through the
group chain. That worked only by convention; a future op with cross-
node data needs would have to rediscover or copy the pattern. Now the
rename op carries an explicit CreateNodeID int set at plan time and
the executor reads pctx[CreateNodeID].ContainerID directly. The
recursive findCreatedIDInChain is gone.
* Stop re-listing containers on every create. execCreateContainer used
to call getContainersByService(ctx, projectName) — a fresh
ContainerList per create — to resolve service references at execute
time. The executor now holds a live containersByService view seeded
from ObservedState (via observed.containersByService()) and grown as
OpCreateContainer nodes complete, so service references resolve from
memory. On OpRemoveContainer the removed container is dropped from
the view via slices.DeleteFunc, so a dependent's create that resolves
network_mode: service:x against the just-removed container cannot
pick up a stale ID (Containers.sorted() orders by canonical name and
would otherwise return the removed container).
* Defensive slices.Clone of op.Service.VolumesFrom in execCreateContainer.
resolveServiceReferences mutates VolumesFrom in place, and the
shallow struct copy of *op.Service still shares the backing array.
Single-execution-per-node makes it safe today, but the clone removes
the trap for any future parallel-execution mode.
* Operation gains a CreateNodeID int (not a *PlanNode pointer) to avoid
a structural cycle between Operation and PlanNode. OperationType
values are pinned to explicit integers so adding an op in the middle
cannot silently shift the others.
* execRenameContainer carries two checks so a missing CreateNodeID and
an empty produced ID are distinguishable in logs. Both are programmer
invariants (prefixed "internal:").
* containersByServiceFromObserved moved from a package-level helper to
a method on *ObservedState.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
ensureProjectVolumes already prompts the user when a volume's config hash
diverges from the compose file (create.go:1626) and recreates it on confirm.
The reconciler ran after ensureProjectVolumes and prompted again with the
exact same message — so a user who declined the first prompt was asked the
same question a second time.
Drop the prompt + recreate call from reconcileVolumes(). Recreation of
diverged volumes stays owned by ensureProjectVolumes; the reconciler only
plans the creation of missing volumes. If the user declined recreation,
the existing container's mounts still match the existing volume name and
hasVolumeMismatch correctly returns false, so containers are not falsely
flagged as obsolete.
Keep the supporting infrastructure available for future use, when
divergence detection migrates fully into the reconciler:
- reconciler.prompt field
- prompt parameter on reconcile()
- planRecreateVolume function (//nolint:unused)
- servicesUsingVolume function (//nolint:unused)
- noPrompt test helper
The reframed test (TestReconcileVolumes_DivergedIsIgnored) asserts the
new contract: a diverged volume produces no plan operations from the
reconciler.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
- plan.go: pin OperationType constants to explicit values so adding an op
in the middle doesn't shift the others.
- executor.go: remove the meaningless `var _ = getContainerProgressName`
line — same-package functions are always accessible.
- reconcile.go: fix the stale switch-default comment that contradicted
the case clause above it.
- reconcile.go: drop the local `serviceLabel` const that shadowed
`api.ServiceLabel` and use the shared constant.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
Replace the "flag any literal env var" check with a key-name heuristic
backed by the upstream DefangLabs keyword detector (password, secret,
token, api_key, …), and convert the hard error into a prompt matching
the existing checkForBindMount / checkForSensitiveData UX. --with-env
silences the env prompt; literal config.content gets its own prompt.
The previous check flagged benign vars like LOG_LEVEL=info, blocking
the 99% case, while still missing low-entropy real secrets the
existing secret-detector skips (MYSQL_ROOT_PASSWORD=toto slips through
on entropy ~1.5).
Refs: docker/compose#13394
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
Pass the active project name as the appId query parameter on the
docker-desktop://dashboard/logs deep link, both from the post-command
hint (compose up -d, compose logs) and the interactive nav menu
('l' key during compose up). The hook subprocess re-runs compose-go's
project loader so the name matches what the parent computed; it skips
the appId when -p, -f, --project-directory, --workdir, or --env-file
is set, since the hook payload does not carry their values. docker
logs stays unfiltered: the CLI hook contract does not expose the
positional container id.
Signed-off-by: Guillaume Lours <[email protected]>
Docker Desktop is removing the "Enable Logs view" beta setting, so drop
the /app/settings check and rely on /features alone. With the setting
gate gone, the compose hook subprocess would print the Logs view hint
regardless of LogsTab; add a flag check in handleHook. Consolidate
engine-label discovery and feature-flag evaluation into internal/desktop.
Signed-off-by: Guillaume Lours <[email protected]>
The stop lifecycle hook added in 672dc14d2 calls stoppingEvent and
stoppedEvent from pkg/compose/plugins.go, but the helpers had been
removed by the earlier wrapper-cleanup in da530c723. The result was a
broken go build for pkg/compose on upstream/main.
Restore the two helpers alongside the symmetric creating/created and
removing/removed pairs so call sites in plugins.go match the existing
pattern used in the same switch.
Signed-off-by: Guillaume Lours <[email protected]>
A previous change extended IsEmpty() to include `Stop == nil`. For a
provider that advertises only a `stop` block, this caused
commandMetadataIsEmpty to be false in setupPluginCommand, which made
the option-forwarding filter reject every key — silently dropping
all provider options on `up` and `down`.
The Stop-presence signal lives independently in stop.go and the
"stop" case of setupPluginCommand, so reverting the IsEmpty check
to its original semantics is sufficient.
Addresses PR #13779 review feedback.
Signed-off-by: Guillaume Lours <[email protected]>
The previous implementation fetched the provider binary path and
metadata twice per service during `compose stop`: once in stop.go
to gate on the `stop` capability, and again inside runPlugin via
setupPluginCommand.
setupPluginCommand now signals "skip" by returning (nil, nil) when
the requested command is absent from the provider's metadata.
stop.go calls runPlugin directly; the skip-when-unadvertised check
moves into runPlugin.
Addresses PR #13779 review feedback.
Signed-off-by: Guillaume Lours <[email protected]>
Provider-backed services were silently skipped on `docker compose stop`,
leaving external resources running after the user expected the stack to
be paused (e.g. after Ctrl+C during `up --watch`).
Compose now invokes `<provider> compose stop <service>` for providers
that advertise a `stop` block in their `metadata` subcommand output.
Providers that do not advertise stop (or do not implement metadata at
all) are silently skipped, preserving backward compatibility with
existing providers that pre-date this hook.
Closes#13772
Signed-off-by: Guillaume Lours <[email protected]>
- Fix stale image/container reuse across test runs - Add registry readiness check and async removal polling
- Skip multi-arch test when docker driver supports it
- Use t.Cleanup for reliable teardown, fix project name mismatches
- Re-enable 4 previously skipped tests that now pass
Signed-off-by: Guillaume Lours <[email protected]>
The compose process performs OCI artifact fetches in-process via
containerd's docker resolver, whose default transport only honors
HTTP_PROXY/HTTPS_PROXY/NO_PROXY env vars. Users behind PAC-only
corporate proxies hit i/o timeouts on `oci://` includes and on
`compose publish`.
When Docker Desktop is the active engine and exposes httpproxy.sock,
route the resolver through it (PAC-aware). Falls back to the default
transport when DD is unavailable or the socket is missing. Modeled on
docker/mcp-gateway PR #354.
Signed-off-by: Guillaume Lours <[email protected]>
The e2e job targets desktop runners (desktop-windows, desktop-macos,
desktop-m1) that are not configured anymore for this project.
Signed-off-by: Guillaume Lours <[email protected]>
Wrap the docker-desktop://dashboard/logs URL in OSC 8 escape sequences
with underline styling so it appears as a clickable link in supported
terminals. Respects NO_COLOR and COMPOSE_ANSI=never to suppress escapes.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
Guillaume Lours