compose watch triggered one rebuild per debounced batch, so a burst of
edits arriving faster than the debounce window (e.g. from a coding
agent editing several files) queued up several rebuilds in a row
instead of a single consolidated one.
Rebuilds now run asynchronously through a small scheduler that
coalesces requests received while a rebuild is in flight into one
trailing rebuild, without adding latency to an isolated edit. This
also closes a race where a plain restart could run concurrently with
an asynchronous rebuild of the same service.
Signed-off-by: Guillaume Lours <[email protected]>
Transposes the applicable findings from the review of the ttyWriter fix
(#14119) to the rewritten renderer:
- Done waits for the refresh goroutine to exit (outside the lock), so
nothing can repaint after Done returns and output printed right after
an operation cannot be garbled by a stray frame
- a nested Start retires the previous cycle instead of leaking its
refresh goroutine until the parent context is cancelled
- build suspension no longer leaks across cycles: Start resets it and
starts a fresh block below whatever buildkit wrote
- drop the dead strings.ToLower in startDependencies: the unexported
start only reads projectName when no Project is passed
The remaining findings are structural to this rewrite: Done is a single
critical section, and per-cycle context cancellation replaces the
doneSignal handshake.
Signed-off-by: Nicolas De Loof <[email protected]>
Compose silently ignored several spec-valid attributes it doesn't
honor outside Swarm mode: deploy.update_config, credential_spec,
ports[].mode: host, and others had no visible effect on a plain
`docker compose up`.
Detection is wired into compose-go's loader as a fail-closed
allowlist: every attribute path the schema declares, minus the
handful this runtime deliberately doesn't implement, plus a few
value-conditional checks for attributes only unsupported for one
specific value. A newly-specified attribute gets flagged
automatically, with no corresponding code change needed here.
Depends on an unreleased compose-go change; go.mod carries a
temporary replace to a fork branch until it's released.
Signed-off-by: Guillaume Lours <[email protected]>
The fake clock is only used by tests to control the debounce timer,
but requires carrying a clock abstraction through production code.
Use the standard time package for the debounce ticker and use
testing/synctest to control time in tests instead.
This removes the clock from composeService and BatchDebounceEvents,
and eliminates the clockwork dependency from the watch implementation.
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Providers could not see the definition of the service they manage:
options had to be duplicated between the compose file and the provider,
or the provider had to re-resolve the model on its own.
A provider may now emit {"type": "get-service-config"} on stdout;
compose answers on the provider's stdin with one JSON line holding the
resolved canonical configuration of the provider's own service,
straight from the in-memory model. The message can be repeated; each
occurrence is answered with one line. Detection is by construction: a
compose that predates the message aborts on it and never writes to
stdin, so the provider treats EOF as 'unsupported, upgrade compose'.
The example provider demonstrates the round trip, backed by an e2e
scenario; unit tests drive executePlugin against a helper-process
provider and cover the injection.
Signed-off-by: Nicolas De Loof <[email protected]>
`down --rmi` removed every dangling image for the project unconditionally,
while its tagged-image counterpart (ImagesToPrune) already skips a service
no longer in the project unless --remove-orphans is set. `down --rmi local`
on a project with an orphaned service therefore kept the tagged image but
deleted its dangling ones - an inconsistent result for one command.
Extract the "list a project's dangling images, remove what a keep predicate
doesn't spare, tolerate individual failures" logic - which down.go and
watch.go's pruneDanglingImagesOnRebuild had each partially and divergently
implemented - into one shared composeService.removeDanglingImages,
parallelized via errgroup like the rest of the codebase's bulk removals.
down.go's keep now mirrors ImagesToPrune's own RemoveOrphans/GetService
check; watch.go keeps its existing built-IDs check, behavior unchanged.
The removal fan-out now also honors --parallel via eg.SetLimit(s.maxConcurrency),
matching push.go/pull.go, instead of always running unbounded. Fixed
TestWatch_Sync's composeService{} literal to set maxConcurrency: -1
(NewComposeService's own default) - the zero value means "allow zero
concurrent goroutines" to errgroup, not "unlimited", and was silently
relying on maxConcurrency being unused before this change.
Signed-off-by: hiroto.toyoda <[email protected]>
`down --rmi` removed every dangling image for the project unconditionally,
while its tagged-image counterpart (ImagesToPrune) already skips a service
no longer in the project unless --remove-orphans is set. `down --rmi local`
on a project with an orphaned service therefore kept the tagged image but
deleted its dangling ones - an inconsistent result for one command.
Extract the "list a project's dangling images, remove what a keep predicate
doesn't spare, tolerate individual failures" logic - which down.go and
watch.go's pruneDanglingImagesOnRebuild had each partially and divergently
implemented - into one shared composeService.removeDanglingImages,
parallelized via errgroup like the rest of the codebase's bulk removals.
down.go's keep now mirrors ImagesToPrune's own RemoveOrphans/GetService
check; watch.go keeps its existing built-IDs check, behavior unchanged.
Signed-off-by: hiroto.toyoda <[email protected]>
`bridge convert -o <dir>` unconditionally ran `os.RemoveAll` on the
output directory and discarded any error, so a typo (`-o .`, `-o
$PWD`, `-o ~`) silently destroyed unrelated user data.
An empty or missing output directory is still created silently, but a
non-empty one now requires explicit confirmation (prompted
interactively, or via the new `--yes` flag for scripts/CI) before
being wiped.
Signed-off-by: Guillaume Lours <[email protected]>
v0.0.6 dropped the Egress policyType/block from generated
NetworkPolicy manifests since it blocked CoreDNS traffic; update
the fixtures to match.
Signed-off-by: Ricardo Branco <[email protected]>
Split SetRecursive into the existing per-GOOS watcher files instead of
a runtime check + nolint: the non-Windows stub never calls the
always-failing SetRecursive, and Windows failures now propagate as a
real error instead of silently downgrading to non-recursive.
Addresses review from thaJeztah on #14164.
Signed-off-by: Guillaume Lours <[email protected]>
- migrate gomodguard to gomodguard_v2 (gomodguard is deprecated since
v2.13 and warns on every run); same blocked modules, flat-list schema
- silence SA4023 in watcher_naive.go: tilt-dev/fsnotify SetRecursive
only succeeds on Windows, so on any single GOOS the newer staticcheck
proves the err == nil comparison constant — the cross-platform code
is correct, the finding is per-GOOS
Signed-off-by: Nicolas De Loof <[email protected]>
The PATH fallback of provider discovery appended a hardcoded ".exe" on
Windows, restricting providers to compiled binaries. exec.LookPath
already implements Windows executable resolution (PATHEXT), which finds
.exe but also .com/.bat/.cmd scripts — the suffix only narrowed the
lookup. Drop the executable() indirection and look up the bare name.
Note this only affects the PATH fallback: the docker CLI plugin channel
(docker-<name>) keeps requiring .exe on Windows, per docker/cli plugin
candidate rules.
Fixes#14157
Signed-off-by: Nicolas De Loof <[email protected]>
Building the dependency graph silently deleted unresolvable optional
depends_on references from the caller's project — and since DependsOn
is a map, even from projects sharing it. What later readers observed
depended on how many graphs had been built before them.
The pruning becomes an explicit step, applied once where the project
consumed by the engines is established — create and start — before the
graph, the dependency waits and the com.docker.compose.depends_on
label read the model, using Project.WithoutUnresolvedOptionalDependencies
introduced by compose-spec/compose-go#922 (go.mod bumped to the
post-merge commit). NewGraph now just skips the edge; required dangling
references still flow to its existing errors (naming the profiles that
would enable the dependency).
No behavior change; epic #14081, Lot 0 step 4 — precondition for a
canonical project object across plan phases.
Signed-off-by: Nicolas De Loof <[email protected]>
A container is orphaned when it is a one-off (compose run) that FINISHED
its task, or when it carries the project labels but its service is not
defined by the compose model — the typical leftover after the compose
file was edited. A still-RUNNING one-off is somebody's live session:
'up --remove-orphans' cleans up leftovers and must never kill it.
'down --remove-orphans' is the explicit stop-the-application action, so
it DOES take running one-offs down — through the per-service removal
loop, not the orphan branch.
That asymmetry was implemented but stated nowhere: isOrphaned's comment
only mentioned the model-absent half, down.go removed running one-offs
as an unexplained side effect of including one-offs in the listing, and
the observed-state collection silently dropped running one-offs with no
hint it was deliberate. No behavior change — the semantics are now
written at all three sites and pinned by tests: the predicate matrix,
the observed-state classification (a running one-off is neither a
service replica nor an orphan), and down's single stop+remove path for
a running one-off of a declared service.
Closes item C.4 of #14074.
Signed-off-by: Nicolas De Loof <[email protected]>
moby/buildkit#5558, which this test was disabled for, is fixed and
released well before the buildkit v0.26.3 currently vendored here.
Verified the re-enabled case passes (3 consecutive runs).
Signed-off-by: hiroto.toyoda <[email protected]>
Eleven comments in the reconciler and its neighbors justified decisions
by pointing at code that no longer exists ('matching the previous
ensureNetwork behavior', 'same way as convergence.go:138-160' — lines
that today hold unrelated code) instead of stating the behavior and its
reason. A reader following those pointers finds nothing, or worse,
something else.
Each is rewritten as a self-contained behavioral statement. Two carry
real weight:
- reconcileService's up-to-date branch now says explicitly that
starting containers is NOT the plan's job (a separate start phase in
start.go does it) — the previous wording invited 'fixing' the
StateCreated case;
- sortContainers now documents that its ordering IS the scale-down
policy (the tail it produces is what i >= expected trims), instead of
referencing stale line numbers.
Part of #14074 (A: the code misdescribes its own structure).
Signed-off-by: Nicolas De Loof <[email protected]>
The convergence type this file was named after was removed in fbea647b9,
but the file kept its name and its doc comment ('convergence manages
service's container lifecycle') ended up attached to the unrelated
getScale(). Anyone asked to 'change the convergence logic' lands here,
while the actual reconciliation lives in reconcile.go.
The file is renamed to service_containers.go with a header stating what
it really contains: the per-service container helpers shared by both
lifecycle engines (naming, service-reference resolution, dependency
waiting, container creation, startup). getScale gets a doc comment that
describes getScale.
Part of #14074 (A: the code misdescribes its own structure).
Signed-off-by: Nicolas De Loof <[email protected]>
The classic-builder rejections said 'set DOCKER_BUILDKIT=1 to use
BuildKit', a leftover from the internal BuildKit builder removed in
af579ebd4. Since then BuildKit builds go through the buildx plugin:
compose falls back to the classic builder when BuildKit is disabled
(DOCKER_BUILDKIT=0) OR when buildx is not installed — so the advertised
fix does nothing for users who are simply missing buildx.
The five messages now state the actual requirement (buildx installed,
DOCKER_BUILDKIT not forced to 0), and the silent-fallback warning when
buildx is missing now says what the fallback implies instead of a bare
'requires buildx plugin to be installed'.
Part of #14074 (A: the code misdescribes its own structure).
Signed-off-by: Nicolas De Loof <[email protected]>
AGENTS.md documents 'Test unit: go test ./pkg/...' and 'E2E tests:
go test -tags e2e ./pkg/e2e/'. Both statements were false: no file in
pkg/e2e carried an e2e build constraint, so the -tags flag was a no-op
and the 'unit' command actually launched the full e2e suite — requiring
a Docker daemon and the locally built binary, creating and destroying
containers, for up to 20 minutes. CI only avoided this by grepping e2e
out of the package list (Dockerfile).
Every *_test.go in pkg/e2e now carries //go:build e2e (merged with the
existing !windows constraints), and the Makefile e2e targets pass the
tag. The documented commands become true:
- go test ./pkg/... -> pkg/e2e reports [no test files]
- go test -tags e2e ./pkg/e2e -> runs the suite
AGENTS.md now also states the daemon/binary prerequisites explicitly.
Part of #14074 (A: the code misdescribes its own structure).
Signed-off-by: Nicolas De Loof <[email protected]>
When a service set `build.provenance: false` (or `sbom: false`), the
bake attest entry was silently omitted (the field is `omitempty`),
so BuildKit applied its defaults and attached a provenance attestation
anyway. The attestation changes the manifest-list digest on every build,
which caused containers to be needlessly recreated on `up --build`.
Emit `"type=<t>,disabled=true"` instead, which BuildKit's
`Attest.UnmarshalText` parses as `Disabled:true` and forwards to
BuildKit as an explicit disable signal.
Extract `appendAttest` to avoid duplicating the three-branch switch
for both provenance and sbom.
Fixes#14111
Signed-off-by: Guillaume Lours <[email protected]>
EncodedAuth now delegates the X-Registry-Auth wire encoding to
moby/api/pkg/authconfig (same module, already required) instead of
hand-rolling the JSON+base64url pair, and push.go reuses it instead of
its own copy. The CLI and moby AuthConfig structs are field-for-field
identical, so a direct type conversion bridges them.
The explicit GetAuthConfigKey normalization drops out of EncodedAuth:
docker/cli's configfile.GetAuthConfig normalizes the Docker Hub domain
itself. It remains for the OCI resolver, whose credential callback
receives the network host actually contacted (registry-1.docker.io),
which the configfile does not map.
Signed-off-by: Nicolas De Loof <[email protected]>
buildx was imported for two narrow purposes, both replaced by
docker/cli machinery compose already uses elsewhere:
- storeutil.GetProxyConfig -> configfile.ParseProxyConfig, the same
lookup docker build uses (also picks up ALL_PROXY and lowercase
variants)
- the dry-run image resolver (buildx builder + imagetools) -> a
DistributionInspect through the daemon with credentials from the CLI
config file, i.e. the same path a real pull/push would take; the
encoded-auth helper moves to internal/registry, shared with pull.go
bake is unaffected: it already execs the buildx CLI plugin. Dropping
the module also drops the transitive aws-sdk-go-v2 footprint
(-328 lines of go.sum).
Signed-off-by: Nicolas De Loof <[email protected]>
Fixes#14114: Done() blocked forever once the render goroutine had
already exited via ctx.Done() (e.g. after SIGTERM), since it sent on an
unbuffered channel with nobody left to receive.
Closing the channel instead of sending never blocks, but a bare close()
panics the moment ttyWriter runs a second Start/Done cycle on the same
shared bus - which already happens today (run's nested create/start,
publish's nested push, rm --stop's sequential stop then remove). Start
now hands out a fresh, once-only-closable signal each cycle instead of
a single shared channel, and run/publish stop opening a redundant
nested bracket around calls already inside one.
Signed-off-by: Guillaume Lours <[email protected]>
A pre_start hook container retained after failure (introduced in the
previous commit) is created without ConfigHashLabel so getContainers()
and every downstream compose operation (down, ps, stop, rm) is blind
to it. Without explicit cleanup it would survive compose down.
This commit adds removePreStartHookContainers(), called from down()
after regular container teardown and before network removal (so hook
containers are gone before their networks are removed). The cleanup is
project-wide on a full compose down; scoped to the requested services
when compose down <svc...> is used. Individual ContainerRemove errors
are logged at warn level and do not abort the teardown.
Changes:
- pkg/compose/down.go: removePreStartHookContainers helper + call in down()
- pkg/compose/filters.go: add //nolint:unparam to hookFilter (param is
architecturally correct; today only preStartHookType is in use)
- pkg/compose/down_test.go: update all existing down tests to expect the
new ContainerList call; add TestDownRemovesRetainedPreStartHookContainers
and TestDownHookContainerRemovalFailureIsNonFatal
Signed-off-by: Guillaume Lours <[email protected]>
A failing pre_start hook container is now retained (AutoRemove: false)
so operators can run 'docker logs <id>' and 'docker ps -a' to diagnose
the failure. On success the container is removed explicitly, mirroring
the old AutoRemove behaviour including anonymous volumes.
Before each pre_start run, stale hook containers from a previous failed
run are detected via project+service+HookLabel filters and force-removed
so they do not accumulate.
Changes:
- pkg/api/labels.go: add HookLabel (com.docker.compose.hook)
- pkg/compose/filters.go: add hookFilter helper
- pkg/compose/pre_start.go:
- AutoRemove: false in createPreStartContainer
- HookLabel added to container labels
- runPreStartHook: explicit ContainerRemove on success; retain on failure
- removeOrphanPreStartContainers: new helper called in runPreStart
- Tests: update all existing pre_start tests for the new flow; add
feature tests (success removes, failure retains, orphan cleanup) and
coverage-gap tests (lowestNumberedContainer, waitPreStart cancel,
preStartResultErr, streamPreStartLogs error paths, old-API network
paths, ExecCreate/Attach/Inspect errors, hookExitError branches)
Coverage after: hook.go 100%, pre_start.go most functions 100%
(was 89% and 64% respectively per Codecov delta).
Signed-off-by: Guillaume Lours <[email protected]>
On top of the output-tail infrastructure from the previous commit:
Stderr bias: split the combined output tail into separate stdout and stderr
buffers. stdcopy.StdCopy(wOut, wErr, reader) now routes each stream to its
own outputTail. The error message prefers stderr (where hooks write their
actual error) and falls back to stdout when stderr is empty:
db hook exited with status 1: Table 'service.sites' doesn't exist
A new hookExitError helper encapsulates this preference.
pre_start parity: streamPreStartLogs previously short-circuited and returned
immediately when listener was nil (detached mode / compose up -d), leaving
the tail buffers empty and failure errors without context. The log stream is
now always opened. The demultiplexed streams fill separate tailOut / tailErr
buffers. On non-zero exit the stderr-biased tail is appended to the error
(same behaviour as exec hooks).
Tests:
- TestRunHook_StderrBias: stdout noise stays out of the error when stderr
has content.
- TestPreStart_DetachedModeAttachesLogs: ContainerLogs is called even with
a nil listener.
- TestPreStart_FailureIncludesTail: a failing pre_start hook carries its
stderr output in the returned error.
Signed-off-by: Guillaume Lours <[email protected]>
startMx exists to serialize ContainerStart calls because the engine
assigns published ports from ranges non-atomically. It is held by the
plan executor's execStartContainer — a nearly dead code path (plans
only emit OpStartContainer for paused/dead containers) — and NOT by
startServiceContainer, the path every up/start actually runs. The
protection is thus inoperative today: concurrent services publishing
overlapping port ranges can race in the engine.
Take the mutex around the real ContainerStart call and document the
contract on the variable. Starts of different services were already
serialized per-service (replicas start sequentially); this serializes
the API call across services too, matching what the plan engine will
do once the start phase converges (#14081).
Part of #14074 (section C) / #14081 (lot 0).
Signed-off-by: Nicolas De Loof <[email protected]>
waitDependency returned nil whenever its context ended, so an expired
wait deadline was reported only when the 500ms poll ticker happened to
win the select race and the condition check then failed on the expired
context. Whether `up --wait --wait-timeout` failed or silently
succeeded after the timeout was a coin toss, and the "timeout waiting
for dependencies" mapping in waitDependencies was unreachable on the
direct path.
Return the context error when the deadline expired; a plain
cancellation (Ctrl-C) is still not reported as a dependency failure.
The unit test pins both sides deterministically (a deadline shorter
than the first tick used to be swallowed with certainty); the e2e
scenario documents the user-visible contract: `up --wait
--wait-timeout` on a never-healthy service fails with "application not
healthy after Xs" while the container keeps running.
Part of #14074 (section C) / #14081 (lot 0).
Signed-off-by: Nicolas De Loof <[email protected]>
The `mocks` target still passed the pre-rename `Service` argument to
mockgen (the interface is `Compose` since the rename) and installed
whatever mockgen happened to be on PATH. Run it through `go run` with a
pinned version so regeneration is reproducible; v0.4.0 is the last
version usable here, as the package mode introduced in v0.5 copies
parameter names from the mocked source and moby's APIClient names its
variadic options `_`, which produces invalid Go code.
Regenerated pkg/mocks with the fixed invocation (header-only change).
Signed-off-by: Nicolas De Loof <[email protected]>
The imperative start engine (startService, startServiceContainer,
restart, waitDependencies) is only locked by e2e tests today. Before
converging it into the plan engine (#14081), pin its observable
behavior with unit characterization tests: start idempotence and the
silent no-op on running replicas, the "no container to start" error,
pre_start gating (mixed replicas skip the hook, all-stopped runs it
once on the lowest-numbered replica), the inject -> start -> post_start
sequence with Started only emitted after hooks, restart's depends_on
restart:true selection and its pre_stop -> ContainerRestart ->
post_start order, --wait's completed_successfully condition mapping,
and waitDependencies' missing/failing dependency semantics for
required and optional dependencies.
No production change.
Part of #14074 (section C) / #14081 (lot 0).
Signed-off-by: Nicolas De Loof <[email protected]>
The --format flag defaulted to "table" instead of "", so the
opts.Format == "" guard in runPs never fired for a bare compose ps.
Setting the default to "" lets the existing config-fallback path work,
matching the behaviour of docker ps.
Fixes#13643
Supersedes #14032
Signed-off-by: Guillaume Lours <[email protected]>
createNetwork built the IPAM object in two separate blocks; the second
silently overwrote createOpts.IPAM with a fresh empty struct and never
forwarded n.Ipam.Options to the daemon. Consolidate into a single block
covering driver, config pool, and options.
Fixes#13785
Supersedes #13936
Signed-off-by: Guillaume Lours <[email protected]>
RunOptions embeds CreateOptions, which suggests every creation option
applies to the one-off's dependencies; in reality only Build,
IgnoreOrphans, RemoveOrphans and QuietPull are propagated
(startDependencies builds a fresh CreateOptions from exactly those
four). Setting Recreate or Inherit on a RunOptions was a silent no-op.
The same struct also serves Exec, which honors only the exec-relevant
subset of its ~20 fields.
Both facts are now stated on the type, the embedding and the
Exec-only Index field.
Part of #14074 (B: pkg/api promises things the implementation does not
honor).
Signed-off-by: Nicolas De Loof <[email protected]>
StartOptions fuses two contracts: the fields s.start honors (Project,
AttachTo-as-project-source, Wait, WaitTimeout) and the fields only Up's
foreground session reads (Attach, OnExit, ExitCodeFrom, Watch,
NavigationMenu). Passing the latter to Start compiled, returned no
error, and did nothing.
Each session field is now marked 'honored by Up's foreground session
only; ignored by Start'. Two implicit behaviors become explicit along
the way: Attach doubles as the detached/interactive mode switch (nil
means Up returns once containers are started), and AttachTo has two
unrelated meanings (project reconstruction source for Start, log
scoping for Up).
Part of #14074 (B: pkg/api promises things the implementation does not
honor).
Signed-off-by: Nicolas De Loof <[email protected]>
Three structs carry a Services field with three different meanings:
ProjectLoadOptions.Services narrows the loaded project (the only real
filter), CreateOptions.Services only selects which recreation policy
applies to which service while the WHOLE project converges, and
StartOptions.Services only scopes Up's log monitor and is ignored by
Start. Nothing said so: an SDK consumer calling
Create(project, CreateOptions{Services: []string{"web"}}) reasonably
expected to create web alone, and got the full project.
The interface doc now states the underlying invariant — methods act on
the whole project they receive; scoping is done by narrowing the
project, and downstream Services fields are intent markers, not
filters — and each of the three fields documents its actual effect.
Part of #14074 (B: pkg/api promises things the implementation does not
honor).
Signed-off-by: Nicolas De Loof <[email protected]>
initialSyncFiles and initialSyncDirectory filtered out any host file
whose mtime predated the image creation time — which is exactly the
files that initial_sync is meant to copy. The imageCreatedTime lookup
(ContainerList + ImageInspect) was also a silent failure point when the
container list returned nothing. Remove the filter and the helper
entirely: sync all files that are not ignored or bind-mounted.
Fixes#13725
Supersedes #14031
Signed-off-by: Guillaume Lours <[email protected]>
Rename-only change, no behavior difference, no exported identifier
renamed (only parameter names of a few exported functions, which are
not part of the Go API).
- names that hide what a value is: parameters holding a *name* or *ID*
while named like the object — project→projectName, service→serviceName,
container→containerID — so string vs struct is readable at every call
site without opening the signature
- misleading receivers: monitor methods used c (reads as
client/container/CLI) → m; jsonWriter used p while its ttyWriter
sibling uses w → w
- cryptic abbreviations with non-trivial scope: nw (meant both
network.Summary and types.NetworkConfig) → network/networkConfig,
svc → service/serviceName/serviceCopy, oc/ocs → observedContainer(s),
cnts → serviceContainers, cnx → attachResponse
- container.Summary loop variables unified on ctr (was a c/ctr mix);
fillBindMounts also dropped its p/s/m single-letters, one of which
was shadowed by a loop variable of a different type
- option-struct identifiers unified on options (opts remains only for
variadic functional options); ambiguous ones get a precise name
(bindOptions, networkCreateOptions, projectOptionsFns)
Signed-off-by: Nicolas De Loof <[email protected]>
The Compose interface promised 'Scale manages numbers of container
instances running per service', but ScaleOptions had no replica count:
callers had to know they must mutate project.Services[x] via SetScale
before calling, and ScaleOptions.Services only tuned the recreation
policy. An SDK consumer following the interface doc got a no-op
convergence of the whole project.
ScaleOptions gains Replicas (service -> count); the backend applies it
to the model itself before converging, and derives the targeted
services from its keys when Services is not set. The CLI now does what
a CLI should: parse SERVICE=REPLICAS tuples and hand them to the
backend, instead of pre-mutating the model. Passing a pre-mutated
project with an empty Replicas map still behaves as before, so
existing SDK callers are unaffected.
Part of #14074 (B: pkg/api promises things the implementation does not
honor).
Signed-off-by: Nicolas De Loof <[email protected]>
The bake progress display goes through containerd/console, which on
Windows only accepts the exact os.Stdin/Stdout/Stderr values (pointer
identity in newMaster) and rejects everything else with "creating a
console from a file is not supported on windows". The _console wrapper
introduced to satisfy buildkit's console.File type-assert therefore
could never pass that check: TTY progress silently fell back to plain
in auto mode, and --progress=tty / --ansi always failed hard with
"failed to get console".
When the CLI stream was constructed from a real file — the interactive
case, where it wraps os.Stdout — hand that genuine *os.File to
NewDisplay so the identity check passes. File-less streams keep the
wrapper, which works on Unix where only the descriptor matters.
Fixes#14086
Co-Authored-By: Claude Fable 5 <[email protected]>
Signed-off-by: Nicolas De Loof <[email protected]>
Several e2e tests used fixed host ports (8070, 8080, 8090, etc.), global
buildx builders registered with --use, and a hardcoded IPC container name.
Under parallel execution these shared daemon-level resources caused port
conflicts, buildkitd container-name collisions, and intermittent failures.
Separately, the standalone make target hardcoded -parallel=1, forcing the
entire standalone suite to run serially even though the same tests run in
parallel in plugin mode.
Changes:
- Convert fixed host ports in four fixtures (network-test, sentences,
build-test, volume-test) to ephemeral bindings; add ServicePublishedPort
helper to framework.go to resolve the actual mapped port at runtime.
- Give each buildx-builder test a unique daemon-scoped name via BuilderName
helper, preventing container-name collisions between parallel goroutines.
Applies to TestBuildPlatformsWithCorrectBuildxConfig, TestBuildPrivileged,
TestBuildBuilder, TestBuildEntitlements, TestBuildTLS.
- Rename the fixed ipc_mode_container to a project-scoped name
(ipc_e2e-src) and update the ipc-test fixture accordingly.
- Replace the hardcoded -parallel=1 in make e2e-compose-standalone with
E2E_PARALLEL_PLUGIN?=4 and E2E_STANDALONE_PARALLEL?=4, aligning both
modes with the effective GOMAXPROCS on 4-core GitHub Actions runners.
Both variables are overridable (E2E_STANDALONE_PARALLEL=1 restores
serial local debugging).
- Raise the poll delay in watch_test.go long-running polls (90s/120s
timeout) from the default 100ms to 1s; bump cascade_test.go and
publish_test.go from 100ms to 500ms. Reduces CPU pressure without
affecting effective test duration.
The logging-driver fixture retains its fixed port 24224: the Docker fluentd
log driver encodes fluentd-address at service-start time, making ephemeral
resolution non-trivial. TestLoggingDriver is serial so there is no
collision risk in practice.
Signed-off-by: Guillaume Lours <[email protected]>
A failing service hook returns only its exit code:
dependency failed to start: db hook exited with status 1
The output that explains why is not there, and for a hook running without a
listener it does not exist anywhere: runHook created the exec with
AttachStdout/AttachStderr false, so the daemon discarded both streams and
runWaitExec only polled for the exit code. There was nothing left to look at
afterwards.
Now stdout and stderr are always attached, and the last lines of them are kept
in a bounded buffer. On non-zero exit that tail goes into the error:
db hook exited with status 1: SQLSTATE[42S02]: Base table or view not found
A successful hook is unchanged - output goes to the listener as before, nothing
is added anywhere. With a listener the lines still stream out, the buffer just
tees them.
The buffer keeps the last 10 lines and 2 KiB, drops the oldest content and never
short-writes, so a chatty hook cannot block on it and cannot grow it.
runWaitExec is removed: it existed only for the unattached branch, and that
branch is what threw the output away.
Signed-off-by: Deniss Solnce <[email protected]>
Map, per container lifecycle scenario, the exact event sequences the
engine emits and the monitor may receive: natural exit, policy-driven
restart (die/start only, no restart action, silence during backoff),
external stop of a running vs restarting container, ContainerRestart,
docker kill, OOM kill, rm -f and rm.
Mapped from moby's daemon sources (LogContainerEvent call sites and
handleContainerExit/containerStop/containerRestart/kill paths) and
verified against a live engine v29.6.1 with 'docker events'. Notably:
the relative order of stop and die on a container stop is not
guaranteed, and no event at all is emitted while a container sits in
restart backoff.
Related to #13985 / #13990.
Signed-off-by: Nicolas De Loof <[email protected]>
- projectNameFor maps the subtest separator to a double dash so
TestFoo/bar and TestFooBar cannot resolve to the same compose project
and interfere when running in parallel
- copyDir preserves the source file mode instead of forcing 0644, so an
executable testdata script stays executable in the scenario's project
directory
- OutputMatches and OutputMatchesCount compile their pattern once at
construction: a malformed pattern panics where the check is declared,
and Eventually loops no longer recompile it on every tick
Signed-off-by: Nicolas De Loof <[email protected]>
The per-container inspect of the previous fix doubled the suite's
wall-clock on the containerd CI job — one docker CLI process per
container per snapshot — pushing it past go test's default 10m and
killing the run mid-flight. Snapshots go back to one batch inspect,
falling back to individual inspects only when the batch fails because
a helper container vanished. The e2e make targets also pass an
explicit -timeout 20m so a slow job fails on its own results, not on
the default timeout.
TestUpStopWithLogsMixed becomes deterministic: with a bounded ping the
service could exit on its own before the abort reached it and the
pre_stop hook never ran; it now pings forever so the abort always
interrupts it.
Signed-off-by: Nicolas De Loof <[email protected]>
Guillaume Lours