extension.md gains the Image distribution section: invocation contract
(--image/--digest/--created/--source/--policy), the get-image request
and its chunked image-stream answer (zero-chunk terminated, stdin
exclusivity and drain-before-next-answer contract), the two-branch
decision guidance (digest as identity, created as ordering fallback,
reproducible builds caveat), and the support-by-presence metadata
convention is now stated as the general rule. The example provider
implements pull end to end — request, stock chunked reader, hash
recorded at PROVIDER_PULL_MARKER — and the e2e scenario locks the
whole path: up builds the service image and streams it to the provider
under the local verdict, compose pull re-invokes it under the
freshness contract.
Signed-off-by: Nicolas De Loof <[email protected]>
Binding the announced gateway alone restricts by destination address, not
by the interface traffic arrives on, so another bridge can still reach it
by IP. A provider that pins the socket to the relay-link bridge's own
interface (SO_BINDTODEVICE on Linux) keeps the endpoint relay-only.
Signed-off-by: Nicolas De Loof <[email protected]>
Unify the up and down paths on one removal helper that tolerates both
NotFound and Conflict and emits the same progress events as removeNetwork,
build one down op per relay-link network gated on the shared limiter (so
the empty-ops warning is reachable again), and share the label filter.
Reword the isolation claim: the relay-link network is joined by nothing
else, but its gateway is a host address that local containers can still
reach by IP; only a provider binding exclusively that address keeps the
endpoint off loopback and LAN-facing addresses.
Signed-off-by: Nicolas De Loof <[email protected]>
get-relay-info answers pointed a locally-bound provider at one of the
dependents' own networks: shared infrastructure every other service on
it could also reach, and no more isolated than any project bridge.
Provision a dedicated, internal:true bridge network scoped to that one
provider service instead, created only when the provider itself sends
get-relay-info -- the provider's own declaration that it binds locally,
never for a provider backing a remote resource. The relay container
joins it alongside the dependents' networks it already joins; it is
torn down with the relay on endpoint changes and on down, including
the no-compose-file down path.
The network's lifecycle is hardened against the races a shared,
project-scoped resource runs into:
- a NetworkCreate conflict on the deterministic name re-checks by
label (not just by name) before adopting the existing network, so a
genuine name clash with an unrelated, user-declared network fails
loudly instead of being silently adopted into the relay's isolation
boundary;
- a NotFound gateway lookup after losing that conflict falls back to
an unresolved answer instead of propagating the error;
- removeRelayLinkNetwork and ensureRelayLinkNetworksDown both tolerate
a NetworkRemove conflict from the daemon's asynchronous endpoint
disconnect, which can race the relay container's own removal;
- ensureRelayLinkNetworksDown accumulates failures with errors.Join
across every relay-link network of a project instead of aborting on
the first one, and skips (warn-only) a network that still has active
endpoints, mirroring removeNetwork's existing tolerance for regular
project networks.
docs/extension.md documents that the "desktop" network name in the
get-relay-info answer (under Docker Desktop) is an opaque placeholder,
never a real Docker network a provider can inspect or connect to.
Signed-off-by: Nicolas De Loof <[email protected]>
publish was promoted to a regular Compose command in 13618756d
(2025-03-12) but was left registered under 'alpha' as well.
Remove the leftover duplicate registration and the now-orphaned
generated docs (compose_alpha_publish.md,
docker_compose_alpha_publish.yaml), and regenerate
docker_compose_alpha.yaml accordingly.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Signed-off-by: Guillaume Lours <[email protected]>
Addresses docker-agent review on #14258 (discussion_r4092089046):
when ReadString fails after Start() succeeds, the pipe read-end was
leaked -- nothing else calls Wait() on this deliberately long-lived
subprocess. Mirrors the existing close on the Start() failure path.
Signed-off-by: Nicolas De Loof <[email protected]>
A provider running its service LOCALLY has to pick the address its
published endpoint binds, and on a standalone Linux engine no address
is both relay-reachable and off the LAN (#14257):
host.docker.internal resolves there to the bridge gateway, which a
loopback-only listener cannot accept, while the wildcard exposes the
port on every host interface. The bundled example provider worked
around it by binding 0.0.0.0.
The new get-relay-info message closes that gap, with compose owning
the platform knowledge so the provider just binds what is announced.
Compose answers with one JSON line listing the networks the relay
would join — the dependents' networks, exactly as selected for the
relay deployment — each with the address a locally-run endpoint should
bind to be reachable from the relay: the network's engine-assigned
gateway on a standalone engine (an address the host owns on that
bridge, reachable from the relay but not from the LAN), and 127.0.0.1
under Docker Desktop — the networks live inside the VM there, and the
host's own loopback is, factually, where a host process is reached
through the Desktop proxy. Resolution is lazy (Desktop detection and
network inspects run only when a provider asks — a remote-resource
provider never does) and best-effort: a missing gateway means "bind
elsewhere".
Compose also announces the message types it accepts in the
COMPOSE_PROVIDER_MESSAGES environment variable of the provider
process: an unknown message stays a fatal protocol error by design
(a provider REQUIRING an unsupported message must fail loudly), and
the announcement is how a provider adapts instead of failing. The
example provider binds the first announced gateway, falling back to
the wildcard only when none is announced; through it,
TestProviderPublishEndpoint exercises the gateway path on Linux CI
and the loopback path on Docker Desktop.
Signed-off-by: Nicolas De Loof <[email protected]>
The 'Connection to a service' section only covered environment-variable
injection, while the relay mechanism was buried in the message-type
reference list. Move the relay narrative into its own subsection there,
illustrated with a sequence diagram showing the full path — the provider
publishing the endpoint, Compose deploying the relay on the dependents'
networks, and the application connecting at the compose-native address.
The publish-endpoint entry in the message list keeps the wire-format
contract and points to the new section.
Signed-off-by: Nicolas De Loof <[email protected]>
A peer that never closes after receiving the relayed FIN used to pin
the forward's goroutine pair and both TCP connections until SIGKILL —
and with them the drain in main. Force-closing both ends as soon as one
direction finishes would have thrown out TCP half-close (a client that
FINs its request and then reads a long response), so the surviving
direction now runs under an idle grace instead: a read deadline
re-armed before every Read once the other direction is done. Active
streams are never cut — only pairs sitting idle past the grace are
reaped. Locked by two tests: the silent-peer pair is reaped at the
grace, and a response still streaming after the client's half-close
survives well past it.
The demo provider's comment also spells out why serve-demo is neither
Wait()ed nor a zombie: the provider exits within seconds, so init has
long adopted — and reaps — the subprocess when its three minutes are up.
Co-Authored-By: Claude Fable 5 <[email protected]>
Signed-off-by: Nicolas De Loof <[email protected]>
The serve-demo subprocess is the demo's provisioned resource: it must
outlive the provider invocation — consumers reach it through the relay
after up returns — and it reaps itself after three minutes. Spell that
out where a reader (or reviewer) would otherwise expect a Kill/Wait.
Co-Authored-By: Claude Fable 5 <[email protected]>
Signed-off-by: Nicolas De Loof <[email protected]>
down handled a provider service by running the plugin alone — but the
service may own a project container, the relay deployed when it
published endpoints. Left running, it kept the project network in use
and `down -v` failed with "Resource is still in use".
The relay is now part of the service's deprovisioning: its containers
are removed first, then the plugin removes the provider's resource —
mirroring up, which provisions the resource before deploying the relay.
The example provider's down used to answer with a hardcoded error (a
leftover no test relied on): it now succeeds, with the failure
simulation kept behind PROVIDER_DOWN_FAILURE.
Co-Authored-By: Claude Fable 5 <[email protected]>
Signed-off-by: Nicolas De Loof <[email protected]>
A provider's resource lives outside the compose network: consumers could
only reach it through injected variables carrying a host-published
address — nothing like the compose-native experience of addressing a
service by name at its well-known port.
A provider may now publish where each endpoint of its resource actually
listens:
{"type": "publish-endpoint", "message": "80=localhost:49152"}
The endpoint is announced as seen from the provider's host: the relay —
the component that knows it runs inside a container — rewrites loopback
or unspecified upstream hosts to host.docker.internal (resolved through
its injected host-gateway extra_host); routable addresses pass through.
When at least one endpoint is published, compose deploys a relay
container in place of the service: a minimal TCP forwarder (new relay/
directory, published as docker/compose-relay, overridable with
COMPOSE_RELAY_IMAGE for internal registries) joining the networks of the
services that depend on the provider service, aliased with the service
name. Consumers then use http://<service>:<port> as if the service were
a regular container.
The relay is a first-class project container — canonical name, standard
compose labels including config-hash (label-driven commands run without
the compose file keep seeing the service: ps, logs, stop, down) — plus
the com.docker.compose.relay label declaring its role:
- the reconciler already leaves provider services' containers alone, and
the relay's identity hash (image + routes) makes up idempotent: kept
when routes are unchanged, recreated otherwise;
- process-level commands (exec, cp) refuse a relay — there is no service
process in it to act on;
- the up monitor excludes relays from the containers whose termination
ends an attached up: they are long-lived infrastructure and would
otherwise keep 'up' waiting forever.
The example provider demonstrates the flow behind PROVIDER_DEMO_ENDPOINT
(a detached helper serving a fixed HTTP response), backed by an e2e
scenario asserting the compose-native address works and exec is refused.
Signed-off-by: Nicolas De Loof <[email protected]>
Providers could not see the definition of the service they manage:
options had to be duplicated between the compose file and the provider,
or the provider had to re-resolve the model on its own.
A provider may now emit {"type": "get-service-config"} on stdout;
compose answers on the provider's stdin with one JSON line holding the
resolved canonical configuration of the provider's own service,
straight from the in-memory model. The message can be repeated; each
occurrence is answered with one line. Detection is by construction: a
compose that predates the message aborts on it and never writes to
stdin, so the provider treats EOF as 'unsupported, upgrade compose'.
The example provider demonstrates the round trip, backed by an e2e
scenario; unit tests drive executePlugin against a helper-process
provider and cover the injection.
Signed-off-by: Nicolas De Loof <[email protected]>
`bridge convert -o <dir>` unconditionally ran `os.RemoveAll` on the
output directory and discarded any error, so a typo (`-o .`, `-o
$PWD`, `-o ~`) silently destroyed unrelated user data.
An empty or missing output directory is still created silently, but a
non-empty one now requires explicit confirmation (prompted
interactively, or via the new `--yes` flag for scripts/CI) before
being wiped.
Signed-off-by: Guillaume Lours <[email protected]>
bridgeCommand and transformersCommand had no RunE, so an unknown
subcommand (e.g. "bridge zzz") fell through cobra's default
non-Runnable path: help printed, exit 0. Scripts couldn't distinguish
a typo from success. Mirrors the same fix already shipped on the
compose root command.
Signed-off-by: Guillaume Lours <[email protected]>
The fenced Go blocks in docs/sdk.md become real code under
docs/examples/sdk, built by the ordinary module build: API drift now
breaks compilation. A unit test pins the markdown to the compiled
files byte-for-byte (first block = main.go from its package clause
down, second block = customService's body), so wording drift breaks
the test instead of silently rotting the docs.
Epic #14074, section G.
Signed-off-by: Nicolas De Loof <[email protected]>
sdk.md advertised a 'progress' package (progress.NewTTYWriter,
NewPlainWriter, NewJSONWriter, NewQuietWriter) that does not exist in
the repository, and typed WithEventProcessor against it. Anyone
following the doc writes code that cannot compile — or recreates the
phantom package.
The section now points at the real surface: api.EventProcessor, the
silent-by-default behavior when no processor is configured, and the
CLI's actual renderers in cmd/display (Full, Plain, JSON, Quiet) with
their real signatures.
Part of #14074 (A: the code misdescribes its own structure).
Signed-off-by: Nicolas De Loof <[email protected]>
- compose_down.md: note that retained pre_start hook containers (those
kept after a hook failure for post-mortem) are automatically removed
by compose down, with the label filter to list them beforehand.
- compose_up.md: note the retention behavior on pre_start hook failure,
how to inspect the retained container, and that it is cleaned up on
the next compose up or compose down.
Signed-off-by: Guillaume Lours <[email protected]>
The --format flag defaulted to "table" instead of "", so the
opts.Format == "" guard in runPs never fired for a bare compose ps.
Setting the default to "" lets the existing config-fallback path work,
matching the behaviour of docker ps.
Fixes#13643
Supersedes #14032
Signed-off-by: Guillaume Lours <[email protected]>
Output was aligned with `docker ps --format json` in v2.21.0 (#10918)
but the docs were never updated.
Fixes#13850
Signed-off-by: Guillaume Lours <[email protected]>
The stop section only mentioned setenv being ignored; rawsetenv
is handled identically by the code but was missing from the docs.
The mermaid sequence diagram also lacked a rawsetenv step.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Signed-off-by: Yohta Kimura <[email protected]>
rawsetenv injects provider variables without the service-name prefix, so
a key can collide with a value already set on the dependent service,
whether declared by the user in environment or emitted by another
provider. Log a warning and overwrite on collision, document the
precedence and the non-deterministic ordering between concurrent
providers, and cover the user-environment override with an e2e test.
Signed-off-by: Yohta Kimura <[email protected]>
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Providers can now send rawsetenv messages to inject environment
variables into dependent services without the automatic service name
prefix. This enables use cases where applications require exact
variable names that cannot be altered.
Closes#13727
Signed-off-by: Yohta Kimura <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
follow-up to 7eeb7de7a2, adding more
links now that the CLI reference for docker logs has anchors for them.
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Link to the corresponding `docker container logs` equivalents, which
contain more details on the accepted formats and use.
The container logs documentation still needs some updates to provide
per-flag sections, so follow-ups can be made once those are done.
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The previous implementation fetched the provider binary path and
metadata twice per service during `compose stop`: once in stop.go
to gate on the `stop` capability, and again inside runPlugin via
setupPluginCommand.
setupPluginCommand now signals "skip" by returning (nil, nil) when
the requested command is absent from the provider's metadata.
stop.go calls runPlugin directly; the skip-when-unadvertised check
moves into runPlugin.
Addresses PR #13779 review feedback.
Signed-off-by: Guillaume Lours <[email protected]>
Provider-backed services were silently skipped on `docker compose stop`,
leaving external resources running after the user expected the stack to
be paused (e.g. after Ctrl+C during `up --watch`).
Compose now invokes `<provider> compose stop <service>` for providers
that advertise a `stop` block in their `metadata` subcommand output.
Providers that do not advertise stop (or do not implement metadata at
all) are silently skipped, preserving backward compatibility with
existing providers that pre-date this hook.
Closes#13772
Signed-off-by: Guillaume Lours <[email protected]>
Most files already grouped imports into "stdlib -> other -> local",
but some files didn't. The gci formatter is similar to goimports, but
has better options to make sure imports are grouped in the expected
order (and to make sure no additional groups are present).
This formatter has a 'fix' function, so code can be re-formatted auto-
matically;
golangci-lint run -v --fix
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Nicolas De Loof