- demuxExecStdout errors on a header cut short mid-frame (was a silent EOF that
dropped the tail) and surfaces a docker systemerr frame instead of discarding it.
- ftester drains telemetry on exit through the bounded observer.Drain instead of
two unbounded ForceFlush calls, so an unreachable collector can't hang it at exit.
- container-listing failures are sorted for a deterministic skipped-entries preview,
logged per-entry at Debug (the detail is already in the response and the endpoint
is hit on every navigation) with names quoted so control bytes in a hostile
filename can't inject into a log line; corrected the stat-failure comment
(a dangling symlink lstats fine, it doesn't fail).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The truncation slice and the demux stdout byte-cap had no docker-layer tests —
only the handler's Truncated wiring was exercised through the fake, so a
mis-slice or a dropped cap would have gone unnoticed. Extract find-output
parsing into a pure parseFindEntries and take the byte cap as a demuxExecStdout
parameter, then unit-test both boundaries (at cap / cap+1 / over-limit stream).
Also bound how many paths one container-files request may list
(maxContainerListPaths), so the per-path entry cap can't be multiplied by an
attacker-chosen path count into a large fan-out or response body.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Container-listing polish on top of the partial-success work:
- A directory whose entries all fail to stat no longer 500s. The find exec
already proved the container alive, so ListContainerDir returns the readable
entries (possibly none) plus the per-entry failures, and the handler serves a
200 partial listing; only a cancelled request or a dir that can't be listed at
all still errors.
- Over the entry cap, list the first page and set a Truncated flag rather than
erroring with end-user copy from the docker layer. The flag flows through to
the Pull dialog, which now warns the user the directory was truncated instead
of silently showing a subset.
- The Pull dialog no longer flashes a skeleton on an in-place refetch (Refresh,
or after a Pull): the listing hook clears its rows only when the path actually
changes, so a same-path reload keeps the current rows under the loading guard.
Tests: an all-entries-failed listing returns 200 with the failures; the
Truncated flag surfaces in the response. Swagger regenerated for the new field.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three defects on the shutdown and container-listing paths:
- Telemetry drain ran the langfuse and otel sinks serially under one deadline.
langfuse's ForceFlush/Shutdown ignore the caller ctx and block on their own
timeout, so a langfuse outage burned the whole budget and the healthy otel
sink's final batch was never flushed. Add observer.Drain, which flushes then
shuts down each sink in its own goroutine bounded by the deadline; main uses
it, retiring the inline flush-then-shutdown dance.
- The partial-success listing put the raw stat/list error into the 200 body's
Failures[].Message, exposing docker-layer detail (container ids, the daemon
address) that response.Error only reveals under develop mode. Send a generic
message to the client and keep the raw text dev-gated, logged server-side.
- A path read successfully by one query could also appear in Failures when
another query failed on it, order-dependent. Reconcile at assembly: drop any
failure whose path was read successfully, so no path is in both arrays.
Also bound the exec stdout buffered before parsing so a compromised sandbox
can't stream unbounded output into memory.
Tests: Drain delivers a healthy otel sink while a langfuse sink blocks and
returns within its deadline; the listing failure message is dev-gated; a path
never lands in both Files and Failures. Live: unreachable collector -> shutdown
stays bounded (~5s) and exits cleanly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
find -print0 already delimits with NUL, but the exec still ran under a TTY,
whose onlcr translation rewrites every \n in the stream to \r\n — including a
\n that is part of a filename — so a file literally named with a newline was
stat'd under the mangled name and reported unreadable. Run the exec without a
TTY and demux the resulting multiplexed stdout/stderr stream by hand (no new
dependency), so filename bytes reach the parser untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Parsing `ls -1` output was unsafe: under the TTY the exec allocates, GNU
coreutils shell-quotes names — so a readable file with a space, quote or
non-ASCII byte was stat'd with the quotes and reported unreadable — and
busybox wraps every entry in ANSI colour codes, so on an alpine/busybox
image the whole listing came back unreadable. Switch to `find -maxdepth 1
-print0`: literal bytes, portable across GNU and busybox, and the NUL
delimiter survives filenames containing newlines. Cap the entry count so a
directory with hundreds of thousands of files can't fan out into that many
Docker API calls, and classify a cancelled request or an all-entries-failed
listing as a directory-level error instead of a misleading empty 200.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A single unreadable directory entry (dangling symlink, a file removed
between ls and stat, a transient /proc entry) used to fail the whole
listing with HTTP 500, blanking the file browser and discarding every
readable sibling. The frontend already expects /proc/sys to not fail
spuriously, but the backend did the opposite.
ListContainerDir now returns a ContainerDirListing{Files, Failures}: per-
entry stat errors no longer abort the batch. GetFlowContainerFiles serves
the readable entries as HTTP 200, carries the failures back in a new
ContainerFiles.Failures field, and logs each skipped entry (capped) plus a
degradation summary. Directory-level faults (not a dir, ls failed, container
gone) still return 500 — there is no partial to show. statContainerEntries
returns successes + failures instead of the lowest-index error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The deleted queue normalized workers<=0 to a default; the errgroup helper
passed the count straight to SetLimit, where 0 deadlocks the first Go() on an
unbuffered semaphore and a negative value disables the bound entirely. The
sole caller hardcodes 20 so this was latent, but clamp it back so a future
config-driven count can't hang the listing or flood the Docker daemon.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ListContainerDir was the only consumer of pkg/queue and discarded its ordered
delivery (the caller dedups and re-sorts). Replace the fan-out with a small
errgroup helper — SetLimit keeps the same 20-way bound on concurrent stats
against the Docker daemon, the caller's context propagates into each call,
results stay in input order, and the lowest-index failure is surfaced
deterministically — then delete pkg/queue. The helper was verified byte-
identical to the previous queue implementation across 600 randomized directory
shapes (sizes, failure positions, out-of-order completion) before removal.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Added methods for non-recursive directory listing and file stat operations in the Docker client.
- Implemented a new API endpoint to retrieve files from a running container's directory.
- Updated documentation to reflect new file operations and API changes.
- Introduced data structures for container file metadata and integrated them into the flow file service.
- Enhanced flow file management capabilities with improved synchronization between local and container file systems.
Sergey Kozyrenko