A single unreadable directory entry (dangling symlink, a file removed
between ls and stat, a transient /proc entry) used to fail the whole
listing with HTTP 500, blanking the file browser and discarding every
readable sibling. The frontend already expects /proc/sys to not fail
spuriously, but the backend did the opposite.
ListContainerDir now returns a ContainerDirListing{Files, Failures}: per-
entry stat errors no longer abort the batch. GetFlowContainerFiles serves
the readable entries as HTTP 200, carries the failures back in a new
ContainerFiles.Failures field, and logs each skipped entry (capped) plus a
degradation summary. Directory-level faults (not a dir, ls failed, container
gone) still return 500 — there is no partial to show. statContainerEntries
returns successes + failures instead of the lowest-index error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The tail reader's no-trailing-newline branch (a torn/truncated final
append, the case M3's fsync defends) and the over-window error branch
had no coverage. Add cases for a newline-free last line, a torn final
append that must be rejected, and a single line exceeding the 64KiB
window.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The comment narrated the replaced sync.Map design and defended the
choice; the fixed-stripe array plus FNV indexing is self-evident and
the rationale already lives in commit d12b984.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
record() called readLastEvidenceReceiptHash on every append, which re-read
and re-hashed the entire receipts.jsonl just to get the last hash and verify
the whole chain — O(N^2) over a flow's receipts, on the tool-call hot path
under the per-path lock (M2).
Read only the last line instead (windowed ReadAt from the end), failing
closed if that line is missing, wrong-schema, or hash-mismatched. This is
O(1) per append and stays correct across the multiple recorder instances
that write one path (L6): the file stays the single source of truth, so
there is no per-instance cache to go stale. Full-chain verification belongs
in a separate read-time verify tool, not on every write.
Off by default (EVIDENCE_RECEIPTS_ENABLED=false). Measured: 1024 concurrent
appends under -race dropped from ~49s to ~7.5s.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The append wrote the receipt line and returned without flushing, so a crash
between the write and the OS flushing its page cache could lose or truncate
the last receipt, after which the fail-closed chain check halts all further
writes until the file is repaired (M3). Sync the file before returning.
Off by default (EVIDENCE_RECEIPTS_ENABLED=false), so standard deployments are
unaffected; the cost lands only when the feature is explicitly enabled.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three guards for the L6 sharded mutex pool (d12b984):
- bounded: 100k distinct flows resolve to <= 256 mutexes, so reverting to a
per-path map (the original unbounded leak) fails the test.
- high concurrency: 16 cross-executor writers on one path keep the hash chain
intact under -race.
- stripe collision: two flows that hash to the same stripe keep their own
chains intact, confirming shared stripes add contention but not corruption.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A sync.Map holding one *sync.Mutex per flow path grew unbounded for the
whole server uptime. A fixed 256-stripe array keyed by FNV-32a hash of
the path caps memory at a constant cost while preserving per-path append
serialization required by the hash-chain invariant.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Added langfuse observability support to the graphiti search tool, allowing for detailed tracking of search operations.
- Introduced a mapping of search types to human-readable titles for better traceability in logs.
- Implemented a new method to build the input payload for langfuse retrievers, encapsulating relevant search parameters.
- Enhanced error handling to include retriever status updates on success and failure, improving observability and debugging capabilities.
- Introduced a process-level singleton for the anonymizer replacer to enhance performance and ensure thread safety.
- The replacer is built once using patterns loaded at startup, allowing it to be shared across all flow executor instances.
- Updated the flow tools executor to utilize the shared replacer, simplifying the creation process and improving efficiency.
Disabled-by-default audit feature (EVIDENCE_RECEIPTS_ENABLED): appends a
hash-chained JSONL receipt per finished/failed toolcall under
<DATA_DIR>/flow-<id>/evidence/receipts.jsonl, recording toolcall
provenance plus SHA-256 hashes of args/result (no raw content).
Integration adapted to current main: the original PR built the receipt
from a database.Toolcall returned by ce.db.UpdateToolcall*Result, which
the executor no longer uses after the ToolCallLogProvider (tclp) refactor.
Receipts are now built from the in-scope toolcall data at the tclp log
sites and recorded non-fatally, so a receipt failure is logged and never
fails an otherwise-successful toolcall.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Introduced `EMBEDDING_MAX_TEXT_BYTES` to limit the maximum byte size of text sent to the embedding model.
- Renamed database connection pool settings: `DATABASE_MAX_OPEN_CONNS`, `DATABASE_MAX_IDLE_CONNS`, and `DATABASE_VECTOR_MAX_CONNS` for improved PostgreSQL connection management.
- Updated relevant documentation to reflect these new configuration options and their usage.
- Adjusted various components to utilize the new settings for enhanced performance and resource management.
- Added ToolCallLogProvider interface with methods for logging tool calls, updating success and failure statuses.
- Introduced proxyToolCallLogProvider to handle ToolCall logging operations.
- Updated flow execution components to integrate ToolCall logging, including flow workers and controllers.
- Enhanced GraphQL schema to support ToolCall logs, including queries and subscriptions for real-time updates.
- Updated documentation to reflect the new ToolCall logging features and their usage.
- Introduced shared connection pooling for PostgreSQL using `*sql.DB` for sqlc and GORM, optimizing resource usage.
- Added new environment variables: `DB_MAX_OPEN_CONNS`, `DB_MAX_IDLE_CONNS`, and `DB_VECTOR_MAX_CONNS` for configurable connection limits.
- Updated documentation to reflect new connection pooling strategy and provide operational commands for monitoring.
- Implemented shared `pgxpool` for pgvector stores to reduce connection overhead and improve performance.
- Adjusted various components to utilize the new connection pooling setup, ensuring efficient database interactions.
- Introduced WaitTaskCompletion method in FlowWorker interface to block until the current task completes or the context expires.
- Implemented signalTaskComplete to manage task completion signaling across goroutines.
- Added waitFlowCompletion tool to handle waiting for task completion with configurable timeout.
- Updated assistant provider to include wait functionality for flow completion.
- Enhanced templates and tool registry to support new wait functionality.
- GraphQL/REST CRUD + semantic search for knowledge documents
- KnowledgeStore with admin/user-scoped filtering, re-embedding on update
- Real-time subscriptions (created/updated/deleted) per user and admin
- user_id tracking in all agent-stored documents (guide/answer/code/memory)
- sqlc queries, goose migrations, privilege grants, user_id backfill
- Memory cleanup on flow deletion; stale orphan purge via migration
- Unit tests for all KnowledgeStore operations including security cases
- Frontend GraphQL schema and TypeScript types regenerated
- Updated `getHTML` and `getMD` methods to return warnings for small content instead of errors.
- Implemented checks for binary URLs, returning descriptive errors when such URLs are encountered.
- Added new tests to validate the updated behavior for small and empty content handling.
- Replaced ambiguous "user's language" guidance in tools/args.go with explicit engagement-log vs technical-channel markers per field, with strong English-only requirement for vector-store and search-engine queries.
- Added a unified LANGUAGE POLICY block to every agent prompt (primary_agent, assistant, pentester, coder, installer, searcher, memorist, generator, refiner, reporter, enricher), tailored per agent based on its actual tool set.
- Extended template variables and tool access (TerminalToolName, FileToolName) for coder, pentester, installer, memorist, generator, refiner, and enricher to match their runtime tool registrations.
- Fixed inverted UseAgents condition and removed misleading vector-store write references in assistant prompt; corrected MEMORY SYSTEM INTEGRATION for mode-specific tool references.
- Compressed COMPLETION REQUIREMENTS across templates and aligned closing-tool guidance with the channel mapping (engagement-log message vs technical-channel result).
Fixes#285.
Co-Authored-By: Octopus <liyuan851277048@icloud.com>
- Added methods for non-recursive directory listing and file stat operations in the Docker client.
- Implemented a new API endpoint to retrieve files from a running container's directory.
- Updated documentation to reflect new file operations and API changes.
- Introduced data structures for container file metadata and integrated them into the flow file service.
- Enhanced flow file management capabilities with improved synchronization between local and container file systems.
- Added new endpoints for managing flow files, including listing, uploading, and deleting files within flow workspaces.
- Introduced FlowFile model to represent file metadata.
- Enhanced GraphQL schema to support flow file operations and subscriptions for real-time updates.
- Updated API documentation to reflect new flow file functionalities.
- new get_flow_status / stop_flow / submit_flow_input / patch_flow_subtasks tools in assistant executor, backed by FlowWorker callbacks
- flowStatusTool supports 5 detail levels with verbose mode, polling for task readiness, and per-size summarization
- summarizer LRU cache (1000 entries, 4 h TTL, SHA-256 key) on flowProvider to skip redundant LLM calls
- updated assistant.tmpl with full flow management protocol: state reference, decision guide, constraints
- updated flow_execution.md with terminal timeout config, new tools, and cache
- Changed default terminal tool timeout from 600 to 1200 seconds.
- Updated related documentation across .env.example, README.md, and config files to reflect the new timeout settings and their constraints.
- Enhanced descriptions in code comments and documentation to clarify timeout behavior, including clamping rules for values outside the accepted range.
Sergey Kozyrenko