[PR #12] feat(stdlib): add file I/O and jq-filter commands #12

New Issue

2026-02-15T18:15:41-05:00

yindo commented

2026-02-15 18:15:41 -05:00

📋 Pull Request Information

Original PR: https://github.com/openclaw/lobster/pull/12
Author: @sfo2001
Created: 2/15/2026
Status: 🔄 Open

Base: main ← Head: feat/file-io-jq-filter

📝 Commits (3)

671dcd6 fix(stdlib): remove unused Transport type and transport variable
1d88b05 feat(stdlib): add file I/O and jq-filter commands
2e6230f feat(stdlib): add --raw flag to jq-filter command

📊 Changes

10 files changed (+814 additions, -4 deletions)

View changed files

📝 .gitignore (+3 -0)
📝 src/cli.ts (+1 -1)
📝 src/commands/registry.ts (+6 -0)
➕ src/commands/stdlib/file_read.ts (+86 -0)
➕ src/commands/stdlib/file_write.ts (+70 -0)
➕ src/commands/stdlib/jq_filter.ts (+84 -0)
📝 src/commands/stdlib/llm_task_invoke.ts (+0 -3)
➕ test/file_read.test.ts (+197 -0)
➕ test/file_write.test.ts (+196 -0)
➕ test/jq_filter.test.ts (+171 -0)

📄 Description

Motivation

Pipelines currently have no native way to read from or write to local files. Loading data requires shelling out (exec cat + manual parsing), and there is no structured JSON transformation primitive beyond map. file.read, file.write, and jq-filter fill these gaps: pipelines can ingest local datasets, persist intermediate results, and apply arbitrary jq expressions
without leaving the pipeline or spawning ad-hoc shell commands.

Summary

Add file.read command: reads files into the pipeline with auto-detection of JSON, JSONL, and plain text formats
Add file.write command: writes pipeline items to disk as JSON, JSONL, or text; supports tee passthrough for downstream stages
Add jq-filter command: applies jq expressions to each pipeline item, with --raw flag for plain string output
50 MB file size guard on file.read; file.write creates parent directories by default (--mkdir)

Changes

Category	Files
New commands	`src/commands/stdlib/file_read.ts`, `file_write.ts`, `jq_filter.ts`
Registry	`src/commands/registry.ts` (+3 registrations)
CLI	`src/cli.ts` (minor)
Tests	`test/file_read.test.ts` (12 tests), `test/file_write.test.ts` (10 tests), `test/jq_filter.test.ts` (10 tests)

+682 lines across 9 files.

Test plan

pnpm build passes
pnpm lint passes (0 warnings)
node --test dist/test/file_read.test.js -- 12 tests
node --test dist/test/file_write.test.js -- 10 tests
node --test dist/test/jq_filter.test.js -- 10 tests
Requires jq on PATH for jq-filter tests

Use of AI

Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/openclaw/lobster/pull/12 **Author:** [@sfo2001](https://github.com/sfo2001) **Created:** 2/15/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `feat/file-io-jq-filter` --- ### 📝 Commits (3) - [`671dcd6`](https://github.com/openclaw/lobster/commit/671dcd66b39ddfb8ba690e0d8571ca140efbcc75) fix(stdlib): remove unused Transport type and transport variable - [`1d88b05`](https://github.com/openclaw/lobster/commit/1d88b0572eb028b71c5e73d87f76b899cb1697cd) feat(stdlib): add file I/O and jq-filter commands - [`2e6230f`](https://github.com/openclaw/lobster/commit/2e6230f3d4217ec6eca0e0547493b5d988f213f8) feat(stdlib): add --raw flag to jq-filter command ### 📊 Changes **10 files changed** (+814 additions, -4 deletions) <details> <summary>View changed files</summary> 📝 `.gitignore` (+3 -0) 📝 `src/cli.ts` (+1 -1) 📝 `src/commands/registry.ts` (+6 -0) ➕ `src/commands/stdlib/file_read.ts` (+86 -0) ➕ `src/commands/stdlib/file_write.ts` (+70 -0) ➕ `src/commands/stdlib/jq_filter.ts` (+84 -0) 📝 `src/commands/stdlib/llm_task_invoke.ts` (+0 -3) ➕ `test/file_read.test.ts` (+197 -0) ➕ `test/file_write.test.ts` (+196 -0) ➕ `test/jq_filter.test.ts` (+171 -0) </details> ### 📄 Description ## Motivation Pipelines currently have no native way to read from or write to local files. Loading data requires shelling out (`exec cat` + manual parsing), and there is no structured JSON transformation primitive beyond `map`. `file.read`, `file.write`, and `jq-filter` fill these gaps: pipelines can ingest local datasets, persist intermediate results, and apply arbitrary jq expressions without leaving the pipeline or spawning ad-hoc shell commands. ## Summary - Add `file.read` command: reads files into the pipeline with auto-detection of JSON, JSONL, and plain text formats - Add `file.write` command: writes pipeline items to disk as JSON, JSONL, or text; supports tee passthrough for downstream stages - Add `jq-filter` command: applies jq expressions to each pipeline item, with `--raw` flag for plain string output - 50 MB file size guard on `file.read`; `file.write` creates parent directories by default (`--mkdir`) ## Changes | Category | Files | |---|---| | New commands | `src/commands/stdlib/file_read.ts`, `file_write.ts`, `jq_filter.ts` | | Registry | `src/commands/registry.ts` (+3 registrations) | | CLI | `src/cli.ts` (minor) | | Tests | `test/file_read.test.ts` (12 tests), `test/file_write.test.ts` (10 tests), `test/jq_filter.test.ts` (10 tests) | +682 lines across 9 files. ## Test plan - [x] `pnpm build` passes - [x] `pnpm lint` passes (0 warnings) - [x] `node --test dist/test/file_read.test.js` -- 12 tests - [x] `node --test dist/test/file_write.test.js` -- 10 tests - [x] `node --test dist/test/jq_filter.test.js` -- 10 tests - [x] Requires `jq` on PATH for jq-filter tests ## Use of AI - Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

yindo added the pull-request label 2026-02-15 18:15:41 -05:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: openclaw/lobster#12