mlx-knife/docs/ADR/ADR-014-Unix-Pipe-Integration.md

ADR-014 — Unix Pipe Integration

• Status: Beta (Phase 1 feature complete)
• Authors: mlx-knife maintainers
• Date: 2025-11-16
• Updated: 2025-12-03
• Target Version: 2.0.4-beta.1 (Phase 1)
• Prerequisite: 2.0.3 (stdout/stderr separation) ✅
• Related: ADR-012 (Vision Support), Issue #26 (Embeddings API)

API Stability

Phase 1 (2.0.4-beta.1): Feature complete behind MLXK2_ENABLE_PIPES=1 gate.

Aspect	API Stability	Notes
stdin -	Stable	Unix standard, will not change
isatty() auto-batch	Stable	Unix standard, will not change
Exit codes 0/1	Stable	Unix standard, will not change
\n\n separator	Stable default	May become configurable, default unchanged
mlx-run wrapper	Stable	Part of toolchain family
SIGPIPE/BrokenPipeError	Stable	Robust Unix pipe handling

Phase 2+ (future): Additive features only, no breaking changes to Phase 1 API.

• mlx-tee tool (new tool, not modifying existing)
• --in/--out flags (additive, - semantics unchanged)

Gate removal: When MLXK2_ENABLE_PIPES=1 is removed (stable release), Phase 1 API is frozen.

Context

Current mlxk run requires all input as command-line arguments. For complex workflows (vision→reasoning chains, multi-stage analysis, preprocessing), users need temporary files:

# Current: Requires temp files
mlxk run vision_model --image x.jpg "describe" > /tmp/desc.txt
mlxk run reasoning_model "$(cat /tmp/desc.txt)" "analyze for hallucinations"

Unix philosophy encourages composable pipelines where tools read stdin and write stdout. Tools like tar, docker, jq, and git support - for pipe integration. Can mlx-knife adopt similar semantics for model chaining?

Motivating use case (vision-reasoning chain):

mlxk run vision_model --image cockpit.jpg "Describe what you see" \
  | mlxk run reasoning_model - "Check this description for technical accuracy"

Goals

1. Enable stdin/stdout pipes for model chaining without temp files
2. Maintain backward compatibility (existing CLI unchanged)
3. Follow Unix conventions (- for stdin, isatty() for output formatting)
4. Support all model types (text, vision, reasoning - generic infrastructure)
5. Keep implementation simple (no complex protocol, just text streams)
6. Establish pipe semantics for local workflows that are topologically equivalent to distributed execution (test locally, deploy to cluster without syntax changes)

Non-Goals

• Binary protocols or structured formats (JSON-RPC, msgpack)
• Bidirectional streaming (stdin→stdout simultaneously)
• Interactive prompts in pipe mode
• Multipart/MIME protocols (use server API for that)
• Cluster orchestration (load balancing, node discovery - see broke-cluster)
• Network protocols for distributed execution (HTTP/gRPC between nodes)
• Production-ready multi-model tools (mlxk-tee is experimental/reference implementation)

Proposed Design

Input: Explicit - (stdin)

Syntax:

mlxk run <model> -                    # Read entire prompt from stdin
mlxk run <model> - "Additional text"  # stdin + CLI arg combined

Semantics:

• - as positional prompt argument → read from stdin until EOF
• If additional text provided → f"{stdin_content}\n\n{cli_arg}"
• Compatible with --image flag: mlxk run vision_model --image x.jpg "describe"

Implementation:

def parse_prompt(args):
    if args.prompt == "-":
        stdin_content = sys.stdin.read()
        if args.additional_text:
            return f"{stdin_content}\n\n{args.additional_text}"
        return stdin_content
    return args.prompt

Why explicit - (not implicit isatty() check)?

• Predictable: User declares intent ("I want stdin")
• Avoids surprises: Empty prompt vs. waiting for stdin
• Unix precedent: tar -, docker load -, jq -
• Scripting-safe: $(mlxk run ...) doesn't accidentally block on stdin

Output: Implicit isatty() (stdout)

Behavior:

if sys.stdout.isatty():
    # TTY → Pretty output with colors, stats, token/sec
    print_colored(response, tokens_per_sec=7.9, ...)
else:
    # Pipe → Clean text only (no ANSI codes, no stats)
    print(response["text"])

Why implicit for stdout?

• User expects clean text in pipes (like git log, ls)
• No debugging confusion: | cat shows what's piped
• Override available: --json forces JSON even in TTY

Critical for Unix tool integration: Plain text output is required for standard Unix tools:

• tee - Log intermediate results
• grep - Filter/search responses
• sed/awk - Text transformations
• head/tail - Truncate output
• wc - Count words/lines
• sort/uniq - Deduplication

Example (only works with plain text):

echo "test" | mlxk run model1 - | tee stage1.txt | grep "keyword" | mlxk run model2 -

Edge cases:

• Reasoning models: Only output content (not reasoning_content) in pipe mode
• Streaming: Works in both modes (SSE in TTY, plain text in pipe)
• Errors: Always stderr (implemented in 2.0.3), exit code 1
• Exit codes: 0 (success), 1 (error) - standard Unix behavior

Example Workflows

Single-Model Chains

1. Vision → Reasoning Chain

mlxk run mlx-community/Llama-3.2-11B-Vision-Instruct-4bit \
  --image cockpit.jpg "Describe the cockpit in detail" \
  | mlxk run mlx-community/DeepSeek-R1-Distill-Qwen-14B - \
  "Analyze this description for hallucinations"

2. Multi-stage Analysis

cat codebase.py \
  | mlxk run model1 - "Extract function signatures" \
  | mlxk run model2 - "Generate unit tests for these"

3. Batch Processing

for img in *.jpg; do
  mlxk run vision_model --image "$img" "describe"
done | mlxk run summarizer_model - "Summarize these descriptions"

4. Interactive (TTY) - No change

mlxk run model "test"  # Pretty output with stats (as today)

Multi-Model Workflows (mlxk-tee)

Motivation: Test distributed topologies locally before scaling to cluster. Different models locally = different nodes in cluster (topologically equivalent).

⚠️ RESOURCE WARNING: Parallel LLM inference on unified memory (Apple Silicon) can cause severe RAM pressure. Focus on embedding use cases where models are small and can run on CPU (--cpu-only), keeping GPU free for primary LLM workload. Advanced users may run parallel LLM inference but must understand RAM constraints.

1. Parallel Embedding (RECOMMENDED - Safe & Practical)

# Embed with multiple models for ensemble retrieval
echo "machine learning tutorial" \
  | mlxk-tee \
      "mlxk embed nomic-embed --cpu-only -" \
      "mlxk embed bge-small --cpu-only -" \
  | combine-embeddings > ensemble.jsonl

Use Case: Multi-model embeddings for robust RAG retrieval Safety: Small models (~500MB each), CPU-only keeps GPU free Cluster equivalent: Two embedding nodes

2. Model A/B Testing (⚠️ ADVANCED - RAM intensive)

# Compare two models on same prompt
# WARNING: Both models loaded simultaneously!
echo "Explain quantum computing" \
  | mlxk-tee \
      "mlxk run phi-3-mini -" \
      "mlxk run deepseek-r1 -" \
  | diff -y --width=160

Use Case: Quality comparison, regression testing ⚠️ Risk: High RAM usage (both models in memory) Safer alternative: Sequential testing or cluster deployment Cluster equivalent: Two nodes with different models

3. Multi-Source Embedding (RECOMMENDED - Safe)

# Index documents with multiple embedding models
cat docs/*.md | while read doc; do
  echo "$doc" | mlxk-tee \
    "mlxk embed semantic-model --cpu-only - >> semantic.jsonl" \
    "mlxk embed keyword-model --cpu-only - >> keyword.jsonl"
done

Use Case: Hybrid search (semantic + keyword embeddings) Safety: CPU-only embeddings, no GPU contention Cluster equivalent: Parallel embedding nodes

4. Ensemble / Majority Voting (⚠️ ADVANCED)

# Three models vote on classification
# WARNING: 3 models loaded simultaneously!
echo "Sentiment: 'meh, it's okay'" \
  | mlxk-tee \
      "mlxk run model1 - 'Answer: positive or negative'" \
      "mlxk run model2 - 'Answer: positive or negative'" \
      "mlxk run model3 - 'Answer: positive or negative'" \
  | python3 majority-vote.py

Use Case: Robust classification, consensus-based decisions ⚠️ Risk: Very high RAM usage (3 models) Safer alternative: Cluster deployment recommended Cluster equivalent: Three nodes for redundancy

5. Parallel Vision + Embedding (RECOMMENDED - Safe pattern)

# Process image for analysis AND indexing simultaneously
mlxk run vision-model --image photo.jpg "describe" \
  | mlxk-tee \
      "cat > description.txt" \
      "mlxk embed vision-embed --cpu-only - >> index.jsonl"

Use Case: RAG indexing while preserving description Safety: Only vision LLM on GPU, embedding on CPU Cluster equivalent: Vision on Node1, Embedding on Node2

6. Speed vs Quality Trade-off (⚠️ ADVANCED)

# Fast preview + thorough analysis in parallel
# WARNING: 2 models loaded simultaneously!
echo "Analyze: $(cat script.py)" \
  | mlxk-tee \
      "mlxk run fast-small - | notify-user" \
      "mlxk run slow-large - | save-to-db"

Use Case: Instant feedback + high-quality archival ⚠️ Risk: RAM contention between models Safer alternative: Sequential execution or cluster Cluster equivalent: Fast node (M1) + slow node (Cloud GPU)

7. Multi-Format Output (Safe - no parallel LLMs)

# Save output to multiple formats
mlxk run model "Implement binary search in Python" \
  | mlxk-tee \
      "tee code.py" \
      "python3 -m py_compile -" \
      "wc -l > metrics.txt"

Use Case: Multiple outputs from single inference Safety: No parallel LLMs, just stream processing Cluster equivalent: Single node, multiple consumers

8. Pipeline Debugging (Safe)

# Log every stage of complex pipeline
cat input.txt \
  | mlxk run stage1 - \
  | mlxk-tee "tee stage1.log" "mlxk run stage2 -" \
  | mlxk-tee "tee stage2.log" "mlxk run stage3 -" \
  | tee final.log

Use Case: Debugging multi-stage workflows Cluster equivalent: Distributed pipeline with centralized logging

Summary: Safe vs Advanced Patterns

✅ SAFE (Recommended for local development):

1. Parallel embeddings with --cpu-only (GPU stays free for LLM)
2. Stream processing (tee, wc, grep - no model loading)
3. Sequential LLM calls (one model at a time)
4. Vision → Embedding (LLM on GPU, embedding on CPU)

⚠️ ADVANCED (Cluster recommended):

1. Parallel LLM inference (2+ large models simultaneously)
2. Ensemble voting (3+ models = high RAM pressure)
3. Speed vs Quality (concurrent large models)

Rule of thumb:

• Local: Embeddings in parallel (safe), LLMs sequential
• Cluster: LLMs in parallel across nodes (distributed RAM)

RAG Use Cases (Low-Hanging Fruit)

Motivation: Pipe integration + embeddings (Issue #26) enables simple RAG workflows without complex infrastructure (FAISS, ChromaDB, LangChain). Unix philosophy: composable tools > monolithic frameworks.

Classic RAG Pipeline (Stateless)

Query → Embed → Vector Search → Retrieve Context → LLM → Answer

As Unix Pipes:

echo "What is MLX?" \
  | mlxk embed model - \
  | vector-search index.db - --top-k 3 \
  | mlxk run chat-model - "Answer based on this context"

What Works TODAY (No Embeddings)

1. Keyword-Based RAG (grep/ripgrep)

# "Poor man's RAG" - surprisingly effective!
rg "stop.*token" docs/ --context 5 \
  | mlxk run chat-model - "Explain stop token handling"

2. LLM as Retriever (Expensive but Creative)

# Two-stage: Filter → Analyze
cat all_documentation.txt \
  | mlxk run small-fast-model - "Extract sections about vision support" \
  | mlxk run large-reasoning-model - "Analyze the vision roadmap"

3. Multi-Document Synthesis

# Combine multiple sources
cat design_doc.md spec.md changelog.md \
  | mlxk run analyst-model - "Compare planned vs implemented features"

What Works WITH Embeddings (Issue #26)

Once mlxk embed is implemented:

1. Codebase Q&A

# Index once (one-time setup)
find . -name "*.py" -exec cat {} \; \
  | mlxk embed code-model - --batch --output codebase.jsonl

# Query repeatedly (fast)
echo "How does stop token detection work?" \
  | mlxk embed code-model - \
  | cosine-search codebase.jsonl - --top-k 5 \
  | mlxk run chat-model - "Explain based on this code"

2. Semantic Code Search

# Find similar implementations
mlxk embed code-model --file runner.py "function generate_batch" \
  | vector-search codebase.jsonl - --top-k 10 \
  | mlxk run code-model - "Compare these implementations for differences"

3. Document Analysis (Vision + RAG)

# Multi-modal RAG: Image → Text → Semantic Search → Analysis
mlxk run vision_model --image invoice.pdf "Extract all text" \
  | tee >(mlxk embed doc_model -) \
  | mlxk run analyst_model - "Extract key financial data"

4. Simple JSON "Vector Store"

# No FAISS/Chroma needed - JSONL is enough for small datasets!
# embeddings.jsonl format:
{"text": "...", "embedding": [0.1, 0.2, ...], "metadata": {...}}

# Search with jq + Python one-liner
cat embeddings.jsonl \
  | python3 -c "import sys, json, numpy as np
query = json.loads(sys.argv[1])
for line in sys.stdin:
    doc = json.loads(line)
    score = np.dot(query['embedding'], doc['embedding'])
    if score > 0.8:
        print(json.dumps({'text': doc['text'], 'score': score}))" \
  "$(echo 'query text' | mlxk embed model -)"

Why Constraints are Features

Property	Limitation	Benefit
Stateless	No persistence between runs	Reproducible, git-versionable, testable
Text Streams	No binary vector formats	Debuggable with | less, | head, tee
Separate Tools	Need external vector search	Composable, Unix-like, no vendor lock-in
No Vector DB	Slower than FAISS (linear scan)	No infrastructure, works on laptop, CI/CD-friendly
JSONL Format	Not optimized for GB-scale	Human-readable, jq-queryable, git-diffable

Sweet spot: 1K-100K documents (most codebases, documentation sets, personal knowledge bases)

External Tools for Vector Search

Lightweight (no dependencies):

# Pure Python (numpy only)
pip install numpy
# cosine-search.py (100 LOC, ships with mlx-knife examples/)

Production-ready:

# sqlite-vss (SQLite extension)
pip install sqlite-vss
sqlite3 embeddings.db "CREATE VIRTUAL TABLE vec USING vss0(...)"

# usearch (fast, no server)
pip install usearch

Framework integration:

# LangChain (if you must)
echo "query" | mlxk embed model - | python langchain_search.py

Issue #26 Motivation

What mlx-knife needs:

• CLI: mlxk embed <model> <text> or mlxk embed <model> -
• Server: /v1/embeddings (OpenAI-compatible)
• Batch mode: --batch for processing large documents
• Output: JSON or JSONL (pipe-friendly)

Low-hanging fruit approach:

1. ✅ Pipe integration (ADR-014) - enables composition
2. 🔄 Embeddings API (Issue #26) - completes the RAG loop
3. 📦 Ship example cosine-search.py (100 LOC, no framework)
4. 📚 Document JSONL-based RAG patterns (README)
5. 🎯 No need for complex RAG frameworks - Unix pipes are the framework!

User benefits:

• RAG without LangChain/LlamaIndex (lighter dependencies)
• Debuggable workflows (set -x, pipe inspection)
• Testable pipelines (fixtures = text files)
• Git-versionable indexes (JSONL in repo)
• CI/CD friendly (no vector DB server needed)

Implementation Plan

Phase 1: stdin/stdout Support for mlxk run (2.0.4-beta.1) ✅ COMPLETE

Critical: This phase MUST be implemented before ADR-015 (embeddings). Minimal viable pipe semantics to enable embed use cases.

Scope: mlxk run with - stdin support

Implementation Tasks:

• Add - parsing to mlxk2/cli.py argument parser
• Update mlxk2/operations/run.py to read stdin when - detected
• Handle combined stdin + additional text: stdin_content + "\n\n" + additional_text
• Detect isatty() in output formatting:
  • TTY: Pretty output (colors, stats, streaming)
  • Pipe: Clean text only (no ANSI, no stats, batch mode)
• Automatic batch mode in pipes: if not sys.stdout.isatty(): stream = False
• Implement mlx-run wrapper tool (pyproject.toml + 10 LOC wrapper)
• SIGPIPE handler (signal.signal(SIGPIPE, SIG_DFL)) for robust pipe termination
• BrokenPipeError handling in streaming + batch output
• Unit tests for stdin edge cases (empty, stdin-only, trailing text)
• Unit tests for SIGPIPE and BrokenPipeError handling
• Regression tests: Ensure TTY behavior unchanged

Enables:

• ✅ Model chains: mlxk run model1 - | mlxk run model2 -
• ✅ ADR-015 implementation: mlxk embed model - uses same stdin semantics
• ✅ RAG pipelines: mlxk embed - | cosine-search - | mlxk run -

Explicitly NOT in Phase 1:

• ❌ --stream flag (YAGNI - automatic batch in pipes is sufficient)
• ❌ Reasoning-specific output handling (defer to ADR-010)
• ❌ Vision integration (defer to ADR-012)
• ❌ --in / --out flags (defer to Phase 2+)

Phase 2: Example Scripts (2.0.4+)

Note: mlx-tee is not a core mlx-knife tool but an example script demonstrating parallel model execution. For production distributed workflows, see broke-cluster.

• examples/mlx-tee.py - Reference implementation (~100 LOC)
  • Broadcast stdin to multiple commands in parallel
  • SSH placeholder for remote execution (@node:command)
  • ThreadPoolExecutor for parallel execution
• examples/cosine-search.py - Vector search for RAG (depends on ADR-015)
• examples/rag-pipeline.sh - End-to-end RAG example

Phase 3: Documentation (2.0.3 or 2.1)

• README: Unix Pipes section with single-model + multi-model examples
• TESTING-DETAILS.md: Pipe mode test cases
• mlxk run --help: Document - syntax
• examples/README.md: mlxk-tee usage guide
• Document topological equivalence (local models ↔ cluster nodes)

Phase 4: Server Integration (Future - 2.4+)

• /v1/completions with stream: true → SSE
• Client-side pipe emulation (broke-nchat)
• Server stays simple (no pipe protocol)

Risks & Mitigation

Risk 1: Parallel LLM Inference (RAM Pressure)

Problem: mlxk-tee with multiple LLMs loads all models simultaneously → OOM on unified memory systems

Mitigation:

• Primary use case: Embeddings (small models, --cpu-only)
• Document safe patterns (embeddings parallel, LLMs sequential)
• Warning in examples: "⚠️ ADVANCED - RAM intensive"
• Recommend cluster for parallel LLM workflows
• Future: RAM budget check before parallel execution

Risk 2: Binary Input (images as stdin)

Problem: mlxk run vision-model - could confuse users (image as binary stdin?)

Mitigation:

• Vision models REQUIRE --image <path> flag (no binary stdin)
• - only for text prompts
• Error message: "Vision models require --image flag, cannot use binary stdin"

Risk 3: Large Input (RAM)

Problem: cat 10GB.txt | mlxk run model - could OOM

Mitigation:

• Document: stdin is buffered in RAM (warn about large inputs)
• Future: Streaming stdin processing (chunked prompts)
• Current: Practical limit ~100MB (acceptable for most use cases)

Risk 4: Prompt Injection via Pipes

Problem: Malicious content in piped data

Mitigation:

• Not our concern (user controls pipeline)
• Document: Sanitize untrusted input before piping
• Same risk as cat untrusted.txt | bash (Unix philosophy)

Risk 5: Backward Compatibility

Problem: Existing scripts using - as prompt text

Mitigation:

• Unlikely: - is unusual prompt text
• Feature gate: MLXK2_ENABLE_PIPES=1 (optional, for alpha)
• Can graduate to default in 2.1 after validation

Alternatives Considered

Alternative 1: Explicit --input - / --output - flags

Rejected: Verbose, breaks Unix conventions (most tools use - directly)

Alternative 2: Implicit stdin (no - needed)

echo "test" | mlxk run model  # Implicit stdin

Rejected: Confusing when stdin is empty, scripting pitfalls, unpredictable

Alternative 3: Named pipes / temp files

Rejected: Not cross-platform, requires cleanup, complexity

Alternative 4: Server-side pipe protocol

Rejected: Scope creep, mlx-knife is CLI-first, server stays simple

Alternative 5: JSON-by-default in pipes

Considered: Make all pipe output JSON for robustness (error propagation, metadata)

Rejected for Phase 1:

• Breaks Unix tool integration (tee, grep, sed, awk, wc, head/tail, sort/uniq)
• JSON output makes standard text processing impossible
• Example broken workflow: mlxk run model - | tee log.txt | grep "keyword"
  • With JSON: grep matches JSON syntax, not response content
  • With plain text: Works as expected

Phase 1 Decision:

• Plain text default (Unix compatibility)
• --json only for terminal output or external tools (jq, broke-cluster)
• Accept limitation: No metadata in pipes (use server API if needed)

Future Evaluation:

• Gather usage data: mlxk→mlxk chains vs mlxk→Unix tool usage
• Consider: stderr JSON metadata, separate metadata files, or mlxk-pipe wrapper
• Defer decision until real-world usage patterns emerge

--json Flag Behavior

Terminal (TTY):

mlxk run model "test" --json
# Output: {"status": "success", "data": {"response": "..."}, ...}

In Pipes (NOT RECOMMENDED):

# Problem: JSON output breaks Unix tools
echo "test" | mlxk run model - --json | grep "keyword"
# grep matches JSON keys/syntax, not response content ❌

# Workaround (if JSON needed for external tools):
echo "test" | mlxk run model - --json | jq -r '.data.response' | mlxk run model2 -
# But this is complex and fragile

Rule for Phase 1:

• Don't use --json when piping to Unix tools or other mlxk commands
• Use plain text (auto-detect) for pipe workflows
• Use --json only for terminal inspection or external consumers (jq scripts, broke-cluster)

Future Consideration:

• Auto-detect JSON input (parse if stdin is mlxk JSON format)
• --json could mean: "JSON output + parse JSON input if available"
• Deferred to post-Phase 1 based on user feedback

Open Questions

1. Reasoning models: Pipe mode outputs only content or also reasoning_content?

   • Proposal: Add --think-only flag for reasoning-only output
   • Default pipe: content (user-facing answer)

2. Error handling: Partial output on error (streaming), or no output?

   • Proposal: stderr for errors, stdout partial results (like grep)

3. Multiline prompts: How to combine stdin + CLI arg?

   • Proposal: "{stdin}\n\n{cli_arg}" (paragraph separator)

4. Feature gate: Ship behind MLXK2_ENABLE_PIPES=1 or direct to main?

   • Proposal: Direct to main (low risk, well-established Unix pattern)

5. Embeddings output format: Should mlxk embed output raw JSON arrays or JSONL with metadata?

   • Proposal: JSONL with {"text": "...", "embedding": [...], "metadata": {...}} (pipe-friendly, extendable)
   • Allows | jq, | grep, easy inspection

6. Embedding execution mode: Should mlxk embed default to --cpu-only?

   • Proposal: YES - Keep GPU free for primary LLM workload
   • Embeddings are small (100MB-1GB) and fast enough on CPU
   • User can override with --gpu if needed
   • Critical for safe parallel embedding workflows (mlxk-tee)

Success Criteria

Phase 1 (Complete)

• cat prompt.txt | mlxk run model - works (basic stdin)
• mlxk run model - "extra" combines stdin + arg
• TTY output unchanged (colors, stats preserved)
• Pipe output clean (no ANSI, only text)
• Exit codes: 0 (success), 1 (error) - pipes abort on error
• SIGPIPE handling: mlxk run model | head -1 terminates cleanly
• BrokenPipeError: Streaming/batch output handles pipe closure gracefully
• No regression in existing CLI behavior
• --json flag documented as NOT for pipe workflows (terminal/jq use only)

Phase 2 (Examples - Complete)

• examples/mlx-tee.py - Parallel model execution with SSH placeholder

Future (Depends on ADR-015 Embeddings)

• Unix tools work: mlxk run model - | tee log.txt | grep "keyword" | wc -l
• Vision chains work: mlxk run vision_model --image x.jpg "describe" | mlxk run chat_model -
• RAG workflow: mlxk embed model - | cosine-search - | mlxk run chat_model -
• examples/cosine-search.py - Vector search for RAG
• examples/rag-pipeline.sh - End-to-end RAG example

Future: Distributed Execution

The pipe semantics defined in this ADR are location-agnostic by design. While mlx-knife focuses on local execution, the same stdin/stdout contracts enable distributed execution.

Topological Equivalence:

• Local: Different models on one machine → Test workflows locally
• Cluster: Different nodes in network → Deploy same workflows distributed

Example (identical syntax):

# Local development (mlx-knife)
echo "test" | mlxk-tee "mlxk run model1 -" "mlxk run model2 -"

# Production cluster (broke-cluster)
echo "test" | broke-tee "node1:model1" "node2:model2"

Out of scope for mlx-knife:

• Cluster orchestration (load balancing, node discovery)
• Network protocols (HTTP/gRPC between nodes)
• Distributed state management

These concerns are addressed in the broke-cluster project, which builds on this ADR's pipe semantics.

Next Steps

1. Validation: Prototype in feature branch (2-3 hours implementation)
2. Testing: Unit tests + manual workflow validation
3. mlxk-tee implementation: Multi-model broadcast tool (examples/)
4. Documentation: README examples + mlxk-tee usage guide
5. Decision: Ship in 2.0.3 (quick win) or 2.1 (with vision support)?
6. Community feedback: Reddit/Discord examples to gauge adoption
7. Issue #26 synergy: Once pipes are stable, embeddings API enables RAG workflows (ship example cosine-search.py)

---

Status: Beta - Phase 1 feature complete (2.0.4-beta.1).

Completed effort:

• Phase 1 (stdin/stdout): ✅ Complete (2025-12-03)
• Phase 2 (examples/mlx-tee.py): ✅ Complete (2025-12-03)

Remaining effort:

• Phase 3 (docs): 0.5 session
• Future examples (cosine-search.py, rag-pipeline.sh): Depends on ADR-015

Compatibility: 100% backward compatible (additive feature, behind gate)

---

Appendix A: Toolchain Familie (Design Exploration)

This ADR establishes pipe semantics for a family of specialized Unix-style tools built on shared MLX infrastructure.

Proposed Toolchain

Core Management:

• mlxk - Model management (list, health, pull, serve, rm)

Execution Tools:

• mlx-run - Direct model execution (Phase 1 - this ADR)
• mlx-tee - Multi-model broadcast (Phase 2 - this ADR)
• mlx-embed - Embeddings (ADR-015)

Vision Tools (ADR-012):

• mlx-img - Image → text (vision models)

Conversion Tools (future):

• mlx-convert - Format conversion (GGUF ↔ MLX, quantization)

Cluster Tools (broke-cluster project):

• broke-run - Distributed execution (topologically equivalent to mlx-run)
• broke-tee - Distributed multi-model (topologically equivalent to mlx-tee)

Design Principles

1. Shared Core: All tools use MLXRunner, mlxk2.core, same model loading
2. Consistent Pipes: All tools follow - for stdin, isatty() for output
3. Specialization: Each tool optimized for specific use case
4. Composability: Tools chain naturally via Unix pipes
5. Topological Equivalence: Local tools (mlx-*) have distributed equivalents (broke-*) with identical syntax

Rationale: Why NOT "implicit run"?

Considered: Making mlxk auto-detect pipe mode and execute models implicitly:

cat prompt.txt | mlxk phi-3    # Implicit run in pipe mode?

Rejected because:

1. Parser Complexity: Must disambiguate commands vs models (expensive fuzzy-matching)
2. Namespace Pollution: Can never use model names matching commands
3. Inconsistency: Different behavior TTY vs Pipe (violates principle of least surprise)
4. Future-Brittleness: Every new command blocks a model name

Preferred Solution: Separate tools with clear purpose:

• mlxk = Management (never ambiguous)
• mlx-run = Execution (compact, pipe-native)
• mlx-tee = Multi-model (specialized)

Implementation (Phase 1)

mlx-run is trivial wrapper around mlxk run:

# pyproject.toml
[project.scripts]
mlxk = "mlxk2.cli:main"
mlx-run = "mlxk2.tools.run:main"

# mlxk2/tools/run.py
def main():
    """mlx-run: Direct model execution."""
    sys.argv.insert(1, 'run')
    from mlxk2.cli import main as mlxk_main
    mlxk_main()

Effort: 10 lines of code, zero parser complexity increase.

User Experience:

# Traditional (always works):
mlxk run phi-3 "test"
cat prompt.txt | mlxk run phi-3 -

# Compact (pipe-optimized):
mlx-run phi-3 "test"
cat prompt.txt | mlx-run phi-3 -

# Power users can alias:
alias m='mlx-run'
cat prompt.txt | m phi-3 -

---

Appendix B: Advanced Input/Output Semantics (WIP)

Status: Design exploration for future phases. NOT part of Phase 1 implementation.

Motivation

Phase 1 provides basic stdin/stdout pipes. Advanced workflows may require:

• Multi-source input: Combine stdin + multiple files (RAG, context injection)
• Pipeline debugging: Capture intermediate outputs without breaking chains
• Structured prompts: CLI equivalent of server's message arrays

Proposed Extensions (Post-Phase 1)

Extension 1: Multi-Source Input (--in)

Use Case: Combine query (stdin) with knowledge bases (files):

echo "How does MLX quantization work?" | mlx-run model \
  --in docs/quantization.md \
  --in examples/4bit.py \
  -

Open Questions:

1. Combination semantics: How to merge inputs?

   • Simple concatenation: file1 + "\n\n" + file2 + "\n\n" + stdin
   • Structured labels: "Context 1:\n" + file1 + "\n\nQuery:\n" + stdin
   • Chat-template aware: Map to message array (see below)

2. Order: --in files first, then stdin? Configurable?

3. Separator control: --separator "\n---\n" flag?

Relationship to Server API:

// Server uses message arrays:
POST /v1/chat/completions
{
  "messages": [
    {"role": "system", "content": "<file1>"},
    {"role": "user", "content": "<stdin>"}
  ]
}

CLI could map to same structure:

mlx-run model --system system.txt --in context.txt - "query"
# → Internally builds message array, applies chat template

Related Issues/Features:

• --system flag (cli.py:131, already stubbed for future)
• ADR-010 (Reasoning Content API) - message-based
• Issue #39 (OpenAI Function Calling) - message-based
• Multi-turn conversations (run.py:132 interactive_chat)

All use Chat Templates + Message Arrays! This suggests a unified approach.

---

Extension 2: Pipeline Debugging (--out)

Use Case: Capture intermediate stages without breaking pipe:

cat input.txt \
  | mlx-run stage1 --out log/stage1.txt - \
  | mlx-run stage2 --out log/stage2.txt - \
  | mlx-run stage3 -

Semantics: Write to file AND stdout (tee-like behavior).

Alternative: Use Unix tee:

cat input.txt | mlx-run stage1 - | tee log/stage1.txt | mlx-run stage2

Trade-off: --out is more compact, but tee already exists and works. Low priority.

---

Design Constraints for Phase 1

To preserve future compatibility with these extensions, Phase 1 MUST:

1. Reserve argument syntax:

   • --in <file> - Reserved for multi-source input
   • --out <file> - Reserved for output capture
   • --system <text|file> - Already reserved (cli.py:131)

2. Semantic foundation:

   • - always means stdin (never conflicts with --in)
   • Additional text after - always allowed: mlx-run model - "instruction"
   • This enables future: mlx-run model --in file - "instruction"

3. Chat template compatibility:

   • Internal prompt construction must support future message array mapping
   • --no-chat-template flag preserves raw mode for testing

4. Output modes remain simple:

   • stdout = main output channel
   • stderr = errors only
   • --out (future) adds file, doesn't replace stdout

Phase 1 Minimal Implementation

What Phase 1 WILL implement:

# Basic stdin:
cat prompt.txt | mlx-run model -

# stdin + additional:
cat context.txt | mlx-run model - "Answer this question"

# TTY mode unchanged:
mlx-run model "prompt"

# Output adapts automatically:
mlx-run model "test"          # TTY: pretty
mlx-run model "test" | cat    # Pipe: clean text

What Phase 1 will NOT implement:

• --in (multi-source)
• --out (tee-like)
• --system (stubbed but not functional)
• Chat-template aware input structuring

Phase 1 is intentionally minimal to validate core pipe semantics before adding complexity.

---

Connection to Server Architecture

The server already implements message-array semantics via Chat Templates:

# mlxk2/core/server_base.py (existing)
def _format_conversation(messages: List[dict]) -> str:
    """Apply chat template to message array."""
    return tokenizer.apply_chat_template(messages, ...)

Future CLI-Server symmetry:

Feature	Server API	CLI (Future)	Shared Implementation
System prompts	messages[0].role = "system"	--system file.txt	_format_conversation()
Multi-turn	messages[] array	--in file1 --in file2	Chat template engine
Context injection	messages[].content	--in context.txt	Token management
Reasoning content	reasoning_content field	--show-reasoning	ADR-010 parser

Benefit: CLI becomes a thin wrapper around server logic, ensuring consistency.

---

Recommendation

Phase 1 (2.0.3-beta.x):

• Implement: Basic stdin (-), stdout adaptation (isatty())
• Reserve: --in, --out, --system syntax
• Document: Minimal viable pipe semantics

Phase 2+ (Post-2.0.3):

• Evaluate: Real-world usage patterns from Phase 1
• Design: Multi-source (--in) semantics based on feedback
• Implement: Chat-template aware input mapping (CLI-Server symmetry)
• Consider: --out if tee proves insufficient

This appendix serves as design documentation, not implementation commitment.