mirror of
https://github.com/Heretek-AI/awesome-harness.git
synced 2026-10-12 02:03:18 +00:00
Awesome Open Source AI Agent Harnesses
A dynamically curated list of open source AI agent testbeds, evaluation harnesses, sandboxed runtimes, and benchmarking frameworks, automatically updated and grouped by GitHub topics.
What is an AI Agent Harness?
An AI Agent Harness is the runtime environment, benchmarking framework, evaluation testbed, or execution sandbox that provides:
- Task Environments & Datasets: Realistic scenarios (codebases, browser interfaces, terminal shells, APIs).
- Execution Runtimes: Isolated sandboxes (Docker, WebContainers, microVMs) to safely run agent actions.
- Observation & Tooling Interfaces: Standardized protocols (e.g., Model Context Protocol / MCP, tool-calling APIs).
- Scoring & Verification: Deterministic or LLM-judged metrics to validate agent completion and test against regressions.
This repository dynamically indexes, categorizes, and tracks open source harnesses across the GitHub ecosystem using GitHub Topics.
Contents
- Categories
- 💻 Software Engineering & Coding Harnesses (46)
- 📊 Evaluation, Leaderboards & General Benchmarks (74)
- 🌐 Web, Browser & OS Agent Environments (39)
- 🐝 Multi-Agent & Swarm Harnesses (18)
- 🛡️ Safety, Security & Red-Teaming Harnesses (2)
- 📦 Execution Sandboxes, Runtimes & Tooling (49)
- 🚀 Emerging & General Agent Harnesses (95)
- Topic Cloud
- How It Works
- Contributing
Categories
💻 Software Engineering & Coding Harnesses
Harnesses and testbeds for evaluating and executing autonomous coding agents against real-world repositories, issues, and pull requests.
| Repository | Stars | License | Language | Description | Key Topics |
|---|---|---|---|---|---|
| openinterpreter | ★ 68.5k |
Apache-2.0 |
Rust |
A coding agent for open models like Kimi K3 and GLM 5.3 | acp coding-agent deepseek kimi |
| jcode | ★ 20.3k |
MIT |
Rust |
High performance coding agent harness written in rust | ai-agent ai-coding-agent claude cli |
| PI-Desktop | ★ 6.5k |
LGPL-3.0 |
TypeScript |
Local-first AI coding agent desktop: Electron + Rust host core + pi Agent Harness + user-installable plugins | ai-agent coding-agent desktop-app electron |
| SWE-bench | ★ 6.0k |
MIT |
Python |
SWE-bench: Can Language Models Resolve Real-world Github Issues? | benchmark language-model software-engineering |
| harnessrouter | ★ 2.9k |
Apache-2.0 |
Python |
HarnessRouter Community Edition: the self-hosted, Apache-2.0 edition of the unified interface for agent harnesses. Run Codex, Claude Code, Hermes, PI, DSH, a... | agent-api agent-framework agent-harness ai-agents |
| openlit | ★ 2.8k |
Apache-2.0 |
TypeScript |
OpenLIT is the open-source agent harness engineering platform: trace, evaluate, guard, and improve everything around the model in your AI agents, on OTEL. | agent-harness agent-harness-engineering ai-engineering ai-observability |
| Tianshu-harness | ★ 1.1k |
Apache-2.0 |
TypeScript |
天枢harness是一个的终端编程智能体，它要回答的核心问题是：模型何以稳定地交付——目标不漂移、完成有证据、验证有闭环，不说"应该修好了"。为此它在模型与真实世界之间建立一层认知执行环境（CVM），把目标、状态、证据、资源、权限与终止条件从对话历史中外部化，由运行时持续管理。同时对deepseek v4极度优化。 | agent-harness ai-agent cli coding-agent |
| dscode | ★ 1.0k |
MIT |
JavaScript |
A DeepSeek coding agent harness: persistent shell, Ultra subagents, auto approval, Chrome MCP and session telemetry | agentic ai-agent cli code-review |
| Repo2RLEnv | ★ 706 |
Apache-2.0 |
Python |
Turn any repository into verifiable RL environments for coding agents - Harbor tasks you can train on, evaluate and share on the Hugging Face Hub | coding-agents datasets harbor huggingface |
| mu | ★ 430 |
MIT |
TypeScript |
mu (μ): a coding agent that thinks before it acts. A small, fast judge makes the routine calls, the big model does the work. Built on pi and AionUi. | agent-harness ai-agent ai-coding aionui |
| harness-remote | ★ 403 |
Apache-2.0 |
JavaScript |
Native-session control plane for Codex, Claude Code, OpenCode, OMP and PI. Run, resume and hand off coding sessions across your machines. | acp agent-harness agent-orchestration ai-agents |
| no_human | ★ 330 |
MIT |
Python |
From ticket to reviewed pull request. Free and open-source, on your machine. | agent-harness agent-orchestration agentic-ai ai-agent |
| CodeAF | ★ 304 |
Apache-2.0 |
Go |
Open-Source Software factory for Open Models | agent-fleet agent-harness agentfield agentic-coding |
| pi-bluebook | ★ 301 |
MIT |
Markdown |
Pi Coding Agent 中文学习蓝皮书：从安装与第一个可验收任务开始，逐步掌握 Session、Context、Skill、Extension、Subagent 与长期 Agent 工作流。 | agent-harness agent-skills ai-agent chinese |
| SE-Agent | ★ 289 |
MIT |
Python |
SE-Agent is a self-evolution framework for LLM Code agents. It enables trajectory-level evolution to exchange information across reasoning paths via Revision... | claude-code code-agent code-fix mcts |
| learn-agent | ★ 282 |
MIT |
JavaScript |
学习Agent开发的笔记，尝试让Agent从可用到可靠 | agent-harness agent-loop ai-agent aider |
| RepoLaunch | ★ 191 |
MIT |
Python |
[NeurIPS 2026 Main] Automate the build, execution and test of software repositories across programming languages and operating systems. | agentic-rl build-tool coding-agent swe-bench |
| crux | ★ 187 |
MIT |
TypeScript |
Taking the pi coding agent to Claude Code-level performance: 0.539 → 0.773 pass@1 on Terminal-Bench 2.1 with the same self-hosted 27B model, by re-engineerin... | agent-evaluation ai-agent coding-agent context-engineering |
| DM-Code-Agent | ★ 153 |
MIT |
Python |
Local-first, auditable Python code agent. Ships its own 30-task hidden-test benchmark plus SWE-bench Verified scored by the official harness -- every number ... | agent-evaluation agent-skills code-agent llm-agent |
| coder_eval | ★ 149 |
Apache-2.0 |
Python |
Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI... | agent-evaluation agent-skills agent-testing anthropic |
| minimal | ★ 143 |
Apache-2.0 |
Rust |
Run your agent in a Minimal box. Ship and run software with isolation on your own computer. | agent-isolation agent-sandbox ai-agents ai-security |
| Flowness | ★ 106 |
Apache-2.0 |
Python |
A work-centered runtime for agentic software engineering. Work persists; agents, context, and graphs assemble around it. | agent-evaluation agent-governance agent-orchestration agent-runtime |
| godmode | ★ 96 |
MIT |
Python |
Production-grade Agent Skills for AI coding agents—composable workflows for planning, TDD, debugging, review, UI/UX, releases, incidents, and evals. | agent-evaluation agent-skills ai-agents ai-coding |
| benchmark-harnesses | ★ 64 |
Apache-2.0 |
Python |
Strands-based agents and harnesses for agentic benchmarks. | agentic agentic-ai benchmarks genai |
| retro-harness | ★ 58 |
MIT |
Python |
RHO: Evolving Agents in the Dark — Retrospective Harness Optimization via Self-Preference. Improving LLM agents from unlabeled past trajectories (arXiv:2606.... | agent-optimization llm-agents prompt-optimization research |
| agentic-loop-engineering-course | ★ 58 |
MIT |
Jupyter Notebook |
An 18 notebook course that isolates and measures each component of agentic loop engineering on real, industry standard software datasets. | agent-orchestration agentic-ai ai-agents code-generation |
| swe-sweep | ★ 57 |
MIT |
Python |
How many bugs can LMs find & fix in large codebases? | ai-agents benchmark benchmarking harbor |
| kotlin-swe-bench | ★ 54 |
Apache-2.0 |
Python |
The official Kotlin benchmark by JetBrains | agent-evaluation ai-agents kotlin swe-bench |
| clawker | ★ 54 |
MIT |
Go |
AI coding agent sandbox platform | agent-sandbox agentic-ai agentic-coding ai-agent |
| Xenon | ★ 53 |
MIT |
Python |
🚀 Xenon 0.9.2 — 可验证的 AI coding agent runtime：单一任务状态机、证据驱动执行、7 种推理范式、MCP、多模型与 Windows/GBK 兼容。 | agent-framework agent-harness ai-coding-assistant cli |
| insights | ★ 51 |
MIT |
TypeScript |
We track and analyze the activity and performance of autonomous code agents in the wild | swe-agent swe-bench |
| why-was-fable-banned | ★ 46 |
MIT |
Python |
Fable-style spec + evidence gate for Claude Code + Codex. Makes Opus/Codex work under Fable-like discipline: blocks every edit until a deterministic spec pas... | agent-harness ai-agent-guardrails ai-coding claude-code |
| NanoHarness | ★ 40 |
MIT |
Python |
Compact AI agent runtime control plane with governed tools, HITL approval, resumable execution, trace-driven evaluation, and SWE-bench-shaped evidence. | agent-evaluation agent-harness coding-agent llm-agent |
| agro | ★ 40 |
Apache-2.0 |
TypeScript |
🏗️ AGRO — Agent Governance Runtime Orchestrator. A portable home for autonomous coding agents. | agent-sandbox ai-agents ai-coding claude |
| AgentPatchCheck | ★ 38 |
Apache-2.0 |
TypeScript |
Coding Agent Runtime & Evaluation Harness for controlled repository-level software repair | agent-runtime benchmarking coding-agent evaluation-harness |
| simple-long-horizon-agent | ★ 34 |
Apache-2.0 |
Python |
Simple Long Horizon Agent - A simple yet effective AI agent for learning, experimentation, and long horizon work. | agent-harness harness harness-engineering long-horizon |
| arbos | ★ 30 |
MIT |
Rust |
Arbos: an open-source, file-system-native agent coordinator with a Cursor-style desktop, full-duplex voice, iPhone app, and a mesh of workers | agent-coordinator ai-agents coding-agent cursor |
| mini-ork | ★ 29 |
Apache-2.0 |
Python |
Prove an AI-written fix actually fixes the bug. mini-ork certify reproduces the bug against your repo's own code, attacks the patch with generated invariants... | agent-framework agentic-ai ai-agents ai-code-review |
| repoagentbench | ★ 28 |
MIT |
Python |
SWE-bench for your codebase — mine your merged PRs into local, contamination-free coding-agent benchmarks. Adapters: claude-code, aider (Opus 4.7 / GPT-5.5 /... | agent-evals ai-agents aider benchmark |
| squeez | ★ 24 |
Apache-2.0 |
Python |
Squeeze verbose LLM agent tool output down to only the relevant lines | coding-agent context-compression lora pytorch |
| coding-agent-kernel | ★ 23 |
Apache-2.0 |
Python |
Independent Python coding-agent kernel with observable runs, tools, sessions, permissions, DeepSeek, and SWE-bench. | agent-kernel coding-agent deepseek llm-agents |
| ai-workflow-benchmark | ★ 14 |
MIT |
Python |
Benchmark harness measuring AI coding tool+workflow performance, not just model capability. 100 tasks, sigmoid scoring, 12 capability dimensions, gap analysis. | ai-agents ai-coding benchmark claude-code |
| mcpbr | ★ 11 |
MIT |
Python |
Benchmark your MCP server. | benchmarking llm-evaluation mcp ml-evaluation |
| agents | ★ 11 |
Apache-2.0 |
Python |
Minimal coding agents, easy to play with (opencode TUI) and easy to benchmark (ACP on BenchFlow) — built around mini-swe-agent | acp ai-agents coding-agent llm-evaluation |
| Agent-Sandbox | ★ 11 |
Apache-2.0 |
TypeScript |
Fast, Multi-Cloud Sandbox Engine for AI Agents | agent-sandbox agentic-rl e2b e2b-compatible |
| harness-bench | ★ 10 |
MIT |
TypeScript |
Benchmark leaderboard for AI coding harnesses by model. | ai-agents benchmark coding-agent leaderboard |
📊 Evaluation, Leaderboards & General Benchmarks
Harnesses designed to evaluate LLMs and agent capabilities across diverse benchmarks, tasks, and standardized leaderboards.
| Repository | Stars | License | Language | Description | Key Topics |
|---|---|---|---|---|---|
| iFixAi | ★ 22.0k |
Apache-2.0 |
Python |
Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is ... | agent-evaluation ai-alignment ai-evaluation ai-governance |
| giskard-oss | ★ 5.9k |
Apache-2.0 |
Python |
🐢 Open-Source Evaluation & Testing library for LLM Agents | agent-evaluation ai-red-team ai-security ai-testing |
| coze-loop | ★ 5.8k |
Apache-2.0 |
Go |
Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities fro... | agent-evaluation agent-observability agentops coze |
| trulens | ★ 3.6k |
MIT |
Python |
Evaluation and Tracking for LLM Experiments and AI Agents | agent-evaluation agentops ai-agents ai-monitoring |
| future-agi | ★ 2.1k |
Apache-2.0 |
Python |
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway ·... | agent-evaluation ai-agents ai-evals ai-gateway |
| any-agent | ★ 1.2k |
Apache-2.0 |
Python |
A single interface to use and evaluate different agent frameworks | a2a agent-evaluation mcp |
| TheAgentCompany | ★ 792 |
MIT |
Python |
An agent benchmark with tasks in a simulated software company. | ai-benchmark ai-research benchmark |
| pandaprobe | ★ 785 |
Apache-2.0 |
Python |
open source agent engineering platform: traces, evals, and metrics to debug and improve your AI agents. Integrates with LangGraph, CrewAI, Claude Agent SDK, ... | agent-engineering agent-evaluation agent-observability agentic-ai |
| SkillEvaluator | ★ 545 |
Apache-2.0 |
Python |
Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent e... | agent-evaluation agent-security agent-skills agentic-ai |
| AdaRubrics | ★ 364 |
Apache-2.0 |
Python |
AdaRubric: Adaptive Dynamic Rubric Evaluator for Agent Trajectories | agent-evaluation llm-evaluation reward-model rlhf |
| benchmark-radar | ★ 283 |
MIT |
Python |
Track 20,710+ AI benchmark, eval, dataset, and data-quality records from 39 public sources, with linked evidence and daily updates. | agent-bench agent-benchmark agent-benchmarking agent-benchmarks |
| agent-leaderboard | ★ 226 |
MIT |
Jupyter Notebook |
Ranking LLMs on agentic tasks | agent-evaluation ai-agents ai-benchmark ai-evaluation |
| AgentMeasure | ★ 218 |
MIT |
Python |
Independent verification of AI support bills: applicable terms, invoice and business records, with reviewable findings. Open-source conformance for agent tel... | agent-economy agent-evaluation agent-observability agentic-ai |
| ai-agents-tutorial | ★ 187 |
Apache-2.0 |
Multi |
Learn AI Agents step by step, from scratch - from function calling to agent loops to multi-agent systems, orchestration, and evaluation. | agent-evaluation agent-loop agent-orchestration ai-agent |
| Awesome-AI4AI | ★ 178 |
MIT |
Python |
AI4AI Survey: can AI reliably improve AI? 223 papers on long-horizon agents, benchmarks, harness design, and recursive self-improvement · updated weekly | agent-benchmarks agent-harness agi ai-agents |
| mirobody-eval | ★ 174 |
MIT |
Python |
An evaluation harness for medical/health AI agents — reproduce and cover multiple benchmarks under one scoring discipline | agent-evals benchmark-reproducibility evaluation-framework evaluation-harness |
| LLM-Agent-Benchmark-List | ★ 171 |
Apache-2.0 |
Multi |
A banchmark list for evaluation of large language models. | benchmark survey |
| agentenv-framework | ★ 151 |
Apache-2.0 |
Python |
Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments ... | a2a agent-evaluation agentic-ai ai-agents |
| eval-view | ★ 137 |
Apache-2.0 |
Python |
Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic. | agent-benchmark agent-evaluation agentic-ai ai-agents |
| model-serving-minefield | ★ 135 |
MIT |
Python |
Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA... | benchmarking chat-template cuda debugging |
| every_eval_ever | ★ 133 |
MIT |
Python |
Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderb... | agent-evaluation ai-evaluation evaluations infra |
| hermes-skilleval | ★ 125 |
MIT |
Python |
Verification-gated skill routing and self-improvement harness for Hermes-style agent skills | agent-evaluation benchmark ci llm-agents |
| AgentKernelArena | ★ 120 |
Apache-2.0 |
Python |
AgentKernelArena provides an end-to-end siloed-benchmarking environment where different LLM-powered agents—such as Cursor Agent, Claude Code, Codex, SWE-agen... | agent-evaluation gpu-kernels llamas |
| claimpilot-harness | ★ 117 |
MIT |
Python |
Crash-test insurance claim AI agents before production. | agent-evaluation ai-agents insurance llm-evals |
| jev-curate | ★ 104 |
MIT |
Rust |
High-throughput synthetic and pretraining dataset sifter for TypeSafe Jev. Rust streaming core, Parquet and JSONL I/O, typed Choice/Score/Noul judgments, spe... | arrow cli data-cleaning data-engineering |
| heurigym | ★ 97 |
Apache-2.0 |
Python |
Agentic Benchmark for LLM-Crafted Heuristics in Combinatorial Optimization (ICLR'26) | benchmark chatgpt optimization |
| checkup | ★ 95 |
AGPL-3.0 |
Shell |
AgentVitals Checkup (/checkup) — an AI agent skill that gives your agent a professional health checkup: dual-axis Stability + Welfare scoring, a personality-... | agent-benchmark agent-health agent-skill agentvitals |
| forsy-trace-skill | ★ 95 |
MIT |
Python |
Open skill for capturing AI agent work as structured traces. | agent-evaluation agent-traces agent-workflows ai-agents |
| awesome-llm-agent-papers | ★ 87 |
MIT |
Python |
A curated, continuously updated reading list of 500+ papers on LLM agents: planning, memory, tool use, multi-agent, evaluation & safety. Companion to the sur... | agent-benchmark agent-survey ai-agents ai-safety |
| AgentCheck | ★ 80 |
Apache-2.0 |
Python |
Behavioral testing for AI agents. | agent-evaluation ai-agents ai-safety developer-tools |
| ai-agents-reality-check | ★ 61 |
Apache-2.0 |
Python |
Benchmarking the gap between AI agent hype and architecture. Three agent archetypes, 73-point performance spread, stress testing, network resilience, and ens... | agent-architecture agent-benchmark agent-evaluation agent-performance |
| HarnessBench | ★ 59 |
Apache-2.0 |
Python |
Benchmark for comparing agent harnesses on everyday online tasks — fixes the base model, varies the harness. Sister project of ClawBench, same scoring pipeline. | agent-evaluation agent-harness agentic-ai ai-agents |
| dokimos | ★ 59 |
MIT |
Java |
LLM and agent evaluation for Java & Kotlin. Runs in JUnit and CI. Spring AI, LangChain4j, Koog, Embabel, and any LLM client. | agent-evaluation agentic-ai embabel evaluation |
| on-panda | ★ 56 |
MIT |
JavaScript |
🐼 Steer your LLMs and agents at the token level: A web app for token visualization & control, model inspection, data annotation, and more. | agent-evaluation agent-trajectory data-annotation llm-ui |
| cloudbox | ★ 55 |
MIT |
TypeScript |
Durable Cloudflare computers for agents: run repos, steer live workspaces, verify results, and return receipts. | agent-computer agent-evaluation ai-agents cloudflare |
| laya-jev-GraphRAG | ★ 50 |
Apache-2.0 |
Python |
A database-agnostic Agentic GraphRAG framework using swappable System One models (local Laya / cloud Jev). A plug-and-play intelligence layer featuring a com... | astar-algorithm eval-harness fine-tuning graphdatabase |
| tianji-ai-agent | ★ 48 |
MIT |
Java |
Production-oriented multi-agent course advisor with deterministic action governance, offline regression evaluation, SSE contracts and observable tool execution. | agent-evaluation java multi-agent rag |
| benchjack | ★ 48 |
Apache-2.0 |
Python |
AI agent benchmark hackability scanner — find evaluation vulnerabilities before they undermine your results | ai-agents ai-security benchmark evaluation |
| ai-agent-eval-harness | ★ 44 |
Apache-2.0 |
Python |
The open-source MultiAgentOps evaluation and verification harness for any industry business workflow. | agent-assurance agent-observability agent-testing agent-verification |
| A2E | ★ 42 |
MIT |
Python |
No description provided. | agent-audit agent-evaluation agent-observability agentic-ai |
| isnad | ★ 42 |
Apache-2.0 |
Python |
Grade every agent, scraper and model in a claim's chain — provenance, trust scoring and audit evidence for LLM pipelines | agent-evaluation agent-observability ai-agents ai-governance |
| agentune | ★ 40 |
Apache-2.0 |
Python |
Tune your AI Agent to best meet its KPI with a cyclic process of analyze, improve and simulate | agent-evaluation agent-optimization agent-simulator ai-agents |
| DataSpace | ★ 33 |
MIT |
Python |
Benchmarking data agents for verifiable analytics over heterogeneous workspaces. | agent-evaluation ai-agents benchmark data-agents |
| money-agent | ★ 33 |
MIT |
Python |
Overnight experiment: can an AI agent make money, and can you trust what it reports? The agent produces claims; a separate verifier it cannot invoke produces... | agent-evaluation agentic-ai ai-agent ai-agents |
| self-bench | ★ 30 |
MIT |
TypeScript |
Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests. | agent-evaluation ai-agents benchmark coding-agents |
| self-care | ★ 28 |
MIT |
JavaScript |
Agent trace analysis and context remediation plugin for Claude Code. Detects quality issues in your AI agent traces — goal drift, hallucinations, missed acti... | agent-evaluation ai-agents claude-code observability |
| TwinRouterBench | ★ 27 |
Apache-2.0 |
Python |
Per-step LLM routing benchmark with 970 static labels, live SWE-bench evaluation, an open data pipeline, and a public leaderboard. | agentic-ai benchmark cost-optimization leaderboard |
| jeju | ★ 27 |
MIT |
Go |
Declarative, local-first runtime for bounded AI agents — define an agent in one manifest, run it headless, and audit every effect. | agent-evaluation agent-harness ai-agents evolution |
| agentshield-benchmark | ★ 27 |
Apache-2.0 |
TypeScript |
Open benchmark for AI agent security tools — prompt injection, data exfiltration, tool abuse, provenance | agent-security ai-security benchmark guardrails |
| Agentic_Evals | ★ 26 |
MIT |
Jupyter Notebook |
Practical evaluation techniques, benchmarks, and examples for production AI agents: RAG, tool use, planning, memory, safety, reliability, and cost. | agent-evaluation agentic-ai ai-agents benchmark |
| foundry-cicd | ★ 25 |
MIT |
Multi |
Enterprise-ready CI/CD reference for Microsoft Foundry AI agents, with parallel GitHub Actions and Azure DevOps pipelines, evaluation-driven quality gates, a... | agent-evaluation ai-agents azure azure-devops |
| langeval | ★ 24 |
Apache-2.0 |
TypeScript |
Evaluation Infrastructure for AI Agents | agent-evaluation ai-evals ai-evaluation |
| calibrate | ★ 23 |
CC-BY-SA-4.0 |
JavaScript |
Core engine behind Calibrate, a framework for evaluating AI agents: speech-to-text, text-to-speech, LLM evaluation, end-to-end simulations | agent-evaluation agent-evaluation-tools ai-agent-evaluation ai-agents |
| oh-my-knowledge | ★ 23 |
MIT |
TypeScript |
OMK — Evidence-backed evaluation and observability for prompts, RAG, skills, agents, and workflows. Native Codex, Claude Code, and DeepSeek Harness support. | agent-evaluation benchmark bootstrap-ci claude |
| amber | ★ 22 |
Apache-2.0 |
Python |
AMBER (sealed-in-amber historical replay evaluation): real-event replay, physical sealing, pre-registered scoring — method spec; cases private, results public | agent-evaluation ai-agents benchmark benchmarking |
| ctf-agent-benchmark | ★ 22 |
GPL-3.0 |
Python |
Benchmarking platform for evaluating AI agents on CTF-style tasks and tool-use workflows. | ai-agent benchmark ctf evaluation |
| basically-agent-evals | ★ 20 |
MIT |
TypeScript |
Your Evals Are Lying To You: a hands-on TypeScript curriculum for LLM evals, from a five-line assertion to a live diagnostic console. By Tejas Kumar. | ai-agents eval-harness evals llm-evaluation |
| ATLAS | ★ 20 |
Apache-2.0 |
Python |
No description provided. | agent-evaluation agent-observability claude-code codex |
| agentrial | ★ 20 |
MIT |
Python |
Statistical evaluation framework for AI agents | agent-evaluation ai-agents ai-testing ci-cd |
| guard-eval-harness | ★ 19 |
MIT |
Python |
One command to benchmark AI guardrails and coding agents across safety, security, jailbreak, prompt-injection, and secure-code tasks. | ai-safety benchmark cli eval-harness |
| browseruse-agent-bench | ★ 19 |
Apache-2.0 |
Python |
Real-world browser-agent benchmark: 210 tasks across 107 websites, multi-agent/multi-browser evaluation, reproducible leaderboard and result submissions. | agent-evaluation ai-agents benchmark browser-agent |
| ratchet | ★ 18 |
MIT |
TypeScript |
BYOA Agent orchestration that ensures alignment end-to-end – with built-in eval system. | agent-orchestration agent-workflow ai-sdlc bdd |
| tau-trait | ★ 18 |
MIT |
Python |
TraitBasis applied to TauBench | agent-benchmark rl-envs rl-training |
| tiny_qa_benchmark_pp | ★ 18 |
Apache-2.0 |
Python |
Tiny QA Benchmark++ a micro-benchmark suite (52-item gold + on-demand multilingual synthetic packs), generator CLI, and CI-ready eval harness for ultra-fast ... | benchmark dataset evaluation hacktoberfest |
| BABY | ★ 17 |
MIT |
TypeScript |
BABY - Build Agent Benchmarks for Yourself | agent-benchmark agent-workflows |
| swe-serve | ★ 17 |
Apache-2.0 |
Python |
SWE-Serve: an agentic benchmark of 53 production inference-engineering tasks derived from merged SGLang pull requests, run with Harbor. | benchmark evaluation llm-agents llm-inference |
| eval-genius | ★ 16 |
Apache-2.0 |
Python |
Teach your agent to work with evals: WHEN you actually need an eval or benchmark, HOW to build one that holds up, and how to read what it tells you. Determin... | agent-skill ai-agents benchmark-framework benchmarks |
| baba-is-harbor | ★ 15 |
Apache-2.0 |
Python |
Baba Is Harbor - AI agent benchmark on Baba is You | ai-agents baba-is-you benchmark game |
| tycoon-learning-environment | ★ 15 |
MIT |
Python |
A JAX transport-economy learning environment for route planning, cargo flow, financing, and replayable agent benchmarks. | benchmark economy jax logistics |
| nasde-toolkit | ★ 14 |
MIT |
Python |
CLI for benchmarks & evals of AI coding agents — on tasks you already understand, using your Claude / Codex / Gemini individual subscriptions or API keys. | agent-benchmark agent-evaluation ai-coding-agents ai-evaluation |
| memory_and_context_eval_harness | ★ 11 |
MIT |
Python |
Open evaluation harness for AI agent memory systems. Runs LoCoMo and LongMemEval against Synap, Mem0, Zep and Supermemory with pluggable provider adapters. | agent-memory ai-agents benchmark evaluation |
| aobench | ★ 10 |
Apache-2.0 |
Python |
Role-aware, permission-enforced benchmark for LLM agents that operate HPC systems — SLURM, telemetry, RBAC, docs, facility. A policy violation hard-fails the... | agent-benchmark agent-evaluation ai-agents ai-safety |
| dojo.md | ★ 10 |
MIT |
TypeScript |
University for AI agents. 92 courses, 4400+ scenarios, any model via OpenRouter. Auto-training loops generate per-model SKILL.md documents. Works with Claude... | agent-benchmark agent-skills ai-agents ai-testing |
| halu-core | ★ 10 |
MIT |
Python |
Scores whether an autonomous AI agent actually did the work or just hallucinated its report, by checking every claim against recorded API calls. The core eng... | agent-benchmark agent-evaluation ai-agents ai-evaluation |
🌐 Web, Browser & OS Agent Environments
Sandbox environments, browser harnesses, and multi-modal testbeds for web navigation, DOM interaction, and operating system automation.
| Repository | Stars | License | Language | Description | Key Topics |
|---|---|---|---|---|---|
| browser-use | ★ 117.4k |
MIT |
Python |
Agents that use the browser. | ai-agents ai-tools browser-automation browser-use |
| UI-TARS-desktop | ★ 39.2k |
Apache-2.0 |
TypeScript |
The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra | agent-tars browser-use computer-use cowork |
| OpenCLI | ★ 29.9k |
Apache-2.0 |
JavaScript |
Make Any Website into CLI & Use your logged-in browser by AI agent. | ai-agent ai-agents ai-tools browser-automation |
| browser-harness | ★ 18.3k |
MIT |
Python |
Browser Harness | Self-healing harness that enables LLMs to complete any task. | ai-agent browser-agent browser-automation browser-use |
| midscene | ★ 15.1k |
MIT |
TypeScript |
GUI Agent for E2E Testing | browser-use computer-use e2e-testing gpt-operator |
| nanobrowser | ★ 14.0k |
Apache-2.0 |
TypeScript |
Open-Source Chrome extension for AI-powered web automation. Run multi-agent workflows using your own LLM API key. Alternative to OpenAI Operator. | ai-agents ai-tools automation browser |
| magentic-ui | ★ 10.1k |
MIT |
Python |
MagenticLite is an experimental agent that works across the browser and local file system | ai-ux autogen browser-use computer-use-agent |
| BrowserSkill | ★ 8.3k |
MIT |
TypeScript |
Let AI agents use your real, logged-in browser without interrupting your work. CLI + extension for browser automation across any shell-capable AI agent. | browser-use dsh-plugin |
| fara | ★ 6.2k |
MIT |
Python |
Fara1.5 – A family of frontier computer use agent models | browser-use computer-use computer-use-agent cua |
| mobile-use | ★ 3.2k |
Apache-2.0 |
Python |
AI agents can now use real Android and iOS apps, just like a human. | browser-use langchain langgraph langgraph-python |
| TuriX-CUA | ★ 3.2k |
MIT |
Python |
This is the official website for TuriX Computer-use-Agent | ai-agents browser-use computer-automation computer-use |
| webcmd | ★ 2.6k |
Apache-2.0 |
TypeScript |
Self-learning agent browser | ai-agents browser-automation browser-use cli |
| AIPex | ★ 1.3k |
MIT |
TypeScript |
AIPex: AI browser automation assistant, no migration and privacy first. Alternative to Manus Browser Operator、 Claude Chrome and Agent Browser | accessbility-testing agent-browser agent-skills browser-automation |
| browserable | ★ 1.2k |
MIT |
JavaScript |
Open source and self-hostable browser automation library for AI agents | ai-agents ai-tools browser-automation browser-use |
| celesto | ★ 1.0k |
Apache-2.0 |
Python |
Secure and persistent computer for AI agents | agent-runtime ai-sandbox browser-agent browser-use |
| ClawBench | ★ 967 |
Apache-2.0 |
Python |
Open-source benchmark for browser AI agents on daily tasks. | agent-evaluation agentic-ai ai-agent-benchmark ai-agents |
| unbrowse | ★ 777 |
MIT |
TypeScript |
Unbrowse - Reverse engineer any website so that your agents can access its APIs directly | agentskills browser-use claude-code-plugin clawdbot |
| desktop | ★ 699 |
MIT |
TypeScript |
Browser Use Desktop App | agent-browser browser browser-agent browser-agents |
| terminal | ★ 651 |
MIT |
Rust |
Terminal UI to get stuff done in the browser | browser-automation browser-use cdp terminal |
| web-agent-protocol | ★ 506 |
MIT |
Python |
🌐Web Agent Protocol (WAP) - Record and replay user interactions in the browser with MCP support | ai-agents ai-tools browser-automation browser-use |
| bux | ★ 453 |
MIT |
Python |
Browser Use Box: a 24/7 Claude Code agent for Playwright-style browser automation with Browser Use Cloud, Telegram, and a real browser on any box you own. | ai-agent ai-automation automation browser-agent |
| opencow | ★ 394 |
Apache-2.0 |
TypeScript |
One task, one agent, delivered. The open-source platform for task-driven autonomous AI agents.OpenCow assigns an autonomous AI agent to every task — features... | ai-agent ai-first autonomous-agent browser-use |
| open-browser-use | ★ 359 |
MIT |
JavaScript |
🔮 Platform-neutral Browser Use for AI agents: real Chrome automation with a CLI + SDKs, no lock-in, dead simple. | ai-agent ai-agents browser-automation browser-use |
| RLVR-World | ★ 280 |
MIT |
Python |
Official repository for "RLVR-World: Training World Models with Reinforcement Learning" (NeurIPS 2025), https://arxiv.org/abs/2505.13934 | grpo real2sim reinforcement-learning-with-verifiable-rewards rlvr |
| Meterless | ★ 250 |
Apache-2.0 |
TypeScript |
Local-first context layer for AI agents. Open-source memory, reasoning, world state, intent, and swarm engines. | agent-memory agentic-ai ai-agents browser-use |
| socai | ★ 236 |
Apache-2.0 |
Rust |
Agent that actually understands social platforms. Fast. Precise. Deep. | browser-agent browser-use claude codex |
| spidercreator | ★ 227 |
AGPL-3.0 |
Python |
Automated web scraping spider generation using Browser Use and LLMs. Streamline the creation of Playwright-based spiders with minimal manual coding. Ideal fo... | automation browser-use crawling low-code |
| os-ai-computer-use | ★ 182 |
Apache-2.0 |
Python |
AI controls your OS. OS AI Computer Use, OS and API agnostic. For now on OpenAI and Anthropic API. Desktop app ready. | ai-agents-framework ai-os anthropic browser-use |
| BrowserPilot | ★ 179 |
MIT |
TypeScript |
Open‑source alternative to Perplexity Comet, director.ai and firecrawl combined | ai-agent ai-agents browser browser-agent |
| bubus | ★ 125 |
MIT |
Python |
📢 Production-ready python event bus library with support for async and sync handlers, forwarding betwen busses w/ parent event tracking + loop prevention, F... | async asyncio browser-use concurrency |
| jev-use | ★ 119 |
MIT |
Swift |
Say it, and your Mac does it. A computer-use harness on Jev that reads the screen through Accessibility. Fast, no vision model | accessibility ai-agents browser-use computer-use |
| dsh-tabbit | ★ 102 |
MIT |
TypeScript |
Tabbit Browser plugins for Deepseek Harness | awesome-dsh-plugin browser-automation browser-use deepseek-harness |
| SwiftAutoGUI | ★ 94 |
MIT |
Swift |
A Swift library for macOS automation — mouse, keyboard, screenshots, image recognition, and AI-powered agents. | ai-agents automation browser-use browser-use-agent |
| clawbrowser | ★ 87 |
MIT |
C++ |
Agent-native browser with fingerprint control and proxy support. | ai-agents ai-tools browser-automation browser-use |
| browser-use-vision | ★ 57 |
MIT |
Python |
Vision-grounding plugin for browser-use agents with SoM, Florence-2, Vision–DOM alignment, adaptive visual context, and objective evaluation. | ai-agent browser-agent browser-use florence-2 |
| agentitest | ★ 48 |
Apache-2.0 |
Python |
Write browser tests in natural language, built on pytest and allure. | allure-report browser-use gemini playwright |
| cappuccino | ★ 45 |
Apache-2.0 |
Python |
Cappuccino is an GUI Agent based on desktop screen. It is a Manus-like AI Agent that can be deployed locally. | browser-use computeruse deepseek omniparser |
| browser-use-rs | ★ 41 |
MIT |
Rust |
A Rust library for browser automation via Chrome DevTools Protocol with built-in AI integration through Model Context Protocol (MCP) | browser-use mcp |
| Operator-Use | ★ 41 |
MIT |
Python |
🐧Operator-Use: AI that can do stuffs on your computer | ai-agents browser-use computer-use openclaw |
🐝 Multi-Agent & Swarm Harnesses
Harnesses and testbeds designed for multi-agent collaboration, swarms, role-playing, and distributed agent execution.
| Repository | Stars | License | Language | Description | Key Topics |
|---|---|---|---|---|---|
| ruflo | ★ 74.0k |
MIT |
TypeScript |
🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive... | agentic-ai agentic-framework agentic-workflow ai-agents |
| anything-llm | ★ 66.8k |
MIT |
JavaScript |
Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience | agent-computer agent-harness agent-orchestration agentic-ai |
| CowAgent | ★ 47.3k |
MIT |
Python |
Open-source personal AI assistant & Agent Harness. Plans tasks, runs tools and skills, self-evolves with memory and knowledge. Multi-agent, multi-model, mult... | ai-agent ai-agents chatgpt-on-wechat claude |
| openhuman | ★ 41.5k |
GPL-3.0 |
Rust |
OpenHuman is the fastest, cheapest, most efficient open-source agent harness. Written in Rust | agent-orchestration ai-agents ai-assistant desktop |
| munder-difflin | ★ 8.5k |
MIT |
TypeScript |
an open-source alternative to the dots, bots and muses of the world, run an office of claude code/codex like agents on your laptop, sandboxes or anywhere, us... | agent-harness agent-orchestration ai-agents autonomous-agents |
| loopx | ★ 6.2k |
Apache-2.0 |
Python |
A control plane with a durable state kernel for long-horizon agents and teams. Keep work moving and improving across sessions, with less human attention. | agent-control-plane agent-harness ai-agents claude-code |
| agentops | ★ 5.9k |
MIT |
Python |
Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI ... | agentops agents-sdk anthropic autogen |
| openrig | ★ 5.6k |
Apache-2.0 |
TypeScript |
Build your own network of agents from Claude Code, Codex and Pi: persistent teams with roles, shared context and owned work. | agent-harness agent-orchestration agent-skills ai-coding |
| devspace | ★ 5.2k |
MIT |
TypeScript |
Minimal Coding Agent Harness on MCP for ChatGPT, Claude, Hermes, Grok Bot, OpenClaw | agent-orchestration chatgpt claude claude-code |
| agentconnect | ★ 1.4k |
Apache-2.0 |
TypeScript |
The open-source, multi-agent alternative to Claude Tag. @ any agent, wherever work happens, they work alongside your team, learning as they go. | agent-client-protocol agent-sandbox ai-agent claude-tag-alternative |
| commonly | ★ 1.4k |
Apache-2.0 |
TypeScript |
Open-source room for humans + cross-vendor AI agents. Every agent gets its own name, memory, skills, and workstation. Any runtime, your infra — no per-agent ... | agent-framework agent-harness agent-protocol agentic |
| oh-my-agent | ★ 1.3k |
MIT |
TypeScript |
Mechanical verification for AI coding agents — skills pack or full harness (stop-hook gates, artifact checks, independent judges). | agent-harness agent-skills claude-code cli |
| Chorus | ★ 1.2k |
AGPL-3.0 |
TypeScript |
The Agent Harness for AI-Human Collaboration, inspired by the AI-DLC (AI-Driven Development Lifecycle) | agent-harness ai-agents ai-dlc claude-code |
| starnet | ★ 1.2k |
MIT |
JavaScript |
A living pixel-art station where real AI agents do real work. Local-first desktop agent harness - bring your own key, watch your crew actually run. | agent-harness ai-agents byok desktop-app |
| mateclaw | ★ 1.1k |
Apache-2.0 |
Java |
🤖 MateClaw — Your second brain with Multi-Agent Orchestration, MCP Protocol, Skills & Memory, Dream, and Multi-Channel Support. Built on Spring AI Alibaba. | agent-harness ai-agent dingtalk-robot feishu-bot |
| my-virtual-office | ★ 339 |
AGPL-3.0 |
Python |
A self-hosted 2D AI workspace for AI Agents | agent-character agent-harness agent-management agent-office |
| water | ★ 337 |
Apache-2.0 |
Python |
The production-ready agent harness framework for Python | agent-harness framework harness multi-agent |
| zenith | ★ 334 |
Apache-2.0 |
Python |
Zenith: a continuous-improvement harness for long-running agent tasks. Turns Claude Code, Codex, or Hermes into a multi-agent mission orchestrator via MCP/ACP. | agent-client-protocol agent-harness ai-agents claude-code |
🛡️ Safety, Security & Red-Teaming Harnesses
Evaluation harnesses and testbeds for agent jailbreaking, prompt injection, security audits, and behavioral safety alignment.
| Repository | Stars | License | Language | Description | Key Topics |
|---|---|---|---|---|---|
| OpenART | ★ 231 |
AGPL-3.0 |
Python |
OpenART is an open-source framework designed to evaluate the safety and robustness of autonomous AI agents in dynamic, long-horizon, and stateful environment... | agent-environment agent-evaluation agent-redteam agent-safety |
| proofagent-harness | ★ 30 |
Apache-2.0 |
Python |
Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI | adversarial-testing agent-evaluation agent-testing ai-agents |
📦 Execution Sandboxes, Runtimes & Tooling
Secure execution sandboxes, containerized runtimes, local code execution environments, and Model Context Protocol (MCP) harnesses.
| Repository | Stars | License | Language | Description | Key Topics |
|---|---|---|---|---|---|
| ECC | ★ 274.6k |
MIT |
JavaScript |
The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Curs... | ai-agents anthropic claude claude-code |
| QwenPaw | ★ 35.5k |
Apache-2.0 |
TypeScript |
Your Personal AI Assistant; easy to install, deploy on your own machine or on the cloud; supports multiple chat apps with easily extensible capabilities. | agent-harness agentscope ai-agent ai-agents |
| harness-sdk | ★ 8.7k |
Apache-2.0 |
Python |
Build an agent harness and control it end-to-end. Open-source SDK for production AI agents in Python & TypeScript - any model, any cloud. | agent-framework agentic agentic-ai ai-agents |
| smolvm | ★ 6.6k |
Apache-2.0 |
Rust |
An embeddable, portable, branchable virtual machine to safely run Agents locally. | agent-sandbox ai-agents code-execution containers |
| nono | ★ 4.4k |
Apache-2.0 |
Rust |
agent runtime security - zero trust, zero setup, zero latency agent sandbox | agent-runtime-security agent-sandbox agent-security ai-agent-sandbox |
| mirage | ★ 3.7k |
Apache-2.0 |
TypeScript |
The World's First Virtual Terminal for AI Agents | agent-sandbox agent-tools ai-agents bash |
| onecli | ★ 3.6k |
Apache-2.0 |
TypeScript |
Open-source sandboxed agent harness for teams. Giving every employee a secured personal agent. | ai-agents cli mcp nodejs |
| agents-best-practices | ★ 2.4k |
MIT |
Multi |
Provider-neutral Agent Skill for Codex, Claude Code, and agentic harness design. | agent-skill agent-skills agentic-workflows ai-agents |
| thClaws | ★ 1.2k |
Apache-2.0 |
Rust |
Open-source AI agent harness in native Rust — GUI, CLI, headless, and webapp from one binary. Multi-provider, MCP, skills, plugins, agent teams. | agent-harness agent-teams ai-agent anthropic |
| gentle-shell | ★ 1.2k |
MIT |
TypeScript |
Gentle Shell is a Pi-native coding-agent harness for controlled development with Organic Driven Development, optional SDD/OpenSpec, subagents, TDD evidence, ... | agent-harness agent-skills ai-agents ai-coding |
| flock | ★ 1.1k |
Apache-2.0 |
Rust |
A desktop multi-agent harness built with Rust, Tauri, and React, powered by langgraph-rust. | chatbot deekseek harness langchain |
| best-of-Agent-Harnesses | ★ 1.1k |
CC-BY-SA-4.0 |
Python |
🏆 Ranked list of 167 AI agent harnesses, plus templates, playbooks, MCP, and learning resources. Rescored weekly. | agent-framework agent-harness agentic-ai ai-agent |
| clawk | ★ 1.0k |
Apache-2.0 |
Go |
Give coding agents a disposable Linux VM, not your laptop | agent-sandbox ai-agent-sandbox ai-agents claude |
| nuwax | ★ 893 |
Apache-2.0 |
TypeScript |
Nuwax Agent OS - An enterprise-grade AI Agent Development and Operation Platform - Providing a complete solution for agent creation and distribution, knowled... | a2a agent-os agent-sandbox agentic |
| browser-use-mcp-server | ★ 848 |
MIT |
Python |
Browse the web, directly from Cursor etc. | browser browser-use cursor mcp |
| BossConsole | ★ 812 |
Apache-2.0 |
Kotlin |
Open-source, multi-platform harness for AI agents - a native, multi-threaded operator's console (JVM, not Electron) to run Claude Code, Codex, Gemini or Open... | agent-harness ai-agents browser claude-code |
| sandbase-harness | ★ 698 |
Apache-2.0 |
TypeScript |
Local-first, self-hosted AI agent runtime and MCP bridge with sandboxed sessions, memory, credentials, audit/replay, and a local Console. | agent-harness agent-observability agent-plugin agent-runtime |
| metaharness | ★ 688 |
MIT |
TypeScript |
🛠️ The meta-harness for AI agents — scaffold your own focused, branded agent harness with its own npx CLI, MCP server, memory, learning loop, and witness-sig... | agent-harness agent-harness-generator agent-scaffolding agentic-ai |
| openclaw.net | ★ 519 |
MIT |
C# |
Self-hosted Enterprise and Personal AI + agent runtime in .NET (NativeAOT-friendly) | agent-harness agent-runtime agentqi ai-agent |
| mantishack | ★ 505 |
Apache-2.0 |
Rust |
Mantis Hack | agent-harness ai-agents autonomous-agents bug-bounty |
| accelerated-container-image | ★ 472 |
Apache-2.0 |
Go |
A production-ready remote/on-demand container/sandbox/vm image format (overlaybd) and snapshotter | agent-sandbox container-images containerd containerd-snapshotter |
| Hypha | ★ 447 |
Apache-2.0 |
TypeScript |
Harness-oriented agent system framework for production-grade LLM agent applications | agent-framework agent-harness agent-memory agent-runtime |
| Sponsio | ★ 440 |
Apache-2.0 |
Python |
Deterministic safety solutions for probabilistic AI agents | agent-guardrails agent-harness agent-runtime agent-safety |
| overlaybd | ★ 420 |
Apache-2.0 |
C++ |
Overlaybd: a block based remote image format for containers/vms/sandboxes. The storage backend of containerd/accelerated-container-image. | agent-sandbox blockdevice container-images iscsi-target |
| Containarium | ★ 296 |
Apache-2.0 |
Go |
Open-source agent runtime — SSH-native isolation, eBPF egress policy, Kubernetes + LXC backends, GPU passthrough, MCP-native CLI | agent-native agent-runtime agent-sandbox agentic-ai |
| agent-sandbox | ★ 218 |
Apache-2.0 |
Go |
Agent-Sandbox is an easy-to-use, enterprise-grade sandbox platform for AI Agents — letting them securely run untrusted LLM-generated code, Browser use, Compu... | agent-sandbox ai-infra ai-sandbox browser-use |
| agent-sandbox | ★ 208 |
MIT |
Python |
Secure local dev environment for collaboration with AI coding agents | agent-harness agent-sandbox coding-agents |
| mosoo | ★ 158 |
Apache-2.0 |
TypeScript |
The open-source Agent Gallery and Gateway for Codex, Claude Agent SDK, and OpenCode. Developers publish an Agent once behind one HTTP API; users run it in an... | agent-api agent-control-plane agent-gallery agent-gateway |
| sample-autonomous-cloud-coding-agents | ★ 157 |
MIT-0 |
TypeScript |
Autonomous background coding agents on AWS. Turn tasks into pull requests via isolated runtimes, with built-in orchestration, observability, and governance. | agent-sandbox agentcore autonomous-agents aws |
| ephemeral-sandbox | ★ 108 |
MIT |
Rust |
Open-source agent sandbox infrastructure for parallel coding agents: isolated workspaces, MCP/CLI control, observability, and atomic publication. | agent-infrastructure agent-sandbox agent-swarm ai-agents |
| sandbox-operator | ★ 80 |
AGPL-3.0 |
Go |
Aggregated Kubernetes apiserver and e2b-compatible data plane for warm-pooled agent sandboxes on microVMs. Sandboxes are served from per-node inventory with ... | agent-sandbox ai-agents cocoon crd |
| intentic | ★ 72 |
MIT |
TypeScript |
An open-source workspace for coding agents, running on your own machine. More work. Less AI waste. Same subscriptions. | agent-infrastructure agent-sandbox agentic-coding ai-agents |
| axern | ★ 65 |
Apache-2.0 |
Go |
Secure, reproducible sandboxes for AI agent evaluation, training, and data synthesis. | agent-sandbox agentic-infrastructure ai-agents cloud-native |
| hivebox | ★ 61 |
BSD-3-Clause |
Python |
Lightweight Linux sandbox for AI agents. Kernel-native isolation (namespaces, cgroups, seccomp, Landlock) with REST API, MCP bridge, and web dashboard. Singl... | agent-sandbox ai-agents ai-infrastructure cli |
| TheGrandQuiz | ★ 56 |
MIT |
Python |
Assessment-driven, local-first learning agent built on an observable Agent Runtime and eval harness—grounded ingestion, trace replay, HITL assessment, durabl... | adaptive-learning agent-harness agent-observability agent-runtime |
| greyproxy | ★ 46 |
MIT |
Go |
SOCKS5 and DNS proxy for greywall | agent-firewall agent-sandbox firewall proxy |
| mvm | ★ 43 |
Apache-2.0 |
Rust |
Manage secure microVMs — one command from zero to secure sessions, locally and remotely. | agent-sandbox agent-security ai-agent-sandbox ai-agent-security |
| boxlite-mcp | ★ 43 |
Apache-2.0 |
Python |
Powered by BoxLite - embeddable sandbox with hardware-level isolation and no daemon. The SQLite of sandbox, coming soon as open source. | ai-agents browser-use code-interpreter computer-use |
| agent-box | ★ 38 |
MIT |
Rust |
Sandboxed containers for AI coding agents with disposable Git/Jujutsu workspaces | agent-sandbox claude-code codex codex-cli |
| sprites-js | ★ 36 |
MIT |
TypeScript |
JavaScript and TypeScript SDK for Fly.io Sprites: computers for agents. Node-style APIs for creating Sprites, running remote commands, moving files, and mana... | agent-sandbox ai-agents development-environments fly-io |
| agent-sandbox | ★ 33 |
ISC |
TypeScript |
Give any AI agent its own disposable Linux machine. Firecracker microVM sandboxing with millisecond boot times, full network access, and native MCP support. | agent-sandbox ai-agents code-execution code-interpreter |
| sprites-go | ★ 31 |
MIT |
Go |
Go SDK for Fly.io Sprites: computers for agents. An exec.Cmd-style API for running commands on persistent, hardware-isolated Linux machines. | agent-sandbox ai-agents development-environments fly-io |
| agent-run | ★ 27 |
GPL-3.0 |
Rust |
Run a coding agent in a sandboxed environment | agent-harness agent-sandbox |
| console | ★ 26 |
AGPL-3.0 |
TypeScript |
Open-source, self-hosted AI gateway for multi-tenant orgs: OpenAI-compatible LLM routing, RAG & vector stores, MCP hub, GPU fleet (MIG slicing), sandboxed co... | agent-sandbox agent-tracing agentic-ai ai-gateway |
| microsandbox-reef | ★ 25 |
MIT |
Rust |
Run OpenClaw, Hermes and your own agents in microsandbox microVMs. One reviewable TOML role sets image, allowed domains and host-bound secrets. No daemon. | agent-sandbox ai-agents cli egress-filtering |
| sprites-ex | ★ 25 |
MIT |
Elixir |
Elixir SDK for Fly.io Sprites: computers for agents. Manage Sprites and run remote commands from Elixir, with APIs that match the language's conventions. | agent-sandbox ai-agents code-execution development-environments |
| sprites-py | ★ 22 |
MIT |
Python |
Python SDK for Fly.io Sprites: computers for agents. Sprite management, remote command execution, filesystems, checkpoints, services, and network policy from... | agent-sandbox ai-agents development-environments fly-io |
| YoloFS | ★ 12 |
GPL-2.0 |
Rust |
Don't Let AI Agents YOLO Your Files: Information and Control in Agent-Native Filesystems | agent-harness agent-sandbox filesystem |
| sample-aws-self-hosted-sandbox | ★ 11 |
MIT-0 |
Python |
a production-grade AI Agent sandbox platform built on AWS, microVM architecture — with lower cost, full data sovereignty, and native Kubernetes integration. | agent-sandbox eks firecracker sandbox |
🚀 Emerging & General Agent Harnesses
Discovered open source agent harnesses and experimental frameworks with emerging topic tags.
| Repository | Stars | License | Language | Description | Key Topics |
|---|---|---|---|---|---|
| ragflow | ★ 91.8k |
Apache-2.0 |
Go |
RAGFlow is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG with Agent capabilities to create a superior context... | agent-harness agentic-ai agentic-nagive agentic-retrieval |
| OpenHands | ★ 90.2k |
MIT |
TypeScript |
🙌 OpenHands: AI-Driven Development | chatgpt claude-ai cli developer-tools |
| learn-claude-code | ★ 78.1k |
MIT |
Python |
Bash is all you need - A nano claude code–like 「agent harness」, built from 0 to 1 | agent-development ai-agent claude claude-code |
| aider | ★ 49.4k |
Apache-2.0 |
Python |
aider is AI pair programming in your terminal | anthropic chatgpt claude-3 cli |
| deepagents | ★ 30.0k |
MIT |
Python |
The batteries-included agent harness. | deepagents harness harness-engineering langchain |
| grok-build | ★ 27.2k |
Apache-2.0 |
Rust |
SpaceXAI's coding agent harness and TUI. Fullscreen, mouse interactive, extensible. | |
| SWE-agent | ★ 20.5k |
MIT |
Python |
SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive... | agent-based-model cybersecurity developer-tools lms |
| evals | ★ 19.6k |
OTHER |
Python |
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks. | |
| DeepCode | ★ 16.7k |
MIT |
Python |
"DeepCode: Open Agentic Coding (Agent Harness & Loop Engineering & Multi-Agent Orchestration)" | agentic-coding harness-engineering llm-agent |
| OpenHarness | ★ 15.9k |
MIT |
Python |
"OpenHarness: Open Agent Harness with a Built-in Personal Agent--Ohmo!" | |
| qm | ★ 15.4k |
MIT |
TypeScript |
Multiplayer agent harness for work. | assistant harness qm |
| Trellis | ★ 14.9k |
AGPL-3.0 |
TypeScript |
The best agent harness. | agentic-coding ai-workflow claudecode codex |
| lm-evaluation-harness | ★ 14.1k |
MIT |
Python |
A framework for few-shot evaluation of language models. | evaluation-framework language-model transformer |
| hive | ★ 11.1k |
Apache-2.0 |
Python |
Multi-Agent Harness for Production AI | agent-framework agent-skills anthropic automation |
| ZCode | ★ 7.5k |
Apache-2.0 |
TypeScript |
Z.ai's coding agent harness. Powerful, intelligent, extensible. | |
| trueforge | ★ 6.1k |
MIT |
TypeScript |
The open-source agent harness - the runtime layer that turns an LLM into a working agent. | agentic-ai harness harness-engineering |
| AgentBench | ★ 3.8k |
Apache-2.0 |
Python |
A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24) | chatgpt gpt-4 llm-agent |
| lazycodex | ★ 3.7k |
MIT |
TypeScript |
The one and only agent harness for complex codebases. Project memory, planning, execution, and verified completion inside Codex. | ai-agents claude claude-code cli |
| knowhere | ★ 3.7k |
Apache-2.0 |
Python |
Knowhere extracts, parses, and outputs structured chunks ready for AI Agents and RAG. | agent-harness ai-agents claude-code codex |
| SoL-Pi | ★ 3.4k |
MIT |
TypeScript |
SoL-Pi: Scaling Auto-Research Loops for Efficient Agent Harnesses | |
| llm-as-a-verifier | ★ 3.3k |
MIT |
Python |
LLM-as-a-Verifier is a general-purpose framework that provides fine-grained feedback for any agent without requiring additional training. It achieves SOTA pe... | |
| Qwen-MM-Plugins | ★ 3.1k |
Apache-2.0 |
Python |
Make any agent harness multimodal-native. | |
| unreal-agent | ★ 2.2k |
MIT |
Go |
Async-first agent harness | |
| dataclaw | ★ 2.1k |
MIT |
Python |
Agent harness to publish your agent chat history as Huggingface datasets. | |
| ante | ★ 2.0k |
Apache-2.0 |
Rust |
Ghost in your shell. Ante is a self-contained agent harness with a highly optimized core. It works like Claude Code or Codex, with none of their dependencies... | agentic-ai ai-agents artifical-life rust |
| waku-agent | ★ 1.9k |
MIT |
Python |
Waku Waku! Waku Agent is a local-first AI agent harness you actually own, including loop, memory, eval, all in code built to stay legible as it grows. | agent-framework ai-agent ai-assistant claude |
| Introduction-to-Quantitative-Finance | ★ 1.8k |
MIT |
Python |
AI+金融（量化）：1.多因子股票量化框架开源教程 2.学界和业界的经典资料收录 3.AI + 金融的相关工作，包括LLM, Agent, benchmark(evaluation), etc. | ai4fin finance investment llm4fin |
| loushang | ★ 1.7k |
Apache-2.0 |
Python |
AI-native agent harness for coding workflows by python: multi-model LLM orchestration, stateful sessions, tool governance, traceable delivery, and provider... | agent-harness agentic chatgpt claude |
| webarena | ★ 1.6k |
Apache-2.0 |
Python |
Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents" | nlp |
| deepagentsjs | ★ 1.6k |
MIT |
TypeScript |
The batteries included agent harness. | deepagents langchain langgraph |
| exo | ★ 1.5k |
MIT |
Rust |
Exo is an agent + harness architecture that is fully recursive, able to safely edit all aspects of itself at runtime to get better at your tasks. | harness self-evolution |
| Autonomous-Agents | ★ 1.4k |
MIT |
Multi |
Autonomous Agents (LLMs) research papers. Updated Daily. | agent-context agent-harness agentic agentic-ai |
| mini-coding-agent | ★ 1.2k |
Apache-2.0 |
Python |
Minimal and readable coding agent harness implementation in Python to explain the core components of coding agents. | llms |
| openharness | ★ 1.1k |
MIT |
Dart |
The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and bu... | 3d-modeling agent-framework agent-harness ai-agents |
| whip | ★ 1.1k |
Apache-2.0 |
Go |
A fast coding-agent harness in Go. Tool-use loop, bubbletea TUI, provider-routable models with live catalog discovery, MCP support, background subagents. One... | |
| bigcode-evaluation-harness | ★ 1.1k |
Apache-2.0 |
Python |
A framework for the evaluation of autoregressive code generation language models. | |
| Mind2Web | ★ 1.0k |
MIT |
Jupyter Notebook |
[NeurIPS'23 Spotlight] "Mind2Web: Towards a Generalist Agent for the Web" -- the first LLM-based web agent and benchmark for generalist web agents | |
| agentic-harness-engineering | ★ 915 |
MIT |
Python |
Official AHE code — Agentic Harness Engineering: observability-driven automatic evolution of coding-agent harnesses (concurrent w/ meta-harness). NexAU-AHE r... | |
| cheetahclaws | ★ 785 |
Apache-2.0 |
Python |
CheetahClaws: A Fast and Easy-to-Use Agent Harness Infrastructure for Long-Horizon, Multi-Model, and Tool-Using AI Systems | agentic-ai claude claude-code memory |
| RSI-Harness | ★ 726 |
MIT |
TypeScript |
RSIH — versionable, shareable agent harness: Pi coding agent + Genome config layer | |
| Awesome-Code-as-Agent-Harness-Papers | ★ 713 |
MIT |
Multi |
A curated list of papers and resources based on the survey "Code as Agent Harness" | |
| waggle | ★ 647 |
Apache-2.0 |
Rust |
Attributed, resolvable artifact references for agent handoffs — a ~30-byte token instead of pasted context. MCP-native; the reference layer for the agent-har... | ai-agents harness-engineering llm-tools loop-engineering |
| vla-evaluation-harness | ★ 639 |
Apache-2.0 |
Python |
One framework to evaluate any VLA model on any robot simulation benchmark. | |
| VulnHunter | ★ 606 |
Apache-2.0 |
Python |
Agentic AI security scanner that hunts exploitable vulnerabilities like an adversary, proves them with executable PoCs, and fixes them test-first. A maintain... | agent-harness agentic-ai appsec exploit-verification |
| Awesome-DeepSeek-Harness-Plugins | ★ 572 |
MIT |
Multi |
Curated DeepSeek Harness (DSH) plugins, extensions, tools, skills, clients, runtimes, integrations, and verified references — English and Chinese. | agent-harness ai-agents awesome-list bilingual |
| inferoa | ★ 564 |
Apache-2.0 |
TypeScript |
Inference-native Tokenmaxxing Agent Harness for Loop Engineering | agent-harness agentic-ai harness-engineering inference |
| genomi | ★ 484 |
Apache-2.0 |
Python |
Local-first, open-source Claude Science alternative, before Claude Science is a thing. Turn your AI agent into personal DNA expert. | agent-harness genome-analysis whole-genome-sequencing |
| ViNote | ★ 470 |
MIT |
Python |
ViNote（视记AI）｜DeepSeek Harness SDK 驱动的开源视频知识 Agent。以工具编排、SQLite 会话恢复和流式交互，串联 YouTube / Bilibili 检索、下载、转写与笔记生成，支持视频问答、知识卡片和思维导图。FastAPI + React，支持 OpenAI 兼容接口模型。 | agent-harness ai-agent bilibili deepseek-harness |
| dryforge | ★ 408 |
Apache-2.0 |
Python |
Bounded-autonomy plugin harness for agents. Intent to implementation: ready, then go. | agent-harness agent-plugins agent-skills agentic-coding |
| learn-workbuddy | ★ 406 |
MIT |
Python |
从 0 复刻 WorkBuddy-style 桌面 AI 助手 Harness：24 章 Python 教程，覆盖 Agent Loop、工具调用、记忆系统、Sidecar、沙盒审计、DeepSeek/OpenAI 评测轨迹 | agent-harness ai-agent anthropic context-engineering |
| Awesome-Agent-Harness | ★ 367 |
CC-BY-4.0 |
Multi |
Agent Harness for Large Language Model Agents: A Survey. Survey on LLM agentharnessengineering with a taxonomy. 110+papers, 23 systems analyzed. | agent-framework agent-harness awesome-list harness |
| AgentClinic | ★ 363 |
MIT |
Python |
Agent benchmark for medical diagnosis | |
| ScienceBuddy | ★ 311 |
MIT |
Python |
ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents | agent-harness aiforscience autoresearch llm-agents |
| lacp | ★ 305 |
MIT |
Shell |
Local-first policy, evidence, memory, and recovery controls for Claude, Codex, Hermes, and other CLI coding agents. | agent-harness ai-agents claude-code codex |
| math-evaluation-harness | ★ 280 |
MIT |
Python |
A simple toolkit for benchmarking LLMs on mathematical reasoning tasks. 🧮✨ | |
| claw-bench | ★ 180 |
Apache-2.0 |
Python |
The Definitive AI Agent Benchmark | |
| shellbench | ★ 141 |
MIT |
Python |
The agent benchmark that scores the full stack — harness, config, and model — not just the LLM. Trace-based scoring, reliability metrics, configuration diagn... | |
| AgentRE-Bench | ★ 124 |
MIT |
Python |
AgentRE-Bench is an agentic benchmark that evaluates state-of-the-art models on long-horizon reverse engineering tasks, measuring their ability to analyze bi... | |
| RODE | ★ 89 |
Apache-2.0 |
Python |
Codes accompanying the paper "RODE: Learning Roles to Decompose Multi-Agent Tasks (ICLR 2021, https://arxiv.org/abs/2010.01523). RODE is a scalable role-base... | |
| ko-lm-evaluation-harness | ★ 81 |
MIT |
Python |
Forked repo from https://github.com/EleutherAI/lm-evaluation-harness/commit/1f66adc | |
| skill-eval-harness | ★ 77 |
MIT |
Python |
Agent Skill evaluation harness for paired variants, trace artifacts, and runner adapters | agent-skills evals skills |
| lemans | ★ 77 |
MIT |
Ruby |
A Ruby harness for running agent benchmarks | evals rails ruby |
| Apex | ★ 76 |
MIT |
Python |
Agents, and RL environment, for optimizing GPU kernels on AMD ROCm using LLM agents. Benchmarks LLM serving workloads end-to-end, profiles bottleneck kernels... | gpu-kernels llamas optimization-pipeline |
| Agent-Benchmarks | ★ 62 |
Apache-2.0 |
Java |
skywalking agent performance test | |
| QwenClawBench | ★ 61 |
MIT |
Python |
General Agent Benchmark for OpenClaw, made by Qwen Team, Alibaba Group. | |
| agent-eval-harness | ★ 45 |
Apache-2.0 |
Python |
No description provided. | |
| eval-harness | ★ 43 |
MIT |
Python |
My own personal evaluation harness | |
| GeoBenchX | ★ 32 |
MIT |
Jupyter Notebook |
LLM-agents benchmark set of geospatial tasks requiring multi-step tool use; and LLM-as-Judge based evaluation framework. | |
| GentPool | ★ 31 |
MIT |
Python |
Gentopia Agent Zoo and Agent Benchmark | |
| NexusBench | ★ 28 |
Apache-2.0 |
Python |
Nexusflow function call, tool use, and agent benchmarks. | |
| RAG-evaluation-harnesses | ★ 27 |
MIT |
Python |
An evaluation suite for Retrieval-Augmented Generation (RAG). | evaluation lm-evaluation rag retrieval-augmented-generation |
| hermes-best-models | ★ 26 |
MIT |
Python |
LLM evals for Hermes agents. Benchmark to find the best models for Hermes | |
| agent-eval-harness | ★ 20 |
MIT |
Python |
An open-source evaluation framework specifically for agentic systems — not just LLM outputs, but full agent behavior. | |
| FORTE | ★ 20 |
MIT |
Python |
FORTE (Full-cycle Office Real-world Task Evaluation) is a general agent benchmark for evaluating AI agents on daily office productivity across 15 corporate p... | |
| write-like-me | ★ 19 |
MIT |
Python |
Measures how you write and builds a voice profile, then enforces it with a deterministic checker: em dashes, filler, hedges, and AI tells are caught by code,... | claude-code claude-skills nlp style-checker |
| agent-runtime-patterns | ★ 19 |
MIT |
Python |
Runtime architecture patterns for production AI/ LLM Agents. Runnable LangGraph + Google ADK code, a composed 90-day contract-renewal example, IBM Telco data... | |
| amazon-log-agent-benchmark-tool | ★ 19 |
MIT-0 |
Go |
Simple log generation and process resource monitor used to benchmark log agents | |
| judgejudy | ★ 18 |
MIT |
Go |
Eval Harness | |
| Neural_Forge | ★ 18 |
MIT |
Python |
NeuralForge is an end-to-end LLM fine-tuning & serving platform. It leverages LoRA (PyTorch/Hugging Face) for domain adaptation, cutting error rates vs. base... | ci-cd containerization docker finetuning |
| Multi-Agent-Benchmark-Tool | ★ 17 |
MIT |
Python |
Interested in running a conspiracy of agents (technical term) on your Local AI Infra? Who isn't! | ai-agents hermes-agent llm-agent llm-inference |
| query-agent-benchmarking | ★ 17 |
BSD-3-Clause |
Jupyter Notebook |
Tools for various benchmarking scenarios of Weaviate's Query Agent. | |
| cloud-attack-range | ★ 17 |
Apache-2.0 |
Python |
Self-hosted vulnerable cloud lab for authorized security training and AI-agent benchmarking | |
| K-BrowseComp | ★ 16 |
MIT |
Python |
Repository for "K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts" | |
| PHMForge-A-Scenario-Driven-Agentic-Benchmark-for-Industrial-Asset-Lifecycle-Maintenance | ★ 15 |
MIT |
Jupyter Notebook |
Agentic approach to predicting RUL, cost while adhering to safety standards. | |
| LongVidSearch | ★ 15 |
MIT |
Python |
An Agentic Benchmark for Multi-hop Evidence Retrieval Planning in Long Videos | |
| Awesome-General-Agents-Benchmark | ★ 14 |
MIT |
Multi |
Awesome list of general agent benchmarks | |
| ACESEvals | ★ 14 |
MIT |
Python |
Security Agent Benchmarking and Evaluation Research | agentic-ai benchmark-framework cyber-security evaluation-framework |
| eval-harness | ★ 13 |
MIT |
Shell |
Typed evaluation for OpenCode skills and AI agents: capability, regression and product checks; provider-neutral evidence grading, provenance, and CI reports. | agent-testing ai-agents anthropic bash |
| fusion-engine | ★ 12 |
MIT |
Python |
Self-hosted Fusion API and OpenRouter Fusion alternative for building reliable multi-model LLM ensembles — fan out one prompt to N models, then synthesize on... | ai-agents benchmarking fastapi fusion-api |
| encoder_eval | ★ 11 |
Apache-2.0 |
Python |
Reproducible encoder downstream fine-tune eval harness | |
| skillroll | ★ 11 |
MIT |
Python |
The simple eval harness for your skills | |
| q-evaluation-harness | ★ 10 |
MIT |
Python |
An open-source framework by KX for evaluating Large Language Models on Q/kdb+ code generation tasks. | |
| lm-evaluation-harness | ★ 10 |
MIT |
Python |
A framework for few-shot evaluation of autoregressive language models. | |
| lm-eval-harness | ★ 10 |
MIT |
Python |
No description provided. | |
| lm-evaluation-harness | ★ 10 |
MIT |
Python |
A framework for few-shot evaluation of autoregressive language models. |
Topic Cloud
Discovered topics across indexed open source harnesses:
#ai-agents (122)
#llm (88)
#agent-harness (58)
#agent-evaluation (56)
#mcp (52)
#claude-code (50)
#agent (43)
#python (43)
#browser-use (41)
#codex (39)
#ai (37)
#llm-evaluation (37)
#agent-sandbox (36)
#agentic-ai (36)
#benchmark (33)
#ai-agent (32)
#developer-tools (29)
#coding-agent (28)
#swe-bench (28)
#claude (25)
#cli (25)
#sandbox (25)
#harness (22)
#multi-agent (22)
#agents (21)
#harness-engineering (21)
#llm-agents (21)
#coding-agents (20)
#openai (20)
#typescript (19)
#browser-automation (17)
#evaluation (17)
#self-hosted (17)
#agent-skills (16)
#computer-use (16)
#model-context-protocol (15)
#anthropic (14)
#rag (14)
#autonomous-agents (13)
#openclaw (13)
#opencode (13)
#rust (13)
#evals (12)
#multi-agent-systems (12)
#open-source (12)
#playwright (12)
#skills (12)
#agent-benchmark (11)
#agent-framework (11)
#agent-orchestration (11)
#deepseek (11)
#llmops (11)
#local-first (11)
#agent-observability (10)
#agent-runtime (10)
#ai-coding (10)
#browser-agent (10)
#evaluation-framework (10)
#llm-agent (10)
#ai-evaluation (9)
#ai-safety (9)
#benchmarking (9)
#dsh-plugin (9)
#loop-engineering (9)
#context-engineering (8)
#deepseek-harness (8)
#docker (8)
#jev (8)
#langchain (8)
#pi-coding-agent (8)
#prompt-injection (8)
#tool-use (8)
#agentic (7)
#ai-security (7)
#ai-tools (7)
#chatgpt (7)
#cursor (7)
#desktop-app (7)
#kubernetes (7)
#langgraph (7)
#mcp-server (7)
#security (7)
#agent-security (6)
#agentic-coding (6)
#automation (6)
#computer-use-agent (6)
#eval-harness (6)
#gemini (6)
#guardrails (6)
#hallucination-detection (6)
#large-language-models (6)
#llm-security (6)
#llm-tools (6)
#machine-learning (6)
#microvm (6)
#observability (6)
#opentelemetry (6)
#sdk (6)
#terminal (6)
#testing (6)
#tool-calling (6)
#web-automation (6)
#awesome-list (5)
#browser (5)
#claude-skills (5)
#code-execution (5)
#codex-cli (5)
#llm-observability (5)
#orchestration (5)
#prompt-engineering (5)
#reinforcement-learning (5)
#remote-execution (5)
#terminal-bench (5)
#tui (5)
#tutorial (5)
#agent-testing (4)
#agentops (4)
#ai-sandbox (4)
#chrome-extension (4)
#claude-code-plugin (4)
#code-review (4)
#development-environments (4)
#evaluation-harness (4)
#fly-io (4)
#gemini-cli (4)
#genai (4)
#harbor (4)
#hermes-agent (4)
#human-in-the-loop (4)
#llm-eval (4)
#llms (4)
#macos (4)
#memory (4)
#nodejs (4)
#openai-compatible (4)
#openrouter (4)
#pi (4)
#pytest (4)
#rag-evaluation (4)
#sprites (4)
#tauri (4)
#workflow (4)
#a2a (3)
#acp (3)
#agent-memory (3)
#agent-skill (3)
#agentic-workflow (3)
#ai-agent-sandbox (3)
#ai-agents-framework (3)
#ai-assistant (3)
#ai-benchmark (3)
#ai-evals (3)
#ai-infrastructure (3)
#ai-observability (3)
#ai-testing (3)
#autogen (3)
#benchmark-framework (3)
#benchmarks (3)
#cdp (3)
#chatbot (3)
#ci-cd (3)
#code-sandbox (3)
#crewai (3)
#cua (3)
#datasets (3)
#desktop-automation (3)
#dsh (3)
#electron (3)
#eval (3)
#fastapi (3)
#generative-ai (3)
#gui (3)
#gui-agent (3)
#hermes (3)
#javascript (3)
#leaderboard (3)
#llm-benchmark (3)
#llm-testing (3)
#long-horizon-agents (3)
#mlops (3)
#multi-tenant (3)
#multimodal (3)
#openai-codex (3)
#parallel-agents (3)
#react (3)
#recursive-self-improvement (3)
#red-teaming (3)
#responsible-ai (3)
#retrieval-augmented-generation (3)
#sdlc (3)
#software-engineering (3)
#spec-driven-development (3)
#spring-ai (3)
#sqlite (3)
#survey (3)
#synthetic-data (3)
#tracing (3)
#typesafe-ai (3)
#vector-database (3)
#verification (3)
#virtualization (3)
#vllm (3)
#vm (3)
#web-agent (3)
#web-agents (3)
#agent-api (2)
#agent-benchmarks (2)
#agent-browser (2)
#agent-client-protocol (2)
#agent-computer (2)
#agent-control-plane (2)
#agent-evals (2)
#agent-infrastructure (2)
#agent-loop (2)
#agent-optimization (2)
#agent-safety (2)
#agent-workflows (2)
#agentic-framework (2)
#agentic-rl (2)
#ai-agent-security (2)
#ai-code-review (2)
#ai-coding-agent (2)
#ai-engineering (2)
#ai-gateway (2)
#ai-governance (2)
#ai-research (2)
#ai-verification (2)
#aider (2)
#artificial-intelligence (2)
#bash (2)
#browser-use-agent (2)
#browser-use-box (2)
#browser-use-cloud (2)
#claude-agent-sdk (2)
#claude-code-skills (2)
#cloud-browser (2)
#cloudflare (2)
#code-agent (2)
#code-generation (2)
#code-interpreter (2)
#comet (2)
#container-images (2)
#containers (2)
#context-management (2)
#cybersecurity (2)
#dataset (2)
#deep-learning (2)
#deep-research (2)
#deepagents (2)
#devtools (2)
#e2b (2)
#e2b-compatible (2)
#eu-ai-act (2)
#evaluation-metrics (2)
#evaluations (2)
#event-sourcing (2)
#fine-tuning (2)
#firecracker (2)
#git-worktree (2)
#github-actions (2)
#golang (2)
#gpt-4 (2)
#gpu-kernels (2)
#grpo (2)
#gui-operator (2)
#hacktoberfest (2)
#harbor-framework (2)
#jailbreak (2)
#java (2)
#jev-model (2)
#kimi (2)
#kotlin (2)
#language-model (2)
#llamas (2)
#llm-as-a-judge (2)
#llm-as-judge (2)
#llm-benchmarking (2)
#llm-evals (2)
#llm-inference (2)
#llm-orchestration (2)
#llm-safety (2)
#llm-tracing (2)
#llm-wiki (2)
#local-ai (2)
#lora (2)
#mcp-security (2)
#model-evaluation (2)
#monitoring (2)
#muse (2)
#nix (2)
#nlp (2)
#ollama (2)
#open-standard (2)
#openai-agents (2)
#openclaw-skills (2)
#optimization (2)
#paper-list (2)
#persistent-browser (2)
#personal-agent (2)
#personal-ai (2)
#personal-ai-assistant (2)
#personal-assistant (2)
#pi-agent (2)
#planning (2)
#plugins (2)
#policy-engine (2)
#provenance (2)
#python-sdk (2)
#pytorch (2)
#qa (2)
#qwen (2)
#rbac (2)
#react-agent (2)
#reasoning-models (2)
#record-replay (2)
#red-team (2)
#regression-testing (2)
#remote-image (2)
#replay (2)
#reproducibility (2)
#research (2)
#rlvr (2)
#sandboxing (2)
#self-improving (2)
#simulation (2)
#skill-evaluation (2)
#software-factory (2)
#spring-boot (2)
#ssh (2)
#strands-agents (2)
#super-agent (2)
#supply-chain-security (2)
#swe-agent (2)
#swift (2)
#telegram-agent (2)
#telegram-bot (2)
#tmux (2)
#trustworthy-ai (2)
#vibe-coding (2)
#vision (2)
#vps-agent (2)
#vulnerability-scanner (2)
#workflow-automation (2)
#world-model (2)
How It Works
This repository is maintained autonomously by a GitHub Actions workflow:
- Discovery: Queries the GitHub Search API on a scheduled cron and on demand using targeted topic queries (
topic:agent-harness,topic:eval-harness,topic:swe-bench, etc.). - Validation: Filters candidates strictly for open-source licenses (OSI-approved SPDX keys), non-archived status, and minimum star thresholds.
- Topic Grouping: Clusters repositories into functional harness categories based on their defined GitHub topics and indexes all raw tags.
- Autonomous Publishing: Generates
README.mdand machine-readabledata/harnesses.json, committing updates automatically if changes are detected.
Contributing
Want to add your AI agent harness or testbed to this list?
Method 1: Tag Your GitHub Repository (Automatic)
Add relevant topics to your GitHub repository (under the repo's "About" section):
- Core topics:
agent-harness,eval-harness,swe-bench,browser-use,agent-benchmark,agent-sandbox. - Ensure your repository has an OSI-approved open source license (
MIT,Apache-2.0,BSD,GPL, etc.). The automated workflow will pick up your repository on its next run!
Method 2: Submit a Pull Request
Add your repository to seed_repos in config/settings.yaml:
seed_repos:
- "your-org/your-agent-harness"
Or propose a new topic category in config/topics.yaml.
License Requirements
All indexed repositories must be licensed under an OSI-approved open source license.