mirror of
https://github.com/crytic/echidna.git
synced 2026-10-12 02:12:32 +00:00
The MCP server is the one part of Echidna that another program talks to, and the Haskell suite cannot say whether that conversation works: it runs in process, while what matters here is what goes over the wire and what the tools answer about a campaign that is actually fuzzing. So this suite starts one. conftest.py launches a real echidna against a fixture contract and waits for the server to accept an MCP initialize rather than merely for the socket to open, then hands the same campaign to every module. Four layers ride on it: test_mcp.py covers the nine tools -- what each one reports about a live campaign, and what it says when it cannot answer. status while the campaign fuzzes, coverage marked up line by line, an LCOV file that is really on disk, an injected sequence that the workers keep fuzzing through, a replayed sequence reported call by call including the revert reason, a sampled function whose counts show up in status a few seconds later. Anything that steers the campaign is undone afterwards, so no test depends on the order it ran in. test_mcp_conformance.py is the regression guard for "real clients can still connect". A notification gets 202 with no body; a GET asking for a server stream gets 405 rather than the JSON blob a strict client tries to read as SSE; initialize negotiates a version and names the server. These are exactly the two behaviours mcp-server 0.2.0.1 fixed, and the reason that version is the floor. test_mcp_codex.py replays Codex's rmcp handshake step by step, and test_mcp_claude.py drives the server with the reference `mcp` SDK through a real session. Both are transport-level and need no API key, so both run in CI; a live-model run is the LangGraph example instead. That example is here too. It reads status, and when the campaign has gone a minute without finding coverage it hands Claude the ABI and the coverage report and injects the sequences that come back -- a demonstration that those tools are enough to close the loop, not a tuned strategy. Some notes on what differs from the version this is adapted from: The tool tests are written against the server as it is rather than the prototype: status answers JSON, show_coverage takes a contract, and a tool that cannot do what it was asked answers with an MCP error result. They assert on all three, where the originals asserted that some substring appeared somewhere in the text. They also go through the shared wire helpers after a handshake, rather than a second raw client that posted tools/call with no handshake and no headers -- asserting leniency next to a suite that asserts strictness said two different things about what the server promises. The `mcp` SDK's 2.0 renamed its transport helper, dropped the third stream it used to yield, and moved its result models to snake_case. Rather than straddle both majors, requirements-test.txt follows the current one and the test skips itself on an older SDK. The other pins are bounded by major version only, since a compatibility suite should be running against what clients actually ship. The fixture contracts lost their spec-kit headers, two properties that could not fail (one returned a literal `true`), and the `test` prefix on their entry points, which means something else in Foundry mode. What is left is a contract that gives the tools something to report on: calls that succeed, calls that revert with a reason, and properties that hold so the campaign keeps running for as long as the session needs it. The workflow now triggers on master and on pull requests against it, like every other workflow here, and only for paths that can change what the server answers -- it builds the whole closure and then fuzzes, which is not something to do on every push to every branch. Its Cachix token is unavailable to pull requests from forks, so those runs read the public cache and push nothing; that is noted where it will be read. tests/mcp/requirements.txt is gone: nothing referenced it, and it contradicted both its sibling and the example's README on what to install. So is pytest.ini, which was only read when pytest was given the directory as an argument -- the one async test carries an explicit marker instead, which works either way. Co-authored-by: Dani Tradito <[email protected]> Co-authored-by: gustavo-grieco <[email protected]>
191 lines
6.6 KiB
Python
191 lines
6.6 KiB
Python
#!/usr/bin/env python3
|
|
"""
|
|
LangGraph agent for Echidna's MCP server.
|
|
|
|
Connects to a running campaign, watches what the fuzzer is reaching, and when
|
|
coverage stalls asks Claude for call sequences to aim it at. A demonstration of
|
|
what the server is for rather than a tuned strategy: the interesting part is
|
|
that `status` and `show_coverage` are enough for a model to decide what to try
|
|
next, and `inject_fuzz_transactions` is enough to act on it.
|
|
|
|
Requirements:
|
|
pip install langchain-anthropic langgraph httpx
|
|
|
|
Usage:
|
|
# Start Echidna with the MCP server. --format text is required: the
|
|
# interactive TUI otherwise owns the terminal.
|
|
echidna MyContract.sol --server 8080 --format text
|
|
|
|
# Run the agent:
|
|
ANTHROPIC_API_KEY=... python examples/mcp_agent.py
|
|
"""
|
|
|
|
import json
|
|
import os
|
|
import re
|
|
import time
|
|
from typing import TypedDict
|
|
|
|
import httpx
|
|
from langchain_anthropic import ChatAnthropic
|
|
from langgraph.graph import END, StateGraph
|
|
|
|
MCP_URL = os.environ.get("ECHIDNA_MCP_URL", "http://127.0.0.1:8080/mcp")
|
|
PROTOCOL_VERSION = "2025-06-18"
|
|
|
|
# How long the campaign may go without finding coverage before the agent steps
|
|
# in, and how often it looks.
|
|
STALL_SECONDS = 60
|
|
INTERVAL_SECONDS = 30
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# MCP client
|
|
# ---------------------------------------------------------------------------
|
|
|
|
def _rpc(method: str, params: dict | None = None, request_id: int | None = 1):
|
|
body = {"jsonrpc": "2.0", "method": method}
|
|
if request_id is not None:
|
|
body["id"] = request_id
|
|
if params is not None:
|
|
body["params"] = params
|
|
headers = {
|
|
"Content-Type": "application/json",
|
|
"Accept": "application/json, text/event-stream",
|
|
}
|
|
if method != "initialize":
|
|
headers["MCP-Protocol-Version"] = PROTOCOL_VERSION
|
|
response = httpx.post(MCP_URL, json=body, headers=headers, timeout=60)
|
|
response.raise_for_status()
|
|
return response.json() if request_id is not None else None
|
|
|
|
|
|
def connect() -> str:
|
|
"""Run the MCP handshake and return the server's name."""
|
|
result = _rpc("initialize", {
|
|
"protocolVersion": PROTOCOL_VERSION,
|
|
"capabilities": {},
|
|
"clientInfo": {"name": "echidna-mcp-agent", "version": "0"},
|
|
}, request_id=0)["result"]
|
|
_rpc("notifications/initialized", request_id=None)
|
|
return result["serverInfo"]["name"]
|
|
|
|
|
|
def call_tool(name: str, arguments: dict | None = None) -> str:
|
|
"""Call a tool and return its report, raising if the tool could not answer."""
|
|
result = _rpc("tools/call", {"name": name, "arguments": arguments or {}})["result"]
|
|
text = "".join(part.get("text", "") for part in result.get("content", []))
|
|
if result.get("isError"):
|
|
raise RuntimeError(f"{name}: {text}")
|
|
return text
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Graph
|
|
# ---------------------------------------------------------------------------
|
|
|
|
class State(TypedDict):
|
|
contract: str
|
|
iterations: int
|
|
coverage: int
|
|
stalled_for: int
|
|
|
|
|
|
def observe(state: State) -> State:
|
|
"""Read how the campaign is going."""
|
|
status = json.loads(call_tool("status"))
|
|
stalled_for = status["time_since_last_coverage_sec"] or 0
|
|
print(f" coverage={status['coverage_points']} "
|
|
f"iterations={status['iterations']} "
|
|
f"last coverage {stalled_for}s ago "
|
|
f"({', '.join(status['recent_covered_functions'][:3]) or 'nothing yet'})")
|
|
return {
|
|
**state,
|
|
"iterations": status["iterations"],
|
|
"coverage": status["coverage_points"],
|
|
"stalled_for": stalled_for,
|
|
}
|
|
|
|
|
|
def inject(state: State) -> State:
|
|
"""Ask Claude which orderings to try, and point the fuzzer at them."""
|
|
# temperature is not accepted on current models; steer with the prompt.
|
|
llm = ChatAnthropic(model="claude-opus-4-8", max_tokens=1024)
|
|
|
|
abi = call_tool("target")
|
|
coverage = call_tool("show_coverage", {"contract": state["contract"]})
|
|
|
|
answer = llm.invoke(
|
|
"You are helping an Echidna fuzzing campaign reach code it has not "
|
|
"reached. It has found no new coverage for a while.\n\n"
|
|
f"The contract under test exposes:\n{abi}\n\n"
|
|
f"Its coverage so far — lines marked * were executed, r reverted, "
|
|
f"o ran out of gas, e errored, blank was never reached:\n"
|
|
f"{coverage[:8000]}\n\n"
|
|
"Reply with up to 3 call sequences that would reach unmarked lines, one "
|
|
"per line and nothing else. Separate the calls in a sequence with ';', "
|
|
"and write '?' for any argument the fuzzer should choose. For example:\n"
|
|
"approve(0x10, ?); transferFrom(?, ?, 100)\n"
|
|
).content
|
|
|
|
# Anything with a call in it; the model's prose, if any, has no parentheses.
|
|
sequences = [line.strip() for line in answer.splitlines()
|
|
if re.search(r"\w+\s*\(", line)]
|
|
|
|
# Drop what was injected last time first, so the fuzzer is not left
|
|
# splitting its budget across every ordering the agent has ever suggested.
|
|
call_tool("clear_fuzz_priorities")
|
|
for sequence in sequences[:3]:
|
|
try:
|
|
print(f" injecting: {sequence}")
|
|
print(f" {call_tool('inject_fuzz_transactions', {'transactions': sequence})}")
|
|
except RuntimeError as complaint:
|
|
print(f" rejected: {complaint}")
|
|
|
|
return state
|
|
|
|
|
|
def route(state: State) -> str:
|
|
return "inject" if state["stalled_for"] >= STALL_SECONDS else END
|
|
|
|
|
|
def build_graph():
|
|
graph = StateGraph(State)
|
|
graph.add_node("observe", observe)
|
|
graph.add_node("inject", inject)
|
|
graph.set_entry_point("observe")
|
|
graph.add_conditional_edges("observe", route, {"inject": "inject", END: END})
|
|
graph.add_edge("inject", END)
|
|
return graph.compile()
|
|
|
|
|
|
def main() -> None:
|
|
if not os.getenv("ANTHROPIC_API_KEY"):
|
|
print("Set ANTHROPIC_API_KEY before running.")
|
|
return
|
|
|
|
try:
|
|
print(f"Connected to {connect()} at {MCP_URL}.")
|
|
except Exception as unreachable:
|
|
print(f"Cannot reach the MCP server: {unreachable}")
|
|
print("Start Echidna with: echidna MyContract.sol --server 8080 --format text")
|
|
return
|
|
|
|
# "Contract: path/to/File.sol:Name" — the name is what show_coverage wants.
|
|
contract = call_tool("target").splitlines()[0].rsplit(":", 1)[1].strip()
|
|
print(f"Watching {contract}. Stepping in after {STALL_SECONDS}s without coverage.")
|
|
|
|
graph = build_graph()
|
|
state: State = {"contract": contract, "iterations": 0, "coverage": 0, "stalled_for": 0}
|
|
|
|
step = 0
|
|
while True:
|
|
step += 1
|
|
print(f"\n--- step {step} ---")
|
|
state = graph.invoke(state)
|
|
time.sleep(INTERVAL_SECONDS)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|