Compare commits

...
Author SHA1 Message Date
Alex Schapiro 470aa45735 Add --skills and --skills-dir CLI flags with STRIX_SKILLS_DIR support
--skills loads named skills into the root agent before the scan starts
and is persisted in run.json so --resume keeps them. --skills-dir and
STRIX_SKILLS_DIR add extra skill roots ahead of the packaged set so
installed builds can add or override skills without rebuilding. Docs
explain when to use an instruction file versus a skill.
2026-10-23 03:47:54 +00:00
10 changed files with 376 additions and 3 deletions

No files matched your search

+4
View File
@@ -145,6 +145,10 @@ affecting the agents that do the actual testing.
Record every LLM request's and tool call's timing in `run.json`, under `llm_requests` and `tool_calls`. Adds about 0.5 MB to a typical scan's `run.json`.
</ParamField>
<ParamField path="STRIX_SKILLS_DIR" type="string">
Extra skill directories, separated by the OS path separator (`:` on Linux and macOS). Each directory uses the `<category>/<name>.md` layout of the built-in skills. Skills in these directories replace built-in skills with the same path. `--skills-dir` adds directories in front of this list. See [Skills](/advanced/skills#add-your-own-skills).
</ParamField>
<ParamField path="TRACELOOP_BASE_URL" type="string">
OTLP/Traceloop base URL for remote OpenTelemetry export. If unset, Strix keeps traces local only.
</ParamField>
+41
View File
@@ -134,6 +134,47 @@ How to bypass common protections.
How to confirm findings and avoid false positives.
```
## Load skills yourself
Agents pick skills from a catalog of names and one-line descriptions. The pick depends on the subtask the agent was given. To make sure the root agent starts with a skill, name it on the command line:
```bash
strix --target https://api.example.com --skills idor,mass_assignment
strix --target https://api.example.com --skills vulnerabilities/idor,custom/my_method
```
Strix loads the named skills into the root agent's system prompt before the scan starts. The root agent can still pass them to child agents with `create_agent(skills=[...])`. An unknown name stops the run and prints the available skills. A resumed scan keeps the skills of the original run.
## Add your own skills
Point Strix at a directory that uses the same `<category>/<name>.md` layout as the built-in skills:
```bash
strix --target https://app.com --skills-dir ./my-skills --skills custom/my_method
```
```text
my-skills/
├── custom/
│ └── my_method.md
└── vulnerabilities/
└── idor.md # replaces the built-in idor skill
```
Repeat `--skills-dir` for more directories, or set `STRIX_SKILLS_DIR` to a list of directories separated by the OS path separator. Strix searches these directories before the packaged skills. A file with the same path as a built-in skill replaces it. This works for installed builds too, so you do not have to rebuild the package to add a skill.
Each file needs the frontmatter `description` field. Agents only see the name and this description in the catalog, so write the description for the subtask that should trigger the skill.
## Skills or instructions?
<Note>
A skill is specialist knowledge for one agent: a vulnerability class, a technology, a tool, or a target-specific quirk. An agent loads a skill when the skill matches its subtask.
An instruction file is the engagement rulebook: scope, credentials, coordination rules, shared artifacts, and completion criteria. Every agent sees the instructions. See [Custom Instructions](/usage/instructions#reusable-methodology).
</Note>
A cross-cutting methodology that spans the whole engagement belongs in an instruction file. If the methodology also needs knowledge that the built-in skills do not have, put that part in a custom skill and name it from the instruction file.
## Contributing Skills
Community contributions are welcome. Create a `.md` file in the appropriate category with YAML frontmatter (`name` and `description` fields). Good skills include:
+14
View File
@@ -37,6 +37,20 @@ strix (--target <target> | --target-list <path>) [options]
Path to a file containing detailed instructions.
</ParamField>
<ParamField path="--skills" type="string">
Comma-separated skills to load into the root agent before the scan starts, for
example `idor,mass_assignment,custom/my_method`. Use the bare name or
`<category>/<name>`. Unknown names stop the run and list the available skills.
See [Skills](/advanced/skills#load-skills-yourself).
</ParamField>
<ParamField path="--skills-dir" type="string">
Directory with extra skills, in the same `<category>/<name>.md` layout as the
built-in set. Repeat the option for more directories. A skill with the same
path as a built-in skill replaces it. Strix also reads `STRIX_SKILLS_DIR`. See
[Add your own skills](/advanced/skills#add-your-own-skills).
</ParamField>
<ParamField path="--workspace-file" type="string">
Path to a file on your machine to place into the sandbox workspace before the
scan starts. Repeat the option for more files. Write `PATH:DEST` to choose the
+17
View File
@@ -72,6 +72,23 @@ strix --target https://api.example.com \
Be specific. Good instructions help Strix prioritize the most valuable attack paths.
</Tip>
## Reusable methodology
Keep a methodology that you reuse across engagements in a version-controlled instruction file. Good content for the file:
- Coordination rules, for example which identities to test as HIGH and LOW privilege
- Shared artifacts and the format of the evidence you expect
- Completion criteria
- The built-in skills to delegate to, by name, for example `idor`, `broken_function_level_authorization`, `mass_assignment`, and `business_logic`
Every agent in the scan sees the instructions. Agents do not see the full text of a skill unless the skill matches their subtask, so a methodology written as a skill is often not loaded. Use `--skills` to load a skill into the root agent at start, and keep custom skills for specialist knowledge the built-in set does not have. See [Skills](/advanced/skills#skills-or-instructions).
```bash
strix --target https://api.example.com \
--instruction-file ./methodology.md \
--skills idor,broken_function_level_authorization,mass_assignment,business_logic
```
## Workspace files
Instructions become part of the prompt. To give Strix a file to work with, such
+1
View File
@@ -96,6 +96,7 @@ async def run_cli(args: Any) -> None: # noqa: PLR0915
"local_sources": getattr(args, "local_sources", None) or [],
"workspace_files": getattr(args, "workspace_files", None) or [],
"scope_mode": getattr(args, "scope_mode", "auto"),
"skills": list(getattr(args, "skills", None) or []),
"diff_base": getattr(args, "diff_base", None),
"resume_instruction": getattr(args, "user_explicit_instruction", None) or "",
}
+50
View File
@@ -6,6 +6,7 @@ import argparse
import os
import sys
from pathlib import Path
from typing import cast
from strix.config import apply_config_override
from strix.config.settings import DEFAULT_MAX_TURNS
@@ -19,6 +20,7 @@ from strix.interface.utils import (
validate_config_file,
)
from strix.report.runs import list_run_summaries
from strix.skills import SKILLS_DIR_ENV, env_skill_dirs, validate_requested_skills
# Severities ``--fail-on`` accepts, most severe first.
@@ -168,6 +170,25 @@ Strix Cloud:
"(e.g., '--instruction-file ./detailed_instructions.txt').",
)
parser.add_argument(
"--skills",
type=str,
metavar="NAMES",
help="Comma-separated skills to load into the root agent before the scan starts "
"(for example '--skills idor,mass_assignment,custom/my_method'). Use the bare name "
"or '<category>/<name>'. Unknown names stop the run and list the available skills.",
)
parser.add_argument(
"--skills-dir",
dest="skills_dir",
action="append",
metavar="PATH",
help="Directory with extra skills, in the same '<category>/<name>.md' layout as the "
"built-in set. Repeat the option for more directories. A skill with the same path as "
f"a built-in skill replaces it. Also read from the {SKILLS_DIR_ENV} environment variable.",
)
parser.add_argument(
"--workspace-file",
type=str,
@@ -334,6 +355,22 @@ Strix Cloud:
# setting it here makes the flag win over the default location.
os.environ["STRIX_MCP_CONFIG"] = str(mcp_config_path)
# The skill registry reads this env var on every lookup, so writing the flag
# into it (ahead of any entries the user already set) is enough to register.
if args.skills_dir:
skill_dirs: list[str] = []
for raw in args.skills_dir:
skills_dir = Path(raw).expanduser()
if not skills_dir.is_dir():
parser.error(f"--skills-dir directory not found: {raw}")
skill_dirs.append(str(skills_dir))
skill_dirs.extend(str(d) for d in env_skill_dirs() if str(d) not in skill_dirs)
os.environ[SKILLS_DIR_ENV] = os.pathsep.join(skill_dirs)
args.skills = parse_skill_names(args.skills)
if args.skills and (skills_error := validate_requested_skills(args.skills, max_skills=None)):
parser.error(f"--skills: {skills_error}")
# The MCP loader reads these as its per-run include/exclude selection.
if args.mcp_server:
os.environ["STRIX_MCP_ONLY"] = ",".join(args.mcp_server)
@@ -444,6 +481,16 @@ def resume_run_list_message(lead: str) -> str:
return "\n".join(lines)
def parse_skill_names(raw: str | None) -> list[str]:
"""Split a comma-separated ``--skills`` value, dropping blanks and repeats."""
names: list[str] = []
for part in (raw or "").split(","):
name = part.strip()
if name and name not in names:
names.append(name)
return names
def load_resume_state(args: argparse.Namespace) -> None:
"""Populate ``args.targets_info`` and friends from a prior run's run.json.
@@ -496,6 +543,9 @@ def load_resume_state(args: argparse.Namespace) -> None:
if args.instruction is None:
args.instruction = state.get("instruction")
if not args.skills:
stored_skills = cast("list[str]", state.get("skills") or [])
args.skills = [str(name) for name in stored_skills if str(name).strip()]
if not getattr(args, "user_instruction", None):
args.user_instruction = state.get("user_instruction") or None
args.local_sources = collect_local_sources(args.targets_info)
+1
View File
@@ -98,6 +98,7 @@ class GoTuiRuntime:
"local_sources": self.args.local_sources or [],
"workspace_files": getattr(self.args, "workspace_files", None) or [],
"scope_mode": self.args.scope_mode,
"skills": list(getattr(self.args, "skills", None) or []),
"diff_base": self.args.diff_base,
"resume_instruction": self.args.user_explicit_instruction or "",
"workspace_mount": getattr(self.args, "workspace_mount", None) or "",
+1
View File
@@ -784,6 +784,7 @@ class ReportState:
"local_sources": config.get("local_sources", []),
"scope_mode": config.get("scope_mode", "auto"),
"diff_base": config.get("diff_base"),
"skills": list(config.get("skills") or []),
}
)
+27 -3
View File
@@ -1,4 +1,5 @@
import logging
import os
import re
from collections import Counter
from collections.abc import Iterator
@@ -26,6 +27,10 @@ _SKILL_TEMPLATES = Environment(
_INTERNAL_SKILL_CATEGORIES: frozenset[str] = frozenset({"scan_modes", "coordination", "analysis"})
_ROOT_SKILL_CATEGORY = "root"
# ``os.pathsep``-separated list of extra skill roots, read on every lookup so the
# CLI can set it before the first prompt renders.
SKILLS_DIR_ENV = "STRIX_SKILLS_DIR"
_EXTRA_SKILL_DIRS: list[Path] = []
_SKILL_METADATA_CACHE: dict[tuple[Path, int, int], dict[str, str]] = {}
@@ -55,9 +60,27 @@ def registered_skill_dirs() -> tuple[Path, ...]:
return tuple(reversed(_EXTRA_SKILL_DIRS))
def env_skill_dirs() -> tuple[Path, ...]:
"""Skill roots named by ``STRIX_SKILLS_DIR``, in the order given."""
raw = os.environ.get(SKILLS_DIR_ENV, "")
dirs: list[Path] = []
for entry in raw.split(os.pathsep):
if not entry.strip():
continue
path = Path(entry.strip()).expanduser()
if path not in dirs:
dirs.append(path)
return tuple(dirs)
def skill_search_dirs() -> tuple[Path, ...]:
"""All existing skill roots, highest precedence first (built-in last)."""
"""All existing skill roots, highest precedence first (built-in last).
Order: directories from :func:`register_skill_dir`, then the entries of
``STRIX_SKILLS_DIR``, then the packaged skills.
"""
roots = [d for d in registered_skill_dirs() if d.is_dir()]
roots.extend(d for d in env_skill_dirs() if d.is_dir() and d not in roots)
builtin = get_strix_resource_path("skills")
if builtin.is_dir():
roots.append(builtin)
@@ -221,13 +244,14 @@ def get_available_skills() -> dict[str, list[dict[str, str]]]:
return grouped
def validate_requested_skills(skill_list: list[str], max_skills: int = 5) -> str | None:
def validate_requested_skills(skill_list: list[str], max_skills: int | None = 5) -> str | None:
"""Validate a list of user-passed skill names.
Returns ``None`` on success, or a model-readable error message
describing what was wrong (count exceeded, unknown names).
``max_skills=None`` skips the count check.
"""
if len(skill_list) > max_skills:
if max_skills is not None and len(skill_list) > max_skills:
return (
f"Cannot specify more than {max_skills} skills per agent; "
f"got {len(skill_list)}. Aim for 1-3 related skills per specialist."
+220
View File
@@ -0,0 +1,220 @@
"""Tests for the --skills and --skills-dir CLI flags and STRIX_SKILLS_DIR."""
from __future__ import annotations
import importlib
import json
import os
import sys
from types import SimpleNamespace
from typing import TYPE_CHECKING, Any
import pytest
from strix import skills as skills_pkg
from strix.report.state import ReportState
if TYPE_CHECKING:
from pathlib import Path
cli_main: Any = importlib.import_module("strix.interface.main")
ARGV = ["strix", "-t", "https://test.com/", "-n"]
def _stub_settings(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(
cli_main,
"load_settings",
lambda: SimpleNamespace(runtime=SimpleNamespace(max_local_copy_mb=1024)),
)
def _write_skill(root: Path, rel: str, description: str) -> Path:
path = root / rel
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(
f"---\ndescription: {description}\n---\n# {rel}\n{description}\n", encoding="utf-8"
)
return path
@pytest.fixture(autouse=True)
def _isolate(monkeypatch: pytest.MonkeyPatch) -> None:
# setenv (not delenv) so the value the parser writes is restored afterwards.
monkeypatch.setenv(skills_pkg.SKILLS_DIR_ENV, "")
monkeypatch.setattr(skills_pkg, "_EXTRA_SKILL_DIRS", [])
def test_skills_flag_parses_names_into_list(monkeypatch: pytest.MonkeyPatch) -> None:
_stub_settings(monkeypatch)
monkeypatch.setattr(
sys, "argv", [*ARGV, "--skills", "idor, vulnerabilities/mass_assignment,idor,"]
)
args = cli_main.parse_arguments()
assert args.skills == ["idor", "vulnerabilities/mass_assignment"]
def test_skills_flag_defaults_to_empty_list(monkeypatch: pytest.MonkeyPatch) -> None:
_stub_settings(monkeypatch)
monkeypatch.setattr(sys, "argv", ARGV)
assert cli_main.parse_arguments().skills == []
def test_skills_flag_rejects_unknown_name(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
_stub_settings(monkeypatch)
monkeypatch.setattr(sys, "argv", [*ARGV, "--skills", "idor,no_such_skill"])
with pytest.raises(SystemExit):
cli_main.parse_arguments()
err = capsys.readouterr().err
assert "--skills:" in err
assert "no_such_skill" in err
def test_skills_flag_has_no_count_limit(monkeypatch: pytest.MonkeyPatch) -> None:
_stub_settings(monkeypatch)
names = [s["name"] for s in skills_pkg.get_available_skills()["vulnerabilities"]][:7]
assert len(names) == 7
monkeypatch.setattr(sys, "argv", [*ARGV, "--skills", ",".join(names)])
assert cli_main.parse_arguments().skills == names
def test_skills_dir_flag_registers_custom_root(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_write_skill(tmp_path, "custom/my_method.md", "Our methodology")
_stub_settings(monkeypatch)
monkeypatch.setattr(
sys, "argv", [*ARGV, "--skills-dir", str(tmp_path), "--skills", "custom/my_method"]
)
args = cli_main.parse_arguments()
assert args.skills == ["custom/my_method"]
assert os.environ[skills_pkg.SKILLS_DIR_ENV] == str(tmp_path)
assert skills_pkg.skill_search_dirs()[0] == tmp_path
assert "my_method" in skills_pkg.load_skills(["custom/my_method"])
def test_skills_dir_flag_is_repeatable_and_prepends_to_env(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
first, second, from_env = tmp_path / "a", tmp_path / "b", tmp_path / "env"
for d in (first, second, from_env):
d.mkdir()
monkeypatch.setenv(skills_pkg.SKILLS_DIR_ENV, str(from_env))
_stub_settings(monkeypatch)
monkeypatch.setattr(
sys, "argv", [*ARGV, "--skills-dir", str(first), "--skills-dir", str(second)]
)
cli_main.parse_arguments()
assert os.environ[skills_pkg.SKILLS_DIR_ENV].split(os.pathsep) == [
str(first),
str(second),
str(from_env),
]
def test_skills_dir_flag_rejects_missing_directory(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
_stub_settings(monkeypatch)
monkeypatch.setattr(sys, "argv", [*ARGV, "--skills-dir", str(tmp_path / "nope")])
with pytest.raises(SystemExit):
cli_main.parse_arguments()
assert "--skills-dir directory not found" in capsys.readouterr().err
def test_env_skill_dir_is_searched_and_shadows_builtin(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_write_skill(tmp_path, "vulnerabilities/idor.md", "Our IDOR playbook")
_write_skill(tmp_path, "custom/gap_assessment.md", "Web/API gap assessment")
monkeypatch.setenv(skills_pkg.SKILLS_DIR_ENV, os.pathsep.join([str(tmp_path), " "]))
assert skills_pkg.env_skill_dirs() == (tmp_path,)
assert skills_pkg.skill_search_dirs()[0] == tmp_path
catalog = skills_pkg.get_available_skills()
assert {"name": "gap_assessment", "description": "Web/API gap assessment"} in catalog["custom"]
idor = [s for s in catalog["vulnerabilities"] if s["name"] == "idor"]
assert idor == [{"name": "idor", "description": "Our IDOR playbook"}]
assert "Our IDOR playbook" in skills_pkg.load_skills(["idor"])["idor"]
def test_registered_dir_wins_over_env_dir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
env_dir, reg_dir = tmp_path / "env", tmp_path / "reg"
_write_skill(env_dir, "custom/x.md", "from env")
_write_skill(reg_dir, "custom/x.md", "from registered")
monkeypatch.setenv(skills_pkg.SKILLS_DIR_ENV, str(env_dir))
skills_pkg.register_skill_dir(reg_dir)
assert skills_pkg.skill_search_dirs()[:2] == (reg_dir, env_dir)
assert "from registered" in skills_pkg.load_skills(["custom/x"])["x"]
def test_scan_config_and_run_record_carry_skills(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_stub_settings(monkeypatch)
monkeypatch.chdir(tmp_path)
monkeypatch.setattr(sys, "argv", [*ARGV, "--skills", "idor,business_logic"])
args = cli_main.parse_arguments()
state = ReportState(args.run_name)
state.set_scan_config({"targets": args.targets_info, "skills": args.skills})
assert state.run_record["skills"] == ["idor", "business_logic"]
def _write_resumable_run(runs_dir: Path, run_name: str, record: dict[str, Any]) -> None:
run_dir = runs_dir / run_name
run_dir.mkdir(parents=True, exist_ok=True)
(run_dir / "run.json").write_text(json.dumps(record), encoding="utf-8")
state_dir = run_dir / ".state"
state_dir.mkdir(exist_ok=True)
(state_dir / "agents.json").write_text("{}", encoding="utf-8")
def _resumable_record(skills: list[str]) -> dict[str, Any]:
return {
"run_name": "pentest_abcd",
"targets_info": [{"type": "web_application", "details": {"original": "https://t.com/"}}],
"local_sources": [],
"instruction": "",
"scan_mode": "deep",
"skills": skills,
}
def test_resume_restores_persisted_skills(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
_stub_settings(monkeypatch)
monkeypatch.chdir(tmp_path)
_write_resumable_run(tmp_path / "strix_runs", "pentest_abcd", _resumable_record(["idor"]))
monkeypatch.setattr(sys, "argv", ["strix", "--resume", "pentest_abcd"])
assert cli_main.parse_arguments().skills == ["idor"]
def test_resume_explicit_skills_flag_wins(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
_stub_settings(monkeypatch)
monkeypatch.chdir(tmp_path)
_write_resumable_run(tmp_path / "strix_runs", "pentest_abcd", _resumable_record(["idor"]))
monkeypatch.setattr(
sys, "argv", ["strix", "--resume", "pentest_abcd", "--skills", "business_logic"]
)
assert cli_main.parse_arguments().skills == ["business_logic"]