[GH-ISSUE #297] [Enhancement]: PentAGI as a General-Purpose Autonomous Testing Platform #94

New Issue

2026-06-06T22:09:06-04:00

yindo commented

2026-06-06 22:09:06 -04:00

Originally created by @epeer1 on GitHub (May 3, 2026).
Original GitHub issue: https://github.com/vxcontrol/pentagi/issues/297

Originally assigned to: @asdek on GitHub.

Target Component

Core Services (Frontend UI/Backend API), AI Agents (Researcher/Developer/Executor), Documentation and User Experience

Enhancement Description

[Proposal] Extend agent system to support pre-deployment QA & load testing — not just security

The observation
I've been studying PentAGI's architecture deeply — the agent hierarchy, the prompt template system, the flow/task/subtask decomposition, the Graphiti knowledge graph, the Docker isolation, the image chooser. And I realized something:
PentAGI is not a pentesting tool. It's a general-purpose autonomous agent framework that happens to have pentesting prompts.
The entire orchestration layer — the orchestrator → researcher → developer → executor pipeline, the vector memory, the chain summarization, the 13 agent types with swappable prompts, the template customization via database — none of it is inherently security-specific. The "pentest" lives in exactly three places: the prompt templates, the tool catalog (nmap/metasploit/sqlmap), and the Docker image (Kali Linux). Everything else is generic.
The opportunity
The riskiest moment in software delivery is deploying to production. Every team has staging environments, but testing them is either manual, scripted (brittle), or limited to CI unit tests that don't catch real-world behavior.
What if PentAGI could also be the thing that autonomously validates a staging environment before deployment? Not just for security — for everything:

QA Tester agent — reads the API spec, systematically tests every endpoint, validates contracts, checks edge cases, reports PASS/FAIL per endpoint
Load Tester agent — runs k6/wrk scripts, ramps traffic, measures p95 latency, finds the degradation point, gives a GO/NO-GO
Scout agent — monitors /metrics and health endpoints while the other agents test, detects anomalies, flags memory leaks or error rate spikes

And here's the thing — the pentester agent still runs alongside them. So you get security + functional + performance + observability in a single swarm. Nobody else offers this.
Why this is a small change
Because of how well PentAGI is architected, this doesn't require touching the core:

3 new ProviderOptionsType constants in providers.go
3 new prompt templates (.tmpl files) — following the exact same structure as pentester.tmpl
3 new result tool constants in tools.go — same pattern as HackResultToolName
1 update to image_chooser.tmpl — add a testing toolbox image alongside Kali
Test data entries in validator/testdata.go

Zero changes to the orchestrator, the memory system, the GraphQL API, the frontend, the summarization system, or any existing agent. Fully additive. Fully backward compatible. Existing pentesting flows work identically.
Why this matters for PentAGI's future
Right now PentAGI competes with NodeZero ($35K/yr), Pentera ($50K/yr), and a handful of other security-only tools. That's a narrow market.
Adding internal testing agents puts PentAGI in a much larger market — every engineering team that deploys software. The pitch becomes: "One framework, one knowledge graph, one swarm — security testing AND deployment validation." No other open-source project does this.
The name even works: PentAGI becomes "testing AGI" — penetration testing is just one mode.
What I'm offering
I have a detailed RFC with the full prompt templates, code change locations, taxonomy extensions, and a phased implementation plan. I'd like to contribute this as a PR if the maintainers are interested.
Happy to start small — just the QA Tester agent as a first PR to prove the concept, then expand to Load Tester and Scout in follow-ups.
Would love to hear your thoughts.

Technical Details

No response

Designs and Mockups

No response

Alternative Solutions

No response

Verification

I have checked that this enhancement hasn't been already proposed
This enhancement aligns with PentAGI's goal of autonomous penetration testing
I have considered the security implications of this enhancement
I have provided clear use cases and benefits

Originally created by @epeer1 on GitHub (May 3, 2026). Original GitHub issue: https://github.com/vxcontrol/pentagi/issues/297 Originally assigned to: @asdek on GitHub. ### Target Component Core Services (Frontend UI/Backend API), AI Agents (Researcher/Developer/Executor), Documentation and User Experience ### Enhancement Description [Proposal] Extend agent system to support pre-deployment QA & load testing — not just security The observation I've been studying PentAGI's architecture deeply — the agent hierarchy, the prompt template system, the flow/task/subtask decomposition, the Graphiti knowledge graph, the Docker isolation, the image chooser. And I realized something: PentAGI is not a pentesting tool. It's a general-purpose autonomous agent framework that happens to have pentesting prompts. The entire orchestration layer — the orchestrator → researcher → developer → executor pipeline, the vector memory, the chain summarization, the 13 agent types with swappable prompts, the template customization via database — none of it is inherently security-specific. The "pentest" lives in exactly three places: the prompt templates, the tool catalog (nmap/metasploit/sqlmap), and the Docker image (Kali Linux). Everything else is generic. The opportunity The riskiest moment in software delivery is deploying to production. Every team has staging environments, but testing them is either manual, scripted (brittle), or limited to CI unit tests that don't catch real-world behavior. What if PentAGI could also be the thing that autonomously validates a staging environment before deployment? Not just for security — for everything: QA Tester agent — reads the API spec, systematically tests every endpoint, validates contracts, checks edge cases, reports PASS/FAIL per endpoint Load Tester agent — runs k6/wrk scripts, ramps traffic, measures p95 latency, finds the degradation point, gives a GO/NO-GO Scout agent — monitors /metrics and health endpoints while the other agents test, detects anomalies, flags memory leaks or error rate spikes And here's the thing — the pentester agent still runs alongside them. So you get security + functional + performance + observability in a single swarm. Nobody else offers this. Why this is a small change Because of how well PentAGI is architected, this doesn't require touching the core: 3 new ProviderOptionsType constants in providers.go 3 new prompt templates (.tmpl files) — following the exact same structure as pentester.tmpl 3 new result tool constants in tools.go — same pattern as HackResultToolName 1 update to image_chooser.tmpl — add a testing toolbox image alongside Kali Test data entries in validator/testdata.go Zero changes to the orchestrator, the memory system, the GraphQL API, the frontend, the summarization system, or any existing agent. Fully additive. Fully backward compatible. Existing pentesting flows work identically. Why this matters for PentAGI's future Right now PentAGI competes with NodeZero ($35K/yr), Pentera ($50K/yr), and a handful of other security-only tools. That's a narrow market. Adding internal testing agents puts PentAGI in a much larger market — every engineering team that deploys software. The pitch becomes: "One framework, one knowledge graph, one swarm — security testing AND deployment validation." No other open-source project does this. The name even works: PentAGI becomes "testing AGI" — penetration testing is just one mode. What I'm offering I have a detailed RFC with the full prompt templates, code change locations, taxonomy extensions, and a phased implementation plan. I'd like to contribute this as a PR if the maintainers are interested. Happy to start small — just the QA Tester agent as a first PR to prove the concept, then expand to Load Tester and Scout in follow-ups. Would love to hear your thoughts. ### Technical Details _No response_ ### Designs and Mockups _No response_ ### Alternative Solutions _No response_ ### Verification - [x] I have checked that this enhancement hasn't been already proposed - [x] This enhancement aligns with PentAGI's goal of autonomous penetration testing - [x] I have considered the security implications of this enhancement - [x] I have provided clear use cases and benefits

yindo added the enhancement label 2026-06-06 22:09:06 -04:00

yindo commented

2026-06-06 22:09:06 -04:00

@jasper-247 commented on GitHub (May 7, 2026):

Target Component

Core Services (Frontend UI/Backend API), AI Agents (Researcher/Developer/Executor), Documentation and User Experience

Enhancement Description

[Proposal] Extend agent system to support pre-deployment QA & load testing — not just security

The observation
I've been studying PentAGI's architecture deeply — the agent hierarchy, the prompt template system, the flow/task/subtask decomposition, the Graphiti knowledge graph, the Docker isolation, the image chooser. And I realized something:
PentAGI is not a pentesting tool. It's a general-purpose autonomous agent framework that happens to have pentesting prompts.
The entire orchestration layer — the orchestrator → researcher → developer → executor pipeline, the vector memory, the chain summarization, the 13 agent types with swappable prompts, the template customization via database — none of it is inherently security-specific. The "pentest" lives in exactly three places: the prompt templates, the tool catalog (nmap/metasploit/sqlmap), and the Docker image (Kali Linux). Everything else is generic.
The opportunity
The riskiest moment in software delivery is deploying to production. Every team has staging environments, but testing them is either manual, scripted (brittle), or limited to CI unit tests that don't catch real-world behavior.
What if PentAGI could also be the thing that autonomously validates a staging environment before deployment? Not just for security — for everything:

QA Tester agent — reads the API spec, systematically tests every endpoint, validates contracts, checks edge cases, reports PASS/FAIL per endpoint
Load Tester agent — runs k6/wrk scripts, ramps traffic, measures p95 latency, finds the degradation point, gives a GO/NO-GO
Scout agent — monitors /metrics and health endpoints while the other agents test, detects anomalies, flags memory leaks or error rate spikes

And here's the thing — the pentester agent still runs alongside them. So you get security + functional + performance + observability in a single swarm. Nobody else offers this.
Why this is a small change
Because of how well PentAGI is architected, this doesn't require touching the core:

3 new ProviderOptionsType constants in providers.go
3 new prompt templates (.tmpl files) — following the exact same structure as pentester.tmpl
3 new result tool constants in tools.go — same pattern as HackResultToolName
1 update to image_chooser.tmpl — add a testing toolbox image alongside Kali
Test data entries in validator/testdata.go

Zero changes to the orchestrator, the memory system, the GraphQL API, the frontend, the summarization system, or any existing agent. Fully additive. Fully backward compatible. Existing pentesting flows work identically.
Why this matters for PentAGI's future
Right now PentAGI competes with NodeZero ($35K/yr), Pentera ($50K/yr), and a handful of other security-only tools. That's a narrow market.
Adding internal testing agents puts PentAGI in a much larger market — every engineering team that deploys software. The pitch becomes: "One framework, one knowledge graph, one swarm — security testing AND deployment validation." No other open-source project does this.
The name even works: PentAGI becomes "testing AGI" — penetration testing is just one mode.
What I'm offering
I have a detailed RFC with the full prompt templates, code change locations, taxonomy extensions, and a phased implementation plan. I'd like to contribute this as a PR if the maintainers are interested.
Happy to start small — just the QA Tester agent as a first PR to prove the concept, then expand to Load Tester and Scout in follow-ups.
Would love to hear your thoughts.

Technical Details

No response

Designs and Mockups

No response

Alternative Solutions

No response

Verification

I have checked that this enhancement hasn't been already proposed

This enhancement aligns with PentAGI's goal of autonomous penetration testing

I have considered the security implications of this enhancement

I have provided clear use cases and benefits

What guarantees do you provide that no raw PII is ever transmitted to external LLM APIs?

@jasper-247 commented on GitHub (May 7, 2026): > ### Target Component > > Core Services (Frontend UI/Backend API), AI Agents (Researcher/Developer/Executor), Documentation and User Experience > > ### Enhancement Description > > [Proposal] Extend agent system to support pre-deployment QA & load testing — not just security > > The observation > I've been studying PentAGI's architecture deeply — the agent hierarchy, the prompt template system, the flow/task/subtask decomposition, the Graphiti knowledge graph, the Docker isolation, the image chooser. And I realized something: > PentAGI is not a pentesting tool. It's a general-purpose autonomous agent framework that happens to have pentesting prompts. > The entire orchestration layer — the orchestrator → researcher → developer → executor pipeline, the vector memory, the chain summarization, the 13 agent types with swappable prompts, the template customization via database — none of it is inherently security-specific. The "pentest" lives in exactly three places: the prompt templates, the tool catalog (nmap/metasploit/sqlmap), and the Docker image (Kali Linux). Everything else is generic. > The opportunity > The riskiest moment in software delivery is deploying to production. Every team has staging environments, but testing them is either manual, scripted (brittle), or limited to CI unit tests that don't catch real-world behavior. > What if PentAGI could also be the thing that autonomously validates a staging environment before deployment? Not just for security — for everything: > > QA Tester agent — reads the API spec, systematically tests every endpoint, validates contracts, checks edge cases, reports PASS/FAIL per endpoint > Load Tester agent — runs k6/wrk scripts, ramps traffic, measures p95 latency, finds the degradation point, gives a GO/NO-GO > Scout agent — monitors /metrics and health endpoints while the other agents test, detects anomalies, flags memory leaks or error rate spikes > > And here's the thing — the pentester agent still runs alongside them. So you get security + functional + performance + observability in a single swarm. Nobody else offers this. > Why this is a small change > Because of how well PentAGI is architected, this doesn't require touching the core: > > 3 new ProviderOptionsType constants in providers.go > 3 new prompt templates (.tmpl files) — following the exact same structure as pentester.tmpl > 3 new result tool constants in tools.go — same pattern as HackResultToolName > 1 update to image_chooser.tmpl — add a testing toolbox image alongside Kali > Test data entries in validator/testdata.go > > Zero changes to the orchestrator, the memory system, the GraphQL API, the frontend, the summarization system, or any existing agent. Fully additive. Fully backward compatible. Existing pentesting flows work identically. > Why this matters for PentAGI's future > Right now PentAGI competes with NodeZero ($35K/yr), Pentera ($50K/yr), and a handful of other security-only tools. That's a narrow market. > Adding internal testing agents puts PentAGI in a much larger market — every engineering team that deploys software. The pitch becomes: "One framework, one knowledge graph, one swarm — security testing AND deployment validation." No other open-source project does this. > The name even works: PentAGI becomes "testing AGI" — penetration testing is just one mode. > What I'm offering > I have a detailed RFC with the full prompt templates, code change locations, taxonomy extensions, and a phased implementation plan. I'd like to contribute this as a PR if the maintainers are interested. > Happy to start small — just the QA Tester agent as a first PR to prove the concept, then expand to Load Tester and Scout in follow-ups. > Would love to hear your thoughts. > > ### Technical Details > > _No response_ > > ### Designs and Mockups > > _No response_ > > ### Alternative Solutions > > _No response_ > > ### Verification > > - [x] I have checked that this enhancement hasn't been already proposed > - [x] This enhancement aligns with PentAGI's goal of autonomous penetration testing > - [x] I have considered the security implications of this enhancement > - [x] I have provided clear use cases and benefits What guarantees do you provide that no raw PII is ever transmitted to external LLM APIs?

yindo commented

2026-06-06 22:09:07 -04:00

@epeer1 commented on GitHub (May 7, 2026):

The proposal assumes a dedicated simulation environment with fully synthetic data, fake users, fake credentials, generated before each run and destroyed after. No real PII ever touches the pipeline.
For extra safety, PentAGI already supports local LLMs via Ollama so nothing leaves the network.

@epeer1 commented on GitHub (May 7, 2026): The proposal assumes a dedicated simulation environment with fully synthetic data, fake users, fake credentials, generated before each run and destroyed after. No real PII ever touches the pipeline. For extra safety, PentAGI already supports local LLMs via Ollama so nothing leaves the network.

yindo commented

2026-06-06 22:09:07 -04:00

@jasper-247 commented on GitHub (May 7, 2026):

I specialize in pushing APIs beyond normal usage repeated calls, edge input, and timing inconsistency. My goal is to uncover instability, not just obvious box, and give you reproducible report.

@jasper-247 commented on GitHub (May 7, 2026): I specialize in pushing APIs beyond normal usage repeated calls, edge input, and timing inconsistency. My goal is to uncover instability, not just obvious box, and give you reproducible report.

yindo commented

2026-06-06 22:09:07 -04:00

@epeer1 commented on GitHub (May 7, 2026):

You've built a platform that naturally fits as a full internal testing harness, not just security. Would you like to collaborate on making that happen, rather than me forking and building it separately?

@epeer1 commented on GitHub (May 7, 2026): You've built a platform that naturally fits as a full internal testing harness, not just security. Would you like to collaborate on making that happen, rather than me forking and building it separately?

yindo commented

2026-06-06 22:09:07 -04:00