[GH-ISSUE #2] [Enhancement]: Updating the same project with complete opensource alternatives #2

New Issue

2026-06-06T22:07:50-04:00

yindo commented

2026-06-06 22:07:50 -04:00

Originally created by @bb1nfosec on GitHub (Jan 21, 2025).
Original GitHub issue: https://github.com/vxcontrol/pentagi/issues/2

Originally assigned to: @asdek on GitHub.

Target Component

AI Agents (Researcher/Developer/Executor)

Enhancement Description

This enhancement rewrites PentAGI to eliminate reliance on public APIs (e.g., OpenAI, Google, Tavily). The new architecture is entirely open-source, self-hosted, and locally modifiable. This enables greater flexibility, control, and cost efficiency while ensuring data privacy.

Key Changes
Local Language Model Integration:

Replace public LLM APIs with self-hosted models (e.g., LLaMA, GPT-J).
Host the model locally using frameworks like LocalAI or FastAPI.

Local Search Engine:
Replace external search APIs (e.g., Google/Tavily) with local indexing tools such as Whoosh or OpenSearch.

Locally Installed Tools:
Use open-source pentesting tools installed on the system or Dockerized (e.g., nmap, sqlmap, Metasploit).

Self-Contained Deployment:
Use Docker Compose to manage all components, ensuring easy deployment and scalability.

Custom Reporting:
Use Jinja2 and WeasyPrint for generating detailed HTML and PDF reports.

Technical Details

Implementation Details

LocalAI Integration

Framework: Host models like GPT-J or LLaMA locally with LocalAI.

API Endpoint: Serve via http://localhost:8000/generate.

import requests

def generate_text(prompt):
    response = requests.post(
        "http://localhost:8000/generate",
        json={"prompt": prompt, "max_tokens": 100}
    )
    return response.json()["response"
```]

2. Local Search Engine

Tool: Whoosh for lightweight indexing and querying.

3. Open-Source Pentesting Tools

Tools: Nmap, SQLmap, Metasploit, OpenVAS.

4. Reporting Engine

HTML Reporting: Use Jinja2 templates.

### Designs and Mockups

Dashboard UI

Main Page:

Input fields for scanning targets and queries.

Buttons to run pentests, search, and generate reports.

Results Page:

Tabs for vulnerability results, search insights, and generated reports.

### Alternative Solutions

LLM Alternatives:

Use smaller models like GPT-2 or Alpaca for lower hardware requirements.

Explore lightweight frameworks like HuggingFace Transformers with quantized models.

Search Alternatives:

For advanced scalability, replace Whoosh with OpenSearch.

Deployment Alternatives:

Use Kubernetes instead of Docker Compose for larger-scale deployments.

### Verification

- [x] I have checked that this enhancement hasn't been already proposed
- [x] This enhancement aligns with PentAGI's goal of autonomous penetration testing
- [x] I have considered the security implications of this enhancement
- [x] I have provided clear use cases and benefits

Originally created by @bb1nfosec on GitHub (Jan 21, 2025). Original GitHub issue: https://github.com/vxcontrol/pentagi/issues/2 Originally assigned to: @asdek on GitHub. ### Target Component AI Agents (Researcher/Developer/Executor) ### Enhancement Description This enhancement rewrites PentAGI to eliminate reliance on public APIs (e.g., OpenAI, Google, Tavily). The new architecture is entirely open-source, self-hosted, and locally modifiable. This enables greater flexibility, control, and cost efficiency while ensuring data privacy. Key Changes Local Language Model Integration: Replace public LLM APIs with self-hosted models (e.g., LLaMA, GPT-J). Host the model locally using frameworks like LocalAI or FastAPI. Local Search Engine: Replace external search APIs (e.g., Google/Tavily) with local indexing tools such as Whoosh or OpenSearch. Locally Installed Tools: Use open-source pentesting tools installed on the system or Dockerized (e.g., nmap, sqlmap, Metasploit). Self-Contained Deployment: Use Docker Compose to manage all components, ensuring easy deployment and scalability. Custom Reporting: Use Jinja2 and WeasyPrint for generating detailed HTML and PDF reports. ### Technical Details Implementation Details 1. LocalAI Integration Framework: Host models like GPT-J or LLaMA locally with LocalAI. API Endpoint: Serve via http://localhost:8000/generate. ``` import requests def generate_text(prompt): response = requests.post( "http://localhost:8000/generate", json={"prompt": prompt, "max_tokens": 100} ) return response.json()["response" ```] 2. Local Search Engine Tool: Whoosh for lightweight indexing and querying. 3. Open-Source Pentesting Tools Tools: Nmap, SQLmap, Metasploit, OpenVAS. 4. Reporting Engine HTML Reporting: Use Jinja2 templates. ### Designs and Mockups Dashboard UI Main Page: Input fields for scanning targets and queries. Buttons to run pentests, search, and generate reports. Results Page: Tabs for vulnerability results, search insights, and generated reports. ### Alternative Solutions LLM Alternatives: Use smaller models like GPT-2 or Alpaca for lower hardware requirements. Explore lightweight frameworks like HuggingFace Transformers with quantized models. Search Alternatives: For advanced scalability, replace Whoosh with OpenSearch. Deployment Alternatives: Use Kubernetes instead of Docker Compose for larger-scale deployments. ### Verification - [x] I have checked that this enhancement hasn't been already proposed - [x] This enhancement aligns with PentAGI's goal of autonomous penetration testing - [x] I have considered the security implications of this enhancement - [x] I have provided clear use cases and benefits

yindo added the enhancement label 2026-06-06 22:07:50 -04:00

yindo closed this issue

2026-06-06 22:07:51 -04:00

yindo commented

2026-06-06 22:07:51 -04:00

@mazamizo21 commented on GitHub (Jan 21, 2025):

If we can also include Deepseek, as I have spent a significant amount of money and have never completed the task due to persistent issues with installing tools that repeatedly fail.
I believe it would be a valuable addition to have a comprehensive Docker container with the most widely available tools. This would allow the AI to focus on penetration testing rather than troubleshooting the installation of tools.

@mazamizo21 commented on GitHub (Jan 21, 2025): If we can also include Deepseek, as I have spent a significant amount of money and have never completed the task due to persistent issues with installing tools that repeatedly fail. I believe it would be a valuable addition to have a comprehensive Docker container with the most widely available tools. This would allow the AI to focus on penetration testing rather than troubleshooting the installation of tools.

yindo commented

2026-06-06 22:07:51 -04:00

@asdek commented on GitHub (Jan 22, 2025):

Hello @bb1nfosec

Thank you for your enhancement request. Let me address each point:

Local Language Model Integration

Currently, you can configure the following environment variables:

LLM_SERVER_URL=
LLM_SERVER_KEY=
LLM_SERVER_MODEL=

These variables allow PentAGI to connect to a local backend (such as vLLM) implementing the OpenAI interaction specification, but it can also be deployed locally. Additionally, you can set up a LiteLLM proxy server, which offers an OpenAI-compatible interface and forwards requests to another local server. I'll going to review the LocalAI specification and get back to you on this. It seems feasible to support this as an option for selecting a custom LLM server.

Local Search Engine

Supporting Whoosh or OpenSearch doesn't appear to be a major issue. However, the data storage structure may impose constraints on configuring such integration. What do you think about developing a simple HTTP-based protocol with a single POST endpoint for search parameters, while the connection logic to local databases is handled separately?

Locally Installed Tools

Currently, when a job is started, a Docker image is automatically selected to serve the flow. All necessary tools are either pre-installed within the image or downloaded as needed. I have considered creating a custom build based on Kali Linux, but have found a suitable image in booyaabes/kali-linux-full:latest. It wouldn't be difficult to create a Dockerfile with the required tools. Nonetheless, the agents will need internet access during execution to download dependencies and tools from package managers, including those from GitHub. You can see the Docker image selection prompt here: image_chooser.tmpl.

Self-Contained Deployment

PentAGI is currently available in a docker-compose format, divided into three parts:

docker-compose.yml — Core services essential for operation; this is the minimum setup for deployment.
docker-compose-langfuse.yml — Adds LangFuse components for viewing each flow and task in an alternative interface.
docker-compose-observability.yml — Includes Grafana, Jaeger, Loki, VictoriaMetrics, OpenTelemetry collector, and other components for error detection.

You can view the installation instructions:

I'm planning to create a video guide on setup and configuration, including LangFuse and observability components. There will also be a separate video and guide on securely setting up the Docker environment, covering docker-in-docker and other deployment options. Could you provide more details on what you mean in this section?

Custom Reporting

Currently, this feature is under development. There will be a standard report for flows and separate reports for each task in markdown and PDF formats. If you have specific requirements regarding the format and content of the reports, please share them with us, and we will incorporate them into our feature analysis.

Thank you for helping us enhance our product.

@asdek commented on GitHub (Jan 22, 2025): Hello @bb1nfosec Thank you for your enhancement request. Let me address each point: ### Local Language Model Integration Currently, you can configure the following environment variables: - `LLM_SERVER_URL=` - `LLM_SERVER_KEY=` - `LLM_SERVER_MODEL=` These variables allow PentAGI to connect to a local backend (such as [vLLM](https://github.com/vllm-project/vllm)) implementing the OpenAI interaction specification, but it can also be deployed locally. Additionally, you can set up a [LiteLLM](https://github.com/BerriAI/litellm) proxy server, which offers an OpenAI-compatible interface and forwards requests to another local server. I'll going to review the LocalAI specification and get back to you on this. It seems feasible to support this as an option for selecting a custom LLM server. ### Local Search Engine Supporting Whoosh or OpenSearch doesn't appear to be a major issue. However, the data storage structure may impose constraints on configuring such integration. What do you think about developing a simple HTTP-based protocol with a single POST endpoint for search parameters, while the connection logic to local databases is handled separately? ### Locally Installed Tools Currently, when a job is started, a Docker image is automatically selected to serve the flow. All necessary tools are either pre-installed within the image or downloaded as needed. I have considered creating a custom build based on Kali Linux, but have found a suitable image in `booyaabes/kali-linux-full:latest`. It wouldn't be difficult to create a Dockerfile with the required tools. Nonetheless, the agents will need internet access during execution to download dependencies and tools from package managers, including those from GitHub. You can see the Docker image selection prompt here: [image_chooser.tmpl](https://github.com/vxcontrol/pentagi/blob/master/backend/pkg/templates/prompts/image_chooser.tmpl). ### Self-Contained Deployment PentAGI is currently available in a docker-compose format, divided into three parts: - [docker-compose.yml](https://github.com/vxcontrol/pentagi/blob/master/docker-compose.yml) — Core services essential for operation; this is the minimum setup for deployment. - [docker-compose-langfuse.yml](https://github.com/vxcontrol/pentagi/blob/master/docker-compose-langfuse.yml) — Adds LangFuse components for viewing each flow and task in an alternative interface. - [docker-compose-observability.yml](https://github.com/vxcontrol/pentagi/blob/master/docker-compose-observability.yml) — Includes Grafana, Jaeger, Loki, VictoriaMetrics, OpenTelemetry collector, and other components for error detection. You can view the installation instructions: - [Quick Start](https://github.com/vxcontrol/pentagi/tree/master?tab=readme-ov-file#-quick-start) - [Advanced Setup](https://github.com/vxcontrol/pentagi/tree/master?tab=readme-ov-file#-advanced-setup) I'm planning to create a video guide on setup and configuration, including LangFuse and observability components. There will also be a separate video and guide on securely setting up the Docker environment, covering docker-in-docker and other deployment options. Could you provide more details on what you mean in this section? ### Custom Reporting Currently, this feature is under development. There will be a standard report for flows and separate reports for each task in markdown and PDF formats. If you have specific requirements regarding the format and content of the reports, please share them with us, and we will incorporate them into our feature analysis. Thank you for helping us enhance our product.

yindo commented

2026-06-06 22:07:51 -04:00

@asdek commented on GitHub (Jan 22, 2025):

Hello @mazamizo21

I apologize for the inconvenience you've experienced. I will try to reproduce this problem on my side. In the meantime, here's a workaround: please ensure that the Docker image booyaabes/kali-linux-full:latest is selected. This image is about 6GB in size but already contains all the necessary base utilities, as can be mentioned in the job description. Additionally, within your job's instructions, you can specify that during execution, it should first verify whether the required utility is already installed instead of downloading and installing it separately.

As an alternative solution, you can add to your prompt that GUI-based utilities should be excluded from use during execution and explicitly list a blacklist of utilities that should neither be installed nor used.

Currently, users can only manage the job text, but we have tasks in our backlog to expand user customization to include system prompts via UI. For better understanding how your task might be executed, please take a look at the list of system prompts here: System Prompts.

Thank you for your patience and understanding.

@asdek commented on GitHub (Jan 22, 2025): Hello @mazamizo21 I apologize for the inconvenience you've experienced. I will try to reproduce this problem on my side. In the meantime, here's a workaround: please ensure that the Docker image `booyaabes/kali-linux-full:latest` is selected. This image is about 6GB in size but already contains all the necessary base utilities, as can be mentioned in the job description. Additionally, within your job's instructions, you can specify that during execution, it should first verify whether the required utility is already installed instead of downloading and installing it separately. As an alternative solution, you can add to your prompt that GUI-based utilities should be excluded from use during execution and explicitly list a blacklist of utilities that should neither be installed nor used. Currently, users can only manage the job text, but we have tasks in our backlog to expand user customization to include system prompts via UI. For better understanding how your task might be executed, please take a look at the list of system prompts here: [System Prompts](https://github.com/vxcontrol/pentagi/tree/master/backend/pkg/templates/prompts). Thank you for your patience and understanding.

yindo commented

2026-06-06 22:07:51 -04:00

@vchaplygim commented on GitHub (Apr 6, 2025):

@asdek Hi, thanks for the great tool — it looks very promising. However, I ran into an issue where the image I was using didn’t have zap-cli preinstalled. As a result, I ended up wasting most of my allocated OpenAI API requests on repeated attempts to download and use zap-cli and not only these tool.

I’m now thinking it might be better to use the official image (https://hub.docker.com/r/kalilinux/kali-rolling) and install the full set of tools (apt update && apt -y install kali-linux-large) upfront, instead of relying on someone’s custom image (booyaabes/kali-linux-full:latest - which was last updated a year ago according to Docker Hub) that might be missing essential components.

@vchaplygim commented on GitHub (Apr 6, 2025): @asdek Hi, thanks for the great tool — it looks very promising. However, I ran into an issue where the image I was using didn’t have zap-cli preinstalled. As a result, I ended up wasting most of my allocated OpenAI API requests on repeated attempts to download and use zap-cli and not only these tool. I’m now thinking it might be better to use the official image (https://hub.docker.com/r/kalilinux/kali-rolling) and install the full set of tools (apt update && apt -y install kali-linux-large) upfront, instead of relying on someone’s custom image (booyaabes/kali-linux-full:latest - which was last updated a year ago according to Docker Hub) that might be missing essential components. <img width="1459" alt="Image" src="https://github.com/user-attachments/assets/d54aef2c-882a-4454-b11f-78928bea1aaf" /> <img width="616" alt="Image" src="https://github.com/user-attachments/assets/385669aa-bfe8-415f-b616-fe77996db595" />

yindo commented

2026-06-06 22:07:52 -04:00

@asdek commented on GitHub (Apr 6, 2025):

Hi @vchaplygim

You're absolutely right, there need to move away from using booyaabes/kali-linux-full. I tried quickly building an image with the necessary packages installed, but without proper configuration, not all the utilities are ready to work out of the box. I also had the idea of creating a set of pre-prepared commands wrapped in scripts to minimize the time spent on installation and debugging. This should be available within the next couple of months, maybe sooner.

In the meantime, you can check out the prompt for selecting a Docker image here.

According to this, you have the ability to specify in the task description, which kicks off the flow, which Docker image the AI agents should use, with a phrase like:

“At the stage of selecting the Docker image to solve this task, you should use {YOUR_IMAGE_NAME}.”

Also, PentAGI allows you to override the default Docker image if it doesn't find anything more suitable for the solution. To do this, you can use the environment variable DOCKER_DEFAULT_IMAGE. However, in your case, it might not be used because the task relates to information security. So unless you explicitly specify the necessary image within the task, it will default to using booyaabes/kali-linux-full.

If you manage to create a working Dockerfile to build an image with the necessary set of utilities based on kalilinux/kali-rolling, and it performs well during testing, please let me know. I'd really appreciate it, and I can base the Docker image with scripts on it.

Thank you for your feedback, it helps us improve the tool! 😉

@asdek commented on GitHub (Apr 6, 2025): Hi @vchaplygim You're absolutely right, there need to move away from using `booyaabes/kali-linux-full`. I tried quickly building an image with the necessary packages installed, but without proper configuration, not all the utilities are ready to work out of the box. I also had the idea of creating a set of pre-prepared commands wrapped in scripts to minimize the time spent on installation and debugging. This should be available within the next couple of months, maybe sooner. In the meantime, you can check out the prompt for selecting a Docker image [here](https://github.com/vxcontrol/pentagi/blob/master/backend/pkg/templates/prompts/image_chooser.tmpl). According to this, you have the ability to specify in the task description, which kicks off the flow, which Docker image the AI agents should use, with a phrase like: *“At the stage of selecting the Docker image to solve this task, you should use `{YOUR_IMAGE_NAME}`.”* Also, PentAGI allows you to override the default Docker image if it doesn't find anything more suitable for the solution. To do this, you can use the environment variable `DOCKER_DEFAULT_IMAGE`. However, in your case, it might not be used because the task relates to information security. So unless you explicitly specify the necessary image within the task, it will default to using `booyaabes/kali-linux-full`. If you manage to create a working Dockerfile to build an image with the necessary set of utilities based on `kalilinux/kali-rolling`, and it performs well during testing, please let me know. I'd really appreciate it, and I can base the Docker image with scripts on it. Thank you for your feedback, it helps us improve the tool! 😉

yindo referenced this issue

2026-06-06 22:08:01 -04:00

[GH-ISSUE #37] [Bug]: in docker log pentAGI have error: Unable to create langfuse client: failed to create langfuse client: public key is required service ssl crt and key already exist #17

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: vxcontrol/pentagi#2