[PR #13] [CLOSED] perf(startup): lazy-load LLM & RAG, defer UI imports to speed startup #15

New Issue

2026-02-27T13:59:12-05:00

yindo commented

2026-02-27 13:59:12 -05:00

📋 Pull Request Information

Original PR: https://github.com/GH05TCREW/pentestagent/pull/13
Author: @giveen
Created: 1/10/2026
Status: ❌ Closed

Base: main ← Head: patch-1

📝 Commits (6)

dd58fd6 Merge tui-cli-cleanup into fork main
de964fa mcp: make DiscoveredServer.tools Optional[List[Dict[str, Any]]] to satisfy type check
42f6b94 startup: lazy-import run_cli/run_tui to reduce startup time
dbe2b79 llm: lazy-init litellm on first use (async guard)
0734c9e rag: lazy-load serialized index and background rebuild if stale
49dde17 rag: load serialized index non-blocking (background rebuild) in CLI and TUI

📊 Changes

6 files changed (+180 additions, -37 deletions)

View changed files

📝 pentestagent/interface/cli.py (+6 -1)
📝 pentestagent/interface/main.py (+8 -2)
📝 pentestagent/interface/tui.py (+13 -4)
📝 pentestagent/knowledge/rag.py (+99 -0)
📝 pentestagent/llm/llm.py (+51 -27)
📝 pentestagent/llm/memory.py (+3 -3)

📄 Description

Summary: Reduce application cold-start time by deferring heavy initialization and running expensive work in background.
Key changes:
- Lazy-import run_cli / run_tui in pentestagent/interface/main.py so UI code is not loaded at module import.
- Lazy-initialize litellm in pentestagent/llm/llm.py (async-guarded) so model/tokenizer clients are imported only on first use.
- Add RAGEngine.load_or_create_index() and background rebuild logic in pentestagent/knowledge/rag.py; wire into CLI/TUI to load serialized index quickly and rebuild asynchronously if stale.
- Wire load_or_create_index() into CLI (pentestagent/interface/cli.py) and TUI (pentestagent/interface/tui.py).
Measured impact:
- Observed package import-time reductions for pentestagent (example runs): ~182 μs -> ~103 μs -> ~97 μs as lazy changes were applied.
- Interactive app now becomes responsive immediately; heavy indexing and model initialization occur on demand or in background threads.
Behavior notes:
- First LLM/index use may still incur initialization latency unless pre-warmed.
- Background indexing is daemonized and errors are handled to avoid blocking startup.
Tests performed:
- Ran python3 -X importtime -c "import pentestagent" and manual TUI/CLI launches to verify responsiveness.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/GH05TCREW/pentestagent/pull/13 **Author:** [@giveen](https://github.com/giveen) **Created:** 1/10/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `patch-1` --- ### 📝 Commits (6) - [`dd58fd6`](https://github.com/GH05TCREW/pentestagent/commit/dd58fd6a0e76bacd1245fddfa9b8cbdc3c1e165f) Merge tui-cli-cleanup into fork main - [`de964fa`](https://github.com/GH05TCREW/pentestagent/commit/de964fa9c82f88f9b7ce384878bdfc106484b847) mcp: make DiscoveredServer.tools Optional[List[Dict[str, Any]]] to satisfy type check - [`42f6b94`](https://github.com/GH05TCREW/pentestagent/commit/42f6b9430845ccfc788424bdd6e0866d09a4feaa) startup: lazy-import run_cli/run_tui to reduce startup time - [`dbe2b79`](https://github.com/GH05TCREW/pentestagent/commit/dbe2b7927d48e28416aa6ac4d9204aad0166b809) llm: lazy-init litellm on first use (async guard) - [`0734c9e`](https://github.com/GH05TCREW/pentestagent/commit/0734c9e39a019b93d4f2a23330a261fe3af25736) rag: lazy-load serialized index and background rebuild if stale - [`49dde17`](https://github.com/GH05TCREW/pentestagent/commit/49dde1756f2d5d6096a016275861ebe380fb09b9) rag: load serialized index non-blocking (background rebuild) in CLI and TUI ### 📊 Changes **6 files changed** (+180 additions, -37 deletions) <details> <summary>View changed files</summary> 📝 `pentestagent/interface/cli.py` (+6 -1) 📝 `pentestagent/interface/main.py` (+8 -2) 📝 `pentestagent/interface/tui.py` (+13 -4) 📝 `pentestagent/knowledge/rag.py` (+99 -0) 📝 `pentestagent/llm/llm.py` (+51 -27) 📝 `pentestagent/llm/memory.py` (+3 -3) </details> ### 📄 Description - Summary: Reduce application cold-start time by deferring heavy initialization and running expensive work in background. - Key changes: - Lazy-import run_cli / run_tui in pentestagent/interface/main.py so UI code is not loaded at module import. - Lazy-initialize litellm in pentestagent/llm/llm.py (async-guarded) so model/tokenizer clients are imported only on first use. - Add RAGEngine.load_or_create_index() and background rebuild logic in pentestagent/knowledge/rag.py; wire into CLI/TUI to load serialized index quickly and rebuild asynchronously if stale. - Wire load_or_create_index() into CLI (pentestagent/interface/cli.py) and TUI (pentestagent/interface/tui.py). - Measured impact: - Observed package import-time reductions for pentestagent (example runs): ~182 μs -> ~103 μs -> ~97 μs as lazy changes were applied. - Interactive app now becomes responsive immediately; heavy indexing and model initialization occur on demand or in background threads. - Behavior notes: - First LLM/index use may still incur initialization latency unless pre-warmed. - Background indexing is daemonized and errors are handled to avoid blocking startup. - Tests performed: - Ran python3 -X importtime -c "import pentestagent" and manual TUI/CLI launches to verify responsiveness. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

yindo added the pull-request label 2026-02-27 13:59:12 -05:00

yindo closed this issue

2026-02-27 13:59:13 -05:00

yindo referenced this issue

2026-02-27 13:59:23 -05:00

[PR #15] [CLOSED] More tui tweaks #20

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: yindo/pentestagent#15