KeygraphHQ/validation-benchmarks
1
0 Fork 0
mirror of https://github.com/KeygraphHQ/validation-benchmarks.git synced 2026-11-12 13:04:23 -10:00
Code Issues Packages Projects Releases Wiki Activity
24 Commits 2 Branches 0 Tags
main
Commit Graph
24 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
ajmallesh 6522f73bfa Merge pull request #1 from KeygraphHQ/removing-hints 
Removing hints
 2025-10-30 23:39:03 -07:00
ajmallesh 6b38ee518e cleanup 2025-10-30 23:38:25 -07:00
ajmallesh 243198d8e5 Update PENTEST_ORCHESTRATION.md to modify Makefile inclusion path and set environment variable for maximum output tokens 2025-10-30 23:36:41 -07:00
ajmallesh 3e373aa54a Update PENTEST_ORCHESTRATION.md 2025-10-27 10:23:26 -07:00
ajmalleshandClaude f780fb2393 fix: resolve build failures for 14 benchmarks 
Fixed build issues introduced during hint removal process:

Build Infrastructure Fixes:
- XBEN-012-24, 013-24, 018-24: Updated Debian repository URLs
  (buster is EOL, using archive.debian.org)
- XBEN-084-24: Added missing 'default' network definition in docker-compose.yml

Dockerfile Path Fixes (caused by directory/file renames during hint removal):
- XBEN-051-24: Fixed CSS paths after xss → styles directory rename
- XBEN-059-24: Fixed CSS paths after xss → challenge directory rename
- XBEN-074-24: Fixed CSS paths after xss → pages directory rename
- XBEN-087-24: Fixed CSS paths after xss → styles directory rename
- XBEN-089-24: Fixed app directory paths
- XBEN-090-24: Fixed app directory paths
- XBEN-091-24: Fixed app directory paths
- XBEN-093-24: Fixed app directory paths
- XBEN-094-24: Fixed app directory paths
- XBEN-100-24: Fixed app directory paths

Added Build Testing Tools:
- test_all_builds.sh: Parallelized script to test all 104 benchmarks
- test_failed_builds.sh: Iterative testing for failed builds only
- Updated .gitignore to exclude build artifacts

Result: All 104 benchmarks now build successfully ✓

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
 2025-10-24 17:16:58 -07:00
ajmalleshandClaude 397b76c255 chore: remove hints from XBEN-097 through XBEN-104 (FINAL BATCH) 
Remove all vulnerability hints from final 8 benchmarks (097-104):

XBEN-097 (Path Traversal):
- Removed 6 CTF references from content text

XBEN-098 (LFI/Path Traversal):
- Removed 6 CTF references from content text
- Removed revealing .htaccess comment about xbow.txt

XBEN-099 (IDOR):
- Renamed database from getthefirst to appdb
- Removed 2 hint response fields revealing target proximity
- Changed app name from "GetTheFirst" to "User Portal"

XBEN-100 (NoSQL Injection/GraphQL):
- Renamed database from viewtheadmins to appdb
- Changed page title from "View The Admins" to "User List"
- Removed fetch comment revealing default behavior

XBEN-101 (Padding Oracle):
- Removed DEBUG print statement
- Removed 6 cryptography-related comments and imports
- Removed padding mechanism hint comments

XBEN-102 (Business Logic):
- Removed CRITICAL HTML comment revealing test:test credentials
- Removed CRITICAL vulnerability comments showing vuln vs correct way
- Removed 10+ descriptive debug and implementation comments

XBEN-103 (AES-CBC):
- Removed HTML comment revealing test1:test1 credentials
- Removed padding implementation comments revealing CBC mechanism

XBEN-104 (SSTI):
- Renamed ssti_loan_calculator to loan_calculator throughout
- Removed "(vulnerable to SSTI)" comment from routes.py
- Removed HTML comment revealing test:test credentials
- Updated all Docker, Apache, MySQL, and WSGI references

All benchmark.json files sanitized to generic format.

This completes hint removal for all 104 XBEN benchmarks.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
 2025-10-24 16:44:30 -07:00
ajmalleshandClaude 59718cf935 chore: remove hints from XBEN-085 through XBEN-096 
Remove all vulnerability hints from benchmarks 085-096 including:

XBEN-085 (Auth Bypass):
- Changed page title from "AUTH 03" to "Springfield Login"
- Removed 6 descriptive comments about IP restrictions
- Neutralized success/error messages

XBEN-086 (SSTI):
- Changed app name from "RepeatMePlease" to "Text Generator"
- Neutralized error messages about special characters

XBEN-087 (XSS):
- Renamed /xss/ directory to /styles/
- Renamed xss*.css to level*.css
- Renamed check_xss() to check_page()
- Removed README with XSS documentation

XBEN-088 (TOCTOU/Race Condition):
- Renamed toctou_admin_router to admin_router
- Removed 10+ descriptive comments
- Updated all Docker, Apache, MySQL references

XBEN-089 (Privilege Escalation):
- Removed comments about SQL injection filtering
- Removed descriptive metadata about 2FA bypass

XBEN-090 (IDOR):
- Removed critical commented-out authorization check
- Removed IDOR vulnerability description

XBEN-091 (JWT):
- Changed /gettheflag route to /dashboard
- Removed JWT manipulation hints from metadata

XBEN-092 (PHP Deserialization):
- Renamed sstitwig2 to webapp
- Renamed skumodel to service
- Removed 15+ TODO and security policy comments

XBEN-093 (SSTI):
- Removed CSRF, user creation, and flag handling comments

XBEN-094 (Command Injection):
- Removed "Vulnerable to OS Command Injection" comment

XBEN-095 (Blind SQLi):
- Renamed ctf database to appdb
- Renamed ctfuser to appuser
- Removed filter and implementation comments

XBEN-096 (XXE):
- Removed XML external entities comment
- Removed book XML example comment

All benchmark.json files sanitized to generic format.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
 2025-10-24 16:39:51 -07:00
ajmalleshandClaude 6f39ff1ed7 chore: remove hints from XBEN-073 through XBEN-084 
Remove all vulnerability hints from benchmarks 073-084 including:

XBEN-073 (Command Injection):
- Removed command injection tags and descriptive metadata
- Removed vulnerability explanation comment from app.py

XBEN-074 (XSS):
- Deleted README.md with XSS documentation
- Renamed /xss/ directory to /pages/
- Renamed xss*.css files to page*.css
- Renamed check_xss() to check_challenge()
- Removed XSS filtering hint comments

XBEN-075 (Insecure Deserialization):
- Renamed deserialization_yaml_bookmarks to bookmarks
- Removed 3 major deserialization vulnerability comments
- Updated all Docker, Apache, and WSGI references

XBEN-076 (SSTI):
- Renamed /ssti-demo-app/ to /app/
- Changed application title from "SSTI Demo" to "Greeting App"

XBEN-077 (XXE):
- Deleted entire /hint endpoint and hint.html template
- Removed list_directory() function
- Removed XXE explanation from metadata

XBEN-078 (SQLi):
- Renamed admins table to users
- Removed injection point comment
- Removed 7 SQL schema hint comments

XBEN-079 (LFI):
- Sanitized ScriptKiddie.php blog post mentioning LFI
- Removed detailed LFI vulnerability explanations

XBEN-080 (SSTI):
- Removed SSTI tags and detailed metadata

XBEN-081 (PHP Deserialization):
- Changed database entries from "Flag" to "Secret"
- Removed vulnerable PHP version hint from Dockerfile

XBEN-082 (Command Injection):
- Removed challenge goal description from index.html
- Removed eval security warning comment
- Removed HTML comments revealing API endpoints

XBEN-083 (SQLi):
- Changed "Secret" category to "Hidden"
- Changed "SECRET POST" to "PRIVATE POST"

XBEN-084 (JWT/Privilege Escalation):
- Removed hidden directories hint from description
- Changed error message to generic "Access denied."

All benchmark.json files sanitized to generic format.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
 2025-10-24 16:33:28 -07:00
ajmalleshandClaude d3567fd209 chore: remove hints from XBEN-061 through XBEN-072 
Remove all vulnerability hints from benchmarks 061-072 including:

XBEN-061 (LFI):
- Removed descriptive name and LFI tags from metadata

XBEN-062 (XSS):
- Renamed /xss/ directory to /challenges/
- Renamed xss*.css files to challenge*.css
- Renamed XSS_CONTENT variables to CHALLENGE_CONTENT
- Renamed check_xss() function to check_solution()
- Deleted README.md with explicit XSS documentation

XBEN-063 (SSTI):
- Renamed ssti_encoding_loan_calculator to loan_calculator
- Removed SSTI vulnerability hint comments from routes.py
- Updated all Docker, Apache, and database references

XBEN-064-065 (XSS):
- Renamed /xss/ directories to /challenge/
- Renamed all XSS constants to generic CONTENT variables
- Removed XSS filtering hint comments
- Sanitized README files

XBEN-066 (HTTP Request Smuggling):
- Renamed hrs_admin_router to admin_router
- Updated all container, file, and configuration references

XBEN-067 (File Upload):
- Renamed /challenge/ directory to /app/
- Removed hints from Dockerfile comments

XBEN-068 (Command Injection):
- Removed command injection tags and descriptive metadata

XBEN-069 (IDOR):
- Renamed idor_eats database to app_db
- Changed IDOR Eats application name to Food App
- Updated all email domains and references
- Renamed ctf-flag element to admin-flag

XBEN-070 (XSS):
- Extensive XSS constant renaming (XSS_CONTENT → CHALLENGE_CONTENT)
- Renamed /xss/ directory to /challenges/
- Removed XSS filtering comments and README documentation

XBEN-071 (SQLi):
- Removed 23 Spanish comments from PHP code
- Removed SQL backup comments

XBEN-072 (Auth Bypass):
- Removed 36 CSS hint comments
- Changed page title from AUTH 02 to Welcome

All benchmark.json files sanitized to generic format.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
 2025-10-24 16:27:11 -07:00
ajmalleshandClaude e17ee5263a chore: remove hints from XBEN-049 through XBEN-060 
Remove all vulnerability hints from benchmarks 049-060 including:

XBEN-049-051 (XSS):
- Renamed /xss/ directories to /styles/ and /challenge/
- Renamed xss*.css files to page*.css
- Renamed functions (check_xss → check_input, xss* → page*)
- Removed XSS filtering hint comments
- Updated IMAGE_ROUTES keys to generic level names

XBEN-052-054 (Auth/SSTI):
- Changed auth challenge titles to generic names
- Removed explicit SSTI exploitation hints from HTML
- Removed vulnerability-revealing tags (http_method_tamper, ssti)

XBEN-055-057 (Auth/SSTI/Deserialization):
- Renamed ssti_secret_art_gallery → art_gallery
- Removed major SSTI vulnerability explanation comments
- Renamed deserialization_pickle_bookmarks → bookmarks
- Removed pickle deserialization hint comments

XBEN-058-060 (IDOR/XSS/Auth):
- Removed demo credentials hint
- Extensive XSS constant renaming (XSS_CONTENT → PAGE_CONTENT)
- Removed critical MD5 hash disclosure from error messages

All benchmark.json files sanitized to generic format.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
 2025-10-24 16:14:24 -07:00
ajmalleshandClaude 51a7e2e2fb feat: remove hints from XBEN-037 through XBEN-048 
Comprehensive hint removal for 12 additional benchmarks while maintaining
full functionality and application behavior.

Changes:
- Sanitized 12 benchmark.json files (removed tags, levels, descriptions)
- Modified 17 code files to remove vulnerability-revealing content
- Renamed 1 file to remove vulnerability reference

Metadata Changes (All 12 benchmarks):
- Removed descriptive names (e.g., "Easy login", "Simple GraphQL", "Cross 20")
- Replaced with generic format: "XBEN-XXX-24"
- Removed all vulnerability tags (command_injection, sqli, xss, graphql, idor, etc.)
- Removed difficulty levels (1, 2)
- Removed hints from descriptions (exploit methods, flag locations)

Code Changes (XBEN-037, 039, 045):
- Removed "# Filtered command injection" comments
- Removed "#vuln" marker on vulnerable code line
- Removed commented-out secure code showing the fix

Code Changes (XBEN-040, 041, 042):
- Removed "// Injection point" comment
- Removed description hints about environment variables
- Removed description revealing hardcoded credentials issue

Code Changes (XBEN-043, 044, 045):
- Renamed idor_broken_authn_trading_platform → trading_platform
- Changed container name: ssti-twig → web-app
- Removed IDOR vulnerability comments from routes.py
- Updated all configuration files (Dockerfile, httpd.conf, docker-compose.yml)

Code Changes (XBEN-046, 047, 048):
- Removed 3 README.md sections explaining XSS challenges
- Removed difficulty comments (EASY, EASY_MED, MEDIUM)
- Removed 11 inline comments explaining filtering logic:
  - script/img/input/a/div/iframe tag removal comments
  - URL regex filtering comments
  - Blacklist explanation comments
- Removed example XSS payloads from documentation

Progress: 48/104 benchmarks completed (46.2%)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
 2025-10-24 16:03:54 -07:00
ajmalleshandClaude b44e3d797d feat: remove hints from XBEN-025 through XBEN-036 
Comprehensive hint removal for 12 additional benchmarks while maintaining
full functionality and application behavior.

Changes:
- Sanitized 12 benchmark.json files (removed tags, levels, descriptions)
- Modified 29 code files to remove vulnerability-revealing content
- Renamed 5+ files/directories to remove vulnerability references

Metadata Changes (All 12 benchmarks):
- Removed descriptive names (e.g., "Art Gallery", "OGNL fun", "Green pickle")
- Replaced with generic format: "XBEN-XXX-24"
- Removed all vulnerability tags (ssti, idor, lfi, sqli, xxe, ssrf, cve, etc.)
- Removed difficulty levels (1, 2)
- Removed hints from descriptions (flag paths, default credentials, exploit methods)

Code Changes (XBEN-025, 027):
- Renamed ssti_debug_art_gallery → art_gallery
- Renamed idor_json_cookie_trading_platform → trading_platform
- Changed MySQL passwords to generic names
- Removed multi-line comments explaining SSTI and IDOR vulnerabilities
- Updated all Python module references and path references

Code Changes (XBEN-028, 029, 030):
- Replaced "LFI" mentions with "vulnerability" in content
- Renamed payroll_db → app_db
- Renamed payroll_db.sql → init_db.sql
- Changed WordPress blogname from "RCE" → "Blog"
- Sanitized error messages and upload notifications

Code Changes (XBEN-031, 032, 033):
- Removed flag injection comments
- Removed Dockerfile comments revealing benchmark context
- Removed descriptions with default credentials

Code Changes (XBEN-034, 035, 036):
- Removed docker-compose exploit mechanism comment
- Removed OGNL injection vulnerability comment
- Renamed pickle_file → data_file throughout XBEN-036
- Changed UI text: "Pickle CTF" → "Web Challenge"
- Removed code snippet hints from HTML templates

Progress: 36/104 benchmarks completed (34.6%)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
 2025-10-24 15:52:33 -07:00
ajmalleshandClaude 14294b0003 feat: remove hints from XBEN-001 through XBEN-024 
Comprehensive hint removal to prevent vulnerability type disclosure while
maintaining full build functionality and application behavior.

Changes:
- Deleted 48 metadata files (24 README.md, 24 benchmark.yaml)
- Sanitized 24 benchmark.json files (removed name hints, descriptions, tags, levels)
- Modified 150+ code files to remove vulnerability-revealing content
- Renamed 15+ files/directories to remove vulnerability type references

Metadata Changes:
- Removed all descriptive challenge names (e.g., "Basic IDOR", "Cross 25", "PHP SSRF")
- Replaced with generic format: "XBEN-XXX-24"
- Removed all tags revealing vulnerability types (idor, xss, sqli, ssrf, ssti, lfi, etc.)
- Removed difficulty level indicators
- Kept required fields: canaries, win_condition

Code Changes (XBEN-001-003):
- Renamed idor_broken_authz_trading_platform → trading_platform_db
- Renamed idor_indirect_trading_platform → trading_platform
- Renamed idor_custom_authn_header_trading_platform → trading_platform
- Removed 2 multi-line comments explaining IDOR vulnerabilities
- Updated all database passwords, connection strings, and path references

Code Changes (XBEN-004-018):
- Renamed all XSS_CONTENT* variables → PAGE_CONTENT* or TEMPLATE_CONTENT*
- Changed check_xss() → check_solution() or check_result()
- Renamed /css/xss/ directories → /css/templates/ or /css/challenges/
- Renamed 21 CSS files: xss*.css → page*.css or template*.css
- Removed X-XSS-Protection header hints
- Changed is_admin → level (privilege escalation hint removal)
- Removed difficulty level comments (EASY, MEDIUM, HARD)
- Sanitized error messages to be less revealing

Code Changes (XBEN-019-024):
- Renamed lfi_static_resource → webapp
- Renamed ssrf-demo-app → web-app
- Renamed ssti_include_loan_calculator → loan_calculator
- Renamed ssti_blind_loan_calculator → loan_calculator
- Changed page titles: "SSRF Demo" → "Company Portal", "AUTH 04" → "Springfield Login"
- Removed major vulnerability explanation comments (SSTI, LFI, SSRF)
- Removed 44-line API documentation revealing hidden endpoints
- Removed credential hints (TODO comments with test:test)

Build Verification:
- 21/24 benchmarks build successfully
- 3 failures (XBEN-012, 013, 018) due to pre-existing Debian Buster EOL issue
- All functional code preserved, only hints removed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
 2025-10-24 15:40:14 -07:00
Khaushik-keygraph 27ec483e5e chore: added .env 2025-10-14 20:52:57 +05:30
Khaushik-keygraph b501136f6f chore: fixed dockerfile 2025-10-14 16:19:04 +05:30
Khaushik-keygraph a5f3273534 chore: fixed phantomjs error 2025-10-14 15:53:49 +05:30
Khaushik-keygraph 0e2db52fc0 chore: fixed dockerfile 2025-10-14 15:02:05 +05:30
Khaushik-keygraph 388d1b399d chore: changed ports 2025-10-14 14:54:57 +05:30
Khaushik-keygraph bdc3f15d57 chore: changed port 2025-10-14 14:51:14 +05:30
Khaushik-keygraph 7cfb03bd62 chore: changed the port 2025-10-14 14:49:29 +05:30
Khaushik-keygraph ff3952e286 chore: added automated script for launching docker contianers 2025-10-13 23:28:08 +05:30
Khaushik-keygraph 800bc7a22a chore: updated dockerfiles 2025-10-13 22:56:31 +05:30
Khaushik-keygraph 41d2d59ef9 fixed docker files 2025-10-08 18:36:42 +05:30
Khaushik-keygraph 86e9e7d01d first commit 2025-10-08 13:27:20 +05:30