Implementation for an Issue #1895

Minor update
Fixes #1664
2026-10-12 02:22:55 +00:00 · 2017-04-06 11:33:59 +02:00 · 2017-03-30 12:05:05 +02:00 · 2017-03-30 11:58:03 +02:00 · 2017-03-30 11:42:34 +02:00 · 2017-03-30 10:24:57 +02:00
347 changed files with 1498 additions and 1105 deletions
@@ -2,4 +2,5 @@
 output/
 .sqlmap_history
 traffic.txt
-*~
+*~
+.idea/
@@ -18,7 +18,7 @@ You can download the latest tarball by clicking [here](https://github.com/sqlmap

 Preferably, you can download sqlmap by cloning the [Git](https://github.com/sqlmapproject/sqlmap) repository:

-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev

 sqlmap works out of the box with [Python](http://www.python.org/download/) version **2.6.x** and **2.7.x** on any platform.

@@ -1,7 +1,7 @@
 COPYING -- Describes the terms under which sqlmap is distributed. A copy
 of the GNU General Public License (GPL) is appended to this file.

-sqlmap is (C) 2006-2016 Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar.
+sqlmap is (C) 2006-2017 Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar.

 This program is free software; you may redistribute and/or modify it under
 the terms of the GNU General Public License as published by the Free
@@ -361,7 +361,6 @@ This license does not apply to the following components:
 * The MultipartPost library located under thirdparty/multipartpost/.
 * The Odict library located under thirdparty/odict/.
 * The Oset library located under thirdparty/oset/.
-* The PageRank library located under thirdparty/pagerank/.
 * The PrettyPrint library located under thirdparty/prettyprint/.
 * The PyDes library located under thirdparty/pydes/.
 * The SocksiPy library located under thirdparty/socks/.
@@ -281,8 +281,6 @@ be bound by the terms and conditions of this License Agreement.

 * The bottle web framework library located under thirdparty/bottle/.
  Copyright (C) 2012, Marcel Hellkamp.
-* The PageRank library located under thirdparty/pagerank/.
-  Copyright (C) 2010, Corey Goldberg.
 * The Termcolor library located under thirdparty/termcolor/.
  Copyright (C) 2008-2011, Volvox Development Team.

@@ -17,7 +17,7 @@ Se puede descargar el "tarball" más actual haciendo clic [aquí](https://github

 Preferentemente, se puede descargar sqlmap clonando el repositorio [Git](https://github.com/sqlmapproject/sqlmap):

-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev

 sqlmap funciona con las siguientes versiones de [Python](http://www.python.org/download/) ** 2.6.x** y ** 2.7.x** en cualquier plataforma.

@@ -26,7 +26,7 @@ Uso

 Para obtener una lista de opciones básicas: 

-    python sqlmap.py -h 
+    python sqlmap.py -h

 Para obtener una lista de todas las opciones:

@@ -17,7 +17,7 @@ Vous pouvez télécharger le plus récent fichier tarball en cliquant [ici](http

 De préférence, télécharger __sqlmap__ en le [clonant](https://github.com/sqlmapproject/sqlmap):

-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev

 sqlmap fonctionne sur n'importe quel système d'exploitation avec la version **2.6.x** et **2.7.x** de [Python](http://www.python.org/download/)  

@@ -18,7 +18,7 @@

 Κατά προτίμηση, μπορείτε να κατεβάσετε το sqlmap κάνοντας κλώνο το [Git](https://github.com/sqlmapproject/sqlmap) αποθετήριο:

-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev

 Το sqlmap λειτουργεί χωρίς περαιτέρω κόπο με την [Python](http://www.python.org/download/) έκδοσης **2.6.x** και **2.7.x** σε όποια πλατφόρμα.

@@ -18,7 +18,7 @@ Možete preuzeti zadnji tarball klikom [ovdje](https://github.com/sqlmapproject/

 Po mogućnosti, možete preuzeti sqlmap kloniranjem [Git](https://github.com/sqlmapproject/sqlmap) repozitorija:

-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev

 sqlmap radi bez posebnih zahtjeva korištenjem [Python](http://www.python.org/download/) verzije **2.6.x** i/ili **2.7.x** na bilo kojoj platformi.

@@ -19,7 +19,7 @@ Anda dapat mengunduh tarball versi terbaru [di sini]

 Sebagai alternatif, Anda dapat mengunduh sqlmap dengan men-_clone_ repositori [Git](https://github.com/sqlmapproject/sqlmap):

-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev

 sqlmap berfungsi langsung pada [Python](http://www.python.org/download/) versi **2.6.x** dan **2.7.x** pada platform apapun.

@@ -18,7 +18,7 @@ Puoi scaricare l'ultima tarball cliccando [qui](https://github.com/sqlmapproject

 La cosa migliore sarebbe però scaricare sqlmap clonando la repository [Git](https://github.com/sqlmapproject/sqlmap):

-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev

 sqlmap è in grado di funzionare con le versioni **2.6.x** e **2.7.x** di [Python](http://www.python.org/download/) su ogni piattaforma.

@@ -19,7 +19,7 @@ wikiに載っているいくつかの機能のデモをスクリーンショッ

 [Git](https://github.com/sqlmapproject/sqlmap) レポジトリをクローンして、sqlmapをダウンロードすることも可能です。:

-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev

 sqlmapは、 [Python](http://www.python.org/download/) バージョン **2.6.x** または **2.7.x** がインストールされていれば、全てのプラットフォームですぐに使用できます。

@@ -19,7 +19,7 @@ Você pode baixar o arquivo tar mais recente clicando [aqui]

 De preferência, você pode baixar o sqlmap clonando o repositório [Git](https://github.com/sqlmapproject/sqlmap):

-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev

 sqlmap funciona em [Python](http://www.python.org/download/) nas versões **2.6.x** e **2.7.x** em todas as plataformas.

@@ -21,7 +21,7 @@ Kurulum

 Veya tercihen, [Git](https://github.com/sqlmapproject/sqlmap) reposunu klonlayarak indirebilirsiniz

-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev

 sqlmap [Python](http://www.python.org/download/) sitesinde bulunan **2.6.x** and **2.7.x** versiyonları ile bütün platformlarda çalışabilmektedir.

@@ -18,7 +18,7 @@ sqlmap 是一个开源的渗透测试工具，可以用来自动化的检测，

 推荐你从 [Git](https://github.com/sqlmapproject/sqlmap) 仓库获取最新的源代码:

-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev

 sqlmap 可以运行在 [Python](http://www.python.org/download/)  **2.6.x**  和  **2.7.x** 版本的任何平台上

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -3,7 +3,7 @@
 """
 beep.py - Make a beep sound

-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -3,7 +3,7 @@
 """
 cloak.py - Simple file encryption/compression utility

-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -3,7 +3,7 @@
 """
 dbgtool.py - Portable executable to ASCII debug script converter

-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -3,7 +3,7 @@
 """
 safe2bin.py - Simple safe(hex) to binary format converter

-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,6 +1,6 @@
 #!/usr/bin/env python

-# Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 # See the file 'doc/COPYING' for copying permission

 # Removes duplicate entries in wordlist like files
@@ -13,7 +13,7 @@ then
    NEW_TAG=$(python -c "import re, sys, time; version = re.search('\"([0-9.]*)\"', sys.argv[1]).group(1); _ = version.split('.'); print '.'.join(_[:-1]) if len(_) == 4 and _[-1] == '0' else ''" "$LINE")
    if [ -n "$NEW_TAG" ]
    then
-        git commit -am "Automatic monthly tagging"
+        #git commit -am "Automatic monthly tagging"
        echo "Creating new tag ${NEW_TAG}"
        git tag $NEW_TAG
        git push origin $NEW_TAG
@@ -10,6 +10,8 @@ PROJECT_FULLPATH=${SCRIPTPATH%/*}/$PROJECT
 SETTINGS_FULLPATH=${SCRIPTPATH%/*}/$SETTINGS
 CHECKSUM_FULLPATH=${SCRIPTPATH%/*}/$CHECKSUM

+git diff $SETTINGS_FULLPATH | grep "VERSION =" > /dev/null && exit 0
+
 if [ -f $SETTINGS_FULLPATH ]
 then
    LINE=$(grep -o ${SETTINGS_FULLPATH} -e 'VERSION = "[0-9.]*"')
@@ -11,7 +11,7 @@ cat > $TMP_DIR/setup.py << EOF
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -55,7 +55,7 @@ cat > sqlmap/__init__.py << EOF
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -107,7 +107,7 @@ If you prefer fetching daily updates, you can download sqlmap by cloning the

 ::

-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev

 sqlmap works out of the box with
 `Python <http://www.python.org/download/>`__ version **2.6.x** and
@@ -1,6 +1,6 @@
 #!/usr/bin/env python

-# Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 # See the file 'doc/COPYING' for copying permission

 import codecs
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -10,10 +10,9 @@ import httplib
 import random
 import re
 import socket
+import subprocess
 import time

-from subprocess import Popen as execute
-
 from extra.beep.beep import beep
 from lib.core.agent import agent
 from lib.core.common import Backend
@@ -21,6 +20,7 @@ from lib.core.common import extractRegexResult
 from lib.core.common import extractTextTagContent
 from lib.core.common import findDynamicContent
 from lib.core.common import Format
+from lib.core.common import getFilteredPageContent
 from lib.core.common import getLastRequestHTTPError
 from lib.core.common import getPublicTypeMembers
 from lib.core.common import getSafeExString
@@ -64,6 +64,7 @@ from lib.core.exception import SqlmapConnectionException
 from lib.core.exception import SqlmapNoneDataException
 from lib.core.exception import SqlmapSilentQuitException
 from lib.core.exception import SqlmapUserQuitException
+from lib.core.settings import CANDIDATE_SENTENCE_MIN_LENGTH
 from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import DUMMY_NON_SQLI_CHECK_APPENDIX
 from lib.core.settings import FI_ERROR_REGEX
@@ -200,7 +201,7 @@ def checkSqlInjection(place, parameter, value):
            if conf.tech and isinstance(conf.tech, list) and stype not in conf.tech:
                debugMsg = "skipping test '%s' because the user " % title
                debugMsg += "specified to test only for "
-                debugMsg += "%s techniques" % " & ".join(map(lambda x: PAYLOAD.SQLINJECTION[x], conf.tech))
+                debugMsg += "%s techniques" % " & ".join(PAYLOAD.SQLINJECTION[_] for _ in conf.tech)
                logger.debug(debugMsg)
                continue

@@ -479,6 +480,26 @@ def checkSqlInjection(place, parameter, value):

                                    injectable = True

+                                elif threadData.lastComparisonRatio > UPPER_RATIO_BOUND and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
+                                    originalSet = set(getFilteredPageContent(kb.pageTemplate, True, "\n").split("\n"))
+                                    trueSet = set(getFilteredPageContent(truePage, True, "\n").split("\n"))
+                                    falseSet = set(getFilteredPageContent(falsePage, True, "\n").split("\n"))
+
+                                    if originalSet == trueSet != falseSet:
+                                        candidates = trueSet - falseSet
+
+                                        if candidates:
+                                            candidates = sorted(candidates, key=lambda _: len(_))
+                                            for candidate in candidates:
+                                                if re.match(r"\A[\w.,! ]+\Z", candidate) and ' ' in candidate and len(candidate) > CANDIDATE_SENTENCE_MIN_LENGTH:
+                                                    conf.string = candidate
+                                                    injectable = True
+
+                                                    infoMsg = "%s parameter '%s' appears to be '%s' injectable (with --string=\"%s\")" % (paramType, parameter, title, repr(conf.string).lstrip('u').strip("'"))
+                                                    logger.info(infoMsg)
+
+                                                    break
+
                            if injectable:
                                if kb.pageStable and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
                                    if all((falseCode, trueCode)) and falseCode != trueCode:
@@ -651,20 +672,20 @@ def checkSqlInjection(place, parameter, value):

                        # Feed with test details every time a test is successful
                        if hasattr(test, "details"):
-                            for dKey, dValue in test.details.items():
-                                if dKey == "dbms":
-                                    injection.dbms = dValue
+                            for key, value in test.details.items():
+                                if key == "dbms":
+                                    injection.dbms = value

-                                    if not isinstance(dValue, list):
-                                        Backend.setDbms(dValue)
+                                    if not isinstance(value, list):
+                                        Backend.setDbms(value)
                                    else:
-                                        Backend.forceDbms(dValue[0], True)
+                                        Backend.forceDbms(value[0], True)

-                                elif dKey == "dbms_version" and injection.dbms_version is None and not conf.testFilter:
-                                    injection.dbms_version = Backend.setVersion(dValue)
+                                elif key == "dbms_version" and injection.dbms_version is None and not conf.testFilter:
+                                    injection.dbms_version = Backend.setVersion(value)

-                                elif dKey == "os" and injection.os is None:
-                                    injection.os = Backend.setOs(dValue)
+                                elif key == "os" and injection.os is None:
+                                    injection.os = Backend.setOs(value)

                        if vector is None and "vector" in test and test.vector is not None:
                            vector = test.vector
@@ -696,7 +717,7 @@ def checkSqlInjection(place, parameter, value):
                                infoMsg = "executing alerting shell command(s) ('%s')" % conf.alert
                                logger.info(infoMsg)

-                                process = execute(conf.alert, shell=True)
+                                process = subprocess.Popen(conf.alert, shell=True)
                                process.wait()

                            kb.alerted = True
@@ -921,8 +942,10 @@ def heuristicCheckSqlInjection(place, parameter):

    origValue = conf.paramDict[place][parameter]
    paramType = conf.method if conf.method not in (None, HTTPMETHOD.GET, HTTPMETHOD.POST) else place
+
    prefix = ""
    suffix = ""
+    randStr = ""

    if conf.prefix or conf.suffix:
        if conf.prefix:
@@ -931,8 +954,6 @@ def heuristicCheckSqlInjection(place, parameter):
        if conf.suffix:
            suffix = conf.suffix

-    randStr = ""
-
    while randStr.count('\'') != 1 or randStr.count('\"') != 1:
        randStr = randomStr(length=10, alphabet=HEURISTIC_CHECK_ALPHABET)

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -65,7 +65,6 @@ from lib.core.settings import REFERER_ALIASES
 from lib.core.settings import USER_AGENT_ALIASES
 from lib.core.target import initTargetEnv
 from lib.core.target import setupTargetEnv
-from thirdparty.pagerank.pagerank import get_pagerank

 def _selectInjection():
    """
@@ -163,9 +162,10 @@ def _showInjections():
        header = "sqlmap resumed the following injection point(s) from stored session"

    if hasattr(conf, "api"):
+        conf.dumper.string("", {"url": conf.url, "query": conf.parameters.get(PLACE.GET), "data": conf.parameters.get(PLACE.POST)}, content_type=CONTENT_TYPE.TARGET)
        conf.dumper.string("", kb.injections, content_type=CONTENT_TYPE.TECHNIQUES)
    else:
-        data = "".join(set(map(lambda x: _formatInjection(x), kb.injections))).rstrip("\n")
+        data = "".join(set(_formatInjection(_) for _ in kb.injections)).rstrip("\n")
        conf.dumper.string(header, data)

    if conf.tamper:
@@ -224,7 +224,7 @@ def _saveToResultsFile():
        return

    results = {}
-    techniques = dict(map(lambda x: (x[1], x[0]), getPublicTypeMembers(PAYLOAD.TECHNIQUE)))
+    techniques = dict((_[1], _[0]) for _ in getPublicTypeMembers(PAYLOAD.TECHNIQUE))

    for injection in kb.injections + kb.falsePositives:
        if injection.place is None or injection.parameter is None:
@@ -238,7 +238,7 @@ def _saveToResultsFile():

    for key, value in results.items():
        place, parameter, notes = key
-        line = "%s,%s,%s,%s,%s%s" % (safeCSValue(kb.originalUrls.get(conf.url) or conf.url), place, parameter, "".join(map(lambda x: techniques[x][0].upper(), sorted(value))), notes, os.linesep)
+        line = "%s,%s,%s,%s,%s%s" % (safeCSValue(kb.originalUrls.get(conf.url) or conf.url), place, parameter, "".join(techniques[_][0].upper() for _ in sorted(value)), notes, os.linesep)
        conf.resultsFP.writelines(line)

    if not results:
@@ -319,7 +319,7 @@ def start():
                if conf.forms and conf.method:
                    message = "[#%d] form:\n%s %s" % (hostCount, conf.method, targetUrl)
                else:
-                    message = "URL %d:\n%s %s%s" % (hostCount, HTTPMETHOD.GET, targetUrl, " (PageRank: %s)" % get_pagerank(targetUrl) if conf.googleDork and conf.pageRank else "")
+                    message = "URL %d:\n%s %s" % (hostCount, HTTPMETHOD.GET, targetUrl)

                if conf.cookie:
                    message += "\nCookie: %s" % conf.cookie
@@ -470,6 +470,12 @@ def start():
                            infoMsg = "skipping %s parameter '%s'" % (paramType, parameter)
                            logger.info(infoMsg)

+                        elif conf.paramExclude and (re.search(conf.paramExclude, parameter, re.I) or kb.postHint and re.search(conf.paramExclude, parameter.split(' ')[-1], re.I)):
+                            testSqlInj = False
+
+                            infoMsg = "skipping %s parameter '%s'" % (paramType, parameter)
+                            logger.info(infoMsg)
+
                        elif parameter == conf.csrfToken:
                            testSqlInj = False

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -77,11 +77,6 @@ def setHandler():
        items.insert(0, _)

    for dbms, aliases, Handler, Connector in items:
-        if conf.dbms and conf.dbms.lower() != dbms and conf.dbms.lower() not in aliases:
-            debugMsg = "skipping test for %s" % dbms
-            logger.debug(debugMsg)
-            continue
-
        handler = Handler()
        conf.dbmsConnector = Connector()

@@ -107,6 +102,8 @@ def setHandler():
                conf.dbmsHandler = max(_ for _ in items if _[0] == kb.resolutionDbms)[2]()
            else:
                conf.dbmsHandler = handler
+
+            conf.dbmsHandler._dbms = dbms
            break
        else:
            conf.dbmsConnector = None
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -529,6 +529,8 @@ class Agent(object):
        elif fieldsSelect:
            fieldsToCastStr = fieldsSelect.group(1)

+        fieldsToCastStr = fieldsToCastStr or ""
+
        # Function
        if re.search("\A\w+\(.*\)", fieldsToCastStr, re.I) or (fieldsSelectCase and "WHEN use" not in query) or fieldsSubstr:
            fieldsToCastList = [fieldsToCastStr]
@@ -1077,5 +1079,20 @@ class Agent(object):

        return query

+    def whereQuery(self, query):
+        if conf.dumpWhere and query:
+            prefix, suffix = query.split(" ORDER BY ") if " ORDER BY " in query else (query, "")
+
+            if "%s)" % conf.tbl.upper() in prefix.upper():
+                prefix = re.sub(r"(?i)%s\)" % re.escape(conf.tbl), "%s WHERE %s)" % (conf.tbl, conf.dumpWhere), prefix)
+            elif re.search(r"(?i)\bWHERE\b", prefix):
+                prefix += " AND %s" % conf.dumpWhere
+            else:
+                prefix += " WHERE %s" % conf.dumpWhere
+
+            query = "%s ORDER BY %s" % (prefix, suffix) if suffix else prefix
+
+        return query
+
 # SQL agent
 agent = Agent()
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -23,6 +23,7 @@ import random
 import re
 import socket
 import string
+import subprocess
 import sys
 import tempfile
 import time
@@ -37,8 +38,6 @@ from StringIO import StringIO
 from difflib import SequenceMatcher
 from math import sqrt
 from optparse import OptionValueError
-from subprocess import PIPE
-from subprocess import Popen as execute
 from xml.dom import minidom
 from xml.sax import parse
 from xml.sax import SAXParseException
@@ -466,6 +465,8 @@ class Backend:

        if not kb:
            pass
+        elif not kb.get("testMode") and conf.get("dbmsHandler") and getattr(conf.dbmsHandler, "_dbms", None):
+            dbms = conf.dbmsHandler._dbms
        elif Backend.getForcedDbms() is not None:
            dbms = Backend.getForcedDbms()
        elif Backend.getDbms() is not None:
@@ -516,10 +517,9 @@ class Backend:
    # Comparison methods
    @staticmethod
    def isDbms(dbms):
-        if Backend.getDbms() is not None:
-            return Backend.getDbms() == aliasToDbmsEnum(dbms)
-        else:
-            return Backend.getIdentifiedDbms() == aliasToDbmsEnum(dbms)
+        if not kb.get("testMode") and all((Backend.getDbms(), Backend.getIdentifiedDbms())) and Backend.getDbms() != Backend.getIdentifiedDbms():
+            singleTimeWarnMessage("identified ('%s') and fingerprinted ('%s') DBMSes differ. If you experience problems in enumeration phase please rerun with '--flush-session'" % (Backend.getIdentifiedDbms(), Backend.getDbms()))
+        return Backend.getIdentifiedDbms() == aliasToDbmsEnum(dbms)

    @staticmethod
    def isDbmsWithin(aliases):
@@ -590,7 +590,7 @@ def paramToDict(place, parameters=None):
                      or re.search(r'\A9{3,}', _) or re.search(r'\A-\d+\Z', _) or re.search(DUMMY_USER_INJECTION, _))\
                      and not parameter.upper().startswith(GOOGLE_ANALYTICS_COOKIE_PREFIX):
                        warnMsg = "it appears that you have provided tainted parameter values "
-                        warnMsg += "('%s') with most probably leftover " % element
+                        warnMsg += "('%s') with most likely leftover " % element
                        warnMsg += "chars/statements from manual SQL injection test(s). "
                        warnMsg += "Please, always use only valid parameter values "
                        warnMsg += "so sqlmap could be able to run properly"
@@ -614,7 +614,8 @@ def paramToDict(place, parameters=None):
                                candidates = OrderedDict()

                                def walk(head, current=None):
-                                    current = current or head
+                                    if current is None:
+                                        current = head
                                    if isListLike(current):
                                        for _ in current:
                                            walk(head, _)
@@ -622,14 +623,15 @@ def paramToDict(place, parameters=None):
                                        for key in current.keys():
                                            value = current[key]
                                            if isinstance(value, (list, tuple, set, dict)):
-                                                walk(head, value)
+                                                if value:
+                                                    walk(head, value)
                                            elif isinstance(value, (bool, int, float, basestring)):
                                                original = current[key]
                                                if isinstance(value, bool):
                                                    current[key] = "%s%s" % (str(value).lower(), BOUNDED_INJECTION_MARKER)
                                                else:
                                                    current[key] = "%s%s" % (value, BOUNDED_INJECTION_MARKER)
-                                                candidates["%s (%s)" % (parameter, key)] = json.dumps(deserialized)
+                                                candidates["%s (%s)" % (parameter, key)] = re.sub("(%s\s*=\s*)%s" % (re.escape(parameter), re.escape(testableParameters[parameter])), r"\g<1>%s" % json.dumps(deserialized), parameters)
                                                current[key] = original

                                deserialized = json.loads(testableParameters[parameter])
@@ -729,7 +731,11 @@ def getManualDirectories():

    directories = normalizePath(directories)

-    if directories:
+    if conf.webRoot:
+        directories = [conf.webRoot]
+        infoMsg = "using '%s' as web server document root" % conf.webRoot
+        logger.info(infoMsg)
+    elif directories:
        infoMsg = "retrieved the web server document root: '%s'" % directories
        logger.info(infoMsg)
    else:
@@ -1261,7 +1267,7 @@ def parseTargetDirect():
    remote = False

    for dbms in SUPPORTED_DBMS:
-        details = re.search("^(?P<dbms>%s)://(?P<credentials>(?P<user>.+?)\:(?P<pass>.*)\@)?(?P<remote>(?P<hostname>.+?)\:(?P<port>[\d]+)\/)?(?P<db>[\w\d\ \:\.\_\-\/\\\\]+?)$" % dbms, conf.direct, re.I)
+        details = re.search("^(?P<dbms>%s)://(?P<credentials>(?P<user>.+?)\:(?P<pass>.*)\@)?(?P<remote>(?P<hostname>[\w.-]+?)\:(?P<port>[\d]+)\/)?(?P<db>[\w\d\ \:\.\_\-\/\\\\]+?)$" % dbms, conf.direct, re.I)

        if details:
            conf.dbms = details.group("dbms")
@@ -1492,11 +1498,12 @@ def getLimitRange(count, plusOne=False):
    count = int(count)
    limitStart, limitStop = 1, count

-    if isinstance(conf.limitStop, int) and conf.limitStop > 0 and conf.limitStop < limitStop:
-        limitStop = conf.limitStop
+    if kb.dumpTable:
+        if isinstance(conf.limitStop, int) and conf.limitStop > 0 and conf.limitStop < limitStop:
+            limitStop = conf.limitStop

-    if isinstance(conf.limitStart, int) and conf.limitStart > 0 and conf.limitStart <= limitStop:
-        limitStart = conf.limitStart
+        if isinstance(conf.limitStart, int) and conf.limitStart > 0 and conf.limitStart <= limitStop:
+            limitStart = conf.limitStart

    retVal = xrange(limitStart, limitStop + 1) if plusOne else xrange(limitStart - 1, limitStop)

@@ -1756,7 +1763,7 @@ def safeStringFormat(format_, params):
                    break
    return retVal

-def getFilteredPageContent(page, onlyText=True):
+def getFilteredPageContent(page, onlyText=True, split=" "):
    """
    Returns filtered page content without script, style and/or comments
    or all HTML tags
@@ -1769,10 +1776,10 @@ def getFilteredPageContent(page, onlyText=True):

    # only if the page's charset has been successfully identified
    if isinstance(page, unicode):
-        retVal = re.sub(r"(?si)<script.+?</script>|<!--.+?-->|<style.+?</style>%s" % (r"|<[^>]+>|\t|\n|\r" if onlyText else ""), " ", page)
-        while retVal.find("  ") != -1:
-            retVal = retVal.replace("  ", " ")
-        retVal = htmlunescape(retVal.strip())
+        retVal = re.sub(r"(?si)<script.+?</script>|<!--.+?-->|<style.+?</style>%s" % (r"|<[^>]+>|\t|\n|\r" if onlyText else ""), split, page)
+        while retVal.find(2 * split) != -1:
+            retVal = retVal.replace(2 * split, split)
+        retVal = htmlunescape(retVal.strip().strip(split))

    return retVal

@@ -1889,7 +1896,7 @@ def getConsoleWidth(default=80):
                FNULL = open(os.devnull, 'w')
            except IOError:
                FNULL = None
-            process = execute("stty size", shell=True, stdout=PIPE, stderr=FNULL or PIPE)
+            process = subprocess.Popen("stty size", shell=True, stdout=subprocess.PIPE, stderr=FNULL or subprocess.PIPE)
            stdout, _ = process.communicate()
            items = stdout.split()

@@ -2328,7 +2335,7 @@ def wasLastResponseDBMSError():

 def wasLastResponseHTTPError():
    """
-    Returns True if the last web request resulted in an errornous HTTP code (like 500)
+    Returns True if the last web request resulted in an erroneous HTTP code (like 500)
    """

    threadData = getCurrentThreadData()
@@ -2346,7 +2353,7 @@ def wasLastResponseDelayed():
    deviation = stdev(kb.responseTimes.get(kb.responseTimeMode, []))
    threadData = getCurrentThreadData()

-    if deviation and not conf.direct:
+    if deviation and not conf.direct and not conf.disableStats:
        if len(kb.responseTimes[kb.responseTimeMode]) < MIN_TIME_RESPONSES:
            warnMsg = "time-based standard deviation method used on a model "
            warnMsg += "with less than %d response times" % MIN_TIME_RESPONSES
@@ -2366,7 +2373,10 @@ def wasLastResponseDelayed():

        return retVal
    else:
-        return (threadData.lastQueryDuration - conf.timeSec) >= 0
+        delta = threadData.lastQueryDuration - conf.timeSec
+        if Backend.getIdentifiedDbms() in (DBMS.MYSQL,):  # MySQL's SLEEP(X) lasts 0.05 seconds shorter on average
+            delta += 0.05
+        return delta >= 0

 def adjustTimeDelay(lastQueryDuration, lowerStdLimit):
    """
@@ -2605,7 +2615,7 @@ def getPublicTypeMembers(type_, onlyValues=False):
    retVal = []

    for name, value in inspect.getmembers(type_):
-        if not name.startswith('__'):
+        if not name.startswith("__"):
            if not onlyValues:
                retVal.append((name, value))
            else:
@@ -2665,7 +2675,7 @@ def extractTextTagContent(page):
        except MemoryError:
            page = page.replace(REFLECTED_VALUE_MARKER, "")

-    return filter(None, (_.group('result').strip() for _ in re.finditer(TEXT_TAG_REGEX, page)))
+    return filter(None, (_.group("result").strip() for _ in re.finditer(TEXT_TAG_REGEX, page)))

 def trimAlphaNum(value):
    """
@@ -2777,11 +2787,11 @@ def removeDynamicContent(page):
            if prefix is None and suffix is None:
                continue
            elif prefix is None:
-                page = re.sub(r'(?s)^.+%s' % re.escape(suffix), suffix.replace('\\', r'\\'), page)
+                page = re.sub(r"(?s)^.+%s" % re.escape(suffix), suffix.replace('\\', r'\\'), page)
            elif suffix is None:
-                page = re.sub(r'(?s)%s.+$' % re.escape(prefix), prefix.replace('\\', r'\\'), page)
+                page = re.sub(r"(?s)%s.+$" % re.escape(prefix), prefix.replace('\\', r'\\'), page)
            else:
-                page = re.sub(r'(?s)%s.+%s' % (re.escape(prefix), re.escape(suffix)), '%s%s' % (prefix.replace('\\', r'\\'), suffix.replace('\\', r'\\')), page)
+                page = re.sub(r"(?s)%s.+%s" % (re.escape(prefix), re.escape(suffix)), "%s%s" % (prefix.replace('\\', r'\\'), suffix.replace('\\', r'\\')), page)

    return page

@@ -3155,7 +3165,16 @@ def unhandledExceptionMessage():
    errMsg += "Operating system: %s\n" % PLATFORM
    errMsg += "Command line: %s\n" % re.sub(r".+?\bsqlmap.py\b", "sqlmap.py", getUnicode(" ".join(sys.argv), encoding=sys.stdin.encoding))
    errMsg += "Technique: %s\n" % (enumValueToNameLookup(PAYLOAD.TECHNIQUE, kb.technique) if kb.get("technique") else ("DIRECT" if conf.get("direct") else None))
-    errMsg += "Back-end DBMS: %s" % ("%s (fingerprinted)" % Backend.getDbms() if Backend.getDbms() is not None else "%s (identified)" % Backend.getIdentifiedDbms())
+    errMsg += "Back-end DBMS:"
+
+    if Backend.getDbms() is not None:
+        errMsg += " %s (fingerprinted)" % Backend.getDbms()
+
+    if Backend.getIdentifiedDbms() is not None and (Backend.getDbms() is None or Backend.getIdentifiedDbms() != Backend.getDbms()):
+        errMsg += " %s (identified)" % Backend.getIdentifiedDbms()
+
+    if not errMsg.endswith(')'):
+        errMsg += " None"

    return errMsg

@@ -3625,13 +3644,31 @@ def randomizeParameterValue(value):
    value = re.sub(r"%[0-9a-fA-F]{2}", "", value)

    for match in re.finditer('[A-Z]+', value):
-        retVal = retVal.replace(match.group(), randomStr(len(match.group())).upper())
+        while True:
+            original = match.group()
+            candidate = randomStr(len(match.group())).upper()
+            if original != candidate:
+                break
+
+        retVal = retVal.replace(original, candidate)

    for match in re.finditer('[a-z]+', value):
-        retVal = retVal.replace(match.group(), randomStr(len(match.group())).lower())
+        while True:
+            original = match.group()
+            candidate = randomStr(len(match.group())).lower()
+            if original != candidate:
+                break
+
+        retVal = retVal.replace(original, candidate)

    for match in re.finditer('[0-9]+', value):
-        retVal = retVal.replace(match.group(), str(randomInt(len(match.group()))))
+        while True:
+            original = match.group()
+            candidate = str(randomInt(len(match.group())))
+            if original != candidate:
+                break
+
+        retVal = retVal.replace(original, candidate)

    return retVal

@@ -3728,7 +3765,6 @@ def isAdminFromPrivileges(privileges):

    # In Firebird there is no specific privilege that means
    # that the user is DBA
-    # TODO: confirm
    retVal |= (Backend.isDbms(DBMS.FIREBIRD) and all(_ in privileges for _ in ("SELECT", "INSERT", "UPDATE", "DELETE", "REFERENCES", "EXECUTE")))

    return retVal
@@ -3811,7 +3847,7 @@ def findPageForms(content, url, raise_=False, addToTargets=False):
                    continue

                # flag to know if we are dealing with the same target host
-                _ = reduce(lambda x, y: x == y, map(lambda x: urlparse.urlparse(x).netloc.split(':')[0], (response.geturl(), url)))
+                _ = checkSameHost(response.geturl(), url)

                if conf.scope:
                    if not re.search(conf.scope, url, re.I):
@@ -3834,6 +3870,23 @@ def findPageForms(content, url, raise_=False, addToTargets=False):

    return retVal

+def checkSameHost(*urls):
+    """
+    Returns True if all provided urls share that same host
+
+    >>> checkSameHost('http://www.target.com/page1.php?id=1', 'http://www.target.com/images/page2.php')
+    True
+    >>> checkSameHost('http://www.target.com/page1.php?id=1', 'http://www.target2.com/images/page2.php')
+    False
+    """
+
+    if not urls:
+        return None
+    elif len(urls) == 1:
+        return True
+    else:
+        return all(urlparse.urlparse(url or "").netloc.split(':')[0] == urlparse.urlparse(urls[0] or "").netloc.split(':')[0] for url in urls[1:])
+
 def getHostHeader(url):
    """
    Returns proper Host header value for a given target URL
@@ -3903,6 +3956,13 @@ def evaluateCode(code, variables=None):
 def serializeObject(object_):
    """
    Serializes given object
+
+    >>> serializeObject([1, 2, 3, ('a', 'b')])
+    'gAJdcQEoSwFLAksDVQFhVQFihnECZS4='
+    >>> serializeObject(None)
+    'gAJOLg=='
+    >>> serializeObject('foobar')
+    'gAJVBmZvb2JhcnEBLg=='
    """

    return base64pickle(object_)
@@ -3913,6 +3973,8 @@ def unserializeObject(value):

    >>> unserializeObject(serializeObject([1, 2, 3])) == [1, 2, 3]
    True
+    >>> unserializeObject('gAJVBmZvb2JhcnEBLg==')
+    'foobar'
    """

    return base64unpickle(value) if value else None
@@ -3959,6 +4021,8 @@ def decodeHexValue(value, raw=False):

    >>> decodeHexValue('3132332031')
    u'123 1'
+    >>> decodeHexValue(['0x31', '0x32'])
+    [u'1', u'2']
    """

    retVal = value
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -150,7 +150,6 @@ FIREBIRD_PRIVS = {
    "U": "UPDATE",
    "D": "DELETE",
    "R": "REFERENCE",
-    "E": "EXECUTE",
    "X": "EXECUTE",
    "A": "ALL",
    "M": "MEMBER",
@@ -184,15 +183,15 @@ DB2_PRIVS = {
 DUMP_REPLACEMENTS = {" ": NULL, "": BLANK}

 DBMS_DICT = {
-    DBMS.MSSQL: (MSSQL_ALIASES, "python-pymssql", "http://pymssql.sourceforge.net/", "mssql+pymssql"),
+    DBMS.MSSQL: (MSSQL_ALIASES, "python-pymssql", "https://github.com/pymssql/pymssql", "mssql+pymssql"),
    DBMS.MYSQL: (MYSQL_ALIASES, "python-pymysql", "https://github.com/petehunt/PyMySQL/", "mysql"),
    DBMS.PGSQL: (PGSQL_ALIASES, "python-psycopg2", "http://initd.org/psycopg/", "postgresql"),
    DBMS.ORACLE: (ORACLE_ALIASES, "python cx_Oracle", "http://cx-oracle.sourceforge.net/", "oracle"),
    DBMS.SQLITE: (SQLITE_ALIASES, "python-sqlite", "http://packages.ubuntu.com/quantal/python-sqlite", "sqlite"),
-    DBMS.ACCESS: (ACCESS_ALIASES, "python-pyodbc", "http://pyodbc.googlecode.com/", "access"),
+    DBMS.ACCESS: (ACCESS_ALIASES, "python-pyodbc", "https://github.com/mkleehammer/pyodbc", "access"),
    DBMS.FIREBIRD: (FIREBIRD_ALIASES, "python-kinterbasdb", "http://kinterbasdb.sourceforge.net/", "firebird"),
    DBMS.MAXDB: (MAXDB_ALIASES, None, None, "maxdb"),
-    DBMS.SYBASE: (SYBASE_ALIASES, "python-pymssql", "http://pymssql.sourceforge.net/", "sybase"),
+    DBMS.SYBASE: (SYBASE_ALIASES, "python-pymssql", "https://github.com/pymssql/pymssql", "sybase"),
    DBMS.DB2: (DB2_ALIASES, "python ibm-db", "https://github.com/ibmdb/python-ibmdb", "ibm_db_sa"),
    DBMS.HSQLDB: (HSQLDB_ALIASES, "python jaydebeapi & python-jpype", "https://pypi.python.org/pypi/JayDeBeApi/ & http://jpype.sourceforge.net/", None),
    DBMS.INFORMIX: (INFORMIX_ALIASES, "python ibm-db", "https://github.com/ibmdb/python-ibmdb", "ibm_db_sa"),
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -176,6 +176,7 @@ class HTTP_HEADER:
    PROXY_CONNECTION = "Proxy-Connection"
    RANGE = "Range"
    REFERER = "Referer"
+    REFRESH = "Refresh"  # Reference: http://stackoverflow.com/a/283794
    SERVER = "Server"
    SET_COOKIE = "Set-Cookie"
    TRANSFER_ENCODING = "Transfer-Encoding"
@@ -286,31 +287,32 @@ class WEB_API:
    JSP = "jsp"

 class CONTENT_TYPE:
-    TECHNIQUES = 0
-    DBMS_FINGERPRINT = 1
-    BANNER = 2
-    CURRENT_USER = 3
-    CURRENT_DB = 4
-    HOSTNAME = 5
-    IS_DBA = 6
-    USERS = 7
-    PASSWORDS = 8
-    PRIVILEGES = 9
-    ROLES = 10
-    DBS = 11
-    TABLES = 12
-    COLUMNS = 13
-    SCHEMA = 14
-    COUNT = 15
-    DUMP_TABLE = 16
-    SEARCH = 17
-    SQL_QUERY = 18
-    COMMON_TABLES = 19
-    COMMON_COLUMNS = 20
-    FILE_READ = 21
-    FILE_WRITE = 22
-    OS_CMD = 23
-    REG_READ = 24
+    TARGET = 0
+    TECHNIQUES = 1
+    DBMS_FINGERPRINT = 2
+    BANNER = 3
+    CURRENT_USER = 4
+    CURRENT_DB = 5
+    HOSTNAME = 6
+    IS_DBA = 7
+    USERS = 8
+    PASSWORDS = 9
+    PRIVILEGES = 10
+    ROLES = 11
+    DBS = 12
+    TABLES = 13
+    COLUMNS = 14
+    SCHEMA = 15
+    COUNT = 16
+    DUMP_TABLE = 17
+    SEARCH = 18
+    SQL_QUERY = 19
+    COMMON_TABLES = 20
+    COMMON_COLUMNS = 21
+    FILE_READ = 22
+    FILE_WRITE = 23
+    OS_CMD = 24
+    REG_READ = 25

 PART_RUN_CONTENT_TYPES = {
    "checkDbms": CONTENT_TYPE.TECHNIQUES,
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -897,20 +897,25 @@ def _setTamperingFunctions():
        for script in re.split(PARAMETER_SPLITTING_REGEX, conf.tamper):
            found = False

+            path = paths.SQLMAP_TAMPER_PATH.encode(sys.getfilesystemencoding() or UNICODE_ENCODING)
            script = script.strip().encode(sys.getfilesystemencoding() or UNICODE_ENCODING)

-            if not script:
-                continue
+            try:
+                if not script:
+                    continue

-            elif os.path.exists(os.path.join(paths.SQLMAP_TAMPER_PATH, script if script.endswith(".py") else "%s.py" % script)):
-                script = os.path.join(paths.SQLMAP_TAMPER_PATH, script if script.endswith(".py") else "%s.py" % script)
+                elif os.path.exists(os.path.join(path, script if script.endswith(".py") else "%s.py" % script)):
+                    script = os.path.join(path, script if script.endswith(".py") else "%s.py" % script)

-            elif not os.path.exists(script):
-                errMsg = "tamper script '%s' does not exist" % script
-                raise SqlmapFilePathException(errMsg)
+                elif not os.path.exists(script):
+                    errMsg = "tamper script '%s' does not exist" % script
+                    raise SqlmapFilePathException(errMsg)

-            elif not script.endswith(".py"):
-                errMsg = "tamper script '%s' should have an extension '.py'" % script
+                elif not script.endswith(".py"):
+                    errMsg = "tamper script '%s' should have an extension '.py'" % script
+                    raise SqlmapSyntaxException(errMsg)
+            except UnicodeDecodeError:
+                errMsg = "invalid character provided in option '--tamper'"
                raise SqlmapSyntaxException(errMsg)

            dirname, filename = os.path.split(script)
@@ -928,7 +933,7 @@ def _setTamperingFunctions():
                sys.path.insert(0, dirname)

            try:
-                module = __import__(filename[:-3])
+                module = __import__(filename[:-3].encode(sys.getfilesystemencoding() or UNICODE_ENCODING))
            except (ImportError, SyntaxError), ex:
                raise SqlmapSyntaxException("cannot import tamper script '%s' (%s)" % (filename[:-3], getSafeExString(ex)))

@@ -1001,7 +1006,7 @@ def _setWafFunctions():
            try:
                if filename[:-3] in sys.modules:
                    del sys.modules[filename[:-3]]
-                module = __import__(filename[:-3])
+                module = __import__(filename[:-3].encode(sys.getfilesystemencoding() or UNICODE_ENCODING))
            except ImportError, msg:
                raise SqlmapSyntaxException("cannot import WAF script '%s' (%s)" % (filename[:-3], msg))

@@ -1335,17 +1340,17 @@ def _setHTTPAuthentication():
        debugMsg = "setting the HTTP authentication type and credentials"
        logger.debug(debugMsg)

-        aTypeLower = conf.authType.lower()
+        authType = conf.authType.lower()

-        if aTypeLower in (AUTH_TYPE.BASIC, AUTH_TYPE.DIGEST):
+        if authType in (AUTH_TYPE.BASIC, AUTH_TYPE.DIGEST):
            regExp = "^(.*?):(.*?)$"
-            errMsg = "HTTP %s authentication credentials " % aTypeLower
+            errMsg = "HTTP %s authentication credentials " % authType
            errMsg += "value must be in format 'username:password'"
-        elif aTypeLower == AUTH_TYPE.NTLM:
+        elif authType == AUTH_TYPE.NTLM:
            regExp = "^(.*\\\\.*):(.*?)$"
            errMsg = "HTTP NTLM authentication credentials value must "
            errMsg += "be in format 'DOMAIN\username:password'"
-        elif aTypeLower == AUTH_TYPE.PKI:
+        elif authType == AUTH_TYPE.PKI:
            errMsg = "HTTP PKI authentication require "
            errMsg += "usage of option `--auth-pki`"
            raise SqlmapSyntaxException(errMsg)
@@ -1362,13 +1367,13 @@ def _setHTTPAuthentication():

        _setAuthCred()

-        if aTypeLower == AUTH_TYPE.BASIC:
+        if authType == AUTH_TYPE.BASIC:
            authHandler = SmartHTTPBasicAuthHandler(kb.passwordMgr)

-        elif aTypeLower == AUTH_TYPE.DIGEST:
+        elif authType == AUTH_TYPE.DIGEST:
            authHandler = urllib2.HTTPDigestAuthHandler(kb.passwordMgr)

-        elif aTypeLower == AUTH_TYPE.NTLM:
+        elif authType == AUTH_TYPE.NTLM:
            try:
                from ntlm import HTTPNtlmAuthHandler
            except ImportError:
@@ -2226,10 +2231,19 @@ def _mergeOptions(inputOptions, overrideOptions):

    if inputOptions.pickledOptions:
        try:
-            inputOptions = base64unpickle(inputOptions.pickledOptions, unsafe=True)
-            if type(inputOptions) == dict:
-                inputOptions = AttribDict(inputOptions)
-            _normalizeOptions(inputOptions)
+            unpickledOptions = base64unpickle(inputOptions.pickledOptions, unsafe=True)
+
+            if type(unpickledOptions) == dict:
+                unpickledOptions = AttribDict(unpickledOptions)
+
+            _normalizeOptions(unpickledOptions)
+
+            unpickledOptions["pickledOptions"] = None
+            for key in inputOptions:
+                if key not in unpickledOptions:
+                    unpickledOptions[key] = inputOptions[key]
+
+            inputOptions = unpickledOptions
        except Exception, ex:
            errMsg = "provided invalid value '%s' for option '--pickled-options'" % inputOptions.pickledOptions
            errMsg += " (%s)" % repr(ex)
@@ -2310,7 +2324,7 @@ def _setProxyList():
        return

    conf.proxyList = []
-    for match in re.finditer(r"(?i)((http[^:]*|socks[^:]*)://)?([\w.]+):(\d+)", readCachedFileContent(conf.proxyFile)):
+    for match in re.finditer(r"(?i)((http[^:]*|socks[^:]*)://)?([\w\-.]+):(\d+)", readCachedFileContent(conf.proxyFile)):
        _, type_, address, port = match.groups()
        conf.proxyList.append("%s://%s:%s" % (type_ or "http", address, port))

@@ -2473,14 +2487,14 @@ def _basicOptionValidation():
    if conf.regexp:
        try:
            re.compile(conf.regexp)
-        except re.error, ex:
+        except Exception, ex:
            errMsg = "invalid regular expression '%s' ('%s')" % (conf.regexp, getSafeExString(ex))
            raise SqlmapSyntaxException(errMsg)

    if conf.crawlExclude:
        try:
            re.compile(conf.crawlExclude)
-        except re.error, ex:
+        except Exception, ex:
            errMsg = "invalid regular expression '%s' ('%s')" % (conf.crawlExclude, getSafeExString(ex))
            raise SqlmapSyntaxException(errMsg)

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -77,7 +77,8 @@ optDict = {
                               "testParameter":     "string",
                               "skip":              "string",
                               "skipStatic":        "boolean",
-                               "dbms":              "string",
+                               "skip":              "string",
+                               "paramExclude":      "string",
                               "dbmsCred":          "string",
                               "os":                "string",
                               "invalidBignum":     "boolean",
@@ -224,11 +225,11 @@ optDict = {
                               "identifyWaf":       "boolean",
                               "mobile":            "boolean",
                               "offline":           "boolean",
-                               "pageRank":          "boolean",
                               "purgeOutput":       "boolean",
                               "skipWaf":           "boolean",
                               "smart":             "boolean",
                               "tmpDir":            "string",
+                               "webRoot":           "string",
                               "wizard":            "boolean",
                               "verbose":           "integer",
                             },
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,15 +1,13 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

 import os
 import re
-
-from subprocess import PIPE
-from subprocess import Popen as execute
+import subprocess

 def getRevisionNumber():
    """
@@ -46,7 +44,7 @@ def getRevisionNumber():
            break

    if not retVal:
-        process = execute("git rev-parse --verify HEAD", shell=True, stdout=PIPE, stderr=PIPE)
+        process = subprocess.Popen("git rev-parse --verify HEAD", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        stdout, _ = process.communicate()
        match = re.search(r"(?i)[0-9a-f]{32}", stdout or "")
        retVal = match.group(0) if match else None
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -19,7 +19,7 @@ from lib.core.enums import DBMS_DIRECTORY_NAME
 from lib.core.enums import OS

 # sqlmap version (<major>.<minor>.<month>.<monthly commit>)
-VERSION = "1.0.12.0"
+VERSION = "1.1.4.0"
 TYPE = "dev" if VERSION.count('.') > 2 and VERSION.split('.')[-1] != '0' else "stable"
 TYPE_COLORS = {"dev": 33, "stable": 90, "pip": 34}
 VERSION_STRING = "sqlmap/%s#%s" % ('.'.join(VERSION.split('.')[:-1]) if VERSION.count('.') > 2 and VERSION.split('.')[-1] == '0' else VERSION, TYPE)
@@ -109,7 +109,7 @@ DUMMY_SEARCH_USER_AGENT = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:49.0) Geck
 TEXT_TAG_REGEX = r"(?si)<(abbr|acronym|b|blockquote|br|center|cite|code|dt|em|font|h\d|i|li|p|pre|q|strong|sub|sup|td|th|title|tt|u)(?!\w).*?>(?P<result>[^<]+)"

 # Regular expression used for recognition of IP addresses
-IP_ADDRESS_REGEX = r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b"
+IP_ADDRESS_REGEX = r"\b(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\b"

 # Regular expression used for recognition of generic "your ip has been blocked" messages
 BLOCKED_IP_REGEX = r"(?i)(\A|\b)ip\b.*\b(banned|blocked|block list|firewall)"
@@ -303,6 +303,7 @@ ERROR_PARSING_REGEXES = (
    r"(?m)^(fatal|error|warning|exception):?\s*(?P<result>[^\n]+?)$",
    r"(?P<result>[^\n>]*SQL Syntax[^\n<]+)",
    r"<li>Error Type:<br>(?P<result>.+?)</li>",
+    r"CDbCommand (?P<result>[^<>\n]*SQL[^<>\n]+)",
    r"error '[0-9a-f]{8}'((<[^>]+>)|\s)+(?P<result>[^<>]+)",
    r"\[[^\n\]]+(ODBC|JDBC)[^\n\]]+\](\[[^\]]+\])?(?P<result>[^\n]+(in query expression|\(SQL| at /[^ ]+pdo)[^\n<]+)"
 )
@@ -358,6 +359,9 @@ MIN_RATIO = 0.0
 # Maximum value for comparison ratio
 MAX_RATIO = 1.0

+# Minimum length of sentence for automatic choosing of --string (in case of high matching ratio)
+CANDIDATE_SENTENCE_MIN_LENGTH = 10
+
 # Character used for marking injectable position inside provided data
 CUSTOM_INJECTION_MARK_CHAR = '*'

@@ -527,7 +531,7 @@ UNION_CHAR_REGEX = r"\A\w+\Z"
 UNENCODED_ORIGINAL_VALUE = "original"

 # Common column names containing usernames (used for hash cracking in some cases)
-COMMON_USER_COLUMNS = ("login", "user", "username", "user_name", "user_login", "benutzername", "benutzer", "utilisateur", "usager", "consommateur", "utente", "utilizzatore", "usufrutuario", "korisnik", "usuario", "consumidor")
+COMMON_USER_COLUMNS = ("login", "user", "username", "user_name", "user_login", "benutzername", "benutzer", "utilisateur", "usager", "consommateur", "utente", "utilizzatore", "usufrutuario", "korisnik", "usuario", "consumidor", "client", "cuser")

 # Default delimiter in GET/POST values
 DEFAULT_GET_POST_DELIMITER = '&'
@@ -587,7 +591,7 @@ BANNER = re.sub(r"\[.\]", lambda _: "[\033[01;41m%s\033[01;49m]" % random.sample
 DUMMY_NON_SQLI_CHECK_APPENDIX = "<'\">"

 # Regular expression used for recognition of file inclusion errors
-FI_ERROR_REGEX = "(?i)[^\n]*(no such file|failed (to )?open)[^\n]*"
+FI_ERROR_REGEX = "(?i)[^\n]{0,100}(no such file|failed (to )?open)[^\n]{0,100}"

 # Length of prefix and suffix used in non-SQLI heuristic checks
 NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH = 6
@@ -692,7 +696,7 @@ MAX_HISTORY_LENGTH = 1000
 MIN_ENCODED_LEN_CHECK = 5

 # Timeout in seconds in which Metasploit remote session has to be initialized
-METASPLOIT_SESSION_TIMEOUT = 300
+METASPLOIT_SESSION_TIMEOUT = 120

 # Reference: http://www.postgresql.org/docs/9.0/static/catalog-pg-largeobject.html
 LOBLKSIZE = 2048
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,18 +1,17 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

 import difflib
 import random
+import thread
 import threading
 import time
 import traceback

-from thread import error as ThreadError
-
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
@@ -47,6 +46,7 @@ class _ThreadData(threading.local):
        self.lastComparisonPage = None
        self.lastComparisonHeaders = None
        self.lastComparisonCode = None
+        self.lastComparisonRatio = None
        self.lastErrorPage = None
        self.lastHTTPError = None
        self.lastRedirectMsg = None
@@ -151,7 +151,7 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio

            try:
                thread.start()
-            except ThreadError, ex:
+            except thread.error, ex:
                errMsg = "error occurred while starting new thread ('%s')" % ex.message
                logger.critical(errMsg)
                break
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,18 +1,16 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

 import locale
 import os
 import re
+import subprocess
 import time

-from subprocess import PIPE
-from subprocess import Popen as execute
-
 from lib.core.common import dataToStdout
 from lib.core.common import getSafeExString
 from lib.core.common import pollProcess
@@ -31,7 +29,7 @@ def update():

    if not os.path.exists(os.path.join(paths.SQLMAP_ROOT_PATH, ".git")):
        errMsg = "not a git repository. Please checkout the 'sqlmapproject/sqlmap' repository "
-        errMsg += "from GitHub (e.g. 'git clone https://github.com/sqlmapproject/sqlmap.git sqlmap')"
+        errMsg += "from GitHub (e.g. 'git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap')"
        logger.error(errMsg)
    else:
        infoMsg = "updating sqlmap to the latest development version from the "
@@ -44,7 +42,7 @@ def update():
        dataToStdout("\r[%s] [INFO] update in progress " % time.strftime("%X"))

        try:
-            process = execute("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=PIPE, stderr=PIPE, cwd=paths.SQLMAP_ROOT_PATH.encode(locale.getpreferredencoding()))  # Reference: http://blog.stastnarodina.com/honza-en/spot/python-unicodeencodeerror/
+            process = subprocess.Popen("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=paths.SQLMAP_ROOT_PATH.encode(locale.getpreferredencoding()))  # Reference: http://blog.stastnarodina.com/honza-en/spot/python-unicodeencodeerror/
            pollProcess(process, True)
            stdout, stderr = process.communicate()
            success = not process.returncode
@@ -57,7 +55,7 @@ def update():
        else:
            if "Not a git repository" in stderr:
                errMsg = "not a valid git repository. Please checkout the 'sqlmapproject/sqlmap' repository "
-                errMsg += "from GitHub (e.g. 'git clone https://github.com/sqlmapproject/sqlmap.git sqlmap')"
+                errMsg += "from GitHub (e.g. 'git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap')"
                logger.error(errMsg)
            else:
                logger.error("update could not be completed ('%s')" % re.sub(r"\W+", " ", stderr).strip())
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -269,6 +269,9 @@ def cmdLineParser(argv=None):
        injection.add_option("--skip-static", dest="skipStatic", action="store_true",
                             help="Skip testing parameters that not appear to be dynamic")

+        injection.add_option("--param-exclude", dest="paramExclude",
+                           help="Regexp to exclude parameters from testing (e.g. \"ses\")")
+
        injection.add_option("--dbms", dest="dbms",
                             help="Force back-end DBMS to this value")

@@ -479,10 +482,10 @@ def cmdLineParser(argv=None):
                               help="Use WHERE condition while table dumping")

        enumeration.add_option("--start", dest="limitStart", type="int",
-                               help="First query output entry to retrieve")
+                               help="First dump table entry to retrieve")

        enumeration.add_option("--stop", dest="limitStop", type="int",
-                               help="Last query output entry to retrieve")
+                               help="Last dump table entry to retrieve")

        enumeration.add_option("--first", dest="firstChar", type="int",
                               help="First query output word character to retrieve")
@@ -735,10 +738,6 @@ def cmdLineParser(argv=None):
                                  action="store_true",
                                  help="Work in offline mode (only use session data)")

-        miscellaneous.add_option("--page-rank", dest="pageRank",
-                                  action="store_true",
-                                  help="Display page rank (PR) for Google dork results")
-
        miscellaneous.add_option("--purge-output", dest="purgeOutput",
                                  action="store_true",
                                  help="Safely remove all content from output directory")
@@ -757,6 +756,9 @@ def cmdLineParser(argv=None):
        miscellaneous.add_option("--tmp-dir", dest="tmpDir",
                                  help="Local directory for storing temporary files")

+        miscellaneous.add_option("--web-root", dest="webRoot",
+                                  help="Web server document root directory (e.g. \"/var/www\")")
+
        miscellaneous.add_option("--wizard", dest="wizard",
                                  action="store_true",
                                  help="Simple wizard interface for beginner users")
@@ -774,6 +776,9 @@ def cmdLineParser(argv=None):
        parser.add_option("--disable-precon", dest="disablePrecon", action="store_true",
                          help=SUPPRESS_HELP)

+        parser.add_option("--disable-stats", dest="disableStats", action="store_true",
+                          help=SUPPRESS_HELP)
+
        parser.add_option("--profile", dest="profile", action="store_true",
                          help=SUPPRESS_HELP)

@@ -897,6 +902,9 @@ def cmdLineParser(argv=None):
            elif len(argv[i]) > 1 and all(ord(_) in xrange(0x2018, 0x2020) for _ in ((argv[i].split('=', 1)[-1].strip() or ' ')[0], argv[i][-1])):
                dataToStdout("[!] copy-pasting illegal (non-console) quote characters from Internet is, well, illegal (%s)\n" % argv[i])
                raise SystemExit
+            elif len(argv[i]) > 1 and u"\uff0c" in argv[i].split('=', 1)[-1]:
+                dataToStdout("[!] copy-pasting illegal (non-console) comma characters from Internet is, well, illegal (%s)\n" % argv[i])
+                raise SystemExit
            elif re.search(r"\A-\w=.+", argv[i]):
                dataToStdout("[!] potentially miswritten (illegal '=') short option detected ('%s')\n" % argv[i])
                raise SystemExit
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -14,13 +14,14 @@ from lib.core.common import UnicodeRawConfigParser
 from lib.core.data import cmdLineOptions
 from lib.core.data import conf
 from lib.core.data import logger
+from lib.core.enums import OPTION_TYPE
 from lib.core.exception import SqlmapMissingMandatoryOptionException
 from lib.core.exception import SqlmapSyntaxException
 from lib.core.optiondict import optDict

 config = None

-def configFileProxy(section, option, boolean=False, integer=False):
+def configFileProxy(section, option, datatype):
    """
    Parse configuration file and save settings into the configuration
    advanced dictionary.
@@ -30,10 +31,12 @@ def configFileProxy(section, option, boolean=False, integer=False):

    if config.has_option(section, option):
        try:
-            if boolean:
+            if datatype == OPTION_TYPE.BOOLEAN:
                value = config.getboolean(section, option) if config.get(section, option) else False
-            elif integer:
+            elif datatype == OPTION_TYPE.INTEGER:
                value = config.getint(section, option) if config.get(section, option) else 0
+            elif datatype == OPTION_TYPE.FLOAT:
+                value = config.getfloat(section, option) if config.get(section, option) else 0.0
            else:
                value = config.get(section, option)
        except ValueError, ex:
@@ -91,8 +94,4 @@ def configFileParser(configFile):
    for family, optionData in optDict.items():
        for option, datatype in optionData.items():
            datatype = unArrayizeValue(datatype)
-
-            boolean = datatype == "boolean"
-            integer = datatype == "integer"
-
-            configFileProxy(family, option, boolean, integer)
+            configFileProxy(family, option, datatype)
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -102,7 +102,7 @@ def forgeHeaders(items=None):
                        message = "you provided a HTTP %s header value. " % HTTP_HEADER.COOKIE
                        message += "The target URL provided its own cookies within "
                        message += "the HTTP %s header which intersect with yours. " % HTTP_HEADER.SET_COOKIE
-                        message += "Do you want to merge them in futher requests? [Y/n] "
+                        message += "Do you want to merge them in further requests? [Y/n] "
                        _ = readInput(message, default="Y")
                        kb.mergeCookies = not _ or _[0] in ("y", "Y")

@@ -168,6 +168,8 @@ def checkCharEncoding(encoding, warn=True):
        encoding = encoding.replace("8858", "8859")  # iso-8858 -> iso-8859
    elif "8559" in encoding:
        encoding = encoding.replace("8559", "8859")  # iso-8559 -> iso-8859
+    elif "8895" in encoding:
+        encoding = encoding.replace("8895", "8859")  # iso-8895 -> iso-8859
    elif "5889" in encoding:
        encoding = encoding.replace("5889", "8859")  # iso-5889 -> iso-8859
    elif "5589" in encoding:
@@ -314,6 +316,12 @@ def decodePage(page, contentEncoding, contentType):
            page = re.sub(r"&([^;]+);", lambda _: chr(htmlEntities[_.group(1)]) if htmlEntities.get(_.group(1), 256) < 256 else _.group(0), page)

            kb.pageEncoding = kb.pageEncoding or checkCharEncoding(getHeuristicCharEncoding(page))
+
+            if kb.pageEncoding and kb.pageEncoding.lower() == "utf-8-sig":
+                kb.pageEncoding = "utf-8"
+                if page and page.startswith("\xef\xbb\xbf"):  # Reference: https://docs.python.org/2/library/codecs.html (Note: noticed problems when "utf-8-sig" is left to Python for handling)
+                    page = page[3:]
+
            page = getUnicode(page, kb.pageEncoding)

            # e.g. &#8217;&#8230;&#8482;
@@ -363,7 +371,7 @@ def processResponse(page, responseHeaders):
                        if readInput(msg, default='N').strip().upper() != 'Y':
                            continue
                        conf.paramDict[PLACE.POST][name] = value
-                conf.parameters[PLACE.POST] = re.sub("(?i)(%s=)[^&]+" % name, r"\g<1>%s" % value, conf.parameters[PLACE.POST])
+                conf.parameters[PLACE.POST] = re.sub("(?i)(%s=)[^&]+" % re.escape(name), r"\g<1>%s" % re.escape(value), conf.parameters[PLACE.POST])

    if not kb.captchaDetected and re.search(r"(?i)captcha", page or ""):
        for match in re.finditer(r"(?si)<form.+?</form>", page):
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -144,6 +144,9 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
            kb.matchRatio = ratio
            logger.debug("setting match ratio for current parameter to %.3f" % kb.matchRatio)

+    if kb.testMode:
+        threadData.lastComparisonRatio = ratio
+
    # If it has been requested to return the ratio and not a comparison
    # response
    if getRatioValue:
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -31,6 +31,7 @@ from extra.safe2bin.safe2bin import safecharencode
 from lib.core.agent import agent
 from lib.core.common import asciifyUrl
 from lib.core.common import calculateDeltaSeconds
+from lib.core.common import checkSameHost
 from lib.core.common import clearConsoleLine
 from lib.core.common import dataToStdout
 from lib.core.common import evaluateCode
@@ -145,9 +146,9 @@ class Connect(object):
        if kb.testMode and kb.previousMethod == PAYLOAD.METHOD.TIME:
            # timed based payloads can cause web server unresponsiveness
            # if the injectable piece of code is some kind of JOIN-like query
-            warnMsg = "most probably web server instance hasn't recovered yet "
+            warnMsg = "most likely web server instance hasn't recovered yet "
            warnMsg += "from previous timed based payload. If the problem "
-            warnMsg += "persists please wait for few minutes and rerun "
+            warnMsg += "persists please wait for a few minutes and rerun "
            warnMsg += "without flag 'T' in option '--technique' "
            warnMsg += "(e.g. '--flush-session --technique=BEUS') or try to "
            warnMsg += "lower the value of option '--time-sec' (e.g. '--time-sec=2')"
@@ -266,7 +267,7 @@ class Connect(object):
            url = urlparse.urljoin(conf.url, url)

        # flag to know if we are dealing with the same target host
-        target = reduce(lambda x, y: x == y, map(lambda x: urlparse.urlparse(x).netloc.split(':')[0], [url, conf.url or ""]))
+        target = checkSameHost(url, conf.url)

        if not retrying:
            # Reset the number of connection retries
@@ -373,9 +374,7 @@ class Connect(object):

            # Reset header values to original in case of provided request file
            if target and conf.requestFile:
-                headers = OrderedDict(conf.httpHeaders)
-                if cookie:
-                    headers[HTTP_HEADER.COOKIE] = cookie
+                headers = forgeHeaders({HTTP_HEADER.COOKIE: cookie})

            if auxHeaders:
                for key, value in auxHeaders.items():
@@ -474,7 +473,7 @@ class Connect(object):
                    return conn, None, None

                # Get HTTP response
-                if hasattr(conn, 'redurl'):
+                if hasattr(conn, "redurl"):
                    page = (threadData.lastRedirectMsg[1] if kb.redirectChoice == REDIRECTION.NO\
                    else Connect._connReadProxy(conn)) if not skipRead else None
                    skipLogTraffic = kb.redirectChoice == REDIRECTION.NO
@@ -482,45 +481,54 @@ class Connect(object):
                else:
                    page = Connect._connReadProxy(conn) if not skipRead else None

-                code = code or conn.code
-                responseHeaders = conn.info()
-                responseHeaders[URI_HTTP_HEADER] = conn.geturl()
+                if conn:
+                    code = conn.code
+                    responseHeaders = conn.info()
+                    responseHeaders[URI_HTTP_HEADER] = conn.geturl()
+                else:
+                    code = None
+                    responseHeaders = {}
+
                page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE))
-                status = getUnicode(conn.msg)
+                status = getUnicode(conn.msg) if conn else None

            kb.connErrorCounter = 0

-            if extractRegexResult(META_REFRESH_REGEX, page) and not refreshing:
-                refresh = extractRegexResult(META_REFRESH_REGEX, page)
+            if not refreshing:
+                refresh = responseHeaders.get(HTTP_HEADER.REFRESH, "").split("url=")[-1].strip()

-                debugMsg = "got HTML meta refresh header"
-                logger.debug(debugMsg)
+                if extractRegexResult(META_REFRESH_REGEX, page):
+                    refresh = extractRegexResult(META_REFRESH_REGEX, page)

-                if kb.alwaysRefresh is None:
-                    msg = "sqlmap got a refresh request "
-                    msg += "(redirect like response common to login pages). "
-                    msg += "Do you want to apply the refresh "
-                    msg += "from now on (or stay on the original page)? [Y/n]"
-                    choice = readInput(msg, default="Y")
+                    debugMsg = "got HTML meta refresh header"
+                    logger.debug(debugMsg)

-                    kb.alwaysRefresh = choice not in ("n", "N")
+                if refresh:
+                    if kb.alwaysRefresh is None:
+                        msg = "sqlmap got a refresh request "
+                        msg += "(redirect like response common to login pages). "
+                        msg += "Do you want to apply the refresh "
+                        msg += "from now on (or stay on the original page)? [Y/n]"
+                        choice = readInput(msg, default="Y")

-                if kb.alwaysRefresh:
-                    if re.search(r"\Ahttps?://", refresh, re.I):
-                        url = refresh
-                    else:
-                        url = urlparse.urljoin(url, refresh)
+                        kb.alwaysRefresh = choice not in ("n", "N")

-                    threadData.lastRedirectMsg = (threadData.lastRequestUID, page)
-                    kwargs['refreshing'] = True
-                    kwargs['url'] = url
-                    kwargs['get'] = None
-                    kwargs['post'] = None
+                    if kb.alwaysRefresh:
+                        if re.search(r"\Ahttps?://", refresh, re.I):
+                            url = refresh
+                        else:
+                            url = urlparse.urljoin(url, refresh)

-                    try:
-                        return Connect._getPageProxy(**kwargs)
-                    except SqlmapSyntaxException:
-                        pass
+                        threadData.lastRedirectMsg = (threadData.lastRequestUID, page)
+                        kwargs["refreshing"] = True
+                        kwargs["url"] = url
+                        kwargs["get"] = None
+                        kwargs["post"] = None
+
+                        try:
+                            return Connect._getPageProxy(**kwargs)
+                        except SqlmapSyntaxException:
+                            pass

            # Explicit closing of connection object
            if conn and not conf.keepAlive:
@@ -626,6 +634,14 @@ class Connect(object):
                if kb.testMode and kb.testType not in (None, PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED):
                    singleTimeWarnMessage("there is a possibility that the target (or WAF/IPS/IDS) is dropping 'suspicious' requests")
                warnMsg = "connection timed out to the target URL"
+            elif "Connection reset" in tbMsg:
+                if not conf.disablePrecon:
+                    singleTimeWarnMessage("turning off pre-connect mechanism because of connection reset(s)")
+                    conf.disablePrecon = True
+
+                if kb.testMode:
+                    singleTimeWarnMessage("there is a possibility that the target (or WAF/IPS/IDS) is resetting 'suspicious' requests")
+                warnMsg = "connection reset to the target URL"
            elif "URLError" in tbMsg or "error" in tbMsg:
                warnMsg = "unable to connect to the target URL"
                match = re.search(r"Errno \d+\] ([^>]+)", tbMsg)
@@ -1037,15 +1053,29 @@ class Connect(object):
                        found = False
                        value = getUnicode(value)

+                        if kb.postHint and re.search(r"\b%s\b" % re.escape(name), post or ""):
+                            if kb.postHint in (POST_HINT.XML, POST_HINT.SOAP):
+                                if re.search(r"<%s\b" % re.escape(name), post):
+                                    found = True
+                                    post = re.sub(r"(?s)(<%s\b[^>]*>)(.*?)(</%s)" % (re.escape(name), re.escape(name)), "\g<1>%s\g<3>" % value, post)
+                                elif re.search(r"\b%s>" % re.escape(name), post):
+                                    found = True
+                                    post = re.sub(r"(?s)(\b%s>)(.*?)(</[^<]*\b%s>)" % (re.escape(name), re.escape(name)), "\g<1>%s\g<3>" % value, post)
+
+                            regex = r"\b(%s)\b([^\w]+)(\w+)" % re.escape(name)
+                            if not found and re.search(regex, (post or "")):
+                                found = True
+                                post = re.sub(regex, "\g<1>\g<2>%s" % value, post)
+
                        regex = r"((\A|%s)%s=).+?(%s|\Z)" % (re.escape(delimiter), re.escape(name), re.escape(delimiter))
+                        if not found and re.search(regex, (post or "")):
+                            found = True
+                            post = re.sub(regex, "\g<1>%s\g<3>" % value, post)
+
                        if re.search(regex, (get or "")):
                            found = True
                            get = re.sub(regex, "\g<1>%s\g<3>" % value, get)

-                        if re.search(regex, (post or "")):
-                            found = True
-                            post = re.sub(regex, "\g<1>%s\g<3>" % value, post)
-
                        if re.search(regex, (query or "")):
                            found = True
                            uri = re.sub(regex.replace(r"\A", r"\?"), "\g<1>%s\g<3>" % value, uri)
@@ -1072,7 +1102,7 @@ class Connect(object):
            elif kb.postUrlEncode:
                post = urlencode(post, spaceplus=kb.postSpaceToPlus)

-        if timeBasedCompare:
+        if timeBasedCompare and not conf.disableStats:
            if len(kb.responseTimes.get(kb.responseTimeMode, [])) < MIN_TIME_RESPONSES:
                clearConsoleLine()

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -63,6 +63,7 @@ def direct(query, content=True):
    elif output:
        infoMsg = "resumed: %s..." % getUnicode(output, UNICODE_ENCODING)[:20]
        logger.info(infoMsg)
+
    threadData.lastQueryDuration = calculateDeltaSeconds(start)

    if not output:
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -364,7 +364,7 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser
        if conf.direct:
            value = direct(forgeCaseExpression if expected == EXPECTED.BOOL else expression)

-        elif any(map(isTechniqueAvailable, getPublicTypeMembers(PAYLOAD.TECHNIQUE, onlyValues=True))):
+        elif any(isTechniqueAvailable(_) for _ in getPublicTypeMembers(PAYLOAD.TECHNIQUE, onlyValues=True)):
            query = cleanQuery(expression)
            query = expandAsteriskForColumns(query)
            value = None
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,10 +1,11 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

+import re
 import types
 import urllib2
 import urlparse
@@ -123,7 +124,12 @@ class SmartRedirectHandler(urllib2.HTTPRedirectHandler):

            req.headers[HTTP_HEADER.HOST] = getHostHeader(redurl)
            if headers and HTTP_HEADER.SET_COOKIE in headers:
-                req.headers[HTTP_HEADER.COOKIE] = headers[HTTP_HEADER.SET_COOKIE].split(conf.cookieDel or DEFAULT_COOKIE_DELIMITER)[0]
+                delimiter = conf.cookieDel or DEFAULT_COOKIE_DELIMITER
+                _ = headers[HTTP_HEADER.SET_COOKIE].split(delimiter)[0]
+                if HTTP_HEADER.COOKIE not in req.headers:
+                    req.headers[HTTP_HEADER.COOKIE] = _
+                else:
+                    req.headers[HTTP_HEADER.COOKIE] = re.sub("%s{2,}" % delimiter, delimiter, ("%s%s%s" % (re.sub(r"\b%s=[^%s]*%s?" % (_.split('=')[0], delimiter, delimiter), "", req.headers[HTTP_HEADER.COOKIE]), delimiter, _)).strip(delimiter))
            try:
                result = urllib2.HTTPRedirectHandler.http_error_302(self, req, fp, code, msg, headers)
            except urllib2.HTTPError, e:
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -25,13 +25,13 @@ from lib.core.shell import autoCompletion
 from lib.request import inject
 from lib.takeover.udf import UDF
 from lib.takeover.web import Web
-from lib.takeover.xp_cmdshell import Xp_cmdshell
+from lib.takeover.xp_cmdshell import XP_cmdshell


-class Abstraction(Web, UDF, Xp_cmdshell):
+class Abstraction(Web, UDF, XP_cmdshell):
    """
    This class defines an abstraction layer for OS takeover functionalities
-    to UDF / Xp_cmdshell objects
+    to UDF / XP_cmdshell objects
    """

    def __init__(self):
@@ -40,7 +40,7 @@ class Abstraction(Web, UDF, Xp_cmdshell):

        UDF.__init__(self)
        Web.__init__(self)
-        Xp_cmdshell.__init__(self)
+        XP_cmdshell.__init__(self)

    def execCmd(self, cmd, silent=False):
        if self.webBackdoorUrl and not isStackingAvailable():
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -351,7 +351,7 @@ class Metasploit:

            self._cliCmd += " E"
        else:
-            self._cliCmd = "%s -x 'use multi/handler; set PAYLOAD %s" % (self._msfConsole, self.payloadConnStr)
+            self._cliCmd = "%s -L -x 'use multi/handler; set PAYLOAD %s" % (self._msfConsole, self.payloadConnStr)
            self._cliCmd += "; set EXITFUNC %s" % exitfunc
            self._cliCmd += "; set LPORT %s" % self.portStr

@@ -601,6 +601,8 @@ class Metasploit:

            except (EOFError, IOError, select.error):
                return proc.returncode
+            except KeyboardInterrupt:
+                pass

    def createMsfShellcode(self, exitfunc, format, extra, encode):
        infoMsg = "creating Metasploit Framework multi-stage shellcode "
@@ -620,7 +622,7 @@ class Metasploit:
        pollProcess(process)
        payloadStderr = process.communicate()[1]

-        match = re.search("(Total size:|Length:|succeeded with size) ([\d]+)", payloadStderr)
+        match = re.search("(Total size:|Length:|succeeded with size|Final size of exe file:) ([\d]+)", payloadStderr)

        if match:
            payloadSize = int(match.group(2))
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -26,6 +26,7 @@ from lib.core.common import ntToPosixSlashes
 from lib.core.common import isTechniqueAvailable
 from lib.core.common import isWindowsDriveLetterPath
 from lib.core.common import normalizePath
+from lib.core.common import parseFilePaths
 from lib.core.common import posixToNtSlashes
 from lib.core.common import randomInt
 from lib.core.common import randomStr
@@ -38,8 +39,10 @@ from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.data import paths
 from lib.core.enums import DBMS
+from lib.core.enums import HTTP_HEADER
 from lib.core.enums import OS
 from lib.core.enums import PAYLOAD
+from lib.core.enums import PLACE
 from lib.core.enums import WEB_API
 from lib.core.exception import SqlmapNoneDataException
 from lib.core.settings import BACKDOOR_RUN_CMD_TIMEOUT
@@ -196,6 +199,60 @@ class Web:
                self.webApi = choices[int(choice) - 1]
                break

+        if not kb.absFilePaths:
+            message = "do you want sqlmap to further try to "
+            message += "provoke the full path disclosure? [Y/n] "
+            getOutput = readInput(message, default="Y")
+
+            if getOutput in ("y", "Y"):
+                headers = {}
+                been = {conf.url}
+
+                for match in re.finditer(r"=['\"]((https?):)?(//[^/'\"]+)?(/[\w/.-]*)\bwp-", kb.originalPage, re.I):
+                    url = "%s%s" % (conf.url.replace(conf.path, match.group(4)), "wp-content/wp-db.php")
+                    if url not in been:
+                        try:
+                            page, _, _ = Request.getPage(url=url, raise404=False, silent=True)
+                            parseFilePaths(page)
+                        except:
+                            pass
+                        finally:
+                            been.add(url)
+
+                url = re.sub(r"(\.\w+)\Z", "~\g<1>", conf.url)
+                if url not in been:
+                    try:
+                        page, _, _ = Request.getPage(url=url, raise404=False, silent=True)
+                        parseFilePaths(page)
+                    except:
+                        pass
+                    finally:
+                        been.add(url)
+
+                for place in (PLACE.GET, PLACE.POST):
+                    if place in conf.parameters:
+                        value = re.sub(r"(\A|&)(\w+)=", "\g<2>[]=", conf.parameters[place])
+                        if "[]" in value:
+                            page, headers = Request.queryPage(value=value, place=place, content=True, raise404=False, silent=True, noteResponseTime=False)
+                            parseFilePaths(page)
+
+                cookie = None
+                if PLACE.COOKIE in conf.parameters:
+                    cookie = conf.parameters[PLACE.COOKIE]
+                elif headers and HTTP_HEADER.SET_COOKIE in headers:
+                    cookie = headers[HTTP_HEADER.SET_COOKIE]
+
+                if cookie:
+                    value = re.sub(r"(\A|;)(\w+)=[^;]*", "\g<2>=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", cookie)
+                    if value != cookie:
+                        page, _ = Request.queryPage(value=value, place=PLACE.COOKIE, content=True, raise404=False, silent=True, noteResponseTime=False)
+                        parseFilePaths(page)
+
+                    value = re.sub(r"(\A|;)(\w+)=[^;]*", "\g<2>=", cookie)
+                    if value != cookie:
+                        page, _ = Request.queryPage(value=value, place=PLACE.COOKIE, content=True, raise404=False, silent=True, noteResponseTime=False)
+                        parseFilePaths(page)
+
        directories = list(arrayizeValue(getManualDirectories()))
        directories.extend(getAutoDirectories())
        directories = list(oset(directories))
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -33,7 +33,7 @@ from lib.core.exception import SqlmapUnsupportedFeatureException
 from lib.core.threads import getCurrentThreadData
 from lib.request import inject

-class Xp_cmdshell:
+class XP_cmdshell:
    """
    This class defines methods to deal with Microsoft SQL Server
    xp_cmdshell extended procedure for plugins.
@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

@@ -1,7 +1,7 @@
 #!/usr/bin/env python

 """
-Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """

--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Miroslav Stampar	cfe34f61b8	Implementation for an Issue #1895	2017-04-06 11:33:59 +02:00
Miroslav Stampar	c1c7ea33fe	Minor update	2017-03-30 12:05:05 +02:00
Miroslav Stampar	4458a443ef	Fixes #1664	2017-03-30 11:58:03 +02:00
Miroslav Stampar	16bd3a1f02	Fixes #2453	2017-03-30 11:42:34 +02:00
Miroslav Stampar	a358bc0a38	Minor update	2017-03-30 10:24:57 +02:00
Miroslav Stampar	aebae6e27b	Added (heuristic) support for #1679	2017-03-30 10:16:35 +02:00
Miroslav Stampar	0a3e771b1b	Fixes #2449	2017-03-28 15:22:53 +02:00
Miroslav Stampar	f82c0497fa	Fixes #2447	2017-03-27 22:36:04 +02:00
Miroslav Stampar	715763885d	Fixes #2306	2017-03-24 14:20:18 +01:00
Miroslav Stampar	4aae5d9a9d	Fixes #2444	2017-03-19 21:34:47 +01:00
Miroslav Stampar	1bc583d358	Another patch related to the #2440	2017-03-17 09:43:45 +01:00
Miroslav Stampar	e506a390db	Minor patch (prevent message spamming of multiple union column possibilities)	2017-03-15 16:18:20 +01:00
Miroslav Stampar	c5b4af8636	Dummy commit (to provoke rehash)	2017-03-15 16:07:52 +01:00
Miroslav Stampar	c29e47f72f	Fixes #2440	2017-03-15 16:04:56 +01:00
Miroslav Stampar	4087213501	Merge pull request #2439 from klensy/mysql-fingerprint-update updated mysql version numbers	2017-03-14 21:15:44 +01:00
klensy	e4725366d3	updated mysql version numbers	2017-03-14 15:11:03 +03:00
Miroslav Stampar	60e8c725f9	Fixes #2437	2017-03-12 23:24:13 +01:00
Miroslav Stampar	5dba32b2e1	Fixes #2431	2017-03-12 09:52:37 +01:00
Miroslav Stampar	ef04c99069	No more dumb usage of '--dbms'	2017-03-06 12:53:04 +01:00
Miroslav Stampar	e2fb16c98c	Fixes #2425	2017-03-06 12:05:58 +01:00
Miroslav Stampar	d2b16c5c91	Fixes #2422	2017-03-01 11:09:55 +01:00
Miroslav Stampar	9f0c42dde0	Minor leftover	2017-03-01 10:09:13 +01:00
Miroslav Stampar	78ca371162	Adding option --web-root (Issue #2419 )	2017-03-01 10:07:26 +01:00
Miroslav Stampar	a35c976759	Proper implementation for an Issue #2418	2017-02-28 14:00:42 +01:00
Miroslav Stampar	89e9f4939d	Merge pull request #2418 from Ekultek/ip_regex IP address regex updated to not provide a false positive	2017-02-28 14:00:00 +01:00
Ekultek	71984fc452	updated IP address regex as to not provide false positive	2017-02-28 06:35:37 -06:00
Miroslav Stampar	a0a6702a4e	Minor patch (reported via ML)	2017-02-28 13:16:19 +01:00
Miroslav Stampar	b18444f215	Issue #2417 (most probably -> most likely)	2017-02-27 22:14:52 +01:00
Miroslav Stampar	7ea524800a	Taking couple of suggestions from #2417	2017-02-27 22:03:15 +01:00
Miroslav Stampar	7960045cf9	Fixes #2277 and #2300	2017-02-27 13:58:07 +01:00
Miroslav Stampar	d253a97a6f	Merge pull request #2411 from bbbbbrie/master Correct typo in basic.py	2017-02-27 12:55:59 +01:00
Brie Carranza	1475ba441c	Correct typo in basic.py	2017-02-26 09:05:36 -05:00
Miroslav Stampar	b2585cc8ea	Patch for #2410	2017-02-25 07:58:59 +01:00
Miroslav Stampar	7b263327cc	Update for #2410	2017-02-25 07:54:54 +01:00
Miroslav Stampar	cd31bf4ecb	Merge pull request #2410 from qnrq/patch-1 Adds option command to api client	2017-02-25 07:47:23 +01:00
Niklas Femerstrand	1b938c758f	Adds option command to api client	2017-02-25 10:24:00 +07:00
Miroslav Stampar	5a08b71999	Minor update	2017-02-23 11:36:37 +01:00
Miroslav Stampar	4b420e7579	Removing Google PageRank as it is dead now	2017-02-23 11:33:39 +01:00
Miroslav Stampar	6b580a682a	Minor update	2017-02-20 10:06:06 +01:00
Miroslav Stampar	d6e7c2acdc	Minor touch	2017-02-19 01:48:12 +01:00
Miroslav Stampar	4d3aa1605c	Merge pull request #2401 from tomahock/master Fix proxyFile regex to properly match an address with a -	2017-02-19 01:31:12 +01:00
Tomahock	7fe1820ce4	Fix proxyFile regex to properly match an address with a -	2017-02-17 23:32:32 +00:00
Miroslav Stampar	98e449e38c	Adding plus2fnconcat tamper script (Issue #2396 )	2017-02-17 10:26:25 +01:00
Miroslav Stampar	9acf122ba6	Patch for an Issue #2396	2017-02-16 16:56:54 +01:00
Miroslav Stampar	2ed144ec85	Patch for wrong encoding reported privately via email	2017-02-16 15:52:07 +01:00
Miroslav Stampar	ec0c103952	Bug fix (reported privately)	2017-02-15 10:30:29 +01:00
Miroslav Stampar	a35d1e5373	Minor patch related to the email from ML	2017-02-14 13:14:35 +01:00
Miroslav Stampar	f5cf22a536	Update for an Issue #2377	2017-02-06 13:57:33 +01:00
Miroslav Stampar	38f16decef	Update for an Issue #2384	2017-02-06 13:28:33 +01:00
Miroslav Stampar	15f86e85b1	Minor update for #2379	2017-02-06 12:03:18 +01:00
Miroslav Stampar	5217efc69b	Fixes #2379	2017-02-06 12:01:46 +01:00
Miroslav Stampar	03bbf552ef	Patch for an Issue #2382	2017-02-06 11:14:45 +01:00
Miroslav Stampar	664684ad8f	Update for #2378	2017-02-03 23:04:38 +01:00
Miroslav Stampar	ddea0bf6e4	Merge pull request #2378 from samogot/patch-1 Minor parse-error extension	2017-02-03 22:56:05 +01:00
samogot	1c1f259df4	Update settings.py fix - looking for any tag	2017-02-03 16:50:16 +02:00
samogot	6249823335	Minor parse-error extension parse errors produced by Yii PHP Framework	2017-02-03 16:36:57 +02:00
Miroslav Stampar	bad3f80a1c	Touch update	2017-01-31 14:18:36 +01:00
Miroslav Stampar	529089ba5b	Merge pull request #2374 from anarcoder/space2morecomment New space 2 more comment bypass	2017-01-31 14:16:53 +01:00
Miroslav Stampar	9851a5703a	Fixes #2373	2017-01-31 14:00:12 +01:00
Daniel Almeida	aa9989ff90	[add] new space 2 more comment bypass	2017-01-31 10:50:14 -02:00
Miroslav Stampar	2a3014b606	Fixes #2367	2017-01-24 18:07:06 +01:00
Miroslav Stampar	16d5e22b72	Fixes #2358	2017-01-21 23:58:37 +01:00
Miroslav Stampar	a8a6dce38b	Fixes #2366	2017-01-21 23:09:15 +01:00
Miroslav Stampar	f542e828d2	Fixes #2364	2017-01-20 13:11:12 +01:00
Miroslav Stampar	cf182882b1	Minor update	2017-01-18 10:40:39 +01:00
Miroslav Stampar	2224ac76aa	Merge pull request #2359 from MyKings/master Adding new WAF script(TencentCloud)	2017-01-18 10:36:39 +01:00
Miroslav Stampar	dd5ac6f1e7	Fixes #2357	2017-01-18 10:33:54 +01:00
Miroslav Stampar	1e7a453ff6	Fixes #2356	2017-01-18 10:19:23 +01:00
MyKings	8a84c252be	Adding new WAF script(TencentCloud)	2017-01-18 16:11:03 +08:00
Miroslav Stampar	138aa6db65	Patch for an Issue #2351	2017-01-16 15:23:38 +01:00
Miroslav Stampar	121f0376ea	Implementation for #2351	2017-01-16 14:29:23 +01:00
Miroslav Stampar	dfc684640a	Proper implementation for #2347	2017-01-16 14:01:44 +01:00
Miroslav Stampar	104fbc80af	Patch for #2348	2017-01-16 13:53:46 +01:00
Miroslav Stampar	cadba37059	Proper implementation for #2350	2017-01-16 13:44:46 +01:00
Miroslav Stampar	750d57ec96	Fixed bug reported privately via email	2017-01-13 14:41:41 +01:00
Miroslav Stampar	9a86365d92	Fixes #2333	2017-01-08 01:21:31 +01:00
Miroslav Stampar	f794d9d5a5	Fixes #2328	2017-01-02 15:26:32 +01:00
Miroslav Stampar	c29db43bfa	Minor refactoring	2017-01-02 15:14:59 +01:00
Miroslav Stampar	e0eeed0a96	Minor update	2017-01-02 14:31:19 +01:00
Miroslav Stampar	55272f7a3b	New version preparation	2017-01-02 14:19:18 +01:00
Miroslav Stampar	6ff07f01eb	Fixes #2326	2016-12-31 13:39:22 +01:00
Miroslav Stampar	1c737d7515	Fixes #2322	2016-12-28 22:11:14 +01:00
Miroslav Stampar	2fa5341879	Merge pull request #2323 from CoresecSystems/master Fix the logic used for --param-exclude	2016-12-28 21:44:46 +01:00
Francisco Blas Izquierdo Riera (klondike)	025e9ac5b4	Fix the logic used for --param-exclude The current logic will skip all existing parameters if no param-exclude is defined. This breaks previous behaviour, makes it harder to use the tool and is quite confusing. The new logic will always check the parameter is set before running any other checks instead of shortcircuit an empoty(always true) regexp.	2016-12-28 12:25:05 +01:00
Miroslav Stampar	89bbf5284c	Adding new option --param-exclude on private request	2016-12-25 23:16:44 +01:00
Miroslav Stampar	44b00d629d	Fixes #2312	2016-12-21 10:33:35 +01:00
Miroslav Stampar	afc3b30c41	Minor refactoring	2016-12-20 09:56:44 +01:00
Miroslav Stampar	17c556a63d	Minor patches (and one bug from ML)	2016-12-20 09:53:44 +01:00
Miroslav Stampar	edc6f47758	Some refactoring	2016-12-19 23:47:39 +01:00
Miroslav Stampar	bb6e8fd4ce	Minor bug fix (reported privately via email)	2016-12-15 16:09:09 +01:00
Miroslav Stampar	c54c2204a1	Fixes #2303	2016-12-12 10:47:05 +01:00
Miroslav Stampar	f7f33bef9f	Minor patches	2016-12-09 23:19:03 +01:00
Miroslav Stampar	4bd7d81cea	Patches #2300	2016-12-09 23:14:18 +01:00
Miroslav Stampar	f6815df5c3	Fixes #2302	2016-12-09 23:10:14 +01:00
Miroslav Stampar	42cea2e03c	Better git clone (faster; without too much commit history)	2016-12-08 11:04:42 +01:00
Miroslav Stampar	52177065ca	Patch for an Issue #2297	2016-12-06 15:43:09 +01:00
Miroslav Stampar	e74149970b	Minor debug update	2016-12-03 22:06:18 +01:00
Miroslav Stampar	90b0ac37c8	New WAF script (AWS WAF)	2016-12-01 23:09:06 +01:00
Miroslav Stampar	63a74777f2	Minor update	2016-12-01 23:08:49 +01:00