While looking into #6009 I noticed the Bing fallback in -g never
returns anything, even when Bing itself serves a normal results page.
Two things changed on Bing's side. The result anchor now carries
target="_blank" before href, so '<h2><a href="..." h=' no longer
matches at all. And the href is no longer the target URL but a
tracking redirect (https://www.bing.com/ck/a?...&u=a1<base64url>),
where the real link sits base64url-encoded after the "a1" prefix
in the u parameter.
The regex now allows extra attributes on both tags and captures
either the u payload of a ck/a redirect or a plain href. The payload
is decoded with decodeBase64(), which already deals with the
URL-safe alphabet and the missing padding. Plain hrefs still work
the same as before (with & unescaped), in case Bing serves the
old markup to some clients.
Checked against a saved live results page (0 -> 10 links) and the
old markup, and --smoke-test passes. Google still can't be scraped
since it only serves a JS page now, so this doesn't fully close
#6009, but Bing is usable again as a fallback when DuckDuckGo blocks
the IP.
* Fix python3 binary - character mismatch in api.py
Convert between text and binary data in api.py call to urllib.request
and response from it.
In python3 sqlmapapi -c it fixes, at least
- not nice output from log/list commands
- any command that POSTs data (including new) crashing sqlmapapi
* Fix python3 binary - character mismatch in search.py
Before:
python3 sqlmap.py -g <long random string>
(...)
[18:35:17] [INFO] using search result page #1
no usable links found. What do you want to do?
[1] (re)try with DuckDuckGo (default)
[2] (re)try with Bing
[3] quit
> 1
[18:35:21] [CRITICAL] unable to connect
After:
python3 sqlmap.py -g asfafw2fwesvzsdvaw
(...)
[18:37:30] [INFO] using search result page #1
no usable links found. What do you want to do?
[1] (re)try with DuckDuckGo (default)
[2] (re)try with Bing
[3] quit
> 1
[18:37:34] [INFO] found 26 results for your search dork expression, 16 of them are testable targets
[18:37:34] [INFO] found a total of 16 targets
URL 1:
GET https:...
do you want to test this URL? [Y/n/q]
Miroslav Štampar