Commit Graph

50 Commits

Author SHA1 Message Date
Sean Flanigan
5df439b7e8
Add enhancements for contrib/mitmproxywrapper.py (#5883)
* Add options for mitmweb and random port

* Clean up proxy settings on ^C

* [autofix.ci] apply automated fixes

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-01-18 09:02:04 +01:00
Sean Flanigan
2bf96678ae
Adapt contrib/mitmproxywrapper.py to Python 3 strings (#5866)
* Adapt mitmproxywrapper.py to Python 3 strings

This fixes errors like this one: 

`TypeError: memoryview: a bytes-like object is required, not 'str'`

* [autofix.ci] apply automated fixes

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-01-13 10:04:13 +01:00
gpiechnik2
12960c0494
har: don't set pages 2022-12-17 14:32:38 +01:00
autofix-ci[bot]
8c2428c9d3
[autofix.ci] apply automated fixes 2022-11-29 13:28:41 +00:00
Maximilian Hils
8361c81cdf add autofix.ci 2022-11-29 14:27:35 +01:00
Maximilian Hils
6b71066141 cleanup mitmproxy.connection, introduce dataclass-based serialization 2022-11-21 01:27:31 +01:00
inceabdullah
be2b9a962e
add websocket messages in HAR file (#5663)
* add websocket messages in HAR file

* use websocket_end instead of websocket_message

* remove requestID_to_entry

* remove request

* add base64-encoded data if it is binary

* har dumper: nits

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-10-21 02:44:01 +02:00
m4ci3k2
c0f25e0470
Make HAR files compatible with Firefox DevTools (#5666)
Firefox Developer Tools reject to load HAR files that do not have
`har.log.pages` element (see
<https://bugzilla.mozilla.org/show_bug.cgi?id=1691240>). Adding a
placeholder entry fixes it.

Co-authored-by: Maciej Zielenkiewicz <maciej.zielenkiewicz@connectpoint.pl>
2022-10-21 01:54:07 +02:00
BigSully
b2303558b2
get real url in transparent mode (#5613)
flow.request.url shows the ip address rather than host name when mitmproxy works on transparent mode
flow.request.pretty_url can show the correct url
2022-10-14 07:44:01 +02:00
Maximilian Hils
c69239bb90 switch to stdlib logging
mitmproxy previously used a homegrown logging mechanism based around
`mitmproxy.ctx.log` and the `add_log` hook. This worked well for everything
we control, but does not work outside the mitmproxy universe.
For now we have simply ignored logging in e.g. tornado or h2, but with the
upcoming introduction of mitmproxy_wireguard we now have a dependency
on some Rust/PyO3 code for which we definitely want logs, but which also
cannot easily be changed to use our homegrown logging (PyO3 does the heavy
lifting to add interoperability with stdlib logging). Long story short,
we want to introduce a log handler for stdlib logging.

Now there are two ways how such a handler could operate:

 1. We could build a handler that forwards all stdlib log events
    into our homegrown mechanism.
 2. We embrace stdlib's logging as the correct way to do things,
    and get rid of our homegrown stuff.

This PR follows the second approach by removing the `add_log` hook and
rewriting the `TermLog` and `EventStore` addons to listen for stdlib log records.
This means that all `mitmproxy.ctx.log.info` events are now simply `logging.info` etc.

One upside of this approach is that many parts of the codebase now don't depend
on the existence of `mitmproxy.ctx` and we can use off-the-shelf things like pytest's
`caplog`. We can also now better colorize log output and/or add timestamps.
2022-09-17 17:28:35 +02:00
Francisco Boni
f9899fdd44
Improve http_manipulate_cookies.py example (#5578)
Improve the example helper functions to use the get_all functions recommended
in the Header class docs for use with cookies data. This will avoid errors when
dealing with multiple headers.

cf655a664f/mitmproxy/http.py (L148)
cf655a664f/mitmproxy/http.py (L91)
2022-09-11 15:58:29 +02:00
Maximilian Hils
fdde9ba3b3 use Python 3.9+ typing 2022-04-26 13:53:23 +02:00
Maximilian Hils
b31ad66225 autoflake --in-place **/*.py 2022-04-26 13:52:46 +02:00
Maximilian Hils
e83ec8390a pyupgrade --py39-plus **/*.py 2022-04-26 13:51:11 +02:00
Will Monge
a863f529ab
Example: manipulate http cookies (#5278)
* added example on how to manipulate (outbound and inbound) cookies for http flows

* added entry to the CHANGELOG
2022-04-22 12:40:23 +02:00
Maximilian Hils
648d3bc0f9
add wsreplay.py, close #5012 2022-03-30 15:14:29 +02:00
Gabriel Corona
66dd158560
Add domain fronting example (#5217)
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-29 18:24:59 +02:00
Maximilian Hils
9d1e3107e8 pyupgrade --keep-runtime-typing --py38-plus 2022-03-29 15:29:37 +02:00
Maximilian Hils
beb49ab121 don't use #noqa for imports 2022-03-29 13:51:08 +02:00
Agustin Bacigalup
33bba38b03
Add httpdump.py example (#5190)
* add httpdump.py example

* update CHANGELOG.md

* delete print() statement

* fix flake8?
2022-03-23 13:04:38 +00:00
Eldad Marciano
cba67aa94c
an ntlm upstream addon for http \ https (#5100)
* an ntlm upstream addon for http \ https

Ntlm upstream proxy for http https all http methods

* fix filename

* lint!

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-15 20:44:09 +00:00
EndUser509
a0cf273484
Merge pull request #5099 from EndUser509/save_streamed_data2
Save streamed data
2022-03-15 21:34:49 +01:00
Maximilian Hils
96f77453cc
fix change_upstream_proxy.py example, fix #4981 (#5007) 2021-12-20 21:10:06 +00:00
Maximilian Hils
3cb87f5a2f split tls_handshake hook into client/server and success/fail variants 2021-11-22 10:23:21 +01:00
Maximilian Hils
a72f61ef57 Merge remote-tracking branch 'origin/main' into ignore-after-clienthello 2021-11-22 09:54:08 +01:00
Marius
301f8cf79d
Create search.py (#4900)
* Create search.py

* Address linter findings

* Address linter findings
2021-11-16 12:47:34 +01:00
Maximilian Hils
7fd887a553 move tls hook data to mitmproxy.tls 2021-09-04 16:37:39 +02:00
Maximilian Hils
9f39e2f387 tests++ 2021-09-04 16:03:06 +02:00
Maximilian Hils
98a3e33477 tls: add tls_handshake, ignore-after-clienthello
this fixes #4702
2021-09-04 00:24:41 +02:00
Maximilian Hils
d518622976
Update examples/contrib/README.md 2021-08-22 14:57:20 +02:00
KORraNpl
52c81c287e Fix DeprecationWarning in har_dump.py 2021-08-11 20:32:39 +02:00
Alexander Prinzhorn
eef6522a5e remove websocket_error event, fixes #4674 2021-07-12 16:35:25 +02:00
Maximilian Hils
d6975e0b80 add next_layer docs + example, minor fixes 2021-06-20 02:15:09 +02:00
Maximilian Hils
15adb2cd79
Update tls_passthrough.py 2021-06-18 08:49:48 +02:00
Brad Dixon
c6ba97eab6
Use emoji and characters as markers. Add ~marker filter syntax. (#4587)
* Use emoji and characters as markers. Add ~marker filter syntax.

* Add a test to please our CI overlords. :)
2021-05-27 11:40:41 +02:00
Peter Dave Hello
123342ea0b
Improve Markdown syntax(styling) (#4496)
* Improve Markdown syntax

- Add missing blank lines
- Remove trailing spaces
- Remove additional blank lines
- Fix indentation consistency and correctness

* Update addons-examples.md

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-03-11 19:13:02 +01:00
Anatoli Babenia
cff98a30e2
Fix HAR dump (#4439)
* Fix `KeyError: 'Unknown options: hardump'` in HAR test_simple

The reason was an invalid import of `connections` which is gone
in d159897d98, but the loading error
was silently ignored (#4438) resulting in a misleading exception
at a later stage.

This still doesn't make the test suite run, because it is better
to fix one error at a time.

The way to run tests from examples.

    ln -sr test/conftest.py examples
    pytest examples/contrib/test_har_dump.py -k simpl

The way to see the execution flow with silenced exceptions.

    from hunter import trace, Q
    trace(Q(stdlib=False), ~Q(filename_contains='site-packages'))

* Fix `TypeError: invoke() takes 3 positional arguments but 4 were given`

* Fix `TypeError: 'bool' object is not callable`
2021-02-11 21:07:09 +01:00
Anatoli Babenia
00f718e6c0
Fix location for contrib har_dump.py in its test (#4424)
To run the test, I had to copy `conftest.py` to examples.

    cp test/conftest.py examples
    pytest examples/contrib/test_har_dump.py

This fixes the `fixture 'tdata' not found` error.
2021-02-03 18:52:56 +01:00
Thomas Kriechbaumer
af3dfac855
Merge pull request #4301 from Kriechi/pyupgrade-py36
pyupgrade --py36-plus mitmproxy/**/*.py
2020-11-21 11:05:16 +01:00
Thomas Kriechbaumer
82b5d61456 revert some minor changes 2020-11-21 10:46:19 +01:00
Thomas Kriechbaumer
38cca379df pyupgrade --py36-plus mitmproxy/**/*.py 2020-11-20 19:25:26 +01:00
Maximilian Hils
e36c790d55
Merge pull request #4300 from anneborcherding/fix/webscanner-helper
[contrib addons] updated imports and style.
2020-11-20 10:06:13 +01:00
anneborcherding
97bf522165
Added addon to suppress the sending of 502 Bad Gateway response… (#4299)
* WIP: Added option to suppress the sending of 502 Bad Gateway responses if the server is not responding

* Revert "WIP: Added option to suppress the sending of 502 Bad Gateway responses if the server is not responding"

This reverts commit 21b74f38b7.

* Added addon which suppresses the bad gateway error messages sent by mitmproxy

* fix type annotation

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2020-11-20 09:15:12 +01:00
Anne Borcherding
d4db078312 [contrib addons] updated imports and style. 2020-11-20 09:10:08 +01:00
Maximilian Hils
dfcc1578e5
Merge pull request #3962 from anneborcherding/feature/selenium-auth-addon
Added add-on with support for proxy authentication using selenium.
2020-07-15 13:54:48 +02:00
Maximilian Hils
83c6ab1393 move selenium example into contrib 2020-07-14 16:46:36 +02:00
Vikas Gupta
1acbcc2437
add addon to check SSL-pinning on client (#3196) 2020-07-14 16:38:13 +02:00
Ed Medvedev
14dfdec8f4
Add a JSON dump script: write data into a file or send to an endpoint as JSON (#2861) 2020-07-14 16:35:03 +02:00
Maximilian Hils
96ce21687e blacklist -> blocklist 2020-07-04 12:09:51 +02:00
Maximilian Hils
08895e9ba6 restructure examples
- restructure examples (fix #4031)
 - remove example dependencies from setup.py,
   we do not need special dependencies for our supported addons.
 - unify how we generate docs from code
 - improve example docs
2020-06-23 16:00:14 +02:00