Commit Graph

69 Commits

Author SHA1 Message Date
fosple
0503fbb63a
Update change_upstream_proxy.py (#6853)
Script was not working as ServerSpec accepts a tuple, which was not supplied. I fixed it so a tuple is supplied to ServerSpec
2024-05-18 12:06:48 +02:00
Alexander Prinzhorn
d6dc12c835
Update all_markers.py (#6849) 2024-05-16 14:15:55 +02:00
pengqiseven
a8603cac0d
fix some typos (#6728) 2024-03-11 08:19:36 +01:00
dependabot[bot]
a91989b7ba
Bump the github-actions group with 3 updates (#6701)
Bumps the github-actions group with 3 updates:
[install-pinned/ruff](https://github.com/install-pinned/ruff),
[apple-actions/import-codesign-certs](https://github.com/apple-actions/import-codesign-certs)
and
[docker/setup-buildx-action](https://github.com/docker/setup-buildx-action).

Updates `install-pinned/ruff` from
fe472defb50a6a2c00ea3a3982534e86e69991e8 to
38b373a3a8635c2be31d92314e816a491fda910a
<details>
<summary>Commits</summary>
<ul>
<li><a
href="38b373a3a8"><code>38b373a</code></a>
update README.md (ruff 0.3.0)</li>
<li><a
href="06af3ea1c3"><code>06af3ea</code></a>
update pins (ruff 0.3.0)</li>
<li><a
href="be1c354876"><code>be1c354</code></a>
update README.md (ruff 0.2.2)</li>
<li><a
href="c9779bbd5b"><code>c9779bb</code></a>
update pins (ruff 0.2.2)</li>
<li><a
href="48831a86ce"><code>48831a8</code></a>
update README.md (ruff 0.2.1)</li>
<li><a
href="6775b5f352"><code>6775b5f</code></a>
update pins (ruff 0.2.1)</li>
<li><a
href="bc12a64c2f"><code>bc12a64</code></a>
update README.md (ruff 0.2.0)</li>
<li><a
href="3b8cceff45"><code>3b8ccef</code></a>
update pins (ruff 0.2.0)</li>
<li>See full diff in <a
href="fe472defb5...38b373a3a8">compare
view</a></li>
</ul>
</details>
<br />

Updates `apple-actions/import-codesign-certs` from
5565bb656f60c98c8fc515f3444dd8db73545dc2 to
493007ed063995cf2d4fbca064704150548f8bb5
<details>
<summary>Commits</summary>
<ul>
<li><a
href="493007ed06"><code>493007e</code></a>
Merge pull request <a
href="https://redirect.github.com/apple-actions/import-codesign-certs/issues/62">#62</a>
from himself65/patch-1</li>
<li><a
href="2e5aa07267"><code>2e5aa07</code></a>
Update README.md</li>
<li>See full diff in <a
href="5565bb656f...493007ed06">compare
view</a></li>
</ul>
</details>
<br />

Updates `docker/setup-buildx-action` from 3.0.0 to 3.1.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/docker/setup-buildx-action/releases">docker/setup-buildx-action's
releases</a>.</em></p>
<blockquote>
<h2>v3.1.0</h2>
<ul>
<li><code>cache-binary</code> input to enable/disable caching binary to
GHA cache backend by <a
href="https://github.com/crazy-max"><code>@​crazy-max</code></a> in <a
href="https://redirect.github.com/docker/setup-buildx-action/pull/300">docker/setup-buildx-action#300</a></li>
<li>build(deps): bump <code>@​babel/traverse</code> from 7.17.3 to
7.23.2 in <a
href="https://redirect.github.com/docker/setup-buildx-action/pull/282">docker/setup-buildx-action#282</a></li>
<li>build(deps): bump <code>@​docker/actions-toolkit</code> from 0.12.0
to 0.17.0 in <a
href="https://redirect.github.com/docker/setup-buildx-action/pull/281">docker/setup-buildx-action#281</a>
<a
href="https://redirect.github.com/docker/setup-buildx-action/pull/284">docker/setup-buildx-action#284</a>
<a
href="https://redirect.github.com/docker/setup-buildx-action/pull/299">docker/setup-buildx-action#299</a></li>
<li>build(deps): bump uuid from 9.0.0 to 9.0.1 in <a
href="https://redirect.github.com/docker/setup-buildx-action/pull/271">docker/setup-buildx-action#271</a></li>
<li>build(deps): bump undici from 5.26.3 to 5.28.3 in <a
href="https://redirect.github.com/docker/setup-buildx-action/pull/297">docker/setup-buildx-action#297</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/docker/setup-buildx-action/compare/v3.0.0...v3.1.0">https://github.com/docker/setup-buildx-action/compare/v3.0.0...v3.1.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="0d103c3126"><code>0d103c3</code></a>
Merge pull request <a
href="https://redirect.github.com/docker/setup-buildx-action/issues/300">#300</a>
from crazy-max/cache-binary</li>
<li><a
href="f19477aacd"><code>f19477a</code></a>
chore: update generated content</li>
<li><a
href="a4180f835d"><code>a4180f8</code></a>
cache-binary input to enable/disable caching binary to GHA cache
backend</li>
<li><a
href="524315340d"><code>5243153</code></a>
Merge pull request <a
href="https://redirect.github.com/docker/setup-buildx-action/issues/299">#299</a>
from docker/dependabot/npm_and_yarn/docker/actions-to...</li>
<li><a
href="3679a54023"><code>3679a54</code></a>
chore: update generated content</li>
<li><a
href="37a22a2fb2"><code>37a22a2</code></a>
build(deps): bump <code>@​docker/actions-toolkit</code> from 0.14.0 to
0.17.0</li>
<li><a
href="65afe610a1"><code>65afe61</code></a>
Merge pull request <a
href="https://redirect.github.com/docker/setup-buildx-action/issues/297">#297</a>
from docker/dependabot/npm_and_yarn/undici-5.28.3</li>
<li><a
href="fcb8f722fd"><code>fcb8f72</code></a>
chore: update generated content</li>
<li><a
href="f62b9a17c0"><code>f62b9a1</code></a>
Merge pull request <a
href="https://redirect.github.com/docker/setup-buildx-action/issues/298">#298</a>
from crazy-max/bump-gha</li>
<li><a
href="74c5b717e5"><code>74c5b71</code></a>
bump codecov/codecov-action from 3 to 4</li>
<li>Additional commits viewable in <a
href="f95db51fdd...0d103c3126">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2024-03-06 21:54:11 +01:00
ReK2
0c91733df8
examples/contrib: link to security plugin 2023-11-06 22:56:24 +01:00
Maximilian Hils
f55ee1d44f
Migrate to ruff (#6433) 2023-10-31 16:03:53 +01:00
Stanley Good
bdcd3e3192
add save HAR addon (#6368)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2023-09-24 13:51:22 +00:00
Stanley Good
eacbb25e93
ReadHAR addon (#6283)
* moved readhar to addons and adjusted tests

* fixed tests for read_har function

* Add har export addon (#6252)

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>

* fixed path issues

* added back contrib/import_har

* [autofix.ci] apply automated fixes

* fixed windows error hopefully

* [autofix.ci] apply automated fixes

* fixed ubuntu 3.10 isoformat error

* [autofix.ci] apply automated fixes

* fixed Path import

* [autofix.ci] apply automated fixes

* need to fix ubuntu 3.10 isoformat

* [autofix.ci] apply automated fixes

* replaced Z with +00:00 before isoformatting

* fixed tox lint

* moved readhar to addons and adjusted tests

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2023-07-29 18:32:23 +02:00
Stanley Good
cb1c824255
Add har export addon (#6252)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2023-07-27 19:37:06 +00:00
Stanley Good
d4ec24d85d
HAR import: request content handling (#6250)
* added handling for request content

* fixed small error in getting request content, and added test HAR file that contains a request with post data

* changed postdata.har to HAR file from reqbin

* Deterministic file_to_flows (#6257)

* made file_to_flows deterministic

* [autofix.ci] apply automated fixes

* made dummy values easier to understand

* [autofix.ci] apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* changed postdata.har to include only one entry

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-07-16 14:59:48 +02:00
Stanley Good
1c8c4ab935
Deterministic file_to_flows (#6257)
* made file_to_flows deterministic

* [autofix.ci] apply automated fixes

* made dummy values easier to understand

* [autofix.ci] apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-07-16 11:52:18 +00:00
Stanley Good
9c99777e42
http version for flow is now based on HAR file (#6248)
* http version for flow is now based on HAR file

* [autofix.ci] apply automated fixes

* http_version is now always added as correct representation, and test files are updated with correct http_version values

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-07-15 09:56:09 +02:00
Stanley Good
c34c86dd90
add HAR import tests (#6231)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2023-07-11 17:17:44 +02:00
Stanley Good
47021baaa1
first draft of read har. (#6190)
* first draft of read har.

* [autofix.ci] apply automated fixes

* Fixed using max's suggestions

* [autofix.ci] apply automated fixes

* updated to snake_case

* [autofix.ci] apply automated fixes

* Made max's changes.

* [autofix.ci] apply automated fixes

* gets server address from har file now

* [autofix.ci] apply automated fixes

* added support for if serverIPAdress wasnt found

* [autofix.ci] apply automated fixes

* editted docstring and changed peername and sockname values

* fix nits

* removed unused variable, should pass tox

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2023-06-30 08:12:55 +02:00
Niles Ingalls
19218bfaf2
update api for sni call (#6077) 2023-04-22 12:48:24 +02:00
Maximilian Hils
555a2d2199
disallow bare except statements (#6017) 2023-03-26 13:57:48 +00:00
autofix-ci[bot]
51670861e6 [autofix.ci] apply automated fixes 2023-02-27 08:37:11 +01:00
Xiao Wang
430833e3d9
Fix server addr issue in tls_passthrough example. (#5904)
* fix server peername is None issue

peername would be None, we should use other not None property as key.

* Update tls_passthrough.py

---------

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2023-02-07 09:57:24 +01:00
dependabot[bot]
dfeddcf4ad
Bump install-pinned/black from 9101a4d68e870eaaaae21c412d1d879b93c9afcb to 13c8a20eb904ba800c87f0b34ccfd932ac2ff81d (#5899)
* Bump install-pinned/black

Bumps [install-pinned/black](https://github.com/install-pinned/black) from 9101a4d68e870eaaaae21c412d1d879b93c9afcb to 13c8a20eb904ba800c87f0b34ccfd932ac2ff81d.
- [Release notes](https://github.com/install-pinned/black/releases)
- [Commits](9101a4d68e...13c8a20eb9)

---
updated-dependencies:
- dependency-name: install-pinned/black
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* [autofix.ci] apply automated fixes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-02-01 23:08:09 +00:00
Sean Flanigan
5df439b7e8
Add enhancements for contrib/mitmproxywrapper.py (#5883)
* Add options for mitmweb and random port

* Clean up proxy settings on ^C

* [autofix.ci] apply automated fixes

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-01-18 09:02:04 +01:00
Sean Flanigan
2bf96678ae
Adapt contrib/mitmproxywrapper.py to Python 3 strings (#5866)
* Adapt mitmproxywrapper.py to Python 3 strings

This fixes errors like this one: 

`TypeError: memoryview: a bytes-like object is required, not 'str'`

* [autofix.ci] apply automated fixes

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-01-13 10:04:13 +01:00
gpiechnik2
12960c0494
har: don't set pages 2022-12-17 14:32:38 +01:00
autofix-ci[bot]
8c2428c9d3
[autofix.ci] apply automated fixes 2022-11-29 13:28:41 +00:00
Maximilian Hils
8361c81cdf add autofix.ci 2022-11-29 14:27:35 +01:00
Maximilian Hils
6b71066141 cleanup mitmproxy.connection, introduce dataclass-based serialization 2022-11-21 01:27:31 +01:00
inceabdullah
be2b9a962e
add websocket messages in HAR file (#5663)
* add websocket messages in HAR file

* use websocket_end instead of websocket_message

* remove requestID_to_entry

* remove request

* add base64-encoded data if it is binary

* har dumper: nits

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-10-21 02:44:01 +02:00
m4ci3k2
c0f25e0470
Make HAR files compatible with Firefox DevTools (#5666)
Firefox Developer Tools reject to load HAR files that do not have
`har.log.pages` element (see
<https://bugzilla.mozilla.org/show_bug.cgi?id=1691240>). Adding a
placeholder entry fixes it.

Co-authored-by: Maciej Zielenkiewicz <maciej.zielenkiewicz@connectpoint.pl>
2022-10-21 01:54:07 +02:00
BigSully
b2303558b2
get real url in transparent mode (#5613)
flow.request.url shows the ip address rather than host name when mitmproxy works on transparent mode
flow.request.pretty_url can show the correct url
2022-10-14 07:44:01 +02:00
Maximilian Hils
c69239bb90 switch to stdlib logging
mitmproxy previously used a homegrown logging mechanism based around
`mitmproxy.ctx.log` and the `add_log` hook. This worked well for everything
we control, but does not work outside the mitmproxy universe.
For now we have simply ignored logging in e.g. tornado or h2, but with the
upcoming introduction of mitmproxy_wireguard we now have a dependency
on some Rust/PyO3 code for which we definitely want logs, but which also
cannot easily be changed to use our homegrown logging (PyO3 does the heavy
lifting to add interoperability with stdlib logging). Long story short,
we want to introduce a log handler for stdlib logging.

Now there are two ways how such a handler could operate:

 1. We could build a handler that forwards all stdlib log events
    into our homegrown mechanism.
 2. We embrace stdlib's logging as the correct way to do things,
    and get rid of our homegrown stuff.

This PR follows the second approach by removing the `add_log` hook and
rewriting the `TermLog` and `EventStore` addons to listen for stdlib log records.
This means that all `mitmproxy.ctx.log.info` events are now simply `logging.info` etc.

One upside of this approach is that many parts of the codebase now don't depend
on the existence of `mitmproxy.ctx` and we can use off-the-shelf things like pytest's
`caplog`. We can also now better colorize log output and/or add timestamps.
2022-09-17 17:28:35 +02:00
Francisco Boni
f9899fdd44
Improve http_manipulate_cookies.py example (#5578)
Improve the example helper functions to use the get_all functions recommended
in the Header class docs for use with cookies data. This will avoid errors when
dealing with multiple headers.

cf655a664f/mitmproxy/http.py (L148)
cf655a664f/mitmproxy/http.py (L91)
2022-09-11 15:58:29 +02:00
Maximilian Hils
fdde9ba3b3 use Python 3.9+ typing 2022-04-26 13:53:23 +02:00
Maximilian Hils
b31ad66225 autoflake --in-place **/*.py 2022-04-26 13:52:46 +02:00
Maximilian Hils
e83ec8390a pyupgrade --py39-plus **/*.py 2022-04-26 13:51:11 +02:00
Will Monge
a863f529ab
Example: manipulate http cookies (#5278)
* added example on how to manipulate (outbound and inbound) cookies for http flows

* added entry to the CHANGELOG
2022-04-22 12:40:23 +02:00
Maximilian Hils
648d3bc0f9
add wsreplay.py, close #5012 2022-03-30 15:14:29 +02:00
Gabriel Corona
66dd158560
Add domain fronting example (#5217)
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-29 18:24:59 +02:00
Maximilian Hils
9d1e3107e8 pyupgrade --keep-runtime-typing --py38-plus 2022-03-29 15:29:37 +02:00
Maximilian Hils
beb49ab121 don't use #noqa for imports 2022-03-29 13:51:08 +02:00
Agustin Bacigalup
33bba38b03
Add httpdump.py example (#5190)
* add httpdump.py example

* update CHANGELOG.md

* delete print() statement

* fix flake8?
2022-03-23 13:04:38 +00:00
Eldad Marciano
cba67aa94c
an ntlm upstream addon for http \ https (#5100)
* an ntlm upstream addon for http \ https

Ntlm upstream proxy for http https all http methods

* fix filename

* lint!

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-15 20:44:09 +00:00
EndUser509
a0cf273484
Merge pull request #5099 from EndUser509/save_streamed_data2
Save streamed data
2022-03-15 21:34:49 +01:00
Maximilian Hils
96f77453cc
fix change_upstream_proxy.py example, fix #4981 (#5007) 2021-12-20 21:10:06 +00:00
Maximilian Hils
3cb87f5a2f split tls_handshake hook into client/server and success/fail variants 2021-11-22 10:23:21 +01:00
Maximilian Hils
a72f61ef57 Merge remote-tracking branch 'origin/main' into ignore-after-clienthello 2021-11-22 09:54:08 +01:00
Marius
301f8cf79d
Create search.py (#4900)
* Create search.py

* Address linter findings

* Address linter findings
2021-11-16 12:47:34 +01:00
Maximilian Hils
7fd887a553 move tls hook data to mitmproxy.tls 2021-09-04 16:37:39 +02:00
Maximilian Hils
9f39e2f387 tests++ 2021-09-04 16:03:06 +02:00
Maximilian Hils
98a3e33477 tls: add tls_handshake, ignore-after-clienthello
this fixes #4702
2021-09-04 00:24:41 +02:00
Maximilian Hils
d518622976
Update examples/contrib/README.md 2021-08-22 14:57:20 +02:00
KORraNpl
52c81c287e Fix DeprecationWarning in har_dump.py 2021-08-11 20:32:39 +02:00