mirror of
https://github.com/pydantic/pydantic-ai.git
synced 2026-10-11 21:59:51 +00:00
Run Pyright in its own CI step instead of the pre-commit typecheck hook (#8906)
This commit is contained in:
@@ -107,7 +107,7 @@ jobs:
|
||||
# All Python/stub files conservatively cover Pyright's include and import graph;
|
||||
# the remaining paths change its command, configuration, dependencies, or this gate.
|
||||
case "$1" in
|
||||
*.py|*.pyi|*/py.typed|py.typed|pyproject.toml|*/pyproject.toml|uv.lock|uv.toml|*/uv.toml|pyrightconfig*.json|*/pyrightconfig*.json|Makefile|.pre-commit-config.yaml|.github/workflows/ci.yml) ;;
|
||||
*.py|*.pyi|*/py.typed|py.typed|pyproject.toml|*/pyproject.toml|uv.lock|uv.toml|*/uv.toml|pyrightconfig*.json|*/pyrightconfig*.json|Makefile|.github/workflows/ci.yml) ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
}
|
||||
@@ -160,6 +160,7 @@ jobs:
|
||||
# FastMCP 4 / MCP SDK v2 lock fork, which `tool.uv.conflicts` bars from co-installing
|
||||
# with the FastMCP 3 dev environment (the `test-fastmcp-4` job covers that universe).
|
||||
- name: Install dependencies
|
||||
id: install
|
||||
run: uv sync --all-extras --no-extra mcp-tasks --all-packages --group lint
|
||||
|
||||
- name: Test GitHub agentic workflow policy
|
||||
@@ -179,8 +180,8 @@ jobs:
|
||||
.github/scripts/test_docs_navigation_workflow.py
|
||||
.github/scripts/test_protect_github_dir.py
|
||||
|
||||
# The pre-commit `typecheck` hook narrows Pyright to the files a change reaches, so
|
||||
# a wrong answer here silently stops type errors reaching a developer's terminal.
|
||||
# `make typecheck-changed` narrows Pyright to the files a change reaches, so a wrong
|
||||
# answer here silently stops type errors reaching a developer's terminal.
|
||||
# It lives outside `testpaths`, in the job that installs Pyright, because one of its
|
||||
# tests runs Pyright for real.
|
||||
- name: Test the incremental typecheck script
|
||||
@@ -265,26 +266,36 @@ jobs:
|
||||
with:
|
||||
extra_args: --all-files --verbose
|
||||
env:
|
||||
SKIP: ${{ needs.classify.outputs.pyright_changed == 'true' && 'no-commit-to-branch' || 'no-commit-to-branch,typecheck' }}
|
||||
SKIP: no-commit-to-branch
|
||||
# pre-commit hooks shell out via `uv run`; without this they would re-sync the venv
|
||||
# down to the root package and drop the `--all-packages` install above.
|
||||
UV_NO_SYNC: "1"
|
||||
|
||||
# The full Pyright check runs here, not in pre-commit. With `CI` set, `make typecheck-changed`
|
||||
# hands the whole project to `make typecheck-pyright`, under the `PYRIGHT_TIME_BUDGET` that only
|
||||
# that script enforces. It runs whenever the install succeeded, so a failing pre-commit hook or
|
||||
# policy check does not hide type errors.
|
||||
- name: Type-check with Pyright
|
||||
if: ${{ !cancelled() && steps.install.outcome == 'success' && needs.classify.outputs.pyright_changed == 'true' }}
|
||||
env:
|
||||
# The `Makefile` leaves pyright single-process by default, for laptops where a worker
|
||||
# per core swaps. A runner has the memory, and this step is the longest in the job, so
|
||||
# CI opts back in.
|
||||
PYRIGHT_THREADS: auto
|
||||
# Fail a passing pyright run that outlasts this many seconds, so a change that makes
|
||||
# pyright itself slow fails its own PR instead of `main`. #8077 made pyright slow and
|
||||
# nothing caught it (see #8177): the hook ran 328 to 490 s over the five `main` runs
|
||||
# nothing caught it (see #8177): pyright ran 328 to 490 s over the five `main` runs
|
||||
# between those two. The budget was 110 (1.1x the p99 of 60 main runs, derived in
|
||||
# https://github.com/pydantic/pydantic-ai/issues/8182#issuecomment-5575016259), but by
|
||||
# 2026-09-23 the hook ran 66 to 111 s on the same day's main runs (runner variance alone
|
||||
# 2026-09-23 pyright ran 66 to 111 s on the same day's main runs (runner variance alone
|
||||
# spans about 30 s); the pyright 1.1.414 bump (#8690) and type-checking docs examples (#8646)
|
||||
# probably lifted the baseline to about 105 to 111 s, and a 110.6 s passing run failed main;
|
||||
# 150 leaves room for that variance while still failing a regression like #8077's 328 to
|
||||
# 490 s. Type-checking `pydantic-ai-harness` and `pydantic-clai2` took a passing run to
|
||||
# 156 s, so 200 keeps the same headroom.
|
||||
PYRIGHT_TIME_BUDGET: "200"
|
||||
# pre-commit hooks shell out via `uv run`; without this they would re-sync the venv
|
||||
# down to the root package and drop the `--all-packages` install above.
|
||||
UV_NO_SYNC: "1"
|
||||
run: make typecheck-changed
|
||||
|
||||
# Run inside `quality checks`, which finishes several minutes before the test/coverage critical path.
|
||||
- name: Check public API compatibility with the latest release
|
||||
@@ -394,7 +405,7 @@ jobs:
|
||||
with:
|
||||
extra_args: --all-files --verbose
|
||||
env:
|
||||
SKIP: no-commit-to-branch,format,lint,typecheck,check-cassettes
|
||||
SKIP: no-commit-to-branch,format,lint,check-cassettes
|
||||
UV_NO_SYNC: "1"
|
||||
|
||||
- name: Test documentation snippets
|
||||
|
||||
@@ -82,18 +82,6 @@ repos:
|
||||
types: [python]
|
||||
language: system
|
||||
pass_filenames: false
|
||||
- id: typecheck
|
||||
name: Typecheck
|
||||
entry: make
|
||||
# Narrows the run to the files a change can reach; when it cannot, it checks every
|
||||
# tracked file Pyright reports on, minus the unchanged `tests/` files. Only `CI`, an
|
||||
# interpreter older than 3.11 and a Pyright configuration it cannot reproduce hand the
|
||||
# whole project to `make typecheck-pyright`. CI skips this hook by id when nothing
|
||||
# Pyright reads has changed, and runs the full check otherwise.
|
||||
args: [typecheck-changed]
|
||||
language: system
|
||||
types: [python]
|
||||
pass_filenames: false
|
||||
- id: check-cassettes
|
||||
name: Check cassettes
|
||||
entry: uv
|
||||
|
||||
@@ -126,12 +126,12 @@ Do not use subprocesses to test logic that can run in-process. Starting another
|
||||
|
||||
## When to verify
|
||||
|
||||
Pre-commit runs `make lint`, `make format`, and `make typecheck-changed` automatically on every commit; that last one skips `tests/` files you did not change since it last passed (the first run checks every file), and CI additionally runs the whole type check and test suite. While iterating, only run targeted checks on the files/tests you have a specific reason to suspect:
|
||||
Pre-commit runs `make lint` and `make format` automatically on every commit. CI runs the repository-wide type check, test, and coverage gates. Do not run repository-wide Pyright, pytest, or coverage locally. While iterating, only run targeted checks on the files/tests you have a specific reason to suspect:
|
||||
|
||||
- typecheck a single file: `PYRIGHT_PYTHON_IGNORE_WARNINGS=1 uv run pyright path/to/file.py`
|
||||
- run a single test: `uv run pytest path/to/test.py::test_name`
|
||||
|
||||
Avoid `make typecheck` and `make test` between edits — both are slow and the pre-commit/CI gates cover them at the right time.
|
||||
Avoid `make typecheck` and `make test` — both are slow and CI covers them at the right time.
|
||||
|
||||
# Coding Guidelines
|
||||
|
||||
|
||||
@@ -51,7 +51,7 @@ typecheck-pyright:
|
||||
|
||||
.PHONY: typecheck-changed
|
||||
typecheck-changed: ## Run static type checking on the files reached by changes since it last passed
|
||||
@# The pre-commit hook's entry point. Whenever the narrowed set is not provably the same answer, it
|
||||
@# CI's Pyright entry point. Locally, whenever the narrowed set is not provably the same answer, it
|
||||
@# runs pyright over every tracked file it reports on, minus the unchanged `tests/` files. Only `CI`, an
|
||||
@# interpreter older than 3.11 and a pyright configuration it cannot reproduce hand the whole project to
|
||||
@# `typecheck-pyright`; see scripts/typecheck_changed.py
|
||||
|
||||
+16
-13
@@ -133,16 +133,20 @@ make
|
||||
|
||||
### Type checking
|
||||
|
||||
`make typecheck` runs Pyright over every file in the project.
|
||||
`make typecheck` runs Pyright over every file in the project. Pre-commit does not run Pyright: CI
|
||||
runs the full check on every pull request that touches something Pyright reads. Locally, type-check
|
||||
the files you changed:
|
||||
|
||||
The pre-commit hook runs `make typecheck-changed` instead, which checks only the files whose content
|
||||
changed since Pyright last passed plus everything that transitively imports them. It records what
|
||||
passed under your git directory, so the record is per-worktree and never committed.
|
||||
```bash
|
||||
PYRIGHT_PYTHON_IGNORE_WARNINGS=1 uv run pyright path/to/file.py
|
||||
```
|
||||
|
||||
CI runs that same hook, and whenever it runs it checks everything: GitHub Actions always sets `CI`,
|
||||
and on seeing it `make typecheck-changed` narrows nothing and hands the whole project to
|
||||
`make typecheck-pyright`. A fresh runner has no record to narrow against in the first place. CI still
|
||||
skips the hook outright on a pull request that touches nothing Pyright reads, as it did before.
|
||||
`make typecheck-changed` checks only the files whose content changed since Pyright last passed plus
|
||||
everything that transitively imports them, but its fallbacks below can check nearly every file, so
|
||||
prefer targeted runs while iterating. It records what passed under your git directory, so the record
|
||||
is per-worktree and never committed. CI runs the same target, and there it checks everything:
|
||||
GitHub Actions always sets `CI`, and on seeing it `make typecheck-changed` narrows nothing and hands
|
||||
the whole project to `make typecheck-pyright`.
|
||||
|
||||
Locally it follows the imports Pyright resolves statically, and never checks a file under `tests/`
|
||||
that did not itself change since the run it recorded. Tests are two thirds of the project's lines
|
||||
@@ -158,15 +162,14 @@ project. It then runs Pyright over every tracked file Pyright reports on, minus
|
||||
that did not change; a first run has no record to compare them against, so it checks all of them.
|
||||
Only three things hand the whole project to `make typecheck-pyright`: `CI`, an interpreter older
|
||||
than Python 3.11, which is what it needs to read `pyproject.toml`, and a Pyright configuration it
|
||||
cannot reproduce. Every other run considers tracked files only, so run `make typecheck` yourself
|
||||
before relying on a green hook for a file you have not added.
|
||||
cannot reproduce.
|
||||
|
||||
A full run is single-process unless `PYRIGHT_THREADS` says otherwise, and CI sets it to `auto`.
|
||||
The variable turns on Pyright's parallel check phase, which reaches the same diagnostics in less
|
||||
wall time: `auto` is up to one worker per logical core, and a positive integer caps them. Only
|
||||
`make typecheck-pyright` reads it, so the hook picks it up only on a run that hands the whole
|
||||
project over: `CI`, an interpreter older than Python 3.11, or a Pyright configuration it cannot
|
||||
reproduce. A run it narrows, or runs itself over the reduced set, stays single-process.
|
||||
`make typecheck-pyright` reads it, so `make typecheck-changed` picks it up only on a run that hands
|
||||
the whole project over: `CI`, an interpreter older than Python 3.11, or a Pyright configuration it
|
||||
cannot reproduce. A run it narrows, or runs itself over the reduced set, stays single-process.
|
||||
|
||||
```bash
|
||||
export PYRIGHT_THREADS=auto
|
||||
|
||||
@@ -613,7 +613,7 @@ def test_a_failing_run_over_the_time_budget_reports_the_failure_not_the_time(
|
||||
def test_ci_over_the_time_budget_fails_and_records_nothing(
|
||||
project: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
):
|
||||
# The pre-commit step that sets `PYRIGHT_TIME_BUDGET` sets `CI` too, so the full run handed
|
||||
# The CI step that sets `PYRIGHT_TIME_BUDGET` sets `CI` too, so the full run handed
|
||||
# to the Makefile is the only one production measures.
|
||||
monkeypatch.setenv('CI', 'true')
|
||||
monkeypatch.setenv('PYRIGHT_TIME_BUDGET', '10')
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
|
||||
`make typecheck` runs Pyright over every file in `[tool.pyright] include`, and takes about
|
||||
as long whether one file changed or a thousand. That is the right trade in CI and the wrong
|
||||
one on every commit, so the pre-commit hook runs this instead: it narrows the run to the
|
||||
one on every local run, so `make typecheck-changed` runs this instead: it narrows the run to the
|
||||
files whose content changed since Pyright last passed, plus everything that transitively
|
||||
imports them.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user