mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-11-10 17:48:43 -10:00
Nicholas TindleandClaude Opus 5.5
f90049a17a
### Why / What / How
**Why:** Agents and AutoPilot can't answer "how many views and clones
did we get?". GitHub's official MCP server, which is in our catalog, has
no traffic tools; there's an open request for them
(github/github-mcp-server#3164). The nightly `github-repo-stats`
workflow was removed in #15169 (an internal collector now archives the
org's own repos), so nothing on the platform reads traffic for a user's
repository. It's on the growth and marketing analytics wish list, next
to PostHog (already in the MCP catalog), Hacker News, Google Analytics
and Search Console.
**What:** One read-only block in the GitHub family, Github Get
Repository Traffic. One run returns what a repository's Insights >
Traffic page shows.
**How:**
- Calls the four traffic endpoints at the same time (`asyncio.gather`)
through the existing `get_api`:
- `GET /repos/{owner}/{repo}/traffic/views?per=day|week`
- `GET /repos/{owner}/{repo}/traffic/clones?per=day|week`
- `GET /repos/{owner}/{repo}/traffic/popular/referrers` (top 10)
- `GET /repos/{owner}/{repo}/traffic/popular/paths` (top 10)
- **Inputs:** GitHub credentials (`repo` scope), `repo_url` (a
github.com link or `owner/repo`, checked by `normalize_repo_path`) and
`period` (`day` by default, or `week`).
- **Outputs:** `views`, `unique_visitors`, `views_by_period`, `clones`,
`unique_cloners`, `clones_by_period`, `referrers` and `popular_content`,
named after the Traffic page's labels. Each popular page also gets its
github.com `url`. Lists only, no per-item outputs: they are short and
read whole.
- **Errors:**
- A 403 saying the caller can't read traffic ("Must have push access to
repository", or "Resource not accessible by ..." for a fine-grained
token) becomes: "GitHub only shows traffic to people with push access to
owner/repo. Connect a GitHub account that can push to it; a fine-grained
token also needs the Administration (read) permission."
- A 404 says the repository wasn't found or the account can't see it.
- Other 403s (rate limits, SAML SSO, IP allow lists) and 401s keep
GitHub's message. A 401 reaches AutoPilot unchanged, so its reconnect
prompt still works.
- An invalid repository is a `BlockInputError`, raised before any
request.
- **Conventions:** each request is a static method the self-test mocks.
`run()` does the gather, error mapping and parsing, so the self-test
parses realistic payloads. `run()` isn't wrapped in a broad
`try/except`. Effect is `READ`, so AutoPilot runs it without asking.
Categories: Developer Tools and Marketing.
### Changes 🏗️
- `backend/blocks/github/traffic.py`: `GithubGetRepositoryTrafficBlock`
(`aa46e092-b04f-492a-ae13-bb920f1469c2`) and its `TrafficCount`,
`TrafficReferrer` and `PopularContent` output types.
- `backend/blocks/github/traffic_test.py` (29 tests) and
`traffic_effects_test.py` (pins `BlockEffect.READ`).
- Docs: `docs/integrations/block-integrations/github/traffic.md`, plus
the generated `README.md` and `SUMMARY.md` entries.
- No existing blocks, settings or env vars change.
- **What users need:** push (write) access to the repository. The GitHub
connection or a classic token needs the `repo` scope; a fine-grained
token needs the Administration repository permission (read). GitHub
keeps 14 days, so the docs suggest a daily run that appends to a Google
Sheet for longer history.
### Agents and large language models used
- Claude Code with Claude Opus 5.5
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Block self-test passes via `execute_block_test`, and
`test_block.py -k GithubGetRepositoryTraffic --noconftest` gives 6
passed (self-test, effect checks, uuid4 id)
- [x] New and existing GitHub tests: 160 passed (`traffic_test.py`,
`traffic_effects_test.py`, `test_github_blocks.py`, `--noconftest`). The
new ones cover:
- URLs and the `per` parameter for each endpoint, and `get_api` built
with `convert_urls=False`
- all four requests in flight at once
- URL and `owner/repo` input, and invalid repositories and periods
rejected before any request
- parsing realistic, empty, missing and null fields, and page links
- error mapping: no push access, fine-grained token, 404, rate limit,
401
- [x] `test_block.py -k "declares_an_effect or ids_valid or effect_none
or irreversible" --noconftest`: 3037 passed, 83 skipped
- [x] `copilot/capabilities/registry_test.py` and
`integrations/provider_logos_test.py`: 146 passed. The copilot listing
is 297 characters, and copilot search ranks the block first for "how
many clones did our github repo get".
- [x] `ruff`, `isort` and `black` clean over the whole backend;
`pyright` 0 errors
- [x] `load_all_blocks()` loads all 624 blocks; `generate_block_docs.py
--check` reports all documentation in sync
- [x] Ran the block itself against GitHub with a real token that can
push to Significant-Gravitas/AutoGPT, weekly breakdown: 40,434 views
(19,507 unique visitors), 186,288 clones (8,347 unique cloners), two
weekly buckets each, 10 referrers and 10 pages with their links. Both
`owner/repo` and a github.com URL worked.
- [x] Errors from the same live run: `torvalds/linux` (no push access)
gave the push-access message, a repository that doesn't exist gave the
not-found message, and `not a repo` was rejected before any request.
- [ ] `test_block.py` against the Docker test stack (not run; the runs
above use `--noconftest`)
**Not verified:**
- The fine-grained token 403 wording ("Resource not accessible by
personal access token") is GitHub's usual message for a missing
permission. I didn't see it from this endpoint.
- In the live check the newest daily entry was yesterday (UTC). I don't
know whether GitHub ever includes the current, partial day.
#### For configuration changes:
- [x] `.env.default` is updated or already compatible with my changes
(no new settings)
- [x] `docker-compose.yml` is updated or already compatible with my
changes (no changes)
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**). There are none.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Low Risk**
> Additive read-only integration with isolated new code and tests; no
auth, persistence, or existing block behavior changes.
>
> **Overview**
> Adds a **read-only** GitHub block, **Github Get Repository Traffic**,
so workflows can pull the same metrics as **Insights → Traffic** (14-day
views/clones, referrers, popular paths).
>
> One run **fans out four REST calls in parallel** (`views`, `clones`,
`popular/referrers`, `popular/paths`) via the existing `get_api` helper
(`convert_urls=False`). Inputs are `repo_url` (normalized with
`normalize_repo_path`), optional `period` (`day`/`week` for
views/clones), and repo-scoped credentials. Outputs aggregate totals,
per-period breakdowns, top referrers, and popular pages with built
`github.com` URLs.
>
> **Error handling** maps traffic-specific 403s and 404s to clearer
`BlockExecutionError` messages; rate limits and 401s are left for
AutoPilot. The block declares **`BlockEffect.READ`** (Developer Tools +
Marketing categories).
>
> Also adds **unit/effect tests** and integration docs (`traffic.md`,
index entries in `README.md` / `SUMMARY.md`). No changes to existing
blocks or configuration.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
f50bfd64e2. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
Co-authored-by: Claude Opus 5.5 (Claude Code) <[email protected]>
3.8 KiB
3.8 KiB
GitHub Traffic
A block for reading a repository's traffic: the views, git clones, referring sites and popular content on its Insights > Traffic page. GitHub only shows traffic to people with push (write) access to the repository. The GitHub connection or a classic personal access token needs the repo scope, and a fine-grained token needs the Administration repository permission set to read.
Github Get Repository Traffic
What it is
Get a GitHub repository's views and clones over the last 14 days, with unique visitors and cloners, plus its top referring sites and most viewed pages. Views and clones are also broken down by day or week. GitHub only shows traffic to accounts with push access to the repository.
How it works
Calls GitHub's four traffic endpoints at the same time: traffic/views and traffic/clones (with per set to day or week), traffic/popular/referrers and traffic/popular/paths. repo_url can be a github.com link or owner/repo. Anything else is rejected before a request is made. If the connected account can't push to the repository, or a fine-grained token lacks the Administration (read) permission, GitHub refuses and the block says which access is missing. A repository that doesn't exist, or that the account can't see, fails as not found. Other errors, such as rate limits, keep GitHub's own message.
GitHub keeps only the last 14 days. Days and weeks start at midnight UTC, and weeks start on Monday. Views and clones refresh every hour; referring sites and popular content refresh once a day. Clones are full git clones, not fetches. Unique counts don't add up across days: someone who visited on three days counts once in unique_visitors but once on each of those days in views_by_period. referrers and popular_content are GitHub's top 10 for the 14 days, and each page comes with its github.com link. GitHub lists each path as it was visited, so the same page can appear twice with different capitalisation.
Inputs
| Input | Description | Type | Required |
|---|---|---|---|
| repo_url | Repository URL or '{owner}/{repo}' | str | Yes |
| period | Whether to break the views and clones down by day or by week. Days and weeks start at midnight UTC, and weeks start on Monday. | "day" | "week" | No |
Outputs
| Output | Description | Type |
|---|---|---|
| error | Error message if the operation failed | str |
| views | Views in the last 14 days | int |
| unique_visitors | Unique visitors in the last 14 days. Someone who visited on several days counts once. | int |
| views_by_period | Views and unique visitors for each day or week, oldest first | List[TrafficCount] |
| clones | Git clones in the last 14 days. Counts full clones, not fetches. | int |
| unique_cloners | Unique cloners in the last 14 days | int |
| clones_by_period | Clones and unique cloners for each day or week, oldest first | List[TrafficCount] |
| referrers | Top 10 sites that sent visitors in the last 14 days, with the views and unique visitors from each | List[TrafficReferrer] |
| popular_content | Top 10 most viewed pages of the repository in the last 14 days, with the views, unique visitors and github.com link of each | List[PopularContent] |
Possible use case
Traffic History: Run the block on a daily schedule and append yesterday's views and clones to a Google Sheet, so you keep more than the 14 days GitHub holds.
Weekly Report: Every Monday, post last week's views, unique visitors, clones and top referrers to Slack.
Launch Check: After a release or a blog post, see whether views and clones went up and which sites sent the visitors.