mirror of
https://github.com/piabrota/mu.git
synced 2026-11-07 06:15:59 -10:00
The repo scripts make repos on an operator's own account ("owner": "user") (#81)
* The repo scripts make repos on an operator's own account ("owner": "user")
Any operator can now make private or public repos on their own GitHub account with the same setup as
ouroboric-dev's. A config with "owner": "user" gets it, with the differences GitHub forces:
- create-repo.sh creates through /user/repos, after checking that the token belongs to the config's
user, so a repo never lands on the wrong account.
- setup-repo.sh writes each user repo's own Actions allowlist (lint.yml's pinned actions, nothing
else), since a user has no org to hold it. On a user's private repo it skips secret scanning, push
protection and private vulnerability reporting, which GitHub doesn't offer there, and says so.
- If GitHub refuses the ruleset (a user's private repo without GitHub Pro), setup-repo.sh fails and
says main is not protected, after every other setting is in place; new-repo.sh then reports the repo
as unprotected. The ruleset itself is unchanged.
An operator's config is theirs, not the company's: it lives in scripts/orgs/local/, which git ignores
except for its README, and --org reads it from there (a name in both folders is refused). Only
generic examples are committed, in scripts/orgs/examples/: a user config and a proprietary licence
text that names no one. Invariant 9 in AGENTS.md says never to commit an operator's config, and a test
fails if any committed config is neither an org's nor an example. The fakes learn GET /user,
POST /user/repos and refused writes.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* check-repos.sh --check operator: no tracked file names the operator (invariant 9)
The check reads each repo's local clone and fails when a tracked file names the operator who runs it: their
login, their home folder's name, or a user account in scripts/orgs/local/, as whole words in any case. The
names come from the machine, so the check serves any operator and the script names no one;
CHECK_REPOS_OPERATOR replaces them. It prints the files, never the names, and fails closed on a missing
clone, an empty name list or a git error. The records of past work keep their paths as written: RECORDS
lists those files one by one, so a new plan beside them is still checked. It is a default check.
The local-qwen design, plan and how-to now give paths relative to the clones, and the design no longer
names the founder's folders. Invariant 9 also covers docs, plans and fixtures.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---------
Co-authored-by: Bruno Guilhermo de Barros Valério <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
This commit is contained in:
23 files changed
+370
-41
No files matched your search
@@ -0,0 +1,4 @@
|
||||
# Operators' own configs, such as a user account's repo config, stay on the operator's machine
|
||||
# (AGENTS.md invariant 9). Only the README that explains the folder is committed.
|
||||
/scripts/orgs/local/*
|
||||
!/scripts/orgs/local/README.md
|
||||
@@ -17,6 +17,10 @@ loosen these.
|
||||
7. Never commit raw sources that should not be published, such as AI-generated rasters or the
|
||||
founder's source images. Commit only final vectors and exports.
|
||||
8. Work on a branch, stage files by path, and open a pull request for every change.
|
||||
9. Never commit an operator's own config, such as a personal account name, a local path or a personal
|
||||
setting. The scripts serve any operator: such config goes in a gitignored folder, such as
|
||||
`scripts/orgs/local/`, and only generic examples are committed. Docs, plans and fixtures never name
|
||||
an operator either: `scripts/check-repos.sh --check operator` fails when they do.
|
||||
|
||||
## Map
|
||||
|
||||
@@ -44,7 +48,7 @@ folder.
|
||||
| Compare reviewer models: costs money unless `--dry-run`; on the planted-fault leaves in `scripts/testdata/reviewer-bench`; runs the OpenCode version gleipnir pins (installed once with npm; `--opencode BIN`, `--any-opencode`) | `scripts/ygg-reviewer-bench.sh --model MODEL --leaf DIR --sessions N --dry-run`, then without it |
|
||||
| A Phase 6 proof run (v0-T290, v0-T291): costs money without `--dry-run`; `--dry-run` first; the `software` graph on the real task with three roles and caps you set, then the grade and the evidence; never reads the suites folder or prints the credentials | `scripts/ygg-proof-run.sh --name NAME --dry-run [--planner AGENT:MODEL --implementer AGENT:MODEL --reviewer AGENT:MODEL --run-usd N ...]`, then without `--dry-run`. `--reviewer-reasoning EFFORT` (low, medium, high or off; default `medium`) sets the reviewer's reasoning effort; `--build` builds the sandbox images first; `--baseline RUN_ID` compares with a run of an earlier proof run; `--continue RUN_ID --approve GATE`, `--reject GATE --reason TEXT` or `--resume` continues a stopped or paused run (a rejection starts sessions, so it needs the credentials as an approval does); `--clip` copies the report. The run's output is in `run.txt` and the evidence in `report.txt`, in `DIR/proof-NAME-TIME`. The example config is `scripts/testdata/proof/factory.proof.example.yaml` |
|
||||
| Build a plan from reference commits, or execute one task by task | `scripts/plan-*` (`plan-generate.py`, `plan-replay.py`, `plan-states.sh`, `plan-red.sh`, `plan-defences.py`; `plan-blocks.py`, `plan-task-start.sh`, `plan-task-done.sh`, `plan-task-land.sh`) |
|
||||
| Check that the repos have the required setup, and that the realms' shared files match | `scripts/as-bot.sh --read all scripts/check-repos.sh`; `--check shared` runs the realms check alone; `--check services` (not a default, once per call) keeps the services' versions equal in niflheim-local, mimir and helheim-go |
|
||||
| Check that the repos have the required setup, and that the realms' shared files match | `scripts/as-bot.sh --read all scripts/check-repos.sh`; `--check shared` runs the realms check alone; `--check operator` (local, no GitHub) checks that no tracked file names you; `--check services` (not a default, once per call) keeps the services' versions equal in niflheim-local, mimir and helheim-go |
|
||||
| Scan git history for secrets | `scripts/scan-history.sh` |
|
||||
| Create a repo with its first commit (`README.md`, licences, base CI) and its protection | `scripts/new-repo.sh --org <org> <repo> "<description>"`. Founder only |
|
||||
| Write the licence files or the base CI into a folder | `scripts/add-licences.sh`, `scripts/add-ci.sh`. Founder only |
|
||||
|
||||
@@ -174,7 +174,11 @@ checks:
|
||||
- `reuse lint`;
|
||||
- the `AGENTS.md` structure;
|
||||
- the pinned `gitleaks` pre-commit hook;
|
||||
- that the pr-bot can reach the repo.
|
||||
- that the pr-bot can reach the repo;
|
||||
- that no tracked file names the operator who runs the check (`operator`, invariant 9): their login,
|
||||
their home folder's name, or a user account in `scripts/orgs/local/`. Docs give paths relative to
|
||||
the folder that holds the clones side by side, or through `$HOME`, and fixtures use the org. The
|
||||
records of past work that the script lists keep their paths as written.
|
||||
|
||||
Run one check with `--check <name>`, or name repos to check only those.
|
||||
|
||||
@@ -195,6 +199,9 @@ Run one check with `--check <name>`, or name repos to check only those.
|
||||
output says. `--private` makes a hidden repo, which is never added. Another org copies
|
||||
[`scripts/orgs/ouroboric-dev.json`](../../scripts/orgs/ouroboric-dev.json) to
|
||||
`scripts/orgs/<org>.json` and changes the values ([the keys](../../scripts/orgs/README.md)).
|
||||
An operator's own account works the same way with `"owner": "user"`. Its config lives in
|
||||
`scripts/orgs/local/`, which git ignores, and is never committed
|
||||
([the differences](../../scripts/orgs/README.md#a-users-own-account)).
|
||||
- **Each step also runs alone**, with the same `--org`.
|
||||
[`create-repo.sh`](../../scripts/create-repo.sh) creates empty repos.
|
||||
[`add-licences.sh`](../../scripts/add-licences.sh) and [`add-ci.sh`](../../scripts/add-ci.sh)
|
||||
|
||||
@@ -52,7 +52,7 @@ Yggdrasil. It wins on limits, privacy, offline use and learning.
|
||||
These make the work a failure even if the model runs:
|
||||
|
||||
- **The Claude account split breaks.** The founder runs two Claude Code accounts, split by folder
|
||||
(work under `~/flink` with `~/.claude`; personal under `~/brunogbv` with `~/.claude-personal`),
|
||||
(a work folder with `~/.claude`, a personal folder with `~/.claude-personal`),
|
||||
with `claude`, `claude-work` and `claude-personal` shell functions in
|
||||
`~/.config/claude-accounts.zsh` and an `account-guard.sh` hook in each config. The local setup MUST
|
||||
NOT change either config dir, either login, or the `claude` function's routing. Local Claude Code
|
||||
|
||||
@@ -117,7 +117,7 @@ Start the server, then run these commands. The figures in each answer come from
|
||||
|
||||
```zsh
|
||||
python3 -I scripts/qwen-measure.py prompt --tokens 95000 --out /tmp/long.txt \
|
||||
"$HOME/brunogbv/ouroboric/yggdrasil/gleipnir/**/*.go"
|
||||
"../yggdrasil/gleipnir/**/*.go"
|
||||
python3 -I scripts/qwen-measure.py speed --runs 1 --max-tokens 256 /tmp/long.txt
|
||||
```
|
||||
|
||||
@@ -139,7 +139,7 @@ the same prompt. Tools are approved without asking, so use a task you trust the
|
||||
|
||||
```zsh
|
||||
scripts/qwen-local.sh start
|
||||
scripts/qwen-compare.sh --repo ~/brunogbv/ouroboric/yggdrasil/odin --commit bc973e3 \
|
||||
scripts/qwen-compare.sh --repo ../yggdrasil/odin --commit bc973e3 \
|
||||
--prompt docs/5-research/2026-10-19-local-qwen-comparison/prompt.txt --out /tmp/qwen-compare-b071
|
||||
```
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ already installed at `/home/linuxbrew/.linuxbrew/bin/claude`), python3 only insi
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Repo: `~/brunogbv/ouroboric/ouroboric`, branch `local-qwen`. Stage explicit paths only; never
|
||||
- Repo: your `ouroboric` clone, branch `local-qwen`. Stage explicit paths only; never
|
||||
`git add -A` (the tree has an untracked `docs/assets/logo/` that is not ours).
|
||||
- Scripts go in `scripts/`, tests in `scripts/tests/<name>.test.sh`, in the style of
|
||||
`scripts/tests/ygg-engine.test.sh`: a temp `HOME`, fake commands, no GPU and no network. Scripts are
|
||||
@@ -85,7 +85,7 @@ already installed at `/home/linuxbrew/.linuxbrew/bin/claude`), python3 only insi
|
||||
- [ ] **Step 1: Create the branch and commit the design and this plan**
|
||||
|
||||
```bash
|
||||
cd ~/brunogbv/ouroboric/ouroboric
|
||||
cd ouroboric # your clone
|
||||
git switch main && git pull --ff-only
|
||||
git switch -c local-qwen
|
||||
git add docs/5-research/2026-10-19-local-qwen-design.md docs/5-research/2026-10-19-local-qwen-plan.md
|
||||
@@ -356,7 +356,7 @@ Expected: only `ok` lines, and no shellcheck output.
|
||||
|
||||
- [ ] **Step 5: Real smoke**
|
||||
|
||||
With the server running, in `~/brunogbv/ouroboric/ouroboric`, run:
|
||||
With the server running, in your `ouroboric` clone, run:
|
||||
`scripts/qwen-opencode.sh run "List the files in scripts/ that start with qwen, using a tool"`.
|
||||
|
||||
Expected: an answer that names the qwen scripts, after at least one tool call. The server log shows
|
||||
@@ -468,12 +468,12 @@ Expected: only `ok` lines, and no shellcheck output.
|
||||
- [ ] **Step 5: Real smoke**
|
||||
|
||||
1. Run `scripts/qwen-claude.sh init`.
|
||||
2. With the server running, in `~/brunogbv/ouroboric/ouroboric`, run
|
||||
2. With the server running, in your `ouroboric` clone, run
|
||||
`scripts/qwen-claude.sh -p "Which scripts in scripts/ start with qwen? Use a tool."`.
|
||||
3. Check the answer. Expected: it names the qwen scripts. No login prompt appears, and
|
||||
`ls ~/.claude-qwen/plugins` shows the three plugins.
|
||||
4. Check the server log. Expected: the server log shows `/v1/messages` requests.
|
||||
5. Check the personal config. Expected: `claude-whoami` in `~/brunogbv` still reports the personal
|
||||
5. Check the personal config. Expected: `claude-whoami` in the personal folder still reports the personal
|
||||
account.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
@@ -504,7 +504,7 @@ git commit -m "qwen-claude.sh: Claude Code on the local Qwen server, in its own
|
||||
- [ ] **Step 2: Speed with about 100K tokens filled**
|
||||
|
||||
Build a prompt of about 95,000 tokens from real source files (for example, the contents of
|
||||
`~/brunogbv/ouroboric/yggdrasil/gleipnir`, repeated). Send it with `max_tokens: 256`, with MTP on.
|
||||
the `gleipnir` clone, repeated). Send it with `max_tokens: 256`, with MTP on.
|
||||
Record the prompt speed, the decode speed and the VRAM in use. If the request fails for lack of
|
||||
memory, lower `QWEN_CTX` in steps of 8192 tokens, but not below `65536`. Record the result.
|
||||
|
||||
|
||||
+46
-2
@@ -1,8 +1,12 @@
|
||||
#!/usr/bin/env bash
|
||||
# Check that ouroboric-dev repos have the required setup. See docs/4-ways-of-working/docs-standard.md.
|
||||
# check-repos.sh [--check settings|security|ruleset|reuse|agents|gitleaks|shared|bot|services]... [repo...]
|
||||
# check-repos.sh [--check settings|security|ruleset|reuse|agents|gitleaks|shared|bot|operator|services]... [repo...]
|
||||
# services is not a default check and runs once per call: it keeps the services' versions equal in niflheim-local's
|
||||
# pack, mimir's client-gallery-lite Dockerfile and helheim-go's template CI.
|
||||
# operator fails when a tracked file in a repo's local clone names the operator who runs the check (invariant 9):
|
||||
# their login, their home folder's name, or a user account in scripts/orgs/local/, matched as whole words in any
|
||||
# case. CHECK_REPOS_OPERATOR (names, space-separated) replaces those names. It prints the files, never the names.
|
||||
# The past records in RECORDS keep their paths as written; every other file is checked.
|
||||
# The bot check also checks that the pr-bot can't reach any hidden repo. Run it under
|
||||
# as-bot.sh --read all, because it reads the pr-bot's own list of repos.
|
||||
# shellcheck disable=SC2015,SC2329 # `cond && ok || bad` reports only (ok and bad just print); checks run by name as check_$c
|
||||
@@ -16,6 +20,23 @@ LATER=" mimir " # standard repos checked once they exist; side-by-side clone
|
||||
REALMS=" midgard-react helheim-go " # build realms; they share scripts, and helheim-go's copy is the source
|
||||
SHARED=(devcontainer/checks/docs.sh devcontainer/checks/protected-common)
|
||||
HIDDEN=" huldra " # private repos that agents must never see
|
||||
# Records of past work that name the founder's paths as they were; a new record names no operator.
|
||||
RECORDS=(
|
||||
yggdrasil:docs/2-engineering/changes/2026-10-18-base-ci-rollout/tasks.md
|
||||
yggdrasil:docs/2-engineering/changes/2026-10-18-repo-bootstrap/tasks.md
|
||||
yggdrasil:docs/2-engineering/changes/2026-10-19-voluspa/tasks.md
|
||||
yggdrasil:docs/2-engineering/changes/2026-10-22-gallery-runtime/tasks.md
|
||||
yggdrasil:docs/2-engineering/plans/2026-10-12-yggdrasil-v0/tasks.md
|
||||
yggdrasil:docs/2-engineering/plans/2026-10-15-yggdrasil-v0-phase-1/tasks.md
|
||||
yggdrasil:docs/2-engineering/plans/2026-10-15-yggdrasil-v0-phase-2/tasks.md
|
||||
yggdrasil:docs/2-engineering/plans/2026-10-16-yggdrasil-v0-phase-3/tasks.md
|
||||
yggdrasil:docs/2-engineering/plans/2026-10-16-yggdrasil-v0-phase-4/tasks.md
|
||||
yggdrasil:docs/2-engineering/plans/2026-10-17-yggdrasil-v0-phase-5a/tasks.md
|
||||
yggdrasil:docs/2-engineering/plans/2026-10-17-yggdrasil-v0-phase-5b/tasks.md
|
||||
yggdrasil:docs/2-engineering/plans/2026-10-18-yggdrasil-v0-phase-5c/tasks.md
|
||||
yggdrasil:docs/2-engineering/plans/2026-10-22-yggdrasil-v0-phase-6/tasks.md
|
||||
yggdrasil:docs/5-research/2026-10-12-sandbox-spike.md
|
||||
)
|
||||
unset BOT_REPOS # set only by bot_repos below, never from the environment
|
||||
gh_api() { env -u GITHUB_TOKEN gh api "$@"; }
|
||||
local_path() { if [[ "$LEAVES" == *" $1 "* ]]; then echo "$ROOT/yggdrasil/$1"; else echo "$ROOT/$1"; fi; }
|
||||
@@ -128,6 +149,29 @@ check_services() { # once per call: the three copies of the services' versions;
|
||||
else ok niflheim-local services "postgres minors differ, allowed: $imgminor (image) and $pkgminor (package)"; fi
|
||||
}
|
||||
|
||||
operator_names() { # the names of the operator who runs the check, one per line
|
||||
local -a given
|
||||
if [ -n "${CHECK_REPOS_OPERATOR+set}" ]; then read -ra given <<<"$CHECK_REPOS_OPERATOR"; printf '%s\n' ${given[@]+"${given[@]}"}; return; fi
|
||||
printf '%s\n' "${USER:-$(id -un)}" "$(basename "$HOME")"
|
||||
local f
|
||||
for f in "${REPO_ORGS_DIR:-$(dirname "${BASH_SOURCE[0]}")/orgs}"/local/*.json; do
|
||||
[ -f "$f" ] && jq -r 'select(.owner == "user") | .org // empty' "$f" 2>/dev/null
|
||||
done
|
||||
}
|
||||
check_operator() {
|
||||
local d n names=() ex=() rec hits rc=0
|
||||
d="$(local_path "$1")"
|
||||
[ -e "$d/.git" ] || { bad "$1" operator "no local clone at $d"; return; }
|
||||
while IFS= read -r n; do [ -z "$n" ] || names+=(-e "$n"); done < <(operator_names | sort -u)
|
||||
[ ${#names[@]} -gt 0 ] || { bad "$1" operator "no operator names to look for"; return; }
|
||||
for rec in "${RECORDS[@]}"; do [ "${rec%%:*}" != "$1" ] || ex+=(":(exclude)${rec#*:}"); done
|
||||
hits="$(git -C "$d" grep -lIiwF "${names[@]}" -- . ${ex[@]+"${ex[@]}"})" || rc=$? # 1: no match
|
||||
case "$rc" in
|
||||
0) bad "$1" operator "names the operator in $(paste -sd' ' - <<<"$hits")" ;;
|
||||
1) ok "$1" operator ;;
|
||||
*) bad "$1" operator "git grep failed in $d" ;;
|
||||
esac
|
||||
}
|
||||
bot_repos() { # the pr-bot's installation repos, once; ERROR if they can't be listed
|
||||
[ -n "${BOT_REPOS+x}" ] && return
|
||||
local out
|
||||
@@ -149,7 +193,7 @@ check_hidden() { # fails closed: a repo agents must never see
|
||||
|
||||
checks=(); repos=()
|
||||
while [ $# -gt 0 ]; do case "$1" in --check) checks+=("${2:?--check needs a name}"); shift 2 ;; *) repos+=("$1"); shift ;; esac; done
|
||||
[ ${#checks[@]} -gt 0 ] || checks=(settings security ruleset reuse agents gitleaks shared bot)
|
||||
[ ${#checks[@]} -gt 0 ] || checks=(settings security ruleset reuse agents gitleaks shared bot operator)
|
||||
for c in "${checks[@]}"; do declare -F "check_$c" >/dev/null || { echo "unknown check: $c"; exit 1; }; done
|
||||
repo_checks=(); services=0 # services compares three repos, so it runs once, not once per repo
|
||||
for c in "${checks[@]}"; do if [ "$c" = services ]; then services=1; else repo_checks+=("$c"); fi; done
|
||||
|
||||
+16
-5
@@ -1,13 +1,15 @@
|
||||
#!/usr/bin/env bash
|
||||
# shellcheck source-path=SCRIPTDIR
|
||||
# Create new, empty repos in the org that a config names. Each repo has no commits and no settings
|
||||
# beyond the visibility, so nothing in it is protected yet. new-repo.sh is the normal way to run this
|
||||
# script: it adds the first commit and protects the repo. If any repo already exists, a name repeats or
|
||||
# Create new, empty repos in the org that a config names, or in a user's own account when the config
|
||||
# has "owner": "user". Each repo has no commits and no settings beyond the visibility, so nothing in it
|
||||
# is protected yet. new-repo.sh is the normal way to run this script: it adds the first commit and
|
||||
# protects the repo. If any repo already exists, a name repeats or
|
||||
# a description is bad, nothing is created. Stops at the first error. A description is one line, with
|
||||
# no control characters and no leading or trailing spaces: new-repo.sh makes it the first paragraph of
|
||||
# README.md.
|
||||
# Founder-run: GH_TOKEN must hold the short-lived admin token, with Administration read and write
|
||||
# (the pr-bot can't create repos).
|
||||
# (the pr-bot can't create repos). For a user config, the token is the user's own, and it must
|
||||
# belong to the config's account, or nothing is created.
|
||||
# GH_TOKEN=... create-repo.sh (--org <name>|--config <path>) [--private] [--check] <repo> "<description>" [<repo> "<description>"]...
|
||||
# --check runs every check, creates nothing and prints nothing. --private makes private repos, such as
|
||||
# huldra, the hidden benchmark suite.
|
||||
@@ -54,6 +56,15 @@ for ((i = 0; i < ${#args[@]}; i += 2)); do
|
||||
[ "$ok" = true ] || die "$r: the description must be one line, with no control characters and no leading or trailing spaces"
|
||||
for ((j = 0; j < i; j += 2)); do [ "${args[j]}" != "$r" ] || die "$r is named twice"; done
|
||||
done
|
||||
# A user's repos are created on the token's own account, so the token must be the config's account.
|
||||
create="/orgs/$RC_ORG/repos"
|
||||
if [ "$RC_OWNER" = user ]; then
|
||||
code="$(request GET /user)"
|
||||
[ "$code" = 200 ] || die "can't tell whose token this is (HTTP $code); nothing was created"
|
||||
login="$(jq -r '.login // empty' "$T/reply")"
|
||||
[ "${login,,}" = "${RC_ORG,,}" ] || die "the token belongs to ${login:-no user}, not $RC_ORG; nothing was created"
|
||||
create=/user/repos
|
||||
fi
|
||||
for ((i = 0; i < ${#args[@]}; i += 2)); do
|
||||
r="${args[i]}"
|
||||
code="$(request GET "/repos/$RC_ORG/$r")"
|
||||
@@ -65,7 +76,7 @@ for ((i = 0; i < ${#args[@]}; i += 2)); do
|
||||
r="${args[i]}"
|
||||
body="$(jq -cn --arg n "$r" --arg d "${args[i + 1]}" --arg v "$visibility" \
|
||||
'{name: $n, description: $d, visibility: $v, auto_init: false, has_issues: false, has_wiki: false, has_projects: false}')"
|
||||
code="$(request POST "/orgs/$RC_ORG/repos" "$body")"
|
||||
code="$(request POST "$create" "$body")"
|
||||
[ "$code" = 201 ] || die "$r: GitHub refused to create it (HTTP $code: $(message))"
|
||||
echo "created $r"
|
||||
done
|
||||
+6
-2
@@ -1,7 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
# shellcheck source-path=SCRIPTDIR
|
||||
# Create new repos in the org that a config names, each with a first commit and the required setup, so
|
||||
# agents only ever see a protected repo. Every check runs before the first repo is created: if any repo
|
||||
# Create new repos in the org that a config names, or in a user's own account for a config with
|
||||
# "owner": "user", each with a first commit and the required setup, so agents only ever see a protected
|
||||
# repo. Every check runs before the first repo is created: if any repo
|
||||
# already exists, a name repeats, a --ci piece is unknown or the config is bad, nothing is created.
|
||||
# Then, for each repo in turn: it builds the first commit in a temporary folder, with README.md (the
|
||||
# name as the title, the description as the first paragraph), the licence files (add-licences.sh) and
|
||||
@@ -13,6 +14,9 @@
|
||||
# repository access "All repositories", with Administration, Contents and Workflows read and write
|
||||
# (the pr-bot can't create repos). A real run confirmed this token on 2026-10-18; a token limited to
|
||||
# selected repositories is untested. The org's Actions allowlist must hold lint.yml's two actions.
|
||||
# For a user config, the token is owned by that user instead, with the same access, and each repo holds
|
||||
# its own allowlist (setup-repo.sh writes it). GitHub refuses rulesets on a user's private repo without
|
||||
# GitHub Pro: then setup-repo.sh fails, and the repo is reported as not protected.
|
||||
# GH_TOKEN=... new-repo.sh (--org <name>|--config <path>) [--private] [--ci <piece>,...] <repo> "<description>" [<repo> "<description>"]...
|
||||
# --private makes private repos, such as huldra, the hidden benchmark suite. --ci replaces the
|
||||
# config's base CI pieces. After each repo, it prints the config's after-create message, such as how
|
||||
|
||||
+28
-4
@@ -1,13 +1,15 @@
|
||||
# Org configs
|
||||
|
||||
One JSON file per org, `scripts/orgs/<org>.json`. The repo scripts read it with `--org <org>`, or with
|
||||
`--config <path>` for a file elsewhere. To add an org, copy `ouroboric-dev.json` as the example and
|
||||
change the values. A key that is not listed here, or a value that breaks a rule, stops the script with
|
||||
One JSON file per org, `scripts/orgs/<org>.json`. An operator's own account, with `"owner": "user"`,
|
||||
has its config in [`local/`](local/README.md), which git ignores: never commit one. The repo scripts
|
||||
read a config with `--org <name>`, from either folder, or with `--config <path>` for a file elsewhere.
|
||||
To add an org, copy `ouroboric-dev.json` as the example and change the values. A key that is not listed here, or a value that breaks a rule, stops the script with
|
||||
exit code 2.
|
||||
|
||||
| Key | Required | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `org` | yes | The GitHub org. It must equal the name given to `--org`. Letters, digits, `.`, `_`, `-`; starts with a letter or digit. |
|
||||
| `org` | yes | The GitHub org, or the user for `"owner": "user"`. It must equal the name given to `--org`. Letters, digits, `.`, `_`, `-`; starts with a letter or digit. |
|
||||
| `owner` | no | `org` (the default) or `user`. [A user's account](#a-users-own-account) gets the same setup, with the differences below. |
|
||||
| `licence` | yes | SPDX id of the main licence, such as `AGPL-3.0-only`. Letters, digits, `.`, `+`, `-`. |
|
||||
| `docs_licence` | with `docs_paths` | SPDX id of the licence for the docs. Set both `docs_licence` and `docs_paths`, or neither. |
|
||||
| `docs_paths` | with `docs_licence` | Non-empty list of path globs that take `docs_licence`, such as `["docs/**", "README.md"]`. |
|
||||
@@ -22,3 +24,25 @@ README as its only docs file, and `reuse lint` fails on a licence text that no f
|
||||
|
||||
Another org should give its bots access to a new repo only after `new-repo.sh` succeeds, and its
|
||||
`after_create` message should say so.
|
||||
|
||||
## A user's own account
|
||||
|
||||
Any operator can make repos on their own account, outside the company, with the same setup. The
|
||||
config lives in [`local/`](local/README.md), copied from [`examples/user.json`](examples/user.json).
|
||||
The example licence is `LicenseRef-Proprietary`, with a text that names no one: the holder is the
|
||||
config's `copyright`, in `REUSE.toml`. The scripts work as for an org, with these differences:
|
||||
|
||||
- The token is fine-grained, short-lived and owned by the user, with the same access as the org's
|
||||
token. `create-repo.sh` checks that the token belongs to the config's user before it creates anything.
|
||||
- `setup-repo.sh` writes each repo's own Actions allowlist: the actions that `lint.yml` pins, at their
|
||||
SHAs, and nothing else. A user has no org to hold the list.
|
||||
- A private repo gets no secret scanning, push protection or private vulnerability reporting, because
|
||||
GitHub offers none of them there. Gitleaks in `lint` is the guard.
|
||||
- GitHub applies rulesets to a user's private repo only with GitHub Pro. Without it, `setup-repo.sh`
|
||||
fails and says that `main` is not protected. Upgrade, or make the repo public, then run
|
||||
`setup-repo.sh` again.
|
||||
- The user is the only person with write access, so nobody else can approve a pull request. The user
|
||||
merges through the admin bypass, which the ruleset still allows only for a pull request.
|
||||
- The pr-bot is installed on ouroboric-dev only, so agents can't reach the repo. To let them in, install
|
||||
a bot on the account for that repo alone.
|
||||
- Agents never write an operator's config. The operator copies the example and fills it in.
|
||||
@@ -0,0 +1,5 @@
|
||||
All rights reserved by the copyright holder named in REUSE.toml.
|
||||
|
||||
This software and its documentation are proprietary. No licence is granted to use, copy, modify,
|
||||
merge, publish, distribute, sublicense or sell them, in whole or in part, without the prior written
|
||||
permission of the copyright holder.
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"org": "your-github-user",
|
||||
"owner": "user",
|
||||
"licence": "LicenseRef-Proprietary",
|
||||
"copyright": "Your Name",
|
||||
"licence_dir": ".",
|
||||
"ci": ["reuse", "markdown", "actions"],
|
||||
"after_create": {
|
||||
"public": "{repo} is on your own account, where the pr-bot is not installed: agents can't reach it until you install a bot for {repo} alone.",
|
||||
"private": "{repo} is on your own account, where the pr-bot is not installed: agents can't reach it until you install a bot for {repo} alone."
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
# Operator configs
|
||||
|
||||
This folder holds each operator's own configs, such as the config for repos on your own GitHub
|
||||
account. Git ignores everything here except this README, so a config never leaves your machine.
|
||||
Never commit one ([invariant 9](../../../AGENTS.md#invariants)).
|
||||
|
||||
`--org <name>` reads `scripts/orgs/<name>.json` or `scripts/orgs/local/<name>.json`. A name in both
|
||||
places stops the script.
|
||||
|
||||
To set up your own account:
|
||||
|
||||
1. Copy [`../examples/user.json`](../examples/user.json) to `<your-github-user>.json` here.
|
||||
2. Set `org` to your GitHub user, `copyright` to your name, and `licence_dir` to `<your-github-user>`.
|
||||
3. Copy [`../examples/LicenseRef-Proprietary.txt`](../examples/LicenseRef-Proprietary.txt) into a
|
||||
folder `<your-github-user>/` here, or put your own licence text there and change `licence`.
|
||||
4. Run `new-repo.sh --org <your-github-user> [--private] <repo> "<description>"` with your own token
|
||||
([the differences from an org](../README.md#a-users-own-account)).
|
||||
+15
-6
@@ -1,14 +1,16 @@
|
||||
#!/usr/bin/env bash
|
||||
# One config file per org, read by create-repo.sh, add-licences.sh, add-ci.sh, setup-repo.sh and
|
||||
# new-repo.sh. Source this file; do not run it. It defines functions only.
|
||||
# Founder and agents run the scripts that source it; the config lives in scripts/orgs/<org>.json
|
||||
# (see scripts/orgs/README.md).
|
||||
# Founder and agents run the scripts that source it; the config lives in scripts/orgs/<org>.json, or
|
||||
# for an operator's own account in scripts/orgs/local/<name>.json, which git ignores (see
|
||||
# scripts/orgs/README.md).
|
||||
#
|
||||
# Use: . repo-config.sh; repo_config_args "$@"; repo_config_load
|
||||
# repo_config_args ARGS... reads a leading --org <name> or --config <path>, given once. Sets RC_FILE (absolute),
|
||||
# repo_config_args ARGS... reads a leading --org <name> or --config <path>, given once. --org looks in orgs/
|
||||
# and orgs/local/, and refuses a name found in both. Sets RC_FILE (absolute),
|
||||
# RC_REST (the other arguments) and RC_FROM (--config "$RC_FILE", to pass on).
|
||||
# repo_config_load checks RC_FILE and sets RC_ORG, RC_LICENCE, RC_DOCS_LICENCE, RC_DOCS_PATHS,
|
||||
# RC_COPYRIGHT, RC_LICENCE_DIR, RC_ISSUES_ON and RC_CI.
|
||||
# RC_COPYRIGHT, RC_LICENCE_DIR, RC_ISSUES_ON, RC_CI and RC_OWNER (org or user).
|
||||
# repo_after_create public|private REPO prints the config's after-create message for REPO, if any.
|
||||
# Every error prints "repo-config: <message>" on stderr and exits 2.
|
||||
|
||||
@@ -25,12 +27,13 @@ RC_CHECK='
|
||||
def isstr: type == "string";
|
||||
def name($re): isstr and test($re);
|
||||
def text: isstr and . != "" and (test("[\"\\\\\n]") | not);
|
||||
["org","licence","docs_licence","docs_paths","copyright","licence_dir","issues_on","ci","after_create"] as $known
|
||||
["org","owner","licence","docs_licence","docs_paths","copyright","licence_dir","issues_on","ci","after_create"] as $known
|
||||
| if type != "object" then "the config must be a JSON object"
|
||||
elif (keys - $known | length) > 0 then "unknown key: \(keys - $known | first)"
|
||||
elif (["org","licence","copyright"] - keys | length) > 0 then "missing required key: \(["org","licence","copyright"] - keys | first)"
|
||||
elif (has("docs_licence") != has("docs_paths")) then "docs_licence and docs_paths go together: set both or neither"
|
||||
elif (.org | name("\\A[A-Za-z0-9][A-Za-z0-9._-]*\\z") | not) then "bad org name"
|
||||
elif has("owner") and (.owner | . == "org" or . == "user" | not) then "owner must be org or user"
|
||||
elif (.licence | name("\\A[A-Za-z0-9][A-Za-z0-9.+-]*\\z") | not) then "bad licence"
|
||||
elif (.copyright | text | not) then "copyright must be non-empty with no \", \\ or newline"
|
||||
elif has("docs_licence") and (.docs_licence | name("\\A[A-Za-z0-9][A-Za-z0-9.+-]*\\z") | not) then "bad docs_licence"
|
||||
@@ -49,7 +52,12 @@ repo_config_args() {
|
||||
[ $# -ge 2 ] || rc_die "--org needs a name"
|
||||
[[ "$2" =~ ^[A-Za-z0-9][A-Za-z0-9-]*$ ]] || rc_die "bad org name: $2"
|
||||
RC_WANT_ORG="$2"
|
||||
RC_FILE="${REPO_ORGS_DIR:-$RC_DIR/orgs}/$2.json"
|
||||
local dir="${REPO_ORGS_DIR:-$RC_DIR/orgs}"
|
||||
RC_FILE="$dir/$2.json"
|
||||
if [ -f "$dir/local/$2.json" ]; then
|
||||
[ ! -f "$RC_FILE" ] || rc_die "$2 is in both $dir and $dir/local: remove one"
|
||||
RC_FILE="$dir/local/$2.json"
|
||||
fi
|
||||
shift 2
|
||||
;;
|
||||
--config)
|
||||
@@ -82,6 +90,7 @@ repo_config_load() {
|
||||
RC_DOCS_PATHS="$(jq -r '[(.docs_paths // [])[] | tojson] | join(", ")' "$RC_FILE")"
|
||||
RC_ISSUES_ON="$(jq -r '(.issues_on // []) | if length == 0 then "" else " " + join(" ") + " " end' "$RC_FILE")"
|
||||
RC_CI="$(jq -r '(.ci // []) | join(",")' "$RC_FILE")"
|
||||
RC_OWNER="$(jq -r '.owner // "org"' "$RC_FILE")"
|
||||
RC_LICENCE_DIR=''
|
||||
dir="$(jq -r '.licence_dir // ""' "$RC_FILE")"
|
||||
if [ -n "$dir" ]; then
|
||||
|
||||
+32
-5
@@ -7,7 +7,13 @@
|
||||
# ci.yml gives "test". A repo with neither workflow requires nothing.
|
||||
# A push after approval dismisses the approval, and the last push needs approval from someone else,
|
||||
# so the pr-bot can't land commits nobody reviewed.
|
||||
# Founder-run: GH_TOKEN must hold the short-lived admin token (agents' tokens can't change settings).
|
||||
# A user config ("owner": "user") gets the same setup, with two changes. Each repo holds its own Actions
|
||||
# allowlist, lint.yml's pinned actions, because a user has no org to hold it. And a private repo gets no
|
||||
# secret scanning, push protection or private vulnerability reporting, because GitHub offers none there.
|
||||
# If GitHub refuses the ruleset, such as on a user's private repo without GitHub Pro, the script fails
|
||||
# and says main is not protected, after every other setting is in place.
|
||||
# Founder-run: GH_TOKEN must hold the short-lived admin token (agents' tokens can't change settings), or
|
||||
# for a user config the user's own token with the same permissions.
|
||||
# GH_TOKEN=... setup-repo.sh (--org <name>|--config <path>) <repo>
|
||||
set -euo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
@@ -24,10 +30,25 @@ gh_api() { env -u GITHUB_TOKEN gh api "$@"; }
|
||||
if [[ "${RC_ISSUES_ON:-}" == *" $repo "* ]]; then ISSUES=true; else ISSUES=false; fi
|
||||
gh_api -X PATCH "$R" -F has_issues=$ISSUES -F has_wiki=false -F has_projects=false \
|
||||
-F allow_squash_merge=true -F allow_merge_commit=false -F allow_rebase_merge=false -F delete_branch_on_merge=true >/dev/null
|
||||
gh_api -X PATCH "$R" --input - >/dev/null <<'EOF'
|
||||
visibility=''
|
||||
[ "$RC_OWNER" != user ] || visibility="$(gh_api "$R" -q .visibility)"
|
||||
if [ "$visibility" = private ]; then
|
||||
echo "setup-repo: $repo is a user's private repo, where GitHub offers no secret scanning, push protection or private vulnerability reporting: gitleaks in lint is the guard" >&2
|
||||
else
|
||||
gh_api -X PATCH "$R" --input - >/dev/null <<'EOF'
|
||||
{"security_and_analysis":{"secret_scanning":{"status":"enabled"},"secret_scanning_push_protection":{"status":"enabled"}}}
|
||||
EOF
|
||||
gh_api -X PUT "$R/private-vulnerability-reporting" --silent
|
||||
gh_api -X PUT "$R/private-vulnerability-reporting" --silent
|
||||
fi
|
||||
# A user has no org to hold the Actions allowlist, so the repo holds it: lint.yml's actions at the SHAs
|
||||
# it pins, and nothing else.
|
||||
if [ "$RC_OWNER" = user ]; then
|
||||
pins="$(sed -n 's/^ *- uses: \([^ ]*\).*/\1/p' "$HERE/templates/ci/lint.yml" | jq -Rnc '[inputs]')"
|
||||
[ "$pins" != '[]' ] || die "templates/ci/lint.yml names no actions"
|
||||
gh_api -X PUT "$R/actions/permissions" --input - >/dev/null <<<'{"enabled":true,"allowed_actions":"selected"}'
|
||||
jq -cn --argjson p "$pins" '{github_owned_allowed:false,verified_allowed:false,patterns_allowed:$p}' \
|
||||
| gh_api -X PUT "$R/actions/permissions/selected-actions" --input - >/dev/null
|
||||
fi
|
||||
RULESET='{"name":"protect-main","target":"branch","enforcement":"active",
|
||||
"conditions":{"ref_name":{"include":["~DEFAULT_BRANCH"],"exclude":[]}},
|
||||
"bypass_actors":[{"actor_id":5,"actor_type":"RepositoryRole","bypass_mode":"pull_request"}],
|
||||
@@ -52,6 +73,12 @@ if [ ${#checks[@]} -gt 0 ]; then
|
||||
fi
|
||||
RULESET+=']}'
|
||||
id="$(gh_api "$R/rulesets" -q '.[]|select(.name=="protect-main")|.id' | head -1)"
|
||||
if [ -n "$id" ]; then gh_api -X PUT "$R/rulesets/$id" --input - >/dev/null <<<"$RULESET"
|
||||
else gh_api -X POST "$R/rulesets" --input - >/dev/null <<<"$RULESET"; fi
|
||||
if [ -n "$id" ]; then write=(-X PUT "$R/rulesets/$id"); else write=(-X POST "$R/rulesets"); fi
|
||||
if ! err="$(gh_api "${write[@]}" --input - <<<"$RULESET" 2>&1 >/dev/null)"; then
|
||||
echo "setup-repo: GitHub refused the ruleset, so main is not protected: $err" >&2
|
||||
if [ "$visibility" = private ] && [[ "$err" == *"HTTP 403"* ]]; then
|
||||
echo "setup-repo: a user's private repo needs GitHub Pro for rulesets. Upgrade, or make $repo public, then run setup-repo.sh again." >&2
|
||||
fi
|
||||
exit 1
|
||||
fi
|
||||
echo "set up ${repo}"
|
||||
@@ -84,4 +84,5 @@ pull request, because it runs every hook for real.
|
||||
|
||||
An org whose Actions policy allows only listed actions (ouroboric-dev's does) must list both of `lint.yml`'s
|
||||
actions, `actions/checkout` and `actions/setup-python`, at the exact SHAs it pins. Otherwise `lint` ends in
|
||||
`startup_failure` and never runs. So a bump of either action SHA also updates the org's allowlist.
|
||||
`startup_failure` and never runs. So a bump of either action SHA also updates the org's allowlist. A
|
||||
user's repo holds its own allowlist: run `setup-repo.sh` on each of them after the bump.
|
||||
@@ -150,4 +150,39 @@ check "services, once per call whatever repos are named" [ "$(services "$r" ouro
|
||||
out="$(CHECK_REPOS_ROOT="$r" "$HERE/../check-repos.sh" --check agents gungnir 2>&1)"
|
||||
check "services, not in the default checks" [ "$(grep -c '^\(FAIL\|PASS\) niflheim-local services' <<<"$out")" = 0 ]
|
||||
check "services, the check never names the hidden suites" [ -z "$(grep -n 'hidden-suites\|huldra\|credentials\|factory.env' <(sed -n '/^check_services/,/^}/p' "$HERE/../check-repos.sh"))" ]
|
||||
# operator: no tracked file names the operator who runs the check. The names come from the machine (the
|
||||
# login and the home folder's name) and from the user accounts in orgs/local/, or from CHECK_REPOS_OPERATOR.
|
||||
O="$T/oroot"
|
||||
# clone <repo path under $O> <file> <text>: a git repo with one tracked file
|
||||
clone() { mkdir -p "$O/$1/$(dirname "$2")"; printf '%s\n' "$3" > "$O/$1/$2"
|
||||
git -C "$O/$1" init -q 2>/dev/null; git -C "$O/$1" add -- "$2"; }
|
||||
operator() { CHECK_REPOS_ROOT="$O" "$HERE/../check-repos.sh" --check operator "$@"; }
|
||||
clone asgard README.md 'cd ~/zed/ouroboric/asgard'
|
||||
clone yggdrasil/odin main_test.go 'Owner: "ouroboric-dev"'
|
||||
out="$(CHECK_REPOS_OPERATOR=zed operator asgard odin)"
|
||||
check "operator, a file that names the operator: FAIL, with the file" grep -qx 'FAIL asgard operator: names the operator in README.md' <<<"$out"
|
||||
check "operator, a repo that names only the org: PASS" grep -qx 'PASS odin operator' <<<"$out"
|
||||
check "operator, the output never prints the names" bash -c "! grep -qw zed <<<'$out'"
|
||||
check "operator, the match ignores case and needs a whole word" bash -c "
|
||||
[ \"\$(CHECK_REPOS_OPERATOR=ZED CHECK_REPOS_ROOT='$O' '$HERE/../check-repos.sh' --check operator asgard | cut -d' ' -f1)\" = FAIL ] &&
|
||||
[ \"\$(CHECK_REPOS_OPERATOR=ze CHECK_REPOS_ROOT='$O' '$HERE/../check-repos.sh' --check operator asgard | cut -d' ' -f1)\" = PASS ]"
|
||||
printf 'zed\n' > "$O/asgard/untracked.txt"
|
||||
check "operator, an untracked file is not checked" [ "$(CHECK_REPOS_OPERATOR=nobody operator asgard)" = 'PASS asgard operator' ]
|
||||
check "operator, without its clone: FAIL" [ "$(CHECK_REPOS_OPERATOR=zed operator gleipnir | cut -d' ' -f1)" = FAIL ]
|
||||
check "operator, from the machine: the login" \
|
||||
[ "$(env -u CHECK_REPOS_OPERATOR USER=zed HOME="$T/h/someone" REPO_ORGS_DIR="$T/none" CHECK_REPOS_ROOT="$O" "$HERE/../check-repos.sh" --check operator asgard | cut -d' ' -f1)" = FAIL ]
|
||||
check "operator, from the machine: the home folder's name" \
|
||||
[ "$(env -u CHECK_REPOS_OPERATOR USER=someone HOME="$T/h/zed" REPO_ORGS_DIR="$T/none" CHECK_REPOS_ROOT="$O" "$HERE/../check-repos.sh" --check operator asgard | cut -d' ' -f1)" = FAIL ]
|
||||
mkdir -p "$T/orgs/local"
|
||||
printf '%s\n' '{"org":"zed","owner":"user","licence":"MIT","copyright":"Z"}' > "$T/orgs/local/zed.json"
|
||||
check "operator, from the machine: a user account in orgs/local/" \
|
||||
[ "$(env -u CHECK_REPOS_OPERATOR USER=someone HOME="$T/h/someone" REPO_ORGS_DIR="$T/orgs" CHECK_REPOS_ROOT="$O" "$HERE/../check-repos.sh" --check operator asgard | cut -d' ' -f1)" = FAIL ]
|
||||
check "operator, no names at all: FAIL, never a silent pass" \
|
||||
[ "$(CHECK_REPOS_OPERATOR=' ' operator odin | cut -d' ' -f1)" = FAIL ]
|
||||
# The past records keep their paths as written; any other file in the same folders is checked.
|
||||
clone yggdrasil docs/2-engineering/plans/2026-10-12-yggdrasil-v0/tasks.md 'cd /home/zed/ouroboric'
|
||||
check "operator, a listed past record: PASS" [ "$(CHECK_REPOS_OPERATOR=zed operator yggdrasil)" = 'PASS yggdrasil operator' ]
|
||||
clone yggdrasil docs/2-engineering/plans/2026-11-01-next/tasks.md 'cd /home/zed/ouroboric'
|
||||
check "operator, a new plan beside the records: FAIL" grep -q 'tasks.md' <<<"$(CHECK_REPOS_OPERATOR=zed operator yggdrasil)"
|
||||
check "operator, in the default checks" grep -q 'operator' <(sed -n 's/.*checks=(\(settings.*\))$/\1/p' "$HERE/../check-repos.sh")
|
||||
exit $fail
|
||||
@@ -73,4 +73,25 @@ check "--private: exit 0" [ "$rc" = 0 ]
|
||||
check "--private creates a private repo" \
|
||||
[ "$(grep '"method": "POST"' "$T/log" | tail -1 | jq -r '.body|fromjson|.visibility')" = private ]
|
||||
|
||||
# owner user: the repo is created on the token's own account, which must be the config's.
|
||||
org_lines="$(logn)"
|
||||
mkdir -p "$T/git/acme"
|
||||
printf '{"org":"acme","owner":"user","licence":"MIT","copyright":"Acme"}\n' > "$T/me.json"
|
||||
GH_TOKEN=ghp_FAKE "$CREATE" --config "$T/me.json" --private kappa "The kappa repo." > "$T/out" 2>&1; rc=$?
|
||||
check "a user config: exit 0" [ "$rc" = 0 ]
|
||||
check "a user config creates the repo on the user's account" \
|
||||
[ "$(grep '"method": "POST"' "$T/log" | tail -1 | jq -r .path)" = /user/repos ]
|
||||
check "a user config keeps the visibility and the settings" \
|
||||
[ "$(grep '"method": "POST"' "$T/log" | tail -1 | jq -cS '.body|fromjson')" = '{"auto_init":false,"description":"The kappa repo.","has_issues":false,"has_projects":false,"has_wiki":false,"name":"kappa","visibility":"private"}' ]
|
||||
check "a user config: the repo is a bare repo under the user" [ -d "$T/git/acme/kappa.git" ]
|
||||
printf '{"org":"someone-else","owner":"user","licence":"MIT","copyright":"Acme"}\n' > "$T/other.json"
|
||||
before="$(posts)"
|
||||
GH_TOKEN=ghp_FAKE "$CREATE" --config "$T/other.json" lambda "The lambda repo." 2> "$T/err"; rc=$?
|
||||
check "a token of another user: exit 2, nothing is created" bash -c "[ $rc = 2 ] && [ '$before' = \$(grep -c '\"method\": \"POST\"' '$T/log') ]"
|
||||
check "a token of another user: both accounts are named" bash -c "grep -q someone-else '$T/err' && grep -q acme '$T/err'"
|
||||
GH_TOKEN=ghp_FAKE "$CREATE" --config "$T/other.json" --check lambda "The lambda repo." 2>/dev/null; rc=$?
|
||||
check "a token of another user fails --check too" [ "$rc" = 2 ]
|
||||
check "an org config never asks who the token is" \
|
||||
bash -c "! head -n '$org_lines' '$T/log' | jq -e 'select(.path==\"/user\")' >/dev/null"
|
||||
|
||||
exit $fail
|
||||
@@ -1,7 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
# Fake gh for the script tests: "gh api" answers GETs from $FAKE_GH_DIR/GET/<path, / as _>.json
|
||||
# (404 when the file is missing; see ERR below) and logs every other request as a JSON line in
|
||||
# $FAKE_GH_DIR/log (with the -F/-f fields as "key=value" strings).
|
||||
# $FAKE_GH_DIR/log (with the -F/-f fields as "key=value" strings). A file at
|
||||
# $FAKE_GH_DIR/ERR/<METHOD>_<path, / as _> makes that write fail, after the log, with its contents.
|
||||
set -uo pipefail
|
||||
[ "${1:-}" = api ] || { echo "fake gh: only 'gh api' is faked" >&2; exit 2; }
|
||||
shift
|
||||
@@ -19,6 +20,8 @@ done
|
||||
if [ "$method" != GET ]; then
|
||||
body=''; [ "$input" = - ] && body="$(cat)"
|
||||
jq -cn --arg m "$method" --arg p "$path" --arg b "$body" --arg f "$fields" '{method:$m,path:$p,body:$b,fields:($f|split("\n")|map(select(.!="")))}' >> "$FAKE_GH_DIR/log"
|
||||
e="$FAKE_GH_DIR/ERR/${method}_${path//\//_}"
|
||||
[ -f "$e" ] && { echo "gh: $(cat "$e")" >&2; exit 1; }
|
||||
echo '{}'; exit 0
|
||||
fi
|
||||
f="$FAKE_GH_DIR/GET/${path//\//_}.json"
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
# Fake GitHub API for the script tests. Logs each request to LOG as one JSON line.
|
||||
# python3 -I fake-github.py LOG PORTFILE
|
||||
# FAKE_PEM: the key the manifest conversion returns. FAKE_GIT_ROOT: where a created repo gets its
|
||||
# bare git repo, so a test can push to file://$FAKE_GIT_ROOT/<org>/<repo>.git.
|
||||
# bare git repo, so a test can push to file://$FAKE_GIT_ROOT/<org>/<repo>.git. FAKE_USER: the login
|
||||
# that GET /user returns, and the account that POST /user/repos creates in (default acme).
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
@@ -17,8 +18,10 @@ def repo_routes(method, path, body):
|
||||
if method == "GET" and (m := re.fullmatch(r"/repos/([^/]+)/([^/]+)", path)):
|
||||
org, name = m.groups()
|
||||
return (200, {"name": name}) if name == "taken" or (org, name) in repos else None
|
||||
if method == "POST" and (m := re.fullmatch(r"/orgs/([^/]+)/repos", path)):
|
||||
org = m.group(1)
|
||||
if method == "GET" and path == "/user":
|
||||
return 200, {"login": os.environ.get("FAKE_USER", "acme")}
|
||||
if method == "POST" and (m := re.fullmatch(r"/orgs/([^/]+)/repos|/user/repos", path)):
|
||||
org = m.group(1) or os.environ.get("FAKE_USER", "acme")
|
||||
name = json.loads(body)["name"]
|
||||
if name == "taken" or (org, name) in repos:
|
||||
return 422, {"message": "Repository creation failed."}
|
||||
|
||||
@@ -13,7 +13,7 @@ B="$T/git/ouroboric-dev"
|
||||
|
||||
mkdir -p "$B"
|
||||
: > "$T/log"
|
||||
FAKE_GIT_ROOT="$T/git" python3 -I "$HERE/fake-github.py" "$T/log" "$T/port" &
|
||||
FAKE_GIT_ROOT="$T/git" FAKE_USER=someone python3 -I "$HERE/fake-github.py" "$T/log" "$T/port" &
|
||||
for _ in $(seq 50); do [ -s "$T/port" ] && break; sleep 0.1; done
|
||||
CREATE_REPO_API="http://127.0.0.1:$(cat "$T/port")"
|
||||
CREATE_REPO_GIT="file://$T/git"
|
||||
@@ -112,4 +112,25 @@ check "a failed setup: stderr says zeta is not protected" grep -qx \
|
||||
'new-repo: zeta was created, but setup-repo.sh failed. It is not protected and agents must not get it yet. Delete it, or push a first commit and run setup-repo.sh.' "$T/err"
|
||||
check "a failed setup: it never says protected" bash -c "! grep -q protected '$T/out'"
|
||||
|
||||
# An operator's user config, end to end: the example, copied into local/ as an operator would.
|
||||
O="$T/orgs"; mkdir -p "$O/local/someone"
|
||||
cp "$HERE/../orgs/examples/LicenseRef-Proprietary.txt" "$O/local/someone/"
|
||||
jq '.org = "someone" | .copyright = "Someone" | .licence_dir = "someone"' "$HERE/../orgs/examples/user.json" > "$O/local/someone.json"
|
||||
printf '#!/bin/sh\necho "$@" >> "%s/user-setup.log"\n' "$T" > "$T/user-setup"
|
||||
chmod +x "$T/user-setup"
|
||||
GH_TOKEN=$TOKEN REPO_ORGS_DIR="$O" NEW_REPO_SETUP="$T/user-setup" "$NEW" --org someone --private mu "The mu repo." \
|
||||
</dev/null > "$T/out" 2> "$T/err"; rc=$?
|
||||
U="$T/git/someone/mu.git"
|
||||
check "a user config: exit 0" [ "$rc" = 0 ]
|
||||
check "a user config: the repo is made on the user's account, private" \
|
||||
[ "$(grep '"method": "POST"' "$T/log" | tail -1 | jq -r '.path + " " + (.body|fromjson|.visibility)')" = '/user/repos private' ]
|
||||
check "a user config: the commit holds the proprietary licence, the base CI and the README" \
|
||||
[ "$(git -C "$U" ls-tree -r --name-only main 2>/dev/null)" = "$(printf '%s\n' .github/workflows/lint.yml .markdownlint.yaml \
|
||||
.pre-commit-config.yaml LICENSE LICENSES/LicenseRef-Proprietary.txt README.md REUSE.toml)" ]
|
||||
check "a user config: REUSE.toml names the proprietary licence and the operator" \
|
||||
bash -c "git -C '$U' show main:REUSE.toml | grep -qx 'SPDX-License-Identifier = \"LicenseRef-Proprietary\"' &&
|
||||
git -C '$U' show main:REUSE.toml | grep -q 'SPDX-FileCopyrightText = \"[0-9]* Someone\"'"
|
||||
check "a user config: setup runs on the user's repo" grep -qx -- "--config $O/local/someone.json mu" "$T/user-setup.log"
|
||||
check "a user config: the message says agents can't reach it" grep -q "agents can't reach it" "$T/out"
|
||||
|
||||
exit $fail
|
||||
@@ -109,4 +109,33 @@ check "a config in a folder with a space" \
|
||||
[ "$(load --config "$T/my org/acme.json" | sed -n 's/^DIR=//p')" = "$T/my org/lic" ]
|
||||
check "a relative --config gives an absolute RC_FILE" bash -c "
|
||||
cd '$T/my org' && . '$HERE/../repo-config.sh' && repo_config_args --config acme.json && [ \"\$RC_FILE\" = '$T/my org/acme.json' ]"
|
||||
|
||||
# owner: the account is an org (the default) or a user.
|
||||
owner() { ( . "$HERE/../repo-config.sh"; repo_config_args "$@"; repo_config_load; printf '%s' "$RC_OWNER" ); }
|
||||
check "without owner, the account is an org" [ "$(owner --config "$T/min.json")" = org ]
|
||||
check "ouroboric-dev is an org" [ "$(owner --org ouroboric-dev)" = org ]
|
||||
cfg u "{$base, \"owner\": \"user\"}"
|
||||
check "owner user loads" [ "$(owner --config "$T/u.json")" = user ]
|
||||
cfg m "{$base, \"owner\": \"team\"}"
|
||||
check "an unknown owner is refused" refused --config "$T/m.json"
|
||||
cfg m "{$base, \"owner\": 1}"
|
||||
check "an owner that is not a string is refused" refused --config "$T/m.json"
|
||||
check "the user example is a user, with its licence text and no docs licence" bash -c "
|
||||
. '$HERE/../repo-config.sh'; repo_config_args --config '$HERE/../orgs/examples/user.json'; repo_config_load
|
||||
[ \"\$RC_OWNER\" = user ] && [ -s \"\$RC_LICENCE_DIR/\$RC_LICENCE.txt\" ] && [ -z \"\$RC_DOCS_LICENCE\" ]"
|
||||
|
||||
# An operator's own config lives in orgs/local/, which git ignores; --org finds it there.
|
||||
mkdir -p "$T/o/local"
|
||||
printf '%s\n' '{"org": "someone", "owner": "user", "licence": "MIT", "copyright": "Someone"}' > "$T/o/local/someone.json"
|
||||
check "--org finds an operator's config in local/" bash -c "
|
||||
. '$HERE/../repo-config.sh'; REPO_ORGS_DIR='$T/o' repo_config_args --org someone; repo_config_load
|
||||
[ \"\$RC_FILE\" = '$T/o/local/someone.json' ] && [ \"\$RC_OWNER\" = user ]"
|
||||
cp "$T/o/local/someone.json" "$T/o/someone.json"
|
||||
REPO_ORGS_DIR="$T/o" check "a name in both orgs/ and local/ is refused" refused --org someone
|
||||
REPO_ORGS_DIR="$T/o" check "a name in neither is refused" refused --org nobody
|
||||
check "git ignores the operator configs in local/" git -C "$HERE/../.." check-ignore -q scripts/orgs/local/someone.json
|
||||
check "git ignores an operator's licence texts in local/" git -C "$HERE/../.." check-ignore -q scripts/orgs/local/someone/LicenseRef-Proprietary.txt
|
||||
check "git keeps local/README.md" bash -c "! git -C '$HERE/../..' check-ignore -q scripts/orgs/local/README.md"
|
||||
check "no operator config is committed: every committed config is an org's or an example" \
|
||||
bash -c "[ -z \"\$(git -C '$HERE/../..' ls-files 'scripts/orgs/*.json' 'scripts/orgs/local/*' | grep -vx -e 'scripts/orgs/ouroboric-dev.json' -e 'scripts/orgs/examples/user.json' -e 'scripts/orgs/local/README.md')\" ]"
|
||||
exit $fail
|
||||
@@ -91,4 +91,52 @@ check "two repo arguments: exit 2" bash -c "GH_TOKEN=fake '$HERE/../setup-repo.s
|
||||
export FAKE_GH_DIR="$T/option"; mkdir -p "$FAKE_GH_DIR/GET"
|
||||
check "an unknown option: exit 2, and gh is never called" \
|
||||
bash -c "GH_TOKEN=fake '$HERE/../setup-repo.sh' --org ouroboric-dev --cl >/dev/null 2>&1; [ \$? = 2 ] && [ ! -s '$FAKE_GH_DIR/log' ]"
|
||||
# owner user: no org to hold the Actions allowlist, so each repo holds it. GitHub offers no secret
|
||||
# scanning or private vulnerability reporting on a user's private repo.
|
||||
printf '%s\n' '{"org":"me","owner":"user","licence":"MIT","copyright":"Me"}' > "$T/me.json"
|
||||
PINS="$(sed -n 's/^ *- uses: \([^ ]*\).*/\1/p' "$HERE/../templates/ci/lint.yml" | jq -Rnc '[inputs]')"
|
||||
# user_repo <dir> <visibility>: a fake for repos/me/widget with no ruleset yet
|
||||
user_repo() { export FAKE_GH_DIR="$T/$1"; mkdir -p "$FAKE_GH_DIR/GET" "$FAKE_GH_DIR/ERR"
|
||||
echo "{\"visibility\":\"$2\"}" > "$FAKE_GH_DIR/GET/repos_me_widget.json"
|
||||
echo '[]' > "$FAKE_GH_DIR/GET/repos_me_widget_rulesets.json"; }
|
||||
# logged <jq filter>: some logged request matches
|
||||
logged() { jq -e "select($1)" "$FAKE_GH_DIR/log" >/dev/null; }
|
||||
allowlisted() { logged '.method=="PUT" and .path=="repos/me/widget/actions/permissions" and (.body|fromjson|.enabled==true and .allowed_actions=="selected")' &&
|
||||
jq -se --argjson pins "$PINS" 'map(select(.method=="PUT" and .path=="repos/me/widget/actions/permissions/selected-actions"))
|
||||
|length==1 and (.[0].body|fromjson|.patterns_allowed==$pins and .github_owned_allowed==false and .verified_allowed==false)' "$FAKE_GH_DIR/log" >/dev/null; }
|
||||
export -f logged allowlisted
|
||||
export PINS
|
||||
check "lint.yml pins two actions, each by a full SHA" bash -c "jq -e 'length==2 and all(test(\"@[0-9a-f]{40}\$\"))' <<<'$PINS' >/dev/null"
|
||||
|
||||
user_repo user-private private
|
||||
GH_TOKEN=fake "$HERE/../setup-repo.sh" --config "$T/me.json" widget > /dev/null 2> "$T/err"; rc=$?
|
||||
check "a private user repo: exit 0" [ "$rc" = 0 ]
|
||||
check "a private user repo: only lint.yml's pinned actions may run" allowlisted
|
||||
check "a private user repo: no secret scanning request" bash -c "! logged '.body|test(\"secret_scanning\")'"
|
||||
check "a private user repo: no private vulnerability reporting request" bash -c "! logged '.path|test(\"private-vulnerability-reporting\")'"
|
||||
check "a private user repo: stderr names what GitHub doesn't offer" \
|
||||
bash -c "grep -q 'secret scanning' '$T/err' && grep -q 'vulnerability reporting' '$T/err'"
|
||||
check "a private user repo: the same ruleset as an org's" \
|
||||
bash -c "jq -rs 'map(select(.path==\"repos/me/widget/rulesets\"))|last|.body' \"\$FAKE_GH_DIR/log\" \
|
||||
| jq -e '([.rules[].type]|sort)==[\"deletion\",\"non_fast_forward\",\"pull_request\"] and .bypass_actors==[{\"actor_id\":5,\"actor_type\":\"RepositoryRole\",\"bypass_mode\":\"pull_request\"}]' >/dev/null"
|
||||
check "a private user repo: the same merge settings as an org's" \
|
||||
logged '.method=="PATCH" and .path=="repos/me/widget" and (.fields|index("allow_squash_merge=true") and index("has_issues=false"))'
|
||||
|
||||
user_repo user-public public
|
||||
GH_TOKEN=fake "$HERE/../setup-repo.sh" --config "$T/me.json" widget > /dev/null 2>&1
|
||||
check "a public user repo: secret scanning and push protection" logged '.path=="repos/me/widget" and (.body|test("secret_scanning_push_protection"))'
|
||||
check "a public user repo: private vulnerability reporting" logged '.method=="PUT" and .path=="repos/me/widget/private-vulnerability-reporting"'
|
||||
check "a public user repo: only lint.yml's pinned actions may run" allowlisted
|
||||
|
||||
user_repo user-free private
|
||||
echo 'Upgrade to GitHub Pro or make this repository public to enable this feature. (HTTP 403)' > "$FAKE_GH_DIR/ERR/POST_repos_me_widget_rulesets"
|
||||
GH_TOKEN=fake "$HERE/../setup-repo.sh" --config "$T/me.json" widget > "$T/out" 2> "$T/err"; rc=$?
|
||||
check "a refused ruleset: exit non-zero" [ "$rc" != 0 ]
|
||||
check "a refused ruleset: stderr says main is not protected" grep -q 'main is not protected' "$T/err"
|
||||
check "a refused ruleset on a private user repo: stderr names GitHub Pro" grep -q 'GitHub Pro' "$T/err"
|
||||
check "a refused ruleset: it never says set up" bash -c "! grep -q 'set up' '$T/out'"
|
||||
check "a refused ruleset: the Actions allowlist is already in place" allowlisted
|
||||
|
||||
export FAKE_GH_DIR="$T/acme"
|
||||
check "an org repo: the Actions allowlist is the org's, so none is written" bash -c "! logged '.path|test(\"actions/permissions\")'"
|
||||
exit $fail
|
||||
Reference in new issue
Block a user