mirror of
https://github.com/codewhale-hq/Codewhale.git
synced 2026-11-11 01:51:34 -10:00
docs(evidence): record model-assisted constitution drafting coverage
Rewrite the guided-constitution examples note for the two authoring paths (guided deterministic, model-assisted via the first configured model) sharing one schema/validator/renderer, reframe the GLM-5.2 example accordingly, and refresh the acceptance notes for the ratification flow. Add QA-matrix rows for the draft offer gating, minimal request payload, untrusted ingestion, failure fallback, and ratification provenance; fix a stale citation (the matrix referenced guided_constitution_commit_emits_structured_payload, which never existed - the real tests are guided_constitution_requires_preview_ before_save and guided_constitution_answers_shape_preview_and_saved_ payload). Refresh the current-build evidence snapshot (date, head, updated suite counts). Tests: cargo test -p codewhale-tui --bin codewhale-tui --locked setup -- --nocapture
This commit is contained in:
Hunter B committed
1 parent
fa7c4b0553
commit
4d9199f9c8
3 files changed
+207
-7
No files matched your search
@@ -5,9 +5,16 @@ constitution-first setup lane. It complements
|
||||
`docs/evidence/v0867-constitution-setup-qa-matrix.md`; it is not a release
|
||||
tag, artifact, or publish record.
|
||||
|
||||
- Date: 2026-07-01T07:03:18Z
|
||||
Representative guided output examples, including a GLM-5.2-oriented profile,
|
||||
are recorded in `docs/evidence/v0867-guided-constitution-examples.md`. As of
|
||||
this snapshot the wizard also offers model-assisted drafting: `A` on the
|
||||
Constitution step asks the first configured model to draft from the guided
|
||||
answers, gated through `UserConstitution::from_untrusted_json`
|
||||
(parse/sanitize/bound) and the same ratification preview + explicit `G` save.
|
||||
|
||||
- Date: 2026-07-02T03:54:02Z
|
||||
- Branch: `claude/v0.8.67-constitution-setup-174rj9`
|
||||
- Head: `3509fe291`
|
||||
- Head: `fa7c4b055`
|
||||
- Workspace version observed in `Cargo.toml`: `0.8.66`
|
||||
|
||||
## Covered Surfaces
|
||||
@@ -41,13 +48,14 @@ Result: 1 passed, 0 failed.
|
||||
cargo test -p codewhale-tui --bin codewhale-tui --locked constitution -- --nocapture
|
||||
```
|
||||
|
||||
Result: 41 passed, 0 failed.
|
||||
Result: 42 passed, 0 failed.
|
||||
|
||||
```sh
|
||||
cargo test -p codewhale-tui --bin codewhale-tui --locked setup -- --nocapture
|
||||
```
|
||||
|
||||
Result: 106 passed, 0 failed.
|
||||
Result: 127 passed, 0 failed (includes the model-draft request/ingestion,
|
||||
ratification, discard-on-tune, and authoring-provenance tests).
|
||||
|
||||
```sh
|
||||
cargo test -p codewhale-tui --bin codewhale-tui --locked context_report -- --nocapture
|
||||
@@ -59,7 +67,14 @@ Result: 10 passed, 0 failed.
|
||||
cargo test -p codewhale-tui --bin codewhale-tui --locked doctor_setup -- --nocapture
|
||||
```
|
||||
|
||||
Result: 2 passed, 0 failed.
|
||||
Result: 5 passed, 0 failed.
|
||||
|
||||
```sh
|
||||
cargo test -p codewhale-config --lib
|
||||
```
|
||||
|
||||
Result: 342 passed, 0 failed (includes the `untrusted_draft_*` ingestion-gate
|
||||
tests and the `constitution_authoring` round-trip/legacy-load tests).
|
||||
|
||||
```sh
|
||||
cargo test -p codewhale-tui --bin codewhale-tui --locked verification_report -- --nocapture
|
||||
|
||||
@@ -7,6 +7,8 @@ checking each surface in isolation.
|
||||
|
||||
Current-build automated text/render evidence is recorded in
|
||||
`docs/evidence/v0867-constitution-setup-current-build-evidence.md`.
|
||||
Guided constitution output examples are recorded in
|
||||
`docs/evidence/v0867-guided-constitution-examples.md`.
|
||||
|
||||
## Gate Commands
|
||||
|
||||
@@ -50,7 +52,12 @@ cargo run -p codewhale-tui --locked -- doctor --context-json | jq '.entries[] |
|
||||
| Scenario | Expected behavior | Evidence |
|
||||
| --- | --- | --- |
|
||||
| Clean home, bundled/default constitution | First-run can complete by choosing language, recording provider readiness as ready or needs-action, reviewing runtime posture, choosing bundled/default, and opening the setup report. | `/setup` `U` on Constitution step; `crates/tui/src/tui/setup/mod.rs::bundled_constitution_commit_marks_checkpoint_complete`; `doctor --json .setup.constitution.choice == "bundled"` |
|
||||
| Clean home, guided user-global constitution | Guided custom save writes `$CODEWHALE_HOME/constitution.json`, records source/validity/hash/version in `setup_state.json`, and previews the rendered block. | `crates/tui/src/tui/setup/mod.rs::guided_constitution_commit_emits_structured_payload`; `persist_user_constitution_choice_writes_constitution_and_state`; `/constitution preview` |
|
||||
| Clean home, guided user-global constitution | Guided custom save writes `$CODEWHALE_HOME/constitution.json`, records source/validity/hash/version/authoring in `setup_state.json`, and previews the rendered block before the ratifying second `G`. | `crates/tui/src/tui/setup/mod.rs::guided_constitution_requires_preview_before_save`; `guided_constitution_answers_shape_preview_and_saved_payload`; `deterministic_ratification_records_guided_authoring`; `persist_user_constitution_choice_writes_constitution_and_state`; `/constitution preview` |
|
||||
| Model-assisted draft offer gating | The `A` "ask your model to draft" action appears and responds only when the first provider/model route is ready (key/local runtime present); without a ready route the key is inert and the deterministic guided flow is unchanged. | `crates/tui/src/tui/setup/mod.rs::model_draft_key_is_inert_without_a_ready_provider`; `model_draft_key_requests_drafting_with_current_answers`; `constitution_card_gates_the_model_draft_invitation` |
|
||||
| Model-assisted draft request payload | The one-shot drafting request carries only the six guided answer labels and the UI language tag — no secrets, env, config, repo contents, or memory — plus injection-resistance and advisory-only guardrails in the system prompt. | `crates/tui/src/tui/setup/model_draft.rs::drafting_request_sends_only_answers_and_language`; `drafting_prompts_carry_the_safety_guardrails` |
|
||||
| Model-assisted draft ingestion | Model output is untrusted: fenced/prose-wrapped JSON parses, invalid or empty output is rejected with a reason, oversized fields are bounded before preview/save, unknown (runtime-policy) keys cannot persist, thinking blocks never reach the parser, and constitution-tag forgery is neutralized. | `crates/tui/src/tui/setup/model_draft.rs` ingestion tests; `crates/config/src/user_constitution.rs::untrusted_draft_*` tests |
|
||||
| Model-assisted draft failure fallback | Provider construction failure, timeout, request error, or bad JSON degrade to a status line; the deterministic guided draft still previews and ratifies. Decline is the default: not pressing `A` (or tuning `1-6`, which discards a stale draft) keeps the guided path. | `crates/tui/src/tui/ui.rs::handle_setup_constitution_model_draft` error arm; `crates/tui/src/tui/setup/mod.rs::cycling_answers_discards_the_model_draft` |
|
||||
| Model-assisted ratification | An installed model draft opens the ratification preview immediately; saving still requires the explicit `G`, records `constitution_authoring = model_drafted` plus the bounded draft's preview hash, and persists through the same single `SetupTransaction`. | `crates/tui/src/tui/setup/mod.rs::installed_model_draft_previews_then_ratifies_with_provenance`; `model_drafted_commit_round_trips_through_the_setup_transaction` |
|
||||
| Existing user update checkpoint | If the v0.8.67 checkpoint is incomplete, interactive launch opens `/setup`; choosing bundled/default is a valid completion. | `crates/tui/src/tui/setup/mod.rs::wizard_resumes_at_constitution_checkpoint_when_update_incomplete`; `crates/tui/src/tui/ui/tests.rs::setup_checkpoint_opens_after_onboarding_when_due` |
|
||||
| First-run onboarding handoff | Finishing the legacy Welcome/Language/API/trust gates opens setup when the checkpoint is due, instead of landing straight in chat. | `crates/tui/src/tui/ui/tests.rs::setup_checkpoint_opens_after_onboarding_when_due`; onboarding copy tests |
|
||||
| Existing valid user-global constitution | `/constitution` reports it as active when setup state does not select bundled/deferred/expert override; prompt assembly injects it as a separate block. | `crates/tui/src/prompts.rs::user_global_constitution_block_is_injected_separately`; `/constitution status` |
|
||||
@@ -79,7 +86,9 @@ candidate:
|
||||
health without secrets.
|
||||
3. `/setup` Runtime Posture card says constitution guidance does not change
|
||||
runtime policy silently.
|
||||
4. `/setup` Constitution step shows bundled/default and guided custom actions.
|
||||
4. `/setup` Constitution step shows bundled/default and guided custom actions,
|
||||
and — once the provider route is ready — the `A` model-draft invitation
|
||||
naming the first configured model.
|
||||
5. `/constitution` overview shows bundled, user-global, repo-local, AGENTS,
|
||||
memory/handoff, preview, and maintenance actions.
|
||||
6. `/setup report` or `codewhale doctor --json | jq '.setup'` shows
|
||||
|
||||
@@ -0,0 +1,176 @@
|
||||
# v0.8.67 Guided Constitution Examples
|
||||
|
||||
These examples show the structured output of the v0.8.67 constitution creator.
|
||||
The wizard has two authoring paths that share one schema, one validator, and
|
||||
one renderer:
|
||||
|
||||
1. **Guided deterministic** — the six guided answers map deterministically
|
||||
into `$CODEWHALE_HOME/constitution.json`. Always available; the standing
|
||||
fallback.
|
||||
2. **Model-assisted** — once the user's first provider/model route is ready,
|
||||
`A` on the Constitution step asks that first configured model (GLM-5.2 on
|
||||
Z.ai, DeepSeek, or any other route) to draft the constitution from the
|
||||
guided answers. The request carries only the six answer labels and the UI
|
||||
language tag. The reply is treated as untrusted data: the first JSON object
|
||||
is extracted, schema-parsed (unknown keys — including any runtime-policy
|
||||
keys — are dropped), sanitized (control characters and
|
||||
`<codewhale_user_constitution>` tag forgery neutralized), and bounded
|
||||
before anyone sees it. Invalid, empty, or failed drafts degrade to the
|
||||
deterministic path with a visible reason.
|
||||
|
||||
Either way, the saved artifact is the same bounded `UserConstitution` JSON,
|
||||
rendered by the same deterministic renderer into the same
|
||||
`<codewhale_user_constitution>` block — the model that drafts the law gains no
|
||||
authority from having written it. Ratification is explicit: the wizard shows
|
||||
the rendered preview and nothing persists until the user confirms with `G`.
|
||||
`setup_state.json` records the provenance (`constitution_authoring`:
|
||||
`guided` or `model_drafted`).
|
||||
|
||||
This matters for provider testing: a GLM-5.2 route receives the same
|
||||
constitution layer as any other route, and may also be the route that drafts
|
||||
it. Provider/model choice affects model behavior, context limits, pricing,
|
||||
and reasoning controls, but it does not change the constitution schema or
|
||||
silently expand runtime authority.
|
||||
|
||||
## Schema Shape
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"language": "en",
|
||||
"about": "short user/work context",
|
||||
"working_style": [
|
||||
"bounded working-style preference"
|
||||
],
|
||||
"priorities": [
|
||||
"bounded standing priority"
|
||||
],
|
||||
"autonomy_preference": "balanced",
|
||||
"notes": "bounded advisory free prose"
|
||||
}
|
||||
```
|
||||
|
||||
All text fields are bounded before save. Empty structured constitutions render
|
||||
no block. Autonomy remains guidance only; it never changes approval policy,
|
||||
sandbox mode, shell access, network defaults, trust, MCP permission, or default
|
||||
mode.
|
||||
|
||||
## Example: GLM-5.2 Coding Workbench
|
||||
|
||||
This is the kind of user-global constitution a Z.ai/GLM-5.2 user might ratify
|
||||
after choosing a coding purpose, ambitious initiative, release evidence,
|
||||
concise communication, strict boundaries, and scoped changes — whether GLM-5.2
|
||||
drafted it via `A` or the wizard rendered it deterministically. A model-drafted
|
||||
version may word the prose differently, but it must land in this same schema,
|
||||
inside these same bounds, and renders through this same block.
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"language": "en",
|
||||
"about": "A CodeWhale user who routes through Z.ai GLM-5.2 for coding work and wants a calm, evidence-first coding workbench.",
|
||||
"working_style": [
|
||||
"Keep code changes scoped to requested behavior and existing repo patterns.",
|
||||
"Keep updates concise and explain important tradeoffs briefly.",
|
||||
"Cite file paths, commands, screenshots, CI, or sources for material claims and release evidence.",
|
||||
"Treat secrets, personal data, credentials, production state, money, and publish actions as stop-and-confirm boundaries."
|
||||
],
|
||||
"priorities": [
|
||||
"Current user requests and live tool evidence outrank memory, stale handoffs, and guesses.",
|
||||
"Batch routine safe work, then stop for destructive, credential, publishing, high-cost, legal, or security-risk actions.",
|
||||
"Stop and ask before reading or spreading sensitive data, touching production systems, spending money, or publishing."
|
||||
],
|
||||
"autonomy_preference": "autonomous",
|
||||
"notes": "Guided answers: purpose=coding workbench; initiative=ambitious; evidence=release receipts; communication=concise; privacy=strict boundaries; principles=scoped changes. Freeform principle: prefer small, reviewable changes and avoid unrelated refactors unless explicitly requested. Freeform principles are advisory and do not change approval, sandbox, shell, network, trust, or MCP permissions."
|
||||
}
|
||||
```
|
||||
|
||||
Rendered block:
|
||||
|
||||
```text
|
||||
<codewhale_user_constitution source="user-global">
|
||||
User-global standing preferences (personal law: subordinate to the current user request and the global Constitution, but applies across all your projects). Treat as durable guidance, not as enforceable runtime policy.
|
||||
|
||||
About the user:
|
||||
A CodeWhale user who routes through Z.ai GLM-5.2 for coding work and wants a calm, evidence-first coding workbench.
|
||||
|
||||
Working style:
|
||||
- Keep code changes scoped to requested behavior and existing repo patterns.
|
||||
- Keep updates concise and explain important tradeoffs briefly.
|
||||
- Cite file paths, commands, screenshots, CI, or sources for material claims and release evidence.
|
||||
- Treat secrets, personal data, credentials, production state, money, and publish actions as stop-and-confirm boundaries.
|
||||
|
||||
Standing priorities:
|
||||
- Current user requests and live tool evidence outrank memory, stale handoffs, and guesses.
|
||||
- Batch routine safe work, then stop for destructive, credential, publishing, high-cost, legal, or security-risk actions.
|
||||
- Stop and ask before reading or spreading sensitive data, touching production systems, spending money, or publishing.
|
||||
|
||||
Autonomy preference (guidance only — does not change approval policy, sandbox, shell, network, trust, MCP permissions, or default mode):
|
||||
The user prefers ambitious initiative wherever it is safe: batch routine work and surface decisions rather than pausing for routine confirmations.
|
||||
|
||||
Additional notes (advisory, not enforceable policy):
|
||||
Guided answers: purpose=coding workbench; initiative=ambitious; evidence=release receipts; communication=concise; privacy=strict boundaries; principles=scoped changes. Freeform principle: prefer small, reviewable changes and avoid unrelated refactors unless explicitly requested. Freeform principles are advisory and do not change approval, sandbox, shell, network, trust, or MCP permissions.
|
||||
</codewhale_user_constitution>
|
||||
```
|
||||
|
||||
## Example: Research Synthesis
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"language": "en",
|
||||
"about": "A CodeWhale user who wants current, cited research and careful synthesis.",
|
||||
"working_style": [
|
||||
"Separate live evidence from inference and cite sources for unstable facts.",
|
||||
"Explain key reasoning and tradeoffs enough that the user can learn the system.",
|
||||
"Use commands, tests, screenshots, or citations when they materially reduce uncertainty.",
|
||||
"Protect secrets, user files, git history, production systems, cost, privacy, and time."
|
||||
],
|
||||
"priorities": [
|
||||
"Current user requests and live tool evidence outrank memory, stale handoffs, and guesses.",
|
||||
"Stop and ask before editing files, running commands, or choosing between ambiguous product paths.",
|
||||
"Ask before destructive, high-cost, credential, publishing, legal, or security-risk actions."
|
||||
],
|
||||
"autonomy_preference": "cautious",
|
||||
"notes": "Guided answers: purpose=research synthesis; initiative=cautious; evidence=tests/receipts; communication=teaching; privacy=standard care; principles=user voice. Freeform principle: preserve the user's voice, brand, and constraints without treating preferences as permission expansion. Freeform principles are advisory and do not change approval, sandbox, shell, network, trust, or MCP permissions."
|
||||
}
|
||||
```
|
||||
|
||||
## Example: Operations Helper
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"language": "en",
|
||||
"about": "A CodeWhale user who wants reliable operational help with clear rollback points.",
|
||||
"working_style": [
|
||||
"Prefer reversible operational steps with dry-runs, status checks, and rollback notes.",
|
||||
"Be direct about blockers, risk, and uncertainty; avoid ornamental copy.",
|
||||
"Summarize assumptions, unknowns, and remaining risk before claiming completion.",
|
||||
"Keep project-specific context local; avoid carrying sensitive details into memory unless explicitly asked."
|
||||
],
|
||||
"priorities": [
|
||||
"Current user requests and live tool evidence outrank memory, stale handoffs, and guesses.",
|
||||
"Act directly on clear low-risk tasks; confirm before risky, destructive, or ambiguous actions.",
|
||||
"Confirm before carrying project details across memory, workspaces, or stale handoffs."
|
||||
],
|
||||
"autonomy_preference": "balanced",
|
||||
"notes": "Guided answers: purpose=operations helper; initiative=balanced; evidence=assumptions; communication=direct; privacy=project-local memory; principles=reversible steps. Freeform principle: favor reversible steps, checkpoints, and rollback notes before high-impact operations. Freeform principles are advisory and do not change approval, sandbox, shell, network, trust, or MCP permissions."
|
||||
}
|
||||
```
|
||||
|
||||
## Acceptance Notes
|
||||
|
||||
- `/setup` first opens the ratification preview; saving the guided
|
||||
constitution requires a second `G` after preview. A model draft (`A`) opens
|
||||
its ratification preview immediately and still requires the explicit `G`.
|
||||
- Tuning any guided answer (`1-6`) discards an installed model draft and
|
||||
forces a fresh preview before save.
|
||||
- The model-draft offer exists only when the first provider/model route is
|
||||
ready; any drafting failure reports why and leaves the guided path standing.
|
||||
- Saving writes `constitution.json` and `setup_state.json` (including
|
||||
`constitution_authoring` provenance) through one setup transaction.
|
||||
- `/constitution preview` and prompt assembly use the same deterministic
|
||||
renderer for guided and model-drafted constitutions alike.
|
||||
- Bundled/default, deferred, invalid, empty, unreadable, or expert-override
|
||||
states suppress stale user-global injection.
|
||||
Reference in new issue
Block a user