mirror of
https://github.com/codewhale-hq/Codewhale.git
synced 2026-11-08 08:04:25 -10:00
chore(ci): record the intentional base-prompt growth in the runtime-contract budget
Hosted Lint one59a119643failed check-runtime-contract-budget on the base stage identity digest. The base system prompt grew from 7231 to 9063 bytes from5ba1aa63bc(eager sub-agents and goals by default, clearer tool text) and080554911c(ask the user with options); both are intended product changes. Re-measured provider-free on macOS aarch64; the budget carries the reason. Verified: check-runtime-contract-budget.py passes (55 metrics exactly at budget). Hosted Linux measurement not yet compared beyond the identity digest. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
This commit is contained in:
CodeWhale BotandClaude Sonnet 5.5 committed
1 parent
e59a119643
commit
69505829ab
1 file changed
+37
-36
@@ -8,7 +8,7 @@
|
||||
"_codex_20261012_bash_description": "2026-10-12: the optional display-only bash description property adds exactly 112 serialized bytes and 28 estimated tokens to each active/full Plan/Act/Operate tool catalog. Catalog names and counts are unchanged; only new sessions pin this schema. Description values remain append-only ToolUse history. Canonical macOS provider-free measurement: CW/artifacts/0102-handoff-20261012/bash-contract-receipt.json.",
|
||||
"_codex_20261031_engine_move_prompt_measurement": "2026-10-31: four existing provider-free metric tests on macOS aarch64 after the Engine relocation reproduce the pre-b984e31290 prompt bytes and every representative-stage identity exactly. b984e31290 had absorbed a 3-byte raw / 5-byte normalized host-dependent decrease without a prompt source change. Restore only those measured fields; tool catalogs remain byte-for-byte at the archive-tool budget. Receipt: CW/artifacts/engine-move-takeover-20261031/runtime-metrics-move.json. This is a local measurement; the following DSH batch pins linux and /bin/bash through the existing recorded-environment guard, independent of host shell detection; all representative identities and byte counts remain exact. No provider usage or billing is measured.",
|
||||
"_codex_20261031_goal_schema_7177": "PR #7177 changes exactly two update_goal description strings: +134 and +122 serialized bytes, +256 total on every full/active Plan, Act and Operate catalog. Counts and name identities are unchanged. Four existing metric tests on Engine binary SHA-256 648735f5b55dadcc8dd113ca462f07b83d01846c940c161d89ce7434ad889f04 confirm all 55 ceilings exactly. Existing prompt/event golden checks pass after re-recording the derived hashes, byte counts and bounded descriptions. Receipts: CW/artifacts/dsh-upgrade-takeover-20261031/runtime-contract.json and *-verify.log. No provider behavior or usage is claimed.",
|
||||
"_comment": "One-way numeric ceilings and exact structural identities for the provider-free runtime contract. Decreases pass; increases or identity changes fail. Lock in decreases with: python3 scripts/check-runtime-contract-budget.py --update The v0.9.8 child-receipt restore grew every production tool surface by 1496 schema bytes / 374 estimated tokens (agent tool). The v0.9.8 workshop read/tool-result byte fields then grew every production tool surface by 371 schema bytes / 93 estimated tokens. Both raises are explicit maintainer decisions; identities stay on the pre-raise digests only if the name set is unchanged \u2014 re-measure on Linux CI if Lint reports identity drift. The v0.9.8 pinned session prefix added the <context_update> sentence to the base prompt (5848 -> 6084 bytes, every representative stage re-hashed), and the host-side Workflow/Goal verbs plus honest child posture grew the tool catalog (active 16531 -> 16602 bytes, full 71473 -> 72371); both are explicit v0.9.8 maintainer decisions measured from the release train. The v0.9.9 configured-skills change hides only custom configured-root paths, preserves discoverable default-root paths, normalizes Windows prompt separators, and trims 50 redundant skills-prompt bytes. The skill/memory/goal/handoff identities were re-measured without raising any ceiling. Explicit maintainer decision for #5473/#5492. The v0.9.10 full surfaces intentionally add the safe read_media tool; their measured schemas remain below the prior byte/token ceilings. Representative prompt byte metrics now use the same host-independent normalized text as their identities; the normalized base is 6089 bytes. The v0.9.11 model-visible sub-agent surface intentionally retires six legacy agents/* tools in favor of the canonical agent tool; all affected schema and prompt metrics decrease. The v0.9.12 plugin prompt-match slice intentionally adds the request_plugin_install tool to the full tool surfaces (plan full: +518 schema bytes / +130 estimated tokens / 29 -> 30 tools) so a strong prompt match can surface the human review CTA; explicit maintainer decision for #5663/#5579. The v0.9.13 profile pins a non-executed bare bash shell so interpreter guidance is reproducible across hosts. The duplicate tts catalog entry is intentionally hidden; speech remains canonical and the alias remains available for saved-transcript dispatch. Explicit v0.9.13 maintainer decision (2026-09-08): after removing 1426 repeated guidance bytes and pinning the bash-v2 fixture, accept only the measured tool byte/token ceilings from all-features macOS source e27735bb63c897f88061c71567701fd971f5d396, verified libtest SHA-256 5e8cbe213f32c4ecdec63494c4de5e31857b4a40134edf7b21a55bca926b1b38: active 13274/3319 in every mode, Plan full 39885/9972, Act/Operate full 67603/16901, with no margin. Against the prior budget, active +390 bytes is agent -41 plus retained bash command syntax +431. Plan full also retains Git commit_plan +253, update_goal progress +583, github bounded local-report guidance +127, review complete-input refusal +35, and send_later dispatching status +14. Act/Operate full instead has github +2151 and additionally speech +230, hidden tts -2120, and tasks/automation exact model-route fields +274 each. The older budget predates v0.9.12: that tag had already removed 361 agent bytes and added the two 274-byte route fields; the retained initial increase versus the tag is 751 source-attributed bytes (agent +320, bash +431), not the +390 budget delta. Only the seven active definitions form the initial request; full catalogs include deferred tools. Estimated tokens use the existing bytes/4 heuristic, not provider usage or billing. Prompt, representative-context, skill-discovery and tool-name identities/ceilings are unchanged. Explicit v0.9.13 maintainer decision (2026-09-09): source ccc5dadfa2279545bf084d37cff3617e41ceaae2 intentionally exposes create_goal, get_goal and update_goal before continuation, so all three initial surfaces now contain ten tools. Measure exact source 4648d148eea64782be857eda6952af2c539cbfcc with the hosted macOS all-features libtest SHA-256 4485c88c7a8b66b8bb9a135807321e417a1e266457ecb74cffc7dfb92f850fc4: four exact provider-free metric tests pass. Active schemas are exactly 17847 bytes / 4462 estimated tokens (+4573 / +1143 for the three eager goal definitions); Plan full is 40597 / 10150 and Act/Operate full is 68315 / 17079, with no margin. The +712 full-catalog bytes are request_user_input guidance +358, update_goal state-change guidance +98, list_dir home-relative path guidance +56, explicit review max_passes schema +197, and three defer_loading true-to-false values +3. The three active name sets/digests, their counts, and measured active/full byte/token ceilings change; full name identities and all prompt, representative-context and skill-discovery measurements remain unchanged. This updates the earlier seven-tool initial-request receipt; bytes/4 remains an estimate, not measLine truncated
|
||||
"_comment": "One-way numeric ceilings and exact structural identities for the provider-free runtime contract. Decreases pass; increases or identity changes fail. Lock in decreases with: python3 scripts/check-runtime-contract-budget.py --update The v0.9.8 child-receipt restore grew every production tool surface by 1496 schema bytes / 374 estimated tokens (agent tool). The v0.9.8 workshop read/tool-result byte fields then grew every production tool surface by 371 schema bytes / 93 estimated tokens. Both raises are explicit maintainer decisions; identities stay on the pre-raise digests only if the name set is unchanged — re-measure on Linux CI if Lint reports identity drift. The v0.9.8 pinned session prefix added the <context_update> sentence to the base prompt (5848 -> 6084 bytes, every representative stage re-hashed), and the host-side Workflow/Goal verbs plus honest child posture grew the tool catalog (active 16531 -> 16602 bytes, full 71473 -> 72371); both are explicit v0.9.8 maintainer decisions measured from the release train. The v0.9.9 configured-skills change hides only custom configured-root paths, preserves discoverable default-root paths, normalizes Windows prompt separators, and trims 50 redundant skills-prompt bytes. The skill/memory/goal/handoff identities were re-measured without raising any ceiling. Explicit maintainer decision for #5473/#5492. The v0.9.10 full surfaces intentionally add the safe read_media tool; their measured schemas remain below the prior byte/token ceilings. Representative prompt byte metrics now use the same host-independent normalized text as their identities; the normalized base is 6089 bytes. The v0.9.11 model-visible sub-agent surface intentionally retires six legacy agents/* tools in favor of the canonical agent tool; all affected schema and prompt metrics decrease. The v0.9.12 plugin prompt-match slice intentionally adds the request_plugin_install tool to the full tool surfaces (plan full: +518 schema bytes / +130 estimated tokens / 29 -> 30 tools) so a strong prompt match can surface the human review CTA; explicit maintainer decision for #5663/#5579. The v0.9.13 profile pins a non-executed bare bash shell so interpreter guidance is reproducible across hosts. The duplicate tts catalog entry is intentionally hidden; speech remains canonical and the alias remains available for saved-transcript dispatch. Explicit v0.9.13 maintainer decision (2026-09-08): after removing 1426 repeated guidance bytes and pinning the bash-v2 fixture, accept only the measured tool byte/token ceilings from all-features macOS source e27735bb63c897f88061c71567701fd971f5d396, verified libtest SHA-256 5e8cbe213f32c4ecdec63494c4de5e31857b4a40134edf7b21a55bca926b1b38: active 13274/3319 in every mode, Plan full 39885/9972, Act/Operate full 67603/16901, with no margin. Against the prior budget, active +390 bytes is agent -41 plus retained bash command syntax +431. Plan full also retains Git commit_plan +253, update_goal progress +583, github bounded local-report guidance +127, review complete-input refusal +35, and send_later dispatching status +14. Act/Operate full instead has github +2151 and additionally speech +230, hidden tts -2120, and tasks/automation exact model-route fields +274 each. The older budget predates v0.9.12: that tag had already removed 361 agent bytes and added the two 274-byte route fields; the retained initial increase versus the tag is 751 source-attributed bytes (agent +320, bash +431), not the +390 budget delta. Only the seven active definitions form the initial request; full catalogs include deferred tools. Estimated tokens use the existing bytes/4 heuristic, not provider usage or billing. Prompt, representative-context, skill-discovery and tool-name identities/ceilings are unchanged. Explicit v0.9.13 maintainer decision (2026-09-09): source ccc5dadfa2279545bf084d37cff3617e41ceaae2 intentionally exposes create_goal, get_goal and update_goal before continuation, so all three initial surfaces now contain ten tools. Measure exact source 4648d148eea64782be857eda6952af2c539cbfcc with the hosted macOS all-features libtest SHA-256 4485c88c7a8b66b8bb9a135807321e417a1e266457ecb74cffc7dfb92f850fc4: four exact provider-free metric tests pass. Active schemas are exactly 17847 bytes / 4462 estimated tokens (+4573 / +1143 for the three eager goal definitions); Plan full is 40597 / 10150 and Act/Operate full is 68315 / 17079, with no margin. The +712 full-catalog bytes are request_user_input guidance +358, update_goal state-change guidance +98, list_dir home-relative path guidance +56, explicit review max_passes schema +197, and three defer_loading true-to-false values +3. The three active name sets/digests, their counts, and measured active/full byte/token ceilings change; full name identities and all prompt, representative-context and skill-discovery measurements remain unchanged. This updates the earlier seven-tool initial-request receipt; bytes/4 remains an estimate, not measureLine truncated
|
||||
"_compaction_archive_recall_20261031": "2026-10-31: the compaction-archive recall slice adds context_archive_get and context_archive_search to every production surface that already carries the other read tools - Plan active 13 -> 15, Plan full 30 -> 32, Act/Operate active 15 -> 17, Act/Operate full 51 -> 53, measured provider-free on Windows x86_64 as +875 active bytes and +1923 full bytes per Act/Operate surface. No tool left a surface. --update also absorbed pre-existing drift from the prompt and tool commits merged after the 2026-10-26 budget lock (4cf2d2cff ran with [skip ci]): the representative base stage and the system-prompt byte counts move by 3-5 bytes with no matching source change in this slice. 55 metrics, 21 identities. bytes/4 remains an estimate, not provider usage or billing.",
|
||||
"_pr7046_jsonpath_description": "2026-10-22: grew by the merged contributor PR #7046 (b961994290), which documented the JSONPath subset the web tool actually accepts. That is one model-visible description string on web_tool's `fields` argument: active catalog +132 bytes, full catalog +295 bytes, the same in act, operate and plan. No tool was added and no identity changed; the growth is the text the model needs to stop sending recursive descent and filters that the parser rejects. Remeasured on afeb9cea31's tree, 55 metrics, 21 identities.",
|
||||
"document_kind": "codewhale.runtime_contract_budget",
|
||||
@@ -16,43 +16,43 @@
|
||||
"fixture_id": "representative-v1",
|
||||
"stages": {
|
||||
"base": {
|
||||
"bytes": 7231,
|
||||
"identity_sha256": "5130806e324482a4b5ec28ae6fc408846b894844db9b7c38f3cfcf48e2ac63d8"
|
||||
"bytes": 9063,
|
||||
"identity_sha256": "ac6ef0e250b17343384a74f0b030b7b8ad7a5ddc08c43d3af91ea1c308324377"
|
||||
},
|
||||
"goal": {
|
||||
"bytes": 9487,
|
||||
"bytes": 11319,
|
||||
"delta_bytes": 81,
|
||||
"identity_sha256": "9815c41c0acea40708388324c8dd306c34a61377ab1b66357e4556e082cac694"
|
||||
"identity_sha256": "e5ac013f1e6dd459ca452437af3655084cf233d8da5833713503d2488db056b4"
|
||||
},
|
||||
"handoff": {
|
||||
"bytes": 9875,
|
||||
"bytes": 11707,
|
||||
"delta_bytes": 388,
|
||||
"identity_sha256": "ff712ad5f95d2ead15445d5b493e5c6917ed337d0ed2f3a2e75fc04bc098f6bb"
|
||||
"identity_sha256": "970d1fb2187f145545abc1b53568b39f8b61c843170ba12c5f34797a1ec0058c"
|
||||
},
|
||||
"instructions": {
|
||||
"bytes": 7677,
|
||||
"bytes": 9509,
|
||||
"delta_bytes": 203,
|
||||
"identity_sha256": "4de57cb6dd78ddf515bb0c391b404033ccc5bc15b41da8e0f2daa71f37e632ae"
|
||||
"identity_sha256": "3bfb9b379d36aa2e39e9ecbad770b44d42837cc8347b0945220b3c0c6b98cc4f"
|
||||
},
|
||||
"memory": {
|
||||
"bytes": 9406,
|
||||
"bytes": 11238,
|
||||
"delta_bytes": 963,
|
||||
"identity_sha256": "2c751505d43f54ea6962313293e8a34c74e8d75c4dab163134abee4b678c4502"
|
||||
"identity_sha256": "ffde491530738cdf4c759240d2fde98ec9156f14b2963623d78800a1ccbe0ed5"
|
||||
},
|
||||
"project": {
|
||||
"bytes": 7474,
|
||||
"bytes": 9306,
|
||||
"delta_bytes": 243,
|
||||
"identity_sha256": "918a55fd0c6da2a1334cdad93b2da61bcee660723ebc0443cd0a7b045979a0f8"
|
||||
"identity_sha256": "d4e4cd2f2538de5f7ad662ac36606b5a72c19e0c633a4e6a55be58f7a2abf167"
|
||||
},
|
||||
"skill": {
|
||||
"bytes": 8443,
|
||||
"bytes": 10275,
|
||||
"delta_bytes": 766,
|
||||
"identity_sha256": "1645ac6802e2221c8545eeaaa2c1a297d080b92aa7cc9c30e2db78c2575de6c4"
|
||||
"identity_sha256": "35ac1154f4ba43638cea7ecc66a840b7bec1bf9f1adf87469fbc8d388a6239d9"
|
||||
}
|
||||
},
|
||||
"system_prompt_blocks": 5,
|
||||
"total_bytes": 9875,
|
||||
"total_tokens_est": 2469
|
||||
"total_bytes": 11707,
|
||||
"total_tokens_est": 2927
|
||||
},
|
||||
"schema_version": 1,
|
||||
"skill_discovery": {
|
||||
@@ -73,22 +73,22 @@
|
||||
"mode_instructions_bytes": 0,
|
||||
"mode_instructions_tokens_est": 0,
|
||||
"system_prompt_blocks": 4,
|
||||
"system_prompt_bytes": 7226,
|
||||
"system_prompt_tokens_est": 1807
|
||||
"system_prompt_bytes": 9058,
|
||||
"system_prompt_tokens_est": 2265
|
||||
},
|
||||
"operate": {
|
||||
"mode_instructions_bytes": 0,
|
||||
"mode_instructions_tokens_est": 0,
|
||||
"system_prompt_blocks": 4,
|
||||
"system_prompt_bytes": 7226,
|
||||
"system_prompt_tokens_est": 1807
|
||||
"system_prompt_bytes": 9058,
|
||||
"system_prompt_tokens_est": 2265
|
||||
},
|
||||
"plan": {
|
||||
"mode_instructions_bytes": 0,
|
||||
"mode_instructions_tokens_est": 0,
|
||||
"system_prompt_blocks": 4,
|
||||
"system_prompt_bytes": 7226,
|
||||
"system_prompt_tokens_est": 1807
|
||||
"system_prompt_bytes": 9058,
|
||||
"system_prompt_tokens_est": 2265
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -97,9 +97,9 @@
|
||||
"modes": {
|
||||
"act": {
|
||||
"active": {
|
||||
"bytes": 41398,
|
||||
"bytes": 40797,
|
||||
"identity_sha256": "1578366e28f28bf12261e2b62b51644767c229f815a342881dcc721e84bdfa92",
|
||||
"tokens_est": 10350,
|
||||
"tokens_est": 10200,
|
||||
"tool_names": [
|
||||
"agent",
|
||||
"bash",
|
||||
@@ -123,9 +123,9 @@
|
||||
"tools": 18
|
||||
},
|
||||
"full": {
|
||||
"bytes": 88625,
|
||||
"bytes": 88220,
|
||||
"identity_sha256": "b63d6fb294f6c92e4c7eedfa9d775886459f51798a3fd22ae4f931d5ed6cf41b",
|
||||
"tokens_est": 22157,
|
||||
"tokens_est": 22055,
|
||||
"tool_names": [
|
||||
"Git",
|
||||
"Run",
|
||||
@@ -187,9 +187,9 @@
|
||||
},
|
||||
"operate": {
|
||||
"active": {
|
||||
"bytes": 41398,
|
||||
"bytes": 40797,
|
||||
"identity_sha256": "1578366e28f28bf12261e2b62b51644767c229f815a342881dcc721e84bdfa92",
|
||||
"tokens_est": 10350,
|
||||
"tokens_est": 10200,
|
||||
"tool_names": [
|
||||
"agent",
|
||||
"bash",
|
||||
@@ -213,9 +213,9 @@
|
||||
"tools": 18
|
||||
},
|
||||
"full": {
|
||||
"bytes": 88625,
|
||||
"bytes": 88220,
|
||||
"identity_sha256": "b63d6fb294f6c92e4c7eedfa9d775886459f51798a3fd22ae4f931d5ed6cf41b",
|
||||
"tokens_est": 22157,
|
||||
"tokens_est": 22055,
|
||||
"tool_names": [
|
||||
"Git",
|
||||
"Run",
|
||||
@@ -277,9 +277,9 @@
|
||||
},
|
||||
"plan": {
|
||||
"active": {
|
||||
"bytes": 38629,
|
||||
"bytes": 38028,
|
||||
"identity_sha256": "bef7948b985f61653725507678de10ece050a4fde69a5f17cff425c95f0366e5",
|
||||
"tokens_est": 9658,
|
||||
"tokens_est": 9507,
|
||||
"tool_names": [
|
||||
"agent",
|
||||
"bash",
|
||||
@@ -301,9 +301,9 @@
|
||||
"tools": 16
|
||||
},
|
||||
"full": {
|
||||
"bytes": 58241,
|
||||
"bytes": 57834,
|
||||
"identity_sha256": "f9578ac50bc6b69d793dd70090ecb4d7c9b2a8ad957dba57726c1da84ecc1cf9",
|
||||
"tokens_est": 14561,
|
||||
"tokens_est": 14459,
|
||||
"tool_names": [
|
||||
"Git",
|
||||
"Web",
|
||||
@@ -344,5 +344,6 @@
|
||||
}
|
||||
},
|
||||
"surface_profile": "production-default-builtins-no-mcp-no-host-interpreters-bash-v2"
|
||||
}
|
||||
},
|
||||
"_claude_20261108_eager_agents_ask_options": "2026-11-08: 5ba1aa63bc (eager sub-agents and goals by default, clearer tool text) and 080554911c (ask the user with options when a choice is theirs) intentionally grow the base system prompt from 7231 to 9063 bytes and change every representative-stage identity; the shared tool-description edits are recorded in their commit messages. Measured provider-free on macOS aarch64 with the existing runtime-contract metric tests on source e59a119643. Hosted Linux Lint on e59a119643 reported only the base-stage identity change, so the numeric ceilings are unchanged in kind."
|
||||
}
|
||||
Reference in new issue
Block a user