mirror of
https://github.com/Heretek-AI/GDevelop-Heretek.git
synced 2026-10-12 02:02:42 +00:00
feat(ai): local multi-agent game studio + engine culling and instancing
Answers the two questions the fork asked, then acts on both. Assessment (AI_GAME_STUDIO_ASSESSMENT.md): a literal Anno-1800-scale 3D city builder is not achievable on this engine - no instancing, no 3D culling, Android is a Cordova WebView around the same HTML5 runtime, and a 15 MB per-resource cloud-build ceiling. A 2D isometric city builder of Anno-1602/1701 scope is reachable today. The hosted backend already is a multi-agent company; the BYOK path can reach none of it (no plan tool, no sub-agent tool, no role prompts, no local spawner, no loop guard). Studio (phases 2-5, 9-11), all under newIDE/app/src/AiGeneration/Studio/: * Roles.js - manager/designer/developer/tester: prompts, exact tool subsets, read-only invariant, turn caps. * PlanStore.js - create_or_update_plan is real under BYOK and returns the shape getLatestActivePlan/OrchestratorPlan already read; the dispatcher merges a new plan over the existing one so a re-plan cannot strip a field another writer set. * SpawnSubAgents.js + customCreateSubAgentAiRequest - spawn_agent (the BYOK tool schema goes 29 -> 39 tools) and a real child AiRequest carrying the role's prompt and tool subset. * FinalizeSubAgents.js + UseStudioRuntime.js - a finished child's report is written onto its parent's function_call_output and the plan task flips. * RequestWriteGate.js - one gate owns writes to an AiRequest, with named block reasons and latest-wins per-request serialization. * LoopGuard.js - a local circuit breaker (repeated identical calls, error storms, turn count) with a steer -> constrain ladder. Ported from chaitanyagiri/munder-difflin (MIT) src/main/breaker.ts, incl. its #377 whole-arguments sha256 key; no MD source is vendored. * One in-flight model turn per request id, and the two in-place mutators made whole-replace so a concurrent turn cannot discard them. Engine (phases 6-8): * 3D culling: gdjs.Object3DCulling shared AABB maths, a per-layer frustum cache, and a 3D branch in both _updateObjectsPreRender implementations. Capability-checked, because CustomRuntimeObject3D does not extend RuntimeObject3D. * Instancing: opt-in Model3D "Use instancing" property + Model3DInstancePool, one InstancedMesh per (resource, material, layer) for single non-skinned meshes; every other model silently keeps the clone path. * TileMap culling generalised to TileMap (not just SimpleTileMap) and to tilemaps nested in custom objects, with bounds clamped to the map. Two bugs found by running the loop end-to-end rather than by tests: the instance pool was page-global (handing out slots on a discarded scene's mesh), and buildSubAgentReport read only `output_text` while the local path emits `text`, so every local sub-agent reported "(no report)".
This commit is contained in:
John Smith
@@ -0,0 +1,327 @@
|
||||
# AI Game Studio — Assessment
|
||||
|
||||
Answers three questions the fork asked, then records what this plan builds on the answers.
|
||||
|
||||
Every claim below carries a `path:line` (or a `path`) that was read while writing this
|
||||
document, in this checkout. Where a figure is quoted from another project, the file that
|
||||
contains it is named.
|
||||
|
||||
---
|
||||
|
||||
## 1. Verdict: Anno 1800 on Android — not achievable on this engine
|
||||
|
||||
A literal Anno-1800-scale 3D city builder is **not** achievable with GDevelop as it stands.
|
||||
Four load-bearing reasons, any one of which is disqualifying on its own:
|
||||
|
||||
**1.1 No instancing anywhere.** Every `Model3DRuntimeObject` instance clones its own scene
|
||||
graph: `Extensions/3D/Model3DRuntimeObject3DRenderer.ts:349` —
|
||||
`const root = THREE_ADDONS.SkeletonUtils.clone(this._originalModel.scene);`. Draw calls
|
||||
therefore scale 1:1 with instance count. A city builder's defining workload is thousands of
|
||||
repeated small meshes (houses, trees, fences, props); nothing in `GDJS/Runtime` or
|
||||
`Extensions/` uses `THREE.InstancedMesh` — a repo-wide search matches only vendored
|
||||
`three.js`, the vendored `ThreeAddons.js`, type declarations under `GDJS/node_modules`, and
|
||||
`SharedLibs/ThreeAddons/src/examples/jsm/**` (upstream Three.js examples). There is no
|
||||
batching, no LOD, no merge pass. Phase 7 adds an opt-in instanced path, and it is narrower
|
||||
than "instancing" sounds: one `THREE.InstancedMesh` per (model resource, material type, layer)
|
||||
draws every instance of a model that is a **single non-skinned mesh**, and the object is
|
||||
otherwise on the clone path. A model with several sub-meshes, a skinned or animated model, or
|
||||
the per-instance "Basic" material substitution cannot share one instanced mesh (the vendored
|
||||
Three.js build does not even export `BufferGeometryUtils` to merge child geometries), and
|
||||
those silently keep the per-object clone. So Phase 7 removes the 1:1 draw-call cost for the
|
||||
common repeated-prop case, not for an arbitrary scene.
|
||||
|
||||
**1.2 No 3D culling.** 3D objects are never culled. Four independent pieces of evidence:
|
||||
|
||||
- Three explicit `TODO (3D) culling` markers left in the render loop:
|
||||
`GDJS/Runtime/runtimescene.ts:524`, `GDJS/Runtime/RuntimeInstanceContainer.ts:496`,
|
||||
`GDJS/Runtime/CustomRuntimeObjectInstanceContainer.ts:263`.
|
||||
- `RuntimeObject3D.getRendererObject()` returns `null`
|
||||
(`Extensions/3D/A_RuntimeObject3D.ts:97-99`), so the 2D AABB culling path in
|
||||
`CustomRuntimeObjectInstanceContainer._updateObjectsPreRender` skips every 3D object
|
||||
entirely: `if (rendererObject) { ... } else { object.updatePreRender(this); }`
|
||||
(`GDJS/Runtime/CustomRuntimeObjectInstanceContainer.ts:266-283`).
|
||||
|
||||
So every 3D object in the scene is submitted to the GPU every frame regardless of where the
|
||||
camera is. Combined with 1.1, a large city is not merely slow — it is bounded by total
|
||||
object count, not visible object count.
|
||||
|
||||
**1.3 Android is a WebView around the same HTML5 runtime.** There is no native renderer.
|
||||
A Cordova export copies the identical JS runtime into a WebView
|
||||
(`GDJS/GDJS/IDE/Exporter.cpp:152-181`: `source = gdjsRoot + "/Runtime/Cordova/www/index.html"`
|
||||
for `options.target == "cordova"`). Two upstream acknowledgements of what that costs:
|
||||
|
||||
- `GDJS/Runtime/Cordova/config.xml:28` —
|
||||
`<preference name="loadUrlTimeoutValue" value="60000" />`, commented
|
||||
`<!-- Increase timeout value for low-end android devices -->`.
|
||||
- WebView processes killed for memory are a first-class editor concern:
|
||||
`newIDE/app/src/Utils/NativeAppLifecycle.js:4-5` ("Declare what the editor is displaying,
|
||||
so that the native mobile app knows what was running if the system killed the WebView
|
||||
(because of the memory used)").
|
||||
|
||||
**1.4 15 MB per resource, cloud build.** `PROJECT_RESOURCE_MAX_SIZE_IN_BYTES = 15 * 1000 *
|
||||
1000` (`newIDE/app/src/Utils/GDevelopServices/Project.js:23`), enforced on upload
|
||||
(`newIDE/app/src/ResourcesList/FileToCloudProjectResourceUploader.js:186`, and the user-facing
|
||||
message at `:319`). An Anno-scale asset set — large terrain meshes, many building models,
|
||||
music, voice — does not fit through that pipe.
|
||||
|
||||
**Not in the four reasons, but worth stating:** the shader-program problem is real and
|
||||
measurable but is not disqualifying on its own — `GDJS/Runtime/profiler.ts:306-318` documents
|
||||
that the 3D renderer compiles a separate program for each new combination of material
|
||||
features/lights/fog/shadows/tone-mapping/instancing, and the cost lands on a single frame
|
||||
("unexplained stutter early in a playthrough"). See section 4 — accepted, not fixed.
|
||||
|
||||
---
|
||||
|
||||
## 2. Verdict: what is achievable — a 2D isometric city builder, today, with no engine work
|
||||
|
||||
An Anno-1602/1701-scope **2D isometric** city builder is reachable on this engine as it
|
||||
stands. The building blocks exist and are already used in shipped extensions:
|
||||
|
||||
| Need | Facility | Evidence |
|
||||
|---|---|---|
|
||||
| The world grid | `TileMap::SimpleTileMap` — the only renderer that culls at all | `Extensions/TileMap/tilemapruntimeobject-pixi-renderer.ts:152-159`; the object type is declared at `Extensions/TileMap/JsExtension.js:996` |
|
||||
| Save/load of the whole world | `SaveState` extension: full state persistence over `getNetworkSyncData` into IndexedDB | `Extensions/SaveState/SaveStateTools.ts:334-403` (`gameNetworkSyncData`, per-object `getNetworkSyncData` at `:384`, per-scene at `:400`); the actual write is `gdjs.indexedDb.saveToIndexedDB` at `:453` |
|
||||
| Unit movement (crowds) | `NavMeshPathfinding` — Recast/Detour navmesh + crowd | `Extensions/NavMeshPathfinding/NavMeshObstacleRuntimeBehavior.ts:174-186` |
|
||||
| Unit movement (grid) | `PathfindingBehavior` — 2D grid A* | `Extensions/PathfindingBehavior/pathfindingruntimebehavior.ts` |
|
||||
| Building ↔ resource links | `LinkedObjects` extension | `Extensions/LinkedObjects/` (`Extension.cpp`, `JsExtension.cpp`, `linkedobjects.ts`) |
|
||||
| Economy, timers | Builtin variables + timers on the scene/project | `Core/GDCore/Project/Layout.h`, `Project.h` |
|
||||
|
||||
The AI side already knows how to drive all of this: 29 tool definitions in the BYOK tool
|
||||
schema (`newIDE/app/src/AI/CustomAIClient.js:219` onwards, names enumerated in section 3.2),
|
||||
including `create_scene`, `put_2d_instances`, `create_or_replace_object`,
|
||||
`add_scene_events`, `add_or_edit_variable`.
|
||||
|
||||
### 2.1 Three quantified caveats
|
||||
|
||||
These are not blockers; they are limits that shape the design.
|
||||
|
||||
**Grid A* has a hard iteration cap and fails silently.** `_maxComplexityFactor: integer = 50`
|
||||
(`Extensions/PathfindingBehavior/pathfindingruntimebehavior.ts:636`); the search budget is
|
||||
`startNode.estimateCost * this._maxComplexityFactor` and on exceeding it the behaviour logs
|
||||
`No path was found after covering N cells.` and returns `false`
|
||||
(`:752-760`). Long routes across a city grid simply give up. Design around it: keep cells
|
||||
small, or route with `NavMeshPathfinding` for long journeys.
|
||||
|
||||
**Navmesh crowd capacity is fixed at build time.** `maxAgents: this.characters.size + 100`
|
||||
(`Extensions/NavMeshPathfinding/NavMeshObstacleRuntimeBehavior.ts:179-182`). The crowd is
|
||||
rebuilt when the obstacle set changes, not per-agent; a settlement that grows past the
|
||||
initial headcount plus 100 agents needs the obstacle to be re-registered.
|
||||
|
||||
**Any tilemap alpha change forces a full re-render.** `TileMapRuntimeObjectPixiRenderer.
|
||||
updateOpacity` calls `this._object.updateTileMap(true)` (`Extensions/TileMap/
|
||||
tilemapruntimeobject-pixi-renderer.ts:92`) after walking every layer. Fading a district in
|
||||
costs a complete tile rebuild, so do not animate tilemap opacity per frame.
|
||||
|
||||
---
|
||||
|
||||
## 3. Verdict: does the existing agent system suffice?
|
||||
|
||||
**The architecture suffices — and it is better than a from-scratch ChatDev-style port would
|
||||
be. But the fork cannot reach it.**
|
||||
|
||||
Upstream's hosted backend *is* the multi-agent company already. It implements:
|
||||
|
||||
| Capability | Where the client already expects it |
|
||||
|---|---|
|
||||
| A task DAG with `dependsOn` | `AiRequestPlanTask` / `AiRequestPlan` types, `newIDE/app/src/Utils/GDevelopServices/Generation.js:40-50` |
|
||||
| Sub-agent launch as a tool call, with the child id stamped on the call | `subAgentAiRequestId?: string` on `AiRequestMessageAssistantFunctionCall`, `Generation.js:53-63`; consumed by `getFunctionCallsToProcess` at `newIDE/app/src/AiGeneration/AiRequestUtils.js:103` (`if (functionCall.subAgentAiRequestId) continue;`) |
|
||||
| Sub-agent classification (edit vs explorer) | `getSubAgentKind`, `AiRequestUtils.js:201` |
|
||||
| Polling, activation and retirement of children | `AiRequestContext.activateSubAgent` (`:1070`), `removeSubAgentIfDone` (`:1128`), `activeSubAgents` in the context value (`:1480`) |
|
||||
| Plan rendering | `AiRequestChat/OrchestratorPlan.js`, read back by `getLatestActivePlan` (`AiRequestUtils.js:268`) |
|
||||
| Edit approval for mutating child calls | `newIDE/app/src/AiGeneration/Utils.js:473-520` |
|
||||
| A real headless tester | `run_gameplay_test`, `newIDE/app/src/EditorFunctions/GameplayTestTools.js:108`; CLI `RUN_ALL_TESTS` at `newIDE/app/src/MainFrame/LocalCliCommandRunner.js:200` |
|
||||
|
||||
The **execution** half already works locally too, and it is shared: `processEditorFunctionCalls`
|
||||
(`newIDE/app/src/EditorFunctions/EditorFunctionCallRunner.js:96`) runs any request's calls;
|
||||
`AskAiEditorContainer` builds `aiRequestsToProcess` as the selected request plus every active
|
||||
sub-agent (`newIDE/app/src/AiGeneration/AskAiEditorContainer.js:1086`-ish, the `useMemo`
|
||||
feeding `useProcessFunctionCalls`).
|
||||
|
||||
The fork's BYOK path strips all of it. `GDEVELOP_OPENAI_TOOLS`
|
||||
(`newIDE/app/src/AI/CustomAIClient.js:219` through the closing `];` before
|
||||
`extractThinkingAndContent`) has 29 tools and **no sub-agent tool**; `buildSystemPrompt`
|
||||
(`:1256`) is a flat seven-guideline string with no role concept; `customCreateAiRequest`
|
||||
hardcodes `toolsVersion: 'v14'` (`:1590`) and nothing local ever creates a child `AiRequest`.
|
||||
|
||||
### 3.1 The five concrete gaps
|
||||
|
||||
**Gap 1 — no plan tool.** `create_or_update_plan` exists in the registry
|
||||
(`newIDE/app/src/EditorFunctions/index.js:10552`, registered at `:11031`) but is a stub:
|
||||
`makeGenericFailure('Unable to create or update plan - this is handled server-side.')`
|
||||
(`:10556-10558`). A Local model that calls it gets a failure. → **Phase 3**.
|
||||
|
||||
**Gap 2 — no sub-agent tool.** `spawn_agent` does not exist in `GDEVELOP_OPENAI_TOOLS`. The
|
||||
hosted equivalents (`run_edit_agent` at `newIDE/app/src/EditorFunctions/index.js:10765`,
|
||||
`run_explorer_agent` above it) are stubs returning `handled server-side`. → **Phase 4**.
|
||||
|
||||
**Gap 3 — no role prompts.** There is exactly one system prompt, built by
|
||||
`buildSystemPrompt` (`newIDE/app/src/AI/CustomAIClient.js:1256-1290`), with no parameter for a
|
||||
role. A manager, a designer, a developer and a QA tester all receive identical instructions.
|
||||
→ **Phase 2** (role registry) feeding **Phase 4** (role-scoped prompts and tool subsets).
|
||||
|
||||
**Gap 4 — no local spawner.** No code path in `newIDE/app/src` creates a child `AiRequest`
|
||||
under BYOK. `activeSubAgents` is only ever populated from what the server already put on the
|
||||
parent's output (`AiRequestContext.activateSubAgent`, `:1070`, is called from
|
||||
`useActivatePendingSubAgents`, which reads `getPendingSubAgentFunctionCalls`). Nothing writes
|
||||
`subAgentAiRequestId`. → **Phase 4b/4c**.
|
||||
|
||||
**Gap 5 — no local loop guard.** Nothing counts turns, detects repeated identical tool calls,
|
||||
or reacts to an error storm. The only breakers are server-emitted error codes the client
|
||||
merely renders (`repeated-tool-call-loop` → `'stuck'`, `context-too-large` → `'too-large'`,
|
||||
`newIDE/app/src/AiGeneration/AiRequestChat/AiRequestErrorRow.js:51-56`) and
|
||||
`MAX_AI_REQUEST_RETRIES_IN_A_ROW = 3`, whose own comment states it is "kept in sync with the
|
||||
API, which is what really enforces it" (`AiRequestUtils.js:34-38`). Under BYOK there is no
|
||||
server: a looping local agent runs until the user stops it. → **Phase 9b**.
|
||||
|
||||
Two supporting gaps that are structural rather than missing features, and are what make a
|
||||
naive multi-agent port dangerous here:
|
||||
|
||||
- **Gap 2b — concurrent whole-request writes.** 31 `updateAiRequest(` call sites in
|
||||
`newIDE/app/src`, of which 15+ pass `() => aiRequest` (a whole replacement that discards
|
||||
whatever the previous state was) — e.g. `AiRequestContext.js:406`, `:451`, `:1439`, `:1453`;
|
||||
`AskAiEditorContainer.js:725`, `:949`, `:1042`, `:1433`. The single state mutator
|
||||
(`AiRequestContext.updateAiRequest`, `:368`) supports both merge and replace. Once a parent
|
||||
and a child are both being polled and both whole-replaced in the same tick, this is a
|
||||
lost-update race. → **Phase 9a** (a write gate) and **Phase 11** (a per-request turn lock).
|
||||
- **Gap 2c — no plan-merge semantics.** Phase 3 writes a whole task array; Phase 5 flips one
|
||||
task's status. Two writers over one plan is exactly the kanban data-loss shape
|
||||
munder-difflin's `mergeTaskLedger`/`patchTaskInLedger` exists to prevent (section 6).
|
||||
→ **Phase 10**.
|
||||
|
||||
### 3.2 The 29 BYOK tools, and how the four roles partition them
|
||||
|
||||
Names, in schema order (`newIDE/app/src/AI/CustomAIClient.js:219` onward):
|
||||
|
||||
```
|
||||
create_scene, create_or_replace_object, add_behavior, put_2d_instances, add_scene_events,
|
||||
run_script, read_events_source, describe_instances, inspect_object_properties_effects,
|
||||
inspect_behavior_properties, inspect_scene_properties_layers_effects,
|
||||
change_scene_properties_layers_effects_groups, change_object_properties_effects,
|
||||
change_behavior_property, change_project_properties_resources,
|
||||
inspect_project_properties_resources, add_or_edit_variable, put_3d_instances,
|
||||
inspect_variables, read_game_project_json, search_object_asset_store, search_resource_store,
|
||||
search_docs, read_full_docs, initialize_project, get_game_starter_summary,
|
||||
create_or_update_plan, run_gameplay_test, change_gameplay_tests
|
||||
```
|
||||
|
||||
`spawn_agent` is added to this list by Phase 4. The four roles and their exact tool names are
|
||||
defined in `newIDE/app/src/AiGeneration/Studio/Roles.js` (Phase 2); the read-only invariant
|
||||
(tester mutates nothing; designer's only mutating tool is `add_or_edit_variable`) is asserted
|
||||
by `Roles.spec.js`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Engine blocker table
|
||||
|
||||
| Symptom | Exact location | User-visible consequence | Phase |
|
||||
|---|---|---|---|
|
||||
| No instancing: every Model3D instance clones its own scene graph | `Extensions/3D/Model3DRuntimeObject3DRenderer.ts:349` (`SkeletonUtils.clone`); no `THREE.InstancedMesh` outside vendored Three.js | Draw calls rise 1:1 with repeated props; framerate collapses on any repeated-model set | **7** |
|
||||
| 3D objects never culled in a scene | `GDJS/Runtime/runtimescene.ts:524` (`TODO (3D) culling`) | Off-camera 3D objects still submitted every frame | **6** |
|
||||
| 3D objects never culled inside a custom object | `GDJS/Runtime/CustomRuntimeObjectInstanceContainer.ts:263` (`TODO (3D) culling`) | A custom object holding many 3D children is never culled as a whole or in part | **6** |
|
||||
| `RuntimeObject3D` reports no renderer object, so the 2D AABB path skips it | `Extensions/3D/A_RuntimeObject3D.ts:97-99` (`getRendererObject() { return null; }`); branch at `GDJS/Runtime/CustomRuntimeObjectInstanceContainer.ts:266-283` | Reinforces the two rows above: even the existing 2D culling cannot see 3D objects | **6** |
|
||||
| Base container has no camera to cull against | `GDJS/Runtime/RuntimeInstanceContainer.ts:496` (`TODO (3D) culling`) | Accepted — see below | accepted, not fixed |
|
||||
| TileMap viewport culling only for SimpleTileMap, only as a direct scene child, only with an unrotated 3D camera | `Extensions/TileMap/tilemapruntimeobject-pixi-renderer.ts:152-159` (`dimX + dimY > 100 && isSimpleTileMap(object) && instanceContainer === scene && (!gdjs.scene3d || rotations are 0)`) | A large `TileMap` (not Simple) or a tilemap nested in a custom object re-renders the whole map every frame | **8** |
|
||||
| Any tilemap opacity change forces a full re-render | `Extensions/TileMap/tilemapruntimeobject-pixi-renderer.ts:90-95` (`updateTileMap(true)`) | Fading a district stutters | accepted, not fixed |
|
||||
| Lazy 3D shader compilation costs a dropped frame | `GDJS/Runtime/profiler.ts:306-318` (documented), `:380-384` (the newly-compiled-program counter) | Unexplained stutter early in a playthrough | accepted, not fixed |
|
||||
| Per-frame `sortableChildren` z-sorting | `GDJS/Runtime/pixi-renderers/layer-pixi-renderer.ts:246`, `runtimescene-pixi-renderer.ts:33`, `CustomRuntimeObject2DPixiRenderer.ts:37` | CPU cost proportional to 2D object count per layer | accepted, not fixed |
|
||||
| `_allInstancesList` rebuilt when object membership changes | `GDJS/Runtime/RuntimeInstanceContainer.ts:653`, `:736` (invalidations), `:556-565` (rebuild) | A cost spike per spawn/despawn, not per frame | accepted, not fixed |
|
||||
| Grid A* gives up on long routes, silently | `Extensions/PathfindingBehavior/pathfindingruntimebehavior.ts:636`, `:739-750` | Units stop pathing with only a console warning | accepted — a design constraint, documented in section 2.1 |
|
||||
| Navmesh crowd capacity fixed at build | `Extensions/NavMeshPathfinding/NavMeshObstacleRuntimeBehavior.ts:179-182` | Growing population needs the obstacle re-registered | accepted — documented in section 2.1 |
|
||||
|
||||
"Accepted, not fixed" means: recorded here, deliberately out of this plan's scope. Phases 6-8
|
||||
fix the three that bound object count; the rest are either design constraints or costs that
|
||||
scale with a quantity the user controls.
|
||||
|
||||
---
|
||||
|
||||
## 5. Roadmap
|
||||
|
||||
Eleven phases, in the order they were planned. Phases 1-5 are the studio; 6-8 are engine
|
||||
culling and instancing; 9-11 are the editor-side modules that make a local multi-agent run
|
||||
safe. Phases 6-11 are independent of each other and of 2-5, with two named ordering edges
|
||||
(10 before 5's task-status write; 11 with or before 4's spawner).
|
||||
|
||||
| # | Phase | Outcome |
|
||||
|---|---|---|
|
||||
| 1 | Assessment document | This file — both verdicts plus the blocker table |
|
||||
| 2 | Role registry | `AiGeneration/Studio/Roles.js`: four roles, their prompts, their exact tool subsets, their turn caps |
|
||||
| 3 | Local plan tool | `create_or_update_plan` becomes real under BYOK, returning the exact shape `getLatestActivePlan` and `OrchestratorPlan` already read |
|
||||
| 4 | Spawn tool + spawner | `spawn_agent` in the tool schema; a spawner that creates and activates a real child `AiRequest` with a role-scoped prompt and tool subset |
|
||||
| 5 | Finalization + runtime hook | A finished child's report is written onto the parent's `function_call_output`, the plan task flips to done, the next goes in-progress, and the parent takes its next model turn |
|
||||
| 6 | 3D culling | `RuntimeObject3D.getAABB3D`/`isInFrustum`, a per-frame frustum on the layer renderer, and a 3D branch in both `_updateObjectsPreRender` implementations |
|
||||
| 7 | Instancing | `Model3DInstancePool` + an opt-in `useInstancing` property; repeated static models share one `THREE.InstancedMesh` |
|
||||
| 8 | TileMap culling generalisation | Culling works for `TileMap` (not just `SimpleTileMap`) and for tilemaps nested in custom objects, with bounds clamped to the map |
|
||||
| 9 | Write gate + loop guard | One gate owns writes to an `AiRequest`; a local circuit breaker trips on repeated identical calls, error storms and turn count |
|
||||
| 10 | Plan merge semantics | `mergePlanTasks` / `patchPlanTask`, so a status flip cannot strip the fields the plan writer wrote |
|
||||
| 11 | Local turn serialization | One in-flight model turn per request id, and the two in-place mutators made whole-replace |
|
||||
|
||||
---
|
||||
|
||||
## 6. Verdict: munder-difflin as a "test and active multi-agent harness"
|
||||
|
||||
Two verdicts, kept separate. The runtime is unusable here; three of its pure modules are
|
||||
worth porting, and this plan ports exactly those.
|
||||
|
||||
`chaitanyagiri/munder-difflin` v0.4.6, MIT, 7,819 stars (GitHub API, read while writing this),
|
||||
last pushed 2026-09-17.
|
||||
|
||||
### 6.1 Its runtime model does not fit this editor
|
||||
|
||||
Munder-difflin wraps **terminal agent CLIs**. Its `package.json` dependencies include
|
||||
`node-pty ^1.0.0` and `@xterm/xterm ^5.5.0`; its agents coordinate by writing JSON files into
|
||||
per-agent `inbox/`/`outbox/` directories inside a git repo the harness owns
|
||||
(`docs/message-queue.md` §1 describes the one gate that types into a PTY;
|
||||
`src/main/workerWake.ts` describes the inbox-wake watchdog).
|
||||
|
||||
A GDevelop agent must call **editor functions** against the in-memory `gd` project built from
|
||||
`libGD.wasm`. A terminal CLI cannot load that wasm module, cannot produce or mutate `gd`
|
||||
objects, and has no access to the running editor's state; it would have to round-trip through
|
||||
the project file. That is strictly worse than the tool-call loop the fork already has, so
|
||||
munder-difflin's agent runtime, its Pixi office floor, its git-as-coordination-layer (which
|
||||
would collide with `newIDE/app/src/ProjectsStorage/`), and its MemPalace semantic memory
|
||||
(which would add a `uv` + Python toolchain to a browser app) are all out of scope.
|
||||
|
||||
Two smaller mismatches:
|
||||
|
||||
- **Its `mcpCatalog` has no landing site.** A search for `mcp`/`MCP` across
|
||||
`newIDE/app/src` returns no matches: this fork has zero MCP surface.
|
||||
- **Its `test/load-ts.cjs` transpile-one-file loader is unnecessary here.** `newIDE/app` is
|
||||
stock Jest 27 through `react-app-rewired` (`config-overrides.js` overrides only
|
||||
`transformIgnorePatterns`), and its specs use `jest.mock` freely. The
|
||||
`jest-light-runner` no-mocking restriction applies to `GDevelop.js` only
|
||||
(`GDevelop.js/package.json:37` declares `jest-light-runner`, `:44` selects it via
|
||||
`"runner"`).
|
||||
|
||||
### 6.2 Three of its modules close holes this fork verifiably has
|
||||
|
||||
| MD module | MD evidence (read while writing this) | GDevelop hole (measured in this checkout) |
|
||||
|---|---|---|
|
||||
| `src/main/breaker.ts` — repeated-tool, error-storm, token-velocity and cost trips, with a `steer → constrain → stop` escalation ladder | `DEFAULTS` = `{ enabled: true, hardStop: false, repeatedToolLimit: 8, errorStormLimit: 5, tokenVelocityPerMin: 60_000 }`; `tick()` escalates at most one level per beat toward `ceiling = hardStop ? 'stopped' : 'constrained'`, de-escalates one level when not tripping, and sets `action` only when the level escalates; `toolKey()` is `name + ':' + sha256(inp)`, where the comment records that the old `input.slice(0, 200)` collided on Bash commands sharing a long preamble and constrained an agent over nine *different* measurements (issue #377) | **No client-side turn cap, repeat-tool detection or cost accounting exists anywhere in the AI path.** The only breakers are two server-emitted error codes the client merely renders (`AiRequestErrorRow.js:51-56`) and `MAX_AI_REQUEST_RETRIES_IN_A_ROW = 3`, whose own comment says the API is what really enforces it (`AiRequestUtils.js:34-38`). Under BYOK there is no server, so a looping local agent runs until the user stops it |
|
||||
| `src/renderer/src/components/terminalAutomation.ts` + `docs/message-queue.md` — one gate owns every write, with a pure predicate returning an explicit block reason | `terminalAutomationBlock(state)` returns `'exited' | 'picker' | 'draft' | 'settling' | null`; `docs/message-queue.md` §1: "One place types automatic messages into a live agent's PTY: the drain loop… This is load-bearing. When the inbox nudge wrote straight into the terminal, it was a second writer with its own idea of when the prompt was free — and its text landed on top of a half-written line"; the drain is "debounced 200 ms … plus a 3 s backstop tick"; `control.ts` `MAX_PENDING_STEERS = 20` drops from the front so the newest instruction still lands | **31 `updateAiRequest(` call sites**, 15+ of them whole-request replacements that ignore previous state (listed in section 3.1). `inFlightFunctionCallIdsRef` (`AiGeneration/Utils.js:386`) is a per-`(requestId, callId)` lock on function-call *processing* only: it does not serialize state writes, and a parent and its sub-agent are both polled and both whole-replaced in the same tick |
|
||||
| `src/shared/taskLedger.ts` — `mergeTaskLedger` + `patchTaskInLedger` | The module's own header records why: the ledger is hand-written and holds fields the renderer's display model does not know about, so "any writer holding a partial model of a card silently DELETED every field it didn't know about, on EVERY card on the board". `mergeTaskLedger` returns `incoming` (its order, its membership — deletion still works) folded over matching existing entries, and "a field the caller DOES send wins, including an explicit `null`"; an entry with no string `id` passes through untouched. `patchTaskInLedger` edits the **raw** entry so a normalizing parser cannot re-emit a coerced value | Phase 3's `buildPlanOutput` writes the whole task array and Phase 5 flips a task's status: two writers over one plan, the exact kanban data-loss shape this function exists to prevent |
|
||||
|
||||
Two supporting patterns worth naming but not porting wholesale: `src/main/workerWake.ts`'s
|
||||
`announcedInboxIds` edge trigger (a worker is re-nudged only when a **new** message id
|
||||
appears — "unchanged undrained mail is never re-announced once a minute forever"), and its
|
||||
`WORKER_WAKE_COOLDOWN_MS = 60_000` / `WORKER_WAKE_HITL_REARM_MS = 5 * 60_000` guards.
|
||||
|
||||
### 6.3 What that means for the roadmap
|
||||
|
||||
Phases 9-11 exist because of section 6.2, and are independent of phases 1-8.
|
||||
|
||||
The false-positive lesson carried over is munder-difflin's `PreCompact` exemption, which
|
||||
exists because the harness's own auto-compact tripped its own breaker
|
||||
(`breaker.ts`, the comment on `recordCompactStart`: "the harness's own auto-compact mission
|
||||
tripping its own breaker on idle agents"). The analogous legitimate-but-long operations here
|
||||
are a `run_script` (capped at 600 editor-function calls,
|
||||
`newIDE/app/src/EditorFunctions/ScriptExecution/ScriptRunner.js:243-266`) and a gameplay test
|
||||
run. So the ported trips are **repeat-tool, error-storm and turn-count, with the turn cap
|
||||
generous by default (60)** — never a per-tool-call rate, never a wall-clock trip, and never a
|
||||
cost trip: the BYOK path has no usage feed at all, which Phase 9b states explicitly instead
|
||||
of inventing one.
|
||||
|
||||
**Provenance.** Only algorithms and exact constants are taken from munder-difflin; no MD
|
||||
source is vendored, no MD module is imported, and no MD dependency is added. The three
|
||||
upstream files are `src/main/breaker.ts`, `src/shared/taskLedger.ts` and
|
||||
`src/renderer/src/components/terminalAutomation.ts` (with `src/main/control.ts` for the
|
||||
pending-steer rule). The MIT grant covers the reuse.
|
||||
@@ -94,6 +94,22 @@ namespace gdjs {
|
||||
|
||||
abstract getRenderer(): gdjs.RuntimeObject3DRenderer;
|
||||
|
||||
/**
|
||||
* Writes the object's conservative axis-aligned box into `box` and returns
|
||||
* it. Allocates nothing. See `gdjs.Object3DCulling.computeAABB`.
|
||||
*/
|
||||
getAABB3D(box: THREE.Box3): THREE.Box3 {
|
||||
return gdjs.Object3DCulling.computeAABB(this, box);
|
||||
}
|
||||
|
||||
/**
|
||||
* True when the object's box intersects `frustum`. Allocates nothing, so it
|
||||
* is safe to call for every object every frame.
|
||||
*/
|
||||
isInFrustum(frustum: THREE.Frustum): boolean {
|
||||
return gdjs.Object3DCulling.isInFrustum(this, frustum);
|
||||
}
|
||||
|
||||
getRendererObject() {
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -78,6 +78,22 @@ namespace gdjs {
|
||||
return this.getRenderer().get3DRendererObject()!;
|
||||
}
|
||||
|
||||
/**
|
||||
* Writes the object's conservative axis-aligned box into `box` and returns
|
||||
* it. Allocates nothing. See `gdjs.Object3DCulling.computeAABB`.
|
||||
*/
|
||||
getAABB3D(box: THREE.Box3): THREE.Box3 {
|
||||
return gdjs.Object3DCulling.computeAABB(this, box);
|
||||
}
|
||||
|
||||
/**
|
||||
* True when the object's box intersects `frustum`. Allocates nothing, so it
|
||||
* is safe to call for every object every frame.
|
||||
*/
|
||||
isInFrustum(frustum: THREE.Frustum): boolean {
|
||||
return gdjs.Object3DCulling.isInFrustum(this, frustum);
|
||||
}
|
||||
|
||||
override extraInitializationFromInitialInstance(
|
||||
initialInstanceData: InstanceData
|
||||
) {
|
||||
|
||||
@@ -380,6 +380,7 @@ module.exports = {
|
||||
.setIncludeFile('Extensions/3D/A_RuntimeObject3D.js')
|
||||
.addIncludeFile('Extensions/3D/A_RuntimeObject3DRenderer.js')
|
||||
.addIncludeFile('Extensions/3D/Model3DRuntimeObject.js')
|
||||
.addIncludeFile('Extensions/3D/Model3DInstancePool.js')
|
||||
.addIncludeFile('Extensions/3D/Model3DRuntimeObject3DRenderer.js');
|
||||
|
||||
// Properties expressions/conditions/actions:
|
||||
|
||||
@@ -0,0 +1,161 @@
|
||||
namespace gdjs {
|
||||
/**
|
||||
* A pool of `THREE.InstancedMesh`, one per (model resource, material type, layer)
|
||||
* so repeated 3D models sharing a model resource can be drawn in one call instead
|
||||
* of one call per instance.
|
||||
*
|
||||
* The pool is opt-in per object (the Model3D "Use instancing" property). It is
|
||||
* only useful, and only correct, for models that share the *same* model resource
|
||||
* and material: a per-instance material substitution (the "Basic" material type
|
||||
* replaces every material with a `MeshBasicMaterial`) cannot be expressed in a
|
||||
* shared instanced material, so the caller must keep those on the per-object
|
||||
* clone path.
|
||||
*
|
||||
* Keys are strings built by `getModelInstanceKey`; a slot is an index into the
|
||||
* `InstancedMesh`'s instance matrix. Entries grow by doubling when the slots run
|
||||
* out, copying the matrices already written.
|
||||
*/
|
||||
|
||||
/** The instance capacity an entry starts with, and how much it grows by. */
|
||||
const INITIAL_INSTANCE_CAPACITY = 16;
|
||||
|
||||
type InstanceEntry = {
|
||||
instancedMesh: THREE.InstancedMesh;
|
||||
/** Rebuilds the mesh with a new capacity, in the same parent. */
|
||||
growMesh: (newCapacity: integer) => THREE.InstancedMesh;
|
||||
/** Indices that were released and can be handed out again. */
|
||||
freeSlots: Array<integer>;
|
||||
/** How many slots have ever been handed out (the used prefix of the mesh). */
|
||||
usedSlotCount: integer;
|
||||
/** Whether the instance matrices changed since the last `flush`. */
|
||||
isMatrixDirty: boolean;
|
||||
};
|
||||
|
||||
export class Model3DInstancePool {
|
||||
private _entries: Map<string, InstanceEntry> = new Map();
|
||||
/** The scratch matrix, reused by `setMatrix`; never returned. */
|
||||
private _temporaryMatrix: THREE.Matrix4 = new THREE.Matrix4();
|
||||
|
||||
/**
|
||||
* Take a free slot for `key`, creating the `InstancedMesh` when the key is
|
||||
* first seen and growing it when the slots run out. Returns the slot index.
|
||||
*/
|
||||
acquire(
|
||||
key: string,
|
||||
createMesh: (capacity: integer) => THREE.InstancedMesh
|
||||
): integer {
|
||||
const entry = this._entries.get(key);
|
||||
if (!entry) {
|
||||
const capacity = INITIAL_INSTANCE_CAPACITY;
|
||||
const instancedMesh = createMesh(capacity);
|
||||
const newEntry: InstanceEntry = {
|
||||
instancedMesh,
|
||||
growMesh: (newCapacity: integer) => {
|
||||
const grown = createMesh(newCapacity);
|
||||
// Move the matrices already written onto the bigger mesh.
|
||||
for (let slot = 0; slot < newEntry.usedSlotCount; slot++) {
|
||||
instancedMesh.getMatrixAt(slot, this._temporaryMatrix);
|
||||
grown.setMatrixAt(slot, this._temporaryMatrix);
|
||||
}
|
||||
grown.instanceMatrix.needsUpdate = true;
|
||||
if (instancedMesh.parent) {
|
||||
instancedMesh.parent.remove(instancedMesh);
|
||||
instancedMesh.parent.add(grown);
|
||||
}
|
||||
instancedMesh.dispose();
|
||||
newEntry.instancedMesh = grown;
|
||||
return grown;
|
||||
},
|
||||
freeSlots: [],
|
||||
usedSlotCount: 0,
|
||||
isMatrixDirty: true,
|
||||
};
|
||||
this._entries.set(key, newEntry);
|
||||
return this._takeSlot(newEntry);
|
||||
}
|
||||
return this._takeSlot(entry);
|
||||
}
|
||||
|
||||
private _takeSlot(entry: InstanceEntry): integer {
|
||||
const freeSlot = entry.freeSlots.pop();
|
||||
if (freeSlot !== undefined) {
|
||||
return freeSlot;
|
||||
}
|
||||
if (entry.usedSlotCount >= entry.instancedMesh.count) {
|
||||
entry.growMesh(entry.instancedMesh.count * 2);
|
||||
}
|
||||
const slot = entry.usedSlotCount;
|
||||
entry.usedSlotCount++;
|
||||
return slot;
|
||||
}
|
||||
|
||||
/** Give a slot back so a later `acquire` for the same key can reuse it. */
|
||||
release(key: string, slot: integer): void {
|
||||
const entry = this._entries.get(key);
|
||||
if (!entry) return;
|
||||
if (slot < 0 || slot >= entry.usedSlotCount) return;
|
||||
if (entry.freeSlots.indexOf(slot) !== -1) return;
|
||||
entry.freeSlots.push(slot);
|
||||
}
|
||||
|
||||
/**
|
||||
* Write the instance matrix of `slot`. Copies into a scratch matrix and marks
|
||||
* the entry dirty; the caller flushes once per frame.
|
||||
*/
|
||||
setMatrix(key: string, slot: integer, matrix: THREE.Matrix4): void {
|
||||
const entry = this._entries.get(key);
|
||||
if (!entry) return;
|
||||
if (slot < 0 || slot >= entry.instancedMesh.count) return;
|
||||
this._temporaryMatrix.copy(matrix);
|
||||
entry.instancedMesh.setMatrixAt(slot, this._temporaryMatrix);
|
||||
entry.isMatrixDirty = true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Upload the instance matrices of every entry whose matrices changed. Call
|
||||
* once per frame, after all the objects of the frame wrote their matrices.
|
||||
*/
|
||||
flush(): void {
|
||||
this._entries.forEach((entry) => {
|
||||
if (!entry.isMatrixDirty) return;
|
||||
entry.instancedMesh.instanceMatrix.needsUpdate = true;
|
||||
entry.isMatrixDirty = false;
|
||||
});
|
||||
}
|
||||
|
||||
/** The `InstancedMesh` of a key, for a caller that needs to check or scene it. */
|
||||
getInstancedMesh(key: string): THREE.InstancedMesh | null {
|
||||
const entry = this._entries.get(key);
|
||||
return entry ? entry.instancedMesh : null;
|
||||
}
|
||||
|
||||
/** The keys currently held, in insertion order. */
|
||||
getKeys(): Array<string> {
|
||||
const keys: Array<string> = [];
|
||||
this._entries.forEach((entry, key) => keys.push(key));
|
||||
return keys;
|
||||
}
|
||||
|
||||
/** Release every mesh this pool created and forget every key. */
|
||||
dispose(): void {
|
||||
this._entries.forEach((entry) => {
|
||||
if (entry.instancedMesh.parent) {
|
||||
entry.instancedMesh.parent.remove(entry.instancedMesh);
|
||||
}
|
||||
entry.instancedMesh.dispose();
|
||||
});
|
||||
this._entries.clear();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The pool key of one Model3D instance: the model resource, the material type and
|
||||
* the layer it is rendered on. Two objects with the same key can share one
|
||||
* `InstancedMesh` because their meshes and material are interchangeable.
|
||||
*/
|
||||
export const getModelInstanceKey = (
|
||||
modelResourceName: string,
|
||||
materialType: string,
|
||||
layerName: string
|
||||
): string => `${modelResourceName}|${materialType}|${layerName}`;
|
||||
}
|
||||
@@ -23,7 +23,8 @@ Model3DObjectConfiguration::Model3DObjectConfiguration()
|
||||
: width(100), height(100), depth(100), rotationX(90), rotationY(0),
|
||||
rotationZ(90), modelResourceName(""), materialType("StandardWithoutMetalness"),
|
||||
originLocation("ModelOrigin"), centerLocation("CenteredOnZ"),
|
||||
keepAspectRatio(true), crossfadeDuration(0.1f), isCastingShadow(true), isReceivingShadow(true) {}
|
||||
keepAspectRatio(true), crossfadeDuration(0.1f), isCastingShadow(true),
|
||||
isReceivingShadow(true), useInstancing(false) {}
|
||||
|
||||
bool Model3DObjectConfiguration::UpdateProperty(const gd::String &propertyName,
|
||||
const gd::String &newValue) {
|
||||
@@ -67,6 +68,10 @@ bool Model3DObjectConfiguration::UpdateProperty(const gd::String &propertyName,
|
||||
return false;
|
||||
return true;
|
||||
}
|
||||
if (propertyName == "useInstancing") {
|
||||
useInstancing = newValue == "1" || newValue == "true";
|
||||
return true;
|
||||
}
|
||||
if (propertyName == "originLocation") {
|
||||
auto normalizedValue = newValue.LowerCase();
|
||||
if (normalizedValue == "modelorigin")
|
||||
@@ -191,6 +196,14 @@ Model3DObjectConfiguration::GetProperties() const {
|
||||
.SetLabel(_("Material"))
|
||||
.SetGroup(_("Lighting"));
|
||||
|
||||
objectProperties["useInstancing"]
|
||||
.SetValue(useInstancing ? "true" : "false")
|
||||
.SetType("boolean")
|
||||
.SetLabel(_("Use instancing"))
|
||||
.SetDescription(_("Draw objects sharing the same 3D model and material with a single instanced draw call. Only takes effect for static models made of a single mesh; animated or multi-part models keep the normal renderer."))
|
||||
.SetGroup(_("Performance"))
|
||||
.SetAdvanced(true);
|
||||
|
||||
objectProperties["originLocation"]
|
||||
.SetValue(originLocation.empty() ? "TopLeft" : originLocation)
|
||||
.SetType("choice")
|
||||
|
||||
@@ -186,6 +186,9 @@ private:
|
||||
bool keepAspectRatio;
|
||||
bool isCastingShadow;
|
||||
bool isReceivingShadow;
|
||||
bool useInstancing;
|
||||
|
||||
bool shouldUseInstancing() const { return useInstancing; };
|
||||
|
||||
std::vector<Model3DAnimation> animations;
|
||||
static Model3DAnimation badAnimation; //< Bad animation when an out of bound
|
||||
|
||||
@@ -45,6 +45,8 @@ namespace gdjs {
|
||||
crossfadeDuration: float;
|
||||
isCastingShadow: boolean;
|
||||
isReceivingShadow: boolean;
|
||||
/** Opt-in sharing of one `THREE.InstancedMesh` between identical models. */
|
||||
useInstancing?: boolean;
|
||||
};
|
||||
}
|
||||
|
||||
@@ -82,6 +84,14 @@ namespace gdjs {
|
||||
_modelResourceName: string;
|
||||
_materialType: gdjs.Model3DRuntimeObject.MaterialType =
|
||||
gdjs.Model3DRuntimeObject.MaterialType.Basic;
|
||||
/**
|
||||
* Whether this instance should share a `THREE.InstancedMesh` with the other
|
||||
* instances of the same model resource, instead of cloning the model.
|
||||
*
|
||||
* The renderer decides whether the model actually allows it; see
|
||||
* `Model3DRuntimeObject3DRenderer._canUseInstancing`.
|
||||
*/
|
||||
_useInstancing: boolean = false;
|
||||
|
||||
/**
|
||||
* The local point of the model that will be at the object position.
|
||||
@@ -138,6 +148,7 @@ namespace gdjs {
|
||||
objectData.content.materialType
|
||||
);
|
||||
this._crossfadeDuration = objectData.content.crossfadeDuration || 0;
|
||||
this._useInstancing = !!objectData.content.useInstancing;
|
||||
|
||||
this.setIsCastingShadow(objectData.content.isCastingShadow);
|
||||
this.setIsReceivingShadow(objectData.content.isReceivingShadow);
|
||||
@@ -505,6 +516,18 @@ namespace gdjs {
|
||||
const originPoint = this._renderer.getOriginPoint();
|
||||
return this.getZ() - this.getDepth() * originPoint[2];
|
||||
}
|
||||
|
||||
/**
|
||||
* Called when the object leaves the scene: give its instance slot back to
|
||||
* the shared pool so another object can reuse it. A no-op for an object on
|
||||
* the clone path.
|
||||
*/
|
||||
override onDeletedFromScene(): void {
|
||||
if (this._renderer.get3DRendererObject()) {
|
||||
this._renderer.releaseInstancing();
|
||||
}
|
||||
super.onDeletedFromScene();
|
||||
}
|
||||
}
|
||||
|
||||
/** @category Objects > 3D Model */
|
||||
|
||||
@@ -81,6 +81,17 @@ namespace gdjs {
|
||||
*/
|
||||
private _modelOriginPoint: FloatPoint3D;
|
||||
|
||||
/**
|
||||
* The instanced path, set when `useInstancing` is on and the model can be
|
||||
* shared (see `_canUseInstancing`). While it is set, `_threeObject3D` is a
|
||||
* transform carrier that is **never added to the scene graph**: the pool
|
||||
* reads its matrix and writes it to the shared `InstancedMesh`.
|
||||
*/
|
||||
private _instanceSlot: integer | null = null;
|
||||
private _instanceKey: string | null = null;
|
||||
private _layerRenderer: gdjs.LayerPixiRenderer | null = null;
|
||||
private static _instanceMatrix: THREE.Matrix4 | null = null;
|
||||
|
||||
constructor(
|
||||
runtimeObject: gdjs.Model3DRuntimeObject,
|
||||
instanceContainer: gdjs.RuntimeInstanceContainer
|
||||
@@ -117,6 +128,170 @@ namespace gdjs {
|
||||
this._animationMixer.update(timeDelta);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether this object can be drawn through a shared `InstancedMesh`.
|
||||
*
|
||||
* The three conditions the property documents, all required:
|
||||
* - the object opted in (`useInstancing`),
|
||||
* - its material is not substituted per instance ("Basic" replaces every
|
||||
* material with a `MeshBasicMaterial`, which a shared instanced material
|
||||
* cannot express),
|
||||
* - the model is a single mesh, because a shared instanced mesh draws one
|
||||
* geometry (a model with materials/skins - anything animated - therefore
|
||||
* never takes this path).
|
||||
*
|
||||
* When any of them is false the object silently uses the clone path: no
|
||||
* error, no warning.
|
||||
*/
|
||||
private _canUseInstancing(): boolean {
|
||||
if (!this._model3DRuntimeObject._useInstancing) return false;
|
||||
if (
|
||||
this._model3DRuntimeObject._materialType ===
|
||||
gdjs.Model3DRuntimeObject.MaterialType.Basic
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
return !!this._getSingleMesh();
|
||||
}
|
||||
|
||||
/**
|
||||
* The model's single mesh, or null when it is not exactly one mesh (a group,
|
||||
* a skinned mesh, several sub-meshes...).
|
||||
*/
|
||||
private _getSingleMesh(): THREE.Mesh | null {
|
||||
const scene = this._originalModel ? this._originalModel.scene : null;
|
||||
if (!scene) return null;
|
||||
let found: THREE.Mesh | null = null;
|
||||
let meshCount = 0;
|
||||
let skippedSkinnedMesh = false;
|
||||
// An empty group contributes nothing; a mesh (or a mesh-less leaf) is one
|
||||
// drawable, so anything but exactly one drawable means no shared geometry.
|
||||
const visit = (node: THREE.Object3D) => {
|
||||
const maybeMesh = node as THREE.Mesh;
|
||||
if (maybeMesh.isMesh) {
|
||||
if (maybeMesh.isSkinnedMesh) {
|
||||
skippedSkinnedMesh = true;
|
||||
return;
|
||||
}
|
||||
meshCount++;
|
||||
if (meshCount === 1) found = maybeMesh;
|
||||
return;
|
||||
}
|
||||
if (node.children.length === 0) {
|
||||
// A leaf that is not a mesh (a light, a camera, an empty).
|
||||
return;
|
||||
}
|
||||
for (const child of node.children) visit(child);
|
||||
};
|
||||
visit(scene);
|
||||
if (skippedSkinnedMesh || meshCount !== 1) return null;
|
||||
return found;
|
||||
}
|
||||
|
||||
/** The layer renderer this object is rendered by. */
|
||||
private _getLayerRenderer(): gdjs.LayerPixiRenderer | null {
|
||||
const layer = this._object
|
||||
.getInstanceContainer()
|
||||
.getLayer(this._object.getLayer());
|
||||
return layer ? layer.getRenderer() : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Take an instance slot for this object, replacing the per-object clone.
|
||||
*
|
||||
* Returns false when instancing is not applicable, in which case the caller
|
||||
* keeps the clone path.
|
||||
*/
|
||||
private _trySetupInstancing(): boolean {
|
||||
if (!this._canUseInstancing()) return false;
|
||||
const mesh = this._getSingleMesh();
|
||||
if (!mesh) return false;
|
||||
const layerRenderer = this._getLayerRenderer();
|
||||
const threeGroup = layerRenderer ? layerRenderer.getThreeGroup() : null;
|
||||
if (!layerRenderer || !threeGroup) return false;
|
||||
|
||||
const layerName = this._object.getLayer();
|
||||
const key = gdjs.getModelInstanceKey(
|
||||
this._model3DRuntimeObject._modelResourceName,
|
||||
String(this._model3DRuntimeObject._materialType),
|
||||
layerName
|
||||
);
|
||||
const geometry = mesh.geometry;
|
||||
const material = mesh.material;
|
||||
|
||||
// The layer owns the pool: the InstancedMesh is added to this layer's 3D
|
||||
// group, so it must never be shared with another layer or another scene.
|
||||
const pool = layerRenderer.getModelInstancePool();
|
||||
this._layerRenderer = layerRenderer;
|
||||
this._instanceSlot = pool.acquire(key, (capacity) => {
|
||||
const instancedMesh = new THREE.InstancedMesh(
|
||||
geometry,
|
||||
material as THREE.Material,
|
||||
capacity
|
||||
);
|
||||
instancedMesh.rotation.order = 'ZYX';
|
||||
// The instance count and matrices are managed by the pool; culling is
|
||||
// per object (Phase 6 culls the carrier's box, not this merged mesh).
|
||||
instancedMesh.frustumCulled = false;
|
||||
instancedMesh.castShadow = this._model3DRuntimeObject._isCastingShadow;
|
||||
instancedMesh.receiveShadow =
|
||||
this._model3DRuntimeObject._isReceivingShadow;
|
||||
threeGroup.add(instancedMesh);
|
||||
return instancedMesh;
|
||||
});
|
||||
this._instanceKey = key;
|
||||
|
||||
// The instanced mesh draws this object now: drop the per-object clone and
|
||||
// take the carrier out of the scene graph. The carrier is never added to
|
||||
// the scene while instancing is on - it exists only to carry the transform
|
||||
// the pool reads, and an empty group left in the graph would still be
|
||||
// traversed (and counted as a 3D object) every frame.
|
||||
const carrier = this.get3DRendererObject();
|
||||
carrier.clear();
|
||||
layerRenderer.remove3DRendererObject(carrier);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Write the carrier's transform into the shared `InstancedMesh`. */
|
||||
private _writeInstanceMatrix(): void {
|
||||
if (this._instanceKey === null || this._instanceSlot === null) return;
|
||||
let matrix = Model3DRuntimeObject3DRenderer._instanceMatrix;
|
||||
if (!matrix) {
|
||||
matrix = new THREE.Matrix4();
|
||||
Model3DRuntimeObject3DRenderer._instanceMatrix = matrix;
|
||||
}
|
||||
const carrier = this.get3DRendererObject();
|
||||
carrier.updateMatrix();
|
||||
matrix.copy(carrier.matrix);
|
||||
if (this._layerRenderer) {
|
||||
this._layerRenderer
|
||||
.getModelInstancePool()
|
||||
.setMatrix(this._instanceKey, this._instanceSlot, matrix);
|
||||
}
|
||||
}
|
||||
|
||||
/** Give the instance slot back; the slot is reusable by another object. */
|
||||
releaseInstancing(): void {
|
||||
if (this._instanceKey === null || this._instanceSlot === null) return;
|
||||
const layerRenderer = this._layerRenderer;
|
||||
if (layerRenderer) {
|
||||
layerRenderer
|
||||
.getModelInstancePool()
|
||||
.release(this._instanceKey, this._instanceSlot);
|
||||
// Put the carrier back: it was taken out of the scene graph when
|
||||
// instancing took over, and the clone path draws through it again.
|
||||
layerRenderer.add3DRendererObject(this.get3DRendererObject());
|
||||
}
|
||||
this._instanceKey = null;
|
||||
this._instanceSlot = null;
|
||||
this._layerRenderer = null;
|
||||
}
|
||||
|
||||
/** The shared pool slot of this object, or null on the clone path. */
|
||||
getInstanceSlot(): integer | null {
|
||||
return this._instanceSlot;
|
||||
}
|
||||
|
||||
override updatePosition() {
|
||||
const originPoint = this.getOriginPoint();
|
||||
const centerPoint = this.getCenterPoint();
|
||||
@@ -128,6 +303,7 @@ namespace gdjs {
|
||||
this._object.getZ() -
|
||||
this._object.getDepth() * (originPoint[2] - centerPoint[2])
|
||||
);
|
||||
this._writeInstanceMatrix();
|
||||
}
|
||||
|
||||
getOriginPoint(): FloatPoint3D {
|
||||
@@ -378,6 +554,11 @@ namespace gdjs {
|
||||
if (isAnimationPaused) {
|
||||
this.pauseAnimation();
|
||||
}
|
||||
|
||||
// Last, so the clone is in place if instancing cannot be used: the
|
||||
// instanced path replaces it, and silently leaves it alone otherwise.
|
||||
this.releaseInstancing();
|
||||
this._trySetupInstancing();
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -0,0 +1,248 @@
|
||||
// @ts-check
|
||||
// @ts-nocheck The expect.js assertions take an optional message argument, which
|
||||
// the installed `expect` typings do not declare.
|
||||
|
||||
describe('TileMap viewport culling', function () {
|
||||
const COLUMN_COUNT = 80;
|
||||
const ROW_COUNT = 80;
|
||||
const TILE_SIZE = 32;
|
||||
|
||||
/**
|
||||
* The scene's camera is what the culling maths reads, so the layer is sized
|
||||
* to the map and positioned per test.
|
||||
* @param {gdjs.RuntimeGame} runtimeGame
|
||||
* @param {string} [layerName]
|
||||
*/
|
||||
const createScene = (runtimeGame, layerName = '') => {
|
||||
const runtimeScene = new gdjs.RuntimeScene(runtimeGame);
|
||||
if (layerName !== '') {
|
||||
runtimeScene.addLayer({
|
||||
name: layerName,
|
||||
renderingType: '3d',
|
||||
cameraType: 'perspective',
|
||||
visibility: true,
|
||||
cameras: [],
|
||||
effects: [],
|
||||
ambientLightColorR: 255,
|
||||
ambientLightColorG: 255,
|
||||
ambientLightColorB: 255,
|
||||
isLightingLayer: false,
|
||||
followBaseLayerCamera: false,
|
||||
});
|
||||
}
|
||||
runtimeScene.addLayer({
|
||||
name: '',
|
||||
visibility: true,
|
||||
cameras: [],
|
||||
effects: [],
|
||||
ambientLightColorR: 255,
|
||||
ambientLightColorG: 255,
|
||||
ambientLightColorB: 255,
|
||||
isLightingLayer: false,
|
||||
followBaseLayerCamera: false,
|
||||
});
|
||||
return runtimeScene;
|
||||
};
|
||||
|
||||
/**
|
||||
* A large SimpleTileMap: `dimX + dimY > 100` is required for culling to apply.
|
||||
* The tile map itself is not loaded (the texture cache needs a real atlas
|
||||
* image); the culling maths only reads the dimensions and the object's grid
|
||||
* transform, both of which are set here.
|
||||
* @param {gdjs.RuntimeInstanceContainer} instanceContainer
|
||||
* @param {string} name
|
||||
*/
|
||||
const createLargeSimpleTileMap = (instanceContainer, name) => {
|
||||
const object = new gdjs.SimpleTileMapRuntimeObject(instanceContainer, {
|
||||
name,
|
||||
type: 'TileMap::SimpleTileMap',
|
||||
variables: [],
|
||||
behaviors: [],
|
||||
effects: [],
|
||||
content: {
|
||||
atlasImage: '',
|
||||
rowCount: ROW_COUNT,
|
||||
columnCount: COLUMN_COUNT,
|
||||
tileSize: TILE_SIZE,
|
||||
tilesWithHitBox: '',
|
||||
},
|
||||
});
|
||||
instanceContainer.addObject(object);
|
||||
// The texture-cache path needs a real atlas image, which the test game has
|
||||
// none of, so the map is built directly: this is the same object the loader
|
||||
// assigns at the end of `_loadTileMap`. The culling maths only reads the
|
||||
// map's dimensions and tile size.
|
||||
object._tileMap = new TileMapHelper.EditableTileMap(
|
||||
TILE_SIZE,
|
||||
TILE_SIZE,
|
||||
COLUMN_COUNT,
|
||||
ROW_COUNT,
|
||||
new Map()
|
||||
);
|
||||
return object;
|
||||
};
|
||||
|
||||
const computeBounds = (object) =>
|
||||
gdjs.computeTileMapCullingBounds(
|
||||
object,
|
||||
COLUMN_COUNT,
|
||||
ROW_COUNT,
|
||||
[0, 0, 0, 0]
|
||||
);
|
||||
|
||||
/** left, right, top, bottom. */
|
||||
const width = (bounds) => bounds[1] - bounds[0];
|
||||
const height = (bounds) => bounds[3] - bounds[2];
|
||||
|
||||
it('culls a large tile map to the camera tile bounds', async () => {
|
||||
const runtimeGame = await gdjs.getPixiRuntimeGameWithAssets();
|
||||
const runtimeScene = createScene(runtimeGame);
|
||||
const tileMap = createLargeSimpleTileMap(runtimeScene, 'BigMap');
|
||||
|
||||
// moving the camera changes the culled tile bounds
|
||||
expect(COLUMN_COUNT + ROW_COUNT).to.be.above(100);
|
||||
|
||||
// Camera over the middle of the map: the drawn window is smaller than the map.
|
||||
runtimeScene.getLayer('').setCameraX(400);
|
||||
runtimeScene.getLayer('').setCameraY(400);
|
||||
const first = computeBounds(tileMap);
|
||||
|
||||
expect(width(first)).to.be.below(COLUMN_COUNT);
|
||||
expect(height(first)).to.be.below(ROW_COUNT);
|
||||
// Inside the map: the window starts after 0 and is narrower than the map.
|
||||
expect(first[0] + first[2]).to.be.above(0);
|
||||
|
||||
// Moving the camera moves the window.
|
||||
runtimeScene.getLayer('').setCameraX(1600);
|
||||
runtimeScene.getLayer('').setCameraY(1600);
|
||||
const second = computeBounds(tileMap);
|
||||
|
||||
expect(second[0]).to.be.above(first[0]);
|
||||
|
||||
// Never more than the whole map.
|
||||
expect(width(second)).to.be.below(COLUMN_COUNT + 1);
|
||||
expect(height(second)).to.be.below(ROW_COUNT + 1);
|
||||
});
|
||||
|
||||
it('clamps the bounds to the map when the camera is far outside it', async () => {
|
||||
const runtimeGame = await gdjs.getPixiRuntimeGameWithAssets();
|
||||
const runtimeScene = createScene(runtimeGame);
|
||||
const tileMap = createLargeSimpleTileMap(runtimeScene, 'BigMap');
|
||||
|
||||
// Way outside, before the map: bounds stay inside [0, dim].
|
||||
runtimeScene.getLayer('').setCameraX(-100000);
|
||||
runtimeScene.getLayer('').setCameraY(-100000);
|
||||
const outside = computeBounds(tileMap);
|
||||
|
||||
expect(outside[0]).to.be.below(1);
|
||||
expect(outside[2]).to.be.below(1);
|
||||
expect(outside[1]).to.be.below(COLUMN_COUNT + 1);
|
||||
expect(outside[3]).to.be.below(ROW_COUNT + 1);
|
||||
// The clamped window draws nothing rather than the whole map.
|
||||
expect(width(outside)).to.be(0);
|
||||
|
||||
// Way outside, past the map: clamps on the far side.
|
||||
runtimeScene.getLayer('').setCameraX(1000000);
|
||||
runtimeScene.getLayer('').setCameraY(1000000);
|
||||
const far = computeBounds(tileMap);
|
||||
|
||||
expect(far[0]).to.be(COLUMN_COUNT);
|
||||
expect(far[2]).to.be(ROW_COUNT);
|
||||
expect(far[1]).to.be.below(COLUMN_COUNT + 1);
|
||||
expect(far[3]).to.be.below(ROW_COUNT + 1);
|
||||
});
|
||||
|
||||
it('culls a tile map nested in a custom object, resolving the scene camera', async () => {
|
||||
const runtimeGame = await gdjs.getPixiRuntimeGameWithAssets();
|
||||
const runtimeScene = createScene(runtimeGame);
|
||||
|
||||
// The tile map's own container is the custom object, which has no camera of
|
||||
// its own: culling must resolve the camera from the scene's layer.
|
||||
const customObject = new gdjs.CustomRuntimeObject3D(runtimeScene, {
|
||||
name: 'MyCustomObject',
|
||||
type: 'MyExtension::MyEventsBasedObject',
|
||||
variant: '',
|
||||
isInnerAreaFollowingParentSize: false,
|
||||
variables: [],
|
||||
behaviors: [],
|
||||
effects: [],
|
||||
content: { width: 0, height: 0, depth: 0 },
|
||||
});
|
||||
runtimeScene.addObject(customObject);
|
||||
|
||||
const tileMap = createLargeSimpleTileMap(
|
||||
// The tile map lives in the custom object's own container, which has no
|
||||
// camera: culling must resolve the camera from the scene's layer.
|
||||
customObject._instanceContainer,
|
||||
'BigMap'
|
||||
);
|
||||
runtimeScene.getLayer('').setCameraX(400);
|
||||
runtimeScene.getLayer('').setCameraY(400);
|
||||
const bounds = computeBounds(tileMap);
|
||||
|
||||
// Before this change, a tile map inside a custom object was never culled
|
||||
// (the old guard required `instanceContainer === scene`).
|
||||
expect(width(bounds)).to.be.below(COLUMN_COUNT);
|
||||
expect(height(bounds)).to.be.below(ROW_COUNT);
|
||||
});
|
||||
|
||||
it('culls on an unrotated 3D camera and keeps the whole map on a tilted one', async () => {
|
||||
const runtimeGame = await gdjs.getPixiRuntimeGameWithAssets();
|
||||
const runtimeScene = createScene(runtimeGame, 'three-d');
|
||||
const tileMap = createLargeSimpleTileMap(runtimeScene, 'BigMap');
|
||||
tileMap.setLayer('three-d');
|
||||
runtimeScene.getLayer('three-d').setCameraX(800);
|
||||
runtimeScene.getLayer('three-d').setCameraY(800);
|
||||
|
||||
// The guard reads these two accessors, so they are what the test drives.
|
||||
const rotations = sinon.stub(gdjs.scene3d.camera, 'getCameraRotationX');
|
||||
const rotationsY = sinon.stub(gdjs.scene3d.camera, 'getCameraRotationY');
|
||||
rotations.returns(0);
|
||||
rotationsY.returns(0);
|
||||
try {
|
||||
// An unrotated 3D camera culls.
|
||||
const unrotated = computeBounds(tileMap);
|
||||
expect(width(unrotated)).to.be.below(COLUMN_COUNT);
|
||||
|
||||
// A camera tilted around Y cannot be culled with axis-aligned tile
|
||||
// bounds: the whole map is drawn.
|
||||
rotationsY.returns(30);
|
||||
const tiltedY = computeBounds(tileMap);
|
||||
expect(tiltedY[0]).to.be(0);
|
||||
expect(tiltedY[2]).to.be(0);
|
||||
expect(tiltedY[1]).to.be(COLUMN_COUNT);
|
||||
expect(tiltedY[3]).to.be(ROW_COUNT);
|
||||
|
||||
// Same for a tilt around X.
|
||||
rotationsY.returns(0);
|
||||
rotations.returns(30);
|
||||
const tiltedX = computeBounds(tileMap);
|
||||
expect(tiltedX[0]).to.be(0);
|
||||
expect(tiltedX[1]).to.be(COLUMN_COUNT);
|
||||
} finally {
|
||||
rotations.restore();
|
||||
rotationsY.restore();
|
||||
}
|
||||
|
||||
// An in-plane camera rotation (not a tilt) still culls, using a window that
|
||||
// covers the rotated viewport's bounding square.
|
||||
runtimeScene.getLayer('three-d').setCameraRotation(45);
|
||||
const spun = computeBounds(tileMap);
|
||||
expect(width(spun)).to.be.below(COLUMN_COUNT);
|
||||
});
|
||||
|
||||
it('does not cull a small map', async () => {
|
||||
const runtimeGame = await gdjs.getPixiRuntimeGameWithAssets();
|
||||
const runtimeScene = createScene(runtimeGame);
|
||||
const tileMap = createLargeSimpleTileMap(runtimeScene, 'SmallMap');
|
||||
|
||||
// dimX + dimY = 100 is the threshold: at it, the whole map is drawn.
|
||||
const bounds = gdjs.computeTileMapCullingBounds(
|
||||
tileMap,
|
||||
50,
|
||||
50,
|
||||
[0, 0, 0, 0]
|
||||
);
|
||||
expect(bounds).to.eql([0, 50, 0, 50]);
|
||||
});
|
||||
});
|
||||
@@ -1,6 +1,96 @@
|
||||
/// <reference path="helper/TileMapHelper.d.ts" />
|
||||
/// <reference path="pixi-tilemap/dist/pixi-tilemap.d.ts" />
|
||||
namespace gdjs {
|
||||
/**
|
||||
* The tile bounds of a tile map to draw for the current camera position, as
|
||||
* `[left, right, top, bottom]`.
|
||||
*
|
||||
* The whole map (`[0, dimX, 0, dimY]`) is returned whenever culling must not
|
||||
* be applied:
|
||||
* - the map is small (`dimX + dimY <= 100`): the maths is not worth paying for;
|
||||
* - the container has no scene or the scene has no such layer;
|
||||
* - the layer's 3D camera is tilted, because the axis-aligned tile-bounds
|
||||
* maths is only valid for an unrotated camera.
|
||||
*
|
||||
* The camera is resolved from the **scene's** layer, so a tile map nested in
|
||||
* a custom object is culled too - its own container has no camera.
|
||||
*
|
||||
* The computed bounds are clamped to `[0, dimX] x [0, dimY]`: a camera outside
|
||||
* the map must not produce out-of-range bounds that differ every frame and so
|
||||
* force a full re-render at every step.
|
||||
*
|
||||
* Written into `result` (a 4-element array) to avoid allocating; returns it.
|
||||
* @category Renderers > Tile Map
|
||||
*/
|
||||
export const computeTileMapCullingBounds = (
|
||||
object: gdjs.TileMapRuntimeObject | gdjs.SimpleTileMapRuntimeObject,
|
||||
dimX: integer,
|
||||
dimY: integer,
|
||||
result: Array<integer>
|
||||
): Array<integer> => {
|
||||
result[0] = 0;
|
||||
result[1] = dimX;
|
||||
result[2] = 0;
|
||||
result[3] = dimY;
|
||||
|
||||
// Don't cull small maps or chunks.
|
||||
if (dimX + dimY <= 100) return result;
|
||||
|
||||
const instanceContainer = object.getInstanceContainer();
|
||||
const scene = instanceContainer.getScene();
|
||||
const layerName = object.getLayer();
|
||||
const layer = scene ? scene.getLayer(layerName) : null;
|
||||
if (!layer) return result;
|
||||
|
||||
// The axis-aligned tile bounds maths is only valid for an unrotated camera.
|
||||
if (
|
||||
gdjs.scene3d &&
|
||||
(gdjs.scene3d.camera.getCameraRotationX(scene, layerName, 0) !== 0 ||
|
||||
gdjs.scene3d.camera.getCameraRotationY(scene, layerName, 0) !== 0)
|
||||
) {
|
||||
return result;
|
||||
}
|
||||
|
||||
const cameraX = layer.getCameraX();
|
||||
const cameraY = layer.getCameraY();
|
||||
let cameraHalfWidth = layer.getCameraWidth() / 2;
|
||||
let cameraHalfHeight = layer.getCameraHeight() / 2;
|
||||
if (layer.getCameraRotation() !== 0) {
|
||||
const hypot = cameraHalfWidth + cameraHalfHeight;
|
||||
cameraHalfWidth = hypot;
|
||||
cameraHalfHeight = hypot;
|
||||
}
|
||||
const [cameraLeftTile, cameraTopTile] =
|
||||
object.getGridCoordinatesFromSceneCoordinates(
|
||||
cameraX - cameraHalfWidth,
|
||||
cameraY - cameraHalfHeight
|
||||
);
|
||||
const [cameraRightTile, cameraBottomTile] =
|
||||
object.getGridCoordinatesFromSceneCoordinates(
|
||||
cameraX + cameraHalfWidth,
|
||||
cameraY + cameraHalfHeight
|
||||
);
|
||||
|
||||
// Clamp to the map.
|
||||
result[0] = Math.max(
|
||||
0,
|
||||
Math.min(dimX, Math.min(cameraLeftTile, cameraRightTile))
|
||||
);
|
||||
result[1] = Math.max(
|
||||
0,
|
||||
Math.min(dimX, Math.max(cameraLeftTile, cameraRightTile) + 1)
|
||||
);
|
||||
result[2] = Math.max(
|
||||
0,
|
||||
Math.min(dimY, Math.min(cameraTopTile, cameraBottomTile))
|
||||
);
|
||||
result[3] = Math.max(
|
||||
0,
|
||||
Math.min(dimY, Math.max(cameraTopTile, cameraBottomTile) + 1)
|
||||
);
|
||||
return result;
|
||||
};
|
||||
|
||||
/**
|
||||
* The PIXI.js renderer for the Tile map runtime object.
|
||||
*
|
||||
@@ -9,10 +99,11 @@ namespace gdjs {
|
||||
*/
|
||||
export class TileMapRuntimeObjectPixiRenderer {
|
||||
private _object:
|
||||
| gdjs.TileMapRuntimeObject
|
||||
| gdjs.SimpleTileMapRuntimeObject;
|
||||
gdjs.TileMapRuntimeObject | gdjs.SimpleTileMapRuntimeObject;
|
||||
|
||||
private _pixiObject: PIXI.tilemap.CompositeTilemap;
|
||||
/** Scratch array reused by the culling maths; never returned. */
|
||||
private _temporaryCullingBounds: Array<integer> = [0, 0, 0, 0];
|
||||
private _lastCullingLeftBound = 0;
|
||||
private _lastCullingRightBound = 0;
|
||||
private _lastCullingTopBound = 0;
|
||||
@@ -24,8 +115,7 @@ namespace gdjs {
|
||||
*/
|
||||
constructor(
|
||||
runtimeObject:
|
||||
| gdjs.TileMapRuntimeObject
|
||||
| gdjs.SimpleTileMapRuntimeObject,
|
||||
gdjs.TileMapRuntimeObject | gdjs.SimpleTileMapRuntimeObject,
|
||||
instanceContainer: gdjs.RuntimeInstanceContainer
|
||||
) {
|
||||
this._object = runtimeObject;
|
||||
@@ -140,49 +230,16 @@ namespace gdjs {
|
||||
const dimX = tileMap.getDimensionX();
|
||||
const dimY = tileMap.getDimensionY();
|
||||
|
||||
let leftBound = 0;
|
||||
let rightBound = dimX;
|
||||
let topBound = 0;
|
||||
let bottomBound = dimY;
|
||||
|
||||
const instanceContainer = this._object.getInstanceContainer();
|
||||
const scene = instanceContainer.getScene();
|
||||
const layerName = this._object.getLayer();
|
||||
const layer = this._object.getInstanceContainer().getLayer(layerName);
|
||||
if (
|
||||
// Don't cull small maps or chunks.
|
||||
dimX + dimY > 100 &&
|
||||
// TODO Handle culling for TileMapRuntimeObject.
|
||||
isSimpleTileMap(object) &&
|
||||
instanceContainer === scene &&
|
||||
(!gdjs.scene3d ||
|
||||
(gdjs.scene3d.camera.getCameraRotationX(scene, layerName, 0) === 0 &&
|
||||
gdjs.scene3d.camera.getCameraRotationY(scene, layerName, 0) === 0))
|
||||
) {
|
||||
const cameraX = layer.getCameraX();
|
||||
const cameraY = layer.getCameraY();
|
||||
let cameraHalfWidth = layer.getCameraWidth() / 2;
|
||||
let cameraHalfHeight = layer.getCameraHeight() / 2;
|
||||
if (layer.getCameraRotation() !== 0) {
|
||||
const hypot = cameraHalfWidth + cameraHalfHeight;
|
||||
cameraHalfWidth = hypot;
|
||||
cameraHalfHeight = hypot;
|
||||
}
|
||||
const [cameraLeftTile, cameraTopTile] =
|
||||
object.getGridCoordinatesFromSceneCoordinates(
|
||||
cameraX - cameraHalfWidth,
|
||||
cameraY - cameraHalfHeight
|
||||
);
|
||||
const [cameraRightTile, cameraBottomTile] =
|
||||
object.getGridCoordinatesFromSceneCoordinates(
|
||||
cameraX + cameraHalfWidth,
|
||||
cameraY + cameraHalfHeight
|
||||
);
|
||||
leftBound = Math.min(cameraLeftTile, cameraRightTile);
|
||||
rightBound = Math.max(cameraLeftTile, cameraRightTile) + 1;
|
||||
topBound = Math.min(cameraTopTile, cameraBottomTile);
|
||||
bottomBound = Math.max(cameraTopTile, cameraBottomTile) + 1;
|
||||
}
|
||||
const cullingBounds = computeTileMapCullingBounds(
|
||||
object,
|
||||
dimX,
|
||||
dimY,
|
||||
this._temporaryCullingBounds
|
||||
);
|
||||
const leftBound = cullingBounds[0];
|
||||
const rightBound = cullingBounds[1];
|
||||
const topBound = cullingBounds[2];
|
||||
const bottomBound = cullingBounds[3];
|
||||
|
||||
if (
|
||||
forceRefresh ||
|
||||
@@ -217,13 +274,6 @@ namespace gdjs {
|
||||
}
|
||||
}
|
||||
|
||||
function isSimpleTileMap(
|
||||
object: gdjs.SimpleTileMapRuntimeObject | gdjs.TileMapRuntimeObject
|
||||
): object is gdjs.SimpleTileMapRuntimeObject {
|
||||
//@ts-ignore We are checking if the methods are present.
|
||||
return object.getGridCoordinatesFromSceneCoordinates;
|
||||
}
|
||||
|
||||
/**
|
||||
* @category Renderers > Tile Map
|
||||
*/
|
||||
|
||||
@@ -10,7 +10,9 @@ namespace gdjs {
|
||||
* @see gdjs.CustomRuntimeObject
|
||||
* @category Core Engine > Instance Container
|
||||
*/
|
||||
export class CustomRuntimeObjectInstanceContainer extends gdjs.RuntimeInstanceContainer {
|
||||
export class CustomRuntimeObjectInstanceContainer
|
||||
extends gdjs.RuntimeInstanceContainer
|
||||
{
|
||||
_debuggerRenderer: gdjs.DebuggerRenderer;
|
||||
_runtimeScene: gdjs.RuntimeScene;
|
||||
/** The parent container that contains the object associated with this container. */
|
||||
@@ -260,18 +262,46 @@ namespace gdjs {
|
||||
*/
|
||||
_updateObjectsPreRender() {
|
||||
const allInstancesList = this.getAdhocListOfAllInstances();
|
||||
// TODO (3D) culling - add support for 3D object culling?
|
||||
for (let i = 0, len = allInstancesList.length; i < len; ++i) {
|
||||
const object = allInstancesList[i];
|
||||
const rendererObject = object.getRendererObject();
|
||||
if (rendererObject) {
|
||||
rendererObject.visible = !object.isHidden();
|
||||
|
||||
// Update effects, only for visible objects.
|
||||
if (rendererObject.visible) {
|
||||
// 3D objects report no `getRendererObject()` (see
|
||||
// `RuntimeObject3D.getRendererObject`), so they need their own culling.
|
||||
// A custom object has no camera of its own: resolve the scene's, for the
|
||||
// layer the 3D object is on.
|
||||
//
|
||||
// Both 3D object families are caught by the capability, not by a base
|
||||
// class: `CustomRuntimeObject3D` does not extend `RuntimeObject3D`.
|
||||
const isThreeD = gdjs.CullableRuntimeObject3D.isCullable3D(object);
|
||||
if (isThreeD) {
|
||||
const scene = this.getScene();
|
||||
const layer = scene ? scene.getLayer(object.getLayer()) : null;
|
||||
const layerRenderer = layer ? layer.getRenderer() : null;
|
||||
const frustum = layerRenderer
|
||||
? layerRenderer.getThreeFrustum()
|
||||
: null;
|
||||
const isVisible =
|
||||
!object.isHidden() && (!frustum || object.isInFrustum(frustum));
|
||||
const threeObject = object.get3DRendererObject();
|
||||
if (threeObject) threeObject.visible = isVisible;
|
||||
|
||||
if (isVisible) {
|
||||
this.getGame()
|
||||
.getEffectsManager()
|
||||
.updatePreRender(object.getRendererEffects(), object);
|
||||
object.updatePreRender(this);
|
||||
}
|
||||
} else {
|
||||
const rendererObject = object.getRendererObject();
|
||||
if (rendererObject) {
|
||||
rendererObject.visible = !object.isHidden();
|
||||
|
||||
// Update effects, only for visible objects.
|
||||
if (rendererObject.visible) {
|
||||
this.getGame()
|
||||
.getEffectsManager()
|
||||
.updatePreRender(object.getRendererEffects(), object);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -288,8 +318,11 @@ namespace gdjs {
|
||||
);
|
||||
}
|
||||
|
||||
// Perform pre-render update.
|
||||
object.updatePreRender(this);
|
||||
// Perform pre-render update, always for non-3D objects (they handle
|
||||
// visibility on their own); 3D objects got theirs above, only if visible.
|
||||
if (!isThreeD) {
|
||||
object.updatePreRender(this);
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
@@ -324,8 +357,7 @@ namespace gdjs {
|
||||
* Get the renderer associated to the RuntimeScene.
|
||||
*/
|
||||
getRenderer():
|
||||
| gdjs.CustomRuntimeObject2DRenderer
|
||||
| gdjs.CustomRuntimeObject3DRenderer {
|
||||
gdjs.CustomRuntimeObject2DRenderer | gdjs.CustomRuntimeObject3DRenderer {
|
||||
return this._customObject.getRenderer();
|
||||
}
|
||||
|
||||
|
||||
@@ -493,7 +493,14 @@ namespace gdjs {
|
||||
*/
|
||||
_updateObjectsPreRender() {
|
||||
const allInstancesList = this.getAdhocListOfAllInstances();
|
||||
// TODO (3D) culling - add support for 3D object culling?
|
||||
// No 3D culling here: a plain instance container has no camera of its own,
|
||||
// so 3D culling is only defined where a camera exists - on the scene
|
||||
// (`RuntimeScene._updateObjectsPreRender`) and inside a custom object,
|
||||
// which resolves the scene's camera for the object's layer
|
||||
// (`CustomRuntimeObjectInstanceContainer._updateObjectsPreRender`).
|
||||
// `RuntimeScene` overrides this method with the culling 2D and 3D objects
|
||||
// both need; this implementation is the first-frame path for a scene and
|
||||
// the every-frame path for a container without a camera.
|
||||
for (let i = 0, len = allInstancesList.length; i < len; ++i) {
|
||||
const object = allInstancesList[i];
|
||||
const rendererObject = object.getRendererObject();
|
||||
|
||||
@@ -210,14 +210,22 @@ namespace gdjs {
|
||||
private _threeGroup: THREE.Group | null = null;
|
||||
private _threeScene: THREE.Scene | null = null;
|
||||
private _threeCamera:
|
||||
| THREE.PerspectiveCamera
|
||||
| THREE.OrthographicCamera
|
||||
| null = null;
|
||||
THREE.PerspectiveCamera | THREE.OrthographicCamera | null = null;
|
||||
private _threeCameraDirty: boolean = false;
|
||||
private _threeEffectComposer: THREE_ADDONS.EffectComposer | null = null;
|
||||
private _basis: Basis | null = null;
|
||||
private static matrix4: THREE.Matrix4 | null = null;
|
||||
|
||||
/**
|
||||
* The 3D camera frustum, rebuilt at most once per frame and read by 3D
|
||||
* object culling (`RuntimeObject3D.isInFrustum`). Both scratch objects are
|
||||
* allocated here, once.
|
||||
*/
|
||||
private _threeFrustum: THREE.Frustum | null = null;
|
||||
private _threeFrustumMatrix: THREE.Matrix4 | null = null;
|
||||
/** Everything the frustum depends on, so it is rebuilt only on a change. */
|
||||
private _threeFrustumSignature: string | null = null;
|
||||
|
||||
// For a 2D+3D layer, the 2D rendering is done on the render texture
|
||||
// and then must be displayed on a plane in the 3D world:
|
||||
private _threePlaneTexture: THREE.Texture | null = null;
|
||||
@@ -315,17 +323,122 @@ namespace gdjs {
|
||||
return this._threeScene;
|
||||
}
|
||||
|
||||
/**
|
||||
* The pool of `THREE.InstancedMesh` for this layer's 3D group.
|
||||
*
|
||||
* Owned per layer, not globally: the meshes live in this layer's
|
||||
* `_threeGroup`, so a pool shared across layers (or across scenes, whose
|
||||
* layer renderers are rebuilt) would hand out slots on a mesh that belongs
|
||||
* to a discarded scene and nothing would be drawn.
|
||||
*/
|
||||
private _modelInstancePool: gdjs.Model3DInstancePool | null = null;
|
||||
|
||||
/**
|
||||
* The instance pool of this layer, created on first use: a layer with no
|
||||
* instanced Model3D never allocates one.
|
||||
*/
|
||||
getModelInstancePool(): gdjs.Model3DInstancePool {
|
||||
if (!this._modelInstancePool) {
|
||||
this._modelInstancePool = new gdjs.Model3DInstancePool();
|
||||
}
|
||||
return this._modelInstancePool;
|
||||
}
|
||||
|
||||
/** The 3D group this layer renders its 3D objects into. */
|
||||
getThreeGroup(): THREE.Group | null {
|
||||
return this._threeGroup;
|
||||
}
|
||||
|
||||
getThreeCamera():
|
||||
| THREE.PerspectiveCamera
|
||||
| THREE.OrthographicCamera
|
||||
| null {
|
||||
THREE.PerspectiveCamera | THREE.OrthographicCamera | null {
|
||||
return this._threeCamera;
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy the layer camera onto the 3D camera. Needed by the frustum before 3D
|
||||
* object culling runs, which happens before the layers are updated for the
|
||||
* frame (`RuntimeScene.render`), so the frustum cannot rely on
|
||||
* `updatePosition` having run yet.
|
||||
*/
|
||||
private _update3DCameraFromLayer(): void {
|
||||
const camera = this._threeCamera;
|
||||
if (!camera) return;
|
||||
|
||||
camera.position.x = this._layer.getCameraX();
|
||||
camera.position.y = -this._layer.getCameraY(); // scene is mirrored on Y
|
||||
camera.rotation.z = -gdjs.toRad(this._layer.getCameraRotation());
|
||||
|
||||
if (camera instanceof THREE.OrthographicCamera) {
|
||||
camera.zoom = this._layer.getCameraZoom();
|
||||
camera.updateProjectionMatrix();
|
||||
camera.position.z = this._layer.getCameraZ(null);
|
||||
} else {
|
||||
camera.position.z = this._layer.getCameraZ(camera.fov);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The 3D camera frustum of this layer, or null when the layer has no 3D
|
||||
* camera.
|
||||
*
|
||||
* Rebuilt only when the camera actually changed: the layer camera state is
|
||||
* cheap to read, so a change in position, zoom or rotation (X, Y or Z)
|
||||
* invalidates the cached frustum. The returned object is shared - read it,
|
||||
* never keep it across frames.
|
||||
*/
|
||||
getThreeFrustum(): THREE.Frustum | null {
|
||||
const camera = this._threeCamera;
|
||||
if (!camera) return null;
|
||||
|
||||
// A cheap signature of everything the frustum depends on.
|
||||
const cameraSignature =
|
||||
this._layer.getCameraX() +
|
||||
'|' +
|
||||
this._layer.getCameraY() +
|
||||
'|' +
|
||||
this._layer.getCameraRotation() +
|
||||
'|' +
|
||||
this._layer.getCameraZoom() +
|
||||
'|' +
|
||||
this._layer.getWidth() +
|
||||
'|' +
|
||||
this._layer.getHeight() +
|
||||
'|' +
|
||||
camera.rotation.x +
|
||||
'|' +
|
||||
camera.rotation.y +
|
||||
'|' +
|
||||
camera.fov +
|
||||
'|' +
|
||||
camera.near +
|
||||
'|' +
|
||||
camera.far;
|
||||
|
||||
if (
|
||||
this._threeFrustum &&
|
||||
this._threeFrustumSignature === cameraSignature
|
||||
) {
|
||||
return this._threeFrustum;
|
||||
}
|
||||
|
||||
if (!this._threeFrustum) {
|
||||
this._threeFrustum = new THREE.Frustum();
|
||||
}
|
||||
if (!this._threeFrustumMatrix) {
|
||||
this._threeFrustumMatrix = new THREE.Matrix4();
|
||||
}
|
||||
|
||||
this._update3DCameraFromLayer();
|
||||
camera.updateMatrixWorld();
|
||||
this._threeFrustumMatrix.multiplyMatrices(
|
||||
camera.projectionMatrix,
|
||||
camera.matrixWorldInverse
|
||||
);
|
||||
this._threeFrustum.setFromProjectionMatrix(this._threeFrustumMatrix);
|
||||
this._threeFrustumSignature = cameraSignature;
|
||||
return this._threeFrustum;
|
||||
}
|
||||
|
||||
getThreeEffectComposer(): THREE_ADDONS.EffectComposer | null {
|
||||
return this._threeEffectComposer;
|
||||
}
|
||||
@@ -833,22 +946,7 @@ namespace gdjs {
|
||||
const runtimeGame = instanceContainer.getGame();
|
||||
|
||||
// Update the 3D camera position and rotation.
|
||||
if (this._threeCamera) {
|
||||
const angle = -gdjs.toRad(this._layer.getCameraRotation());
|
||||
this._threeCamera.position.x = this._layer.getCameraX();
|
||||
this._threeCamera.position.y = -this._layer.getCameraY(); // scene is mirrored on Y
|
||||
this._threeCamera.rotation.z = angle;
|
||||
|
||||
if (this._threeCamera instanceof THREE.OrthographicCamera) {
|
||||
this._threeCamera.zoom = this._layer.getCameraZoom();
|
||||
this._threeCamera.updateProjectionMatrix();
|
||||
this._threeCamera.position.z = this._layer.getCameraZ(null);
|
||||
} else {
|
||||
this._threeCamera.position.z = this._layer.getCameraZ(
|
||||
this._threeCamera.fov
|
||||
);
|
||||
}
|
||||
}
|
||||
this._update3DCameraFromLayer();
|
||||
|
||||
let effectivePixiZoom = 1;
|
||||
const angle = -gdjs.toRad(this._layer.getCameraRotation());
|
||||
@@ -1186,6 +1284,12 @@ namespace gdjs {
|
||||
}
|
||||
this._threeCameraDirty = false;
|
||||
}
|
||||
|
||||
// Upload the instance matrices written during object pre-render of this
|
||||
// frame, before the layer is drawn.
|
||||
if (this._modelInstancePool) {
|
||||
this._modelInstancePool.flush();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -5,7 +5,8 @@
|
||||
*/
|
||||
namespace gdjs {
|
||||
export interface AbstractRuntimeObject3D
|
||||
extends gdjs.RuntimeObject,
|
||||
extends
|
||||
gdjs.RuntimeObject,
|
||||
gdjs.Base3DHandler,
|
||||
gdjs.Resizable,
|
||||
gdjs.Scalable,
|
||||
@@ -55,6 +56,151 @@ namespace gdjs {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* A 3D object that can cull itself against a camera frustum.
|
||||
*
|
||||
* Implemented by both 3D object families - `gdjs.RuntimeObject3D` and
|
||||
* `gdjs.CustomRuntimeObject3D` - which are unrelated classes (the custom one
|
||||
* extends `gdjs.CustomRuntimeObject`), so culling tests for this capability
|
||||
* rather than for a base class.
|
||||
*/
|
||||
export interface CullableRuntimeObject3D {
|
||||
/**
|
||||
* Writes the object's conservative axis-aligned box into `box` and returns
|
||||
* it. Allocates nothing.
|
||||
*/
|
||||
getAABB3D(box: THREE.Box3): THREE.Box3;
|
||||
/** True when the object's box intersects `frustum`. */
|
||||
isInFrustum(frustum: THREE.Frustum): boolean;
|
||||
}
|
||||
|
||||
/** @category Objects > 3D Objects */
|
||||
export namespace CullableRuntimeObject3D {
|
||||
export const isCullable3D = (
|
||||
object: gdjs.RuntimeObject
|
||||
): object is gdjs.AbstractRuntimeObject3D & CullableRuntimeObject3D => {
|
||||
//@ts-ignore We are checking if the methods are present.
|
||||
return !!(object.getAABB3D && object.isInFrustum);
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The accessors the shared AABB maths needs. Both 3D object families have all
|
||||
* of them (`gdjs.RuntimeObject3D` and `gdjs.CustomRuntimeObject3D`).
|
||||
*/
|
||||
export type Object3DBoxSource = {
|
||||
getDrawableX(): float;
|
||||
getDrawableY(): float;
|
||||
getDrawableZ(): float;
|
||||
getWidth(): float;
|
||||
getHeight(): float;
|
||||
getDepth(): float;
|
||||
getRotationX(): float;
|
||||
getRotationY(): float;
|
||||
angle: float;
|
||||
};
|
||||
|
||||
/**
|
||||
* The AABB maths of `CullableRuntimeObject3D`, shared by both 3D object
|
||||
* families so the two cannot drift apart.
|
||||
* @category Objects > 3D Objects
|
||||
*/
|
||||
export namespace Object3DCulling {
|
||||
/** Scratch objects: never returned to callers, never read across calls. */
|
||||
const temporaryBox = new THREE.Box3();
|
||||
const temporaryMatrix = new THREE.Matrix4();
|
||||
const temporaryEuler = new THREE.Euler();
|
||||
|
||||
/**
|
||||
* Write the object's conservative axis-aligned box into `box` and return it.
|
||||
*
|
||||
* The box takes the position, the size and the object's Euler rotation
|
||||
* (`ZYX` order, like the renderers) into account: it is the bounding box of
|
||||
* the rotated box, so a visible object can never be culled by mistake. It is
|
||||
* conservative, not tight - a non-axis-aligned rotation yields a larger box
|
||||
* than the object's silhouette, which is the safe direction for culling.
|
||||
*
|
||||
* Allocates nothing.
|
||||
*/
|
||||
export const computeAABB = (
|
||||
object: Object3DBoxSource,
|
||||
box: THREE.Box3
|
||||
): THREE.Box3 => {
|
||||
const halfWidth = object.getWidth() / 2;
|
||||
const halfHeight = object.getHeight() / 2;
|
||||
const halfDepth = object.getDepth() / 2;
|
||||
const centerX = object.getDrawableX() + halfWidth;
|
||||
const centerY = object.getDrawableY() + halfHeight;
|
||||
const centerZ = object.getDrawableZ() + halfDepth;
|
||||
|
||||
if (
|
||||
object.getRotationX() === 0 &&
|
||||
object.getRotationY() === 0 &&
|
||||
object.angle === 0
|
||||
) {
|
||||
// No rotation: the box is already axis-aligned.
|
||||
box.min.set(
|
||||
centerX - halfWidth,
|
||||
centerY - halfHeight,
|
||||
centerZ - halfDepth
|
||||
);
|
||||
box.max.set(
|
||||
centerX + halfWidth,
|
||||
centerY + halfHeight,
|
||||
centerZ + halfDepth
|
||||
);
|
||||
return box;
|
||||
}
|
||||
|
||||
// Half-extents of the rotated box: for each world axis, the sum of the
|
||||
// absolute contributions of the three local half-extents.
|
||||
temporaryEuler.set(
|
||||
gdjs.toRad(object.getRotationX()),
|
||||
gdjs.toRad(object.getRotationY()),
|
||||
gdjs.toRad(object.angle),
|
||||
'ZYX'
|
||||
);
|
||||
temporaryMatrix.makeRotationFromEuler(temporaryEuler);
|
||||
const elements = temporaryMatrix.elements;
|
||||
const halfExtentX =
|
||||
Math.abs(elements[0]) * halfWidth +
|
||||
Math.abs(elements[4]) * halfHeight +
|
||||
Math.abs(elements[8]) * halfDepth;
|
||||
const halfExtentY =
|
||||
Math.abs(elements[1]) * halfWidth +
|
||||
Math.abs(elements[5]) * halfHeight +
|
||||
Math.abs(elements[9]) * halfDepth;
|
||||
const halfExtentZ =
|
||||
Math.abs(elements[2]) * halfWidth +
|
||||
Math.abs(elements[6]) * halfHeight +
|
||||
Math.abs(elements[10]) * halfDepth;
|
||||
|
||||
box.min.set(
|
||||
centerX - halfExtentX,
|
||||
centerY - halfExtentY,
|
||||
centerZ - halfExtentZ
|
||||
);
|
||||
box.max.set(
|
||||
centerX + halfExtentX,
|
||||
centerY + halfExtentY,
|
||||
centerZ + halfExtentZ
|
||||
);
|
||||
return box;
|
||||
};
|
||||
|
||||
/**
|
||||
* True when the object's box intersects `frustum`. Uses the shared scratch
|
||||
* box, so calling it for every object every frame allocates nothing.
|
||||
*/
|
||||
export const isInFrustum = (
|
||||
object: Object3DBoxSource,
|
||||
frustum: THREE.Frustum
|
||||
): boolean => {
|
||||
computeAABB(object, temporaryBox);
|
||||
return frustum.intersectsBox(temporaryBox);
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* An axis-aligned bounding box. Used to represents a box around an object for example.
|
||||
* @category Core Engine > Object
|
||||
@@ -1760,7 +1906,7 @@ namespace gdjs {
|
||||
* remove null ones.
|
||||
*/
|
||||
updateForces(elapsedTime: float): void {
|
||||
for (let i = 0; i < this._instantForces.length; ) {
|
||||
for (let i = 0; i < this._instantForces.length;) {
|
||||
const force = this._instantForces[i];
|
||||
const multiplier = force.getMultiplier();
|
||||
if (multiplier === 1) {
|
||||
|
||||
@@ -521,11 +521,43 @@ namespace gdjs {
|
||||
// instead.
|
||||
// - objects having effects rendering outside of their visibility AABB.
|
||||
|
||||
// TODO (3D) culling - add support for 3D object culling?
|
||||
this._updateLayersCameraCoordinates(2);
|
||||
const allInstancesList = this.getAdhocListOfAllInstances();
|
||||
for (let i = 0, len = allInstancesList.length; i < len; ++i) {
|
||||
const object = allInstancesList[i];
|
||||
|
||||
// 3D objects report no `getRendererObject()` (see
|
||||
// `RuntimeObject3D.getRendererObject`), so they need their own
|
||||
// culling: their renderer object is the THREE object, and visibility
|
||||
// is decided against the 3D camera frustum of their layer. No safety
|
||||
// margin is used here (unlike the 2D "scale of 2" above): the box is
|
||||
// the object's true conservative extent, so a visible object cannot
|
||||
// be culled by mistake.
|
||||
//
|
||||
// Both 3D object families are caught by the capability, not by a base
|
||||
// class: `CustomRuntimeObject3D` does not extend `RuntimeObject3D`.
|
||||
if (gdjs.CullableRuntimeObject3D.isCullable3D(object)) {
|
||||
const layer = this.getLayer(object.getLayer());
|
||||
const layerRenderer = layer ? layer.getRenderer() : null;
|
||||
const frustum = layerRenderer
|
||||
? layerRenderer.getThreeFrustum()
|
||||
: null;
|
||||
const isVisible =
|
||||
!object.isHidden() && (!frustum || object.isInFrustum(frustum));
|
||||
const threeObject = object.get3DRendererObject();
|
||||
if (threeObject) threeObject.visible = isVisible;
|
||||
|
||||
// Update effects and perform the pre-render update only if the
|
||||
// object is visible, like the 2D path below.
|
||||
if (isVisible) {
|
||||
this._runtimeGame
|
||||
.getEffectsManager()
|
||||
.updatePreRender(object.getRendererEffects(), object);
|
||||
object.updatePreRender(this);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
const rendererObject = object.getRendererObject();
|
||||
if (rendererObject) {
|
||||
if (object.isHidden()) {
|
||||
|
||||
@@ -172,6 +172,11 @@ module.exports = function (config) {
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/TextObject/textruntimeobject-pixi-renderer.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/3D/A_RuntimeObject3D.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/3D/A_RuntimeObject3DRenderer.js',
|
||||
// Loaded before the object that uses them.
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/3D/Scene3DTools.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/3D/Model3DInstancePool.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/3D/Model3DRuntimeObject.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/3D/Model3DRuntimeObject3DRenderer.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/3D/Cube3DRuntimeObject.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/3D/Cube3DRuntimeObjectPixiRenderer.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/3D/CustomRuntimeObject3D.js',
|
||||
@@ -190,7 +195,10 @@ module.exports = function (config) {
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/TileMap/TileMapRuntimeManager.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/TileMap/TileMapBehavior.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/TileMap/AbstractTileMapRuntimeObject.js',
|
||||
// The tile map renderer needs the pixi-tilemap plugin at construction.
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/TileMap/pixi-tilemap/dist/pixi-tilemap.umd.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/TileMap/tilemapruntimeobject-pixi-renderer.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/TileMap/simpletilemapruntimeobject.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/TileMap/tilemapruntimeobject.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/TileMap/collision/TileMapCollisionMaskRenderer.js',
|
||||
'./newIDE/app/resources/GDJS/Runtime/Extensions/TileMap/collision/TransformedTileMap.js',
|
||||
@@ -234,6 +242,13 @@ module.exports = function (config) {
|
||||
served: true,
|
||||
nocache: false,
|
||||
},
|
||||
{
|
||||
pattern: './GDJS/tests/tests-utils/assets/*.glb',
|
||||
watched: false,
|
||||
included: false,
|
||||
served: true,
|
||||
nocache: false,
|
||||
},
|
||||
{
|
||||
pattern: './GDJS/tests/tests-utils/simple-tiled-map/*.json',
|
||||
watched: false,
|
||||
|
||||
Binary file not shown.
@@ -0,0 +1,191 @@
|
||||
// @ts-check
|
||||
// @ts-nocheck The expect.js assertions take an optional message argument, which
|
||||
// the installed `expect` typings do not declare.
|
||||
|
||||
describe('3D object culling', function () {
|
||||
/**
|
||||
* A scene with one 3D layer, and a custom 3D object whose inner child is a
|
||||
* plain 2D sprite (so the legacy 2D AABB path cannot decide the 3D object's
|
||||
* visibility - only the new frustum branch can).
|
||||
*
|
||||
* @return {Promise<{runtimeGame: gdjs.RuntimeGame, runtimeScene: gdjs.RuntimeScene, customObject: gdjs.CustomRuntimeObject3D}>}
|
||||
*/
|
||||
const makeSceneWith3DLayer = async () => {
|
||||
const runtimeGame = await gdjs.getPixiRuntimeGameWithAssets();
|
||||
const runtimeScene = new gdjs.RuntimeScene(runtimeGame);
|
||||
runtimeScene.addLayer({
|
||||
name: '',
|
||||
renderingType: '3d',
|
||||
cameraType: 'perspective',
|
||||
visibility: true,
|
||||
cameras: [],
|
||||
effects: [],
|
||||
ambientLightColorR: 255,
|
||||
ambientLightColorG: 255,
|
||||
ambientLightColorB: 255,
|
||||
isLightingLayer: false,
|
||||
followBaseLayerCamera: false,
|
||||
});
|
||||
|
||||
// The event-based object declaration comes from
|
||||
// getPixiRuntimeGameWithAssets.
|
||||
const customObject = new gdjs.CustomRuntimeObject3D(runtimeScene, {
|
||||
name: 'MyCustomObject',
|
||||
type: 'MyExtension::MyEventsBasedObject',
|
||||
variant: '',
|
||||
isInnerAreaFollowingParentSize: false,
|
||||
variables: [],
|
||||
behaviors: [],
|
||||
effects: [],
|
||||
content: { width: 0, height: 0, depth: 0 },
|
||||
});
|
||||
runtimeScene.addObject(customObject);
|
||||
|
||||
return { runtimeGame, runtimeScene, customObject };
|
||||
};
|
||||
|
||||
/**
|
||||
* The scene only culls from its second frame on (`RuntimeScene` defers to the
|
||||
* base container on the first frame), so step it twice before asserting.
|
||||
*/
|
||||
const advancePastFirstFrame = (runtimeScene) => {
|
||||
runtimeScene.getTimeManager().update(16, 60);
|
||||
runtimeScene.getTimeManager().update(16, 60);
|
||||
};
|
||||
|
||||
it('culls a 3D object outside the camera frustum and keeps one inside it', async () => {
|
||||
const { runtimeScene, customObject } = await makeSceneWith3DLayer();
|
||||
|
||||
customObject.setPosition(0, 0);
|
||||
customObject.setZ(0);
|
||||
customObject.setWidth(50);
|
||||
customObject.setHeight(50);
|
||||
customObject.setDepth(50);
|
||||
|
||||
const threeObject = customObject.get3DRendererObject();
|
||||
// the custom object has a 3D renderer object
|
||||
expect(threeObject).to.be.ok();
|
||||
|
||||
const layerRenderer = runtimeScene.getLayer('').getRenderer();
|
||||
// the 3D layer has a frustum
|
||||
expect(layerRenderer.getThreeFrustum()).to.be.ok();
|
||||
|
||||
advancePastFirstFrame(runtimeScene);
|
||||
|
||||
// Put the camera far from the object: it must be culled.
|
||||
runtimeScene.getLayer('').setCameraX(100000);
|
||||
runtimeScene.getLayer('').setCameraY(100000);
|
||||
runtimeScene.render();
|
||||
// an object 100000 pixels away is culled
|
||||
expect(threeObject.visible).to.be(false);
|
||||
|
||||
// Put the camera back on the object: it must be visible again.
|
||||
runtimeScene.getLayer('').setCameraX(25);
|
||||
runtimeScene.getLayer('').setCameraY(25);
|
||||
runtimeScene.render();
|
||||
// an object under the camera is visible
|
||||
expect(threeObject.visible).to.be(true);
|
||||
});
|
||||
|
||||
it('never culls a 3D object on a layer that has no 3D camera', async () => {
|
||||
const runtimeGame = await gdjs.getPixiRuntimeGameWithAssets();
|
||||
const runtimeScene = new gdjs.RuntimeScene(runtimeGame);
|
||||
runtimeScene.addLayer({
|
||||
name: '',
|
||||
// A plain 2D layer: no 3D camera is set up for it.
|
||||
visibility: true,
|
||||
cameras: [],
|
||||
effects: [],
|
||||
ambientLightColorR: 255,
|
||||
ambientLightColorG: 255,
|
||||
ambientLightColorB: 255,
|
||||
isLightingLayer: false,
|
||||
followBaseLayerCamera: false,
|
||||
});
|
||||
|
||||
const customObject = new gdjs.CustomRuntimeObject3D(runtimeScene, {
|
||||
name: 'MyCustomObject',
|
||||
type: 'MyExtension::MyEventsBasedObject',
|
||||
variant: '',
|
||||
isInnerAreaFollowingParentSize: false,
|
||||
variables: [],
|
||||
behaviors: [],
|
||||
effects: [],
|
||||
content: { width: 0, height: 0, depth: 0 },
|
||||
});
|
||||
runtimeScene.addObject(customObject);
|
||||
|
||||
// without a frustum, visibility only follows isHidden()
|
||||
expect(runtimeScene.getLayer('').getRenderer().getThreeFrustum()).to.be(
|
||||
null
|
||||
);
|
||||
|
||||
advancePastFirstFrame(runtimeScene);
|
||||
customObject.setPosition(100000, 100000);
|
||||
runtimeScene.render();
|
||||
expect(customObject.get3DRendererObject().visible).to.be(true);
|
||||
|
||||
customObject.hide(true);
|
||||
runtimeScene.render();
|
||||
// a hidden object stays hidden
|
||||
expect(customObject.get3DRendererObject().visible).to.be(false);
|
||||
});
|
||||
|
||||
it('computes a conservative axis-aligned box that accounts for rotation', async () => {
|
||||
const { customObject } = await makeSceneWith3DLayer();
|
||||
|
||||
// Put the object at a known place, and read its extent through the box.
|
||||
customObject.setPosition(0, 0);
|
||||
customObject.setZ(0);
|
||||
customObject.setWidth(100);
|
||||
customObject.setHeight(100);
|
||||
customObject.setDepth(100);
|
||||
customObject.setAngle(0);
|
||||
customObject.setRotationX(0);
|
||||
customObject.setRotationY(0);
|
||||
customObject.setCenterXInScene(50);
|
||||
customObject.setCenterYInScene(50);
|
||||
customObject.setCenterZInScene(50);
|
||||
|
||||
const box = new THREE.Box3();
|
||||
customObject.getAABB3D(box);
|
||||
// The box is centered on the object's center, with the object's size.
|
||||
expect(Math.round(box.max.x - box.min.x)).to.be(100);
|
||||
expect(Math.round(box.max.y - box.min.y)).to.be(100);
|
||||
expect(Math.round(box.max.z - box.min.z)).to.be(100);
|
||||
expect(Math.round((box.min.x + box.max.x) / 2)).to.be(50);
|
||||
expect(Math.round((box.min.y + box.max.y) / 2)).to.be(50);
|
||||
expect(Math.round((box.min.z + box.max.z) / 2)).to.be(50);
|
||||
const unrotatedWidth = box.max.x - box.min.x;
|
||||
|
||||
// Rotating 45 degrees around Z grows the X and Y extents (never shrinks
|
||||
// them): a conservative box cannot cull a visible object.
|
||||
customObject.setAngle(45);
|
||||
customObject.getAABB3D(box);
|
||||
expect(box.max.x - box.min.x).to.be.above(unrotatedWidth);
|
||||
expect(box.max.y - box.min.y).to.be.above(unrotatedWidth);
|
||||
// Z is untouched by a Z-axis rotation.
|
||||
expect(Math.round(box.max.z - box.min.z)).to.be(100);
|
||||
|
||||
// The center never moves.
|
||||
expect(Math.round((box.min.x + box.max.x) / 2)).to.be(50);
|
||||
expect(Math.round((box.min.y + box.max.y) / 2)).to.be(50);
|
||||
});
|
||||
|
||||
it('reuses its scratch box, so culling allocates nothing per object', async () => {
|
||||
const { customObject } = await makeSceneWith3DLayer();
|
||||
customObject.setPosition(0, 0);
|
||||
customObject.setWidth(10);
|
||||
customObject.setHeight(10);
|
||||
customObject.setDepth(10);
|
||||
|
||||
const first = new THREE.Box3();
|
||||
customObject.getAABB3D(first);
|
||||
const second = new THREE.Box3();
|
||||
customObject.getAABB3D(second);
|
||||
// The caller's box may be a different object; the class scratch is not
|
||||
// returned. Reading twice in a row must give the same values.
|
||||
expect(second.min.x).to.be(first.min.x);
|
||||
expect(second.max.x).to.be(first.max.x);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,194 @@
|
||||
// @ts-check
|
||||
// @ts-nocheck The runtime types do not describe the internals these tests drive
|
||||
// (the instance pool slot, the resource manager registry), which the built
|
||||
// runtime does expose.
|
||||
|
||||
describe('Model3D instancing', function () {
|
||||
/** The single-mesh model shipped for these tests. */
|
||||
const MODEL_RESOURCE_NAME = 'model3D/tests-utils/assets/single-mesh.glb';
|
||||
|
||||
/**
|
||||
* A game with one loaded single-mesh model resource, so the instanced path
|
||||
* can apply. `getPixiRuntimeGameWithAssets` has no 3D resource, so the model
|
||||
* is loaded through the manager from the test assets folder.
|
||||
* @return {Promise<gdjs.RuntimeGame>}
|
||||
*/
|
||||
const makeGameWithSingleMeshModel = async () => {
|
||||
const runtimeGame = await gdjs.getPixiRuntimeGameWithAssets();
|
||||
const model3DManager = runtimeGame.getModel3DManager();
|
||||
// The resource has to be declared for the manager to find its file.
|
||||
const resourceLoader = model3DManager._resourceLoader;
|
||||
resourceLoader.privateResourceManager._resources.set(MODEL_RESOURCE_NAME, {
|
||||
name: MODEL_RESOURCE_NAME,
|
||||
kind: 'model3D',
|
||||
file: 'base/GDJS/tests/tests-utils/assets/single-mesh.glb',
|
||||
metadata: '',
|
||||
userAdded: true,
|
||||
});
|
||||
await model3DManager.loadResource(MODEL_RESOURCE_NAME);
|
||||
return runtimeGame;
|
||||
};
|
||||
|
||||
const createSceneWith3DLayer = (runtimeGame) => {
|
||||
const runtimeScene = new gdjs.RuntimeScene(runtimeGame);
|
||||
runtimeScene.addLayer({
|
||||
name: '',
|
||||
renderingType: '3d',
|
||||
cameraType: 'perspective',
|
||||
visibility: true,
|
||||
cameras: [],
|
||||
effects: [],
|
||||
ambientLightColorR: 255,
|
||||
ambientLightColorG: 255,
|
||||
ambientLightColorB: 255,
|
||||
isLightingLayer: false,
|
||||
followBaseLayerCamera: false,
|
||||
});
|
||||
return runtimeScene;
|
||||
};
|
||||
|
||||
/**
|
||||
* @param {gdjs.RuntimeInstanceContainer} instanceContainer
|
||||
* @param {string} name
|
||||
* @param {boolean} useInstancing
|
||||
* @param {string} [materialType]
|
||||
*/
|
||||
const createModel3D = (
|
||||
instanceContainer,
|
||||
name,
|
||||
useInstancing,
|
||||
materialType = 'StandardWithoutMetalness'
|
||||
) => {
|
||||
const object = new gdjs.Model3DRuntimeObject(instanceContainer, {
|
||||
name,
|
||||
type: 'Scene3D::Model3DObject',
|
||||
variables: [],
|
||||
behaviors: [],
|
||||
effects: [],
|
||||
content: {
|
||||
width: 100,
|
||||
height: 100,
|
||||
depth: 100,
|
||||
rotationX: 0,
|
||||
rotationY: 0,
|
||||
rotationZ: 0,
|
||||
keepAspectRatio: false,
|
||||
modelResourceName: MODEL_RESOURCE_NAME,
|
||||
materialType,
|
||||
originLocation: 'TopLeft',
|
||||
centerLocation: 'ObjectCenter',
|
||||
animations: [],
|
||||
crossfadeDuration: 0,
|
||||
isCastingShadow: true,
|
||||
isReceivingShadow: true,
|
||||
useInstancing,
|
||||
},
|
||||
});
|
||||
instanceContainer.addObject(object);
|
||||
return object;
|
||||
};
|
||||
|
||||
const getThreeChildren = (runtimeScene) =>
|
||||
runtimeScene.getLayer('').getRenderer().getThreeGroup().children;
|
||||
|
||||
/** The instance pool of the scene's 3D layer. */
|
||||
const getLayerPool = (runtimeScene) =>
|
||||
runtimeScene.getLayer('').getRenderer().getModelInstancePool();
|
||||
|
||||
// `constructor.name` is minified in the built runtime, so count with the
|
||||
// Three.js type flags instead.
|
||||
const countInstancedMeshes = (runtimeScene) =>
|
||||
getThreeChildren(runtimeScene).filter((child) => child.isInstancedMesh)
|
||||
.length;
|
||||
|
||||
const countGroups = (runtimeScene) =>
|
||||
getThreeChildren(runtimeScene).filter((child) => child.isGroup).length;
|
||||
|
||||
it('shares one InstancedMesh between instanced models and keeps the clone path for the others', async () => {
|
||||
const runtimeGame = await makeGameWithSingleMeshModel();
|
||||
const runtimeScene = createSceneWith3DLayer(runtimeGame);
|
||||
|
||||
const first = createModel3D(runtimeScene, 'First', true);
|
||||
const second = createModel3D(runtimeScene, 'Second', true);
|
||||
const third = createModel3D(runtimeScene, 'Third', true);
|
||||
// Not opted in: must keep the per-object clone.
|
||||
const cloned = createModel3D(runtimeScene, 'Cloned', false);
|
||||
|
||||
runtimeScene.render();
|
||||
|
||||
// the three instanced models share one InstancedMesh
|
||||
expect(countInstancedMeshes(runtimeScene)).to.be(1);
|
||||
// the fourth, non-instanced model still gets its own clone
|
||||
expect(countGroups(runtimeScene)).to.be(1);
|
||||
|
||||
// moving the first instance rewrites its matrix
|
||||
expect(getLayerPool(runtimeScene).getKeys().length).to.be(1);
|
||||
expect(first.getRenderer().getInstanceSlot()).to.be(0);
|
||||
expect(second.getRenderer().getInstanceSlot()).to.be(1);
|
||||
expect(third.getRenderer().getInstanceSlot()).to.be(2);
|
||||
expect(cloned.getRenderer().getInstanceSlot()).to.be(null);
|
||||
});
|
||||
|
||||
it('writes the instance matrix of each instance when it moves', async () => {
|
||||
const runtimeGame = await makeGameWithSingleMeshModel();
|
||||
const runtimeScene = createSceneWith3DLayer(runtimeGame);
|
||||
|
||||
const first = createModel3D(runtimeScene, 'First', true);
|
||||
const second = createModel3D(runtimeScene, 'Second', true);
|
||||
runtimeScene.render();
|
||||
|
||||
const key = getLayerPool(runtimeScene).getKeys()[0];
|
||||
const instancedMesh = getLayerPool(runtimeScene).getInstancedMesh(key);
|
||||
expect(instancedMesh).to.be.ok();
|
||||
|
||||
const matrix = new THREE.Matrix4();
|
||||
instancedMesh.getMatrixAt(0, matrix);
|
||||
const firstX = matrix.elements[12];
|
||||
|
||||
first.setX(1000);
|
||||
runtimeScene.render();
|
||||
instancedMesh.getMatrixAt(0, matrix);
|
||||
expect(matrix.elements[12]).to.be.above(firstX);
|
||||
|
||||
instancedMesh.getMatrixAt(1, matrix);
|
||||
const secondX = matrix.elements[12];
|
||||
second.setX(2000);
|
||||
runtimeScene.render();
|
||||
instancedMesh.getMatrixAt(1, matrix);
|
||||
// the freed slot 0 is handed to the next instanced model
|
||||
expect(matrix.elements[12]).to.be.above(secondX);
|
||||
});
|
||||
|
||||
it('frees a slot when an instanced model leaves the scene, for reuse', async () => {
|
||||
const runtimeGame = await makeGameWithSingleMeshModel();
|
||||
const runtimeScene = createSceneWith3DLayer(runtimeGame);
|
||||
|
||||
const first = createModel3D(runtimeScene, 'First', true);
|
||||
const second = createModel3D(runtimeScene, 'Second', true);
|
||||
runtimeScene.render();
|
||||
|
||||
expect(first.getRenderer().getInstanceSlot()).to.be(0);
|
||||
expect(second.getRenderer().getInstanceSlot()).to.be(1);
|
||||
|
||||
// Removing the first object gives slot 0 back.
|
||||
runtimeScene.markObjectForDeletion(first);
|
||||
runtimeScene.render();
|
||||
|
||||
const third = createModel3D(runtimeScene, 'Third', true);
|
||||
runtimeScene.render();
|
||||
expect(third.getRenderer().getInstanceSlot()).to.be(0);
|
||||
});
|
||||
|
||||
it('silently keeps the clone path for a Basic material', async () => {
|
||||
const runtimeGame = await makeGameWithSingleMeshModel();
|
||||
const runtimeScene = createSceneWith3DLayer(runtimeGame);
|
||||
|
||||
// "Basic" replaces every material per instance: not shareable.
|
||||
const object = createModel3D(runtimeScene, 'Basic', true, 'Basic');
|
||||
runtimeScene.render();
|
||||
|
||||
expect(object.getRenderer().getInstanceSlot()).to.be(null);
|
||||
expect(countInstancedMeshes(runtimeScene)).to.be(0);
|
||||
expect(countGroups(runtimeScene)).to.be(1);
|
||||
});
|
||||
});
|
||||
@@ -7,6 +7,7 @@
|
||||
// session keys, or anything that requires unpredictability.
|
||||
|
||||
import axios from 'axios';
|
||||
import { getStudioRole, getToolsForRole } from '../AiGeneration/Studio/Roles';
|
||||
import {
|
||||
type AiRequest,
|
||||
type AiRequestMessage,
|
||||
@@ -158,13 +159,57 @@ export const getEndpointUrl = (
|
||||
const localAiRequestsCache: { [id: string]: AiRequest } = {};
|
||||
|
||||
/**
|
||||
* Reset CustomAIClient state (for testing).
|
||||
* One in-flight model turn per request id.
|
||||
*
|
||||
* A turn reads `localAiRequestsCache[id]`, awaits a full model call, then writes
|
||||
* the whole updated request back. Once the studio spawns a sub-agent, a parent
|
||||
* and its child are both taking turns, so that read-to-write window is a real
|
||||
* lost-update race: without this lock the later write discards the earlier one's
|
||||
* appended message.
|
||||
*
|
||||
* Different request ids never block each other.
|
||||
*/
|
||||
const localAiTurnTails: { [aiRequestId: string]: Promise<void> } = {};
|
||||
|
||||
/**
|
||||
* Run `turn` with the exclusive turn lock for `aiRequestId`, releasing it when
|
||||
* the turn settles. The lock is a promise tail, so a queued turn only starts
|
||||
* after the previous one has released it, and different request ids never block
|
||||
* each other.
|
||||
*/
|
||||
export const withLocalAiTurnLock = async <T>(
|
||||
aiRequestId: string,
|
||||
turn: () => Promise<T>
|
||||
): Promise<T> => {
|
||||
const tail = localAiTurnTails[aiRequestId] || Promise.resolve();
|
||||
let release: () => void;
|
||||
const held = new Promise<void>(resolve => {
|
||||
release = resolve;
|
||||
});
|
||||
const newTail = tail.then(() => held);
|
||||
localAiTurnTails[aiRequestId] = newTail;
|
||||
|
||||
await tail;
|
||||
try {
|
||||
return await turn();
|
||||
} finally {
|
||||
release();
|
||||
// Drop the entry once nothing is queued behind this turn.
|
||||
if (localAiTurnTails[aiRequestId] === newTail) {
|
||||
delete localAiTurnTails[aiRequestId];
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
/** Reset CustomAIClient state (for testing). */
|
||||
export const _resetCustomAiClientForTesting = () => {
|
||||
cachedConfig = null;
|
||||
for (const key of Object.keys(localAiRequestsCache)) {
|
||||
delete localAiRequestsCache[key];
|
||||
}
|
||||
for (const key of Object.keys(localAiTurnTails)) {
|
||||
delete localAiTurnTails[key];
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -1048,6 +1093,539 @@ export const GDEVELOP_OPENAI_TOOLS: Array<{|
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'create_extension',
|
||||
description:
|
||||
'Create a new events-functions extension in the project. Optionally copy an existing one.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
extension_name: {
|
||||
type: 'string',
|
||||
description:
|
||||
'Name of the extension to create. It is made unique automatically if taken.',
|
||||
},
|
||||
duplicated_extension_name: {
|
||||
type: 'string',
|
||||
description:
|
||||
'Name of an existing extension to copy into the new one. Omit to create an empty extension.',
|
||||
},
|
||||
full_name: {
|
||||
type: 'string',
|
||||
description: 'Human-readable title shown in the editor.',
|
||||
},
|
||||
short_description: {
|
||||
type: 'string',
|
||||
description: 'One-line description of the extension.',
|
||||
},
|
||||
description: {
|
||||
type: 'string',
|
||||
description: 'Full description of the extension.',
|
||||
},
|
||||
category: { type: 'string', description: 'Extension category.' },
|
||||
tags: {
|
||||
type: 'string',
|
||||
description: 'Comma-separated tags.',
|
||||
},
|
||||
author: { type: 'string', description: 'Extension author.' },
|
||||
},
|
||||
required: ['extension_name'],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'change_extension_properties',
|
||||
description:
|
||||
'Change properties of an extension, its dependencies, or delete the extension.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
extension_name: {
|
||||
type: 'string',
|
||||
description: 'Name of the extension to change.',
|
||||
},
|
||||
new_name: {
|
||||
type: 'string',
|
||||
description: 'New name for the extension, to rename it.',
|
||||
},
|
||||
delete_this_extension: {
|
||||
type: 'boolean',
|
||||
description: 'Set to true to delete the extension.',
|
||||
},
|
||||
delete_even_if_used: {
|
||||
type: 'boolean',
|
||||
description:
|
||||
'Allow deleting an extension that is still used. Only when delete_this_extension is true.',
|
||||
},
|
||||
changed_properties: {
|
||||
type: 'array',
|
||||
description: 'Properties to set on the extension.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
property_name: {
|
||||
type: 'string',
|
||||
description:
|
||||
'One of: fullName, shortDescription, description, category, tags, version, author, helpPath, previewIconUrl, iconUrl, dimension.',
|
||||
},
|
||||
new_value: { type: 'string' },
|
||||
},
|
||||
required: ['property_name', 'new_value'],
|
||||
},
|
||||
},
|
||||
changed_dependencies: {
|
||||
type: 'array',
|
||||
description: 'Dependencies to create, edit or delete.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
dependency_name: { type: 'string' },
|
||||
dependency_type: { type: 'string' },
|
||||
new_name: { type: 'string' },
|
||||
export_name: { type: 'string' },
|
||||
version: { type: 'string' },
|
||||
delete_this_dependency: { type: 'boolean' },
|
||||
extra_settings: {
|
||||
type: 'object',
|
||||
description: 'Dependency-specific settings, as strings.',
|
||||
},
|
||||
},
|
||||
required: ['dependency_name'],
|
||||
},
|
||||
},
|
||||
},
|
||||
required: ['extension_name'],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'create_custom_object',
|
||||
description:
|
||||
'Create a custom object (an events-based object) in an extension, optionally as a copy of an existing one.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
extension_name: {
|
||||
type: 'string',
|
||||
description: 'Extension to create the custom object in.',
|
||||
},
|
||||
custom_object_name: {
|
||||
type: 'string',
|
||||
description: 'Name of the new custom object.',
|
||||
},
|
||||
duplicated_custom_object_name: {
|
||||
type: 'string',
|
||||
description: 'Existing custom object to copy.',
|
||||
},
|
||||
duplicated_from_extension_name: {
|
||||
type: 'string',
|
||||
description:
|
||||
'Extension holding the custom object to copy. Defaults to extension_name.',
|
||||
},
|
||||
area: {
|
||||
type: 'object',
|
||||
description:
|
||||
'Default area of the object, as numbers: minX, minY, minZ, maxX, maxY, maxZ.',
|
||||
properties: {
|
||||
minX: { type: 'number' },
|
||||
minY: { type: 'number' },
|
||||
minZ: { type: 'number' },
|
||||
maxX: { type: 'number' },
|
||||
maxY: { type: 'number' },
|
||||
maxZ: { type: 'number' },
|
||||
},
|
||||
},
|
||||
},
|
||||
required: ['extension_name', 'custom_object_name'],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'change_custom_object',
|
||||
description:
|
||||
'Change a custom object: rename, delete, change its settings, its variants or its properties.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
extension_name: {
|
||||
type: 'string',
|
||||
description: 'Extension holding the custom object.',
|
||||
},
|
||||
custom_object_name: {
|
||||
type: 'string',
|
||||
description: 'Name of the custom object to change.',
|
||||
},
|
||||
new_name: { type: 'string', description: 'New name, to rename it.' },
|
||||
delete_this_custom_object: {
|
||||
type: 'boolean',
|
||||
description: 'Set to true to delete the custom object.',
|
||||
},
|
||||
delete_even_if_used: {
|
||||
type: 'boolean',
|
||||
description:
|
||||
'Allow deleting a custom object that is still used. Only when delete_this_custom_object is true.',
|
||||
},
|
||||
fit_area_to_children: {
|
||||
type: 'string',
|
||||
description:
|
||||
'Whether to fit the area to the children (true/false).',
|
||||
},
|
||||
changed_settings: {
|
||||
type: 'array',
|
||||
description: 'Settings to change on the custom object.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
setting_name: { type: 'string' },
|
||||
new_value: {
|
||||
description:
|
||||
'String, boolean or number depending on the setting.',
|
||||
},
|
||||
},
|
||||
required: ['setting_name', 'new_value'],
|
||||
},
|
||||
},
|
||||
changed_variants: {
|
||||
type: 'array',
|
||||
description: 'Variants to create or delete.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
variant_name: { type: 'string' },
|
||||
duplicated_from_variant_name: { type: 'string' },
|
||||
delete_this_variant: { type: 'boolean' },
|
||||
},
|
||||
required: ['variant_name'],
|
||||
},
|
||||
},
|
||||
changed_properties: {
|
||||
type: 'array',
|
||||
description: 'Properties to set on the custom object.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
property_name: { type: 'string' },
|
||||
new_value: { type: 'string' },
|
||||
},
|
||||
required: ['property_name', 'new_value'],
|
||||
},
|
||||
},
|
||||
},
|
||||
required: ['extension_name', 'custom_object_name'],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'create_custom_behavior',
|
||||
description:
|
||||
'Create a custom behavior (an events-based behavior) in an extension.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
extension_name: {
|
||||
type: 'string',
|
||||
description: 'Extension to create the behavior in.',
|
||||
},
|
||||
custom_behavior_name: {
|
||||
type: 'string',
|
||||
description: 'Name of the new custom behavior.',
|
||||
},
|
||||
duplicated_custom_behavior_name: {
|
||||
type: 'string',
|
||||
description: 'Existing custom behavior to copy.',
|
||||
},
|
||||
duplicated_from_extension_name: {
|
||||
type: 'string',
|
||||
description:
|
||||
'Extension holding the behavior to copy. Defaults to extension_name.',
|
||||
},
|
||||
full_name: {
|
||||
type: 'string',
|
||||
description: 'Human-readable title.',
|
||||
},
|
||||
description: { type: 'string' },
|
||||
object_type: {
|
||||
type: 'string',
|
||||
description:
|
||||
'Restrict the behavior to objects of this type. Omit for all objects.',
|
||||
},
|
||||
},
|
||||
required: ['extension_name', 'custom_behavior_name'],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'change_custom_behavior',
|
||||
description:
|
||||
'Change a custom behavior: rename, delete, change its settings, its shared properties or its properties.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
extension_name: {
|
||||
type: 'string',
|
||||
description: 'Extension holding the custom behavior.',
|
||||
},
|
||||
custom_behavior_name: {
|
||||
type: 'string',
|
||||
description: 'Name of the custom behavior to change.',
|
||||
},
|
||||
new_name: { type: 'string', description: 'New name, to rename it.' },
|
||||
delete_this_custom_behavior: {
|
||||
type: 'boolean',
|
||||
description: 'Set to true to delete the custom behavior.',
|
||||
},
|
||||
delete_even_if_used: {
|
||||
type: 'boolean',
|
||||
description:
|
||||
'Allow deleting a behavior that is still used. Only when delete_this_custom_behavior is true.',
|
||||
},
|
||||
changed_settings: {
|
||||
type: 'array',
|
||||
description: 'Settings to change on the custom behavior.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
setting_name: { type: 'string' },
|
||||
new_value: {
|
||||
description:
|
||||
'String, boolean or number depending on the setting.',
|
||||
},
|
||||
},
|
||||
required: ['setting_name', 'new_value'],
|
||||
},
|
||||
},
|
||||
changed_shared_properties: {
|
||||
type: 'array',
|
||||
description: 'Shared properties of the behavior to change.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
property_name: { type: 'string' },
|
||||
new_value: { type: 'string' },
|
||||
},
|
||||
required: ['property_name', 'new_value'],
|
||||
},
|
||||
},
|
||||
changed_properties: {
|
||||
type: 'array',
|
||||
description: 'Properties of the behavior to change.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
property_name: { type: 'string' },
|
||||
new_value: { type: 'string' },
|
||||
},
|
||||
required: ['property_name', 'new_value'],
|
||||
},
|
||||
},
|
||||
},
|
||||
required: ['extension_name', 'custom_behavior_name'],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'create_custom_function',
|
||||
description:
|
||||
'Create a function in an extension, a custom object or a custom behavior.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
scope: {
|
||||
type: 'object',
|
||||
description:
|
||||
'Where to create the function. type "extension" needs extension_name; "custom_object" needs extension_name and custom_object_name; "custom_behavior" needs extension_name and custom_behavior_name.',
|
||||
properties: {
|
||||
type: {
|
||||
type: 'string',
|
||||
enum: ['extension', 'custom_object', 'custom_behavior'],
|
||||
},
|
||||
extension_name: { type: 'string' },
|
||||
custom_object_name: { type: 'string' },
|
||||
custom_behavior_name: { type: 'string' },
|
||||
},
|
||||
required: ['type'],
|
||||
},
|
||||
function_name: {
|
||||
type: 'string',
|
||||
description: 'Name of the new function.',
|
||||
},
|
||||
duplicated_function_name: {
|
||||
type: 'string',
|
||||
description: 'Existing function to copy.',
|
||||
},
|
||||
function_type: {
|
||||
type: 'string',
|
||||
description:
|
||||
'Action, Condition, Expression, StringExpression or ActionWithOperator.',
|
||||
},
|
||||
getter_name: {
|
||||
type: 'string',
|
||||
description:
|
||||
'For ActionWithOperator: the getter function to build from.',
|
||||
},
|
||||
full_name: { type: 'string' },
|
||||
description: { type: 'string' },
|
||||
group: { type: 'string' },
|
||||
sentence: {
|
||||
type: 'string',
|
||||
description: 'Sentence shown in the events sheet.',
|
||||
},
|
||||
parameters: {
|
||||
type: 'array',
|
||||
description: 'Parameters to declare on the function.',
|
||||
items: { type: 'object' },
|
||||
},
|
||||
},
|
||||
required: ['scope', 'function_name'],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'change_custom_function',
|
||||
description:
|
||||
'Change a function: rename, delete, change its settings or its parameters.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
scope: {
|
||||
type: 'object',
|
||||
description:
|
||||
'Where the function lives. type "extension" needs extension_name; "custom_object" needs extension_name and custom_object_name; "custom_behavior" needs extension_name and custom_behavior_name.',
|
||||
properties: {
|
||||
type: {
|
||||
type: 'string',
|
||||
enum: ['extension', 'custom_object', 'custom_behavior'],
|
||||
},
|
||||
extension_name: { type: 'string' },
|
||||
custom_object_name: { type: 'string' },
|
||||
custom_behavior_name: { type: 'string' },
|
||||
},
|
||||
required: ['type'],
|
||||
},
|
||||
function_name: {
|
||||
type: 'string',
|
||||
description: 'Name of the function to change.',
|
||||
},
|
||||
new_name: { type: 'string', description: 'New name, to rename it.' },
|
||||
delete_this_function: {
|
||||
type: 'boolean',
|
||||
description: 'Set to true to delete the function.',
|
||||
},
|
||||
changed_settings: {
|
||||
type: 'array',
|
||||
description: 'Settings to change on the function.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
setting_name: { type: 'string' },
|
||||
new_value: {
|
||||
description:
|
||||
'String, boolean or number depending on the setting.',
|
||||
},
|
||||
},
|
||||
required: ['setting_name', 'new_value'],
|
||||
},
|
||||
},
|
||||
changed_parameters: {
|
||||
type: 'array',
|
||||
description: 'Parameters to change on the function.',
|
||||
items: { type: 'object' },
|
||||
},
|
||||
},
|
||||
required: ['scope', 'function_name'],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'inspect_extension',
|
||||
description:
|
||||
'Read the structure of an extension: its custom objects, custom behaviors, functions and variants.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
extension_name: {
|
||||
type: 'string',
|
||||
description: 'Name of the extension to inspect.',
|
||||
},
|
||||
custom_object_name: {
|
||||
type: 'string',
|
||||
description: 'Restrict the output to this custom object.',
|
||||
},
|
||||
custom_behavior_name: {
|
||||
type: 'string',
|
||||
description: 'Restrict the output to this custom behavior.',
|
||||
},
|
||||
function_name: {
|
||||
type: 'string',
|
||||
description: 'Restrict the output to this function.',
|
||||
},
|
||||
variant_name: {
|
||||
type: 'string',
|
||||
description: 'Restrict the output to this variant.',
|
||||
},
|
||||
},
|
||||
required: ['extension_name'],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'spawn_agent',
|
||||
description:
|
||||
'Delegate one task to a studio sub-agent, which works on it and reports back. Only the studio lead may call this. Sub-agents never delegate further.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
role: {
|
||||
type: 'string',
|
||||
description:
|
||||
'Which specialist to use: "designer" writes the design document as GDD_ project variables; "developer" changes the project to implement a task; "tester" writes and runs a gameplay test and reports pass or fail.',
|
||||
enum: ['designer', 'developer', 'tester'],
|
||||
},
|
||||
short_title: {
|
||||
type: 'string',
|
||||
description:
|
||||
'A 2-5 word label shown in the chat for this sub-agent, e.g. "Build the town grid".',
|
||||
},
|
||||
task: {
|
||||
type: 'string',
|
||||
description:
|
||||
'The complete, self-contained instruction for the sub-agent. It cannot see this conversation, so state everything it needs: the scene, the objects, the expected result.',
|
||||
},
|
||||
context: {
|
||||
type: 'string',
|
||||
description:
|
||||
'Extra facts the sub-agent needs: relevant variable values, previous findings, constraints.',
|
||||
},
|
||||
related_task_id: {
|
||||
type: 'string',
|
||||
description:
|
||||
'The id of the create_or_update_plan task this agent works on. That task is marked done when the agent reports back.',
|
||||
},
|
||||
},
|
||||
required: ['role', 'short_title', 'task'],
|
||||
},
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
/**
|
||||
@@ -1252,17 +1830,35 @@ export const transformGDevelopMessagesToOpenAi = (
|
||||
|
||||
/**
|
||||
* Build the system prompt with project structure and guidelines.
|
||||
*
|
||||
* `role` and `spawnContextNote` are the studio's additions (see
|
||||
* `AiGeneration/Studio/`). Both are optional: with neither, the prompt is exactly
|
||||
* what the chat and the hosted-style agent turns have always used.
|
||||
*/
|
||||
export const buildSystemPrompt = ({
|
||||
gameProjectJson,
|
||||
projectSpecificExtensionsSummaryJson,
|
||||
mode,
|
||||
role,
|
||||
spawnContextNote,
|
||||
}: {|
|
||||
gameProjectJson?: string | null,
|
||||
projectSpecificExtensionsSummaryJson?: string | null,
|
||||
mode?: 'chat' | 'agent' | 'orchestrator',
|
||||
role?: string | null,
|
||||
spawnContextNote?: string | null,
|
||||
|}): string => {
|
||||
let prompt = `You are GDevelop AI Assistant, an expert game engine developer. You help users build games in GDevelop 5.
|
||||
const rolePrompt = role ? getStudioRole((role: any)).systemPrompt : null;
|
||||
|
||||
let prompt = rolePrompt
|
||||
? `${rolePrompt}
|
||||
|
||||
You are working inside GDevelop, the game engine this project is built with. The tools you can call are restricted to the ones your role is granted: if you need something you cannot do, report it instead of trying another route.
|
||||
|
||||
`
|
||||
: '';
|
||||
|
||||
prompt += `You are GDevelop AI Assistant, an expert game engine developer. You help users build games in GDevelop 5.
|
||||
|
||||
Guidelines:
|
||||
1. Always use available tools/functions to inspect or modify the project.
|
||||
@@ -1283,6 +1879,9 @@ Guidelines:
|
||||
if (projectSpecificExtensionsSummaryJson) {
|
||||
prompt += `\nInstalled Project Extensions:\n${projectSpecificExtensionsSummaryJson}\n`;
|
||||
}
|
||||
if (spawnContextNote) {
|
||||
prompt += `\n${spawnContextNote}\n`;
|
||||
}
|
||||
|
||||
return prompt;
|
||||
};
|
||||
@@ -1619,33 +2218,120 @@ export const customAddMessageToAiRequest = async ({
|
||||
projectSpecificExtensionsSummaryJson?: string | null,
|
||||
mode?: 'chat' | 'agent' | 'orchestrator',
|
||||
|}): Promise<AiRequest> => {
|
||||
const existing = localAiRequestsCache[aiRequestId] || {
|
||||
id: aiRequestId,
|
||||
createdAt: new Date().toISOString(),
|
||||
updatedAt: new Date().toISOString(),
|
||||
userId: LOCAL_BYOK_USER_ID,
|
||||
status: 'ready',
|
||||
output: [],
|
||||
error: null,
|
||||
};
|
||||
return withLocalAiTurnLock(aiRequestId, async () => {
|
||||
const existing = localAiRequestsCache[aiRequestId] || {
|
||||
id: aiRequestId,
|
||||
createdAt: new Date().toISOString(),
|
||||
updatedAt: new Date().toISOString(),
|
||||
userId: LOCAL_BYOK_USER_ID,
|
||||
status: 'ready',
|
||||
output: [],
|
||||
error: null,
|
||||
};
|
||||
|
||||
const output = [...(existing.output || [])];
|
||||
const output = [...(existing.output || [])];
|
||||
|
||||
if (functionCallOutputs && functionCallOutputs.length > 0) {
|
||||
for (const fcOutput of functionCallOutputs) {
|
||||
if (functionCallOutputs && functionCallOutputs.length > 0) {
|
||||
for (const fcOutput of functionCallOutputs) {
|
||||
output.push({
|
||||
type: 'function_call_output',
|
||||
call_id: fcOutput.call_id,
|
||||
output: fcOutput.output,
|
||||
messageId: `msg-fco-${Date.now()}-${Math.random()
|
||||
.toString(36)
|
||||
.substr(2, 4)}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (userMessage && userMessage.trim()) {
|
||||
output.push({
|
||||
type: 'function_call_output',
|
||||
call_id: fcOutput.call_id,
|
||||
output: fcOutput.output,
|
||||
messageId: `msg-fco-${Date.now()}-${Math.random()
|
||||
.toString(36)
|
||||
.substr(2, 4)}`,
|
||||
type: 'message',
|
||||
status: 'completed',
|
||||
role: 'user',
|
||||
content: [
|
||||
{
|
||||
type: 'user_request',
|
||||
status: 'completed',
|
||||
text: userMessage,
|
||||
},
|
||||
],
|
||||
messageId: `msg-user-${Date.now()}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (userMessage && userMessage.trim()) {
|
||||
output.push({
|
||||
const systemPrompt = buildSystemPrompt({
|
||||
gameProjectJson,
|
||||
projectSpecificExtensionsSummaryJson,
|
||||
mode: mode || existing.mode,
|
||||
});
|
||||
|
||||
const openAiMessages = transformGDevelopMessagesToOpenAi(
|
||||
output,
|
||||
systemPrompt
|
||||
);
|
||||
|
||||
const assistantResponse = await sendChatCompletion({
|
||||
messages: openAiMessages,
|
||||
tools: GDEVELOP_OPENAI_TOOLS,
|
||||
});
|
||||
|
||||
const assistantMsgId = `msg-asst-${Date.now()}`;
|
||||
const assistantMessage = parseAssistantMessage(
|
||||
assistantResponse,
|
||||
assistantMsgId
|
||||
);
|
||||
output.push(assistantMessage);
|
||||
|
||||
const currentCachedRequest = localAiRequestsCache[aiRequestId] || existing;
|
||||
const updatedAiRequest: AiRequest = {
|
||||
// Read the cache again, inside the lock: a suspend or a suggestion write
|
||||
// that landed while the model was answering must not be discarded.
|
||||
...currentCachedRequest,
|
||||
updatedAt: new Date().toISOString(),
|
||||
// A request suspended during the turn stays suspended: writing the answer
|
||||
// back must not silently resume it. It keeps the new output so the user
|
||||
// can read what the agent produced before it was stopped.
|
||||
status:
|
||||
currentCachedRequest.status === 'suspended' ? 'suspended' : 'ready',
|
||||
output,
|
||||
};
|
||||
|
||||
localAiRequestsCache[aiRequestId] = updatedAiRequest;
|
||||
saveLocalAiRequests();
|
||||
|
||||
return updatedAiRequest;
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* Create the `AiRequest` of one studio sub-agent: a child of `parentAiRequestId`,
|
||||
* with the role's prompt and tool subset, that runs exactly one model turn.
|
||||
*
|
||||
* The child is never listed as a top-level chat: it has a `parentAiRequestId`, so
|
||||
* the history ignores it (`AiRequestContext.updateAiRequest`).
|
||||
*/
|
||||
export const customCreateSubAgentAiRequest = async ({
|
||||
parentAiRequestId,
|
||||
roleId,
|
||||
userRequest,
|
||||
gameProjectJson,
|
||||
projectSpecificExtensionsSummaryJson,
|
||||
spawnContextNote,
|
||||
}: {|
|
||||
parentAiRequestId: string,
|
||||
roleId: string,
|
||||
userRequest: string,
|
||||
gameProjectJson: string | null,
|
||||
projectSpecificExtensionsSummaryJson: string | null,
|
||||
spawnContextNote?: string | null,
|
||||
|}): Promise<AiRequest> => {
|
||||
const reqId = `local-ai-${Date.now()}-${Math.random()
|
||||
.toString(36)
|
||||
.substr(2, 7)}`;
|
||||
|
||||
return withLocalAiTurnLock(reqId, async () => {
|
||||
const userMessage: AiRequestUserMessage = {
|
||||
type: 'message',
|
||||
status: 'completed',
|
||||
role: 'user',
|
||||
@@ -1653,47 +2339,62 @@ export const customAddMessageToAiRequest = async ({
|
||||
{
|
||||
type: 'user_request',
|
||||
status: 'completed',
|
||||
text: userMessage,
|
||||
text: userRequest,
|
||||
},
|
||||
],
|
||||
messageId: `msg-user-${Date.now()}`,
|
||||
};
|
||||
|
||||
const output: Array<AiRequestMessage> = [userMessage];
|
||||
|
||||
const systemPrompt = buildSystemPrompt({
|
||||
gameProjectJson,
|
||||
projectSpecificExtensionsSummaryJson,
|
||||
mode: 'agent',
|
||||
role: roleId,
|
||||
spawnContextNote,
|
||||
});
|
||||
}
|
||||
|
||||
const systemPrompt = buildSystemPrompt({
|
||||
gameProjectJson,
|
||||
projectSpecificExtensionsSummaryJson,
|
||||
mode: mode || existing.mode,
|
||||
const openAiMessages = transformGDevelopMessagesToOpenAi(
|
||||
output,
|
||||
systemPrompt
|
||||
);
|
||||
|
||||
const assistantResponse = await sendChatCompletion({
|
||||
messages: openAiMessages,
|
||||
tools: getToolsForRole((roleId: any), GDEVELOP_OPENAI_TOOLS),
|
||||
});
|
||||
|
||||
const assistantMessage = parseAssistantMessage(
|
||||
assistantResponse,
|
||||
`msg-asst-${Date.now()}`
|
||||
);
|
||||
output.push(assistantMessage);
|
||||
|
||||
const now = new Date().toISOString();
|
||||
const aiRequest: AiRequest = {
|
||||
id: reqId,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
userId: LOCAL_BYOK_USER_ID,
|
||||
gameProjectJson: gameProjectJson || null,
|
||||
status: 'ready',
|
||||
mode: 'agent',
|
||||
aiConfiguration: { presetId: 'default' },
|
||||
toolsVersion: 'v14',
|
||||
toolOptions: null,
|
||||
parentAiRequestId: parentAiRequestId,
|
||||
error: null,
|
||||
output,
|
||||
lastUserMessagePriceInCredits: 0,
|
||||
totalPriceInCredits: 0,
|
||||
};
|
||||
|
||||
localAiRequestsCache[reqId] = aiRequest;
|
||||
saveLocalAiRequests();
|
||||
|
||||
return aiRequest;
|
||||
});
|
||||
|
||||
const openAiMessages = transformGDevelopMessagesToOpenAi(
|
||||
output,
|
||||
systemPrompt
|
||||
);
|
||||
|
||||
const assistantResponse = await sendChatCompletion({
|
||||
messages: openAiMessages,
|
||||
tools: GDEVELOP_OPENAI_TOOLS,
|
||||
});
|
||||
|
||||
const assistantMsgId = `msg-asst-${Date.now()}`;
|
||||
const assistantMessage = parseAssistantMessage(
|
||||
assistantResponse,
|
||||
assistantMsgId
|
||||
);
|
||||
output.push(assistantMessage);
|
||||
|
||||
const updatedAiRequest: AiRequest = {
|
||||
...existing,
|
||||
updatedAt: new Date().toISOString(),
|
||||
status: 'ready',
|
||||
output,
|
||||
};
|
||||
|
||||
localAiRequestsCache[aiRequestId] = updatedAiRequest;
|
||||
saveLocalAiRequests();
|
||||
|
||||
return updatedAiRequest;
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -1756,10 +2457,16 @@ export const customGetAiRequestStatuses = (
|
||||
export const customSuspendAiRequest = (aiRequestId: string): AiRequest => {
|
||||
const existing = localAiRequestsCache[aiRequestId];
|
||||
if (existing) {
|
||||
existing.status = 'suspended';
|
||||
existing.updatedAt = new Date().toISOString();
|
||||
// Replace the cached request rather than mutating it in place: a model turn
|
||||
// may be holding a reference to it and will write its own copy back.
|
||||
const suspended: AiRequest = {
|
||||
...existing,
|
||||
status: 'suspended',
|
||||
updatedAt: new Date().toISOString(),
|
||||
};
|
||||
localAiRequestsCache[aiRequestId] = suspended;
|
||||
saveLocalAiRequests();
|
||||
return existing;
|
||||
return suspended;
|
||||
}
|
||||
return customGetAiRequest(aiRequestId);
|
||||
};
|
||||
@@ -1857,10 +2564,23 @@ Return your response STRICTLY as a JSON object with this format:
|
||||
.trim();
|
||||
const parsed: AiRequestSuggestions = JSON.parse(clean);
|
||||
|
||||
if (req.output && req.output.length > 0) {
|
||||
const lastMsg = req.output[req.output.length - 1];
|
||||
// Re-read the cache: a model turn may have written the request back while
|
||||
// the suggestion model was answering.
|
||||
const latest = localAiRequestsCache[aiRequestId] || req;
|
||||
if (latest.output && latest.output.length > 0) {
|
||||
const lastMsgIndex = latest.output.length - 1;
|
||||
const lastMsg = latest.output[lastMsgIndex];
|
||||
if (lastMsg.type === 'message' && lastMsg.role === 'assistant') {
|
||||
lastMsg.suggestions = parsed;
|
||||
// Replace the message and the request rather than mutating in place: a
|
||||
// model turn holding this request will write its own copy back, and an
|
||||
// in-place edit would be silently discarded by that write.
|
||||
const updatedOutput = [...latest.output];
|
||||
updatedOutput[lastMsgIndex] = { ...lastMsg, suggestions: parsed };
|
||||
localAiRequestsCache[aiRequestId] = {
|
||||
...latest,
|
||||
output: updatedOutput,
|
||||
updatedAt: new Date().toISOString(),
|
||||
};
|
||||
}
|
||||
}
|
||||
saveLocalAiRequests();
|
||||
|
||||
@@ -14,6 +14,7 @@ import {
|
||||
LOCAL_BYOK_USER_ID,
|
||||
_resetCustomAiClientForTesting,
|
||||
customCreateAiRequest,
|
||||
customAddMessageToAiRequest,
|
||||
customGetAiRequest,
|
||||
customGetAiRequests,
|
||||
customGetAiRequestStatuses,
|
||||
@@ -476,4 +477,143 @@ describe('CustomAIClient', () => {
|
||||
expect(resourceResult.results).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Concurrent local model turns', () => {
|
||||
/** A completion that answers after `delayMs` with a distinct message. */
|
||||
const answerAfter = (content: string, delayMs: number) => {
|
||||
axios.post.mockImplementationOnce(
|
||||
() =>
|
||||
new Promise(resolve =>
|
||||
setTimeout(
|
||||
() =>
|
||||
resolve({
|
||||
status: 200,
|
||||
data: {
|
||||
choices: [{ message: { role: 'assistant', content } }],
|
||||
},
|
||||
}),
|
||||
delayMs
|
||||
)
|
||||
)
|
||||
);
|
||||
};
|
||||
|
||||
const createRequest = async (userRequest: string) => {
|
||||
answerAfter('ok', 0);
|
||||
return customCreateAiRequest({
|
||||
userRequest,
|
||||
gameProjectJson: null,
|
||||
projectSpecificExtensionsSummaryJson: null,
|
||||
mode: 'agent',
|
||||
aiConfiguration: { presetId: 'default' },
|
||||
gameId: null,
|
||||
});
|
||||
};
|
||||
|
||||
it('serializes two concurrent turns on one request without losing a write', async () => {
|
||||
const aiRequest = await createRequest('Start');
|
||||
|
||||
// The first turn is slow; the second starts while it is still answering.
|
||||
answerAfter('first answer', 20);
|
||||
const firstTurn = customAddMessageToAiRequest({
|
||||
aiRequestId: aiRequest.id,
|
||||
userMessage: '',
|
||||
functionCallOutputs: [
|
||||
{ type: 'function_call_output', call_id: 'call-1', output: '{}' },
|
||||
],
|
||||
});
|
||||
answerAfter('second answer', 0);
|
||||
const secondTurn = customAddMessageToAiRequest({
|
||||
aiRequestId: aiRequest.id,
|
||||
userMessage: '',
|
||||
functionCallOutputs: [
|
||||
{ type: 'function_call_output', call_id: 'call-2', output: '{}' },
|
||||
],
|
||||
});
|
||||
|
||||
const [firstResult, secondResult] = await Promise.all([
|
||||
firstTurn,
|
||||
secondTurn,
|
||||
]);
|
||||
|
||||
const finalRequest = await customGetAiRequest(aiRequest.id);
|
||||
const assistantTexts = (finalRequest.output || [])
|
||||
.filter(
|
||||
message => message.type === 'message' && message.role === 'assistant'
|
||||
)
|
||||
.map(message => (message: any).text || '');
|
||||
|
||||
// Both answers are present: the second turn did not overwrite the first.
|
||||
expect(assistantTexts).toContain('first answer');
|
||||
expect(assistantTexts).toContain('second answer');
|
||||
// Both tool results are present too.
|
||||
const outputCallIds = (finalRequest.output || [])
|
||||
.filter(message => message.type === 'function_call_output')
|
||||
.map(message => (message: any).call_id);
|
||||
expect(outputCallIds).toEqual(
|
||||
expect.arrayContaining(['call-1', 'call-2'])
|
||||
);
|
||||
// Each turn's own return value carries its own answer.
|
||||
expect(
|
||||
firstResult.output ? firstResult.output.length : 0
|
||||
).toBeGreaterThan(0);
|
||||
expect(
|
||||
secondResult.output ? secondResult.output.length : 0
|
||||
).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('does not block concurrent turns on different requests', async () => {
|
||||
const firstRequest = await createRequest('One');
|
||||
const secondRequest = await createRequest('Two');
|
||||
|
||||
answerAfter('slow answer', 20);
|
||||
const slowTurn = customAddMessageToAiRequest({
|
||||
aiRequestId: firstRequest.id,
|
||||
userMessage: '',
|
||||
functionCallOutputs: [],
|
||||
});
|
||||
answerAfter('fast answer', 0);
|
||||
const fastTurn = customAddMessageToAiRequest({
|
||||
aiRequestId: secondRequest.id,
|
||||
userMessage: '',
|
||||
functionCallOutputs: [],
|
||||
});
|
||||
|
||||
// The fast one resolves while the slow one is still waiting.
|
||||
await fastTurn;
|
||||
await slowTurn;
|
||||
|
||||
const first = await customGetAiRequest(firstRequest.id);
|
||||
const second = await customGetAiRequest(secondRequest.id);
|
||||
expect(JSON.stringify(first.output || '').includes('slow answer')).toBe(
|
||||
true
|
||||
);
|
||||
expect(JSON.stringify(second.output || '').includes('fast answer')).toBe(
|
||||
true
|
||||
);
|
||||
});
|
||||
|
||||
it('keeps a suspension that lands during an in-flight turn', async () => {
|
||||
const aiRequest = await createRequest('Suspend me');
|
||||
|
||||
answerAfter('answer after suspend', 20);
|
||||
const turn = customAddMessageToAiRequest({
|
||||
aiRequestId: aiRequest.id,
|
||||
userMessage: '',
|
||||
functionCallOutputs: [],
|
||||
});
|
||||
|
||||
// Suspend while the model is answering.
|
||||
customSuspendAiRequest(aiRequest.id);
|
||||
await turn;
|
||||
|
||||
const finalRequest = await customGetAiRequest(aiRequest.id);
|
||||
// The turn write-back must not resume a suspended request.
|
||||
expect(finalRequest.status).toBe('suspended');
|
||||
// The answer the model produced is still readable.
|
||||
expect(JSON.stringify(finalRequest.output || '')).toContain(
|
||||
'answer after suspend'
|
||||
);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -77,6 +77,7 @@ import { listAllExamples } from '../Utils/GDevelopServices/Example';
|
||||
import UrlStorageProvider from '../ProjectsStorage/UrlStorageProvider';
|
||||
import { prepareAiUserContent } from './PrepareAiUserContent';
|
||||
import { AiRequestContext } from './AiRequestContext';
|
||||
import { useStudioRuntime } from './Studio/UseStudioRuntime';
|
||||
import { getAiConfigurationPresetsWithAvailability } from './AiConfiguration';
|
||||
import {
|
||||
setEditorHotReloadNeeded,
|
||||
@@ -546,6 +547,7 @@ export const AskAiEditor: React.ComponentType<Props> = React.memo<Props>(
|
||||
requestEditApproval,
|
||||
resolveEditApproval,
|
||||
setIsFetchingSuggestions,
|
||||
activateSubAgent,
|
||||
} = React.useContext(AiRequestContext);
|
||||
const {
|
||||
getEditorFunctionCallResults,
|
||||
@@ -1103,6 +1105,7 @@ export const AskAiEditor: React.ComponentType<Props> = React.memo<Props>(
|
||||
const {
|
||||
onProcessFunctionCalls,
|
||||
clearApprovedEditBatches,
|
||||
enqueueRequestWrite,
|
||||
} = useProcessFunctionCalls({
|
||||
project,
|
||||
resourceManagementProps,
|
||||
@@ -1129,6 +1132,21 @@ export const AskAiEditor: React.ComponentType<Props> = React.memo<Props>(
|
||||
getIsAutoEditEnabled,
|
||||
suspendAiRequest,
|
||||
requestEditApproval,
|
||||
activateSubAgent,
|
||||
updateAiRequest,
|
||||
isSendingAiRequest,
|
||||
});
|
||||
|
||||
// Closes the loop for locally spawned studio sub-agents: waits for a child
|
||||
// to finish, reports it back to its parent, and updates the plan. A no-op
|
||||
// when no sub-agent is active, so the hosted path is untouched.
|
||||
useStudioRuntime({
|
||||
aiRequests,
|
||||
activeSubAgents,
|
||||
getEditorFunctionCallResults,
|
||||
updateAiRequest,
|
||||
onSendEditorFunctionCallResults,
|
||||
enqueueRequestWrite,
|
||||
});
|
||||
|
||||
// Wrap onProcessFunctionCalls to bind the selected AI request for the chat UI.
|
||||
|
||||
@@ -67,6 +67,11 @@ const gd: libGDevelop = global.gd;
|
||||
const alwaysAutoEditEnabled = () => true;
|
||||
const noOpSuspendAiRequest = async () => {};
|
||||
const alwaysApproveEdit = async () => true;
|
||||
// The stand-alone form has no studio runtime: it never spawns a sub-agent, and
|
||||
// the shared request-store mutators are not mounted here.
|
||||
const noOpActivateSubAgent = () => {};
|
||||
const noOpUpdateAiRequest = () => {};
|
||||
const neverSendingAiRequest = () => false;
|
||||
|
||||
type Props = {|
|
||||
project: ?gdProject,
|
||||
@@ -649,6 +654,10 @@ export const AskAiStandAloneForm = ({
|
||||
getIsAutoEditEnabled: alwaysAutoEditEnabled,
|
||||
suspendAiRequest: noOpSuspendAiRequest,
|
||||
requestEditApproval: alwaysApproveEdit,
|
||||
// The form never spawns a sub-agent and has no request store to write to.
|
||||
activateSubAgent: noOpActivateSubAgent,
|
||||
updateAiRequest: noOpUpdateAiRequest,
|
||||
isSendingAiRequest: neverSendingAiRequest,
|
||||
});
|
||||
|
||||
const onProcessFormFunctionCalls = React.useCallback(
|
||||
|
||||
@@ -0,0 +1,140 @@
|
||||
// @flow
|
||||
|
||||
import { type AiRequest } from '../../Utils/GDevelopServices/Generation';
|
||||
import { type EditorFunctionCallResult } from '../../EditorFunctions';
|
||||
import {
|
||||
getAllSubAgentFunctionCalls,
|
||||
getFunctionCallsToProcess,
|
||||
getPendingSubAgentFunctionCalls,
|
||||
} from '../AiRequestUtils';
|
||||
import { getStudioRole, type StudioRoleId } from './Roles';
|
||||
import { isSpawnAgentCall, parseSpawnAgentArgs } from './SpawnSubAgents';
|
||||
|
||||
/**
|
||||
* The role a studio sub-agent was spawned with, read back from its parent's
|
||||
* `spawn_agent` call. Null for a hosted sub-agent (`run_edit_agent` /
|
||||
* `run_explorer_agent`) or when the launch call cannot be resolved.
|
||||
*/
|
||||
export const getSubAgentRoleId = ({
|
||||
aiRequest,
|
||||
aiRequests,
|
||||
}: {|
|
||||
aiRequest: AiRequest,
|
||||
aiRequests: { [string]: AiRequest },
|
||||
|}): StudioRoleId | null => {
|
||||
if (!aiRequest.parentAiRequestId) return null;
|
||||
const parentRequest = aiRequests[aiRequest.parentAiRequestId] || null;
|
||||
if (!parentRequest) return null;
|
||||
const launchingCall = getAllSubAgentFunctionCalls({
|
||||
aiRequest: parentRequest,
|
||||
}).find(functionCall => functionCall.subAgentAiRequestId === aiRequest.id);
|
||||
if (!launchingCall || !isSpawnAgentCall(launchingCall)) return null;
|
||||
const parsedArgs = parseSpawnAgentArgs(launchingCall);
|
||||
return parsedArgs ? parsedArgs.role : null;
|
||||
};
|
||||
|
||||
/** How many model turns a sub-agent has taken: one assistant message each. */
|
||||
export const countAssistantTurns = (aiRequest: AiRequest): number =>
|
||||
(aiRequest.output || []).filter(
|
||||
message => message.type === 'message' && message.role === 'assistant'
|
||||
).length;
|
||||
|
||||
/**
|
||||
* True when a studio sub-agent has used up its role's turn budget.
|
||||
*
|
||||
* The dispatcher stops processing its remaining calls at that point and the
|
||||
* studio runtime finalizes it, so a runaway sub-agent cannot run forever.
|
||||
*/
|
||||
export const isSubAgentAtTurnCap = ({
|
||||
aiRequest,
|
||||
aiRequests,
|
||||
}: {|
|
||||
aiRequest: AiRequest,
|
||||
aiRequests: { [string]: AiRequest },
|
||||
|}): boolean => {
|
||||
const roleId = getSubAgentRoleId({ aiRequest, aiRequests });
|
||||
if (!roleId) return false;
|
||||
return countAssistantTurns(aiRequest) >= getStudioRole(roleId).maxTurns;
|
||||
};
|
||||
|
||||
/**
|
||||
* A sub-agent is finished when it has nothing left to do locally: it is not
|
||||
* working, it has no unexecuted function calls, and it has no pending sub-agent
|
||||
* of its own.
|
||||
*
|
||||
* A hosted sub-agent (`run_edit_agent`) is polled until the server retires it;
|
||||
* this only decides when the *studio* may write its report back.
|
||||
*/
|
||||
export const isSubAgentFinished = ({
|
||||
subAgentRequest,
|
||||
editorFunctionCallResults,
|
||||
}: {|
|
||||
subAgentRequest: AiRequest,
|
||||
editorFunctionCallResults: Array<EditorFunctionCallResult> | null,
|
||||
|}): boolean => {
|
||||
if (subAgentRequest.status !== 'ready') return false;
|
||||
if (
|
||||
getFunctionCallsToProcess({
|
||||
aiRequest: subAgentRequest,
|
||||
editorFunctionCallResults,
|
||||
}).length > 0
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
getPendingSubAgentFunctionCalls({ aiRequest: subAgentRequest }).length > 0
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
editorFunctionCallResults &&
|
||||
editorFunctionCallResults.some(result => result.status === 'working')
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
};
|
||||
|
||||
/**
|
||||
* The text written into the parent's `function_call_output`: the sub-agent's
|
||||
* last assistant message, prefixed with the role's display name.
|
||||
*
|
||||
* The message's text lives in a `content` entry whose type is `'output_text'`
|
||||
* for a message the server produced, and `'text'` for one the local BYOK path
|
||||
* parsed (`parseAssistantMessage` emits `'text'`); both are read, and the
|
||||
* message's own `text` field is the fallback, so a report is never lost to the
|
||||
* difference.
|
||||
*/
|
||||
export const buildSubAgentReport = (subAgentRequest: AiRequest): string => {
|
||||
const output = subAgentRequest.output || [];
|
||||
for (let i = output.length - 1; i >= 0; i--) {
|
||||
const message = output[i];
|
||||
if (message.type !== 'message' || message.role !== 'assistant') continue;
|
||||
const textContent = message.content.find(
|
||||
content => content.type === 'output_text' || content.type === 'text'
|
||||
);
|
||||
const textFromContent = textContent ? (textContent: any).text : null;
|
||||
if (typeof textFromContent === 'string' && textFromContent) {
|
||||
return textFromContent;
|
||||
}
|
||||
const messageText = (message: any).text;
|
||||
if (typeof messageText === 'string' && messageText) return messageText;
|
||||
return '(no report)';
|
||||
}
|
||||
return '(no report)';
|
||||
};
|
||||
|
||||
/**
|
||||
* The label shown for the sub-agent's report. Falls back to the generic name for
|
||||
* a hosted sub-agent, whose role this fork does not know.
|
||||
*/
|
||||
export const getSubAgentReportLabel = ({
|
||||
aiRequest,
|
||||
aiRequests,
|
||||
}: {|
|
||||
aiRequest: AiRequest,
|
||||
aiRequests: { [string]: AiRequest },
|
||||
|}): string => {
|
||||
const roleId = getSubAgentRoleId({ aiRequest, aiRequests });
|
||||
return roleId ? getStudioRole(roleId).displayName : 'Agent';
|
||||
};
|
||||
@@ -0,0 +1,483 @@
|
||||
// @flow
|
||||
import {
|
||||
isSubAgentFinished,
|
||||
buildSubAgentReport,
|
||||
getSubAgentReportLabel,
|
||||
countAssistantTurns,
|
||||
isSubAgentAtTurnCap,
|
||||
} from './FinalizeSubAgents';
|
||||
import { buildPlanStatusUpdateOutput } from './UseStudioRuntime';
|
||||
import { buildPlanOutput } from './PlanStore';
|
||||
|
||||
const assistantMessage = (text: string, functionCalls?: Array<any>): any => ({
|
||||
type: 'message',
|
||||
status: 'completed',
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{ type: 'output_text', status: 'completed', text, annotations: [] },
|
||||
...(functionCalls || []),
|
||||
],
|
||||
messageId: `msg-asst-${text}`,
|
||||
});
|
||||
|
||||
const spawnCall = (
|
||||
callId: string,
|
||||
subAgentId: string | null,
|
||||
relatedTaskId: string | null
|
||||
): any => ({
|
||||
type: 'function_call',
|
||||
status: 'completed',
|
||||
call_id: callId,
|
||||
name: 'spawn_agent',
|
||||
arguments: JSON.stringify({
|
||||
role: 'developer',
|
||||
short_title: 'Build the grid',
|
||||
task: 'Build it.',
|
||||
related_task_id: relatedTaskId === null ? undefined : relatedTaskId,
|
||||
}),
|
||||
...(subAgentId ? { subAgentAiRequestId: subAgentId } : {}),
|
||||
});
|
||||
|
||||
const makeSubAgent = (overrides: any): any => ({
|
||||
id: 'child-1',
|
||||
createdAt: '2026-01-01T00:00:00.000Z',
|
||||
updatedAt: '2026-01-01T00:00:00.000Z',
|
||||
userId: 'local-byok-user',
|
||||
status: 'ready',
|
||||
error: null,
|
||||
output: [assistantMessage('Done.')],
|
||||
...overrides,
|
||||
});
|
||||
|
||||
describe('FinalizeSubAgents', () => {
|
||||
describe('isSubAgentFinished', () => {
|
||||
it('is false while the sub-agent is not ready', () => {
|
||||
expect(
|
||||
isSubAgentFinished({
|
||||
subAgentRequest: makeSubAgent({ status: 'working' }),
|
||||
editorFunctionCallResults: null,
|
||||
})
|
||||
).toBe(false);
|
||||
expect(
|
||||
isSubAgentFinished({
|
||||
subAgentRequest: makeSubAgent({ status: 'suspended' }),
|
||||
editorFunctionCallResults: null,
|
||||
})
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('is false while a function call is still to be executed', () => {
|
||||
const subAgentRequest = makeSubAgent({
|
||||
output: [
|
||||
assistantMessage('Creating the scene.', [
|
||||
{
|
||||
type: 'function_call',
|
||||
status: 'completed',
|
||||
call_id: 'call-scene',
|
||||
name: 'create_scene',
|
||||
arguments: '{"scene_name":"Town"}',
|
||||
},
|
||||
]),
|
||||
],
|
||||
});
|
||||
expect(
|
||||
isSubAgentFinished({ subAgentRequest, editorFunctionCallResults: null })
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('is false while a sub-agent of its own is pending', () => {
|
||||
const subAgentRequest = makeSubAgent({
|
||||
output: [
|
||||
assistantMessage('Delegating.', [
|
||||
spawnCall('call-sub', 'child-2', null),
|
||||
]),
|
||||
],
|
||||
});
|
||||
expect(
|
||||
isSubAgentFinished({ subAgentRequest, editorFunctionCallResults: null })
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('is false while a result is still marked working', () => {
|
||||
const subAgentRequest = makeSubAgent({});
|
||||
expect(
|
||||
isSubAgentFinished({
|
||||
subAgentRequest,
|
||||
editorFunctionCallResults: [
|
||||
{ status: 'working', call_id: 'call-x' },
|
||||
{
|
||||
status: 'finished',
|
||||
call_id: 'call-y',
|
||||
success: true,
|
||||
output: {},
|
||||
},
|
||||
],
|
||||
})
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('is true for a clean ready sub-agent with no pending work', () => {
|
||||
expect(
|
||||
isSubAgentFinished({
|
||||
subAgentRequest: makeSubAgent({}),
|
||||
editorFunctionCallResults: null,
|
||||
})
|
||||
).toBe(true);
|
||||
expect(
|
||||
isSubAgentFinished({
|
||||
subAgentRequest: makeSubAgent({}),
|
||||
editorFunctionCallResults: [
|
||||
{
|
||||
status: 'finished',
|
||||
call_id: 'call-y',
|
||||
success: true,
|
||||
output: {},
|
||||
},
|
||||
],
|
||||
})
|
||||
).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildSubAgentReport', () => {
|
||||
it('returns the last assistant message text', () => {
|
||||
const subAgentRequest = makeSubAgent({
|
||||
output: [
|
||||
assistantMessage('Starting.'),
|
||||
assistantMessage('Finished: created the scene Town.'),
|
||||
],
|
||||
});
|
||||
expect(buildSubAgentReport(subAgentRequest)).toBe(
|
||||
'Finished: created the scene Town.'
|
||||
);
|
||||
});
|
||||
|
||||
it('reads the local BYOK message shape, whose text entry is `text`', () => {
|
||||
// `parseAssistantMessage` (the local path) emits `type: 'text'` where the
|
||||
// server emits `type: 'output_text'`: reading only the latter reported
|
||||
// "(no report)" for every local sub-agent.
|
||||
const subAgentRequest = makeSubAgent({
|
||||
output: [
|
||||
{
|
||||
type: 'message',
|
||||
status: 'completed',
|
||||
role: 'assistant',
|
||||
text: 'Done: created the Town scene.',
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
status: 'completed',
|
||||
text: 'Done: created the Town scene.',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(buildSubAgentReport(subAgentRequest)).toBe(
|
||||
'Done: created the Town scene.'
|
||||
);
|
||||
});
|
||||
|
||||
it('falls back to the message text when the content entry has none', () => {
|
||||
const subAgentRequest = makeSubAgent({
|
||||
output: [
|
||||
{
|
||||
type: 'message',
|
||||
status: 'completed',
|
||||
role: 'assistant',
|
||||
text: 'Only the message text.',
|
||||
content: [],
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(buildSubAgentReport(subAgentRequest)).toBe(
|
||||
'Only the message text.'
|
||||
);
|
||||
});
|
||||
|
||||
it('returns a placeholder when the last message has no text', () => {
|
||||
const subAgentRequest = makeSubAgent({
|
||||
output: [
|
||||
{
|
||||
type: 'message',
|
||||
status: 'completed',
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{
|
||||
type: 'function_call',
|
||||
status: 'completed',
|
||||
call_id: 'call-1',
|
||||
name: 'create_scene',
|
||||
arguments: '{}',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(buildSubAgentReport(subAgentRequest)).toBe('(no report)');
|
||||
});
|
||||
|
||||
it('returns a placeholder for a request with no assistant message', () => {
|
||||
expect(buildSubAgentReport(makeSubAgent({ output: [] }))).toBe(
|
||||
'(no report)'
|
||||
);
|
||||
expect(buildSubAgentReport(makeSubAgent({ output: undefined }))).toBe(
|
||||
'(no report)'
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('getSubAgentReportLabel', () => {
|
||||
it('uses the role display name of the spawn call', () => {
|
||||
const parent: any = makeSubAgent({
|
||||
id: 'parent-1',
|
||||
output: [
|
||||
assistantMessage('Delegating.', [
|
||||
spawnCall('call-1', 'child-1', null),
|
||||
]),
|
||||
],
|
||||
});
|
||||
const child = makeSubAgent({ parentAiRequestId: 'parent-1' });
|
||||
|
||||
expect(
|
||||
getSubAgentReportLabel({
|
||||
aiRequest: child,
|
||||
aiRequests: { 'parent-1': parent },
|
||||
})
|
||||
).toBe('Developer');
|
||||
});
|
||||
|
||||
it('falls back for a hosted sub-agent and for a missing parent', () => {
|
||||
const hostedParent: any = makeSubAgent({
|
||||
id: 'parent-1',
|
||||
output: [
|
||||
assistantMessage('Delegating.', [
|
||||
{
|
||||
type: 'function_call',
|
||||
status: 'completed',
|
||||
call_id: 'call-1',
|
||||
name: 'run_edit_agent',
|
||||
arguments: '{"short_title":"Editing"}',
|
||||
subAgentAiRequestId: 'child-1',
|
||||
},
|
||||
]),
|
||||
],
|
||||
});
|
||||
expect(
|
||||
getSubAgentReportLabel({
|
||||
aiRequest: makeSubAgent({ parentAiRequestId: 'parent-1' }),
|
||||
aiRequests: { 'parent-1': hostedParent },
|
||||
})
|
||||
).toBe('Agent');
|
||||
expect(
|
||||
getSubAgentReportLabel({
|
||||
aiRequest: makeSubAgent({ parentAiRequestId: 'ghost' }),
|
||||
aiRequests: {},
|
||||
})
|
||||
).toBe('Agent');
|
||||
});
|
||||
});
|
||||
|
||||
describe('turn cap', () => {
|
||||
const parentWithRole = (role: string, subAgentId: string): any =>
|
||||
makeSubAgent({
|
||||
id: 'parent-1',
|
||||
output: [
|
||||
assistantMessage('Delegating.', [
|
||||
{
|
||||
type: 'function_call',
|
||||
status: 'completed',
|
||||
call_id: 'call-1',
|
||||
name: 'spawn_agent',
|
||||
arguments: JSON.stringify({
|
||||
role,
|
||||
short_title: 'Work',
|
||||
task: 'Work.',
|
||||
}),
|
||||
subAgentAiRequestId: subAgentId,
|
||||
},
|
||||
]),
|
||||
],
|
||||
});
|
||||
|
||||
it('counts assistant turns', () => {
|
||||
expect(countAssistantTurns(makeSubAgent({}))).toBe(1);
|
||||
expect(
|
||||
countAssistantTurns(
|
||||
makeSubAgent({
|
||||
output: [assistantMessage('a'), assistantMessage('b')],
|
||||
})
|
||||
)
|
||||
).toBe(2);
|
||||
});
|
||||
|
||||
it('is false below the role cap and true at it', () => {
|
||||
const parent = parentWithRole('tester', 'child-1');
|
||||
const aiRequests = { 'parent-1': parent };
|
||||
|
||||
// The tester's cap is 30.
|
||||
const belowCap = makeSubAgent({
|
||||
parentAiRequestId: 'parent-1',
|
||||
output: Array.from({ length: 29 }, (_, i) => assistantMessage(`t${i}`)),
|
||||
});
|
||||
expect(isSubAgentAtTurnCap({ aiRequest: belowCap, aiRequests })).toBe(
|
||||
false
|
||||
);
|
||||
|
||||
const atCap = makeSubAgent({
|
||||
parentAiRequestId: 'parent-1',
|
||||
output: Array.from({ length: 30 }, (_, i) => assistantMessage(`t${i}`)),
|
||||
});
|
||||
expect(isSubAgentAtTurnCap({ aiRequest: atCap, aiRequests })).toBe(true);
|
||||
});
|
||||
|
||||
it('is false for a sub-agent whose role cannot be resolved', () => {
|
||||
expect(
|
||||
isSubAgentAtTurnCap({
|
||||
aiRequest: makeSubAgent({}),
|
||||
aiRequests: {},
|
||||
})
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildPlanStatusUpdateOutput', () => {
|
||||
const withPlan = (tasks: Array<any>): any => {
|
||||
const parent: any = makeSubAgent({
|
||||
id: 'parent-1',
|
||||
output: [
|
||||
{
|
||||
type: 'function_call_output',
|
||||
call_id: 'plan-call',
|
||||
output: JSON.stringify(buildPlanOutput(tasks)),
|
||||
},
|
||||
assistantMessage('Delegating.', [
|
||||
spawnCall('call-1', 'child-1', 'grid'),
|
||||
]),
|
||||
],
|
||||
});
|
||||
return parent;
|
||||
};
|
||||
|
||||
it('flips the related task to done and the next ready task to in_progress', () => {
|
||||
const parent = withPlan([
|
||||
{
|
||||
id: 'grid',
|
||||
title: 'Build the grid',
|
||||
description: 'A 20x20 grid.',
|
||||
status: 'pending',
|
||||
dependsOn: [],
|
||||
},
|
||||
{
|
||||
id: 'units',
|
||||
title: 'Add units',
|
||||
description: 'Units walk the grid.',
|
||||
status: 'pending',
|
||||
dependsOn: ['grid'],
|
||||
},
|
||||
]);
|
||||
|
||||
const updatedOutput = buildPlanStatusUpdateOutput(parent, 'call-1');
|
||||
expect(updatedOutput).not.toBeNull();
|
||||
const planMessage: any = (updatedOutput || []).find(
|
||||
(message: any) =>
|
||||
message.type === 'function_call_output' &&
|
||||
message.call_id === 'plan-call'
|
||||
);
|
||||
expect(planMessage).toBeDefined();
|
||||
const plan = JSON.parse(planMessage.output).plan;
|
||||
expect(plan.tasks[0].status).toBe('done');
|
||||
expect(plan.tasks[1].status).toBe('in_progress');
|
||||
// The transcript is not grown: the plan message is rewritten.
|
||||
expect(updatedOutput).toHaveLength(parent.output.length);
|
||||
});
|
||||
|
||||
it('keeps a sibling field the writer does not model', () => {
|
||||
const parent = withPlan([
|
||||
{
|
||||
id: 'grid',
|
||||
title: 'Build the grid',
|
||||
description: 'A 20x20 grid.',
|
||||
status: 'pending',
|
||||
dependsOn: [],
|
||||
agentCallId: 'call-1',
|
||||
},
|
||||
{
|
||||
id: 'units',
|
||||
title: 'Add units',
|
||||
description: 'Units walk the grid.',
|
||||
status: 'pending',
|
||||
dependsOn: ['grid'],
|
||||
agentCallId: 'call-2',
|
||||
},
|
||||
]);
|
||||
|
||||
const plan = JSON.parse(
|
||||
((buildPlanStatusUpdateOutput(parent, 'call-1') || [])[0]: any).output
|
||||
).plan;
|
||||
expect(plan.tasks[0].agentCallId).toBe('call-1');
|
||||
expect(plan.tasks[1].agentCallId).toBe('call-2');
|
||||
expect(plan.tasks[1].description).toBe('Units walk the grid.');
|
||||
});
|
||||
|
||||
it('returns null when there is no plan, no related task id, or a missing task', () => {
|
||||
// No plan at all.
|
||||
expect(
|
||||
buildPlanStatusUpdateOutput(
|
||||
makeSubAgent({ id: 'parent-1', output: [assistantMessage('Hi.')] }),
|
||||
'call-1'
|
||||
)
|
||||
).toBeNull();
|
||||
|
||||
// A spawn call with no related_task_id.
|
||||
const noRelated: any = makeSubAgent({
|
||||
id: 'parent-1',
|
||||
output: [
|
||||
{
|
||||
type: 'function_call_output',
|
||||
call_id: 'plan-call',
|
||||
output: JSON.stringify(
|
||||
buildPlanOutput([
|
||||
{
|
||||
id: 'grid',
|
||||
title: 'T',
|
||||
description: 'D',
|
||||
status: 'pending',
|
||||
dependsOn: [],
|
||||
},
|
||||
])
|
||||
),
|
||||
},
|
||||
assistantMessage('Delegating.', [
|
||||
spawnCall('call-1', 'child-1', null),
|
||||
]),
|
||||
],
|
||||
});
|
||||
expect(buildPlanStatusUpdateOutput(noRelated, 'call-1')).toBeNull();
|
||||
|
||||
// A related_task_id naming a task that is not in the plan.
|
||||
const missingTask = withPlan([
|
||||
{
|
||||
id: 'other',
|
||||
title: 'T',
|
||||
description: 'D',
|
||||
status: 'pending',
|
||||
dependsOn: [],
|
||||
},
|
||||
]);
|
||||
expect(buildPlanStatusUpdateOutput(missingTask, 'call-1')).toBeNull();
|
||||
|
||||
// A call id that is not a spawn call.
|
||||
const parent = withPlan([
|
||||
{
|
||||
id: 'grid',
|
||||
title: 'T',
|
||||
description: 'D',
|
||||
status: 'pending',
|
||||
dependsOn: [],
|
||||
},
|
||||
]);
|
||||
expect(buildPlanStatusUpdateOutput(parent, 'unknown-call')).toBeNull();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,213 @@
|
||||
// @flow
|
||||
|
||||
import jsSHA from '../../Utils/Sha256';
|
||||
|
||||
/**
|
||||
* The local loop guard: a circuit breaker for a BYOK agent.
|
||||
*
|
||||
* Ported from munder-difflin's `src/main/breaker.ts`, narrowed to what this
|
||||
* editor can actually observe. Under BYOK there is no server, so nothing counts
|
||||
* a runaway local agent: the hosted `repeated-tool-call-loop` error can never
|
||||
* fire, `MAX_AI_REQUEST_RETRIES_IN_A_ROW` is a UI gate the API enforces
|
||||
* (`AiRequestUtils.js`), and a model looping on *fresh* call ids is otherwise
|
||||
* unbounded.
|
||||
*
|
||||
* Three trips, all from munder-difflin:
|
||||
* - `repeatedToolLimit` consecutive identical tool calls
|
||||
* (`'looping: N identical calls to <name>'`);
|
||||
* - `errorStormLimit` consecutive failed calls
|
||||
* (`'error storm: N consecutive failures'`);
|
||||
* - `maxTurnsWithoutUserInput` assistant turns, cleared only by `reset`
|
||||
* (`'turn limit: N turns without user input'`) - the trip with no
|
||||
* munder-difflin analogue that this editor needs.
|
||||
*
|
||||
* Escalation is munder-difflin's ladder (`breaker.ts` `tick`): one level per
|
||||
* `evaluate`, capped at `constrained` (there is no `hardStop` here - the guard
|
||||
* never kills a request, it steers and constrains). A non-tripping `evaluate`
|
||||
* de-escalates one level. `action` is non-`'none'` only when the level escalated,
|
||||
* so a durable steer is not re-sent every tick.
|
||||
*
|
||||
* Deliberately NOT ported: the cost and token-velocity trips (the BYOK path has
|
||||
* no usage feed), the `hardStop` level, and the `PreCompact` exemption (there is
|
||||
* no auto-compaction here).
|
||||
*/
|
||||
|
||||
export type LoopGuardLevel = 'healthy' | 'steering' | 'constrained' | 'stopped';
|
||||
export type LoopGuardAction = 'none' | 'steer' | 'constrain' | 'stop';
|
||||
|
||||
export type LoopGuardDecision = {|
|
||||
level: LoopGuardLevel,
|
||||
reason: string,
|
||||
action: LoopGuardAction,
|
||||
/** True when the level changed since the previous `evaluate` (either way). */
|
||||
changed: boolean,
|
||||
|};
|
||||
|
||||
export const DEFAULT_LOOP_GUARD_CONFIG = {
|
||||
enabled: true,
|
||||
/** Consecutive identical tool calls (same name AND same arguments) allowed. */
|
||||
repeatedToolLimit: 8,
|
||||
/** Consecutive failed tool calls allowed. */
|
||||
errorStormLimit: 5,
|
||||
/**
|
||||
* Assistant turns allowed without a user message in between. Generous on
|
||||
* purpose: a legitimate long task (a `run_script` doing hundreds of calls, a
|
||||
* gameplay test) is not a loop.
|
||||
*/
|
||||
maxTurnsWithoutUserInput: 60,
|
||||
};
|
||||
|
||||
export type LoopGuardConfig = {
|
||||
enabled: boolean,
|
||||
repeatedToolLimit: number,
|
||||
errorStormLimit: number,
|
||||
maxTurnsWithoutUserInput: number,
|
||||
};
|
||||
|
||||
const LEVELS: Array<LoopGuardLevel> = [
|
||||
'healthy',
|
||||
'steering',
|
||||
'constrained',
|
||||
'stopped',
|
||||
];
|
||||
|
||||
const rank = (level: LoopGuardLevel): number => LEVELS.indexOf(level);
|
||||
|
||||
const actionFor = (level: LoopGuardLevel): LoopGuardAction => {
|
||||
if (level === 'steering') return 'steer';
|
||||
if (level === 'constrained') return 'constrain';
|
||||
if (level === 'stopped') return 'stop';
|
||||
return 'none';
|
||||
};
|
||||
|
||||
/**
|
||||
* The identity of a tool call: its name and a hash of its **whole** arguments
|
||||
* string.
|
||||
*
|
||||
* Hashing the whole string is the point. munder-difflin's #377 records that its
|
||||
* old `input.slice(0, 200)` collided on Bash commands sharing a long identical
|
||||
* preamble - nine *different* measurements in a row read as nine identical calls
|
||||
* and constrained the agent. Different arguments must produce different keys,
|
||||
* however long their common prefix.
|
||||
*/
|
||||
export const toolCallKey = (name: string, argsJson: string): string => {
|
||||
const shaObj = new jsSHA('SHA-256', 'TEXT', { encoding: 'UTF8' });
|
||||
shaObj.update(argsJson || '');
|
||||
return `${name}:${shaObj.getHash('HEX')}`;
|
||||
};
|
||||
|
||||
export type LoopGuard = {|
|
||||
recordToolCall: (call: {| name: string, arguments: string |}) => void,
|
||||
recordError: () => void,
|
||||
recordTurn: () => void,
|
||||
reset: () => void,
|
||||
evaluate: () => LoopGuardDecision,
|
||||
|};
|
||||
|
||||
export const createLoopGuard = (
|
||||
config?: Partial<LoopGuardConfig>
|
||||
): LoopGuard => {
|
||||
const effectiveConfig: LoopGuardConfig = {
|
||||
...DEFAULT_LOOP_GUARD_CONFIG,
|
||||
...(config || {}),
|
||||
};
|
||||
|
||||
let level: LoopGuardLevel = 'healthy';
|
||||
let reason = '';
|
||||
let repeatKey: string | null = null;
|
||||
let repeatCount = 0;
|
||||
let errorCount = 0;
|
||||
let turnCount = 0;
|
||||
|
||||
const evaluateTrips = (): {| tripping: boolean, reason: string |} => {
|
||||
if (repeatCount >= effectiveConfig.repeatedToolLimit) {
|
||||
return {
|
||||
tripping: true,
|
||||
reason: `looping: ${repeatCount} identical calls to ${
|
||||
repeatKey ? repeatKey.split(':')[0] : 'a tool'
|
||||
}`,
|
||||
};
|
||||
}
|
||||
if (errorCount >= effectiveConfig.errorStormLimit) {
|
||||
return {
|
||||
tripping: true,
|
||||
reason: `error storm: ${errorCount} consecutive failures`,
|
||||
};
|
||||
}
|
||||
if (turnCount >= effectiveConfig.maxTurnsWithoutUserInput) {
|
||||
return {
|
||||
tripping: true,
|
||||
reason: `turn limit: ${turnCount} turns without user input`,
|
||||
};
|
||||
}
|
||||
return { tripping: false, reason: '' };
|
||||
};
|
||||
|
||||
return {
|
||||
recordToolCall: call => {
|
||||
const key = toolCallKey(call.name, call.arguments);
|
||||
if (key === repeatKey) {
|
||||
repeatCount++;
|
||||
} else {
|
||||
repeatKey = key;
|
||||
repeatCount = 1;
|
||||
// A distinct tool call is progress: it clears the error storm
|
||||
// (munder-difflin `breaker.ts` `recordToolUse`).
|
||||
errorCount = 0;
|
||||
}
|
||||
},
|
||||
recordError: () => {
|
||||
errorCount++;
|
||||
},
|
||||
recordTurn: () => {
|
||||
turnCount++;
|
||||
},
|
||||
reset: () => {
|
||||
// A new user input: the counters start over, but the ladder is NOT
|
||||
// jumped back to healthy - it de-escalates one level per `evaluate`, so
|
||||
// the caller is not handed a steer it already acted on, and a user who
|
||||
// speaks during a constrained run sees it recover over the next ticks.
|
||||
repeatKey = null;
|
||||
repeatCount = 0;
|
||||
errorCount = 0;
|
||||
turnCount = 0;
|
||||
},
|
||||
evaluate: () => {
|
||||
if (!effectiveConfig.enabled) {
|
||||
const wasChanged = level !== 'healthy';
|
||||
level = 'healthy';
|
||||
reason = '';
|
||||
return {
|
||||
level,
|
||||
reason,
|
||||
action: 'none',
|
||||
changed: wasChanged,
|
||||
};
|
||||
}
|
||||
|
||||
const trip = evaluateTrips();
|
||||
// The ceiling has no `hardStop` here: the guard steers and constrains,
|
||||
// it never kills a request.
|
||||
const ceiling: LoopGuardLevel = 'constrained';
|
||||
let target: LoopGuardLevel;
|
||||
if (trip.tripping) {
|
||||
target = LEVELS[Math.min(rank(level) + 1, rank(ceiling))];
|
||||
} else {
|
||||
target = LEVELS[Math.max(rank(level) - 1, 0)];
|
||||
}
|
||||
const changed = target !== level;
|
||||
const escalated = rank(target) > rank(level);
|
||||
level = target;
|
||||
reason = trip.tripping ? trip.reason : changed ? 'recovering' : reason;
|
||||
|
||||
return {
|
||||
level,
|
||||
reason,
|
||||
// Only an escalation carries an action, so a durable steer is not
|
||||
// re-sent on every tick.
|
||||
action: escalated ? actionFor(target) : 'none',
|
||||
changed,
|
||||
};
|
||||
},
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,222 @@
|
||||
// @flow
|
||||
import {
|
||||
createLoopGuard,
|
||||
toolCallKey,
|
||||
DEFAULT_LOOP_GUARD_CONFIG,
|
||||
} from './LoopGuard';
|
||||
|
||||
describe('LoopGuard', () => {
|
||||
describe('toolCallKey', () => {
|
||||
it('keys different arguments differently, however long the common prefix', () => {
|
||||
// munder-difflin #377: `input.slice(0, 200)` made nine different
|
||||
// measurements read as nine identical calls. Hashing the whole string is
|
||||
// what fixes it.
|
||||
const preamble = 'x'.repeat(300);
|
||||
const first = toolCallKey('run_script', `${preamble}first-measurement`);
|
||||
const second = toolCallKey('run_script', `${preamble}second-measurement`);
|
||||
expect(first).not.toBe(second);
|
||||
});
|
||||
|
||||
it('keys identical arguments identically, and different tools differently', () => {
|
||||
expect(toolCallKey('create_scene', '{"a":1}')).toBe(
|
||||
toolCallKey('create_scene', '{"a":1}')
|
||||
);
|
||||
expect(toolCallKey('create_scene', '{"a":1}')).not.toBe(
|
||||
toolCallKey('create_scene', '{"a":2}')
|
||||
);
|
||||
expect(toolCallKey('create_scene', '{}')).not.toBe(
|
||||
toolCallKey('put_2d_instances', '{}')
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('repeated tool call trip', () => {
|
||||
const call = (guard: any, args: string) =>
|
||||
guard.recordToolCall({ name: 'create_scene', arguments: args });
|
||||
|
||||
it('trips at the configured limit and names the tool', () => {
|
||||
const guard = createLoopGuard({ repeatedToolLimit: 8 });
|
||||
for (let i = 0; i < 7; i++) call(guard, '{"scene_name":"Town"}');
|
||||
// Seven identical calls: not yet.
|
||||
expect(guard.evaluate().level).toBe('healthy');
|
||||
|
||||
guard.recordToolCall({
|
||||
name: 'create_scene',
|
||||
arguments: '{"scene_name":"Town"}',
|
||||
});
|
||||
const decision = guard.evaluate();
|
||||
expect(decision.level).toBe('steering');
|
||||
expect(decision.reason).toContain('looping');
|
||||
expect(decision.reason).toContain('create_scene');
|
||||
expect(decision.action).toBe('steer');
|
||||
});
|
||||
|
||||
it('is reset by an intervening different call', () => {
|
||||
const guard = createLoopGuard({ repeatedToolLimit: 3 });
|
||||
call(guard, '{"a":1}');
|
||||
call(guard, '{"a":1}');
|
||||
call(guard, '{"a":2}');
|
||||
// Only two identical in a row now.
|
||||
expect(guard.evaluate().level).toBe('healthy');
|
||||
call(guard, '{"a":2}');
|
||||
call(guard, '{"a":2}');
|
||||
expect(guard.evaluate().level).toBe('steering');
|
||||
});
|
||||
|
||||
it('does not trip on two long arguments differing past character 300', () => {
|
||||
// The direct #377 regression, at the default limit.
|
||||
const guard = createLoopGuard({ repeatedToolLimit: 2 });
|
||||
const preamble = 'y'.repeat(300);
|
||||
guard.recordToolCall({
|
||||
name: 'run_script',
|
||||
arguments: `${preamble}A`,
|
||||
});
|
||||
guard.recordToolCall({
|
||||
name: 'run_script',
|
||||
arguments: `${preamble}B`,
|
||||
});
|
||||
expect(guard.evaluate().level).toBe('healthy');
|
||||
});
|
||||
});
|
||||
|
||||
describe('error storm trip', () => {
|
||||
it('trips after the configured number of consecutive failures', () => {
|
||||
const guard = createLoopGuard({ errorStormLimit: 5 });
|
||||
for (let i = 0; i < 4; i++) guard.recordError();
|
||||
expect(guard.evaluate().level).toBe('healthy');
|
||||
|
||||
guard.recordError();
|
||||
const decision = guard.evaluate();
|
||||
expect(decision.level).toBe('steering');
|
||||
expect(decision.reason).toContain('error storm');
|
||||
expect(decision.reason).toContain('5');
|
||||
});
|
||||
|
||||
it('is reset by a successful (distinct) tool call', () => {
|
||||
const guard = createLoopGuard({ errorStormLimit: 3 });
|
||||
guard.recordError();
|
||||
guard.recordError();
|
||||
guard.recordToolCall({ name: 'create_scene', arguments: '{}' });
|
||||
guard.recordError();
|
||||
guard.recordError();
|
||||
// Only two consecutive errors since the successful call.
|
||||
expect(guard.evaluate().level).toBe('healthy');
|
||||
});
|
||||
});
|
||||
|
||||
describe('turn limit trip', () => {
|
||||
it('trips at the configured turn count', () => {
|
||||
const guard = createLoopGuard({ maxTurnsWithoutUserInput: 60 });
|
||||
for (let i = 0; i < 59; i++) guard.recordTurn();
|
||||
expect(guard.evaluate().level).toBe('healthy');
|
||||
|
||||
guard.recordTurn();
|
||||
const decision = guard.evaluate();
|
||||
expect(decision.level).toBe('steering');
|
||||
expect(decision.reason).toContain('turn limit');
|
||||
});
|
||||
|
||||
it('is cleared only by reset, not by tool calls or errors', () => {
|
||||
const guard = createLoopGuard({ maxTurnsWithoutUserInput: 3 });
|
||||
guard.recordTurn();
|
||||
guard.recordTurn();
|
||||
guard.recordTurn();
|
||||
// Progress does not clear the turn counter.
|
||||
guard.recordToolCall({ name: 'a', arguments: '{}' });
|
||||
guard.recordToolCall({ name: 'b', arguments: '{}' });
|
||||
guard.recordToolCall({ name: 'c', arguments: '{}' });
|
||||
guard.recordError();
|
||||
guard.recordError();
|
||||
expect(guard.evaluate().level).toBe('steering');
|
||||
|
||||
guard.reset();
|
||||
expect(guard.evaluate().level).toBe('healthy');
|
||||
});
|
||||
});
|
||||
|
||||
describe('escalation ladder', () => {
|
||||
it('climbs at most one level per evaluate and never past constrained', () => {
|
||||
const guard = createLoopGuard({
|
||||
repeatedToolLimit: 1,
|
||||
maxTurnsWithoutUserInput: 1000,
|
||||
});
|
||||
// One call is already a "loop" at limit 1.
|
||||
guard.recordToolCall({ name: 'a', arguments: '{}' });
|
||||
|
||||
expect(guard.evaluate().level).toBe('steering');
|
||||
expect(guard.evaluate().level).toBe('constrained');
|
||||
// Capped: no 'stopped' level (there is no hardStop here).
|
||||
expect(guard.evaluate().level).toBe('constrained');
|
||||
expect(guard.evaluate().level).toBe('constrained');
|
||||
});
|
||||
|
||||
it('reports an action only when the level escalates', () => {
|
||||
const guard = createLoopGuard({
|
||||
repeatedToolLimit: 1,
|
||||
maxTurnsWithoutUserInput: 1000,
|
||||
});
|
||||
guard.recordToolCall({ name: 'a', arguments: '{}' });
|
||||
|
||||
const first = guard.evaluate();
|
||||
expect(first.level).toBe('steering');
|
||||
expect(first.action).toBe('steer');
|
||||
|
||||
// Second tick escalates to constrained.
|
||||
const second = guard.evaluate();
|
||||
expect(second.level).toBe('constrained');
|
||||
expect(second.action).toBe('constrain');
|
||||
|
||||
// Third tick does not escalate: no action, so a durable steer is not
|
||||
// re-sent.
|
||||
const third = guard.evaluate();
|
||||
expect(third.level).toBe('constrained');
|
||||
expect(third.action).toBe('none');
|
||||
});
|
||||
|
||||
it('de-escalates a level per healthy evaluate', () => {
|
||||
const guard = createLoopGuard({
|
||||
repeatedToolLimit: 1,
|
||||
maxTurnsWithoutUserInput: 1000,
|
||||
});
|
||||
guard.recordToolCall({ name: 'a', arguments: '{}' });
|
||||
guard.evaluate(); // steering
|
||||
guard.evaluate(); // constrained
|
||||
|
||||
// Clear the trip: the guard recovers one level per tick.
|
||||
guard.reset();
|
||||
const recovering = guard.evaluate();
|
||||
expect(recovering.level).toBe('steering');
|
||||
expect(recovering.reason).toBe('recovering');
|
||||
expect(recovering.action).toBe('none');
|
||||
|
||||
expect(guard.evaluate().level).toBe('healthy');
|
||||
expect(guard.evaluate().level).toBe('healthy');
|
||||
});
|
||||
|
||||
it('stays healthy and actionless when nothing trips', () => {
|
||||
const guard = createLoopGuard();
|
||||
guard.recordToolCall({ name: 'a', arguments: '{}' });
|
||||
const decision = guard.evaluate();
|
||||
expect(decision.level).toBe('healthy');
|
||||
expect(decision.action).toBe('none');
|
||||
expect(decision.changed).toBe(false);
|
||||
});
|
||||
|
||||
it('reports healthy with no trip when disabled', () => {
|
||||
const guard = createLoopGuard({ enabled: false, repeatedToolLimit: 1 });
|
||||
guard.recordToolCall({ name: 'a', arguments: '{}' });
|
||||
const decision = guard.evaluate();
|
||||
expect(decision.level).toBe('healthy');
|
||||
expect(decision.action).toBe('none');
|
||||
});
|
||||
});
|
||||
|
||||
it('defaults are the documented ones', () => {
|
||||
expect(DEFAULT_LOOP_GUARD_CONFIG).toEqual({
|
||||
enabled: true,
|
||||
repeatedToolLimit: 8,
|
||||
errorStormLimit: 5,
|
||||
maxTurnsWithoutUserInput: 60,
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,237 @@
|
||||
// @flow
|
||||
|
||||
/**
|
||||
* The local plan store: the shape `create_or_update_plan` writes and the studio
|
||||
* reads back.
|
||||
*
|
||||
* The plan shape itself is upstream's (`AiRequestPlan` / `AiRequestPlanTask` in
|
||||
* `Utils/GDevelopServices/Generation.js`), and so is the reading half:
|
||||
* `AiRequestUtils.getLatestActivePlan` pulls the last `function_call_output`
|
||||
* whose JSON has `.plan.tasks`, and `AiRequestChat/OrchestratorPlan.js` renders
|
||||
* it. This module only builds that exact shape and answers questions about it,
|
||||
* so the hosted and BYOK paths return the same thing for the same tool.
|
||||
*/
|
||||
|
||||
export type StudioPlanTaskStatus =
|
||||
| 'pending'
|
||||
| 'in_progress'
|
||||
| 'done'
|
||||
| 'voided';
|
||||
|
||||
export type StudioPlanTask = {|
|
||||
id: string,
|
||||
title: string,
|
||||
description: string,
|
||||
status: StudioPlanTaskStatus,
|
||||
dependsOn: Array<string>,
|
||||
/**
|
||||
* Set by the studio when the task is delegated, so the plan row can be linked
|
||||
* to the sub-agent's function call (matches
|
||||
* `AiRequestMessageAssistantFunctionCall.taskId`).
|
||||
*/
|
||||
agentCallId?: string,
|
||||
|};
|
||||
|
||||
const PLAN_TASK_STATUSES: Array<StudioPlanTaskStatus> = [
|
||||
'pending',
|
||||
'in_progress',
|
||||
'done',
|
||||
'voided',
|
||||
];
|
||||
|
||||
export const isStudioPlanTaskStatus = (value: any): boolean =>
|
||||
typeof value === 'string' && PLAN_TASK_STATUSES.includes((value: any));
|
||||
|
||||
/**
|
||||
* The output of `create_or_update_plan`: the exact nesting
|
||||
* `getLatestActivePlan` parses (`output.plan.tasks`).
|
||||
*/
|
||||
export const buildPlanOutput = (tasks: Array<StudioPlanTask>): Object => ({
|
||||
success: true,
|
||||
plan: { tasks },
|
||||
});
|
||||
|
||||
/** Read a task array back out of a plan tool's output. Null when malformed. */
|
||||
export const getPlanFromOutput = (output: Object): ?Array<StudioPlanTask> => {
|
||||
if (!output || typeof output !== 'object') return null;
|
||||
const plan: any = (output: any).plan;
|
||||
if (!plan || typeof plan !== 'object') return null;
|
||||
const tasks: any = plan.tasks;
|
||||
if (!Array.isArray(tasks)) return null;
|
||||
return tasks;
|
||||
};
|
||||
|
||||
/**
|
||||
* True when every id in `dependsOn` names a task that is done. An id matching no
|
||||
* task is NOT satisfied: a dependency on missing work is not met.
|
||||
*/
|
||||
export const areTaskDependenciesSatisfied = (
|
||||
task: StudioPlanTask,
|
||||
tasks: Array<StudioPlanTask>
|
||||
): boolean => {
|
||||
const doneIds = new Set(
|
||||
tasks
|
||||
.filter(otherTask => otherTask.status === 'done')
|
||||
.map(otherTask => otherTask.id)
|
||||
);
|
||||
return task.dependsOn.every(dependencyId => doneIds.has(dependencyId));
|
||||
};
|
||||
|
||||
/** The first pending task whose dependencies are satisfied, or null. */
|
||||
export const getNextReadyTask = (
|
||||
tasks: Array<StudioPlanTask>
|
||||
): ?StudioPlanTask =>
|
||||
tasks.find(
|
||||
task =>
|
||||
task.status === 'pending' && areTaskDependenciesSatisfied(task, tasks)
|
||||
) || null;
|
||||
|
||||
/**
|
||||
* Fold `incoming` tasks over `existing`, matching by `id`.
|
||||
*
|
||||
* The result is `incoming` — its order and its membership, so a task dropped
|
||||
* from the list is gone. What survives are the fields on the matching existing
|
||||
* task that `incoming` does not mention. A field the caller DOES send wins,
|
||||
* including an explicit null: that is how a field is cleared. A task without a
|
||||
* string id passes through untouched.
|
||||
*
|
||||
* Ported from munder-difflin's `mergeTaskLedger` (`src/shared/taskLedger.ts`),
|
||||
* which exists because a writer holding a partial model of a card silently
|
||||
* deleted every field it did not know about.
|
||||
*/
|
||||
export const mergePlanTasks = (
|
||||
existing: Array<StudioPlanTask>,
|
||||
incoming: Array<StudioPlanTask>
|
||||
): Array<StudioPlanTask> => {
|
||||
const existingList = Array.isArray(existing) ? existing : [];
|
||||
const incomingList = Array.isArray(incoming) ? incoming : [];
|
||||
const byId = new Map<string, any>();
|
||||
for (const task of existingList) {
|
||||
if (task && typeof task.id === 'string' && task.id && !byId.has(task.id)) {
|
||||
byId.set(task.id, task);
|
||||
}
|
||||
}
|
||||
return incomingList.map((task: any) => {
|
||||
if (!task || typeof task.id !== 'string' || !task.id) return task;
|
||||
const prior = byId.get(task.id);
|
||||
return prior ? { ...prior, ...task } : task;
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* Apply `patch` to one task, leaving every other task — and every other field of
|
||||
* the patched task — untouched.
|
||||
*
|
||||
* This is what a status flip must use. Rebuilding the task array from a model
|
||||
* strips whatever fields the model does not know about (`agentCallId`,
|
||||
* `description`, a future field), which is exactly what
|
||||
* `patchTaskInLedger` exists to prevent in munder-difflin
|
||||
* (`src/shared/taskLedger.ts`).
|
||||
*/
|
||||
export const patchPlanTask = (
|
||||
tasks: Array<StudioPlanTask>,
|
||||
taskId: string,
|
||||
patch: Partial<StudioPlanTask>
|
||||
): Array<StudioPlanTask> => {
|
||||
const list = Array.isArray(tasks) ? tasks : [];
|
||||
return list.map(task =>
|
||||
task && task.id === taskId ? { ...task, ...patch } : task
|
||||
);
|
||||
};
|
||||
|
||||
export type StudioPlanTaskValidation =
|
||||
| {| success: true, tasks: Array<StudioPlanTask> |}
|
||||
| {| success: false, message: string |};
|
||||
|
||||
const describeValue = (value: any): string => {
|
||||
if (typeof value === 'string') return JSON.stringify(value);
|
||||
if (value === null) return 'null';
|
||||
if (Array.isArray(value)) return 'an array';
|
||||
return typeof value;
|
||||
};
|
||||
|
||||
/**
|
||||
* Validate the `tasks` argument of `create_or_update_plan` as a local model
|
||||
* sent it, naming the first offending index. A model mistake becomes a normal
|
||||
* failed tool call, never a thrown error inside the editor.
|
||||
*/
|
||||
export const validatePlanTasks = (rawTasks: any): StudioPlanTaskValidation => {
|
||||
if (!Array.isArray(rawTasks)) {
|
||||
return {
|
||||
success: false,
|
||||
message: 'create_or_update_plan requires a tasks array.',
|
||||
};
|
||||
}
|
||||
|
||||
const tasks: Array<StudioPlanTask> = [];
|
||||
for (let index = 0; index < rawTasks.length; index++) {
|
||||
const rawTask = rawTasks[index];
|
||||
const invalid = (detail: string): StudioPlanTaskValidation => ({
|
||||
success: false,
|
||||
message: `Invalid plan task at index ${index}: ${detail}`,
|
||||
});
|
||||
|
||||
if (!rawTask || typeof rawTask !== 'object' || Array.isArray(rawTask)) {
|
||||
return invalid(`expected an object, got ${describeValue(rawTask)}.`);
|
||||
}
|
||||
if (typeof rawTask.id !== 'string' || !rawTask.id.trim()) {
|
||||
return invalid(
|
||||
`\`id\` must be a non-empty string, got ${describeValue(rawTask.id)}.`
|
||||
);
|
||||
}
|
||||
if (typeof rawTask.title !== 'string' || !rawTask.title.trim()) {
|
||||
return invalid(
|
||||
`\`title\` must be a non-empty string, got ${describeValue(
|
||||
rawTask.title
|
||||
)}.`
|
||||
);
|
||||
}
|
||||
if (
|
||||
typeof rawTask.description !== 'string' ||
|
||||
!rawTask.description.trim()
|
||||
) {
|
||||
return invalid(
|
||||
`\`description\` must be a non-empty string, got ${describeValue(
|
||||
rawTask.description
|
||||
)}.`
|
||||
);
|
||||
}
|
||||
if (!isStudioPlanTaskStatus(rawTask.status)) {
|
||||
return invalid(
|
||||
`\`status\` must be one of ${PLAN_TASK_STATUSES.join(
|
||||
', '
|
||||
)}, got ${describeValue(rawTask.status)}.`
|
||||
);
|
||||
}
|
||||
const rawDependsOn: any = rawTask.dependsOn;
|
||||
let dependsOn: Array<string>;
|
||||
if (rawDependsOn == null) {
|
||||
dependsOn = [];
|
||||
} else if (
|
||||
Array.isArray(rawDependsOn) &&
|
||||
rawDependsOn.every((id: any) => typeof id === 'string')
|
||||
) {
|
||||
dependsOn = rawDependsOn;
|
||||
} else {
|
||||
return invalid(
|
||||
`\`dependsOn\` must be an array of strings, got ${describeValue(
|
||||
rawDependsOn
|
||||
)}.`
|
||||
);
|
||||
}
|
||||
|
||||
const task: StudioPlanTask = {
|
||||
id: rawTask.id,
|
||||
title: rawTask.title,
|
||||
description: rawTask.description,
|
||||
status: (rawTask.status: StudioPlanTaskStatus),
|
||||
dependsOn,
|
||||
};
|
||||
if (typeof rawTask.agentCallId === 'string') {
|
||||
task.agentCallId = rawTask.agentCallId;
|
||||
}
|
||||
tasks.push(task);
|
||||
}
|
||||
|
||||
return { success: true, tasks };
|
||||
};
|
||||
@@ -0,0 +1,301 @@
|
||||
// @flow
|
||||
import {
|
||||
buildPlanOutput,
|
||||
getPlanFromOutput,
|
||||
areTaskDependenciesSatisfied,
|
||||
getNextReadyTask,
|
||||
validatePlanTasks,
|
||||
mergePlanTasks,
|
||||
patchPlanTask,
|
||||
} from './PlanStore';
|
||||
import { getLatestActivePlan } from '../AiRequestUtils';
|
||||
|
||||
const makeTask = (overrides: Object) => ({
|
||||
id: 'task-1',
|
||||
title: 'Build the town grid',
|
||||
description: 'A 20x20 tile grid with roads.',
|
||||
status: 'pending',
|
||||
dependsOn: [],
|
||||
...overrides,
|
||||
});
|
||||
|
||||
describe('PlanStore', () => {
|
||||
describe('buildPlanOutput / getPlanFromOutput', () => {
|
||||
it('round-trips through getLatestActivePlan', () => {
|
||||
const tasks = [
|
||||
makeTask({ id: 'gdd', title: 'Write the GDD', status: 'done' }),
|
||||
makeTask({ id: 'grid', title: 'Build the grid', dependsOn: ['gdd'] }),
|
||||
];
|
||||
|
||||
const aiRequest: any = {
|
||||
output: [
|
||||
{
|
||||
type: 'function_call_output',
|
||||
call_id: 'c',
|
||||
output: JSON.stringify(buildPlanOutput(tasks)),
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const plan = getLatestActivePlan(aiRequest);
|
||||
expect(plan).not.toBeNull();
|
||||
expect(plan && plan.tasks).toEqual(tasks);
|
||||
});
|
||||
|
||||
it('returns the tasks a plan output holds', () => {
|
||||
const tasks = [makeTask({})];
|
||||
expect(getPlanFromOutput(buildPlanOutput(tasks))).toEqual(tasks);
|
||||
});
|
||||
|
||||
it('returns null for every malformed output', () => {
|
||||
// $FlowFixMe[incompatible-call]
|
||||
expect(getPlanFromOutput(null)).toBeNull();
|
||||
// $FlowFixMe[incompatible-call]
|
||||
expect(getPlanFromOutput(undefined)).toBeNull();
|
||||
// $FlowFixMe[incompatible-call]
|
||||
expect(getPlanFromOutput('not an object')).toBeNull();
|
||||
expect(getPlanFromOutput({})).toBeNull();
|
||||
expect(getPlanFromOutput({ plan: null })).toBeNull();
|
||||
expect(getPlanFromOutput({ plan: {} })).toBeNull();
|
||||
expect(getPlanFromOutput({ plan: { tasks: null } })).toBeNull();
|
||||
expect(getPlanFromOutput({ plan: { tasks: 'nope' } })).toBeNull();
|
||||
expect(getPlanFromOutput({ plan: { tasks: {} } })).toBeNull();
|
||||
expect(getPlanFromOutput({ success: false, message: 'nope' })).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('dependencies', () => {
|
||||
it('is satisfied only when every dependency is done', () => {
|
||||
const doneTask = makeTask({ id: 'a', status: 'done' });
|
||||
const pendingTask = makeTask({ id: 'b', status: 'pending' });
|
||||
|
||||
expect(
|
||||
areTaskDependenciesSatisfied(makeTask({ dependsOn: [] }), [doneTask])
|
||||
).toBe(true);
|
||||
expect(
|
||||
areTaskDependenciesSatisfied(makeTask({ dependsOn: ['a'] }), [doneTask])
|
||||
).toBe(true);
|
||||
expect(
|
||||
areTaskDependenciesSatisfied(makeTask({ dependsOn: ['b'] }), [
|
||||
pendingTask,
|
||||
])
|
||||
).toBe(false);
|
||||
expect(
|
||||
areTaskDependenciesSatisfied(makeTask({ dependsOn: ['a', 'b'] }), [
|
||||
doneTask,
|
||||
pendingTask,
|
||||
])
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('treats a dependency on a missing task as unsatisfied', () => {
|
||||
expect(
|
||||
areTaskDependenciesSatisfied(makeTask({ dependsOn: ['ghost'] }), [
|
||||
makeTask({ id: 'a', status: 'done' }),
|
||||
])
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('treats a voided dependency as unsatisfied', () => {
|
||||
expect(
|
||||
areTaskDependenciesSatisfied(makeTask({ dependsOn: ['a'] }), [
|
||||
makeTask({ id: 'a', status: 'voided' }),
|
||||
])
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('getNextReadyTask', () => {
|
||||
it('returns the first pending task whose dependencies are satisfied', () => {
|
||||
const tasks = [
|
||||
makeTask({ id: 'a', status: 'done' }),
|
||||
makeTask({ id: 'b', status: 'pending', dependsOn: ['a'] }),
|
||||
makeTask({ id: 'c', status: 'pending', dependsOn: ['b'] }),
|
||||
];
|
||||
const readyB = getNextReadyTask(tasks);
|
||||
expect(readyB && readyB.id).toBe('b');
|
||||
});
|
||||
|
||||
it('skips a task whose dependency is not done, and a task already started', () => {
|
||||
const tasks = [
|
||||
makeTask({ id: 'a', status: 'pending' }),
|
||||
makeTask({ id: 'b', status: 'in_progress' }),
|
||||
makeTask({ id: 'c', status: 'pending', dependsOn: ['b'] }),
|
||||
makeTask({ id: 'd', status: 'pending' }),
|
||||
];
|
||||
const readyA = getNextReadyTask(tasks);
|
||||
expect(readyA && readyA.id).toBe('a');
|
||||
});
|
||||
|
||||
it('returns null when nothing is ready', () => {
|
||||
expect(
|
||||
getNextReadyTask([
|
||||
makeTask({ id: 'a', status: 'done' }),
|
||||
makeTask({ id: 'b', status: 'pending', dependsOn: ['a', 'zz'] }),
|
||||
])
|
||||
).toBeNull();
|
||||
expect(getNextReadyTask([])).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('validatePlanTasks', () => {
|
||||
it('accepts a well-formed plan and defaults dependsOn to an empty array', () => {
|
||||
const result = validatePlanTasks([
|
||||
{
|
||||
id: 'a',
|
||||
title: 'Write the GDD',
|
||||
description: 'Sections: overview, economy.',
|
||||
status: 'pending',
|
||||
},
|
||||
]);
|
||||
expect(result.success).toBe(true);
|
||||
if (result.success) {
|
||||
expect(result.tasks[0].dependsOn).toEqual([]);
|
||||
expect(result.tasks[0].status).toBe('pending');
|
||||
}
|
||||
});
|
||||
|
||||
it('rejects a non-array', () => {
|
||||
const result = validatePlanTasks(undefined);
|
||||
expect(result.success).toBe(false);
|
||||
if (!result.success) {
|
||||
expect(result.message).toBe(
|
||||
'create_or_update_plan requires a tasks array.'
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it('rejects a malformed task, naming its index', () => {
|
||||
const valid = {
|
||||
id: 'a',
|
||||
title: 'Ok',
|
||||
description: 'Ok.',
|
||||
status: 'pending',
|
||||
};
|
||||
const result = validatePlanTasks([
|
||||
valid,
|
||||
{ id: '', title: 'No id', description: 'x', status: 'pending' },
|
||||
]);
|
||||
expect(result.success).toBe(false);
|
||||
if (!result.success) {
|
||||
expect(result.message).toContain('index 1');
|
||||
expect(result.message).toContain('`id`');
|
||||
}
|
||||
});
|
||||
|
||||
it('rejects an unknown status and a non-string dependsOn', () => {
|
||||
const bad = validatePlanTasks([
|
||||
{ id: 'a', title: 'T', description: 'D', status: 'later' },
|
||||
]);
|
||||
expect(bad.success).toBe(false);
|
||||
if (!bad.success) expect(bad.message).toContain('`status`');
|
||||
|
||||
const bad2 = validatePlanTasks([
|
||||
{
|
||||
id: 'a',
|
||||
title: 'T',
|
||||
description: 'D',
|
||||
status: 'pending',
|
||||
dependsOn: [1, 2],
|
||||
},
|
||||
]);
|
||||
expect(bad2.success).toBe(false);
|
||||
if (!bad2.success) expect(bad2.message).toContain('`dependsOn`');
|
||||
});
|
||||
});
|
||||
|
||||
describe('mergePlanTasks', () => {
|
||||
it('keeps a field the writer never mentioned', () => {
|
||||
const existing: any = [
|
||||
makeTask({ id: 'a', agentCallId: 'call-1', description: 'Original.' }),
|
||||
];
|
||||
const incoming: any = [makeTask({ id: 'a', status: 'in_progress' })];
|
||||
|
||||
const merged = mergePlanTasks(existing, incoming);
|
||||
expect(merged[0].agentCallId).toBe('call-1');
|
||||
// An unmentioned field keeps its existing value; the sent field wins.
|
||||
expect(merged[0].status).toBe('in_progress');
|
||||
});
|
||||
|
||||
it('lets an explicit null clear a field and a missing key not clear it', () => {
|
||||
const existing: any = [makeTask({ id: 'a', agentCallId: 'call-1' })];
|
||||
const cleared: any = mergePlanTasks(existing, [
|
||||
{ ...makeTask({ id: 'a' }), agentCallId: null },
|
||||
]);
|
||||
expect(cleared[0].agentCallId).toBeNull();
|
||||
|
||||
const untouched: any = mergePlanTasks(existing, [makeTask({ id: 'a' })]);
|
||||
expect(untouched[0].agentCallId).toBe('call-1');
|
||||
});
|
||||
|
||||
it('takes the incoming order and membership, so a dropped task is gone', () => {
|
||||
const existing: any = [
|
||||
makeTask({ id: 'a' }),
|
||||
makeTask({ id: 'b' }),
|
||||
makeTask({ id: 'c' }),
|
||||
];
|
||||
const merged = mergePlanTasks(existing, [
|
||||
makeTask({ id: 'c' }),
|
||||
makeTask({ id: 'a' }),
|
||||
]);
|
||||
expect(merged.map(task => task.id)).toEqual(['c', 'a']);
|
||||
});
|
||||
|
||||
it('passes an id-less task through untouched', () => {
|
||||
const idLess: any = { title: 'Hand-written', status: 'pending' };
|
||||
const merged: any = mergePlanTasks([makeTask({ id: 'a' })], [idLess]);
|
||||
expect(merged[0]).toBe(idLess);
|
||||
});
|
||||
|
||||
it('adds a task that has no existing counterpart', () => {
|
||||
const merged = mergePlanTasks(
|
||||
[makeTask({ id: 'a' })],
|
||||
[makeTask({ id: 'b', title: 'New' })]
|
||||
);
|
||||
expect(merged).toHaveLength(1);
|
||||
expect(merged[0].id).toBe('b');
|
||||
});
|
||||
});
|
||||
|
||||
describe('patchPlanTask', () => {
|
||||
it('leaves a sibling task byte-identical, including its agentCallId', () => {
|
||||
const tasks: any = [
|
||||
makeTask({ id: 'a', status: 'in_progress', agentCallId: 'call-1' }),
|
||||
makeTask({
|
||||
id: 'b',
|
||||
description: 'A description the writer wrote.',
|
||||
agentCallId: 'call-2',
|
||||
}),
|
||||
];
|
||||
const patched: any = patchPlanTask(tasks, 'a', { status: 'done' });
|
||||
|
||||
expect(patched[0].status).toBe('done');
|
||||
// The sibling is the same object: nothing was rebuilt.
|
||||
expect(patched[1]).toBe(tasks[1]);
|
||||
expect(patched[1].agentCallId).toBe('call-2');
|
||||
expect(patched[1].description).toBe('A description the writer wrote.');
|
||||
// The patched task keeps the fields the patch did not mention.
|
||||
expect(patched[0].agentCallId).toBe('call-1');
|
||||
});
|
||||
|
||||
it('is a no-op for an unknown id and for a non-array', () => {
|
||||
const tasks: any = [makeTask({ id: 'a' })];
|
||||
const patched: any = patchPlanTask(tasks, 'ghost', { status: 'done' });
|
||||
// Same task objects, unchanged.
|
||||
expect(patched).toEqual(tasks);
|
||||
expect(patched[0]).toBe(tasks[0]);
|
||||
// A non-array is tolerated at runtime.
|
||||
expect(patchPlanTask((null: any), 'a', { status: 'done' })).toEqual([]);
|
||||
});
|
||||
|
||||
it('survives a round-trip through buildPlanOutput and getPlanFromOutput', () => {
|
||||
const tasks: any = [
|
||||
makeTask({ id: 'a', status: 'in_progress' }),
|
||||
makeTask({ id: 'b', agentCallId: 'call-2' }),
|
||||
];
|
||||
const patched: any = patchPlanTask(tasks, 'a', { status: 'done' });
|
||||
const reread: any = getPlanFromOutput(buildPlanOutput(patched));
|
||||
expect(reread).toEqual(patched);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,158 @@
|
||||
// @flow
|
||||
|
||||
/**
|
||||
* One gate owns every write to an `AiRequest`.
|
||||
*
|
||||
* Ported from munder-difflin's "one gate, one writer" (`src/renderer/src/
|
||||
* components/terminalAutomation.ts` + `docs/message-queue.md` §1). There, a
|
||||
* second writer with its own idea of when the terminal was free landed its text
|
||||
* on a half-written line; here, the studio runtime, the shared dispatcher and the
|
||||
* user's own send all write to the same request, and a write that lands in the
|
||||
* middle of a function-call batch can be silently replaced by the next one
|
||||
* (`updateAiRequest(id, () => aiRequest)` discards previous state).
|
||||
*
|
||||
* The gate does two things:
|
||||
* - `getRequestWriteBlock` says, with a named reason, why a write must wait;
|
||||
* - `createRequestWriteQueue` serializes the writes per request, keeping at most
|
||||
* one pending write (latest wins) and retrying a blocked one rather than
|
||||
* dropping it.
|
||||
*/
|
||||
|
||||
export type RequestWriteBlock =
|
||||
// The request is suspended: the editor has cleared its function-call results
|
||||
// and a write now would re-populate them (`AiRequestContext:1438`).
|
||||
| 'suspended'
|
||||
// A user message is in flight for this request.
|
||||
| 'sending'
|
||||
// A function call of this request is being processed by the editor.
|
||||
| 'call-in-flight'
|
||||
// Nothing blocks a write.
|
||||
| null;
|
||||
|
||||
export const getRequestWriteBlock = ({
|
||||
aiRequestId,
|
||||
request,
|
||||
isSending,
|
||||
inFlightCallIds,
|
||||
}: {|
|
||||
aiRequestId: string,
|
||||
request: ?{ +status?: string },
|
||||
isSending: boolean,
|
||||
inFlightCallIds: Set<string>,
|
||||
|}): RequestWriteBlock => {
|
||||
if (request && request.status === 'suspended') return 'suspended';
|
||||
if (isSending) return 'sending';
|
||||
const prefix = `${aiRequestId}:`;
|
||||
for (const callId of inFlightCallIds) {
|
||||
if (callId.startsWith(prefix)) return 'call-in-flight';
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
type PendingWrite = {|
|
||||
write: () => Promise<void>,
|
||||
isBlocked: () => boolean,
|
||||
resolve: () => void,
|
||||
reject: (error: Error) => void,
|
||||
|};
|
||||
|
||||
type QueueEntry = {
|
||||
isRunning: boolean,
|
||||
/** At most one pending write per request: a newer one replaces it. */
|
||||
pending: PendingWrite | null,
|
||||
};
|
||||
|
||||
export type RequestWriteQueue = {|
|
||||
enqueue: (
|
||||
aiRequestId: string,
|
||||
write: () => Promise<void>,
|
||||
isBlocked: () => boolean
|
||||
) => Promise<void>,
|
||||
isIdle: (aiRequestId: string) => boolean,
|
||||
|};
|
||||
|
||||
/**
|
||||
* A per-request FIFO holding at most one pending write.
|
||||
*
|
||||
* A newer write for the same request replaces the older pending one (latest
|
||||
* wins, matching munder-difflin's `MAX_PENDING_STEERS` drop-from-the-front rule
|
||||
* in `src/main/control.ts`). A write whose `isBlocked()` is true is retried on
|
||||
* the next `enqueue` for that request rather than dropped: the caller is not
|
||||
* asked to poll, and the last enqueue always resolves when the queue drains.
|
||||
*/
|
||||
export const createRequestWriteQueue = (): RequestWriteQueue => {
|
||||
const entries: Map<string, QueueEntry> = new Map();
|
||||
|
||||
const getEntry = (aiRequestId: string): QueueEntry => {
|
||||
let entry = entries.get(aiRequestId);
|
||||
if (!entry) {
|
||||
entry = { isRunning: false, pending: null };
|
||||
entries.set(aiRequestId, entry);
|
||||
}
|
||||
return entry;
|
||||
};
|
||||
|
||||
const runEntry = async (aiRequestId: string): Promise<void> => {
|
||||
const entry = getEntry(aiRequestId);
|
||||
if (entry.isRunning) return;
|
||||
|
||||
entry.isRunning = true;
|
||||
try {
|
||||
// Keep draining until there is nothing left to run: a write enqueued
|
||||
// while this loop awaited has to be picked up by the same loop, not left
|
||||
// to a second `runEntry` that would see `isRunning` and return.
|
||||
while (entry.pending) {
|
||||
const pending = entry.pending;
|
||||
if (pending.isBlocked()) {
|
||||
// Do not drop it: the next enqueue retries it. Nothing more to do
|
||||
// now, because the queue only advances when the caller enqueues.
|
||||
break;
|
||||
}
|
||||
entry.pending = null;
|
||||
try {
|
||||
await pending.write();
|
||||
pending.resolve();
|
||||
} catch (error) {
|
||||
pending.reject(error);
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
entry.isRunning = false;
|
||||
if (!entry.pending) {
|
||||
entries.delete(aiRequestId);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
return {
|
||||
enqueue: (
|
||||
aiRequestId: string,
|
||||
write: () => Promise<void>,
|
||||
isBlocked: () => boolean
|
||||
): Promise<void> => {
|
||||
const entry = getEntry(aiRequestId);
|
||||
// Latest wins: replace whatever was waiting, and settle it as superseded
|
||||
// (resolved, not rejected - the caller asked for a write that a newer one
|
||||
// made pointless).
|
||||
if (entry.pending) {
|
||||
const superseded = entry.pending;
|
||||
entry.pending = null;
|
||||
superseded.resolve();
|
||||
}
|
||||
return new Promise<void>((resolve, reject) => {
|
||||
entry.pending = { write, isBlocked, resolve, reject };
|
||||
if (!entry.isRunning) {
|
||||
runEntry(aiRequestId).catch(error => {
|
||||
// `runEntry` handles its own per-write errors; this is a safety net
|
||||
// so a bug there cannot produce an unhandled rejection.
|
||||
console.error('[studio] request write queue failure:', error);
|
||||
});
|
||||
}
|
||||
});
|
||||
},
|
||||
isIdle: (aiRequestId: string): boolean => {
|
||||
const entry = entries.get(aiRequestId);
|
||||
return !entry || (!entry.isRunning && !entry.pending);
|
||||
},
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,246 @@
|
||||
// @flow
|
||||
import {
|
||||
getRequestWriteBlock,
|
||||
createRequestWriteQueue,
|
||||
} from './RequestWriteGate';
|
||||
|
||||
describe('RequestWriteGate', () => {
|
||||
describe('getRequestWriteBlock', () => {
|
||||
it('blocks a suspended request', () => {
|
||||
expect(
|
||||
getRequestWriteBlock({
|
||||
aiRequestId: 'req-1',
|
||||
request: { status: 'suspended' },
|
||||
isSending: false,
|
||||
inFlightCallIds: new Set(),
|
||||
})
|
||||
).toBe('suspended');
|
||||
});
|
||||
|
||||
it('blocks while a user send is in flight', () => {
|
||||
expect(
|
||||
getRequestWriteBlock({
|
||||
aiRequestId: 'req-1',
|
||||
request: { status: 'ready' },
|
||||
isSending: true,
|
||||
inFlightCallIds: new Set(),
|
||||
})
|
||||
).toBe('sending');
|
||||
});
|
||||
|
||||
it('blocks while one of this request function calls is being processed', () => {
|
||||
expect(
|
||||
getRequestWriteBlock({
|
||||
aiRequestId: 'req-1',
|
||||
request: { status: 'ready' },
|
||||
isSending: false,
|
||||
inFlightCallIds: new Set(['req-1:call-1']),
|
||||
})
|
||||
).toBe('call-in-flight');
|
||||
// Another request's call must not block this one.
|
||||
expect(
|
||||
getRequestWriteBlock({
|
||||
aiRequestId: 'req-1',
|
||||
request: { status: 'ready' },
|
||||
isSending: false,
|
||||
inFlightCallIds: new Set(['req-2:call-1']),
|
||||
})
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('reports the suspension first', () => {
|
||||
// A suspended request that also has a call in flight stays 'suspended':
|
||||
// that is the path that cleared the results, so it is the decisive one.
|
||||
expect(
|
||||
getRequestWriteBlock({
|
||||
aiRequestId: 'req-1',
|
||||
request: { status: 'suspended' },
|
||||
isSending: true,
|
||||
inFlightCallIds: new Set(['req-1:call-1']),
|
||||
})
|
||||
).toBe('suspended');
|
||||
});
|
||||
|
||||
it('does not block a clean request', () => {
|
||||
expect(
|
||||
getRequestWriteBlock({
|
||||
aiRequestId: 'req-1',
|
||||
request: { status: 'ready' },
|
||||
isSending: false,
|
||||
inFlightCallIds: new Set(),
|
||||
})
|
||||
).toBeNull();
|
||||
expect(
|
||||
getRequestWriteBlock({
|
||||
aiRequestId: 'req-1',
|
||||
request: null,
|
||||
isSending: false,
|
||||
inFlightCallIds: new Set(),
|
||||
})
|
||||
).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('createRequestWriteQueue', () => {
|
||||
it('runs a write and reports the queue idle afterwards', async () => {
|
||||
const queue = createRequestWriteQueue();
|
||||
const order = [];
|
||||
await queue.enqueue(
|
||||
'req-1',
|
||||
async () => {
|
||||
order.push('write');
|
||||
},
|
||||
() => false
|
||||
);
|
||||
expect(order).toEqual(['write']);
|
||||
expect(queue.isIdle('req-1')).toBe(true);
|
||||
});
|
||||
|
||||
it('never interleaves two writes for one request', async () => {
|
||||
const queue = createRequestWriteQueue();
|
||||
const events = [];
|
||||
|
||||
const first = queue.enqueue(
|
||||
'req-1',
|
||||
async () => {
|
||||
events.push('first-start');
|
||||
await new Promise(resolve => setTimeout(resolve, 20));
|
||||
events.push('first-end');
|
||||
},
|
||||
() => false
|
||||
);
|
||||
const second = queue.enqueue(
|
||||
'req-1',
|
||||
async () => {
|
||||
events.push('second-start');
|
||||
events.push('second-end');
|
||||
},
|
||||
() => false
|
||||
);
|
||||
|
||||
await Promise.all([first, second]);
|
||||
|
||||
// The second write starts only after the first finished.
|
||||
expect(events).toEqual([
|
||||
'first-start',
|
||||
'first-end',
|
||||
'second-start',
|
||||
'second-end',
|
||||
]);
|
||||
});
|
||||
|
||||
it('does not block writes for different request ids', async () => {
|
||||
const queue = createRequestWriteQueue();
|
||||
const events = [];
|
||||
|
||||
const slow = queue.enqueue(
|
||||
'req-1',
|
||||
async () => {
|
||||
events.push('slow-start');
|
||||
await new Promise(resolve => setTimeout(resolve, 20));
|
||||
events.push('slow-end');
|
||||
},
|
||||
() => false
|
||||
);
|
||||
const fast = queue.enqueue(
|
||||
'req-2',
|
||||
async () => {
|
||||
events.push('fast-start');
|
||||
events.push('fast-end');
|
||||
},
|
||||
() => false
|
||||
);
|
||||
|
||||
await Promise.all([slow, fast]);
|
||||
|
||||
// The second request's write completed while the first was still running.
|
||||
expect(events).toEqual([
|
||||
'slow-start',
|
||||
'fast-start',
|
||||
'fast-end',
|
||||
'slow-end',
|
||||
]);
|
||||
});
|
||||
|
||||
it('keeps only the latest pending write for a request', async () => {
|
||||
const queue = createRequestWriteQueue();
|
||||
const events = [];
|
||||
|
||||
const first = queue.enqueue(
|
||||
'req-1',
|
||||
async () => {
|
||||
events.push('first');
|
||||
await new Promise(resolve => setTimeout(resolve, 20));
|
||||
},
|
||||
() => false
|
||||
);
|
||||
// These two are enqueued while `first` runs: the middle one is superseded.
|
||||
const second = queue.enqueue(
|
||||
'req-1',
|
||||
async () => {
|
||||
events.push('second');
|
||||
},
|
||||
() => false
|
||||
);
|
||||
const third = queue.enqueue(
|
||||
'req-1',
|
||||
async () => {
|
||||
events.push('third');
|
||||
},
|
||||
() => false
|
||||
);
|
||||
|
||||
await Promise.all([first, second, third]);
|
||||
expect(events).toEqual(['first', 'third']);
|
||||
});
|
||||
|
||||
it('retries a blocked write rather than dropping it', async () => {
|
||||
const queue = createRequestWriteQueue();
|
||||
let blocked = true;
|
||||
const events = [];
|
||||
|
||||
const write = queue.enqueue(
|
||||
'req-1',
|
||||
async () => {
|
||||
events.push('ran');
|
||||
},
|
||||
() => blocked
|
||||
);
|
||||
|
||||
// Give the queue a chance to observe the block.
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
expect(events).toEqual([]);
|
||||
|
||||
// Unblock and enqueue again: the retried write runs.
|
||||
blocked = false;
|
||||
await queue.enqueue(
|
||||
'req-1',
|
||||
async () => {
|
||||
events.push('ran');
|
||||
},
|
||||
() => blocked
|
||||
);
|
||||
await write;
|
||||
expect(events).toEqual(['ran']);
|
||||
});
|
||||
|
||||
it('rejects the caller when its write throws', async () => {
|
||||
const queue = createRequestWriteQueue();
|
||||
let caught = null;
|
||||
try {
|
||||
await queue.enqueue(
|
||||
'req-1',
|
||||
async () => {
|
||||
throw new Error('boom');
|
||||
},
|
||||
() => false
|
||||
);
|
||||
} catch (error) {
|
||||
caught = error;
|
||||
}
|
||||
expect(caught && caught.message).toBe('boom');
|
||||
// A failed write does not wedge the queue.
|
||||
expect(queue.isIdle('req-1')).toBe(true);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,234 @@
|
||||
// @flow
|
||||
|
||||
/**
|
||||
* The role registry of the local (BYOK) AI game studio.
|
||||
*
|
||||
* Upstream's hosted backend runs a multi-agent company server-side: it decides
|
||||
* which sub-agents exist and what each may do. Under BYOK there is no server, so
|
||||
* the fork has to declare that here: for every role, the prompt it runs with, the
|
||||
* exact tools it may call, whether it may change the project, and how many
|
||||
* assistant turns it gets before it is finalized.
|
||||
*
|
||||
* Roles are a fork concept. Nothing in the hosted path reads this file: the
|
||||
* server's own `run_edit_agent`/`run_explorer_agent` split is unrelated and keeps
|
||||
* working untouched (see `EditorFunctions/index.js`).
|
||||
*/
|
||||
|
||||
export type StudioRoleId = 'manager' | 'designer' | 'developer' | 'tester';
|
||||
|
||||
export type StudioRole = {|
|
||||
id: StudioRoleId,
|
||||
/** Shown in the chat for the sub-agent. Becomes `short_title` of the spawn call. */
|
||||
displayName: string,
|
||||
/** Prepended to the shared system prompt for this role's requests. */
|
||||
systemPrompt: string,
|
||||
/**
|
||||
* Exact tool names exposed to this role. Any name not listed here is never
|
||||
* sent to the model (see `getToolsForRole`), so a role cannot be talked into
|
||||
* using a tool it was not granted.
|
||||
*/
|
||||
allowedToolNames: Array<string>,
|
||||
/**
|
||||
* True when the role's tools may not mutate the project. Drives the read-only
|
||||
* `run_script` restriction and skips the edit approval prompt.
|
||||
*/
|
||||
readOnly: boolean,
|
||||
/**
|
||||
* Hard cap on assistant turns for one sub-agent request. On reaching it, the
|
||||
* agent is finalized with its last message (see `FinalizeSubAgents`).
|
||||
*/
|
||||
maxTurns: number,
|
||||
|};
|
||||
|
||||
/** The tools that only read the project, shared by the roles that inspect. */
|
||||
export const READ_ONLY_TOOL_NAMES: Array<string> = [
|
||||
'read_game_project_json',
|
||||
'read_events_source',
|
||||
'describe_instances',
|
||||
'inspect_variables',
|
||||
'inspect_object_properties_effects',
|
||||
'inspect_scene_properties_layers_effects',
|
||||
'inspect_project_properties_resources',
|
||||
'inspect_behavior_properties',
|
||||
'inspect_extension',
|
||||
'search_object_asset_store',
|
||||
'search_resource_store',
|
||||
'search_docs',
|
||||
'read_full_docs',
|
||||
];
|
||||
|
||||
/** Every tool that changes the project, in schema order. */
|
||||
export const MUTATING_TOOL_NAMES: Array<string> = [
|
||||
'run_script',
|
||||
'create_scene',
|
||||
'create_or_replace_object',
|
||||
'change_object_properties_effects',
|
||||
'add_behavior',
|
||||
'change_behavior_property',
|
||||
'put_2d_instances',
|
||||
'put_3d_instances',
|
||||
// `generate_events` is a registry alias of this one, not a separate tool the
|
||||
// schema exposes (see EditorFunctions/index.js).
|
||||
'add_scene_events',
|
||||
'change_scene_properties_layers_effects_groups',
|
||||
'change_project_properties_resources',
|
||||
'add_or_edit_variable',
|
||||
'create_extension',
|
||||
'change_extension_properties',
|
||||
'create_custom_object',
|
||||
'change_custom_object',
|
||||
'create_custom_behavior',
|
||||
'change_custom_behavior',
|
||||
'create_custom_function',
|
||||
'change_custom_function',
|
||||
];
|
||||
|
||||
export const STUDIO_ROLES: { [StudioRoleId]: StudioRole } = {
|
||||
manager: {
|
||||
id: 'manager',
|
||||
displayName: 'Studio lead',
|
||||
readOnly: false,
|
||||
maxTurns: 60,
|
||||
allowedToolNames: [
|
||||
'create_or_update_plan',
|
||||
'spawn_agent',
|
||||
'read_game_project_json',
|
||||
'describe_instances',
|
||||
'read_events_source',
|
||||
'inspect_variables',
|
||||
'inspect_scene_properties_layers_effects',
|
||||
'inspect_object_properties_effects',
|
||||
'inspect_extension',
|
||||
'inspect_project_properties_resources',
|
||||
'run_gameplay_test',
|
||||
'search_docs',
|
||||
'read_full_docs',
|
||||
],
|
||||
systemPrompt: `You are the studio lead of a small game studio building a game in GDevelop.
|
||||
|
||||
Your job is to plan and delegate. You never edit the project yourself - you have no tool to do so.
|
||||
|
||||
Follow this loop, every time:
|
||||
1. Decompose the goal into a short list of concrete tasks with create_or_update_plan. Give each task an id, a title, a description of what "done" means, and any dependsOn.
|
||||
2. Delegate each ready task with spawn_agent. Pick the role that matches the work: designer for the design document, developer for anything that changes the project, tester for verifying a feature. Give each sub-agent a complete, self-contained instruction - it cannot see this conversation.
|
||||
3. Pass the plan task's id as related_task_id, so the task is marked done when the agent reports back.
|
||||
4. Wait for a sub-agent's report before treating its task as done. Do not spawn a second agent for the same task.
|
||||
5. If a report says the work failed or is incomplete, spawn a follow-up agent with the specifics, or report the problem to the user.
|
||||
|
||||
Finish with a short summary of what was built, what was verified, and what is left. If a task genuinely cannot be done with the tools available, say so plainly instead of delegating it again.`,
|
||||
},
|
||||
|
||||
designer: {
|
||||
id: 'designer',
|
||||
displayName: 'Designer',
|
||||
readOnly: false,
|
||||
maxTurns: 30,
|
||||
// The only mutating tool: a design document has to survive into the next
|
||||
// role, and project variables are the only artifact a later agent can read.
|
||||
allowedToolNames: [...READ_ONLY_TOOL_NAMES, 'add_or_edit_variable'],
|
||||
systemPrompt: `You are the designer of a small game studio. You produce the game design document for the studio lead.
|
||||
|
||||
You write the design as GDevelop project variables: one variable per section, named with the GDD_ prefix (for example GDD_Overview, GDD_Economy, GDD_Progression, GDD_Controls), with a string value holding the section's text. Project variables are the only artifact the next role can read - never write your design only as a chat message.
|
||||
|
||||
You never create scenes, objects or events: another role does that. Your tools that change anything are limited to add_or_edit_variable.
|
||||
|
||||
Read the project first with read_game_project_json and describe_instances, so the design fits what already exists. Be specific and numeric: name the resources, the costs, the rates and the win condition, so the developer does not have to invent them. Report which GDD_ variables you wrote.`,
|
||||
},
|
||||
|
||||
developer: {
|
||||
id: 'developer',
|
||||
displayName: 'Developer',
|
||||
readOnly: false,
|
||||
maxTurns: 80,
|
||||
allowedToolNames: [
|
||||
...MUTATING_TOOL_NAMES,
|
||||
'read_game_project_json',
|
||||
'read_events_source',
|
||||
'describe_instances',
|
||||
'inspect_variables',
|
||||
'inspect_object_properties_effects',
|
||||
'inspect_behavior_properties',
|
||||
'inspect_scene_properties_layers_effects',
|
||||
'inspect_extension',
|
||||
'inspect_project_properties_resources',
|
||||
'search_object_asset_store',
|
||||
'search_resource_store',
|
||||
'search_docs',
|
||||
'read_full_docs',
|
||||
],
|
||||
systemPrompt: `You are the developer of a small game studio. You implement exactly the task you were given, and nothing else.
|
||||
|
||||
Read the GDD_ project variables first: they hold the design you are implementing. If the task you were given contradicts them, follow the task and say so in your report.
|
||||
|
||||
Prefer run_script when a change needs many calls (placing a grid of instances, setting up many objects): one script doing fifty calls beats fifty tool calls. Await every call inside the script, and keep the script under a few hundred calls.
|
||||
|
||||
Inspect before you write: read the project, describe the instances and inspect the objects you are about to change, so you edit what actually exists.
|
||||
|
||||
When you are done, report what you changed, naming the scenes, objects, variables and events you touched, and state anything you could not do. A partial implementation reported honestly is worth more than a silent failure.`,
|
||||
},
|
||||
|
||||
tester: {
|
||||
id: 'tester',
|
||||
displayName: 'QA tester',
|
||||
readOnly: true,
|
||||
maxTurns: 30,
|
||||
// Nothing here changes the project: the tester writes a test, runs it, and
|
||||
// reports. `change_gameplay_tests` is the one exception - it edits a test
|
||||
// definition, never the game.
|
||||
allowedToolNames: [
|
||||
'run_gameplay_test',
|
||||
'change_gameplay_tests',
|
||||
'read_game_project_json',
|
||||
'read_events_source',
|
||||
'describe_instances',
|
||||
'inspect_variables',
|
||||
'inspect_object_properties_effects',
|
||||
'inspect_scene_properties_layers_effects',
|
||||
'inspect_behavior_properties',
|
||||
],
|
||||
systemPrompt: `You are the QA tester of a small game studio. You verify the feature you were asked about, and you never change game content.
|
||||
|
||||
Write a gameplay test with run_gameplay_test, using harness assertions that actually exercise the named feature - assert on the state the feature is supposed to produce, not on it merely existing. Read the project first so your test names the real objects, scenes and variables.
|
||||
|
||||
Run the test, then report PASS or FAIL and quote the assertions and any errors verbatim. If it fails, say which assertion failed and what the game did instead. If the feature is not implemented at all, say that plainly: a missing feature is a finding, not a test to make pass.`,
|
||||
},
|
||||
};
|
||||
|
||||
/** The roles a `spawn_agent` call may name. A sub-agent never spawns sub-agents. */
|
||||
export const SPAWNABLE_ROLE_IDS: Array<StudioRoleId> = [
|
||||
'designer',
|
||||
'developer',
|
||||
'tester',
|
||||
];
|
||||
|
||||
export const isStudioRoleId = (value: any): boolean =>
|
||||
typeof value === 'string' && !!STUDIO_ROLES[(value: any)];
|
||||
|
||||
export const isSpawnableRoleId = (value: any): boolean =>
|
||||
isStudioRoleId(value) && SPAWNABLE_ROLE_IDS.includes((value: StudioRoleId));
|
||||
|
||||
export const getStudioRole = (roleId: StudioRoleId): StudioRole => {
|
||||
const role = STUDIO_ROLES[roleId];
|
||||
if (!role) {
|
||||
throw new Error(`Unknown studio role: ${String(roleId)}.`);
|
||||
}
|
||||
return role;
|
||||
};
|
||||
|
||||
/**
|
||||
* The subset of `allTools` a role may call, in `allTools` order.
|
||||
*
|
||||
* A role naming a tool that is not in `allTools` is not an error: the tool is
|
||||
* skipped. That keeps a role declaration usable against an older tool schema
|
||||
* (and lets `Roles.spec.js` state the invariant without importing a second
|
||||
* source of truth).
|
||||
*/
|
||||
export const getToolsForRole = (
|
||||
roleId: StudioRoleId,
|
||||
allTools: Array<Object>
|
||||
): Array<Object> => {
|
||||
const allowedNames = new Set(getStudioRole(roleId).allowedToolNames);
|
||||
return allTools.filter(
|
||||
tool => !!tool && !!tool.function && allowedNames.has(tool.function.name)
|
||||
);
|
||||
};
|
||||
@@ -0,0 +1,153 @@
|
||||
// @flow
|
||||
import {
|
||||
STUDIO_ROLES,
|
||||
READ_ONLY_TOOL_NAMES,
|
||||
MUTATING_TOOL_NAMES,
|
||||
getStudioRole,
|
||||
getToolsForRole,
|
||||
} from './Roles';
|
||||
import { GDEVELOP_OPENAI_TOOLS } from '../../AI/CustomAIClient';
|
||||
|
||||
describe('Studio roles', () => {
|
||||
const allToolNames: Array<string> = GDEVELOP_OPENAI_TOOLS.map(
|
||||
tool => tool.function.name
|
||||
);
|
||||
// `spawn_agent` is part of the studio's own tool schema addition (Phase 4),
|
||||
// not of the upstream-derived part of `GDEVELOP_OPENAI_TOOLS`.
|
||||
const availableToolNames = new Set([...allToolNames, 'spawn_agent']);
|
||||
|
||||
const roleIds = Object.keys(STUDIO_ROLES);
|
||||
|
||||
it('declares exactly the four studio roles', () => {
|
||||
expect(roleIds.sort()).toEqual([
|
||||
'designer',
|
||||
'developer',
|
||||
'manager',
|
||||
'tester',
|
||||
]);
|
||||
});
|
||||
|
||||
it('every role only names tools that exist', () => {
|
||||
roleIds.forEach(roleId => {
|
||||
const unknown = STUDIO_ROLES[(roleId: any)].allowedToolNames.filter(
|
||||
name => !availableToolNames.has(name)
|
||||
);
|
||||
expect({ roleId, unknown }).toEqual({ roleId, unknown: [] });
|
||||
});
|
||||
});
|
||||
|
||||
it('declares every role id and display name consistently', () => {
|
||||
roleIds.forEach(roleId => {
|
||||
const role = getStudioRole((roleId: any));
|
||||
expect(role.id).toBe(roleId);
|
||||
expect(role.displayName.length).toBeGreaterThan(0);
|
||||
expect(role.systemPrompt.length).toBeGreaterThan(0);
|
||||
expect(role.maxTurns).toBeGreaterThan(0);
|
||||
// A role with no allowed tool could only talk - it would waste a turn.
|
||||
expect(role.allowedToolNames.length).toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
|
||||
it('throws on an unknown role', () => {
|
||||
expect(() => getStudioRole(('architect': any))).toThrow(
|
||||
'Unknown studio role'
|
||||
);
|
||||
});
|
||||
|
||||
it('getToolsForRole filters by name and preserves schema order', () => {
|
||||
const developerTools = getToolsForRole('developer', GDEVELOP_OPENAI_TOOLS);
|
||||
const developerToolNames = developerTools.map(tool => tool.function.name);
|
||||
|
||||
expect(developerToolNames.length).toBeGreaterThan(0);
|
||||
// Nothing outside the declared subset leaks in.
|
||||
developerToolNames.forEach(name =>
|
||||
expect(STUDIO_ROLES.developer.allowedToolNames).toContain(name)
|
||||
);
|
||||
// Order is the schema's, not the declaration's.
|
||||
const schemaOrder = allToolNames.filter(name =>
|
||||
developerToolNames.includes(name)
|
||||
);
|
||||
expect(developerToolNames).toEqual(schemaOrder);
|
||||
|
||||
// The role's whole declared set is offered, since every developer tool is in
|
||||
// the schema.
|
||||
expect(developerToolNames.sort()).toEqual(
|
||||
[...STUDIO_ROLES.developer.allowedToolNames].sort()
|
||||
);
|
||||
});
|
||||
|
||||
it('getToolsForRole skips a declared tool that the schema does not have', () => {
|
||||
// No throw, no placeholder entry.
|
||||
const tools = getToolsForRole('manager', [
|
||||
{ type: 'function', function: { name: 'not_a_real_tool' } },
|
||||
{ type: 'function', function: { name: 'spawn_agent' } },
|
||||
]);
|
||||
expect(tools.map(tool => tool.function.name)).toEqual(['spawn_agent']);
|
||||
});
|
||||
|
||||
it('the tester cannot change the project and the designer can only write variables', () => {
|
||||
const testerToolNames = new Set(STUDIO_ROLES.tester.allowedToolNames);
|
||||
expect(STUDIO_ROLES.tester.readOnly).toBe(true);
|
||||
expect(testerToolNames.has('run_script')).toBe(false);
|
||||
expect(testerToolNames.has('create_scene')).toBe(false);
|
||||
expect(testerToolNames.has('create_or_replace_object')).toBe(false);
|
||||
expect(testerToolNames.has('put_2d_instances')).toBe(false);
|
||||
expect(testerToolNames.has('put_3d_instances')).toBe(false);
|
||||
expect(testerToolNames.has('add_scene_events')).toBe(false);
|
||||
expect(testerToolNames.has('create_extension')).toBe(false);
|
||||
|
||||
// The designer writes only variables: every other tool it may call is one
|
||||
// of the declared read-only tools.
|
||||
const designerToolNames = STUDIO_ROLES.designer.allowedToolNames;
|
||||
expect(designerToolNames).toContain('add_or_edit_variable');
|
||||
const designerReadOnlyTools = designerToolNames.filter(
|
||||
name => name !== 'add_or_edit_variable'
|
||||
);
|
||||
expect(
|
||||
designerReadOnlyTools.filter(name => !READ_ONLY_TOOL_NAMES.includes(name))
|
||||
).toEqual([]);
|
||||
// And the tester is granted no mutating tool at all.
|
||||
const testerMutations = STUDIO_ROLES.tester.allowedToolNames.filter(name =>
|
||||
MUTATING_TOOL_NAMES.includes(name)
|
||||
);
|
||||
expect(testerMutations).toEqual([]);
|
||||
});
|
||||
|
||||
it('only the manager may delegate, so a sub-agent never spawns a sub-agent', () => {
|
||||
expect(STUDIO_ROLES.manager.allowedToolNames).toContain('spawn_agent');
|
||||
['designer', 'developer', 'tester'].forEach(roleId => {
|
||||
expect(STUDIO_ROLES[(roleId: any)].allowedToolNames).not.toContain(
|
||||
'spawn_agent'
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
it('gives the developer every mutating tool the schema offers, and the tester none', () => {
|
||||
const mutatingNames = [
|
||||
'run_script',
|
||||
'create_scene',
|
||||
'create_or_replace_object',
|
||||
'change_object_properties_effects',
|
||||
'add_behavior',
|
||||
'change_behavior_property',
|
||||
'put_2d_instances',
|
||||
'put_3d_instances',
|
||||
'add_scene_events',
|
||||
'change_scene_properties_layers_effects_groups',
|
||||
'change_project_properties_resources',
|
||||
'add_or_edit_variable',
|
||||
'create_extension',
|
||||
'change_extension_properties',
|
||||
'create_custom_object',
|
||||
'change_custom_object',
|
||||
'create_custom_behavior',
|
||||
'change_custom_behavior',
|
||||
'create_custom_function',
|
||||
'change_custom_function',
|
||||
];
|
||||
mutatingNames.forEach(name => {
|
||||
expect(STUDIO_ROLES.developer.allowedToolNames).toContain(name);
|
||||
expect(STUDIO_ROLES.tester.allowedToolNames).not.toContain(name);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,192 @@
|
||||
// @flow
|
||||
|
||||
import { makeSimplifiedProjectBuilder } from '../../EditorFunctions/SimplifiedProject/SimplifiedProject';
|
||||
import { customCreateSubAgentAiRequest } from '../../AI/CustomAIClient';
|
||||
import { getStudioRole, isSpawnableRoleId, type StudioRoleId } from './Roles';
|
||||
import { type AiRequestMessageAssistantFunctionCall } from '../../Utils/GDevelopServices/Generation';
|
||||
|
||||
const gd: libGDevelop = global.gd;
|
||||
|
||||
/** Whether a function call is the studio's delegation tool. */
|
||||
export const isSpawnAgentCall = (
|
||||
functionCall: AiRequestMessageAssistantFunctionCall
|
||||
): boolean => functionCall.name === 'spawn_agent';
|
||||
|
||||
export type ParsedSpawnAgentArgs = {|
|
||||
role: StudioRoleId,
|
||||
shortTitle: string,
|
||||
task: string,
|
||||
context: string,
|
||||
relatedTaskId: string | null,
|
||||
|};
|
||||
|
||||
/**
|
||||
* Read and validate a `spawn_agent` call's arguments. Null when the call is
|
||||
* malformed: the caller then fails the call instead of spawning an agent with
|
||||
* half an instruction.
|
||||
*/
|
||||
export const parseSpawnAgentArgs = (
|
||||
functionCall: AiRequestMessageAssistantFunctionCall
|
||||
): ?ParsedSpawnAgentArgs => {
|
||||
let parsedArguments: any;
|
||||
try {
|
||||
parsedArguments = JSON.parse(functionCall.arguments);
|
||||
} catch (error) {
|
||||
return null;
|
||||
}
|
||||
if (
|
||||
!parsedArguments ||
|
||||
typeof parsedArguments !== 'object' ||
|
||||
Array.isArray(parsedArguments)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// The model's own JSON: each field is checked, and anything unexpected makes
|
||||
// the whole call invalid (the caller then fails it instead of spawning with
|
||||
// half an instruction).
|
||||
const role: any = parsedArguments.role;
|
||||
if (!isSpawnableRoleId(role)) return null;
|
||||
|
||||
const rawShortTitle: any = parsedArguments.short_title;
|
||||
const rawTask: any = parsedArguments.task;
|
||||
const rawContext: any = parsedArguments.context;
|
||||
const rawRelatedTaskId: any = parsedArguments.related_task_id;
|
||||
|
||||
if (typeof rawShortTitle !== 'string') return null;
|
||||
const shortTitle = rawShortTitle.trim();
|
||||
if (!shortTitle) return null;
|
||||
|
||||
if (typeof rawTask !== 'string') return null;
|
||||
const task = rawTask.trim();
|
||||
if (!task) return null;
|
||||
|
||||
const context = typeof rawContext === 'string' ? rawContext.trim() : '';
|
||||
const relatedTaskId =
|
||||
typeof rawRelatedTaskId === 'string' && rawRelatedTaskId.trim()
|
||||
? rawRelatedTaskId.trim()
|
||||
: null;
|
||||
|
||||
return { role, shortTitle, task, context, relatedTaskId };
|
||||
};
|
||||
|
||||
/**
|
||||
* The `GDD_*` project variables, as a note appended to a sub-agent's context.
|
||||
*
|
||||
* The designer writes the design document as project variables (it is the only
|
||||
* artifact a later role can read), so the developer and the tester are handed
|
||||
* them explicitly rather than having to discover them.
|
||||
*
|
||||
* Project variables live at `globalVariables` on the simplified project (the
|
||||
* `properties` object holds resolution and orientation, not variables).
|
||||
*/
|
||||
export const buildGddContextNote = (project: ?gdProject): string => {
|
||||
if (!project) return '';
|
||||
let simplifiedProject: any;
|
||||
try {
|
||||
simplifiedProject = makeSimplifiedProjectBuilder(gd).getSimplifiedProject(
|
||||
project,
|
||||
{}
|
||||
);
|
||||
} catch (error) {
|
||||
// A project the simplifier cannot read is not a reason to refuse to spawn:
|
||||
// the agent simply gets no design note.
|
||||
console.warn(
|
||||
'Unable to read the project variables for a sub-agent:',
|
||||
error
|
||||
);
|
||||
return '';
|
||||
}
|
||||
|
||||
const projectVariables: Array<any> =
|
||||
(simplifiedProject && simplifiedProject.globalVariables) || [];
|
||||
const gddVariables = projectVariables.filter(
|
||||
variable =>
|
||||
variable &&
|
||||
typeof variable.variableName === 'string' &&
|
||||
variable.variableName.startsWith('GDD_')
|
||||
);
|
||||
if (gddVariables.length === 0) {
|
||||
return 'No GDD_ project variable is set on this project yet: the design document has not been written.';
|
||||
}
|
||||
|
||||
const lines = gddVariables.map(
|
||||
variable =>
|
||||
`- ${variable.variableName}: ${
|
||||
typeof variable.value === 'string'
|
||||
? variable.value
|
||||
: JSON.stringify(variable.variableChildren || variable.type)
|
||||
}`
|
||||
);
|
||||
return `The studio's design document, as GDD_ project variables:\n${lines.join(
|
||||
'\n'
|
||||
)}`;
|
||||
};
|
||||
|
||||
export type SpawnSubAgentResult =
|
||||
| {| subAgentAiRequestId: string, shortTitle: string, callId: string |}
|
||||
| {| error: string |};
|
||||
|
||||
/**
|
||||
* Create the child `AiRequest` for one `spawn_agent` call.
|
||||
*
|
||||
* The caller is responsible for stamping `subAgentAiRequestId` onto the call
|
||||
* (see `onStamped`) and for activating the sub-agent; this function only creates
|
||||
* the request, so it stays testable without the React context.
|
||||
*/
|
||||
export const spawnSubAgent = async ({
|
||||
parentAiRequestId,
|
||||
functionCall,
|
||||
gameProjectJson,
|
||||
projectSpecificExtensionsSummaryJson,
|
||||
project,
|
||||
onStamped,
|
||||
}: {|
|
||||
parentAiRequestId: string,
|
||||
functionCall: AiRequestMessageAssistantFunctionCall,
|
||||
gameProjectJson: string | null,
|
||||
projectSpecificExtensionsSummaryJson: string | null,
|
||||
project: ?gdProject,
|
||||
onStamped: (subAgentAiRequestId: string) => void,
|
||||
|}): Promise<SpawnSubAgentResult> => {
|
||||
const parsedArgs = parseSpawnAgentArgs(functionCall);
|
||||
if (!parsedArgs) {
|
||||
return { error: 'Invalid spawn_agent arguments.' };
|
||||
}
|
||||
|
||||
const role = getStudioRole(parsedArgs.role);
|
||||
if (!role) {
|
||||
// `isSpawnableRoleId` already gated this, but keep the failure a normal
|
||||
// result rather than a throw, so a model mistake is a failed tool call.
|
||||
return { error: `Unknown role: ${String(parsedArgs.role)}.` };
|
||||
}
|
||||
|
||||
let userRequest = `Task: ${parsedArgs.task}`;
|
||||
if (parsedArgs.context) {
|
||||
userRequest += `\n\nContext: ${parsedArgs.context}`;
|
||||
}
|
||||
|
||||
// The developer and the tester need the design document; the designer writes
|
||||
// it, so handing it back to the designer would be noise.
|
||||
const spawnContextNote =
|
||||
parsedArgs.role === 'developer' || parsedArgs.role === 'tester'
|
||||
? buildGddContextNote(project)
|
||||
: null;
|
||||
|
||||
const subAgentRequest = await customCreateSubAgentAiRequest({
|
||||
parentAiRequestId,
|
||||
roleId: parsedArgs.role,
|
||||
userRequest,
|
||||
gameProjectJson,
|
||||
projectSpecificExtensionsSummaryJson,
|
||||
spawnContextNote,
|
||||
});
|
||||
|
||||
onStamped(subAgentRequest.id);
|
||||
|
||||
return {
|
||||
subAgentAiRequestId: subAgentRequest.id,
|
||||
shortTitle: parsedArgs.shortTitle,
|
||||
callId: functionCall.call_id,
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,174 @@
|
||||
// @flow
|
||||
import {
|
||||
isSpawnAgentCall,
|
||||
parseSpawnAgentArgs,
|
||||
buildGddContextNote,
|
||||
} from './SpawnSubAgents';
|
||||
|
||||
const makeCall = (name: string, args: any): any => ({
|
||||
type: 'function_call',
|
||||
status: 'completed',
|
||||
call_id: 'call-1',
|
||||
name,
|
||||
arguments: typeof args === 'string' ? args : JSON.stringify(args),
|
||||
});
|
||||
|
||||
describe('SpawnSubAgents', () => {
|
||||
describe('isSpawnAgentCall', () => {
|
||||
it('matches only the studio delegation tool', () => {
|
||||
expect(isSpawnAgentCall(makeCall('spawn_agent', {}))).toBe(true);
|
||||
expect(isSpawnAgentCall(makeCall('create_scene', {}))).toBe(false);
|
||||
expect(isSpawnAgentCall(makeCall('run_edit_agent', {}))).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('parseSpawnAgentArgs', () => {
|
||||
it('parses a valid call and defaults context and related_task_id', () => {
|
||||
const parsed = parseSpawnAgentArgs(
|
||||
makeCall('spawn_agent', {
|
||||
role: 'developer',
|
||||
short_title: 'Build the town grid',
|
||||
task: 'Place a 20x20 grid of ground tiles in Scene1.',
|
||||
})
|
||||
);
|
||||
expect(parsed).toEqual({
|
||||
role: 'developer',
|
||||
shortTitle: 'Build the town grid',
|
||||
task: 'Place a 20x20 grid of ground tiles in Scene1.',
|
||||
context: '',
|
||||
relatedTaskId: null,
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps context and related_task_id when given', () => {
|
||||
const parsed = parseSpawnAgentArgs(
|
||||
makeCall('spawn_agent', {
|
||||
role: 'tester',
|
||||
short_title: 'Verify the grid',
|
||||
task: 'Write and run a test asserting 400 tiles exist.',
|
||||
context: 'The grid is built by the developer.',
|
||||
related_task_id: 'grid',
|
||||
})
|
||||
);
|
||||
expect(parsed && parsed.context).toBe(
|
||||
'The grid is built by the developer.'
|
||||
);
|
||||
expect(parsed && parsed.relatedTaskId).toBe('grid');
|
||||
});
|
||||
|
||||
it('returns null for a missing or empty task', () => {
|
||||
expect(
|
||||
parseSpawnAgentArgs(
|
||||
makeCall('spawn_agent', {
|
||||
role: 'developer',
|
||||
short_title: 'Title',
|
||||
})
|
||||
)
|
||||
).toBeNull();
|
||||
expect(
|
||||
parseSpawnAgentArgs(
|
||||
makeCall('spawn_agent', {
|
||||
role: 'developer',
|
||||
short_title: 'Title',
|
||||
task: ' ',
|
||||
})
|
||||
)
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('returns null for a missing or empty short_title', () => {
|
||||
expect(
|
||||
parseSpawnAgentArgs(
|
||||
makeCall('spawn_agent', { role: 'developer', task: 'Do it.' })
|
||||
)
|
||||
).toBeNull();
|
||||
expect(
|
||||
parseSpawnAgentArgs(
|
||||
makeCall('spawn_agent', {
|
||||
role: 'developer',
|
||||
short_title: ' ',
|
||||
task: 'Do it.',
|
||||
})
|
||||
)
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('returns null for an unknown or missing role', () => {
|
||||
expect(
|
||||
parseSpawnAgentArgs(
|
||||
makeCall('spawn_agent', {
|
||||
role: 'architect',
|
||||
short_title: 'Title',
|
||||
task: 'Do it.',
|
||||
})
|
||||
)
|
||||
).toBeNull();
|
||||
// The manager plans, it does not get spawned as a sub-agent.
|
||||
expect(
|
||||
parseSpawnAgentArgs(
|
||||
makeCall('spawn_agent', {
|
||||
role: 'manager',
|
||||
short_title: 'Title',
|
||||
task: 'Do it.',
|
||||
})
|
||||
)
|
||||
).toBeNull();
|
||||
expect(
|
||||
parseSpawnAgentArgs(
|
||||
makeCall('spawn_agent', { short_title: 'Title', task: 'Do it.' })
|
||||
)
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('returns null for arguments that are not JSON', () => {
|
||||
expect(
|
||||
parseSpawnAgentArgs(makeCall('spawn_agent', 'not json at all'))
|
||||
).toBeNull();
|
||||
expect(parseSpawnAgentArgs(makeCall('spawn_agent', '[]'))).toBeNull();
|
||||
expect(parseSpawnAgentArgs(makeCall('spawn_agent', 'null'))).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildGddContextNote', () => {
|
||||
let project: gdProject;
|
||||
|
||||
beforeEach(() => {
|
||||
const gd: libGDevelop = global.gd;
|
||||
// $FlowFixMe[invalid-constructor]
|
||||
project = new gd.ProjectHelper.createNewGDJSProject();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
project.delete();
|
||||
});
|
||||
|
||||
it('returns an empty note without a project', () => {
|
||||
expect(buildGddContextNote(null)).toBe('');
|
||||
// $FlowFixMe[incompatible-call]
|
||||
expect(buildGddContextNote(undefined)).toBe('');
|
||||
});
|
||||
|
||||
it('reads the GDD_ project variables and ignores the others', () => {
|
||||
const variables = project.getVariables();
|
||||
const gddOverview = variables.insertNew('GDD_Overview', 0);
|
||||
gddOverview.setString('A 20x20 isometric town.');
|
||||
const unrelated = variables.insertNew('Score', 1);
|
||||
unrelated.setValue(42);
|
||||
|
||||
const note = buildGddContextNote(project);
|
||||
expect(note).toContain('GDD_Overview: A 20x20 isometric town.');
|
||||
expect(note).not.toContain('Score');
|
||||
expect(note).toContain('design document');
|
||||
});
|
||||
|
||||
it('says the document is missing when no GDD_ variable exists', () => {
|
||||
project
|
||||
.getVariables()
|
||||
.insertNew('Score', 0)
|
||||
.setValue(1);
|
||||
|
||||
const note = buildGddContextNote(project);
|
||||
expect(note).toContain('has not been written');
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,303 @@
|
||||
// @flow
|
||||
|
||||
import * as React from 'react';
|
||||
import { type AiRequest } from '../../Utils/GDevelopServices/Generation';
|
||||
import { type EditorFunctionCallResult } from '../../EditorFunctions';
|
||||
import {
|
||||
getAllSubAgentFunctionCalls,
|
||||
getLatestActivePlan,
|
||||
} from '../AiRequestUtils';
|
||||
import {
|
||||
isSubAgentFinished,
|
||||
buildSubAgentReport,
|
||||
getSubAgentReportLabel,
|
||||
} from './FinalizeSubAgents';
|
||||
import {
|
||||
buildPlanOutput,
|
||||
patchPlanTask,
|
||||
getNextReadyTask,
|
||||
type StudioPlanTask,
|
||||
} from './PlanStore';
|
||||
import { isSpawnAgentCall, parseSpawnAgentArgs } from './SpawnSubAgents';
|
||||
|
||||
type ActiveSubAgent = {|
|
||||
parentAiRequestId: string,
|
||||
callId: string,
|
||||
|};
|
||||
|
||||
/**
|
||||
* How many sub-agents this hook may finalize in one pass. The pass runs on every
|
||||
* render, so an unbounded loop over a large fleet would stall the chat.
|
||||
*/
|
||||
const MAX_FINALIZATIONS_PER_PASS = 8;
|
||||
|
||||
/**
|
||||
* Flip the plan task a spawn call named to `done`, and the next ready task to
|
||||
* `in_progress`.
|
||||
*
|
||||
* The plan message already in the parent's output is rewritten in place - same
|
||||
* `call_id`, new JSON - rather than appended to: the plan tool call's own result
|
||||
* is that message, and `getLatestActivePlan` reads the last output carrying
|
||||
* `.plan.tasks`, so rewriting it keeps the transcript consistent and needs no
|
||||
* new tool call the model never made.
|
||||
*
|
||||
* The tasks are patched, not rebuilt: rebuilding strips the fields the writer
|
||||
* does not model (see `PlanStore.patchPlanTask`).
|
||||
*
|
||||
* Returns the new output array, or null when there is nothing to change.
|
||||
*/
|
||||
export const buildPlanStatusUpdateOutput = (
|
||||
parentRequest: AiRequest,
|
||||
callId: string
|
||||
): ?Array<any> => {
|
||||
const plan = getLatestActivePlan(parentRequest);
|
||||
if (!plan || !Array.isArray(plan.tasks)) return null;
|
||||
const tasks: Array<StudioPlanTask> = (plan.tasks: any);
|
||||
if (tasks.length === 0) return null;
|
||||
|
||||
const spawnCall = getAllSubAgentFunctionCalls({
|
||||
aiRequest: parentRequest,
|
||||
}).find(functionCall => functionCall.call_id === callId);
|
||||
if (!spawnCall || !isSpawnAgentCall(spawnCall)) return null;
|
||||
const parsedArgs = parseSpawnAgentArgs(spawnCall);
|
||||
const relatedTaskId = parsedArgs ? parsedArgs.relatedTaskId : null;
|
||||
if (!relatedTaskId) return null;
|
||||
if (!tasks.some(task => task && task.id === relatedTaskId)) return null;
|
||||
|
||||
let patchedTasks = patchPlanTask(tasks, relatedTaskId, { status: 'done' });
|
||||
const nextReadyTask = getNextReadyTask(patchedTasks);
|
||||
if (nextReadyTask) {
|
||||
patchedTasks = patchPlanTask(patchedTasks, nextReadyTask.id, {
|
||||
status: 'in_progress',
|
||||
});
|
||||
}
|
||||
|
||||
const planOutput = JSON.stringify(buildPlanOutput(patchedTasks));
|
||||
const output = parentRequest.output || [];
|
||||
|
||||
// Find the message the plan lives in, from the end, exactly as
|
||||
// `getLatestActivePlan` does.
|
||||
for (let i = output.length - 1; i >= 0; i--) {
|
||||
const message = output[i];
|
||||
if (message.type !== 'function_call_output' || !message.output) continue;
|
||||
try {
|
||||
const parsed = JSON.parse(message.output);
|
||||
if (parsed && parsed.plan && parsed.plan.tasks) {
|
||||
const updatedOutput = [...output];
|
||||
updatedOutput[i] = { ...message, output: planOutput };
|
||||
return updatedOutput;
|
||||
}
|
||||
} catch (error) {
|
||||
// Ignore parse errors, like `getLatestActivePlan` does.
|
||||
}
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
/**
|
||||
* The studio runtime: closes the loop between a spawned sub-agent and its parent.
|
||||
*
|
||||
* A child `AiRequest` is polled and driven by the shared machinery
|
||||
* (`AiRequestContext` + `useProcessFunctionCalls`), exactly like a hosted
|
||||
* sub-agent. What is missing under BYOK is the write-back: the hosted backend
|
||||
* posts a `function_call_output` for the sub-agent's call when its sub-agent is
|
||||
* done, and the client only reacts to it. This hook does that write locally -
|
||||
*
|
||||
* 1. wait until the child has nothing left to do (`isSubAgentFinished`);
|
||||
* 2. flip the plan task the spawn call named to `done`, and the next ready task
|
||||
* to `in_progress`, patching the raw tasks so nothing else is lost;
|
||||
* 3. post the child's report as its call's `function_call_output`, which the
|
||||
* shared BYOK path turns into the parent's next model turn and which
|
||||
* `removeSubAgentIfDone` reads to retire the child.
|
||||
*
|
||||
* A sub-agent cannot spawn a sub-agent: `spawn_agent` is not in any sub-agent
|
||||
* role's tool subset (`Roles.getToolsForRole`), so the recursion depth is one.
|
||||
*/
|
||||
export const useStudioRuntime = ({
|
||||
aiRequests,
|
||||
activeSubAgents,
|
||||
getEditorFunctionCallResults,
|
||||
updateAiRequest,
|
||||
onSendEditorFunctionCallResults,
|
||||
enqueueRequestWrite,
|
||||
}: {|
|
||||
aiRequests: { [string]: AiRequest },
|
||||
activeSubAgents: { [string]: ActiveSubAgent },
|
||||
getEditorFunctionCallResults: string => Array<EditorFunctionCallResult> | null,
|
||||
updateAiRequest: (
|
||||
aiRequestId: string,
|
||||
updateFn: (previousAiRequest: ?AiRequest) => AiRequest
|
||||
) => void,
|
||||
onSendEditorFunctionCallResults: (
|
||||
aiRequestId: string,
|
||||
editorFunctionCallResults: Array<EditorFunctionCallResult>,
|
||||
options: Object
|
||||
) => Promise<void>,
|
||||
/**
|
||||
* Runs a write once the request is free (see `RequestWriteGate`). Optional so
|
||||
* the hook can be used without the gate; the write then runs immediately.
|
||||
*/
|
||||
enqueueRequestWrite?: (
|
||||
aiRequestId: string,
|
||||
write: () => Promise<void>
|
||||
) => Promise<void>,
|
||||
|}): void => {
|
||||
// Call ids already finalized in this session. Without it, a render landing
|
||||
// between writing the output and the parent's state propagating would finalize
|
||||
// the same agent twice, and the parent would take two turns for one agent.
|
||||
const finalizedCallIdsRef = React.useRef<Set<string>>(new Set());
|
||||
// The pass awaits a message send, so a second pass can start before the first
|
||||
// finished. Serialize them.
|
||||
const isRunningRef = React.useRef<boolean>(false);
|
||||
|
||||
React.useEffect(
|
||||
() => {
|
||||
if (isRunningRef.current) return;
|
||||
|
||||
const subAgentIds = Object.keys(activeSubAgents);
|
||||
if (subAgentIds.length === 0) return;
|
||||
|
||||
const pendingFinalizations: Array<{|
|
||||
subAgentRequest: AiRequest,
|
||||
parentAiRequestId: string,
|
||||
callId: string,
|
||||
|}> = [];
|
||||
|
||||
for (const subAgentId of subAgentIds) {
|
||||
if (pendingFinalizations.length >= MAX_FINALIZATIONS_PER_PASS) break;
|
||||
const subAgentInfo = activeSubAgents[subAgentId];
|
||||
if (!subAgentInfo) continue;
|
||||
// Already finalized in an earlier pass.
|
||||
if (finalizedCallIdsRef.current.has(subAgentInfo.callId)) continue;
|
||||
|
||||
const subAgentRequest = aiRequests[subAgentId];
|
||||
if (!subAgentRequest) continue;
|
||||
const parentRequest = aiRequests[subAgentInfo.parentAiRequestId];
|
||||
if (!parentRequest) continue;
|
||||
|
||||
// The parent already carries this call's output: the child is on its way
|
||||
// out (`removeSubAgentIfDone` is what removes it). Stay compatible.
|
||||
const parentOutput = parentRequest.output || [];
|
||||
const parentAlreadyHasOutput = parentOutput.some(
|
||||
message =>
|
||||
message.type === 'function_call_output' &&
|
||||
message.call_id === subAgentInfo.callId
|
||||
);
|
||||
if (parentAlreadyHasOutput) continue;
|
||||
|
||||
if (
|
||||
!isSubAgentFinished({
|
||||
subAgentRequest,
|
||||
editorFunctionCallResults: getEditorFunctionCallResults(subAgentId),
|
||||
})
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
|
||||
pendingFinalizations.push({
|
||||
subAgentRequest,
|
||||
parentAiRequestId: subAgentInfo.parentAiRequestId,
|
||||
callId: subAgentInfo.callId,
|
||||
});
|
||||
}
|
||||
|
||||
if (pendingFinalizations.length === 0) return;
|
||||
|
||||
isRunningRef.current = true;
|
||||
(async () => {
|
||||
const startedCallIds: Array<string> = [];
|
||||
try {
|
||||
for (const {
|
||||
subAgentRequest,
|
||||
parentAiRequestId,
|
||||
callId,
|
||||
} of pendingFinalizations) {
|
||||
// Mark it before any await: a pass landing during the await must not
|
||||
// pick the same call up again.
|
||||
finalizedCallIdsRef.current.add(callId);
|
||||
startedCallIds.push(callId);
|
||||
if (finalizedCallIdsRef.current.size > 500) {
|
||||
// Bound the guard like the other in-flight sets of the AI path.
|
||||
finalizedCallIdsRef.current.clear();
|
||||
finalizedCallIdsRef.current.add(callId);
|
||||
}
|
||||
console.info(
|
||||
`[studio] finalizing sub-agent #${
|
||||
subAgentRequest.id
|
||||
} for call ${callId}.`
|
||||
);
|
||||
|
||||
const parentRequest = aiRequests[parentAiRequestId];
|
||||
if (parentRequest) {
|
||||
const updatedOutput = buildPlanStatusUpdateOutput(
|
||||
parentRequest,
|
||||
callId
|
||||
);
|
||||
if (updatedOutput) {
|
||||
const writePlan = async () => {
|
||||
updateAiRequest(parentAiRequestId, currentRequest =>
|
||||
currentRequest
|
||||
? { ...currentRequest, output: updatedOutput }
|
||||
: // The request disappeared: hand back the same empty shell
|
||||
// the mutator expects rather than null.
|
||||
({
|
||||
id: parentAiRequestId,
|
||||
createdAt: new Date().toISOString(),
|
||||
updatedAt: new Date().toISOString(),
|
||||
userId: 'local-byok-user',
|
||||
status: 'ready',
|
||||
error: null,
|
||||
output: updatedOutput,
|
||||
}: any)
|
||||
);
|
||||
};
|
||||
if (enqueueRequestWrite) {
|
||||
await enqueueRequestWrite(parentAiRequestId, writePlan);
|
||||
} else {
|
||||
await writePlan();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const label = getSubAgentReportLabel({
|
||||
aiRequest: subAgentRequest,
|
||||
aiRequests,
|
||||
});
|
||||
await onSendEditorFunctionCallResults(
|
||||
parentAiRequestId,
|
||||
[
|
||||
{
|
||||
status: 'finished',
|
||||
call_id: callId,
|
||||
success: true,
|
||||
output: {
|
||||
message: `Report from the ${label}:\n\n${buildSubAgentReport(
|
||||
subAgentRequest
|
||||
)}`,
|
||||
},
|
||||
},
|
||||
],
|
||||
{}
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[studio] error while finalizing a sub-agent:', error);
|
||||
// Let the next pass retry: a failed write must not be marked done.
|
||||
startedCallIds.forEach(callId =>
|
||||
finalizedCallIdsRef.current.delete(callId)
|
||||
);
|
||||
} finally {
|
||||
isRunningRef.current = false;
|
||||
}
|
||||
})();
|
||||
},
|
||||
[
|
||||
aiRequests,
|
||||
activeSubAgents,
|
||||
getEditorFunctionCallResults,
|
||||
updateAiRequest,
|
||||
onSendEditorFunctionCallResults,
|
||||
enqueueRequestWrite,
|
||||
]
|
||||
);
|
||||
};
|
||||
@@ -61,6 +61,14 @@ import { retryIfFailed } from '../Utils/RetryIfFailed';
|
||||
import { makeSimplifiedProjectBuilder } from '../EditorFunctions/SimplifiedProject/SimplifiedProject';
|
||||
import { prepareAiUserContent } from './PrepareAiUserContent';
|
||||
import { extractGDevelopApiErrorStatusAndCode } from '../Utils/GDevelopServices/Errors';
|
||||
import { isCustomEndpointEnabled } from '../AI/CustomAIClient';
|
||||
import { isSpawnAgentCall, spawnSubAgent } from './Studio/SpawnSubAgents';
|
||||
import { createLoopGuard } from './Studio/LoopGuard';
|
||||
import { mergePlanTasks } from './Studio/PlanStore';
|
||||
import {
|
||||
createRequestWriteQueue,
|
||||
getRequestWriteBlock,
|
||||
} from './Studio/RequestWriteGate';
|
||||
import UnsavedChangesContext from '../MainFrame/UnsavedChangesContext';
|
||||
import {
|
||||
type FileMetadata,
|
||||
@@ -264,6 +272,9 @@ export const useProcessFunctionCalls = ({
|
||||
getIsAutoEditEnabled,
|
||||
suspendAiRequest,
|
||||
requestEditApproval,
|
||||
activateSubAgent,
|
||||
updateAiRequest,
|
||||
isSendingAiRequest,
|
||||
}: {|
|
||||
i18n: I18nType,
|
||||
project: ?gdProject,
|
||||
@@ -319,12 +330,30 @@ export const useProcessFunctionCalls = ({
|
||||
getIsAutoEditEnabled: () => boolean,
|
||||
suspendAiRequest: (aiRequestId: string) => Promise<void>,
|
||||
requestEditApproval: (request: EditApprovalRequest) => Promise<boolean>,
|
||||
// Activates a studio sub-agent so the context polls it and the editor
|
||||
// processes its function calls (see `AiRequestContext.activateSubAgent`).
|
||||
activateSubAgent: (
|
||||
subAgentAiRequestId: string,
|
||||
parentAiRequestId: string,
|
||||
callId: string
|
||||
) => void,
|
||||
// Writes a request (used by the studio loop guard to mark a request failed).
|
||||
updateAiRequest: (
|
||||
aiRequestId: string,
|
||||
updateFn: (previousAiRequest: ?AiRequest) => AiRequest
|
||||
) => void,
|
||||
// Whether a user send is in flight for a request (read by the write gate).
|
||||
isSendingAiRequest: (aiRequestId: string | null) => boolean,
|
||||
|}): {
|
||||
onProcessFunctionCalls: (
|
||||
aiRequest: AiRequest,
|
||||
functionCalls: Array<AiRequestMessageAssistantFunctionCall>
|
||||
) => Promise<void>,
|
||||
clearApprovedEditBatches: () => void,
|
||||
enqueueRequestWrite: (
|
||||
aiRequestId: string,
|
||||
write: () => Promise<void>
|
||||
) => Promise<void>,
|
||||
} => {
|
||||
const { ensureExtensionInstalled } = useEnsureExtensionInstalled({
|
||||
project,
|
||||
@@ -403,6 +432,11 @@ export const useProcessFunctionCalls = ({
|
||||
// `call:<callId>` (for a single direct modifying call like generate_events).
|
||||
const approvedEditBatchKeysRef = React.useRef<Set<string>>(new Set());
|
||||
|
||||
// One loop guard per local AI request (see `Studio/LoopGuard.js`): it counts
|
||||
// consecutive identical calls, consecutive failures and turns, so a local
|
||||
// agent that loops cannot run forever.
|
||||
const loopGuardsRef = React.useRef<Map<string, any>>(new Map());
|
||||
|
||||
// Forget all previously-granted edit approvals so the next modifying call
|
||||
// asks again. Called when the user toggles auto-edit: turning it on then off
|
||||
// again means they want to review the upcoming edits, even within a sub-agent
|
||||
@@ -541,6 +575,142 @@ export const useProcessFunctionCalls = ({
|
||||
}))
|
||||
);
|
||||
|
||||
// The local loop guard: with no server under BYOK, nothing else stops an
|
||||
// agent looping on fresh call ids. Only recorded for local requests - the
|
||||
// hosted backend enforces its own `repeated-tool-call-loop` error, and a
|
||||
// second client-side trip over the same condition would double-report.
|
||||
if (isCustomEndpointEnabled()) {
|
||||
const guard = createLoopGuard();
|
||||
loopGuardsRef.current.set(aiRequest.id, guard);
|
||||
functionCallsToProcess.forEach(functionCall => {
|
||||
guard.recordToolCall({
|
||||
name: functionCall.name,
|
||||
arguments: functionCall.arguments,
|
||||
});
|
||||
});
|
||||
guard.recordTurn();
|
||||
|
||||
const decision = guard.evaluate();
|
||||
if (decision.action === 'steer' || decision.action === 'constrain') {
|
||||
const loopGuardMessage = `The local agent looks stuck (${
|
||||
decision.reason
|
||||
}). Stop repeating this action and change approach, or report the problem.`;
|
||||
addEditorFunctionCallResults(aiRequest.id, [
|
||||
{
|
||||
status: 'finished',
|
||||
// A synthetic id: the model sees the warning in its transcript
|
||||
// as the output of a call it did not make, which is enough for
|
||||
// it to change course.
|
||||
call_id: `loop-guard-${aiRequest.id}-${Date.now()}`,
|
||||
success: false,
|
||||
output: { message: loopGuardMessage },
|
||||
},
|
||||
]);
|
||||
}
|
||||
if (decision.action === 'constrain') {
|
||||
// Stop issuing further turns for this request. The request is marked
|
||||
// as failed with the error code the existing error UI already
|
||||
// classifies as "The AI got stuck" (`AiRequestErrorRow.js`), so no
|
||||
// new UI is needed for it.
|
||||
console.warn(
|
||||
`[studio] loop guard constrained the local request ${
|
||||
aiRequest.id
|
||||
}: ${decision.reason}`
|
||||
);
|
||||
updateAiRequest(aiRequest.id, currentRequest => ({
|
||||
...(currentRequest || {
|
||||
id: aiRequest.id,
|
||||
createdAt: new Date().toISOString(),
|
||||
userId: 'local-byok-user',
|
||||
error: null,
|
||||
output: [],
|
||||
}),
|
||||
status: 'error',
|
||||
updatedAt: new Date().toISOString(),
|
||||
error: {
|
||||
code: 'repeated-tool-call-loop',
|
||||
message: `The local agent was stopped after ${decision.reason}.`,
|
||||
},
|
||||
}));
|
||||
try {
|
||||
await suspendAiRequest(aiRequest.id);
|
||||
} catch (error) {
|
||||
console.error(
|
||||
'Error while suspending a request stopped by the loop guard:',
|
||||
error
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// The local studio's delegation tool is not an editor function: it creates
|
||||
// a child AI request and stamps the call so the shared loop stops treating
|
||||
// it as a call for the editor to execute (`getFunctionCallsToProcess`
|
||||
// skips calls carrying a `subAgentAiRequestId`). Only under BYOK: the
|
||||
// hosted backend never emits this tool name, so hosted behaviour is
|
||||
// untouched and this pre-pass is skipped entirely.
|
||||
const spawnCalls = isCustomEndpointEnabled()
|
||||
? functionCallsToProcess.filter(isSpawnAgentCall)
|
||||
: [];
|
||||
const spawnHandledCallIds = new Set<string>();
|
||||
if (spawnCalls.length > 0) {
|
||||
for (const spawnCall of spawnCalls) {
|
||||
let spawnResult;
|
||||
try {
|
||||
spawnResult = await spawnSubAgent({
|
||||
parentAiRequestId: aiRequest.id,
|
||||
functionCall: spawnCall,
|
||||
gameProjectJson: aiRequest.gameProjectJson || null,
|
||||
projectSpecificExtensionsSummaryJson: null,
|
||||
project,
|
||||
onStamped: subAgentAiRequestId => {
|
||||
// Stamp the call in place so the shared loop skips it from here
|
||||
// on, then activate the child so AiRequestContext polls it and
|
||||
// the editor processes its own function calls.
|
||||
spawnCall.subAgentAiRequestId = subAgentAiRequestId;
|
||||
activateSubAgent(
|
||||
subAgentAiRequestId,
|
||||
aiRequest.id,
|
||||
spawnCall.call_id
|
||||
);
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Error while spawning a studio sub-agent:', error);
|
||||
spawnResult = { error: 'The sub-agent could not be started.' };
|
||||
}
|
||||
|
||||
spawnHandledCallIds.add(spawnCall.call_id);
|
||||
if (spawnResult.error) {
|
||||
// Surface it as a failed call the model sees, rather than a crash.
|
||||
addEditorFunctionCallResults(aiRequest.id, [
|
||||
{
|
||||
status: 'finished',
|
||||
call_id: spawnCall.call_id,
|
||||
success: false,
|
||||
output: { message: spawnResult.error },
|
||||
},
|
||||
]);
|
||||
}
|
||||
}
|
||||
}
|
||||
const callsForEditor = spawnHandledCallIds.size
|
||||
? functionCallsToProcess.filter(
|
||||
functionCall => !spawnHandledCallIds.has(functionCall.call_id)
|
||||
)
|
||||
: functionCallsToProcess;
|
||||
if (callsForEditor.length === 0) {
|
||||
// Nothing left for the editor: the spawn results were already recorded,
|
||||
// and the parent's next turn is driven by the sub-agent reporting back.
|
||||
callsForEditor.forEach(functionCall => {
|
||||
inFlightFunctionCallIdsRef.current.delete(
|
||||
`${aiRequest.id}:${functionCall.call_id}`
|
||||
);
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
// The "modified outside editor" callbacks each refresh the editor and can
|
||||
// trigger an in-game editor hot reload. Firing them once per function
|
||||
// call would, for a batch of modifying calls (e.g. a sub-agent adding 20
|
||||
@@ -643,7 +813,7 @@ export const useProcessFunctionCalls = ({
|
||||
// Explorer sub-agent scripts are read-only: restrict their
|
||||
// `run_script` to non-mutating functions (defense in depth).
|
||||
runScriptReadOnly: isReadOnlyScriptContext,
|
||||
functionCalls: functionCallsToProcess.map(functionCall => ({
|
||||
functionCalls: callsForEditor.map(functionCall => ({
|
||||
name: functionCall.name,
|
||||
arguments: functionCall.arguments,
|
||||
call_id: functionCall.call_id,
|
||||
@@ -731,7 +901,41 @@ export const useProcessFunctionCalls = ({
|
||||
|
||||
const newResults = addEditorFunctionCallResults(aiRequest.id, results);
|
||||
|
||||
await onSendEditorFunctionCallResults(aiRequest.id, newResults, {
|
||||
// A plan written by the plan tool is folded over the plan the request
|
||||
// already has, matching tasks by id: the tool sends the tasks it knows
|
||||
// about, so a field another writer set (the studio's `agentCallId` when
|
||||
// it delegates a task, Phase 5) must survive. Without this, re-planning
|
||||
// would silently strip it.
|
||||
const mergedResults = newResults.map(result => {
|
||||
if (result.status !== 'finished' || !result.success) return result;
|
||||
let output;
|
||||
try {
|
||||
output = JSON.parse(result.output);
|
||||
} catch (error) {
|
||||
return result;
|
||||
}
|
||||
if (!output || !output.plan || !Array.isArray(output.plan.tasks)) {
|
||||
return result;
|
||||
}
|
||||
const existingPlan = getLatestActivePlan(aiRequest);
|
||||
const existingTasks = existingPlan ? existingPlan.tasks : null;
|
||||
if (!existingTasks) return result;
|
||||
return {
|
||||
...result,
|
||||
output: JSON.stringify({
|
||||
...output,
|
||||
plan: {
|
||||
tasks: mergePlanTasks(
|
||||
// $FlowFixMe[incompatible-type]
|
||||
existingTasks,
|
||||
output.plan.tasks
|
||||
),
|
||||
},
|
||||
}),
|
||||
};
|
||||
});
|
||||
|
||||
await onSendEditorFunctionCallResults(aiRequest.id, mergedResults, {
|
||||
createdSceneNames,
|
||||
createdExternalLayoutNames,
|
||||
createdProject,
|
||||
@@ -756,7 +960,7 @@ export const useProcessFunctionCalls = ({
|
||||
|
||||
// Release the lock so these calls can be retried if needed
|
||||
// (e.g. after an error or a suspension).
|
||||
functionCallsToProcess.forEach(functionCall => {
|
||||
callsForEditor.forEach(functionCall => {
|
||||
inFlightFunctionCallIdsRef.current.delete(
|
||||
`${aiRequest.id}:${functionCall.call_id}`
|
||||
);
|
||||
@@ -792,6 +996,8 @@ export const useProcessFunctionCalls = ({
|
||||
getIsAutoEditEnabled,
|
||||
suspendAiRequest,
|
||||
requestEditApproval,
|
||||
activateSubAgent,
|
||||
updateAiRequest,
|
||||
]
|
||||
);
|
||||
|
||||
@@ -835,9 +1041,50 @@ export const useProcessFunctionCalls = ({
|
||||
[onProcessFunctionCalls, allFunctionCallsToProcess]
|
||||
);
|
||||
|
||||
// One write gate per hook instance: the studio's writes to a request go
|
||||
// through it, so a write cannot land in the middle of another one for the
|
||||
// same request (see `Studio/RequestWriteGate.js`).
|
||||
const requestWriteQueueRef = React.useRef<Object | null>(null);
|
||||
if (!requestWriteQueueRef.current) {
|
||||
requestWriteQueueRef.current = createRequestWriteQueue();
|
||||
}
|
||||
|
||||
/**
|
||||
* Run `write` once the request is free, through the gate. The reason a write
|
||||
* waited is logged, which is the whole point of naming the three block
|
||||
* reasons rather than collapsing them to a boolean.
|
||||
*/
|
||||
const enqueueRequestWrite = React.useCallback(
|
||||
(aiRequestId: string, write: () => Promise<void>): Promise<void> => {
|
||||
const queue = requestWriteQueueRef.current;
|
||||
if (!queue) return write();
|
||||
return queue.enqueue(aiRequestId, write, () => {
|
||||
const block = getRequestWriteBlock({
|
||||
aiRequestId,
|
||||
request: (aiRequestsRef.current[aiRequestId]: any) || null,
|
||||
isSending: isSendingAiRequest(aiRequestId),
|
||||
inFlightCallIds: inFlightFunctionCallIdsRef.current,
|
||||
});
|
||||
if (block) {
|
||||
console.info(
|
||||
`[studio] deferred a write to ${aiRequestId}: ${block}.`
|
||||
);
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
});
|
||||
},
|
||||
// `isSendingAiRequest` is read through the latest-render closure: the gate
|
||||
// must see the current value at write time, not the one captured when the
|
||||
// callback was created, so it is deliberately not a dependency.
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
[]
|
||||
);
|
||||
|
||||
return {
|
||||
onProcessFunctionCalls,
|
||||
clearApprovedEditBatches,
|
||||
enqueueRequestWrite,
|
||||
};
|
||||
};
|
||||
|
||||
|
||||
@@ -26,6 +26,9 @@ export const NON_SCRIPTABLE_FUNCTION_NAMES: Set<string> = new Set([
|
||||
'report_fulfilment_problem',
|
||||
'run_edit_agent',
|
||||
'run_explorer_agent',
|
||||
// The local studio's delegation tool: a script must not spawn agents (they
|
||||
// would run outside the script's sequential-call contract).
|
||||
'spawn_agent',
|
||||
'get_game_starter_summary',
|
||||
// Events are generated by the orchestrator (`generate_events`), never inside
|
||||
// an edit/explorer script (v1 scope).
|
||||
|
||||
@@ -19,18 +19,49 @@ describe('server-side handled tools', () => {
|
||||
project.delete();
|
||||
});
|
||||
|
||||
const launch = (functionName: string) =>
|
||||
const launch = (functionName: string, args: Object = {}) =>
|
||||
editorFunctions[functionName].launchFunction({
|
||||
...makeFakeLaunchFunctionOptionsWithProject(project),
|
||||
args: {},
|
||||
args,
|
||||
});
|
||||
|
||||
it('create_or_update_plan fails as it is handled server-side', async () => {
|
||||
const result = await launch('create_or_update_plan');
|
||||
expect(result.success).toBe(false);
|
||||
expect(result.message).toBe(
|
||||
'Unable to create or update plan - this is handled server-side.'
|
||||
it('create_or_update_plan builds the plan locally', async () => {
|
||||
const result = await launch('create_or_update_plan', {
|
||||
tasks: [
|
||||
{
|
||||
id: 'gdd',
|
||||
title: 'Write the GDD',
|
||||
description: 'Sections: overview, economy.',
|
||||
status: 'pending',
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(result.success).toBe(true);
|
||||
expect((result: any).plan).toEqual({
|
||||
tasks: [
|
||||
{
|
||||
id: 'gdd',
|
||||
title: 'Write the GDD',
|
||||
description: 'Sections: overview, economy.',
|
||||
status: 'pending',
|
||||
dependsOn: [],
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it('create_or_update_plan rejects a bad tasks argument instead of throwing', async () => {
|
||||
const missingTasks = await launch('create_or_update_plan');
|
||||
expect(missingTasks.success).toBe(false);
|
||||
expect(missingTasks.message).toBe(
|
||||
'create_or_update_plan requires a tasks array.'
|
||||
);
|
||||
|
||||
const badTask = await launch('create_or_update_plan', {
|
||||
tasks: [{ id: 'a', title: '', description: 'd', status: 'pending' }],
|
||||
});
|
||||
expect(badTask.success).toBe(false);
|
||||
expect(badTask.message).toContain('index 0');
|
||||
});
|
||||
|
||||
it('report_fulfilment_problem fails as it is handled server-side', async () => {
|
||||
|
||||
@@ -158,6 +158,10 @@ import {
|
||||
} from './Extensions/CustomFunctionFunctions';
|
||||
import { capScriptExecutionResult } from './ScriptExecution/CapScriptOutput';
|
||||
import { isNoOpConsideredSuccess } from './IsNoOpConsideredSuccess';
|
||||
import {
|
||||
buildPlanOutput,
|
||||
validatePlanTasks,
|
||||
} from '../AiGeneration/Studio/PlanStore';
|
||||
import {
|
||||
inspectExtension,
|
||||
type InspectedCustomBehavior,
|
||||
@@ -10550,12 +10554,19 @@ const inspectVariables: EditorFunction = {
|
||||
};
|
||||
|
||||
const createOrUpdatePlan: EditorFunction = {
|
||||
// No renderForEditor: handled server-side and shown separately via the
|
||||
// OrchestratorPlan component, so nothing to render as a function call.
|
||||
// No renderForEditor: the plan is shown separately via the OrchestratorPlan
|
||||
// component, so nothing to render as a function call.
|
||||
//
|
||||
// Under BYOK there is no server to build the plan, so the editor does it: the
|
||||
// output is the exact shape `getLatestActivePlan` parses (`output.plan.tasks`)
|
||||
// and `OrchestratorPlan` renders, which is what the hosted backend returns for
|
||||
// the same tool.
|
||||
launchFunction: async ({ args }) => {
|
||||
return makeGenericFailure(
|
||||
`Unable to create or update plan - this is handled server-side.`
|
||||
);
|
||||
const validation = validatePlanTasks(args ? args.tasks : undefined);
|
||||
if (!validation.success) {
|
||||
return makeGenericFailure(validation.message);
|
||||
}
|
||||
return buildPlanOutput(validation.tasks);
|
||||
},
|
||||
modifiesProject: false,
|
||||
};
|
||||
@@ -10782,6 +10793,35 @@ const runEditAgent: EditorFunction = {
|
||||
modifiesProject: true,
|
||||
};
|
||||
|
||||
/**
|
||||
* The local studio's delegation tool.
|
||||
*
|
||||
* Only the BYOK runtime handles this call, before `processEditorFunctionCalls`
|
||||
* runs: it creates a child AI request with the role's prompt and tool subset
|
||||
* (see `AiGeneration/Studio/SpawnSubAgents.js`). Reaching the registry means the
|
||||
* local endpoint is off, or a `run_script` tried to call it - both are failures,
|
||||
* not silent no-ops.
|
||||
*/
|
||||
const spawnAgent: EditorFunction = {
|
||||
renderForEditor: ({ args }) => {
|
||||
const shortTitle = SafeExtractor.extractStringProperty(args, 'short_title');
|
||||
if (shortTitle && shortTitle.trim()) {
|
||||
return {
|
||||
text: truncateSubAgentTitleByWords(shortTitle),
|
||||
};
|
||||
}
|
||||
return {
|
||||
text: <Trans>Delegating to a studio agent.</Trans>,
|
||||
};
|
||||
},
|
||||
launchFunction: async ({ args }) => {
|
||||
return makeGenericFailure(
|
||||
'spawn_agent is handled by the local studio runtime.'
|
||||
);
|
||||
},
|
||||
modifiesProject: true,
|
||||
};
|
||||
|
||||
const runTests: EditorFunction = {
|
||||
renderForEditor: ({ args }) => {
|
||||
const newTest = SafeExtractor.extractObjectProperty(args, 'new_test');
|
||||
@@ -11032,6 +11072,7 @@ export const editorFunctions: { [string]: EditorFunction } = {
|
||||
|
||||
run_explorer_agent: runExplorerAgent,
|
||||
run_edit_agent: runEditAgent,
|
||||
spawn_agent: spawnAgent,
|
||||
run_tests: runTests,
|
||||
run_gameplay_test: runGameplayTest,
|
||||
change_gameplay_tests: changeGameplayTests,
|
||||
|
||||
Reference in New Issue
Block a user