commit f1290982c18d349481bfb362dc42d40df0d019c2 Author: hermes Date: Tue Oct 27 18:53:18 2026 -0900 chore: orchestration-ready layout (docs/, scripts/, README, AGENTS) diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..ed3d742 --- /dev/null +++ b/.gitignore @@ -0,0 +1,7 @@ +build/ +Library/ +*.user +bin/ +obj/ +.vs/ +.idea/ diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..97f251c --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,20 @@ +# AGENTS.md — game-engine + +## Invariants +- Native owns all frame state (`native/`). Managed (`managed/`) holds no gameplay state; zero-copy only via `EcsChunkView*` + `[UnmanagedCallersOnly]` + `SuppressGCTransition`. +- Every change to `native/bridge/engine_abi.h` must regenerate `managed/Engine.Bindings/NativeApi.g.cs` via `tools/abi_codegen` and run layout assertions. +- No `malloc`/`new` on frame path: `LinearFrameAllocator` (reset/frame) + `PageArenaAllocator` (virtual memory). Flecs columns `alignas(64)`. +- `IComponent` structs are blittable only (no `string`/`class`/`interface`/arrays/LINQ/closures in hot path). Analyzers ENG0001–ENG0003 enforce. +- Hot-reload is stateless: component data stays in Flecs; `GameScripts.dll` loads from memory stream via collectible ALC; schema evolution migrates tables. + +## Workflow (Hermes orchestration) +- Board: `game-engine`. One task per issue `M1-01`…`M5-04`. Worker claims atomically, works in `hermes -w` worktree, opens PR, `qa-gate` reviews. +- Profiles: `chief-architect` (routes + gates), `cpp-ecs-core`, `dotnet-roslyn-bridge`, `vulkan-renderer`, `editor-cli-ux`, `qa-gate` (adversarial verify, must pass CMake + `dotnet test` + Lavapipe headless before merge). +- Memory: record ABI breaks, Flecs/Jolt gotchas, failed-build postmortems. Prefer `flecs-jolt-patterns`, `zero-copy-abi-guard`, `roslyn-incremental-gen` skills. + +## Verification gates (definition of done per milestone) +- M1: Benchmark1M <1.5ms, `GC.GetAllocatedBytesForCurrentThread()==0`. +- M2: edit-save → 100k entities change behavior <100ms, no state loss, exceptions fault-isolate. +- M3: `vulkaninfo` + Lavapipe headless pass; 250k draws via `vkCmdDrawIndexedIndirectCount`, <0.5ms CPU render thread. +- M4: 30k bodies locked 60 FPS, `Physics.RaycastBatch` parallel, collision ring-buffer → `ReadOnlySpan`. +- M5: drop `.glb`/`.png` → visible <16ms/1 frame; `engine-cli scene tree|entity set|snapshot capture|profile --frames 60`; `.pak` + `dotnet publish -r /p:PublishAot=true`, cold boot <1.0s. diff --git a/README.md b/README.md new file mode 100644 index 0000000..85619e5 --- /dev/null +++ b/README.md @@ -0,0 +1,41 @@ +# Game Engine — Native C++20 + .NET 10 + +100% world state in native C++20 (Flecs / Jolt / Vulkan 1.3 / miniaudio), 0% gameplay state on the .NET GC heap. `.NET 10` (`hostfxr`) is a stateless logic runner mutating native memory zero-copy via `Span` + `delegate* unmanaged[SuppressGCTransition]`. + +End users write Unity-like `IComponent` scripts. Roslyn Incremental Source Generators compile them to burst-vectorized native chunk loops. Compile-time analyzers forbid hot-path allocations. + +## Layout + +```text +engine-root/ +├── docs/ # Architecture roadmap + M1–M5 milestones + orchestration plan +├── scripts/ # bootstrap-github-issues.sh (labels, milestones, issues) +├── native/ # libengine_core (C++20): core, ecs, renderer, physics, audio, bridge +├── managed/ # Engine.Bindings / Runtime / Analyzers / Editor / Cli +├── tools/ # abi_codegen (bridge JSON -> C# bindings) +└── samples/ # Benchmark1M, stress scenes, FPS template +``` + +## Milestones & Gates + +| Milestone | Gate | +|---|---| +| M1 Spine (native ECS + zero-copy host) | 1M `LocalTransform` from C# in <1.5ms, 0 GC bytes | +| M2 Illusion compiler (Roslyn + hot-reload) | <100ms state-preserving C# hot-reload | +| M3 GPU-driven Vulkan 1.3 renderer | 250k PBR meshes + SDF GI @120+ FPS, <0.5ms CPU | +| M4 Jolt physics + spatial audio | 30k rigidbodies + zero-alloc C# callbacks @60+ FPS | +| M5 Editor UX + asset pipeline + CLI | <16ms asset streaming, `engine-cli` IPC, 1-click NativeAOT export | + +## Orchestration (Hermes-native) + +- 6 profiles: `chief-architect`, `cpp-ecs-core`, `dotnet-roslyn-bridge`, `vulkan-renderer`, `editor-cli-ux`, `qa-gate` +- Work queue: `hermes kanban` board `game-engine` (tasks `M1-01`…`M5-04`) +- Memory: `lore` active; `mem0` OSS optional on `https://llm.heretek.one/v1` (`lemonade/embeddinggemma-2`, 768 dims) +- Bootstrap GitHub: `./scripts/bootstrap-github-issues.sh` + +## Start (M1-01) + +```bash +hermes kanban list --board game-engine +hermes -p cpp-ecs-core -w # isolated worktree mode for code tasks +``` diff --git a/docs/00-ARCHITECTURE-ROADMAP.md b/docs/00-ARCHITECTURE-ROADMAP.md new file mode 100644 index 0000000..50bc8d2 --- /dev/null +++ b/docs/00-ARCHITECTURE-ROADMAP.md @@ -0,0 +1,67 @@ +# Architecture & Master Roadmap: Native Core + Zero-Copy .NET 10 Engine + +## 1. Executive Summary & Core Thesis +This engine is architected to surpass **Unreal Engine 5** in deterministic CPU/GPU frame times and entity scale while matching or exceeding **Unity** in C# developer ergonomics and iteration speed. + +### The Core Architectural Split +1. **100% of World State in Native C++20 (`native/`):** All ECS component arrays (Flecs v4), physics islands (Jolt), GPU command buffers (Vulkan 1.3), and audio streams (`miniaudio`) reside in 64-byte cache-line-aligned native memory. +2. **0% of Gameplay State on the .NET GC Heap (`managed/`):** The embedded `.NET 10` (`hostfxr`) runtime acts as a stateless logic runner and Editor UI host. It mutates native C++ memory directly in-place via zero-copy `Span` views and `delegate* unmanaged[SuppressGCTransition]` function pointer tables. +3. **The Illusion Layer (`Engine.Analyzers`):** End users write intuitive, Unity-like `IComponent` scripts. Roslyn Incremental Source Generators secretly compile those methods into burst-vectorized, multithreaded native chunk loops at build time while compile-time analyzers forbid heap allocations in hot paths. + +--- + +## 2. Monorepo Topology + +```text +engine-root/ +├── CMakeLists.txt # Master C++20 build (Ninja + Clang/MSVC) +├── Engine.slnx # .NET 10 Solution for Host, Editor, Analyzers, CLI +├── Directory.Build.props # Strict C# flags, AllowUnsafeBlocks, Zero-Warn +├── deps/ # Vendored/Submodule Native Dependencies +│ ├── flecs/ # v4.x Archetype/Sparse ECS (C99/C++) +│ ├── JoltPhysics/ # Multi-core deterministic physics +│ ├── volk/ & VulkanMemoryAllocator/ # Vulkan 1.3 loader + VMA +│ ├── slang/ # Shader compiler (SPIR-V / DXIL / MSL) +│ └── miniaudio/ # Low-latency native audio +├── native/ # THE NATIVE CORE (libengine_core.so / .dll) +│ ├── core/ # Arenas, SIMD math, SDL3 windowing, JobScheduler +│ ├── ecs/ # Flecs world wrapper, hierarchy/prefab bridges +│ ├── renderer/ # Vulkan 1.3 GPU-driven bindless renderer & FrameGraph +│ ├── physics/ # Jolt <-> Flecs transform sync & BroadPhase layers +│ ├── audio/ # miniaudio spatial graph +│ └── bridge/ # C99 ABI boundary + JSON schema dumper +├── tools/ +│ └── abi_codegen/ # Parses bridge JSON -> emits C# unmanaged structs/ptrs +├── managed/ # THE .NET 10 HOST & ILLUSION LAYER +│ ├── Engine.Bindings/ # Auto-generated SuppressGCTransition P/Invoke tables +│ ├── Engine.Runtime/ # Stateless ALC host, Component base, Query, Time +│ ├── Engine.Analyzers/ # Roslyn Source Generators (OOP -> ECS) & GC Guardrails +│ ├── Engine.Editor/ # ImGui/Docking Viewport, AssetDB, Prefabs, Undo/Redo +│ │ ├── AssetsDatabase/ # Async background BC7/glTF streaming + .meta files +│ │ └── Cli/ # Built-in IPC/JSON-RPC server for AI/CI automation +│ └── Engine.Cli/ # Standalone terminal client +└── samples/ # Validation scenes (1M Entity Stress Test, FPS Template) +``` + +--- + +## 3. Milestone Overview & Verification Gates + +| Milestone | Document | Target Timeline | Verification Gate ("Definition of Done") | +| :--- | :--- | :--- | :--- | +| **M1: The Spine** | `docs/01-MILESTONE-1-THE-SPINE.md` | Months 1–2 | C# script mutates 1,000,000 `LocalTransform` structs in native Flecs memory in $< 1.5\text{ ms}$ with $0\text{ bytes}$ of GC allocation. | +| **M2: The Illusion Compiler** | `docs/02-MILESTONE-2-ILLUSION-COMPILER.md` | Months 3–4 | Edit a C# movement script, hit save, and see 100,000 moving entities change direction in $< 100\text{ ms}$ without losing velocity state. | +| **M3: GPU-Driven Renderer** | `docs/03-MILESTONE-3-VULKAN-RENDERER.md` | Months 5–7 | Render 250,000 unique dynamic meshes with PBR materials and real-time GI at $120+\text{ FPS}$ with $< 0.5\text{ ms}$ CPU render thread time. | +| **M4: Physics & Audio** | `docs/04-MILESTONE-4-PHYSICS-AND-AUDIO.md` | Months 8–9 | Spawn 30,000 colliding rigidbodies in a pit with active C# collision callbacks running at a locked $60+\text{ FPS}$. | +| **M5: Editor UX & CLI** | `docs/05-MILESTONE-5-EDITOR-AND-CLI.md` | Months 10–12 | Build and export a standalone, NativeAOT-compiled 3D FPS playable demo from the Editor in one click, fully controllable via `engine-cli`. | + +--- + +## 4. Standard GitHub Labels +Use the accompanying `scripts/bootstrap-github-issues.sh` script to automatically create these labels in your repository: +* `area:native-core` (`#d73a4a`) — C++20 memory, job system, Flecs ECS, OS abstraction +* `area:abi-bridge` (`#0075ca`) — `hostfxr`, C99 ABI headers, `abi_codegen`, zero-copy marshaling +* `area:roslyn-illusion` (`#a2eeef`) — Roslyn Incremental Source Generators & GC Diagnostic Analyzers +* `area:renderer` (`#7057ff`) — Vulkan 1.3 bindless, Slang shaders, GPU culling, SDF GI +* `area:physics-audio` (`#008672`) — Jolt Physics & `miniaudio` integration +* `area:editor-cli` (`#e4e669`) — Asset pipeline, Editor UI, Prefabs, IPC/CLI protocol \ No newline at end of file diff --git a/docs/01-MILESTONE-1-THE-SPINE.md b/docs/01-MILESTONE-1-THE-SPINE.md new file mode 100644 index 0000000..d3048ba --- /dev/null +++ b/docs/01-MILESTONE-1-THE-SPINE.md @@ -0,0 +1,58 @@ +# Milestone 1: The Spine (Native ECS + Zero-Copy .NET 10 Host) + +* **Timeline:** Months 1–2 +* **Verification Gate:** A managed C# script iterates and mutates $1,000,000$ `LocalTransform` structs stored in native C++ Flecs tables in $< 1.5\text{ ms}$ across worker threads with $0\text{ bytes}$ of GC allocation. + +--- + +## GitHub Issues Breakdown + +### Issue M1-01: Initialize Monorepo, CMake C++20 Build, and `.slnx` Workspace +* **Labels:** `area:native-core`, `priority:critical` +* **Description:** Set up the unified build pipeline compiling `libengine_core` (`native/`) via CMake + Ninja + Clang/MSVC (C++20) alongside the `.NET 10` (`Engine.slnx`) managed solution. +* **Acceptance Criteria:** + - [ ] Configure root `CMakeLists.txt` with strict C++20 flags (`-fno-exceptions`, `-fno-rtti` in hot modules, `-mavx2`). + - [ ] Vendor `flecs` v4.x and `enkiTS` in `deps/`. + - [ ] Create `Engine.slnx` and `Directory.Build.props` targeting `net10.0` with `true` and `true`. + - [ ] Add GitHub Actions CI workflow verifying clean compilation on Windows, Linux, and macOS. + +--- + +### Issue M1-02: Native Memory Arenas & 64-Byte Cache-Aligned Allocators +* **Labels:** `area:native-core` +* **Description:** Eliminate general-purpose `malloc`/`new` from frame execution by implementing custom native allocators in `native/core/memory/`. +* **Acceptance Criteria:** + - [ ] Implement `LinearFrameAllocator` with atomic bump pointer that resets at every frame boundary. + - [ ] Implement `PageArenaAllocator` backed by OS virtual memory (`VirtualAlloc` / `mmap`). + - [ ] Override Flecs `ecs_os_api_t` memory hooks (`malloc_`, `calloc_`, `realloc_`, `free_`) to enforce 64-byte cache-line alignment (`alignas(64)`) on all component table columns. + +--- + +### Issue M1-03: Work-Stealing Job Scheduler & Flecs Hierarchy Pipeline +* **Labels:** `area:native-core` +* **Description:** Integrate a lock-free work-stealing thread pool (`enkiTS`) with $N - 1$ worker threads pinned to physical CPU cores and wire it into Flecs' staging pipeline. +* **Acceptance Criteria:** + - [ ] Bind `enkiTS` worker threads to Flecs' multi-threaded task API (`ecs_os_api.task_new_`). + - [ ] Implement core built-in native components: `LocalTransform` (`vec3` pos, `quat` rot, `vec3` scale) and `WorldTransform` (`mat4x4`). + - [ ] Implement breadth-first hierarchical transform propagation (`EcsCascade` over `EcsChildOf`) using SIMD `AVX2`/`NEON` matrix multiplication. + +--- + +### Issue M1-04: C99 ABI Contract (`engine_abi.h`) & Automated C# Binding Generator +* **Labels:** `area:abi-bridge`, `priority:critical` +* **Description:** Create the single-entry-point C99 ABI header (`engine_get_api_table()`) and a build tool (`tools/abi_codegen`) that emits zero-overhead C# function pointers. +* **Acceptance Criteria:** + - [ ] Define `EcsChunkView`, `ManagedSystemCallback`, and `EngineApiTable` in `native/bridge/engine_abi.h`. + - [ ] Build `tools/abi_codegen` to parse `engine_abi.h` and emit `managed/Engine.Bindings/NativeApi.g.cs`. + - [ ] Ensure all generated C# function pointers use `delegate* unmanaged[Cdecl, SuppressGCTransition]<...>` so P/Invoke calls compile to a single raw `call` instruction. + +--- + +### Issue M1-05: Embed `.NET 10` (`hostfxr`) & 1M Entity Zero-Copy Benchmark +* **Labels:** `area:abi-bridge`, `priority:critical` +* **Description:** Embed the CoreCLR runtime inside `libengine_core` via `hostfxr` and execute the Milestone 1 verification benchmark. +* **Acceptance Criteria:** + - [ ] Implement `DotNetHost` in `native/bridge/` using `hostfxr_initialize_for_runtime_config` and `load_assembly_and_get_function_pointer`. + - [ ] Pass raw Flecs table pointers (`EcsChunkView*`) from C++ worker threads into an `[UnmanagedCallersOnly(CallConvs = [typeof(CallConvCdecl)])]` C# callback. + - [ ] Create `samples/Benchmark1M/` spawning $1,000,000$ entities with `LocalTransform` and `Velocity`. + - [ ] Verify frame update completes in $< 1.5\text{ ms}$ with $0\text{ bytes}$ allocated on the managed heap (`GC.GetAllocatedBytesForCurrentThread() == 0`). \ No newline at end of file diff --git a/docs/02-MILESTONE-2-ILLUSION-COMPILER.md b/docs/02-MILESTONE-2-ILLUSION-COMPILER.md new file mode 100644 index 0000000..a6de09f --- /dev/null +++ b/docs/02-MILESTONE-2-ILLUSION-COMPILER.md @@ -0,0 +1,47 @@ +# Milestone 2: The Illusion Compiler (Roslyn Source Generators & 100ms Hot-Reload) + +* **Timeline:** Months 3–4 +* **Verification Gate:** Edit a C# gameplay script (`IComponent`), press `Ctrl+S`, and observe $100,000$ active entities update their behavior in $< 100\text{ ms}$ without losing world state or pausing the engine. + +--- + +## GitHub Issues Breakdown + +### Issue M2-01: Ergonomic `IComponent`, `Entity`, and Phase Attributes API +* **Labels:** `area:roslyn-illusion` +* **Description:** Design the user-facing C# scripting surface in `Engine.Runtime` so developers write intuitive per-entity methods without touching pointers or ECS queries. +* **Acceptance Criteria:** + - [ ] Define `public interface IComponent`, `readonly record struct Entity(ulong Id)`, and `[UpdateInPhase(Phase)]` attributes. + - [ ] Provide blittable collection & string types (`FixedString64`, `TempList` backed by the native `LinearFrameAllocator`, `AssetRef`). + - [ ] Expose SIMD-accelerated math types (`float3`, `float4`, `quaternion`, `float4x4`) mapped directly to `System.Numerics` / `System.Runtime.Intrinsics`. + +--- + +### Issue M2-02: Roslyn `IIncrementalGenerator` (`IComponent` to Native Chunk Job) +* **Labels:** `area:roslyn-illusion`, `priority:critical` +* **Description:** Build the Roslyn Source Generator in `Engine.Analyzers` that intercepts any `partial struct : IComponent` with a `Tick(...)` method and compiles it into a zero-copy native chunk runner. +* **Acceptance Criteria:** + - [ ] Parse `ref` (read-write) and `in` (read-only) component parameters on user `Tick()` methods. + - [ ] Emit auto-registration code that registers the struct size, alignment (`sizeof(T)`), and component dependencies with Flecs via `NativeApi.EcsRegisterComponent` and `NativeApi.EcsRegisterSystem`. + - [ ] Emit an `[UnmanagedCallersOnly]` static method that unpacks `EcsChunkView*` into `Span` / `ReadOnlySpan` slices and executes a tight, bounds-check-eliminated loop. + +--- + +### Issue M2-03: Compile-Time Performance & Memory Safety Roslyn Analyzers +* **Labels:** `area:roslyn-illusion` +* **Description:** Prevent GC spikes and memory corruption at compile time by flagging illegal C# patterns inside hot ECS systems as the user types in their IDE. +* **Acceptance Criteria:** + - [ ] **`ENG0001` (Heap Allocation in Hot Path):** Flag `new` class allocations, string interpolations, LINQ queries, and lambda closures inside `[UpdateInPhase]` methods. Provide a Roslyn CodeFix to replace `new List()` with `TempList`. + - [ ] **`ENG0002` (Non-Blittable Component Field):** Forbid managed reference types (`string`, `class`, `interface`, arrays) inside any struct implementing `IComponent`. + - [ ] **`ENG0003` (Blocking Async/Wait):** Forbid `.Wait()`, `.Result`, or `Thread.Sleep()` on engine threads. + +--- + +### Issue M2-04: Stateless Collectible `AssemblyLoadContext` 100ms Hot-Reload +* **Labels:** `area:abi-bridge`, `area:roslyn-illusion`, `priority:critical` +* **Description:** Implement indestructible live code reloading by isolating user gameplay scripts inside a collectible `AssemblyLoadContext` while keeping 100% of component data in native C++ Flecs memory. +* **Acceptance Criteria:** + - [ ] Implement `ScriptHostManager` in `Engine.Runtime` loading `GameScripts.dll` from an in-memory stream (preventing OS file locks on `.dll` / `.pdb` files). + - [ ] At the frame sync barrier, unbind old `ManagedSystemCallback` pointers in Flecs, unload the old `AssemblyLoadContext`, load the newly compiled assembly, and re-bind the updated `[UnmanagedCallersOnly]` callbacks. + - [ ] Support schema evolution: if a user adds or removes a field on an `IComponent` struct during hot-reload, automatically invoke Flecs component table migration so existing entity values remain intact. + - [ ] Wrap managed system invocations in a native/managed fault barrier: if a user script throws an exception, log the stack trace, disable that system, and keep the engine/editor running without crashing. \ No newline at end of file diff --git a/docs/03-MILESTONE-3-VULKAN-RENDERER.md b/docs/03-MILESTONE-3-VULKAN-RENDERER.md new file mode 100644 index 0000000..f7aeb3c --- /dev/null +++ b/docs/03-MILESTONE-3-VULKAN-RENDERER.md @@ -0,0 +1,46 @@ +# Milestone 3: Zero-Config GPU-Driven Vulkan 1.3 Renderer + +* **Timeline:** Months 5–7 +* **Verification Gate:** Render $250,000$ dynamic PBR mesh instances with cascaded/virtual shadows and real-time diffuse global illumination at $120+\text{ FPS}$ with $< 0.5\text{ ms}$ CPU render thread overhead. + +--- + +## GitHub Issues Breakdown + +### Issue M3-01: SDL3 Windowing, Vulkan 1.3 Bootstrap (`volk` + `VMA`), and Render Graph +* **Labels:** `area:renderer`, `priority:critical` +* **Description:** Initialize Vulkan 1.3 with dynamic rendering (`VK_KHR_dynamic_rendering`), synchronization2 (`VK_KHR_synchronization2`), `buffer_device_address`, and a declarative DAG Render Graph. +* **Acceptance Criteria:** + - [ ] Integrate SDL3 window creation and Vulkan surface management in `native/renderer/`. + - [ ] Initialize `volk` and `VulkanMemoryAllocator` (VMA) with dedicated ReBAR (Resizable BAR) ring buffers for fast CPU-to-GPU transform uploads. + - [ ] Build a declarative `RenderGraph` that automatically computes image layout transitions, memory aliasing, and pipeline barriers per frame. + +--- + +### Issue M3-02: Bindless Descriptor Table & GPU-Driven Indirect Drawing +* **Labels:** `area:renderer`, `priority:critical` +* **Description:** Eliminate CPU draw-call loops by storing all scene geometry, materials, and transforms in GPU SSBOs and rendering via `vkCmdDrawIndexedIndirectCount`. +* **Acceptance Criteria:** + - [ ] Create a single global bindless descriptor set (`VK_EXT_descriptor_indexing`) holding all textures, samplers, and material buffers. + - [ ] Stream dirty `WorldTransform` matrices from Flecs tables directly to a persistent GPU Scene SSBO at the end of every simulation frame. + - [ ] Implement a two-phase GPU Compute Culling pass (Frustum culling + Hierarchical-Z Pyramid occlusion culling) that emits draw commands to an indirect draw buffer. + +--- + +### Issue M3-03: Slang Shader Compiler Pipeline & C# `GraphicsCompositor` API +* **Labels:** `area:renderer`, `area:abi-bridge` +* **Description:** Integrate the **Slang** shader compiler for modular shaders and expose the `RenderGraph` to C# so users can write custom render passes easily. +* **Acceptance Criteria:** + - [ ] Embed `slang` in `native/renderer/shaders/` with automatic hot-reloading when `.slang` files are modified on disk. + - [ ] Implement a PBR Metallic-Roughness visibility buffer / forward+ shading pipeline. + - [ ] Expose `RenderPass` registration across the ABI bridge so C# developers can inject custom compute or post-processing passes into the frame graph. + +--- + +### Issue M3-04: Zero-Bake Real-Time GI (Signed Distance Fields + Radiance Cascades) +* **Labels:** `area:renderer` +* **Description:** Deliver "it just works" real-time 3D lighting out of the box without static lightmap baking progress bars. +* **Acceptance Criteria:** + - [ ] Generate sparse 3D Signed Distance Field (SDF) bricks for imported meshes on background worker threads. + - [ ] Implement Screen-Space + World-Space SDF Radiance Cascades for real-time multibounce diffuse global illumination and sky occlusion. + - [ ] Implement Virtual Shadow Maps (or SDSM) with soft contact shadows and ACES tonemapping. \ No newline at end of file diff --git a/docs/04-MILESTONE-4-PHYSICS-AND-AUDIO.md b/docs/04-MILESTONE-4-PHYSICS-AND-AUDIO.md new file mode 100644 index 0000000..555081e --- /dev/null +++ b/docs/04-MILESTONE-4-PHYSICS-AND-AUDIO.md @@ -0,0 +1,44 @@ +# Milestone 4: Deterministic Jolt Physics & Spatial Audio + +* **Timeline:** Months 8–9 +* **Verification Gate:** Simulate $30,000$ active colliding rigidbodies with zero-allocation C# collision callbacks and spatialized 3D audio running at a locked $60+\text{ FPS}$. + +--- + +## GitHub Issues Breakdown + +### Issue M4-01: Jolt Physics Integration & Shared Work-Stealing Thread Pool +* **Labels:** `area:physics-audio`, `priority:critical` +* **Description:** Vendor and integrate `JoltPhysics` in `native/physics/`, routing its job execution directly into the engine's `enkiTS` scheduler so ECS and Physics share the same worker threads. +* **Acceptance Criteria:** + - [ ] Implement `Jolt::JobSystem` wrapper backed by the engine's `enkiTS` scheduler. + - [ ] Configure `BroadPhaseLayerInterface` (`Static`, `Dynamic`, `Trigger`, `Debris`) and custom `TempAllocator` backed by the engine's `PageArenaAllocator`. + - [ ] Define native ECS components: `RigidBody`, `BoxCollider`, `SphereCollider`, `CapsuleCollider`, `MeshCollider`, and `CharacterController`. + +--- + +### Issue M4-02: Zero-Glue Parallel Flecs <-> Jolt Transform Synchronization +* **Labels:** `area:physics-audio`, `area:native-core` +* **Description:** Synchronize active rigidbody transforms between Jolt Physics islands and Flecs `LocalTransform` tables in parallel without per-object virtual calls. +* **Acceptance Criteria:** + - [ ] Before `PhysicsSystem::Update()`, push user-modified kinematic/dynamic transforms from Flecs to Jolt via `BodyInterface::SetPositionAndRotationWhenChanged`. + - [ ] After `PhysicsSystem::Update()`, iterate active (non-sleeping) Jolt bodies in parallel and write updated positions/rotations directly into Flecs `LocalTransform` columns. + - [ ] Buffer collision/trigger events (`OnCollisionEnter`, `OnCollisionExit`, `OnTriggerEnter`) into a lock-free native ring buffer and dispatch them to C# systems as contiguous `ReadOnlySpan` batches. + +--- + +### Issue M4-03: Zero-Cost C# Spatial Queries (`Physics.Raycast` / `OverlapSphere`) +* **Labels:** `area:physics-audio`, `area:abi-bridge` +* **Description:** Expose Jolt's spatial queries to C# via `[SuppressGCTransition]` unmanaged function pointers for sub-microsecond raycasting. +* **Acceptance Criteria:** + - [ ] Expose `Physics.Raycast`, `Physics.SphereCast`, and `Physics.OverlapSphere` in `Engine.Runtime`. + - [ ] Add batch raycast API (`Physics.RaycastBatch(ReadOnlySpan rays, Span hits)`) that executes across native worker threads in parallel. + +--- + +### Issue M4-04: Low-Latency 3D Spatial Audio Graph (`miniaudio`) +* **Labels:** `area:physics-audio` +* **Description:** Integrate `miniaudio` in `native/audio/` and bind `AudioSource` and `AudioListener` components to Flecs `WorldTransform` tables. +* **Acceptance Criteria:** + - [ ] Initialize `ma_engine` and `ma_resource_manager` for lock-free streaming of `.wav`, `.ogg`, and `.mp3` assets. + - [ ] Implement a native Flecs system that updates 3D spatialization cones, Doppler velocity, and attenuation from `WorldTransform` components every frame. \ No newline at end of file diff --git a/docs/05-MILESTONE-5-EDITOR-AND-CLI.md b/docs/05-MILESTONE-5-EDITOR-AND-CLI.md new file mode 100644 index 0000000..93f4733 --- /dev/null +++ b/docs/05-MILESTONE-5-EDITOR-AND-CLI.md @@ -0,0 +1,48 @@ +# Milestone 5: Effortless Editor UX, Instant Asset Streaming & Agent-Ready CLI + +* **Timeline:** Months 10–12 +* **Verification Gate:** Drop raw `.glb` and `.png` files into `Assets/` with $< 16\text{ ms}$ viewport appearance, inspect/mutate the live scene via `engine-cli`, and export a standalone NativeAOT-compiled 3D game in one click. + +--- + +## GitHub Issues Breakdown + +### Issue M5-01: Non-Blocking Background Asset Pipeline & `.meta` GUID Database +* **Labels:** `area:editor-cli`, `priority:critical` +* **Description:** Eliminate Unity-style "Importing Assets..." modal progress bars by streaming raw assets into the viewport immediately while compressing artifacts on background threads. +* **Acceptance Criteria:** + - [ ] Implement `AssetWatcher` and human-readable `.meta` sidecar files with stable 128-bit `GUID`s. + - [ ] Memory-map newly dropped `.glb`/`.gltf` and `.png`/`.jpg` files so uncompressed mipmaps appear in the Vulkan viewport within 1 frame ($< 16\text{ ms}$). + - [ ] Run background worker jobs using `meshoptimizer` (vertex cache + LOD generation) and `bc7enc`/`astcenc` (texture compression) to populate `Library/ArtifactCache/` asynchronously. + +--- + +### Issue M5-02: Native-Backed Editor UI (Hierarchy, Inspector, Prefabs, Gizmos) +* **Labels:** `area:editor-cli`, `priority:critical` +* **Description:** Build the docking Editor UI in `Engine.Editor` directly inspecting Flecs world tables and rendering the Vulkan 1.3 viewport with 3D transform gizmos. +* **Acceptance Criteria:** + - [ ] Implement Scene Hierarchy panel backed by Flecs `EcsChildOf` queries and `EcsIsA` Prefab instances. + - [ ] Implement Reflection/Source-Generated Inspector panel that edits native Flecs component memory in-place with full Command-pattern Undo/Redo support. + - [ ] Implement GPU Object ID picking and 3D Translate/Rotate/Scale viewport gizmos. + - [ ] Implement instant Play-In-Editor (PIE) mode using Flecs world snapshots (`ecs_world_snapshot`) so entering and exiting Play Mode takes $< 10\text{ ms}$. + +--- + +### Issue M5-03: First-Class IPC / JSON-RPC Server & `engine-cli` Control Plane +* **Labels:** `area:editor-cli` +* **Description:** Embed a high-speed Unix-Socket / Named-Pipe server inside the Editor so CI pipelines, terminal scripts, and AI coding agents can inspect and control the engine programmatically. +* **Acceptance Criteria:** + - [ ] Implement `CliServer` inside `Engine.Editor` and standalone `engine-cli` binary in `Engine.Cli`. + - [ ] Support `engine-cli scene tree` (JSON hierarchy dump) and `engine-cli entity set `. + - [ ] Support `engine-cli snapshot capture --out frame.png --pass [final|albedo|normals|depth|wireframe]` for automated visual regression testing and AI agent visual feedback. + - [ ] Support `engine-cli profile --frames 60` returning structured CPU/GPU pass timings and per-system ECS microsecond breakdowns. + +--- + +### Issue M5-04: One-Click Standalone NativeAOT Game Pak Exporter +* **Labels:** `area:editor-cli`, `area:abi-bridge` +* **Description:** Package the native C++ runtime, compiled Slang SPIR-V shaders, compressed `ArtifactCache` assets, and NativeAOT-compiled C# gameplay assemblies into a zero-dependency standalone executable. +* **Acceptance Criteria:** + - [ ] Pack all referenced assets by `GUID` into a memory-mappable `.pak` archive with LZ4/Zstd block compression. + - [ ] Invoke `dotnet publish -r /p:PublishAot=true` on the user's `GameScripts` project to compile all C# scripts directly to native machine code (or embed the stripped `hostfxr` runtime). + - [ ] Verify cold-boot time of the exported standalone game is $< 1.0\text{ second}$. \ No newline at end of file diff --git a/docs/06-ORCHESTRATION-PLAN.md b/docs/06-ORCHESTRATION-PLAN.md new file mode 100644 index 0000000..eac8710 --- /dev/null +++ b/docs/06-ORCHESTRATION-PLAN.md @@ -0,0 +1,176 @@ +To run an unattended, multi-week software factory inside a Proxmox LXC container building a hybrid **C++20 (Flecs/Jolt/Vulkan) \+ .NET 10 (Roslyn/Editor)** game engine powered by **Muse Spark 1.3 (muse-spark-1.3-contributor)** and **Unsloth embeddinggemma-2-GGUF**, every layer of your stack must be chosen based on how its actual source code behaves under autonomous load—not marketing claims. +Below is the fact-based architectural breakdown of the harnesses, the 5 orchestrators you listed, the memory layer, the LSP/AST MCP stack, and the exact LXC deployment blueprint. + +## **Part 1: The Harness Verdict (Hermes vs. Claude Code vs. Pi)** + +We are driving muse-spark-1.3-contributor (1,048,576 token context window, exposed at \[https\://api.meta.ai/v1\](https\://api.meta.ai/v1) with both OpenAI and Anthropic SDK drop-in compatibility). How do these three harnesses actually handle LSP, AST, and multi-hour unattended loops? + +| Feature / Mechanism | Pi (@mariozechner/pi-coding-agent) | Claude Code (@anthropic-ai/claude-code) | Hermes Agent (NousResearch/hermes-agent) | +| :---- | :---- | :---- | :---- | +| **LSP Integration** | **S-Tier (samfoy/pi-lsp-extension)**: Persistent background daemon (dist/lsp-daemon.js). **Auto-diagnostics hook**: Every time the agent runs write or edit, the extension intercepts the result and automatically appends up to 10 lines of compiler errors from the running LSP directly into the tool output. Zero wasted turns. | **B-Tier (Built-in since v2.0.74 / \$ENABLE\_LSP\_TOOL=1)**: Uses .lsp.json via claude-code-lsps marketplace. **Fatal Flaw**: Claude Code's hardcoded system prompt aggressively instructs the model to prefer Grep and Glob over LSP. In unattended runs, the model routinely ignores LSP tools unless forced via hooks. | **A-Tier (rewasa/hermes-code-intel-plugin)**: Python plugin (\~/.hermes/plugins/) providing 11 LSP tools (code\_definition, code\_callers, code\_impact, code\_diagnostics) with automatic fallback to Tree-sitter AST if a language server crashes. | +| **AST / Structural Refactor** | **Native in pi-lsp-extension**: Exposes ast\_search, code\_rewrite (dry-run by default), and code\_overview (Tree-sitter symbol map). | **None Native**: Requires external MCP (@notprolands/ast-grep-mcp or ast-grep/ast-grep-mcp) plus CLAUDE.md rules. | **Native Optional Skill \+ Plugin**: Ships with optional-skills/software-development/ast-grep/SKILL.md (with mechanical regex-validation helpers) \+ rewasa plugin's 8 Tree-sitter tools (code\_refactor, code\_capsule). | +| **Unattended Loop & Context** | **Deterministic & Hackable**: TypeScript core, zero bloat, pairs with clang.engineer/pi-workloop for bounded automatic continuation. | **Proprietary Heuristics**: Hardcoded Anthropic prompt-caching assumptions; when pointed at ANTHROPIC\_BASE\_URL=\[https\://api.meta.ai/v1\](https\://api.meta.ai/v1) for Muse Spark, system-prompt steering fights custom instructions. | **Built-in Learning Loop**: SQLite WAL \+ FTS5 session store, auto-compaction with Resolved/Pending state handoffs, ThreadPoolExecutor (8 parallel tool workers), and autonomous SKILL.md self-creation/curation (hermes curator). | +| **Subagent Spawning** | Handled via extensions or external orchestrator (openrig). | Native Task/Subagent tool, isolated context. | Built-in subagent delegation (max depth 2, 3 concurrent children, restricted toolsets). | + +### **The Harness Decision: Dual-Harness (Pi for Workers \+ Hermes for Lead/Curator)** + +> 1. **Use Pi (pi-coding-agent \+ samfoy/pi-lsp-extension) as your Primary Coding Worker Harness:** When writing C++20/Vulkan and C\#13 unsafe pointer code, **Pi's auto-diagnostics-on-edit** is unbeatable. The millisecond Pi writes a C++ or C\# file, clangd or csharp-ls validates it and injects any compiler error straight into the same tool turn. The agent fixes its own syntax/type errors before finishing its thought. +> 2. **Use Hermes Agent (v0.14.0 \+ rewasa/hermes-code-intel-plugin) as your Architect / QA / Memory Curator:** Hermes has a native closed learning loop (MEMORY.md, hermes curator, and code\_impact blast-radius analysis). When a worker finishes a complex milestone, Hermes audits the diff, extracts reusable engine patterns into new SKILL.md files, and prunes stale skills. +> 3. *(Optional)* Keep **Claude Code** in the container strictly as a fallback adapter for orchestrators that default to it, pointed at Muse Spark via ANTHROPIC\_BASE\_URL=\[https\://api.meta.ai/v1\](https\://api.meta.ai/v1). + +## **Part 2: The Orchestrator Showdown (Hard Facts on the 5 Repos)** + +You listed five specific orchestrators. You need **agents driving agents**, **Kanban visibility**, **isolated Git worktrees**, and **non-stop execution inside an LXC container** until all 24 GitHub issues across Milestones 1–5 are merged and verified. + +| Orchestrator Repo | Stack & Architecture | Native Harnesses | How It Actually Works Under the Hood | Fatal Flaw for Unattended LXC | +| :---- | :---- | :---- | :---- | :---- | +| **1\. agent-of-empires/agent-of-empires (AoE v1.21.0)** | Rust (79%) \+ TypeScript, tmux session manager \+ Web UI | Claude Code, OpenCode, Pi.dev, Gemini CLI, Codex | Wraps each agent in a persistent tmux session \+ Git worktree. Great mobile web UI and status detection (working, waiting, idle). | **No Autonomous Loop:** AoE is a *human-driven multiplexer*. It does not decompose backlogs, route CI failures back to workers, or have agents drive other agents. | +| **2\. OrchestratorInc/agent-orchestrator** | Go daemon \+ Electron/React UI \+ Live Kanban | 30+ including **Claude Code, Pi, Muse, OpenCode 2** | Has a persistent "Project Orchestrator" agent that breaks plans into Kanban cards (Needs you, In review, Ready to merge), spawns workers in isolated Git worktrees, and routes PR/CI failures back to the worker. | **Desktop-First Coupling:** While a headless Docker workflow exists in CI, the primary supervisor UX is built around its local desktop app rather than a pure declarative server daemon. | +| **3\. sortie-ai/sortie (v1.30.0)** | Single Go binary (95% Go) \+ embedded SQLite (.sortie.db) \+ /ui/ Web Dashboard | Claude Code, OpenCode, Codex, Copilot, Kiro | Polls **GitHub Issues** (agent-ready label) via WORKFLOW.md (Go text/template). Clones worktree (after\_create), runs agent, opens PR (after\_run), swaps label to review. Has stall\_timeout\_ms (kills/requeues hung agents) and routes failing GitHub Actions CI checks back to the agent. | **Flat Ticket Runner:** Sortie is a relentless 1-ticket-to-1-PR state machine, but it has no concept of persistent multi-agent pods talking to each other (dev-cpp asking dev-csharp a question mid-task). | +| **4\. mvschwarz/openrig (v0.7.4)** | TypeScript/Hono daemon \+ SQLite \+ tmux \+ **Built-in MCP Server** | **Claude Code, Codex, Pi** | Declarative YAML RigSpec (pods, members, edges, CULTURE.md). Every agent gets a stable address (e.g., cpp-core@engine-rig). **Crucial Feature:** Exposes an MCP server (rig\_up, rig\_send, rig\_ps, rig\_snapshot\_create) *to the agents themselves*, so a Lead Agent can spawn, message, and inspect specialist agents over SQLite inbox/outbox queues. | **No Native GitHub Issue Poller / Business Kanban:** It excels at East-West inter-agent topology and tmux persistence, but relies on an operator or external trigger to feed high-level backlog items. | +| **5\. paperclipai/paperclip** | Node.js 24.11+ server \+ React UI \+ Postgres/SQLite | **Claude Code, Pi, Hermes, OpenCode, Codex**, HTTP/Process | **The "Virtual Company" Control Plane.** Organizes agents into an **Org Chart** (CEO \-\> Tech Lead \-\> Engineers) with **Atomic Task Checkout** (execution locks so two agents never claim the same ticket), **Heartbeat Loops** (HEARTBEAT.md wakes agents on a schedule to check their queue/subordinates), **Goal Ancestry**, and **Runtime Skill Injection**. | **High-Level Focus:** Manages the "Who, Why, and Task Lock" brilliantly, but doesn't manage low-level C++/C\# tmux sub-topologies as tightly as OpenRig. | + +### **The Ultimate Unattended Architecture: The 3-Layer "Russian Doll" Stack** + +Because **tokens are no issue** and you want this running inside an LXC container until the entire engine is built, **do not pick just one.** Combine the three that complement each other's exact mechanical boundaries: + +┌─────────────────────────────────────────────────────────────────────────────┐ +│ LAYER 1: MACRO GOVERNANCE & KANBAN — Paperclip (paperclipai/paperclip) │ +│ • Runs on Port 3000 (React Kanban \+ Org Chart \+ Heartbeat Scheduler) │ +│ • Holds the 5 Milestones & 24 Engine Tasks with Goal Ancestry │ +│ • Wakes the "Chief Architect" (Hermes Agent) every 10m via Heartbeat │ +└──────────────────────────────────────┬──────────────────────────────────────┘ + │ + ┌─────────────────────────────┴─────────────────────────────┐ + ▼ (Polls GitHub Issues / PRs) ▼ (Invokes \`rig send\` via MCP) +┌─────────────────────────────────────────┐ ┌─────────────────────────────────┐ +│ LAYER 2A: CI/PR WATCHDOG — Sortie │ │ LAYER 2B: SWARM TOPOLOGY — │ +│ (sortie-ai/sortie single Go binary) │ │ OpenRig (mvschwarz/openrig) │ +│ • Polls GitHub Issues labeled │ │ • Declarative YAML RigSpec │ +│ \`agent-ready\` │ │ • Persistent tmux seats \+ SQLite│ +│ • Enforces \`stall\_timeout\_ms: 300000\` │ │ Inbox/Outbox queues │ +│ • Automatically routes failing CMake / │ │ • Lead Agent uses OpenRig MCP │ +│ dotnet test CI logs back to the │ │ to drive 6 specialist Pi & │ +│ exact branch worker until green │ │ Hermes seats in parallel │ +└─────────────────────────────────────────┘ └─────────────────────────────────┘ + +> 1. **Paperclip (paperclipai/paperclip)** is your **Top-Level Kanban & Heartbeat Engine**: It guarantees the system *never stops*. Even if every terminal session finishes a turn, Paperclip's heartbeat scheduler wakes up your Chief-Architect (running **Hermes**) and Milestone-Lead (running **Pi**) every 5–10 minutes with their HEARTBEAT.md instructions: *"Check open tasks, review blocked PRs, verify Milestone gates, and dispatch next sub-tasks."* +> 2. **OpenRig (mvschwarz/openrig)** is your **Inter-Agent Mesh ("Agents Driving Agents")**: Your Milestone-Lead seat has OpenRig's MCP server (packages/cli/src/mcp-server.ts) attached. It uses rig send cpp-architect@engine-rig "..." and rig send roslyn-specialist@engine-rig "..." to coordinate cross-boundary work (like making sure engine\_abi.h in C++ matches NativeApi.g.cs in C\#). +> 3. **Sortie (sortie-ai/sortie)** is your **Relentless Issue-to-PR Execution Loop**: Whenever the Architect marks a GitHub Issue as agent-ready, Sortie atomically checks it out into an isolated Git worktree, spawns the worker, monitors stall\_timeout\_ms (if an agent hangs on a CMake build or segfault, Sortie kills and retries with exponential backoff), opens the PR, and feeds any failing GitHub Action / test output right back into the agent until the PR is mergeable. + +## **Part 3: Wiring the Memory Layer (unsloth/embeddinggemma-2-GGUF)** + +Let's look at the actual specs of **unsloth/embeddinggemma-2-GGUF** (released Oct 2026): + +* **Architecture:** 740M total parameters (or 270M text+code only sub-model), **8,192 token context window**, Matryoshka output dimensions (**768**, 512, 256, or 128). +* **Critical Requirement (Do Not Miss This):** EmbeddingGemma 2 requires **L2 normalization** and **explicit task prefixes** on inputs (SearchQuery prefix for queries, Document prefix for indexed code/memories) to hit peak retrieval accuracy. + +### **The 2-Tier Memory Architecture in Your LXC Container** + +Assuming your embeddinggemma-2-GGUF is served via llama-server (or Lemonade/LiteLLM) with \--embeddings \--pooling mean exposing an OpenAI-compatible /v1/embeddings endpoint at http\://embedding-host:8080/v1: + +> 1. **Semantic Codebase & Architecture Memory (Qdrant \+ Custom FastMCP Bridge):** + * Run a local **Qdrant** container/service (localhost:6333) inside the LXC. + * Why not standard mcp-server-qdrant? Because standard mcp-server-qdrant defaults to local FastEmbed CPU models (all-MiniLM-L6-v2) and doesn't inject EmbeddingGemma-2's mandatory query/document prefixes. + * Instead, run a 60-line Python FastMCP server (engine-memory-mcp) that hits your embeddinggemma-2-GGUF endpoint at **768 dimensions**, L2-normalizes the vector, and stores/queries two Qdrant collections: + * engine\_architecture\_memory: Architectural decisions, ABI contracts, Flecs/Jolt gotchas, and post-mortems of failed builds. + * engine\_code\_symbols: AST-chunked C++ headers and C\# interfaces indexed automatically by a Git post-commit hook. +> 2. **Procedural & Episodic Memory (Hermes Native MEMORY.md \+ SQLite FTS5):** + * Hermes Agent natively maintains \~/.hermes/MEMORY.md (scanned for prompt injection before every write) and full-text FTS5 search over all historical agent transcripts (\~/.hermes/sessions.db). + * Symlink \~/.hermes/MEMORY.md into the shared workspace (/workspace/SHARED\_MEMORY.md) so every **Pi** worker seat in OpenRig reads the exact same curated institutional memory on startup. + +## **Part 4: The Exact MCP Servers & Skills Stack** + +To build a C++20 (CMake/Flecs/Jolt/Vulkan) \+ .NET 10 (Roslyn/ImGui) engine without agents hallucinating APIs or breaking the ABI, every agent seat needs a locked-in set of **LSPs**, **MCP servers**, and **Skills**. + +### **1\. Language Servers (Wired into pi-lsp-extension & hermes-code-intel-plugin)** + +Install these binaries directly in the LXC image so pi-lsp-extension auto-detects them and appends compiler errors after every file edit: + +* **C++20 (clangd-19 or clangd-20):** Configured with \--background-index \--clang-tidy \--header-insertion=never \--compile-commands-dir=/workspace/build. +* **C\# / .NET 10 (csharp-ls or Microsoft.CodeAnalysis.LanguageServer):** Installed via dotnet tool install \--global csharp-ls. Gives Pi instant diagnostics on unsafe blocks, Span\, and Roslyn IIncrementalGenerator code. +* **Shaders (slangd / glsl\_analyzer):** Language server for .slang and .glsl compute/mesh shaders. + +### **2\. The 6 Mandatory MCP Servers (.mcp.json)** + +Attach these across your OpenRig / Paperclip seats: + +> 1. **openrig-mcp (@openrig/cli built-in MCP):** + * Gives the Lead Agent rig\_up, rig\_down, rig\_ps, rig\_send, and rig\_snapshot\_create to command the worker seats. +> 2. **clangd-mcp (uv tool install git+\[https\://github.com/schuay/clangd-mcp.git\](https\://github.com/schuay/clangd-mcp.git)):** + * Exposes C++ call hierarchies (get\_callers, get\_callees, find\_implementations, list\_file\_symbols) over compile\_commands.json for agents that need deep cross-TU C++ inspection beyond single-file diagnostics. +> 3. **roslyn-mcp (JoshuaRamirez/RoslynMcpServer or sphcode/roslyn-mcp-server):** + * Exposes 60+ Roslyn semantic workspace tools (find all implementations of IComponent, inspect generated .g.cs syntax trees in memory, verify blittable struct layout). +> 4. **ast-grep-mcp (\[github.com/ast-grep/ast-grep-mcp\](https\://github.com/ast-grep/ast-grep-mcp) or @notprolands/ast-grep-mcp):** + * Exposes dump\_syntax\_tree, find\_code, find\_code\_by\_rule, and test\_match\_code\_rule. Essential for bulk codemods across C++ and C\#. +> 5. **engine-memory-mcp (Backed by unsloth/embeddinggemma-2-GGUF \+ Qdrant):** + * Exposes memory\_store\_decision, memory\_search\_architecture, and codebase\_semantic\_search. +> 6. **renderdoc-vulkan-mcp (Headless GPU Verification Wrapper):** + * A lightweight local MCP wrapper around vulkaninfo, glslangValidator, spirv-Cross, and headless ctest runner (ctest \--output-on-failure) using **Lavapipe** (mesa-vulkan-drivers software rasterizer) or passthrough /dev/dri GPU so agents can run and screenshot Vulkan 1.3 tests inside the LXC container without a physical monitor. + +### **3\. The Skills Library (\~/.agents/skills/ & \~/.claude/skills/)** + +Install these skills into the shared skills directory so Pi, Hermes, and Claude Code load them automatically: + +* **openrig/skills/rigs & refocusing**: Built-in OpenRig skills for inter-agent message passing (rig send), queue management, and context refocusing when a task drifts. +* **paperclip-skill (paperclip/doc/SPEC.md)**: Allows agents to query goal ancestry, update ticket status, post build metrics, and spawn follow-up tasks in the Paperclip Kanban. +* **hermes/ast-grep (nousresearch/hermes-agent/optional-skills/software-development/ast-grep/SKILL.md)**: Enforces the two-pass dry-run invariant (helper validate \-\> helper search \-\> helper replace \--apply). +* **Custom Engine Domain Skills (We will seed these on boot):** + * zero-copy-abi-guard: Enforces that every change to native/bridge/engine\_abi.h is immediately followed by running python3 tools/abi\_codegen/generate.py and dotnet test on the layout assertions. + * flecs-jolt-patterns: Contains canonical C99/C++20 idioms for Flecs v4 ecs\_query\_t, alignas(64) component registration, and Jolt BodyInterface lock-free batching. + * roslyn-incremental-gen: Enforces strict IIncrementalGenerator value-equatable pipeline rules so generated code never breaks IDE responsiveness. + +## **Part 5: How to Configure Muse Spark 1.3 Across All Harnesses** + +Meta's Model API (\[https\://api.meta.ai/v1\](https\://api.meta.ai/v1)) is natively drop-in compatible with **both** the OpenAI API and the Anthropic API (muse-spark-1.3-contributor, 1M context window). Put these environment variables in /etc/environment inside the LXC container: + +Bash +\# Master Muse Spark 1.3 Contributor Credentials +export MODEL\_API\_KEY="your\_meta\_model\_api\_key" + +\# 1\. For OpenAI-compatible Harnesses (Pi, Hermes, OpenCode, Sortie) +export OPENAI\_API\_BASE="https\://api.meta.ai/v1" +export OPENAI\_BASE\_URL="https\://api.meta.ai/v1" +export OPENAI\_API\_KEY="\${MODEL\_API\_KEY}" + +\# 2\. For Anthropic-compatible Harnesses (Claude Code, Paperclip Default Adapter) +export ANTHROPIC\_BASE\_URL="https\://api.meta.ai/v1" +export ANTHROPIC\_AUTH\_TOKEN="\${MODEL\_API\_KEY}" +export ANTHROPIC\_API\_KEY="\${MODEL\_API\_KEY}" +export ANTHROPIC\_MODEL="muse-spark-1.3-contributor" +export ANTHROPIC\_DEFAULT\_SONNET\_MODEL="muse-spark-1.3-contributor" +export ANTHROPIC\_DEFAULT\_OPUS\_MODEL="muse-spark-1.3-contributor" +export ANTHROPIC\_DEFAULT\_HAIKU\_MODEL="muse-spark-1.2" + +\# 3\. For Unsloth EmbeddingGemma-2-GGUF Server +export EMBEDDING\_BASE\_URL="http\://127.0.0.1:8081/v1" \# Or your Proxmox host IP +export EMBEDDING\_MODEL="unsloth/embeddinggemma-2-GGUF" + +## **Part 6: The Turnkey LXC Factory Blueprint (Generated Files)** + +To make this immediately actionable, I am generating the 4 core configuration files you need to drop into your LXC container to start this factory and let it run until all 5 milestones are complete: + +> 1. **lxc-factory/setup-lxc-factory.sh**: Complete provisioning script for an Ubuntu 24.04 Proxmox LXC container (installs C++20/Clang-19/CMake/Ninja/Vulkan-SDK/Lavapipe, .NET 10 SDK, Node 24, Go 1.23, Qdrant, pi-coding-agent \+ pi-lsp-extension, hermes-agent \+ hermes-code-intel-plugin, claude-code, openrig, sortie, and paperclip, plus systemd services so everything survives reboots). +> 2. **lxc-factory/openrig-engine-team.yaml**: The declarative **OpenRig RigSpec** defining your 6 persistent agent seats (chief-architect, cpp-ecs-core, dotnet-roslyn-bridge, vulkan-renderer, editor-cli-ux, and adversarial-qa-gate), their harnesses (Pi \+ Hermes), and their inter-agent communication edges. +> 3. **lxc-factory/WORKFLOW.md**: The **Sortie** autonomous issue-runner configuration that polls your GitHub repo for agent-ready issues, manages isolated Git worktrees, enforces a 5-minute stall watchdog, and loops failed CMake/.NET CI checks back to the worker until green. +> 4. **lxc-factory/mcp-embeddinggemma-qdrant.py**: The custom FastMCP server that connects your agents' long-term memory directly to your **unsloth/embeddinggemma-2-GGUF** server with proper SearchQuery/Document prefixes and L2-normalized 768-dim vectors. + +lxc-factory/setup-lxc-factory.sh \[generated with file\_gen tool\] +lxc-factory/openrig-engine-team.yaml \[generated with file\_gen tool\] +lxc-factory/WORKFLOW.md \[generated with file\_gen tool\] +lxc-factory/mcp-embeddinggemma-qdrant.py \[generated with file\_gen tool\] + +## **Part 7: How the Unattended Loop Actually Runs (From Boot to Completion)** + +Once you run setup-lxc-factory.sh inside your Proxmox LXC container and run ./scripts/bootstrap-github-issues.sh (from our previous step) to populate the 24 GitHub issues, here is the exact mechanical loop that keeps running 24/7 until the engine is finished: + +> 1. **The Heartbeat Trigger (Every 10 Minutes):**systemd runs engine-watchdog.timer (and Paperclip runs its internal heartbeat scheduler). If the openrig swarm is idle, it wakes chief-architect@engine-factory (running **Hermes Agent** on muse-spark-1.3-contributor). +> 2. **Milestone Gating & Dispatch:**chief-architect queries engine-memory-mcp (backed by your embeddinggemma-2-GGUF server) to check what was completed in the last session, inspects the GitHub Issue board via gh issue list, and applies the agent-ready label to the next unblocked issues (e.g., M1-01 and M1-02). +> 3. **Isolated Worktree Checkout & Execution:Sortie** (sortie-factory.service) detects the agent-ready label within 30 seconds, clones an isolated Git worktree, and launches **Pi** (pi-coding-agent) with WORKFLOW.md. +> 4. **LSP-Guided Coding \+ Cross-Agent Consultation:** + * As the **Pi** worker writes C++20 or C\#13 code, pi-lsp-extension keeps clangd and csharp-ls hot in the background. Every file edit immediately returns compiler/type diagnostics in the same tool turn. + * If the C\# worker needs a change to the C99 ABI struct, it runs rig send cpp-ecs-core@engine-factory "Need EcsChunkView updated with entity\_ids pointer" over **OpenRig**. +> 5. **Adversarial Verification & Self-Healing:** Before opening a PR, the worker must pass the local CMake \+ .NET 10 \+ Lavapipe headless Vulkan test suite. If a test fails or if an agent stalls for \> 5 minutes (stall\_timeout\_ms: 300000), Sortie kills the runaway process, feeds the failure log back into the retry queue, and loops until the PR is green. +> 6. **Memory Consolidation & Skill Evolution:** Once adversarial-qa-gate@engine-factory approves and merges the PR, chief-architect stores the architectural summary into Qdrant via embeddinggemma-2-GGUF, runs hermes curator to refine its local SKILL.md files with any new Flecs/Jolt/Vulkan lessons learned, and unlocks the next issue on the Kanban board. + +Would you like me to generate the **Paperclip Org Chart & Heartbeat seed script (seed-paperclip-org.js)** so Paperclip's UI on Port pre-loads these exact 6 agents and 5 milestones automatically on first boot, or should we generate the **Day-1 C++20/Flecs/.NET 10 seed repository files** so the agents have a clean initial commit to start from? \ No newline at end of file diff --git a/scripts/bootstrap-github-issues.sh b/scripts/bootstrap-github-issues.sh new file mode 100755 index 0000000..d26db81 --- /dev/null +++ b/scripts/bootstrap-github-issues.sh @@ -0,0 +1,146 @@ +#!/usr/bin/env bash +# ============================================================================== +# GitHub Roadmap, Labels, Milestones & Issues Bootstrapper +# Prerequisite: Install GitHub CLI (https://cli.github.com/) and run `gh auth login` +# Usage: chmod +x scripts/bootstrap-github-issues.sh && ./scripts/bootstrap-github-issues.sh +# ============================================================================== + +set -euo pipefail + +echo "==> 1. Creating Architecture Labels..." +gh label create "area:native-core" --color "d73a4a" --description "C++20 memory, job system, Flecs ECS, OS abstraction" --force +gh label create "area:abi-bridge" --color "0075ca" --description "hostfxr, C99 ABI headers, abi_codegen, zero-copy marshaling" --force +gh label create "area:roslyn-illusion" --color "a2eeef" --description "Roslyn Incremental Source Generators & GC Analyzers" --force +gh label create "area:renderer" --color "7057ff" --description "Vulkan 1.3 bindless, Slang shaders, GPU culling, SDF GI" --force +gh label create "area:physics-audio" --color "008672" --description "Jolt Physics & miniaudio integration" --force +gh label create "area:editor-cli" --color "e4e669" --description "Asset pipeline, Editor UI, Prefabs, IPC/CLI protocol" --force +gh label create "priority:critical" --color "b60205" --description "Core architectural blocker for milestone gate" --force + +echo "==> 2. Creating 5 Roadmap Milestones..." +REPO=$(gh repo view --json nameWithOwner -q .nameWithOwner) + +create_milestone() { + local title="$1" + local desc="$2" + gh api "repos/${REPO}/milestones" -f title="$title" -f description="$desc" >/dev/null || true +} + +create_milestone "M1: The Spine (Native ECS + Zero-Copy Host)" "Gate: 1M LocalTransform updates from C# in <1.5ms with 0 GC allocations." +create_milestone "M2: The Illusion Compiler (Roslyn + Hot Reload)" "Gate: <100ms state-preserving C# hot-reload + OOP-to-ECS source generation." +create_milestone "M3: GPU-Driven Vulkan 1.3 Renderer" "Gate: 250k dynamic PBR meshes + real-time SDF GI at 120+ FPS (<0.5ms CPU)." +create_milestone "M4: Deterministic Jolt Physics & Audio" "Gate: 30k colliding rigidbodies + zero-cost C# raycasts & spatial audio at 60+ FPS." +create_milestone "M5: Editor UX, Asset Pipeline & Agent CLI" "Gate: <16ms background asset streaming, engine-cli IPC server, 1-click NativeAOT export." + +echo "==> 3. Populating Issues across Milestones..." + +# --- MILESTONE 1 --- +gh issue create --title "[M1-01] Initialize Monorepo, CMake C++20 Build, and .slnx Workspace" \ + --milestone "M1: The Spine (Native ECS + Zero-Copy Host)" \ + --label "area:native-core,priority:critical" \ + --body "See \`docs/01-MILESTONE-1-THE-SPINE.md\` (Issue M1-01) for full specification and acceptance criteria." + +gh issue create --title "[M1-02] Native Memory Arenas & 64-Byte Cache-Aligned Allocators" \ + --milestone "M1: The Spine (Native ECS + Zero-Copy Host)" \ + --label "area:native-core" \ + --body "See \`docs/01-MILESTONE-1-THE-SPINE.md\` (Issue M1-02) for full specification and acceptance criteria." + +gh issue create --title "[M1-03] Work-Stealing Job Scheduler & Flecs Hierarchy Pipeline" \ + --milestone "M1: The Spine (Native ECS + Zero-Copy Host)" \ + --label "area:native-core" \ + --body "See \`docs/01-MILESTONE-1-THE-SPINE.md\` (Issue M1-03) for full specification and acceptance criteria." + +gh issue create --title "[M1-04] C99 ABI Contract (engine_abi.h) & Automated C# Binding Generator" \ + --milestone "M1: The Spine (Native ECS + Zero-Copy Host)" \ + --label "area:abi-bridge,priority:critical" \ + --body "See \`docs/01-MILESTONE-1-THE-SPINE.md\` (Issue M1-04) for full specification and acceptance criteria." + +gh issue create --title "[M1-05] Embed .NET 10 (hostfxr) & 1M Entity Zero-Copy Benchmark" \ + --milestone "M1: The Spine (Native ECS + Zero-Copy Host)" \ + --label "area:abi-bridge,priority:critical" \ + --body "See \`docs/01-MILESTONE-1-THE-SPINE.md\` (Issue M1-05) for full specification and acceptance criteria." + +# --- MILESTONE 2 --- +gh issue create --title "[M2-01] Ergonomic IComponent, Entity, and Phase Attributes API" \ + --milestone "M2: The Illusion Compiler (Roslyn + Hot Reload)" \ + --label "area:roslyn-illusion" \ + --body "See \`docs/02-MILESTONE-2-ILLUSION-COMPILER.md\` (Issue M2-01) for full specification and acceptance criteria." + +gh issue create --title "[M2-02] Roslyn IIncrementalGenerator (IComponent to Native Chunk Job)" \ + --milestone "M2: The Illusion Compiler (Roslyn + Hot Reload)" \ + --label "area:roslyn-illusion,priority:critical" \ + --body "See \`docs/02-MILESTONE-2-ILLUSION-COMPILER.md\` (Issue M2-02) for full specification and acceptance criteria." + +gh issue create --title "[M2-03] Compile-Time Performance & Memory Safety Roslyn Analyzers" \ + --milestone "M2: The Illusion Compiler (Roslyn + Hot Reload)" \ + --label "area:roslyn-illusion" \ + --body "See \`docs/02-MILESTONE-2-ILLUSION-COMPILER.md\` (Issue M2-03) for full specification and acceptance criteria." + +gh issue create --title "[M2-04] Stateless Collectible AssemblyLoadContext 100ms Hot-Reload" \ + --milestone "M2: The Illusion Compiler (Roslyn + Hot Reload)" \ + --label "area:abi-bridge,area:roslyn-illusion,priority:critical" \ + --body "See \`docs/02-MILESTONE-2-ILLUSION-COMPILER.md\` (Issue M2-04) for full specification and acceptance criteria." + +# --- MILESTONE 3 --- +gh issue create --title "[M3-01] SDL3 Windowing, Vulkan 1.3 Bootstrap (volk + VMA), and Render Graph" \ + --milestone "M3: GPU-Driven Vulkan 1.3 Renderer" \ + --label "area:renderer,priority:critical" \ + --body "See \`docs/03-MILESTONE-3-VULKAN-RENDERER.md\` (Issue M3-01) for full specification and acceptance criteria." + +gh issue create --title "[M3-02] Bindless Descriptor Table & GPU-Driven Indirect Drawing" \ + --milestone "M3: GPU-Driven Vulkan 1.3 Renderer" \ + --label "area:renderer,priority:critical" \ + --body "See \`docs/03-MILESTONE-3-VULKAN-RENDERER.md\` (Issue M3-02) for full specification and acceptance criteria." + +gh issue create --title "[M3-03] Slang Shader Compiler Pipeline & C# GraphicsCompositor API" \ + --milestone "M3: GPU-Driven Vulkan 1.3 Renderer" \ + --label "area:renderer,area:abi-bridge" \ + --body "See \`docs/03-MILESTONE-3-VULKAN-RENDERER.md\` (Issue M3-03) for full specification and acceptance criteria." + +gh issue create --title "[M3-04] Zero-Bake Real-Time GI (Signed Distance Fields + Radiance Cascades)" \ + --milestone "M3: GPU-Driven Vulkan 1.3 Renderer" \ + --label "area:renderer" \ + --body "See \`docs/03-MILESTONE-3-VULKAN-RENDERER.md\` (Issue M3-04) for full specification and acceptance criteria." + +# --- MILESTONE 4 --- +gh issue create --title "[M4-01] Jolt Physics Integration & Shared Work-Stealing Thread Pool" \ + --milestone "M4: Deterministic Jolt Physics & Audio" \ + --label "area:physics-audio,priority:critical" \ + --body "See \`docs/04-MILESTONE-4-PHYSICS-AND-AUDIO.md\` (Issue M4-01) for full specification and acceptance criteria." + +gh issue create --title "[M4-02] Zero-Glue Parallel Flecs <-> Jolt Transform Synchronization" \ + --milestone "M4: Deterministic Jolt Physics & Audio" \ + --label "area:physics-audio,area:native-core" \ + --body "See \`docs/04-MILESTONE-4-PHYSICS-AND-AUDIO.md\` (Issue M4-02) for full specification and acceptance criteria." + +gh issue create --title "[M4-03] Zero-Cost C# Spatial Queries (Physics.Raycast / OverlapSphere)" \ + --milestone "M4: Deterministic Jolt Physics & Audio" \ + --label "area:physics-audio,area:abi-bridge" \ + --body "See \`docs/04-MILESTONE-4-PHYSICS-AND-AUDIO.md\` (Issue M4-03) for full specification and acceptance criteria." + +gh issue create --title "[M4-04] Low-Latency 3D Spatial Audio Graph (miniaudio)" \ + --milestone "M4: Deterministic Jolt Physics & Audio" \ + --label "area:physics-audio" \ + --body "See \`docs/04-MILESTONE-4-PHYSICS-AND-AUDIO.md\` (Issue M4-04) for full specification and acceptance criteria." + +# --- MILESTONE 5 --- +gh issue create --title "[M5-01] Non-Blocking Background Asset Pipeline & .meta GUID Database" \ + --milestone "M5: Editor UX, Asset Pipeline & Agent CLI" \ + --label "area:editor-cli,priority:critical" \ + --body "See \`docs/05-MILESTONE-5-EDITOR-AND-CLI.md\` (Issue M5-01) for full specification and acceptance criteria." + +gh issue create --title "[M5-02] Native-Backed Editor UI (Hierarchy, Inspector, Prefabs, Gizmos)" \ + --milestone "M5: Editor UX, Asset Pipeline & Agent CLI" \ + --label "area:editor-cli,priority:critical" \ + --body "See \`docs/05-MILESTONE-5-EDITOR-AND-CLI.md\` (Issue M5-02) for full specification and acceptance criteria." + +gh issue create --title "[M5-03] First-Class IPC / JSON-RPC Server & engine-cli Control Plane" \ + --milestone "M5: Editor UX, Asset Pipeline & Agent CLI" \ + --label "area:editor-cli" \ + --body "See \`docs/05-MILESTONE-5-EDITOR-AND-CLI.md\` (Issue M5-03) for full specification and acceptance criteria." + +gh issue create --title "[M5-04] One-Click Standalone NativeAOT Game Pak Exporter" \ + --milestone "M5: Editor UX, Asset Pipeline & Agent CLI" \ + --label "area:editor-cli,area:abi-bridge" \ + --body "See \`docs/05-MILESTONE-5-EDITOR-AND-CLI.md\` (Issue M5-04) for full specification and acceptance criteria." + +echo "==> Done! All labels, milestones, and 20 issues have been created." \ No newline at end of file