langchain-ai/deepagentsjs
1
0
Fork 0
mirror of https://github.com/langchain-ai/deepagentsjs.git synced 2026-07-18 16:04:44 -04:00
Code Issues 42 Packages Projects Releases 10 Wiki Activity

[PR #524] [MERGED] fix(quickjs): individual repl sessions use individual wasm module causing inefficient memory usage #537

New Issue
Closed
opened 2026-06-05 17:23:36 -04:00 by yindo · 0 comments
yindo commented 2026-06-05 17:23:36 -04:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/langchain-ai/deepagentsjs/pull/524
Author: @colifran
Created: 5/6/2026
Status: ✅ Merged
Merged: 5/6/2026
Merged by: @colifran

Base: main ← Head: colifran/mem

📝 Commits (5)

📊 Changes

4 files changed (+234 additions, -33 deletions)

View changed files

➕ .changeset/wet-tables-notice.md (+5 -0)
➕ libs/providers/quickjs/src/eval-queue.test.ts (+77 -0)
➕ libs/providers/quickjs/src/eval-queue.ts (+33 -0)
📝 libs/providers/quickjs/src/session.ts (+119 -33)

📄 Description

Summary

This PR optimizes memory usage by allowing us to use a single WASM module. To do this we needed to fix two failure modes that occur when multiple sessions share a single WASM module. Both are caused by the async module loader creating asyncify suspensions (unwind/rewind cycles on a module-global stack buffer) during import resolution.

Failure 1: Concurrent eval crash. The asyncify variant allows only one async operation per module at a time. If two sessions eval simultaneously and both hit an import, the second eval tries to suspend asyncify while the first is already suspended. This crashes with QuickJSAsyncifySuspended: Already suspended.
Fix: AsyncEvalQueue queues evalCodeAsync calls so only one runs at a time.

Failure 2: Silent disposal corruption. When a session imports a multi-file skill (e.g. an entry point that imports greet.js and shout.js), each import causes an asyncify unwind/rewind cycle. The eval succeeds. But when the session's runtime is disposed, the disposal does not fully reset the module's asyncify stack buffer and the unwind/rewind cycles left residual state (saved stack frames, rewind pointers) that emscripten manages at the module level, not the runtime level. The next session creates a fresh runtime on the same module, but the corrupted asyncify state causes module loader callbacks to silently never fire.
Fix: synchronous module loader. Skill files are preloaded into a Map before eval starts. The loader callback does a plain map.get(). No await, no asyncify suspension, no residual state on disposal.

Note: The eval queue alone does not fix failure 2. The corruption happens at runtime disposal, which runs outside the queue. Both fixes are required.

What This PR Adds

  • Replaces per-session WASM module instantiation with a process-global singleton shared by all sessions. Each session still gets its own runtime and context (full isolation); only the underlying WASM module is shared. Reduces WASM memory from ~16 MB per session to ~16 MB total.
  • Adds AsyncEvalQueue to serialize evalCodeAsync calls across sessions, enforcing the asyncify one-concurrent-async-call-per-module constraint.
  • Makes the module loader synchronous by preloading skill files into an in-memory cache before eval (preloadReferencedSkills). Why is this needed? An async module loader causes asyncify suspensions on each import. Disposing a runtime after multi-file skill imports corrupts the shared module's asyncify state and silently breaks the loader for all subsequent sessions. The sync loader eliminates asyncify suspensions from imports entirely.

Synchronous Module Loading

As stated above, synchronous module loading is a necessary change due to async module loading causing asyncify suspensions on each import which corrupts the singleton WASM module's asyncify state which silently breaks the loader for all subsequent sessions. The following demonstrates the before and after regarding this change:

Before:
User code: import { run } from "@/skills/demo"

  1. eval() is called
  2. evalCodeAsync() runs the code
  3. QuickJS hits the import, calls the module loader callback
  4. The callback is async: await ensureSkillLoaded("demo")
    → fetches /skills/demo/index.js from the backend (network I/O)
    → this await causes the first asyncify suspension in the WASM module
  5. index.js has: import { greet } from "./greet.js"
  6. QuickJS calls the loader again for greet.js
    → await ensureSkillLoaded() → network fetch
    → The second asyncify suspension in the WASM module is hit
  7. index.js has: import { shout } from "./shout.js"
  8. QuickJS calls the loader again for shout.js
    → await ensureSkillLoaded() → network fetch
    → third asyncify suspension in the WASM module is hit
  9. eval completes, runtime is disposed
  10. runtime.dispose() is called to clean up the session
  11. The runtime disposal does NOT fully reset the module's asyncify stack buffer. The 3 unwind/rewind cycles left residual state (saved stack frames, rewind pointers, etc.) that the dispose path doesn't know to clean up, because asyncify bookkeeping is managed by emscripten at the module level, not by QuickJS at the runtime level.
  12. Next session creates a fresh runtime on the same module, but the module's asyncify machinery is corrupted. When QuickJS tries to call the new session's module loader, the stale asyncify state causes the callback to silently not fire. No crash, no error. Imports just resolve to nothing.

After:
User code: import { run } from "@/skills/demo"

  1. eval() is called
  2. preloadReferencedSkills() runs BEFORE evalCodeAsync():
    → scans source code for "@/skills/*" specifiers
    → finds "demo"
    → fetches all 3 files from the backend into a Map:
    "@/skills/demo" → source code
    "@/skills/demo/greet.js" → source code
    "@/skills/demo/shout.js" → source code
    → (this is async but happens OUTSIDE the WASM module so no asyncify)
  3. evalCodeAsync() runs the code
  4. QuickJS hits the import, calls the module loader callback
  5. The callback is sync: map.get("@/skills/demo") → returns source immediately
    → NO asyncify suspension
  6. QuickJS resolves greet.js → map.get("@/skills/demo/greet.js") → immediate
    → NO asyncify suspension
  7. QuickJS resolves shout.js → map.get("@/skills/demo/shout.js") → immediate
    → NO asyncify suspension
  8. eval completes, runtime is disposed
  9. Zero asyncify suspensions occurred → no residual state → module is clean
  10. Next session works perfectly

Tests

  • 4 new AsyncEvalQueue unit tests: serialization (no interleaving of concurrent ops), return values, error isolation (rejected op doesn't break queue), insertion order preservation
  • All 160 existing tests pass (session, skills, middleware, transform, utils, eval queue)

Memory Benchmarking

Per-session module (before)

Sessions RSS Delta (MB) Per-Session RSS (MB) WASM Memory (MB) Module Init (ms) Setup (ms)
1 0.53 0.53 16 11.4 15.9
5 2.13 0.43 80 30.2 37.9
10 5.28 0.53 160 31.1 42.8
15 5.53 0.37 240 57.4 72.1
20 6.80 0.34 320 73.6 91.1

Singleton (after)

Sessions RSS Delta (MB) Per-Session RSS (MB) WASM Memory (MB) Module Init (ms) Setup (ms)
1 0.61 0.61 16 6.7 9.1
5 0.59 0.12 16 12.5 28.5
10 0.91 0.09 16 12.2 37.0
15 1.81 0.12 16 12.0 45.1
20 1.70 0.09 16 10.4 50.4

Comparison at 20 sessions (realistic workload)

Metric Before After Improvement
WASM virtual memory 320 MB 16 MB 20x reduction
RSS delta 6.80 MB 1.70 MB 4x reduction
Module instantiation 73.6 ms 10.4 ms 7x reduction
Setup time 91.1 ms 50.4 ms 1.8x reduction

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/langchain-ai/deepagentsjs/pull/524 **Author:** [@colifran](https://github.com/colifran) **Created:** 5/6/2026 **Status:** ✅ Merged **Merged:** 5/6/2026 **Merged by:** [@colifran](https://github.com/colifran) **Base:** `main` ← **Head:** `colifran/mem` --- ### 📝 Commits (5) - [`e8f2b66`](https://github.com/langchain-ai/deepagentsjs/commit/e8f2b66a33d7e1c3b0c4742bb028ed26a9628338) implement async eval queue - [`23172ea`](https://github.com/langchain-ai/deepagentsjs/commit/23172eacd9450d2153c4be6fba745080646b64dd) eval queue and wiring - [`2952fc3`](https://github.com/langchain-ai/deepagentsjs/commit/2952fc386acd0a1ba262856f26e9e8df5984bd1a) eval queue tests - [`1331cd7`](https://github.com/langchain-ai/deepagentsjs/commit/1331cd78ab8164cce73b7d6f478efc4c4931fa20) update docstrings - [`ab8df37`](https://github.com/langchain-ai/deepagentsjs/commit/ab8df3718fbf19e9419bfd23a40c7c3ac1abf4b9) changeset ### 📊 Changes **4 files changed** (+234 additions, -33 deletions) <details> <summary>View changed files</summary> ➕ `.changeset/wet-tables-notice.md` (+5 -0) ➕ `libs/providers/quickjs/src/eval-queue.test.ts` (+77 -0) ➕ `libs/providers/quickjs/src/eval-queue.ts` (+33 -0) 📝 `libs/providers/quickjs/src/session.ts` (+119 -33) </details> ### 📄 Description ### Summary This PR optimizes memory usage by allowing us to use a single WASM module. To do this we needed to fix two failure modes that occur when multiple sessions share a single WASM module. Both are caused by the async module loader creating asyncify suspensions (unwind/rewind cycles on a module-global stack buffer) during import resolution. __Failure 1__: Concurrent eval crash. The asyncify variant allows only one async operation per module at a time. If two sessions eval simultaneously and both hit an import, the second eval tries to suspend asyncify while the first is already suspended. This crashes with QuickJSAsyncifySuspended: Already suspended. Fix: AsyncEvalQueue queues evalCodeAsync calls so only one runs at a time. __Failure 2__: Silent disposal corruption. When a session imports a multi-file skill (e.g. an entry point that imports greet.js and shout.js), each import causes an asyncify unwind/rewind cycle. The eval succeeds. But when the session's runtime is disposed, the disposal does not fully reset the module's asyncify stack buffer and the unwind/rewind cycles left residual state (saved stack frames, rewind pointers) that emscripten manages at the module level, not the runtime level. The next session creates a fresh runtime on the same module, but the corrupted asyncify state causes module loader callbacks to silently never fire. Fix: synchronous module loader. Skill files are preloaded into a Map before eval starts. The loader callback does a plain map.get(). No await, no asyncify suspension, no residual state on disposal. Note: The eval queue alone does not fix failure 2. The corruption happens at runtime disposal, which runs outside the queue. Both fixes are required. #### What This PR Adds - Replaces per-session WASM module instantiation with a process-global singleton shared by all sessions. Each session still gets its own runtime and context (full isolation); only the underlying WASM module is shared. Reduces WASM memory from ~16 MB per session to ~16 MB total. - Adds AsyncEvalQueue to serialize evalCodeAsync calls across sessions, enforcing the asyncify one-concurrent-async-call-per-module constraint. - Makes the module loader synchronous by preloading skill files into an in-memory cache before eval (preloadReferencedSkills). Why is this needed? An async module loader causes asyncify suspensions on each import. Disposing a runtime after multi-file skill imports corrupts the shared module's asyncify state and silently breaks the loader for all subsequent sessions. The sync loader eliminates asyncify suspensions from imports entirely. #### Synchronous Module Loading As stated above, synchronous module loading is a necessary change due to async module loading causing asyncify suspensions on each import which corrupts the singleton WASM module's asyncify state which silently breaks the loader for all subsequent sessions. The following demonstrates the before and after regarding this change: __Before__: User code: `import { run } from "@/skills/demo"` 1. eval() is called 2. evalCodeAsync() runs the code 3. QuickJS hits the import, calls the module loader callback 4. The callback is async: await ensureSkillLoaded("demo") → fetches /skills/demo/index.js from the backend (network I/O) → this await causes the first asyncify suspension in the WASM module 5. index.js has: import { greet } from "./greet.js" 6. QuickJS calls the loader again for greet.js → await ensureSkillLoaded() → network fetch → The second asyncify suspension in the WASM module is hit 7. index.js has: import { shout } from "./shout.js" 8. QuickJS calls the loader again for shout.js → await ensureSkillLoaded() → network fetch → third asyncify suspension in the WASM module is hit 9. eval completes, runtime is disposed 10. runtime.dispose() is called to clean up the session 11. The runtime disposal does NOT fully reset the module's asyncify stack buffer. The 3 unwind/rewind cycles left residual state (saved stack frames, rewind pointers, etc.) that the dispose path doesn't know to clean up, because asyncify bookkeeping is managed by emscripten at the module level, not by QuickJS at the runtime level. 12. Next session creates a fresh runtime on the same module, but the module's asyncify machinery is corrupted. When QuickJS tries to call the new session's module loader, the stale asyncify state causes the callback to silently not fire. No crash, no error. Imports just resolve to nothing. __After__: User code: `import { run } from "@/skills/demo"` 1. eval() is called 2. preloadReferencedSkills() runs BEFORE evalCodeAsync(): → scans source code for "@/skills/*" specifiers → finds "demo" → fetches all 3 files from the backend into a Map: "@/skills/demo" → source code "@/skills/demo/greet.js" → source code "@/skills/demo/shout.js" → source code → (this is async but happens OUTSIDE the WASM module so no asyncify) 3. evalCodeAsync() runs the code 4. QuickJS hits the import, calls the module loader callback 5. The callback is sync: map.get("@/skills/demo") → returns source immediately → NO asyncify suspension 6. QuickJS resolves greet.js → map.get("@/skills/demo/greet.js") → immediate → NO asyncify suspension 7. QuickJS resolves shout.js → map.get("@/skills/demo/shout.js") → immediate → NO asyncify suspension 8. eval completes, runtime is disposed 9. Zero asyncify suspensions occurred → no residual state → module is clean 10. Next session works perfectly ### Tests - 4 new AsyncEvalQueue unit tests: serialization (no interleaving of concurrent ops), return values, error isolation (rejected op doesn't break queue), insertion order preservation - All 160 existing tests pass (session, skills, middleware, transform, utils, eval queue) ### Memory Benchmarking #### Per-session module (before) | Sessions | RSS Delta (MB) | Per-Session RSS (MB) | WASM Memory (MB) | Module Init (ms) | Setup (ms) | |----------|----------------|----------------------|------------------|------------------|------------| | 1 | 0.53 | 0.53 | 16 | 11.4 | 15.9 | | 5 | 2.13 | 0.43 | 80 | 30.2 | 37.9 | | 10 | 5.28 | 0.53 | 160 | 31.1 | 42.8 | | 15 | 5.53 | 0.37 | 240 | 57.4 | 72.1 | | 20 | 6.80 | 0.34 | 320 | 73.6 | 91.1 | --- #### Singleton (after) | Sessions | RSS Delta (MB) | Per-Session RSS (MB) | WASM Memory (MB) | Module Init (ms) | Setup (ms) | |----------|----------------|----------------------|------------------|------------------|------------| | 1 | 0.61 | 0.61 | 16 | 6.7 | 9.1 | | 5 | 0.59 | 0.12 | 16 | 12.5 | 28.5 | | 10 | 0.91 | 0.09 | 16 | 12.2 | 37.0 | | 15 | 1.81 | 0.12 | 16 | 12.0 | 45.1 | | 20 | 1.70 | 0.09 | 16 | 10.4 | 50.4 | --- #### Comparison at 20 sessions (realistic workload) | Metric | Before | After | Improvement | |------------------------|---------|--------|-----------------| | WASM virtual memory | 320 MB | 16 MB | 20x reduction | | RSS delta | 6.80 MB | 1.70 MB| 4x reduction | | Module instantiation | 73.6 ms | 10.4 ms| 7x reduction | | Setup time | 91.1 ms | 50.4 ms| 1.8x reduction | --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-06-05 17:23:36 -04:00
yindo closed this issue 2026-06-05 17:23:36 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
enhancement
help wanted
needs-response
pull-request
Mirrored from GitHub Pull Request
 stale
triage: high-impact
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/deepagentsjs#537
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?